Job Title

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.WHY JOIN US If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!ABOUT THE ROLE As a Data Engineer specializing in Java and Apache Spark, you will help build and evolve large-scale data processing systems that power experimentation and user insights. Working within a cloud-based AWS EMR environment, you’ll contribute to improving data infrastructure reliability, scalability, and cost efficiency. This role offers the opportunity to shape critical datasets and analytics capabilities while collaborating with platform and data teams to support high-impact experimentation and decision-making.WHAT YOU WILL DO - Develop and maintain index builder products, including user session index builders, user session–derived index builders, and experimentation platform index builders; - Investigate and resolve reported issues related to index builders; - Assist with user inquiries regarding the platform and its datasets; - Improve index builder stability and reliability; - Support efforts to optimize compute costs across the platform; - Contribute to the Central Exposure Dataset effort, including building a consolidated dataset for experiment analysis; - Work toward meeting code freshness goals; - Persist Yarn logs and Spark history for terminated clusters; - Capture metrics from UserCohort; - Optimize resource allocation for platform infrastructure; - Help reduce the number of core instances for platform clusters; - Support the deprecation of legacy index builders used for experiment analysis.MUST HAVES -4+ years experience in software development ; -Bachelor’s degree in Computer Science or equivalent practical experience ; -Significant practical experience with Java (4+ years) ; -Practical experience implementing Apache Spark jobs , including partitioning, grouping, joins, importing data into the cluster, and exporting data from the cluster; -Practical experience working with AWS, specifically AWS EMR(or ability to pick it up fast); -Upper-intermediate English level .NICE TO HAVES - Basic knowledge of Kubernetes; - Experience with Spark Operator; - Experience with Airflow; - Experience with Scala.PERKS AND BENEFITS -Professional growth:Mentorship, TechTalks, and personalized growth roadmaps. -Competitive compensation:USD-based pay with education, fitness, and team activity budgets. -Exciting projects:Modern solutions with Fortune 500 and top product companies. -Flextime:Flexible schedule with remote and office options.