Job Title

Our client is seeking a Senior PySpark Developer with strong experience in building and supporting scalable data pipelines using PySpark, Python, and AWS cloud services. This role will involve collaborating closely with business and technical teams to design, develop, test, and support data solutions that align with business needs and established quality frameworks. Key Responsibilities Analyze and understand systems and business processes related to assigned data products. Collaborate with business partners to develop, maintain, and optimize data solutions using Python, PySpark, and related tools. Write clean, high-quality, and maintainable code while participating in peer code reviews. Develop, enhance, and execute automated tests to improve coverage and reduce manual effort. Ensure testing and release activities adhere to the Client's quality assurance and change management standards. Create, maintain, and update technical documentation including product design and ETL mapping. Prepare project-related reports, documentation, and presentations as needed. Provide operational support, troubleshooting, and enhancements to existing solutions. Must-Have Skills & Experience Extensive experience building scalable PySpark data pipelines. Strong expertise in Python and SQL programming. Hands-on experience with DevOps practices, automated testing, and CI/CD pipelines. Ability to manage multiple priorities in a fast-paced environment. Strong analytical skills with the ability to translate business requirements into technical solutions. Excellent problem-solving skills with a proactive, collaborative approach. Strong communication skills in English (written and verbal). Technical Requirements Advanced Python & SQL skills; ability to write modular and maintainable code. Strong knowledge of MySQL or similar relational database technologies. Expertise with Spark DataFrames, Spark SQL, and PySpark for distributed data processing. Proficiency with Git and development best practices. Experience with Apache Iceberg (open table format). Experience using Airflow or similar orchestration tools. Hands-on experience developing in an AWS environment, including: Amazon Redshift Amazon EMR AWS Lambda AWS Cloud Development Kit (CDK) for Python