About the Role:Join our data engineering team to build and maintain large-scale data pipelines that power analytics across various products. In this role, you will process a large amount of data to deliver actionable insights for product teams and executives.What You'll Do:● Develop Apache Airflow DAGs and PySpark ETL pipelines for high volume data processing.● Write optimized SQL queries for data transformation and aggregation.● Build data products serving Business Process, Executive KPIs, and Product Analytics.● Implement data quality and monitoring solutions.● Optimize pipeline performance and troubleshoot production issues.● Collaborate with cross-functional teams.● Production Pipeline Monitoring (KLO).Qualifications:Required Skills● 10+ years of data engineering experience with a minimum of 7 years dedicated to the Big data stack.● Expert in Python and PySpark (DataFrame API, Spark SQL).● Advanced SQL skills (window functions, complex queries).● Production experience with Apache Airflow.● Solid background in data warehousing and dimensional modelling.Preferred Skills● Experience with SQL, Trino, Apache Iceberg.● Knowledge of Tableau CRM/CLOUD, Salesforce platforms.● AWS/cloud data services experience.
Job Title
Lead Data Engineer - PySpark