Job Title

Lead Engineer-Azure Location: India, Noida(Hybrid) Experience: 5-8 Years Algoworks About the Company We are a global team of engineers, architects, designers, researchers, operators and innovators who share a passion for achieving client goals. Our engineering services help businesses thrive at the intersection of technology and people. From the latest AI implementations to legacy platform migrations and everything in between, our services span the enterprise technology spectrum. Our world class experience transformation playbook elevates digital success and increases ROI with a relentless focus on the human experience. Our customer base includes Fortune 500 companies around the globe. We’ve got the skills and insights and we’re also fun to work with. Our global team spans a diverse cultural spectrum, with wide ranging interests, enabling us to bring personality and depth to every engagement. Follow the video below to know about US! Clipchamp Role Overview The Senior Data Engineer will be responsible for building, optimizing, and maintaining scalable data pipelines across Bronze, Silver, and curated layers. This role focuses on high-performance ETL/ELT development, Delta Lake optimization, and close collaboration with DWH teams for downstream consumption. The engineer will act as a senior contributor, ensuring reliability, performance, and data quality across the platform. Responsibilities Pipeline Development Experience Build and maintain data pipelines using Azure Databricks and Azure Data Factory. Implement ingestion and transformation logic across bronce and silver layers. Strong hands-on experience building pipelines on Azure. Curated Layer Logic Implement hydration, merge and upsert logic using Delta Lake. Ensure Curated datasets meet data quality and business requirements. Handle late arriving data and incremental updates. Performance & Storage Optimization Optimize Delta Lake tables for performance and cost. Select and tune appropriate storage formats (Parquet / Delta). Apply partitioning, compaction, and file sizing strategies. Tune Spark jobs for large-scale data processing. Downstream & DWH Collaboration Work closely with DWH and BI teams to support downstream consumption. Provide optimized datasets for Synapse and reporting workloads. Support data validation and reconciliation with gold layer outputs. Engineering Best Practices Implement basic CI/CD practices for data pipelines. Follow coding standards, documentation, and version control practices. Support production troubleshooting and performance tuning. Required Skills & Experiences Bachelor’s or Master’s degree in Computer Science or related field. Experience 5–8 years of experience in data engineering. Strong hands-on experience building pipelines on Azure. Technical Skills Strong proficiency in PySpark. Hands-on experience with Azure Databricks and Data Factory. Deep knowledge of Delta Lake tuning and optimization. Experience with storage optimization (Parquet, Delta). Strong SQL skills for transformation and validation. Tools & Practices Experience with Git and basic CI/CD pipelines. Familiarity with data quality and validation techniques. Experience working in Agile delivery models. Soft Skills Strong analytical and problem-solving skills. Ability to work independently on complex pipelines. Good communication and collaboration skills. Nice to have Skills Experience supporting Synapse Dedicated SQL Pool. Exposure to streaming or near real-time pipelines. Familiarity with data governance or metadata tools. Interview Process 2 Rounds of Discussion