Job Title

Role: Senior Data Engineer - AzureExperience Level: 4 to 6 YearsWork location: Mumbai, Bangalore, Trivandrum (Hybrid)Notice Perid: 0-30 daysKey Responsibilities:● Design and build scalable ETL/ELT pipelines using Azure Data Factory (ADF), Azure Databricks (Spark), and Azure Synapse Analytics.● Mandatory hands-on experience with Microsoft Azure and Databricks.● Understanding of the Medallion Architecture (specifically the Bronze/Raw layer) and data modeling concepts for data lakes.● Experience building robust Orchestration pipelines (scheduling, error handling, logging) and managing full/incremental data loads.● Knowledge of Delta format best practices, including partitioning, Z-ordering, vacuuming, and storage tiering for cost efficiency.● Familiarity with CI/CD pipelines, automated deployment scripts, version control (Git), and Unit/Integration testing frameworks.● Experience implementing Data Quality rules, PII handling/masking, and metadata management.● Develop and optimize PySpark/Spark SQL jobs for large-scale batch and streaming data transformations.● Ingest data from various sources including Apache Kafka, REST APIs, and RDBMS, ensuring real-time or near-real-time availability.● Implement data modeling strategies (star schema, snowflake schema) for analytics consumption layers in Synapse or ADLS.● Collaborate with DevOps teams to automate deployment using CI/CD pipelines (Azure DevOps, GitHub Actions, etc.).● Monitor, troubleshoot, and optimize data workflows for performance, cost-efficiency, and reliability.● Follow coding standards, participate in peer reviews, and maintain version-controlled code in Git repositories.● Support data quality checks, logging, alerting, and observability mechanisms for production workloads.● Participate in sprint ceremonies and contribute to task estimation and delivery planning.Must have skills:● 3+ years of experience in data engineering roles.● Hands-on experience with:○ Azure Data Factory (ADF) – building pipelines, triggers, linked services.○ Azure Databricks – building and managing Spark jobs in PySpark.○ Azure Synapse Analytics – data warehousing, SQL queries, workspace orchestration.○ Apache Kafka – consuming and processing real-time data streams.● Strong in SQL, Python, and Spark for data manipulation and transformation.● Exposure to CI/CD practices (Azure DevOps, Git workflows, build/release pipelines).● Understanding of data lake architecture and modern data warehousing principles.