Job Title

K&K Global Talents is an international recruiting agency that has been providing technical resources globally since 1993. This position is with one of our clients in India who is actively hiring candidates to expand their teams.Title: Senior Big Data Engineer with Azure Locations: Bhopal, Bangalore, Chennai, Gurugram, Jaipur, PuneEmployment Type: Contract to HireRole:We are looking for a highly skilled Senior Big Data Engineer with 8+ years of overall IT experience and strong expertise in Azure, Databricks, DevOps, and large-scale data engineering. The ideal candidate will have deep experience building scalable data pipelines, optimizing Spark workloads, managing cloud infrastructure, and contributing to robust DevOps practices.Responsibilities:Big Data EngineeringDesign, develop, and optimize Spark/Scala-based data pipelines for large-scale datasetsBuild robust ETL/ELT pipelines across diverse data formats (CSV, JSON, XML, etc.)Perform data profiling, analysis, and mining to support business insightsDevelop scalable data solutions for onboarding new data sources and enabling analytics use casesAzure & Cloud EngineeringWork extensively with Azure ecosystem including Data Lake, Databricks, Blob Storage, Azure SQL, Functions, and VMsEnsure efficient cloud architecture with focus on performance, scalability, and cost optimizationManage cloud-level configurations including networking, monitoring, and identity accessDatabricks & Data GovernanceConfigure and manage Databricks environments (clusters, workspaces, CI/CD integration)Implement Unity Catalog for data governance, lineage, access control, and auditingMaintain structured and governed data platforms across environmentsDevOps & AutomationBuild and maintain Azure DevOps (ADO) YAML pipelinesImplement CI/CD workflows for data platforms and pipelinesUse Terraform for infrastructure automation and multi-stage deploymentsWork with Git & Jenkins for version control and automationRequired Skills:Strong experience with Azure Databricks (Infra + Development)Hands-on expertise in Unity Catalog (Data Governance)Proficiency in Spark / ScalaExperience with Azure Cloud Services (Storage, Compute, Networking, PaaS)Strong working knowledge of Azure DevOps (YAML pipelines)Expertise in Terraform (Infrastructure as Code)Experience with CI/CD, GitOps, and JenkinsStrong debugging, troubleshooting, and performance tuning skillsPreferred Skills:Experience with Cosmos DB / DynamoDB / Azure SQLKnowledge of Shell/Bash scriptingExposure to distributed data systems and large-scale architectures