Skip to Main Content

Job Title


Lead Databricks Engineer


Company : Newscape Consulting


Location : Pune, Maharashtra


Created : 2025-12-18


Job Type : Full Time


Job Description

We’re Hiring: Senior or Lead Databricks DeveloperLocation: Baner, Pune (Onsite, 5 days a week)Work Mode: Full-time | Office-based⏳ Availability: Immediate Joiners / Notice Period ≤ 30 Days⚠️ Note: Only local Pune candidates may apply.* L2 interviews will be conducted in person — apply only if you are open to face-to-face interviews.About Newscape Consulting-Newscape is a fast-growing digital services company specializing in healthcare technology and data engineering. We help healthcare organizations modernize legacy systems, integrate cloud-native platforms, and drive digital transformation through data.If you’re passionate about solving complex data challenges and contributing to meaningful innovation in healthcare, this is the place for you.Position Summary -This role requires a seasoned Databricks professional with strong technical depth and leadership capability. You will design, architect, and optimize data pipelines using Databricks, Apache Spark, Delta Lake, and cloud-native services (Azure, AWS, or GCP). As a Senior or Lead engineer, you will guide best practices, mentor junior engineers, and collaborate closely with data architects, data scientists, and business stakeholders.Key Responsibilities- Architect, design, and maintain large-scale data pipelines using Databricks, PySpark, Spark SQL, and optionally Scala. - Lead the adoption and implementation of Databricks components including Workspace, Jobs, DLT (Delta Live Tables), Repos, and Unity Catalog. - Build and optimize Delta Lake solutions aligned with Lakehouse and Medallion architecture standards. - Partner with architects, engineering teams, and business stakeholders to translate complex business requirements into robust technical solutions. - Establish coding standards, best practices, and reusable components for Databricks-based development. - Drive CI/CD automation for Databricks deployments using Azure DevOps, GitHub Actions, or equivalent tools. - Ensure end-to-end data governance, data quality, metadata management, and lineage tracking leveraging Unity Catalog or Azure Purview. - Utilize orchestration tools such as Apache Airflow, Azure Data Factory, or Databricks Workflows for scheduling and monitoring. - Conduct performance tuning for Spark jobs, cluster optimization, and cost-efficient compute strategies. - Guide data modeling, SQL transformations, and data warehousing implementation using industry-standard best practices. - Mentor and provide technical leadership to junior and mid-level data engineers. - Participate in architecture reviews, solution design discussions, and roadmap planning.Required Skills and Qualifications- Strong hands-on experience in data engineering with specific and extensive expertise in Databricks. - Deep proficiency with Databricks Workspace, Jobs, DLT, Repos, Unity Catalog, and Lakehouse architecture. - Advanced programming experience with PySpark and Spark SQL; Scala is an added advantage. - Strong experience with at least one major cloud platform: Azure, AWS, or GCP. - Proven expertise with Delta Lake, Lakehouse, and Medallion architecture patterns. - Hands-on experience implementing CI/CD pipelines for Databricks using DevOps tooling. - Familiarity with orchestration tools such as Airflow, ADF, or Databricks Workflows. - Strong understanding of data governance, security frameworks, and enterprise data management. - Experience integrating Databricks datasets with Power BI, including creating optimized data models, implementing Direct Lake or DirectQuery connections, and enabling performant, enterprise-grade reporting solutions. - Experience leading technical teams or acting as a senior SME in critical data projects. - Excellent analytical abilities, communication skills, and stakeholder management capability. - Prior experience in healthcare data environments is a plus (not mandatory).Work Model- This is a full-time, onsite role based in Baner, Pune. - 5 days onsite per week is mandatory.Ready to make an impact?Apply now or DM us if you’re a Pune-based data enthusiast ready for your next big challenge!How to Apply:Send your resume toswapnil.supe@Subject Line: Application – Azure Databricks Developer+91 8233829595