Position Name : Senior Data Engineer – DatabricksLocation : Pune / Hyderabad / BengaluruExperience : Min 6 to 10 yearsAbout Jade GlobalJade Global is a premier global IT services and solutions firm committed to helping clients drive business transformation through enterprise applications, AI/ML, cloud platforms, automation, and managed services. As a trusted partner for industry leaders across Hi-Tech, Healthcare, Manufacturing, and Financial Services, we bring deep domain knowledge, agility, and a relentless focus on client success.About PositionWe are seeking a highly skilled and experienced Senior Data Engineer – Databricks to join our Data Engineering team. The ideal candidate will have strong hands-on expertise in Databricks, Apache Spark, Delta Lake, and cloud-based data platforms. You will be responsible for building scalable, high-performance data pipelines and enabling advanced analytics to support data-driven decision-making across the organization.Key Responsibilities:- Design, develop, and implement scalable ETL/ELT data pipelines using Apache Spark on Databricks. - Build, optimize, and manage data lakes and data warehouses using Delta Lake and related technologies. - Develop, schedule, and monitor workflows using Databricks Workflows, Airflow, or other orchestration tools. - Collaborate with data scientists, analysts, and business stakeholders to understand requirements and deliver robust data solutions. - Tune and optimize Spark jobs for performance, scalability, and cost efficiency. - Implement CI/CD pipelines and follow DevOps best practices for data engineering projects. - Ensure data security, governance, and compliance using tools such as Unity Catalog. - Troubleshoot and resolve issues related to data ingestion, transformation, and integration. - Participate in data platform migration initiatives, including migrations from: - On-premise or Hadoop platforms - Legacy ETL tools - Other cloud or data warehouse platforms to Databricks - Mentor junior engineers and contribute to technical excellence, best practices, and innovation within the team.Required Skills & Experience:- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field. - 6+ years of hands-on experience in Data Engineering / Big Data development. - 3+ years of strong experience with Databricks, including Apache Spark, Delta Lake, and MLflow. - Strong programming skills in Python and SQL. - Solid understanding of distributed systems and Spark performance tuning. - Hands-on experience with cloud platforms: Azure, AWS, or GCP (Azure Databricks preferred). - Experience working with structured, semi-structured, and unstructured data. - Familiarity with DevOps practices, including: - Git-based version control - CI/CD pipelines - Infrastructure as Code (Terraform, ARM templates) - Strong problem-solving abilities and capability to explain complex technical concepts to non-technical stakeholders. - Proven experience in Databricks migration projects is highly desirable.Behavioral & Soft Skills:- Ability to thrive in a fast-paced, client-facing environment with high delivery expectations. - Strong leadership and interpersonal skills. - Eager to contribute to a collaborative, team-oriented culture. - Excellent prioritization and multitasking skills with a proven track record of meeting deadlines. - Creative and analytical approach to problem-solving. - Strong verbal and written communication skills. - Adaptable to new technologies, environments, and processes. - Comfortable managing ambiguity and solving undefined problems.Why Jade Global?- Award-winning enterprise partner (Oracle, Salesforce, ServiceNow, Boomi, and more) - Diverse global culture with over 2,000 employees - High-growth environment with opportunities for innovation and leadership - Hybrid work flexibility and inclusive leadership culture
Job Title
Senior Data Engineer – Databricks