Job Title:Azure Databricks Engineer – Unity Catalog Migration (Offshore Contract) Location:Remote (Offshore) Contract Duration:6+ Months Work Hours:7.30 PM to 4.30 AM IST.Job Overview: We are seeking a highly skilled Azure Databricks Engineer with proven experience in migrating to Unity Catalog. This contract role focuses on leading and executing the end-to-end transition from a legacy Hive metastore to Unity Catalog, supporting multiple departments and business units. You will play a key role in architecting, migrating, securing, and optimizing data and metadata assets within Azure Databricks environments.Key Responsibilities: Lead the migration from Hive metastore to Unity Catalog, including planning, execution, and validation. Design and implement the 3-part namespace architecture across Dev, Test, and Prod environments. Migrate managed/external tables, notebooks, jobs, and related dependencies (including reporting assets). Leverage tools like the UCX Assessment Tool or manual methods for efficient data catalog migration. Configure and manage Unity Catalog security, including schema-level permissions, security groups, and access control. Collaborate closely with business stakeholders to define catalog and schema structures based on their domains or business units. Implement workspace-per-department architecture to support scalable self-service analytics. Integrate Databricks with other Azure services such as Power BI (Fabric), Azure Data Factory, and Data Mirroring.Required Skills & Experience: Strong experience in Azure Databricks and Unity Catalog migration projects. Proficient in Python and PySpark for data engineering and automation.In-depth understanding of: Unity Catalog Metastore and architecture. Migration of notebooks, jobs, and workflows. Security design in Unity Catalog (schemas, groups, access policies). Familiarity with the UCX Assessment Tool or manual migration methodologies. Experience working across multiple business units or data domains. Strong communication skills and ability to coordinate with offshore/onshore teams.
Job Title
Azure Data Engineer