Lead the migration from Glue/Redshift to Databricks10+ years of professional data engineering experience, with at least 2+ years in a tech-lead or lead engineer capacity.5-6 years of experience in each of the following areas:Deep, hands-on experience with Apache Spark (PySpark and Spark SQL), including performance tuning (partitioning, shuffles, skew, caching, file sizing).Hands-on production experience with Databricks Delta Lake, Unity Catalog, Workflows, and either Delta Live Tables or a comparable declarative pipeline framework.Strong production experience with AWS Glue (jobs, crawlers, Data Catalog, triggers) and Amazon Redshift(including Redshift Spectrum, workload management, distribution and sort keys, performance tuning, and Redshift SQL / stored procedures).Proven experience leading at least one substantial data platform migration or re-platforming effort end-to-end.Expert SQL skills and strong software-engineering fundamentals: Python, version control, testing, and CI/CD.Deep knowledge of AWS data services (S3, IAM, KMS, VPC, Lambda, Step Functions, MSK/Kinesis, CloudWatch).Strong understanding of data modeling for analytics (dimensional, Data Vault, or medallion patterns) and lakehouse architecture.Excellent communication skills; able to translate between executive stakeholders and engineers, and to write clear design docs and migration plans.
Job Title
Data Engineer - Minimum 12 years experience