Job Title

Job Purpose:We are seeking a talented and experienced Data Engineer with expertise in both Azure and AWS cloud platforms to join our team. The ideal candidate will have experience in designing, building, and maintaining data solutions on both Azure and AWS cloud platforms. You will work closely with cross-functional teams to understand business requirements and translate them into scalable and efficient data pipelines and architectures across both ecosystems.Responsibilities:Design, develop, and deploy data solutions on the Azure cloud platform, leveraging services such as Azure Data Factory, Azure Databricks, Azure SQL Database, and Azure Synapse Analytics.Design, develop, and deploy data solutions on the AWS cloud platform, utilizing services such as AWS Glue, Amazon Redshift, AWS Lambda, Amazon S3, and Amazon Athena.Collaborate with analysts and other stakeholders to understand business requirements and translate them into technical solutions.Build and maintain scalable data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data across Azure and AWS environments.Optimize data storage, retrieval, and processing techniques to ensure high performance and cost efficiency in both Azure and AWS ecosystems.Monitor and troubleshoot data pipelines to identify and resolve performance issues and bottlenecks across both cloud platforms.Document data solutions, including architecture diagrams, technical specifications, and operational procedures for Azure and AWS-based solutions.Collaborate with cross-functional teams to understand business requirements and translate them into solutions, ensuring designs align with business needs in both Azure and AWS contexts.Be involved in analysis and design phases of a project, participate in application design sessions, and make solution recommendations for hybrid or multi-cloud architectures.Work cooperatively in a team environment and remain flexible on changing workloads.Implement and maintain CI/CD pipelines for data solutions using Azure DevOps and AWS CodePipeline/CodeBuild for seamless deployment and integration.Qualifications:BE/BTech/MCA degree in computer science or related field.Minimum 5-10 years of experience in data engineering, with a focus on designing and building data solutions on both Azure and AWS cloud platforms.Hands-on experience with Azure services such as Azure Data Factory, Azure Databricks, Azure SQL Database, Azure Synapse Analytics, and Azure Data Lake.Hands-on experience with AWS services such as AWS Glue, Amazon Redshift, AWS Lambda, Amazon S3, Amazon Athena, and AWS Data Pipeline.Experience with programming languages such as C#, Python for data processing and automation in both Azure and AWS environments.Experience using DBT (Data Build Tool) for building and managing data pipelines and transformations in a data warehouse environment.Familiarity with data warehousing concepts, dimensional modeling, data modeling, and data integration patterns ETL/ELT processes.Knowledge in Data migration to Dynamics CRM using SSIS Kingsway Soft, ADF, or AWS Database Migration Service (DMS).Proficiency in SQL and experience working with relational databases such as SQL Server, MySQL, Amazon RDS, or Aurora, and non-relational databases such as Cosmos DB or Amazon DynamoDB.Hands-on experience with SSIS components such as Control Flow tasks, Data Flow transformations, Connection Managers, and Event Handlers.Strong communication and collaboration skills, with the ability to work effectively in a cross-functional team environment.Proficient in Azure DevOps (ADO) practices (Boards, Git Repos, CI/CD pipelines) and AWS DevOps tools (CodeCommit, CodePipeline, CodeBuild).Demonstrated analytical, problem-solving, organizational, and interpersonal communication skills.Preferred Qualifications:Microsoft Azure Data Engineer Associate, AWS Certified Data Analytics – Specialty, MCSD, or Power BI Analyst certification.Knowledge of Automobile, Online Auctions, Classifieds industry.Experience with multi-cloud or hybrid cloud architectures, integrating Azure and AWS services for optimized data solutions.