About the Job We are seeking a skilled Data Engineer to architect, build, and optimise scalable data platforms on cloud infrastructure. The role involves close collaboration with cross-functional teams to deliver robust, secure, and high-performance data solutions that support analytics and business operations. Core Responsibilities - Design, develop, and maintain scalable ETL/ELT pipelines, data lakes, and data warehouse solutions. - Build and optimize data ingestion frameworks for batch and real-time processing. - Develop and deploy containerised applications using Docker, Amazon ECR, and Amazon ECS. - Design and implement RESTful APIs for system integrations using modern frameworks (e.g., FastAPI). - Implement Infrastructure as Code (IaC) using Terraform and AWS CloudFormation. - Establish and maintain CI/CD pipelines for automated build, test, and deployment workflows. - Ensure adherence to data security, governance, and compliance standards (e.g., encryption, access control). - Monitor, troubleshoot, and optimise data workflows for performance and reliability. Required Skills - Strong proficiency in Python and SQL for data processing and transformation. - Hands-on experience with FastAPI for API development. - Experience with distributed data processing frameworks such as PySpark. - Solid experience with AWS services, including: Amazon S3, AWS Glue, Amazon Redshift, Amazon Athena AWS Lambda, AWS DMS, API Gateway - Experience with containerization and orchestration (Docker, ECS). - Strong understanding of cloud-native architecture and best practices. - Excellent problem-solving, communication, and collaboration skills. Nice to have - Exposure to Generative AI and Agentic AI concepts. - Hands-on experience with LLM frameworks such as LangChain, LlamaIndex, or CrewAI. - Experience working with LLMs and NLP models (e.g., GPT, BERT, LLaMA, Mistral, Gemini). - Familiarity with LLM-as-a-Service platforms such as AWS Bedrock or Hugging Face. - Basic understanding of ML model deployment and lifecycle management. Qualifications - Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field. - 3–5 years of hands-on experience in Data Engineering and AWS cloud environments.
Job Title
AWS Data Engineer