Job Title

Job Title: Data Engineer Experience: 5-7 years Relevant Experience – 5 years Work location: Bangalore/ Pune/Hyderabad/Chennai/Noida Job Description Data engineers design, build, and maintain the infrastructure for an organization's data, ensuring it is reliable, accessible, and optimized for use by data scientists and analysts. Key responsibilities include creating and managing data pipelines (ETL processes), optimizing data storage and processing, ensuring data quality and security, and collaborating with other teams to meet data needs. Responsibilities Design and build data infrastructure: Create and maintain scalable data pipelines, databases, and data warehouses to handle large volumes of data from various sources. Ensure data quality and reliability: Implement data cleansing, validation, and quality checks to ensure the data is accurate and trustworthy. Optimize data systems: Monitor and tune performance of databases and data processing tasks to ensure efficiency and speed. Collaborate with stakeholders: Work with data scientists, analysts, and other teams to understand their data requirements and make data accessible for their use. Key skills Data Platform Project Autonomous level people Skillsets needed (Mandatory) Pyspark, Spark SQL - Databricks Data Governance – Unity CatalogPython – Pandera Framework REST API Frameworks Nice to have Cloud platforms: Experience with cloud services (e.g. GCP Services - Cloud build, BigQuery, Firestore etc… Airbyte/Fivetran Nutter Framework – Experience in writing unit test in Nutter Framework Terraform Mandatory. CI/CD Concepts OAuth 2 Protocol Analytical and problem-solving skills: Strong analytical abilities are needed to troubleshoot issues and find solutions.