Job Title

Candidates ready to join immediately can share their details via email for quick processing.CCTC | ECTC | Notice Period | Location Preferencenitin.patil@Act fast for immediate attention! ⏳Key Responsibilities- Design, develop, and maintain scalable data pipelines using PySpark - Build and manage batch and real-time data processing systems - Develop and integrate Kafka-based streaming solutions - Optimize Spark jobs for performance, cost, and scalability - Work with cloud-native services to deploy and manage data solutions - Ensure data quality, reliability, and security across platforms - Collaborate with data scientists, analysts, and application teams - Participate in code reviews, design discussions, and production supportMust-Have Skills- Strong hands-on experience with PySpark / Apache Spark - Solid understanding of distributed data processing concepts - Experience with Apache Kafka (producers, consumers, topics, partitions) - Hands-on experience with any one cloud platform: - AWS (S3, EMR, Glue, EC2, IAM) or - Azure (ADLS, Synapse, Databricks) or - GCP (GCS, Dataproc, BigQuery) - Proficiency in Python - Strong experience with SQL and data modeling - Experience working with large-scale datasets - Familiarity with Linux/Unix environments - Understanding of ETL/ELT frameworks - Experience with CI/CD pipelines for data applicationsGood-to-Have Skills- Experience with Spark Structured Streaming - Knowledge of Kafka Connect and Kafka Streams - Exposure to Databricks - Experience with NoSQL databases (Cassandra, MongoDB, HBase) - Familiarity with workflow orchestration tools (Airflow, Oozie) - Knowledge of containerization (Docker, Kubernetes) - Experience with data lake architectures - Understanding of security, governance, and compliance in cloud environments - Exposure to Scala or Java is a plus - Prior experience in Agile/Scrum environments