Job Title

Location- Bangalore Experience- 5-8 Yrs Important Note: We are looking for immediate joiners who are based in Bangalore. Additional Benefit: A special Immediate Joiner Bonus will be offered to candidates who can join without delay. About Scientist Technologies: At Scientist Technologies, we believe in driving global progress through scientific innovation, engineering expertise, and policy collaboration. Our mission is to create solutions that empower businesses, societies, and public institutions to tackle the most pressing challenges of our time. We offer a dynamic, collaborative environment where your expertise contributes to groundbreaking projects that weave together science, business, and policy for equitable human progress. Overview: We are looking for a skilled and passionate Senior Data Engineer(Scala) with 5–8 years of experience in designing and implementing scalable big data solutions. The ideal candidate should have hands-on experience with Spark (preferably using Scala), a strong understanding of the Hadoop ecosystem, and a deep knowledge of big data processing and ingestion techniques. You will be responsible for designing and implementing robust data solutions for personalization and analytics platforms. Key Responsibilities: Develop and optimize Spark-based data pipelines using Scala. Handle large-scale data ingestion and transformation from various sources. Work with the Hadoop ecosystem including HDFS, Hive, Oozie, and Sqoop. Design and maintain efficient big data workflows, ensuring reliability and performance. Collaborate with cross-functional teams in an Agile environment. Implement streaming solutions using Spark Streaming and Kafka (nice to have). Write and optimize complex SQL queries and shell scripts. Collaborate with data scientists to implement feature engineering solutions for personalization systems Translate data science requirements into scalable production code Optimize algorithms for large-scale distributed processing Implement data quality checks and monitoring systems Key Qualifications: 5–8 years of hands-on experience in big data technologies. Strong command of Spark architecture and components. Proficient in Scala (or Java) and scripting languages like Shell. Experience working with Hadoop tools (HDFS, Hive, Oozie, Sqoop). Familiarity with Kafka and real-time data pipelines is a plus. Strong analytical, problem-solving, and communication skills. Comfortable working independently and as part of a team in Agile setups. Technical Stack: Scala, spark Good understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming, Driver Node, Worker Node, Stages, Executors and Tasks. Should have experienced with Hadoop ecosystem - Hdfs, hive (must), Oozie, Sqoop, etc. Should have prior experience working with big data ingestion from various sources. Handling TB's of data. Excellent knowledge of SQL and coding skills. Good to have Kafka experience as well. Experience with scripting languages such as shell. Effective working in a team as well as working independently. Experience of working in Agile delivery.