Job Title

Job Summary You will play a pivotal role in designing and implementing cutting"‘edge data solutions using technologies like PySpark, BigQuery and Apache Spark. With a focus on cloud"‘based platforms such as Cloud Dataproc and Cloud Dataflow you will ensure seamless data processing and analytics. Your expertise in Python will be crucial in developing scalable and efficient data architectures. This hybrid role offers the flexibility of working both remotely and on"‘site during day shifts wit Responsibilities Lead the design and development of data architectures using PySpark and Apache Spark to optimize data processing workflows. Oversee the integration of BigQuery for efficient data storage and retrieval ensuring high performance and scalability. Provide technical guidance in utilizing Cloud Dataproc and Cloud Dataflow for seamless cloud"‘based data processing solutions. Collaborate with cross"‘functional teams to identify data requirements and translate them into actionable solutions. Develop Python scripts to automate data processing tasks enhancing efficiency and reducing manual intervention. Ensure data security and compliance with industry standards safeguarding sensitive information across all platforms. Implement best practices for data governance and management maintaining data integrity and accuracy. Monitor and troubleshoot data pipelines proactively addressing any issues to minimize downtime and ensure continuity. Optimize data workflows to improve processing speed and resource utilization contributing to cost"‘effective solutions. Innovate and propose new methodologies for data analytics driving insights that support business objectives. Coordinate with stakeholders to understand business needs and align data solutions with organizational goals. Evaluate emerging technologies and tools recommending adoption to enhance data capabilities and performance. Document architectural designs and processes providing clear guidelines for future development and maintenance. Qualifications Possess extensive experience in PySpark and Apache Spark demonstrating proficiency in data processing and analytics. Have a strong background in BigQuery showcasing ability to manage large datasets with efficiency. Demonstrate expertise in cloud platforms like Cloud Dataproc and Cloud Dataflow ensuring seamless integration and operation. Exhibit proficiency in Python programming with a focus on developing scalable and automated solutions. Show understanding of data security protocols and compliance standards ensuring protection of sensitive information. Display knowledge of data governance practices maintaining high standards of data integrity and accuracy. Have experience in troubleshooting and optimizing data pipelines ensuring minimal downtime and maximum efficiency. #J-18808-Ljbffr