Hourly rate or salaried option availablePeoplebank are recruiting for a Senior Data Engineer to work in Adelaide on a 12 month contract (with high likelihood of extension).In this NEWLY created role, the Data Engineer will be responsible for designing and implementing a highly scalable and secure, integrated data platform for this business-critical system that is the first stages of its multi-year iterative design, build and implementation. The data architecture will enable the collection, transformation and processing of large volumes of structured and unstructured data from disparate dataset to form mastered golden records.This role will suit a problem-solving individual who is passionate about driving process improvements, such as the promotion of reusable code, automation and DevOps practices (e.g. CI/CD), and who possesses the ability to apply the right balance of agility and predictability when planning to ensure timelines are met and deliverables are consistently maintained to a high-quality standard.Key Skills and Experience will include (but not be limited to):10+ years'' experience as a Data Engineer designing, developing and deploying domain-specific data solutions with high volume, complex disparate datasets on cloud platforms, preferably Azure.Demonstrated experience designing production, operational data solutions that underpin business critical applications.Experience developing data pipelines that meet availability, latency, and reliability requirements for business-critical processes.Proven ability to independently identify and translate complex requirements from a variety of sources (e.g. functional and non-functional requirements, user stories, etc.) into technical data solution designs.Proven experience implementing data quality controls, validation rules, reconciliation processes, and exception handling across different layers.Experience designing auditability, traceability, and reprocessing mechanisms to support operational assurance and compliance.Experience implementing data security controls, including role-based access control (RBAC).Proven experience designing, implementing and maintaining scalable ETL/ELT pipelines with complex transformations across Azure Data Factory, Databricks and SQL services that includes ingestion data from disparate datasets and sources (e.g. Databricks, SQL, API) and with various file formats (e.g. CSV, JSON, Excel, Parquet).Advanced SQL, Python, Apache Spark, Databricks and Azure service engineering skills.Extensive understanding of data lake and lake warehouse protocols, schema evolution and data-quality frameworks.Experience creating data models, including those associated with software/application design, administration functionality, data process (e.g. data mastery/MDM, data lake) and presentation in reports and dashboards.Experience with Azure DevOps (e.g. repositories, branching strategies, CI/CD, etc.).Understanding of Australian government data governance, security and compliance standards, policies and industry best practice.This is a unique opportunity to work on a meaningful project, where you will get the chance to create something special!So don''t delay, APPLY NOW, or call Jerry Belialba on 08 8112 7406 for a confidential chat about the role.Peoplebank and Leaders IT are committed to creating a diverse and inclusive workplace where everyone belongs. We welcome applications from people of all backgrounds, identities, and experiences. If you need adjustments to the recruitment process due to your circumstances, please let us knowwe''re here to support you.Unlock job insightsHirer responsiveness Salary match Number of applicants #J-18808-Ljbffr
Job Title
Senior Data Engineer