We are seeking an Azure Data Engineer with strong, hands-on Microsoft Fabric experience to build and operate a scalable middle-layer data and integration platform. The role focuses on ingesting data from diverse sources, implementing transformation and business rules, and delivering curated, consumption-ready datasets/models for analytics and downstream applications. Key Responsibilities Fabric Implementation Work on the fabrics platform to design and implement robust data solutions, including One Lake architecture for efficient data storage and processing. Build & Optimize Data Pipelines Design, develop, and maintain scalable ingestion and transformation pipelines using Microsoft Fabric (Data Factory in Fabric / Pipelines), ADF/Synapse Pipelines, OneLake storage patterns, PySpark, Python, and SQL across structured and unstructured data. Develop pipelines that ingest data from external APIs on a scheduled basis and initiate end-to-end downstream processing, supporting one or multiple daily runs through to curated and consumption-ready layers. Data Ingestion & Integration Integrate data from cloud and on-prem sources including databases, third-party systems, files, and REST/SOAP APIs (auth, throttling, pagination, retries, and error handling). Build curated layers and consumption-ready models; implement incremental and batch processing logic; apply data modelling and transformation best practices aligned to reporting/analytics needs. SQL Development & Tuning Develop and optimize complex queries, stored procedures, views, and datasets for efficient analytics and reporting; partner with analytics teams to meet performance SLAs. Performance Tuning & Cost Optimization Tune Spark jobs, ADF data flows and SQL workloads (partitioning, caching, parallelism, cluster sizing/configs) to improve reliability and reduce runtime/cost. Business Logic Implementation Translate requirements into scalable rules (validation, eligibility, availability calculations), manage exceptions, audit logging, and ensure data consistency across systems. Data Quality & Validation Implement automated data quality checks, validation frameworks, reconciliation, and monitoring to ensure trusted datasets. Security & Compliance Implement secure access via Azure AD, Managed Identities, RBAC, least privilege, and secure connectivity to data lake, Fabric/Synapse, and APIs. Automation & CI/CD Build deployment automation using Azure DevOps/Git, promoting code across environments with consistent release practices; support testing and release activities. Monitoring & Troubleshooting Monitor pipelines and jobs using Spark UI and Azure Log Analytics; triage failures, perform root-cause analysis, and improve resiliency/runbooks. Work closely with architects, platform/DevOps engineers, analysts, and data scientists; participate in design sessions and code reviews; operate within Agile/Scrum delivery. Tools & Technologies Fabric Microsoft Fabric Workspaces OneLake Fabric Pipelines / Data Factory in Fabric Lakehouse/Warehouse (as applicable) ADLS Gen2 Blob Storage Synapse Analytics App Service (as needed) Languages PySpark Python SQL (T-SQL) DevOps Git Terraform (preferred) Monitoring Spark UI Data Governance AI Tools Role Experience Experience: 5+ years Role Level: Individual Contributor (Senior Engineer) About Symbiotic Digital. Symbiotic Digital provides the IT Experts you need that solve problems and get things done. See what the top 2% in their field can do for you. (Symbiotic Digital is a division of Symbiotic Group Inc.) We serve customers in two ways, we provide: 1) Digital Experts: IT Consulting expertise. Bright Minds That Produce Proven Results 2) Recruitment: Find The Right People You Need Serving IT & Business leaders, Symbiotic Digital provides a business system that enable us to provide the Expert Technical staff needed to solve challenging business and IT problems. IT leaders in the most reputable organizations in Canada have relied on Symbiotic Digital to help them staff difficult assignments with proven Experts uniquely evaluated by our proven QMS- Quality Management System Staff Development Model. We are a 100% Indigenous owned company. #J-18808-Ljbffr
Job Title
Azure Data Engineer