Job Title

Your work days are brighter here. Were obsessed with making hard work pay off, for our people, our customers, and the world around us. As a Fortune 500 company and a leading AI platform for managing people, money, and agents, were shaping the future of work so teams can reach their potential and focus on what matters most. The minute you join, youll feel it. Not just in the products we build, but in how we show up for each other. Our culture is rooted in integrity, empathy, and shared enthusiasm. Were in this together, tackling big challenges with bold ideas and genuine care. We look for curious minds and courageous collaborators who bring sun-drenched optimism and drive. Whether you''''re building smarter solutions, supporting customers, or creating a space where everyone belongs, youll do meaningful work with Workmates whove got your back. In return, well give you the trust to take risks, the tools to grow, the skills to develop and the support of a company invested in you for the long haul. So, if you want to inspire a brighter work day for everyone, including yourself, youve found a match in Workday, and we hope to be a match for you too. About the Team This is a very exciting opening in the AI Platform team. We believe if you do what you love, youll love what you do. Theres a lot to love at Workday. We are part of a global, high-growth technology company and our team has the opportunity to develop the next generation of Workdays groundbreaking collaborative products. You will take on sophisticated problems and influence teams across Workday as you build enterprise software. Thrive in our fun, people-first culture that builds an environment focused on your success and ability to do your best work. The Agent Evaluation Platform team is the ''''Ground Truth'''' engine for Workdays AI transformation. As Workday infuses AI Agents into every facet of our enterprise suite, our team provides the critical infrastructure and algorithms needed to prove they workand make them better. We build the platform that enables agent engineering teams to be empowered with rigorous, data-driven optimization, evaluation and validation of their agents. Our roadmap is ambitious: building cloud-based evaluation scaling, real-time online monitoring, and automated optimization loops that fine-tune prompts and drive optimal model selection across complex agentic graphs. Workdays AI Platform organization is bringing AI first products to life at every step of the Workday product offering. Were looking for highly creative, results-focused, and deeply skilled machine learning engineers/scientists to work with us on a range of these challenges. About the Role We are looking for a highly skilled and pragmatic Machine Learning Engineer to work with us on the applied research, development, deployment, and optimization of advanced ML systems and products. You will help to build the future of AI Agent observability, evaluation, and optimization. In this role, you won''''t just be evaluating models; you will be building the systems that automate the evaluation and optimization of thousands of agent configurations. You will solve ''''Evaluation-and-Optimization-at-Scale'''' challengessimulating multi-tenant environments, orchestrating massive Kubeflow pipeline runs, and designing the metrics that define success for autonomous enterprise workflows. You will use Workdays vast computing resources on rich, exclusive datasets to deliver value that transforms the way our customers make decisions and run their businesses. We will challenge you to apply your best creative thinking, analysis, problem-solving, and technical abilities to make an impact on thousands of enterprises and millions of users. Impact: You are the gatekeeper of quality for products reaching 31 million users. Innovation: You''''ll be working at the absolute frontier of Agentic AI - shifting from ''''how do we build an agent'''' to ''''how do we validate, scale, and optimize an agent.'''' Scale: Your work will optimize compute costs and performance across the entire Workday AI portfolio. In this role, you would: Agent Optimization (Meta-ML): Develop algorithms for automated node-level optimization within agent graphsdetermining the best LLM model and prompt configuration for every step of a workflow. Work on enabling lifelong agent learning. Architect Evaluation Pipelines: Design and scale offline evaluation systems using Kubeflow on the cloud to handle massive, distributed test suites for LLM agents. Build recommender systems for engineering teams to drive optimal evaluation for their agents. Work on failure attribution as an outcome of evaluation. Develop Bespoke Metrics: Build a Python-native framework allowing developers to define complex, domainspecific metrics for agent accuracy, safety, and hallucination rates. Enable Online Evaluation and Observability: Implement realtime evaluation and A/B testing frameworks for live agents, enabling teams to experiment with new architectures in production safely. Simulation, Multitenancy, and Globalization: Build the capability to evaluate agents across different languages and simulated Workday tenant environments to ensure global reliability. Centralized Analytics: Drive the design of our metrics warehouse, ensuring evaluation data is queryable and provides actionable insights via executive and engineering dashboards. AI Agent Engineering: Design, build, and deploy sophisticated AI agents (e.g. orchestration agents, reasoning agents, planning and execution node agents, tool selection agents, autonomous workflow agents, conversational interface agents, swarm agents) that interact seamlessly with enterprise data. Work on continuous learning for the agents. Prompt Engineering & Optimization: Develop, test, and maintain advanced prompt engineering and prompt optimization strategies and guardrails to ensure LLMpowered features are accurate, safe, and reliable at scale. Production and MLOps: Own the entire ML lifecycle, ensuring highquality, scalable deployment, monitoring, and continuous improvement of all models and agents in production environments. Own exploration, design and execution of advanced ML models, algorithms and frameworks that deliver value to our users. Collaborate with other ML engineers, software engineers, product managers, and across teams to deliver your products through Workday enduser applications. Be given autonomy and ownership over your work, but with the support of the entire organization. Keep abreast of the latest advancements in ML/AI, Agentic AI, Generative AI, NLP research, techniques, and tools. Have extraordinary opportunities for career growth and learning in a fastgrowing, forwardlooking company. About You Basic Qualifications 3+ years of professional experience as a Machine Learning Engineer, focusing on researching, developing, building, training, and deploying deep learning, NLP solutions, and generative/agentic AI systems into production. Demonstrated experience in building and evaluating AI agents, including familiarity with agent frameworks, RAG architectures, and agent evaluation frameworks (e.g., DeepEval, RAGAS, or custom internal systems). Expertise in prompt engineering, prompt optimization, and developing robust strategies for integrating LLMs into userfacing products. Proven theoretical and practical understanding of statistical analysis and machine learning algorithms, natural language processing, optimization, recommender systems, especially for supervised, unsupervised and selfsupervised methods. Solid understanding of A/B testing, statistical significance, and how to design experiments that differentiate signal from noise in nondeterministic LLM outputs. Expertise in language model finetuning techniques (e.g., parameterefficient finetuning, domain adaptation) and building models with mid and large model architectures (e.g., BERT family, as well as LLMs). Proficiency in modern ML and deep learning frameworks (e.g. PyTorch, TensorFlow, Huggingface), and agentic frameworks (e.g. LangChain/LangGraph/LangSmith). Programming Mastery: Expertlevel Python skills, specifically for building modular libraries and frameworks that other engineers will use, and including experience with AI coding tools. Experience with topics relating to multithreading, api design, matrix processing, runtime memory design, and asynchronous call patterns. System Design: Experience designing ''''Agentintheloop'''' systems and a deep understanding of agentic frameworks (LangGraph, LangChain, or similar). Solid understanding and experience with MLOps, scalability, and cloud services (e.g., AWS, GCP, Azure), containerization technologies (e.g. Docker), Kubernetes, Kubeflow, and largescale ML systems. Other Qualifications A relevant advanced degree (Masters or Ph.D.) in Computer Science, Machine Learning, Data Science, Computer/Software engineering or a related quantitative programming field. OptimizationFocused: Interest in prompt optimization techniques (e.g., DSPy) and costbenefit analysis of different LLM architectures. Data Engineering: Proficiency with PySpark, Pandas, and SQL for managing largescale evaluation datasets and centralized metrics storage. Experience with advanced techniques such as reinforcement learning, imitation learning, graph neural networks, and multimodal models. Experience with A/B testing, and online evaluation experimentation. Experience with recommender systems, optimization techniques, and largescale data ingestion pipelines. Standout leader and colleague, strong communication skills, with experience working across functions and teams, and working on ambiguous problems. Bonus points for machine learning related research publications. Resilience to obstacles, and the ability to lead the solving of problems independently. Workday Pay Transparency Statement The annualized base salary ranges for the primary location and any additional locations are listed below. Workday pay ranges vary based on work location. As a part of the total compensation package, this role may be eligible for the Workday Bonus Plan or a rolespecific commission/bonus, as well as annual refresh stock grants. Recruiters can share more detail during the hiring process. Each candidates compensation offer will be based on multiple factors including, but not limited to, geography, experience, skills, job duties, and business need, among other things. For more information regarding Workdays comprehensive benefits, please click here. Primary Location: CAN.ON.Toronto Primary CAN Base Pay Range: $128,000 - $192,000 CAD Additional CAN Location(s) Base Pay Range: $128,000 - $192,000 CAD Our Approach to Flexible Work With Flex Work, were combining the best of both worlds: inperson time and remote. Our approach enables our teams to deepen connections, maintain a strong community, and do their best work. We know that flexibility can take shape in many ways, so rather than a number of required days inoffice each week, we simply spend at least half (50%) of our time each quarter in the office or in the field with our customers, prospects, and partners (depending on role). This means you''''ll have the freedom to create a flexible schedule that caters to your business, team, and personal needs, while being intentional to make the most of time spent together. Those in our remote ''''home office'''' roles also have the opportunity to come together in our offices for important moments that matter. Pursuant to applicable Fair Chance law, Workday will consider for employment qualified applicants with arrest and conviction records. Workday is an Equal Opportunity Employer including individuals with disabilities and protected veterans. At Workday, we are committed to providing an accessible and inclusive hiring experience where all candidates can fully demonstrate their skills. If you require assistance or an accommodation at any point, please email [email protected]. Are you being referred to one of our roles? If so, ask your connection at Workday about our Employee Referral process! At Workday, we value our candidates privacy and data security. Workday will never ask candidates to apply to jobs through websites that are not Workday Careers. Please be aware of sites that may ask for you to input your data in connection with a job posting that appears to be from Workday but is not. In addition, Workday will never ask candidates to pay a recruiting fee, or pay for consulting or coaching services, in order to apply for a job at Workday. #J-18808-Ljbffr