Job Title

Job Summary: ETS is a global education and talent solutions organization dedicated to advancing the science of measurement for fair skill assessment. The Associate AI Engineer will contribute to the development of advanced adaptive learning technologies and support AI product development by building and maintaining LLM-driven content generation and review systems. Responsibilities: • Adhere to ethical standards and comply with the laws and regulations applicable to the job function • Contributing to the development of AI-enabled services and workflows, including LLM-based content generation and review pipelines, to achieve objectives as outlined by Senior or Principal AI Engineers • Work with data scientists to assist in building data ingest and data transformation infrastructures • Provide support to AI Engineers on the support assessment content, metadata, and evaluation pipelines during development, deployment, and recalibration • Provide support to evaluate, benchmark, and monitor AI and LLM system behavior and provide metrics to document and track their performance. • Explore the data and help to identify differences in data distribution that could impact performance when deployed in prototypes • Utilize best practices around the support for the design, coding, automated unit, regression testing, and deployment of AI-enabled services and LLM-based workflows to production • Keep current of latest AI research and applied advances in large-language-model platforms, orchestration tools, and evaluation methods relevant to educational assessment and content development • Utilize best practices in the responsible use of AI • Work closely with learning scientists and data scientists to define collection events and data transformations needed to drive personalized learning and to support assessment content development and review workflows • Work closely with Product Owners to understand potential and limitations of AI and LLM-based systems in the product • Work effectively in the agile-at-scale framework • Clearly communicate findings, implementation details, and system behavior to senior AI engineers, product owners, and assessment domain experts Qualifications: Required: • A Master’s Degree in Computer Science, Applied Mathematics, Engineering or a related field • 1+ year of training or experience in software engineering, data engineering, or applied AI system development (including LLM-based systems) • Some experience with Python-based software and data engineering frameworks, Java, and/or R • Some experience with APIs and services for large-language models and related tooling (for example, model APIs, prompt-driven workflows, or orchestration platforms such as Dify or similar tools) • Knowledge of statistical analysis and mathematical modeling concepts and principles • Familiarity with JSON-based APIs, data pipelines, and cloud-based services used to integrate AI capabilities into production systems • Excellent communication skills • Ability to work effectively in a cross-functional team • Adhere to ethical standards and comply with the laws and regulations applicable to the job function • Contributing to the development of AI-enabled services and workflows, including LLM-based content generation and review pipelines, to achieve objectives as outlined by Senior or Principal AI Engineers • Work with data scientists to assist in building data ingest and data transformation infrastructures • Provide support to AI Engineers on the support assessment content, metadata, and evaluation pipelines during development, deployment, and recalibration • Provide support to evaluate, benchmark, and monitor AI and LLM system behavior and provide metrics to document and track their performance. • Explore the data and help to identify differences in data distribution that could impact performance when deployed in prototypes • Utilize best practices around the support for the design, coding, automated unit, regression testing, and deployment of AI-enabled services and LLM-based workflows to production • Keep current of latest AI research and applied advances in large-language-model platforms, orchestration tools, and evaluation methods relevant to educational assessment and content development • Utilize best practices in the responsible use of AI • Work closely with learning scientists and data scientists to define collection events and data transformations needed to drive personalized learning and to support assessment content development and review workflows • Work closely with Product Owners to understand potential and limitations of AI and LLM-based systems in the product • Work effectively in the agile-at-scale framework • Clearly communicate findings, implementation details, and system behavior to senior AI engineers, product owners, and assessment domain experts Preferred: • Interest in educational assessment, learning science, or standardized testing environments, and in building human-in-the-loop AI systems for content development and review Company: ETS is a global education and talent solutions organization, enabling lifelong learners to be future ready. Founded in 1947, the company is headquartered in Princeton, USA, with a team of 1001-5000 employees. The company is currently Late Stage.