Job Title

Job Summary:We are seeking an experienced QA Manager with deep expertise in AI-specific testing to lead the quality assurance efforts for our Generative AI and Large Language Model (LLM) solutions. This role is critical in ensuring the performance, reliability, and integration of AI-driven systems through rigorous test planning, automation, and documentation.Key Responsibilities:Manage AI-specific defect identification, triage, and resolution, working closely with engineering and product teams.Oversee the design, development, and execution of test cases for Generative AI/LLM-based systems.Implement and maintain AI testing documentation, including traceability matrices, test plans, and test case repositories.Ensure comprehensive functional, regression, and integration testing across AI components and services.Generate detailed QA reports and dashboards, tracking coverage, issues, and test effectiveness.Lead automation efforts for LLM testing using industry-standard frameworks and tools.Collaborate cross-functionally with ML engineers, data scientists, and DevOps teams to ensure seamless delivery.Required Qualifications:Bachelor's or Master’s degree in Engineering or Computer Science from a top-tier university.10+ years in software QA, with at least 3+ years in AI/ML or LLM testing.Hands-on experience with automation frameworks (e.g., Selenium, PyTest, Robot Framework) and scripting (Python preferred).Deep understanding of Generative AI concepts, LLM architectures, and AI system behaviors.Familiarity with API testing tools (e.g., Postman, REST-assured) and performance testing tools (e.g., JMeter).Experience managing test strategies, test coverage, defect life cycle, and continuous testing in Agile environments.Kore.ai is PreferablePreferred Qualifications:Exposure to open-source LLMs (e.g., LLaMA, GPT, Claude) and fine-tuning/testing workflows.Familiarity with MLOps and CI/CD pipelines for AI model deployments.Experience working in cloud environments (Azure, AWS, GCP) for AI testing infrastructure.Certifications in ISTQB, AI/ML, or Test Automation.