Job Title

Focus: Testing and evaluating AI-driven featuresTeam: AI InnovationsThe RoleSeeking a QA professional with 2+ years of experience, including 6+ months dedicated to evaluating and testing AI-driven features, with a focus on model accuracy, safety, and non-deterministic output validation.As a QA Engineer in the AI Innovations team, you won''t be looking for bugs in a traditional way - you''ll be the guardian of accuracy for the next generation of AI-driven features.Testing AI is fundamentally different from traditional QA; we deal in probabilities, not just "pass/fail" binaries. You will be responsible for defining the gold standard of what "good" looks like, building the frameworks to catch hallucinations before our users do, and ensuring that our AI remains a safe, scientifically sound companion for millions of people. If you are a technical problem-solver who is fascinated by how LLMs "think" and want to help shape the testing culture of a fast-paced AI startup environment, this is your seat at the table.What You''ll DoModel Performance Evaluation: Create and maintain "Golden Datasets" (ground truth) to evaluate model performance and the accuracy of our AI-driven features.Functional & Model Testing: Test user-facing AI features for accuracy, latency, and "hallucinations."Prompt Engineering QA: Collaborate with developers to test and refine prompts to ensure output is nutritionally sound and follows safety guardrails.Data Integrity: Audit the internal automated workflows.Edge Case Discovery: Proactively find where the AI fails.Automation: Build automated pipelines to run batch tests against our AI models whenever they are updated.Regression for Non-Deterministic Systems: Develop strategies to ensure that updating an AI model to improve one feature doesn''t degrade performance in another.Preferred ExperienceExperience testing LLMs (OpenAI, Claude, etc.).Experience in a "Start-up" style environment where you help define the testing process from scratch.Python Knowledge: Ability to write basic scripts to automate the processing of large datasets (images/audio) for batch testing.Postman: Experience with Postman for API testing, managing environments and collections.API: Basic understanding of RESTful APIs, JSON schema validation.What We''re Looking ForAnalytical Mindset: Comfortable working with "probability" rather than just binary Pass/Fail.Problem Solver: You can identify "Edge Cases" that a model might miss.AI Curiosity: You don''t need to be a Data Scientist, but you must understand how LLMs behave (and misbehave).Metric-Driven: You enjoy working with numbers and statistical distributions rather than just "Pass/Fail" results.fatsecret Passion: An interest in nutrition, health data, and improving the user experience for millions of dieters.#J-18808-Ljbffr