Job Title

Generalist - English [$45/hour]Location: Geography restricted to US, UK, Canada Type: Full-time or Part-time Contract Work Fluent Language Skills Required: EnglishRole ResponsibilitiesEvaluate LLM-generated responses on their ability to effectively answer user queriesConduct fact-checking using trusted public sources and external toolsGenerate high-quality human evaluation data by annotating response strengths, areas for improvement, and factual inaccuraciesAssess reasoning quality, clarity, tone, and completeness of responsesEnsure model responses align with expected conversational behavior and system guidelinesApply consistent annotations by following clear taxonomies, benchmarks, and detailed evaluation guidelinesGood CandidatureBachelors degree holder Significant experience using large language models (LLMs) and understand how and why people use themExcellent writing skills and can clearly articulate nuanced feedbackStrong attention to detail and consistently notice subtle issues others may overlookAdaptable and comfortable moving across topics, domains, and customer requirementsBackground or experience in domains requiring structured analytical thinking (e.g., research, policy, analytics, linguistics, engineering)Excellent college-level mathematics skillsNice-to-Have SpecialtiesPrior experience with RLHF, model evaluation, or data annotation workExperience writing or editing high-quality written contentExperience comparing multiple outputs and making fine-grained qualitative judgmentsFamiliarity with evaluation rubrics, benchmarks, or quality scoring systems