Skip to Main Content

Job Title


AI Agent Evaluation Analyst


Company : Mindrift


Location : Ballarat, Australia


Created : 2025-11-03


Job Type : Full Time


Job Description

Overview At Mindrift, innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI. We are looking for curious and intellectually proactive contributors who doublecheck assumptions and play devils advocate. This flexible, projectbased opportunity is wellsuited for analysts, researchers, consultants, students, and parttime nonpermanent seekers. About the project We are on the hunt for QAs for autonomous AI agents for a new project focused on validating and improving complex task structures, policy logic, and agent evaluation frameworks. Throughout the project, you will balance quality assurance, research, and logical problemsolving. What you will be doing Review evaluation tasks and scenarios for logic, completeness, and realism. Identify inconsistencies, missing assumptions, or unclear decision points. Define clear expected behaviors (gold standards) for AI agents. Annotate causeeffect relationships, reasoning paths, and plausible alternatives. Think through complex systems and policies as a human would to ensure agents are tested properly. Work closely with QA, writers, or developers to suggest refinements or edgecase coverage. Requirements Excellent analytical thinking: reason about complex systems and logical implications. Strong attention to detail: spot contradictions, ambiguities, and vague requirements. Familiarity with structured data formats: read JSON/YAML. Holistic assessment: identify missing or unrealistic elements. Good communication and clear writing in English. We also value applicants with experience in policy evaluation, logic puzzles, case studies, structured scenario design, consulting, academia, olympiads, research, LLMs, prompt engineering, AIgenerated content, QA or testcase thinking, or evaluation scoring. Benefits Pay up to $38/hour depending on skills, experience, and project needs. Flexible, remote, freelance project that fits around your primary professional or academic commitments. Advanced AI project and valuable portfolio experience. Influence how future AI models understand and communicate in your field of expertise. #J-18808-Ljbffr