About the RoleWe are seeking experienced software engineers to help evaluate how well AI models handle real software engineering problems. This is a short-term, high-impact role where your engineering expertise will directly influence the training and assessment of next-generation AI systems built for developers.You’ll be working with a global AI company on a specialized project focused on analyzing code produced by Large Language Models (LLMs). The goal: make these models better at reading, writing, and understanding real-world code.What You’ll DoAssess AI-generated code for quality, correctness, readability, and efficiencyCompare multiple code outputs and rank them using clear guidelinesReview code diffs from actual GitHub projects and judge their effectivenessWrite concise explanations to support each ranking decisionIdentify edge cases or confusing outputs that indicate AI weaknessesWork with a team of experts to improve evaluation standards and datasetsMust-Have RequirementsPlease do not apply unless you meet all of the following baseline requirements:Experience:5+ years of overall professional software engineering experience (experience working as a data scientist will not be considered)2+ years working full-time as a Fullstack Engineer at a top-tier tech product companyCompanies include Google, Datadog, Shopify, Meta, Canva, Amazon, and others.Note: Contract-only or part-time roles will not be considered as experienceYou are skilled at reading and analyzing Git-style diffsYou can write clear, structured reasoning to explain technical choicesYou follow rubrics and guidelines to ensure fair, structured evaluationsNice to HaveExposure to LLM-generated code or prior experience evaluating model outputsDegree from a top university (not required, but preferred)Background in developer tools, automation, or open-source contributionsExperience with AI research or evaluation workflowsEngagement DetailsType: Contract (independent contractor)Duration: 1 month to start (with possible extensions)Hours: Flexible work hours with commitment of 10-20 hours/week (must have some overlap with Pacific Time)Compensation: $50–$150/hour (based on experience and skill level)Start Date: Immediate openings available (next week)About Turing:Turing is one of the world’s fastest-growing AI companies, pushing the boundaries of AI-assisted software development. Our mission is to empower the next generation of AI systems to reason about and work with real-world software repositories. You’ll be working at the intersection of software engineering, open-source ecosystems, and frontier AI.Why Join Turing?Be part of a cutting-edge AI project helping shape how developers work with codeCollaborate with world-class AI researchers and engineersApply your skills in a meaningful way—on real software, not theoretical examples
Job Title
Remote Senior Software Consultant (LLM) - 34953