About Company:Delta Exchange is a dynamic and rapidly growing crypto exchange that specializes in facilitating derivatives trading, with a primary focus on Options trading. We are India’s largest and world’s 2nd largest Crypto Options Exchange with over $200MN in daily trading volumes. Our platform enables users to trade derivatives on Bitcoin, Ethereum, and Solana and other cryptos. With a solid foundation, having raised over $15MN in funding, and a passionate team of 100+ professionals coming from backgrounds in Finance, Technology, Startups, IITs, IIMs, and ISB, we are poised for exponential growth and market leadership in the crypto derivatives space.The Role:Experience Level: 2-4 Years AI Feature Validation & Tool Auditing: Act as the primary gatekeeper for AI quality (pseudo-QA) by rigorously testing features and verifying that the LLM correctly triggers and utilizes backend tools (function calling) during conversations.Conversation Analysis & Tone Guardrailing: Monitor and grade the semantic quality and tone of AI interactions to ensure empathy and brand alignment, while flagging hallucinations or negative sentiment patterns for immediate resolution.Root Cause Investigation: High Level backend interactions to diagnose failure points (e.g., analyzing why a model like Gemini Flash failed to parse an image attachment) and determine if the issue lies in the model capabilities or not.Prompt Engineering Support & Model Testing: Assist the PM in refining system prompts to fix edge cases and conduct A/B testing on different models to provide data-backed recommendations for model selection
Job Title
AI Product Analyst/ Senior Analyst