About UsAt Tally, we’re driven by one core mission - making business simpler for everyone. As a pioneer in business management software, we’ve been the trusted partnerof choice for over 2.5 million businesses across 100+ countries - many of them Micro, Small, and Medium Enterprises (MSMEs) that form the backbone of global economies.Our journey is rooted in innovation, guided by deep empathy for customers, and driven by a commitment to excellence. With a legacy of trust and a future fueled by technology, we are shaping the way businesses manage accounting, inventory, compliance, and much more.We believe in honouring our people, nurturingpotential, creating fearlessly, and mastering our craft - all while staying true to our values. If you are looking to make a real impact, grow with purpose, and be part of something meaningful, you’ll find your place at Tally.As we continue to grow, we are working towards a larger ambition - to become the technology backbone for global economic progress.About Engineering Our Engineering center is located in Bangalore while our sales offices and partners are spread across the country and specific regions outside India. Our Engineering team consists of highly talented engineers who live a purpose and dream to develop the software that will accomplish our goal ‘To be the technology fabric that drives the economic growth of the world’. To build this network of businesses Tally likes to build its own technology stack to deliver the required products. Major components of the stack are highlighted below.Operating system: We use a trimmed-down version of the Linux Database system: An Object-oriented database written by Tally to support single view, replication, distributed and multi-tenancy.Web server & app server: We shall write our own hosting platform that can handle millions of connections per server. Engineering @ Tally is responsible for the Design, Development & Testing of all the delightful and flawless products that we release for our customers. We at Engineering do deep technology innovation to deliver unique experiences and capabilities at scale for simplifying business operations across sectors and segments.Role: SDE/ Senior SDE (AI/ML)Relevant Experience: 1 - 5 yearsLocation: BangaloreWhat You Will OwnIn this role, you require a solid foundation in traditional machine learning, the primary focus is on Generative AI. You will be responsible for building, optimizing, and scaling advanced AI systems, moving beyond basic prompt engineering to architect complex Agentic RAG workflows and high-performance inference engines.Experience You Should BringGenerative AI & LLM Orchestration (Core Focus) Agentic Workflows: Proven experience designing and deploying Agentic RAG systems where LLMs autonomously plan and execute tasks. Frameworks: Hands-on expertise with LangGraph (or similar orchestration libraries) to manage state, cycles, and complex decision-making capabilities in agents. Architectural Patterns: Understanding of ReAct patterns, chain-of-thought reasoning, and tool-use integration.Model Optimization & Architecture (Good to Have) Compression: Practical knowledge or deep theoretical understanding of Quantization (e.g., GPTQ, AWQ, FP8) to run large models on constrained hardware without significant accuracy loss. Distillation: Experience with Knowledge Distillation techniques to train smaller, efficient student models from larger teacher models for cost-effective deployment. Fine-tuning: Experience with PEFT, LoRA, and QLoRA for domain adaptation. High-Performance Inference Serving Engines: Experience optimizing inference throughput using high-performance serving frameworks such as vLLM or SGLang. Latency Engineering: Ability to debug and optimize token-per-second (TPS) and time-to-first-token (TTFT) metrics. Core Machine Learning & Data Engineering Foundations: Experience with Python, PyTorch, or TensorFlow. Strong grasp of algorithms, exploratory data analysis (EDA), and statistical hypothesis testing. Data Pipelines: Proficiency in SQL and vector databases (e.g., Pinecone, Milvus, Qdrant) for managing high-dimensional data. Feature Engineering: Solid knowledge of data cleaning, normalization, and transformation pipelines using Pandas/NumPy. What You Will Be DoingBuild Agentic Systems: Implement multi-agent workflows using LangGraph to solve complex, multi-step user queries. Optimize for Scale: Implement Quantization and vLLM/SGLang strategies to reduce inference costs and latency in production. End-to-End Deployment: Collaborate with backend engineers to deploy models using Docker and Kubernetes, ensuring CI/CD pipelines are robust. Research & Distillation: Actively research state-of-the-art open-source models (e.g., Qwen3, Mistral) and apply Knowledge Distillation to create smaller, faster proprietary models. Monitor & Improve: Continuously monitor drift, hallucination rates, and retrieval quality in production systems. Code Quality and Best Practices: Ensure that code adheres to best practices, including clean code principles, testing, and documentation. Note: Only shortlisted candidates will be contacted for further steps in the hiring process.
Job Title
AIML Engineer