At Inc42’s The CTO Summit 2026 last week, leaders from Swiggy, Rapido, Razorpay, ShareChat, Shadowfax, and Meesho emphasized the growing importance of AI model evaluations, or evals, in product development. These systematic tests measure how well AI models perform specific tasks, helping teams decide which models to deploy and when to update them, according to inc42.com.
The summit discussions revealed that product teams often face challenges when integrating new AI models, as models that perform well in demos may fail under real-world conditions. Madhusudhan Rao, CTO at Swiggy, shared how early design choices without proper evaluation led to costly issues. The consensus was that evaluation and monitoring must be embedded into the AI model lifecycle to ensure reliability and performance in production environments.
This shift towards continuous evaluation marks a change in AI strategy among major Indian tech companies. Instead of selecting a model once, teams now treat model choice as an ongoing production decision supported by evidence from evals. This approach helps prevent deploying models that might be cheaper or faster but underperform with actual users, addressing a critical gap in AI product management.
Swiggy’s experience at the summit underscored the practical need for robust evaluation frameworks in AI development. The CTO Summit 2026 highlighted that as AI platforms scale, integrating evals into workflows is becoming a key differentiator for companies aiming to maintain product quality and customer satisfaction, inc42.com reported.