The Hidden Cost of AI Agents: Designing for Predictable Spend at Scale - Anannya Roy Chowdhury

This talk examines the unit economics of agentic AI systems in production, focusing on the reasons behind cost accumulation. The speaker discusses how orchestration overhead from uncontrolled reasoning loops, excessive tool calls, and unmanaged memory contribute to unpredictability and risk in enterprise systems. By drawing on real deployments, the session highlights practical strategies for addressing these issues, such as agent loop budgeting, tiered model routing for sub-tasks, caching strategies for tool outputs, and implementing observability patterns to enhance cost predictability at scale. Attendees will gain insights into designing efficient agent workflows that control spending effectively.