We’re all excited to build and deliver agentic AI services. But what about running at the exponentially greater scale that agents create? LLMs suffer from poor latency and availability issues. More frequent model training drives more frequent updates to agentic services. Most of all, the LLM cost of running at an agentic scale breaks the bank—fast. So, what can you do?
In this session, we’ll dig into how engineering and operations can address:
- Making agentic services fail-proof when their LLMs are not
- Managing a two-order-of-magnitude increase in TPS, including a 2M TPS RAG case
study - Navigating cost vs. quality tradeoffs, with LLMs costing up to 100,000x more than a
database transaction - Continuously redeploying agents that require frequent retraining
After this session, we invite you to attend part 2 of the discussion: “From Concept to Code: Navigating Agentic AI Services"
Speaker
Duncan DeVore
Sr. Director & Architect Advocate @Akka
Duncan DeVore is the co-author of Reactive Application Development, a hands-on guide for building reliable enterprise applications using reactive design patterns. An avid Java developer since 2001, he has earned three patents for innovative software design and led the launch of one of the first large-scale distributed reactive applications in 2012. Duncan is an expert in open source development, an enthusiastic proponent of AI, and a regular writer and speaker on both topics.
Find Duncan DeVore at:
Session Sponsored By
Akka is used to develop resilient, low latency, large scale, cloud-to-edge distributed applications.
Date
Monday Apr 7 / 01:35PM BST ( 50 minutes )
Location
Westminster (4th Fl.)
Video
Video is not available
