You are viewing content from a past/completed conference.
From Fan-Out to Fast: Sub-100ms API Design in Distributed Systems
Abstract
A “simple” API request rarely stays simple. In distributed systems, one call quickly turns into fan-out across gateways, services, caches, and databases — and your p99 becomes the sum of every hop and every flaky dependency. Worse, it’s often not a clean outage; it’s grey failures and intermittent slowdowns that are hard to reproduce and easy for customers to feel.
In this session, I’ll share a practical playbook for designing sub-100ms APIs when fan-out is unavoidable. We’ll start with latency budgets, so performance becomes a design constraint, not a hope. Then we’ll cover the patterns that keep tail latency predictable: safe parallelism, timeouts and retries that don’t amplify failure, idempotency, bulkheads/circuit breakers with fallbacks, and caching strategies where invalidation is treated as a correctness problem. We’ll close with trace-driven observability — the minimal signals that let you quickly answer: where did the milliseconds go, what changed, and is it us or a dependency?
Main takeaways:
- How to budget latency across service boundaries and enforce it with guardrails
- How to use timeouts/retries/idempotency + bulkheads without creating new p99 spikes
- How to use traces + a few key metrics to pinpoint the slow hop fast
Speaker
Saranya Vedagiri
Senior Staff Engineer @eBay
Saranya Vedagiri is a Staff Engineer at eBay, where she designs and operates large-scale distributed systems with a focus on reliability and low-latency performance. Her work spans API design, service-to-service communication, caching strategies, and resilience patterns that keep critical flows fast under real production traffic. Saranya is passionate about performance as a product feature, engineering culture, and mentoring teams to build systems that stay predictably fast as they scale.
Find
Saranya Vedagiri
at:
From the same track
Session
Platform Engineering
APIs for Agents: Rethinking API Programs in the MCP Era
Monday Mar 16 / 01:35PM GMT
As API programs mature, a familiar gap emerges: some teams operate with strong standards, reusable platforms, and clear governance, while others rely on informal guidance and best-effort consistency.
Jim Gough
Distinguished Engineer, API Platform Lead Architect @Morgan Stanley, Co-Author of Optimizing Java
Andreea Niculcea
Vice President @Morgan Stanley
APIs for Agents: Rethinking API Programs in the MCP Era
Session
architecture
Managing Asynchronous APIs at Scale
Monday Mar 16 / 05:05PM GMT
When event-driven architectures are small, teams can reason about events through word-of-mouth. They know who publishes what, who consumes it, and how messages flow through the system. Teams manage their own infrastructure or raise tickets to request changes.
Ian Cooper
Senior Principal Engineer @Just Eat Takeaway
Managing Asynchronous APIs at Scale
Session
Observability
Uncorking Queueing Bottlenecks with OpenTelemetry
Monday Mar 16 / 11:45AM GMT
Queues are the backbone of scalable, asynchronous systems, but they can easily create a tangled web of complexity. When things slow down, the bottleneck could be anywhere, from producer lag to consumer exhaustion, and standard metrics often fail to show the full picture.
Julian Wreford
Team Lead of Operability Team @Gearset, Software Engineer Turned Accidental SRE
Oli Lane
Engineering Team Lead @Gearset, Focusing on Engineering Culture, Observability, and Platform Reliability
Uncorking Queueing Bottlenecks with OpenTelemetry
Session
AI
Enchant Your AI and APIs with eBPF Magic 🪄
Monday Mar 16 / 03:55PM GMT
It is a common occurrence to see applications thrown over the fence, landing somewhere in production without a second thought about their lifecycle or how they may need maintaining in the future to connect to more efficient API endpoints.
Dan Finneran
Principal Community Advocate at Isovalent @Cisco
Enchant Your AI and APIs with eBPF Magic 🪄
