Performance: Adventures in Thread-per-Core Async with Redpanda and Seastar

Thread-per-core programming models are well known in software domains where latency is important. Pinning application threads to physical cores and handling data in a core-affine way can yield impressive results, but what is it like to build a large scale general purpose software product within these constraints?

Redpanda is a high performance persistent stream engine, built on the Seastar framework. This session will describe practical experience of building high performance systems with C++20 in an asynchronous runtime, the unexpected simplicity that can come from strictly mapping data to cores, and explore the challenges & tradeoffs in adopting a thread-per-core architecture.


Speaker

John Spray

Engineer @redpandadata

John works in the Core Engineering group at Redpanda, building the high-throughput heart of the Redpanda streaming platform.  His background is in high scale systems, especially distributed storage systems such as Ceph and Lustre.  He enjoys writing async code in Rust and C++, with a particular focus on the tradeoffs in turning high performance designs into systems that are robust at scale.

Read more