Performance: Adventures in Thread-per-Core Async with Redpanda and Seastar

Thread-per-core programming models are well known in software domains where latency is important. Pinning application threads to physical cores and handling data in a core-affine way can yield impressive results, but what is it like to build a large scale general purpose software product within these constraints?

Redpanda is a high performance persistent stream engine, built on the Seastar framework. This session will describe practical experience of building high performance systems with C++20 in an asynchronous runtime, the unexpected simplicity that can come from strictly mapping data to cores, and explore the challenges & tradeoffs in adopting a thread-per-core architecture.


Speaker

John Spray

Engineer @redpandadata

John works in the Core Engineering group at Redpanda, building the high-throughput heart of the Redpanda streaming platform.  His background is in high scale systems, especially distributed storage systems such as Ceph and Lustre.  He enjoys writing async code in Rust and C++, with a particular focus on the tradeoffs in turning high performance designs into systems that are robust at scale.

Read more

Date

Wednesday Mar 29 / 11:50AM BST ( 50 minutes )

Location

Whittle (3rd Fl.)

Topics

performance latency streaming case study

Slides

Slides are not available

Share

From the same track

Session performance

What We Talk About When We Talk About Networks

Wednesday Mar 29 / 02:55PM BST

Networks, and the applications they support, sometimes treat each other as strangers.

Keith Winstein

Associate Professor of Computer Science and, by courtesy, of Electrical Engineering @Stanford

Session performance

Providing a Personalized Experience to Millions of Users @BBC

Wednesday Mar 29 / 01:40PM BST

Personalization can be a very effective and impactful way for your organisation to attract new customers as well as to retain your existing customer base – if you get it right!

Manisha Lopes

Principal Software Engineer @BBC

Session performance

Adventures in Performance

Wednesday Mar 29 / 10:35AM BST

The end of the economical version of Moore's Law (exponential decrease in transistor unit costs), the advent of metered cloud computing, and the shift of the economy toward digital goods and SaaS-based business models has shifted performance engineering from the fringes ("CPU progress will fix it

Thomas Dullien

Distinguished Software Engineer @Elastic

Session performance

WebGPU is Not Just About the Web

Wednesday Mar 29 / 04:10PM BST

Since OpenGL has been deprecated, writing portable native apps that leverage the massively parallel computing capabilities of GPUs has become tedious because no single graphics API can fit all targets.

Élie Michel

Research Scientist @Adobe