Multi-Region Data Streaming with Redpanda

QCon London 2023

Session raft

Multi-Region Data Streaming with Redpanda

Monday Mar 27 / 04:10PM BST, Mountbatten (6th Fl.)

Abstract

Real time data streaming platforms such as Redpanda have become a mission critical component in enterprise infrastructure. Multi-region deployments of streaming applications can provide important benefits, such as improved resiliency, better performance and cost reduction.

Developers and architects must consider various trade-offs in multi-region streaming: the level of data consistency required, the desired behaviour on failures, and how to configure data placement for best performance. This talk will introduce the design of Redpanda’s Multi-Region feature, and describe how we leveraged Raft protocol properties, a hierarchical constraint solver, automatic data balancing, and tiered storage to provide an easy to use and flexible model of building reliable multi-region data streaming applications

Topics

raft streaming enterprise infrastructure resiliency performance data balancing tiered storage
76% senior dev or higher
1:11 speaker ratio
60+ practitioners

QCon London 2023 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.

Share

From the same track

Monday 27 March

10:35 Mountbatten (6th Fl.) Session A New Era for Database Design with TigerBeetle Joran Greef Founder and CEO @TigerBeetle 11:50 Windsor (5th Fl.) Session Apache Pinot Speed of Apache Pinot at the Cost of Cloud Object Storage with Tiered Storage Neha Pawar Founding Engineer @StarTree 13:40 Churchill (Ground Fl.) Session Microservices Change Data Capture for Microservices Gunnar Morling Senior Staff Software Engineer @Decodableco 14:55 Mountbatten (6th Fl.) Session transactions Amazon DynamoDB Distributed Transactions at Scale Akshat Vig Senior Principal Engineer NoSQL databases @awscloud 16:10 Mountbatten (6th Fl.) Session raft Multi-Region Data Streaming with Redpanda Michał Maślanka Software Engineer @Redpanda 17:25 Mountbatten (6th Fl.) Session processing techniques In-Process Analytical Data Management with DuckDB Hannes Mühleisen Co-founder and CEO @duckdblabs