Abstract
Microservices represent complex business domains in the form of loosely coupled systems, but these don't exist in isolation: services need to propagate data changes amongst each other, in a reliable and scalable way. Also, oftentimes, you don't get to start from scratch, but need to migrate from monolithic legacy applications to microservices.
In this session we'll discuss and showcase how change data capture (CDC) and stream processing can help developers with typical challenges they often face when working on microservices. Come and join us to learn how to:
- Employ the outbox pattern for reliable, eventually consistent data exchange between microservices, without incurring unsafe dual writes or tight coupling
- Gradually extract microservices from existing monolithic applications, using CDC
- Deal with concerns like schema changes and backfilling of change data feeds
We'll explore how open-source technologies, like Debezium and Apache Flink, can help with these tasks; discussing best practices gained from applying these tools in the real world.
Topics
QCon London 2023 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.
From the same track
Monday 27 March
10:35 Mountbatten (6th Fl.) Session A New Era for Database Design with TigerBeetle Joran Greef Founder and CEO @TigerBeetle The pre-recorded video of this presentation will become available within the next few hours. 11:50 Windsor (5th Fl.) Session Apache Pinot Speed of Apache Pinot at the Cost of Cloud Object Storage with Tiered Storage Neha Pawar Founding Engineer @StarTree For real-time analytics, you need systems that can provide ultra low latency (milliseconds) and extremely high throughput (hundreds of thousands of queries per second). 13:40 Churchill (Ground Fl.) Session Microservices Change Data Capture for Microservices Gunnar Morling Senior Staff Software Engineer @Decodableco Microservices represent complex business domains in the form of loosely coupled systems, but these don't exist in isolation: services need to propagate data changes amongst each other, in a reliable and scalable way. 14:55 Mountbatten (6th Fl.) Session transactions Amazon DynamoDB Distributed Transactions at Scale Akshat Vig Senior Principal Engineer NoSQL databases @awscloud NoSQL databases are popular for their high availability, high scalability, and predictable performance. 16:10 Mountbatten (6th Fl.) Session raft Multi-Region Data Streaming with Redpanda Michał Maślanka Software Engineer @Redpanda Real time data streaming platforms such as Redpanda have become a mission critical component in enterprise infrastructure. Multi-region deployments of streaming applications can provide important benefits, such as improved resiliency, better performance and cost reduction. 17:25 Mountbatten (6th Fl.) Session processing techniques In-Process Analytical Data Management with DuckDB Hannes Mühleisen Co-founder and CEO @duckdblabs Analytical data management systems have long been monolithic monsters far removed from the action by ancient protocols. Redesigning them to move into the application process greatly streamlines data transfer, deployment, and management.