Presentation: Staying in Sync: From Transactions to Streams
Location:
- Churchill, G flr.
Duration
Day of week:
- Monday
Key Takeaways
- Learn approaches to keeping datastores in sync in the face of failure and latency.
- Explore event streams and Kafka use cases.
- Understand consistency guarantees and trade offs with keeping multiple data stores sync’d.
Abstract
For the very simplest applications, a single database is sufficient, and then life is pretty good. But as your application needs to do more, you often find that no single technology can do everything you need to do with your data. And so you end up having to combine several databases, caches, search indexes, message queues, analytics tools, machine learning systems, and so on, into a heterogeneous infrastructure...
Now you have a new problem: your data is stored in several different places, and if it changes in one place, you have to keep it in sync in the other places, too. It's not too bad if all your systems are up and running smoothly, but what if some parts of your systems have failed, some are running slow, and some are running buggy code that was deployed by accident?
It's not easy to keep data in sync across different systems in the face of failure. Distributed transactions and 2-phase commit have long been seen as the "correct" solution, but they are slow and have operational problems, and so many systems can't afford to use them.
In this talk we'll explore using event streams and Kafka for keeping data in sync across heterogeneous systems, and compare this approach to distributed transactions: what consistency guarantees can it offer, and how does it fare in the face of failure?
Interview
Similar Talks
Tracks
Covering innovative topics
Monday, 7 March
-
Back to Java
What to expect in Java 9 and Spring 5
-
Stream Processing @ Scale
Big data, fast-moving data. Practical implementation lessons on Real-time Data
-
DevOps & CI/CD
Lessons/stories on optimizing the deployment pipeline
-
Head-to-Tail Functional Languages
Free-range Monads, Tackling immutability, tales from production, and more...
-
Architecting for Failure
Your system will fail. Take control before it takes you with it
-
21st Century Culture from Geeks on the Ground
New ways to organise technology companies and workplace culture
Tuesday, 8 March
-
Architectures You've Always Wondered about
In-depth technical case studies from giants like: Microsoft, Netflix, Google, Twitter, and more...
-
Close to the Metal
Get efficiency back into your code, concepts like: cache efficient algorithm and lock free data structures
-
Containers (in production)
Real-world lessons on scalability and reliability in production container deployments
-
Modern CS in the real world
Real-world Industry adoption of modern CS ideas
-
Security, Incident Response & Fraud Detection
Master-level classes on building security into your system and responding to incidents when things go wrong.
-
Optimizing You
Keeping life in balance is always a challenge. Learning lifehacks
Wednesday, 9 March
-
Disrupting Finance
Technology advances in finance (blockchain, P2P, Machine Learning, API's)
-
Modern Native Languages
Modern native languages: Safe efficiency with Go, Rust, Swift
-
Full Stack Javascript
Level up Javascript with topics like Angular, React/ReactNative, Node, Mongo/Couch/Other, Falcor, GraphQL, etc
-
Data Science & Machine Learning Methods
A developer's data science and machine learning toolkit
-
Microservices for Mega-Architectures
Practical lessons on Microservices success.
-
Modern Agile Development
Revisiting Agile today and tackling challenges we are seeing in the wild