Conference:March 6-8, 2017
Workshops:March 9-10, 2017
Presentation: The Distributed Pit of Success @Deliveroo
Location:
- Fleming, 3rd flr.
Duration
Day of week:
- Tuesday
Level:
- Intermediate
Persona:
- Architect
Key Takeaways
- Understand that microservices aren’t necessarily the best way of splitting apart a monolith - data needs to be owned
- Learn about the differences between a data bus and an event bus
- Discover how HATEOS and REST can be used to structure communications between systems
Abstract
In just two years Deliveroo has expanded from central London to hundreds of cities in twelve countries, and the engineering team has grown at a similar rate. To allow us to continue innovating rapidly we need to be able to scale the team horizontally. But building distributed systems is hard, and typically requires hordes of very senior engineers with many years of experience and past failures behind them. This talk covers how Deliveroo is using domain driven design principles and powerful building blocks to remove this limitation and allow engineers of any level to quickly and successfully deploy new systems into production.
Interview
Deliveroo is four years old now; a couple of years ago we were only delivering in cental London - but since we've expanded to over 12 countries and 3 continents.
When I joined we were running on a single server; now we’re running on a ruby stack across hundreds of machines.
There are around 100 developers working on the product, which is primarily still a monolith. It is being broken out into a federation services, but this is an early-stage transform.
The theme is a report on how we’re splitting up our monolith into different services, some of the architecture that we’re moving into, and some of the problems that we found along the way. We’ll look into scaling issues with Ruby and dispatch, and how to monitor the state of the system and events.
We’re building an event bus (rather than a data bus) - services will have their have bounded contexts, but with a CRUD over the event bus. We’ll cover more of the details in the talk itself. We handle 10s of millions of events we process per day, because we’re getting pings from writers and restaurants. We need to make exceptions for this kind of data volume for streaming
We think that microservices are flawed approach and we’ll cover the domain architecture and why we think that’s an improvement, applying strong ownership to avoid some of the data traps that end up with a massive data lake.
The service owns the canonical data for the content it manages; for example, the identity service, it will manage not only the users, but will also manage sessions, oauth, and other identities such as external profiles, restaurants and so on. It owns everything it needs to know about the domain rather than being broken into different microservices. We use REST and HATEOS to show links between different services.
We’ve scaled from 7 devs initially to 100 last year, and are aiming to move to 300 people later this year. The kinds of developers interested in the Ruby stack are mainly skewed to the junior and mid-range end; so we’re looking at how that affects our architecture. In other technologies, like the JVM, you find a lot more people skewed to the more senior end of the spectrum.
This talk is aimed at technical leads and architects. Although the examples and experience are Ruby based, the architecture could be applied to any kind of language.
Similar Talks
Tracks
-
Architecting for Failure
Building fault tolerate systems that are truly resilient
-
Architectures You've Always Wondered about
QCon classic track. You know the names. Hear their lessons and challenges.
-
Modern Distributed Architectures
Migrating, deploying, and realizing modern cloud architecture.
-
Fast & Furious: Ad Serving, Finance, & Performance
Learn some of the tips and technicals of high speed, low latency systems in Ad Serving and Finance
-
Java - Performance, Patterns and Predictions
Skills embracing the evolution of Java (multi-core, cloud, modularity) and reenforcing core platform fundamentals (performance, concurrency, ubiquity).
-
Performance Mythbusting
Performance myths that need busting and the tools & techniques to get there
-
Dark Code: The Legacy/Tech Debt Dilemma
How do you evolve your code and modernize your architecture when you're stuck with part legacy code and technical debt? Lessons from the trenches.
-
Modern Learning Systems
Real world use of the latest machine learning technologies in production environments
-
Practical Cryptography & Blockchains: Beyond the Hype
Looking past the hype of blockchain technologies, alternate title: Weaselfree Cryptography & Blockchain
-
Applied JavaScript - Atomic Applications and APIs
Angular, React, Electron, Node: The hottest trends and techniques in the JavaScript space
-
Containers - State Of The Art
What is the state of the art, what's next, & other interesting questions on containers.
-
Observability Done Right: Automating Insight & Software Telemetry
Tools, practices, and methods to know what your system is doing
-
Data Engineering : Where the Rubber meets the Road in Data Science
Science does not imply engineering. Engineering tools and techniques for Data Scientists
-
Modern CS in the Real World
Applied, practical, & real-world dive into industry adoption of modern CS ideas
-
Workhorse Languages, Not Called Java
Workhorse languages not called Java.
-
Security: Lessons Learned From Being Pwned
How Attackers Think. Penetration testing techniques, exploits, toolsets, and skills of software hackers
-
Engineering Culture @{{cool_company}}
Culture, Organization Structure, Modern Agile War Stories
-
Softskills: Essential Skills for Developers
Skills for the developer in the workplace
Conference for Professional Software Developers
