You are viewing content from a past/completed QCon

Track: Next Generation Microservices: Building Distributed Systems the Right Way

Location: Fleming, 3rd flr.

Day of week: Monday

There has been a meteoric adoption of large scale distributed systems following the advent of microservices architecture. Although there has been widespread adoption of this architecture the practice is still rapidly evolving.

We see these complex distributed systems springing up everywhere but rarely see well built versions. 

We need to answer so many questions to build a distributed system well 

  • How do we migrate our current monolithic architecture to microservices? 
  • What does sensible testing look like in a landscape of autonomous loosely coupled services? 
  • How do we avoid building a distributed monolith? 
  • What do highly available and reliable services look like in this architecture?
  • How do we support and maintain the systems we build?

On this track, we are bringing together some of the early adopters and thought leaders of this approach. They will share their insights and experience to help you avoid some of the pitfalls and to answer the questions you need to address.

Track Host: Nicky Wrightson

Principal Engineer @Skyscanner

Nicky has extensive experience delivering large scale cloud native architectures previously at the Financial Times and now at Skyscanner. She passionately promotes operability as a first class concern in developing these large distributed systems. She works now on the data platform at Skyscanner where the huge scale means a whole different set of problems to solve while still striving to be operable, cost effective and maintainable.

10:35am - 11:25am

Monolith Decomposition Patterns

Patterns to help you incrementally migrate from a monolith to microservices. Big Bang rebuilds of systems are so 20th century. With our users expecting new functionality to be shipped more frequently than ever before, we no longer have the luxury of a complete system rebuild. In fact, a big bang migration of a monolithic architecture into a microservice architecture can be especially problematic, as we’ll explore in this talk. We want to ship features, but we also want to improve our architecture, and for many of us this means breaking down existing systems into microservices. But how do you do this while still regularly releasing new features? In this talk, I’ll share with you some key principles and a number of patterns which you can use to incrementally decompose an existing system into microservices. I’ll also cover off patterns that can work to migrate functionality out of systems you can’t change, which are useful when working with very old systems or vendor products. We'll look at the use of strangler patterns, change data capture, database decomposition and more.

Sam Newman, Microservice, Cloud, CI/CD Expert

11:50am - 12:40pm

Why Distributed Systems Are Hard

Every company that has adopted microservices architecture operates a complex distributed system. It's basically a full-time endeavor to keep up with the ever-changing landscape of technologies and tools to build, maintain, and scale these towering production systems, but the fundamentals of distributed computing theory have remained relatively constant in the last few decades. So, why are distributed systems known for being notoriously difficult to wrangle?

This talk will cover a brief history of distributed computing, present a survey of key academic contributions to distributed systems theory including the CAP theorem and the FLP correctness result, and dig into why network partitions are inevitable today. Though operating in a distributed fashion is full of unknowns, mathematics (consensus algorithms) and engineering (designing for observability) can work together to mitigate these risks. We'll also take a look at how to design systems for greater resilience by studying human factors, which can help reduce the impact of programmatic uncertainty when you're at the helm of a sprawling ecosystem of microservices.

Denise Yu, Senior Software Engineer @GitHub

1:40pm - 2:30pm

Beyond the Distributed Monolith: Rearchitecting the Big Data Platform

The BBC’s Audience Platform Data team collects, transforms and delivers billions of events each day from audience interactions with mobile apps and web sites such as BBC News, BBC Sport,  iPlayer and Sounds.

Last year we migrated to a new analytics provider and we took this as an opportunity to re-architect our distributed monolith. We will share the lessons learnt from operating it for nearly 3 years, how we designed our new microservices architecture so that it is easier to test, scale to cater for increasing demand, keep track of the message flow and replay errors without stopping the rest of the messages from being processed. 

We will also discuss the ideas behind the tooling we have developed which helps us operate our pipeline and has helped new members of the team share the understanding required to troubleshoot problems.

We have been in production for over a year and as demand from our big data platform increases we are beginning to discuss what our platform may look like in the future and the steps we will go through to achieve it.

Blanca Garcia-Gil, Principal Engineer on data platform @BBC

2:55pm - 3:45pm

To Microservices and Back Again

From the start, Segment embraced a microservice architecture in our control plane and data plane. Microservices have many benefits: improved modularity, reduced testing burden, better functional composition, environmental isolation, and development team autonomy, etc. but when implemented wrong their benefits can quickly become burdens.  

After years of continuing to add to our microservice architecture we found ourselves in a spot where our developer velocity was quickly declining and we were constantly tripping over our microservice architecture and its complexity. In this session you'll learn what microservice antipatterns to avoid, the trade offs between microservices and a monolith, how to identify when it's time to take a step back and make a big change, and how moving to a monolith was the solution that worked for us.

Alexandra Noonan, Software Engineer @segment

4:10pm - 5:00pm

Monitoring All the Things: Keeping Track of a Mixed Estate

Monitoring all of a team’s systems can be tricky when you have a microservice architecture. But what happens when you have many teams, each building systems using totally different technology stacks? Add in decades of legacy systems and a sprinkling of third-party tools and you’ve got plenty of fun in store. Discover how to approach monitoring an estate of many technologies and find out what the Financial Times did to improve visibility across systems built by all its teams.

Luke Blaney, Principal Engineer Operations and Reliability Programme @FT

5:25pm - 6:15pm

Panel: Microservices - Are they still worth it?

Lots of us have moved away from monolithic architectures and embraced microservices but do we see the bang for the buck? Is the impact they are having a positive one or negative one? Is there an alternative middle ground? Have we learnt how to wrangle all the operational complexity inherent with large distributed systems?  

Our panel have moved from the monolith to microservices and in some cases back again. They have strong opinions on monorepos, on operating distributed systems and on the best way to structure your organisation to make a success of this architecture.

Luke Blaney, Principal Engineer Operations and Reliability Programme @FT
Alexandra Noonan, Software Engineer @segment
Manuel Pais, IT Organizational Consultant and co-author of Team Topologies
Matt Heath, Senior Staff Engineer @Monzo

Last Year's Tracks