Architecture
Past Presentations
Building and Trusting a Cloud Bank
"Fail fast and fail often" - not only does the Silicon Valley mantra speak volumes on the relentless pursuit of innovation, it also highlights technology's power of unpredictability. But when creating a bank from scratch, the art is in combining pace of change and stability. So when your boss...
Cluster Consensus: When Aeron Met Raft
Consensus protocols enable distributed systems to agree a common view of shared state. This common view allows a cluster to continue service while a majority of its members are available. Raft was designed to be understandable. Raft succeeded in this goal and became popular. Aeron was designed...
Cloud Native Data Pipelines With Apache Kafka
As microservices, data services and serverless APIs proliferate in a cloud-native world, analysts still need to report on the business as a whole. Data engineers need to collect and standardize data in an increasingly complex and diverse system. Luckily, the problem is also the solution – the...
Tesla Virtual Power Plant
A Virtual Power Plant (VPP) is a network of distributed energy-resources (often solar, wind, and batteries) that are aggregated to provide smarter and more flexible power generation, distribution, and availability. A VPP leverages assets for more than one purpose, and, in doing so, decentralizes...
Crushing Tech Debt Through Automation at Coinbase
In the past 4 years Coinbase has grown from one service in production to over 100. Our team has grown 50x, we’ve launched in 30 countries and reinvented ourselves several times as we mature and our industry evolves. We believe a failure mode of financial companies is the death of innovation...
The Present and Future of Serverless Observability
As engineers, we're empowered by advancements in cloud platforms to build ever more complex systems that can achieve amazing feats at a scale previously only possible for the elite few. The monitoring tools have evolved over the years to accommodate our growing needs with these increasingly...
Interviews
What Lies Between: The Challenge of Operationalising Microservices
What is the work you’re doing today?
I build and operate software platforms for monitoring, aggregating, optimizing, and controlling renewable-energy assets, including energy production, battery storage, grid services, and electric-vehicle charging. These platforms leverage microservices and reactive-systems principles. I am most focused on mentoring and growing teams,...
Read Full InterviewBeyond the Distributed Monolith: Rearchitecting the Big Data Platform
What are you doing now?
I am a Principal Systems Engineer at the BBC, and within the BBC I work in the area that deals with our personalizations in services. What this means is that when you sign up with an account with the BBC, then you enter all this personalization features and you can get show recommendations, you can follow shows, stay up to date by...
Read Full InterviewDatabases and Stream Processing: A Future of Consolidation
Tell us a little bit about yourself and what you are doing today.
I work at Confluent, which is one of the companies that sits behind Apache Kafka. Originally I worked on Kafka Core where I worked on a number of features, including the latest version of the replication protocol. I did some work on throttling and a few other things too. These days I run what we call the Office of the CTO, which is a...
Read Full InterviewMachine Learning Through Streaming at Lyft
What is the work you are doing today?
I work for a ride sharing company called Lyft. Our mission is to improve people's lives through the world's best transportation. For the last two and a half years, I have been working on the streaming platform team. In the ride sharing world, it is imperative to build the most recent state of the world and take decisions based...
Read Full InterviewLessons From DAZN: Scaling Your Project with Micro-Frontends
What work are you doing today?
I'm VP of Architecture at a company called DAZN. It's similar to Netflix but for live sports mainly and video-on-demand. It's a platform dedicated to sports. We are available in nine countries and we have a ton of devices that we are targeting between consoles, TVs, set top boxes, web browsers and mobile. I had the...
Read Full InterviewScaling Facebook Live Videos to a Billion Users
QCon: What is your role at Facebook today, and what are the types of problems you’re focused on?
I’m currently working on the backend for Facebook Live, Facebook Video, and Facebook Messenger. Most recently. I’ve been spending a lot of time on scaling the Facebook Live stack. Facebook Live let’s anybody broadcast across the world using just the camera in their pocket. What this means (in real terms) is the backend has to be...
Read Full Interview