You are viewing content from a past/completed QCon -

Track: Architectures You've Always Wondered about

Location: Fleming, 3rd flr.

Day of week:

Behind every successful tech company, there is an architecture that enables it to function, innovate and scale. What best practices, patterns, and decisions have these companies employed on their road to success? What mistakes did they make along the way and what lessons did they learn? You will hear from companies that have scaled exponentially, handled billions of transactions and millions of users, revised processes and approaches, and ideated platforms that empower companies like yours. In this track, we hope to inspire you through their stories.

Track Host: Sid Anand

Hacker at Large, Co-chair @QCon & Data Council, PMC & Committer @ApacheAirflow

Sid Anand recently served as PayPal's Chief Data Engineer, focusing on ways to realize the value of data. Prior to joining PayPal, he held several positions including Agari's Data Architect, a Technical Lead in Search & Data Analytics @ LinkedIn, Netflix’s Cloud Data Architect, Etsy’s VP of Engineering, and several technical roles at eBay. Sid earned his BS and MS degrees in CS from Cornell University, where he focused on Distributed Systems. In his spare time, he is a maintainer/committer on Apache Airflow, a co-chair for QCon, and a frequent speaker at conferences. When not working, Sid enjoys spending time with family and friends.

Architectural Overhaul: Ad Serving @Spotify Scale

In a hyper growth product company continuous evolution of systems is inevitable as technology improves everyday, product and business needs evolve regularly and change in scale impacts operating costs. At Spotify the ads systems have undergone several changes fundamentally over the years. The performance needs from Spotify's ad technology stack are very demanding - high volume, high availability and low latency. Any downtime or outage has a direct and real revenue impact. Overtime as newer consumer platforms have emerged, backend and data infrastructure technologies have matured and Spotify products have evolved; the needs of the ad technology systems have changed as well. This has resulted in continuous evolution of individual components and the overall architecture. In this talk the speaker will share his experiences about lessons learnt while evolving Spotify's Ad systems without disrupting daily activity.

Kinshuk Mishra, Ad Serving Engineering Director @Spotify

Scaling Facebook Live Videos to a Billion Users

Facebook Live is a live streaming platform available to all Facebook users (1 billion daily active people) from Facebook apps as well as an API. It enables citizen journalism, makes it easy to share everyday moments with your friends and allows celebrities to interact with their fans directly. Facebook Live has seen a tremendous growth in usage since being launched to the public.

Building a successful Live streaming platform requires live streams to have low latency and high quality. Broadcasting and viewing live streams has to work on a wide variety of devices across varying network capabilities. Interactivity during live streams is a key part of the overall experience. In this talk, we will talk about why Live matters, how Facebook Live was architected for these requirements and how it is setup to be flexible in adding features like multi-person live streaming.

Sachin Kulkarni, Engineering Director @Facebook focused on Live, Videos, and Messenger

Low Latency Trading Architecture at LMAX Exchange

Building a financial exchange like the one that LMAX Exchange operates presents a unique challenge. Our customers trade over $2 trillion a year through our platform, while demanding low and predictable latency with ever increasing volumes of transactions. In this talk we will present an overview of the architecture we use to deliver this, and share our experience of how taking a scientific approach to testing and tuning software has helped us to build a high-availability stateful system that regularly processes many tens of thousands of transactions per second in a single thread, with round-trip latencies through our systems measuring just 80 microseconds - all the while running on commodity Intel servers and written in plain old Java.

Sam Adams, Head of Software Engineering @LMAX Exchange

The Distributed Pit of Success @Deliveroo

In just two years Deliveroo has expanded from central London to hundreds of cities in twelve countries, and the engineering team has grown at a similar rate. To allow us to continue innovating rapidly we need to be able to scale the team horizontally. But building distributed systems is hard, and typically requires hordes of very senior engineers with many years of experience and past failures behind them. This talk covers how Deliveroo is using domain driven design principles and powerful building blocks to remove this limitation and allow engineers of any level to quickly and successfully deploy new systems into production.

Greg Beech, Lead Engineer @Deliveroo

Scaling Instagram Infrastructure

Instagram is a social network mobile app that allows people to share the world's moments as they happen. It serves 300 millions users on a daily basis throughout the world.

In this talk, we will give an overview on the infrastructure that supports its users on this large scale.

Topics will include:

  • a brief history of infrastructure evolution
  • overall architecture and multi-data center support
  • tuning of uwsgi parameters for scaling
  • performance monitoring and diagnosis
  • and django/python upgrade (why, challenges and lessons learned)

Lisa Guo, Software Engineer @Instagram


This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.