You are viewing content from a past/completed QCon

Presentation: Realtime & Personalized Notifications @Twitter

Track: Modern Distributed Architectures

Location: Fleming, 3rd flr.

Day of week: Wednesday

Slides: Download Slides

Level: Intermediate

Persona: Architect

Share this on:

What You’ll Learn

  • Hear about major challenges in scaling notifications infrastructure at Twitter
  • Understand Twitter’s architecture used to deliver Notifications
  • Learn about the tradeoffs involved in building systems that deliver realtime and personalized notifications

Abstract

Twitter Notifications Infrastructure enables hundreds of millions of users to stay informed about what’s going on in their Twitter world. Our systems process large volumes of data (aka the Twitter firehose) and deliver realtime and personalized notifications to all kinds of users, ranging from Katy Perry with ~95M followers to brand new users trying out our product for the very first time. We will give an overview of unique challenges in building our notifications infrastructure, present how we solved our challenges, and share key lessons learned along the way.

Question: 

What does your team do at Twitter?

Answer: 

We work on the Notifications team at Twitter. One of our main company goals is to help users stay informed about the world, and our team plays a central role in enabling this. We help notify users about what’s happening in their world in realtime.

Question: 

What can you share about the scale Twitter is dealing with?

Answer: 

We serve more than 300 million monthly active users, who create hundreds of millions of Tweets per day.

Question: 

What will the talk contain?

Answer: 

We will give an overview of unique challenges in building notifications infrastructure at Twitter, present how we solved our challenges, and share key lessons learned along the way.

Specifically we will talk about:

  • How we architected our systems to ensure low latency and high resiliency
  • How we handle massive spikes in traffic and keep up with Twitter’s firehose
  • How we personalize notifications and challenges involved in building out personalization infrastructure
  • Trade offs involved in building a system that delivers both realtime and personalized notifications  
Question: 

Who is the talk aimed at?

Answer: 

It will be of interest to tech leads, senior developers and architects. We hope attendees will be able to apply our strategies and lessons learned to help scale out their own systems.

Speaker: Gary Lam

Tech Lead Notifications, Staff Software Engineer @ Twitter

Gary Lam is the technical lead of the Notifications timeline team at Twitter. He and his team are responsible for Twitter's notifications platform and notifications timeline. Prior to working on Notifications he was the technical lead for Twitter's Machine Learning Platform and helped build out Twitter's underlying machine learning infrastructure.

Find Gary Lam at

Speaker: Saurabh Pathak

Leads Notifications Team @Twitter

Saurabh Pathak is an Engineering Manager at Twitter where he leads the Notifications timeline team which is a product & platform team rolled into one. Prior to joining Twitter, Saurabh was an engineer at Netflix where he architected & built their social infrastructure from scratch. He also worked on Netflix recommender system that powers personalized ranking for Netflix users. Before that, Saurabh was at Yahoo where he led a team that built real time infrastructure from scratch to power social and real time experiences across various Yahoo properties including Yahoo Search. Prior to that, he was part of Yahoo Search team.

Find Saurabh Pathak at

Last Year's Tracks