You are viewing content from a past/completed QCon -

Presentation: Deep Learning @Google Scale: Smart Reply in Inbox

Track: Modern Learning Systems

Location: Mountbatten, 6th flr.

Day of week:

Level: Advanced

Persona: Data Scientist

What You’ll Learn

  • Hear how end-to-end learning against data has been achieved today in an applied use case from Google Inbox by Gmail
  • Revisit some of the foundational elements of neural networks that Google used to build the Smart Reply feature
  • Dive into the details of how Google implemented a Recurrent Neural Network in an effort to provide contextual short responses for Inbox users


Anjuli will describe the algorithmic, scaling and deployment considerations involved in an extremely prominent application of cutting-edge deep learning in a user-facing product: the Smart Reply feature of Google Inbox.


What is the type of work you’re focused on at Google today?


I’m part of a group called the Google Brain team. The way I think of our team is that we approach machine learning from all angles: some of my teammates work on research in machine learning, some of them work on machine learning software (like TensorFlow), and some work on applications of machine learning around Google. I personally consider myself to be at the boundary between research and applications. My main interest is applying machine learning to language understanding. I’ve worked a little bit on the application side with Gmail (which is what my talk is about), and more recently I’ve been working on the research side.


Can you talk a bit about what it means to apply natural language understanding in Gmail?


Natural language understanding is about applications where there is written or spoken language, and we want the computer to interpret it, then do something with it, possibly generating some language in return. So you can imagine Gmail is a really great place to apply these techniques: the entire application is about the exchange of written language and every day they’re helping people wade through their inboxes. So if Gmail can imbue the product with more language understanding, they can really help their users.


Can you give me a use case inside Gmail that I might recognize?


An easy one to understand are the three tags at the top of the interface (Promote, Social, and Inbox). The system is doing some basic understanding and analysis of the email contents to decide what category to put each email into.


What’s your goal for the talk?


My talk is going to be about Smart Reply. Smart Reply is a feature in Inbox by Gmail that makes it easier and faster to respond on your mobile device. So, if you’re like me, you probably look at emails on your mobile phone, but often postpone responding because typing on a mobile phone is slow and cumbersome and error-prone. What Smart Reply does is it analyzes the contents of the email, and, if the system thinks that a short response is appropriate, it surfaces three suggestions for short responses. Then you, as the person replying, can tap a suggestion (or add to/modify a suggestion) however you want and send it. So the use case here is just helping to make it easier, faster, and less frustrating to respond to messages on your phone device. My goal in talking about Smart Reply is to showcase the power of end-to-end learning from data. In order to have a system that can predict a response in Inbox, the system has to understand how messages and responses work. It’s all about understanding the dialog. Historically, systems that understood dialogue did so in a very rule based way. People would manually program rules and the responses to those rules. Smart Reply is a very different approach. We used a deep learning model called a Recurrent Neural Network. This model learns completely end-to-end from the data itself. Without any feature engineering or manual crafting of rules, we've built a fully data-driven model of conversation. So my goal is to really demonstrate how powerful this technique is to extract complex relationships in the data.


Are the techniques you’re discussing something that is approachable for me? I mean will I be able to incorporate some of your work into my own apps?

I think so, on a couple levels. First of all, the high level idea of my talk is really the power of these models for language understanding. So certainly if you do anything with language understanding the talk is very applicable. Maybe you need to understand text queries on your website, or maybe you want to classify the sentiment of customer reviews. For Smart Reply we had the benefit of a lot of data, but there's a lot of research on how to transfer the techniques I’ll be discussing to smaller data sets as well. But even if you’re not working with language understanding, the broader idea is really just about machine learning and learning end-to-end from data. I hope the ideas can inspire and motivate developers to use concepts from machine learning in new ways.

Speaker: Anjuli Kannan

Software Engineer @GoogleBrain

Anjuli Kannan is a senior software engineer at Google. She is a member of the Brain Team, which works to advance the field of machine intelligence through a combination of basic research, software (TensorFlow), and applications that improve people's lives. Anjuli is especially interested applications of machine learning to problems in natural language understanding. Recently she was a core member of the team that brought the Smart Reply feature to Inbox by Gmail. Launched in 2015, Smart Reply was the first Google-scale application to effectively apply recurrent neural networks in language understanding, as well as the first to leverage Google's open-source TensorFlow.

Find Anjuli Kannan at

Preliminary tracks

Discover some of the topics you will see at QCon London. *The schedule is subject to change

Event-Based Architectures: The Hard Parts

Architectures You've Always Wondered About

Building High Performing Teams

Scaling Security, from Device to Cloud

From Remote to Hybrid Teams: Return to Office?

Performance/Mechanical Sympathy

Next Generation Microservices: Building Distributed Systems the Right Way

The Cloud Operating Model

Chaos and Resilience: Architecting for Success

Modern Data Pipelines & Streams

Scaling Frontend

Modern JVM Innovations

MLOps: Implementing ML Across the Enterprise

Architecting a Modern Financial Institution

Crafting the Developer Experience