You are viewing content from a past/completed QCon -

Presentation: DSSTNE: Deep Learning at Scale

Track: Modern Learning Systems

Location: Mountbatten, 6th flr.

Day of week:

Slides: Download Slides

Level: Advanced

Persona: Data Scientist

Abstract

DSSTNE (Deep Sparse Scalable Tensor Network Engine) is a deep learning framework for working with large sparse data sets. It arose out of research into the use of deep learning for product recommendations after we realized existing frameworks were limited to a single GPU or data-parallel scaling and that they handled sparse datasets incredibly inefficiently. DSSTNE provides nearly free sparse input layers for neural networks and stores such data in a CSR-like format that allowed us to train on data sets that would otherwise have consumed Terabytes of memory and/or bandwidth. Further, DSSTNE implements a new approach to model parallel training that automatically minimizes communication costs such that for a GM204 GPU, one can attain nearly 100% efficient scaling given sufficiently large layer width (~1000 units per GM204 in use). In mid-2016 Amazon open-sourced DSSTNE in exactly the same form as it is used in production in the hopes of advancing the use of deep learning for large sparse data sets wherever the may be.

Speaker: Scott Le Grand

Deep Learning Engineer @Teza (ex-Amazon, ex-NVidia)

Scott is a senior scientist at Teza Technologies. He spent four years at Amazon where he was the lead author of DSSTNE, the Deep Scalable Sparse Tensor Network. Before that he spent ten years at NVidia, doing work that resulted in 14 GPU-related patents.

Find Scott Le Grand at

Preliminary tracks

Discover some of the topics you will see at QCon London. *The schedule is subject to change

Event-Based Architectures: The Hard Parts

Building High Performing Teams

Scaling Security, from Device to Cloud

From Remote to Hybrid Teams: Return to Office?

Architectures You've Always Wondered About

Performance/Mechanical Sympathy

Next Generation Microservices: Building Distributed Systems the Right Way

Modern Data Pipelines & Streams

The Cloud Operating Model

Chaos and Resilience: Architecting for Success

Scaling Frontend

Modern JVM Innovations

MLOps: Implementing ML Across the Enterprise

Architecting a Modern Financial Institution

Crafting the Developer Experience