Presentation: Building Robust Machine Learning Systems



1:40pm - 2:30pm

Day of week:



Key Takeaways

  • Learn how to evolve machine learning practices to more closely match software engineering practices
  • Discover strategies for validating training data and introspecting models
  • Hear techniques for unit testing models


Machine learning is powering huge advances in products that we know and love. As a result, ever growing parts of the systems we build are changing from the deterministic to the probabilistic. The accuracy of machine learning applications can quickly deteriorate in the wild without strategies for testing models, instrumenting their behaviour and the ability to introspect and debug incorrect predictions. Wouldn't it be nice to have the best of the software engineering and machine learning worlds when building our systems? This session will take an applied view from my experience at Ravelin, and will provide useful practices and tips to help ensure your machine learning systems are robust, well audited, avoid embarrassing predictions, and introspectable, so you can hopefully sleep a little better at night.


What is the focus of your work today?

I’m a cofounder of Ravelin, where we use machine learning, along with graph databases, to help protect merchants from credit card fraudsters. I work on building our machine learning infrastructure, and ensuring that our models are always getting smarter.

What’s the motivation for your talk?

Machine learning systems are complex beasts, and usually have to be treated differently than normal services. They’re hard to debug, difficult to test, and a pain to upgrade. People don’t often talk about the operational experience of running machine learning systems successfully, so I wanted to bring the topic out in the open.

How you you describe the persona of the target audience of this talk?

It’s targeted towards anyone building or developing machine learning systems in production - so primarily data/software engineers, machine learning engineers, and data scientists.

QCon targets advanced architects and sr development leads, what do you feel will be the actionable that type of persona will walk away from your talk with?

They’ll walk away with concrete examples of how to make their machine learning systems more robust, along with experience that I’ve gained from running systems in production.

What do you feel is the most disruptive tech in IT right now?

Deep learning, and specifically the success of transfer learning. It’s remarkable that we can take pretrained ImageNet models, and use the learnt representations on many different problems with a small amount of fine tuning to the task at hand. It will take a lot less time and resources to build advanced systems.

Speaker: Stephen Whitworth

Co-founder and Machine Learning Engineer @Ravelin

Stephen is a co-founder and machine learning engineer at Ravelin, helping fight fraudsters online. He previously worked at Hailo, where he built data products and simulations to understand how people move around cities. He also started golearn, one of the most popular machine learning libraries for Go

Find Stephen Whitworth at

Similar Talks

CTO who understands the science around helping people do their best
Senior Software Engineer @IBM, Committer on Apache Aries
Distributed Systems Engineer Working on Cache @Twitter
Gold Badges Java, JVM, Memory, & Performance @StackOverflow / Lead developer of the OpenHFT project
Research Lead, Software Correctness @Galois


Conference for Professional Software Developers