You are viewing content from a past/completed QCon

Track: Solutions Track IV

Location: Westminster, 4th flr.

Day of week: Wednesday

Industry practitioners and technical product managers from leading vendors demonstrate solutions to some of today's toughest software development challenges in the areas of performance monitoring, Big Data, software delivery, scalability, and more.

Track Host: Nitin Bharti

Managing Editor and Product Manager, QCon and InfoQ

Over the last decade, Nitin has helped build several notable online developer communities including, DZone, and The Code Project. He is known for his extensive editorial work in the Enterprise Java, .NET, SOA, and Agile communities. As Managing Editor and Product Manager at C4Media - the producer of and QCon events - Nitin continues to pursue his primary passion: helping spread knowledge and innovation throughout the enterprise software development community.

10:35am - 11:25am

Data Inferno: 9 Circles of Data Tests With Apache Airflow

Continuous delivery is a given nowadays. This goes hand in hand with a lot of automated testing. For 'normal' applications, such testing is well known and documented in the form of unit tests, integration tests, regression tests etc. For big data applications, however, another dimension of complexity is added: that of the data itself. The truth is: real data sucks, it always surprises you by how it differs from what you expect. Unreliable data, in turn, can result in unreliable applications, which makes for unhappy users. In this talk, we'll take you on a journey through our Nine Circles of Data Tests which ensure the data is correct and makes sense. We use Airflow to do this, testing our data and logic at several steps, in order to avoid having to debug such issues over the weekend.

Topics include:

  • CI tests for your data deployments
  • Integrating data tests into your DAG
  • DTAP-ing your data deployments
  • Integrating data science models into this engineering world
  • How we went nuclear with GIT
  • How Chuck Norris keeps us honest
  • Local Airflow in Docker

John Müller, Data Engineer WB Advanced Analytics @ING_news (ING Bank)

11:50am - 12:40pm

Why Developers Shouldn't Care About Containers

Imagine a world where developers don't even need to know what a container is? A world where writing code is literally all a developer needs to worry about. Turns out that world already exists - and in multiple forms! Ed will cover how to increase development efficiency and will use the open source Cloud Foundry Application Runtime as an example. The talk will include a live demonstration of taking code, written in any language or framework, and running it on multiple clouds with a single command. The talk will also explore platform components, architecture and share with you what types of workload are best suited to different deployments.

Ed Shee, Developer Advocate @IBM

Last Year's Tracks