Conference: March 5-7, 2018

Workshops: March 8-9, 2018

Track:

Stream Processing in the Modern Age

Location: Whittle, 3rd flr.

Day of week: Monday

Compelling applications of stream processing & recent advances in the field

Track Host:
Tyler Akidau
Engineer @Google & Founder/Committer on Apache Beam

Tyler Akidau is a senior staff software engineer at Google, where he is the technical lead for the Data Processing Languages & Systems group, responsible for Google's Apache Beam efforts, Google Cloud Dataflow, and internal data processing tools like Google Flume, MapReduce, and MillWheel. His also a founding member of the Apache Beam PMC. Though deeply passionate and vocal about the capabilities and importance of stream processing, he is a firm believer in batch and streaming as two sides of the same coin, with the real endgame for data processing systems the seamless merging between the two. He is the author of the 2015 Dataflow Model paper, the Streaming 101 and Streaming 102 articles, and the upcoming Streaming Systems book. His preferred mode of transportation is by cargo bike, with his two young daughters in tow.

10:35am - 11:25am

by Aris Koliopoulos
CTO @Drivetribe

by Hamish Dickson
Backend engineer @Drivetribe
Drivetribe: A Social Network on Streams

Drivetribe is the world's biggest motoring destination, as envisioned by Jeremy Clarkson, Richard Hammond, and James May. Built on top of the Event Sourcing/CQRS pattern, the Drivetribe platform uses Apache Kafka as its source of truth and Apache Flink as its processing backbone. This talk aims to elaborate on how common problems in social media, such as analysing user sessions, counting big numbers and making real time recommendations can be resolved with a...

11:50am - 12:40pm

by Jibran Saithi
Lead Architect @Qubit
Lessons From a ~Yearly Re-Write of a Data Pipeline

Every year, we’ve set ourselves a goal of dramatically improving the performance and efficiency of our core data pipelines. We’ve done this by re-writing, effectively from scratch, the streaming pipelines that are responsible for processing over 120,000 events per second to deliver realtime personalisation to millions of web and mobile clients.
From our initial custom ETL system to the latest generation powered by Apache Beam, we’ve learnt to both respect and ignore the common wisdom...

1:40pm - 2:30pm

by Stephan Ewen
Committer @ApacheFlink, CTO @dataArtisans
Next Steps in Stateful Streaming with Apache Flink

Come learn how Apache Flink is making stateful stream processing even more expressive and flexible to support applications in streaming that were previously not considered streamable.

Over the last years, data stream processing has redefined how many of us build data pipelines. Apache Flink is one of the systems at the forefront of that development: With its versatile APIs (event-time streaming, Stream SQL, events/state) and...

2:55pm - 3:45pm
Stream Processing Open Space
4:10pm - 5:00pm

by Tyler Akidau
Engineer @Google & Founder/Committer on Apache Beam
Streaming SQL Foundations: Why I ❤ Streams+Tables

What does it mean to execute robust streaming queries in SQL? What is the relationship of streaming queries to classic relational queries? Are streams and tables the same thing conceptually, or different? And how does all of this relate to the programmatic frameworks like we’re all familiar with? This talk will address all of those questions in two parts.

First, we’ll explore the relationship between the Beam Model (as...

5:25pm - 6:15pm

by Akara Sucharitakul
Principal MTS, Architect @PayPal

by Anil Gursel
Software Engineer @PayPal
Streaming Reactive Systems & Data Pipes w. squbs

Reactive libraries are nothing new to the JVM. Reactive Streams as an SPI has even made its way into Java 9. However, their uses within microservice components are still for relatively narrow purposes like service orchestration. But we think differently. Our whole presence and universe can be thought of as streams of events and activities. And that's the core concept of a new kind of system we design/built - streams end-to-end.
 
This talk will focus on modeling and building...

Last Year's Tracks

Monday, 5 March

Tuesday, 6 March

Wednesday, 7 March

This Year's Schedule

Learn trends from innovator and early adopter companies that you can bring home to your team

Sign up to be informed when registration for QCon London 2019 (Mar 4-8) opens!

Sign Up