Cloud Dataflow
Past Presentations
Straggler Free Data Processing in Cloud Dataflow
One of the main causes of performance problems in distributed data processing systems (from the original MapReduce to modern Spark and Flink) is "stragglers." Stragglers are parts of the input that take an unexpectedly long time to process, delaying the completion of the whole job, and wasting...
Drivetribe: A Social Network on Streams
Drivetribe is the world's biggest motoring destination, as envisioned by Jeremy Clarkson, Richard Hammond, and James May. Built on top of the Event Sourcing/CQRS pattern, the Drivetribe platform uses Apache Kafka as its source of truth and Apache Flink as its processing backbone. This talk aims...
Streaming SQL Foundations: Why I ❤ Streams+Tables
What does it mean to execute robust streaming queries in SQL? What is the relationship of streaming queries to classic relational queries? Are streams and tables the same thing conceptually, or different? And how does all of this relate to the programmatic frameworks like we’re all familiar...