Streaming Data Architectures

Today’s modern systems generate, as well as process data faster, and in larger volumes than ever before. Whether it’s IOT sensor data, business or social media feeds, or even internal system logs, data often arrives as a continuous stream of events needing attention. Whilst there is no one size fits all, it's crucial to ensure you have a solid streaming architecture in place capable of handling everything thrown at it. Architectures which can scale and are resilient to failures and data surges, and likewise allow for intelligent, timely decision making processes (including AI/ML models) to be embedded and executed at the most appropriate time.

Join us on this track to learn from innovators and engineers in the trenches on how they are designing systems and leveraging modern data stream processing platforms to deal with such challenges. Hear war stories and gain insights into the concrete approaches and solutions applied, as well as pitfalls to avoid. Finally, peer into the future world of the event streaming database - an emerging new category of database that looks to focus on 'data that moves’.

From this track

SESSION + Live Q&A Interview Available

Streaming a Million likes/second: Real-time Interactions on Live Video

When a broadcaster like BBC streams a live video on LinkedIn, tens of thousands of viewers will watch it concurrently. Typically, hundreds of likes on the video will be streamed in real-time to all of these viewers. That amounts to a million likes/second streamed to viewers per live video. How do...

Akhilesh Gupta

Sr. Staff Software Engineer @LinkedIn

SESSION + Live Q&A Interview Available

Internet of Tomatoes: Building a Scalable Cloud Architecture

Five years ago we started on a journey of building a website monitoring tool. Little did I know that this would land up morphing into a full IoT based agriculture platform. Discussing if tomatoes need dark hours to sleep was not the type of question I had anticipated having to answer. But...

Flavia Paganelli

CTO and Founder @30Mhz

SESSION + Live Q&A London

Databases and Stream Processing: A Future of Consolidation

Are databases and stream processors wholly different things, or are they really two sides of the same coin? Certainly, stream processors feel very different from traditional databases when you use them. In this talk, we’ll explore why this is true, but maybe more importantly why it's...

Benjamin Stopford

Author of “Designing Event Driven Systems” & Senior Director @confluentinc

SESSION + Live Q&A Interview Available

From Batch to Streaming to Both

In this talk I walk through how the streaming data platform at Skyscanner evolved over time. This platform now processes hundreds of billions of events per day, including all our application logs, metrics and business events. But streaming platforms are hard, and we did not get it right on day...

Herman Schaaf

Senior Software Engineer @Skyscanner

SESSION + Live Q&A Silicon Valley

Machine Learning Through Streaming at Lyft

Uses of Machine Learning are pervasive in today’s world. From recommendations systems to ads serving. In the world of ride sharing we use Machine Learning to make a lot of decisions in realtime, for example: supply/demand curves are used to get an accurate ETA(estimated time of arrival) and...

Sherin Thomas

Senior Software Engineer @Lyft

UNCONFERENCE + Live Q&A Silicon Valley

Streaming Data Architectures Open Space

Details to follow.