Track: Stream Processing @ Scale
Location:
- Mountbatten, 6th flr.
Day of week:
- Monday
The current growth of data volumes and connectivity are driving the active development of stream processing. We frequently see demands to process more data faster to get near real-time insights and decisions. Lambda and Kappa architectures are some of the subsequent popular results. Technologies like Apache Spark and Apache Flink are exciting frameworks at our disposal to implement these or other paradigms.
Stream processing has many facets from architecture for scale and resilience to processing paradigms like micro-batching versus event-driven. The Stream processing at scale track focuses on bringing together current knowledge, and developments in the area from Internet giants to Open Source projects.
by Alexey Kharlamov
VP of Technology @IntegralAdScience
Modern data streaming systems process millions of messages per second. To extract value from their data, organizations employ horizontally scalable distributed event processors such as Apache Storm. Such architectures are frequently designed under the assumption that data loss and calculation errors are acceptable.
In other cases, Kappa architecture is used to fulfill performance requirements without sacrificing consistency and reliability. And given the typical data consistency and...
by Robert Metzger
PMC member and committer Apache Flink project
Data streaming is gaining popularity, as more and more organizations are realizing that the nature of their data production is continuous and unbounded, and can be better served with a streaming architecture. Streaming architectures promise decreased latency from signal to decision, a radically simplified data infrastructure architecture, and the ability to cope with new data that is generated continuously. Apache Flink is a full-featured true stream processing framework with:
- ...
by Ben Stopford
Core Kafka team @Confluent
The world of Microservices is a little different to standard service oriented architectures. They play to an opinionated score of decentralisation, isolation and automation. Stream processing comes from a different angle though. One where analytic function is melded to a firehose of in-flight events. Yet business applications increasingly need to be data intensive, nimble and fast to market. This isn’t as easy as it sounds.
This talk will look at the implications of mixing toolsets...
by Manuel Fahndrich
SE @Google working on horizontal auto-scaling batch/streaming pipelines
Resource allocation and tuning of large data-parallel pipelines has traditionally been a manual process based on human oversight and as such is costly, wasteful, and high latency. Pipelines might see spikes in input rates, organic traffic growth, or fall behind due to outages or throttling of other services. Typically, such variation forces operators to either overprovision their resources for the worst-case, or to manually monitor and adjust resources when necessary. Both of these...
by Richard Kasperowski
Author of The Core Protocols: A Guide to Greatness
Open Space
by Sudhir Tonse
Engineering Manager @Uber - Marketplace Data & Forecasting
Uber is making rapid strides as a thriving Marketplace/Logistics platform. The Marketplace system consists of a set of services that are responsible for handling rider requests and other fulfillment requests from UberEats, Rush etc. To make our Marketplace system efficient and intelligent, we need to extract deep and timely insights from our carefully curated data. We also have to make the insights easily accessible for both people (Operations, Data Scientists and Engineers) and Machines to...
Tracks
Covering innovative topics
Monday, 7 March
-
Back to Java
What to expect in Java 9 and Spring 5
-
Stream Processing @ Scale
Big data, fast-moving data. Practical implementation lessons on Real-time Data
-
DevOps & CI/CD
Lessons/stories on optimizing the deployment pipeline
-
Head-to-Tail Functional Languages
Free-range Monads, Tackling immutability, tales from production, and more...
-
Architecting for Failure
Your system will fail. Take control before it takes you with it
-
21st Century Culture from Geeks on the Ground
New ways to organise technology companies and workplace culture
Tuesday, 8 March
-
Architectures You've Always Wondered about
In-depth technical case studies from giants like: Microsoft, Netflix, Google, Twitter, and more...
-
Close to the Metal
Get efficiency back into your code, concepts like: cache efficient algorithm and lock free data structures
-
Containers (in production)
Real-world lessons on scalability and reliability in production container deployments
-
Modern CS in the real world
Real-world Industry adoption of modern CS ideas
-
Security, Incident Response & Fraud Detection
Master-level classes on building security into your system and responding to incidents when things go wrong.
-
Optimizing You
Keeping life in balance is always a challenge. Learning lifehacks
Wednesday, 9 March
-
Disrupting Finance
Technology advances in finance (blockchain, P2P, Machine Learning, API's)
-
Modern Native Languages
Modern native languages: Safe efficiency with Go, Rust, Swift
-
Full Stack Javascript
Level up Javascript with topics like Angular, React/ReactNative, Node, Mongo/Couch/Other, Falcor, GraphQL, etc
-
Data Science & Machine Learning Methods
A developer's data science and machine learning toolkit
-
Microservices for Mega-Architectures
Practical lessons on Microservices success.
-
Modern Agile Development
Revisiting Agile today and tackling challenges we are seeing in the wild