Conference:March 6-8, 2017
Workshops:March 9-10, 2017
Workshop: [SOLD OUT] Real-Time Streaming Data Pipelines
Location:
- Rutherford, 4th flr.
When:
- Friday
Prerequisites
- Basic knowledge of Apache Kafka concepts is very helpful, but is not required. Kafka Streams applications are written in Java, but sample solutions will be provided for those not familiar with the language.
- To fully participate, you should bring a laptop with VirtualBox 5.x installed.
- Our VM requires 3GB of RAM, so your laptop should have at least 4GB of RAM installed, and must be able to run a 64-bit virtual machine (so VT-X should be enabled in the BIOS on Windows machines).
- Your laptop should have at least 15GB of free disk space. A link to download the VM will be provided two weeks prior to the event.
This workshop is sold out.
In this workshop, we will show how Kafka Connect and Kafka Streams can be used together to build a real-world, real-time data pipeline. Using Kafka Connect, we will ingest data from a relational database into Kafka topics as the data is being generated. We then will process and enrich the data in real time using Kafka Streams, before writing it out for further analysis.
We’ll see how easy it is to use Connect to ingest and export data (no code is required), and how the Kafka Streams Domain Specific Language (DSL) means that developers can concentrate on business logic without worrying about the low-level plumbing of streaming data processing. Because Streams is a Java library, developers can build real-time applications without needing a separate cluster to run an external stream processing framework.
Key takeaways:
- Configure Kafka Connect to move data between external systems and Apache Kafka
- Write a real-time stream processing application using the Kafka Streams DSL
- See how easy it is to scale Connect and Streams as your data volume increases
Other Workshops:
Tracks
-
Architecting for Failure
Building fault tolerate systems that are truly resilient
-
Architectures You've Always Wondered about
QCon classic track. You know the names. Hear their lessons and challenges.
-
Modern Distributed Architectures
Migrating, deploying, and realizing modern cloud architecture.
-
Fast & Furious: Ad Serving, Finance, & Performance
Learn some of the tips and technicals of high speed, low latency systems in Ad Serving and Finance
-
Java - Performance, Patterns and Predictions
Skills embracing the evolution of Java (multi-core, cloud, modularity) and reenforcing core platform fundamentals (performance, concurrency, ubiquity).
-
Performance Mythbusting
Performance myths that need busting and the tools & techniques to get there
-
Dark Code: The Legacy/Tech Debt Dilemma
How do you evolve your code and modernize your architecture when you're stuck with part legacy code and technical debt? Lessons from the trenches.
-
Modern Learning Systems
Real world use of the latest machine learning technologies in production environments
-
Practical Cryptography & Blockchains: Beyond the Hype
Looking past the hype of blockchain technologies, alternate title: Weaselfree Cryptography & Blockchain
-
Applied JavaScript - Atomic Applications and APIs
Angular, React, Electron, Node: The hottest trends and techniques in the JavaScript space
-
Containers - State Of The Art
What is the state of the art, what's next, & other interesting questions on containers.
-
Observability Done Right: Automating Insight & Software Telemetry
Tools, practices, and methods to know what your system is doing
-
Data Engineering : Where the Rubber meets the Road in Data Science
Science does not imply engineering. Engineering tools and techniques for Data Scientists
-
Modern CS in the Real World
Applied, practical, & real-world dive into industry adoption of modern CS ideas
-
Workhorse Languages, Not Called Java
Workhorse languages not called Java.
-
Security: Lessons Learned From Being Pwned
How Attackers Think. Penetration testing techniques, exploits, toolsets, and skills of software hackers
-
Engineering Culture @{{cool_company}}
Culture, Organization Structure, Modern Agile War Stories
-
Softskills: Essential Skills for Developers
Skills for the developer in the workplace
