Speaker
Abstract
Data Mesh principles have brought data products to the forefront of data architecture discussions. However, due to the variety of applications and the vast technology landscape, it is hard to bring those principles into practice.
Data products also provide a new lens to think about data ownership, encapsulation, data sharing, and even team structures across an organization.
In this presentation, we will bring insights based on our experience implementing data products and Data Mesh principles in different organizations, covering topics such as:
- What constitutes a data product and what are the different types of data products
- How data products support data architecture at different levels, from inside an application all the way up to an enterprise level
- Skills and team topologies to support data product thinking
- The data platform capabilities required to build an operating model based around data product teams
Interview
I help our client's data leaders to create value from their data, by advising on their Data & AI strategy, and how to bring it to life through modern data architecture and engineering.
I have many conversations with different clients, and there's a lot of confusion and different interpretations around what a data product is. It gets more interesting when you start looking around how to architect and build them, so I thought this was a topic worth exploring.
I expect this would be of interest to Data Architects, Data Engineers, or Technology Leaders with an intermediate to advanced understanding of data architecture in general.
I want them to gain a shared understanding of what a data product is, what their different types are, and how that impacts the technical architecture and the team dynamics needed to build and support them.
Topics
QCon London 2024 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.
Part of the track
Architectures You've Always Wondered About Hosted by Cassandra Shum Vice President of Field Engineering @RelationalAI, Previously @ThoughtworksFrom the same track
Monday 8 April
10:35 Fleming (3rd Fl.) Session architecture Beyond Platform Thinking at RB Global – Build Things No One Expects, in a Place No One Expects It Ranbir Chawla SVP of Engineering @RB Global, Previously SME @Thoughtworks Digital Platform Strategy Team, 30+ Years in Software Engineering and Entrepreneurship Ever wondered what is it really like to move from a poorly integrated COTS legacy architecture to a well factored API driven cloud native platform and event driven architecture, globally deployed in AWS EKS with modern observability and daily software releases? 11:45 Fleming (3rd Fl.) Session 0 → 1, shipping Threads in 5 months Zahan Malkani Software Engineer @Meta In Jan 2023, we received word that we’d need to build a microblogging service to compete with Twitter in a couple of months. This is the (technical) story of the small team that was assembled to take on that challenge, and that shipped a new social network in July. 13:35 Whittle (3rd Fl.) Session Data Architecture Architecting for Data Products Danilo Sato Global Head of Technology - Data & AI @Thoughtworks, Author of DevOps in Practice, CD4ML, and DataIQ UK’s 100 Most Influential People in Data in 2022–2023 Data Mesh principles have brought data products to the forefront of data architecture discussions. However, due to the variety of applications and the vast technology landscape, it is hard to bring those principles into practice. 14:45 Fleming (3rd Fl.) Session platforms How We Created a High-Scale Notification System at Duolingo That Delivered Millions of Messages Within Seconds During a Super Bowl Commercial Break Vitor Pellegrino, Zhen Zhou Building a notification system may seem trivial, but what about building one that could reach million of users within a few seconds? What about doing that right after your advertisement airs? 15:55 Fleming (3rd Fl.) Session Everything Is a Plugin: How the Backstage Architecture Helps Platform Teams at Spotify and Beyond Spread Ownership and Deliver Value Pia Nilsson, Mike Lewis Back in 2014, platform engineers at Spotify identified a problem: the number of internal tools was growing rapidly, and as a result engineers around the company were struggling to find the tools and information they needed to work efficiently. 17:05 Rutherford (4th Fl.) Unconference Unconference: Architectures You've Always Wondered About An unconference is a participant-driven meeting. Attendees come together, bringing their challenges and relying on the experience and know-how of their peers for solutions.