Architecting for Data Products

Data Mesh principles have brought data products to the forefront of data architecture discussions. However, due to the variety of applications and the vast technology landscape, it is hard to bring those principles into practice.

Data products also provide a new lens to think about data ownership, encapsulation, data sharing, and even team structures across an organization.

In this presentation, we will bring insights based on our experience implementing data products and Data Mesh principles in different organizations, covering topics such as:

  • What constitutes a data product and what are the different types of data products
  • How data products support data architecture at different levels, from inside an application all the way up to an enterprise level
  • Skills and team topologies to support data product thinking
  • The data platform capabilities required to build an operating model based around data product teams

Interview:

What's the focus of your work these days?

I help our client's data leaders to create value from their data, by advising on their Data & AI strategy, and how to bring it to life through modern data architecture and engineering.

What's the motivation for your talk at QCon London 2024?

I have many conversations with different clients, and there's a lot of confusion and different interpretations around what a data product is. It gets more interesting when you start looking around how to architect and build them, so I thought this was a topic worth exploring.

How would you describe your main persona and target audience for this session?

I expect this would be of interest to Data Architects, Data Engineers, or Technology Leaders with an intermediate to advanced understanding of data architecture in general.

Is there anything specific that you'd like people to walk away with after watching your session?

I want them to gain a shared understanding of what a data product is, what their different types are, and how that impacts the technical architecture and the team dynamics needed to build and support them.


Speaker

Danilo Sato

Global Head of Technology - Data & AI @Thoughtworks, Author of DevOps in Practice, CD4ML, and DataIQ UK’s 100 Most Influential People in Data in 2022–2023

Danilo helps organisations win with Data & AI by delivering value-led transformations. As a VP, Global Head of Technology - Data & AI, he advises executive leaders on topics ranging from data strategy and governance to building the products and platforms that bring the strategy to life. A member of Thoughtworks' Global Technology team, he is responsible for developing our unique approaches to Data & AI, such as Data Mesh and CD4ML. Author of Devops in Practice, Danilo was named by DataIQ as one of UK's 100 most influential people in data in 2022 and 2023. His work with ITV has won the DataIQ 'Shared data, shared culture' award in 2022.

Read more
Find Danilo Sato at:

Date

Monday Apr 8 / 01:35PM BST ( 50 minutes )

Location

Whittle (3rd Fl.)

Topics

Data Architecture architecture Data Data mesh Data Products Team Topologies

Share

From the same track

Session architecture

Beyond Platform Thinking at RB Global – Build Things No One Expects, in a Place No One Expects It

Monday Apr 8 / 10:35AM BST

Ever wondered what is it really like to move from a poorly integrated COTS legacy architecture to a well factored API driven cloud native platform and event driven architecture, globally deployed in AWS EKS with modern observability and daily software releases?

Speaker image - Ranbir Chawla

Ranbir Chawla

SVP of Engineering @RB Global, Previously SME @Thoughtworks Digital Platform Strategy Team, 30+ Years in Software Engineering and Entrepreneurship

Session

Everything Is a Plugin: How the Backstage Architecture Helps Platform Teams at Spotify and Beyond Spread Ownership and Deliver Value

Monday Apr 8 / 03:55PM BST

Back in 2014, platform engineers at Spotify identified a problem: the number of internal tools was growing rapidly, and as a result engineers around the company were struggling to find the tools and information they needed to work efficiently.

Speaker image - Pia Nilsson

Pia Nilsson

Director of Engineering @Spotify

Speaker image - Mike Lewis

Mike Lewis

Staff Engineer @Spotify

Session platforms

How We Created a High-Scale Notification System at Duolingo That Delivered Millions of Messages Within Seconds During a Super Bowl Commercial Break

Monday Apr 8 / 02:45PM BST

Building a notification system may seem trivial, but what about building one that could reach million of users within a few seconds? What about doing that right after your advertisement airs?

Speaker image - Vitor Pellegrino

Vitor Pellegrino

Site Reliability Engineering @Duolingo

Speaker image - Zhen Zhou

Zhen Zhou

Software Engineer at @Duolingo, Previous Theoretical Computer Science Enthusiast @CMU

Session

Unconference: Architectures You've Always Wondered About

Monday Apr 8 / 05:05PM BST

An unconference is a participant-driven meeting. Attendees come together, bringing their challenges and relying on the experience and know-how of their peers for solutions.

Session

0 → 1, shipping Threads in 5 months

Monday Apr 8 / 11:45AM BST

In Jan 2023, we received word that we’d need to build a microblogging service to compete with Twitter in a couple of months. This is the (technical) story of the small team that was assembled to take on that challenge, and that shipped a new social network in July.

Speaker image - Zahan Malkani

Zahan Malkani

Software Engineer @Meta