Speaker
Abstract
Many organizations mistake the adoption of a data warehouse, like BigQuery, as the golden ticket to solving all their data challenges. But without a robust data strategy and architecture, you’re simply shifting chaos into the cloud. This talk explores why a warehouse is just one piece of the puzzle and dives into the tools, processes, and structures that can enhance your architecture beyond the warehouse. We’ll also cover practical strategies for recovering from poorly implemented data systems and building sustainable and adaptable data infrastructure.
Interview
I specialise in data software engineering and architecture, that enables effective data management to meet business needs. My work often involves designing and building systems that can handle increasing complexity - whether due to data volume, multiple sources, various integrations, or innovative applications of data.
I've been discussing this topic for years because I've seen firsthand how poor data architecture can hinder companies. My goal is to help change that by sharing insights and best practices to change the status quo in data tech.
My talk is for architects, engineers, and anyone responsible for shaping how data flows within an organisation.
A clear understanding of what good data architecture looks like, practical ideas for improving their own systems, and the realisation that they don’t have to accept the status quo. Every data system can be improved.
I think we'll see an expansion of AI in automation. Whether one agrees with this direction or not, it will demand higher-quality data and more robust security practices than we currently have today.
Topics
QCon London 2025 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.
Part of the track
Modern Data Architectures Hosted by Fabiane Nardon Data Expert, Java Champion & Data Platform Director @totvsFrom the same track
Wednesday 9 April
10:35 Whittle (3rd Fl.) Session Data Architecture Reliable Data Flows and Scalable Platforms: Tackling Key Data Challenges Matthias Niehoff Head of Data and Data Architecture @codecentric AG, iSAQB Certified Professional for Software Architecture There are a few common and mostly well-known challenges when architecting for data. For example, many data teams struggle to move data in a stable and reliable way from operational systems to analytics systems. 11:45 Whittle (3rd Fl.) Session AI/ML Achieving Precision in AI: Retrieving the Right Data Using AI Agents Adi Polak Director, Advocacy and Developer Experience Engineering @Confluent, Author of "Scaling Machine Learning with Spark" and "High Performance Spark 2nd Edition" In the race to harness the power of generative AI, organizations are discovering a hidden challenge: precision. 13:35 Fleming (3rd Fl.) Session Panel: Modern Data Architectures 14:45 Mountbatten (6th Fl.) Session AI/ML The Data Backbone of LLM Systems Paul Iusztin Senior ML/AI Engineer, MLOps, Founder @Decoding ML Any LLM application has four dimensions you must carefully engineer: the code, data, models and prompts. Each dimension influences the other. That's why you must learn how to track and manage each. The trick is that every dimension has particularities requiring unique strategies and tooling. 15:55 Whittle (3rd Fl.) Session Data Architecture Beyond the Warehouse: Why BigQuery Alone Won’t Solve Your Data Problems Sarah Usher Data & Backend Engineer, Community Director, Mentor Many organizations mistake the adoption of a data warehouse, like BigQuery, as the golden ticket to solving all their data challenges. But without a robust data strategy and architecture, you’re simply shifting chaos into the cloud.