Speaker
Abstract
The operation and maintenance of large scale production machine learning systems has uncovered new challenges which require fundamentally different approaches to that of traditional software. The field of security in data & machine learning infrastructure has seen a growing rise in attention due to the critical risks being identified as it expands into more demanding real-world use-cases. In this talk we will introduce the motivations and the importance of security in data & machine learning infrastructure through a set of practical examples showcasing "Flawed Machine Learning Security".
These "Flawed ML security" examples are analogous to the annual "OWASP Top 10" report that highlights the top vulnerabilities in the web space, and will highlight common high risk touchpoints. We'll cover a practical example covering how we can mitigate these critical security vulnerabilities. We will cover concepts such as RBAC for ML system artifacts and resources, encryption and access restrictions of data in transit and at rest, best practices for supply chain vulnerability mitigation, tools for vulnerability scans, and templates that practitioners can introduce to ensure best practices.
Interview
My current focus is on building platforms that enable Data Scientists and ML Engineers to iterate throughout the Model Development Life Cycle. This takes a lot of learnings from DevOps, like the “shift left” paradigm, to abstract as many details as possible from the underlying systems while letting them “own” their own models, training code, etc. at the abstraction level that is useful for them.
In the last few years there has been a huge increase in literature around MLOps, and how to do ML systems in the right way. However, there has been very little focus around security in the traditional “cybersecurity” / SecOps sense. As a fellow of the Institute for Ethical AI, one of our key concerns is to spread awareness over this gap and kickstart a conversation about what’s the right way to secure our ML systems.
The talk is quite broad in focus, so it should interest to AI practitioners (i.e. Data Scientists / ML Engineers), to DevOps / Architects and everyone in between. It doesn’t need a deep technical level, however some familiarity with the Model Development Life Cycle may be useful.
I’ll be 100% honest: you won’t learn any silver bullets from this talk. When it comes to security, every solution will be a combination of processes, tools and humans. Instead, the goal is make the attendee aware of this current gap in ML Systems Design and to introduce them to the field of MLSecOps.
QCon London 2024 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.
Part of the track
Architecture for the Age of AI Hosted by Fabiane Nardon Data Expert, Java Champion & Data Platform Director @totvsFrom the same track
Tuesday 9 April
10:35 Churchill (Ground Fl.) Session When AIOps Meets MLOps: What Does It Take To Deploy ML Models at Scale Ghida Ibrahim Chief Architect, Head of Data @Sector Alarm Group, Ex-Facebook/Meta In this talk, we introduce the concept of AIOps referring to using AI and data-driven tooling to provision, manage and scale distributed IT infra. We particularly focus on how AIOps can be leveraged to help train and deploy machine learning models and pipelines at scale. 11:45 Fleming (3rd Fl.) Session AI/ML Mind Your Language Models: An Approach to Architecting Intelligent Systems Nischal HP Vice President of Data Science @Scoutbee, Decade of Experience Building Enterprise AI As large language models (LLMs) emerge from the realm of proof-of-concept (POC) and into mainstream production, the demand for effective architectural strategies intensifies. 13:35 Rutherford (4th Fl.) Event Connecting the Dots: Applying Generative AI (Limited Space - Registration Required) Details coming soon. 14:45 Windsor (5th Fl.) Session Flawed ML Security: Mitigating Security Vulnerabilities in Data & Machine Learning Infrastructure with MLSecOps Adrian Gonzalez-Martin Senior MLOps Engineer, Previously Leader of the MLServer Project @Seldon The operation and maintenance of large scale production machine learning systems has uncovered new challenges which require fundamentally different approaches to that of traditional software. 15:55 Fleming (3rd Fl.) Session Large Language Models for Code: Exploring the Landscape, Opportunities, and Challenges Loubna Ben Allal Machine Learning Engineer @Hugging Face In the rapidly evolving landscape of software development, Large Language Models (LLMs) for code have emerged as a groundbreaking tool for code completion, synthesis and analysis. 17:05 Whittle (3rd Fl.) Session AI/ML Lessons Learned From Building LinkedIn’s AI Data Platform Felix GV Principal Staff Engineer @LinkedIn Taking AI from lab to business is notoriously difficult. It is not just about picking which model flavor of the day to use. More important is making every step of the process reliable and productive.