Speaker
Abstract
Finding statistically significant changes in performance results has always been challenging but now that most of our code runs on hardware and infrastructure we don't own, we need methods and tools for detecting performance changes in noisy data. This talk will provide tips for understanding noise, where it comes, and techniques for fighting it. Plus some anecdotes and real-life examples from the world of open-source.
Interview
I'm leading the development of Nyrkiö's performance analysis product.
I struggled with getting reliable benchmark results for a long time until I learned the tools and tricks discussed in my talk and I want to share these with everyone.
Senior engineers or architects.
The knowledge of how to handle noise in benchmark results and the latest techniques for doing so.
QCon London 2024 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.
From the same track
Monday 8 April
10:35 Mountbatten (6th Fl.) Session performance A Walk Along the Complexity-Performance Curve Richard Startin Senior Software Engineer @Datadog Software performance and complexity are related. It’s common for refactoring to introduce unanticipated regressions, and for performance optimisations to attract scrutiny in code review; how much performance improvement is worth a perceived loss of readability? 11:45 Windsor (5th Fl.) Session Linux kernel Opening the Box: Diagnosing Operating-System Task-Scheduler Behavior on Highly Multicore Machines Julia Lawall Senior Scientist @INRIA An operating system task scheduler is responsible for placing tasks on cores and for selecting which task is allowed to run, at what time. As such, the scheduler is a critical component of any operating system and has a major impact on application performance. 13:35 Windsor (5th Fl.) Session Pitfalls of Unified Memory Models in GPUs Joe Rowell Founding Engineer @poolside.ai, Low-Level Performance Engineer, Cryptographer and PhD Candidate @RHUL Modern GPUs offer support for so-called unified memory, providing a universal address space for both CPUs and GPUs. 14:45 Rutherford (4th Fl.) Unconference Unconference: Performance Engineering Unleashed An unconference is a participant-driven meeting. Attendees come together, bringing their challenges and relying on the experience and know-how of their peers for solutions. 15:55 Windsor (5th Fl.) Session Practical Benchmarking: How To Detect Performance Changes in Noisy Results Matt Fleming CTO @Nyrkiö, Former Linux Kernel Maintainer @Intel and @SUSE Finding statistically significant changes in performance results has always been challenging but now that most of our code runs on hardware and infrastructure we don't own, we need methods and tools for detecting performance changes in noisy data. 17:05 Windsor (5th Fl.) Session Panel: What Does the Future of Computing Look Like Julia Lawall, Matt Fleming, Joe Rowell The future of computing promises to be revolutionary. This panel dives into cutting-edge advancements that will redefine how we interact with technology. We'll explore groundbreaking concepts and discuss their potential to transform our world.