You are viewing content from a past/completed QCon

Presentation: Continuous Profiling in Production: What, Why and How

Track: Bare Knuckle Performance

Location: Whittle, 3rd flr.

Duration: 4:10pm - 5:00pm

Day of week: Tuesday

Share this on:

This presentation is now available to view on InfoQ.com

Watch video with transcript

What You’ll Learn

 

  1. Learn that regardless of your language stack, you can do continuous profiling in production.
  2. Hear about ways to do very efficient profiling with a low overhead in production.
  3. See how you can leave continuous profiling in all the time and then when something goes wrong, you've got data already.

Abstract

Everyone wants to understand what their application is really doing in production, but this information is normally invisible to JVM developers. Profilers tell you what code your application is running but few developers profile and mostly on their development environments. Thankfully production profiling is now a practical reality that can help you solve and avoid performance problems.

 

Profiling in development can be problematic because it’s rare that you have a realistic workload or performance test for your Java Virtual Machine. Even if you’ve got accurate perf tests maintaining these and validating that they represent production systems is hugely time consuming and hard. Not only that but often the hardware and operating system that you run in production are different from your development environment.

 

This pragmatic talk will help you understand the ins and outs of profiling in a production system. You’ll learn about different techniques and approaches that help you understand what’s really happening with your system. This helps you to solve new performance problems, regressions and undertake capacity planning exercises

Question: 

What is continuous profiling?

Answer: 

Performance is crucial to make the system much cheaper. We always found profiling in production an incredibly laborious process. Every time we did it it was a huge win. But we do it very frequently because of how laborious and how risky it is. Engineers don't like going into production doing things because basically, they get the wrong people shouting at them.

Continuous profiling is about how many of these problems with production can go away if you can do it all the time. Continuous profiling is about unknown unknowns. The things that you don't know beforehand and you don't think to actually monitor. It's about finding things you never knew were actually a problem.

Question: 

Are you going to be talking about specific tools or strategies for implementing tooling?

Answer: 

We talk about different information coming out of existing monitoring systems in terms of metrics and how instrumentation works. We look at some profiling techniques which are JVM specific but would apply to any language that's running on the JVM, Java, Scala and Clojure.

We look for ways how you can do very efficient profiling, efficient enough to do in production. It's about what information you can get out of tools and how you can use these things and what are the benefits to you as a developer of taking this kind of approach.

Question: 

How focused on the JVM is this talk?

Answer: 

A lot of our discussions about different metrics and implementation tradeoffs are pretty much applicable to anyone running a software system. JVM specific is when we talk about how you can do low overhead JVM production profiling.

Question: 

What do you want people to leave the talk with?

Answer: 

One thing that people often do when they diagnose performance problems is basically reproduce those performance problems in a development environment and then try to fix that. People do that because they don't have enough information to see what their production system is doing. One of the big takeaways is that you don't want to be doing that. It can often be very difficult to do, very time consuming to get an accurate reproduction. Your software stack, your hardware and software, can all be very different in your development environment than in production. You might fix the wrong problem.

Continuous profiling entails being able to profile your production system without a significant impact to the users. You can just leave it in all the time and then when something goes wrong you've got that data already, you don't need to hurt anyone.

Question: 

What do you feel is the next potentially disruptive technology for software?

Answer: 

Where we are on the AI adoption process in terms of technologies, we're very very early on. Even though it's a current disruptor it feels like it will still be a huge disruptive factor for the next 10 to 15 years.

Speaker: Richard Warburton

Top Performance-Minded Java Engineer & cofounder of Opsian.com

Richard is a Software Engineer, Teacher and Java Champion. He is the cofounder of Opsian.com and has a long-standing passion for improving Java performance. He’s worked as a developer in different areas including Developer Tools, HFT and Network Protocols. He has written the book “Java 8 Lambdas” for O’Reilly and helps developers learn via http://iteratrlearning.com and http://www.pluralsight.com/author/richard-warburton. Richard is an experienced conference speaker having spoken at dozens of events and sat on conference committees for some of the biggest conferences in Europe and the USA. He holds a PhD in Computer Science from The University of Warwick.

Find Richard Warburton at

Speaker: Sadiq Jaffer

Director at Opsian

Sadiq has for years consulted for multi-national companies designing and implementing highly scalable intelligent platforms. His experience has included deep learning systems, embedded platforms, desktop and mobile games development. He's delivered talks on performance at top software engineering conferences and holds a PhD in autonomous mobile robotics.

Find Sadiq Jaffer at

Tracks

  • Architectures You've Always Wondered About

    Hard-earned lessons from the names you know on scalability, reliability, security, and performance.

  • Machine Learning: The Latest Innovations

    AI and machine learning is more approachable than ever. Discover how ML, deep learning, and other modern approaches are being used in practice.

  • Kubernetes and Cloud Architectures

    Learn about cloud native architectural approaches from the leading industry experts who have operated Kubernetes and FaaS at scale, and explore the associated modern DevOps practices.

  • Evolving Java

    JVM futures, JIT directions and improvements to the runtimes stack is the theme of this year’s JVM track.

  • Next Generation Microservices: Building Distributed Systems the Right Way

    Microservice-based applications are everywhere, but well-built distributed systems are not so common. Early adopters of microservices share their insights on how to design systems the right way.

  • Chaos and Resilience: Architecting for Success

    Making systems resilient involves people and tech. Learn about strategies being used, from cognitive systems engineering to chaos engineering.

  • The Future of the API: REST, gRPC, GraphQL and More

    The humble web-based API is evolving. This track provides the what, how, and why of future APIs.

  • Streaming Data Architectures

    Today's systems move huge volumes of data. Hear how the innovators in this space are designing systems and leveraging modern data stream processing platforms.

  • Modern Compilation Targets

    Learn about the innovation happening in the compilation target space. WebAssembly is only the tip of the iceberg.

  • Leaving the Ivory Tower: Modern CS Research in the Real World

    Thoughts pushing software forward, including consensus, CRDT's, formal methods & probabilistic programming.

  • Bare Knuckle Performance

    Crushing latency and getting the most out of your hardware.

  • Leading Distributed Teams

    Remote and distributed working are increasing in popularity, but many organisations underestimate the leadership challenges. Learn from those who are doing this effectively.

  • Full Cycle Developers: Lead the People, Manage the Process & Systems

    "Full cycle developers" is not just another catch phrase; it's about engineers taking ownership and delivering value, and doing so with the support of their entire organisation. Learn more from the pioneers.

  • JavaScript: Pushing the Client Beyond the Browser

    JavaScript is not just the language of the web. Join this track to learn how the innovators are pushing the boundaries of this classic language and ecosystem.

  • When Things Go Wrong: GDPR, Ethics, & Politics

    Privacy, confidentiality, safety and security: learning from the frontlines, from both good and bad experiences

  • Growing Unicorns in the EU: Building, Leading and Scaling Financial Tech Start Ups

    Learn how EU FinTech innovators have designed, built, and led both their technologies and organisations.

  • Building High Performing Teams

    To have a high-performing team, everybody on it has to feel and act like an owner. Learn about cultivating culture, creating psychological safety, sharing the vision effectively, and more

  • Scaling Security, from Device to Cloud

    Implementing effective security is vitally important, regardless of where you are deploying software applications.