You are viewing content from a past/completed QCon

Presentation: Continuous Profiling in Production: What, Why and How

Track: Bare Knuckle Performance

Location: Whittle, 3rd flr.

Duration: 4:10pm - 5:00pm

Day of week: Tuesday

Share this on:

This presentation is now available to view on InfoQ.com

Watch video with transcript

What You’ll Learn

 

  1. Learn that regardless of your language stack, you can do continuous profiling in production.
  2. Hear about ways to do very efficient profiling with a low overhead in production.
  3. See how you can leave continuous profiling in all the time and then when something goes wrong, you've got data already.

Abstract

Everyone wants to understand what their application is really doing in production, but this information is normally invisible to JVM developers. Profilers tell you what code your application is running but few developers profile and mostly on their development environments. Thankfully production profiling is now a practical reality that can help you solve and avoid performance problems.

 

Profiling in development can be problematic because it’s rare that you have a realistic workload or performance test for your Java Virtual Machine. Even if you’ve got accurate perf tests maintaining these and validating that they represent production systems is hugely time consuming and hard. Not only that but often the hardware and operating system that you run in production are different from your development environment.

 

This pragmatic talk will help you understand the ins and outs of profiling in a production system. You’ll learn about different techniques and approaches that help you understand what’s really happening with your system. This helps you to solve new performance problems, regressions and undertake capacity planning exercises

Question: 

What is continuous profiling?

Answer: 

Performance is crucial to make the system much cheaper. We always found profiling in production an incredibly laborious process. Every time we did it it was a huge win. But we do it very frequently because of how laborious and how risky it is. Engineers don't like going into production doing things because basically, they get the wrong people shouting at them.

Continuous profiling is about how many of these problems with production can go away if you can do it all the time. Continuous profiling is about unknown unknowns. The things that you don't know beforehand and you don't think to actually monitor. It's about finding things you never knew were actually a problem.

Question: 

Are you going to be talking about specific tools or strategies for implementing tooling?

Answer: 

We talk about different information coming out of existing monitoring systems in terms of metrics and how instrumentation works. We look at some profiling techniques which are JVM specific but would apply to any language that's running on the JVM, Java, Scala and Clojure.

We look for ways how you can do very efficient profiling, efficient enough to do in production. It's about what information you can get out of tools and how you can use these things and what are the benefits to you as a developer of taking this kind of approach.

Question: 

How focused on the JVM is this talk?

Answer: 

A lot of our discussions about different metrics and implementation tradeoffs are pretty much applicable to anyone running a software system. JVM specific is when we talk about how you can do low overhead JVM production profiling.

Question: 

What do you want people to leave the talk with?

Answer: 

One thing that people often do when they diagnose performance problems is basically reproduce those performance problems in a development environment and then try to fix that. People do that because they don't have enough information to see what their production system is doing. One of the big takeaways is that you don't want to be doing that. It can often be very difficult to do, very time consuming to get an accurate reproduction. Your software stack, your hardware and software, can all be very different in your development environment than in production. You might fix the wrong problem.

Continuous profiling entails being able to profile your production system without a significant impact to the users. You can just leave it in all the time and then when something goes wrong you've got that data already, you don't need to hurt anyone.

Question: 

What do you feel is the next potentially disruptive technology for software?

Answer: 

Where we are on the AI adoption process in terms of technologies, we're very very early on. Even though it's a current disruptor it feels like it will still be a huge disruptive factor for the next 10 to 15 years.

Speaker: Richard Warburton

Top Performance-Minded Java Engineer & cofounder of Opsian.com

Richard is a Software Engineer, Teacher and Java Champion. He is the cofounder of Opsian.com and has a long-standing passion for improving Java performance. He’s worked as a developer in different areas including Developer Tools, HFT and Network Protocols. He has written the book “Java 8 Lambdas” for O’Reilly and helps developers learn via http://iteratrlearning.com and http://www.pluralsight.com/author/richard-warburton. Richard is an experienced conference speaker having spoken at dozens of events and sat on conference committees for some of the biggest conferences in Europe and the USA. He holds a PhD in Computer Science from The University of Warwick.

Find Richard Warburton at

Speaker: Sadiq Jaffer

Director at Opsian

Sadiq has for years consulted for multi-national companies designing and implementing highly scalable intelligent platforms. His experience has included deep learning systems, embedded platforms, desktop and mobile games development. He's delivered talks on performance at top software engineering conferences and holds a PhD in autonomous mobile robotics.

Find Sadiq Jaffer at

Last Year's Tracks

Monday, 4 March

Tuesday, 5 March

Wednesday, 6 March