Workshop: Profiling JVM Applications in Production

Level: 
Intermediate

When:

1:00pm - 4:00pm

Prerequisites

  • experience with Java application development or operations
  • bring a laptop with a modern browser - hands-on labs will be performed through the browser

Profiling JVM applications it not very easy in the first place. Many profilers, such as JVisualVM and jstack, will simply lie to your face about which call stacks are hottest and where your bottlenecks lie. Profiling in production environments has even more challenges, because you have to carefully manage overhead, account for other processes running on the system, and choose non-invasive tools that don't require an application restart. This is the workshop for you -- we will learn, understand, and experiment first-hand with JVM profiling tools designed for production use.

We will build a simple checklist for verifying JVM application performance, and finding the area to focus on in a closer investigation. Then, we will experiment with two approaches for CPU profiling on Linux: the perf multi-tool, combined with perf-map-agent, and the async-profiler project, an innovative tool that brings perf together with traditional JVM profiling techniques. We will visualize stack traces using flame graphs, and understand where the CPU bottlenecks lie, through a series of hands-on labs. In the second half of this workshop, we will talk about more complicated scenarios: diagnosing errors when opening files, tracing database queries, monitoring system I/O load, understanding the reasons for excessive garbage collection, figuring out why threads are blocked off-CPU, and more.

Outline:

  • Linux and JVM performance information sources: tracepoints, {k,u}probes, USDT, JMX, JVMTI, Java SA, JFR, GC/JIT logs
  • Linux JVM performance checklist: top, sar, free, iostat, pidstat, vmstat, jstack, jstat (based on hsperfdata), jcmd, jattach
  • Linux perf, CPU sampling, getting stack reports, problem with symbols
  • Generating perf maps with perf-map-agent, inline frames, source info
  • Visualizing JVM profiles with flame graphs
  • JVMTI profiling with an agent & jattach: GetAllStackTraces, exceptions, class loads, monitor contention [link] etc.
  • Full-stack JVM profiling with async-profiler, how it works, pros and cons
  • Introduction to BPF
  • BPF scenarios and the BCC toolkit, Java-specific tools, JVM USDT probes, -XX:+ExtendedDTraceProbes
  • Dedicated BCC tools: fileslower, opensnoop, gethostlatency, runqlat, cpudist
  • General-purpose BCC tools: trace, argdist, funccount, stackcount, funclatency
  • Heap allocation profiling (based on TLAB sampling) with async-profiler vs. funccount/stackcount vs. grav

Speaker: Sasha Goldshtein

CTO of Sela Group, a Microsoft MVP and Regional Director

Sasha Goldshtein is the CTO of Sela Group, a Microsoft MVP and Regional Director, Pluralsight and O'Reilly author, and international consultant and trainer. Sasha is the author of two books and multiple online courses, and a prolific blogger. He is also an active open source contributor to projects focused on system diagnostics, performance monitoring, and tracing -- across multiple operating systems and runtimes. Sasha authored and delivered training courses on Linux performance optimization, event tracing, production debugging, mobile application development, and modern C++. Between his consulting engagements, Sasha speaks at international conferences world-wide.

Find Sasha Goldshtein at

Other Workshops:

Day: Thursday [Full Day]
Day: Friday [Half Day]
by
Day: Thursday [Full Day]
by

.

Tracks

  • Architecting for Failure

    If you're not architecting for failure you're heading for failure

  • Architectures You've Always Wondered About

    Topics like next-gen architecture, self-driving cars, deep learning, and lessons around massive scale.

  • Bare Knuckle Performance

    Killing latency and getting the most out of your hardware

  • Building Great Engineering Cultures & Organizations

    Stories of cultural change in organizations

  • DevEx: The Next Evolution of DevOps

    Removing friction from the developer experience.

  • Distributed Stateful Systems

    Architecting and leveraging NoSQL revisitied

  • Evolving Java and the JVM: Mobile, Micro and Modular

    Although the Java language is holding strong as a developer favourite, new languages and paradigms are being embraced on JVM.

  • JavaScript and Beyond: The Future of the Frontend

    Exploring the great frontend frameworks that make JavaScript so popular and theg JavaScript-based languages revolutionising frontend development.

  • Leading Edge Backend Languages

    Code the future! How cutting-edge programming languages and their more-established forerunners can help solve today and tomorrow’s server-side technical problems.

  • Microservices/Serverless: Patterns and Practices

    Stories of success and failure building modern service and function-based applications, including event sourcing, reactive, decomposition, & more.

  • Modern CS in the Real World

    Applied trends in Computer Science that are likely to affect Software Engineers today.

  • Next Gen Banking: It’s not all Blockchains and ICOs

    Great technologies like Blockchain, smartphones and biometrics must not be limited to just faster banking, but better banking.

  • Observability: Logging, Alerting and Tracing

    Observability in modern large distributed computer systems

  • Operating Systems: LinuxKit, Unikernels, & Beyond

    Applied, practical, & real-world deep-dive into industry adoption of OS, containers and virtualisation, including Linux on Windows, LinuxKit, and Unikernels

  • Security: Red XOR Blue Team

    Security from the defender's AND the attacker's point of view

  • Stream Processing in the Modern Age

    Compelling applications of stream processing & recent advances in the field

  • Tech Ethics in Action

    In this track we will explore real world examples of where companies have made technology decisions driven by ethical as much as technological reasons and what we can learn from their experiences.

  • The Practice & Frontiers of AI

    Learn about machine learning in practice and on the horizon