Presentation: TornadoVM: Java for GPUs and FPGAs
Share this on:
What You’ll Learn
- Hear about Tornado VM, what it is and what it is good for.
- Find out what are the latest developments on heterogeneous hardware, and where TornadoVM fits.
Abstract
There is no single computer architecture that is best for executing all types of workloads efficiently. Therefore, the proliferation of heterogeneous hardware in recent years means that every system we program is likely to include a mix of computing elements; each of these with different hardware characteristics that enable programmers to improve performance while decreasing energy consumption. These new heterogeneous devices include multi-core CPUs, Graphics Processing Units (GPUs), and Field Programmable Gate Arrays (FPGAs). This trend has been followed by changes in software development norms that do not necessarily favor programmers.
A prime example is the two most popular heterogeneous programming languages, CUDA and OpenCL, which expose several low-level features to the API, making them difficult to use by non-expert users. Instead of using low-level programming languages, developers in industry and academia tend to use higher-level, object-oriented programming languages, typically executed on managed runtime environments, such as Java, R, Python, and JavaScript. Although many programmers might expect that such programming languages would have already been adapted for transparent execution on heterogeneous hardware, the reality is that their support is either very limited or absent.
In this talk, we present TornadoVM, a plugin for OpenJDK that allows Java programmers to automatically run on Heterogeneous Hardware such as multi-core CPUs, GPUs, and FPGAs. Furthermore, TornadoVM can perform live task-migration from one device to another entirely transparent for the user. To do so, TornadoVM consists of three software components: 1) A simple API for composing pipelines of existing Java methods. 2) An optimizing JIT compiler that extends the Graal compiler with hardware-aware optimizations that generate OpenCL C code. 3) A runtime system that executes TornadoVM specific bytecodes, it performs memory management. It also schedules the code for execution on GPUs, multi-core CPUs, and FPGAs, and it performs live task-migration between devices.
What is the work you're doing today?
I'm currently a research associate at the University of Manchester, UK, working at the Advanced Processor Technology Group (APT). What we do is research on new language virtual machines to take advantage of new hardware accelerators as transparent as possible with the aim to increase performance and decrease energy consumption. The use-case we have is TornadoVM, a plugin to OpenJDK that automatically runs Java programs and on GPUs and FPGAs. That's one direction. Another direction we are investigating is how to integrate this into distributed programming frameworks such as Flink and Spark.
What are your goals for your QCon talk?
I would say the main goals are to get the audience to know that current computer systems are heterogeneous. Mobile devices have multicore CPUs and GPUs, as well as laptops and even data centers, that are including reconfigurable hardware (FPGAs) as well. So heterogeneous computing is here to stay. That's great, but we need to program them somehow. We need to use them. And thus the goal of my talk, how to use them efficiently from managed programming languages like Java.
Another goal is to introduce people to the device selection problem. This is because now, we have the choice of where to execute code on the best possible hardware for each program (e.g, is it on a GPU? on an FPGA?) This is a hard problem because it depends on many factors that are only finally decided at runtime, such as input-sizes. This is an important problem that TornadoVM automatically tackles and I will show how.
What are some of the key takeaways from your talk?
Getting people to know TornadoVM and the reasons to use it. There are many alternatives. TornadoVM is just one proposal but we offer more features and code coverage than others. Additionally, although TornadoVM is developed inside academia, we are starting to get the attention from the industry. So it can be used outside of academia. We have cases for deep learning, computer vision, machine learning, fintech, etc..
Finally, I would like people to remember that TornadoVM is not a replacement for the Java Virtual Machine (JVM), but rather a complement to execute specialized workloads much faster by transparently using the right hardware. For example, I will show that TornadoVM can achieve up to 4500x speedup compared to Java OpenJDK when running on GPUs and FPGAs.
Similar Talks
Scaling N26 Technology Through Hypergrowth
Software Engineer and Tech Lead @N26
Folger Fonseca
How to Debug Your Team
Senior VP Engineering @spring_health, former VP Engineering @Meetup
Lisa van Gelder
Trust, the Secret Ingredient in High Performing Team
Site Reliability Engineer @immersivelabsuk
Jaycee Cheong
Lessons Learned from Reviewing 150 Infrastructures
CTO / CEO @scalefactory
Jon Topper
Beyond the Distributed Monolith: Rearchitecting the Big Data Platform
Principal Engineer on data platform @BBC
Blanca Garcia-Gil
Software Profiling for Fun & Profit
Chief Architect for the OpenLDAP Project and CTO of Symas Corporation
Howard Chu
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
Java Performance Engineer @Oracle
Sergey Kuksenko
Monitoring All the Things: Keeping Track of a Mixed Estate
Principal Engineer Operations and Reliability Programme @FT
Luke Blaney
Streaming a Million likes/second: Real-time Interactions on Live Video
Sr. Staff Software Engineer @LinkedIn
Akhilesh Gupta
Tracks
Monday, 2 March
-
Next Generation Microservices: Building Distributed Systems the Right Way
Microservice-based applications are everywhere, but well-built distributed systems are not so common. Early adopters of microservices share their insights on how to design systems the right way.
-
Streaming Data Architectures
Today's systems process huge volumes of continuously changing data. Hear how the innovators in this space are designing systems and leveraging modern data stream processing platforms.
-
Driving Full Cycle Engineering Teams at Every Level
"Full cycle developers" is not just another catch phrase; it's about engineers taking ownership and delivering value, and doing so with the support of their entire organisation. Learn more from the pioneers.
-
When Things Go Wrong: GDPR, Ethics, & Politics
Privacy, confidentiality, safety and security: learning from the frontlines, from both good and bad experiences
-
JavaScript: Pushing the Client Beyond the Browser
JavaScript is not just the language of the web. Join this track to learn how the innovators are pushing the boundaries of this classic language and ecosystem
-
Modern CS in the Real World
Head back to academia to solve today's problems in software engineering.
Tuesday, 3 March
-
Architectures You've Always Wondered About
Hard-earned lessons from the names you know on scalability, reliability, security, and performance.
-
The Future of the API: REST, gRPC, GraphQL and More
The humble web-based API is evolving. This track provides the what, how, and why of future APIs.
-
Building High Performing Teams
There are many discussions outlining the secret sauce of high-performing teams. Learn how to balance the essential ingredients of high performing teams such as trust and delegation, as well as recognising the pitfalls and problems that will ruin any recipe.
-
Machine Learning: The Latest Innovations
AI and machine learning is more approachable than ever. Discover how ML, deep learning, and other modern approaches are being used in practice.
-
Bare Knuckle Performance
Crushing latency and getting the most out of your hardware.
-
Modern Compilation Targets
Learn about the innovation happening in the compilation target space. WebAssembly is only the tip of the iceberg.
Wednesday, 4 March
-
Growing Unicorns in the EU: Building, Leading and Scaling Financial Tech Start Ups
Learn how EU FinTech innovators have designed, built, and led both their technologies and organisations.
-
Kubernetes and Cloud Architectures
Learn about cloud native architectural approaches from the leading industry experts who have operated Kubernetes and FaaS at scale, and explore the associated modern DevOps practices.
-
Chaos and Resilience: Architecting for Success
Making systems resilient involves people and tech. Learn about strategies being used, from cognitive systems engineering to chaos engineering.
-
Leading Distributed Teams
Remote and distributed working are increasing in popularity, but many organisations underestimate the leadership challenges. Learn from those who are doing this effectively.
-
Scaling Security, from Device to Cloud
Implementing effective security is vitally important, regardless of where you are deploying software applications
-
Evolving Java
JVM futures, JIT directions and improvements to the runtimes stack is the theme of this year’s JVM track.
