Everything Is a Plugin: How the Backstage Architecture Helps Platform Teams at Spotify and Beyond Spread Ownership and Deliver Value

Back in 2014, platform engineers at Spotify identified a problem: the number of internal tools was growing rapidly, and as a result engineers around the company were struggling to find the tools and information they needed to work efficiently. Reducing the number of tools is the obvious way to address this problem, but how do you consolidate all the tools in one place without disempowering and slowing down the teams working on them?

Fast-forward 10 years, and Backstage has had a massive impact on developer productivity at Spotify, making teams that use it heavily about 2.3x more active on GitHub, and 30% more productive overall. And it doesn’t stop there: in 2020, we open-sourced the Backstage framework to help other companies build their own internal developer portal – and thousands of companies are using it today.

Backstage is now a central pillar of our software development workflow: it catalogs our software ecosystem, helps our engineers follow best-practices and measure the tech health of the components they own, and centralizes and streamlines repeatable tasks. But how can we bring all this together in Backstage? And how does the value we get from it internally translate to adopters of the open-source framework? The answer is plugins.

In this talk we explain how the Backstage plugin system brings disparate pieces of functionality together in one place, helping platform teams to build independently while still shipping a cohesive product for our engineering teams to use. We’ll explain the architecture of the plugin system, and walk through some specific examples of how Backstage (and individual plugins themselves) can be extended and interconnected, and how we make use of this to distribute ownership at Spotify.

Finally, we’ll talk about how the extensibility model of Backstage helps companies offering commercial products in the IDP space add value to the open-source Backstage offering, from specific plugins to entirely new IDP products like Spotify Portal for Backstage. 

Interview:

What's the focus of your work these days?

Pia: I lead the Platform Developer Experience Studio at Spotify, which means I'm supporting teams to build a greater developer experience internally, and bring some of these great products to the market as open-source or in our commercial Backstage product suite.

Mike: I'm the tech lead for Backstage at Spotify, which means I spend my days thinking about how to make Backstage a better product for adopters, internal users, contributors, and customers.

What's the motivation for your talk at QCon London 2024?

We're excited to share the benefits we've seen from building a powerful extensibility model into Backstage, in terms of both encouraging contributions in the open-source framework, and enabling distributed ownership of capabilities in our internal instance. 

How would you describe your main persona and target audience for this session?

This talk is aimed at senior engineers and architects with an interest in building extensible software solutions. No prior experience with Backstage is required!

Is there anything specific that you'd like people to walk away with after watching your session?

We're hoping to share the things we've learned building an extensible framework, and inspire others to incorporate more extensibility along with open- and inner-source contribution models into their software.


Speaker

Pia Nilsson

Director of Engineering @Spotify

Pia Nilsson is the Platform Developer Experience (PDX) lead at Spotify. She started her career off as a backend engineer for 14 years working across telecom, pharma, retail, banking and gaming. Half of that time was spent as a self employed contractor, where she began recognizing patterns in the challenges engineers were experiencing. This interest and strong belief that teams can be more effective and have more fun(!) led her to focus more and more on platform-oriented work. She was an early adopter of devops principles and pair-programming, and also led classes in clean code and test driven development. Pia joined Spotify to lead the CI (Continuous Integration) platform and gradually expanded into other domains connected to developer experience and effectiveness. She led the team that built Backstage, where her focus was to increase the speed of iteration so that Spotify could build products faster. She continues to lead that team while also working to create an exceptional developer experience through the tools and infrastructure her team provides to every engineer at Spotify.

Read more

Speaker

Mike Lewis

Staff Engineer @Spotify

Mike Lewis is a Staff Engineer at Spotify, and has been focused exclusively on Backstage for the last few years after an initial stint working on Spotify Premium. Mike has contributed to significant features in the open-source Backstage framework, as well as Spotify commercial offerings related to Backstage including the Soundcheck and RBAC plugins.

Read more

Date

Monday Apr 8 / 03:55PM BST ( 50 minutes )

Location

Fleming (3rd Fl.)

Share

From the same track

Session architecture

Beyond Platform Thinking at RB Global – Build Things No One Expects, in a Place No One Expects It

Monday Apr 8 / 10:35AM BST

Ever wondered what is it really like to move from a poorly integrated COTS legacy architecture to a well factored API driven cloud native platform and event driven architecture, globally deployed in AWS EKS with modern observability and daily software releases?

Speaker image - Ranbir Chawla

Ranbir Chawla

SVP of Engineering @RB Global, Previously SME @Thoughtworks Digital Platform Strategy Team, 30+ Years in Software Engineering and Entrepreneurship

Session Data Architecture

Architecting for Data Products

Monday Apr 8 / 01:35PM BST

Data Mesh principles have brought data products to the forefront of data architecture discussions. However, due to the variety of applications and the vast technology landscape, it is hard to bring those principles into practice.

Speaker image - Danilo Sato

Danilo Sato

Global Head of Technology - Data & AI @Thoughtworks, Author of DevOps in Practice, CD4ML, and DataIQ UK’s 100 Most Influential People in Data in 2022–2023

Session platforms

How We Created a High-Scale Notification System at Duolingo That Delivered Millions of Messages Within Seconds During a Super Bowl Commercial Break

Monday Apr 8 / 02:45PM BST

Building a notification system may seem trivial, but what about building one that could reach million of users within a few seconds? What about doing that right after your advertisement airs?

Speaker image - Vitor Pellegrino

Vitor Pellegrino

Site Reliability Engineering @Duolingo

Speaker image - Zhen Zhou

Zhen Zhou

Software Engineer at @Duolingo, Previous Theoretical Computer Science Enthusiast @CMU

Session

Unconference: Architectures You've Always Wondered About

Monday Apr 8 / 05:05PM BST

An unconference is a participant-driven meeting. Attendees come together, bringing their challenges and relying on the experience and know-how of their peers for solutions.

Session

0 → 1, shipping Threads in 5 months

Monday Apr 8 / 11:45AM BST

In Jan 2023, we received word that we’d need to build a microblogging service to compete with Twitter in a couple of months. This is the (technical) story of the small team that was assembled to take on that challenge, and that shipped a new social network in July.

Speaker image - Zahan Malkani

Zahan Malkani

Software Engineer @Meta