You are viewing content from a past/completed QCon

Presentation: Who Broke Prod? - Growing Teams Who Can Fail Without Fear

Track: DevOps & DevEx: Remove Friction, Ship Code, Add Value

Location: Churchill, G flr.

Duration: 1:40pm - 2:30pm

Day of week: Tuesday

Share this on:

This presentation is now available to view on

Watch video with transcript

What You’ll Learn

  1. Listen about the culture of blame, what it is and how it impacts a team’s performance.
  2. Learn what are some practical things to do in order to eliminate the culture of blame.
  3. Hear how to transition into a DevOps role to achieve better results in your team.


In a world where we can deploy software and infrastructure changes within seconds, it’s reasonably easy to break stuff. When the inevitable bad stuff happens, humans tend to look for someone to blame. Research has shown that the most productive teams are those who don’t have to fear blame. On high-performing teams, people feel safe taking risks, making mistakes, and learning from them – so how do you build an environment where it’s OK to fail? In this talk we’ll explore the factors that contribute to psychological safety on a delivery team. We’ll look at some of the steps that leaders and team members can take to foster a culture of blameless failure that encourages innovation and collaboration. We’ll look at soft-skills, CI/CD practices, retrospective processes and tooling that can help you to build a culture of trust and ownership within your own team.


What is your talk about?


It's quite common for me to come across groups of people who instinctively point the finger of blame at each other. And we know from some of the great research around DevOps that the most effective teams are teams that trust each other, and can collaborate together and where you've got a culture of blame that stands in the way of success. The talk is really my experience and some tangible changes that you can make, either as a leader or as a team member, to help breakdown that culture of blame.


What are some specific things that you are going to talk about?


I am going to talk about blameless postmortems, but trying not to talk about the scary bit of postmortem and talk more about all the good stuff that can come out of it. I'm going to talk about some little subtle things that you might not think about around the CI/CD processes, while everybody is rushing to go and embrace continuous delivery and use all these wonderful tools which integrate their development processes, maybe they're contributing to the blame culture by ensuring that there's something in the middle that you can blame rather than working together to solve a problem? I talk about the culture of experimentation, what can you do as a leader to help foster a culture of experimentation. The more that we experiment, the braver we become at preparing for failure. And that can be small things such as allowing developers time to experiment with something entirely new, to go and play with a shiny piece of technology that you’ve wanted to for a while.


Who's the audience you're talking to?


It's definitely targeted at teams members and team leaders; people who have been working in a team and can see that there are things that could change, that you could do better.


When you say team leaders, are you talking about individual contributors or management types?


Both. As somebody who has been a developer and then has moved up through the ranks, I believe that for a team to be effective you don't necessarily need a manager but you need some people, whether those are your technical leaders or people leaders, some people on your team who help the team to improve. In an effective DevOps team, everybody is helping the team to improve. Your natural leaders, your senior developers, your technical architects and software development managers.


In your abstract you talk about psychological safety. How do you build that in your team?


The whole talk is around these real things that you can do to make changes .Regarding psychological safety, I talk a bit around retrospective techniques. Even if you're not running Scrum, to reflect regularly, providing some ideas of things you can do in your retrospectives to make the environment safer to provide honest and constructive feedback. For example, for teams who are finding it hard to talk honestly in front of each other, to use an online tool. Remote teams use an online tool to conduct their retrospectives. Even if you are not remote, using an online tool to capture the talking points and talk from there. This gets over the first hurdle of people being brave enough to raise their concerns.


What do you want someone who comes to your talk to walk away with?


I want them to have a list of things they want to try, and to feel inspired to go and make a couple of small changes on their team which could help eliminate the culture of blame.

Speaker: Emma Button

COO and Co-founder @nubego Cloud Consulting

Emma is an agile software delivery specialist with 16 years’ experience building and deploying enterprise software. In recent years, Emma has specialised in the cultural, technology and process changes needed to grow high-performing software & operations teams responsible for building and maintaining applications in the Cloud. Emma leads engineering teams going through cultural change; inspiring team members to transition from traditional working methods, through agile and lean practices and into the DevOps mindset.

Find Emma Button at


  • Architectures You've Always Wondered About

    Hard-earned lessons from the names you know on scalability, reliability, security, and performance.

  • Machine Learning: The Latest Innovations

    AI and machine learning is more approachable than ever. Discover how ML, deep learning, and other modern approaches are being used in practice.

  • Kubernetes and Cloud Architectures

    Learn about cloud native architectural approaches from the leading industry experts who have operated Kubernetes and FaaS at scale, and explore the associated modern DevOps practices.

  • Evolving Java

    JVM futures, JIT directions and improvements to the runtimes stack is the theme of this year’s JVM track.

  • Next Generation Microservices: Building Distributed Systems the Right Way

    Microservice-based applications are everywhere, but well-built distributed systems are not so common. Early adopters of microservices share their insights on how to design systems the right way.

  • Chaos and Resilience: Architecting for Success

    Making systems resilient involves people and tech. Learn about strategies being used, from cognitive systems engineering to chaos engineering.

  • The Future of the API: REST, gRPC, GraphQL and More

    The humble web-based API is evolving. This track provides the what, how, and why of future APIs.

  • Streaming Data Architectures

    Today's systems move huge volumes of data. Hear how the innovators in this space are designing systems and leveraging modern data stream processing platforms.

  • Modern Compilation Targets

    Learn about the innovation happening in the compilation target space. WebAssembly is only the tip of the iceberg.

  • Leaving the Ivory Tower: Modern CS Research in the Real World

    Thoughts pushing software forward, including consensus, CRDT's, formal methods & probabilistic programming.

  • Bare Knuckle Performance

    Crushing latency and getting the most out of your hardware.

  • Leading Distributed Teams

    Remote and distributed working are increasing in popularity, but many organisations underestimate the leadership challenges. Learn from those who are doing this effectively.

  • Full Cycle Developers: Lead the People, Manage the Process & Systems

    "Full cycle developers" is not just another catch phrase; it's about engineers taking ownership and delivering value, and doing so with the support of their entire organisation. Learn more from the pioneers.

  • JavaScript: Pushing the Client Beyond the Browser

    JavaScript is not just the language of the web. Join this track to learn how the innovators are pushing the boundaries of this classic language and ecosystem.

  • When Things Go Wrong: GDPR, Ethics, & Politics

    Privacy, confidentiality, safety and security: learning from the frontlines, from both good and bad experiences

  • Growing Unicorns in the EU: Building, Leading and Scaling Financial Tech Start Ups

    Learn how EU FinTech innovators have designed, built, and led both their technologies and organisations.

  • Building High Performing Teams

    To have a high-performing team, everybody on it has to feel and act like an owner. Learn about cultivating culture, creating psychological safety, sharing the vision effectively, and more

  • Scaling Security, from Device to Cloud

    Implementing effective security is vitally important, regardless of where you are deploying software applications.