QCon10Years
Conference:March 7-9, 2016
Workshops:March 10-11, 2016
Sign up to be informed when registration opens for QCon London 2017 (3/6 - 3/10)

Presentation: Effortless Eventual Consistency with Weave Mesh

Track: 
Modern CS in the real world

Location:

  • Mountbatten, 6th flr.

Duration

Duration: 
10:35am - 11:25am

Day of week:

  • Tuesday

More talks on:

Key Takeaways

  • Gain a functioning understanding of what Gossip is and a way to contextualize it in a broader space of communication protocols and distributed systems
  • Learn various implementations of Gossip and understand where each can be applied successfully.
  • Learn how Gossip can be used to build interesting abstractions, such as a Raft consensus algorithm on top of Weave Net.

Abstract

We built Weave Net to make it easy to network your Docker containers however you like. At its core, Weave Net is powered by an eventually-consistent, partition-tolerant, completely decentralised mesh. We're packaging this technology as Weave Mesh, so that you can build your own distributed applications with the same simplicity and reliability as Weave Net. In this talk we will explain the theory behind Weave Mesh, some of the key features we learned were important, and demonstrate some exciting use-cases, like distributed caching and state replication.

Interview

Question: 
QCon: What is Weave Mesh?
Answer: 
Peter: Weave Net is the primary product from Weaveworks. It’s a SDN for the container and Docker ecosystem. It’s totally free, totally open source. At the core of Weave Net is a gossip-based communication protocol, which we developed specifically for the SDN.
Recently, we decided it would be interesting to try to extract that, and turn it into its own thing. We’ve been experimenting with this idea since then: Weave Mesh is the gossip core of the Weave Net SDN, plus a set of layers around it that allow you to do interesting things.
At the lowest level, Weave Mesh gives you: - membership - multicast-style broadcast - basic UDP-style, peer to peer packet connection
With those things, you can build interesting abstraction layers. One I’m working on now is bootstrapping the Raft consensus algorithm on top of Weave Mesh.
The idea with it is if you deploy or embed Weave Mesh, it should make creating a clustered application very easy. And because it’s extracted from the battle-hardened Weave Net SDN, you can trust that it’s reliable and robust. From there, you can build a lot of abstractions that are actually really difficult right now, in this Cloud Native computing world. The sort of things where you would normally need to talk to ZooKeeper to find your peers, or use Consul to do service discovery, or something like this—Weave Mesh will hopefully be able to solve a lot of these problems with a lot fewer caveats.
Question: 
QCon: What is the focus of the talk?
Answer: 
Peter: The talk is going to first of all be about gossip. We will discuss what gossip is, and then walk through the principles and applications of gossip protocols.
We will go through several gossip implementations. We’ll talk about what makes them interesting, and then (given this foundation) we’ll then walk through what can you build on top them.
These things apply generally to any gossip foundation, but for this example, we will talk about the specific things that we have built on top of Mesh.
Question: 
QCon: What do you want the attendees to walk away with from this talk?
Answer: 
Peter: The biggest takeaway is that the people in the audience will gain a functioning understanding of what gossip is, and a way to contextualize it in a broader space of communication protocols and distributed systems.
What is gossip and what are its semantics? What it is good for? How do gossip-y protocols compare to Paxos, Raft and other strongly consistent protocol? How do these things compare to your standard RDBMS? You will be able to better know when would pick one over the other and understand their capabilities?
Then, based on that, you can decide what makes sense to use for your application. And we show some interesting examples.

Peter Bourgon Elsewhere

GitHub

Similar Talks

Hot code is faster code - addressing JVM warm-up
Building a Modern Security Engineering Team
Real-Time Fraud Detection with Graphs
Successful Go program design, 6 years on
Rust: Systems Programming for Everyone
Using Pony for Fintech
Lead the Revolution by Being Ordinary
Mens Sana in Corpore Sano: Optimising Mind & Body
The Microservices and DevOps Journey

Similar Talks

Hot code is faster code - addressing JVM warm-up
Building a Modern Security Engineering Team
Real-Time Fraud Detection with Graphs
Successful Go program design, 6 years on
Rust: Systems Programming for Everyone
Using Pony for Fintech
Lead the Revolution by Being Ordinary
Mens Sana in Corpore Sano: Optimising Mind & Body
The Microservices and DevOps Journey

Tracks

Covering innovative topics

Monday, 7 March

Tuesday, 8 March

Wednesday, 9 March

FULL SCHEDULE

Conference for Professional Software Developers

Follow QCon

Contact

Menu

QCons around the World

Qcon