Conference:March 6-8, 2017
Workshops:March 9-10, 2017
Presentation: Causal Consistency For Large Neo4j Clusters
Location:
- Fleming, 3rd flr.
Duration
Day of week:
- Wednesday
Level:
- Advanced
Persona:
- Developer
Key Takeaways
- Understand what is meant by Causal clustering, and it's application
- Understand how to build applications with Raft
- Decipher whether you can you mix asynchronous replication for scale with Raft for safety (yes!)
Abstract
In this talk we'll explore the new Causal clustering architecture for Neo4j. We'll see how Neo4j uses the Raft protocol for a robust underlay for intensive write operations, and how the asynchronous new scale-out mechanism provides enormous capacity for very demanding graph workloads.
We'll discuss the cluster architecture's new causal consistency model. Causal consistency is a big leap forward compared to the commonplace eventual consistency which makes it convenient to write applications that use the full capacity of the cluster. In particular we'll show how despite the mixture of concensus protocols and asynchronous replication, that Neo4j allows users to read their own writes straightforwardly and discuss why this is such a difficult achievement in distributed systems.
For the application developer, we'll show how Neo4j's Causal Clustering optimised drivers makes it easyto write applications that scale smoothly from a single server to a large, distributed cluster: a practical motivation for the distributed systems enthusiast.
Interview
I work on the Neo4j graph database, and focus on fault tolerance and scale generally, and have recently been working on consensus protocols and strong consistency models.
Eventual consistency is great for databases but less good for developers. Strong consistency is the opposite. I’ve been working on some database middleware that provides causal consistency - quite strong, but still allowing large scale.
People who are interested in distributed systems, especially databases. Also those who are planning to deploy databases. It’s a technical-ish talk so anyone in that kind of role might enjoy it.
Beginner. If you try to do complex distributed systems talks in 50 minutes, you’ll fail. This talk is deliberately accessible, partly as an homage to the design of the key protocol it uses - Raft - which strives to be humane where other protocols have tried to be clever.
They will be able to build a system with Neo4j after this.
I’m biased, but I think it’s neo4j. Longer term I actually think hardware architecture is going to stop being dull and start being radical again.
Similar Talks
Tracks
-
Architecting for Failure
Building fault tolerate systems that are truly resilient
-
Architectures You've Always Wondered about
QCon classic track. You know the names. Hear their lessons and challenges.
-
Modern Distributed Architectures
Migrating, deploying, and realizing modern cloud architecture.
-
Fast & Furious: Ad Serving, Finance, & Performance
Learn some of the tips and technicals of high speed, low latency systems in Ad Serving and Finance
-
Java - Performance, Patterns and Predictions
Skills embracing the evolution of Java (multi-core, cloud, modularity) and reenforcing core platform fundamentals (performance, concurrency, ubiquity).
-
Performance Mythbusting
Performance myths that need busting and the tools & techniques to get there
-
Dark Code: The Legacy/Tech Debt Dilemma
How do you evolve your code and modernize your architecture when you're stuck with part legacy code and technical debt? Lessons from the trenches.
-
Modern Learning Systems
Real world use of the latest machine learning technologies in production environments
-
Practical Cryptography & Blockchains: Beyond the Hype
Looking past the hype of blockchain technologies, alternate title: Weaselfree Cryptography & Blockchain
-
Applied JavaScript - Atomic Applications and APIs
Angular, React, Electron, Node: The hottest trends and techniques in the JavaScript space
-
Containers - State Of The Art
What is the state of the art, what's next, & other interesting questions on containers.
-
Observability Done Right: Automating Insight & Software Telemetry
Tools, practices, and methods to know what your system is doing
-
Data Engineering : Where the Rubber meets the Road in Data Science
Science does not imply engineering. Engineering tools and techniques for Data Scientists
-
Modern CS in the Real World
Applied, practical, & real-world dive into industry adoption of modern CS ideas
-
Workhorse Languages, Not Called Java
Workhorse languages not called Java.
-
Security: Lessons Learned From Being Pwned
How Attackers Think. Penetration testing techniques, exploits, toolsets, and skills of software hackers
-
Engineering Culture @{{cool_company}}
Culture, Organization Structure, Modern Agile War Stories
-
Softskills: Essential Skills for Developers
Skills for the developer in the workplace