Fast Eventual Consistency: Inside Corrosion, the Distributed System Powering Fly.io

Fast Eventual Consistency: Inside Corrosion, the Distributed System Powering Fly.io

Presented by Somtochi Onyekwere, this talk delves into the design and implementation of Corrosion, an open-source distributed system designed to handle fast eventual consistency across Fly.io's cloud platform, particularly focusing on:

• The background and necessity of Corrosion, developed as a replacement for Consul to improve scalability and consistency in data dissemination across nodes.
• Corrosion's architecture, which uses:
  • SQLite: Each node has an SQLite database for local data management.
  • Conflict-free Replicated Data Types (CRDTs): Used for conflict resolution when replicating data across the distributed system.
  • SWIM Gossip Protocol: Manages cluster membership, ensuring nodes are aware of each other’s state.
  • QUIC Transport Protocol: Enables efficient data packet exchange between nodes.
• The role of CRDTs in maintaining eventual consistency by allowing independent updates which synchronize without requiring a fixed sequence of operations, ensuring replicas eventually converge to the same state.
• The system's decision-making process under CAP theorem constraints, focusing on availability and partition tolerance at the expense of immediate consistency.
• The significance of efficient data routing and dissemination to provide fast operations across Fly.io's globally distributed network infrastructure.
• Potential issues, including the lack of built-in authorization or authentication and challenges with destructive changes or schema management.
• Lessons learned from transitioning from Consol to Corrosion, emphasizing the importance of optimizing data write/read processes for distributed environments.

Implemented in Rust for its memory safety and effectiveness as a systems programming language, Corrosion is portrayed as a robust solution for modern cloud-based application requirements, especially those demanding a high degree of data consistency across distributed networks.

This is the end of the AI-generated content.

This talk explores Corrosion, an open-source solution built on SQLite that delivers fast, eventually consistent replication across distributed nodes. Initially developed at Fly to replace Consul for service discovery, Corrosion has evolved to power the platform, enabling the rapid dissemination of machine and service data across numerous nodes with a p99 latency of one second. Leveraging technologies such as Conflict-free Replicated Data Types (CRDTs) for conflict resolution, the SWIM gossip protocol for membership management, and the QUIC transport protocol, Corrosion is implemented in Rust and is open-source. In this talk, we will explore these technologies, dive into architectural and design decisions, and discuss how Corrosion is used in production at Fly. Whether you're familiar with these concepts or they're new to you, join us for an in-depth look at Corrosion.