Stateful Cloud Services at Neon: Navigating Design Decisions and Trade-Offs

QCon London 2024

Session storage

Stateful Cloud Services at Neon: Navigating Design Decisions and Trade-Offs

Monday Apr 8 / 05:05PM BST, Churchill (Ground Fl.)

Abstract

Cloud-native engineering talks frequently focus on implementing and operating the stateless parts of applications. Designing and deploying stateful services such as databases and message brokers, involves additional concerns: we must ensure data durability as well as service availability, and consider the cost/performance tradeoffs of placing data on local disk, block storage, or object storage.

This talk delves into the often-overlooked complexities of stateful cloud service design, using Neon Serverless Postgres as a case study. We will discuss the key decision points: where do you put your data, and how many copies? How do we ensure availability when starting a new node with an empty cache drive? How can we efficiently scale a service that requires local disk storage? Additionally, we'll examine Kubernetes' role in this landscape and the cost implications of achieving data durability across multiple availability zones or regions.

Topics

storage Platform Engineering SaaS
76% senior dev or higher
1:11 speaker ratio
60+ practitioners

QCon London 2024 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.

Share

From the same track

Monday 8 April

10:35 Churchill (Ground Fl.) Session AWS Efficient Serverless Development: Latest Patterns and Practices on AWS Yan Cui AWS Serverless Hero & Independent Consultant 11:45 Churchill (Ground Fl.) Session DevSecOps Efficient DevSecOps Workflows With a Little Help From AI Michael Friedrich Senior Developer Advocate @GitLab 13:35 Rutherford (4th Fl.) Unconference Unconference: Cloud-Native Engineering 14:45 Churchill (Ground Fl.) Session architecture Building SaaS From Scratch Using Cloud-Native Patterns: A Deep Dive Into a Cloud Startup Joni Collinge Founding Software Engineer @Diagrid 15:55 Churchill (Ground Fl.) Session Infrastructure as Code Borderless Cloud: Designing, Implementing, and Securing Apps Across Multiple Clouds Adora Nwodo Senior Software Engineer, Founder of NexaScale, Multi-Published Author 17:05 Churchill (Ground Fl.) Session storage Stateful Cloud Services at Neon: Navigating Design Decisions and Trade-Offs John Spray Storage Engineering Lead @neon.tech, Formerly Redpanda, Inktank (Ceph), Whamcloud (Lustre)