Stateful Cloud Services at Neon: Navigating Design Decisions and Trade-Offs

QCon London 2024

Session storage

Stateful Cloud Services at Neon: Navigating Design Decisions and Trade-Offs

Monday Apr 8 / 05:05PM BST (50 minutes), Churchill (Ground Fl.)

Abstract

Cloud-native engineering talks frequently focus on implementing and operating the stateless parts of applications. Designing and deploying stateful services such as databases and message brokers, involves additional concerns: we must ensure data durability as well as service availability, and consider the cost/performance tradeoffs of placing data on local disk, block storage, or object storage.

This talk delves into the often-overlooked complexities of stateful cloud service design, using Neon Serverless Postgres as a case study. We will discuss the key decision points: where do you put your data, and how many copies? How do we ensure availability when starting a new node with an empty cache drive? How can we efficiently scale a service that requires local disk storage? Additionally, we'll examine Kubernetes' role in this landscape and the cost implications of achieving data durability across multiple availability zones or regions.

Topics

76% senior dev or higher
1:11 speaker ratio
60+ practitioners

QCon London 2024 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.

Slides & video

Slides Not available

Share

From the same track

Monday 8 April

10:35 Churchill (Ground Fl.) Session AWS Efficient Serverless Development: Latest Patterns and Practices on AWS Yan Cui AWS Serverless Hero & Independent Consultant 11:45 Churchill (Ground Fl.) Session DevSecOps Efficient DevSecOps Workflows With a Little Help From AI Michael Friedrich Senior Developer Advocate @GitLab 13:35 Rutherford (4th Fl.) Unconference Unconference: Cloud-Native Engineering 14:45 Churchill (Ground Fl.) Session architecture Building SaaS From Scratch Using Cloud-Native Patterns: A Deep Dive Into a Cloud Startup Joni Collinge Founding Software Engineer @Diagrid 15:55 Churchill (Ground Fl.) Session Infrastructure as Code Borderless Cloud: Designing, Implementing, and Securing Apps Across Multiple Clouds Adora Nwodo Senior Software Engineer, Founder of NexaScale, Multi-Published Author 17:05 Churchill (Ground Fl.) Session storage Stateful Cloud Services at Neon: Navigating Design Decisions and Trade-Offs John Spray Storage Engineering Lead @neon.tech, Formerly Redpanda, Inktank (Ceph), Whamcloud (Lustre)