When StatefulSets are not enough

📺 Click to view the youtube embed player and accept their cookies.
Session Abstract

K8s StatefulSets present significant hurdles for scaling and migrating large-scale cloud database workloads. We’ll cover scaling strategies beyond vanilla StatefulSets and share lessons on executing zero-downtime live migrations using custom controllers, durable execution workflows, and tackling complex synchronization problems in ClickHouse Cloud.

Session Description

This is a densely packed technical talk that teaches you Auto Scaling architecture, Kubernetes StatefulSets and their limitations, various scaling strategies and statefulset alternatives. We also look at building custom kubernetes controllers for the purpose of changing our orchestration code-path, and investigate leveraging durable execution workflows like Temporal for managing zero downtime migrations.

You will understand the Pros and Cons of Break-First and Make-First scaling models and which to use when. We focus on the challenges that prevent doing Make-first with traditional StatefulSets. We discuss open source projects such as Advanced StatefulSets, OpenKruise and a custom Multi-StatefulSet approach. We go into the story of moving from one mode of orchestrating StatefulSet to another via a Live Migration, without breaking the running queries. Finally we end with some ClickHouse specific problems we encountered during the migrations and how we solved them.

Frannz Salon
16.Jun 2025
14:00pm - 14:40pm
Talk