You should never run stateful systems on Kubernetes. Or should you? Kubernetes has come a long way since its inception as the way to orchestrate modern stateless applications. In this episode, Vinay and Eddie take a trip through its journey toward increasing adoption in stateful workloads.

Like the technology, their discussion is wide ranging and covers topics from how its continued development and changes in software development and operations thinking have supported this, how application, environmental, and organizational considerations should guide your decision to implement it, and more.

If you’re looking to get the ins and outs of Kubernetes and stateful workloads today, and where it is going. This is your episode.

Key insights

⚡DBaaS is not the turnkey, “low ops” choice it promised to be

DBaaS has taken a lot of the heavy lifting off of the DBA’s shoulders and is considered the low ops choice for stateful workloads but Vinay and Eddie question this assumption. They argue that, although it has made aspects of the operational milieu more efficient and reliable, it is not the turnkey solution it promises to be, it still requires a lot of expert administration. 

⚡Kubernetes is the great environment equalizer for modern workloads

Due to Kubernetes’s ability to act as a standardized base layer across environments, sophisticated tools such as operators, various interfaces and services, and robust community, it is fast becoming the way to implement modern applications and their databases. This is being proven by its increasing inclusion in on-prem and cloud environments, and various workloads.  

⚡Kubernetes illustrates that software is more like gardening than engineering

In engineering, you’re expected to get the same result as the next if you follow the blueprint. In gardening, the same inputs can produce different results — software is the same. Kubernetes isn’t for everything, always, including databases. Instead, its selection should be determined by needs and the context, whether that context be application stage, team capabilities, etc.

Episode highlights

💡 Eddie’s elevator pitch for placing databases on Kubernetes[00:04:24]

Vinay asks Eddie to give his elevator pitch on why you should run your databases on Kubernetes today. Beginning his response with, “The answer is, why wouldn’t you run your database on Kubernetes,” Eddie explains that, similar to VMs and bare metal, Kubernetes provides a standard that allows you to forget everything below your applications. So whether you’re talking about networking, monitoring, resiliency, storage, etc., Kubernetes can provide that, and more, out-of-box for your stateful systems.

💡K8s has developed to satisfy stateful workload needs[00:08:47]

After establishing that Kubernetes is historically associated with stateless systems, but is now increasingly being adopted for stateful ones as well, Vinay and Eddie explore what has actually changed for this shift to happen. There has been a lot of work, such as the creation of StatefulSets, adoption of storage classes, integration of CNIs and CSIs, that has ultimately made incorporating database workloads into Kubernetes environments more suitable by increasing the their reliability.

💡K8s is your great equalizer across environments[00:22:23]

Eddie discusses how Kubernetes enables organizations to efficiently move workloads across environments by acting as a common denominator, highlighting how operators are an example of this through their codification of best practices and detailing their mechanics such as reconcile loops work. He then speculates on how they can be extended via AI in the future.

💡Kubernetes is sharp knife, not a silver bullet[00:29:17]

When should an organization not use Kubernetes for their stateful systems? Eddie suggests that, at minimum, you need specific skillsets or services such as Kubernetes as a Service (KaaS) and not have special hardware requirements. To underline the point, Vinay and Eddie review a few cases where simple changes can result in catastrophic consequences for your databases.

💡Scale is still a limiter but not completely[00:31:38]

Kubernetes tops out at 5000 nodes per cluster and 110 pods per node. However, Eddie suggests that the days of gigantic clusters are nearing their end and micro-clusters will take their place at dawn. He not only discusses how this is a continuation of the “micro” trend, but how K8s enables this and why it might be better, e.g. reduced bottlenecks, blast radius, etc.

💡Want Day 2? Go with operators[00:40:58]

Although they aren’t the only way to do so, Eddie believes that operators are the way to go when running databases on Kubernetes. They not only handle the Day 1, but Day 2 ops as well due to their unique mechanics and can be used in combination in various patterns to handle complex operational workflows; it’s not the only way, but the way to go.