Understanding the Challenges with Databases and DevOps

DevOps has become one of the fastest growing terms in IT for the past five years, and this comes to no surprise. DevOps is a term that is a compound of the words development and operations, where it refers to the collaboration and communication between (software) developers and information technology operations professionals. It changes the way these two groups of people work in a cultural and environmental way.

This improves the building, testing and releasing of software and allows more reliable, frequent and rapid deployments.

In DevOps the traditional development process is changed to a continuous process of developing and releasing the product. This circular process is much more suitable for short development cycles, where the cycle iterates over code (plan), build (create), test (verify), package, release, configure and monitor.

As quick and continuous release cycles dictate frequent updates from development to production, how do databases fit in this picture?

Most RDBMS databases are mostly built around securing the integrity of the data. This means that certain trade-off s have been made on how the database copes with (schema) changes. In general any change to the structure of the data in an RDBMS will involve locking and take a painfully long to apply. To overcome this problem, many DevOps will rather favor schemaless datastores like MongoDB. Schemaless datastores have made the trade-off of flexibility over consistency.

Database Collaboration in DevOps

As the DevOps method requires more close collaboration, this shifts a lot in the roles of the people that are part of the team and some of them will now share parts of their roles. The developer not only develops the application, but also is part of the release process. The developer is more likely to know most about the application, so the developer should also know how to monitor the application best. Similarly, QA has to know what they are going to test, how it works and what environment it is being hosted in. The operations members will likely know more about the internals of the application they are hosting, saving valuable time when troubleshooting issues.

With the shift in roles, additional responsibilities will be inevitable. The developer can no longer hide behind the fact that “she is just the developer” when it comes to operational issues, similarly for the other roles. As the whole team now owns the application, the team members will also feel more connected to the product that they are responsible for. Naturally they will feel more responsible when an outage occurs.

So who will be responsible for the role of DBA in a DevOps environment? In most cases this will be a collaborative role between the developer and the ops roles. The developer will drive the initiative, changes and performance aspects while the ops will handle the consistency and security.

What Does This Mean for Databases?

As mentioned before relational databases are less flexible by nature, while DevOps actually requires more flexibility. It will be a continuous trade-off between Dev and Ops for the solution chosen. Regardless of the solution chosen, there are many challenges that need to be overcome.

The first challenge will be deployment automation. As continuous deployments will be part of the DevOps process, the entire environment needs to be fully automated, provisioned and primed with the necessary data.

The second challenge will be the incompatibility between relational databases and microservice architectures. Microservices have, by definition, a shared-nothing architecture. The reasoning behind this is to lift any dependencies on other microservices and to prevent the outage of one microservice to affect the other. This means that in its purest form a microservice will be nothing more than a single table in a single schema on a database cluster.

The third challenge is collaboration. Collaboration between the members of a DevOps team will be key to its success. Communication is the most important element in collaboration, so it is essential that every member of the team is up to date with the latest information. Communication channels, also known as ChatOps, will play a big role in this.

In this whitepaper we will touch upon each of these challenges one by one. We will also see how Severalnines ClusterControl can be used to address these challenges.