Tips for Upgrading Percona XtraDB Cluster to 8.0

MySQL 8.0 has been available for a while and Percona XtraDB Cluster 8.0 has also been available for some time as well. Also, MySQL 5.6 is coming up to its End-of-Life and people using PXC 5.6 will also want to migrate to a more recent version soon. Let’s take a look at the process and share some tips around it.

Remember, it is MySQL Underneath…

For starters, you have to keep in mind that Galera is just a plugin that works with MySQL so you have to ensure that you adhere to the migration rules and processes for your particular MySQL version that’s used in your PXC. If we are running PXC 5.7 and you want to upgrade to PXC 8.0, you need to first check all the boxes on the MySQL upgrade checklist. We covered some of that in our blog describing the upgrade process between MySQL 5.7 and MySQL 8.0. It also means that only upgrade paths supported by MySQL are viable – for example, you cannot upgrade PXC 5.6 directly to PXC 8.0 as direct MySQL upgrade from 5.6 to 8.0 is not supported.

Read the Upgrade Documentation

As usual, whenever you plan an upgrade, you should familiarize yourself with the documentation prepared by the vendor. Percona has a web page dedicated to explaining the upgrade process and caveats around it. We’ll go over some of those points in this blog post.

Configuration Changes

There are a couple of changes in the default configuration settings that may trigger issues in the upgrade process – pxc_strict_mode is set to ENFORCING by default. This mode blocks any kind of configuration that is experimental and may affect the stability of the cluster. Checks cover, among other things, used storage engines, binary log format, existence of primary keys and some other things. When upgrading, you may want to set it to PERMISSIVE to track the incompatibilities in the error log but otherwise let the PXC run even if some things are not in the best shape.

Another setting, pxc_encrypt_cluster_traffic, is also enabled by default, enforcing encryption of the Galera communication between nodes. It is not possible to mix nodes with encrypted and unencrypted nodes in the same cluster therefore you either have to set it up on all of the nodes or ensure you disable the pxc_encrypt_cluster_traffic on the new, PXC 8.0 nodes.

Change in the Default Authentication Plugin

We mentioned this in our blog post on migration to MySQL 8.0, but the change is so important that we want to reiterate it here as well – with MySQL 8.0 the default authentication plugin changes to caching_sha2_password – this may make some of the older applications incompatible with MySQL 8.0. Of course, you can change this setting but if you do not take it under consideration, it may backfire after the upgrade is complete.

Upgrade Processes

For starters, please keep in mind that, while not recommended, with some it is possible to have both PXC 5.7 and PXC 8.0 nodes running in the same cluster. This opens an opportunity to do an in-place, live upgrade. The process would be quite simple – stop PXC 5.7 node, upgrade it to PXC 8.0 by installing a new version and starting it. In the process data directory will be upgraded to the new version and the new PXC 8.0 node should be able to start properly and join the cluster. Then you proceed with the nodes one by one, migrating them from PXC 5.7 to PXC 8.0. Please keep in mind that you should avoid SST as a data directory from PXC 8.0 node cannot be used on 5.7. The other way around should work ok. To prevent SST from happening, ensure that the gcache size is large enough to accommodate writes and allow IST to happen. IST itself will work just fine.

If you have more free nodes, you can always perform the upgrade with two clusters on different major versions running in parallel and connected through asynchronous replication. What is important, such approach will let you test the PXC 8.0 more thoroughly as you will have it up and running for a while (basically for as long as you need it) and you can test your application on this cluster – at any point in time you can move some of the read-only workload to the PXC 8.0 seeing how queries are handled, if there are any errors or performance issues. 

If you use ClusterControl, this can be accomplished by using the “Create Slave Cluster” job.

You will be asked to decide where the initial data should come from, master PXC node or from the backup.

You will also need to pass the connectivity details like SSH user and key path.

Then you will be asked to pick the vendor and version – you probably want to use PXC 5.7 for now, you will upgrade the nodes later, testing the upgrade process itself.

Finally, you have to pass the node hostnames or IP addresses for ClusterControl to connect to and start setting the nodes up.

As the result you will have a second Percona XtraDB Cluster running in version 5.7, replicating from the original cluster. That cluster can be upgraded to the new version and, eventually, you can switch your traffic towards it. We have explained this process in detail in our previous blog post.

We hope that this short blog post will help you to prepare better to upgrade your Percona XtraDB Cluster to version 8.0.