How to Deploy Galera Cluster for MySQL using Docker Containers

Virtual Machines are great, and very useful when trying out new software. However, they might be an unnecessarily heavyweight solution when testing clusters, especially if these consist of multiple nodes running exactly the same software. Each VM runs a full-blown OS image. On the other hand, Linux Containers (LXC) are an efficient alternative to OS-level virtualization and can run multiple isolated systems on a single host. Docker is a wrapper around LXC, it automates the deployment of applications inside containers.

A notable advantage if you run with Docker + LXC is you can run many containers on a single host. They all share the same OS as the host, and when possible, the same binaries. Deployment can be extremely fast. Using Docker could be a good method if you want to spin multiple Galera nodes within a single host.

In this post, we will create MySQL Galera Cluster containers using Docker, and fire them up to form a database cluster running on a single host.

Install LXC and Docker

1. Docker works best on kernel 3.8 (due to a bug in LXC, at the time of writing). Install the kernel image for Raring:

   $ sudo apt-get update
   $ sudo apt-get install -y linux-image-generic-lts-raring linux-headers-generic-lts-raring
   $ sudo init 6

2. Verify the kernel version:

   $ uname -a
   Linux lxc 3.8.0-35-generic #52~precise1-Ubuntu SMP Thu Jan 30 17:24:40 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

3. Add the repository keychain locally, add Docker repository, update the package list and perform installation:

   $ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
   $ sudo sh -c "echo deb http://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list"
   $ sudo apt-get update
   $ sudo apt-get install -y lxc-docker mysql-client

4. Modify the forwarding policy for ufw:

   $ sudo nano /etc/default/ufw

   Modify the following line:

   DEFAULT_FORWARD_POLICY="ACCEPT"

Create Docker Image

1. Pull Ubuntu images from Docker repository:

   $ sudo docker pull ubuntu

2. Create Docker build script file:

   $ vim Dockerfile

   And paste the following lines:

   # # MySQL Galera Cluster 5.5.34-25.9/Galera 2.8/Ubuntu 12.04 64bit
   FROM ubuntu:12.04
   MAINTAINER Severalnines AB 
    
   RUN apt-key adv --keyserver keys.gnupg.net --recv-keys 1C4CBDCDCD2EFD2A
   RUN sed -i '/repo.percona.com/d'  /etc/apt/sources.list
   RUN sh -c 'echo "deb http://repo.percona.com/apt precise main" >> /etc/apt/sources.list'
   RUN sh -c 'echo "deb-src http://repo.percona.com/apt precise main" >> /etc/apt/sources.list'
   RUN apt-get -q -y update
   RUN LC_ALL=en_US.utf8 DEBIAN_FRONTEND=noninteractive apt-get -o Dpkg::Options::='--force-confnew' -qqy install nano wget psmisc libdbi-perl libdbd-mysql-perl libwrap0 perl libaio1 mysql-client percona-xtrabackup libssl0.9.8 libssl1.0.0 rsync netcat
   RUN wget --no-check-certificate https://launchpad.net/codership-mysql/5.5/5.5.34-25.9/+download/mysql-server-wsrep-5.5.34-25.9-amd64.deb
   RUN wget --no-check-certificate https://launchpad.net/galera/2.x/25.2.8/+download/galera-25.2.8-amd64.deb
   RUN dpkg -i galera-25.2.8-amd64.deb
   RUN dpkg -i mysql-server-wsrep-5.5.34-25.9-amd64.deb
   ADD ./my.cnf /etc/mysql/my.cnf
   EXPOSE 3306 4444 4567 4568

3. Create a default MySQL configuration file for our Galera nodes that will be used by Docker (as shown at the ADD line above):

   $ vim my.cnf

   And paste following lines:

   [mysqld]
   wsrep_provider=/usr/lib/galera/libgalera_smm.so
   wsrep_cluster_address=gcomm://
   wsrep_sst_method=rsync
   wsrep_cluster_name=galera_cluster
   binlog_format=ROW
   default_storage_engine=InnoDB
   innodb_autoinc_lock_mode=2
   innodb_locks_unsafe_for_binlog=1

4. Build the image:

   $ sudo docker build -t ubuntu_precise/galera:codership .

5. Verify whether the newly created image is listed by Docker:

   $ sudo docker images
   REPOSITORY             TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
   ubuntu_precise/galera  codership           ade20ad1d10b        3 minutes ago       425.8 MB
   ubuntu                 13.10               9f676bd305a4        46 hours ago        178 MB
   ubuntu                 saucy               9f676bd305a4        46 hours ago        178 MB
   ubuntu                 13.04               eb601b8965b8        46 hours ago        166.5 MB
   ubuntu                 raring              eb601b8965b8        46 hours ago        166.5 MB
   ubuntu                 12.10               5ac751e8d623        46 hours ago        161 MB
   ubuntu                 quantal             5ac751e8d623        46 hours ago        161 MB
   ubuntu                 lucid               9cc9ea5ea540        46 hours ago        180.8 MB
   ubuntu                 10.04               9cc9ea5ea540        46 hours ago        180.8 MB
   ubuntu                 latest              9cd978db300e        46 hours ago        204.4 MB
   ubuntu                 12.04               9cd978db300e        46 hours ago        204.4 MB
   ubuntu                 precise             9cd978db300e        46 hours ago        204.4 MB

Launch Containers

1. Launch three Galera Containers (this just takes like less than 5 seconds!):

   $ for n in {1..3}; do sudo docker run -name galera$n -i -t -d ubuntu_precise/galera:codership /bin/bash; done

2. Verify that the Containers have been created correctly:

   $ sudo docker ps
   CONTAINER ID        IMAGE                            COMMAND             CREATED              STATUS              PORTS                                    NAMES
   be2b2927c40e        ubuntu_precise/galera:codership   /bin/bash           About a minute ago   Up About a minute   3306/tcp, 4444/tcp, 4567/tcp, 4568/tcp   galera3
   ddb1e1db236d        ubuntu_precise/galera:codership   /bin/bash           About a minute ago   Up About a minute   3306/tcp, 4444/tcp, 4567/tcp, 4568/tcp   galera2
   99e33954479a        ubuntu_precise/galera:codership   /bin/bash           About a minute ago   Up About a minute   3306/tcp, 4444/tcp, 4567/tcp, 4568/tcp   galera1 

Bootstrap the Cluster

1. Before we can bootstrap the Galera Cluster, we need to add the IP of all nodes into wsrep_cluster_address inside each container’s /etc/mysql/my.cnf. You can get a rough view of Galera containers details (ID and IP address) just by using the ps command as below:

   $ ps -ef | grep docker
   root      1202  1195  0 Feb05 ?        00:00:28 /usr/bin/docker -d
   root      8825  1202  0 Feb05 pts/1    00:00:00 lxc-start -n 99e33954479a79051ceea10036ec4f52a860f6d3d1871f6549d6f9ec1df5dcb8 -f /var/lib/docker/containers/99e33954479a79051ceea10036ec4f52a860f6d3d1871f6549d6f9ec1df5dcb8/config.lxc -- /.dockerinit -g 172.17.42.1 -i 172.17.0.27/16 -mtu 1500 -- /bin/bash
   root      8876  1202  0 Feb05 pts/3    00:00:00 lxc-start -n ddb1e1db236defdce77d9feff8dd930e726d8b16cd91de056787be63fc22691a -f /var/lib/docker/containers/ddb1e1db236defdce77d9feff8dd930e726d8b16cd91de056787be63fc22691a/config.lxc -- /.dockerinit -g 172.17.42.1 -i 172.17.0.28/16 -mtu 1500 -- /bin/bash
   root      9019  1202  0 00:01 pts/6    00:00:00 lxc-start -n be2b2927c40e5fa3053a0edb6d989167a57a3f52f089c200c915ff983f018dd8 -f /var/lib/docker/containers/be2b2927c40e5fa3053a0edb6d989167a57a3f52f089c200c915ff983f018dd8/config.lxc -- /.dockerinit -g 172.17.42.1 -i 172.17.0.30/16 -mtu 1500 -- /bin/bash

   From the output we can determine that:
   99e33954479a79051ceea10036ec4f52a860f6d3d1871f6549d6f9ec1df5dcb8 → 172.17.0.27
   ddb1e1db236defdce77d9feff8dd930e726d8b16cd91de056787be63fc22691a → 172.17.0.28
   be2b2927c40e5fa3053a0edb6d989167a57a3f52f089c200c915ff983f018dd8 → 172.17.0.30

2. Now log into galera1:

   $ sudo lxc-attach -n 99e33954479a79051ceea10036ec4f52a860f6d3d1871f6549d6f9ec1df5dcb8

   And modify following line into /etc/mysql/my.cnf:

   wsrep_cluster_address=gcomm://172.17.0.27,172.17.0.28,172.17.0.30

3. Repeat the same step for galera2 and galera3. Once done, we will need to bootstrap the cluster on galera1, and have galera2 and galera3 join the cluster.

4. To bootstrap the cluster, from galera1:

   $ service mysql start --wsrep-cluster-address=gcomm://

5. Next, log into galera2 and galera3 (one node at a time) and run:

   $ service mysql restart

   You can verify that Galera is running properly by looking at wsrep_local_state_comment and wsrep_cluster_size:

   > SHOW STATUS LIKE ‘wsrep%’;

All the above steps are recorded in following asciinema recording. We were using an AWS small instance of Ubuntu Server 12.04.3 LTS 64 bit:

Happy clustering!