Severalnines Blog
The automation and management blog for open source databases

How to Cluster Liferay with MySQL Galera and Ceph for High Availability and Performance

Posted in:

Liferay is an open-source content management system written in Java. It is used by a number of high traffic sites, as this survey suggests. 

Clustering Liferay and other components such as the database and the file system is a good way to handle the performance requirements of a high traffic site. The latest Liferay version has introduced features that simplify clustering, such as built-in support for Ehcache clustering, Lucene replication, read/write splitting capabilities for database (in case if you run on master-slave architecture) and support for various file systems for the portal repository. 

In this post, we are going to show you how to cluster Liferay in a multi-node load-balanced setup. The database backend will be based on Galera Cluster for MySQL, and the file system clustered using Ceph FS.


We will have a three-node database cluster, with two of the MySQL instances co-located with Liferay portal. The third MySQL instance is co-located with ceph-admin. Another two nodes, ceph-osd0 and ceph-osd1 will be used as a storage pool for Liferay repository using CephFS. ClusterControl will be hosted on ceph-osd1. We will be using Liferay Portal 6.2 Community Edition GA1 and all hosts are running on CentOS 6.4 64bit. All commands shown below are executed in root environment. SElinux and iptables are turned off.

Our hosts definition in all nodes:		liferay1 galera1		lifeary2 galera2		ceph-mds ceph-admin ceph-mon1 galera3		ceph-osd0 ceph-mon2 haproxy1		ceph-osd1 ceph-mon3 haproxy2 clustercontrol

Deploying MySQL Galera Cluster

1. Use the Galera Configurator to deploy a three-node Galera Cluster. Use galera1, galera2 and galera3 for the MySQL nodes, and ceph-osd1 ( for the ClusterControl node. Once deployed, create a schema for Liferay by using Manage >> Schema and Users >> Create a database called “liferay”. Then assign all privileges to the schema with a wildcard host (‘%’).

We will use the MySQL JDBC connector for load balancing and failover of database requests. 

Installing Ceph Storage Cluster

Ceph is a free clustered storage platform that provides unified object, block and file storage. It requires at least three nodes, one for the admin/monitor node and the rest for the storage cluster nodes (OSD). We’ll also add a monitor role to the storage nodes, for resiliency in case the admin node goes down. 

Please note that Ceph FS is relatively new and not yet production ready. If you’re considering this for production, you may instead want to look at another clustered file system. We covered the following in previous blog posts; GlusterFSOCFS2, or GFS2.

Our distributed storage layer is illustrated below:

We’ll use a nice helper script to simplify the Ceph deployment called ceph-deploy. This script requires passwordless SSH setup between the admin node and OSD nodes. Then, we will mount it using the FUSE method on clients. 

The following steps should be performed on ceph-admin node.

1. SSH into ceph-admin and generate SSH key:

$ ssh-keygen -t rsa # press Enter for all questions

2. Copy the SSH key to OSDs to allow passwordless SSH:

$ ssh-copy-id -i ~/.ssh/id_rsa ceph-osd0
$ ssh-copy-id -i ~/.ssh/id_rsa ceph-osd1

3. Install ceph-deploy dependencies, install ceph-deploy package and create a directory for Ceph deployment:

$ yum install python-argparse python-distribute -y
$ rpm -Uhv
$ mkdir ~/ceph-deploy
$ cd ~/ceph-deploy

4. Start the Ceph deployment:

$ ceph-deploy new ceph-admin ceph-osd0 ceph-osd1
$ ceph-deploy install ceph-admin ceph-osd0 ceph-osd1
$ ceph-deploy mon create-initial

5. Define the public network for Ceph communication on ~/ceph-deploy/ceph.conf:

$ cat “public_network =>> ceph.conf

Then, push the configuration file to all Ceph nodes:

$ ceph-deploy --overwrite-conf config push ceph-admin
$ ceph-deploy --overwrite-conf config push ceph-osd0
$ ceph-deploy --overwrite-conf config push ceph-osd1

6. Create the Ceph monitor daemons:

$ ceph-deploy mon create ceph-admin ceph-osd0 ceph-osd1
$ ceph-deploy gatherkeys ceph-admin

7. On each OSD node, create a directory for OSD daemons:

$ ssh ceph-osd0 "mkdir /var/local/osd0"
$ ssh ceph-osd1 "mkdir /var/local/osd1"

8. Prepare the directory for OSDs and activate it:

$ ceph-deploy osd prepare ceph-osd0:/var/local/osd0 ceph-osd1:/var/local/osd1
$ ceph-deploy osd activate ceph-osd0:/var/local/osd0 ceph-osd1:/var/local/osd1

9. Copy the configuration file and admin key to all Ceph nodes so we can manage it without specifying keyring: 

$ ceph-deploy admin ceph-osd0 ceph-osd1 ceph-admin


10. Check the ceph deployment status. Make sure you achieve active+clean state which indicates that it has finished peering:

$ ceph -s
    cluster 8d8f1f38-ddb4-4d7b-a219-c4a886101788
     health HEALTH_WARN clock skew detected on mon.ceph-osd0, mon.ceph-osd1
     monmap e3: 3 mons at {ceph-admin=,ceph-osd0=,ceph-osd1=}, election epoch 8, quorum 0,1,2 ceph-admin,ceph-osd0,ceph-osd1
     osdmap e9: 2 osds: 2 up, 2 in
      pgmap v26: 192 pgs, 3 pools, 0 bytes data, 0 objects
            14792 MB used, 9260 MB / 25339 MB avail
                 192 active+clean

11. Create the metadata server on ceph-admin:

$ ceph-deploy mds create ceph-admin


Creating CephFS

Following steps should be performed on ceph-admin.

1. Create a separate pool for ceph client to mount for CephFS:

$ rados mkpool cephfs
$ ceph osd pool set cephfs size 2

2. Create and associate keyring and assign client privilege for the CephFS:

$ ceph auth get-or-create client.cephfs mon 'allow r' osd 'allow rwx pool=cephfs' -o ~/ceph-deploy/client.cephfs.keyring
$ ceph-authtool -p -n client.cephfs ~/ceph-deploy/client.cephfs.keyring > ~/ceph-deploy/client.cephfs

3. Copy the CephFS configuration file and keyring to liferay1 and liferay2:

$ scp ceph.conf client.cephfs client.cephfs.keyring liferay1:/etc/ceph/
$ scp ceph.conf client.cephfs client.cephfs.keyring liferay2:/etc/ceph/


Mounting Ceph

Following steps should be performed on liferay1 and liferay2.

1. Install ceph-fuse package:

$ yum install ceph-fuse -y

2. Create the mount point for CephFS and mount using FUSE:

$ mkdir /storage
$ ceph-fuse --keyring /etc/ceph/client.cephfs.keyring --name client.cephfs /storage

3. Verify if CephFS is mounted correctly:

$ mount | grep ceph
ceph-fuse on /storage type fuse.ceph-fuse (rw,nosuid,nodev,allow_other,default_permissions)

4. Add the following line into /etc/rc.local to allow automount on system startup:

cat/usr/bin/ceph-fuse --keyring /etc/ceph/client.cephfs.keyring --name client.cephfs /storage” >> /etc/rc.local


Installing Liferay

Following steps should be performed on liferay1 and liferay2 unless specified.

1. Liferay requires the latest Java from Oracle JDK (not OpenJDK available in yum repository). Download it by using following command:

$ wget --no-cookies --header "Cookie:" "" --no-check-certificate -O jdk-7u51-linux-x64.rpm

2. Install Oracle JDK:

$ rpm -Uhv jdk-7u51-linux-x64.rpm

3. Check the Java version. Make sure it installs correctly:

$ java -version
java version "1.7.0_51"
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)

4. Download Liferay Portal Community Edition (bundled with Tomcat), install Liferay dependencies and extract the downloaded file into /home directory:

$ yum install -y unzip ImageMagick
$ wget
$ unzip -d /home

5. Create a symbolic link to simplify the Liferay path:

$ ln -sf /home/liferay-portal-6.2.0-ce-ga1/ /home/liferay

6. On liferay1, create a directory for Liferay clustered storage:

$ mkdir /storage/liferay


7. Create a file called to replace some default parameters in Liferay:

$ vim /home/liferay/tomcat-7.0.42/webapps/ROOT/WEB-INF/classes/

And add the following lines:

8. On liferay1, start Liferay:

$ /home/liferay/tomcat-7.0.42/bin/

You can monitor the Liferay bootstrap process at /home/liferay/tomcat-7.0.42/logs/catalina.out. You will see “INFO: Server startup in 130336 ms” indicates the Liferay Portal has started.

9. Connect to liferay1 Tomcat server at and perform the initial setup. Once done, you will be redirected to a welcome page, similar to screenshot below:

10. Once completed, repeat step 8 for the other Liferay node (liferay2). If the bootstrapping completed successfully, both Liferay portals should run in active-active clustered mode now. You should see some activities on your database cluster, as seen from the ClusterControl dashboard:

Finally, you may add load balancers in front of the Liferay web apps such as HAproxy, Apache (with mod_jk, mod_rewrite and mod_proxy) or load balancing appliances.



Related Post

Planets9s - 2016’s most popular s9s resources

Planets9s is a weekly communication on all the latest Severalnines resources and technologies around automation and management of open source databases such as MySQL, MariaDB, PostgreSQL & MongoDB.

Posted in:

Planets9s - Online schema change for MySQL & MariaDB, MySQL storage engine & backups … and more

Planets9s is a weekly communication on all the latest Severalnines resources and technologies around automation and management of open source databases such as MySQL, MariaDB, PostgreSQL & MongoDB.

Posted in:

Planets9s - Download our new DevOps Guide to Database Backups for MariaDB & MySQL

Planets9s is a weekly communication on all the latest Severalnines resources and technologies around automation and management of open source databases such as MySQL, MariaDB, PostgreSQL & MongoDB.

Posted in:

Planets9s - On stable MySQL Replication setups, running MySQL on Docker and cloud lockin

Planets9s is a weekly communication on all the latest Severalnines resources and technologies around automation and management of open source databases such as MySQL, MariaDB, PostgreSQL & MongoDB.

Posted in: