Using Redis to Offload Galera Cluster from Session Management Data in Moodle

There is a huge demand for a highly available Moodle system to have a Galera Cluster as the database backend due to its growing popularity and robustness. Galera Cluster turns the popular MySQL and MariaDB server into a multi-master replication with an automated member provisioning and joining process and no data loss guarantee with a proper configuration. However, in a busy Moodle environment, a multi-writer Galera Cluster could face a degrading issue due to high contention on hotspot tables, especially sessions and caches.

In this blog post, we are going to look at how to install and configure Redis to offload our Galera Cluster from session management data in Moodle. Our architecture consists of a single Moodle web server, connected to a three-node MariaDB Cluster via an HAProxy instance. The HAProxy configuration is set for multi-writer configuration, where every Galera node will process read/write in a round-robin fashion. 

Moodle’s Session and Cache Management

A PHP session stores data on the server rather than the user’s computer. In a session-based environment, every user is identified through a unique number called session identifier or SID. This unique session ID is used to link each user with their own information on the server. Without a PHP session, a user will be logged out of the Moodle system.

Moodle needs to store the session data in storage. It supports a number of persistent storage endpoints like filesystem, database, memcached and Redis, to store the following components:

• Default file store for application caches.
• Default session store for session caches.

We chose Redis because it is simple to install (available directly from the OS package manager) and easy to configure out-of-the-box.

Installing Redis

You can use standalone Redis or a clustered Redis called Redis Sentinal. For simplicity reasons, we are going to deploy only a single standalone Redis server on the same host as the Moodle server. The Redis session driver is available only on Moodle 3.1.3 onwards, which requires a Redis server and the Redis PHP extension to be installed.

First of all, install php-redis extension:

$ apt -y install php-redis

Load the module into Apache by restarting the Apache webserver:

$ systemctl restart apache2

Install Redis tools and server:

$ apt -y install redis

Modify the following line inside /etc/redis/redis.conf, to protect the Redis server with password authentication and configure systemd as the supervision tree:

$ vim /etc/redis/redis.conf
requirepass redispass123
supervised systemd

Restart Redis to load the new changes:

$ systemctl restart redis

Test connecting to the Redis server via redis-cli:

$ redis-cli
127.0.0.1:6379> AUTH redispass123
OK
127.0.0.1:6379> 

We are now good to configure Moodle’s session and cache management to Redis. 

Configure Moodle with Redis

Log in to Moodle as administrator, then go to Site administrator -> Plugins -> Caching -> Configuration. Under “Installed cache stores” section, click on the “Add instance” for Redis and enter the following details:

The Password text field is equivalent to the requirepass value inside /etc/redis/redis.conf, which in this case is “redispass123”. Click “Save Changes”. Next, go to the “Stores used when no mapping is present” section (at the bottom of the page) and click on the “Edit mappings”, and set the “Application” and “Session” as the following:

Click on the “Save changes” button. Our Moodle system is now integrated with the Redis server for caches and session storage. To test, log in to the Moodle system and you can then check if the Redis server has stored the session and/or cache data by using the redis-cli command line:

$ redis-cli
127.0.0.1:6379> AUTH redispass123
OK
127.0.0.1:6379> KEYS *
  1) "lciaqttsndkgoa6b61th03ad7o"
  2) "mo8bbm8hoonug4b3dndsbitpto"
...
...

Or if you prefer the GUI, you may use phpRedisAdmin to get the same result:

Our Redis installation and integration with the Moodle system is now complete. 

Performance Improvement

To get a summary of the Redis performance, we used the Redis Stats, where it will produce a summary like this:

The key components that we need to focus on are the Hits and Misses values, the Used and Peak Memory values and the number of keys stored in db0 (default for Moodle). Make sure the hit rate percentage is high around the peak (the pie chart), indicating an efficient cache system, while the memory section shows how much memory to store and serve the 202 keys in db0, giving a good visualization of the total cache size and memory capacity allocation of the server.

When comparing the database queries with and without Redis, depending on the user’s activities, we could see up to 10 database hits for the same session ID on multiple tables. The following lines are taken from MySQL’s general log involving all queries related to session ID ‘fc02352nsgrohlhnv961p7blm1’:

2021-01-21T12:12:12.985149Z      6403 Query     INSERT INTO mdl_sessions (state,sid,sessdata,userid,timemodified,timecreated,lastip,firstip) VALUES('0','fc02352nsgrohlhnv961p7blm1',NULL,'26','1611231132','1611231132','13.229.218.239','13.229.218.239')
2021-01-21T12:12:13.031471Z      6404 Query     SELECT id, sid, state, userid, lastip, timecreated, timemodified FROM mdl_sessions WHERE sid = 'fc02352nsgrohlhnv961p7blm1'
2021-01-21T12:12:13.048408Z      6405 Query     SELECT id, sid, state, userid, lastip, timecreated, timemodified FROM mdl_sessions WHERE sid = 'fc02352nsgrohlhnv961p7blm1'
2021-01-21T12:12:27.407469Z      6447 Query     SELECT id, sid, state, userid, lastip, timecreated, timemodified FROM mdl_sessions WHERE sid = 'fc02352nsgrohlhnv961p7blm1'
2021-01-21T12:12:42.708191Z      6482 Query     SELECT id, sid, state, userid, lastip, timecreated, timemodified FROM mdl_sessions WHERE sid = 'fc02352nsgrohlhnv961p7blm1'
2021-01-21T12:12:55.205215Z      6525 Query     SELECT id, sid, state, userid, lastip, timecreated, timemodified FROM mdl_sessions WHERE sid = 'fc02352nsgrohlhnv961p7blm1'
2021-01-21T12:12:55.205977Z      6525 Query     SELECT * FROM mdl_sessions WHERE sid = 'fc02352nsgrohlhnv961p7blm1'
2021-01-21T12:12:55.206212Z      6525 Query     DELETE FROM mdl_external_tokens WHERE sid = 'fc02352nsgrohlhnv961p7blm1' AND tokentype = '1'
2021-01-21T12:12:55.206482Z      6525 Query     DELETE FROM mdl_sessions WHERE sid = 'fc02352nsgrohlhnv961p7blm1'
2021-01-21T12:12:55.221436Z      6525 Query     INSERT INTO mdl_logstore_standard_log (eventname,component,action,target,objecttable,objectid,crud,edulevel,contextid,contextlevel,contextinstanceid,userid,courseid,relateduserid,anonymous,other,timecreated,origin,ip,realuserid) VALUES ('\core\event\user_loggedout','core','loggedout','user','user','26','r','0','1','10','0','26','0',NULL,'0','{"sessionid":"fc02352nsgrohlhnv961p7blm1"}','1611231175','web','13.229.218.239',NULL)

With Redis is enabled, regardless of the user’s activities, it will only make 2 database queries per session ID, only on the mdl_sessions table, as shown below for session ID ’00b3fsn4k78oj9dq0oeo1j39pi’:

2021-01-21T12:38:44.060790Z      9381 Query     INSERT INTO mdl_sessions (state,sid,sessdata,userid,timemodified,timecreated,lastip,firstip) VALUES('0','00b3fsn4k78oj9dq0oeo1j39pi',NULL,'0','1611232724','1611232724','13.229.218.239','13.229.218.239')
2021-01-21T12:38:44.090198Z      9382 Query     SELECT id, sid, state, userid, lastip, timecreated, timemodified FROM mdl_sessions WHERE sid = '00b3fsn4k78oj9dq0oeo1j39pi'

From this observation, we can tell that with the help of Redis, we would see a steady 2 database queries per session, otherwise, the database server has to process more queries for cache and session handling, which is up to 5 times if Redis is out of the picture.

In conclusion, when configuring Moodle to use Redis for session and cache storage, we can save a notable amount of database hits, especially on the hotspot tables like mdl_sessions and mdl_external_tokens. This configuration will significantly improve the Galera Cluster replication performance by reducing the number of write conflicts for a multi-master configuration. In Galera Cluster, due to cluster-wide optimistic locking, the conflicted transaction will have to retry (default is wsrep_retry_autocommit=1) or rollback (after out of retry), depending on the database server’s configuration. Note that a rollback is generally a slower operation than a commit and when rollbacks keep happening, it will have an impact on the replication performance as a whole.

Note that, for a busy Moodle system, it still can cause problems with the concurrency especially the contention on the mdl_sessions table because the current implementation still leaves some parts of the session managed by the core functions. There is an improvement request here to offload all session-related data management to Redis and we hope it will get implemented in the near future.