The use of in memory databases as a data structure layer for storing frequently accessed data is on the rise. Infrastructure built for high concurrent traffic typically use an in memory database layer to implement caching strategies.
Redis is a key-value data store where the data is stored in memory for speedy access without disk overhead.
Other use cases that can be done through Redis include message queue, leaderboard, and Pub (Publisher) /Sub (Subscriber).
In this blog, we will review how to set up Redis Master-Slave architecture.
How does Redis Replication work?
The Redis replication uses an asynchronous method to transfer data from master to slave. The slave periodically acknowledges the received data sent by the master node, and also the master node can have many slaves. Redis can support cascading replication, so the slave can be connected to another slave.
It is good practice to enable persistence options in the master and slave nodes, otherwise one needs to avoid restarting an instance in order not to lose all data. Every Redis has a replication ID which consists of a large pseudo random string and the offset value on the master, this replication ID is the key replication process. The offset is incremented for every byte of streaming from master to slaves. On the other side, slaves connect to the master and use PSYNC to send their replication ID and the offset that has been processed in the slave. In this way, the master node will need to send the incremental data that is needed.
On Centos, the package installation does not come along as the default repository in CentOS operating system, thus we need to install the epel-release before we go through installation. While on the Ubuntu/Debian based systems, we can add the official Redis repository into Ubuntu PPA (Personal Package Archive). In this blog, we will use Ubuntu as a base operating system for Redis installation.
First, we need to add Redis repository as below:
vagrant@n1:~$ sudo add-apt-repository ppa:redislabs/redis Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries and streams. Redis has built-in replication, Lua scripting, LRU eviction, transactions and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster. More info: https://launchpad.net/~redislabs/+archive/ubuntu/redis Press [ENTER] to continue or Ctrl-c to cancel adding it. 0% [Working] Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease Get:2 http://ppa.launchpad.net/redislabs/redis/ubuntu bionic InRelease [15.9 kB] Get:3 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] Get:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] Get:5 http://ppa.launchpad.net/redislabs/redis/ubuntu bionic/main i386 Packages [1,020 B] Get:6 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] Get:7 http://ppa.launchpad.net/redislabs/redis/ubuntu bionic/main amd64 Packages [1,024 B] Get:8 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [1,006 kB] Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [2,129 kB] Get:10 http://ppa.launchpad.net/redislabs/redis/ubuntu bionic/main Translation-en [584 B] Get:11 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,783 kB] Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,310 kB] Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [421 kB] Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [25.8 kB] Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [389 kB] Get:16 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [329 kB] Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [52.8 kB] Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,738 kB] Get:19 http://security.ubuntu.com/ubuntu bionic-security/restricted i386 Packages [19.2 kB] Get:20 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [365 kB] Get:21 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [48.9 kB] Get:22 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [983 kB] Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,568 kB] Get:24 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,131 kB] Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [371 kB] Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [26.6 kB] Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,792 B] Get:28 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [256 kB] Fetched 14.2 MB in 29s (492 kB/s) Reading package lists... Done
And then, continue with the update using apt-update command:
vagrant@n1:~$ sudo apt-get update Hit:1 http://ppa.launchpad.net/redislabs/redis/ubuntu bionic InRelease Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease Hit:5 http://security.ubuntu.com/ubuntu bionic-security InRelease Reading package lists... Done
Last thing, install the Redis package using apt package manager:
vagrant@n1:~$ sudo apt-get install redis Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: redis-server redis-tools Suggested packages: ruby-redis The following NEW packages will be installed: redis redis-server redis-tools 0 upgraded, 3 newly installed, 0 to remove and 31 not upgraded. Need to get 1,172 kB of archives. After this operation, 6,716 kB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 http://ppa.launchpad.net/redislabs/redis/ubuntu bionic/main amd64 redis-tools amd64 6:6.2.4-1rl1~bionic1 [1,057 kB] Get:2 http://ppa.launchpad.net/redislabs/redis/ubuntu bionic/main amd64 redis-server amd64 6:6.2.4-1rl1~bionic1 [79.3 kB] Get:3 http://ppa.launchpad.net/redislabs/redis/ubuntu bionic/main amd64 redis all 6:6.2.4-1rl1~bionic1 [36.2 kB] Fetched 1,172 kB in 5s (235 kB/s) Selecting previously unselected package redis-tools. (Reading database ... 39766 files and directories currently installed.) Preparing to unpack .../redis-tools_6%3a6.2.4-1rl1~bionic1_amd64.deb ... Unpacking redis-tools (6:6.2.4-1rl1~bionic1) ... Selecting previously unselected package redis-server. Preparing to unpack .../redis-server_6%3a6.2.4-1rl1~bionic1_amd64.deb ... Unpacking redis-server (6:6.2.4-1rl1~bionic1) ... Selecting previously unselected package redis. Preparing to unpack .../redis_6%3a6.2.4-1rl1~bionic1_all.deb ... Unpacking redis (6:6.2.4-1rl1~bionic1) ... Setting up redis-tools (6:6.2.4-1rl1~bionic1) ... Setting up redis-server (6:6.2.4-1rl1~bionic1) ... Setting up redis (6:6.2.4-1rl1~bionic1) ... Processing triggers for systemd (237-3ubuntu10.47) ... Processing triggers for man-db (2.8.3-2ubuntu0.1) ... Processing triggers for ureadahead (0.100.0-21) ...
We will see the status of the running Redis process through the command :
root@n1:~# systemctl status redis-server ● redis-server.service - Advanced key-value store Loaded: loaded (/lib/systemd/system/redis-server.service; disabled; vendor preset: enabled) Active: active (running) since Thu 2021-07-01 07:54:06 UTC; 6min ago Docs: http://redis.io/documentation, man:redis-server(1) Main PID: 2645 (redis-server) Status: "Ready to accept connections" Tasks: 5 (limit: 1106) CGroup: /system.slice/redis-server.service └─2645 /usr/bin/redis-server 127.0.0.1:6379 Jul 01 07:54:05 n1 systemd: Starting Advanced key-value store... Jul 01 07:54:06 n1 systemd: Started Advanced key-value store.
Repeat the installation process on the other nodes.
Setting up Redis Master and Slave
After the installation is finished, the next part is we need to configure the Redis master and slave node. Redis configuration resides on the /etc/redis/redis.conf path and there are some parameters we need to adjust to ensure the service can be accessed from another host. The parameters are:
bind 127.0.0.1 10.10.10.10 masterauth thisreplicationpassword masteruser replicationuser protected-mode no
The bind parameter listens to the Redis port on the interface address, while the master user and masterauth is used for authentication between the master and slave.
In the Redis slave nodes, we need to configure the Redis master node, master user and auth as below:
bind: 127.0.0.1 10.10.10.11 masterauth thisreplicationpassword masteruser replicationuser replicaof 10.10.10.10 6379
After that, we need to restart the Redis service and test and check the replication process using the info command as below:
root@n1:~# redis-cli -h 10.10.10.10 -p 6379 10.10.10.10:6379> info # Replication role:master connected_slaves:1 master_failover_state:no-failover master_replid:32e7ae56532d3308d3874d7b577f699244d69bf1 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0
Failover in Redis is simple, we just need to run commands on a slave node to promote it to become an independent service and accept traffic. Login to the Redis slave node:
root@n1:~# redis-cli -h 10.10.10.11 -p 6379 10.10.10.11:6379> SLAVEOF NO ONE
If we have more than one slave we can set the ip address of the slave for promotion to master.
10.10.10.11:6379> SLAVEOF 192.168.1.152 6379
So with the above, you can easily promote one of the slaves to become the new master, and have the rest of the slave replicate from the new master. We can see that manual failover is easy, but the problem with it is that it is manual – an administrator needs to log into the systems, understand what’s going on, and then run the commands. In the meantime, your application performance may degrade or in the worse case, you will experience downtime. In order to minimize downtime in your production environment, you need to think about an automatic failover mechanism. In our next blog, we’ll show you how to achieve this with Sentinel. Stay tuned…