Setting Up a High Availability Redis Cluster

1. Overview

In this article, we are going to set up a Redis cluster in a docker-based environment. We will start up by setting up the 3 redis instances with data replication enabled and after that, we will set up 3 sentinel services to the cluster. We will be also going to set up a client application to interact with the Redis cluster.

Check out my course on Hyperledger Fabric Deployment on MultiHost

2. Replication

At the base of Redis replication (excluding the high availability features provided as an additional layer by Redis Cluster or Redis Sentinel), there is a very simple to use and configure leader follower (master-slave) replication: it allows replica Redis instances to be exact copies of master instances. The replica will automatically reconnect to the master every time the link breaks and will attempt to be an exact copy of it regardless of what happens to the master.

3. Authentication

It is very important to run redis with a strong password because redis by default is not configured to use the password, so anyone can connect to it as a part of replication. We have to configure a strong password for each of the replicas and also configure each of the replicas has master node password also.

And this configured using the configuration file redis.conf

#redis-1 configuration
protected-mode no
port 6379
slaveof redis-0 6379

#authentication
masterauth a-very-complex-password-here
requirepass a-very-complex-password-here

slaveof property defines that this instance is the slave of redis-0 instance. And this slaveof property makes the redis-0 master and replication work. And requirepass is the password to authenticate this node, anyone who wants to connect to this instance must have this password. we have to also specify the master node password using the masterauth property.

4. Cluster Setup

In order to set up the redis cluster, we need to have a network over which all the redis instance, sentinel services, and the client application will communicate.

To create a redis network run:

docker network create redis

Once the network is created, we can setup the redis instances:

docker run -d --rm --name redis-0 \
 --net redis \
 -v ${PWD}/redis-0:/etc/redis/ \
 redis:6.0-alpine redis-server /etc/redis/redis.conf
docker run -d --rm --name redis-1 \
--net redis \
-v ${PWD}/redis-1:/etc/redis/ \
redis:6.0-alpine redis-server /etc/redis/redis.conf
docker run -d --rm --name redis-2 \
--net redis \
-v ${PWD}/redis-2:/etc/redis/ \
redis:6.0-alpine redis-server /etc/redis/redis.conf

5. High Availability

Redis Sentinel provides high availability for Redis. In practical terms, this means that using Sentinel we can create a Redis deployment that resists without human intervention certain kinds of failures.

Redis Sentinel provides features like monitoring, notifications, Automatic failover, and Configuration provider.

To set up the sentinel, we need at least 3 Redis nodes in order to get the majority of votes for a quorum.

6. Configure Sentinel

In order to set up the sentinel process, we need to have some configuration files.

port 5000
sentinel monitor mymaster redis-0 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000
sentinel parallel-syncs mymaster 1
sentinel auth-pass mymaster a-very-complex-password-here

port 5000 defines the port on which the sentinel process will run. mymaster is the master-group-name. redis-0 is the DNS name for redis-0 and 6379 is the port on which redis-0 is running and the number 2 is a quorum, so this setup requires minimum 2 votes to come to any descision.

After that, we are defining the down-after-milliseconds as 5000 milliseconds, which means that if the master node doesn’t respond within 5000 milliseconds then, it will be considered as dead or how much would sentinel wait for the master to initialize failover.

parallel-syncs sets the number of replicas that can be reconfigured to use the new master after a failover at the same time.

And at the end, we are defining the auth password using auth-pass.

7. Starting Sentinel Service

To start the sentinel service run:

docker run -d --rm --name sentinel-0 --net redis \
-v ${PWD}/sentinel-0:/etc/redis/ \
redis:6.0-alpine \
redis-sentinel /etc/redis/sentinel.conf
docker run -d --rm --name sentinel-1 --net redis \
-v ${PWD}/sentinel-1:/etc/redis/ \
redis:6.0-alpine \
redis-sentinel /etc/redis/sentinel.conf
docker run -d --rm --name sentinel-2 --net redis \
-v ${PWD}/sentinel-2:/etc/redis/ \
redis:6.0-alpine \
redis-sentinel /etc/redis/sentinel.conf

and verify the logs and in the logs, we should see the other sentinel nodes, if you are not seeing that, it means there is some configuration issue.

docker logs sentinel-0

8. Client Application

I have written a client application on nodejs, which is storing the current timestamp onto the Redis cluster. The client application is communicating with the sentinel service not with the Redis instance directly and this ensures that if our master Redis instance dies, our application will be still functioning properly as our read/write request will be forwarded to the current master Redis instance that is elected from the quorum.

const redis = new Redis({
db: 0,
password: "a-very-complex-password-here",
sentinels: [
{ host: host1, port: port },
{ host: host2, port: port },
{ host: host3, port: port },
],
name: "mymaster",
});

To build the client application run :

docker build ./applications/client/ -t aditya/redis-client:v1.0.0

Run the client application:

docker run -it --net redis \
--env-file .env \
-p 80:80 \
aditya/redis-client:v1.0.0

Now open the browser and visit http://localhost:80 and every time you refresh the page, we add storing the value to redis.

To test the high availablity, we can kill the master redis instance by

docker rm -f redis-0

and now if you try to access the client application we can still see that the application is working.

And to check which instance is the current master node we can run these commands:

docker logs sentinel-0
docker exec -it sentinel-0 sh
redis-cli -p 5000
info
sentinel master mymaster

Summary

In this article, we learned to set up 3 node Redis cluster, we started by setting up authentication and application, then we set the sentinel services, and at the end, we set up the client applications and test the high availability. You can find the source code here.

If you find this article helpful do hit the clap button and follow me for more such informative articles.

You can find me on Linkedin or stalk me on GitHub? If that’s too social for you, just drop a mail to adityaprakashjoshi1@gmail.com if you wish to talk tech with me.