Redis for DevOps Engineers / SRE Part-1

Zayn Korai
Level Up Coding
Published in
5 min readJan 7, 2021

--

I always have wondered how much I need to know about Redis as DevOps Engineer. Why developers use it and why it is hot in the tech community?

Redis is an open-source key-value store and it is mostly used for Cache but Redis is much more than a cache. It has an in-memory data structure store, used as a database, cache, and message broker.

What you must need to know about Redis

Upgrade Redis version
Newer versions of Redis can derive the benefits of performance optimization in many ways, not just in memory management. Redis version 4.0 or later, and version 5.0 has a better strategy. If you are using the Redis version lower than 4.0 consider upgrading it to higher.

Disable Swap Completely
If an operating system has SWAP configured — it can dump some Redis’ data to the disk and later when Redis will try to access them — it can take a long time to read them back to the memory.

sysctl -w vm.swappiness=0

More Cores != Better++
Redis is a single-threaded process and will, at most, consume two cores if you have persistence enabled. you will not need more than two cores for a Redis instance.

It’s Expensive at Scale
Redis isn’t silver bullet and is really expensive at scale as it store data in memory, where you don’t need the kind of in-memory level performance avoid it and move to more cost effective and easy to operate.

Commands you need to know about Redis.

To Know Average latency
To quickly check the average latency of the Redis instance, you can use:

redis-cli — latency

Redis-Benchmark to study latency
To make sure your redis-server isn’t behaving abnormally, quickly run a benchmark to study latency, you can launch:

redis-benchmark -q -n 10000 -c 1 -d average_size_of_your_objects_in_bytes

This isn’t a true “load test”.
Other than Redis-benchmark. There are few more tools available to benchmark testing for Redis.
- Memtier-benchmark
- YCSB
- PerfKit Benchmark

To Find Out What’s Slowing Down Redis
As Redis doesn’t have verbose logs. It’s often hard to track down what exactly is going on inside your instance. Redis provides you commandstat utility to show you this:

127.0.0.1:6379> INFO commandstats
cmdstat_get:calls=12,usec=608,usec_per_call=2.4
cmdstat_setex:calls=15,usec=71,usec_per_call=4.20
cmdstat_keys:calls=21,usec=42,usec_per_call=1.00

Commandstats gives you a breakdown of all the commands, the number of microseconds it took to execute (total and avg per call), and how many times they’ve been run.

What is Happening to the Database
MONITOR is a debugging command that streams back every command processed by the Redis server. It can help in understanding what is happening to the database.

redis-cli monitor

But there is cost running MONITOR As it streams back all commands, its use comes at a cost.

Reset the Redis Statistics
To reset this simply run CONFIG RESETSTAT, and you’ve got a brand new slate.

CONFIG RESETSTAT

To Know memory-related Issues
The MEMORY DOCTOR command reports about different memory-related issues that the Redis server experiences, and advises about possible remedies.

MEMORY DOCTOR

Memory usage of the Server.
The MEMORY STATS command returns an Array reply about the memory usage of the server. List of memory stats it reports. Read more about it here.

MEMORY STATS

To know the role of a Redis Instance
ROLE command returns the information of the instance is currently a master, slave, or sentinel. The command also returns additional information about the state of the replication
Outputs as one of the following three strings:
“master”
“slave”
“sentinel”

Clustering

Don’t Flood One Instance
With the increase of workload on your production, split up the workload on multiple Redis instances. Redis Cluster is now available since version 3.0.0. Redis Cluster allows you to break apart keys amongst sets of given masters/slaves based on key ranges.
If clustering is not an option, consider namespacing and distributing your keys among multiple instances.
An amazing write-up on partitioning your data can be found on the redis.io website here.

High Availability and Replication

Redis has two primary categories of running multiple instances. There is standard replication and Redis Cluster. To manage Replication you can use Redis Sentinel. Cluster mostly manages itself but you can combine the two for extended experience, if complex, HA+Replication.

Redis Sentinel is a very well battle-tested high availability solution and many users been running it in production. Always consider Redis Sentinel as HA (high availability) solution.

How to setup replication or clusters in Redis to spreading out load and surpassing single-instance memory capabilities.

Coming in Part-2

Why developers use Redis?

  • It is blazingly fast!
  • They use it for caching which helps to save database calls and decrease data access latency, increase throughput, and ease the load off on relational or
    NoSQL databases. which also helps to save some cost as well,
  • Redis helps serve frequently requested items at sub-millisecond response times. It enables the developer to easily scale for higher loads without growing the costlier back-end.
  • Different developers use it for different use cases like Database query results caching, persistent session caching, web page caching, and caching of frequently used objects such as images, files, and metadata are all popular examples of caching with Redis.
  • Some developers even use it for real-time analytics. Redis with streaming solutions such as Apache Kafka as an in-memory data store to ingest, process, and analyze real-time data with sub-millisecond latency.
  • We all know data structures in programs are really important and Developers use data structures of their respective programming languages extensively but in some exceptions, they use Redis as a data structure server. Yes Redis has data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyper log logs, geospatial indexes with radius queries and streams

Further Readings
https://redislabs.com/wp-content/uploads/2016/03/15-Reasons-Caching-is-best-with-Redis-RedisLabs-1.pdf
https://gist.github.com/JonCole/925630df72be1351b21440625ff2671f
https://rtfm.co.ua/en/draft-eng-redis-main-configuration-parameters-and-performance-tuning-overview/

If this post was helpful, please click the clap 👏 button below a few times to show your support for the author! ⬇

--

--