Redis on Murat Useinov

Redis 7 functions, same old rule

Thu, 05 May 2022 20:35:00 +0300

FCALL update_if_newer 1 user:42 17. One round trip, a name instead of a SHA, and it survives a restart. Redis 7 came out last week, and Functions are the feature I care about. Everything EVAL scripts should have been.

The problem they solve is real. Take compare-and-update: set a value only if the stored version is older. From PHP it is three round trips and a race between them. WATCH/MULTI works but is ugly and retries under contention. A Lua script does it atomically in one trip, and that is why we all wrote Lua scripts. But scripts are anonymous blobs. The application carries the source, sends it on every deploy, and debugging starts with “which SHA is this”. Functions fix the operational part: the library lives in Redis, has a name, has a version.

Redis Streams is a log, not a better queue

Fri, 20 Aug 2021 22:05:00 +0300

“Should we move the queue to Streams? They are the proper way now.” A colleague, this week, about our Redis list queue. Wrong question. Streams are not a newer version of the same thing.

A list queue is a hand-off. LPUSH on one side, BRPOP on the other, and once a consumer takes the message, it is gone. Crash after the pop and the message died with you, unless you built the pending-list dance yourself. No history, no second reader. For “send this email eventually” it is honestly enough.

TTL is not an invalidation strategy

Wed, 24 Feb 2021 22:35:00 +0300

Support ticket: a user changed his name, the site still shows the old one. The profile sits in Redis with TTL one hour. Fine, we set five minutes. A week later the hit rate drops and MySQL feels it. This seesaw has no good position. TTL is insurance, not invalidation.

The obvious upgrade is cache-aside with explicit delete. Read: try the cache, miss, load from the database, write to the cache. Write: update the row, delete the key. Looks correct. It has a race.

A Redis lock is a promise you cannot fully keep

Wed, 26 Feb 2020 21:45:00 +0300

Two workers, one order, processed twice. Every project gets this day. Someone writes SETNX, calls it a distributed lock, closes the ticket. I want to slow down here, because the ticket is not closed.

The small bugs first. A lock needs a TTL, or a crashed worker holds it forever. A lock needs an owner token, or worker A releases the lock of worker B:

$token = bin2hex(random_bytes(16));
$ok = $redis->set('lock:order:'.$orderId, $token, ['nx', 'ex' => 30]);

And the release must be atomic: compare the token and delete in one Lua script. Check in PHP, delete in a second command, and there is a gap. Something will land in that gap.

What Redis does when memory ends

Tue, 02 Apr 2019 15:10:00 +0300

Users logged out at random. Not all, not always. Only sometimes, only in the afternoon. Afternoon is when traffic peaks. Traffic peaks fill the cache. The cache lived in the same Redis as the sessions, the instance hit maxmemory, and the eviction policy was allkeys-lru. Redis did exactly what we asked: threw away the least recently used keys, and some of them were sessions of people who went for lunch.

Horizon, or the queue becomes visible

Sat, 20 Jan 2018 10:30:00 +0300

redis-cli llen queues:default and hope. That was my queue monitoring for years. Horizon replaces it with a real dashboard, and I did not know how much I needed one until I saw it.

The mental shift matters more than the UI. dispatch() is the beginning of the work, not the end. The job still has to wait in Redis, run, maybe fail, maybe retry. All of that was invisible. Now it is on one screen: throughput, wait time per queue, failed jobs with the full payload and the exception.

Cache stampede: the hot key problem

Tue, 07 Feb 2017 21:30:00 +0300

Cache hit rate 98 percent. The database still fell over. Both facts are true, and the second one does not care about the first.

What happened. One key holds the result of a heavy query for the main page. TTL one minute. At some second the key expires. In that second a few hundred requests all get a miss, and all of them go to recompute the same heavy query. Redis is fine. MySQL is not. This is a cache stampede, and hit rate will not warn you, because hit rate is an average. The stampede lives in the worst second.

Redis cache and Redis queue are different databases

Fri, 21 Aug 2015 19:45:00 +0300

Two lines from redis.conf on one project:

maxmemory 2gb
maxmemory-policy allkeys-lru

Correct for a cache. Then sessions moved into the same instance, because Redis was already there. Then the job queue, same reason. One process, three tenants. This works until the day it does not.

Cache data is disposable by definition, and those two lines embrace that. Memory fills up, Redis evicts the coldest keys, the application rebuilds them on demand. Persistence is optional. After a restart a cold cache is an inconvenience, not an incident.