Redis Atomic Operations, Pub/Sub, Streams and Distributed Locks

A hackathon has 100 seats and registration opens at noon. Two hundred people click "join" in the same second. The naive code reads the remaining seat count, checks that it is above zero, then writes the new count. Two requests can both read 1 before either writes, and the venue ends up with 101 people. This is a race condition, and Redis offers several ways to remove it, each with a different strength and a different limit.

Atomic single commands and Lua scripts

Redis runs one command at a time, so a single command is atomic by itself. DECR seats:hackathon reads, subtracts and writes with nobody able to interleave. The trouble is the check-then-act pair. For a sequence of several commands, you have two main options: MULTI/EXEC queues commands and runs them together (but cannot branch on a value you just read), and a Lua script runs on the server as one unit and can branch freely.

127.0.0.1:6379> SET seats:hackathon 3
OK
127.0.0.1:6379> DECR seats:hackathon
(integer) 2
127.0.0.1:6379> EVAL "local n = tonumber(redis.call('GET', KEYS[1])); if n and n > 0 then return redis.call('DECR', KEYS[1]) else return -1 end" 1 seats:hackathon
(integer) 1
127.0.0.1:6379> EVAL "local n = tonumber(redis.call('GET', KEYS[1])); if n and n > 0 then return redis.call('DECR', KEYS[1]) else return -1 end" 1 seats:hackathon
(integer) 0
127.0.0.1:6379> EVAL "local n = tonumber(redis.call('GET', KEYS[1])); if n and n > 0 then return redis.call('DECR', KEYS[1]) else return -1 end" 1 seats:hackathon
(integer) -1

The script returns the remaining seats after a successful claim, and -1 once the seats have run out, so the counter never goes negative. Keep scripts short: while one runs, every other client waits. Pass the keys you touch through KEYS and everything else through ARGV, which also keeps scripts compatible with Redis Cluster.

Pub/sub: broadcast with no memory

SUBSCRIBE announcements in one client and PUBLISH announcements "lunch is ready" in another delivers the message to every client currently subscribed. PUBLISH replies with the number of subscribers that received it, which is (integer) 0 if nobody was listening. That is the whole limitation: pub/sub is at-most-once and has no history. A subscriber that is disconnected, slow or restarting simply misses messages, and nothing is stored. It suits live notifications where a missed message is harmless, such as pushing a "new announcement" ping to open dashboards.

Streams: a log you can read in groups

When messages must not be lost, use a stream: an append-only log stored under a key. Producers XADD; consumers in a consumer group each receive different entries and must XACK them after processing, so an entry that was delivered but never acknowledged can be inspected and retried. The following session uses explicit entry ids (1-0, 2-0) so the replies are predictable; in real use you normally pass * and Redis generates ids from the time.

127.0.0.1:6379> XADD checkins 1-0 attendee asha
"1-0"
127.0.0.1:6379> XADD checkins 2-0 attendee ben
"2-0"
127.0.0.1:6379> XLEN checkins
(integer) 2
127.0.0.1:6379> XGROUP CREATE checkins desk 0
OK
127.0.0.1:6379> XREADGROUP GROUP desk d1 COUNT 1 STREAMS checkins >
1) 1) "checkins"
   2) 1) 1) "1-0"
         2) 1) "attendee"
            2) "asha"
127.0.0.1:6379> XACK checkins desk 1-0
(integer) 1

The special id > means "entries never delivered to anyone in this group". Entry 2-0 is still waiting for the next XREADGROUP, and an entry delivered but not yet acknowledged sits in the group's pending list until a consumer claims it again. Streams give you durable queues with replay, but they are not a replacement for a dedicated log platform at large scale; for that, see Apache Kafka. Remember that streams live in memory, so trim them (XADD ... MAXLEN or XTRIM) or they will grow indefinitely.

A simple lock, and what it can and cannot promise

Sometimes you want only one worker to print the badges at a time. The building blocks are SET key token NX PX 30000 (set only if absent, expire after 30 seconds) to acquire, and a script that deletes the key only if it still holds your token, to release. The token check matters: without it, a slow worker whose lock already expired would delete the lock now held by someone else.

import uuid

import redis

r = redis.Redis(host="localhost", port=6379, decode_responses=True)

RELEASE = r.register_script("""
if redis.call('GET', KEYS[1]) == ARGV[1] then
    return redis.call('DEL', KEYS[1])
else
    return 0
end
""")


def print_badges_exclusively():
    lock_key = "lock:badge-printing"
    token = uuid.uuid4().hex

    if not r.set(lock_key, token, nx=True, px=30000):
        print("another worker is printing; skip this run")
        return

    try:
        ...  # do the work; it must finish well inside 30 seconds
    finally:
        RELEASE(keys=[lock_key], args=[token])

Be honest about the guarantees. If the worker pauses (a long garbage-collection pause, a stalled network) for longer than the expiry, the lock disappears while the worker still believes it holds it, and two workers run together. If Redis fails over to a replica that had not yet received the lock, the same can happen. For tasks where a duplicate run is merely wasteful (cache rebuilds, scheduled cleanups), a Redis lock is a fine efficiency tool. For correctness (never double-charge, never oversell), rely on a database constraint or transaction, or pass a fencing token that the protected resource checks. Distributed-lock algorithms built on several Redis instances exist and are debated; understand the trade-offs before choosing one.

Common mistakes and the symptoms they cause

• Read-then-write from the client. Symptom: oversold seats, negative stock, lost updates under load. Use INCR/DECR, a script or WATCH with MULTI.
• Expecting `MULTI`/`EXEC` to roll back. If one queued command fails at run time, the others still run. EXEC is atomic in ordering, not in all-or-nothing outcome.
• Using pub/sub for jobs. Symptom: work silently vanishes whenever a worker restarts. Use a stream with acknowledgements.
• Releasing a lock with a plain `DEL`. Symptom: workers occasionally run together because one deleted another's lock after its own expired.
• Lock without expiry. SET lock 1 NX with no PX: if the holder crashes, the lock is held forever and the job never runs again.
• Unbounded streams. Symptom: memory rises steadily with no keys obviously growing. Trim with MAXLEN.

How this shows up in real projects and interviews

"Design a flash-sale counter" and "how would you implement a distributed lock?" are interview staples. A mature answer shows the atomic claim (Lua or DECR with a guard), then volunteers the failure cases of the lock rather than waiting to be asked, and names the database as the final arbiter for correctness. In production code, scripts and locks are used sparingly and are always accompanied by metrics, because their failure modes are subtle and only visible under load.