Streams

Verified on Redis 8.10.2 (redis:8.10.2-alpine), ioredis 6.0.0, Node 24, Apple-silicon Mac. The # cli/... transcripts are one session in document order, replayed by the course harness; IDs produced by * are timestamps, so yours will differ. Streams reference: Redis Streams docs.

Streams — an append-only log

A Redis Stream is a data structure that behaves like an ordered, append-only log. Unlike Pub/Sub, where a message is gone the moment it is delivered, a stream keeps its entries and lets any consumer read them again from any position, so a late or restarted consumer can catch up.

Each entry has two parts:

• ID — <milliseconds>-<sequence>, unique and strictly increasing within the stream. Pass * to let Redis generate it from the server clock, or supply your own.
• Fields — one or more field-value pairs, like a small hash.

Where do the entries live? In memory, like every other Redis type: the docs describe a radix tree of macro nodes holding many entries each. They reach disk only through the persistence you configured (RDB snapshots, the AOF), exactly like a list or a hash. “Stream” means an in-memory log, not a file on disk, and the size section below shows what that costs.

XADD — appending entries

XADD key id field value [field value ...] appends an entry and returns its ID. With * the ID is <current ms>-0 (the sequence grows when several entries share a millisecond). XLEN counts entries, TYPE says stream:

cli/pubsub-streams-streams-1.redis

127.0.0.1:6379> XADD events * action "login" user "ada"
"1790750093107-0"
127.0.0.1:6379> XADD events * action "purchase" user "ada" item "book"
"1790750093167-0"
127.0.0.1:6379> XLEN events
(integer) 2
127.0.0.1:6379> TYPE events
stream
127.0.0.1:6379> XRANGE events - +
1) 1) "1790750093107-0"
   2) 1) "action"
      2) "login"
      3) "user"
      4) "ada"
2) 1) "1790750093167-0"
   2) 1) "action"
      2) "purchase"
      3) "user"
      4) "ada"
      5) "item"
      6) "book"
127.0.0.1:6379> XRANGE events - + COUNT 1
1) 1) "1790750093107-0"
   2) 1) "action"
      2) "login"
      3) "user"
      4) "ada"
127.0.0.1:6379> XREVRANGE events + - COUNT 1
1) 1) "1790750093167-0"
   2) 1) "action"
      2) "purchase"
      3) "user"
      4) "ada"
      5) "item"
      6) "book"
127.0.0.1:6379> XREAD COUNT 10 STREAMS events 0
1) 1) "events"
   2) 1) 1) "1790750093107-0"
         2) 1) "action"
            2) "login"
            3) "user"
            4) "ada"
      2) 1) "1790750093167-0"
         2) 1) "action"
            2) "purchase"
            3) "user"
            4) "ada"
            5) "item"
            6) "book"

XRANGE key start end [COUNT n] reads oldest to newest with - (smallest) and + (largest) as open ends; XREVRANGE key end start goes newest first. XREAD [COUNT n] STREAMS key id returns entries after the ID you pass (0 = from the start). Each reply is an entry ID plus a flat [field, value, ...] list.

XADD events * action "login" user "ada"
XADD events * action "purchase" user "ada" item "book"
XLEN events
XRANGE events - +
XREAD COUNT 10 STREAMS events 0

Append and read stream entries ⧉

ID rules, trimming and deleting

IDs must strictly increase, and you can pin part of them (1000-* lets Redis pick the sequence). Deleting and trimming are different operations:

• XDEL key id removes one entry but leaves a tombstone, so XLEN drops and the stream remembers the highest deleted ID (max-deleted-entry-id in XINFO STREAM).
• XTRIM key MAXLEN n (keep the newest n) or MINID id (drop everything older) removes from the head. XADD key MAXLEN = n ... trims while appending. NOMKSTREAM prevents XADD from creating the stream.

cli/pubsub-streams-streams-2.redis

127.0.0.1:6379> XADD fixed 1000-1 a 1
"1000-1"
127.0.0.1:6379> XADD fixed 1000-1 a 2
(error) ERR The ID specified in XADD is equal or smaller than the target stream top item
127.0.0.1:6379> XADD fixed 999-0 a 3
(error) ERR The ID specified in XADD is equal or smaller than the target stream top item
127.0.0.1:6379> XADD fixed 1000-* a 4
"1000-2"
127.0.0.1:6379> XADD log 1-0 n 1
"1-0"
127.0.0.1:6379> XADD log 2-0 n 2
"2-0"
127.0.0.1:6379> XADD log 3-0 n 3
"3-0"
127.0.0.1:6379> XADD log 4-0 n 4
"4-0"
127.0.0.1:6379> XADD log 5-0 n 5
"5-0"
127.0.0.1:6379> XADD log 6-0 n 6
"6-0"
127.0.0.1:6379> XDEL log 3-0
(integer) 1
127.0.0.1:6379> XLEN log
(integer) 5
127.0.0.1:6379> XRANGE log 2-0 5-0
1) 1) "2-0"
   2) 1) "n"
      2) "2"
2) 1) "4-0"
   2) 1) "n"
      2) "4"
3) 1) "5-0"
   2) 1) "n"
      2) "5"
127.0.0.1:6379> XTRIM log MAXLEN 4
(integer) 1
127.0.0.1:6379> XRANGE log - +
1) 1) "2-0"
   2) 1) "n"
      2) "2"
2) 1) "4-0"
   2) 1) "n"
      2) "4"
3) 1) "5-0"
   2) 1) "n"
      2) "5"
4) 1) "6-0"
   2) 1) "n"
      2) "6"
127.0.0.1:6379> XTRIM log MINID 5-0
(integer) 2
127.0.0.1:6379> XADD log MAXLEN = 2 7-0 n 7
"7-0"
127.0.0.1:6379> XRANGE log - +
1) 1) "6-0"
   2) 1) "n"
      2) "6"
2) 1) "7-0"
   2) 1) "n"
      2) "7"
127.0.0.1:6379> XADD nostream NOMKSTREAM * a 1
(nil)
127.0.0.1:6379> EXISTS nostream
(integer) 0

Every line above is real output. MAXLEN 4 removed 1 entry (the stream had 5 after the XDEL), and MINID 5-0 removed the two older ones.

Exact vs approximate trimming. MAXLEN ~ n is allowed to stop early, because Redis frees whole internal nodes (stream-node-max-entries, 100 by default). On a 1,000-entry stream, XTRIM s MAXLEN ~ 250 removed 700 and left 300; the exact XTRIM s MAXLEN 250 then removed 50 more to reach 250. The approximate form is cheaper and is what you want on a hot path: XADD key MAXLEN ~ 10000 * ....

XREAD BLOCK from Node.js

XREAD BLOCK ms STREAMS key $ waits for entries newer than “now” ($), and BLOCK 0 waits forever. A blocked command holds its connection, so give the reader its own. The reply is null when the timeout passes with nothing new:

src/pubsub-streams-streams-basic.ts

// @run
import { Redis } from 'ioredis';

const url = process.env.REDIS_URL!;
const redis = new Redis(url);
const reader = new Redis(url);   // XREAD BLOCK holds its connection, so it gets its own
const key = 'b2:events';
await redis.del(key);

// XADD returns the generated ID: "<ms timestamp>-<sequence>"
const id1 = await redis.xadd(key, '*', 'action', 'login', 'user', 'ada');
console.log('first id:', id1, ' has ms-seq shape:', /^\d{13}-\d+$/.test(id1!));

// a reader that blocks for up to 2 s waiting for entries newer than "$"
const waiting = reader.xread('BLOCK', 2000, 'STREAMS', key, '$');
await new Promise((r) => setTimeout(r, 100));
await redis.xadd(key, '*', 'action', 'purchase', 'user', 'ada', 'item', 'book');
const res = await waiting;   // [ [ streamKey, [ [ id, [field, value, ...] ], ... ] ] ]  or null on timeout
const [, entries] = res![0]!;
const [id2, fields] = entries[0]!;
console.log('blocked reader got', id2, fields);                       // [ 'action', 'purchase', 'user', 'ada', 'item', 'book' ]

// fields are a FLAT [field, value, ...] array: turn it into an object
const obj: Record<string, string> = {};
for (let i = 0; i < fields.length; i += 2) obj[fields[i]!] = fields[i + 1]!;
console.log(obj);

// nothing new within the timeout -> null, not an error
console.log(await reader.xread('BLOCK', 200, 'STREAMS', key, '$'));   // null

// bound the stream while appending; ~ trims in whole internal nodes, so the length lands near the target, not on it
for (let i = 0; i < 1000; i++) await redis.xadd(key, 'MAXLEN', '~', 250, '*', 'n', i);
console.log('length after 1000 appends with MAXLEN ~ 250:', await redis.xlen(key));   // 250..349

// XDEL leaves a tombstone: XLEN drops, XINFO remembers the highest deleted id
const [firstId] = (await redis.xrange(key, '-', '+', 'COUNT', 1))[0]!;
await redis.xdel(key, firstId);
const info = await redis.xinfo('STREAM', key) as (string | number)[];
const at = (name: string) => info[info.indexOf(name) + 1];
console.log('XINFO length:', at('length'), ' max-deleted-entry-id is the id we deleted:', at('max-deleted-entry-id') === firstId);

await redis.del(key);
reader.disconnect();
await redis.quit();

Real output (this Mac; IDs differ per run):

first id: 1790750068817-0  has ms-seq shape: true
blocked reader got 1790750068920-0 [ 'action', 'purchase', 'user', 'ada', 'item', 'book' ]
{ action: 'purchase', user: 'ada', item: 'book' }
null
length after 1000 appends with MAXLEN ~ 250: 302
XINFO length: 301  max-deleted-entry-id is the id we deleted: true

XDELEX and XACKDEL (8.2+)

With consumer groups (next lesson) an entry can be deleted from the stream while a group still lists its ID as pending, leaving a dangling reference. Redis 8.2 added XDELEX key [KEEPREF|DELREF|ACKED] IDS n id... and XACKDEL key group IDS n id... to delete safely. ACKED refuses to delete an entry that some group has not acknowledged. Reply per ID: 1 deleted, 2 not deleted because a reference remains, -1 no such entry (XDELEX). They need Redis 8.2+ (not present on Redis 7); I did not test Valkey.

cli/pubsub-streams-streams-3.redis

127.0.0.1:6379> XADD jobs 1-0 task a
"1-0"
127.0.0.1:6379> XADD jobs 2-0 task b
"2-0"
127.0.0.1:6379> XADD jobs 3-0 task c
"3-0"
127.0.0.1:6379> XGROUP CREATE jobs workers 0
OK
127.0.0.1:6379> XREADGROUP GROUP workers w1 COUNT 3 STREAMS jobs >
1) 1) "jobs"
   2) 1) 1) "1-0"
         2) 1) "task"
            2) "a"
      2) 1) "2-0"
         2) 1) "task"
            2) "b"
      3) 1) "3-0"
         2) 1) "task"
            2) "c"
127.0.0.1:6379> XDELEX jobs ACKED IDS 1 1-0
1) (integer) 2
127.0.0.1:6379> XACK jobs workers 1-0
(integer) 1
127.0.0.1:6379> XDELEX jobs ACKED IDS 1 1-0
1) (integer) 1
127.0.0.1:6379> XACKDEL jobs workers IDS 1 2-0
1) (integer) 1
127.0.0.1:6379> XRANGE jobs - +
1) 1) "3-0"
   2) 1) "task"
      2) "c"
127.0.0.1:6379> XPENDING jobs workers
1) (integer) 1
2) "3-0"
3) "3-0"
4) 1) 1) "w1"
      2) "1"

XDELEX ACKED on the unacknowledged 1-0 answered 2 and kept it; after XACK it answered 1. XACKDEL acknowledges and deletes in one atomic step. Entry 3-0 was delivered to w1 but never acknowledged, so it is still in the stream and in the pending list.

Reading AI-generated code

An assistant adds click tracking to an API. Every request appends to a stream that the analytics job drains once a day:

🐛 Spot the bug

app.use(async (req, _res, next) => {
await redis.xadd('clicks', '*', 'path', req.path, 'user', req.user?.id ?? 'anon');
next();
});

Show the answer

Nothing ever trims clicks, and a Stream lives in RAM. I measured a stream of small entries (three fields each) with MEMORY USAGE:

Entries	Memory
1,000	23,597 bytes
10,000	240,726 bytes
100,000	2,510,255 bytes
100,000 appended with MAXLEN ~ 1000	25,753 bytes (XLEN 1000)

About 25 bytes per entry of this shape, linear forever. At real traffic that is a slow memory leak that ends in eviction or OOM command not allowed. Cap it on write with an approximate MAXLEN sized to your consumer’s worst downtime (or MINID for “keep 24 hours”), and make the daily job read with XREAD/XRANGE from the last ID it processed:

await redis.xadd('clicks', 'MAXLEN', '~', 100000, '*', 'path', req.path, 'user', req.user?.id ?? 'anon');

The docs list the same idea under “Capped Streams”. The runnable pubsub-streams-streams-basic.ts above shows the capped append (MAXLEN ~ 250 gave 302 entries after 1,000 appends).

Mental Model

A stream is a log file that lives in RAM: append at the tail, read from any position by ID, drop from the head. It solves what Pub/Sub cannot (catching up, replay), and inherits what an in-memory log costs (you decide how much history to keep). The ID is both the ordering and the bookmark a consumer stores.

Trade-offs

Choice	Benefit	Cost
Streams keep entries	Re-readable, late consumers and restarts can catch up	Memory: about 25 bytes per small entry, unbounded without a cap
XADD with *	Time-ordered ID with no bookkeeping	Tied to the server clock; supply your own ID for business meaning
MAXLEN ~ n	Cheap trimming on the write path	May keep up to a node (100 entries) more than n; use exact MAXLEN when it matters
XREAD BLOCK	Push-like delivery without polling	Holds a connection; handle timeouts and reconnects; no ack (use consumer groups)
XDEL	Removes one entry	Tombstone stays; groups may keep a dangling reference; prefer XDELEX/XACKDEL

Common Misconceptions

• “A stream keeps everything on disk.” It is in memory; disk only via RDB/AOF, and history is whatever you did not trim.
• “XLEN shows how many entries were ever added.” It shows entries currently present; XINFO STREAM has entries-added.
• “MAXLEN ~ 250 leaves exactly 250.” It left 300 (1,000-entry example); use exact when you need exact.
• “Streams give exactly-once delivery.” Consumer groups give at-least-once; process idempotently.

Common mistakes

• No MAXLEN/MINID on a hot stream — memory grows linearly (see the exercise).
• XRANGE - + without COUNT on a big stream — page with COUNT and the last ID you saw.
• Sharing a blocking XREAD connection with other commands — the connection is busy until the timeout.
• XDEL on entries a group still tracks — use XDELEX ACKED (8.2+).

Real-world example

Redis’s own documentation positions Streams as the durable, replayable alternative to Pub/Sub and documents “Capped Streams” (MAXLEN/MINID) as the way to bound them (Redis Streams).

What does passing * as the ID to XADD do?

Appends to all streams Lets Redis generate a <ms>-<seq> ID from the server clock Overwrites the last entry Creates a wildcard entry

Which XRANGE argument means 'the earliest entry in the stream'?

0 $ - ^

You run XADD s 1000-1 a 1 and then XADD s 999-0 a 2. What happens?

Both succeed and are ordered by ID The second fails: the ID must be greater than the stream's top ID The second overwrites the first Redis renumbers the stream

Where does a Redis Stream keep its entries?

In a file on disk that Redis reads on demand In memory, and on disk only through RDB/AOF persistence In a separate Streams database Nowhere, entries are discarded after being read

You append 100,000 entries to one stream with XADD s MAXLEN ~ 1000 * .... Roughly how many entries does it hold?

Exactly 1000 100000 About 1000 (may exceed it by up to one internal node) 0

XDELEX s ACKED IDS 1 1-0 returns 2. What does that mean?

The entry was deleted twice It was not deleted: a consumer group still has it un-acknowledged Two entries were deleted The stream does not exist

Submit

Mark as complete