Redis Pub/Sub vs Streams: Ephemeral or Durable

Redis provides Pub/Sub through PUBLISH and retained stream entries through XADD. I treated them as interchangeable until a ticker subscriber disconnected for about a second. Pub/Sub could not replay the messages it missed. The stream could return entries that were still retained.

The stock ticker series compares Kafka and Redis Streams. This post compares two Redis features: Pub/Sub and Streams.

Same broker, two unrelated promises

PUBLISH and SUBSCRIBE are Redis's original messaging feature, older than Streams by years. A publisher calls PUBLISH channel message, and Redis pushes that message to every client currently subscribed to channel. The return value is a number: how many subscribers it reached.

XADD is the command I pulled apart in Redis Streams Fundamentals. It appends to a log that lives inside a Redis key. Every entry gets an ID of the form <millisecondsTime>-<sequence>, and the entry stays in the stream until something trims it, not until someone reads it.

Whether the message sticks around after delivery separates them. Replay, consumer groups, at least once delivery: all of it falls out of the answer.

Pub/Sub: you get what was listening

Pub/Sub sends a message to clients currently subscribed to the channel. It retains no history for a later subscriber. If no subscriber is present, PUBLISH returns zero and the message is not stored.

The smallest test I could think of: publish to a channel with nobody subscribed yet, and look at what comes back.

const CHANNEL = "demo:s04:ticks";

// Pub/Sub - publish with no subscriber
const receivers0 = await pub.publish(CHANNEL, "BBCA@9500");
console.log(`PUBLISH reached ${receivers0} subscribers -> message LOST`);
// receivers0 is 0. The message does not exist anywhere any more.

Same call again, this time after a subscriber has connected:

const received: string[] = [];
sub.on("message", (_ch, msg) => received.push(msg));
await sub.subscribe(CHANNEL);

const receivers1 = await pub.publish(CHANNEL, "BBCA@9512");
// receivers1 is 1, and `received` now contains "BBCA@9512"

The subscription state determines the result. Pub/Sub provides no processing acknowledgment or replay for a late subscriber. Its receiver count does not prove that a client completed the work.

Streams: a late reader gets retained entries

Then I ran the same shape of test against a stream. Write a batch of ticks first, then read them back with a client that was not present for a single one of the writes:

const STREAM = "demo:s04:stream";
const sim = new MarketSimulator({ seed: 7, volatilityScale: 4 });

for (const tick of sim.batch(12)) {
  await pub.xadd(STREAM, "*", ...tickToRedis(tick));
}

// A reader that arrives only now, after every write already happened:
const history = await pub.xrange(STREAM, "-", "+");
console.log(`a consumer that arrived AFTER the writes still sees all ${history.length} entries`);
// history.length is 12. Every tick survived.

The XRANGE key - + command reads retained entries from the lowest to highest ID. My late reader retrieved all twelve entries. Consumer groups can also read independently, but their starting IDs determine which history they receive. Retention and persistence settings still limit recovery.

What a disconnect costs you

Both halves of that test are one experiment run under a single varying condition: was anyone listening yet. Turn that into a subscriber that goes offline and comes back, and the picture looks like this:

sequenceDiagram
    participant P as Producer
    participant Pub as "Pub/Sub Channel"
    participant S as "Stream Key"
    participant Sub as Subscriber
    Sub--)Pub: disconnects
    P->>Pub: PUBLISH "BBCA@9500"
    Note over Pub: 0 receivers, message discarded
    P->>S: XADD "BBCA@9500"
    Note over S: entry appended, stays until trimmed
    Sub->>Pub: reconnects, SUBSCRIBE
    Note over Sub: nothing published during the gap is recoverable
    Sub->>S: XRANGE - +
    Note over Sub: replays every entry, including "BBCA@9500"

A disconnected Pub/Sub subscriber misses messages sent during the gap. A stream reader can recover retained entries through XRANGE or XREADGROUP. Redis Streams consumer groups describes the Pending Entries List. It tracks deliveries that need acknowledgement. Another consumer can reclaim them if the original consumer stops. Recovery still requires the entry data to remain available.

Stream retention and persistence have separate limits. MAXLEN can remove old entries from memory. RDB snapshots and AOF settings determine what Redis can recover from disk. A disconnected reader alone does not remove entries, but trimming, deletion, or failures can make them unavailable.

Two fan out shapes, side by side

graph TD
    subgraph "Pub/Sub - fire and forget"
        A["PUBLISH tick"] --> B{"Subscriber connected right now?"}
        B -->|No| C["Message discarded, 0 receivers"]
        B -->|Yes| D["Delivered once, no history kept"]
    end
    subgraph "Streams - durable log"
        E["XADD tick"] --> F["Entry appended to the log"]
        F --> G["Late reader: XRANGE replays full history"]
        F --> H["Consumer group: XREADGROUP + PEL + XACK"]
    end

When Pub/Sub is exactly right

Use Pub/Sub when a later update replaces an earlier one and missed updates are acceptable. Examples include cursor positions, typing indicators, and a request for connected clients to refresh current state. The application must still obtain current state after a reconnect.

A price ticker's UI layer broadcasting the latest tick to whoever has the page open is a fair Pub/Sub use case for that exact reason. A client that missed three intermediate prices while reconnecting has no need to replay them. It needs the current one. The price of that convenience is that Pub/Sub cannot answer "what happened while I was gone," because it was never designed to keep an answer around.

When it loses data you needed

Failures arrive as missed messages rather than crashes or exceptions, which is what makes them dangerous. A worker can restart during a deploy and miss every message published in the two seconds it was down. A client behind a flaky mobile connection can drop for a moment during exactly the window a price alert fires. Nothing in Pub/Sub tells you any of that happened. PUBLISH returned a number, the number was smaller than you assumed, and the system moved on.

If work must survive a consumer disconnect, use retained storage and a recovery process. Streams or Kafka can support this. Neither makes an external business effect execute exactly once without application coordination.

Which messages survive

• PUBLISH fans a message out only to clients subscribed at that exact moment. No subscribers means the message is discarded, with no error and no trace.
• XADD appends entries to a stream. A late reader can retrieve retained entries with XRANGE. Each consumer group tracks its own position and pending deliveries through XREADGROUP.
• My publish call returned 0 receivers when no subscribers were connected. A late stream reader recovered all 12 entries from the test.
• MAXLEN limits stream retention. Disk recovery also depends on RDB or AOF configuration.
• Use Pub/Sub for disposable, superseding signals: presence, cursors, "refetch now." Use Streams for anything that must survive a disconnect or be processed reliably.

My ticker puts BBCA price ticks and alerts in matching partition numbers across two Kafka topics. With matching consumer assignments and available state, it can join these records within one process.