Redis Pub/Sub vs Streams: Ephemeral or Durable

Redis ships two commands that both look like fan out: PUBLISH and XADD. Send a message to a channel, or append a message to a stream, and every interested reader picks it up. I treated them as interchangeable for longer than I would like to admit. Then a subscriber in my ticker went offline for about a second, and I learned that only one of them keeps a message around for a reader that was not connected.

Most of what I have written while building the same stock ticker on Kafka and on Redis Streams sets one broker against the other. Not this time. Redis's own two fan-out primitives confuse people more than the Kafka comparison ever does, so I want to stay inside Redis for a post.

Same Broker, Two Unrelated Promises

PUBLISH and SUBSCRIBE are Redis's original messaging feature, older than Streams by years. A publisher calls PUBLISH channel message, and Redis pushes that message to every client currently subscribed to channel. The return value is a number: how many subscribers it reached.

XADD is the command I pulled apart in Redis Streams Fundamentals. It appends to a log that lives inside a Redis key. Every entry gets an ID of the form <millisecondsTime>-<sequence>, and the entry stays in the stream until something trims it, not until someone reads it.

Does the message stick around after it has been delivered? That question is what separates them. Replay, consumer groups, at-least-once delivery: all of it falls out of the answer.

Pub/Sub: You Get What Was Listening

Pub/Sub has no memory. PUBLISH never checks whether anyone will want the message later. It fans the message out to whoever is connected and subscribed at that instant. If that number is zero, the message is gone the moment the command returns. No queue, no error, no signal that anything was lost.

The smallest test I could think of: publish to a channel with nobody subscribed yet, and look at what comes back.

const CHANNEL = "demo:s04:ticks";

// Pub/Sub - publish with no subscriber
const receivers0 = await pub.publish(CHANNEL, "BBCA@9500");
console.log(`PUBLISH reached ${receivers0} subscribers -> message LOST`);
// receivers0 is 0. The message does not exist anywhere any more.

Same call again, this time after a subscriber has connected:

const received: string[] = [];
sub.on("message", (_ch, msg) => received.push(msg));
await sub.subscribe(CHANNEL);

const receivers1 = await pub.publish(CHANNEL, "BBCA@9512");
// receivers1 is 1, and `received` now contains "BBCA@9512"

Same channel, same kind of message, two different outcomes. The only variable is whether a subscriber happened to be connected at the exact moment PUBLISH ran. Fire-and-forget in its purest form: no acknowledgement, no delivery guarantee, no way for a late subscriber to ask what it missed.

Streams: The Late Reader Gets Everything

Then I ran the same shape of test against a stream. Write a batch of ticks first, then read them back with a client that was not present for a single one of the writes:

const STREAM = "demo:s04:stream";
const sim = new MarketSimulator({ seed: 7, volatilityScale: 4 });

for (const tick of sim.batch(12)) {
  await pub.xadd(STREAM, "*", ...tickToRedis(tick));
}

// A reader that arrives only now, after every write already happened:
const history = await pub.xrange(STREAM, "-", "+");
console.log(`a consumer that arrived AFTER the writes still sees all ${history.length} entries`);
// history.length is 12. Every tick survived.

XRANGE key - + reads the whole stream, lowest possible ID to highest. The reader connecting after all twelve XADD calls had finished changes nothing: those entries were never tied to a live connection. They were written into the log, and they stay there. Two independent consumer groups reading the same stream would each get their own full copy of it. Durable fan-out, the property Pub/Sub structurally cannot offer.

What a Disconnect Costs You

Both halves of that test are one experiment run under a single varying condition: was anyone listening yet. Turn that into a subscriber that goes offline and comes back, and the picture looks like this:

sequenceDiagram
    participant P as Producer
    participant Pub as "Pub/Sub Channel"
    participant S as "Stream Key"
    participant Sub as Subscriber
    Sub--)Pub: disconnects
    P->>Pub: PUBLISH "BBCA@9500"
    Note over Pub: 0 receivers, message discarded
    P->>S: XADD "BBCA@9500"
    Note over S: entry appended, stays until trimmed
    Sub->>Pub: reconnects, SUBSCRIBE
    Note over Sub: nothing published during the gap is recoverable
    Sub->>S: XRANGE - +
    Note over Sub: replays every entry, including "BBCA@9500"

For a channel, that gap is permanent data loss with no error raised anywhere. For a stream, the same gap is a slightly later XRANGE call, or an XREADGROUP against a consumer group. I walked through that version in Redis Streams consumer groups: entries a consumer has not acknowledged sit in the group's Pending Entries List until it acks them or another consumer reclaims them. A crashed or disconnected consumer does not lose its unacknowledged work. It leaves the work pending until someone picks it back up.

Streams durability has limits of its own, though. The log lives in RAM, bounded by MAXLEN, and Redis persistence to disk (RDB snapshots or AOF) has its own recovery window on top of that. My claim is not that Streams never lose anything. Streams lose data on your terms, through a trim policy or a persistence setting you chose, instead of losing it without warning as soon as a subscriber disconnects.

Two Fan-Out Shapes, Side by Side

graph TD
    subgraph "Pub/Sub - fire and forget"
        A["PUBLISH tick"] --> B{"Subscriber connected right now?"}
        B -->|No| C["Message discarded, 0 receivers"]
        B -->|Yes| D["Delivered once, no history kept"]
    end
    subgraph "Streams - durable log"
        E["XADD tick"] --> F["Entry appended to the log"]
        F --> G["Late reader: XRANGE replays full history"]
        F --> H["Consumer group: XREADGROUP + PEL + XACK"]
    end

When Pub/Sub Is Exactly Right

None of this makes Pub/Sub the worse tool, only a different one. Reach for it whenever the newest message fully supersedes the last one, so losing an in-between update costs you nothing: presence ("who is online right now"), live cursor positions in a collaborative UI, typing indicators, or a lightweight nudge telling already-connected clients that something changed and they should refetch.

A price ticker's UI layer broadcasting the latest tick to whoever has the page open is a fair Pub/Sub use case for that exact reason. A client that missed three intermediate prices while reconnecting has no need to replay them. It needs the current one. The price of that convenience is that Pub/Sub cannot answer "what happened while I was gone," because it was never designed to keep an answer around.

When It Loses Data You Needed

The failure mode is never a crash or an exception, which is what makes it dangerous. It is a worker that restarts during a deploy and misses every message published in the two seconds it was down. It is a client behind a flaky mobile connection that drops for a moment during exactly the window a price alert fires. Nothing in Pub/Sub tells you any of that happened. PUBLISH returned a number, the number was smaller than you assumed, and the system moved on.

If a message represents something that must be acted on exactly once, eventually, even if the consumer was briefly unavailable, an alert, an order state change, anything with business consequences, that requirement alone rules Pub/Sub out. That is a Streams job, or a Kafka job, which is where the rest of this series lives.

The Short Version

• PUBLISH fans a message out only to clients subscribed at that exact moment. No subscribers means the message is discarded, with no error and no trace.
• XADD appends to a durable log. A reader that shows up after every write still sees everything via XRANGE, and consumer groups each get their own full copy via XREADGROUP.
• My own test showed both halves directly: a publish with zero subscribers returned 0 receivers and vanished, while a late reader against the stream recovered all 12 entries I had written.
• Streams durability is bounded, not infinite: the log lives in memory until MAXLEN trims it, and disk persistence (RDB or AOF) has its own recovery characteristics on top of that.
• Use Pub/Sub for disposable, superseding signals: presence, cursors, "refetch now." Use Streams for anything that must survive a disconnect or be processed reliably.

Back to comparing the two brokers next. The piece of my ticker I am fondest of is the one that pins every BBCA record, price ticks and alerts alike, to the same partition number in two different topics, so a join between them never has to leave one machine.