Demo: Cross-broker Publishing
A publish that arrives at a broker which does not own the shard is forwarded to the one that does, and acknowledged only once that broker has written it. The client does nothing special: it connects to a broker and publishes.
45 seconds, no audio. Everything happens in the terminals.
What the three panes are
Section titled “What the three panes are”| Pane | What it is |
|---|---|
| Top left | The cluster: a control plane and three broker processes, with the shard ownership it settled on |
| Bottom left | A subscriber, attached to the broker that owns the stream |
| Right | A publisher, choosing which broker to publish through |
What to watch for
Section titled “What to watch for”published "hello" to orders via broker-0 → forwarded to broker-2 → acknowledged
broker-0 received the publish, resolved that
broker-2 owns the shard, sent it there over the internal protocol, waited for
broker-2 to write it durably, and only then acknowledged. The record then
appears in the subscriber pane, attached to broker-2.
published "direct" to orders via broker-2 (the owner) — written locally, no hop
The same command aimed at the owner, so there is no hop. The forward happens only when it has to, and the client sees the same result either way.
The two bursts at the end. Six records sent one at a time arrive in order.
Twenty-four sent twelve at a time do not (par-004, par-006, par-005 and so
on). This is expected. Concurrent publishes through different brokers have no
defined relative order, and the owner assigns offsets as it commits them. Felix
keeps each publisher’s own sequence in order, but not the order across three
racing publishers.
Running it yourself
Section titled “Running it yourself”task cluster:demo:tmux # the three panes abovetask cluster:demo # the same story in one pane, no tmux neededBoth start a real cluster and tear it down afterwards. The owning broker is
chosen by hashing the shard key against the node ids, so it differs between
runs. The video happens to show broker-2.
See the cluster harness for what it does and does not exercise. In particular, the brokers are real processes while the control plane runs in the harness itself.