Skip to content

Felix 0.6.0-preview.7

Released 2026-10-11 as a preview. The full history is in CHANGELOG.md.

The seventh preview of 0.6.0. Subscriber memory on the broker is now bounded by bytes end to end: each subscriber queue, and the writer lanes behind it, stop at FELIX_SUB_QUEUE_MAX_BYTES (256 MiB by default) as well as at their entry count. The bound is sized so a subscriber keeping up at line rate with 4 MiB batches never reaches it. Under the block policy, one full subscriber no longer holds up a batch’s fanout to the others. Revoking a principal now ends the streams its tokens already opened, with the new revoked error code, and a subscription whose stream is deleted ends with stream_deleted for clients that offer it. ClusterClient::with_identity lets a gateway act as another user over shared cluster connections, and a delegating gateway can hold a connection-only broker token instead of a grant it never uses. Replication and placement fixes: group and dead-letter logs ride each follower’s exchange, a new Quorum copy is seated only once the leader names it caught up, and a counters-only cache restore is no longer dropped as stalled.

Upgrade notes.

  • A Raft control plane needs every member upgraded before revocation (metadata level 7) and counter-offset reports (level 8) take effect. Until then the revoke endpoint answers 409 after revoking the refresh tokens, and reports are recorded without counter offsets.
  • A Postgres control plane runs migrations 0025 and 0026 on start.
  • Subscriber queues on the broker are bounded by bytes as well as entries. An entry is a whole published batch, so the count alone let one slow subscriber hold up to 4096 batches of up to 16 MiB each. Each queue now also stops at FELIX_SUB_QUEUE_MAX_BYTES of payload (256 MiB by default, 0 for none), and the queue policy handles a full byte bound as it handles a full count: drop_new drops the batch for that subscriber, with the same offset gap and drop counters, and block waits. A batch larger than the bound still goes into an empty queue. FELIX_SUB_QUEUE_MAX_BYTES_TOTAL (off by default) caps the bytes held across all subscriber queues, at the cost of isolation between subscribers. New metrics: felix_sub_queue_bytes and felix_sub_queue_byte_dropped_total{bound}. Broker gains with_subscriber_queue_max_bytes and with_subscriber_queue_max_bytes_total (#1157, #1189).
  • A delegating gateway no longer needs a broker grant it never uses. A principal holding token.delegate:tenant:{tenant_id} can exchange for a connection-only broker token, its own subject with no permissions, by sending "requested": [], "permissions": [] to /token/exchange (or the dev-token bootstrap endpoint). Its shared connections authenticate with it, and every request still runs under the user’s delegated token. Refresh checks token.delegate again and ends the chain once it is removed. Anyone else asking for no permissions still gets 403 no permissions (#1130).
  • ClusterClient::with_identity(tenant_id, token_provider) and ClusterClient::with_identity_token act as another user over the cluster client’s connections (#1129), as Client::with_identity does for one broker. The result is a ClusterClient with its own routing state that follows moved shards, redirects and failovers, reaching each broker over the parent’s connection to it, so a gateway sharing connections keeps a user’s subscription across a shard move and still holds one connection per broker. The Python Client.with_identity and AsyncClient.with_identity and the TypeScript Client.withIdentity wrap it with a fixed token. No wire change.
  • Revoking a principal ends what its tokens already opened, on every broker, within about one control-plane sync interval (#1132). Until now a revoked principal kept its open subscriptions, cache watches, publish streams and Kafka connections until its token expired. POST /v1/tenants/{t}/refresh-tokens/revoke now also records a revocation, and so does removing an RBAC policy or role assignment, for everyone the rule reached (the whole tenant when it reached an IdP group). Brokers poll the new GET /v1/principal-revocations/changes feed, refuse broker tokens issued at or before a revocation, and end the streams they opened with the new revoked error code (16, fatal): an error on a control stream, or as the last frame on a subscription’s or cache watch’s event stream. A Kafka connection is closed at its next request. Under the Raft store this is metadata version 7, and until every member has it the revoke endpoint answers 409 after revoking the refresh tokens. See docs/auth.md.
  • CacheWatch::refusal in the Rust client says why the broker ended a watch with an error, such as revoked. A subscription ended that way returns the BrokerError from next_event instead of an “unexpected message” error (#1132).
  • The subscriber byte bounds also cover the broker’s writer lanes. A batch’s bytes stay held until the frame carrying it is written to the subscriber’s QUIC stream, so frames queued behind a flow-controlled subscription count toward FELIX_SUB_QUEUE_MAX_BYTES instead of piling up in the lanes by entry count alone (up to 64 frames of up to 16 MiB each per subscription). Under drop_new the next batch is dropped at the subscriber queue as before; publishers still never wait. SubscriptionReceiver gains recv_held and try_recv_held, and HeldBytes is public (#1163).
  • A subscription ended because its stream was deleted now says so. The broker sends stream_deleted as its last frame to a client that offered the new extended feature bit FEATURE_STREAM_DELETED; any other client sees the stream end as before. Removing a stream used to leave its subscribers waiting on a feed nothing would end. The Rust client reports it as Subscription::stream_deleted and ClusterSubscription::stream_deleted, and a sharded subscription yields ShardEvent::StreamDeleted for each shard instead of trying to resubscribe it (StreamDeleted in Python, streamDeleted in TypeScript) (#1119).
  • A broker closing a shard it no longer has any role in tells a reader still attached that the shard moved, naming its current leader, rather than just ending the stream. Broker::close_stream_shard takes the handoff (#1119).
  • Under the block queue policy, one subscriber at its byte or entry bound no longer holds up the rest of a batch’s fanout. The broker queues the batch for every subscriber with room first and waits only on the full ones, so a fast subscriber gets each batch as soon as it is durable. Each subscriber still receives batches in publish order. The latency demo now passes FELIX_SUB_QUEUE_MAX_BYTES and FELIX_SUB_QUEUE_MAX_BYTES_TOTAL to the broker it builds (#1179).
  • The Node client returns null, as index.d.ts says, for an absent field on a delivered item: laggedResumeFrom, change and shardMoved on a cache watch item, value on a cache change, offset on an event, event, lostError and recovered on a sharded item, and resumeFrom, nodeId and addr on a shard move. They came back undefined, so a caller testing !== null read every cache watch item as a lag notice (#1140).
  • The cluster test harness’s wait_for_replication now waits until the control plane holds a report naming a follower at the leader’s current tail. It used to return once the leader had shipped anything, which the harness’s start-up publish already guaranteed, so a test that killed the leader next could fail over without the records it had just published (#1136).
  • Right after a broker died, a stream with consumer groups could hold each replication pass on it for a request timeout or a dial: the group cursor and dead-letter logs shipped to every follower at the end of the pass, and the next publish waited behind that pass. They now ride each follower’s exchange, as a Quorum cache’s counters do since #1137, and so do a non-Quorum cache’s counters (#1143).
  • A new copy of a Quorum cache is no longer seated before it holds the cache’s counter log. Seating checked only the cache log’s offsets, so the set could count toward its majority a copy that promotion would refuse. A seat on a Quorum stream or cache now also waits until the leader’s report names the copy caught up (#1139).
  • A restore of a cache used only for counters is no longer dropped as stalled while it copies a large counter log. The stall window (FELIX_SHARD_RESTORE_STALL_MS) watched only the cache log’s offset, which stays at 0 for such a cache. Replica reports now carry each replica’s counter-log position (replica_counter_offsets, omitted for streams and by older brokers), and placement counts a move there as progress. The control plane stores it in the report (Postgres migration 0026_replica_report_counter_offsets, Raft metadata level 8; below level 8 reports are recorded without it) (#1164).