Felix

Early Development

Felix is in early active development and has not been run in production by anyone. The design and implementation are still moving.

Where these pages describe something that is not built, they say so. The status table is the one place that is kept current per capability, and it is the page to trust when another disagrees with it.

Felix is a low-latency distributed data backend that serves streams, caches and work queues from one replicated log, over a single QUIC-based transport layer.

Design goals

New here? Why Felix? explains in plain terms why you might run one system instead of a log, a cache and a queue, and who should not use it. The cards below make the same case for readers who already know the area.

Low tail latency

Designed to keep p99/p999 predictable under load, with aggregate throughput second. QUIC gives multiplexed streams without head-of-line blocking, and explicit backpressure stops one slow path cascading into the rest.

Unified primitives

One core log abstraction behind several semantics: streams (per-subscription fanout cursors), queues (shared consumer-group cursors with acks), and cache (key → value with TTL). You run one system instead of three.

QUIC from the ground up

Encrypted by default with TLS 1.3, multiplexed streams per connection, built-in flow control and congestion awareness, and resistance to head-of-line blocking.

Tunable, explicitly

Connection pooling, stream multiplexing, time- and count-bounded batching, flow-control window sizing, and optional per-stage telemetry. You set all of it; none of it is hidden behind defaults.

Already using Kafka?

Kafka compatibilityPoint existing Kafka producers and partition-assigning consumers at a Felix broker. Topics are streams, partitions are shards, offsets are Felix's. No consumer groups or transactions.

See it behave

Each demo runs a real broker and real clients, and the slow-consumer, state-divergence and queue demos also assert the property they show as a test.

Slow-consumer isolationOne consumer stalls. Under drop_new the healthy consumers never notice; under block it takes down the publisher. The same workload, twice.

Local state divergenceThe counterpart: what an at-most-once configuration costs a consumer that holds local state, and what lossless delivery costs in throughput.

How It Works

Felix uses a framed protocol (felix-wire) over QUIC streams to unify event streaming and request/response caching, with explicit control over multiplexing, batching, and flow control.

Streams, caches and queues are three ways of reading the same bytes. They share one durability path, one recovery path, one placement rule and one replication path.

Pub/Sub data flow

1. The client opens a bidirectional control stream to publish/subscribe and receive acknowledgements.
2. The broker validates scope, enqueues publish jobs, and fans out to subscribers.
3. Each subscription gets a dedicated unidirectional event stream for delivery.
4. Events travel as binary EventBatch frames with configurable batching.

Cache data flow

1. The client maintains a cache connection pool with long-lived stream workers.
2. Cache requests carry a request_id and are multiplexed over those streams.
3. The broker processes request frames in a read loop and replies on the same stream.
4. This avoids per-request stream setup cost and improves tail latency under concurrency.

Queue data flow

1. A consumer polls a group rather than being pushed to, because only it knows when it has capacity.
2. The broker claims records against a visibility timeout and hands them over with their attempt counts.
3. An acknowledgement settles one record; the group’s durable cursor advances over a contiguous run of them.
4. Anything unanswered is redelivered, and dead-lettered once it has been attempted too many times.

Composed data flow

Because every semantic is a reading of one log, they compose. Each flow below covers something applications usually build by hand:

1. Watch a key or prefix (keyed watch): config push and cache invalidation. Every instance learns of a change the moment it lands, with an offset to resume from instead of a poll loop.
2. Current state first, then changes (retained delivery): presence and state sync. Join a room, get the roster at once, and stay current; the confirmation marks the exact moment your state is complete.
3. Deltas folded into a durable sum (counters): rate limits, usage metering and live tallies. Increment and learn where you stand in one round trip, with a sum that survives restart, compaction, and failover.

Key Features

Feature	What it gives you
QUIC transport	Modern, multiplexed, encrypted transport layer
High fanout	Efficient delivery to many subscribers, with isolation between them
Connection pooling	Reusable connections and streams, so hot paths skip setup cost
Batch processing	Time- and count-bounded batching for throughput
TTL support	Time-to-live on cache entries with lazy expiration
Work queues	Consumer groups with acknowledgements, redelivery and dead letters
Kafka compatibility	Kafka producers, and consumers that assign their own partitions, work against durable streams
Backpressure	Explicit flow control at six checkpoints to prevent cascade failures
Metrics	Prometheus-compatible metrics endpoint
Observability	Structured logging with optional per-stage telemetry

Use Cases

Felix is a good fit today for:

• Real-time streaming with high fanout and tunable latency/throughput trade-offs
• Event pipelines with batch publishing and batch delivery
• Low-latency caching over QUIC with predictable tail latency under load
• Work distribution across consumers, where each job goes to one of them and is retried if nobody says it was handled
• Microservice communication with unified stream, cache and queue semantics

See What Felix Is For for where Felix fits and where it does not.

Current Status

Working today

• Multi-broker clusters with rendezvous-hashed shard placement
• Durable log-structured storage, with retention and torn-tail repair
• Replication with quorum acknowledgement and fenced failover
• Online rebalancing: live shard moves, broker drain and join, with no refused publish or lost record
• A log-backed cache: routed to one owner, replicated, put/get/delete
• Consumer groups: poll, acknowledge, redeliver, dead-letter, redrive
• Kafka compatibility: Kafka producers (idempotent ones included) and partition-assigning consumers against durable streams; no Kafka consumer groups or transactions
• A control plane over REST, holding metadata in Raft (no external database) or in Postgres, and capability negotiation on the wire
• Tenant-scoped tokens, RBAC, and OIDC token exchange
• Mutually authenticated broker-to-broker QUIC, when certificates are configured

Not built yet

• Load-aware placement: shards are balanced by count, not by traffic, disk or CPU
• End-to-end payload encryption, encryption at rest, and audit logging
• Tiered storage and cold-tier reads
• Cross-region bridges with explicit data movement
• Clients in Go or C# (Rust, Python and TypeScript exist today)

Quick Start

1. Clone the repository.

   git clone https://github.com/gabloe/felix.git
   cd felix

2. Build the workspace.

   cargo build --workspace --release

3. Run a demo. Each one is self-contained and starts its own broker.

   task demo:slow-consumer
   task demo:state-divergence

4. Or start a real cluster (control plane, credentials and three brokers) and publish through one broker to a shard another one owns.

   cargo run --release -p felix-cluster -- up --nodes 3
   cargo run --release -p felix-cluster -- subscribe orders   # second window
   cargo run --release -p felix-cluster -- publish orders hi  # third

See the Demos Overview or the Quickstart Guide for the full set.

Community & Support

Source on GitHubThe workspace, the demos, and the CI that runs them.

IssuesReport a bug or request a feature.

ContributingHow to build, test, and land a change.

How Felix WorksThe end-to-end tour, from a publish call to a subscriber's socket.