Kafka destinations
A Kafka destination delivers committed Causet events to a topic on a cluster you control (Confluent Cloud, Amazon MSK, Redpanda, self-hosted). Causet never runs your brokers — it only produces to them after ledger commit.
Create a destination
In Settings → Destinations → New → Kafka, provide:
| Field | Required | Notes |
|---|---|---|
| Name | Yes | Display name in the UI |
| Bootstrap servers | Yes | Comma-separated host:port |
| Topic | Yes | Target topic name |
| Security protocol | Yes | PLAINTEXT, SASL_SSL, SSL, … |
| SASL mechanism | If SASL | PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 |
| Username / password | If SASL | Password stored encrypted; never returned in API responses |
| Acks | No | Default all |
| Compression | No | e.g. lz4 |
| Client ID | No | Optional producer identity |
Then attach one or more subscriptions so the engine knows which event types to export.
What gets produced
Each matched committed event becomes one Kafka record with the customer export envelope (specVersion 1.0):
| Kafka field | Source |
|---|---|
| Key | Partition-key strategy (default stream_id, fallback event_id) |
| Value | JSON body: id, type, source, subject, time, data, causet, … |
| Headers | content-type, causet-spec-version, causet-event-id, causet-event-type, causet-delivery-id, causet-destination-id, causet-subscription-id, causet-replay, plus optional sequence / correlation / traceparent |
Consumers must dedupe on causet-event-id (header) or body id. Delivery is at-least-once. Full field list: Event envelope.
Local development
Point a destination at a local broker (for example Redpanda or Kafka on localhost:9092) with PLAINTEXT, create a subscription on *, and submit intents. Confirm with:
# example — adjust for your tooling
kcat -C -b localhost:9092 -t your.topic -JOr inspect the Delivery Log in Settings → Destinations for status, attempts, and last error.
Operations
| Action | Effect |
|---|---|
| Test connection | Produces a probe message; does not require a matching subscription |
| Pause / resume | Stops or resumes job dispatch for that destination |
| Rotate credentials | Updates the encrypted secret without changing the destination id |
| Delivery Log | Per-event delivery history (success, retry, dead-letter) |