Event Flow

This page traces the complete lifecycle of a single intent from client submission to a queryable projection row.

End-to-End Flow

Phase 1: Intent Submission

The client submits an intent to the SaaS API (or directly to causet-runtime for server-to-server calls):

POST /v1/intents
Authorization: Bearer <api-key>
 
{
  "action": "FOLLOW_ARTIST",
  "entityId": "user-123",
  "forkId": "prod",
  "payload": {
    "user_id": "user-123",
    "artist_id": "artist-456"
  }
}

The SaaS API authenticates the request, resolves the fork, and forwards it to causet-runtime.

Phase 2: Rules Evaluation (Synchronous)

causet-runtime processes the intent synchronously:

1. Resolves active IR version for the fork.
2. Acquires per-entity cursor lock.
3. Loads entity snapshot (current state).
4. Runs preflight rules — validation. Rejection at this stage returns immediately to the caller, no writes occur.
5. Runs core rules — produces ledger_events.
6. Runs side-effects rules — may emit additional events, submit downstream intents, or schedule delayed intents.

Phase 3: Ledger Commit (Synchronous, Durable)

Produced events are appended to ledger_events in the causet PostgreSQL database within a single R2DBC transaction:

INSERT INTO ledger_events
  (id, entity_id, fork_id, event_type, payload, ts, ir_version)
VALUES
  ($1, $2, $3, $4, $5, $6, $7);

The entity_snapshots table is updated and intent_status is written in the same transaction.

The caller receives the response after the ledger commit completes — the write is durable before the response is sent.

Phase 4: Kafka Publish (Asynchronous, Fire-and-Forget)

After the ledger transaction commits, causet-runtime publishes enriched projection events to causet.projection-events.v1. This is asynchronous and does not block the response to the caller.

The projection event envelope contains:

• The original ledger_event payload
• Fork ID and tenant schema name
• IR version
• Event sequence number

Note: If the Kafka publish fails transiently, the write response has already been returned. Projection lag or gaps can be recovered by replaying events from the ledger. See Replay.

Phase 5: Projection Materialization (Asynchronous)

The projection-worker consumes events from causet.projection-events.v1:

1. Deserializes the projection event envelope.
2. Resolves the active IR version and loads causet.projections.json from S3 (cached).
3. Matches the event type against declared source_events in the projections IR.
4. For each matching projection:
   • Evaluates derive expressions to compute field values.
   • Executes the configured mutation (upsert, delete, or replace) against the tenant schema.
5. Commits Kafka consumer offset after successful processing.

Phase 6: Query

Once the projection row exists, query-service can serve it:

GET /v1/queries/artist_followers?artist_id=artist-456&fork_id=prod

The query service executes a named, pre-compiled SQL query against the projections database, scoped to the tenant schema.

Timing

Clients should not assume projection data is available immediately after a successful write response. Use the WebSocket gateway (ws-gateway) or poll the query service for time-sensitive reads.

Failure Scenarios

Preflight rejection

The intent is rejected before any write. The caller receives a structured error with a rejection code and message. No ledger event is written.

{
  "error": "PREFLIGHT_REJECTION",
  "code": "CANNOT_FOLLOW_SELF",
  "message": "A user cannot follow themselves."
}

Ledger write failure

If the database write fails (e.g. connection loss, constraint violation), the intent fails and no events are emitted. The caller receives a 5xx error. The intent can be safely re-submitted (no partial state was written).

Kafka publish failure

The ledger commit has succeeded. The caller has already received a 200. The projection worker will not receive this event until a replay is triggered. This is expected — Causet’s projection pipeline is eventually consistent by design.

Projection worker failure

The projection worker retries failed events with exponential backoff. After exhausting retries, the event is sent to causet.projection-dlq.v1. The projection row may not exist or may be stale until the DLQ is processed. See Dead Letter Queues.

Query service failure

The query service is stateless and reads from PostgreSQL. Failures are transient. Retry the query.

Retry and DLQ

Projection events that fail processing are retried by the projection-worker with exponential backoff. After the configured maximum retry count, the event is written to causet.projection-dlq.v1 with:

• The original event payload
• The error message and stack trace
• The retry count
• The handler name that failed

DLQ events can be inspected, fixed (by deploying a corrected projection definition), and re-consumed. See Dead Letter Queues.

Related Pages

• Runtime Architecture — how causet-runtime processes intents
• Projection Pipeline — detailed view of the projection worker
• Retries — retry policies
• Dead Letter Queues — handling projection failures
• Replay — rebuilding projections from the ledger