Replay

The ledger is the source of truth. Entity snapshots, projection tables, and query results are all derived — they can be rebuilt or inspected by re-processing history. Because rules are deterministic, the same event sequence always produces the same outcome.

Use this page when a workflow failed halfway and you need to inspect → fork → replay → repair without guessing from logs.

Failed workflow walkthrough

Example: access request notification failed after approval.

request submitted      ✓  access_requested
approval succeeded     ✓  access_approved
notification failed    ✗  notification_failed
projection stale          status still pending_notification

1. Inspect the timeline

causet inspect timeline --stream access_request_stream --entity ar_123

Or open Decision Timeline in the control plane. Confirm which events committed and which side effect failed. See Timeline.

2. Understand event history

Entity state may show approved while your app DB / projection still shows pending_notification.

3. Simulate a failure

On a fork (not main):

1. Branch from the cursor after access_approved
2. Force or re-run the notification path with a bad provider / mock failure
3. Confirm notification_failed appears and the projection lags

This teaches the team what the timeline looks like before a production incident.

4. Fork from a known point

From Decision Timeline → Branch from cursor, or create a sandbox fork. Explore repairs without touching production.

5. Replay with corrected input

• Fix the notification template / provider config
• Re-submit a guarded notification intent on the fork (idempotent)
• Or rebuild the entity snapshot / projection after deploying corrected handlers

Gate side effects with is_replay or idempotency keys so you do not double-email on main.

6. Repair downstream projection state

If the read model is stale:

1. Deploy corrected projection IR if the bug was in derive / handlers
2. Rebuild the projection from the ledger (UPSERTs are idempotent)
3. Optionally update your existing app DB from the repaired events (webhook/consumer bridge)

Do not treat a one-off SQL UPDATE as the long-term fix — the ledger should remain the source of truth.

What replay means in Causet

The source of truth

Everything downstream of commit is derived:

• Entity snapshots — materialized state per stream/entity/fork
• Projection tables — read models for named queries
• Vector memory — re-ingested from source events

If derived state is wrong or missing, replay from the ledger. The ledger itself cannot be reconstructed from snapshots or projections — protect it accordingly.

Entity Inspector: live vs replay mode

The Deep Entity Viewer supports two modes:

In replay mode, the scrubber loads state at cursor from the ledger (not the live snapshot). Use this to answer: what did this entity look like when intent X committed?

You can also compare two cursors (state diff) to see exactly which fields changed between commits.

Rebuilding an entity snapshot

When a snapshot is stale or corrupted, rebuild it from the ledger for one entity:

1. Open the entity in Entities or the Deep Entity Viewer.
2. Run Rebuild — the runtime re-applies ledger commits in order and writes a fresh snapshot.

Rebuild is scoped to a single (stream, entity, fork). It does not re-run side effects or re-emit Kafka events for projections — it fixes entity materialized state.

When to use:

• Snapshot drift after an incident
• After fixing a rules bug (deploy fixed IR first, then rebuild affected entities)
• Verifying ledger vs snapshot during an investigation

Decision Timeline replay and branch

From Decision Timeline (per entity), you can:

Replay and branch here are investigation tools tied to timeline. They complement the Entity Inspector scrubber: decisions show how rules ran; replay mode shows what state was.

For the same jobs from the terminal (sandbox), use causet forensics: replay-check for snapshot drift, and repair-plan / repair-apply for an inverse patch on one timeline step.

Projection rebuild

Projection tables are filled asynchronously from the event stream. To rebuild read models:

1. Deploy IR with corrected projection handlers or schema.
2. Reset the projection worker consumer to replay events from the needed offset (or from the beginning for a full backfill).
3. The worker re-applies handlers — UPSERTs are idempotent, so re-processing is safe.

When to use:

• New projection added — backfill from history
• derive: expression or handler bug fixed
• Column type change after schema migration

Projection rebuild does not re-run intent rules or side effects — only projection handlers. No risk of duplicate emails or charges from projection replay alone.

Determinism

Core rules are deterministic: no wall clock, no randomness, no external I/O in preflight or core. Replaying the same events with the same ruleset version yields the same state.

Same events + same rules = same result. That is why rebuild and scrubber replay are trustworthy after incidents.

Side effects and replay

Warning: Snapshot rebuild and decision replay paths that re-execute rules can re-fire side effects (emails, webhooks, submit, AI calls) if your rules are not guarded.

Patterns:

• Pass is_replay: true in intent payload and gate side_effects rules.
• Use idempotency keys for external calls.
• Run risky replays on a branch fork, not main.

Choosing the right tool

See also

• Forensics CLI — causet forensics drift check and per-decision repair
• Entity Inspector — browse, scrub, diff, rebuild
• Timeline — step-by-step rule trace
• Ledger Events — commits and intent status
• Idempotency — safe retries and replays
• Schema Versioning — when projection rebuild is required