Storage

Causet uses five categories of storage: two event-store PostgreSQL databases, a read-model PostgreSQL database, a Redis cache, and an S3-compatible artifact store.


PostgreSQL: causet (Event Store)

The causet database is the primary event store. It is the source of truth for all application state.

TablePurpose
ledger_eventsAppend-only log of all events across all entities and forks
entity_snapshotsMaterialized current state of each entity, derived from ledger events
intent_statusRecord of each intent’s processing result (success, rejection, error)
decision_routesRule execution trace for forensics and Decision Log (Debugging)

ledger_events schema

CREATE TABLE ledger_events (
    id              UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    entity_id       TEXT        NOT NULL,
    fork_id         TEXT        NOT NULL,
    event_type      TEXT        NOT NULL,
    payload         JSONB       NOT NULL,
    ts              BIGINT      NOT NULL,
    ir_version      TEXT        NOT NULL,
    sequence_number BIGINT      NOT NULL,
    intent_id       UUID        REFERENCES intent_status(id)
);

Events are never updated or deleted. The table grows monotonically.

entity_snapshots schema

CREATE TABLE entity_snapshots (
    entity_id   TEXT    NOT NULL,
    fork_id     TEXT    NOT NULL,
    stream      TEXT    NOT NULL,
    state       JSONB   NOT NULL,
    version     BIGINT  NOT NULL,
    updated_at  BIGINT  NOT NULL,
    PRIMARY KEY (entity_id, fork_id, stream)
);

Snapshots are upserted on every successful intent. They cache the current state for fast rule evaluation — the runtime does not replay the full ledger on each intent.

Access pattern

All access via R2DBC (reactive). No JDBC. Writes are transactional: ledger_events append + entity_snapshots upsert + intent_status insert happen in a single R2DBC transaction.


PostgreSQL: projections (Read Models)

The projections database holds all read-model tables. Tables are created and managed by the IR at deploy time — not by Flyway.

Schema isolation

Each fork gets its own PostgreSQL schema:

{platformId}_{applicationId}_{forkId}

Schema names are sanitized (alphanumeric + underscores only) and truncated to ≤ 63 characters (the PostgreSQL identifier limit).

All projection reads and writes use SET LOCAL search_path = {tenant_schema} to scope operations.

Table structure

Tables are declared in the DSL as projections:

projections:
  artist_followers:
    target:
      table: artist_followers
      primary_key: [user_id, artist_id]
    fields:
      user_id:     TEXT
      artist_id:   TEXT
      followed_at: BIGINT

The IR compiler generates DDL for each projection table. DDL is applied at deploy time when a new IR version is activated for a fork. This includes CREATE TABLE IF NOT EXISTS and any required index definitions.

Access pattern

  • Writes: projection-worker uses R2DBC. Schema is set per-event via SET LOCAL search_path.
  • Reads: query-service uses R2DBC. Schema is set per-query via SET LOCAL search_path.

PostgreSQL: causet_control_plane

The control plane database stores platform, application, fork, and release metadata managed by causet-saas-cloud.

Schema is managed by Flyway migrations, unlike the projections DB which is IR-driven.

Contents

  • Platform and application registrations
  • Fork configurations
  • Release records (IR version per release)
  • Active release per fork
  • Deployment history

Redis (IR Artifact Cache)

Redis serves two caching roles:

IR artifact cache

Both projection-worker and query-service cache the loaded causet.projections.json IR artifact in Redis:

key:   ir:{irVersion}
value: serialized IR artifact
TTL:   configurable (default: 5 minutes)

On a version promotion, the cache entry for the old version naturally expires. The new version is loaded from S3 on the first request using the new irVersion.

Query result cache

query-service optionally caches query results in Redis:

key:   query:{queryName}:{forkId}:{paramHash}
value: serialized query result
TTL:   configurable per query in the DSL

Result caching is opt-in per named query. Stale results are served until the TTL expires.


S3 / MinIO (IR Artifacts)

Compiled IR artifacts are stored in S3 (AWS) or MinIO (local development) at:

s3://{bucket}/{irVersion}/causet.runtime.json
s3://{bucket}/{irVersion}/causet.projections.json

causet.runtime.json is consumed by causet-runtime. causet.projections.json is consumed by projection-worker and query-service.

Each compile of an application produces a new irVersion. Old versions remain in S3 for rollback and audit purposes. Lifecycle policies can be configured to expire old versions after a retention period.

Note: For local development, MinIO is the default S3-compatible backend. The artifact URL format is identical — only the endpoint and credentials differ.


DDL Management

StoreDDL Strategy
causet DBFixed schema, managed via Flyway on causet-runtime startup
projections DBIR-driven, applied at deploy time per fork from the projections IR
causet_control_plane DBFlyway migrations on causet-saas-cloud startup

Projection tables are not managed by Flyway. The IR compiler generates DDL from the DSL projection definitions. When a new IR version is deployed to a fork, the runtime applies any DDL changes (add columns, create new tables) before the new version is activated.

Warning: Removing or renaming a projection field is a breaking change. The DDL is not applied destructively — columns are not dropped automatically. You must manage column removal manually or via a migration script.


Schema Isolation

Per-fork schema isolation in the projections database provides:

  • Logical tenant isolation: Each fork’s data is in its own schema, inaccessible to queries in other fork schemas.
  • IR version independence: Different forks can have different active IR versions with different table schemas simultaneously.
  • Safe staging environments: A staging fork can have a different projection schema than production.

Backup and Restore

StoreStrategy
causet DB (ledger)Point-in-time recovery (PITR) via PostgreSQL WAL archiving. The ledger is the source of truth — restoring it fully restores application state.
projections DBCan be rebuilt entirely by replaying causet.projection-events.v1 from Kafka. A backup is a convenience, not a necessity.
causet_control_plane DBPITR or snapshot backup. Losing this database loses fork/release metadata — it should be backed up with the same durability as the ledger.
RedisRedis persistence is optional. IR cache and result cache are re-populated automatically. Treat Redis as ephemeral.
S3 artifactsS3 versioning or MinIO redundancy. IR artifacts are immutable once written — loss means re-compiling and re-deploying.