System Events

Causet can emit operational events for observability, alerting, and audit. These system events describe infrastructure behavior rather than domain facts.

Note: System events are partially implemented and partially aspirational. Each event below is marked with its current status.

What System Events Are For

System events let you:

• Alert on projection failures without scraping logs.
• Track replay lifecycle in the audit trail.
• Build internal dashboards on processing health.
• Feed observability tooling (Datadog, Grafana) from the same event pipeline as your domain events.

Event Catalog

PROJECTION_FAILED

Note: Emitted by projection-worker — implemented.

Emitted when a projection handler fails after exhausting all retries. Written to causet.projection-dlq.v1 as a DLQ record and optionally emitted as a system event.

events:
  PROJECTION_FAILED:
    kind: system
    state: _system
    entity_expr: event.entity_id
    payload:
      entity_id:       string
      event_type:      string
      projection_name: string
      fork_id:         string
      tenant_schema:   string
      error:           string
      retry_count:     int
      failed_at:       datetime

HANDLER_RETRIED

Note: Planned — not yet implemented.

Emitted each time a projection handler retries a failed event. Useful for detecting systemic retry storms.

events:
  HANDLER_RETRIED:
    kind: system
    state: _system
    entity_expr: event.entity_id
    payload:
      entity_id:       string
      event_type:      string
      projection_name: string
      fork_id:         string
      retry_attempt:   int
      error:           string
      next_retry_in_ms: int

REPLAY_STARTED

Note: Planned — not yet implemented.

Emitted when a projection rebuild or snapshot replay begins.

events:
  REPLAY_STARTED:
    kind: system
    state: _system
    entity_expr: event.replay_id
    payload:
      replay_id:       string
      replay_type:     string   # "projection" or "snapshot"
      projection_name: string   # null for snapshot replays
      fork_id:         string
      initiated_by:    string
      started_at:      datetime

REPLAY_COMPLETED

Note: Planned — not yet implemented.

Emitted when a replay completes successfully.

events:
  REPLAY_COMPLETED:
    kind: system
    state: _system
    entity_expr: event.replay_id
    payload:
      replay_id:       string
      replay_type:     string
      events_processed: int
      duration_ms:     int
      completed_at:    datetime

INTENT_REJECTED

Note: Tracked in intent_status — a dedicated event is planned but not yet emitted to a system stream.

Currently, rejections are recorded in the intent_status table. A system event for rejections would enable projection-based rejection dashboards.

How to Subscribe to System Events

System events can be consumed in projections and listeners like any other event:

projections:
  projection_failure_log:
    source_events: [PROJECTION_FAILED]
    target:
      table: projection_failure_log
      primary_key: [entity_id, failed_at]
    fields:
      entity_id:       TEXT
      event_type:      TEXT
      projection_name: TEXT
      fork_id:         TEXT
      error:           TEXT
      retry_count:     INT
      failed_at:       BIGINT
    derive:
      entity_id:       event.entity_id
      event_type:      event.event_type
      projection_name: event.projection_name
      fork_id:         event.fork_id
      error:           event.error
      retry_count:     event.retry_count
      failed_at:       event.ts
    mutations:
      PROJECTION_FAILED: { op: upsert }

This materializes a projection_failure_log table queryable via query-service.

Recommended Operational Events to Track

For production monitoring, implement projections and alerts on:

System Events vs Structured Logs

System events and structured logs serve different purposes:

For immediate alerting (SLA-critical), use log-based alerts. For historical analysis and dashboards queryable by application logic, use system events materialized into projections.

Related Pages

• Defining Events — how to declare system events in the DSL
• Concepts: Projections — how system events are materialized