causet replay

Proposed: causet replay is on the roadmap. See Roadmap. The current approach is a manual Kafka consumer group offset reset via the Kafka CLI.

Resets the Kafka consumer group offset for a named projection, triggering the projection worker to reprocess events from the specified starting point.

Usage

causet replay \
  --projection <projection-name> \
  [--from-beginning] \
  [--from-timestamp <ts>] \
  [--from-offset <offset>] \
  --platform <platform> \
  --app <app> \
  --fork <fork> \
  --env <env>

Example — full rebuild from the beginning:

causet replay \
  --projection user_following \
  --from-beginning \
  --platform my-platform \
  --app concert-app \
  --fork main \
  --env production

Example — replay from a specific timestamp:

causet replay \
  --projection artist_show_directory \
  --from-timestamp "2026-06-01T00:00:00Z" \
  --platform my-platform \
  --app concert-app \
  --fork main \
  --env production

Flags

If none of --from-beginning, --from-timestamp, or --from-offset are specified, the command defaults to --from-beginning.

Safety Confirmation

Before resetting, the command displays a confirmation prompt:

Projection: user_following
Fork:       main
Platform:   my-platform / concert-app
Reset:      from beginning

This will truncate the 'user_following' table and reset the consumer group offset.
All rows will be rebuilt from the event stream. This cannot be undone.

Type the projection name to confirm: 

Type the projection name exactly to proceed. Use --no-confirm only in scripted contexts where you have already validated the inputs.

What the Command Does

1. Pauses event consumption for the named projection (or stops the worker)
2. Truncates the projection table in the tenant schema
3. Deletes the projection’s checkpoint record
4. Resets the Kafka consumer group offset for causet.projection-events.v1
5. Resumes the projection worker

Monitoring Recovery

After replay starts, monitor consumer lag to track progress:

kafka-consumer-groups.sh \
  --bootstrap-server $KAFKA_BOOTSTRAP_SERVERS \
  --describe \
  --group causet-projection-worker

Or check projection metrics:

causet_projection_lag_seconds{projection="user_following"}

Replay is complete when causet_projection_lag_seconds reaches 0 for the target projection.

Replay Duration Estimation

estimated_duration = (total_events × avg_processing_time_ms) / 1000

For a projection with 1 million events at 2ms per event: ~33 minutes.

Factors that increase replay time:

• Complex derive: expressions with many joins
• Large event payloads
• Slow PostgreSQL writes (connection pool exhaustion)
• Running multiple replays simultaneously

Warnings

Warning: Do not replay projections that trigger external side effects (notifications, billing, email). Replaying will re-execute the derive expressions for each event, and any side-effect actions that were triggered by those events will not re-run — but data that was conditionally set during original processing may behave differently if entity state has since changed.

Warning: Do not replay from a --from-timestamp unless you are certain the projection handles partial event windows correctly. Projections that use delete mutations may leave orphaned rows if the replay window does not include the original insert events.

Manual Replay (current approach)

Until causet replay is available, use the Kafka CLI directly:

# Stop the projection worker first (or scale to 0)
 
# Truncate the projection table
psql $PROJECTIONS_DB_URL -c "
  TRUNCATE TABLE my_platform_concert_app_main.user_following;
  DELETE FROM projection_checkpoints WHERE projection_name = 'user_following';
"
 
# Reset consumer group offset
kafka-consumer-groups.sh \
  --bootstrap-server $KAFKA_BOOTSTRAP_SERVERS \
  --group causet-projection-worker \
  --reset-offsets \
  --to-earliest \
  --topic causet.projection-events.v1 \
  --execute
 
# Restart the projection worker