PII Handling

The Causet event ledger is append-only. Events committed to the ledger cannot be deleted through normal application operations. This creates a compliance obligation: any PII written into event payloads is effectively permanent.

Design your events to avoid this problem from the start.

The core problem

If a user’s email address appears in an ARTIST_FOLLOWED event payload:

# Do NOT do this
events:
  ARTIST_FOLLOWED:
    payload:
      user_id:    string
      user_email: string   # PII in the ledger
      artist_id:  string

That email address is now committed to ledger_events and cannot be removed without corrupting the audit trail. If the user exercises GDPR right to erasure, you cannot comply without deleting ledger rows.

Recommended design: opaque IDs

Use opaque identifiers in event payloads. Resolve PII at read time from a mutable store.

# Do this instead
events:
  ARTIST_FOLLOWED:
    payload:
      user_id:   string   # opaque ID only
      artist_id: string

Your PII (email, name, phone number) lives in a separate mutable database table outside Causet. When you need to display the user’s name, join the opaque user_id against that table at query time.

This pattern:

• Keeps the ledger free of PII
• Allows PII updates without touching the ledger (user changes their email — update the user table, not the ledger)
• Supports right to erasure — delete the user’s row in the mutable table; the ledger still has the user_id but no PII resolves for it

What to put in events vs what not to

Avoid PII in Kafka messages

Kafka messages contain the event payload and are retained for a configurable period (default: 7 days in most configurations). PII in Kafka messages propagates to consumers and log archival systems.

The causet.projection-events.v1 topic contains the full event payload. If event payloads contain PII, it will appear in this topic, in projection worker logs, and in any consumer of this topic.

GDPR right to erasure

If you have PII in the ledger (e.g., in a legacy deployment), three options exist:

Option 1: Cryptographic erasure

Encrypt PII fields in event payloads with a per-user encryption key. To erase a user’s data, delete their encryption key. The ciphertext remains in the ledger but is unreadable.

ARTIST_FOLLOWED payload (stored in ledger):
{
  "user_id": "user-1",
  "user_email_enc": "AES256(user-1-key, 'alice@example.com')",
  "artist_id": "artist-1"
}

On erasure request: delete user-1-key from your key management system. The encrypted email is now unrecoverable. The rest of the event record (user_id, artist_id, timestamp) remains.

This approach is recognized by some data protection authorities as equivalent to deletion, but verify with your legal counsel.

Option 2: Mutable projection tables

Projection tables are writable by normal SQL operations. If PII needs to be erased, update or delete the row in the projection table:

UPDATE my_platform_app_prod.user_profile
SET email = NULL, name = 'Deleted User'
WHERE user_id = 'user-1';

The ledger event is unchanged. The projection table reflects the erasure. This works when the user-facing view of the data is served from projections, not directly from the ledger.

Option 3: Pseudonymization at write time

Replace identifying values with pseudonyms at intent time before writing to the ledger. The pseudonym is stored in a mapping table outside the ledger. To erase, delete the mapping entry:

Real: alice@example.com  →  Pseudonym: pii_token_xyz

ARTIST_FOLLOWED payload in ledger:
{
  "user_id": "user-1",
  "user_email": "pii_token_xyz"   # pseudonym, not real email
}

On erasure: delete the mapping for pii_token_xyz. The ledger contains a pseudonym that no longer maps to a real identity.

PII in logs

Structured logs ship to Grafana Loki and may be retained for months. Do not log PII:

// Bad
log.info("Intent received: user={} email={}", userId, userEmail);
 
// Good
log.info("Intent received: userId={}", userId);

Review your log fields for any fields that might contain PII from the intent payload. At DEBUG level, full payloads may be logged — ensure DEBUG is not enabled in production, or apply a payload sanitizer before logging.

Recommendations

1. Design events with IDs only — make this a team convention enforced in code review
2. Maintain a PII store — a mutable table (in your own database, not Causet) that maps opaque IDs to PII values
3. Resolve at read time — join the PII store when building user-facing responses
4. Audit your event schemas — periodically review all event payload fields for PII
5. Document your GDPR strategy — decide which option (cryptographic erasure, pseudonymization) you will use before going live