Runbooks

Rotating the encryption key

Full procedure in appsterisk-svc/docs/KEY-ROTATION.md.

The shape: the data key never changes, so rotation re-wraps 32 bytes rather than re-encrypting every sealed column. Adding a key and activating it are separate deployments — during a rolling update both configurations exist, and a pod on the old one cannot unwrap what a pod on the new one wrote.

Two things stated there without softening: rotating does not undo a compromise, and losing the key makes every sealed column permanently unreadable.

:::danger Never executed The procedure has been written and never run. A runbook that has not been run is a hypothesis. :::

Seeding

DATABASE_URL=… npm run seed

Idempotent and non-destructive. Run it after any change to the seed files; nothing does it automatically.

Restoring the database

Neon provides point-in-time recovery within its retention window, and that is the only backup that currently exists.

:::danger No independent backup There is no dump outside Neon. Neon's own retention is the entire recovery story, so a Neon-side failure or an account problem has no second copy behind it. BACKUP-RESTORE.md does not exist yet. :::

Investigating a namespace

ssh root@appsterisk
kubectl -n appsterisk-prd logs deploy/appsterisk-svc --tail=100

Ownership history is append-only and records every change with actor and reason — start there when ownership is in question rather than with the current row, which only shows the end state.

Monitoring

:::warning Not built There are no dashboards, no alerts and no synthetic checks. The OpenTelemetry exporters are wired and disabled. Nothing will tell you the service is down — which is exactly how Webhookr lost seven days in August. :::