Monitoring & alerting

The system emits everything you need; this page tells you what to actually look at. Three layers: health (is it up), quality (is it deciding well), and abuse (is someone attacking it).

Health

Check	How	Alert when
Liveness	GET /health every 30–60s	Non-200 for >2 min
Decision latency	Time from selfie upload (capture.selfie_uploaded audit event) to verification.decided	p95 > 60s sustained
Pipeline errors	decision_reasons containing pipeline error: / audit action verification.pipeline_error	Any occurrence — these are bugs or dependency failures (AI API down, model init failure)
Webhook failures	Your side: expected deliveries that never arrived	Reconcile-poll finds decided verifications you weren’t notified about

Quality (the numbers that tell you the system is drifting)

Track these as daily/weekly ratios; alert on step changes, not absolute values:

Metric	Healthy shape	A step change means
Approval rate	Stable for your population	Up sharply: check thresholds weren’t loosened. Down sharply: capture UX regression or attack wave
Manual-review rate	Low double digits early, falling as you tune	Rising: capture quality problem, new document type you don’t handle, or fraud probing
Review overturn rate (reviewer approves what pipeline flagged)	Meaningful minority	Very high: thresholds too strict. Near zero: thresholds may be too loose to bother reviewing
Rejection reasons distribution	Diverse	One reason dominating = one specific failure mode (e.g. every reject is expired → date parsing bug or population quirk)
Time-in-review	Hours	Days: your review staffing is the bottleneck, users are churning

Abuse (the suspicious-activity layer)

All queries run against the audit_events and verifications tables — the audit chain is designed to be your SIEM feed.

Retry-until-lucky — same user reference, many attempts:

SELECT external_user_ref, COUNT(*) AS attempts,
       SUM(CASE WHEN status = 'rejected' THEN 1 ELSE 0 END) AS rejections
FROM verifications
WHERE created_at > now() - interval '24 hours' AND external_user_ref IS NOT NULL
GROUP BY external_user_ref
HAVING COUNT(*) >= 3
ORDER BY attempts DESC;

Forgery template circulating — one tamper signal repeating across users:

SELECT reason, COUNT(*) FROM verifications,
     jsonb_array_elements_text(decision_reasons::jsonb) AS reason
WHERE created_at > now() - interval '7 days' AND reason LIKE 'tamper signal:%'
GROUP BY reason HAVING COUNT(*) >= 3 ORDER BY count DESC;

Leaked API key — key fingerprint active at unusual hours/volumes:

SELECT actor, date_trunc('hour', created_at) AS hr, COUNT(*)
FROM audit_events
WHERE action = 'verification.created' AND created_at > now() - interval '48 hours'
GROUP BY actor, hr ORDER BY hr DESC, count DESC;

Identity-swap attempts — the strongest single fraud indicator; alert on every occurrence:

SELECT verification_id, created_at, detail
FROM audit_events
WHERE action = 'verification.decided'
  AND detail::text LIKE '%identity changed between frames%';

Audit-chain integrity — run daily; any break is a critical incident:

-- Recompute each event's hash from its fields + prev_hash and compare.
-- A convenience verifier script belongs in your ops toolbox; the chain fields
-- (actor, action, verification_id, detail, created_at, prev_hash, hash) are all
-- in the audit_events table.

Where to send it

Anything that speaks Postgres + HTTP works. Pragmatic starting stack: a scheduled job (cron/Railway) running the SQL above, posting to Slack/email; /health on any uptime monitor; decision-latency and rate metrics derived from the audit table. When volume justifies it, ship audit_events to a real SIEM — it is append-only and PII-light by design, so exporting it is low-risk.