Audit guardrail refusals as `status: "refused"` — design

Date: 2026-07-09
Status: spec
Source: integration prompt INTEGRATION-chatbot-refused-audit-status.md.

Problem

The observatory's probing rule (sec_error_probing) watches per-actor refusal
rates in ai_calls, but the chatbot never tells the audit trail a refusal
happened. When the L1 input guard blocks or the L2 planner returns refuse=true,
the only audit event for that turn is the planning LLM call — logged status:ok.
So a real jailbreak attempt is structurally invisible. The observatory side is
already fixed + deployed (2026-07-09): its rule counts status NOT IN ('ok','completed') rows whose error_code is not a gateway infra code. The
moment this repo emits status:"refused", those rows light up the rule with zero
observatory changes.

Design

One emit choke point

Every structured refusal (L1 PII/entity guard, no-tools, L2 refuse=true,
unparseable plan) flows through plan() → the isRefusal(p) branch in
orchestrate.ts. A single fire-and-forget emit there covers them all, placed
before streaming the refusal copy (but never able to delay/break it).

Refusal category → `error_code`

Refusal (@ahu/orchestrator-types) gains an optional machine-readable code
(RefusalCode union), set at each creation site:

Path	Code
L1 PII pattern	`GUARDRAIL_INPUT_PII`
L1 entity/off-topic/custom block	`GUARDRAIL_INPUT_ENTITY`
L2 planner `refuse=true`	`GUARDRAIL_PLAN_REFUSED`
No active tools	`NO_TOOLS`
Unparseable plan	`PLAN_UNPARSEABLE`

GUARDRAIL_* is a stable namespace (none match the infra UPSTREAM_*/HTTP_*
prefixes, so all count as probing evidence). NO_TOOLS/PLAN_UNPARSEABLE are
honesty cases — refusal-shaped config/parse issues, audited distinctly.

Emitter (`apps/public-web/src/lib/audit/emit.ts`)

New module — public-web-only for now, so it lives in the app (moves to a package
only when a second app needs it). Mirrors the anon-limits lazy-singleton
fail-open ioredis pattern, but on a separate env var / instance:

AUDIT_REDIS_URL — no default; unset ⇒ no-op (the §6 dormancy
contract). AUDIT_STREAM — default ahu.ai.audit.
Fire-and-forget (§5): lazyConnect, maxRetriesPerRequest:1, no retry
strategy, commandTimeout:500, every failure swallowed after a rate-limited
console.warn. Never throws.
XADD <stream> * json <event-json> — schema-v1 event; event_id = uuidv7
(never blank — the ingester treats blank as poison). Body columns omitted;
the user's message text never enters the event (metadata only).

Event fields: engine:"ahu-chatbot", surface = turn surface, user_id = the
turn's X-User-Id (req.sessionId), trace_id = the turn's X-Request-Id
(turnTraceId — joins the refusal to the turn's other calls), status:"refused",
error_code = the code, operation:"chat", traffic_class:"interactive",
total_ms = turn start → refusal, config_version = policy version.

Deploy wiring (dormant)

The platform audit Redis is ahu-platform-redis on the external
ahu-platform_default network (confirmed: it holds ahu.ai.audit). public-web
joins that network in compose.public.yaml so the host resolves once flipped,
but AUDIT_REDIS_URL stays unset by default → ships dormant. Flip is a
deploy-time decision for Efran.

Out of scope (flagged)

The staff/internal native-Agno path has no structured refusal signal (refusals
there are prompt-driven text) — it cannot emit refused events yet, and we will
NOT string-match agent output. Follow-up.

Tests (TDD)

Refusal code per path; buildRefusalEvent schema + no message text; emit XADDs
once to the right stream with correct fields; dormant when unset; fire-and-forget
when Redis throws; non-refusal turns emit nothing.

Acceptance

[ ] Every structured refusal emits one schema-v1 status:"refused" event with
the right GUARDRAIL_*/NO_TOOLS/PLAN_UNPARSEABLE code; trace_id =
turn X-Request-Id; no message content.
[ ] Unset AUDIT_REDIS_URL ⇒ byte-identical; dead Redis ⇒ user stream intact.
[ ] pnpm check green; no deploy without go-ahead.

Audit guardrail refusals as status: "refused" — design