Background-job owner identity — completion notes

Date: 2026-07-08
Spec: specs/2026-07-08-job-owner-identity-design.md
Plan: plans/2026-07-08-job-owner-identity.md
Status: implemented, gated green, deployed + observatory-verified end-to-end
(commit 0ce6a72).

Applies the OCR engine's job-owner pattern to the chatbot: carry the
submitter's id into detached background jobs so their AI egress attributes to a
person, not to nobody.

What changed

• src/lib/jobs/types.ts (new) — shared EvalJob / ReingestJob payloads
  with an ownerId? field (imported by the route + both workers, no drift).
• api/admin/jobs/route.ts — at enqueue (under requireMutationSession,
  admin session live) stamps ownerId = opaqueStaffId(auth.user.email) onto the
  job payload, overriding any client-sent value (server-authoritative).
• workers/eval-nightly.ts — extracted runEvalJob(data); each agent
  /runs call now carries X-User-Id: data.ownerId ?? "system:nightly-eval",
  a unique X-Request-Id: ${sessionId}-t0-${nonce} per question, and
  X-Priority: batch.
• workers/knowledge-reingest.ts — extracted runReingestJob(data);
  forwards X-User-Id (owner or system:knowledge-reingest) + trace id to
  ai-ahu-rag's /admin/ingest.

Precedence: explicit > job ownerId > system:<queue>. Every eval today is
admin-triggered, so an owner is always present; system:<queue> is the reserved
fallback for a future cron-fired (ownerless) job — labelled, never blank.

Report back (per the prompt)

• Where jobs are created/tracked: POST /api/admin/jobs (the only enqueue
  site) via BullMQ (@ahu/queue). Job record = the BullMQ payload; the new
  state is one ownerId string on it.
• Where ownerId was added: injected in the route from the verified session;
  read back in each worker handler.
• Where the worker execution path was wrapped: the workers set X-User-Id
  directly on their outbound egress (the chatbot has no AsyncLocalStorage
  identity context like OCR — identity travels as an explicit header, so the
  header IS the restore mechanism; the agent's RequestIdMiddleware seeds
  user_id_var from it, attributing the model + embedder egress).
• Precedence now reads owner-before-system: yes.
• Genuinely ownerless job types: none today (all admin-triggered). A future
  cron/repeatable eval would be the ownerless case → system:nightly-eval.

Dormancy

Additive X-* headers only; URLs/bodies untouched. Ignored by plain vLLM when
MODEL_GATEWAY_URL is unset.

Tests

• tests/workers/eval-nightly.test.ts — owner restored as X-User-Id on every
  call + unique per-question trace; ownerless → system:nightly-eval.
• tests/workers/knowledge-reingest.test.ts — owner forwarded; ownerless →
  system:knowledge-reingest.
• tests/api/jobs-owner.test.ts — enqueue captures opaqueStaffId(email) and
  overrides a client-sent ownerId.

Gate: pnpm check green (internal-web 65 / public-web 128 / streams 17 +
typecheck + conventions).

Deploy + verification (done 2026-07-08)

Only the ahu-ai-chatbot-internal image changed (the enqueue route + the
esbuild worker bundle both live in internal-web). Rebuilt that one image
(0ce6a72), verified the bundle carries the fix, shipped it, and recreated the
ahu-internal + ahu-workers stacks (agent-internal left untouched). Both
containers healthy, 0 restarts, running 0ce6a72; workers logged
worker ready for both queues.

End-to-end confirmed. Logged in as the seed admin, enqueued a nightly-eval
(agent=internal, limit=1). Worker log:

[eval-nightly] agent=internal limit=1 url=http://internal-agent:8000 owner=staff-604287292ba92ba0

The observatory recorded the eval's egress (model + embedder), both rows
attributed:

surface  = internal
trace_id = eval-2026-07-08T11-46-22-936Z-t0-b88bae01-5fac-4698-82a6-3196d6bcba62   (shared across both sub-calls)
user_id  = staff-604287292ba92ba0                                                   (was blank/ULID before)

This also closes the one open item from the prior task: the staff opaque-id
actor ahu-chatbot/internal/staff-<hex> now appears live in the observatory
Security view, replacing …/unattributed.