Route public RAG /search through the gateway — design

Date: 2026-07-08
Status: spec
Source: integration prompt prompt-ai-ahu-chatbot-rag-proxy.md. Follow-up to
the identity/trace work
(specs/2026-07-08-audit-identity-and-trace-uniqueness-design.md).

Problem

The public orchestrator's RagTool does POST ${RAG_BASE_URL}/search directly
to ai-ahu-rag:8110. Post-split, public-web runs in the DMZ and ai-ahu-rag
in the Internal zone, so that call crosses DMZ→Internal. The platform decision
is to route it through the gateway (:8200, the sanctioned cross-zone path),
which exposes POST /search as a verbatim facade to the RAG service
(audited operation=search). So this is a base-URL swap + the standard §2
headers, not a client rewrite.

Design

Seam — RAG_SEARCH_URL (falls back to RAG_BASE_URL)

RAG_BASE_URL has three consumers: the public orchestrator's /search
(this flip), the knowledge-reingest worker's /admin/ingest, and the admin
search proxy. The gateway only proxies /search, so overloading RAG_BASE_URL
would break the other two. A scoped RAG_SEARCH_URL (fallback
RAG_BASE_URL) flips only the public retrieval path; everything else stays on
ai-ahu-rag. This is the self-documenting seam the prompt offers.

Headers (CONVENTIONS §2) — reuse the identity threading

RagTool gains an optional RagIdentity (surface, userId, requestId,
idempotencyBase), threaded by buildTools from the same per-turn context the
LLM calls already use:

• X-Tenant-Id: ahu-chatbot, X-Priority: interactive (always).
• X-Surface = req.surface, X-User-Id = req.sessionId (same stable
  per-session actor as the chat/LLM calls).
• X-Request-Id = the turn's trace id (makeTurnTraceId) — so /search and
  the synthesis LLM event share one trace_id.
• Idempotency-Key = ${sessionId}:t${turn}:rag:${sha256(query)[:16]} —
  deterministic (built from the nonce-free base, not the trace id) so a
  retried retrieval of the same query within a turn dedupes.

Body ({query}) and response ({query, hits[]} → passages/references) are
unchanged — the gateway relays both verbatim.

Backpressure (CONVENTIONS §3)

The gateway may answer 429/503 + Retry-After when the RAG pool is saturated —
graceful backpressure, not a hard error. RagTool returns ok:false (the
orchestrator already proceeds without RAG context — evaluate.ts skips
status:"error"), but now flags retryable:true + retry_after_sec (additive
ToolResult fields) to distinguish "busy" from "broken", mirroring the LLM
client's Retry-After handling. 500/parse failures stay non-retryable.

Dormancy

With RAG_SEARCH_URL unset the call still resolves to ai-ahu-rag:8110; only
additive X-* headers differ, which the RAG service ignores → byte-identical
behavior. The env swap to the gateway is a separate operational cutover.

Browser-direct path (item 4 — report, don't change)

The public Tanya chat does not call RAG from the browser: it uses
useAgentStream("doc") → OrchestratorProvider → same-origin /api/orchestrate
→ server-side RagTool. NEXT_PUBLIC_DOC_RAG_API_URL in public-web is used only
server-side in /api/threads/doc/* (thread history via ai-ahu-rag
/sessions) — a separate server-side DMZ→Internal path the gateway /search
facade does not cover. Flagged for Efran; not changed here.

Tests

Extend RagTool tests: /search carries the §2 header set + a deterministic
Idempotency-Key; identity-less call omits per-actor headers (dormant); 429
degrades (retryable + retry_after) without throwing; 503 retryable, 500 not.

Acceptance

• [ ] RagTool sends §2 headers with the turn's trace id + per-session user id;
  deterministic idempotency key.
• [ ] 429/503 degrade gracefully; pnpm check green; dormant when
  RAG_SEARCH_URL unset.