Route public RAG /search through the gateway — completion notes

Date: 2026-07-08
Spec: specs/2026-07-08-rag-search-gateway-design.md
Plan: plans/2026-07-08-rag-search-gateway.md
Status: implemented, gated green, deployed dormant + dormancy-verified
(commit cf49dbb). The gateway cutover (RAG_SEARCH_URL env) is a separate
operational flip — not done, awaiting Efran.

What changed

• packages/orchestrator-types/tools.ts — additive retryable? +
  retry_after_sec? on ToolResult (backpressure signal).
• tools/rag.ts — RagTool takes an optional RagIdentity; /search now
  carries the CONVENTIONS §2 header set + a deterministic Idempotency-Key;
  429/503 degrade gracefully.
• orchestrate.ts — buildTools builds RagTool from
  env("RAG_SEARCH_URL", env("RAG_BASE_URL")) and threads the turn identity;
  runOrchestrator passes the nonce-free idempotency base
  ${sessionId}:t${turn}:rag.
• infra/env/public.env.example — documents RAG_SEARCH_URL.

Report back (per the prompt)

• Seam chosen: RAG_SEARCH_URL (falls back to RAG_BASE_URL), not an
  overload of RAG_BASE_URL. Reason: RAG_BASE_URL is also read by the
  knowledge-reingest worker (/admin/ingest) and the admin search proxy —
  neither of which the gateway's /search-only facade can serve. A scoped var
  flips just the public retrieval path.
• Gateway-headers helper reused: the same per-turn identity the LLM path
  established — X-Tenant-Id: ahu-chatbot, X-Surface = req.surface,
  X-User-Id = req.sessionId (the stable per-session id), X-Request-Id = makeTurnTraceId(req) (the turn's trace), X-Priority: interactive. Threaded
  through buildTools as a RagIdentity object (no second identity pattern).
• How the trace id + user id reached RagTool: runOrchestrator mints the
  turn trace id once and passes it (+ req.sessionId + the idempotency base) to
  buildTools, which constructs RagTool with them — so /search shares the
  turn's trace_id and actor with the synthesis LLM call.
• Idempotency-Key: ${sessionId}:t${turn}:rag:${sha256(query)[:16]} —
  deterministic, built from the nonce-free base (not the trace id), so a retried
  retrieval of the same query within a turn dedupes at the gateway.
• 429/503 degradation: RagTool returns ok:false, status:"error" (so the
  orchestrator proceeds without RAG context — evaluate.ts already skips
  status:"error", non-fatal) plus retryable:true + retry_after_sec (from
  Retry-After). 500/parse failures stay non-retryable. Mirrors the LLM
  client's Retry-After handling — no new degradation flow invented.
• Browser-direct (NEXT_PUBLIC_DOC_RAG_API_URL) finding: the public Tanya
  page does not call RAG from the browser — useAgentStream("doc") →
  OrchestratorProvider → same-origin /api/orchestrate → server-side
  RagTool. In public-web, NEXT_PUBLIC_DOC_RAG_API_URL is used only
  server-side, in /api/threads/doc/* (thread history via ai-ahu-rag
  /sessions). That is a separate server-side DMZ→Internal path the gateway's
  /search facade does not cover — its own cross-zone question (needs a gateway
  facade for /sessions or a scoped path). Flagged; not changed. No
  browser-direct exposure to fix.

Dormancy — verified on staging

Deployed ahu-ai-chatbot-public:cf49dbb (only image affected; ahu-public
recreated, agent-public untouched). Left RAG_SEARCH_URL unset → /search
still resolves to ai-ahu-rag:8110. A real public query
("Apa syarat mendirikan PT perorangan?") streamed a full grounded answer with a
references event — retrieval works byte-identically; the additive X-*
headers are ignored by the RAG service. Container healthy, 0 restarts.

Tests

tests/orchestrator/tools.test.ts (+5): §2 header set on /search (trace id =
turn's, per-session user id) + body verbatim; deterministic Idempotency-Key
(same query → same key, distinct across queries); identity-less call omits
per-actor headers (dormant); 429 degrades to retryable+retry_after_sec
without throwing; 503 retryable, 500 not. Gate: pnpm check green
(public-web 133 / internal-web 65 / streams 17 + typecheck + conventions).

Operational flip (Efran) + post-flip verification

Set RAG_SEARCH_URL=http://192.168.83.20:8200 in infra/env/public.env and
recreate ahu-public. Then run one public Tanya query and confirm in the
observatory an operation=search, engine=ahu-chatbot, surface=public event
sharing the same trace_id as that turn's synthesis LLM call — retrieval
and answer stitched into one trace. (This is a cross-zone routing change — left
to Efran's go-ahead.)