think
16px
820px
← all files

Integration prompt — ai-ahu-chatbot: route public RAG-search through the Gateway

Paste into a Claude Code session inside the ai-ahu-chatbot repo. Follow-up to the gateway-integration + identity prompts (prompt-ai-ahu-chatbot.md, -audit-hardening.md, -identity.md). The gateway side is DONE + deployed: the gateway now exposes POST /search as a verbatim sync-facade proxy to ai-ahu-document-rag (upstream ai-ahu-rag, audited as operation=search). This prompt flips the public chatbot to call the gateway instead of the RAG service directly. No gateway change is required.

Why (read first)

Today the public orchestrator calls the Document-RAG service directly: RagTool (in apps/public-web/src/lib/orchestrator/tools/rag.ts) does POST ${RAG_BASE_URL}/search, with RAG_BASE_URL=http://ai-ahu-rag:8110. After the production split, public-web runs in the DMZ zone and ai-ahu-rag runs in the Internal zone — so that call crosses the DMZ→Internal boundary. The platform decision (migration plan #1) was to route it through the Gateway (the sanctioned cross-zone data path) rather than open a firewall exception, because a direct exception would expose an application data path into the Internal engines zone.

The gateway forwards POST /search verbatim — same SearchRequest body, same {query, hits[]} response — so this is a base-URL swap plus the standard gateway headers, not a client rewrite. It's dormant-safe: with the env still pointing at the RAG service, the added X-* headers are ignored by RAG and behavior is byte-identical.

Scope

1. Base-URL seam (the flip itself)

RagTool already reads its base URL from RAG_BASE_URL via env("RAG_BASE_URL") in apps/public-web/src/lib/orchestrator/orchestrate.ts (new RagTool(env("RAG_BASE_URL"), ...)). Because RagTool appends /search to the base and the gateway mounts POST /search at its root, flipping is purely an env change — point RAG_BASE_URL at the gateway base (http://<gateway-host>:8200) and ${baseUrl}/search resolves to <gateway>/search. Confirm no code assumes the ai-ahu-rag host/port anywhere else on this path. Keep the env pointed at ai-ahu-rag:8110 until the cutover — the header work below is what you land now; the env swap is the operational flip.

  • If you prefer an explicit, self-documenting seam over overloading RAG_BASE_URL, introduce RAG_SEARCH_URL (falls back to RAG_BASE_URL when unset) — but do not require a second new var if the single base-URL swap is cleaner. Pick one and document it.

2. Send the CONVENTIONS §2 headers on the /search call

RagTool.call() currently sends only Content-Type: application/json. The gateway needs the standard header set so the retrieval is audited and attributed (and so it shares one trace with the chat turn it serves). Reuse whatever gateway-headers assembly the LLM path already uses in this app (there is an established helper/pattern from the identity work — find it; do not invent a second one). On the /search fetch, add:

  • X-Tenant-Id: ahu-chatbot
  • X-Surface: public
  • X-User-Id — the same stable per-session id the identity work threads onto the chat/LLM calls (retrieval is part of the same user's request footprint; it must attribute to the same actor). Never PII.
  • X-Request-Id — the same trace id as the chat turn that triggered this retrieval, so the observatory shows the /search event and the synthesis LLM event sharing one trace_id. Thread it from the orchestrator's per-turn context (the orchestrator already has a request/turn context — reuse it; don't mint a fresh id here).
  • X-Priority: interactive — a user is waiting on the retrieval.
  • Idempotency-Key — deterministic for this retrieval (e.g. derived from the turn/trace id + a stable hash of the query), never random-per-attempt (CONVENTIONS §2). A retried retrieval within a turn must carry the same key.

The request body ({ query }) and the response handling ({ query, hits[] } → passages/references) are unchanged — the gateway relays both verbatim.

3. Graceful degradation (queue semantics — CONVENTIONS §3)

The gateway may answer 429/503 with Retry-After when the RAG upstream's pool is saturated — this is graceful backpressure, not a hard error. RagTool.call() currently maps any non-r.ok to { ok:false, status:"error" }. At minimum, keep that safe (the orchestrator already treats a failed RAG tool as "no context"), but prefer: on 429/503, surface a distinct retryable signal (the orchestrator can proceed with a "sistem sedang sibuk"/low-confidence path rather than a hard failure). Match the degradation pattern the LLM client already uses for 429/503 in this repo — don't design a new one.

4. Browser-direct path — flag, don't silently leave

NEXT_PUBLIC_DOC_RAG_API_URL (and NEXT_PUBLIC_AGNO_API_URL) point the browser at 192.168.83.20:8110 directly — a separate exposure from the server-side orchestrator this prompt fixes. That path can't simply adopt server-side gateway headers (it's client-side, and NEXT_PUBLIC_* is baked into the bundle). Do not change it blind — but report whether the public Tanya page still calls RAG directly from the browser, because post-split that is its own DMZ-entry question (it would need to enter via the DMZ's public ingress, not reach Internal directly). Surface it; leave the decision to Efran.

Dormancy + verification

  • Dormancy proof: with RAG_BASE_URL still http://ai-ahu-rag:8110, confirm the /search request URL/body are unchanged and only the additive X-* headers differ — the RAG service ignores them, so behavior is byte-identical.
  • Tests: extend RagTool tests to assert the /search call carries X-Tenant-Id: ahu-chatbot, X-Surface: public, X-Priority: interactive, a stable X-User-Id, an X-Request-Id equal to the turn's trace, and a deterministic Idempotency-Key; assert a 429 response degrades rather than throwing.
  • Post-flip check (after the env swap): run one public Tanya query, then in the observatory confirm a operation=search, engine=ahu-chatbot, surface=public event appears, sharing the same trace_id as that turn's synthesis LLM call — retrieval and answer visibly stitched into one trace.

Report back

Which seam you chose (env-only RAG_BASE_URL swap vs. new RAG_SEARCH_URL); the gateway-headers helper reused; how the turn's trace id + per-session user id reached RagTool; the 429/503 degradation behavior landed; and the browser-direct (NEXT_PUBLIC_DOC_RAG_API_URL) finding.