Gateway-Proxied RAG-Search Route — Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add a gateway route that proxies the Document-RAG POST /search endpoint so the public chatbot never calls ai-ahu-rag directly (it crosses the DMZ→Internal zone boundary after the production split; migration-plan decision #1 chose the Gateway-proxy over a firewall exception).

Architecture: RAG search is a synchronous JSON POST — exactly the shape the gateway's existing generalized sync-facade adapter already forwards verbatim (the same mechanism backing the PaddleOCR/doc-classifier native façades: EnableNativeFacades → jobs.Facade.SyncForward, which already does tenant resolution, on-prem enforcement, priority class, pool admission, GPU-shed, verbatim relay, and audit emission). The ONLY gap: the sync-facade adapter is gated on upstream type ∈ {ocr-http, classify}, and the audit operation allowlist is {ocr, classify}. RAG is neither. So this adds one new upstream type rag-http with audit operation search, and wires a config block. No new handler, no new façade, no change to the async job machinery, no engine-repo change (the chatbot flip is a separate hands-off integration prompt).

Tech Stack: Go 1.25 gateway (ahu-gpu-manager), YAML config, go test.

Global Constraints (from CONVENTIONS.md v1.0 — every task inherits these)

All model/OCR/search egress goes through the gateway once an engine is flipped; engine changes are dormant until env flip — with the RAG upstream unconfigured, the gateway is byte-identical to today (the new type only activates when a type: rag-http upstream is present in config).
Audit operation must come from the upstream's configured operation value, never hardcoded (CONVENTIONS §4). RAG emits operation=search.
On-prem enforcement (CONVENTIONS §5): RAG is class: on_prem; production refuses external_dev. The existing checkOnPrem in SyncForward already enforces this — no new code.
Headers on every AI call (CONVENTIONS §2): X-Tenant-Id, X-Surface, X-User-Id, X-Request-Id (→ audit trace_id), X-Priority, Idempotency-Key. SyncForward/emitSync already read these — no new code; the chatbot prompt (Task 3) is what makes the client send them.
Do NOT add rag-http to registry.IsJobUpstream — RAG is a native sync-facade, not an async job upstream; making it a job upstream would wrongly trigger job-store/manager wiring in cmd/gateway/main.go.
Reserved-path collision: /search must not collide with reserved façade paths (/v1, /jobs, /metrics, /health, gpu-server paths) — the existing facadePathCollision validation enforces this; a test asserts /search passes.

Task 1: New `rag-http` upstream type — config validation + audit operation

Files:
- Modify: internal/audit/event.go (add OperationSearch constant)
- Modify: internal/config/config.go (validation gate ~line 278; operation default/allowlist ~lines 348-357)
- Test: internal/config/config_test.go (add cases)

Interfaces:
- Consumes: existing Upstream struct (Type, Adapter, SyncPaths, OperationValue), existing sync-facade validation branch, existing reservedFacadePaths/facadePathCollision.
- Produces: audit.OperationSearch = "search"; a type: rag-http + adapter: sync-facade upstream now validates, defaults operation to search, and mounts its sync_paths exactly as an ocr-http sync-facade does.

[ ] Step 1: Add the audit constant. In internal/audit/event.go, beside OperationOCR/OperationClassify, add:

    OperationSearch   = "search"

[ ] Step 2: Write failing config tests. In internal/config/config_test.go, add tests (adapt to the file's existing test style — most likely a Load-from-bytes or a struct-built validate helper; match whatever the neighbouring sync-facade tests use):
rag-http + adapter: sync-facade + one sync_paths entry {path: /search, model: rag-search} + operation unset → loads OK, and the upstream's OperationValue == "search" (default applied).
Same but with explicit operation: search → loads OK.
rag-http sync-facade with no sync_paths → error mentioning sync_paths.
rag-http sync-facade with sync_paths: [{path: /v1, model: x}] → error mentioning collision with reserved /v1.
A rag-http upstream with operation: ocr explicitly set → still loads OK (any of the three allowed operations is legal on any of these types; do not couple operation to type beyond the default).
Guard the existing behavior: an ocr-http sync-facade still defaults operation to ocr (unchanged).
[ ] Step 3: Run tests to verify they fail.
Run: go test ./internal/config/ -run RAG -v
Expected: FAIL (rag-http rejected as unknown type / operation search rejected).
[ ] Step 4: Widen the validation gate. In internal/config/config.go, change the job/adapter gate (currently if u.Type == "ocr-http" || u.Type == "classify" {) to also admit rag-http:

        if u.Type == "ocr-http" || u.Type == "classify" || u.Type == "rag-http" {

Leave the switch u.Adapter body unchanged — rag-http uses case "sync-facade", which already validates sync_paths, models, collisions, and cache TTL.

[ ] Step 5: Default + allow the search operation. In the same block (the if u.OperationValue == "" defaulting + the allowlist check ~lines 348-357), set the default for rag-http and widen the allowlist. Replace:

            if u.OperationValue == "" {
                if u.Type == "classify" {
                    u.OperationValue = "classify"
                } else {
                    u.OperationValue = "ocr"
                }
            }
            if u.OperationValue != "ocr" && u.OperationValue != "classify" {
                return nil, fmt.Errorf("upstream %s: invalid operation %q (want ocr|classify)", u.ID, u.OperationValue)
            }

with:

            if u.OperationValue == "" {
                switch u.Type {
                case "classify":
                    u.OperationValue = "classify"
                case "rag-http":
                    u.OperationValue = "search"
                default:
                    u.OperationValue = "ocr"
                }
            }
            if u.OperationValue != "ocr" && u.OperationValue != "classify" && u.OperationValue != "search" {
                return nil, fmt.Errorf("upstream %s: invalid operation %q (want ocr|classify|search)", u.ID, u.OperationValue)
            }

[ ] Step 6: Run tests to verify they pass.
Run: go test ./internal/config/ -v && go test ./...
Expected: PASS (new RAG cases green; full suite green — no regression).
[ ] Step 7: Commit.

git add internal/audit/event.go internal/config/config.go internal/config/config_test.go
git commit -m "feat(rag): rag-http upstream type + search audit operation for the sync-facade RAG proxy"

Task 2: End-to-end RAG sync-facade path test + documented config block

Files:
- Test: internal/server/facade_smoke_test.go (or a new internal/server/rag_facade_test.go — match the existing server-test harness that boots a Server with a stub upstream, e.g. facade_smoke_test.go)
- Modify: deploy/gateway.example.yaml (add the RAG upstream block, commented-in with deploy notes)

Interfaces:
- Consumes: server.New + EnableNativeFacades (already wired in cmd/gateway/main.go), jobs.Facade.SyncForward, httptest stub backend.
- Produces: proof that POST /search on a rag-http sync-facade upstream forwards the body verbatim, relays the upstream status+body, and emits an audit event with operation=search, model=<sync_paths.model>, engine from X-Tenant-Id.

[ ] Step 1: Write the failing integration test. Model it on the existing sync-facade server smoke test. Stand up an httptest.Server as the stub RAG backend: POST /search echoes a canned {"query":"x","hits":[...]} (200) and asserts it received the verbatim request body; GET /health returns 200 (so the prober keeps the endpoint healthy — or bypass the prober as the existing smoke test does). Build a config.Config with one upstream {id: ai-ahu-rag, type: rag-http, class: on_prem, adapter: sync-facade, endpoints: [stub.URL], models: [rag-search], probe_path: /health, sync_paths: [{path: /search, model: rag-search}], slots: {total: 4}}. Boot server.New(...) with a capturing audit emitter (reuse the test emitter the other façade tests use), call EnableNativeFacades(), then POST /search with headers X-Tenant-Id: ahu-chatbot, X-Surface: public, X-Priority: interactive. Assert: response 200 + body equals the stub's; the captured audit event has Operation == "search", Model == "rag-search", Engine == "ahu-chatbot", Upstream == "ai-ahu-rag", Status == "ok".
[ ] Step 2: Run it to verify it fails.
Run: go test ./internal/server/ -run RAG -v
Expected: FAIL initially only if a wiring gap exists; if it passes immediately, that CONFIRMS Task 1 fully enabled the path (acceptable — note it in the report). Do NOT weaken the assertions to force a red; the value here is the end-to-end guarantee.
[ ] Step 3: (If needed) fix any wiring gap surfaced. No new production code is expected — if the test fails, diagnose whether it's a test-harness issue vs. a real gap and fix the smaller/correct one.
[ ] Step 4: Run the full suite.
Run: go test ./...
Expected: PASS.
[ ] Step 5: Add the documented config block. In deploy/gateway.example.yaml, after the doc-classifier block, add (commented-in style consistent with the file — a real block, not commented out, since this is an intended production upstream; but keep the deploy-caveat comments):

  # ---- Document-RAG search proxy (RAG-P1) ----
  # So the public chatbot never calls ai-ahu-rag directly across the DMZ→Internal
  # boundary post-split (migration decision #1). sync-facade forwards POST /search
  # verbatim to the RAG service and audits it as operation=search. The engine flip
  # is a single base-URL swap (RAG search URL -> this gateway) — see
  # docs/integration/prompt-ai-ahu-chatbot-rag-proxy.md. Dormant until configured:
  # with this upstream absent the gateway is byte-identical to before.
  - id: ai-ahu-rag
    type: rag-http
    class: on_prem
    adapter: sync-facade
    operation: search
    endpoints: ["http://192.168.83.20:8110"]   # ai-ahu-rag, host-published :8110 (verify reachability from the gateway container at deploy)
    probe_path: /health
    models: ["rag-search"]        # registry key; sync_paths[].model is the audit label
    sync_paths:
      - {path: /search, model: rag-search}   # cache_ttl_s omitted -> 0 -> no caching (retrieval must stay fresh as the corpus changes)
    slots: {total: 8, batch_max: 8}

[ ] Step 6: Verify the example config parses (guards YAML typos against the real loader):
Run: go test ./internal/config/ -run Example (if such a test exists) — otherwise add a one-line test that Loads deploy/gateway.example.yaml and asserts no error, if one isn't already present.
Expected: PASS.
[ ] Step 7: Commit.

git add internal/server/ deploy/gateway.example.yaml
git commit -m "test(rag): end-to-end sync-facade /search forward+audit; document RAG upstream block"

Task 3: Chatbot integration prompt (hands-off — no engine-repo edits)

Files:
- Create: docs/integration/prompt-ai-ahu-chatbot-rag-proxy.md

Interfaces:
- Consumes: the live /search gateway route from Tasks 1-2; the chatbot's existing RAG client + the gateway-headers helper pattern already established by prompt-ai-ahu-chatbot-identity.md.
- Produces: a prompt a Claude Code session inside ai-ahu-chatbot can execute to flip the public RAG-search client to the gateway.

[ ] Step 1: Write the prompt. It must: (a) explain WHY (DMZ→Internal boundary; the gateway is the sanctioned cross-zone path); (b) identify the public-agent's RAG-search client (whatever calls ai-ahu-rag's POST /search today — the session must locate it) and introduce a base-URL seam so a single env (e.g. RAG_SEARCH_URL / reuse MODEL_GATEWAY_URL + path) flips it to <gateway>/search; (c) require the CONVENTIONS §2 headers via the existing gateway-headers helper — X-Tenant-Id: ahu-chatbot, X-Surface: public, X-User-Id (the same stable per-session id from the identity prompt), X-Request-Id (shared with the chat turn's trace so retrieval and synthesis share one trace_id), X-Priority: interactive, deterministic Idempotency-Key; (d) state the request/response shape is unchanged (gateway forwards /search verbatim — same SearchRequest/SearchResponse); (e) be dormant-safe (env unset → calls RAG directly as today); (f) give a post-flip check: run one public query, confirm in the observatory that an event with operation=search, engine=ahu-chatbot, surface=public appears sharing the chat turn's trace_id. Match the tone/structure of the existing docs/integration/prompt-ai-ahu-chatbot-identity.md.
[ ] Step 2: Commit.

git add docs/integration/prompt-ai-ahu-chatbot-rag-proxy.md
git commit -m "docs(integration): chatbot RAG-search proxy flip prompt"

Post-execution (controller, after all tasks review-clean)