Integration prompt — `ai-ahu-chatbot`

Paste this into a Claude Code session running inside the ai-ahu-chatbot repo. Fill in <GATEWAY_URL> before pasting (e.g. http://192.168.83.20:8200), or leave it and only the commented env entries will reference it.

We are introducing a central GPU/LLM gateway (ahu-gpu-manager) that all model traffic must flow through. It is OpenAI-compatible, adds priority queueing, and emits audit events. Your job is to prepare this repo for it. Hard constraint: every change must be backwards-compatible and dormant — with no new env vars set, behavior must be byte-for-byte identical to today. Do not flip any live env values in infra/env/*.env; only add commented-out entries showing the future values.

1. Header propagation in the Agno agents

In apps/internal-agent/dash/agents.py and its mirror apps/public-agent/dash/agents.py, the models built by _resolve_model() / _build_model_from_slot() (via OpenAILike) currently send no identifying headers — only the orchestrator's direct calls do. Add default headers to the model construction:

X-Tenant-Id: ahu-chatbot
X-Surface: from the existing SURFACE env (internal | public)
X-Priority: planning for the tool-loop model, synthesis for the synthesis model (mirroring what apps/public-web/src/lib/orchestrator/orchestrate.ts already sends)

If Agno supports per-run extra headers, also propagate the per-run user_id as X-User-Id; if it only supports client-level headers, add the static ones and leave a # TODO(gateway) noting the per-run limitation. Verify against the installed agno==2.4.7 API — do not guess.

2. Close the bare-OpenAI egress

apps/internal-agent/app/api/knowledge.py:421 (and the public-agent mirror if present) constructs a bare OpenAI() client calling model="gpt-4o" — this bypasses MODEL_GATEWAY_URL and hits api.openai.com directly. Change it to construct the client from the same gateway-resolution logic the agents use (base_url from MODEL_GATEWAY_URL + /v1 when set, model name from a new env KNOWLEDGE_LLM_MODEL defaulting to the current gpt-4o). When MODEL_GATEWAY_URL is unset, behavior must be unchanged.

3. Queue-awareness in the orchestrator LLM client

In apps/public-web/src/lib/orchestrator/llm/client.ts:

On 429 or 503 responses, read Retry-After and X-Queue-Depth headers and include them in the thrown error object so callers can degrade with a friendly "sistem sedang sibuk" message instead of a generic failure.
Add opt-in structured queue events: when a new env LLM_QUEUE_EVENTS=on is set, send header X-Queue-Events: on on streaming requests and parse SSE frames of the form event: queue / data: {"position":N,"eta_ms":M} that may arrive before the first chat chunk. Surface them via a callback so orchestrate.ts can forward a status event to the browser stream (reuse the existing SSE status/error event channel) — the user should see "Antrian ke-N (±Xs)" rather than a spinner. Unknown SSE event types must be ignored gracefully (today's vLLM sends none).
Send an Idempotency-Key header on each request, derived from the existing per-request/session identifiers.

4. Env preparation (commented, not active)

In infra/env/internal.env, infra/env/public.env, infra/env/shared.env, add commented blocks:

# --- ahu-gpu-manager (do not enable until gateway is live) ---
# MODEL_GATEWAY_URL=<GATEWAY_URL>
# SYNTHESIS_GATEWAY_URL=<GATEWAY_URL>/synthesis
# EMBEDDER_BASE_URL=<GATEWAY_URL>/v1
# LLM_QUEUE_EVENTS=on

Note in a comment that once the gateway is live, the DashScope SYNTHESIS_API_KEY moves server-side into the gateway registry and gets removed from this repo.

5. Verification

Run the existing test suites; nothing may regress.
With no new envs set, confirm (by reading the constructed clients or a local smoke run) that request URLs, headers minus the new static X- headers, and bodies are unchanged against the current vLLM endpoint.
Type-check and lint clean.