think
16px
820px
← all files

Gateway Integration Prep Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Make every model-egress path gateway-ready (headers, queue semantics, idempotency, env prep) while byte-for-byte dormant with no new env vars set.

Architecture: TS orchestrator gains a typed SSE event stream + structured HTTP errors in LlmClient, forwarded through compose to the existing browser SSE status/error channel. Python agents gain a light dash/gateway.py helper providing static identity headers and the gated knowledge-LLM client config; agents.py/knowledge.py consume it. Env files get commented flip blocks only.

Tech Stack: Next.js/TypeScript + vitest (public-web); FastAPI/Agno 2.4.7 + pytest (agents); pnpm workspace.

Global Constraints

  • Dormant: with no new env vars set, request URLs/bodies unchanged; only sanctioned new static headers (X-Tenant-Id, X-Surface, X-Priority, X-Request-Id, Idempotency-Key) may appear.
  • Do NOT change any live value in infra/env/*.env — commented additions only.
  • apps/internal-agent and apps/public-agent mirrored files stay byte-identical (diff must be empty).
  • Header values: X-Tenant-Id: ahu-chatbot; X-Priority: interactive (Python agents); model names unchanged (Qwen/Qwen3.6-35B-A3B-FP8, qwen3.5-397b-a17b, Qwen/Qwen3-Embedding-4B).
  • Queue status copy: Dalam antrean — posisi N (±M menit) (CONVENTIONS §3.5).
  • Idempotency keys deterministic — never random-per-attempt.
  • pnpm check green; agent pytest baseline: only tests/test_embedder.py (+ new tests) run DB-free locally; policy/middleware tests need Postgres (pre-existing, run in Docker at ship time).

Task 1: LlmClient — structured errors, Idempotency-Key, streamEvents

Files:
- Modify: apps/public-web/src/lib/orchestrator/llm/client.ts
- Test: apps/public-web/tests/orchestrator/llm-client.test.ts

Interfaces produced:
- class LlmHttpError extends Error { status: number; retryAfterSec?: number; queueDepth?: number }
- type LlmStreamEvent = { type: "delta"; text: string } | { type: "queue"; position: number; etaMs: number } | { type: "error"; status?: number; code?: string }
- interface LlmClientOpts { idempotencyBase?: string; queueEvents?: boolean } — 6th constructor param
- LlmClient.streamEvents(messages, extras?): AsyncGenerator<LlmStreamEvent>; stream() unchanged signature (delta-only wrapper)

  • [x] Step 1: Add failing tests to llm-client.test.ts (429 enrichment, idempotency sequence, absence without base, streamEvents queue/error/unknown frames, X-Queue-Events opt-in, stream delta-only)
  • [x] Step 2: pnpm --filter @ahu/public-web test → new tests FAIL
  • [x] Step 3: Implement client.ts changes
  • [x] Step 4: Tests pass
  • [x] Step 5: Commit

Task 2: compose — queue/error pass-through

Files:
- Modify: apps/public-web/src/lib/orchestrator/steps/compose.ts
- Test: apps/public-web/tests/orchestrator/steps.test.ts

Interfaces produced:
- ComposeEvent gains { kind: "queue"; position: number; etaMs: number }
- class ComposeUpstreamError extends Error { status?: number; code?: string }
- compose consumes llm.streamEvents() (existing compose tests' mocks switch from stream to streamEvents)

  • [x] Steps: failing tests (queue pass-through, error frame throws) → red → implement → green → commit

Task 3: orchestrate — X-Request-Id, idempotency bases, queue status, busy degradation

Files:
- Modify: apps/public-web/src/lib/orchestrator/orchestrate.ts
- Test: apps/public-web/tests/orchestrator/orchestrate.test.ts

Changes: baseHeaders + "X-Request-Id": \${sessionId}-t${history.length}`; planning/synthesis clients getidempotencyBase(${sessionId}:t${turn}:plan|syn) and synthesis getsqueueEvents: process.env.LLM_QUEUE_EVENTS === "on"; compose loop forwardskind:"queue"→emitStatus("Dalam antrean — posisi N (±M menit)"); catch block:LlmHttpError429/503 andComposeUpstreamError429/503 →emitError("Sistem sedang sibuk...")(+ Retry-After seconds when known). Test mocks: llm/client + steps/compose factories switch toimportOriginalspread so realLlmHttpError/ComposeUpstreamError` classes survive.

  • [x] Steps: failing tests (queue→status frame, 429→friendly error) → red → implement → green → commit

Task 4: Python dash/gateway.py + tests (both agents, mirrored)

Files:
- Create: apps/internal-agent/dash/gateway.py + byte-identical apps/public-agent/dash/gateway.py
- Test: apps/internal-agent/tests/test_gateway.py + mirror (file-path loader pattern from test_embedder.py — do NOT import dash)

Interfaces produced:
- gateway_default_headers(priority: str = "interactive") -> dict[str, str] — X-Tenant-Id/X-Priority always; X-Surface only when SURFACE env set
- knowledge_llm_config() -> tuple[dict, str] — ({}, "gpt-4o") when KNOWLEDGE_LLM_MODEL unset (dormant); kwargs with base_url=MODEL_GATEWAY_URL+"/v1", api_key=PLANNING_API_KEY|EMPTY, default_headers=... when model+gateway set

  • [x] Steps: failing tests → red → implement → green in both apps → mirror diff empty → commit

Task 5: Wire agents.py + knowledge.py (both agents, mirrored)

Files:
- Modify: apps/{internal,public}-agent/dash/agents.py — default_headers=gateway_default_headers("interactive") on the three OpenAILike(...) sites (_build_model_from_slot, _resolve_model, _resolve_synthesis_model); one # TODO(gateway) comment re: no per-run X-User-Id in agno 2.4.7
- Modify: apps/{internal,public}-agent/app/api/knowledge.py — client_kwargs, model = knowledge_llm_config(); client = OpenAI(**client_kwargs); model=model in the create call

  • [x] Steps: implement both apps → mirror diff empty → python -c "import ast" syntax check (heavy import unavailable locally) → commit

Task 6: Env flip blocks (live + example files)

Files:
- Modify: infra/env/internal.env, infra/env/public.env — append commented block (gateway URLs, EMBEDDER_BASE_URL, LLM_QUEUE_EVENTS, KNOWLEDGE_LLM_MODEL, SYNTHESIS_API_KEY obsolescence note)
- Modify: infra/env/shared.env — same block minus KNOWLEDGE_LLM_MODEL/SYNTHESIS note
- Modify: the three committed *.env.example files with the same blocks (repo mandate)

  • [x] Steps: append blocks → verify no live line changed (git diff on examples; manual diff on ignored files shows additions only) → commit

Task 7: Verification + completion notes

  • [x] pnpm check (typecheck + all suites + conventions) green
  • [x] pytest tests/test_gateway.py tests/test_embedder.py green in both agent apps
  • [x] Mirror diff empty for dash/gateway.py, dash/agents.py, app/api/knowledge.py
  • [x] Dormancy: unit tests assert no Idempotency-Key/X-Queue-Events without opts; knowledge_llm_config() returns ({}, "gpt-4o") with staging-shaped env (MODEL_GATEWAY_URL set, KNOWLEDGE_LLM_MODEL unset)
  • [x] Optional live check: chat turn via http://192.168.83.20:8200 with contract headers; /metrics shows gateway_requests_total{tenant="ahu-chatbot"}
  • [x] Completion notes in docs/superpowers/; upload session .md files