AHU AI Platform Conventions — v1.0

This file is the canonical contract between the platform (ahu-gpu-manager, ahu-ai-observatory) and every engine (chatbot, Document Intelligence/OCR, doc classifier, and all future engines). Engines and platform services MUST conform. Changes happen by PR to this file with a version bump; the audit ingester supports schema version N and N−1 so upgrades never require lockstep deploys.

Copies/summaries elsewhere (workspace CLAUDE.md, integration prompts) are pointers — on conflict, this file wins.

1. Network & port allocations

Audit transport: Redis Stream, key ahu.ai.audit.

2. HTTP header contract (engine → gateway)

3. Queue & timeout semantics

1. Interactive requests are never held past the client's timeout. Hold budget per class = known client timeout − 15 s (minimum 5 s). Engines declare their client timeout when registering a tenant; changing an engine timeout means updating the tenant config.
2. Past hold budget → 429 (pool saturated) or 503 (upstream down) + Retry-After (seconds) + X-Queue-Depth. Engines MUST treat these as graceful degradation, not errors to retry-storm.
3. Batch: a queued job is not a slow job. Poll statuses are queued (with queue_position, eta_ms) → processing (with stage, progress) → completed | failed. Engine poll clients run their processing-timeout clock only during processing; total queue wait is bounded separately by GPU_QUEUE_WAIT_MS (default 1 800 000 = 30 min).
4. A poll for an unknown job returns 404 {code: "JOB_UNKNOWN"} → the engine resubmits with the same Idempotency-Key (the gateway dedupes; a completed job returns its existing result).
5. Queue state must reach the human. UIs render queued as "Dalam antrean — posisi N (±X menit)", never a generic spinner, whenever position/ETA is available.
6. A 4xx is a permanent verdict on the input; never retry it. Only 408/429/503 + Retry-After and 5xx are retryable. An engine that retries a 4xx burns capacity on an input that can never succeed — 8 corrupt documents retried ~5 800 times over five weeks once produced 98 % of all platform error volume, hiding every real incident underneath. Record the failure with the upstream's own reason and surface it to the user ("PDF rusak / tidak dapat dibaca"); do not translate a 4xx into a 5xx, which reads as transient and invites the retry.
7. The gateway remembers permanent refusals (refusal memory). When an upstream refuses a document with a status that is a verdict on the bytes (default 400, 413, 415, 422), the gateway records that refusal keyed on the document's content fingerprint — not on the caller — and replays the upstream's original status and body, byte-identical, to every later caller. Engines see no new status code and need no change: a refusal looks exactly like the upstream answering. Three consequences engines should know:
   - A replayed refusal carries X-Negative-Cache: HIT (plus X-Negative-Cache-Hits and X-Negative-Cache-First-At) and no Retry-After — retrying is precisely what cannot help.
   - It is audited as error_code = REFUSED_HTTP_nnn with cache_hit=true and upstream_ms=0, deliberately distinct from the live HTTP_nnn, so a poison pill can be separated from a real incident in every rate-based signal.
   - 401/403/404/408/429 are never remembered (they report a caller, credential, route, or backpressure problem, not a bad document), and neither is any 5xx. The gateway refuses such a config at load rather than accepting it.

This does not relieve an engine of rule 6 — it makes the property hold fleet-wide, including for instances still on older code, which per-process engine guards cannot do.

4. Egress rules

1. Once an engine's gateway env is flipped, no engine calls a model/OCR server directly — all LLM, VLM, embedding, and OCR traffic goes through the gateway. New engines integrate via the gateway from day one.
2. Model-server URLs appear only in env vars, never hardcoded. Gateway base env is MODEL_GATEWAY_URL (no /v1 suffix; clients append it).
3. Upstream API keys/secrets live only in the gateway registry, never in engine repos or env files.
4. Upstreams are classed on_prem | external_dev. Production runs allow_external_upstreams=false; cloud endpoints (DashScope, OpenAI) are development-phase conveniences that the gateway refuses in prod.
5. In-process GPU tenants (anything loading models into local CUDA, e.g. the classifier) can't be proxied — they MUST use the SDK pattern instead: heartbeat to MANAGER_URL, audit events to ahu.ai.audit, fire-and-forget, dormant when env unset.

5. Audit event schema (v1)

One event per AI call, emitted by the gateway (proxied traffic) or the engine SDK (in-process tenants). Fire-and-forget; on stream failure, spool to local disk and replay — never block or fail the user request because auditing is down.

event_id (ULID/uuid4) · schema_ver=1 · ts (ISO-8601 UTC) · engine · surface · user_id · trace_id
upstream · model · upstream_class (on_prem|external_dev) · traffic_class
operation (chat|embed|ocr|classify|search|job|job_child) · status · error_code
queue_ms · upstream_ms · total_ms · tokens_in · tokens_out · doc_hash · pages
job_id · parent_job_id · config_version · request_body · response_body (capped 10 MB, zstd; truncation recorded with full-content hash)

Additive fields = no version bump. Renames/removals/semantic changes = schema_ver+1.

6. Naming

• Registry model ids: kebab-case — qwen-397b, qwen-35b, cleanup-3b, tei-embeddings, paddleocr, azure-di-layout-v4, …
• Compose projects: ahu-platform (services), ahu-platform-agent (GPU hosts).
• Env vars: gateway-related config in engines uses existing names where they exist (MODEL_GATEWAY_URL, VLLM_BASE_URL, …); new platform-owned vars are prefixed GATEWAY_, AUDIT_, or MANAGER_.

7. Stack conventions

• Platform services: Go (static binaries). Dashboard: React + Vite + Tailwind/shadcn. Engines keep their own stacks; only the wire contract is mandatory.
• All platform services expose Prometheus-format /metrics (costs nothing, matches the DCGM exporter and the dormant Prometheus/Grafana configs already on x056); the custom dashboard remains the product deliverable.
• Every platform service: /healthz liveness endpoint; graceful drain on SIGTERM (finish in-flight, 503 + Retry-After for new work, never silent connection drops).

8. Engine compliance checklist

Every engine, before its gateway flip is approved:

• [ ] Sends the §2 header set on all model/OCR calls
• [ ] Handles 429/503 + Retry-After as graceful degradation
• [ ] Treats a 4xx as permanent — records the upstream's reason, surfaces it, does not retry (§3.6)
• [ ] Job polling distinguishes queued vs processing (clock rule §3.3) and handles JOB_UNKNOWN
• [ ] Idempotency keys are deterministic (§2)
• [ ] Queue state is user-visible (§3.5)
• [ ] No hardcoded model-server URLs; no upstream secrets in repo
• [ ] All changes dormant until env flip (unset env = unchanged behavior)