AHU AI Platform — Engineering Handoff
Prepared: 2026-07-07 · Scope: chatbot, OCR (legacy + tidy), gateway, observatory, classifier
Canonical contract: ahu-gpu-manager/docs/CONVENTIONS.md (v1.0) — on any conflict, that file wins.
The mental model (read this first)
The platform is a few engines (chatbot, OCR, classifier) that all send their model/OCR traffic through one Gateway. The Gateway holds the upstream keys, enforces on-prem-only, and writes an audit event for every call onto a Redis stream. The Observatory reads that stream into a database and shows it on a dashboard for governance.
One choke point (Gateway) for all AI calls → one audit trail (Observatory). Everything else is an engine that plugs into that seam.
Today everything runs on the single GPU box ai-ahu (192.168.83.20). The production target splits it into 5 trust zones — see ahu-gpu-manager/docs/deploy/2026-07-06-server-topology-and-specs.md.
Diagram 1 — Deployment topology (trust zones)
public RAG Q&A"] end subgraph intz["Internal engines"] intbot["Internal chatbot · 'Tanya Data'
+ Admin console"] rag["RAG service
(ai-ahu-rag)"] pgv[("pgvector
knowledge vectors")] agents["Data agents (Agno)
data-agent · public_dash"] ocr["OCR · akta notaris"] ocrdb[("PostgreSQL
extraction + review")] reg[("Registry systems
SABH · PP · Apostille")] end subgraph ctrl["Control plane"] gw["GATEWAY
single AI egress + audit"] redis[("Redis
leases + ahu.ai.audit stream")] end subgraph gpuz["GPU host · 2x H100 (on-prem)"] llm["LLMs · vLLM
qwen-35b · cleanup-3b"] emb["Embeddings · TEI"] ocreng["OCR engines
PaddleOCR · Azure DI"] clf["Classifier · LayoutLMv3"] end subgraph govz["Governance"] ing["Observatory ingester"] tsdb[("TimescaleDB
audit store · hash-chain")] api["Query API"] dash["Dashboard"] end ext["Alibaba Cloud · DashScope
Qwen 397B big model
TEMPORARY — dev only, refused in prod"]:::ext users --> pubbot staff --> intbot ocruser -->|RBAC gated| ocr pubbot --> rag pubbot --> agents intbot --> agents intbot -. manage KB .-> rag rag --> pgv ocr --> ocrdb ocr --> reg pubbot --> gw rag --> gw agents --> gw ocr --> gw gw --> llm gw --> emb gw --> ocreng gw --> clf gw -. big-model synthesis (temporary) .-> ext gw ==>|audit event / call| redis redis ==> ing ing --> tsdb api --> tsdb dash --> api classDef ext fill:#fbeaea,stroke:#c0392b,color:#7b241c classDef actor fill:#eaf0fb,stroke:#334455
Solid arrow = call · dotted = admin/dormant/temporary · thick ==> = audit pipeline.
Engine→backend model calls are drawn through the Gateway (the intended state); a few engine flips are still env-gated (see "What's left"). The big synthesis model currently runs off-prem on Alibaba Cloud — a temporary dev arrangement that production will refuse; it needs an on-prem home before go-live.
Diagram 2 — Engine repos & how they talk
+ orchestrator (in-process)"] iw["internal-web
Tanya Data + Admin"] pa["public-agent (Agno)"] ia["internal-agent (Agno)"] end rag["repo: ai-ahu-rag
Doc-RAG service"] pgv[("pgvector")] subgraph ocrrepo["repo: ahu-ocr-akta-notaris — ONE codebase, 2 worktrees"] direction TB ocrlegacy["legacy branch
no gateway wiring"] ocrtidy["tidyup branch
gateway client (dormant)"] end clf["repo: ahu-doc-classifier"] gw["repo: ahu-gpu-manager
GATEWAY (single AI egress)"] pw -->|/search| rag pw -->|/runs| pa iw -->|/runs| ia iw -. manage KB .-> rag rag --> pgv pw --> gw pa --> gw ia --> gw rag -->|embeddings| gw ocrtidy -->|OCR + LLM jobs| gw ocrtidy -->|/classify| clf ocrlegacy -->|direct today| clf
The chatbot is a monorepo of 4 apps plus an in-process orchestrator. The OCR is one repo with two git worktrees — a legacy branch and a hardening "tidyup" branch. RAG, classifier, and the Gateway are standalone repos.
Repo summary
| Repo | Role | Stack | Status | Gateway wired? |
|---|---|---|---|---|
| ahu-ai-chatbot | Public RAG "Tanya" + staff data-agent + admin console | Next.js 15 monorepo + Python/Agno + pgvector | Most mature; staging only | Wired, dormant (env flip) |
| ahu-ocr-akta-notaris (legacy) | OCRs notary deeds → SABH registry | Bun/Hono/Prisma + PaddleOCR + Azure DI | Working PoC; no auth, db push deploy |
No |
| ahu-ocr-tidyup (tidy) | Same product, hardened | + contract layer, security, migrations, gateway client | Live on staging; security in shadow mode | Wired, dormant (env flip) |
| ahu-gpu-manager (gateway) | Sole model egress + Job API + audit | Go 1.25, single binary | Gateway done (P0–P2.7). Controller/node-agent not built | — (this is the gateway) |
| ahu-ai-observatory (API) | Audit stream → TimescaleDB → query API | Go 1.25 + TimescaleDB + hash-chain | Working end-to-end | — |
| ahu-observatory-dashboard (UI) | Exec / Operator / Auditor views | React 19 + Vite + IBM Carbon | Live, wired to real API | — |
| ahu-doc-classifier | Classifies ~20 AHU document types | Python + LayoutLMv3-large (GPU) | Working; no audit/heartbeat yet | Via gateway classify upstream |
"Gateway wired but dormant" means the code path exists but is off until an env var is set — with the var unset, behavior is byte-identical to today. This is intentional per the contract (§4).
A few notes the table can't hold
- The two OCR repos are one codebase.
ahu-ocr-tidyupis a git worktree/branch ofahu-ocr-akta-notaris, not a separate rewrite. "Tidy" is a hardening branch (adds auth, migrations, a shared contract, and the dormant gateway client) that keeps the old flows running side-by-side as a correctness check. - The chatbot's orchestrator runs in-process. The plan/execute/compose logic lives inside
ahu-ai-chatbot(public-web). It began as a separate Python service (ahu-chatbot-orchestrator) that is now archived and unused — ignore that repo. - "gpu-manager" is really just the Gateway today. The repo name implies gateway + controller + node-agent, but only the Gateway is built. The Controller (:8210) and node-agent (:8220) are designed but have no code yet.
- The classifier is the one engine not yet on the audit trail. It works, but it doesn't emit heartbeat/audit events the way the contract expects. Either add that, or route it entirely through the Gateway (which would audit it for you).
What works vs. what's left
Working today: chatbot (staging), OCR-tidy (staging), the Gateway (all phases through P2.7 merged), and the full Observatory pipeline (Gateway → Redis → TimescaleDB → dashboard). One live gateway flip is done (the OCR gpu-server path).
Known gaps to hand off:
1. Big model is still off-prem (temporary). Chatbot synthesis uses Alibaba Cloud DashScope Qwen-397B — classed external_dev, allowed only in dev and refused in production. It needs an on-prem home (the future B200 cluster) before go-live; the current 2× H100 box cannot hold 397B.
2. Gateway P3 not built — no controller/node-agent, so no autoscaling or GPU/queue telemetry. The dashboard shows these as labeled placeholders.
3. Classifier emits no audit/heartbeat events — the one in-process GPU tenant isn't on the audit trail yet.
4. Observatory: no data-retention job (audit bodies grow forever), no cost tracking, only avg/max latency (no p50/p95/p99).
5. OCR security is shadow-only — enforcement is blocked until the PDF viewer / file downloads use signed URLs.
6. Most gateway flips are still dormant — chatbot and OCR-tidy have the client wired but env-gated.
7. Port drift: the dashboard deploys on 8320, but the contract says 8310. Pick one before prod.
8. All on one GPU box — production wants 5 trust zones; documented migration order is: public chatbot → DMZ first, then Observatory+DB → governance, then Gateway+Redis → control plane.
Port reference
| Service | Port | Notes |
|---|---|---|
| Gateway | 8200 | live |
| Controller | 8210 | planned, not built |
| Node-agent | 8220 | planned, not built |
| Observatory API | 8300 | live |
| Observatory dashboard | 8310 → 8320 | ⚠ deploys on 8320, contract says 8310 |
| Public chatbot web | 3500 | staging host |
| Internal chatbot web | 3510 | staging host |
| Chatbot Agno agents | 8000 | container |
Doc-RAG (ai-ahu-rag) |
8110 | |
| TEI embeddings | 8100 | |
| OCR backend | 3520 | tidy, staging host |
| Doc classifier | 5004 | |
| DCGM exporter | 9400 | pre-existing on GPU host |
Before touching any model/OCR call path in any repo, read ahu-gpu-manager/docs/CONVENTIONS.md. The engine repos (ahu-ai-chatbot, ahu-ocr-*, ahu-doc-classifier) are live and may be edited concurrently — coordinate before modifying.