GPU host survey & deployment plan — 192.168.83.20 ("x056")

Surveyed: 2026-07-04 over SSH (read-only; the only change made was installing Efran's workstation SSH public key into ~/.ssh/authorized_keys — recommend rotating the password that was shared in chat).

1. What the box is

Item	Value
OS	Ubuntu 24.04.3 LTS, kernel 6.8, x86_64
GPU	2× NVIDIA H100 NVL, 96 GB each (driver 580.159.03). GPU0 ~86 GB used, GPU1 ~29 GB used at survey time
CPU / RAM	12 cores / 122 GB (≈85 GB available)
Disk	502 GB, 92% full — 42 GB free. docker system df: 176.5 GB (79%) of images reclaimable
Docker	29.3.1, Compose v5.1.1
Reverse proxy	Host nginx (80/443): ahu-azure.conf (x056.ahu-azure.val.id), plus configs for grafana/prometheus/flower (those containers not currently running)
GPU telemetry	ahu-dcgm-exporter already running on :9400 — the controller scrapes this; no new telemetry plumbing needed

Corrections to earlier assumptions:

• This is two H100 NVLs, not one. Two GPUs means the placement/registry model matters on day one (which replica on which GPU), not only post-B200.
• Host port :8001 → ahu-vllm (container port 8000). The OCR backend's AKTA_TXN_CLASSIFIER_URL=…:8001/v1 therefore hits the same vLLM instance the chatbot uses via ahu-vllm:8000 — one 35B upstream in the registry, two names today.

Running AI upstreams and reachability from off-box:

Service	Host binding	Reachable from services VM?
ahu-vllm (Qwen 35B)	0.0.0.0:8001	✅ direct
ahu-cleanup-llm (3B)	0.0.0.0:8003	✅ direct
tei-qwen3-embed	0.0.0.0:8100	✅ direct
ahu-paddle-ocr	0.0.0.0:8108	✅ direct
Azure DI layout 3.1/4.0, custom, id, studio	0.0.0.0:5001/5011/5003/…	✅ direct
Azure DI read 3.1/4.0	127.0.0.1:5000/5012	⚠️ loopback-only
ahu-gpu-server (FastAPI/Celery)	127.0.0.1:8000	⚠️ loopback-only (nginx path /api/gpu works)
ahu-classifier	127.0.0.1:5004	⚠️ loopback-only (nginx path /api/classifier works)

Loopback-bound services: at env-flip time, either re-publish on 192.168.83.20 (compose port change, done with the engine go-ahead) or route via the existing nginx HTTPS paths. Until then the node agent (on-box) sees them regardless.

Ports 8200 / 8210 / 8220 / 8300 / 8310 are free — our allocations (see CONVENTIONS.md) fit without conflict.

2. How ahu-gpu-manager taps in

On the GPU host — one thing only: the node agent.

• Compose project ahu-platform-agent, one container, LAN-bound :8220.
• Mounts /var/run/docker.sock (read-only in P1 for inventory/health; read-write from P3 when the controller manages replica lifecycle).
• Scrapes the existing DCGM exporter (:9400) for GPU telemetry; probes local model-server health endpoints (including the loopback-bound ones — this is why the agent, not the controller, does probing).
• Adding a future B200 node = install this one binary/container there.

Everything else runs off-box on the services VM(s) — and the disk situation upgrades this from "recommended" to "do it now": a full-capture audit store on a 42 GB-free disk is a non-starter. Since VM provisioning is easy: provision the services VM first, deploy there directly, skip the interim-on-GPU-host step entirely.

• Services stack (compose project ahu-platform): gateway :8200 (×2 behind VIP when the second VM exists), controller :8210, observatory-ingester, observatory-api :8300, dashboard :8310, postgres+timescale, redis (+ sentinels when ×2).
• Gateway reaches upstreams via the host-published ports above (192.168.83.20:8001 etc.).
• Engines reach the gateway at http://<services-vm>:8200 — that's the value that goes into MODEL_GATEWAY_URL / *_URL env flips (see docs/integration/).
• Optional later: nginx location /gateway/ for TLS if anything outside the LAN must reach it.

3. Housekeeping recommendations (not executed — your call)

1. docker image prune -a after confirming no stopped-container images are still wanted → frees ~176 GB (92% → ~57% disk). The 6 stopped containers include what the grafana/prometheus nginx configs point at.
2. Rotate the SSH password for efran@192.168.83.20 (it appeared in a chat transcript); key auth is now in place.
3. ahu-ai-ocr-web is currently unhealthy — unrelated to this work but worth a look.

4. Deployment sequence (maps to rollout phases)

Phase	On GPU host	On services VM
P0	nothing	gateway (single), stub ingester; staging chatbot flipped
P1	node agent (telemetry-only, docker.sock ro)	gateway, ingester + tiered store, minimal dashboard; all /v1 egress flipped
P2	loopback ports re-published or nginx-routed	job API + OCR adapters live
P3	agent gains lifecycle control (docker.sock rw)	controller placement/autoscaler (dry-run → enforce)
P4	B200 nodes join with agents	driver decision (k8s/llm-d vs compose) executed