DI Engine (OCR) for Kementerian Lingkungan Hidup — Infra Sizing & 2-Year Costing

Prepared: 2026-07-16 · Scope: Document Intelligence (OCR) Engine only — not the full AHU AI platform
FX planning rate: Rp 18.000 / USD (spot ~Rp 18.060 on 15-Jul-2026; rupiah −10,88% YoY → budget an FX buffer)

1. Headline numbers (recommended)

Range across infra options: Rp 6,8 – 7,8 miliar. See §6–7. Two numbers you must confirm can swing this by ~Rp 0,9 miliar each: (a) maintenance = 20%/year vs 20% total, and (b) who owns/hosts the infra ("rent to us").

2. GPU decision: RTX PRO 6000 Blackwell, not DGX Spark

Your stack is capacity-light but bandwidth-heavy. All candidate cards have enough VRAM; the differentiator for a public, concurrent endpoint is memory bandwidth (drives VLM + LLM token throughput and per-request latency).

Why not Spark for production: independent benchmarks converge — Spark is excellent for small-to-mid models at low concurrency and "gets bandwidth-pinned at higher batch sizes or larger models." A public endpoint (even gated) sees bursts; the 273 GB/s wall throttles MinerU's VLM and the LLM decode, and per-request latency degrades several-fold. 2× Spark gives you HA + ~2× aggregate throughput on prototyping hardware, but still slow per request — I'd take 1× RTX PRO 6000 over 2× Spark.

Why RTX PRO 6000 wins: 6,5× the bandwidth, ECC, 96 GB runs the entire stack co-resident with room for a 30B LLM if you ever need it, and one card sustains real concurrency. It's a genuine production inference card, not a dev kit.

Naming note: the "A6000 Blackwell" you're thinking of = RTX PRO 6000 Blackwell (lineage: RTX A6000 Ampere → RTX 6000 Ada → RTX PRO 6000 Blackwell). For a rack/colo build, pick the Server Edition (passive-cooled, 1.6 TB/s); Workstation Edition (fan, 1.8 TB/s) for a tower.

3. VRAM budget — the whole stack on one card

• Fits comfortably in 96 GB with headroom for batching and a bigger LLM. Would be tight on a 48 GB card (L40S) and would force capped MinerU concurrency.
• LLM pick: default Qwen2.5-7B-Instruct (AWQ) or a SEA-tuned model (Sailor2 / SEA-LION) for stronger Bahasa Indonesia; step up to 14B for hard docs (still fits easily). 3B only if latency is critical and fields are simple.
• Cost-saver ("make it light"): gate MinerU behind a "complex-layout detected" trigger. Simple letters/forms → PaddleOCR + LLM only (skip the VLM). This is where most of your GPU savings come from.

Serve everything with vLLM (MinerU 2.5 + the LLM) for batching + queue admission.

4. Throughput — one RTX PRO 6000 has large headroom

A ministry document-intake workload is realistically hundreds to low-thousands of documents/day, bursty. One card covers that many times over. The gates (below) cap concurrency so peak load queues instead of crashing the engine.

5. Architecture & the "1–2 gates" (don't get obliterated)

Reuse the AHU platform's existing gateway + queue pattern — it already solves this:

Public users
   │
 ┌─▼─ GATE 1 — Edge/API gateway ────────────────┐
 │  TLS · auth (API key / gov SSO) · WAF ·       │
 │  per-tenant rate-limit + quota · IP allowlist │
 └─┬─────────────────────────────────────────────┘
   │
 ┌─▼─ GATE 2 — Async job queue + admission ──────┐
 │  bounded GPU concurrency · 429/503 Retry-After │
 │  "Dalam antrean — posisi N" (never a bare spin)│
 └─┬─────────────────────────────────────────────┘
   │
   ▼  DI Engine on GPU: PaddleOCR → [MinerU 2.5] → LLM → JSON

• Gate 1 = authentication + rate limiting at the edge (stops abuse/DDoS-lite).
• Gate 2 = the queue in front of the GPU with bounded concurrency (protects the engine; excess load waits, per your CONVENTIONS.md queue semantics: GPU_QUEUE_WAIT_MS, 429/503 + Retry-After). This is the mechanism that stops the public from obliterating you.
• Supporting (CPU) tier: public web/upload front-end, object storage for docs + extracted JSON (audit), Postgres (job/audit state), Redis (queue + audit stream), Prometheus/Grafana.

6. Data residency — this pushes you to own + host in Indonesia

KLH is a government ministry; Indonesian government data under UU PDP (Indonesia's PDP Law) strongly favors on-shore processing. That means on-prem at KLH or an Indonesian Tier-3 colo — not global GPU clouds (Runpod/Lambda/etc.) for production data. This is a compliance argument and it makes buy-and-host economically better than 24/7 global cloud rental.

7. Infrastructure options & 2-year cost

Hardware landed-Indonesia = US street × Rp 18.000 × ~1,30 (PPN 11% + import/PPh22 + logistics + distributor margin).

Design 1 — Lean production (RECOMMENDED — matches "make it light")

Design 2 — HA production (in-box redundancy, SLA-grade)

Design 3 — KLH hosts / provides the DC ("rents infra to us")

If KLH supplies datacenter + power (or hosts on their own facility) and we bring the GPUs:
| Line | Cost |
|---|---|
| 1–2× RTX PRO 6000 (hardware we supply) + integration + 24-mo ops | ≈ Rp 0,4 – 0,6 miliar |

Cloud-rental reference (for comparison only — likely fails gov data-residency): a dedicated RTX PRO 6000 runs ~$1,7–2,1/hr (committed ~$1,3/hr); an L40S ~$0,9–1,5/hr. A 2-node 24/7 setup ≈ Rp 0,7–1,4 miliar / 24 mo. Owning is cheaper for a steady 24/7 workload and keeps data on-shore — hence Designs 1–2 are preferred.

8. Total deal cost — 2 years

(All figures Rp juta unless noted; "mrd" = miliar.)

Recommended headline: ≈ Rp 7,3 miliar for 2 years = Engine Rp 4,5 mrd + Maintenance Rp 1,8 mrd + Lean data-resident infra ~Rp 1,0 mrd.

9. Assumptions to confirm (each moves the number)

1. Maintenance basis — 20% of engine per year (Rp 1,8 mrd) vs 20% total for 2 yr (Rp 0,9 mrd). [assumed per year]
2. Infra ownership/hosting — "they rent infra to us": does KLH provide the DC/hardware (Design 3, ~Rp 0,5 mrd), or do we procure + host in an Indonesian colo (Design 1, ~Rp 1,0 mrd)? [assumed we procure + host]
3. Data residency — assumed mandatory on-shore (gov + UU PDP) → rules out global cloud.
4. Volume & SLA — expected docs/day + peak, and uptime target. Low/medium volume → Design 1 (1 GPU + Spark DR). SLA-grade HA → Design 2 (2× RTX PRO 6000).
5. Tax treatment — engine is quoted incl. tax; maintenance + infra shown here should have PPN 11% treatment aligned (quote all-in to match).

10. Sources

• RTX PRO 6000 Blackwell pricing — Thunder Compute (Jul 2026) · Tom's Hardware — price to $13,250 · BIZON specs (1.792 GB/s) · NVIDIA Server Edition
• DGX Spark specs & price — NVIDIA · aitooldiscovery review (273 GB/s bottleneck) · Notebookcheck price
• MinerU 2.5 hardware — HuggingFace · DeepWiki hardware compatibility · MinerU2.5 paper
• PaddleOCR PP-OCRv5 GPU throughput — GIGAGPU · PP-OCRv5 docs
• RTX PRO 6000 cloud rental — getdeploying · Runpod · L40S rental — getdeploying
• Jakarta colocation pricing report 2026 · USD/IDR — Trading Economics

11. Buy + self-host — GPU count & A/B/C infra costing (×3 markup)

Decision (2026-07-16): buy hardware, self-host in Indonesia (data residency mandatory, UU PDP).

How many RTX PRO 6000 Blackwell? → 2×

Throughput needs only one card (one 96 GB / 1.8 TB/s card runs the full stack with large headroom — §3–4). The second card is resilience, not compute: since we host and own the uptime/SLA, a single GPU is a single point of failure with multi-week RMA lead time in Indonesia. 2× = N+1 (active-active behind the queue, or active + warm standby), ideally in two separate hosts so a host failure can't take both.
- 1× GPU = budget option — fine on throughput, but accept the SPOF (mitigate with a cold-spare card or the DGX Spark as fallback).

Costing basis (all figures Rp juta = millions of Rp)

FX Rp 18.000/USD · hardware landed = US street × 18.000 × ~1,30 (PPN 11% + import/PPh22 + logistics + distributor margin) · colo ~Rp 5jt/mo per ~1 kW GPU node (Indonesian Tier-3, data-resident) · GPU card ≈ Rp 270jt each (GDDR7 shortage + FX → volatile; lock early).

Per-node building blocks: GPU card 270 · Full host (dual-EPYC-class, 512 GB ECC, 2× enterprise NVMe, redundant PSU) 245 · Min-spec self-contained host (24-core, 256 GB, 2× NVMe) 150 · Min-to-run-GPU host (16-core, 128 GB, 1× 2 TB NVMe) 90.

• A — Full, self-contained: generous GPU hosts + full supporting tier (2× app/DB nodes, object storage, backup, monitoring) — all owned. Over-spec for a light workload.
• B — GPU compute only: minimum host just to run each GPU; web/API, DB, queue, storage, monitoring run on existing (in-country) infra. Cheapest — but adds a dependency and the non-GPU tier must already exist on-shore.
• C — Minimum spec, self-contained: modest-but-sufficient GPU hosts running a lean local supporting stack — independent, cost-optimized. (recommended balance)

2× GPU (recommended, HA)

1× GPU (budget / accept SPOF)

×3 meaning: our all-in 2-yr infra cost × 3 = opening/list price. It absorbs profit + margin + negotiation headroom — you can concede down toward ~2× and still keep healthy margin. (Engine & maintenance are already sell-side numbers; the ×3 applies to infra only. Note OpEx/colo is recurring pass-through — if KLH pushes back, bill colo near-cost and keep the ×3 on hardware+setup.)

Total deal value per scenario (Engine 4.500 + Maint 1.800 + Infra×3)

Recommendation: Scenario C at 2× GPU — self-contained, minimum viable, HA. Our cost Rp 1,23 mrd, list Rp 3,69 mrd, total contract ≈ Rp 10 mrd. Pick B (2×) only if in-country existing infra can carry the non-GPU tier (cheapest at Rp 3,08 mrd list); A is over-spec for this workload.

12. FINAL CONFIGURATION (selected 2026-07-16) — Full package, NO HA, 1× GPU

Complete self-contained stack (Scenario A spec), single-instance (no HA), 1× RTX PRO 6000 Blackwell. Priced for the Indonesian market (landed, data-resident). This supersedes §11's matrix for the chosen path.

Bill of materials — CapEx (Rp juta)

Operating — 24 months (Rp juta)

Price

Optional (not HA — a safety net): keep a cold-spare RTX PRO 6000 on the shelf → +Rp 270jt cost / +Rp 810jt at ×3. Cuts RMA downtime from weeks to hours without paying for a full 2nd node.

Total deal — 2 years

(If maintenance is 20% total for 2 yr, subtract Rp 0,9 mrd → ≈ Rp 8,88 mrd. ×3 sell is shown tax-aligned with the engine; back out ÷1,11 if you quote DPP + PPN separately.)

No-HA caveat: a GPU/host failure = downtime until RMA (weeks in Indonesia for a pro card). The async queue absorbs short blips, but quote a maintenance-window SLA, not 24/7 — or take the cold-spare add-on.

13. Pricing formula (corrected 2026-07-16)

Sell price = (our cost + PPN 11% on cost) × 3. PPN is applied to our input cost before the ×3 margin.

DI-only: (1.160.000.000 + 11%) × 3 = 1.287.600.000 × 3 = Rp 3.862.800.000 (supersedes the Rp 3,48 mrd in §12)
→ DI-only total deal: 4.500 + 1.800 + 3.862,8 = Rp 10,16 mrd

14. Build option — OCR + Chatbot (DI + Knowledge Intelligence)

GPU count: 2× — workload separation, NOT redundancy. Still no HA.

The two workloads have opposite latency profiles and fight each other on one card:

A MinerU pass on a complex 50-page document pins the GPU; chatbot tokens stall mid-answer for a user who is watching them stream. Queueing is invisible for OCR and unacceptable for chat. So: one card per workload.

VRAM budget

Both keep headroom for concurrency spikes and a larger model later.

Spec delta vs DI-only

Both GPUs sit in one dual-GPU server — with no HA required there's no reason to pay for a second chassis.

Costing (Rp juta)

Cheaper variant — 1× GPU shared (compromised)

Chat model capped ~14B instead of ~32B, OCR batches throttled, chat latency degrades during batch runs.
CapEx 930 + OpEx 300 = cost 1.230 → ×1,11 → 1.365,3 → ×3 = Rp 4,10 mrd.
Saves ~Rp 1,37 mrd of sell price — but stakes a Rp 7 mrd engine on a stuttering chatbot. Not recommended.

Total deal — OCR + Chatbot (2× GPU)

15. REVISED (2026-07-16) — KI scoped to INTERNAL REPORTING (Dash) only

Efran: "We are going to only serve KI for internal report (Dash)." This supersedes §14 — the workload is a text-to-SQL data agent, not a public streaming chatbot.

GPU count reverts to 1×

My 2× recommendation in §14 was predicated on a public, streaming chatbot whose tokens a user watches appear. An internal dashboard is a different animal:

→ 1× GPU. No public burst risk, small KV cache, latency-tolerant.
→ Trigger to revisit: if KI is ever opened to the public (the original 3-engine proposal envisioned exactly that — public status self-service), the 2nd GPU returns.

Model stack

Why the specialist: XiYanSQL-QwenCoder-32B ≈ 69 BIRD vs Llama-3.3-70B ≈ 49,2 under a matched protocol — the specialist beats a generalist at 2× its size, at <½ the VRAM. Wrap with self-correction (execute → on error, retry) and optionally self-consistency (N samples, vote); both trade tokens for accuracy, which a dashboard can afford.

Cost-down to evaluate: XiYanSQL-QwenCoder-14B halves the VRAM. Decide by A/B against KLH's real schema in Discovery — not from benchmarks.

Benchmark caveat: BIRD scores are unreliable (documented annotation errors → relative shifts −3% to +31%, rank changes up to 3 places), and BIRD ≠ KLH's schema. The only number that matters is accuracy on their DB with their questions. Build a golden set of ~50–100 real leadership questions and gate on it — same pattern as the existing obs-golden-v1 / goldeneval tooling.

VRAM budget — single 96 GB card

Costing (Rp juta)

Infra comparison

Adding the whole KI engine costs +Rp 94jt of our cost / +Rp 313jt of sell price — it rides the same card.

Total deal — OCR + KI (internal Dash)

16. BUDGET FIT (2026-07-16) — Rp 20 mrd / 2 yr · Rp 10 mrd / 1 yr

The original 4× H200 spec is unfundable — by ~2,2×

Costing the Lampiran — Rekomendasi Hardware spec at our landed cost (H200 NVL confirmed $31–32k each):

The math against the budget:
- Engines (DI 4,5 + KI 7,0) + maintenance 2 yr (4,6) = Rp 16,1 mrd — 80% of budget, before a single server.
- Remaining for infra: Rp 3,9 mrd.
- Original spec at ×1,11×3 = 8.536 × 3,33 = Rp 28,42 mrd — more than the entire Rp 20 mrd budget on its own. Full deal would be Rp 44,5 mrd = 2,2× over.
- Even giving the hardware away at zero margin: 16,1 + 8,54 = Rp 24,6 mrd — still over budget.

→ The 4× H200 configuration cannot be sold into this envelope under any margin structure.

The original spec's own principle says so

From §2 Prinsip Sizing of that same document:

"GPU mudah over-provisioning: seluruh pipeline inferensi muat dalam kapasitas memori satu GPU kelas terkini"
"Kapasitas ditentukan oleh lapisan pemrosesan dokumen (OCR), database, dan alur proses" — not GPU count.

The 4 GPUs exist there for role isolation + HA + public access + enterprise scale. Every one of those is now descoped: no HA, KI is internal-Dash only, no public channel, no Trust & Governance engine. Apply the document's own logic to the actual scope → 1 GPU.

Recommended hardware — fits the Rp 3,9 mrd infra envelope

Package A — 2 years, fits Rp 20 mrd

Bills at ~Rp 10,0 mrd/year — matching an annual ceiling if that's what the budget is.

Package B — 1 year, fits Rp 10 mrd → DI only

Both engines (4,5 + 7,0 = 11,5) exceed Rp 10 mrd before maintenance or hardware. A 1-year deal is therefore DI Engine only:

Sales lever: hardware CapEx is one-time, so year 2 of infrastructure costs only +Rp 0,47 mrd (+13,8%) over the 1-year price (3,44 → 3,91). Two years of infra for 14% more than one. Push the 2-year deal.

The number that breaks this: documents/day

The original spec's own volume model (§7): 1 OCR replica ≈ 125 complex docs/hr at ~28 s/doc (CPU-based). GPU-accelerated OCR changes that by an order of magnitude — but the ceiling still matters:

That document explicitly defers this: "Jumlah final ditetapkan setelah baseline wall-clock per dokumen dan target volume KLH dikonfirmasi." It still isn't confirmed, and it is the single variable that decides whether Package A is real.

If volume is 10.000/day: the budget cannot hold both engines + our-hosted infra. Levers, in order of preference — (a) KLH hosts on-prem and buys the hardware (original spec's actual framing; collapses our infra line), (b) discount the KI engine, (c) 24-hour processing window instead of 8, (d) gate the layout VLM harder.

Note: hosting model conflict

The original spec is on-premise at KLH; our current build assumes we buy + host in our colo. Under a budget squeeze, reverting to KLH-hosted is the biggest single lever available — it removes hardware CapEx and colo from our line entirely.

17. FINAL — SPEC-ONLY (2026-07-16). Supersedes §16 packaging

Efran's directives: (1) budget Rp 20 mrd / 2 thn adalah plafon; target realistis Rp 10–16 mrd / 2 thn. (2) Spec saja, TANPA merek/tipe — merek di kontrak = risiko BAST; harga pasar Indonesia dia yang pegang, estimasi USD-ku tidak dipakai. (3) Skeptis LLM 32B "takut ga ngangkat" → jawabannya VRAM headroom sampai kelas 70B, bukan janji model.

Spesifikasi hardware (BAST-safe: semua "minimal/≥", terukur, tanpa merek)

Node A — Komputasi AI (×1)
| Parameter | Minimum | Rekomendasi |
|---|---|---|
| Akselerator AI | 1× VRAM ≥96 GB | 2× VRAM ≥96 GB (total ≥192 GB) |
| Bandwidth memori akselerator | ≥1,5 TB/s per unit (datasheet) | sama |
| Presisi inferensi | FP16 + kuantisasi 8-bit & 4-bit | sama |
| ECC (VRAM & RAM) | ya | ya |
| CPU | ≥32 core server-class ≥2,4 GHz | sama |
| RAM | ≥512 GB ECC | sama |
| Storage | 2× ≥7,68 TB NVMe (mirror) | sama |
| PSU | redundan, chassis+PSU wajib sanggup akselerator ke-2 double-width tanpa ganti server | terisi 2 |
| Manajemen | BMC/IPMI, 2× ≥10 GbE | sama |

Node B — Aplikasi & Database (×1): CPU ≥32 core · RAM ≥384 GB ECC (min ≥256) · 2× ≥3,84 TB NVMe (mirror) · ≥10 GbE · BMC. Menjalankan web/API, antrian, DB relasional + vector index, dashboard pelaporan, monitoring.

Penyimpanan & backup: usable ≥15 TB (dokumen + audit + riwayat laporan) · target backup ≥30 TB · snapshot harian.

Jaringan & fasilitas: switch managed ≥10 GbE · firewall · UPS ≥3 kVA · rak · DC Tier-3 di Indonesia · bandwidth redundan · daya ~1,7–2,3 kW.

Kenapa spec ini menjawab keraguan 32B

• Minimum (96 GB): DI stack + LLM 32B = ~61–78 GB — muat, tapi mentok di 32B.
• Rekomendasi (2× 96 GB): GPU-1 = DI stack (~33–48 GB); GPU-2 = LLM KI — 32B (~30 GB) SAMPAI 70B INT4 (~50–65 GB) muat. Kalau 32B ternyata kurang, naik ke 70B tanpa beli server baru — tinggal model swap.
• Jalur hemat: beli chassis dual-capable + 1 kartu dulu; kartu ke-2 = opsi tambah belakangan (klausa chassis di atas yang mengunci jalur ini).
• Catatan: untuk text-to-SQL, specialist 32B ≈/> generalist 70B di benchmark (§15) — tapi keputusan tetap lewat golden-set uji di Discovery, dan spec ini memberi escape hatch apa pun hasilnya.

Bentuk paket 2 tahun (angka infra = placeholder formulaku (cost ×1,11 ×3); ganti dengan cost pasar riil)

KI "Dash Edition" @ Rp 3,5 mrd = usulanku (bukan angka yang pernah disepakati): lisensi KI dengan scope internal reporting saja — tanpa kanal publik, tanpa self-service status masyarakat, tanpa RAG publik. Price fence yang jujur (setengah harga untuk setengah scope), dan meninggalkan ruang upsell KI penuh nanti.

18. FINAL v2 (2026-07-16) — Rp 20 mrd KHUSUS INFRA · 3 engine full price · retain ≥4 tahun

Reframe Efran: budget Rp 20 mrd = infrastruktur saja; engine (DI 4,5 + T&G 3,5 + KI 7,0 = 15,0 full price, walau awalnya dipakai internal) dan maintenance di luar budget itu. Infra + engine di-retain ≥4 tahun. Pertanyaan: 2× 96 GB (192 GB) cukup, atau tambah?

Jawaban: TAMBAH → 4× ≥96 GB (total ≥384 GB) pada 2 node AI. Alasan struktural, bukan sekadar "mumpung ada budget":
1. Scope naik lagi — 3 engine full berarti KI penuh (kanal publik + RAG + self-service) dan T&G (gateway + guardrail + audit). Chat publik = interaktif/streaming → wajib terpisah dari batch OCR (§14); T&G butuh porsi inferensi kecil (guardrail/embedding).
2. 4 tahun — tren ukuran model naik; VRAM adalah satu-satunya hedge yang tak bisa ditambal belakangan tanpa beli besi baru.
3. 4 tahun layanan publik tanpa redundansi = janji yang tak bisa dipegang. N+1 kembali wajib (RMA kartu di Indonesia = mingguan). 2 node → node-level failover (degraded mode), persis arsitektur Lampiran hardware kita sendiri — hanya di kelas ekonomi 96 GB, bukan kelas HBM 141 GB yang terbukti memecahkan budget (§16: platform lengkap kelas HBM ≈ sell Rp 28 mrd).

Pembagian peran (mengikuti Lampiran §3, brand-free):
| Akselerator | Peran | Headroom 4 tahun |
|---|---|---|
| 0 (Node A1) | LLM ekstraksi DI | naik 70B INT4 satu kartu |
| 1 (Node A1) | OCR-assist + layout + klasifikasi | pelatihan terjadwal off-peak |
| 2 (Node A2) | KI: LLM + RAG + embedding (publik) | 32B→70B; TP 2-kartu dalam node = kelas ~100B+ kuantisasi |
| 3 (Node A2) | Failover / burst + guardrail T&G | menutup peran mana pun |

Spesifikasi (BAST-safe, semua "minimal/≥", tanpa merek)

Node AI ×2 (identik):
- 2× akselerator VRAM ≥96 GB per unit (per node 192; total sistem ≥384 GB) · bandwidth ≥1,5 TB/s per unit · FP16 + kuantisasi 8/4-bit · ECC
- CPU ≥48 core ≥2,4 GHz · RAM ≥768 GB ECC · 2× ≥3,84 TB NVMe (OS, mirror) + 2× ≥7,68 TB NVMe (data) · endurance ≥1 DWPD
- PSU redundan · chassis sanggup ≥4 akselerator double-width (jalur ekspansi ke 8 total) · BMC · 2× ≥25 GbE

Node OCR/worker ×2 (CPU): ≥48 core ≥2,6 GHz · ≥256 GB ECC · 2× ≥1,92 TB NVMe · ≥10 GbE. (replika OCR CPU per model sizing Lampiran §7: ±125 dok kompleks/jam/replika)

Node Database ×2 (HA): ≥32 core · ≥384 GB ECC · 8× ≥3,84 TB NVMe (RAID10, ±15 TB) · ≥25 GbE. (DB relasional + vector index, replikasi + pooling)

Node Aplikasi/API ×2 (HA): ≥24 core · ≥128 GB ECC · 2× ≥1,92 TB NVMe. (di belakang load balancer)

Object storage (arsip 4 thn): usable ≥60 TB redundant (erasure/replikasi) — dokumen + audit append-only + riwayat + KB.
Backup: target ≥120 TB · snapshot harian · uji restore berkala.
Jaringan: 2× ToR ≥25 GbE (MLAG) · switch manajemen OOB · firewall/NGFW pasangan-HA (kanal publik) · load balancer HA.
Fasilitas: DC Tier-3 Indonesia · UPS ≥10 kVA · daya ~5–7 kW · bandwidth redundan.
Garansi & suku cadang (klausa 4 tahun): garansi pabrikan ≥3 thn (target 5) atau unit cadangan on-site: 1× akselerator cold-spare + PSU/NVMe spare kit.

Envelope harga (placeholder cost-ku ×1,11×3 — ganti dengan cost pasar riilmu)

Turun-spec bila mau ruang: pangkas cold-spare + kecilkan object storage → sell ~17. Naik-spec (isi plafon): +2 akselerator (6 total) → sell ~20,9 — lewat; jadi 4+spare adalah sweet spot plafon.

Kenapa bukan kartu VRAM lebih besar (kelas 141/192 GB HBM)? §16 sudah membuktikan: 4× kelas HBM + platform = cost ~8,5 mrd → sell 28+ mrd, pecah plafon. Dan beban kerja ini butuh jumlah kartu (isolasi peran + redundansi), bukan kartu tunggal raksasa. 96 GB × 4 = arsitektur Lampiran, ekonomi yang masuk.

Referensi nilai total (engine full price, di luar budget infra)

Engine 15,0 + maintenance (di luar, TBD) + infra ≤20 → total program ~Rp 35–41 mrd tergantung struktur maintenance.

19. Dua konfigurasi final — DI-only vs DI+KI+T&G (2026-07-16)

Horizon retain ≥4 tahun untuk keduanya. Semua spek "minimal/≥", tanpa merek (BAST-safe). Harga = placeholder unit-economics-ku; formula: raw → +PPN 11% → ×3.

Spek berdampingan

Harga (placeholder — ganti raw dengan cost pasar riil; struktur tetap)

Opsi turun DI-only: hapus cold-spare → raw 3.000 → sell ≈ 9,99 mrd (pas di bawah 10).
Keduanya di bawah plafon infra Rp 20 mrd; gap DI-only→full ≈ Rp 8 mrd sell = harga masuknya KI publik + T&G + HA penuh.

20. BENCHMARK EMPIRIS (2026-07-18) — Specialist 32B vs MoE 397B, text-to-SQL

Klaim §15 yang diuji: "specialist SQL model beats bigger generalists." Efran menantang klaim ini → benchmark riil dijalankan, bukan angka paper.

Setup (protokol identik untuk keduanya): BIRD Mini-Dev, 120 soal stratified (36 simple / 60 moderate / 24 challenging, seed 42, 11 database) · prompt sama: DDL schema + evidence + question · temperature 0 · single-shot (tanpa self-correction/self-consistency) · skor = execution accuracy (hasil query dieksekusi di SQLite, dibandingkan set-equality vs gold).
- Qwen3.5-397B-A17B (MoE, reasoning mode default) via gateway :8200 → DashScope cloud, header CONVENTIONS lengkap, prioritas batch.
- XiYanSQL-QwenCoder-32B-2504 (dense specialist) Q4_K_M GGUF (19,9 GB) di GPU-1 H100 ai-ahu, vLLM 0.24 + vllm-gguf-plugin, util 0.55 — berjalan BERDAMPINGAN dengan seluruh workload produksi; 35B Qwen tidak pernah disentuh. Host dibersihkan penuh setelah selesai (disk & GPU kembali ke keadaan awal).

Hasil

Head-to-head: 66 sama-sama benar · 13 hanya 397B · 13 hanya specialist · 28 sama-sama salah. McNemar p≈1,0 → statistical dead tie.

Verdict — jujur

• Klaim "specialist BEATS" TIDAK terbukti vs MoE reasoning modern — hasilnya seri persis. (Klaim §15 aslinya vs Llama-3.3-70B dense generalist; lawan 397B MoE + reasoning, keunggulan akurasi hilang.)
• Klaim deployment TERBUKTI: file 20 GB ter-kuantisasi 4-bit, di satu GPU, menyamai model 397B cloud dengan reasoning — sambil 30% lebih cepat, ~29× lebih hemat token output, tanpa biaya cloud per-query, tanpa data keluar (residensi ✓). Untuk KI internal Dash, specialist 32B on-prem tetap pilihan yang tepat — bukan karena menang akurasi, tapi karena akurasi setara dengan biaya & kedaulatan yang jauh lebih baik.
• Specialist unggul di challenging (54,2% vs 45,8%, n=24 — tidak signifikan, tapi arah yang menarik).

Caveats: specialist di-handicap Q4 (bf16 mungkin +1–3 pt); 397B diuntungkan reasoning mode (bayar 1.500 token/query); single-shot tanpa teknik; BIRD ≠ skema KLH → keputusan final tetap lewat golden-set di Discovery.

Artefak: bench-sql/ (bench.py, cfg_*.json, results_cloud.jsonl, results_specialist.jsonl).

21. SOTA SPECIALIST BARU + ARSITEKTUR 12× B200 (2026-07-18)

Konteks Efran: prod akan punya 16× B200, maksimal 12 untuk chatbot. Pertanyaan: specialist SQL yang lebih baru/SOTA?

Peta specialist text-to-SQL terkini

Benchmark tambahan (harness & 120 soal yang sama)

Arctic-R1-7B bf16 dijalankan di GPU-1 ai-ahu (prod tak tersentuh, host dibersihkan penuh):
| | 397B MoE | XiYanSQL-32B Q4 | Arctic-R1-7B |
|---|---|---|---|
| Overall EX | 65,8% | 65,8% | 62,5% |
| simple / mod / chall | 86,1/61,7/45,8 | 83,3/60,0/54,2 | 86,1/55,0/45,8 |
| Median latency | 23,75 s | 16,71 s | 2,47 s |

7B menempel 3 pt di bawah dua model raksasa dengan latency 7–10× lebih rendah. (Arctic-R1-32B belum termuat di sisa VRAM H100 hari ini — kandidat kuat #1 untuk diuji di B200.)

Angka yang mengubah strategi: oracle ensemble

Dari 3 run di atas, per-soal: union (min. 1 benar) = 95/120 = 79,2% · union 2 specialist saja = 75,0% · majority vote = 66,7% · Arctic uniquely correct = 3 soal.
→ Ceiling ensemble ~79% vs ~66% single-model — +13 pt tersedia lewat test-time scaling, dan itu persis resep Agentar-Scale-SQL (81,67% test). Dengan 12× B200, compute untuk N-kandidat + eksekusi + seleksi adalah pembulatan kecil.

Rekomendasi arsitektur chatbot 12× B200 (2.304 GB VRAM)

Alur SQL (execution-guided selection): N kandidat (2 specialist + MoE) → eksekusi semua di replika DB read-only → hasil sepakat → kirim; beda → selection judge memilih. Data kami: majority naif cuma 66,7%, tapi seleksi berbasis eksekusi bergerak menuju ceiling 79,2%.

Yang perlu diuji berikutnya (di B200 / setelah ruang tersedia): Arctic-R1-32B bf16 single-shot + konfigurasi ensemble 3-generator dengan selection judge, di-gate golden-set KLH.

22. ROUND FINAL (2026-07-18) — Arctic-32B tidak dirilis; Q8 test; ensemble TERUKUR

Otorisasi Efran: boleh matikan 35B di GPU 0 untuk bench Arctic-R1-32B "properly".

Temuan pertama: Arctic-Text2SQL-R1-32B (71,83% BIRD test) TIDAK PERNAH DIRILIS. Snowflake hanya mempublikasikan bobot 7B di HF; 14B/32B paper-only. Klaim SOTA-nya tidak bisa diverifikasi atau di-deploy. → §21 harus dikoreksi: specialist 32B ter-deploy terbaik yang benar-benar ada tetap XiYanSQL-QwenCoder-32B; Arctic hanya menyumbang 7B.

Substitusi yang dijalankan (pakai window downtime yang sama): XiYanSQL-32B Q8_0 (near-lossless, 34,8 GB) di GPU 0 kosong — menguji apakah Q4 menghandicap specialist pada hasil seri §20. Prosedur: build-cache docker di-prune (+38 GB, aman), 35B di-restart selama download (downtime diminimalkan ke ~25 menit), bench 120 soal identik, bersih-bersih penuh, 35B kembali healthy (verified).

Hasil Q8 vs Q4

Kuantisasi BUKAN penyebab seri. ~66% adalah ceiling asli specialist ini pada protokol single-shot. Seri vs 397B = hasil yang solid.

Ensemble TERUKUR (bukan oracle) — execution-guided selection offline

Kandidat dieksekusi → dikelompokkan berdasar kesamaan result-set → klaster terbesar menang (tanpa label gold):
| Konfigurasi | Selected-EX |
|---|---|
| Single model terbaik | 65,8% |
| 3-generator DIVERSE (XiYan-32B + 397B + Arctic-7B) | 69,2% (+3,4 pt) |
| 4-generator dengan Q4+Q8 (redundan) | 67,5% — TURUN |
| Oracle ceiling (§21) | 79,2% |

Dua pelajaran keras untuk desain B200: (1) diversity > jumlah generator — menambah generator kembar justru merusak voting; (2) gap 69→79 adalah ruang selection-judge (LLM judge saat kandidat berbeda, bukan sekadar klaster mayoritas).

Konsekuensi untuk §21 (alokasi 12× B200)

Pool specialist SQL = XiYanSQL-32B (bf16/FP8) + Arctic-7B + generator MoE — tiga keluarga berbeda, bukan varian sekeluarga. Arctic-32B dicoret sampai Snowflake merilis bobot. Selection-judge (2× B200 di §21) adalah komponen dengan ROI akurasi tertinggi: +3,4 pt sudah terbukti dengan klaster naif, ~+10 pt lagi tersedia menuju oracle.

Host: 35B healthy kembali, disk 50 GB free (build-cache prune netto +21 GB vs sebelum eksperimen), GPU baseline, semua artefak bench dihapus. Hasil: bench-sql/results_q8.jsonl, ensemble.py.

23. KEPUTUSAN MODEL PRODUKSI (2026-07-18) — 397B atau specialist?

Reframe: bukan either/or. Chatbot KLH punya DUA pekerjaan yang berbeda sifat, dan benchmark kita HANYA menguji satu (NL→SQL). Jawabannya = pakai keduanya, dibagi per peran.

Kenapa specialist untuk SQL (data terukur):
- Akurasi SERI: XiYanSQL-32B = 397B = 65,8% (Q4 & Q8 identik → bukan artefak kuantisasi).
- Specialist ~3× lebih cepat (8,6 vs 23,8 s), ~29× lebih hemat token (52 vs 1.504), dan ~10× throughput/GPU (32B vs 397B) → di skala publik ini menentukan.
- 397B sebagai generator SQL tunggal = bayar mahal untuk hasil yang sama. Jangan.
- Ensemble 3-model DIVERSE + seleksi berbasis eksekusi = 69,2% (+3,4 pt terukur), ceiling 79,2%. Divergasi antar-generator = sinyal "tidak yakin" bawaan → untuk dashboard pimpinan, lebih baik lapor "perlu verifikasi" daripada angka salah dengan percaya diri.

Kenapa tetap butuh MoE besar: specialist SQL tidak bisa mengarang jawaban RAG, bercakap, atau menarasikan hasil. Itu ranah generalis. 397B open-weight → self-host di B200 (bukan DashScope cloud; prod menolak external_dev per CONVENTIONS §5, residensi UU PDP).

397B vs 235B untuk peran generalis: untuk SQL keduanya tak relevan (kalah dari 32B). Untuk kualitas RAG/percakapan — belum diukur di sini. Default efisien = 235B-A22B (2× B200, bebaskan 2 kartu untuk konkurensi/specialist); ambil 397B (4× B200) hanya jika mau kualitas jawaban publik maksimum dan kartu cukup. Putuskan lewat A/B RAG-quality di Discovery, bukan asumsi.

Rakitan akhir (SQL): fan-out 3 generator → eksekusi semua di replika DB read-only → klaster result-set, terbesar menang → jika divergen, selection-judge (MoE rumah) memilih atau tandai low-confidence → narasi ke Bahasa Indonesia oleh MoE. Gate final = golden-set pertanyaan pimpinan KLH (pola obs-golden-v1/goldeneval).

24. PROTOTIPE ENSEMBLE + SELECTION-JUDGE — DIBANGUN & DIUJI (2026-07-18)

Prototipe dibangun (bench-sql/ensemble_judge.py): 3 generator diverse (XiYanSQL-32B Q8 + Arctic-R1-7B + Qwen3.5-397B, di-replay dari prediksi temp=0 deterministik) → eksekusi di DB read-only → klaster result-set → judge LIVE (397B via gateway) saat divergen. Confidence tier = HIGH (semua sepakat) / MED / LOW.

Hasil — dan koreksi keras terhadap §21/§23

Temuan (negatif, penting): LLM selection-judge TIDAK mengalahkan penghitungan suara naif. Pada 50 soal divergen: judge 22–24/50, naive 23/50 — setara noise. Koreksi: klaim §21/§23 "selection-judge = komponen ROI tertinggi" SALAH, dibantah oleh tes sendiri. Sebabnya fundamental: memilih jawaban benar saat kandidat berbeda sama sulitnya dengan menghasilkannya (kalau judge tahu mana yang benar, generator sudah menghasilkannya) — dan 397B adalah salah satu generator, jadi bias ke jawabannya sendiri.

Yang BENAR-BENAR bernilai: confidence gating

Divergasi antar-generator = sinyal ketidakpastian yang akurat. Untuk dashboard pimpinan, ini produk sesungguhnya: sistem tahu KAPAN ia benar. Auto-serve hanya tier HIGH → ~85% akurasi pada 57% pertanyaan; sisanya diarahkan, bukan ditebak.

Konsekuensi arsitektur (revisi §21 alokasi 12× B200)

• Coret selection-judge sebagai lever akurasi. Pakai naive largest-cluster (tanpa latency/biaya judge).
• 2× B200 yang tadinya untuk judge → realokasi ke (a) lebih banyak generator diverse (menaikkan ceiling 79% + kesepakatan) dan (b) jalur human-in-the-loop untuk tier MED/LOW.
• Lever akurasi sebenarnya, berurutan: schema-linking + few-shot KLH (menaikkan base rate jauh di atas 65% BIRD yang sengaja sulit) > lebih banyak/lebih baik generator (menaikkan ceiling) > confidence gating (memutuskan kapan serve). Selection-judge: tidak.

Artefak: bench-sql/ensemble_judge.py, results_ensemble_judge.jsonl (single), results_ensemble_judge_sc3.jsonl (self-consistency).

25. REKAP KESEPAKATAN GPU + EVALUASI OPSI L40S (2026-07-18)

Yang sudah disepakati (rekap)

Ke KLH (spec-only, tanpa merek — §18/§19): akselerator VRAM ≥96 GB per unit, bandwidth ≥1,5 TB/s, FP16 + kuantisasi 8/4-bit, ECC. DI-only = 1 node AI × 2 unit (≥192 GB) + 1 cold-spare. DI+KI+T&G = 2 node AI × 2 unit (total ≥384 GB) + spare. Chassis wajib muat ≥4 unit.
Kelas acuan internal: RTX PRO 6000 Blackwell 96 GB (~270jt landed) — dipilih mengalahkan DGX Spark (bandwidth 273 GB/s = prototyping), dan menggantikan 4× H200 NVL dari Lampiran (cost ~8,54 mrd → sell 28+ mrd, memecahkan plafon §16).
(Terpisah: prod kita sendiri = 16× B200, 12 untuk chatbot — bukan konfigurasi KLH.)

L40S — data

Inti ekonomi: L40S = 74% harga untuk 50% VRAM dan 48% bandwidth → per-GB ±1,5× LEBIH MAHAL.

Konsekuensi biaya (formula (raw+PPN)×3)

Daya: DI-only L40S hemat ~500 W; tapi pada VRAM setara (8 kartu) justru 2.800 W vs 2.400 W — lebih boros.

Apakah 48 GB cukup? (pakai angka terukur kita)

Rekomendasi

1. DI-only → L40S LAYAK dan lebih murah (hemat ~Rp 700jt sell). Temuan Q4=Q8 kita membuat 32B muat nyaman. Ambil bila procurement mengutamakan ketersediaan/harga stabil.
2. Full package (DI+KI+T&G, publik, 4 tahun) → JANGAN L40S. VRAM setara justru melewati plafon; jumlah kartu sama mengorbankan seluruh cerita headroom. Tetap 96 GB-class.
3. Paling cerdas: armada campuran — L40S untuk tier OCR/DI (murah, hemat daya, pas ukuran), 96 GB-class untuk tier LLM/KI (butuh headroom). Ini konsisten dengan Lampiran sendiri, yang memang memisahkan node GPU-Inference (H200) dari node OCR/Doc-AI (0–1× L4) — jadi bukan menyalahi dokumen yang sudah dilihat KLH.

Keunggulan nyata L40S (jujur): ada di katalog OEM mana pun (server card pasif dual-slot, rack-friendly, sourcing Indonesia mudah = aman untuk BAST); tidak terpapar kelangkaan GDDR7 yang menaikkan harga kartu 96 GB sampai 55% — penting untuk kontrak harga tetap 2 tahun; daya per kartu lebih rendah.
Risikonya: rilis Agustus 2023 → sudah 3 tahun sekarang, 7 tahun di akhir kontrak; tanpa FP4/NVFP4 padahal distribusi model sudah bergerak ke sana (build NVFP4 resmi Qwen3.5-397B ada di HF).

⚠ KONSEKUENSI DOKUMEN: spek yang kita tulis ke KLH berbunyi "VRAM ≥96 GB per unit" — L40S (48 GB) TIDAK MEMENUHI. Bila pindah ke L40S, bahasa spek WAJIB diubah ke basis total (mis. "total ≥192 GB pada ≥4 unit, masing-masing ≥48 GB"), atau BAST akan mismatch.