AHU OCR — Target Architecture Design (for judgment)
Date: 2026‑07‑01
Strategy (already chosen): hybrid strangler, in‑tree — refactor the forked orchestration/rules behind seams that already exist; greenfield‑in‑place the two structural slices (frontend review engine, persistence); keep the per‑flow forks alive as an equivalence oracle and migrate flow‑by‑flow.
How this was produced: 6 pillar architects designed against the real code in parallel, then an adversarial integration pass reconciled the shared interfaces and stress‑tested whether "add a flow = a config entry" actually holds. It caught 2 critical interface conflicts (now resolved) — those are in §7 so you can judge the honesty of the fit.
Companion doc:
docs/audits/2026-07-01-jankiness-assessment-and-tidyup-plan.md(the assessment this design answers).
1. TL;DR — the shape
One idea: model the flow matrix as data, not code. Today each legal flow (PT/PP/Apostille × pendirian/perubahan/perbaikan/…) is a hand‑cloned stack of route + processor + cross‑validator + diff + frontend page + hook. The target replaces that with compiler‑forced registries keyed by SubmissionType, each driving one generic engine:
┌─────────────────────────── contract/ (SSOT, zod data only, no React/Hono) ─┐
│ enums.gen.ts (from Prisma) · ReviewDataSchema (normalised) · SectionSchema │
│ ErrorCode/AppError · RouteContract │
└──────────────▲───────────────────────────────▲──────────────────────────────┘
│ imported by both sides (alias @contract)
BACKEND │ │ FRONTEND
┌─────────────────────────────────────┴──────┐ ┌───────────┴───────────────────────────────┐
│ FLOW_REGISTRY: Record<SubmissionType, │ │ REGISTRY: Record<SubmissionType, │
│ FlowConfig> │ │ FrontendReviewDescriptor>│
│ ├─ runFlow(id, cfg) ── processor │ │ └─ <ReviewEngine descriptor={…} │
│ ├─ reMatchAndValidateFlow ── SAME cfg ◀──┼────────────┼────── (mode: author|verifier|readonly) │
│ ├─ runValidations(cfg) ── async-partition Promise.all one shell, N data descriptors │
│ └─ persistValidationResults ─ via validationRepo (txn)│ createReviewHook(endpoints) │
│ cfg.rules = [...SHARED_RULE_SET, ...flowRules] (bodies │ components/review/* primitives │
│ lifted BIT-FOR-BIT from cross-validator.ts) │ (one-way import boundary) │
└───────▲───────────────▲───────────────▲────────────────┘ └────────────────────────────────┘
│ │ │
ExtractionProvider persistence/ security/ (installSecurity(app), one call)
(docType×provider) repos + typed JWT+TokenVerifier · ROUTE_POLICIES · resource-authz
saveExtraction() JSON codecs + upload-guard · rate-limit · PII lifecycle · real health
gpuJob<Req,Res>() satellites + ──────────────────────────────────────────────────────
gpu /extract/{name} single outbox Python: gpu-server → 1 generic /extract/{name} + spec registry
paddle-ocr-service KEPT verbatim
Verdict from the integration architect: "Judge‑worthy and fundamentally sound; the six pillars share one genuine philosophy and it's faithfully applied. Approve the architecture; block merge until the ReviewData shape and the FlowConfig/FE‑descriptor boundary are reconciled (both done in §7). With those two fixes it is a coherent, strangler‑safe, compiler‑forced target worth building."
The money metric — cost to add a new flow (e.g. PELEBURAN_PT):
Today (AKUISISI_PT actual) |
Target | |
|---|---|---|
| Backend processor | 815‑line fork of perubahan‑processor | ~15–40‑line FlowConfig entry |
| Cross‑validator | new forked runner | rules: [...PT_AKTA_RULE_SET, koranWindowRule] |
| Dispatch | 2 edits (dispatch + reMatch if‑chain) | 0 — both resolve the same registry |
| Route file | forked CRUD | 0 (reuses routeFamily) |
| Frontend page | ~700‑line page + forked hook | ~60–120‑line data descriptor, 0 new hook |
| Security | (nothing — it's open) | 1 RoutePolicy object (CI fails if missing) |
| Net | ~2,500 LOC across 6+ files | ~5 config edits + optional 1 rule + 1 hook |
A missing/misspelled SubmissionType key becomes a compile error in three registries at once — the only adoption mechanism that has historically held here (contrast: field-confirmation.ts is used by 1 of 6 confirm endpoints today).
2. The reconciled canonical interfaces (the contract you're judging)
These are the load‑bearing types; every pillar consumes or produces them. Signatures are the reconciled versions (after the integration pass fixed the conflicts).
// contract/enums.gen.ts — GENERATED from Prisma (db:generate), FE-importable, parity-tested
type SubmissionType =
| 'PENDIRIAN_PT' | 'PERUBAHAN_PT' | 'AKUISISI_PT' | 'PERBAIKAN_DATA_PT'
| 'PENDIRIAN_PP' | 'PERUBAHAN_PP' | 'PERBAIKAN_DATA_PP' | 'PEMBUBARAN_PP'
| 'PERALIHAN_PP_KE_PT' | 'APOSTILLE' // EXACT Prisma spellings (schema.prisma:58)
type ValidationStatus = 'PASS' | 'WARNING' | 'FAIL' | 'SKIPPED' // no -ED suffix
// ── Flow Engine (Pillar 1) owns these ──────────────────────────────────────
interface FlowConfig<Ctx = RuleContext> {
type: SubmissionType
engine: 'v1-fork' | 'v2-generic' // migration flag; delete when all flipped
primaryAkta: { classifiedType: string; subtype: AktaSubtype; missingError: string; extractError: string } | null
skipExtractionTypes: string[]
companyLookup: boolean
extraction: ExtractionPlan // → OCR pillar
buildContext(id: string): Promise<Ctx> // builds RuleContext ONCE per run
rules: Rule<Ctx>[] // [...SHARED, ...flowSpecific]
sections: SectionSchema[] // serialisable → published to FE via contract/
rematch: RematchConfig
hooks?: ProcessorHooks<Ctx> // the ONLY sanctioned per-flow behaviour
routeFamily: RouteFamily // string discriminator ('perubahan')
policy?: RoutePolicy // → Security pillar
customProcess?(id: string, cfg: FlowConfig<Ctx>): Promise<void> // escape hatch (apostille/PP)
labels: { akta: string; flow: string }
// NOTE: NO `review` field — a backend config cannot hold JSX (see §7 conflict #2)
}
interface Rule<Ctx = RuleContext> { // the atomic unit
code: string; label: string
severityFloor?: ValidationStatus // clamp (e.g. koran window ≤ WARNING)
async?: boolean // partitioned to Promise.all (SABH round-trips)
run(ctx: Ctx): RuleResult | RuleResult[] | Promise<RuleResult | RuleResult[]>
}
interface RuleResult { ruleCode: string; ruleLabel: string; status: ValidationStatus; message: string; details?: Record<string, unknown> }
// ^ VERBATIM from cross-validator.ts:148 so every existing pure validator satisfies it unchanged.
// A shared rule is literally `{ code, label, run: ctx => validateNikKtpAkta(ctx.input) }` — wrap, don't rewrite.
interface ProcessorHooks<Ctx = RuleContext> {
mapChangeTypes?(ct: ChangeTypeDetectionResult): ChangeTypeDetectionResult // akuisisi.forcePeralihanOn
afterAktaExtraction?(a: { submissionId: string; aktaDocumentId: string; rawText: string }): Promise<void> // koran
onFinalize?(tx, submissionId: string): Promise<void> // enqueueOutbox for SABH push
}
// ── Persistence (Pillar 3) co-owns the ONE read-model ──────────────────────
interface ReviewData { // NORMALISED / flow-agnostic (the reshape, see §7)
submissionId: string; type: SubmissionType; status: SubmissionStatus
sections: SectionState[]; fields: Record<string, ReviewFieldVM>
rosters: Record<string, RosterRowVM[]> // pemegang saham / direksi / komisaris / kbli / BO
validationResults: ValidationResultVM[]; sectionApprovals: Record<string, boolean>
requiredSections: string[]; declarations: string[]
documents: DocumentVM[]; pdfSources: DocumentSource[]
diff?: DiffModel; registryEntity?: RegistryEntityVM
}
interface SectionSchema { key: string; label: string; approvable: boolean; required: boolean } // shared FE/BE — kills the section-key drift class
// ── OCR/LLM (Pillar 4) owns these ──────────────────────────────────────────
interface ExtractionProvider { readonly id: string; extract(ctx: ExtractionContext): Promise<ExtractionResult> }
interface ExtractionResult { fields: Record<string, unknown>; confidence: Record<string, number>; bboxes: FieldBBox[]; meta: { engine; model; processingTimeMs; pageCount; rawText? } }
async function processDocument(documentId: string, opts?: { classificationOverride?; aktaSubtype? }): Promise<ExtractionResult> // facade KEPT, now returns result so hooks get rawText
// ── Security (Pillar 6) + Contract (Pillar 2) share the request context ────
interface RequestContext { requestId: string; clientIp: string | null; userAgent: string | null; principal: RequestPrincipal; log: Logger }
interface RoutePolicy { auth: 'public'|'required'|'optional'; roles?: Role[]; resource?: 'submission'|'document'|'none'; rateLimit?: {windowMs;max}; upload?: {maxBytes;allowedMime[]} }
// ── ONE outbox for the future SABH write-back (Persistence owns; Security drains) ──
function enqueueOutbox(tx, args: { submissionId; targetSystem: 'SABH'; operation; contentVersion; payload }): Promise<void>
3. The six pillars — shape, and the calls that need your judgment
Pillar 1 — Flow Engine Core
Shape: FLOW_REGISTRY: Record<SubmissionType, FlowConfig> (total → compiler forces an entry per enum). dispatchProcessor(id,type) → runFlow(id, flowFor(type)); reMatchAndValidate(id) → reMatchAndValidateFlow(id, flowFor(type)). One registry, two consumers — this structurally kills the silent‑validation‑wipe class (there is no longer a second, un‑exhaustive dispatch to forget). The generic processor reproduces the PT‑akta Phase 1–5 skeleton, config‑gated (primaryAkta:null/companyLookup:false short‑circuit for non‑akta flows). Rule bodies are wrapped, not rewritten.
Collapses: akuisisi's 815‑line fork → ~15 LOC; the 10 copy‑pasted persist loops → 1 transactional persistValidationResults; the two dispatch tables → one.
Judgment calls:
- Context typing — FlowConfig<Ctx> generic (existential in the registry, typed within a flow via defineFlow<Ctx>()), not one mega‑context union (which would rebuild the 114‑line god‑interface we're killing).
- Legal rules — wrap during strangle (rules/ imports & calls the existing validateX from cross-validator.ts), relocate bodies only after parity proves out. Keeps the equivalence oracle honest (same function runs on both paths).
- Persist semantics — unify on notIn‑delete + override‑preserving upsert + transaction (fixes the override‑wipe uniformly) — but this is a behavioural change for flows that currently blanket‑delete; gated by the parity oracle with a blanketDelete opt‑out.
Pillar 2 — API Contract Spine
Shape: a repo‑root contract/ dir (zod data only, aliased @contract on both sides) is the SSOT. AppError + ErrorCode enum + global app.onError make the {error,code} envelope real; apiFetch upgrades to throw AppError preserving {code,blockers,…} so the FE can finally branch on failure class.
Collapses: 405 unchecked apiFetch<T> casts → per‑route validated calls; the 3 live enum drifts → gone (generated + parity‑tested); the ~1,900 LOC of hand‑mirrored FE types → @contract imports.
Judgment calls (this is the most opinionated pillar):
- Typed client — hand‑rolled RouteContract descriptors, explicitly rejecting Hono RPC (hc<AppType> blows up as "type instantiation excessively deep" across 30 routers × 751 c.json shapes, can't type the error union, and forces a big‑bang incompatible with strangler), ts‑rest (server adapter fights 495 hand‑written handlers), and OpenAPI‑gen (adds the codegen build step you want to avoid). Hand‑rolled gives per‑call generics (fast tsc), incremental adoption, no framework.
- Response validation — request always; response dev/test/CI‑only (gated by env) to avoid CPU cost on the large ReviewData hot path, while the FE apiClient still parses client‑side.
- Contract location — repo‑root contract/, not an npm workspace (no root package.json today; workspaces would change install/deploy for the whole PoC). Cost: one Vite fs.allow['..'], tsconfig paths both sides, relaxed backend rootDir — must be done properly, not @ts-ignore'd (see §7 medium).
Pillar 3 — Persistence & Data Model
Shape: thin Submission core + per‑family satellite tables (PtSubmission/PpSubmission/ApostilleSubmission) so required‑per‑flow fields become NOT NULL; typed JsonCodec<T> per JSON column kills the 107 as unknown as reach‑ins; one shared ReviewState HITL column‑set (repo‑enforced, schema‑linted). Migrations re‑baselined (db pull → 0000_baseline → migrate resolve --applied on every env → CI migrate diff --exit-code). One SubmissionOutbox seam for the future SABH push.
Collapses: the god‑table's ~40 nullable columns; the HITL quintet (×10 models) + raw‑OCR trio (×6); 6 forked HITL confirm paths → 1 repo.
Judgment calls:
- Don't collapse the 8 card + 7 akta‑role tables into an EAV/JSON mega‑table — that re‑imports the reach‑in problem this pillar exists to kill. Unify provenance + HITL only; keep genuinely‑typed columns typed.
- JSON columns — split by access pattern: branch‑on‑it data (attestations, selectedJenis) → normalize to tables; opaque external snapshots (oldData) → zod‑validated JSON (normalizing SABH's shape buys nothing).
- Bit‑for‑bit rule‑value preservation: lift money columns without re‑typing (Float vs string jumlahSetor) in the same migration, or you risk silently flipping a FAIL/PASS.
Pillar 4 — Frontend Review Architecture
Shape: one <ReviewEngine> shell (mode: author|verifier|readonly) fed by REGISTRY: Record<SubmissionType, FrontendReviewDescriptor> (pure data). Sections are a discriminated union of renderers (field-list | roster | attestation | registry-entity | diff | custom). One createReviewHook(endpoints) factory replaces the 8 copy‑pasted hooks. The de‑facto shared kit is promoted components/pendirian → components/review/* behind an ESLint‑enforced one‑way boundary (primitives may never import descriptors).
Collapses: 8 forked review pages (~9,400 LOC) → 1 shell + N descriptors; the byte‑identical 85‑line hook block; the 35 catalogued UI inconsistencies (one stepper source, one confidence table, buildPdfSources/ValidationRow/AttestationChecklist primitives).
Judgment calls:
- Full engine + descriptor registry, not a shared‑hook‑only refactor that keeps 8 thin pages — a hook doesn't stop pages re‑forking; a Record makes a fork a compile error.
- Perubahan diff is a DiffSection kind inside the one engine, not a second engine (shares ~90% of the skeleton) — but DiffSectionVM is accepted as the single hardest descriptor and migrated last.
- Verifikator + read‑only apostille become a mode axis, not forked pages.
- Delete the legacy VerificationPage/verification/* (~4,800 LOC) only after carving out the live document-preview viewer subtree it still owns.
Pillar 5 — OCR / LLM Service Boundary
Shape: ExtractionProvider strategy keyed by (docType, provider); one saveExtraction() owns the deleteMany/createMany/status scaffold (document-processor.ts shrinks 2,515 → ~700 LOC); one gpuJob<Req,Res>() submit/poll replaces ~8 hand‑rolled clients; gpu-server becomes one generic POST /extract/{name} driven by an ExtractorSpec registry (13 triples → 1). TS↔Python contract is generated from the pydantic models (gpu-contract.ts, CI diff). paddle-ocr-service kept verbatim.
Judgment calls:
- Contract — codegen TS from pydantic model_json_schema + CI regenerate‑and‑diff (turns the prose contract into a build failure on drift). Field‑level values stay guarded by the FlowRegistry's zod schemas.
- Determinism — temp 0 + seed for field extractors, prompt‑hash mandatory in the cache key (closes the 9‑of‑12 stale‑extraction gap); "reproducible" = "given the pinned server build" (assert vLLM digest via /info).
- saveExtraction — generic scaffold + small writeDomain hook (typed domain tables are heterogeneous; full reflection would be fragile).
Pillar 6 — Security & Cross‑Cutting Spine
Shape: installSecurity(app) — one call. Stateless JWT behind a TokenVerifier interface (LocalJwt now, SabhSsoVerifier later, selected by iss), data‑driven ROUTE_POLICIES with a CI exhaustiveness test (a mount with no policy fails the build), object‑level authz via Submission.ownerId/Document.ownerId (null‑owner falls back to role‑gating), upload guard, rate limit, PII lifecycle (AES‑GCM + retention), real liveness/readiness probes, and a sibling append‑only SecurityAuditEvent reusing the proven DB immutability trigger.
Rollout: shadow‑then‑flip per family behind SECURITY_ENFORCE (a false‑deny is a self‑inflicted outage; shadow logs prove coverage before any rejection).
Judgment calls:
- JWT + TokenVerifier over server‑side sessions (statelessness fits one on‑prem box; mirrors the codebase's own env‑selected‑engine pattern and the apostille redirect‑token direction) or static API keys.
- Principal‑derived data enters RuleContext via buildContext, never the principal object — so rules stay pure/testable even where apostille needs the account NIK (see §7 conflict #4).
4. Directory layout (target)
/ (repo root — NO workspaces; @contract alias only)
├── contract/ # SSOT: enums.gen.ts · errors.ts · http.ts · review.ts (normalised) · sections.ts · routes/*
├── backend/
│ ├── prisma/schema/ # split <200 LOC: core/pt/pp/apostille/review/audit/outbox.prisma
│ ├── prisma/migrations/ # RE-BASELINED: 0000_baseline (db pull) + incremental migrate deploy
│ └── src/
│ ├── flow-engine/ # registry · types(defineFlow) · rule · processor · rematch · validation-runner
│ │ ├── context/pt-akta-context.ts # was forked 3×
│ │ ├── rules/pt-akta/* # PT_AKTA_RULE_SET — 30 sync + 3 async validators WRAPPED (bodies verbatim)
│ │ ├── rules/{rups,akuisisi,pp-*,apostille}/*
│ │ └── flows/*.ts # one FlowConfig per SubmissionType (akuisisi ~15 LOC)
│ ├── persistence/ # submission-repo · review-repo(1 impl, was 6) · validation-repo · extraction-repo · outbox · codecs/*
│ ├── ocr/ # provider(kept) · gpu-job · gen/gpu-contract.ts · extraction/{provider,registry,*-provider} · service-contracts(/info)
│ ├── security/ # principal · token-verifier · authenticate · authorize · route-policies · resource-authz · upload-guard · rate-limit · pii-lifecycle · index(installSecurity)
│ ├── observability/ # logger · metrics · error-tracking
│ ├── health/probes.ts # real DB/GPU/external/disk (kills the static liar)
│ ├── middleware/request-context.ts # EXTENDED: +principal +log
│ ├── config.ts # EnvSchema.parse (fail-fast)
│ └── services/
│ ├── document-processor.ts # 2515 → ~700 (resolve→extract→saveExtraction)
│ ├── submission-dispatch.ts # thin: engine v2→runFlow else legacy fork (deleted at end)
│ └── *-processor.ts / *-cross-validator.ts # legacy forks = oracle, deleted flow-by-flow
├── frontend/src/
│ ├── review/ # ReviewEngine · SectionRenderer · createReviewHook · descriptors/{*,registry.ts}
│ ├── components/review/ # promoted PRIMITIVE kit (one-way boundary) + pdf/ (carved from verification/)
│ └── lib/{api-client(AppError) · auth · role(token-derived) · types → @contract shims then deleted}
├── gpu-server/app/ # extractors/{registry,runner} · queue(13→1 task) · main(/extract/{name},/info) · schemas(pydantic SoT → TS codegen)
└── paddle-ocr-service/ # KEPT verbatim
5. Add‑a‑flow walkthrough (end‑to‑end, target state)
- Persistence — add the enum value to
schema.prisma;db:generate. This regeneratescontract/enums.gen.tsand immediately makesFLOW_REGISTRY, the FEREGISTRY, and everyRecord<SubmissionType,…>a missing‑key compile error — adoption forced across three pillars from one edit. Reuses the family satellite → zero schema change; a genuinely new concept → +1 typed‑JSON column + codec + one additivemigrate deploy. - OCR — reuses an existing extractor → point
FlowConfigat itsextractorName, zero code. New document only →prompts/<name>.txt+ oneExtractorSpec(Python) + regenerategpu-contract.ts. - Flow Engine — one
FlowConfigviadefineFlow<Ctx>():rules: [...SHARED_RULE_SET, ...newRules], reusebuildPtAktaContext,rematch,hooksonly if net‑new behaviour. ~15–40 LOC. NewRulefiles only for genuinely‑new legal behaviour (akuisisi added exactly one: koran‑window). - Security — one
RoutePolicyonFlowConfig.policy. The policy‑exhaustiveness test fails CI if you forget it — an open route can't ship silently. Audit/logging/metrics: zero flow‑specific work. - API Contract — enum auto‑flows both sides (parity‑tested). Shared
ReviewData→ no new wire type; flow‑specific fields → oneRouteContract. - Routes — reused
routeFamily→ no route file.dispatchProcessor/reMatchAndValidateuntouched — both resolve the sameFlowConfig. - Frontend — one
descriptors/<flow>.tsx(pure data: sections reusing renderers, labels, terminal, endpoints). ~60–120 LOC.requiredSectionKeysderives from the sharedSectionSchema(no FE/BE drift). Add the registry key (compile‑forced). No new page, no new hook. - Tests — add
flow-parity.test.ts(old fork snapshot vsrunValidations(config)— deep‑equal on{ruleCode,status,message,details}). Enum‑parity, policy‑exhaustiveness, section‑schema‑lint already guard the seams.
6. Where config‑driven honestly breaks down (disclosed, not hidden)
These are the escape hatches — each is 1–3 tiny files, not a fork. This is the correct answer, not a defect:
- Apostille (KTP‑first, redirect‑token, two‑axis sworn‑translation, per‑doc manual‑confirm — auto‑confirm was deliberately removed) does not fit the PT‑akta Phase 1–5 skeleton → gets
customProcess?()that reuses the samerunValidations+persist+buildContextmachinery without bending the akta skeleton. "Generic" means shared runner/persist/context, not "one processor body fits all" (which would accrete boolean flags into a fork‑with‑ifs). - Peralihan PP→PT cross‑entity match → a cross‑entity
buildContext+ amatchAsalPpreMatch pre‑step. - Perubahan diff view → a
DiffSectionrenderer +DiffModelin the normalisedReviewData; the 3 detail modals become reusablecomponents/reviewprimitives. Migrated last, once the flat flows prove the engine.
7. The reconciliations the adversarial pass forced (judge the honesty here)
The pillars did not compose as first written. These are the real seams — resolved, but you should weigh them:
CRITICAL #1 — ReviewData was three incompatible shapes. The contract pillar assumed it could freeze today's document‑centric shape ({akta, identityDocuments, buktiSetor, …}) as a mechanical dedup; the frontend needs a normalised {sections, rosters, diff?} shape. Resolution: adopt the normalised ReviewData as canonical; Persistence.loadReview projects the akta/KTP/bukti‑setor data into sections+rosters. This makes the contract pillar's "net −400 LOC dedup" actually a re‑shape, and it's a hard synchronization point — Pillars 2/4/5 must co‑design the roster + diff VMs before any of them ships ReviewData code. This is the single most load‑bearing coupling in the whole plan.
CRITICAL #2 — FlowConfig.review: FrontendReviewDescriptor is physically impossible. The FE descriptor holds JSX + select() closures; a backend module can't import React (no shared package). Resolution: drop review from the backend FlowConfig. Backend publishes only serialisable SectionSchema[]; the FE descriptor registry lives entirely on the frontend, keyed by the same SubmissionType. Linkage is SectionSchema data via contract/. Both sides keep independent compiler‑forced Records.
HIGH — the generic runner must not serialise async rules. runAllValidations runs 3 SABH‑hitting rules via Promise.all; a naïve sequential runner triples validation latency on every field edit. → partition by rule.async, keep Promise.all, add a wall‑clock non‑regression assertion to the oracle.
HIGH — two outboxes (Persistence vs Security) for one SABH purpose → collapse to one (Persistence owns; Security owns only the drain worker's creds + the onFinalize wiring).
HIGH — typed‑JSON migration ordering. If Persistence drops oldData/changeTypeResult before Flow‑Engine's buildContext adopts the codec, the context silently breaks — and legal rules read from it. → codec + dual‑read first, buildContext switches, then drop the column (the drop makes any surviving raw access uncompilable).
MEDIUM — persist‑semantics change (blanket‑delete → notIn+preserve‑override) alters which stale validation rows survive → gated by the parity oracle with a blanketDelete opt‑out.
8. Migration sequence (cross‑pillar, reconciled)
- Persistence Phase‑0 FIRST (unblocks everything):
db pull→0000_baseline→migrate resolve --appliedon every env → deletedb pushfrom shared paths → CI drift gate. No behaviour change. - Contract Step‑0/1 (whole‑repo leverage, additive):
app.onError+jsonError, structured logger,apiFetch→throwsAppError,EnvSchema, stand upcontract/+ enum generator + parity test (fails on the 3 current drifts, forcing the fix). Every endpoint improves, zero route edits. - Security Phase‑0 (behind
SECURITY_ENFORCE=off):installSecurityin shadow (verify+log, never reject); real health probes; metrics. - Persistence Phase‑1 (additive tables, dual‑write): satellites,
DocumentExtraction, singleSubmissionOutbox,ownerIdcolumns,SecurityAuditEvent— all alongside existing columns; audit‑only data ⇒ zero backfill. - 🔴 CRITICAL SYNC POINT — co‑design
ReviewData(Pillars 2+4+5) before any of them writes read‑model code. - OCR Step‑1/2:
gpuJob()+gpu-contract.ts; thensaveExtraction()writing through the persistence dual‑write seam; carve per‑docType behindEXTRACTION_DISPATCH=legacy|registry. (Prompt‑hash‑mandatory cold‑invalidates the cache → deploy off‑peak.) - Flow‑Engine on the highest‑duplication pair (perubahan+akuisisi): add the engine as pure addition (
engine:'v1-fork'), author both configs, keep the 796+823‑LOC forks alive as the oracle, add parity tests, flip tov2-genericonce green. - Frontend: promote the kit (re‑export shims); build
ReviewEngine; migrate the simplest flow first (PpPembubaran463 LOC), keep old page as oracle; perubahan‑delta last. - Strangle remaining flows one at a time across all pillars (forks‑as‑oracle): pendirian‑pt → PP → perbaikan → peralihan → apostille last (the hardest fit).
- Security Phase‑4/5: flip
SECURITY_ENFORCEper family (lowest‑traffic first), watch shadow logs, then widen; delete the localStorage role. - Cleanup / compiler‑forced completion: drop legacy god‑table columns (surviving reach‑ins fail to compile), delete the dispatch switch + reMatch if‑chain, delete the legacy FE verification generation + old gpu‑server endpoints, flip
verifyServiceContractsto hard‑fail.
9. The calls I'd like your ruling on
- Contract mechanism — hand‑rolled
RouteContract+contract/dir (my rec), vs. bite the bullet on Hono RPC / ts‑rest / OpenAPI‑gen. This is the most reversible‑expensive decision. - Persistence depth — full satellite‑table split + typed‑JSON codecs now (my rec), vs. the cheaper "keep god‑table, add zod on JSON" (defers the structural fix but less migration risk).
ReviewDatanormalisation — accept that it's a co‑designed re‑shape on the critical path (my rec), vs. a thinner FE‑adapter approach that keeps the backend shape but leaves the FE collapse only partial.- Security scope for the PoC window — full Phase‑0 auth spine now (my rec, given the PDF‑by‑id leak), vs. a minimal token+CORS+PDF‑authz stopgap and defer RBAC/PII depth.
- Sequencing appetite — the whole 10‑step program, vs. lock only Phase 0–2 (safety scaffold + contract + shadow security) as the first executable plan and re‑judge.
10. Accepted decisions (ADR) — 2026‑07‑01
Status: accepted, planning phase — no code written. All §9 recommendations approved.
| # | Decision | Chosen | Rejected alternatives |
|---|---|---|---|
| 1 | Contract mechanism | Hand‑rolled RouteContract + repo‑root contract/ (zod SSOT, @contract alias) |
Hono RPC, ts‑rest, OpenAPI‑gen |
| 2 | Persistence depth | Thin Submission core + per‑family satellites + typed JsonCodec; migrations re‑baselined off db push |
keep god‑table + zod‑on‑JSON |
| 3 | ReviewData |
Normalised, co‑designed re‑shape (sections/rosters/diff) — treated as a critical sync point (Pillars 2+4+5 co‑design before any read‑model code) | freeze today's document‑centric shape; thin FE‑adapter |
| 4 | Security scope | Full Phase‑0 spine now (JWT + TokenVerifier, data‑driven ROUTE_POLICIES + CI exhaustiveness, object‑level PDF authz, shadow‑then‑flip) |
minimal token+CORS+PDF stopgap |
| 5 | Flow rules | Wrapped, not rewritten — bodies lifted bit‑for‑bit; fork kept live as equivalence oracle | re‑derive/clean up during port |
| 6 | Sequencing | Whole 10‑step program = roadmap; next executable slice = Phase 0–2 (migration baseline · contract spine · shadow security), re‑judge after | commit the full program up‑front |
Dev isolation: the architecture is in‑tree (new modules beside old — contract/, flow-engine/, persistence/, security/, …), not a new project directory. Implementation runs in a git worktree on a dedicated branch (see §11 rationale), with its own database name + dev ports so it can't disrupt other sessions sharing the dev Postgres / GPU box.
11. Release & scale — future state (queues, model management)
The system has two independent load axes, and each gets its own queue, owned by whichever service owns the scarce resource. Do not build these now — the seams below already exist in the design; at release you thicken them.
| Concern | Owner | Why | Seam already in this design |
|---|---|---|---|
| Inference queue + admission/batching | GPU / model‑mgmt service | GPU tokens/sec is the true bottleneck (prod = 16×B200). One gateway must own GPU admission across all AI engines (OCR is one of several: KI, chatbot). vLLM does batching; the gateway does admission + priority. | gpuJob<Req,Res>() (Pillar 5) — swap poll for a real broker/gateway without touching callers |
| Model routing / mixture‑by‑role + lifecycle | GPU / model‑mgmt service | Deciding 35B‑A3B (volume) / 122B (hard akta) / 397B‑VL (apostille tail), load/unload, versioning, the /info contract — this is model management; shared platform, not per‑app |
service-contracts.ts /info assertion (Pillar 5); ExtractionProvider id/model in meta |
| Durable submission workflow (saga) | OCR backend | Processing a submission is multi‑step business orchestration over the app's DB (classify → extract N docs → lookup → validate → ready), must be idempotent, retryable, restart‑surviving, progress‑reporting. Today it's fire‑and‑forget .catch(console.error) (a flagged gap). |
Flow‑engine phases (Pillar 1) become durable, persisted, idempotent steps — the phase structure is the seam |
| SABH write‑back queue | OCR backend | Transactional outbox is a data‑integrity pattern bound to the app DB; the drain worker authenticates to SABH (Security owns the creds) | SubmissionOutbox + enqueueOutbox + hooks.onFinalize (Pillar 3/6) — already in the schema |
| Edge / per‑user rate‑limit + admission | OCR backend (Security) | Protects the app and shields the GPU queue from floods; per‑account quotas are an app concern | RoutePolicy.rateLimit with a Redis‑backed interface designed now, in‑memory for PoC (Pillar 6) |
The contract between the two queues: the backend's durable submission jobs submit inference tasks to the GPU service's queue and await results via gpuJob. Backpressure flows backward — a saturated GPU gateway returns queue‑depth/429, and the backend applies admission control at its edge (Pillar 6) rather than piling work onto the GPU. Two queues, two owners, one contract.
So, to your question directly: the inference queue + model management belong in the GPU/model‑management service (and should become a shared platform across engines); the submission‑workflow queue and the SABH outbox belong in the backend. This is a post‑migration "release readiness" phase — the current plan only needs to preserve the seams (which it does), not build the queues.
Produced by a 7‑agent design workflow (6 pillar architects + adversarial integration), plus accepted‑decisions + release‑state added 2026‑07‑01. This document changes no code.