W3 report — shared GPU job client (Phase B) + CONVENTIONS §3.3 clocks

Status: DONE

Branch: refactor/tidyup-foundation. Worktree: /home/efran/remote-development/poc-ahu-ai/ahu-ocr-tidyup.

Commits

1. 31da028 — feat(gateway): shared GPU job submit/poll client (W3 Phase B foundation)
   New backend/src/lib/gpu-job-client.ts + backend/src/lib/__tests__/gpu-job-client.test.ts (26 tests, TDD — written and watched fail before the implementation existed).
2. 324f9d4 — refactor(gateway): migrate the llm/ job callers onto the shared GPU job client
ktp-cleanup.ts, override-scrutiny.ts, pendirian-pp-extract.ts, perbaikan-advisory.ts, surat-pernyataan-extract.ts + documentId threading at 3 call sites.
3. 42aa147 — refactor(gateway): migrate gpu-server.ts and azure-on-prem-bukti-setor.ts onto the shared job client
4. af966aa — refactor(gateway): migrate perbaikan-ai-summary-worker.ts and document-processor.ts's direct job fetches
   The 5 inline submit-poll blocks living directly in document-processor.ts (3 fail-soft LLM-cleanup paths + 2 throw-through extraction paths).
5. 5913c9b — refactor(gateway): domisili — §2 headers on the Azure DI seam, migrate its GPU-server legacy path
azure-on-prem-domisili.ts's two entry points, handled differently (see below).

Gate at every commit: ./node_modules/.bin/tsc --noEmit clean, full backend suite green.

Test summary

Full backend suite: 2056 pass / 1 skip / 0 fail (baseline 2030/1 + 26 new client tests), tsc --noEmit clean, at every commit above. No existing test asserted the old all-polls-count clock accounting, so no caller test needed clock-specific updates (there simply are no pre-existing tests for any of the migrated files or for document-processor.ts — confirmed by search before starting; this file has never had a dedicated test suite, and the 8 named llm/+ocr/ job callers had none either).

Deliverable 1 — backend/src/lib/gpu-job-client.ts

submitAndPollJob<T>(args): Promise<T> implements the exact interface from the brief plus a handful of additive, backward-compatible knobs:

• submitFailedVerb/pollFailedVerb ("submit"|"submission", "poll"|"polling") and unknownErrorText — let the client's generic error templates reproduce most callers' exact legacy wording without any caller-side code (see below).
• gpuQueueWaitMs — optional override of config.gpuQueueWaitMs, for test injectability.
• onQueueUpdate?(pos?, etaMs?) — surfaces queue_position/eta_ms from a queued poll body, per the brief (not wired to UI this wave).
• now?()/sleep?() — injectable clock/delay so the 26 tests never sleep real time.

§3.3 clock split (the one deliberate behavior change): two independent accumulators (processingElapsedMs, queueElapsedMs) are incremented by the real wall-clock delta (now() before/after each poll round-trip) attributed to whichever status (processing vs queued) was just observed. processingElapsedMs > args.processingTimeoutMs throws GpuJobTimeoutError; queueElapsedMs > gpuQueueWaitMs (default config.gpuQueueWaitMs, 1,800,000ms) throws GpuJobQueueWaitExceededError. Queued time never touches the processing clock and vice versa — tested independently (queued-only run with a tiny processingTimeoutMs never fires; mixed queued+processing run fires only once processing time itself exceeds the threshold; queue-only run capped by gpuQueueWaitMs fires independent of an enormous processingTimeoutMs).

JOB_UNKNOWN: a 404 with {code:"JOB_UNKNOWN"} triggers exactly one resubmit using the same deterministic Idempotency-Key (recomputed from the same documentId/stage/contentHash — no manual key storage needed), then polling continues against the new job_id. A second JOB_UNKNOWN for that resubmission throws GpuJobUnknownResubmitError (no loop). A plain 404 (no code field — today's gpu-server shape) throws GpuJobNotFoundError instead, preserving each caller's current "job not found" behavior.

Error-wording strategy (the interface's errorPrefix field): rather than hand-parsing which of the 8+ callers' message shapes the literal quoted template (`${errorPrefix} submit failed (${status}): ${text}`) exactly matches, I audited every current caller's exact string for all 6 failure categories (submit-failed, poll-failed, plain-404, completed-without-result, failed-status, timeout) and found:
- not-found and completed-without-result are worded identically (`${prefix} job ${id} not found` / `...completed without result`) across every one of the 8 named callers — the client's generic templates cover all of them with zero caller-side code.
- submit-failed/poll-failed split roughly 1-vs-6 between "submit"/"poll" (override-scrutiny) and "submission"/"polling" (the rest) — covered by the additive submitFailedVerb/pollFailedVerb knobs, again zero caller-side code for 7 of 8.
- failed-status's only variance across those 7 is the fallback text ("Unknown error" vs "unknown") — covered by unknownErrorText.
- Two callers (ocr/gpu-server.ts, ocr/azure-on-prem-bukti-setor.ts) use structurally different wording with an extra word baked in ("GPU server processing failed" / "GPU server processing timed out"; "GPU server Bukti Setor extraction failed (status): body" with no submit/submission keyword at all). For these two — plus a third caller my own reading turned up, processDomisiliLegacy inside azure-on-prem-domisili.ts, which mirrors bukti-setor's exact wording pattern and was missed by the wave's caller survey — I used the brief's explicit escape hatch: the client throws typed errors (GpuJobSubmitFailedError, GpuJobFailedError, GpuJobTimeoutError, all carrying status/statusText/bodyText/jobId/jobError/processingTimeoutMs) and each caller catches them in a local try/catch and re-throws its own byte-identical legacy string. gpu-server.ts additionally keeps its 413 (File too large) / 422 (GPU server validation error: <detail>) submit special-casing, reconstructed from the caught error's status/bodyText.
- The two throw-through blocks in document-processor.ts (BA RUPS, Jual-beli extraction) use yet another shape (`X submit failed: ${status} ${statusText}` — no response body at all, and timeouts reported in seconds via the old Math.ceil(timeoutMs/1000) formula) — same catch-and-remap treatment.

Deliverable 2 — caller migrations

All 10 named callers migrated, plus 2 the survey missed that I found while reading (processDomisiliLegacy, and confirming document-processor.ts's 5 direct fetch blocks are genuinely direct, not routed through GpuServerOcrProvider). Final sweep (grep -rn "jobs/\${" across backend/src, excluding tests and the client itself) turns up zero remaining unmigrated /jobs/:id poll call sites.

ocr/azure-on-prem-domisili.ts's processDomisili() is Azure DI's own operation-location submit+poll, not the gpu-server /jobs/:id shape — per the brief it stays out of the shared client. It gained the CONVENTIONS §2 header set (buildGatewayHeaders({traceId: currentTraceId(), priority: "batch"}), merged alongside its existing Content-Type/Ocp-Apim-Subscription-Key) on its submit call only; the operation-location poll loop is untouched, and a file comment marks its full gateway migration for the W4 Azure-DI seam.

Accepted deltas (all reviewed, none observable/tested)

1. Default 60s submit timeout added where none existed. 6 of the 8 named callers (all but gpu-server.ts and azure-on-prem-bukti-setor.ts, which already had one) had an unbounded submit fetch; the client's submitTimeoutMs defaults to 60,000ms per the brief. Same for the contact-info LLM-cleanup block in document-processor.ts, which also had no poll timeout. All of these are either short JSON POSTs to the same LAN GPU box, or (for contact-info) already fail-soft, so a 60s cap is very unlikely to ever fire and is a net robustness improvement, not a regression.
2. Debug-log fidelity loss on schema-validation failure. pendirian-pp-extract.ts and surat-pernyataan-extract.ts logged the GPU job's job_id alongside the raw payload when Zod validation failed; submitAndPollJob resolves to the bare result, not {jobId, result}, so that one log line no longer includes the id. Not user-facing, not tested.
3. BA RUPS / Jual-beli's unreachable "completed but falsy result" branch tightened. Both blocks previously had if (status.status === "completed" && status.result) {...} with no else — a job reported completed with a falsy result would silently keep polling until the outer timeout fired. The shared client's contract (matching all 8 other callers) throws GpuJobEmptyResultError immediately instead. This path has no test coverage and cannot be triggered by the real gpu-server (which always populates result on completed); it's a defensive-code tightening, not a loosening.
4. documentId threading. Added an optional trailing documentId param to cleanupKtpExtraction, extractPendirianPp, extractSuratPernyataanPerbaikan, GpuServerOcrProvider.process, processBuktiSetor, and processDomisiliLegacy, threaded through at every call site where it was already in local scope (all in document-processor.ts / perbaikan-processor.ts), for a more precise Idempotency-Key and future audit doc_hash linkage. Left unset for scrutinizeOverrides and generatePerbaikanAdvisory, which operate at submission scope (multiple documents' edits), not a single document — the idempotency key falls back to "no-doc", unchanged from what a bare stage-only key would give.
5. services/document-processor.ts line count. The architecture fitness test (file-size.test.ts) grandfathers this file at a 2564-line ceiling. My net edits (5 inline fetch/poll blocks replaced by much shorter submitAndPollJob calls, documentId threading) leave it at 2560 lines — under the ceiling, no ratchet violation.

Dormancy proof

gpu-job-client.ts never resolves against MODEL_GATEWAY_URL — submitUrl/pollUrlBase are exactly what each caller already resolves via config.gpuServerUrl today (Phase B's gateway routing flips in a later wave, per the plan). Captured against a mocked fetch, ktp-cleanup.ts's submit (GPU_SERVER_URL=http://localhost:8000, inside runWithTrace("demo-trace-id", ...), documentId: "doc-demo-1"):

URL: http://localhost:8000/ktp/cleanup          (unchanged — same URL as before migration)
method: POST
headers: {
  "Content-Type": "application/json",           (caller's own header, preserved)
  "X-Tenant-Id": "ahu-ocr",                      (additive)
  "X-Surface": "internal",                       (additive)
  "X-Request-Id": "demo-trace-id",               (additive)
  "X-Priority": "batch",                         (additive)
  "Idempotency-Key": "doc-demo-1:ktp-cleanup:<sha256 of the body>"   (additive)
}
body: {"raw_fields":{"nama":"Jane Doe"},"ocr_text":"raw ocr text here"}   (byte-identical to before)

and the following poll GET (http://localhost:8000/jobs/job-demo-1) carries no headers at all beyond the default AbortSignal — no Idempotency-Key, confirming polls never carry one. Today's gpu-server ignores unknown headers and never returns 404 {code:"JOB_UNKNOWN"}, so the resubmit path is inert; the only live behavior delta against the current production server is the §3.3 clock re-accounting (which is strictly more permissive during queued time and only starts a real processing clock once the server reports processing).

Concerns for the reviewer

• Two additional callers beyond the brief's named list were migrated (processDomisiliLegacy, plus confirming the 5 document-processor.ts direct blocks) — flagged prominently above in case the scope expansion should be reviewed separately.
• The catch-and-remap approach in gpu-server.ts/azure-on-prem-bukti-setor.ts/azure-on-prem-domisili.ts(processDomisiliLegacy)/document-processor.ts (BA RUPS, Jual-beli) is mechanical but adds boilerplate per file; worth a second pair of eyes on the exact string reconstructions since there is no test harness asserting those legacy strings (no pre-existing tests existed for any of these files to gate the migration).