The box lever, measured on 500 clean officers
2026-07-15 · Apostille signature fast-track · tasks B + C of "let's do all lmao"
Phase K.24 · eval script sigtrain/b_tighten_eval.py · nothing shipped to the scoring path (by design)
TL;DR
Three numbers settle the auto-tighten question:
- The box lever is real — a perfectly tight box gains +6.6 AUC points / +9.0pp detect@p95 / +0.044 genuine cosine over the production loose box, on 500 clean officers. K.20's single-anchor finding holds at corpus scale.
- The CV auto-tightener captures none of it — a wash (pairwise win rate 45%, mean Δ −0.005). K.20's two anecdotal failures are now a measured, corpus-scale negative.
- The OCR-text-mask idea has a ceiling of about half the oracle gain — and that's with perfect text boxes. Real OCR only does worse, and costs seconds per verify.
Decision: ship nothing. The verifikator's redraw (with K.20's "gambar kotak serapat mungkin" guidance) remains the box lever. Also: task C (Bukti Setor rebuild) turned out to be already built — details at the end.
Why the old benchmark had to go
K.20 crowned the box "the real lever" from one case: the real Darsani anchor, where a hand-tight box moved the genuine specimen from #3 to #1. But K.23 proved that anchor is three-quarters contamination — of Darsani's four registry specimens, three carry other people's signatures. "Rank the genuine #1 against those distractors" was never a box test; it was an attribution test, and attribution now answers it deterministically.
The honest remaining question: on clean, correctly-attributed pairs, does a tighter box raise the genuine score?
The experiment
500 clean officers (from the 5,954-officer c6 corpus, excluding every officer production's SigNet-c5 was trained on, md5-ordered, fully deterministic). Production-faithful scoring: both sides through isolate_signature, dino_w = 0.3, the baked service weights. GPU 0 — GPU 1 was running Efran's training job untouched.
The loose box is simulated to K.20's measured geometry (detector box 393×249 around a 175×105 signature): a 2.2×2.4 canvas, the degraded signature off-centre, a printed name + NIP line inside the box — the real thing the detector hands the model.
(to_document)"] --> lb["loose_box()
2.2x canvas + printed name + NIP"] lb --> A["loose
isolate → embed
production today"] lb --> B["autotight
CV tighten → isolate → embed
box_tighten_probe"] lb --> C["textmask
tighten with TRUE text boxes erased
ceiling of the OCR-mask idea"] doc --> D["oracle
isolate the unpadded crop
hand-tight ceiling"]
The textmask row is the clever part: instead of running OCR on 500 synthetic crops (hours), the simulator knows where it drew the printed text — so erasing those boxes from the ink mask measures the best case of "use OCR word boxes to suppress printed text before tightening". Real OCR misses text; this doesn't.
Results
| row | AUC | det@p95 | gen_mean | pairwise win vs loose |
|---|---|---|---|---|
| loose (production) | 0.701 | 31.8% | 0.464 | — |
| autotight (CV) | 0.698 | 34.2% | 0.459 | 45% |
| textmask (OCR ceiling) | 0.739 | 37.2% | 0.480 | 51% |
| oracle (perfectly tight) | 0.767 | 40.8% | 0.508 | 68% |
And the guardrail that matters as much as the headline: applied to an already-tight specimen crop, the tightener hurts — xSHEET AUC drops 0.780 → 0.740, genuine mean 0.548 → 0.487. Auto-tighten must never run unconditionally; sometimes the detector's box is already right, and cutting into it amputates strokes.
What this means
- Don't ship the CV tightener. Its win rate is below a coin flip on the pairs it exists to help, and it actively damages the clean path. This is the third failure of pure-CV tightening (K.20 found two on real anchors) — but now it's a measurement, not an anecdote.
- Don't build the OCR-mask path either, for now. Its ceiling is +3.8 AUC points (half the oracle), bought with an OCR round-trip per verify. The measured best-case doesn't justify the latency and coupling; if that trade ever looks different, the number to beat is on record.
- The human redraw is the right affordance, and now we know its worth: a verifikator who draws a tight box buys up to +9pp detect / +0.044 genuine cosine on a hard pair. K.20's guidance text ("serapat mungkin — jangan sertakan stempel, nama tercetak, NIP") is not a nicety; it is the single highest-leverage user action in the flow.
Task C: the Bukti Setor rebuild was already built
The third task of the wave — rebuild the PENDIRIAN_PT review's Bukti Setor section — turned out to be done since 2026-07-10 (Phase H5, the same day Efran spec'd it): the PaddleOCR + grounded-3B setoran-facts pass fills BuktiSetorExtraction.transferFacts, and the review renders a "Daftar Setoran" roster with dynamic columns and the source doc in a Detail expander — no file/jenis listing. Re-verified live today on staging submission 341b7930: 18 setoran rows compiled across slip bank + rekening koran + SK bank, with bank / rekening penerima / jumlah / transaction id / tanggal transfer all populated.
The saved reminder memory still claimed it was open ("stopgap … needs an extractor pass"), which nearly caused this session to re-plan built work. The memory is corrected; what item 1 actually awaits is Efran's smoke test, same as item 2 (the SP Pendirian PP generator).
Eval deterministic end-to-end (md5 seeds, seeded k-means) · 500 officers, 2000 impostor pairs (detect@p95 SE ≈ 2.1pp) · GPU 1 untouched.