Specimen Matching — Robustness Check + Crash Fix
Date: 2026-07-14 · Deploy: ahu-signature-verify rebuilt (content-sniffing _to_images) · commit aaee38f6
Efran: "ensure spesimen matching works, maybe try altering the images (flip, rotate, stretch, etc.) too."
I validated the live matcher on the real Darsani case (doc b01bb469 vs the 4 "Darsani" registry specimens), stress-tested it with image transforms, and fixed a real crash found along the way.
1. Does matching work? Yes — directionally, as a weak advisory signal
Document signature vs each real specimen (/verify, v4 model):
| specimen | who it really is | match % |
|---|---|---|
| 2655 | genuine Darsani | 43.2% |
| 290994327 | Sari Nurlistiana (successor) | 38.4% |
| 291063455 | Sari Nurlistiana (successor) | 29.9% |
| 291092585 | Galih Yudha Praja (3rd person) | 53.5% ⚠️ |
The genuine 2655 beats the two contaminated Nurlistiana files — but the wrong-person cover-letter sheet (291092585) outscores it. So the AI cannot be trusted to auto-rank the genuine specimen. This is the known cross-domain limit (a stamped ID-document signature vs a clean specimen sheet is the model's hardest case). It's exactly why the design keeps the score advisory and puts the real decision in the verifikator's hands — the specimen scroll, the doc-relative crops, and the manual verdict. On 291092585 the verifikator sees "Galih Yudha Praja" and rejects it regardless of the score.
2. Is it robust to altered images? Yes — for realistic scan variations
Genuine 2655 vs contaminated 290994327, each transformed the same way (raw sheet → /verify; margin = genuine − contaminated, higher is better discrimination):
| transform | genuine | contaminated | margin |
|---|---|---|---|
| identity | 58.7% | 39.9% | +18.8 |
| rotate 90° / 180° | ~57% | ~40% | +13 / +19 |
| skew ±10° | 53.6% | 21.0% | +32.6 |
| stretch W / H ×1.4 | 46–72% | 27–34% | +19 / +38 |
| brightness ×1.3 / ×0.7 | 33–57% | 15–49% | +8 / +18 |
| gaussian noise σ14 | 47.1% | 50.0% | −2.9 ✗ |
| horizontal mirror | 22.9% | 47.8% | −24.9 ✗ |
Robust to rotation, skew, stretch, and brightness — the genuine-vs-contaminated margin survives (the pipeline isolates strokes, tries 4 rotations, and scores with a dinov2+SigNet ensemble). The two failures are heavy noise (real but unlikely at that level; isolate_signature already drops mild noise) and horizontal mirror — which never happens in a scan, so I deliberately did not add mirror test-time augmentation (it would only give impostors extra chances to match and hurt discrimination, as the −24.9 row shows).
3. Crash fixed: mislabeled registry files
While testing, 291092585 returned SERVICE_ERROR. Cause: it's a JPEG served with a .pdf filename, and _to_images trusted the extension → poppler tried to parse a JPEG as PDF → PDFPageCountError crashed the whole comparison (so its doc-relative crop 404'd too). Fix (aaee38f6): sniff the magic bytes — %PDF- → PDF, otherwise open as an image, with a name-based PDF fallback only if the image decode fails. 291092585 now verifies and crops (status OK). Some registry files are mislabeled, so this hardens every comparison, not just this one.
Bottom line
- Matching works and is robust to real scan transforms — good enough to be a helpful advisory nudge.
- It is not reliable enough to auto-select the genuine specimen (cross-domain scores are weak; a wrong-person sheet can win). The verifikator's visual comparison + manual verdict stay authoritative — which is the design.
- The durable lever for stronger matching is more real doc-vs-specimen training pairs (accumulating via the attestation flow), not test-time image tricks.
Optional next step: a corpus-wide robustness eval (all 520 frozen-corpus officers, not just Darsani) would turn the N=1 findings above into population numbers — it needs a GPU slot (both H100s are currently saturated by the training job).