C5 — Signature matching: contamination filter + cross-domain model + calibration fix

Deployed: ahu-signature-verify v4 (sigverify-v4-ens-dinov2-signetc5) · verified live on submission e4229ea8 (Darsani). Efran: "spesimen matching is very important."

The problem C5 had to fix

Genuine Darsani's document signature scored ~4% against his real specimen under v3 — a genuine match reading as a non-match. Investigation found three distinct causes, and C5 addresses all three:

1. Registry contamination — many "Darsani" registry entries actually hold a different official's signature (the successor-attestation FORMULIR-02 files; flagged to AHU separately). Training on those creates false "same-person" pairs.
2. Domain gap — v3 was fine-tuned on clean specimen-vs-specimen pairs, but production compares a stamped, scanned document signature against a clean specimen. v3 never learned that shift.
3. Calibration bug — v3's score→% mapping was anchored on the cross-sheet distribution (easier), so it systematically under-scored the harder cross-domain task that production actually runs.

What C5 did

1. Self-consistency contamination filter (also quantifies the AHU issue)

For each officer with ≥2 registry files, embed all crops and keep only the largest mutually-consistent cluster (files whose signatures agree with each other); drop outliers. No OCR needed. Result across 2,149 multi-file officers:

• 758 (35%) had inconsistent files — hard evidence the wrong-person contamination is systemic, not a one-off.
• Clean positive set: 1,391 officers / 2,858 crops (metric learning only needs within-group same-person, so even a consistently-mislabeled cluster is a valid positive).

2. Synthetic document-domain augmentation

We can't get thousands of real document signatures, so we synthesize the document domain from clean specimens: a lighter-than-ink circular official stamp overlay, security-paper texture, ink/contrast jitter, geometric skew, and JPEG/scan softness — then re-isolate (exactly what production does to real documents). Injected into ~half of each training batch so the batch-hard triplets become clean↔document positives, teaching the invariance that matters.

3. Calibration anchored on the right distribution

v4's score→% knots are computed from the held-out cross-domain (document↔specimen) impostor percentiles, not cross-sheet. This alone lifts genuine matches from "looks like 4%" to a meaningful score, because it finally measures the task production actually performs.

Results (held-out frozen corpus — 95 officers, never trained on)

Metric	v3 (prod)	v4 (c5)
Cross-sheet AUC	0.754	0.760
Cross-domain AUC	0.727	0.743
Cross-domain genuine-detect @ imp-p95	35.8%	46.3% (+10.5 pts)
Self-pair sanity	100%	100%

The headline is the operating-point metric: at a fixed ~5% false-accept rate, v4 clears 46% of genuine cross-domain matches over the "cukup mirip" bar vs v3's 36%. Ensemble weighting was tested (50/50 vs dino-heavy vs dino-only); 50/50 won on the population.

Real Darsani anchor (his actual Kutipan document vs the genuine 2655 specimen, on the live submission): ~4% → 45.8%. A genuine match now sits right at the "cukup mirip, periksa manual" threshold instead of reading as a non-match.

Honest limits

• v4 improves the population modestly (+10.5pp detect) and fixes the calibration, but the synthetic augmentation can't fully close the pathological real gap — Darsani's genuine specimen (2655) is a 2008 rotated, faint scan, near the limit of what's recoverable. It lands at ~46%, not 80%+.
• The real lever remains real document-vs-specimen training pairs, which are now being accumulated as verifikators approve submissions under the mandatory signature-attestation checkbox. That data — real signings, real stamps, real scan conditions — is what will push the hard cases past 80%. C5's harness (signature-verify-service/sigtrain/c5_*.py) is ready to retrain on it.
• The AHU contamination fix (flagged separately) is the other half: while the registry serves successor signatures under retired officers' names, the right specimen (2655) must be manually picked. The candidate-ranking + "Spesimen ✓" chips already help the verifikator do that.

Artifacts

• Model: models-local/signet-ft.pth on the GPU host (c5 last.pth), backup valbox:~/signet-c5-20260714.pth. Baked into the image via the existing Dockerfile hook.
• Calibration: knots 0.409/0.561/0.625/0.741 ↔ 0/50/80/100% (cross-domain-anchored).
• Reproducible harness: sigtrain/c5_augment.py (doc augmentation), c5_clean.py (consistency filter), c5_train.py (cross-domain triplet), c5_eval.py (held-out + weighting sweep).