Why We Use PaddleOCR — OCR Engine Selection Report

Audience: other engineering teams on the AHU AI platform who want to understand why this project's default OCR engine is PaddleOCR rather than Azure Document Intelligence, and where that default does and doesn't apply.
Basis: six benchmark studies run 2026-06-12 through 2026-06-29 (raw docs in docs/benchmarks/2026-06-*.md), consolidated here into a single decision record. Compiled 2026-07-13.

Decision

We use PaddleOCR (PP-OCRv5) as the default OCR engine, replacing Azure Document Intelligence, because it matched or beat Azure on every field that matters across every document type we tested, while being 1.6x to 6.6x faster and removing a licensed, non-on-prem dependency. That result held up not as a proxy OCR-text comparison, but on the metric that actually matters to us: whether real documents run through our production LLM extraction pipeline come out with the same field values. Azure is retained as a fallback and Azure-parity target, and two structure-aware engines (PaddleOCR-VL and MinerU2.5) are wired in behind config flags for the specific document types where PaddleOCR's plain-text output isn't good enough on its own. Nothing here is "PaddleOCR because it's the newest thing" — it's PaddleOCR because it won on our own documents, under our own extraction pipeline, and we adversarially audited the result before trusting it.

Why we needed to decide anything at all

The platform's on-prem/no-external-API constraint means a licensed cloud OCR container is a liability we want to shed if a local alternative is good enough. "Good enough" needed to be proven, not assumed, before we touched a pipeline that files real legal documents (akta notaris, KTP, NPWP, apostille/legalisasi paperwork) with AHU. So before flipping the OCR_PROVIDER switch, we ran a structured evaluation rather than a spot check.

The evidence

Speed and accuracy on the core document type (akta notaris)

The first and most important test used a real 31-page scanned deed with no text layer — a worst case, not a cherry-picked clean scan. We swept eight PaddleOCR model configurations against the production Azure Document Intelligence call path:

The winning configuration (v5_mobile_det detector + the Latin-script recognizer + document unwarping) reached the highest recall of any config tested while still running 1.6x faster than Azure; a speed-tier configuration without unwarping is 6.6x faster at some cost to recall on messy scans. Notably, the newer PP-OCRv6 model was disqualified outright — it was the only configuration that silently dropped a field we actually need.

Token recall against Azure's own output is only a proxy, though, so we ran the test that actually settles the question: the same documents' OCR text from each engine were fed through our real production LLM extraction pipeline, and the extracted field values were diffed, across 6 real documents:

On clean digital documents, extraction was identical to Azure's, field for field. On scanned documents, every high-value field survived intact — company name, notaris name, dates, capital amounts, decree numbers, shareholder and director counts. The only discrepancies were a partial suffix on one decree number and a single-digit difference in one document number, both attributable to ordinary OCR noise and no worse than mistakes Azure's own engine made on the same pages. One document failed on all three engines tested, including Azure — that turned out to be an unrelated bug in our extraction prompt, not an OCR difference, and doesn't bear on the engine choice.

Identity documents (KTP, NPWP)

For KTP and NPWP, Azure's advantage was a custom-trained extraction model, not general OCR, so this needed a separate test: can PaddleOCR's plain text, combined with the deterministic pattern-matching we already had in the codebase, replace that model entirely with no LLM involved at all? We compared field-by-field against production Azure output across 5 real KTP cards (~14 comparable fields each) and 8 real NPWP cards:

PaddleOCR tied or beat Azure on 52 of 66 comparable fields, and the 14 losses broke down into fixable causes rather than genuine OCR-quality gaps: our pattern-matching had been hand-tuned to Azure's exact line layout and needed retuning for PaddleOCR's reading order (accounting for 10 of the 14), with the remaining losses on a single low-quality scan. The ID number matched exactly on 4 of 5 cards, and address fields came through on every card where Azure's own model actually returned nothing at all. The comparison also surfaced that NPWP extraction was already effectively running without any Azure model in production, since that custom model wasn't even deployed.

Every other document type we handle

We extended the comparison to apostille/legalisasi paperwork (population records, education certificates, police clearance, ID cards) and to the full range of PT company-formation documents, adding three additional engines to the field: Azure's general-purpose reader, and two smaller vision-language OCR models (DeepSeek-OCR-2, Unlimited-OCR) that looked promising on paper.

Apostille/legalisasi documents — field recall by engine:

PT company-formation documents — field recall by engine (5 engines, incl. a 21-page akta):

Both tables tell the same story: on general-purpose document types, PaddleOCR is at parity with Azure on field-level accuracy while running 4-6x faster, and it holds that parity across every document family we handle. The two smaller vision-language models were ruled out outright: 7 to 30 times slower, prone to garbling dense or bilingual text (one model misread a signer's name entirely), and in several cases unable to finish processing certain document pages at all even after tuning. dots.ocr likewise failed in a way that matters for a production pipeline — not lower accuracy, but zero output on an entire document type. None of these four are part of our pipeline.

The one place PaddleOCR's plain-text output genuinely falls short is dense, multi-page structured documents — the notarial deed itself, and multi-column financial statements. PaddleOCR reads the same words everyone else does there, but dumps them as fragmented, unstructured lines rather than reconstructing paragraphs, section headings, or table cells:

Two purpose-built document-parsing models, PaddleOCR-VL and MinerU2.5, recover that structure at a real but bounded speed cost (roughly 1.3–7 seconds per page, versus PaddleOCR's under 1 second). We then tested whether that structure actually changes the accuracy of the values our extraction pipeline pulls out — because recovering nice-looking markdown is not the same as improving the answer:

The answer is nuanced: for a simple, single-grid table (the common case), plain PaddleOCR text was already 100% correct and the structured OCR added seven seconds of latency for no accuracy gain. For a genuinely multi-column layout such as a two-column balance sheet, plain PaddleOCR's flattening jumbled the numbers and our lightweight extraction model got every value wrong — so for that specific case, and only that case, the structure-aware engine is worth the cost. That's why our production OCR ensemble treats this as a routing decision by document type rather than a single global engine choice.

The validation step other teams should know about

Before trusting any of the above in production, we ran a dedicated adversarial correctness audit of the PaddleOCR migration branch, independent of the benchmark numbers. It caught a critical, silent data-corruption bug: the NPWP tax-ID pattern was truncating modern 16-digit IDs down to a malformed 15-digit number with no error or warning. It also caught six further defects, mostly around how confidence scoring and bounding-box placement interacted with our human-review gate — bugs that, if left in place, would have let bad OCR values slip past a reviewer undetected. Every one of those defects was fixed and re-verified before this engine choice was considered production-ready; the fix is traceable in our git history as a dedicated commit series culminating in KTP and NPWP extraction reaching 98% and 100% parity with Azure respectively. We mention this because "we benchmarked it and it won" is not, on its own, a reason to trust an OCR migration with legal documents — the audit is the part that makes the win trustworthy.

What this looks like in production today

PaddleOCR is the default engine. An OCR ensemble mode (env-gated, currently off by default) exists to run PaddleOCR alongside Azure and, when configured, PaddleOCR-VL and MinerU2.5, keeping PaddleOCR's coordinate boxes (which the rest of the pipeline depends on for highlighting fields in the reviewer UI) while selecting the best available reading-order text using reliability weights taken directly from these benchmarks. In other words, the "route by document type" conclusion above isn't just a recommendation sitting in a report — it's the actual selection logic wired into the code, gated behind configuration so it can be turned on per environment without a redeploy.

Sources

The full raw benchmark write-ups, including exact per-document numbers, reproduction instructions, and the complete audit findings, are preserved in docs/benchmarks/: 2026-06-12-paddleocr-vs-azure-di.md, 2026-06-13-no-llm-card-extraction.md, 2026-06-14-paddleocr-correctness-audit.md, 2026-06-29-ocr-engines-apostille.md, 2026-06-29-ocr-engines-pt.md, and 2026-06-29-table-extraction-accuracy.md. This report summarizes and cross-references them; teams that need the exact per-field breakdowns or want to reproduce a result should start there.