The 63 invoices and purchase orders from the Legacy IDP, the same 11 fields and the same scoring. Seen: the documents the legacy rules were written on. Unseen: documents nobody wrote rules for. The full site (document viewer, how each pipeline works, live tries) is being built; this table is measured, not estimated.
| pipeline | runs on | fields right · seen | straight through · seen | fields right · unseen | straight through · unseen | time per document |
|---|---|---|---|---|---|---|
| Qwen3-VL-8B: image to JSON vision-language model | RunPod GPU | 91.5% | 29 / 45 | 92.4% | 8 / 18 | 3.0 s |
| PaddleOCR-VL-1.6 reads, Qwen3-VL-8B extracts document parser + LLM | RunPod GPU | 94.7% | 33 / 45 | 91.9% | 9 / 18 | 5.1 s |
| Qwen3-VL-4B (4-bit): image to JSON vision-language model | laptop GPU | 78.8% | 15 / 45 | 83.3% | 8 / 18 | 24.2 s |
| Qwen3-VL-2B: image to JSON vision-language model | laptop GPU | 72.5% | 8 / 45 | 77.3% | 5 / 18 | 18.0 s |
| PP-OCRv6 (PaddleOCR) + the same rules better OCR + rules | browser | 99.4% | 44 / 45 | 55.6% | 3 / 18 | 26.4 s |
| Legacy: TrOCR + 36 hand-written rules OCR + rules | browser | 94.9% | 35 / 45 | 55.1% | 3 / 18 | - |
| Legacy with a fine-tuned TrOCR + the same rules OCR + rules | browser | 85.7% | 9 / 45 | 51.0% | 0 / 18 | - |