*** September 2026 Major Release ***Read and watch

What Happens When a Document AI System Encounters a Document It Was Not Trained On

Share On :

Template-based document AI has a hard boundary: documents inside the training distribution process correctly, documents outside it fail. In production the outside case is the normal condition, because document sources, vendor formats, and regulatory requirements change continuously.

How Template-Based Systems Fail

Template systems store extraction rules as field coordinates tied to specific layouts. A document that does not match fails in one of three ways.

  • Silent empty returns. Fields that cannot be located come back empty, the pipeline continues with missing data, and the failure stays invisible until something downstream breaks.
  • Wrong-field extraction. When a new layout places a different value near the coordinates a known field used to occupy, the system returns that value with no signal that it is wrong.
  • Hard rejection. The system refuses the document. This is the least harmful case because the failure is at least visible, though it still needs manual routing.

All three share a root cause: the system pattern-matches against stored templates and returns whatever the match produces, with no means of interpreting structure it was never given rules for.

How OCR-Plus-LLM Systems Fail

These stacks flatten the document to text before passing it to a language model, and flattening discards the structure the document encoded spatially: column boundaries, table cell relationships, form field positions.

The model then reconstructs that structure from flat text. On standard layouts with well-separated fields this works adequately. On merged tables, multi-column layouts, overlapping form fields, or mixed text and visual content, it produces confident-looking output that does not correspond to the source.

How ADE Parses Unseen Documents

ADE reads documents as visual systems rather than text streams, identifying structure geometrically. It handles any document layout without templates or training, so an unfamiliar format goes through the same parsing path as a familiar one.

Parsing returns reading-order Markdown plus a structure tree of typed blocks, and the type set is fixed rather than per-document: text, table, table_cell, figure, marginalia, attestation, logo, card, and scan_code. A vendor's first-ever invoice layout yields the same block types as their hundredth.

Every block carries grounding with its page, its range in the Markdown, and a bounding box in normalized page coordinates, so output from an unseen document is as traceable as output from a known one.

Extraction stays stable across formats because the schema names the fields you want and their types rather than where they sit. Adding a new source needs no extraction code change, and alternative names let one schema absorb the same field appearing under different headings across issuers.

What ADE Reports When It Cannot Parse Something

The more useful question is not whether a system ever fails, but whether it tells you. ADE reports failure explicitly at every level.

  • Some pages fail, the rest succeed. Parse returns HTTP 206 with the successful pages intact, lists the failed page numbers in metadata.failed_pages, and marks each failed page in the structure tree with status: "failed" and a reason. A document with three bad pages still delivers the rest.
  • Every page fails. Parse returns HTTP 500 with the full response shape, every page carrying its own status and reason.
  • The file is not supported. Validation failures return HTTP 422 with a stable code and a message naming the problem, so pipelines branch on the code rather than parsing strings.
  • A field is not in the document. Extraction returns null for primitives and an empty array for lists, with objects keeping their shape and null fields inside. That is documented behaviour rather than an error, and it is distinguishable from an extraction failure.
  • A value was inferred rather than read. When the model synthesizes a value instead of copying it from the source, both the value and its ranges are null, so inference is separable from transcription.
  • The schema asked for something unextractable. Extraction returns warnings and schema_violation_error alongside the partial result, or fails with 422 if you set options.strict to true.

Credits apply only to 200 and 206 responses, so a document ADE cannot process costs nothing.

Signals for Reviewing Unfamiliar Documents

  • Grounding points a reviewer at the exact page region and text span behind any value, so verifying an extraction from a new format takes seconds rather than a full read.
  • Atomic grounding narrows that further, to the visual line with DPT-3 Pro or the word with DPT-3 Fast.
  • Transcription confidence is available on DPT-3 Fast, which scores every word it transcribes in text, marginalia, and table cells from 0 to 1, rolling the lowest score up to the block and page. The documented response to a low score is to route that content to review or re-parse the document with DPT-3 Pro. DPT-3 Pro does not return confidence scores.
  • Your own validation rules carry the rest, checking extracted values against business logic and routing failures to review with grounding attached.

What This Means for Growing Document Sets

Production document sets grow more varied over time. A KYC workflow handling 20 counterparties in year one may handle 200 by year three, each bringing its own format.

A template system needs a new template per source. ADE runs all of them through the same pipeline with the same schema, and reports explicitly when a page, a file, or a field does not work out.

Validate a schema against a sample of the new format in the Schema Wizard or the Playground before committing it to production.

FAQ

What does parsing without training mean in practice?

ADE identifies document structure geometrically rather than by matching stored templates, so the same parsing path handles a known vendor invoice and an unknown one. Onboarding a new document type needs no template creation, no labeled examples, and no retraining.

How does a template system fail silently, and why does that matter?

A new layout either lacks a field at the expected coordinates or places a different value there. The system returns whatever sits at those coordinates and the pipeline continues, so downstream systems receive wrong or missing data with no error signal. The problem may not surface until it reaches a report, an audit, or a transaction.

Can ADE handle a brand-new document type with no setup?

Parsing needs none. Extraction needs a schema naming the fields you want, which does not require knowing where those fields appear in the new format.

What happens when a layout is genuinely ambiguous?

Fields ADE cannot locate return empty rather than guessed, values it inferred rather than read are marked as such, and grounding shows exactly where any returned value came from. Those three signals give a pipeline something concrete to route on instead of a value with unknown provenance.

Does handling unseen formats require an Enterprise plan?

No. Layout-agnostic parsing is a property of the model, not a plan tier. Enterprise plans add higher rate limits, VPC and on-premises deployment, SSO, SLAs, and dedicated support, as listed on the plans page.