This post shares an end-to-end pipeline that turns a backlog of documents into query-ready rows in your warehouse — built on LandingAI's Agentic Document Extraction (ADE) Gen2 with the new DPT-3 models and streaming ingestion straight into Snowflake. It pairs layout-aware parsing and zero-shot, visually grounded extraction with a high, continuous insertion rate, so results land in your tables the whole time a batch is running rather than all at the end.
Invoices are the worked example, but the pattern is document-agnostic: swap the schema and the same flow handles bills of lading, statements, forms, or anything else.
Starter code:
Workflows/Snowflake/High_Volume_ADE_with_DPT3_to_Snowflake_DPT3
Video Walkthrough
Watch the pipeline parse a folder of documents and stream the rows into Snowflake in real time:
The document understanding problem
In enterprise environments, unstructured documents — scanned invoices, forms, certificates, and contracts — pile up faster than teams can process them. Reviewing them by hand is slow and error-prone, and the usual template-and-OCR approach breaks the moment a layout changes. What organizations actually need is true document understanding: a system that reads a page the way a person does, adapts to new formats without retraining, and hands off clean, structured data to the systems that run the business.
That's what Agentic Document Extraction Gen2 delivers. ADE Gen2 pairs Parse v2 with Extract v2, and DPT-3 — LandingAI's latest generation of document-native vision models — is the engine behind Parse v2: it reads the raw document and produces clean, structured layout. Extract v2 then works over that structure to pull the fields you ask for. No templates, no training, just accurate structure out of any layout. ADE Gen2 is built from the ground up to read a document the way a careful person does: starting with the overall layout and working down to individual lines and table cells. Two things it does better than the previous generation matter especially for a warehouse pipeline:
- Line-level grounding. Citations are anchored to the smallest structural units of a document — down to the individual line, each with its own bounding box — rather than to a whole page or paragraph. Every value you load into Snowflake can be traced back to the exact line it came from.
- A cleaner output structure. DPT-3 returns a consistent hierarchy of pages → blocks → lines, with tables delivered as HTML so the row-and-column structure that plain text flattens is preserved. The output is more compact and predictable, which means less cleanup before it becomes tidy warehouse rows.
A turnkey pipeline built for high volume
Organizations in financial services, insurance, healthcare, legal, energy and utilities, and logistics handle thousands of documents a day — vendor invoices, forms, certificates, contracts, and applications — and processing them takes more than an API call in a loop. This pipeline is built for performance, accuracy, and scale from day one: ADE Gen2 parses documents with DPT-3 and extracts your fields across many documents at once, and each document's results are streamed into Snowflake the moment it's ready. Your warehouse fills continuously instead of waiting for the whole batch to finish, so a folder of scanned PDFs becomes rows you can query in near real time.
Layout-aware extraction with line-level citation
Every extracted field is visually grounded with pixel coordinates and page references — and because DPT-3's parse output is line-level, that grounding traces back to the exact line and bounding box a value came from rather than a whole page or paragraph. That precision is what makes auditability real in regulated industries like finance, insurance, and healthcare, and in human-in-the-loop review workflows, where "where did this number come from?" needs an exact answer. Because Extract v2 is zero-shot, you define what you want in a simple schema and ADE Gen2 fills it in — highly accurate field extraction with no layout-specific training and no per-template tuning.
You can prototype a schema and see grounded results on your own documents in the Visual Playground before wiring anything up.
Optimized for scale: streaming into Snowflake
The pipeline is designed to scale from day one. It supports mini-batching, efficient file staging, and parallel Snowflake insertions so large volumes don't overwhelm resources — using Snowflake's high-throughput load path, staged files bulk-loaded rather than inserted row by row, and streaming each document as soon as it's parsed. The result is a steady, continuous insertion rate that keeps both the extractor and the warehouse busy the entire run, instead of leaving one idle while the other works.
Ready for the full Snowflake ecosystem
Once the data lands, it's just Snowflake. Header fields, line items, parsed blocks, and full document markdown arrive in governed warehouse tables, ready for downstream joins, table enrichment, BI dashboards, and alerts. Because everything is native SQL, it plugs straight into the rest of the Snowflake ecosystem — Streams and Tasks, Snowpark, and the BI tools your team already uses — moving you from raw pixels to governed tables in minutes.
Fully customizable for your workflows
The repo is a flexible starting point, not a black box. It follows a modular Python structure — with field mapping and validation, retry handling, and logging — making it easy to integrate with existing ETL frameworks or trigger via scheduled tasks. Point it at your own documents, define a schema for your fields, and tailor the entire flow to your document types and governance requirements. New users get complimentary credits to test on real documents.
With ADE Gen2 — DPT-3 parsing plus zero-shot extraction — and streaming Snowflake insertion, teams get a scalable toolkit to automate document ingestion and unlock structured, queryable data at scale.
Get the full pipeline on GitHub: ade-sample-projects → High_Volume_ADE_with_DPT3_to_Snowflake_DPT3.