Extraction for RAG and LLM Context Pipelines

Prepare PDF content for a retrieval-augmented generation (RAG) pipeline in this order: parse the file, recover its structure, chunk along that structure, then embed. Structure recovery comes before chunking because it decides whether a table stays whole and whether a heading stays attached to its paragraphs. Smart Data Extraction recovers headings, lists, tables and reading order on a deterministic foundation, so the same PDF returns the same structure on every run.

Export the result as Markdown for your LLM context window, or as JSON with page numbers and bounding boxes when a retrieved value must trace back to its page. Your own code then chunks the output and hands it to your own embedding model and vector store. Extraction runs on your servers and sends no document content to Apryse or any third party.

Naive extraction reads a PDF as a stream of text and discards the document's structure. A PDF describes where to draw characters on a page. It does not describe headings, table rows or paragraphs unless the file is tagged, so a text-only pass returns characters in the order the file drew them. The page on document parsing covers that first stage on its own.

Most teams start by splitting that text into fixed-size chunks. Here's where this goes wrong.

A size limit knows nothing about rows. When the limit falls inside a table, the first chunk keeps the column names and the second keeps orphaned values. A query about Tier 3 then retrieves a row of numbers with nothing to tell the model that 60 means net days.

The example below packs the flat text of one page from a vendor payment report into chunks of at most 200 characters, breaking only between lines.

Flat text from page 4, packed into chunks of at most 200 characters

Chunk 2 opens on Tier 3 with no column names. It also carries the page footer, the repeated page header and the next section's heading, none of which belong to section 3.2.

Text extraction that follows drawing order reads straight across a two-column page. A sentence from the left column continues into the right column, and a table cell lands between two paragraphs. The embedding of that chunk describes neither passage.

Running headers and footers repeat on every page. Flat extraction places them between body paragraphs, so a footer such as Page 4 of 12 sits in the middle of a section and gets embedded with it. Every chunk that spans a page break carries boilerplate that competes with the content.

Retrieval quality changes because chunk boundaries follow the document's own boundaries. Document structure recovery returns the logical structure of a PDF: headings, paragraphs, lists, tables, images, headers and footers, with reading order across them. Your chunker then splits at a heading, keeps a table whole and drops a footer before anything is embedded. In an AI document extraction pipeline, that decides what the embedding model sees.

The foundation is deterministic: the same PDF produces the same structure on every run. A general-purpose model that re-derives the layout from page content on each run offers no such guarantee. Its output varies between runs and degrades on multi-column, nested and long-form documents. With a deterministic foundation, re-ingesting an unchanged document yields identical chunks, so your pipeline can skip anything that has not changed.

Purpose-built AI works on top of that foundation for questions structure alone cannot answer. Document classification assigns each page one of 24 predefined categories, so your pipeline can pick a chunking strategy per page type. Key-value extraction identifies pairs such as an invoice number and its value in documents with no form layout, and form field identification detects fields in flat PDFs. Both return structured data you can attach to chunks as metadata.

Form field identification, key-value extraction, CAD title-block extraction and document classification each return a confidence score with every result. Table extraction does not return a confidence score. Your pipeline owns the thresholds and the routing of low-scoring results.

Markdown output is generated from the document structure that Smart Data Extraction recovers, not from flat text. A heading in the Markdown is a heading in the PDF, not a short line that happens to sit above a paragraph. That is why headings, lists, tables and reading order carry through to the format your RAG and LLM pipelines ingest.

The Markdown preserves:

For image-based PDFs, hidden text is reprocessed with OCR before the Markdown export, and the OCR Module needs its own add-on license. The sample below shows the shape of the output for the same page 4 used above.

Illustrative Markdown for page 4 of the vendor payment report

The table keeps its header row, and the heading sits directly above it. Markdown carries structure and order. It does not carry positions on the page, which is the job of the JSON.

Use JSON when a retrieved answer must point back to the page it came from. Smart Data Extraction exports JSON with page numbers and bounding boxes, built for overlay or annotation use. Your chunker copies the page number and box onto each chunk record. Your app can then open the source page and draw the box when a user asks where an answer came from. To see that output in full, read how to convert PDF content into structured JSON.

The record below is one your pipeline assembles for the table above. The field names belong to your record. The page number and box values come from the Smart Data Extraction JSON for the same table, and the numbers shown are illustrative.

A chunk record your pipeline assembles, with page and bounding box from the JSON

Both formats describe the same recovered structure. The table in the Markdown and the box in the JSON are one element seen two ways. Send the Markdown to the model and keep the JSON beside it for provenance.

The order is parse, structure, chunk, embed. Each stage consumes the output of the one before it, so an error early in the sequence reaches your index intact.

Open the file and read what is on each page. If you embed this output directly, every failure in the naive example applies.

Smart Data Extraction turns page content into headings, lists, tables and reading order, and exports Markdown or JSON. Run it before chunking. If you chunk first, the boundary cuts a table and the damage is already in your data.

Your code splits along the recovered structure, at headings, with each table kept whole. This is the stage you write.

Your embedding model turns each chunk into a vector, and your vector store indexes it. If you embed before chunking, each vector averages a whole document and retrieval returns files instead of passages. Apryse produces the structured input and nothing further.

The script below is the chunk stage. It splits the Markdown at headings, keeps each table whole and carries the page number onto every chunk. It needs only the Python standard library. Setup, installation and the extraction calls live in the Smart Data Extraction workflow guide.

Chunk stage: split Markdown at headings and keep every table whole

Output for the Markdown above

Chunk 1 keeps the heading, the intro sentence and all four rows with their column names. A query about Tier 3 now retrieves the label for 60 along with it. A table larger than MAX_CHARS becomes its own chunk. If your embedding model cannot take it, split by rows and repeat the header row in each piece.

Token cost is an implication of how you extract, not part of the RAG argument. The retrieval case above holds at any price. The cost case holds only if you pay per token for a hosted model.

Retrieved chunks become input tokens on every query. Structured chunks carry the section that answers the question, with headings and tables written as compact Markdown. Flat text carries repeated headers, footers and unrelated rows into the same prompt. If you pay per token, that difference grows with your query volume.

Apryse does not supply a cost figure for your pipeline. Token counts depend on your tokenizer, your chunk size and your model's rate, so measure them on your own documents. The post on the LLM token cost of document extraction compares extraction approaches and what each costs. The capability page covers how to cut LLM token costs.

Smart Data Extraction runs inside your own infrastructure on Windows or Linux, x64 only. It processes documents locally on CPU, and no GPU is required. Document content is not transmitted to Apryse or any third party during operation.

The extraction step writes Markdown or JSON to storage you control. Any call to a model is code your pipeline adds, to a model you choose and host where you choose.

Apryse supplies building blocks for an intelligent document processing (IDP) workflow that your team owns. It is not a turnkey IDP platform. The table shows where each stage sits.

Install the add-on into the language binding you already use.

For other language bindings, start from the Server SDK get-started guide.