On this page

You hand an AI agent a 42-page PDF and ask for the text of every page. It answers in smooth, confident prose. Is that text what the pages say, or what the agent made of them? And if you ask again tomorrow, do you get the same text?

Each reader read the same document three times (the mid-size model four times). On the 15 plain text pages every reader is close to perfect: 0.00 to 0.29 % of the words wrong. On the 27 slide pages, where the text sits in an image, the word error rate without word order is 1.8 to 3.1 % for the large model, 1.1 to 2.3 % for the mid-size model and 2.1 to 2.6 % for the OCR API. The numbers are where the readers part: out of 1,661 numbers in the document, the single-reading pipeline lost 71 and misread 52 with no hint, the large model lost 0 to 9 and misread 4 to 14. The API's single-reading pipeline is 15 to 29 times faster than the agents and puts a box and a confidence on every word. The agents differ from run to run; that pipeline did not (runs of 13 and 14 November 2026, folder blog-agentic-ocr-20261113/drei-laeufe).

Which means: on numbers the agents lead today, and the result must be read per page type and per kind of error, not as one average. It is a state of today and is measured again with every release.

Agentic OCR test: the document, the readers, the OCR API

The document is the public Q3 2026 earnings call transcript of Schaeffler AG, 42 pages, original PDF at schaeffler.com, retrieved 13 November 2026. The text belongs to Schaeffler; this post quotes single words for the comparison and shows no page. It suits the test because it has two kinds of page:

  • 15 pages of plain text (pages 2, 3 and 24 to 36): 5,211 words that are also in the text layer.
  • 27 pages with presentation slides (pages 1, 4 to 23, 37 to 42): 6,659 words of slide text that sit in an image, plus 47 words that exist only as a logo or title graphic. The text layer of the transcript holds none of them: reading only that text layer loses 6,706 of the 15,272 words.

The truth for the plain pages is the text layer, read with pdftotext -layout; why that tool is in the PDF renderer measurement. The truth for the slides is the text layer of the presentation that Schaeffler published with the call, checked against the page images; it differs from an earlier hand-typed truth in 6 of 6,706 words and in no number. Words that exist only as a graphic are counted apart and never as an invention.

Reader What it got
A Large general-purpose model, as an agent, 3 runs the 42 page images, no OCR tool, no text layer
B Mid-size model, as an agent, 4 runs the same
C The OCR API, pipeline with a single reading per word, 3 runs the PDF
D The OCR API, pipeline where a language model reads and a CTC engine checks every word, 3 runs the PDF

The agents got one task and no hint of what is measured: Read this document and write out the text of every page in full. Models are not named: the finding holds for the two classes, not for a brand. The API jobs ran one after the other, formats=json,md.

Does an LLM copy a page? The plain text pages

On the plain text pages all four readers copy. The share of wrong words is 0.00 % for the large model in all three runs, 0.08 % on average (0.00 to 0.29 %) for the mid-size model, and 0.06 % for each of the two API pipelines. An earlier single run on 13 November looked different: the large model returned about a fifth of the words, a summary, and noted a copyright notice on the slides as the reason. In none of the three runs here did that happen. Both are measured. The first shows that an agent can turn a copy into a summary without a word to the next step; only a count against the text layer shows it.

Where the readers differ: slide pages and numbers

42 pages, mean (range) of the runs Large model Mid-size model OCR API, single reading OCR API, language model plus CTC check
Wrong words without order, 27 slide pages 2.4 % (1.8 to 3.1) 1.6 % (1.1 to 2.3) 2.6 % (same in 3 runs) 2.1 % (2.1 to 2.1)
Wrong words without order, whole document 1.6 % (1.2 to 2.1) 1.1 % (0.8 to 1.6) 1.8 % (same in 3 runs) 1.4 % (1.4 to 1.4)
Numbers missing (of 1,661) 5 (0 to 9) 13 (0 to 48) 71 (same in 3 runs) 39 (37 to 40)
Wrong numbers with no hint 8 (4 to 14) 4 (0 to 7) 52 (same in 3 runs) 16 (16 to 17)
Wrong numbers that carry a hint 0 0 28 54 (53 to 55)
Seconds, whole document 583 (420 to 810) 299 (282 to 318) 20 (18 to 25) 92 (89 to 97)
Tokens used by the agent 118,000 (53,000 to 161,000) 106,000 (103,000 to 111,000) not applicable not applicable
Box, confidence, source per word no no yes yes

Runs of 13 and 14 November 2026, truth is the text layer of the transcript and of the presentation. "Wrong words without order" is the larger of the missing and the invented words divided by the words of the truth: it does not ask in which order the readers wrote a slide, because a slide has no fixed reading order. A "wrong number" is a number in the answer that the truth does not have. A "hint" is a flag that tells a person to look; only the API gives one. Seconds are for the whole task; the three runs of the large model ran at the same time as each other.

What the table says, without dressing it up: on numbers the agents are ahead today. In words the four readers are within one percentage point of each other on the slide pages; in numbers the single-reading pipeline loses 71 and misreads 52, where the large model loses 5 and misreads 8 on average. The API pipelines are far faster, and they hand over a box, a confidence and a source for every word, which the agents do not. In the language-model pipeline a wrong number carries a hint more often (54 of 70) than in the single-reading pipeline (28 of 80), so a person is sent to the place. That is not the same as being right.

Same document, same reader, a different answer?

Run to run, the agents move. The large model's share of wrong words on the slide pages lies between 1.8 and 3.1 %, its tokens between 53,000 and 161,000 and its time between 420 and 810 seconds. The mid-size model was steadier in time, 282 to 318 seconds, but one of its four runs missed 48 numbers and the other three at most 4. These are observations from three and four runs. They fit a reader that decides each time how to proceed; the cause was not tested.

The API pipelines are written to repeat (determinism). The single-reading pipeline returned the same words and the same confidences on all 42 pages in all three runs. The language-model pipeline did so on 41 of 42 pages in three runs; page 42, an organisation chart, came back with 411 words in one run and 412 in another. The repeat is not perfect where a language model reads a chart.

Bar chart of seconds for the 42 pages and tokens used per reader; the mean of the runs is a bar and each run is a dot.

Seconds and tokens per reader, mean and each run, runs of 13 and 14 November 2026. The API pipelines take 20 and 92 seconds, the agents 299 and 583 on average, and the large model's own runs span 420 to 810.

Bar chart of the share of wrong words without word order on the slide pages and of the wrong numbers with no hint per reader, mean and range of the runs.

Wrong words without order on the slide pages, and wrong numbers with no hint, per reader. The API pipelines show one value because their runs came back the same.

OCR confidence score and second reading: where the OCR API misread

On the 15 plain text pages the single-reading pipeline made three misreads in 5,211 words: AI as Al (page 28), quarter as guarter (page 31), frequently as freguently (page 35), at confidences 0.996, 0.994 and 0.951.

  1. In the single-reading pipeline, a confidence threshold alone would have caught none of them. The number says how sure the engine is, and an engine can be sure and wrong.
  2. The second reading catches them. In the language-model pipeline, 6 of 5,213 words on these pages are marked as unconfirmed, and they sit on exactly the three places above. A person who checks those 6 words has checked the page.

On the slide pages the language-model pipeline reports model_output_incomplete for 21 of the 42 pages: the model left out text that the detector found, and the service added the missing words with flag: not_in_model and verified: false. A model alone leaves such a gap without a word.

Test set and scope of the agentic OCR measurement

One English document, three runs per reader (four for the mid-size model). The 15 plain text pages have a full truth from the text layer. The slide truth comes from the text layer of the presentation. An earlier truth, typed with the readers' answers as an aid, gave the same numbers and the same word sets; only its word order came from the readers and favoured the large model, which is why this post gives no word error rate that depends on order. Slides have no fixed reading order, so which order counts is a choice, not a truth. Other users can load the API at any time, so its seconds move. The mix of plain and slide pages changes time and cost of your own stack, so run a sample of your own pages first: send a job and compare the words with a page you know.

Agentic OCR with checks: let the agent look only at shaky words

An agent that reads documents does not have to read every word again. The OCR API returns the text of every page with a box, a confidence and a source for every word (API guide). Install the MCP server for agents with pip install ocr-software-mcp. Its tool uncertain_words lists the words of a result with a confidence below 0.9 or a flag for a differing second reading: page, word, confidence, the second reading and the box. Applied to the language-model pipeline's result for the 15 plain text pages, that selection is 6 words in place of 5,213, on pages 28, 31 and 35; this is the tool's rule applied to the saved result, not a run of the server.

Which means: the agent keeps its strength, which is to understand a document and act on it. The OCR keeps the job of writing down what is on the page. And a person, or the agent itself, checks a short list instead of the whole result.