OCR pipeline models: reader, orientation, layout, tables, image-to-text

On this page

• Text reader: word bounding boxes and a confidence score for every word
• Page orientation classifier: rotated scans turned upright before OCR
• Layout analysis: regions, tables and figures found on the page
• Table structure recognition: rows, columns and spans for table extraction
• Image-to-text model: writes the page, the CTC text reader checks every word
• Rendering resolution (dpi) behind the measurements
• Sources

Which model touched your page, and where can it be wrong? Five models sit inside the pipelines. Four find or read what is printed: a turn, regions, lines, a grid. Only the fifth, the image-to-text model, writes text, so it is the one that can invent words; the combination catches that with a second reading. Small test sets, one test GPU.

Which means: when a result is wrong, you know which step to look at.

Which model sits in which pipeline: all pipelines.

Text reader: word bounding boxes and a confidence score for every word

It is the OCR engine behind the fast pipelines. A detector finds the text lines, a CTC OCR engine reads each line character by character, and from its frames a box and a confidence are derived for every word. It writes only what the line image shows; a character can still be misread. In the other pipelines it is the base: it gives the lines for tables, the boxes and the confidence for the combination, and the second reading that flags a word when the image-to-text model reads it differently.

Measured:

• Word boxes of the text reader: on 6 pages with 1,411 words, 98.5 % of the boxes overlap the true word by at least half along the line (word-box test).
• One test GPU, 1 worker, 200 dpi: 4,216 pages/h; with 4 processes 11,662 (throughput test). Neither number is a promise for your volume.
• Same digits with 4 workers as with 1 worker on all pages of that test-bench run.

Where it is weak:

• On the ten hard pages (hardest-pages test) it found 105 of 135 focus values (pages 01 to 06) and read 5 of them wrongly, without a warning. Character error rate, mean of pages 07 to 10: 0.461, against the text layer of those pages. Those wrong values are the ones the second reading must catch.
• Superscripts: a footnote number after a year (2021 followed by a raised 1) comes out as 20211, with an unremarkable confidence of 0.906 (one of the ten hard pages).
• Box shape: the 2D overlap of word boxes is lower than along the line (median 0.735 against 0.944). Digits: 19 of 477 numeric words are off in position, 18 of them on one page whose reference boxes are about seven times wider than the ink (word-box test).
• Rotated pages are read as delivered; the turn is the next model's job.

Licence: published by the authors with the model; texts and notices are under third-party software.

Page orientation classifier: rotated scans turned upright before OCR

A small model looks at the page and proposes whether it is upright or turned by 90, 180 or 270 degrees; its input is the page image alone. The proposal is checked: the page is read as delivered and turned, and the reading with the better page score stays. If there is no proposal, the page is read exactly as by the plain text reader. The boxes of a turned page are calculated back into the frame of the page as delivered, so they fit your original. Where the need showed up: rotated pages and word boxes.

Measured (rotation test: 80 public pages, each in four turns, 350 images in the run, rendered at 200 dpi, reference engine):

• 231 of 240 turned images got the right turn proposed; where the turn was applied at score threshold 0.8, 223 of 240 came back upright.
• 30 of 30 slightly skewed images (1 to 3 degrees) were never turned.
• 77 of 77 pages without a proposal gave bit-identical text to the plain reading.
• Cost of the check: about +88 % time (+1.6 s), only on pages where a turn was proposed. Memory peak 4,098 MiB.

Where it is weak:

• 8 of the 240 turned images were never proposed (thin pages), so they stay turned.
• 1 of 80 upright pages was turned wrongly; the check catches most wrong proposals.

Licence: model card and code of the authors; notices under third-party software.

Layout analysis: regions, tables and figures found on the page

A layout detector. It draws boxes around the regions of a page and gives each a class: title, text, table, figure and so on. Its output is regions and classes. In the table pipeline every region of the class table goes on to the table engine; a table the detector misses is missing from the result.

Measured:

• Layout test, ten pages of the smoke set: precision 0.621, recall 0.610, F1 0.615. 75 ms per page on the GPU, memory peak 704 MiB, model file 130.4 MB.
• Tables found: 22 of 35 test tables, 18 of 23 digital ones (table test).
• In the pipeline it adds about 0.075 s per page (pipeline test).

Where it is weak:

• It misses tables: only 4 of 12 on scans. With its boxes instead of the true box, the table score falls from 0.763 to 0.532 (table test).
• On all 20 pages including the hard ones, F1 is 0.605 (layout test).

Licence: tag of the model as published by its authors; notices under third-party software.

Table structure recognition: rows, columns and spans for table extraction

A table-structure model. Given the box of a table, it finds the rows, columns and spans of the grid. The words in the cells come from the text reader. A table with a broken grid (overlap, a cell outside the table) makes only that page fail, and such a page costs nothing. The post on which tools keep the grid shows what the cells look like.

Measured (table test, 35 test tables, with the true table box):

• Table score (TEDS, 0 to 1: how closely the returned table matches the true one, in grid and cell text) 0.763; digital tables 0.838, scans 0.617; structure alone 0.814. A repeat run differs by at most 0.006.
• 732 ms per table (median), +244 MiB memory over idle, peak 1,506 MiB.

Where it is weak:

• Behind the layout engine the score is 0.532 (digital 0.675, scans 0.257), because tables it does not find are missing.
• Scans: cell digits at the right position only 0.121 (as a set of values 0.559).

Licence: model card of the authors and package metadata; notices under third-party software.

Image-to-text model: writes the page, the CTC text reader checks every word

This is the vision-language model behind the language-model pipeline. It looks at the whole page image and writes the text, tables included, and that text is taken word for word. Boxes and confidence come from the text reader. Where the two read a word differently, the word is flagged and both readings stay in the result. Because the model writes text, the pipeline is marked as one that can invent text; the flag is the answer to that. How it reads spot by spot, next to six other models: seven OCR models on the same hard pages.

Measured:

• Ten hard pages, degraded copies of public pages (hardest-pages test): 130 of 135 focus values found (pages 01 to 06), 0 wrong; numbers on pages 07 to 10: 418 of 425, 0 wrong. Character error rate, mean of pages 07 to 10: 0.334, against the text layer of those pages.
• In the combination with the second OCR engine (offline test, 20 public pages, 4,406 words): 99.4 % of the words get a box, and the text of the model was changed in 0 blocks.
• Speed of the model alone with a serving engine: 375, 1,740 and 3,267 pages/h with 1, 8 and 32 pages at once. The whole combination: 2,086 pages/h at 32 at once and 3,157 pages/h at 64 at once (200 requests in 228 s; runs of 2026-11-06 on the test GPU).

Where it is weak:

• It changes accents: a Ě becomes Ė, a Ć becomes Č (two public pages, shown in the post on seven models).
• Word positions come from the text reader: the model's own spotting gave 533 entries on ten pages, blocks rather than words (own-spotting test).
• Text variation under load. One page at a time, the model's text repeats exactly. Run alone at temperature 0, 6 to 11 % of the pages were text-identical to the run with parallelism 1. In the combination, 20 of 20 pages were identical one after another, and 3 of 100 pages differed in their words at 32 pages at once and 16 of 200 at 64 at once; counting only lines, words and characters, 2 of 100 and 9 of 200 differ. Details: reproducible OCR and batch size.
• On a separate run over 300 diverse pages (a different prompt, so not comparable with the figures above), 3 of 100 pages from scientific papers ended in a loop or were cut off. In the pipeline the cut-off is capped and reported as a warning, with the missing text marked.

Licence: model card of the authors, checked 2026-11-04, also for the base model it was built on; notices under third-party software.

Rendering resolution (dpi) behind the measurements

The service renders PDF pages at 150 dpi unless told otherwise, and the result names the dpi per page. Two of the figures above were measured at 200 dpi (throughput, orientation test); the rendering resolution is named wherever it is known. The image-to-text model scales every page image down to its own working size before it reads.

Sources