Remove line breaks from OCR text
You search for a sentence and find nothing, because it crosses a line end. The text is there, chopped into pieces, and a person has to glue it. Which means: with the right output format the sentence arrives in one piece.
Read one page of your own free and see its lines with their boxes, ready to join into paragraphs.
Line Three lines, each with its box and a confidence of 0.99, 0.99 and 0.99. Boxes: as returned by a pipeline.
Which OCR output format gives you paragraphs
text: one line per line on the page. Faithful to the page, one line per line.md: one section per page; where the pipeline finds paragraphs and tables, it keeps them. Start here.json: every line with its words and boxes. Use it whenmdis not enough and you want to join lines yourself.
Join the lines of a paragraph in five steps
- Ask for
format=json: each line has words with a box. - Measure the gap between lines. Equal gaps stay in one paragraph; a larger gap starts a new one.
- A line that starts further right than its neighbors often starts a paragraph.
- Join the lines of one paragraph with a space. Join a hyphen at a line end only when the next line starts in lower case:
infor-andmationbecomeinformation. - Check the word count before and after. It must be the same, apart from joined hyphens.
Step five protects every word: if the count differs, a line was lost or doubled, and you see it before anyone reads the text.
Tidy OCR text without a language model rewriting it
A model that rewrites
Also writes words that are not on the page.
Joining lines by position
Keeps every word as it is.
Source: run on 300 public pages. Where a language model reads together with a character OCR engine, the text and boxes stay as read: in a run on 20 pages, 99.4 % of 4,406 words carry their own box.
Two-column pages, sidebars and wrong words
Joining is by position and every word stays as read, so the word count is the check (step 5). Wrong words show in their confidence; a person corrects those. For two-column pages and sidebars, group the lines into columns by their boxes before step 2.
Try OCR text cleanup free
Read one of your own pages free, or open the OCR demo gallery to see the lines of a page with their boxes before any joining.
Production use, high volume or your own server?
Private use and tests are free. For a company in production, a server of your own or a pipeline built on your pages, send the kind of page and the volume, and get a written offer, or a clear no when the fit is not there.
