OCR for receipts and invoices: ten systems, one spot

On this page

• OCR accents and diacritics: Ě read as Ė, É or ŘE on a Czech receipt
• OCR errors that look like words: Prosimé for Prosíme
• Missing price and cut product code: OCR output that looks complete
• Flagging OCR differences: how a second OCR engine marks these spots
• How the OCR readings are scored: printed text as reference
• Sources

Which OCR can you trust with an amount, an accent or a product code? A receipt is short, printed small, and full of numbers, names and accents. The same spot of a public receipt is shown here with what ten OCR engines wrote under it, next to two small crops of the printed spots. Most readings agree; the tables show where they differ.

The systems are labelled A to J; System A is the image-to-text model of the combined pipeline and System H is the CTC text reader. The comparison covers one receipt and one invoice.

OCR accents and diacritics: Ě read as Ė, É or ŘE on a Czech receipt

The headline of a Czech café receipt reads NA VŠE, CO DĚLÁME. The E has a caron.

Six systems get it right. Four write another letter on the E: an E with a dot, an E with an acute, an R next to the E, and in one case a mess of letters with tone marks that exist in another script. Nothing in the output tells you which line is wrong. The line looks like text.

OCR errors that look like words: Prosimé for Prosíme

At the bottom of the same receipt is the request to keep the receipt. The print reads Prosíme pečlivě uschovejte.

Here a different group fails: two of the systems write Prosimé pečlivé, with an é where the print has an e or ě and an i without its accent. They are still words you could believe. A spell checker for Czech would catch it; a check that only looks at digits would not.

Missing price and cut product code: OCR output that looks complete

The first line of the items is a menu with the price 248.00 on the right.

Two systems leave the price out of their output altogether: the number is not there. For an invoice that is the worse case, because the line looks complete.

The same happens to a product code on the invoice, a line from a public invoice:

Three systems cut the line after the product name; the code SDSQUAR-064G-GN6MA is gone. The article number is the part a bookkeeper searches for.

Flagging OCR differences: how a second OCR engine marks these spots

The combined pipeline is built on the first system in the tables, a model that writes the page, plus a second OCR engine that finds every word on its own and reads it again. The model wrote DĖLÁME, in the headline. The second OCR engine read DĚLÁME,, and the word is flagged. At the request to keep the receipt, the model wrote Prosimé, and the OCR engine Prosím; flagged, together with the neighbouring words.

In the first case the OCR engine's reading is the one that matches the print. In the second case the OCR engine's reading cuts the word short. The second OCR engine makes its own mistakes, and the combination shows the split: the word is marked with its box, both readings and the OCR engine's confidence.

How the OCR readings are scored: printed text as reference

Prints and readings are compared by script, character by character, with the printed text as the reference for each spot. The two crops come from a public receipt.

Sources