On this page

Your accounting program needs an amount and the spot on the page where it stood. Ask for JSON: per page the lines, per line the words, and for every word its text, its box and a confidence. It is the only format that keeps all of it; to read the words alone, Markdown or plain text is lighter.

OCR to JSON: word text, bounding box, confidence score and source per word

One object per job, with the pipeline by name and version. Per word you get the text, the box in pixels of the page image, the confidence and where the word came from.

One word with arrows to its text, box, confidence and source in JSON.
One word, four fields.

OCR to JSON example: word bounding boxes and confidence in a real API answer

The first words of a Czech till receipt, as the answer comes back:

{
  "schema_version": 1,
  "pipeline": "scan-ctc@2",
  "pages": [{
    "page": 1, "status": "done", "width": 622, "height": 1960,
    "lines": [{"words": [
      {"text": "NA",   "confidence": 0.92, "source": "ctc", "verified": false, "box": [95, 298, 132, 354]},
      {"text": "VŠE,", "confidence": 0.92, "source": "ctc", "verified": false, "box": [148, 299, 211, 356]},
      {"text": "CO",   "confidence": 0.92, "source": "ctc", "verified": false, "box": [229, 299, 263, 356]}
    ]}]
  }]
}

Excerpt from a run over a public Czech till receipt with a free licence (CC0), shortened. Boxes and confidences are those of the character OCR engine; numbers are rounded.

Four lines of a Czech till receipt, from further down the page, each word in a box.
Words with their boxes, as the JSON gives them.

What to build on word bounding boxes and confidence: ERP amounts, search highlights, archive, agents

  • Amounts and dates for ERP. Read the value with its confidence. Send words below a threshold of your choice to a person.
  • Search with a marked hit. Index the words and keep the box, so a hit can be drawn on the page image.
  • Archive. Keep the JSON next to the scan. A later check goes back to the exact place on the page.
  • Agents. A fixed schema is easier for a program than free text, and the agent sees which words are unsure and can ask instead of guessing.

Word-level and line-level JSON schema: which pipeline returns which boxes

A box for every word comes with the pipelines that read words; the others give one box per line. The pipeline is chosen by that line in the format table. A page that failed carries status failed and an error with a fixed code, so a failed page is always visible in the answer. The call to fetch the file is in the API guide, and the same answer next to the page is in the gallery.