Self-hosted OCR API: installation guide for a GPU server

Which NVIDIA GPU does the OCR server need? Cards per pipeline

The start check tells you which pipelines your card carries: the fast reading pipelines from compute capability 7.5 (GTX 16xx, RTX 20xx) with 8,192 MiB of video memory free, and the language-model pipeline from 8.0 (RTX 30 series and newer) with 17,920 MiB free. A GTX 10xx (compute capability 6.1) is below the limit of every pipeline.

The language-model pipeline reads a page with a language model and with a character OCR engine and compares both. That needs video memory: an NVIDIA card with compute capability 8.0 or higher (RTX 30 series and newer), driver 580 or newer and at least 17.5 GiB (17,920 MiB) of video memory free. Measured on an NVIDIA RTX PRO 4500 Blackwell with 32 GB.

Find your own numbers first. Run this on the server:

nvidia-smi --query-gpu=name,compute_cap,memory.free,driver_version --format=csv

The columns are the card, its compute capability, the video memory that is free right now, and the driver. Free means free: a desktop session or another program holding the card lowers it. Compare with the table; the start command in step 5 reads the same numbers from the card and compares them with the limits of the pipeline.

The start command in step 5 checks compute capability, driver and free memory and stops with exit code 4 if one is too low; it says what your card does carry. Above the limit it works out how much memory the language model gets from the memory that is free on your card (the rule is in the file deploy/gpu-vlm.conf of the package) and prints "same memory size as the card we measured (only the RTX PRO 4500 Blackwell has run)" for every card within three percent of the test card (32,623 MiB), "not verified on this card size, it may fail to start or run out of memory" for the others. Only the first card (GPU 0) is checked and used, also on a server with several: the services run with CUDA_DEVICE_ORDER=PCI_BUS_ID and CUDA_VISIBLE_DEVICES=0, so the card that is checked is the card that is used.

• The language-model process held 16,544 MiB on the 32 GB card, set to use half of it (own measurement, test GPU, 6 Nov 2026).

• If the start fails or the language model runs out of memory, the command stops with exit code 6 and names the service; each service gets up to eleven minutes to start, then the API up to ten more, so this can take longer than eleven minutes. A failed start stops everything it began, ocrd too, so the services show as inactive and the reason is in the log. Read why with

  journalctl -u 'ocr*' --since '30 minutes ago' --no-pager
  

  (the log of all services of this package), close other programs that hold the card, and start again; the second start stops the package services first and measures again. If it fails again, send the card, the line "model sidecar: ..." and the log through the contact page.

• The fast reading pipeline used 3.5 GB of video memory with one worker and 14.1 GB with four on the test card (own measurement, 5 Nov 2026); the start check asks for 8,192 MiB free.

GPU too small: what the start check reports

• Before it installs anything, install.sh asks nvidia-smi -L. If no card answers, it stops with exit code 4 and starts nothing.
• The start command (step 5) reads the compute capability, the free memory and the driver from nvidia-smi and compares them with the limits of the pipeline (file deploy/gpu-grenzen.tsv of the package). If one is below, it stops with exit code 4, names the number and the cards it does carry, and starts nothing. It also stops if the memory left for the language model is below its least. Only GPU 0 is read and used, also for foreign processes. A second start stops the services of this package that are running (only those, never a foreign process), waits five seconds and measures again (if the card has not freed its memory by then, the check stops with exit code 4 and "does not fit"); if the check then fails, the workers of this package no longer run; ocrd keeps running.
• Before each service starts, a check stops it with a clear line in the log if another program already computes on GPU 0. It kills nothing; stop that program first.
• If a page does not fit into memory anyway, it comes back as failed with an error code. It is never returned empty and never guessed. A failed page costs nothing.
• Then start a pipeline from the table that your card passes (--pipeline), or use a bigger card. A card below compute capability 7.5, such as every GTX 10xx, passes none.

Driver and CUDA: NVIDIA driver 580 or newer with the open kernel module.

Install the OCR server and read your first page

Run these in the folder with the package. Steps 1 to 3 can be repeated without harm. Step 4 can be repeated after it has finished; if it was cut off, run it again with --rebuild (see step 4). Step 5 stops the services of this package that are running before it measures the card, so a running service is interrupted. Step 6 makes a new key every time.

1. Check.

   sha256sum -c ocr-software-2026.11.11-352765c-linux-amd64.tar.gz.sha256
   

   Must print OK. If not, download again.

2. Unpack.

   tar -xzf ocr-software-2026.11.11-352765c-linux-amd64.tar.gz
   

   and

   cd ocr-software-2026.11.11-352765c-linux-amd64
   

3. Install.

   sudo ./install.sh
   

   First checks that python3.10 with the venv module exists; if not, it stops with exit code 7 and installs nothing (see "Install errors"). Then checks every file and installs. It listens on 127.0.0.1 only. On the first install add --listen 0.0.0.0:8080 to open it to your network; later runs keep your config.yaml and say so, then edit listen: in /etc/ocr/config.yaml.

4. Fetch the models.

   sudo /opt/ocr/current/deploy/ocr-fetch-models.sh
   

   Shows source, licence and size, then asks "Fetch this now? [y/N]" each time you run it; without a terminal add --yes. The only step that downloads, several GiB of models. It fetches all environments of the package, also those the language-model pipeline does not need. If GPU 0 is below the limit of the language-model pipeline, this step prints a warning before it asks; it does not stop. Step 5 checks the card before it looks at the models. Check your card against the table above first: the warning points to it. With --list it only shows, and needs no root. If step 4 was cut off and the next run says "venv differs", an environment was left half built: run

   sudo /opt/ocr/current/deploy/ocr-fetch-models.sh --rebuild
   

   (the script prints this hint itself and exits with code 4). A finished step 4 is skipped on a second run.

5. Start.

   sudo /opt/ocr/current/install.sh --start --pipeline scan-ctc@2
   

   Starts only the services this pipeline needs, after the card check above. Needs the models and a GPU; only GPU 0 is used. If the card is too small it starts nothing. If you start again, it first stops the services of this package that are running, and only those. It waits until the API answers; each service gets up to eleven minutes to start, then the API up to ten more, before an abort comes. If it stops earlier, it prints the reason and an exit code; the table under "Install errors" explains them. If a service does not start (exit code 6) or the language model runs out of memory on a 24 GB card, the start stops everything it began, so the services show as inactive; read why with

   journalctl -u 'ocr*' --since '30 minutes ago' --no-pager
   

   To see what runs after a good start:

   systemctl status 'ocr*' --no-pager
   

6. Make a key.

   sudo /opt/ocr/bin/ocrctl key new -id mykey -paid -keys /etc/ocr/keys.txt
   

   The key is printed once. Keep it.

7. Read a page. The calls below go to your own server, http://127.0.0.1:8080. The hosted API at https://api.ocr-software.com takes the same calls.

export OCR_KEY=<your key>
curl -s -X POST "http://127.0.0.1:8080/v1/jobs?pipeline=scan-ctc@2&formats=json" \
  -H "Authorization: Bearer $OCR_KEY" --data-binary @page.pdf

The answer names a job_id. Ask for the job until its state is done, then fetch the result (pipeline choice, reading the result, errors and retries: API guide):

JOB_ID=<the job_id from the answer>
curl -s "http://127.0.0.1:8080/v1/jobs/$JOB_ID" -H "Authorization: Bearer $OCR_KEY"
curl -s "http://127.0.0.1:8080/v1/jobs/$JOB_ID/result?format=json" -H "Authorization: Bearer $OCR_KEY" -o result.json

Each word in result.json carries its text, a confidence and where it came from. The format is described on the page about JSON output.

Step 3 installs to /opt/ocr/releases/<version>; the installer creates the configuration and the key file only if they are missing, so a second run never overwrites yours. Step 5 stops at the first problem and names the service. Check the API with

curl -s http://127.0.0.1:8080/healthz

OCR speed: seconds per page per pipeline

How fast each pipeline reads, in seconds per page and pages per second (multiply by 3,600 for pages per hour), with the run behind each figure, is in the speed table on the pipelines page. Your card and your pages will differ; measure a sample of your own.

More pages at a time make the language-model pipeline faster. Set it with scheduler.vlm_slots in the server config (default 8, at most 64). Read one page at a time, the output is the same after every restart; use a fast reading pipeline for identical bytes at any load.

Install troubleshooting: errors and exit codes

Its exit code tells you where to look; 0 means it worked.

• The API does not answer. Check

  curl -s http://127.0.0.1:8080/healthz
  

  the first start can take ten minutes.

• No access from other machines. Default is localhost only, plain HTTP: put TLS or a firewall in front.

• A new release does not start. The installer switches back by itself.

• Not solved. Send the exit code and the log through the contact page.

Support, managed server and custom OCR pipeline

Running it for third parties, support, maintenance or a fixed update promise come with a contract. Prices are on the pricing page.