OCR and AI pipeline

The OCR and AI pipeline converts files into structured document data. eDocify treats this as a governed pipeline, not a single black-box request.

Pipeline stages

flowchart TD
  A["File received"] --> B["PDF text layer extraction"]
  B --> C["OCR engine"]
  C --> D["AI / rules structuring"]
  D --> E["Field processing"]
  E --> F["Validation checks"]
  F --> G["Confidence and review reasons"]
  G --> H["Verification queue"]

Provider types

Azure Document Intelligence

Best for invoice structure extraction and strong baseline recognition. It can return both OCR text and structured invoice fields.

OpenAI / Azure OpenAI

Used for structured JSON extraction, verification, reasoning, field suggestions, command-based filtering, and assistant actions.

Mistral OCR / Mistral AI

Used for OCR and LLM-assisted structuring. Useful as a second provider in benchmark and fallback strategies.

Local Tesseract

Cost-controlled OCR option. Works well when documents are clear and rules can structure the text. Useful for region OCR and low-cost background processing.

Local RapidOCR

Local OCR option for fast, lightweight recognition and provider comparison. Useful in hybrid routes where eDocify needs a local fallback, an offline benchmark, or a cost-controlled second opinion before using premium providers.

Local PaddleOCR

Local OCR option for image-heavy or scanned documents. Can be combined with the same eDocify Rules structuring layer.

Local Ollama / Qwen3 structuring

Planned local LLM route for turning OCR text into a strict invoice JSON contract. The experimental path is:

flowchart LR
  A["PDF / JPG invoice"] --> B["RapidOCR or PaddleOCR"]
  B --> C["OCR text"]
  C --> D["Ollama qwen3:8b or qwen3:30b"]
  D --> E["Strict JSON schema"]
  E --> F["Validation and verification queue"]

This route is useful for private experiments, cost-controlled processing, and offline provider comparison. It should be treated as R&D until it passes a golden dataset benchmark, JSON schema validation, and human verification review.

eDocify Rules

Deterministic parser for invoices and known patterns. It is especially useful for supplier-specific rules, totals, VAT, dates, IBAN, and line extraction candidates.

Provider routing

Provider routing can be configured by:

• tenant;
• client group;
• company;
• document type;
• supplier;
• confidence requirement;
• cost limit;
• data residency requirement;
• fallback policy;
• region OCR policy.

Example:

Scenario	Recommended route
New client pilot	Azure Document Intelligence + LLM verifier.
Known supplier template	Local OCR + eDocify Rules.
Low-value high-volume invoices	Tesseract, RapidOCR, or PaddleOCR + rules, fallback only on low confidence.
Local/private extraction experiment	RapidOCR or PaddleOCR -> Ollama qwen3:8b or qwen3:30b -> strict JSON schema -> verification.
Critical invoices	Premium provider + second-pass AI verification.
Customer BYOK	Customer provider first, eDocify fallback if allowed.

Hybrid recognition strategy

eDocify should not depend on one provider. The strongest route is usually a hybrid:

1. Extract text and geometry from PDF text layer or OCR.
2. If the provider supports invoice semantics, read structured invoice fields directly.
3. Normalize the result through the eDocify invoice contract.
4. Run deterministic rules for dates, totals, VAT, IBAN, supplier identity, and line candidates.
5. Ask an LLM verifier to check inconsistencies, missing fields, and ambiguous line items.
6. Store OCR snapshots, provider version, prompt version, raw response, and field provenance.

This makes Azure Document Intelligence, Mistral OCR, OpenAI, Tesseract, RapidOCR, PaddleOCR, and eDocify Rules complementary instead of competing black boxes.

The local Ollama/Qwen3 route adds another option: OCR locally first, then ask a local LLM to structure the OCR text into the same JSON contract used by cloud providers. This keeps the rest of the eDocify pipeline unchanged: validation, confidence rules, verification, AI Learning, ERP export, and archive can read the same schema.

JSON contract

LLM providers should return a strict JSON contract. The prompt should request:

• header fields;
• line items;
• confidence per field;
• source evidence;
• page or region where possible;
• validation warnings;
• duplicate or anomaly reasons;
• empty value handling.

The application should reject unparseable or schema-incompatible responses and record them as provider errors.

For local LLM routes, schema validation is mandatory. The system should record model id, model size, prompt version, input OCR provider, raw OCR text hash, JSON validation result, and any rejected fields.

For invoices, the contract must include line items, not only header fields. A useful invoice response includes:

• supplier and buyer identity;
• invoice number, dates, currency, payment terms;
• net amount, VAT amount, gross amount, VAT rate;
• IBAN and bank account evidence;
• line description, quantity, unit, unit price, VAT, totals;
• suggested cost center, project, and line approver when available;
• confidence and evidence for every field and line.

The same contract lets UI, field processing, approval rules, ERP export, AI Learning, and Accuracy Studio read predictable data.

Confidence rules

Confidence is not the same as truth. It should be calibrated against golden datasets and human corrections.

Recommended confidence bands:

• 95-100: safe candidate;
• 85-94: review when critical field;
• 70-84: likely needs verifier attention;
• below 70: require manual review or second provider;
• missing critical field: always review.

OCR snapshots

For every run, store:

• provider key and model;
• prompt/rule version;
• OCR raw text;
• structured response;
• fields and line items;
• processing duration;
• cost estimate;
• error message;
• confidence summary;
• document id and tenant id.

Snapshots are needed for audit, AI learning, regression testing, and customer quality explanations.

Snapshots also keep the original OCR text. This matters because later services can re-run field processing, compare providers, train local rules, or explain a verifier correction without OCRing the file again.

Enterprise release governance

A new provider, prompt, or rules version should pass:

• golden dataset benchmark;
• critical field regression check;
• line item regression check;
• cost comparison;
• latency comparison;
• data residency review;
• rollback plan.