Turn public tender PDFs into JSON with evidence for each extracted field.

RFP requirements extraction should produce values you can trace back to a particular document version and page. A clean JSON response is useful only if it preserves the source wording, exposes unresolved fields and distinguishes an automatic match from a reviewed correction.

This guide includes a runnable Python example using a real six-page public procurement PDF. The first run finds five of ten scoped fields. Its biggest problem is easy to miss: the cover page produces no text, even though it visibly contains the submission deadline, pricing instructions and delivery month.

We preserve that automatic result and supply a separate field-by-field visual review. The reviewed output contains eight extracted fields, one ambiguous deadline and one budget field marked not found within this PDF. These are results from one document-specific example, not an accuracy benchmark or a claim that every requirement was captured.

Start with one accessible public document

The source is RFQ 2026-CE01, published by the U.S. Court of Appeals for the Sixth Circuit. It concerns furniture-related delivery, design and installation. It is a request for quotation, not a request for proposal; we use it as a small example of the evidence-handling workflow needed for RFP data extraction. Original court PDF.

The bundle pins the file downloaded on September 23, 2026, with its source URL, download completion timestamp, byte count and SHA-256. Every extracted field therefore belongs to specific bytes. The historical June deadline is not a current bidding opportunity, and this exercise does not verify later amendments or solicitation status.

The scope is deliberately small: ten fields from the six-page PDF. A separately referenced Excel quote sheet and clauses incorporated through external references are outside the collection. That boundary matters when interpreting a missing value or assessing whether the output is complete enough for a bid workflow.

If you have not yet established that your source files can be retrieved, start with tender document and attachment coverage. This example begins after successful file retrieval.

Define values, evidence and review states

Our schema separates the field value from the evidence supporting it. Each field has a name, value, status, evidence array, reviewed-page list and explanatory note. The parent record carries the source URL and hash.

An empty search result starts as needs_review. It does not become not_found just because a regular expression returned no match. In this sample, that distinction prevents a missing text layer from being mistaken for missing procurement instructions.

The reviewed deadline field has the following shape; the full record also contains the source hash and URL:

{
  "field": "submission_deadline",
  "value": "2026-06-26T16:00:00",
  "status": "ambiguous",
  "evidence": [{
    "page": 1,
    "quote": "Quotes due no later than: 26 Jun 2026, 1600, Hours",
    "method": "visual-transcription"
  }],
  "reviewed_pages": [1],
  "note": "Timezone unstated on the cover; do not convert to UTC without clarification."
}

The date and clock time can be read, but the cover does not state a timezone. An office address is not enough to silently turn that value into a UTC deadline. Keep the ambiguity available to the application that would otherwise schedule alerts or reject late records.

Keep the cover page even when text extraction fails

The example uses pypdf to extract text separately from every page. In the pinned file, page 1 yields an empty string; pages 2–6 yield text. Visual inspection shows that the cover contains substantial content. We retain it as physical page 1 and record it in pages_without_text.

Dropping empty pages would create two problems: losing the cover's fields and shifting every subsequent citation. The quote-sheet identifier belongs to PDF page 2, regardless of whether the parser returns text for the page before it.

pypdf extracts an existing text layer; it does not perform OCR on page images. Empty text should trigger inspection or a separate OCR step, not a conclusion that the page contains nothing. This example uses recorded visual transcription for four cover-page fields and does not run an OCR engine. pypdf text-extraction documentation.

Retain the original file even when you later add OCR. Store the OCR result, tool version and page mapping separately so that a correction remains attributable to its source.

Reproduce the automatic and reviewed JSON

Unzip the download and install the two pinned dependencies in a Python environment. The example was tested with Python 3.12.3.

python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install -r requirements.txt
python3 -B extract.py --output reproduced
python3 -B -m unittest test_extract.py

Windows activation instructions are in the README. After dependency installation, execution is offline and needs no account or model API key.

The script first checks that source.pdf matches the receipt hash. It then extracts page text, applies narrow document-specific rules and writes automatic.json. A separate visual-review.json supplies the recorded review decisions, producing reviewed.json. page-text.json lets you inspect exactly what the PDF parser returned.

The rules are intentionally specific to this document. They demonstrate evidence retention, duplicate-match handling and review boundaries; they are not a general-purpose tender document extractor. A different document needs its own extraction logic and review.

Replaying the review file does not conduct a new review. It reconstructs the published result from a version-bound set of editorial decisions. The program rejects a changed PDF or a review overlay carrying a different source hash.

Inspect the field-by-field corrections

All six rendered pages were visually inspected during AI-assisted editorial preparation. This was not an independent human review. The downloadable CSV records the baseline status, final status, action, pages, value and review note for every scoped field.

The substantive corrections are the four fields added from the cover. The budget changes state only after reviewing all six pages. That finding remains limited to this PDF; the external workbook was not inspected.

Several smaller choices prevent misleading structured data. Delivery is 2026-10, not an invented October 1 deadline. Site-visit availability does not become a mandatory-visit flag. Conditional permission for substitutes retains its standards qualification. The receiving-report requirement retains the event that triggers it.

The complete sentences remain available in the JSON evidence. A downstream application can show a short summary while still letting a reviewer open the source page and inspect the wording.

Validate citations without claiming complete accuracy

The example performs two types of validation. JSON Schema checks the record structure and allowed combinations of status and value. Additional code checks field uniqueness, page bounds and whether text-derived evidence actually occurs on the cited page after whitespace normalization. JSON Schema conditional validation.

For a not_found decision, the implementation requires a null value and a recorded review of all six pages. Visually transcribed evidence must identify a reviewed page, but its correctness still depends on visual comparison: the program cannot verify those words against an empty text layer.

Eleven tests cover changed source bytes, mismatched review versions, wrong-page citations, invented text evidence, duplicate matches and fields, invalid values, incomplete absence review, preservation of uncertainty, the empty cover and reproduction of the packaged outputs.

None of those checks proves that every obligation has been found. A quote can genuinely occur on a page while supporting the wrong interpretation. A complete-looking object can omit an exception elsewhere in the document. The ten selected fields are neither a compliance checklist nor a representative extraction benchmark.

Adapt the workflow to another document set

For a new source, agree a small field specification before choosing extraction technology. State which values are required, which may be null, how conditions are represented and which decisions require review. Keep document identity and physical page numbering throughout the pipeline.

Separate retrieval failures, unreadable pages, unmatched fields and substantive ambiguity. They require different fixes. Another download attempt cannot resolve an unstated timezone, and a better prompt cannot recover an attachment that was never collected.

If you add an LLM, require the same evidence-bearing output. Treat document text as source material rather than instructions to the application, validate its response and check cited passages. A schema constrains response shape; it does not establish factual correctness. Record model and prompt versions alongside the source hash so changes can be investigated.

Evaluate on additional documents with independently reviewed expected fields. Include scanned covers, tables, cross-page sentences, alternative requirements, referenced attachments and amendments. Measure missed requirements and unsupported values separately from schema-valid responses. Establish amendment precedence before using a value operationally.

Scope recurring collection and structured fields

Run the public example first, then identify the sources, document types and fields your product needs. A useful scoped request includes a few representative files, expected output, acceptable missing-value states and how page evidence should be delivered.

WebTruffle can assess recurring collection and structured-field delivery against that scope, subject to source access and technical feasibility. This example does not announce an existing hosted AI or OCR product. The procurement data evaluation pack provides a broader framework for turning a sample into acceptance criteria.

Frequently asked questions