Evaluate historical procurement data before buying the backfill.

Evaluate historical procurement data against a defined period, source population and date field before buying an archive. Ask for expected partitions, actual first and last dates, failed downloads, parser coverage and a reproducible sample. A large row count is useful only when you know what it represents.

We tested that approach on three official TED daily XML archives, covering August 11–13, 2026. The audit parsed 9,650 publications and selected 21 notice versions referencing an Irish buying organisation and an IT-related CPV code. All three expected editions were present. That verifies this package inventory; it does not prove complete Irish procurement coverage.

Download the receipts, full member inventory, scoped extract and editable backfill specification below. The Python helper reproduces the report from the pinned official archives without an account or API key.

Define the period and population before counting records

Our worked question is: Which publications in OJ S editions 153–155 of 2026 reference an Irish buying organisation and any CPV code beginning with 72? We retain all notice types, rather than presenting historical awards as open opportunities.

| Requirement | Definition used in this audit | | --- | --- | | Source | Official TED daily XML packages | | Period | August 11 through August 13, 2026, inclusive | | Date field | Publication metadata's PublicationDate | | Buyer scope | At least one referenced buying organisation has postal country IRL | | Category | Any recorded CPV classification starts with 72 | | Grain | One source publication; retain notice ID, version and procedure ID | | Notice types | All types in the packages; no open-deadline filter | | Observation date | Archives retrieved September 17, 2026 |

Buyer country is not place of performance. Publication date is not award date, dispatch date or the day our collector first encountered the record. CPV matching across a notice is not proof that every lot is IT-related. These choices materially change which records a supplier should deliver.

The helper resolves buyer references to organisation blocks in the XML. It does not guess buyer identity from a matching organisation name or treat every organisation in a notice as a buyer. For a named-buyer purchase, agree the relevant identifiers and historical name changes before running the extraction.

Inspect the three-edition coverage report

The expected partition set is explicit: 202600153, 202600154, 202600155. The helper compares every accepted XML member's publication date and gazette identifier with its assigned partition.

| OJ S edition | Publication date | XML members parsed | Result | | --- | --- | ---: | --- | | 153/2026 | August 11, 2026 | 3,176 | Verified | | 154/2026 | August 12, 2026 | 3,257 | Verified | | 155/2026 | August 13, 2026 | 3,217 | Verified | | Defined window | August 11–13 | 9,650 | 0 missing or failed editions |

These counts come from the actual 153 archive, 154 archive and 155 archive, downloaded and hashed for this example. archive-manifest.json records URLs, byte counts and SHA-256 values. The member inventory records a hash for each XML file as well.

The first and last publication dates in both the full inventory and scoped extract are August 11 and August 13. There are 9,650 unique canonical publication numbers, with no duplicate publication number across these files. The 21 selected records comprise 12 can-standard, 6 cn-standard and 3 cn-social notice types.

“Zero missing” is relative to the three expected edition keys. It does not mean every day of August was covered. A longer tender data archive needs an explicit publication calendar or authoritative edition index, not a generated list of weekdays that silently assumes every weekday has a package.

Nor did this audit independently reconcile the packages against TED Search or another publication index. That is a separate control to require before making a stronger source-completeness claim. TED documents anonymous access to published notices through its Search API; our TED collection guide explains edition-level reconciliation.

Check what the parser actually retained

An archive download can succeed while a parser silently loses records. In this corpus, three SDK values occur:

| SDK identifier | XML members | | --- | ---: | | eforms-sdk-1.12 | 1,042 | | eforms-sdk-1.13 | 5,943 | | eforms-sdk-1.14 | 2,665 |

The root types also matter: 4,928 ContractAwardNotice, 4,553 ContractNotice, 158 PriorInformationNotice and 11 BusinessRegistrationInformationNotice documents. The last group has no procurement buyer or CPV fields in this corpus. Those 11 records remain in the inventory and fall outside our buyer/category selection; they are not silently discarded as malformed tenders.

There were no unresolved buyer references in the parsed corpus. The 11 rows without buyer countries and the 11 without CPV codes belong to the business-registration group. A generic “every row must have a CPV” rule would misdescribe that source population.

The supplied parser handles the eForms families observed here. Parsing success is not XSD validation or validation of every procurement business rule, and the example does not claim legacy TEDXML support. When a longer archive introduces an unsupported family, stop and report the failed partition rather than accepting only the rows the parser happens to understand.

Reconstruct the window from pinned archives

Use Python 3.10 or later. The helper needs only the standard library:

python3 -B audit_archive.py --download --cache cache --output reproduced
python3 -B -m unittest test_audit_archive.py

It downloads the three files sequentially, checks the pinned size and hash, reads XML members without extracting archive paths, and validates the publication date and edition before accepting each partition. Run without --download to audit previously cached files.

The output includes a partition report, full publication inventory, scoped Irish IT extract and coverage report. All four files were reproduced byte-for-byte from the pinned archives during preparation. Eight checks additionally cover identity handling, unsupported XML, missing fields, duplicate publications, absent partitions, corrupt files and the bundled inventory totals.

A missing or failed partition produces a report and a nonzero exit status. Its partially parsed records are not accepted. That matters commercially: a file with zero matching notices is an observed result; a file that was never retrieved is an unknown. They should not share the same “0 records” cell in a delivery report.

If an official package later has a different hash, investigate and record a new retrieval version. Automatically replacing the expected hash would erase the very evidence this check is intended to preserve.

Separate publication history, observation history and latest state

Before buying historical tender data, ask which kind of history the supplier is offering.

| Kind of history | Question it can support | Evidence needed | | --- | --- | --- | | Source publication history | What publications carry dates in the requested period? | Source publication identifiers, dates and retained source documents | | Collector observation history | What did the collector actually have at a past point in time? | Timestamped observations and immutable retained bytes | | Latest state | What is the most recently known state of a record? | Defined update precedence and links to source observations |

This example reconstructs August publications from packages retrieved in September. It does not prove what a collector knew on August 11, or exclude later changes to the distributed archive. A point-in-time research requirement needs stronger observation evidence.

Likewise, 21 selected notice versions are not 21 distinct contracts or procurements. Procedure links, amendments, lots and awards need their own model. The government contracts database guide covers those relationships; this evaluation focuses on whether the historical window and source material are fit for the intended job.

Check WebTruffle's public-history boundaries

Official TED archives and WebTruffle's public normalized releases are different evidence sources. On September 17, 2026, the public government-tenders-rfps release list contained 31 release entries, including revision tags, with date tags spanning July 15 through September 1. The dates between those endpoints are not a continuous daily archive.

In particular, there were no date-tagged public releases for August 11, 12 or 13 in that listing. This is a release-list observation, not proof that no older records occur in other files or that the source itself has missing publications. We reconstructed this worked window from the official TED packages instead.

The downloadable webtruffle-public-history.json records the dated observation, API route and listed tags. We did not audit every public release's contents, private retention, attachments or a multi-year normalized archive. Do not infer those capabilities from the earliest and latest tags—or from the existence of today's public sample.

For a requested historical backfill, WebTruffle should confirm the source, period and deliverable fields against actual availability before agreeing coverage. Public download access is useful for evaluation; it is not a blanket promise of complete historical coverage.

Write an acceptance contract for the backfill

The downloadable specification turns the worked question into a starting contract. Replace its three-edition window with your required period, then agree the remaining decisions before commissioning extraction.

Require an expected-partition inventory, raw-source retention, deterministic identifiers, a parser/schema report and explicit handling of missing or failed partitions. Specify whether later corrections belong in the delivery, whether attachments are required, and how the supplier will demonstrate completeness for the declared source population.

For the worked example, all expected partitions must parse, edition/date mismatches and duplicate publication numbers must be zero, and unresolved buyer references must be reported and investigated. A production completeness claim additionally needs the independent reconciliation that this example has not performed.

The specification also leaves commercial and operational questions open: longer-range availability, destination format, delivery cadence, repair time, reuse constraints and price. Those are requirements to agree, not capabilities silently inferred from a successful three-day test.

Connect the backfill to continuing delivery

A backfill becomes more useful when the historical and ongoing datasets follow the same definitions. Keep the buyer/category rule, notice grain and date semantics consistent. Agree where the backfill stops and the recurring collector starts, with overlap to detect late publications or corrections.

Retain source receipts and failed-partition records so repairs can be replayed. When a schema changes, quarantine unsupported input and report the gap. Do not let a successful scheduled job conceal a smaller accepted population.

Frequently asked questions