Compare semantic tender search against a keyword baseline you can inspect.

Semantic tender search in Python can rank procurement text by similarity to a buyer's description. It still needs a keyword baseline, source-aware eligibility checks and a relevance review. An embedding score cannot tell you whether a notice is open or whether its documents contain the service you sell.

We ran a local comparison on 217 public notice versions, using BM25 keyword ranking and a pinned multilingual embedding model. Across four illustrative queries, both methods returned the same top result, before and after metadata filtering. Lower ranks differed, and both methods produced unsuitable results.

That is the useful finding here: this small title-only experiment does not demonstrate a semantic-search advantage. It does provide a reproducible notebook, saved vectors, query set and 20 documented query/result reviews that you can inspect and extend.

Start with the text you actually have

The corpus is the pinned September 1, 2026 public tender sample used in our CPV filtering example. It contains 197 TED, 13 Contracts Finder and 7 Find a Tender notice versions, preselected because at least one classification starts with CPV 72.

We index only title, unchanged. Some titles contain a country and English category prefix followed by text in another language. Neither method receives descriptions, tender attachments, lot requirements or extracted document text. The publisher's category prefix is therefore part of the evidence being ranked, not an independent relevance label.

The query set covers four needs:

| Query | What it probes | | --- | --- | | Cyber security monitoring and incident response | A specific operational service | | Software implementation and data migration | Work that may be described in different languages | | Library refurbishment project management | A non-software need present inside an IT-preselected corpus | | Voice transcription for clinical consultations | A plausible match whose notice stage may make it unsuitable |

The queries were written before the first ranking run and were not changed afterward. They were designed with this corpus topic in mind, not sampled from independent users. Four queries are an illustrative test set, not a held-out benchmark or evidence of market-wide retrieval quality.

Build a BM25 keyword baseline

The baseline uses BM25 with k1=1.5 and b=0.75, Unicode word tokenization and case folding. It does not stem, translate, expand synonyms or remove stopwords. This is a modest, inspectable baseline; production lexical retrieval could improve on it.

BM25 rewards overlapping terms while accounting for how common they are and how long the title is. A document with no query-token overlap receives zero and is not emitted as a lexical candidate. The script can therefore return fewer than three keyword results.

keyword_scores = bm25(titles, query)
keyword_candidates = [
    i for i in eligible_indices if keyword_scores[i] > 0
]
keyword_top = ranked(keyword_scores, keyword_candidates, k=3)

BM25 statistics are fitted on all 217 titles; the eligibility gate controls which records may enter the displayed result set. This distinction is recorded in the download so the baseline can be reproduced.

Encode the same titles with a pinned model

The embedding run uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2, pinned to revision e8f8c211226b894fcb81acc59f3b34ba3efd5f42. We executed its quantized AVX2 ONNX build locally on CPU, with attention-masked mean pooling and L2 normalization. Ranking uses cosine similarity.

The model produces 384-dimensional vectors and its published architecture uses a 128-token maximum sequence length. Our encoder applies that limit, so long titles are truncated for embeddings while BM25 sees the full title. The model card documents the model, pooling and license.

# documents and query_vector are L2-normalized arrays.
semantic_scores = documents @ query_vector
semantic_top = ranked(semantic_scores, eligible_indices, k=3)

The download includes vectors for all 217 titles and four queries, plus the exact model and tokenizer URLs and hashes. Replaying rankings requires no hosted inference. Fresh encoding is an optional separate step; model weights are not bundled.

Cosine and BM25 scores have different scales. Neither score is a probability that a tender is relevant or winnable. This example has no validated minimum cosine threshold, which allows the semantic top three to contain poor matches when the corpus does not contain a strong answer.

Filter stage, age and deadlines before ranking

The fixed retrospective cutoff is September 2, 2026, at 14:00 UTC. A row passes the example's gate only if it has opportunity stage, no listed closed/cancelled status, a usable publication date within the previous 30 days, and a known deadline after the cutoff.

| First failing check, or pass | Notice versions | | --- | ---: | | Not opportunity stage | 124 | | Unknown deadline | 71 | | Deadline elapsed | 6 | | Eligible for review | 16 |

These are first-failure categories, not overlapping counts of every issue. Stale publication and explicit closed-status handling are covered by synthetic tests; they do not appear as separate failure categories in this actual report.

One parsing detail matters: this corpus includes XML-style dates such as 2026-09-01+02:00. The helper interprets those at midnight in the stated offset. Plain dates use midnight UTC; for a date-only deadline that is a conservative start-of-day gate, not a claimed exact closing time. Naive date-time values without offsets remain unknown. A production collector should preserve source precision and define its own source-specific date policy.

An unknown deadline is not an unlimited bidding period. And passing these historical rules does not establish that a notice is open today, that no later cancellation exists, or that your company is eligible to bid.

Inspect the side-by-side results

After filtering to the 16 eligible-for-review records, both rankers produce the same first result for each query:

| Query | Shared top result | Title-level assessment | | --- | --- | --- | | Security monitoring and incident response | Irish SOC, SIEM and Managed Incident Response service | Plausible: the title explicitly names the requested services | | Software implementation and migration | “Velora 365,” under a software-package category | Insufficient: the short title does not establish implementation or migration scope | | Library refurbishment project management | University of Limerick library-refurbishment project-management consultancy | Plausible from the title | | Clinical consultation transcription | Garda victim/suspect interview video-recording equipment | Off-topic context despite related recording concepts |

The last result is a concrete failure. Police interview equipment is not evidence of clinical transcription work. Returning it first with a numeric score does not make it a useful sales lead.

Before filtering, both methods rank “Tortus Ambient Voice Technology” first for the clinical-transcription query. It is a plausible discovery cue, but the short title alone does not confirm clinical use, and its stage in the sample is contract. The metadata gate removes it. Both methods also initially rank an SAP S4Hana migration record first for the implementation query, but that row is an award.

Lower-ranked semantic results add different candidates, including application-maintenance work for the migration query. The review classifies those as adjacent rather than confirmed matches. The keyword baseline also produces noise, partly because broad terms and common words remain in its query representation.

The downloadable review covers every distinct query/document pair in the union of the two filtered top-three lists, plus the two unfiltered examples above: 20 pairs in total. These are editorial assessments of supplied titles, with “plausible,” “partial,” “insufficient” and “off-topic” judgments. No independent human labelling or full procurement-document review was performed. We do not report precision, recall or an accuracy uplift from this selected pool.

Reproduce the notebook and review

Extract the bundle, install the small replay dependency and run:

python -m pip install -r requirements.txt
python -B search.py --output reproduced
python -B -m unittest test_search.py

Open semantic-tender-search.ipynb in Jupyter using the same environment. The executed notebook shows the corpus, queries, filtering counts, top results and review entries. rankings.csv contains both the unfiltered and filtered results, with source identifiers, links, scores, stage and deadline alongside each title.

For fresh encoding:

python -m pip install -r requirements-embed.txt
python -B embed.py --cache model-cache --output regenerated-embeddings.npz

This optional command verifies the pinned model files and generates a separate vector file. The provided build targets x86-64 CPU; other architectures may require a different tested build. Compare arrays and rankings before recording a new receipt. Numerical scores can vary slightly across runtime or hardware combinations.

The notebook was executed during preparation. Twelve tests exercise lexical behavior, deterministic tie ordering, date parsing and eligibility edge cases. The saved-input workflow reproduces the three bundled output files on the tested environment. These checks establish implementation behavior, not business relevance.

Evaluate more than a convincing top result

For a stronger evaluation, collect real user queries and label results against explicit requirements. Include hard negatives, multilingual descriptions, sparse records and cases where no suitable notice exists. Keep a held-out query set when tuning lexical preprocessing, embeddings or a hybrid ranker.

Review the full available text before concluding that a title match is relevant. If attachments supply the actual requirements, title-only embeddings cannot stand in for document extraction. Measure retrieval quality separately from data coverage, deadline availability and duplicate procedure handling.

A hybrid approach may be worth testing, but this experiment does not establish its advantage either. The immediate lessons are narrower: exact lexical cues worked well for some queries, semantic neighbors sometimes crossed into the wrong context, and missing or unsuitable metadata removed most of the corpus from the candidate pool.

Keep the search index fed with current records

The input data remains a separate operational requirement. Preserve notice versions, source links, stages, deadlines and correction events so an index can remove or update stale records. Agree how missing dates enter a review queue rather than silently becoming open tenders.

For loading and normalizing the public records, continue with government tender data in Python. For category selection, use the CPV hierarchy and filter comparison.

WebTruffle's next step here is a maintained procurement data feed for your search index. This tutorial does not imply an existing hosted WebTruffle AI matching product.

Frequently asked questions