Choose a web scraping vendor with evidence, not claims.

Choose a web scraping vendor by testing whether it can deliver your defined data outcome under representative production conditions—not by comparing network size, feature lists, client logos, or the lowest quote. Give every candidate the same buying brief. Remove any option that fails a non-negotiable gate. Score the evidence behind the remaining claims, reproduce the most important claims in a buyer-controlled pilot, and move proven commitments into the contract.

That process matters because “web scraping vendor” can describe very different offers: a proxy network, a scraping API, a no-code tool, custom development, a fully managed recurring feed, or a licensed dataset. A provider can be excellent at one model and unsuitable for another. A comparison is fair only when each candidate is being asked to own the same outcome and responsibility boundary.

An evidence-first web scraping vendor selection process makes that boundary explicit before price, features, or presentation quality can distort the comparison.

This guide assumes you have decided to evaluate an external provider. If the operating model is still open, begin with the in-house versus outsourced web scraping framework. If the decision has already been made, use the 20 criteria below to turn a shortlist into an auditable selection.

The scorecard is deliberately evidence-first. A policy, sample report, independent examination, comparable reference, and result reproduced on your workload do not deserve the same score. The evidence register preserves that difference and shows where each proven item must land: the output specification, SLA, security schedule, pricing terms, change process, or exit plan.

Define the buying brief before you score vendors

Do not ask vendors to define the service while they are being scored on their ability to sell it. Write one brief before opening proposals. Every candidate should receive the same required outcome, known constraints, demand forecast, evaluation method, and response format.

The brief should define at least:

• source list or discovery boundary, including countries, locales, devices, sessions, and access conditions that affect the output;
• required entities, fields, relationships, identifiers, history, and source evidence;
• coverage, completeness, validity, accuracy, uniqueness, consistency, and timeliness rules that matter to the intended use;
• cadence, delivery windows, destinations, formats, manifests, replay, and backfill requirements;
• expected source, page, record, transfer, and change volumes across normal and peak periods;
• handling of unavailable, uncertain, duplicate, changed, quarantined, and corrected records;
• monitoring, incident communication, recovery, and consumer-notification expectations;
• security, access, retention, deletion, location, subprocessor, and audit requirements;
• data ownership, documentation, portability, termination, and transition expectations; and
• commercial horizon, forecast scenarios, procurement timetable, and pilot constraints.

Describe exclusions and unknowns as carefully as requirements. If login-based sources, historical collection, entity matching, image extraction, multilingual normalization, or business-rule enrichment are not included, say so. Otherwise one provider may price a complete operation while another quietly assumes that the buyer owns the hard work.

Use an existing production record when one is available. The maintenance-cost ledger can show incident patterns, engineering effort, difficult sources, change volume, and buyer-side work that a replacement must address. Use the web scraping TCO model separately to compare whole-life cost; the selection score should not hide retained buyer labor inside a vendor’s price.

The UK Government’s Sourcing Playbook is written for public-sector procurement, so its mandatory language does not automatically apply to private buying. Its sequence is still useful: define the service, establish objective evaluation criteria, use proportionate KPIs, test through a pilot, assess whole-life value, and plan continuity and exit. It also warns against allowing the lowest bid to dominate complex service selection. (UK Government Sourcing Playbook)

Make the comparison unit an accepted data service

Do not compare vendors on “pages scraped” if the business consumes accepted products, listings, companies, articles, or other records. Pages are an input. The buying unit is the recurring data service that reaches an agreed destination and passes its acceptance rules.

A compact service statement might read:

Deliver the approved product schema from the agreed retailer scope by 06:00 UTC each day, with source timestamps, run manifests, reason-coded coverage, field-level validation, quarantined exceptions, and a tested correction and replay process.

That sentence does more selection work than a long list of technologies. It tells vendors what outcome must be demonstrated while leaving room for different implementation methods. The operational data feeds overview provides another way to frame the recurring outcome and ownership boundary.

Give every vendor the same response structure

For each requirement, ask the vendor to provide:

1. its interpretation of the requirement;
2. assumptions, exclusions, dependencies, and buyer responsibilities;
3. the proposed method and accountable owner;
4. a dated artifact, demonstration, or independent source of assurance;
5. the method your team can use to verify the claim;
6. the price basis and change treatment; and
7. the contract section where the commitment should appear.

This makes missing evidence visible. It also stops proposal quality from becoming a proxy for delivery quality: a beautifully written answer without proof should not outrank a plainer answer supported by a relevant artifact and reproducible result.

Separate gates, weights, and evidence

Keep four distinct fields in the decision model. Mixing them into one score creates false precision.

Gates are pass/fail requirements. A high score elsewhere cannot offset a failed gate. Typical gates include source feasibility, required delivery method, data ownership, project-specific legal approval, mandatory security controls, processor terms when applicable, essential geographic constraints, and workable exit rights.

Weights express the relative importance of criteria that can trade off. Set a small scale such as 1–3 before reading proposals. A pricing team may weight change predictability heavily; a safety-critical monitoring use case may weight recovery and provenance more heavily. There is no universal weighting that fits every dataset.

Fit scores describe how well the vendor satisfies a criterion. Define criterion-specific 0–4 anchors before reading proposals, where 0 fails the requirement and 4 meets the desired condition in full. Fit comes from the observed or proposed result—not from how much proof exists.

Evidence levels describe confidence in that fit score. Use the 0–4 level as a ceiling: an assertion can support at most 1, documentation at most 2, a relevant demonstration at most 3, and a buyer-reproduced result makes the full fit range available. Strong proof can confirm a poor result; it does not improve that result. A bad outcome reproduced by the buyer is fit 0 with evidence level 4, not a score of 4.

For weighted criteria, a simple comparison is:

awarded fit = min(observed fit score, evidence ceiling)
weighted criterion result = pre-set weight × awarded fit / 4

Keep the raw fit score and evidence level visible beside the awarded fit and total. Two vendors can receive the same weighted result for very different reasons: one may show mediocre fit with strong proof, while another claims excellent fit without enough evidence to credit it fully. The decision record should preserve that distinction.

Do not publish a universal passing total. The twenty criteria are a practical structure, not an industry standard, and two 0–4 scales do not turn judgment into measurement science. Define what earns each fit score and evidence level, name the evaluator, record disagreements, and keep gates outside the arithmetic.

Build an evidence register

Use one row per requirement with these fields:

• requirement and consequence if missed;
• gate or weighted criterion;
• weight, fit anchors, raw fit score, evidence level, and any ceiling applied;
• vendor claim and proposed owner;
• artifact or demonstration requested;
• buyer verification method and owner;
• evidence scope, date, period, limitations, and expiry;
• observed pilot result and unresolved exception;
• contract destination; and
• recheck event or date.

This register prevents a common procurement failure: useful evidence is reviewed during selection, then disappears into email while the signed contract contains only generic promises.

1. Outcome and technical fit

The first four criteria test whether the vendor understands the same service and can plausibly deliver it. They do not reward a specific stack. A provider may use custom collectors, APIs, browsers, licensed sources, manual review, or a hybrid method. Score the fit between the method and the required outcome.

1. Does the vendor restate the required outcome accurately?

Ask the vendor to return an annotated version of the brief with assumptions, exclusions, dependencies, and buyer-owned work. A strong response distinguishes extraction from discovery, normalization, identity, validation, history, delivery, and ongoing operations. It explains what happens when a required field is absent at the source instead of quietly promising universal completeness.

Warning signs include a proposal that mostly describes the vendor’s platform, treats every URL as equivalent, or answers a managed-feed requirement with an API feature list. An unstated responsibility is likely to return later as a change request or a gap between teams.

2. Is source feasibility demonstrated at target level?

Do not accept “we can scrape any site.” Ask for a target-level feasibility log covering representative normal sources, difficult sources, edge cases, access dependencies, available fields, expected failure modes, and known constraints. The log should use reason codes rather than marking every missing result as a generic failure.

Feasibility is a gate when a required source, location, cadence, or field cannot be delivered through an approved method. A vendor should be willing to say that a target is unsuitable, a field is unavailable, or an assumption needs buyer approval. Honest constraints are evidence of mature discovery.

3. How will the provider measure coverage and discovery?

Coverage needs a denominator. Ask how the source universe is created and updated, how attempted and delivered entities are reconciled, how removals and new items are detected, and which exclusions are reported. For discovery-based work, require an observable method for showing what was searched and what was found.

A provider-selected sample can show that some records are possible. It cannot prove that the intended population is covered. Require a run manifest or equivalent report that distinguishes attempted, delivered, unchanged, unavailable, rejected, and failed items.

4. Does the responsibility boundary fit the integration?

Ask for a responsibility matrix spanning requirements, source approval, credentials, collection, parsing, normalization, validation, delivery, monitoring, incidents, corrections, replays, downstream acceptance, and consumer communication. Pair it with the proposed schema, transfer method, manifest, authentication, replay interface, and environment plan.

Test the delivery path early. A file is not integrated merely because it opens. Confirm schema compatibility, identifiers, time zones, encodings, partitions, retry behavior, duplicate handling, transfer integrity, and how consumers know a delivery is complete.

2. Data quality and proof

The next four criteria test whether “quality” is defined, measured, evidenced, and corrected. Do not ask for one accuracy percentage without a denominator, truth method, sampling design, field scope, and treatment of unavailable source values.

The Government Data Quality Framework treats quality as fitness for purpose and encourages assessment throughout the data lifecycle rather than one-size-fits-all assurance. Its commonly used dimensions include completeness, uniqueness, consistency, timeliness, validity, and accuracy. Select only the dimensions that matter to the data’s intended use and define them precisely. (Government Data Quality Framework)

5. Is there a measurable data contract?

Request a versioned schema with field definitions, data types, units, time semantics, null meanings, identity rules, source mapping, required-versus-optional status, and acceptance thresholds. Define the accepted output and the rules that stop a delivery, quarantine a record, or create a warning.

Use the dedicated web scraping data-quality framework to define coverage, completeness, validity, freshness, duplicates, source fidelity, and delivery integrity. Do not let a vendor substitute its generic quality policy for your project’s acceptance contract.

6. Can the provider show validation and provenance evidence?

Ask for a sample quality report, validation-rule inventory, source timestamps, lineage or source-evidence fields, schema-change record, and example exception report. A relevant artifact should show actual results, not only a blank template.

WebTruffle’s synthetic sample data-quality report illustrates the type of evidence a buyer can request: acceptance rules, observed results, quarantined exceptions, and delivery evidence remain separate. It is a specimen, not proof about any vendor.

7. Is the pilot representative and buyer-controlled?

Give each finalist the same target basket and approved output contract. Include ordinary sources, difficult sources, edge cases, multiple collection cycles, and a buyer-selected holdout sample that the provider did not choose. Record which parts of the pilot differ from production, including volume, support coverage, security, destination, and change frequency.

A polished demonstration on known pages can validate presentation. It cannot validate coverage, silent-error detection, recurring delivery, recovery, or change handling.

8. What happens to uncertain or defective data?

Ask the vendor to walk through one rejected record, one uncertain match, one late delivery, and one correction. Require the quarantine rule, owner, consumer impact, approval path, revalidation, corrected delivery, replay or backfill, and evidence that closes the defect.

Warning signs include silently filling absent values, overwriting prior good data with a bad delivery, changing schemas without a version, or treating downstream repair as the buyer’s unpriced responsibility.

3. Reliability and change

A scraper can return successful requests while the dataset is incomplete, stale, shifted into the wrong fields, or delivered to the wrong partition. The next four criteria test whether the provider operates the data service beyond initial extraction.

9. Does monitoring cover the delivered outcome?

Request dashboards and alert examples for source coverage, record counts, required-field completeness, validity, freshness, duplicates, distribution shifts, schema changes, delivery completion, and destination acknowledgements where relevant. Ask which signals page an operator, which hold delivery, and which create a review task.

Use the web scraping monitoring runbook to test whether monitoring spans collection, data quality, and delivery. A dashboard screenshot is documented evidence; a pilot alert reproduced on your workload is stronger.

10. Is incident ownership explicit?

Ask for severity definitions, coverage hours, on-call or escalation paths, acknowledgement and update expectations, buyer contacts, decision rights, and one anonymized incident timeline. The example should show detection, triage, containment, communication, repair, validation, redelivery, and prevention—not merely a ticket closure time.

Check whether the team presented during sales will operate the service. If responsibilities pass across product support, engineering, account management, and subcontractors, the handoffs should be visible.

11. Can the provider recover, replay, and backfill?

Require evidence from a recovery exercise or comparable incident. The proof should state the triggering condition, missing interval, retained source material, replay method, duplicate controls, revalidation, redelivery, and reconciliation with the destination.

Recovery promises are incomplete without retention and cost treatment. Clarify how long raw and processed evidence is retained, which recovery actions are included, when reprocessing consumes billable units, and who approves a historical backfill.

12. Can the provider handle change and sustain delivery capacity?

Ask for the change workflow from request through impact analysis, estimate, approval, development, validation, rollout, compatibility, and documentation. Test one controlled change during the pilot and record elapsed time, buyer effort, cost treatment, and any interruption.

Also examine operating-team continuity. Request role coverage, knowledge-management practices, documentation ownership, key-person dependencies, and the process for transferring context when staff or subcontractors change.

For a material service, add proportionate organizational and financial-capacity diligence: confirm the contracting entity and ownership, relevant delivery history, staffing and coverage, material concentration or key-person risks, and financial evidence appropriate to the contract’s scale and criticality. A reference, ratio, or account is a risk input—not a guarantee. Record mitigations and the signals that should trigger review after award.

4. Governance and risk

Governance evidence must be proportional to the service’s criticality and the data handled. Do not demand every certificate from a low-risk provider, and do not let a logo replace project-specific review for a high-risk feed.

NIST’s Cybersecurity Framework 2.0 supply-chain outcomes cover supplier criticality, contractual requirements, pre-contract due diligence, ongoing monitoring, incident involvement, and post-contract provisions. Its guidance is voluntary and non-prescriptive; alignment is not certification and does not prove that a specific data flow is secure. (NIST Cybersecurity Framework 2.0)

NIST’s July 2026 due-diligence guide describes supplier investigation across ownership and control, provenance, resilience, foundational cybersecurity practices, and supply-chain tiers. It is an ICT cybersecurity guide, not a complete commercial or data-quality assessment. (NIST SP 1326)

13. Is responsible collection evaluated per project?

Ask for the provider’s source-review workflow, escalation criteria, exception records, complaint or removal path, and the decisions it expects the buyer to make. Review actual sources, access methods, data categories, contracts, intended use, retention, and jurisdictions with your organization’s qualified counsel.

“Compliant vendor” is not a durable project conclusion. The same provider can support one approved source and use while another requires different rights, controls, or a decision not to collect. Treat project-specific approval as a gate and preserve who approved what.

14. Does security evidence match the service scope?

Request the current architecture and data flow, identity and access model, credential handling, encryption, logging, vulnerability process, incident process, assurance scope, exceptions, remediation status, and customer-dependent controls. Verify the legal entity, systems, locations, period, and subservice organizations covered by each artifact.

AICPA describes SOC as a family of CPA reporting services concerning controls at service organizations. Use the term SOC 2 report or examination, not “SOC 2 certification.” Review the auditor’s opinion, period, scope, exceptions, subservice organizations, and controls the customer must operate. The logo alone does not prove that your project is covered. (AICPA SOC overview)

The UK National Cyber Security Centre recommends a risk-based approach to gaining confidence in supply-chain security. Use your critical assets and exposure to decide how deep the assessment must go; do not send the same questionnaire to every supplier and call the process complete. (NCSC supply-chain assessment guidance)

15. Are privacy, retention, deletion, and minimization explicit?

Map what data the provider receives or creates, why it is needed, where it is processed, who can access it, how long each class is retained, how it is deleted, and what evidence confirms deletion. Separate source content, credentials, raw captures, extracted records, logs, backups, and support attachments.

Where a provider acts as a processor under applicable data-protection law, the contract may require documented instructions, security terms, assistance, audit information, subprocessor controls, and return or deletion at termination. The ICO’s checklist is UK-focused; determine the actual jurisdiction and roles before applying it. (ICO contracts and data sharing guidance)

16. Are subprocessors and continuity dependencies visible?

Ask for the subprocessor and critical-dependency inventory, service and data location, access, notification or approval process, downstream obligations, concentration risks, and alternatives. A provider may depend on cloud hosting, proxy networks, annotation teams, delivery services, or specialist data suppliers that materially affect the service.

For critical feeds, request business-continuity and disaster-recovery objectives, dependency maps, exercise dates, observed results, open corrective actions, and the buyer’s role. A plan that has never been exercised is documented intent, not demonstrated recovery.

5. Commercials and exit

The final four criteria test whether the proposal stays comparable after demand changes, incidents occur, and the relationship ends. Commercial clarity is an operating control, not only a negotiation topic.

17. Is the pricing model tied to forecast demand?

Request the charging unit, included work, setup, minimum commitments, tiers, overages, retries, failed attempts, browser or compute multipliers, bandwidth, storage, support, QA, change requests, backfills, new markets, indexation, and taxes where relevant. Run the same base, growth, difficult-source, incident, and exit scenarios through every proposal.

Keep the vendor-selection score separate from cost. Then use the web scraping TCO worksheet to include buyer-side integration, acceptance, governance, incidents, change, and exit beside the supplier fee.

18. Are service commitments measurable by both parties?

For every commitment, record the metric definition, numerator and denominator, data source, measurement owner, time zone, observation window, exclusions, report, escalation, and remedy or corrective action. “High accuracy,” “enterprise support,” and “near real time” are not service levels.

Do not import universal uptime, accuracy, response-time, or service-credit thresholds from another contract. Choose targets from the business consequence, source behavior, feasible method, and cost. A well-defined metric with an unsuitable threshold is still a poor commitment.

19. Is change governance priced and owned?

Ask what counts as defect remediation, routine source maintenance, configuration, minor change, and new scope. Define the intake, impact assessment, estimate, approval, priority, test, deployment, compatibility, documentation, and dispute path. Record lead-time ranges only when the vendor can tie them to a defined change class and evidence.

The governance rhythm should also be visible: operational reviews, quality reviews, commercial reviews, risk reviews, improvement actions, and named decision-makers. Avoid a relationship where every operational issue must be renegotiated through sales.

20. Can the buyer exit with usable data and knowledge?

Define ownership and permitted use of collected data, code or configuration where relevant, schemas, validation rules, documentation, manifests, history, incident records, and custom logic. Request export formats, transfer method, assistance, timing, fees, credential revocation, retention closure, and deletion evidence.

The Sourcing Playbook recommends planning exit early and connecting the outgoing supplier’s plan with the incoming supplier or in-house operation. It also treats supplier insolvency as a continuity risk for critical services. Adapt that pattern to the importance of your feed: require a usable exit path, not a vague promise to “provide the data.” (UK Government Sourcing Playbook)

Portability is not absolute. A useful managed capability may justify some dependency. Make the trade-off explicit by estimating exit time, buyer work, supplier assistance, data and documentation available, and which parts cannot be transferred.

Run a representative pilot before final selection

The pilot is not a free sample and not a miniature production promise. It is a controlled attempt to reproduce the most decision-relevant claims. Give finalists the same frozen brief, target basket, output contract, delivery destination, schedule, scoring rules, and disclosure requirements.

The Sourcing Playbook describes pilots as a way to learn the environment, constraints, requirements, risks, and opportunities of a service and to improve technical specifications. That principle applies here: a pilot should reduce uncertainty, reveal wrong assumptions, and produce contract-ready evidence—not manufacture a passing screenshot. (UK Government Sourcing Playbook)

Include ordinary work, hard work, and change

A representative pilot should include:

• normal sources that dominate expected volume;
• difficult sources and edge cases that drive risk or maintenance;
• multiple scheduled runs so freshness, change detection, and delivery can be observed;
• production-like authentication, destination, schema, manifests, and acknowledgements;
• a buyer-selected holdout sample for truth and source-fidelity checks;
• one controlled source, field, schema, cadence, or volume change;
• one recovery, replay, or backfill exercise; and
• a record of vendor effort, buyer effort, consumed units, exclusions, and unresolved defects.

Do not disclose every audit record in advance. Vendors need a fair specification, but the buyer should select some validation samples after delivery so the evidence is not limited to prepared examples.

Reconcile the result rather than arguing over one percentage

For every run, reconcile the intended scope to attempted, delivered, unchanged, unavailable, quarantined, rejected, and failed items. Use reason codes. Sample critical fields against retained source evidence or an approved truth set. Measure delivery timing from the agreed event, not from whichever timestamp makes performance look best.

Record known limitations. A conditional pass can be appropriate when an issue has a credible corrective action, owner, deadline, retest, and contract treatment. A failed gate remains a failure even when the aggregate pilot result looks strong.

Put each proven claim in its contract home

Map the final evidence register into:

• the service description and responsibility matrix;
• output schema, acceptance rules, and reporting schedule;
• service levels, incident, recovery, and change terms;
• security, privacy, retention, subprocessor, and audit schedules;
• pricing, forecast, overage, and backfill treatment; and
• ownership, portability, termination, transition, and deletion terms.

If the contract weakens a pilot commitment, lower the evidence level or record the exception. A result reproduced once without an ongoing reporting method, contractual owner, and recheck trigger will decay back into a claim.

Record the selection and the reasons

Create a short decision record that another person can understand without reconstructing the procurement inbox. Include:

• the approved service brief and versions evaluated;
• candidates and service models considered;
• gate results, exceptions, approvers, and rejected options;
• weights fixed before proposals and any approved changes;
• evidence scores with artifact scope, dates, owners, and limitations;
• pilot results, unresolved defects, corrective actions, and retest status;
• normalized TCO scenarios kept beside—not inside—the quality score;
• material contract deviations from the evaluated proposal;
• the selected provider and the reasons it fit the required outcome;
• minority or red-team concerns; and
• evidence recheck dates and material-change triggers.

Do not delete the evidence for losing vendors immediately if procurement policy permits retention. It can explain the decision, support negotiation, reveal whether the selected provider’s evidence has weakened, and reduce repeated work if the shortlist must be reopened.

Re-evaluate after signature

Vendor selection becomes vendor management. Recheck evidence when the service adds material sources, data categories, locations, subprocessors, delivery paths, security dependencies, or business-critical uses. Also recheck after severe incidents, repeated quality misses, ownership changes, financial distress signals, assurance exceptions, and major contract renewal.

There is no universal review interval. Set the cadence from criticality and change rate. NIST’s supply-chain guidance treats supplier risk across the lifecycle, not as a questionnaire completed once before signature. (NIST SP 1305)

Keep the shortlist decision separate from migration

Selection proves that a provider is the best-supported option under the defined evaluation. It does not prove that production can switch immediately. After contract and security conditions are met, follow the managed-service migration playbook for inventory, dual running, reconciliation, failure drills, cutover, rollback, and decommissioning.

Frequently asked questions