Enterprise AI RFP Scoring: How Buyers Compare Vendors | Hokstad Consulting

Enterprise AI RFP Scoring: How Buyers Compare Vendors

Enterprise AI RFP Scoring: How Buyers Compare Vendors

The highest-scoring AI vendor should not win if it fails a mandatory requirement. Set pass/fail checks first, then compare qualifying bids using published weights, 1–5 ratings and verified proof.

A starting scorecard covers six areas:

  • AI capability and use-case fit: 25%
  • Security, privacy and governance: 20%
  • Architecture, integration and deployment: 20%
  • Evidence, delivery and support: 15%
  • Total cost and commercial terms: 15%
  • Portability, exit and operating risk: 5%

Agree outcomes, owners and minimum scores before issuing the RFP. Test vendors on the same tasks, check similar customer deployments and compare costs in £ over the same contract term.

Before award, check whether changes in usage or delivery assumptions affect the ranking. Record the reasons for selection, secure approvals and put agreed commitments into the contract.

::: @figure Enterprise AI RFP Scoring: From Mandatory Checks to Award{Enterprise AI RFP Scoring: From Mandatory Checks to Award} :::

1. Separate must-have requirements from scored criteria

Bring the agreed requirements register into the RFP, separating eligibility checks from scored criteria. Screen proposals against mandatory checks first, then score only those that qualify. A low score affects the award only where a published threshold says it does.

Publish the mandatory schedule, scoring method, thresholds and clarification rules before evaluation. Make clear which failures lead to exclusion.

Set pass-or-fail requirements

Keep mandatory checks focused on legal, operational and compliance needs: UK GDPR, data handling, access controls, audit logs, hosting, availability, backup, disaster recovery, resilience, incident notification and security assurance. Include contractual rights covering data export, termination, transition support, and data deletion or return.

Keep residency policy separate from legal duties. UK-only hosting may be your policy, while restricted international transfers require appropriate safeguards. Ask vendors to disclose where production data and backups are held, where support access takes place, and where subprocessors are located - not just the main hosting region.

Specify the proof needed for each check. This could include architecture or data-flow diagrams, independent assurance reports, sample audit logs, service-level schedules, incident-response procedures, disaster-recovery test results or contractual clauses. Check each item’s date, scope, system version and relevance to the proposed service.

Record each requirement as Pass, Fail or Clarification required, with an evidence reference and the reason for the decision. Resolve clarifications before scoring without allowing a materially different proposal. Exclude failures unless the published rules permit a lawful, documented exception approved in writing.

Define evidence-based ratings from 1 to 5

Set measurable rating anchors for each scored criterion and link them to the agreed procurement outcome. Use the 1–5 scale below, defining the required result and evidence for each criterion:

Rating Published standard
5 - Materially exceeds Exceeds the target in a relevant, measurable way, supported by verified evidence.
4 - Fully meets Meets the target in full, with sound evidence and no material limitation.
3 - Meets minimum Meets the published minimum, with acceptable limitations or dependencies.
2 - Below minimum Falls below the minimum or depends on major conditions.
1 - Fails or unsupported Does not meet the criterion or lacks adequate evidence.

For every rating, record the evidence reference, assumptions, why the anchor applies and any unresolved concerns. Score the evidence, not the supplier’s reputation. Use these records to support the weighted scorecard and award decision.

2. Build a weighted scorecard

Turn the requirements register into weighted categories, minimum ratings and scoring rules before opening bids.

Set weights for 6 categories

Hold a short weighting workshop with procurement, security, data protection, IT, operations, finance, legal and the business owner. Each stakeholder allocates 100 points. The group then agrees a final set of weights before issuing the RFP.

Start with: AI capability and use-case fit 25%; security, privacy and governance 20%; architecture, integration and deployment 20%; evidence, delivery and support 15%; total cost and commercial terms 15%; portability, exit and operating risk 5%.

Give security and governance more weight if the system will process sensitive data. Put more weight on architecture and integration if it must connect to legacy systems. Record the reasons for these choices.[8]

Category Weight What it measures Required evidence Minimum acceptable rating
AI capability and use-case fit 25% Accuracy, relevance, usability, safety and fit against agreed use cases Standardised test results, product documentation, model limitations, sample outputs and performance metrics 3/5
Security, privacy and governance 20% Controls for confidentiality, access, resilience, privacy, auditability and responsible AI Certifications, architecture diagrams, data-processing terms, retention policy, incident process and audit reports 4/5
Architecture, integration and deployment 20% Compatibility, APIs, identity, data flows, deployment options and scalability Technical design, integration plan, API documentation, performance assumptions and implementation schedule 3/5
Evidence, delivery and support 15% Strength of references, implementation method, skills, service levels and support model Relevant case studies, named team, delivery plan, SLAs, support coverage and escalation process 3/5
Total cost and commercial terms 15% Predictability, affordability, transparency and value over the contract term Five-year or agreed-term total-cost model, price card, usage assumptions, renewal terms and overage rates 3/5
Portability, exit and operating risk 5% Ease of migration, data export, dependency management and continuity risk Exit plan, export formats, transition assistance, subcontractor details and business-continuity controls 3/5
Total 100%

Approve these minimums before issuing the RFP. Score performance above the baseline, rather than awarding points for compliance alone. Count each benefit in one category only: task accuracy belongs under capability, export usability under portability, and export cost under commercial terms.

Calculate scores and set thresholds

Use weighted score = Σ[(rating ÷ 5) × weight], with weights expressed in percentage points.

A rating of 4 at 25% contributes 20 points out of 100.

If a category has subcriteria, give each its own weight and add their contributions. Do not score the category again.

Set and publish the subcriterion weights, permitted rating increments, rounding rule, overall threshold, category minimums and any critical subcriterion minimums, such as privacy, together. Keep calculations unrounded until the final total.

Carry forward the mandatory checks from section 1. Rank only bids that pass all three stages: mandatory checks first, category minimums next, and the overall threshold last.[8][9]

Compare costs over the same contract term

Ask every respondent to complete the same five-year cost schedule in £, excluding VAT, with VAT shown separately. Use the same expected-usage, high-usage and low-adoption scenarios, service levels and delivery scope.

Before reviewing bids, define the cost-to-rating formula or bands. Also agree how the 15% category will be split between price and commercial terms. Use a proportional method that awards the highest price score to the lowest compliant total. Resolve any missing scope before scoring.[10]

Five-year cost component Vendor A (£, excl. VAT) Vendor B (£, excl. VAT) Common assumptions to record
Implementation and configuration £ £ Same scope and milestones
Integration and data migration £ £ Same interfaces, data volumes and testing
Usage or licence fees £ £ Included volumes, minimum commitments and overage rates
Hosting and infrastructure £ £ Region, compute, storage and environments
Support and service management £ £ Same coverage, response targets and service levels
Training and change adoption £ £ Same learner numbers and roles
Buyer staffing £ £ Buyer-validated effort and common day rates
Security assurance £ £ Testing, audits and assurance throughout the term
Exit, export and transition £ £ Export format, assistance period and transition duties
Total contract cost £ £ Include annual price changes and renewal assumptions

With the scorecard fixed, test each vendor against the same evidence and tasks.

3. Assess vendor evidence consistently

Once the scorecard is fixed, assess every vendor against the same evidence standard. Use that evidence to confirm pass/fail decisions and apply the published rating scale.

Separate proven capabilities from claims

Use one evidence template for every bidder. Record the requirement ID, rating, evidence reference and type, assumptions, risks, dependencies, evaluator notes and follow-up questions.

Label each capability as available now, configuration-dependent, partner-dependent, pilot-only or roadmap. Label claims as proven, partially proven, unproven or contradicted. A contractual promise alone does not prove that a capability exists today.

Check performance through buyer-controlled tests and control documents, then make commitments enforceable through contract terms. Request model limitations, data provenance, architecture assumptions, governance controls, a named support owner, service levels and pricing dependencies. Apply the template consistently so each rating can be audited later.[4][6][11]

Use this record format for every scored criterion.

Criterion Vendor A: rating Vendor A: evidence reference Vendor A: assumptions Vendor A: risks Vendor A: follow-up questions Vendor B: rating Vendor B: evidence reference Vendor B: assumptions Vendor B: risks Vendor B: follow-up questions
Production accuracy Enter 1–5 Test report and version Data and configuration Error patterns Missing proof Enter 1–5 Test report and version Data and configuration Error patterns Missing proof
Governance and auditability Enter 1–5 Reviewed control documents Retention and access Control gaps Audit rights Enter 1–5 Reviewed control documents Retention and access Control gaps Audit rights
Support and service levels Enter 1–5 Service schedule Support tier and partners Exclusions Accountable owner Enter 1–5 Service schedule Support tier and partners Exclusions Accountable owner

Test vendors on the same AI tasks

After reviewing the documents, run the same buyer-controlled test pack for every shortlisted vendor. Keep instructions, approved data, expected outputs, evaluation criteria, time limits and tool access identical. Agree the measures and thresholds below before testing.

Keep raw outputs, failures, prompts, model versions, configuration, retrieval sources, hosting conditions and records of vendor interventions. Results apply only to the conditions tested. Complete the table with measured results, not demonstration claims.[4][5][11]

Test measure Acceptance threshold Vendor A result Vendor B result Conditions to record Observed limitations
Task success rate Agreed minimum % Record measured % Record measured % Task count and completion rules Failed task types
Factual accuracy Agreed minimum % Record measured % Record measured % Answer key and review method Error categories
Unsupported answers Agreed maximum % Record measured % Record measured % Refusal instructions and source access Unsupported claims
Response latency Agreed time limit Record seconds Record seconds Request count and concurrent load Slow cases
Human-review effort Agreed time budget Record minutes Record minutes Same case volume and review guidance Corrections required
Cost per task completed Approved £ limit Record £ Record £ Model, retrieval and platform charges Excluded costs

Align evaluators and resolve scoring differences

Calibrate the team with sample responses and the published 1-to-5 definitions. Before discussing results, collect independent scores with written reasons and evidence references.

Set a review trigger, such as a two-point difference or any disagreement over pass/fail status. Keep original scores, revised scores and reasons for each change. Declare conflicts of interest and exclude conflicted evaluators where necessary. Share clarifications consistently where procurement rules require it.[12]

Check references from similar deployments

Use references to check how deployments work in practice, validate test results and identify delivery risks that documents miss. They should not replace testing or contract review.

Ask customers with similar scope, scale, sector, data sensitivity and integrations the same questions. Cover implementation time, production accuracy, latency, support response, change handling, unexpected costs, incident handling, data export and delivered roadmap commitments.

Record the deployment date, scale, scope, model configuration and differences from the proposed project. Turn concerns into documented risks, follow-up actions or contract conditions. Adjust ratings only under the published rules.[11]

4. Check and document the award decision

Once scoring and reference checks are complete, check whether the recommendation is ready for approval. Work through the agreed gates in order: mandatory compliance → critical thresholds → risk review → weighted ranking → award conditions. A high overall score cannot make up for a failed mandatory requirement, a missed critical threshold, an unacceptable risk or an unresolved condition that blocks award.[9][13]

Check how assumptions affect the ranking

Keep the published award calculation fixed. Test only material assumptions that could change the final ranking, changing one variable at a time across plausible ranges. If a small change reverses the ranking, address the evidence gap through the published procurement process. Do not choose the scenario that favours a bidder.[9][13]

Scenario Change tested Ranking outcome Effect on ranking Action
Baseline Approved weights and forecast usage Vendor A leads A leads by 2.5 points Proceed to checks
Higher usage AI consumption rises by 20% Vendor B leads B leads by 1.1 points Validate consumption assumptions
Security emphasis Security weight rises from 15% to 25% Vendor A leads A remains first No ranking change
Longer implementation Delivery extends by three months Vendor B leads B leads by 0.6 points Investigate delivery evidence

Use these sensitivity checks to assess whether the ranking holds before drafting the award record.

Record the decision and approval reasons

Create one row per vendor, matching the final scorecard. Include mandatory outcomes, critical thresholds, category scores, overall score, total contract cost, weaknesses, unresolved assumptions, reference findings and award status. Use only four statuses: pass, fail, conditional pass, not evidenced. Compare total contract costs using the same contract term and demand forecast.[13]

Make the final decision easy to audit later.

Vendor Mandatory outcomes Critical thresholds Category scores Overall score Total contract cost Weaknesses and assumptions Reference findings Award status
Vendor A Pass Pass Security 4, delivery 4, value 3 78/100 £1.20m Integration effort requires validation Two comparable references satisfactory Preferred, conditional
Vendor B Pass Fail: resilience Security 3, delivery 5, value 4 82/100 £1.05m Recovery-time evidence incomplete One reference raised support concerns Excluded
Vendor C Fail: data residency Not assessed Security 4, delivery 3, value 4 - £1.10m Mandatory requirement unmet References not completed Excluded

State exactly what the approval covers: a pilot, permitted negotiation, final award or conditional award. Explain why the other bidders were not selected.

For each outstanding condition, record an objective closure test, owner, deadline and authorised approver. Document permitted exceptions and the reasons for them, but do not rewrite the published criteria or scoring retrospectively. Obtain the relevant budget, technical, security, legal, data-protection and operational approvals.

Carry requirements into contracts and delivery

Put material commitments into contract schedules and delivery milestones. Cover capabilities, acceptance measures, service levels, security duties, data-use restrictions, pricing and exit provisions. Give each remaining risk an owner, verification task, deadline and consequence if the required action fails. Keep pre-award conditions separate from go-live conditions.[6][7]

Conclusion: Follow agreed rules and verify evidence

Once evaluation is complete, check the results against the agreed outcomes and published award rules. Complete the mandatory checks, weighted scoring and evidence review using the published pass/fail gates, weights and 1–5 rating anchors.[8][9]

Use the same evidence, test tasks and reference questions for every vendor. Resolve scoring differences, and compare total costs using the same term, scope and usage assumptions.[5][13]

The scorecard informs judgement; it does not replace it. Record an auditable award decision that explains why the chosen vendor can deliver the agreed outcomes. Base it on verified evidence, risk review and recorded approvals - not the score alone.[9][13]

FAQs

How do we avoid double-counting overlapping criteria?

Define clear criteria that don’t overlap. Group related requirements in one category, and assign the same identifier to approved equivalents. Use a structured scorecard to link test evidence to each category. This prevents factors such as integration effort or cost from being rewarded or penalised more than once.

Have trained reviewers resolve any overlaps by checking whether the criteria cover separate needs or the same underlying business requirement.

What if a vendor cannot share confidential evidence?

Approval stays on hold until all mandatory evidence is provided and the residual risk is acceptable [1]. Include audit rights in the contract so you can check practices. Ask for specific details, such as the encryption algorithms used, rather than vague ‘industry-standard’ claims [2].

If remedial work cannot meet these requirements, reject the purchase [1]. If a blocker remains, put the project on hold rather than proceed without the required documentation [3].

How should we break a scoring tie?

Look beyond the numbers when comparing compatibility and risk. Weigh each option against your priorities: ease of integration, ecosystem support and long-term cost of ownership. Check whether your team already has the skills needed to achieve the expected technical performance. Any skills gaps can affect implementation success and the resources required.

Prioritise the vendor whose strengths reduce the remaining risk and best fit your existing workflows and technical dependencies.

Need help with your DevOps, cloud or AI plans?

Hokstad Consulting helps companies with DevOps transformation, cloud architecture and hands-on AI development — pragmatic consulting with measurable results.

Our services: DevOps on Retainer · Hosting & Cloud · AI Development & Strategy