I use small models for tasks I can check, stronger models for harder cases, and separate approval gates for actions. A model that can reason better should not get more permission to change records or spend money.
Here’s the approach I recommend:
- Choose by test results: compare accuracy, response times and total cost per successful task - not just the price of a model call.
- Check before accepting: validate outputs against source records. Start with a limit of two corrected-context retries and one model-tier escalation.
- Approve before acting: a £250 refund proposal is not permission to pay. Bind approval to the exact action, then recheck records and permissions before execution.
- Start read-only: log decisions, name owners and address UK data protection duties before extending access.
My rule: escalate capability, not authority. Expand only when measured quality, safety and cost meet your agreed targets.
::: @figure 
{AI Agent Routing: Escalate Capability, Not Authority}
:::
LLM Model Routing: Stop Overpaying for Simple Tasks | Day 5
Choose Model Tiers by Task Difficulty and Risk
Keep authority controls separate from model choice. Then choose the lowest-cost model that can complete the task safely.
Set measurable thresholds before choosing a model. Test representative, labelled requests, including rare failures. Select the lowest-cost model that meets every mandatory quality and latency threshold. Compare factual accuracy, schema validity, error severity, p95 latency and cost per completed task with a fixed-model baseline. A cheaper call saves nothing if it leads to more corrections.[5][6]
Routing Before Generation vs Escalation After Generation
Routing chooses a model before generation, using rules, metadata or a classifier. Post-generation cascading starts with a small model and escalates only if its output fails checks. Route directly when difficulty is clear upfront. Use cascading when you can check the first output reliably and cheaply.[5][7]
| Approach | Decision point | Cost profile | Latency | Main failure mode | Suitable tasks |
|---|---|---|---|---|---|
| Pre-generation routing | Before generation, using rules, metadata or a classifier | Usually one generation call, plus routing overhead | Avoids an unnecessary first attempt | A difficult request goes to an underpowered model | Predictable tasks with useful signals, such as language detection, document type classification or known workflow categories |
| Post-generation cascading | After checking the smaller model’s output | Escalated requests incur the cost of multiple calls | Fast when accepted; slower when escalated | The checker accepts a plausible but incorrect answer, or escalation causes an unacceptable delay | Tasks with reliably verifiable outputs, such as extraction, classification and constrained summarisation |
| Fixed-model execution | One predetermined model per task class | Predictable, but may overpay for easy work | Simpler latency profile | The model costs too much for routine work or lacks capability for edge cases | Stable workloads with little variation, or where consistent latency matters more than optimisation |
Define Bounded Tasks for Small Models
Give each task a written contract covering input limits, allowed fields or labels, the output schema and acceptance checks. Classification needs an enumerated label set and calibrated confidence checks. Extraction needs source-span and type checks. Summaries of supplied context need claim-to-source checks. For format conversion, prefer deterministic code where possible, and check for data loss or invalid dates.
Tool-argument preparation also suits a bounded contract, provided you check account, amount, currency and idempotency fields. Schema-valid arguments propose an action; they do not authorise execution. Keep execution access out of the preparation stage.
Build a Task and Risk Routing Matrix
Use one matrix to separate task difficulty from action risk. Assess difficulty through ambiguity, reasoning depth, context length and required factual precision. Assess risk separately through reversibility and financial and privacy impact. Instructions can be easy to parse but dangerous to execute. Difficult read-only analysis may need a more capable model without needing action approval.
Conflicting records should trigger escalation - not a silent choice of whichever value seems most plausible. Record permission, spending and approval controls separately from model tier. The matrix below turns that separation into routing rules.
| Task class and difficulty/risk signals | Default model tier | Verification method | Escalation trigger | Permitted tools | Approval requirement |
|---|---|---|---|---|---|
| Fixed-label classification; short input, low consequence | Small | Label validation and calibrated confidence | Ambiguous or conflicting intent; confidence below threshold | Read-only ticket lookup | None for classification alone |
| Known-document extraction; precise fields | Small | Schema, types and source spans | Missing, duplicated or conflicting values | Read-only document retrieval | Approval only if later used for an action with serious effects |
| Supplied-context summary; bounded length | Small or medium | Claim-to-source and length checks | Unsupported claims, contradictory sources or context overflow | Read-only retrieval | Usually none for an internal draft |
| Format conversion; fixed transformation | Small or deterministic code | Parser, schema and round-trip checks | Data loss, invalid syntax or unexpected fields | No write tools by default | Required before a write with serious effects |
| Tool-argument preparation; possible financial or privacy impact | Small or medium | Schema, identity, allow-list and policy checks | Unknown recipient, unusual amount, missing idempotency key or policy violation | Preparation only; no execution | Required before any record-changing or spending tool |
| Conflicting records or multi-step exceptions | Medium or large | Step-by-step validation, record reconciliation and secondary review | Unresolved conflict, unfamiliar path or failed intermediate check | Restricted read-only tools initially | Human approval before action with serious effects |
Keep Quality Checks Separate from Action Approval
Check Output Quality and Set Escalation Rules
Once routing selects a model for a bounded task, validation decides whether to accept, retry, escalate or stop. Run deterministic checks outside the model. Schema validation catches malformed fields, business rules test eligibility, and evidence checks verify each claim against approved sources. Use parameterised, read-only SQL restricted to approved tables and columns. Test calculations and duplicate detection with representative fixtures.
Passing these checks does not prove correctness or entitlement. Set acceptance thresholds using labelled evaluations, not the model’s stated confidence. [1]
A pass means the output is accepted - not that execution is approved. Retry with corrected context for recoverable errors. Escalate conflicting evidence or reasoning beyond the task’s scope, and ask for clarification when facts are missing. Stop for prohibited access, suspected fraud or repeated failure.
Start with a cap of two corrected-context retries and one model-tier escalation. Restrict context to relevant records, policy and correspondence. Log the model, context version, validation results and escalation reason for every attempt. [1] Only after these checks pass should the execution gate decide whether the action can proceed.
Require Approval Before Tool Execution
Place the authorisation gate in the execution service, immediately before any record change or payment. Show the exact tool, version, arguments, affected records, amount, reason, evidence and policy.
Allow autonomous changes only through a narrow allowlist of explicitly pre-authorised, low-risk, reversible actions. Sensitive actions need human approval; prohibited actions stay blocked. Reviewers need the evidence, authority and skills to reject a proposal - not just confirm it. [2]
| Dimension | Model escalation | Human approval |
|---|---|---|
| Purpose | Improve the proposed answer | Permit or deny a specific action |
| Trigger | Conflicting evidence or failed quality checks | Sensitive or policy-controlled operation |
| Decision-maker | Routing policy selects a stronger model | Human with authority for that action |
| Control point | Within the quality workflow | Immediately before tool execution |
| Audit evidence | Model, context, failures and escalation reason | Approver, exact arguments, decision, timestamp and policy version |
Bind approval to the exact action and context. A generic user_confirmed flag is not enough. Verify the approver’s authority for the account, action and amount.
Before execution, re-read records and recheck permissions, eligibility and policy. Changed details invalidate approval. Denial, timeout or an unavailable policy service must fail closed. Check and consume approval atomically to prevent replay. A stronger model cannot bypass these controls. [8]
Example: A UK Refund Workflow
A refund workflow shows how these checks differ:
In this hypothetical UK e-commerce workflow, a small model extracts an order reference and a £250 refund request. Read-only checks reveal a partial-refund record, so a stronger model reviews the conflict. If eligibility supports a further £250 refund, the result remains a proposal; an authorised human must approve it separately. The gate shows the refund tool, order, payment method, amount, evidence and policy reason. Changed refund details invalidate approval, and the service rechecks prior refunds before execution.
Implement and Govern the Routing System
Once routing and approval rules are set, put them into a managed operating model. Small-model routing improves efficiency, escalation improves quality, and approval controls determine who can authorise actions.
Design the Routing System and Calculate Total Cost
Build a versioned pipeline: task classifier → routing policy → model gateway → small-model tier → evaluator → escalation controller. Use a separate tool-policy service to enforce execution rules at runtime, with telemetry recording decisions, outcomes and costs. Keep credentials and permission enforcement outside model control.
Assign named owners for model quality, security, data protection, finance limits, tool integrations and incident response. Choose hosted, private-cloud or hybrid deployment based on data sensitivity, location, retention and operating needs. Document supplier processing terms, training exclusions and exit arrangements. [4]
Measure cost per completed task, not just inference cost. Add spending on inference, evaluation, retries, escalation, retrieval, tool execution, caching, infrastructure, observability and human review, then divide by successful completions. For each task class, compare this with the cost of sending tasks directly to a stronger model. Frequent retries and escalation can make the small-model route more expensive.
Set spending limits in pounds (£) per task, workflow and month, with alerts before hard caps. When a limit is reached, queue work or stop safely. Never remove required checks to save money.
Evaluate Performance and Meet UK Governance Requirements
Test success cases and failure modes before extending access. Include easy, borderline, difficult and adversarial requests, covering prompt injection, missing information and write actions. Use labelled evaluations and expert review where outcomes cannot be observed directly. Apply consistent metric definitions:
| Measure | Definition |
|---|---|
| Acceptance rate | Accepted outputs ÷ evaluated outputs |
| Escalation rate | Tasks escalated to another model tier ÷ routed tasks |
| False acceptance | Incorrect or unsafe accepted outputs ÷ accepted outputs |
| False escalation | Unnecessarily escalated tasks ÷ tasks that could be completed safely without escalation |
| Approval volume | Human approval requests ÷ completed tasks |
| Completion quality | Correctly completed tasks ÷ eligible requests |
| End-to-end latency | p50 and p95 elapsed time from request to final outcome, including review waits |
| Cost per completed task | All workflow costs ÷ successfully completed tasks |
Record trace identifiers, model and policy versions, evaluation results, proposed tool arguments, approval decisions and execution outcomes. Minimise personal data, keep sensitive payloads separate from operational metadata, and apply access controls and retention limits.
Under the UK GDPR and Data Protection Act 2018, document applicable lawful bases, supplier arrangements and safeguards for significant automated decisions. Complete a DPIA before processing likely to create high risks to individuals, and make sure any required human review is meaningful. Reassess thresholds and relevant DPIA findings after changes to models, prompts, tools or policies. [3][9][10][11]
Start with a Read-Only Pilot and Roll Out in Stages
Begin with verifiable read-only tasks in shadow mode, without affecting users or records. Then introduce small-model routing with bounded escalation. Add approval controls before enabling sensitive tools.
Expand only while agreed quality, safety, latency and cost targets hold across representative traffic. Assign owners for policy changes, exception reviews and rollback. Approval bypasses, policy violations, or cost and latency breaches should trigger containment: disable affected tools, pin a known-good version or return work to the established process.
Conclusion: Escalate Capability, Not Authority
The rule is simple: escalate capability, not authority. Use small models for clear tasks whose results you can check. Switch to a stronger model when quality falls below the agreed threshold - not to grant broader permissions. A stronger model can improve a recommendation, but it cannot authorise execution.
Keep the approval gate separate from routing. Model escalation improves output; human approval permits action. Tie approval to the exact proposed action and its parameters. If either changes materially, require new approval.
Start with one read-only workflow and a documented routing matrix. Before rollout, agree on acceptance criteria: the required accuracy, complete audit records, zero writes without authorisation and a lower cost per completed task than the baseline. Expand only when the evidence supports it, not when the model merely sounds certain.
FAQs
How much test data do I need to set routing thresholds?
Start with 20 to 50 representative tasks, covering failed runs, successes and complex edge cases. Use these to set initial routing thresholds based on your quality and cost requirements.
Test the thresholds against a larger evaluation dataset, then validate your chosen routing policy on a separate holdout set. Monitor results and adjust thresholds to balance performance against costs in £, while keeping every segment above your minimum quality requirements.
What should happen if the strongest model still fails validation?
Stop automated retries to avoid unnecessary costs or incorrect actions, then follow the defined escalation path for human review.
Hokstad Consulting recommends saving the current state and hand-off record before blocking further tool or model dispatches. Include the run ID, stop reason, timestamp, best verified checkpoint and diagnostic context. This gives a qualified reviewer the evidence needed to resolve the failure, especially when money is being spent or changes cannot be undone.
How can I prevent approval fatigue for sensitive actions?
Use risk-based routing instead of requiring human sign-off for every task. Automate low-risk actions that can be reversed, and use post-action audits and execution logs to check what happened. For medium-risk tasks, trigger a review only when performance metrics or confidence scores fall below set thresholds.
Keep manual approval gates for high-risk actions: those with a major financial impact, a risk of exposing sensitive data or production changes that can’t be undone. This keeps safeguards in place while cutting unnecessary reviews.