Pydantic Validates AI Tool Arguments Before Execution | Hokstad Consulting

Pydantic Validates AI Tool Arguments Before Execution

Pydantic Validates AI Tool Arguments Before Execution

AI-generated tool arguments must pass validation before any tool runs. Pydantic v2 can reject incorrect types, missing fields and invalid values - for example, refusing "5" when a quantity must be an integer.

A safe tool flow has four steps:

  • Validate first: use strict types and side-effect-free checks, then pass on only the validated model.
  • Check permission and stock: valid arguments do not mean an action is allowed.
  • Return safe feedback: allow up to 2 correction attempts, then stop for human review. Permission denials stop immediately.
  • Test and log the boundary: rejected requests must leave stock unchanged, while duplicate requests must not repeat a stock change.

The guide uses Python 3.10+ and a stock-adjustment example to show how these checks fit together.

::: @figure Pydantic AI Tool Validation: Safe Execution Flow{Pydantic AI Tool Validation: Safe Execution Flow} :::

Pydantic Tool Validation: Stop Bad AI Agent Inputs

1. Define the argument model and strict types

This model acts as a gate between generated arguments and tool calls.

Make all five tool argument fields required by leaving out defaults. Keep only metadata optional. With extra="forbid", Pydantic rejects unknown fields with an extra_forbidden error instead of silently ignoring them.[2]

from typing import Literal

from pydantic import BaseModel, ConfigDict, Field, StrictInt, StrictStr

class InventoryArgs(BaseModel):
    model_config = ConfigDict(extra="forbid")

    request_id: StrictStr
    product_code: StrictStr
    reason: StrictStr
    quantity: StrictInt = Field(gt=0, le=10_000)
    action: Literal["add", "remove"]
    metadata: dict[str, StrictStr] | None = None

Reject type coercion for identifiers, reasons and quantities. StrictInt rejects "5", 5.0 and True, while the bounds allow 1–10,000 units per request. Literal limits the allowed actions; it does not control the types of other fields.[3][4]

Validate dictionaries with model_validate()

Validate raw agent output before running any business logic. Handle malformed JSON with a separate error before calling model_validate(). This helper accepts a dictionary or JSON string and returns a validated InventoryArgs.

import json

def validate_payload(raw: str | dict) -> InventoryArgs:
    if isinstance(raw, str):
        try:
            payload = json.loads(raw)
        except json.JSONDecodeError as exc:
            raise ValueError("Arguments are not valid JSON") from exc
    else:
        payload = raw

    if not isinstance(payload, dict):
        raise ValueError("Tool arguments must be a JSON object")

    return InventoryArgs.model_validate(payload)

Catch ValidationError at the boundary and return feedback so the arguments can be corrected. Do not proceed to execution.[2] Pass the validated model to the next validation layer, and execute the tool only if it passes that check.

Choose field, model or per-call strictness

Use one strictness control at a time. Per-call strictness overrides field settings. Test the exact validation path your agent uses: in strict mode, model_validate_json() can behave differently from model_validate().[4][5][6] Once the schema is strict, add business rules that Pydantic cannot express.

Validation approach Behaviour Appropriate use Risk for agent arguments
quantity: int = Field(strict=True) Strict for one field Sensitive fields New fields may lack strictness
StrictInt / StrictStr Strict field types Explicit contracts Other fields remain unchanged
ConfigDict(extra="forbid", strict=True) Strict by default Model-wide policy May reject expected representations
InventoryArgs.model_validate(payload, strict=True) Strict for this call Validation boundary Other paths may differ

2. Add side-effect-free business validators

Check product codes and reasons

Once the schema is strict, add validators for rules about field values. Place these methods inside InventoryArgs, with import re and from pydantic import field_validator above the class. Product codes must contain at least two uppercase letters, followed by one hyphen and exactly four digits. Reject invalid codes rather than silently changing identifiers.[7][8]

@field_validator("product_code", mode="after")
@classmethod
def check_product_code(cls, value: str) -> str:
    if re.fullmatch(r"[A-Z]{2,}-[0-9]{4}", value) is None:
        raise ValueError(
            "product_code must contain at least two uppercase "
            "letters, a hyphen and four digits"
        )
    return value

@field_validator("reason", mode="after")
@classmethod
def clean_reason(cls, value: str) -> str:
    value = value.strip()
    if not value:
        raise ValueError("reason must not be blank")
    return value

These after validators run after field-type validation, so .strip() receives a validated string. Always return the resulting value. Use ValueError, which Pydantic converts into ValidationError - not assert.[7][9]

Check field combinations and defaults

Define this nested model before InventoryArgs, using the existing imports. It replaces the metadata dictionary with named, typed fields. Setting validate_default=True sends the default None through validation too.

class AdjustmentMetadata(BaseModel):
    audit_reference: str | None = Field(
        default=None, validate_default=True
    )
    source: str | None = None

Next, import model_validator, replace the metadata declaration inside InventoryArgs, and add the method below. This after model validator checks the validated action and metadata together. For removals, it rejects missing or whitespace-only audit text and trims accepted text. It returns self when the rule passes.

metadata: AdjustmentMetadata = Field(default_factory=AdjustmentMetadata)

@model_validator(mode="after")
def require_remove_audit_reference(self) -> "InventoryArgs":
    if self.action == "remove":
        reference = self.metadata.audit_reference
        if reference is None or not reference.strip():
            raise ValueError(
                "metadata.audit_reference is required for remove actions"
            )
        self.metadata.audit_reference = reference.strip()
    return self

Keep permissions, approval checks and stock updates in execution code. Validators must never perform the requested action.

3. Return safe errors and bound retries

Turn validation errors into correction feedback

Turn ValidationError into safe, structured correction feedback. Pydantic groups multiple failures into one exception. Select safe details from errors() without exposing agent-generated arguments, tracebacks or internal messages.[7] Use the mapping below to keep feedback consistent across failure types.

Failure category Agent-facing feedback Internal diagnostic
Type error quantity must be an integer, not text. Pydantic type code, model name, validator version and correlation ID
Missing field Provide the required reason field. Missing-field code and validation path
Custom-rule failure Product code must match the approved format. Internal rule identifier and validator name, without exposing implementation details
Unexpected field Remove unsupported argument debug_mode. Extra-field code and schema version

Apply this mapping in safe_errors() to turn Pydantic failures into fixed retry feedback. Use messages defined by your application, and map an empty model-level error location to ["__model__"]. Redact any locations that should not appear in responses or logs.

Leave out raw ctx. Custom validators can put exception objects there, making direct serialisation unsafe.[7]

Allow up to 2 correction attempts

The code below keeps validation output separate from execution. The initial proposal is attempt 0, so MAX_CORRECTIONS = 2 permits three validation passes.

from pydantic import ValidationError
from uuid import uuid4

MAX_CORRECTIONS = 2

SAFE_MESSAGES = {
    "int_type": (
        "invalid_type", "Quantity must be an integer.",
        "Provide a whole-number quantity.",
    ),
    "missing": (
        "missing_field", "A required argument is missing.",
        "Provide the indicated field.",
    ),
    "extra_forbidden": (
        "unexpected_field", "The argument is not supported.",
        "Remove the indicated field.",
    ),
    "value_error": (
        "business_rule", "The argument fails a validation rule.",
        "Provide a value that satisfies the tool requirements.",
    ),
}

def safe_errors(exc: ValidationError) -> list[dict]:
    result = []
    for item in exc.errors():
        loc = list(item.get("loc", ()))
        code, message, action = SAFE_MESSAGES.get(
            item.get("type"),
            ("invalid_argument", "The arguments are invalid.",
             "Review the argument requirements and try again."),
        )
        result.append({
            "loc": loc or ["__model__"],
            "type": code,
            "message": message,
            "action": action,
        })
    return result

def run_tool(initial_proposal, request_correction,
             log_validation, authorise, execute_once):
    proposal = initial_proposal

    for attempt in range(MAX_CORRECTIONS + 1):
        correlation_id = f"val_{uuid4().hex}"
        try:
            args = InventoryArgs.model_validate(proposal)
        except ValidationError as exc:
            feedback = {
                "status": "validation_failed",
                "correlation_id": correlation_id,
                "attempt": attempt,
                "max_corrections": MAX_CORRECTIONS,
                "retryable": attempt < MAX_CORRECTIONS,
                "errors": safe_errors(exc),
            }
            log_validation(feedback)
            if attempt == MAX_CORRECTIONS:
                return {
                    "status": "human_review_required",
                    "feedback": feedback,
                }
            proposal = request_correction(feedback)
            continue

        if not authorise(args):
            return {
                "status": "authorisation_denied",
                "correlation_id": correlation_id,
                "retryable": False,
            }
        return execute_once(args)

Treat each failure separately: validation_failed allows correction within the retry limit; authorisation_denied stops the request. Execution failures must stay outside the correction loop.

4. Test the validation-to-execution flow

Run this harness with Pydantic v2 installed. The simulated agent uses fixed proposals, so you don’t need network access or credentials. Take the actor and operation ID from the application, not the generated payload.

The harness tests the boundary: invalid arguments fail first. Authorisation and stock checks run only after the input passes validation.

from typing import Literal
from pydantic import BaseModel, Field, ValidationError, model_validator

MAX_CORRECTIONS = 2

class StockRequest(BaseModel):
    product_code: str = Field(pattern=r"^SKU-[A-Z]{3}-\d{3}$")
    quantity: int = Field(gt=0)
    reason: Literal["customer_order", "replenishment"]
    purchase_order: str | None = None

    @model_validator(mode="after")
    def check_purchase_order(self):
        if self.reason == "replenishment":
            if not self.purchase_order or not self.purchase_order.strip():
                raise ValueError("Purchase order required")
        return self

def safe_errors(exc):
    messages = {
        "int_type": "Provide an integer quantity.",
        "greater_than": "Quantity must be greater than zero.",
        "missing": "Provide the required field.",
        "string_pattern_mismatch": "Use the approved SKU format.",
        "literal_error": "Use customer_order or replenishment.",
        "value_error": "Replenishment requires a purchase order.",
    }
    allowed = set(StockRequest.model_fields)
    return [
        {
            "loc": [e["loc"][0]]
                   if e["loc"] and e["loc"][0] in allowed
                   else ["__model__"],
            "type": e["type"] if e["type"] in messages
                    else "invalid_argument",
            "message": messages.get(e["type"], "Review the arguments."),
        }
        for e in exc.errors(include_input=False, include_context=False)
    ]

class SimulatedAgent:
    def __init__(self, proposals):
        self.proposals = proposals
        self.feedback = []

    def propose(self, feedback=None):
        if feedback is not None:
            self.feedback.append(feedback)
        index = min(len(self.feedback), len(self.proposals) - 1)
        return dict(self.proposals[index])

class StockTool:
    def __init__(self):
        self.stock = {"SKU-RED-001": 12, "SKU-BLU-002": 0}
        self.calls = 0
        self.completed = {}

    def reserve_stock(self, request: StockRequest):
        assert isinstance(request, StockRequest)
        assert type(request.quantity) is int
        self.calls += 1
        self.stock[request.product_code] -= request.quantity
        return {
            "status": "reserved",
            "remaining": self.stock[request.product_code],
        }

def authorise(actor, request):
    # This actor can reserve stock only for customer orders.
    return (
        actor in {"agent-warehouse-1"}
        and request.reason == "customer_order"
    )

def check_current_stock(request, stock):
    if request.product_code not in stock:
        return "unknown_product"
    if stock[request.product_code] < request.quantity:
        return "insufficient_stock"
    return None

def run(agent, tool, actor, operation_id):
    feedback = None
    for attempt in range(MAX_CORRECTIONS + 1):
        raw = agent.propose(feedback)
        try:
            request = StockRequest.model_validate(raw, strict=True)
        except ValidationError as exc:
            feedback = {
                "status": "validation_failed",
                "attempt": attempt,
                "retryable": attempt < MAX_CORRECTIONS,
                "errors": safe_errors(exc),
            }
            if attempt == MAX_CORRECTIONS:
                return {"status": "human_review_required",
                        "feedback": feedback}
            continue

        if not authorise(actor, request):
            return {"status": "authorisation_denied"}

        key = (actor, operation_id)
        if key in tool.completed:
            previous, result = tool.completed[key]
            if previous != request.model_dump():
                return {"status": "idempotency_conflict"}
            return dict(result)

        failure = check_current_stock(request, tool.stock)
        if failure:
            return {"status": failure}

        result = tool.reserve_stock(request)
        tool.completed[key] = (request.model_dump(), dict(result))
        return result

valid = {
    "product_code": "SKU-RED-001",
    "quantity": 2,
    "reason": "customer_order",
}
actor = "agent-warehouse-1"

# A correction succeeds; a duplicate returns the saved result.
tool = StockTool()
agent = SimulatedAgent([dict(valid, quantity="2"), valid])
result = run(agent, tool, actor, "reservation-001")
assert result == {"status": "reserved", "remaining": 10}
assert len(agent.feedback) == 1
assert agent.feedback[0]["errors"][0]["type"] == "int_type"
assert tool.calls == 1
assert run(SimulatedAgent([valid]), tool, actor,
           "reservation-001") == result
assert tool.calls == 1
assert tool.stock["SKU-RED-001"] == 10

# The test checks both the response and the inventory mutation.

# Each rejected path leaves inventory and call count unchanged.
cases = [
    (dict(valid, quantity="2"), actor, "human_review_required"),
    (dict(valid, quantity=0), actor, "human_review_required"),
    (dict(valid, reason="replenishment"), actor, "human_review_required"),
    (valid, "unauthorised-agent", "authorisation_denied"),
    (dict(valid, quantity=20), actor, "insufficient_stock"),
    (dict(valid, product_code="SKU-GRN-003"), actor, "unknown_product"),
]
for payload, caller, expected in cases:
    tool = StockTool()
    before = dict(tool.stock)
    agent = SimulatedAgent([payload])
    result = run(agent, tool, caller, "reservation-002")
    assert result["status"] == expected
    assert tool.calls == 0
    assert tool.stock == before
    if expected == "human_review_required":
        assert len(agent.feedback) == MAX_CORRECTIONS

print("All boundary checks passed.")

Assert both the returned status and the side effect. Here, corrected input reserves exactly 2 units, leaving 10. Each rejected path checks for zero tool calls and unchanged stock. The assertions also check that the agent gets two correction rounds before human review, and that a duplicate reservation returns the saved result without changing stock again.

Conclusion: keep validation separate from execution

At the boundary, the rule is simple: treat generated arguments as untrusted until they pass validation. Pass only the validated model to the tool. Keep authorisation, current-state checks and idempotency in execution code. Use strict types and side-effect-free validators to keep that boundary clear.

Return safe, field-level feedback so errors can be corrected. Allow no more than two correction attempts, then stop for human review. Permission denials must stop the request immediately.

For monitoring, log validation failures, exhausted retries and successful executions separately. Record only the tool name, schema version, error category, attempt count and correlation ID.[1][10]

For UK organisations putting these controls into practice, Hokstad Consulting provides AI strategy, implementation and custom agent development.

FAQs

How do I keep tool schemas and validators in sync?

Treat tool definitions as shared, versioned contracts. Keep names, types, enums and constraints in one source, then use it to generate both the model-facing schema and backend validation. This avoids maintaining the same rules by hand in two places.

At each deployment, run consumer-driven contract tests against a production baseline to block incompatible schema changes. Test every supported agent–schema–adapter path for backend compatibility.

Before execution, validate arguments against the live server schema. Reject extra properties and unsupported enum values.

How should my agent handle errors after validation?

Do not retry permanent validation or authorisation errors. Instead, return structured errors with stable codes, field paths and messages explaining how to correct the problem. For transient or recoverable schema-validation errors, allow limited retries - typically three to five - with exponential back-off and jitter.

Before retrying any operation that changes state, check its external status using an idempotency key to prevent duplicates. Quarantine tasks with unknown or conflicting outcomes for human review rather than retrying them autonomously.

How can I prevent duplicate actions during concurrent requests?

Make service handlers idempotent: repeated inputs must not repeat side effects. Build a stable idempotency key from the workflow ID, node ID and logical operation - not the attempt number.

Before executing or retrying an operation, check its external status using that key. This is especially important after a timeout. If the operation has completed, record the receipt rather than repeat the call. Use atomic checks when consuming approval records so that only one successful execution persists.

Need help with your DevOps, cloud or AI plans?

Hokstad Consulting helps companies with DevOps transformation, cloud architecture and hands-on AI development — pragmatic consulting with measurable results.

Our services: DevOps on Retainer · Hosting & Cloud · AI Development & Strategy