Reliability is a contract, not a prompt
The four disciplines that turn a convincing AI demo into a system people quietly depend on: contracts, evals, human-in-the-loop, graceful failure.
from pydantic import BaseModel, ValidationErrorclass Action(BaseModel):tool: strargs: dict[str, str]confidence: float # model's own estimate, 0..1def parse_action(raw: str) -> Action | None:try:action = Action.model_validate_json(raw)except ValidationError as err:# Feed the error back to the model for one bounded repair attempt.return repair(raw, err)# Low-confidence actions route to a human instead of executing.return action if action.confidence >= 0.8 else None