Mark Korotin
Applied AI

Reliability is a contract, not a prompt

The four disciplines that turn a convincing AI demo into a system people quietly depend on: contracts, evals, human-in-the-loop, graceful failure.

Stable · 8 min read

Reliability is a contract, not a prompt

A demo proves a model can produce the right answer. Production asks a harder
question: can it produce a usable answer, every time, or fail in a way the
surrounding system can absorb? The gap between those two is not model quality —
it is engineering. Reliable AI is a systems problem.

Four disciplines carry most of the weight:

  • Structured-output contracts — the model returns data against a schema, not
    free text you hope to parse. Validation is a gate, not an afterthought.
  • Evaluation loops — every change is measured against a fixed suite before it
    reaches a user. “It felt better” is not a metric.
  • Human-in-the-loop — consequential actions pause for approval. The agent
    proposes; a person disposes.
  • Graceful failure — when a step fails, the system degrades to a safe,
    legible state instead of guessing.

1. Make the output a contract

Ask for JSON against a schema and reject anything that does not conform. A
parse failure is a signal, not a crash — retry with the validator’s error fed
back, then fall back to a safe default. The contract is the seam where a
probabilistic model meets deterministic code:

Loading code…

The schema does double duty: it constrains the model and it documents the
tool surface. When the contract changes, the type error finds every caller.

2. Close the evaluation loop

You cannot improve what you do not measure. A small, versioned eval suite —
golden inputs with graded outputs — turns “the new prompt feels smarter” into a
number that either went up or down. Trace every run: inputs, tool calls,
latencies, the failing assertion. The trace is what you debug at 2am, not the
prompt.

3. Keep a human on the consequential path

The reliability pipeline is a sequence of gates. Each gate can pass, block, or
escalate. Autonomy is earned per-step, never granted wholesale:

flowchart LR
    A[Request] --> B[Agent plans]
    B --> C{Contract check}
    C -- valid --> D[Eval & trace]
    C -- invalid --> R[Repair / reject]
    R --> B
    D --> E{Human approval}
    E -- approve --> F[Execute: machines / files]
    E -- hold --> G[Queue for review]

The blocked path is not an error condition — it is the product working. A system
that can refuse is a system you can trust with real actions.

4. Fail toward the safe state

When a tool times out or a contract cannot be satisfied, the wrong move is to
improvise. The right move is to degrade predictably: return the partial result
you did validate, surface the failure to the operator, and never take a
mutating action on unverified data. Reliability, in the end, is just the
accumulated discipline of deciding — in advance — what happens when things go
wrong.