Structured LLM Outputs: JSON Schema, Validation, Retries, and Repair

A structured output contract has four layers. The schema names fields and allowed values. The prompt explains the task and expected meaning. The validator decides whether the model output can enter the application. The consumer assumes the validated shape is true enough to act on. JSON is only the serialization format between those layers.

The running example is ticket-classifier. Input is a support ticket. Success is { category, priority, confidence, reasons } where category and priority are enums, confidence is a bounded number, and reasons are short evidence strings. The intentional failure is a model returning a plausible but invalid priority such as urgent-ish; the app must reject or repair it before a queue route changes.

The path starts before the model call. Normalize the ticket input, choose the schema version, and send the task with clear field semantics. After the model responds, parse the JSON, validate it against the schema, run cross-field checks, and only then publish the classification or update a ticket. If validation fails, decide whether the error is repairable.

Repair is useful for formatting mistakes, missing optional fields, or enum spelling that can be mapped safely. It is not a license to keep asking until the model says what you wanted. If the model gives a valid shape with weak evidence, that is a quality or confidence failure, not a syntax failure. Route it to human review or a safer default instead of hiding it under retries.

Every retry is another model call with cost, latency, and possible new failure. Use retries for bounded, observable classes: malformed JSON, missing required field, or enum value outside the schema. Give the repair prompt the validation error and the original raw output, then ask for the same schema again. Cap attempts and record that repair occurred.

Do not retry semantic disagreement forever. If the schema is valid but confidence is low, evidence is missing, or the category conflicts with a business rule, the right terminal state may be needs_review. A repair loop should make the output conform to the contract, not launder uncertainty into a confident answer.

Structured output becomes an API the moment another component consumes it. A dashboard, queue worker, or CRM integration may depend on the exact enum names and required fields. Adding escalationTeam, renaming priority, or changing confidence from 0..1 to 0..100 can break consumers even if the model still returns valid JSON.

Version the output contract deliberately. Add optional fields first, keep old fields during migration, and publish versioned events or records. If an old consumer sees a new version it does not understand, it should fail closed or route to review rather than guessing. Structured outputs deserve the same compatibility discipline as REST payloads or message schemas.

Trigger. A prompt edit asks the model to be more nuanced about urgency. Symptom. Some tickets start publishing priority: "urgent-ish", and the queue worker silently maps unknown priority to normal. Root mechanism. The app parsed JSON but did not validate the enum before publishing, and the consumer had a permissive default.

Recovery: reject unknown enum values at the producer, add a repair attempt that maps only to allowed priorities, remove the consumer's permissive default, and replay affected tickets from raw traces. Prevention: add a regression case for invalid enum values, require a typed schema_failed state, and block prompt releases unless the golden set includes malformed JSON, invalid enum, low confidence, and valid happy path.