Return structured AI output without trusting its JSON

Your AI extraction feature returns valid JSON containing an invoice total, currency, and due date. The total is negative, the currency is unsupported, and the due date was inferred from a phrase that did not specify one. Parsing succeeded while the application contract failed.

Structured output reduces ambiguity about format. It does not establish that the values are correct or authorized. This article uses an illustrative invoice-extraction feature to separate syntax, schema, semantics, and evidence so machine-readable model output can enter a normal application safely.

Define the contract before the prompt

Write down the fields the application actually needs. For an invoice, that might include invoice number, currency, total in minor units, due date, and evidence references. Specify optional fields and an explicit representation for missing information.

Do not let the model's convenient response shape become the database schema automatically. A document may contain several totals, multiple currencies, or no due date. The application needs a policy for ambiguity rather than a prompt that demands a value at any cost.

Define which fields are proposals and which can become trusted state. Extraction should normally create a reviewable candidate record before it triggers payment or accounting actions. A parseable object is still derived from untrusted content and probabilistic interpretation.

Validate shape at the boundary

Use a runtime schema to check types, required fields, allowed enum values, and additional properties according to your contract. TypeScript interfaces alone do not validate a response arriving from a network call.

A conceptual schema can require an integer total, a currency from an allowed set, and either a valid date string or null. It can reject unknown fields if accepting them would create ambiguity. Keep the schema versioned with the application code so changes are deliberate.

Distinguish parsing from validation errors. Invalid JSON, a missing required property, and an unsupported currency have different causes and recovery options. Record safe error categories rather than dumping full document text into logs. Validation should occur before persistence and before any privileged operation.

Check business semantics separately

An integer total can still be wrong. Validate whether negative totals are allowed for the document type, whether the value falls within the product's supported range, and whether currency is consistent with account policy. These checks depend on the application, not JSON syntax.

Dates need semantic attention too. A string matching a date pattern may represent an impossible date or an unsupported format. Decide how timezone and date-only values are represented. Do not let a generic parser quietly reinterpret ambiguous day and month ordering.

Cross-field rules matter. If the document lists line items, compare their sum with the extracted total within the agreed rounding rules. A discrepancy should become a review state, not an automatic correction that invents a different invoice. The validator should expose inconsistency rather than erase evidence.

Require evidence for important values

Store the document location or text span supporting fields that matter. A total extracted from the wrong section can pass shape and business-range checks. Evidence references allow a reviewer or downstream verification step to inspect the basis of the value.

Validate that references actually point into the provided document. A model can invent a page number or quote. If a quote is required, check it against the source text with a documented normalization policy. Exact matching may need allowances for OCR artifacts, but those allowances should remain bounded.

Treat unsupported values as missing or uncertain according to the contract. Asking the model to always produce a due date encourages guessing when the invoice has none. An explicit null with a reason can be more useful than a confidently formatted invention.

Implement a bounded recovery path

When validation fails, decide whether to retry, ask for clarification, or route to human review. A retry can include concise validation feedback, but it should have a strict attempt limit and should not bypass the same checks.

Do not repeatedly request “valid JSON” if the underlying problem is missing evidence. A repair prompt may fix a property name while preserving an invented total. Recovery should distinguish a formatting defect from a semantic or factual defect.

Return a product-level outcome such as ready for review, unsupported document, or extraction failed. The UI should preserve the uploaded document and explain the next action. A generic internal error message leaves users unsure whether they should reupload, correct a field, or wait.

Use AI assistance to generate adversarial cases

Ask a coding assistant to create synthetic outputs that challenge the validator: unknown currencies, extra keys, nulls, strings instead of integers, huge totals, impossible dates, and contradictory evidence. Review the cases against your actual policy.

An illustrative prompt is: “Generate candidate extraction responses that are valid JSON but violate this contract. For each, state the rule violated. Do not include real invoices or private data.” This tests a common blind spot: confusing successful parsing with trustworthy structured data.

Build deterministic tests for these cases and verify that invalid responses never reach persistence or action execution. The related guide on small evaluation sets covers evaluating extraction quality beyond the validator's binary checks.

Know what validation cannot prove

A schema can enforce shape. Business checks can reject impossible or disallowed combinations. Evidence checks can confirm that a value appears in the document. None alone proves that the model selected the correct value for the intended meaning.

A document may itself be fraudulent or contradictory. The extraction feature should not claim to authenticate it unless separate controls do that work. Human review or deterministic reconciliation may be necessary for consequential use.

Provider-supported structured generation can improve compliance with an output shape, but your application boundary should still enforce its own contract. Providers, models, and configurations can change. Keeping validation under application control makes behavior explicit and preserves a defensible failure path when generation does not meet expectations.

Keep the validated candidate separate from the raw provider response. This allows a reviewer to inspect the original extraction without exposing unvalidated fields to ordinary consumers. Store only the source material and metadata permitted by your retention policy. When a reviewer corrects a total, preserve the distinction between the generated suggestion and the approved value. That distinction helps investigate future extraction failures and prevents a corrected record from being misrepresented as evidence that the model was originally accurate.

Sources

JSON Schema reference documents schema concepts for validating structured data.

Anthropic: Building effective agents discusses programmatic checks in model workflows.