Blog · October 27, 2026

Valid JSON from your model can still be wrong.

Turning on structured output feels like the moment an AI feature grows up. You hand the API a schema, the model is constrained to it, and the regex that used to fish JSON out of chatty prose can finally go. The parse errors stop. That part is real. What it doesn’t buy you is a correct answer, and the bugs that remain are quieter than the ones you just fixed, because now they arrive looking well-formed.

What the schema actually enforces

Structured output (Gemini’s responseSchema, OpenAI’s strict json_schema mode, Anthropic’s equivalent) works by restricting which tokens the model is allowed to emit next. If the schema says the next thing is a string from a fixed list, every token that would leave that list is masked out. The output is guaranteed to have the right keys, the right types, and only values your enums allow.

That is a guarantee about grammar. It says nothing about meaning. A field typed NUMBER will contain a number; the schema doesn’t know whether 0.93 is a sensible place to put a phone on a screenshot. A field typed STRING will contain a string; it doesn’t know you meant a hex colour. The model is still predicting the most likely next token, it’s just doing it inside a fence.

Range and format keywords (minimum, maximum, pattern, maxLength) are where people expect the fence to do more. Support for them differs between providers and has changed more than once, and a keyword the API accepts isn’t always one the decoder enforces. Read the current docs for the model you’re calling, and treat anything beyond types, required keys and enums as a hint rather than a contract.

Four ways a perfect parse goes wrong

All four of these come back as valid JSON that matches the schema exactly.

In range for the type, out of range for the product. A layout coordinate of 1.4 is a fine number and an off-screen phone.
The right type in the wrong shape. A bounding box declared as an array of integers can come back with three integers, or with the bottom edge above the top one.
A required field with nothing true to say. If the schema demands a category and the input gives the model no basis for one, it will still produce one. Required means “always answer”, and a model told to always answer will guess.
Cut off before it closed. The constraint shapes each token; it doesn’t stop the output limit. Hit maxOutputTokens halfway through and you get the first half of a valid object, which isn’t valid JSON at all.

Design the schema so “I don’t know” is a legal answer

The third failure is the one to fix in the schema itself, because it’s the one you caused. Our screenshot extractor asks a vision model to find the app’s UI inside a photo or a framed marketing image so we can crop to it. The obvious schema is just the box. The one we ship has a boolean first:

{ "found": BOOLEAN,   // required
  "box":   [INTEGER]  // optional
}
// box = ymin, xmin, ymax, xmax
// on a 0–1000 scale

With only box required, an image with no app screen in it would still get a box, somewhere, because the schema leaves no other way to answer. With found available, “nothing here” is a valid output and the model takes it. The same idea works anywhere: a nullable field, an enum that includes "unknown", a status before the payload. Give the model a way to decline, and it guesses less.

Order matters for a related reason. The model writes keys in sequence, and later tokens are conditioned on earlier ones. A short reason or found field placed before the answer lets the model commit to its judgement first. Gemini exposes this directly with a propertyOrdering field on the schema; elsewhere, the order you declare properties in is the order to check.

Refusals are the other honest exit. OpenAI’s strict mode returns a separate refusal field when the model declines, instead of forcing the refusal into your schema. Code that only reads the parsed object treats a refusal as an empty success.

The parser is still a validator

Once the response arrives, the work that used to be “parse this” becomes “check this”. In ShotCanvas the AI designer returns a layout: where the phone sits, how big it is, the gradient angle, the shadow. The schema guarantees those are numbers. The parser then clamps every one of them to the range our templates can actually render: the phone’s horizontal centre to between 0.2 and 0.8 of the canvas, its tilt to ±8 degrees, the shadow blur to a band that reads as a shadow rather than a smudge. Each clamp has a fallback for when the value is missing or isn’t a number at all.

The extractor does the same for its box. Values are clamped to the image, a box with fewer than four numbers counts as not found, and a box too thin or too small to be a real screen is rejected. A crop that would leave the user with a 12-pixel sliver is worse than no crop, so a degenerate answer falls back to the original image.

Neither check is clever. What matters is where they sit: between the model and anything the user sees. A layout engine shouldn’t have to wonder whether its inputs are sane.

Why enums are worth more than the rest of the schema

If you take one thing from structured output, take the enums. They are the one place where the grammar guarantee and the meaning guarantee line up: a value from the list is, by construction, a value your code knows how to handle. Our design schema builds its font enum from the same list of fonts the renderer bundles, and its composition enum from the same recipe table the parser reads. Generating both from one source means a font added to the app shows up in the schema on the next call, and a font removed can never be suggested.

Free-text fields are where trouble collects. Wherever you can turn “describe the style” into “pick one of these six styles”, do it. Keep free strings for things that are genuinely open, like headline copy, and validate those against their own limits after the fact.

Keep the fallback for when the call fails outright

Truncation and refusals mean you will still, occasionally, receive text that doesn’t parse. Check the finish reason before you parse: a response that stopped on the length limit should be retried with more room, not handed to jsonDecode in hope. And keep a default ready for the case where every attempt fails. For our designer that’s a palette pulled from the screenshot itself, so the user still sees a sensible design instead of an error.

A checklist for any schema you ship: does every required field have a truthful answer for every possible input? If not, add a way to say “none”. Is every number clamped to a range your product can use, with a fallback? Is every free string either turned into an enum or validated after the fact? Do you check the finish reason and the refusal field before parsing? Four yeses, and a valid response is one you can trust.

Structured output removed a whole class of bugs from our AI features. It just moved the line: the model now always speaks your format, and checking what it says is still your job.

Try AI screenshot design → Testing features that never repeat