English for OpenAI Structured Outputs

“The model returned invalid JSON” used to be a routine complaint before structured outputs existed, and now the more useful conversation is about schema design and strict mode — the vocabulary that actually determines whether a response is guaranteed to be well-formed.

Key Vocabulary

Structured outputs — a feature that constrains a model’s response to conform to a provided JSON schema, guaranteeing valid, schema-conformant JSON rather than relying on prompting alone to produce parseable output. “We used to validate and retry on malformed JSON constantly — switching to structured outputs removed that whole failure mode, because the response is guaranteed to match the schema before it’s even returned.”

JSON schema — a formal specification describing the expected shape of a JSON object, including field names, types, and required fields, provided to the model so it knows the exact structure to produce. “The extraction is failing because our JSON schema marks that field as required, but the source document doesn’t always contain that information — either make the field optional or handle its absence downstream.”

Strict mode — a setting that enforces the JSON schema exactly, disallowing any deviation such as additional properties or type coercion, as opposed to a looser mode that permits some flexibility. “Turn on strict mode for this schema — without it, the model can technically add extra fields we’re not expecting, and our downstream parser isn’t defensive enough to just ignore them.”

Constrained decoding — the underlying mechanism where the model’s token generation is restricted at each step to only tokens that keep the output valid according to the schema, which is what makes the JSON guarantee possible rather than just likely. “This isn’t the model being well-prompted — it’s constrained decoding. The model is structurally incapable of generating a token that would produce invalid JSON against this schema, which is a much stronger guarantee than prompt-based formatting.”

Refusal — a structured response indicating the model declined to complete the request, distinct from a normal schema-conforming answer, which needs to be checked for separately since it doesn’t fit the primary schema. “Don’t assume every response matches our schema — check for a refusal first. If the model declines the request, the response comes back in a different shape, and skipping that check is a common source of downstream parsing errors.”

Common Phrases

• “Are we using structured outputs here, or still parsing free-form text?”
• “Is this field actually required in the JSON schema, or should it be optional?”
• “Is strict mode enabled, or could the model add fields we’re not expecting?”
• “Is this a schema guarantee from constrained decoding, or just a strongly-worded prompt?”
• “Are we checking for a refusal before assuming the response matches the schema?”

Example Sentences

Explaining a reliability improvement to a teammate: “We stopped seeing malformed JSON from this endpoint entirely after switching to structured outputs. It’s not that the model got better at following instructions — constrained decoding makes it structurally impossible to emit a token that breaks the schema.”

Reviewing a schema design: “I’d mark email as optional in this JSON schema, not required — some of our source records genuinely don’t have one, and with strict mode on, a required field with no valid value will make the whole extraction fail instead of degrading gracefully.”

Flagging a missing check in code review: “This code assumes every response matches the schema and parses it directly — we need to check for a refusal first, since a declined request comes back in a different shape and would throw an unhandled error here.”

Professional Tips

• Say structured outputs, not “we prompted it to return JSON,” when the schema is actually enforced — the distinction matters because one is a guarantee and the other is a best effort that can still fail.
• Design the JSON schema around what the data can actually guarantee, not what would be convenient downstream — marking a field required that isn’t always present in the source data causes systematic extraction failures.
• Default to strict mode unless you have a specific reason not to — the looser mode’s flexibility is rarely worth the unpredictability it introduces into a system expecting an exact shape.
• Explain constrained decoding when a teammate assumes structured outputs is just clever prompting — understanding it’s a decoding-level guarantee changes how much validation logic is actually still necessary downstream.
• Always check for a refusal before parsing a structured response — treating every response as schema-conformant without that check is a common source of silent failures.

Practice Exercise

1. Explain the difference between structured outputs and simply prompting a model to return JSON.
2. Describe what strict mode changes about how a JSON schema is enforced.
3. Write a sentence explaining why a refusal needs to be checked for separately from the main schema.