benchrWhen it breaks
It returns JSON most of the time, then wraps it in prose and breaks my parser
Also asked as the model added text around the JSON · invalid JSON from the API · it invented a field that isn't in my schema · my parser crashes randomly
Cause documented by a vendor or a published paper Not tested by benchr First logged Last checked
What is actually happening
You asked for JSON in words. Words are a request, not a constraint, so the model complies at whatever rate it complies.
Why
- Free-form prompting for JSON has no enforcement behind it. Retry loops around it are treating the symptom.
- Schema-constrained decoding exists and is documented: you supply the schema and the response is constrained to it.
- Even with the schema enforced there are documented escape hatches — a refusal, a safety rejection, or a response cut off at the token limit — so valid schema does not mean completed response.
The quick fix
Stop asking for JSON and start supplying a schema. The API feature is called structured outputs and it is a request parameter, not a prompt trick.
The real fix
Supply a flat schema, handle the three documented escape hatches explicitly, and still validate values at the boundary. The shape is guaranteed; the content is not.
Step by step
- Write the schema first, and keep it flat — deep optional nesting is where the unsupported features live.
- Confirm the model you are calling supports it before assuming the parameter is honoured.
- Branch on refusal, safety rejection and truncation separately from parse failure.
- Validate the values, not just the shape. A correctly shaped object can still be wrong.
- Never send user-generated text that has nothing to do with the schema — the documentation warns the model will fill it anyway, which is a hallucination path rather than a parse error.
If you are seeing an API error
Grounded in
- Get JSON that always matches your schemaThe adherence guarantee, the support floor and the three escape hatches were read from the vendor guide and are recorded there with the date.