---
title: "Fix invalid JSON from an LLM - causes, repair and retry"
description: "Why LLMs return broken JSON (code fences, prose, max_tokens cut-offs, trailing commas) and a tested Python function that repairs what is safe and rejects the rest."
url: https://broadrouter.com/blog/fix-invalid-json-from-llm/
lang: en
updated: 2026-11-15
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# How to fix invalid JSON from an LLM: causes, repair, retry

By the BroadNet Inference team · 7 min read · Published 2026-11-15 · Updated 2026-11-16

Check finish\_reason first: a reply cut off at max\_tokens cannot be repaired safely, so raise the limit and resend. Otherwise strip code fences and surrounding prose, decode the first JSON value, then fix only unambiguous mistakes such as trailing commas. Validate against a schema, and if that fails, send the exact error back once or twice. Structured output prevents most of this where supported.

## Key takeaways

-   Most broken JSON is wrapping, not bad JSON: a code fence, a sentence before or after, or a reasoning block. Python's raw\_decode reads the first value and ignores the rest.
-   finish\_reason "length" means the JSON was cut off. Closing the brackets makes it parse but silently loses data, so resend with more room instead.
-   Fix only what has one meaning: trailing commas, Python-style True/None and single quotes. Reject everything else and re-ask.
-   Python's json module accepts NaN and Infinity by default; RFC 8259 does not. Reject them, or other parsers downstream will fail.
-   On BroadNet Inference, only GLM 5.3 and Qwen3.8 27B document structured output. On the other models, use prompt-only JSON plus this repair and validation.

**On this page**

1.  Why do LLMs return invalid JSON?
2.  Why not repair truncated JSON?
3.  A repair function that only fixes the safe cases
4.  How should you retry after repair fails?
5.  Does structured output make this unnecessary?
6.  Where BroadNet Inference fits

Your code calls `json.loads()` on a model reply and gets `JSONDecodeError: Expecting value: line 1 column 1`. Or it parses, and a field is a string where you wanted a number. This post sorts the causes by what you can safely do about each, gives a tested Python function that repairs only the unambiguous cases, and shows a retry loop that knows when a retry can’t help.

Choosing between [JSON mode](https://broadrouter.com/glossary/json-mode/) and `json_schema` is covered in [JSON mode vs json\_schema](https://broadrouter.com/blog/structured-output-json-schema/). This post is about what to do with the reply you actually got.

## Why do LLMs return invalid JSON?

A model writes text one token at a time. Unless the server constrains decoding, nothing stops it adding a friendly sentence or running out of room. The common failures:

| What you see | Cause | Safe to repair in code? |
| --- | --- | --- |
| ` ```json ` … ` ``` ` around the object | Chat formatting habit | Yes: strip the fence |
| “Here is the JSON:” before, “Let me know…” after | Same habit | Yes: read the first value only |
| `<think>...</think>` before the object | Reasoning text in the content field on some servers | Yes: remove the block |
| Ends mid-string, brackets left open | Hit `max_tokens` | No: resend with more room |
| Empty content | Reasoning used the whole budget | No: resend with more room |
| `{"a": 1,}` | Trailing comma; JavaScript allows it, [RFC 8259](https://www.rfc-editor.org/rfc/rfc8259) does not | Yes: one meaning |
| `{'a': True, 'b': None}` | Python syntax | Yes: `ast.literal_eval` reads it safely |
| `NaN`, `Infinity` | Not JSON; Python accepts them by default | No: reject |
| Valid JSON, wrong types or missing fields | Schema not enforced | No: re-ask with the error |

The dividing line is ambiguity. If there is exactly one thing the model could have meant, fix it in code and log that you did. If you would have to guess, reject the reply.

## Why not repair truncated JSON?

Because the repaired version looks right and isn’t. We ran [json\_repair](https://github.com/mangiucugna/json_repair) 0.64.9, a popular MIT-licensed library, on a reply cut off mid-word:

```text
'{"name": "Omar", "city": "Jed'  ->  {'name': 'Omar', 'city': 'Jed'}
```

It returned valid JSON with the city “Jed”. Its README is open about this: it “automatically completes missing values in JSON fields with reasonable defaults”. That is useful for reading logs. For data you store or act on, it turns a visible failure into a silent one.

Check `finish_reason` before you parse. `"length"` means the reply hit [`max_tokens`](https://broadrouter.com/glossary/max-tokens/). Resending the same request with the same limit fails the same way, so double the limit or ask for less per call. Reasoning models make this more common: BroadNet’s [API reference](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-fix-invalid-json-from-llm&utm_content=body) says DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B reason before they answer, and the reasoning counts toward `max_tokens`. With a small limit, the answer can come back empty. Leave 1,000 tokens or more, or set `reasoning_effort` to `low`; [the reasoning budget post](https://broadrouter.com/blog/reasoning-model-max-tokens-budget/) has the numbers.

## A repair function that only fixes the safe cases

This uses only the standard library. It returns the parsed value and a list of the repairs it made, so you can count them.

```python
import ast, json, re

class Truncated(Exception): pass
class NotJSON(Exception): pass

FENCE = re.compile(r"```(?:json|JSON)?\s*\n(.*?)```", re.S)
THINK = re.compile(r"<think>.*?</think>", re.S)

def _reject_constant(name):                 # NaN, Infinity: Python accepts them, JSON does not
    raise NotJSON(f"non-standard constant {name}")

_decoder = json.JSONDecoder(parse_constant=_reject_constant)

def _first_value(s):
    """Decode the first JSON object or array in s, ignoring prose before and after it."""
    for i, ch in enumerate(s):
        if ch in "{[":
            try:
                return _decoder.raw_decode(s, i)[0]
            except json.JSONDecodeError:
                continue                    # a stray brace in prose; try the next one
    raise NotJSON("no JSON object or array found")

def _scan(s):
    """Drop commas before } or ] outside strings; report brackets left open."""
    out, stack, in_str, esc = [], [], False, False
    for i, ch in enumerate(s):
        if in_str:
            if esc:
                esc = False
            elif ch == "\\":
                esc = True
            elif ch == '"':
                in_str = False
        elif ch == '"':
            in_str = True
        elif ch in "{[":
            stack.append(ch)
        elif ch in "}]" and stack:
            stack.pop()
        elif ch == ",":
            rest = s[i + 1:].lstrip()
            if rest[:1] in ("}", "]"):
                continue                    # trailing comma: skip it
        out.append(ch)
    return "".join(out), len(stack) + (1 if in_str else 0)

def _python_literal(s):
    """Output in Python syntax: single quotes, True/False/None, trailing commas."""
    start = min((i for i in (s.find("{"), s.find("[")) if i >= 0), default=-1)
    end = max(s.rfind("}"), s.rfind("]"))
    if start < 0 or end <= start:
        raise NotJSON("no braces")
    value = ast.literal_eval(s[start:end + 1])       # literals only; never runs code
    if not isinstance(value, (dict, list)):
        raise NotJSON("not an object or array")
    json.dumps(value, allow_nan=False)               # must round-trip as strict JSON
    return value

def parse_llm_json(content, finish_reason=None):
    """Return (value, repairs). Raises Truncated or NotJSON."""
    if finish_reason == "length":
        raise Truncated("hit max_tokens; raise the limit or shorten the output")
    if not content or not content.strip():
        raise NotJSON("empty content (reasoning may have used the whole budget)")
    repairs = []
    s = content.strip().lstrip("\ufeff")
    if THINK.search(s):
        s = THINK.sub("", s).strip(); repairs.append("removed <think> block")
    m = FENCE.search(s)
    if m:
        s = m.group(1).strip(); repairs.append("removed code fence")
    try:
        value = _decoder.decode(s)
        return value, repairs
    except json.JSONDecodeError:
        pass
    try:
        value = _first_value(s); repairs.append("ignored text around JSON")
        return value, repairs
    except NotJSON:
        pass
    fixed, open_count = _scan(s)
    if open_count:
        raise NotJSON(f"incomplete JSON, {open_count} bracket(s) or string left open: treat as truncated")
    if fixed != s:
        try:
            value = _first_value(fixed); repairs.append("removed trailing commas")
            return value, repairs
        except NotJSON:
            pass
    try:
        value = _python_literal(s); repairs.append("parsed Python-style literal")
        return value, repairs
    except (ValueError, SyntaxError, TypeError, MemoryError, RecursionError, NotJSON) as e:
        raise NotJSON(f"unrepairable: {e}") from None
```

How each step works:

1.  **Truncation and empty replies** raise before any parsing, so they never become data.
2.  **Fences and reasoning blocks** are stripped with two regular expressions.
3.  **`raw_decode`** is the key tool. Python’s docs say it decodes a document “from a string that may have extraneous data at the end”, and returns where it stopped. Starting at the first `{` or `[`, it skips prose before the object and ignores prose after it. If a brace in prose fails to decode, it tries the next one.
4.  **The bracket scan** walks the text tracking strings and escapes. Brackets left open mean the reply was cut off even if `finish_reason` didn’t say so; that raises. Commas directly before `}` or `]` outside strings are dropped.
5.  **`ast.literal_eval`** reads Python-style output with single quotes, `True` and `None`. It evaluates literals only and never runs code. The result must then serialise as strict JSON.
6.  **`NaN` and `Infinity`** raise, through `parse_constant`. Python’s `json` accepts them by default; RFC 8259 says they “are not permitted”, and JavaScript’s `JSON.parse` rejects them.

We ran it on 17 test strings: clean JSON, fenced JSON, Arabic keys and values in a fence, prose before and after, a `{name}` placeholder in prose before the object, a `<think>` block, trailing commas in objects and arrays, commas and brackets inside strings, escaped quotes, a Windows path ending in a backslash, Python literals, `NaN`, an empty reply, a reply cut off with and without `finish_reason: "length"`, and two objects in a row. All behaved as the table above says. One test caught a real bug: an early version flipped its escape flag before checking for the closing quote, so `"he said \"hi,]\""` lost its comma. Test your own repair code on escaped quotes.

Two limits to know. Given two objects in a row, it returns the first; if you want several items, ask for one JSON array. And single-quoted Python output that contains a double quote inside a string can be misread by the bracket scan and rejected. Rejecting is the safe failure.

## How should you retry after repair fails?

Repair handles format. Validation handles content: right fields, right types, sane values. This loop validates with the `jsonschema` package and picks the retry by cause:

```python
import json, os
from jsonschema import Draft202012Validator
from openai import OpenAI
from llm_json import parse_llm_json, Truncated, NotJSON

client = OpenAI(base_url=os.environ.get("BASE_URL", "https://inference.broadnet.ai/v1"),
                api_key=os.environ["BROADNET_API_KEY"])
SCHEMA = {"type": "object", "additionalProperties": False,
          "required": ["name", "city", "amount"],
          "properties": {"name": {"type": "string", "minLength": 1},
                         "city": {"type": ["string", "null"]},
                         "amount": {"type": "number", "minimum": 0}}}
validator = Draft202012Validator(SCHEMA)

def extract(text, model="broadnet/qwen3.8-27b", max_tokens=1500, attempts=3, log=print):
    messages = [{"role": "system", "content": "Extract name, city and amount. Reply with one JSON object only, no code fence. "
                                              "Schema: " + json.dumps(SCHEMA)},
                {"role": "user", "content": text}]
    for attempt in range(1, attempts + 1):
        r = client.chat.completions.create(model=model, messages=messages, max_tokens=max_tokens,
                                           reasoning_effort="low", temperature=0)
        c = r.choices[0]
        try:
            value, repairs = parse_llm_json(c.message.content, c.finish_reason)
            errors = [f"{'/'.join(map(str, e.path)) or '(root)'}: {e.message}" for e in validator.iter_errors(value)]
            if not errors:
                log({"attempt": attempt, "repairs": repairs, "ok": True})
                return value
            problem = "The JSON does not match the schema: " + "; ".join(errors)
        except Truncated:
            log({"attempt": attempt, "ok": False, "why": "truncated"})
            max_tokens *= 2                 # same prompt, more room; a re-ask would be cut off too
            continue
        except NotJSON as e:
            problem = f"That reply was not valid JSON ({e})."
        log({"attempt": attempt, "ok": False, "why": problem})
        messages += [{"role": "assistant", "content": c.message.content or ""},
                     {"role": "user", "content": problem + " Return the corrected JSON object only."}]
    raise RuntimeError(f"no valid JSON after {attempts} attempts")
```

The rules in that loop:

-   **Truncated: resend the same messages with double the limit.** Adding a “please fix” message would make the prompt longer and the output no shorter.
-   **Not JSON, or wrong shape: send the exact error back.** “amount: ‘1200’ is not of type ‘number’” gives the model something specific to fix. A bare retry often repeats the mistake.
-   **Cap attempts at three.** Every retry resends the whole conversation as [input tokens](https://broadrouter.com/glossary/input-and-output-tokens/). After three, log the input and route it elsewhere.
-   **Log the repairs list.** A rising count of “removed code fence” after a prompt change tells you the prompt got worse, even though nothing failed.

We ran this loop against a local mock server that played back scripted replies. A fenced reply with a trailing comma and `"amount": "1200"` as a string was repaired, failed validation, and was fixed on the second call with the error sent back. A reply cut off with `finish_reason: "length"` was resent with `max_tokens` raised from 1,500 to 3,000 and succeeded. The mock recorded the request each time, confirming the message count and limit the loop sent. A real model can fail differently, so run it on 100 of your own inputs before trusting the numbers.

## Does structured output make this unnecessary?

It removes most of it. A server that constrains decoding to your schema cannot emit a fence, a trailing comma or a missing field. vLLM, a common open-model server, does this with the xgrammar or guidance backends. Two failures remain even then: truncation at `max_tokens`, and refusals. OpenAI notes that JSON mode output is valid “except for in some edge cases” and that your application “must detect and handle” output that is not a complete object. So keep the `finish_reason` check and the validator, even with `json_schema`.

What each approach prevents:

| Approach | Fences and prose | Syntax errors | Wrong shape | Truncation |
| --- | --- | --- | --- | --- |
| Prompt only (“reply with JSON”) | Reduced | Reduced | Reduced | No |
| JSON mode (`json_object`) | Yes | Yes | No | No |
| Structured output (`json_schema`, enforced) | Yes | Yes | Yes | No |

With JSON mode, OpenAI also says you “must always instruct the model to produce JSON” in a message; otherwise it may emit whitespace until it hits the token limit. Other OpenAI-compatible servers vary, so test the mode on the model you ship. The test is in [the structured output post](https://broadrouter.com/blog/structured-output-json-schema/).

To check a reply by hand, paste it into the [LLM JSON output validator](https://broadrouter.com/tools/llm-json-output-validator/), or build a strict schema from a sample object with the [JSON schema generator](https://broadrouter.com/tools/json-schema-generator/).

## Where BroadNet Inference fits

BroadRouter is BroadNet’s site. BroadNet’s API reference lists structured output for [GLM 5.3](https://broadrouter.com/models/glm-5.3/) and [Qwen3.8 27B](https://broadrouter.com/models/qwen3.8-27b/) only, and does not name the request parameter, so test `json_schema` on the model before relying on it. On [DeepSeek V4.1 Flash](https://broadrouter.com/models/deepseek-v4.1-flash/) and [Gemma 4 VL 26B](https://broadrouter.com/models/gemma-4-vl-26b/), use prompt-only JSON with the repair function and validator above. Three of the four models reason before answering, so budget `max_tokens` for it. For bulk extraction, the [batch runner post](https://broadrouter.com/blog/llm-batch-processing-without-batch-api/) shows how to log `finish_reason` per item. Prompts and replies are not retained after each request.

Free tools for this

-   [OpenAI to BroadNet migration helper](https://broadrouter.com/tools/openai-to-broadnet-migration/) Paste OpenAI SDK code; get base URL, key and model edits, a diff and the checks that remain.
-   [max\_tokens calculator for reasoning models](https://broadrouter.com/tools/reasoning-token-budget/) Answer length + reasoning budget + margin = a max\_tokens that does not cut the answer off.
-   [JSON schema generator for LLM structured output](https://broadrouter.com/tools/json-schema-generator/) Sample JSON to strict schema, Pydantic, Zod and a ready request.
-   [LLM JSON output validator](https://broadrouter.com/tools/llm-json-output-validator/) Parse errors by line and column, safe repairs, cut-off detection and JSON Schema checks for model output.

[All free tools](https://broadrouter.com/tools/)

## Terms in this article

[Token](https://broadrouter.com/glossary/token/)

A token is the unit of text a language model reads and writes: a word, part of a word, a punctuation mark or a byte sequence produced by the model's tokenizer. LLM APIs count, limit and usually bill usage in tokens.

[JSON mode](https://broadrouter.com/glossary/json-mode/)

JSON mode is an LLM API setting, response\_format {"type": "json\_object"} in OpenAI-style APIs, that makes the model's reply valid, parseable JSON. Unlike structured outputs with a JSON Schema, it does not guarantee which keys, types or values the object contains.

[Structured output](https://broadrouter.com/glossary/structured-output/)

Structured output is an LLM API feature that constrains a model's reply to a format the developer specifies, usually a JSON Schema, so code can parse it reliably. It is stricter than JSON mode, which guarantees valid JSON but not a particular shape.

[JSON Schema](https://broadrouter.com/glossary/json-schema/)

JSON Schema is a vocabulary, written in JSON, for describing and validating the structure of JSON data: types, required fields, allowed values and nesting. The current version is 2020-12. LLM APIs use it to define tool parameters and the exact shape a structured output must follow.

[Tool calling](https://broadrouter.com/glossary/tool-calling/)

Tool calling, also called function calling, is an LLM API feature in which the model returns a structured request to run a function the developer has defined, with JSON arguments. The application runs the function and sends the result back; the model never runs code itself.

[Context window](https://broadrouter.com/glossary/context-window/)

A context window is the maximum number of tokens a language model can work with in a single request. It covers the prompt, conversation history, tool definitions, images and the generated output, including any reasoning tokens. Anything that does not fit must be cut, summarised or split across several requests.

[Input and output tokens](https://broadrouter.com/glossary/input-and-output-tokens/)

Input tokens are the tokens a language model reads in a request, including the system prompt, conversation history, tool definitions and images. Output tokens are the tokens it generates in reply, including any reasoning. LLM APIs usually price them separately, and output usually costs more.

[vLLM](https://broadrouter.com/glossary/vllm/)

vLLM is an open-source library for LLM inference and serving, first developed at UC Berkeley's Sky Computing Lab. It manages KV cache memory with PagedAttention, batches incoming requests continuously, supports many quantization formats and exposes an OpenAI-compatible HTTP server.

## Questions

### Why does my LLM return JSON inside a markdown code block?

Chat models usually format code for people, and a code fence is how chat interfaces show it. Ask for "JSON only, no code fence", but strip fences in code anyway: the instruction lowers the rate, it does not remove it.

### How do I fix JSONDecodeError on LLM output?

Look at finish\_reason first. If it is "length", the reply was cut off: raise max\_tokens and resend. If not, strip code fences and decode the first JSON value with raw\_decode, which ignores text after it. Fix trailing commas, then validate. Re-ask with the error if it still fails.

### Should I use a JSON repair library?

For logs and debugging, yes. For data you act on, be careful. Libraries such as json\_repair complete missing values with defaults, so a truncated city "Jed" comes back as a valid "Jed". Reject truncated output and resend instead.

### Why is my LLM JSON output cut off?

The reply hit max\_tokens. On reasoning models such as DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B, reasoning counts toward that limit, so a long think can leave too little room. Raise the limit, set reasoning\_effort low, or ask for less output per call.

### Does JSON mode guarantee valid JSON?

Nearly. OpenAI says JSON mode output is valid JSON "except for in some edge cases", such as reaching the token limit, and it does not enforce your schema. Structured output with json\_schema adds schema adherence where the provider enforces it. Validate either way.

### Which BroadNet models support structured output?

BroadNet's API reference lists structured output for GLM 5.3 and Qwen3.8 27B, checked on 15 November 2026. It does not list it for DeepSeek V4.1 Flash or Gemma 4 VL 26B, so use prompt-only JSON with repair and validation on those.

## Keep reading

-   [JSON mode vs json\_schema: structured output that holds up](https://broadrouter.com/blog/structured-output-json-schema/)
-   [Tool calling with open models: schemas, parallel calls, retries](https://broadrouter.com/blog/tool-calling-open-models/)
-   [Setting max\_tokens for reasoning models: a budgeting method](https://broadrouter.com/blog/reasoning-model-max-tokens-budget/)
-   [Arabic text classification with an LLM: sentiment, intent, triage](https://broadrouter.com/blog/arabic-text-classification-llm/)
-   [LLM batch processing without a Batch API: a resumable runner](https://broadrouter.com/blog/llm-batch-processing-without-batch-api/)
-   [Translate app strings to Arabic with an LLM: placeholders and plurals](https://broadrouter.com/blog/translate-app-strings-to-arabic-llm/)
-   [A private endpoint starts with a clear workload](https://broadrouter.com/blog/private-endpoint-planning/)
-   [Before fine-tuning: define the problem to solve](https://broadrouter.com/blog/fine-tuning-readiness/)
-   [Guide: Why a reasoning model returns an empty answer](https://broadrouter.com/guides/reasoning-tokens-empty-response/)
-   [Guide: Use the OpenAI Python SDK with BroadNet Inference](https://broadrouter.com/guides/openai-python-sdk/)
-   [Guide: Read Arabic invoices and receipts with a vision model](https://broadrouter.com/guides/read-arabic-invoices-with-vision/)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-fix-invalid-json-from-llm&utm_content=page-cta) ·

**Sources**

-   [BroadNet Inference API reference: tools and structured output listed for GLM 5.3 and Qwen3.8 27B only; DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B reason, and reasoning counts toward max\_tokens (checked 2026-11-16)](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-fix-invalid-json-from-llm&utm_content=sources)
-   [OpenAI Structured Outputs guide: JSON mode vs schema adherence, instruct the model to produce JSON, whitespace until the token limit, incomplete output, refusals (checked 2026-11-15)](https://developers.openai.com/api/docs/guides/structured-outputs)
-   [IETF RFC 8259, The JSON Data Interchange Format: object grammar, double-quoted strings, Infinity and NaN not permitted (checked 2026-11-15)](https://www.rfc-editor.org/rfc/rfc8259)
-   [Python documentation, json module: JSONDecoder.raw\_decode, NaN and Infinity accepted by default, parse\_constant (checked 2026-11-15)](https://docs.python.org/3/library/json.html)
-   [json\_repair on GitHub (MIT): repairs malformed JSON and fills missing values with defaults (version 0.64.9 tested; checked 2026-11-15)](https://github.com/mangiucugna/json_repair)
-   [vLLM documentation, structured outputs: json\_schema, regex, choice and grammar with xgrammar or guidance backends (checked 2026-11-15)](https://docs.vllm.ai/en/latest/features/structured_outputs.html)
