Chapter 7 · AI-Curious Developer

JSON output guardrails: force and validate

JSON output · validate schema · retry on failureDEV-40650 of 84

Sometimes you don't want prose from a model — you want structured data your program can use directly. You ask it to reply in JSON, then build a guardrail around the answer, because the model will sometimes hand you malformed JSON, or valid JSON with the wrong shape.

The guardrail is four steps:

Parse — json.loads(text) turns a JSON string into a Python dict/list. It raises json.JSONDecodeError on bad JSON, so wrap it in try/except and return None on failure rather than crashing.
Validate the schema — parsing succeeding doesn't mean the data is *right*. Check it's a dict and that every required key is present: all(k in data for k in required_keys).
Combine — parse, then validate; if either fails, return None.
Retry — models are non-deterministic, so a call that returns junk now may return good JSON on the next try. Loop up to max_attempts, return the first valid result, give up cleanly if none work.

The stub below returns good JSON on even calls and junk on odd ones, so the retry loop is exercised deterministically. With a real model the variability is genuine — same pattern.

Syntax

import json

try:
    data = json.loads(text)            # parse
except json.JSONDecodeError:
    data = None                        # bad JSON -> handle, don't crash

# validate schema:
ok = isinstance(data, dict) and all(k in data for k in required_keys)

Worked examples

Parse safely with try / except JSONDecodeError

import json

def parse_json_response(text):
    try:
        return json.loads(text)
    except json.JSONDecodeError:
        return None

print(parse_json_response('{"name": "Riley", "score": 42}'))
print(parse_json_response("Sorry, I can't."))
# Output:
# {'name': 'Riley', 'score': 42}
# None

Validate the schema (right shape, required keys)

def validate_schema(data, required_keys):
    if not isinstance(data, dict):
        return False
    return all(k in data for k in required_keys)

print(validate_schema({"name": "Sam", "role": "admin"}, ["name", "role"]))
print(validate_schema({"name": "Sam"}, ["name", "role"]))
print(validate_schema([1, 2], ["name"]))
# Output:
# True
# False
# False

Retry until valid (give up cleanly)

import json

calls = 0
def flaky_llm(prompt):
    global calls
    calls += 1
    if calls % 2 == 0:
        return json.dumps({"name": "Riley", "status": "active"})
    return "I cannot do that."   # junk on odd calls

def extract_valid_json(text, required_keys):
    data = parse_json_response(text)
    if data is None or not validate_schema(data, required_keys):
        return None
    return data

def retry_until_valid(llm_fn, prompt, required_keys, max_attempts=3):
    for _ in range(max_attempts):
        result = extract_valid_json(llm_fn(prompt), required_keys)
        if result is not None:
            return result
    return None

print(retry_until_valid(flaky_llm, "get user", ["name", "status"]))
print(retry_until_valid(lambda p: "junk", "x", ["k"]))
# Output:
# {'name': 'Riley', 'status': 'active'}
# None

Common mistakes & gotchas

Catching the wrong exception (or bare except)

Catch json.JSONDecodeError specifically. A bare except: also swallows KeyboardInterrupt and real bugs in your own code, hiding problems you actually want to see. Be precise about what you expect to fail.

Valid JSON is not valid data

json.loads('"hello"') succeeds — it returns the string "hello", not a dict. json.loads("[1,2,3]") returns a list. Parsing only proves the *syntax* is JSON; you still have to check the *shape* (it's a dict, the keys you need are there) before using it.

Retrying forever (or with no backstop)

A retry loop with no max_attempts can hammer a real model indefinitely (slow and, in production, expensive) when it just won't comply. Always cap the attempts and return a clear None/error so the caller can decide what to do.

Why it matters

When a model's output feeds the next step automatically — populating a record, triggering an action — you can't eyeball it. Parse-validate-retry is the guardrail that turns unreliable generated text into data you can build on, and it's the foundation under every "structured output" / "JSON mode" feature you'll meet.