Chapter 12 · AI Architect / Lead

The confident-but-wrong output

happy-path green ≠ correctAIR-110179 of 84

AI writes code that *looks* right, reads cleanly, and returns the correct answer for the obvious example. None of that proves it works. The model optimises for fluent, plausible code — not for code that survives the inputs nobody demoed.

The classic tell is a function that is green on the happy path and wrong everywhere else. A median that returns nums[len(nums) // 2] gives 2 for [1, 2, 3] (correct, by luck) — but it never sorts, so [3, 1, 2] returns 1 instead of 2. The demo hides the bug; the bug is in the logic.

The move is always the same: don't argue with the code, write a test that breaks it. Pick an input where you can compute the right answer by hand and the code can't. If the test fails, you've proven the flaw — not guessed at it. If it passes, you've earned a little trust.

assert actual == expected is the whole tool. One line that either stays silent (pass) or raises AssertionError (fail, with the values in front of you).

Syntax

# The AI's code — green on the example it was shown:
def median(nums):
    return nums[len(nums) // 2]   # never sorts -> wrong

# The test that exposes it (you compute the answer yourself):
assert median([3, 1, 2]) == 2   # AssertionError -> flaw proven

# Pattern: pick an input the demo skipped + assert the TRUE answer.

Worked examples

The plausible-but-buggy AI function

def median(nums):
    # "the median is the middle one" — confident, and wrong
    return nums[len(nums) // 2]

print(median([1, 2, 3]))   # 2   <- looks correct, ship it?
print(median([3, 1, 2]))   # 1   <- WRONG, the median is 2

A test that FAILS on the buggy version (proves the flaw)

def median(nums):
    return nums[len(nums) // 2]

assert median([3, 1, 2]) == 2
# Traceback (most recent call last):
#   ...
# AssertionError
# -> the test caught it: 1 != 2

The fix — sort first, average the middle pair on even length

def median(nums):
    s = sorted(nums)
    n = len(s)
    mid = n // 2
    if n % 2 == 1:
        return s[mid]
    return (s[mid - 1] + s[mid]) / 2

assert median([3, 1, 2]) == 2       # passes
assert median([1, 2, 3, 4]) == 2.5  # passes
print("all green")
# Output:
# all green

Common mistakes & gotchas

Mistaking a passing demo for a correctness proof

An example the AI chose to show you is the *least* informative test — it's the case the code was written to satisfy. Green on [1, 2, 3] tells you nothing about [3, 1, 2], [], or duplicates. Always test the cases the demo skipped.

Being persuaded by the comment, not the code

# the median is the middle one reads like the author knew what they were doing. Comments are prose the model also hallucinated — they describe intent, not behaviour. Verify against what the code *does*, never against what its comment *claims*.

Reviewing by reading instead of running

Eyeballing AI logic feels like review but catches a fraction of bugs. The bar is a test that fails when the code is wrong. If you can't break it with an input where you know the answer, you don't yet know whether it's correct.

Why it matters

This is the founding principle of the whole stage: a green happy path is the start of verification, not the end of it. The engineers who get value from AI are the ones who reflexively reach for a breaking test instead of a thumbs-up.