The missing test
AI happily ships code with a test that proves almost nothing. A slugify function arrives with exactly one assertion — assert slugify('Hello World') == 'hello-world' — and that single happy-path check sails past multiple spaces, punctuation, leading/trailing junk, and already-clean input, which is precisely where real implementations break.
Your job is the verification the AI skipped: a battery of asserts covering the cases it ignored. A good test isn't one that passes — it's one that *fails when the code is wrong*. A test that can't fail proves nothing; it's theatre.
The sharpest way to prove a test has teeth is a mutation test: run your verification against a deliberately *broken* implementation and confirm it raises. If verify(broken) stays silent, your test is hollow. A real regression test pins down behaviour so tightly that any drift — a future edit, a different model's rewrite — trips it.
Write tests by asking *what would a lazy/wrong version get wrong?* text.lower().replace(' ', '-') (the obvious shortcut) turns 'a b' into 'a---b', leaves the comma in 'Hello, World!', and gives '--hi--' for ' hi '. Each of those is one assert that catches it.
Syntax
# The AI shipped ONE test — it proves almost nothing:
assert slugify('Hello World') == 'hello-world'
# The verification you write — covers the cases it skipped:
def verify(fn):
assert fn('Hello World') == 'hello-world'
assert fn('a b') == 'a-b' # multiple spaces collapse
assert fn('Hello, World!') == 'hello-world' # punctuation gone
assert fn(' hi ') == 'hi' # trim leading/trailing
assert fn('already-clean') == 'already-clean'
# Mutation test — prove it can FAIL on a broken impl:
# verify(broken) -> must raise AssertionErrorWorked examples
The verification the AI never wrote
import re
def slugify(text):
s = text.lower()
s = re.sub(r'[^a-z0-9]+', '-', s)
return s.strip('-')
def verify(fn):
assert fn('Hello World') == 'hello-world'
assert fn('a b') == 'a-b'
assert fn('Hello, World!') == 'hello-world'
assert fn(' hi ') == 'hi'
assert fn('already-clean') == 'already-clean'
verify(slugify) # silent -> all pass
print('verify passed')
# Output:
# verify passedMutation test — the verification CATCHES a broken slugify
def broken(text):
# the obvious shortcut: only handles single spaces
return text.lower().replace(' ', '-')
caught = False
try:
verify(broken) # 'a b' -> 'a---b', not 'a-b'
except AssertionError:
caught = True
print('caught the bug?', caught)
# Output:
# caught the bug? TrueWhy the single happy-path assert was worthless
# The AI's only test:
assert 'hello-world' == 'Hello World'.lower().replace(' ', '-') # passes!
# ...so the broken version passes the AI's test too. It only breaks on
# the inputs the AI never checked:
print('a b'.lower().replace(' ', '-')) # a---b <- wrong
print(' hi '.lower().replace(' ', '-')) # --hi-- <- wrongCommon mistakes & gotchas
A test that can't fail isn't a test
If your verification passes the broken implementation as readily as the correct one, it measures nothing. Always sanity-check by mentally (or actually) running it against a wrong version — if it doesn't raise, add the assert that would.
Only testing the happy path
One assert on the example you were given re-tests what the code was written to do. Coverage means the *edges*: empty, multiple, punctuation, already-correct, boundary values. The bug lives in the case nobody asserted.
Mistaking 'the code ran' for 'the code is verified'
AI loves to write code, run it once on its own example, and declare it done. Execution without a discriminating test is a demo, not a verification. The deliverable is the failing-then-passing test, not the green run.
Why it matters
AI will generate functionality faster than anyone can read it — so the bottleneck becomes *trust*, and trust comes from tests that bite. Writing the regression test the model skipped is how you pin AI output down: it's the difference between code that happens to work today and code that stays correct after the next edit.