Accept / reject / revise
Everything in this stage composes into one senior-review act: making the call. For each AI-written sample you deliver a verdict — accept, reject, or revise — and crucially, *with evidence*: a failing test, a security citation, or a correctness argument. A verdict without evidence is just a vibe.
The rule of thumb the framework encodes:
- accept — clean code that meets the spec and survives the edges. Evidence: it passes a real battery of tests.
- reject — a security hole (
security) or broken core logic (logic). The function does the wrong thing or opens a door; it has to be rewritten, not patched. - revise — sound code with a fixable gap, typically a missing edge-case guard (
edge-case). The shape is right; it needs one more line.
The machine that produces evidence at scale is a verification harness: a verify(fn) that runs a candidate implementation against the full spec, collects every mismatch or crash as structured evidence, and returns a verdict. A crash is captured as a failure, never propagated — the harness judges code, it doesn't fall over with it.
The proof a harness has teeth is the mutation test: it must accept the genuinely-correct implementation *and* reject every planted-defect variant (no thousands separator, sign in the wrong place, off-by-100 scaling, an outright crash) — each with evidence. A suite that can't fail a broken build is worthless. That's the capstone: build the thing that proves AI code correct or finds the defect, then deliver the verdict.
Syntax
# decision: 'accept' | 'reject' | 'revise'
# reason: 'ok' | 'security' | 'logic' | 'edge-case'
# security hole / broken logic -> reject
# sound code, missing guard -> revise
# clean + passes the battery -> accept
# The verification harness — evidence, not vibes:
def verify(fn):
failures = []
for arg, expected in SPEC_CASES:
try:
got = fn(arg)
except Exception as e:
got = f'{type(e).__name__}: {e}' # capture, never propagate
if got != expected:
failures.append({'input': arg, 'expected': expected, 'got': got})
return {'verdict': 'accept' if not failures else 'reject',
'failures': failures}Worked examples
The judgement framework — verdict with a reason category
_VERDICTS = {
'add': {'decision': 'accept', 'reason': 'ok'}, # clean
'login': {'decision': 'reject', 'reason': 'security'}, # SQL injection
'average': {'decision': 'revise', 'reason': 'edge-case'}, # no empty guard
'is_even': {'decision': 'reject', 'reason': 'logic'}, # inverted (n%2==1)
}
def review_sample(name):
return _VERDICTS[name]
print(review_sample('login'))
# Output:
# {'decision': 'reject', 'reason': 'security'}The verification harness + a correct implementation
SPEC_CASES = [(0, '$0.00'), (100, '$1.00'),
(123456, '$1,234.56'), (-500, '-$5.00')]
def verify(fn):
failures = []
for cents, expected in SPEC_CASES:
try:
got = fn(cents)
except Exception as e:
got = f'{type(e).__name__}: {e}'
if got != expected:
failures.append({'input': cents, 'expected': expected, 'got': got})
return {'verdict': 'accept' if not failures else 'reject', 'failures': failures}
def format_money(cents):
sign = '-' if cents < 0 else ''
c = abs(cents)
dollars, rem = divmod(c, 100)
return f'{sign}${dollars:,}.{rem:02d}'
print(verify(format_money)['verdict']) # accept
# Output:
# acceptThe mutation test — the harness must REJECT a defect, with evidence
# A crashing implementation must be captured, not propagated:
def mut_crash(cents):
raise ValueError('boom')
result = verify(mut_crash) # does NOT raise
print(result['verdict']) # reject
print(result['failures'][0])
# Output:
# reject
# {'input': 0, 'expected': '$0.00', 'got': 'ValueError: boom'}Common mistakes & gotchas
Delivering a verdict without evidence
'Looks fine' / 'seems off' is not review. Every call needs a backing artefact — the failing test, the security cite (pickle.loads is RCE), the correctness argument. Evidence is what makes the verdict reviewable by someone else and defensible to your future self.
Confusing reject with revise
Reject is for code that's *wrong at its core* — a security hole or inverted logic; rewrite it. Revise is for code that's *right but incomplete* — sound logic missing an edge guard; one line fixes it. Mislabel them and you either churn good code or ship a breach.
A harness that can't fail a broken build
If verify accepts the planted-defect mutants as readily as the correct implementation, it proves nothing — the same trap as the missing test, now at the harness level. Always confirm your suite rejects the no-separator, wrong-sign, off-by-100 and crashing variants before you trust its 'accept'.
Why it matters
This is the product's whole thesis. AI can write the code; the durable, compounding, human skill is *judging* it — proving it correct or finding the defect, then making the call with evidence. As models write more, the engineers who matter aren't the fastest typists — they're the ones who can look at confident AI output and say, with proof, accept, reject, or revise.