Chapter 8 · Integrations / Automation Engineer

Idempotent operations

idempotency · state guards · safe re-runsAUT-40253 of 84

An operation is idempotent if running it twice produces the same outcome as running it once. Sending an email is *not* idempotent — run it twice and the customer gets two emails. Setting a flag to True *is* — it's already true the second time.

This matters because automation gets re-triggered. A cron job double-fires, a worker crashes mid-run and restarts, someone clicks the button twice. If your script isn't safe to re-run, every one of those becomes a duplicate charge, a doubled notification, or corrupted state.

The pattern is a state guard: before doing the work, check whether it's already been done.

Keep a record of completed work — a seen-set of processed IDs, usually persisted to a file or database.
Before acting, ask is_done(task_id). If yes, skip.
If no, do the work *and then* record the ID so the next run skips it.

The order is load-amount: do the work first, *then* mark it done. If you mark it done before the work succeeds and the work crashes, you've recorded a lie.

Syntax

# Guard pattern
if is_done(task_id):
    return  # already handled — skip
result = do_the_work()
mark_done(task_id)        # record only after success

# seen-set persisted as a JSON list
done = json.load(f)      # e.g. ['job-1', 'job-2']
return task_id in done   # the guard check

Worked examples

A seen-set guard, persisted to disk

import json

def is_done(state_file, task_id):
    try:
        with open(state_file) as f:
            return task_id in json.load(f)
    except FileNotFoundError:
        return False   # nothing recorded yet

def mark_done(state_file, task_id):
    try:
        with open(state_file) as f:
            done = json.load(f)
    except FileNotFoundError:
        done = []
    if task_id not in done:        # no duplicates in the set
        done.append(task_id)
    with open(state_file, 'w') as f:
        json.dump(done, f)

print(is_done('/tmp/state.json', 'job-1'))   # False
mark_done('/tmp/state.json', 'job-1')
print(is_done('/tmp/state.json', 'job-1'))   # True
# Output:
# False
# True

run_once: do it the first time, skip the rest

import json

def _done(f):
    try:
        with open(f) as fh: return json.load(fh)
    except FileNotFoundError: return []

def run_once(state_file, task_id, fn):
    if task_id in _done(state_file):
        return {'skipped': True, 'result': None}
    result = fn()
    done = _done(state_file)
    done.append(task_id)
    with open(state_file, 'w') as fh: json.dump(done, fh)
    return {'skipped': False, 'result': result}

sent = []
def send_invoice():
    sent.append('INV-99')
    return 'sent'

print(run_once('/tmp/inv.json', 'INV-99', send_invoice))
print(run_once('/tmp/inv.json', 'INV-99', send_invoice))
print('times actually sent:', len(sent))
# Output:
# {'skipped': False, 'result': 'sent'}
# {'skipped': True, 'result': None}
# times actually sent: 1

Re-running the whole batch is harmless

processed = set()

def handle(order_id):
    if order_id in processed:
        return 'skip'
    processed.add(order_id)
    return 'charged'

batch = ['A', 'B', 'A', 'C', 'B']
print([handle(o) for o in batch])
# Output:
# ['charged', 'charged', 'skip', 'charged', 'skip']

Common mistakes & gotchas

Marking done before the work succeeds

If you record the ID *before* doing the work and the work then crashes, the next run skips it — the work never happens but the system thinks it did. Always do the work first, mark done only on success.

An in-memory set forgets on restart

A set() held in a variable resets to empty the moment the process restarts — exactly the situation idempotency is meant to survive. The seen-set has to be persisted (file, DB, cache) to guard across runs.

Idempotency keys that aren't actually unique

The guard is only as good as the ID. Keying on customer_name when two customers share a name, or on a timestamp that two events share, lets real duplicates through and wrongly skips distinct work. Use a stable, genuinely-unique key per unit of work.

Why it matters

Every scheduled job, webhook handler, and message consumer in production will be delivered or triggered more than once — that's a guarantee, not an edge case. Designing operations to be idempotent is what lets you re-run, retry, and recover without leaving a trail of duplicate charges and doubled emails behind you.