Home/Methods

Process · AI agents

Working with a fleet of AI coding agents without losing work

We run many AI coding agents at once, often on the same files and the same systems. Almost nothing we lost came from bad code. It came from coordination: agents sharing a scratch space, two runs writing one live file, a deploy that published someone else's half-finished work, and runs that died together and resumed blind. These are the rules we follow now.

Updated 2026-10-11

What this is for

Parallel agents multiply output, and they multiply the ways to lose it. Every incident below happened to us, with our own agents, on our own systems. In none of them did a model write bad code. Each was a place where two pieces of work met and neither knew the other was there.

The rules are ordered roughly by where the damage starts: shared space, live files, deploys, runs that stop, and the checks that decide a fix is finished.

Shared space is not private space

We started 14 agents in parallel, each on its own file, with the instruction to back up the source before changing it and restore it afterwards. They all chose the same obvious backup name, in the session's scratch directory. That directory looks private but is shared by every agent in the session. The backups overwrote one another, and the restores wrote one project's code into another. 9 of 16 files were affected.

A diff said the restored files were identical to the originals. Both sides of the comparison were already contaminated. What caught it was a hash taken before the run and compared after. Production was untouched only because the deploy had happened before the fan-out.

Rules. Any fan-out prompt demands unique backup names, or puts backups next to the file they protect. Measure the starting state instead of comparing two states that may both be wrong. Put the tree under version control, so the next incident is a checkout; that project went under git the same day.

One owner per live file

Two of our runs, started in parallel on related jobs, could both write the same live script. The first replaced it during its repair stage. The second had already built on the old version and was verifying when its verifier found the file changed underneath it. It reported, as critical, an uncoordinated actor. The actor was our other run. Nothing harmful happened only because neither had applied anything yet.

Each prompt described the system as it was at launch, and neither knew the other was allowed to write. Parallel runs collide at the first repair, not the first read.

Rules. Before starting parallel runs, list the live files each one can write. Give every file one owner and tell the other runs it is read-only. Where a step depends on live state, check its checksum immediately before the step, not the one recorded at launch. When a run reports an unknown actor, look at your own other runs first.

Inside a shared folder the same idea works at a smaller scale. Make targeted edits rather than whole-file rewrites, and re-read just before editing, so that a stale match fails safely instead of overwriting. Complete another session's intent rather than replacing it. On an area that belongs to someone else, a small, clear correctness fix can be applied and noted; a judgement call is reported with the exact fix, not made.

A deploy publishes the disk, not your intent

One deploy script synced the whole tree to production. We deployed six times without touching another agent's files and still published its work in progress, because a deploy does not ask who wrote a file. The build passed and the health check answered. Nothing signalled it.

On another day we published two of our apps with code we had not intended: one carried another app's file, the other showed a debug probe in its panel. We had deployed from a tree that a parallel run was modifying. The push reported success, correctly, because the transfer worked. A successful push says the transfer worked, not what was transferred.

We then made it worse. We said the probe had not reached users, on the strength of a search through the git history. It had never been committed, so the search could not have found it. Absence from git is not evidence of absence from production.

Rules. Before deploying from a shared tree, look for recently modified files you did not write, and tell their owner before, not after. Do not deploy while a run is touching source. After deploying, compare the served source with the local source, file by file. Our deploy script now does this, stops before switching versions if they differ, and refuses to start with uncommitted changes. Its first run found a second contamination we had not been looking for, which is how we knew the check was worth having.

Runs die together; resume from evidence

An account's session limit stops every agent at the same moment. Once, nine runs died together. Work was left uncommitted in worktrees, and one step that wrote to production was half done. A resumed agent starts from zero with the same prompt, unaware that half its steps have already happened.

Interrupting the main session stops every background run too, with no failure notice. Eleven died at once that way. And not every interruption is what it looks like: one run's agents showed interruptions we blamed on new messages, but the run's own log showed a stall watchdog restarting them after about twenty-eight minutes without progress.

Sending a message to an agent inside a run did not reach the running agent. It started a second copy from its transcript, and the two worked in the same worktree, each convinced that another agent was overwriting its files. This happened twice in one day.

Rules. Every step that writes to production keeps a line-by-line log. Before resuming, read the real state: worktree status, the action log, the dead agent's working files. Put it in a resume note that says what is done, what was started and what was never logged, and tell the agent to review rather than redo. Relaunch in waves of at most three, in priority order. Do not message agents inside a run; apply new decisions after it ends.

A fix is not finished until something else checks it

Twice in one day a code run ended with its repair round reporting everything green and no verifier after it. In one of them, an independent verifier then found the repair was partial: a "cancel all" command opened the dialog for the opposite action, across the whole list. A fixer tests its own understanding of the problem. The decision to ship comes from an independent verifier on the final commit, rerunning its own attacks and the previous verifier's.

A single verifier can also cancel an angle by construction. We ran five independent angles, each followed by an adversarial verifier, but wrote that verifier for the main question. The fifth angle asked something different, so its findings had a different shape, and the verifier rejected every one: 3 raw findings, 0 confirmed. One of them was real and more urgent than the question we started with. Verifiers are now written per angle, and reports show raw findings next to confirmed ones, so a zero made by the harness cannot pass for a zero made by reality.

Where we were wrong

Three times in one day we repaired a rule that was not running. One was called by nothing. One was switched off by configuration everywhere it could apply. One had a guard that compared ranks with greater-or-equal and so excluded exactly the case the rule existed for: it resolved 0 of 2,258 cases, and 47 once the comparison was corrected. A fourth time, a test had disabled a guard so it could exercise the main path, and the guard itself was wrong in production.

Code that does not run looks identical to code that runs and finds nothing. Before changing a rule we now ask it two numbers: how many times it ran, and on how many cases it produced a result, measured where it is actually called.

We also tested a fix only where it was needed. A new rule looked flawless on 160 failing cases. Run on 400 cases known to work, it would have rewritten 91 of them. The failing set holds no counterexamples, because anything looks like an improvement on something that does not work. A fix is tested on what already works, and the target there is zero changes.

Limits

These rules reduce collisions; they do not detect all of them. They depend on knowing which files a run can write, and agents do not always say. Hashes and served-source comparisons prove a state at one moment, not that nothing changed afterwards. Waves and independent verifiers cost time, and we pay it knowingly.

Checklist

Unique names in any shared space. One owner per live file, and a checksum immediately before any step that depends on it. No deploy while a run edits source; compare served with local afterwards. Production steps log line by line. Resume from state you have read, with a resume note, in small waves. No messages into a running workflow. An independent verifier is the last step, written per angle, reporting raw counts. Ask a rule how often it ran before you fix it, and test the fix on what already works.

Questions and answers

How do you stop parallel AI coding agents from overwriting each other's work?

Parallel AI coding agents are kept from overwriting each other's work by giving every live file one owner and telling the other runs it is read-only. Fan-out prompts should demand unique backup names, because a shared scratch directory looks private but is not. Agents should make targeted edits, re-read just before editing, and check a file's checksum immediately before any step that depends on it.

How should you resume AI agent runs after a session limit stops them?

AI agent runs stopped by a session limit should be resumed from evidence, not restarted blind. Read the real state first: worktree status, the action log and the dead agent's working files. Write a resume note saying what is done, what was started and what was never logged, and tell the agent to review rather than redo. Relaunch in small waves in priority order, and keep line-by-line logs for production steps.

Why can a deploy publish another agent's unfinished work?

A deploy can publish another agent's unfinished work because a deploy syncs what is on disk and does not ask who wrote each file. The build can pass and the health check answer while the wrong code ships. Before deploying from a shared tree, look for recently modified files you did not write, do not deploy while a run edits source, and compare served with local source afterwards.

Is it safe to send a message to an AI agent while a workflow is running?

Sending a message to an AI agent inside a running workflow is not safe, because the message may start a second copy of the agent from its transcript instead of reaching the running one. The two copies can then work in the same worktree, each convinced another agent is overwriting its files. New decisions should be applied after the workflow ends.

How do you know an automated rule is actually running before you fix it?

An automated rule is shown to be running by asking it two numbers before changing it: how many times it ran, and on how many cases it produced a result, measured where it is actually called. Code that does not run looks identical to code that runs and finds nothing. Rules can be called by nothing, switched off by configuration, or excluded by a faulty guard.

All methods