benchrWhen it breaks
My coding agent fixed the bug and broke three other things
Also asked as the agent broke working code · it refactored files I didn't ask about · it deleted my tests · the diff is enormous
Cause documented by a vendor or a published paper Not tested by benchr First logged Last checked
What is actually happening
The agent was given a goal and no boundary, so it optimized the goal. Everything it touched on the way was fair game.
Why
- Repository-wide agents are documented as being able to read and edit across a whole project. That reach is the feature; without a stated boundary it is also the risk.
- A model cannot tell which of your files are load-bearing unless something tells it.
- Maintainers absorbing agent-written pull requests at scale responded with exactly this: written repo rules, mandatory test plans, and coverage gates that fail the change rather than trusting it.
The quick fix
Work on a branch, and read the diff before the tests. A large diff for a small request is the signal, not the test result.
The real fix
Write the boundary down where the agent will read it — an AGENTS.md or CLAUDE.md at the repository root naming the build command, the test command, the files that must not be touched, and what a change must include. Then let CI, not attention, be the gate.
Step by step
- Never let an agent work directly on the default branch.
- State the scope in the request: which files, and explicitly which files are out of bounds.
- Put the rules in a file at the repository root rather than repeating them in every prompt.
- Require the test command to pass in the same run that produced the change.
- Review the diff by size first. Ask why anything unrelated moved.
Grounded in
- Work through an entire repository, not a pasted fileRepository-wide read and write is documented as a capability. The reach is the feature; without a stated boundary it is also the risk.
- Open-source projects are now writing their contributor guides for agentsThe concrete controls named here are the ones AutoGPT adopted, reported on the GitHub Blog in August 2026.