benchrSomeone did this
They gave a coding agent the eval harness and let it tune the agent
Reported by Magnus Müller, Browser UseMarch 25, 2026Not tested by benchr
What happened
Magnus Müller of Browser Use describes handing Claude Code a command-line interface to their own evaluation platform plus a prompt to run in a loop, twenty cycles, in parallel — "a search tree over the space of possible best agents". The resulting agent scored 97% on Online-Mind2Web, a benchmark of 300 tasks across 136 real websites (83 easy, 143 medium, 74 hard). The post calls it the highest score ever recorded on it.
How it was done
- InA browser agent that already worked, and its eval platform
- ThenClaude Code got a CLI to the evals and a prompt to run in a loop
- ThenTwenty cycles, in parallel, split into train and validation sets
- Out97% on Online-Mind2Web
How to try it
The reusable idea is smaller than the headline: put your evaluation behind a command the agent can run itself, then let it iterate against a validation split it cannot see during tuning. Without the split you are measuring memorisation.
What this does not prove
- The authors name overfitting as the loop's natural tendency and use a train/validation split against it.
- Remaining failures are attributed to unavailable sites, ambiguous prompts, and websites that changed since the benchmark was built.
- The benchmark itself omits harder shapes of work; the post gives "extract 1,000 products with subpages and compare them across platforms" as an example it does not cover.
- The run is self-reported. benchr has not reproduced it and no independent replication is linked.
Someone did this · Not tested by benchr
Partly reproducibleRecorded by benchr on
First-party: the team is reporting on its own product, and the score is self-run. The post is unusually candid about that — it notes elsewhere that when a competitor does not publish its data, "independent verification isn't possible".