benchr

How verification works here

Two questions, kept apart on purpose: does the provider document it, and has benchr run it. The second is the harder one, and benchr has not answered it yet for any capability.

These records describe what providers officially document. benchr has not run these capabilities itself, so nothing here is a test result. Each record shows the vendor page it was read from and the date.

Ledger updated: September 3, 2026

Where verification stands today

No verification run has been accepted yet, so no capability carries the Benchr Verified badge. All 20 records currently sit at “Not tested by benchr” on the testing axis.

4 self-reported runs are held for independent review. They were produced by the same automated session that maintains this repository, which makes them a starting point and not evidence, so their outcomes are not published here.

The benchr verification protocol

Seven steps. A run is void if any of them is missing from the record.

  1. State the claim as a single testable sentence before running anything, so the run cannot be reinterpreted after the fact.
  2. Name the subject exactly: model and version, the surface or interface, and the version of that surface.
  3. Record the environment: operating system, network conditions, and anything that could make the result local rather than general.
  4. Record the inputs and the exact instruction or prompt, verbatim. A paraphrased prompt is not evidence.
  5. Record the settings that were in force, including any left at their defaults.
  6. Record the observed output and the result. Partial and fail are published exactly like a pass.
  7. Record the evidence and the reproduction recipe, then list the limitations of the run - starting with what it did not test.

Documentation axis

  • DocsOfficially documentedThe provider's own current documentation describes this. benchr read that page on the date shown.
  • DocsDocumented in partThe provider documents the mechanism but not the result, or documents it only inside stated limits.
  • DocsDeprecatedThe provider has announced withdrawal, closed new access, or set a retirement date.
  • DocsNeeds recheckThe documentation has not been re-read inside the recheck window. Computed, never hand-set.
  • DocsNot documentedNo official source describes this. Submissions land here until one is found.

Testing axis

  • TestBenchr Verifiedbenchr ran this and reproduced the result. Every run is published with its model, surface, inputs, settings, environment, output and evidence.
  • TestPartly reproducedA benchr run succeeded under narrower conditions than the claim.
  • TestNeeds retestThe last passing run is older than the retest window, or a dependency changed.
  • TestBrokenA benchr run failed. The run is published alongside the record.
  • TestNot tested by benchrbenchr has not run this. This is the honest default and it is where every record starts.