AI tools and surfaces

A tool page should answer one question: what can I actually do with this? Each entry lists the documented capabilities it carries, the models behind it, and what it refuses to do.

These records describe what providers officially document. benchr has not run these capabilities itself, so nothing here is a test result. Each record shows the vendor page it was read from and the date.

Ledger updated: September 3, 2026

Claude Code

Coding agent

AnthropicDocs checked September 1, 2026

An agentic coding tool that reads a codebase, edits files, runs commands and works with git - in a terminal, an IDE, a desktop app and the browser.

What you can actually do with it

  • Build features and trace bugs across many files, then run the tests and fix what fails.
  • Stage changes, write commit messages, create branches and open pull requests directly.
  • Load project rules from CLAUDE.md, run hooks around edits and commits, and package repeated procedures as skills.
  • Connect to your own systems through MCP, and run unattended on a schedule or from a GitHub event.

What it will not do

  • Work without a Claude subscription or Console account on most surfaces.
  • Auto-update when installed through Homebrew or WinGet - only the native install does.
  • Constrain itself: permissions and hooks are configuration, not defaults you can assume.

Status

  • Work through an entire repository, not a pasted fileDocumented
  • Hand off a task and walk away from the machineDocumented
  • Teach the model a procedure once and reuse itDocumented
  • One screenshot in, a working page outPartly documented
  • Give a model your own tools and data through MCPDocumented
  • Drive a real browser, not a scraperDocumented

Claude API

Developer API

AnthropicDocs checked September 1, 2026

The surface that carries the server-side tools: browser control, desktop control, sandboxed code execution, web search with citations, batching and prompt caching.

What you can actually do with it

  • Run Anthropic-executed tools inside a turn, so search and code execution do not need a client-side loop.
  • Cut repeat-context cost to a tenth with prompt caching, and bulk work to half price with the Batches API.
  • Read PDFs up to 32 MB and 600 pages, and images up to 8000x8000 px, by file reference rather than base64.
  • Run Agent Skills inside the code execution container by skill_id.

What it will not do

  • Offer every tool on every platform - browser use, code execution and web search each have their own platform gaps across Bedrock, AWS, Google Cloud and Microsoft Foundry.
  • Give code execution or Skills zero-data-retention eligibility.
  • Allow Skills on the API to reach the network or install packages at runtime.

Status

  • Drive a real browser, not a scraperDocumented
  • Operate a desktop applicationDocumented
  • Let the model write and run real code mid-answerDocumented
  • Read a 600-page PDF including its chartsDocumented
  • Answer from the live web with citations attachedDocumented
  • Stop paying twice for the same fileDocumented
  • Run the whole job overnight at half priceDocumented
  • Teach the model a procedure once and reuse itDocumented
  • One screenshot in, a working page outPartly documented
  • Fit a whole codebase in one prompt, and find its limitPartly documented

OpenAI API

Developer API

OpenAIDocs checked September 1, 2026

Schema-guaranteed output, transcription with speaker separation, and a realtime speech-to-speech session protocol.

What you can actually do with it

  • Constrain a response to a JSON Schema so a required key is never missing and an enum value is never invented.
  • Transcribe audio, with a separate model that returns speaker-attributed output and another that returns word-level timestamps.
  • Run a live voice session over WebRTC, WebSocket or SIP on gpt-realtime-2.1.

What it will not do

  • Accept new fine-tuning users - the platform is documented as winding down.
  • Support the whole of JSON Schema; some features are excluded for performance or technical reasons.
  • Take an audio file over 25 MB in one transcription request.

Status

  • Get JSON that always matches your schemaDocumented
  • Transcribe a recording and label who said whatDocumented
  • Talk to it out loud and hear it answerDocumented
  • Fine-tune a hosted OpenAI modelWithdrawn
  • Give a model your own tools and data through MCPDocumented

Gemini API

Developer API

GoogleDocs checked September 1, 2026

Million-token context, a Python sandbox with charting, image generation that renders readable text, and video generation.

What you can actually do with it

  • Take a whole corpus - text, video or audio - in one request up to 1M tokens.
  • Run Python in a sandbox with roughly forty scientific and data libraries and return Matplotlib figures inline.
  • Generate and conversationally edit images, including images with legible text, with up to 14 reference images.
  • Generate short video from text or a still, with native audio on Veo 3.1.

What it will not do

  • Keep single-query accuracy when one question needs several separate facts from a long context.
  • Run sandboxed code for more than 30 seconds, or return media artefacts other than inline plots.
  • Publish output length, resolution or rate limits for video generation on the page benchr checked.

Status

  • Fit a whole codebase in one prompt, and find its limitPartly documented
  • Let the model write and run real code mid-answerDocumented
  • Generate an image with legible text inside itDocumented
  • Generate video from a prompt or a stillPartly documented

claude.ai and the Claude apps

End-user app

AnthropicDocs checked September 1, 2026

The no-code surface: upload an image or a document, use the pre-built document skills, and add custom skills from settings.

What you can actually do with it

  • Use pre-built PowerPoint, Excel, Word and PDF skills with no setup when creating documents.
  • Upload custom skills as a zip on Pro, Max, Team and Enterprise plans with code execution enabled.
  • Take images by upload or drag and drop, up to 20 per turn and 10 MB each.

What it will not do

  • Share custom skills across an organisation - they are per user, with no admin-level distribution.
  • Sync skills to the API or to Claude Code; each surface is installed separately.
  • Guarantee network access inside skills - it varies with user and admin settings.

Status

  • One screenshot in, a working page outPartly documented
  • Read a 600-page PDF including its chartsDocumented
  • Teach the model a procedure once and reuse itDocumented

MCP servers

Protocol and connectors

Open standardDocs checked September 1, 2026

The connector layer. Write a server once and every client that speaks the protocol can use it - Claude, ChatGPT, VS Code, Cursor and others.

What you can actually do with it

  • Expose data sources, callable tools and workflow prompts to any MCP client.
  • Let a coding agent reach a design file, a ticket tracker or an internal database without a per-client integration.
  • Keep credentials on the server rather than handing them to the model.

What it will not do

  • Make what you connect safe. The protocol standardises the connection, not the blast radius.
  • Guarantee feature parity across clients - support varies by feature.
  • Stay still. The spec is published in dated revisions, so pin the one you build against.

Status

  • Give a model your own tools and data through MCPDocumented

Local runtimes

Local runtime

Open-weight ecosystemDocs checked August 15, 2026

Open-weight models on your own hardware, where the question is memory arithmetic rather than a price per token.

What you can actually do with it

  • Keep every prompt and document on the machine, with no request leaving it.
  • Work offline, which is the whole argument in regulated and disconnected environments.
  • Trade a per-token bill for a fixed hardware ceiling.

What it will not do

  • Promise throughput. benchr's memory figures are deployment-planning estimates, never measurements.
  • Make extended context free - it can need runtime-specific scaling and much more cache memory.
  • Match a hosted frontier model on quality for most workloads.

Status

  • Run a capable model on your own machinePartly documented
Sources
/local