benchr
GPT-Realtime-2.1, announced July 6, is not a text model with a microphone attached. The interesting part is the loop around speech: what happens when a caller interrupts, spells an account number, triggers a tool, or talks over background noise.
Test the messy call, not the studio demo
OpenAI says the update improves alphanumeric recognition, silence and noise handling, interruptions, instruction following, and tool use. Build those failures into the test. Include barge-in, a noisy room, spelling-sensitive names, a slow tool call, and a recovery turn. A clean scripted transcript skips the work this model is meant to do.
The audio line is the budget
Text costs $4 input and $24 output per million tokens. Audio costs $32 input and $64 output. A text-only spreadsheet can make this model look cheap while a call replay tells a different story. Track text, audio, and cached input separately, then price a completed call rather than a token category.
Do not keep every turn forever
A 128K context window and 32K output ceiling give you room, not a call-history policy. Decide what must survive an interruption, when tool results go stale, and when the transcript should be summarized. Score an end-to-end trace for turn-taking and task completion.
| Field | Verified record |
|---|---|
| API model ID | gpt-realtime-2.1 |
| Text pricing | $4 input / $24 output per 1M |
| Audio pricing | $32 input / $64 output per 1M |
| Capabilities | Speech-to-speech, configurable reasoning effort, tool use |