benchr
GPT-Realtime-2.1 is not a text model with a microphone attached. The product surface includes turn detection, interruption, media transport, tool calls, session state, credentials, and a recovery policy. A strong model inside a brittle loop still produces a bad call.
Pick the voice architecture before the model
OpenAI's voice-agent guide separates native speech-to-speech from a chained pipeline of speech recognition, a text model, and speech synthesis. Speech-to-speech is the natural choice when fluid conversation and low perceived latency dominate. A chain is often easier to inspect when the product needs an explicit transcript, predictable intermediate text, approval before speaking, or independent control over each stage.
| Requirement | Recommended path | Why |
|---|---|---|
| Browser or mobile conversation | Speech-to-speech over WebRTC | Client media handling and natural turn-taking |
| Server-owned media stream | Speech-to-speech over WebSocket | Server controls the event and media pipeline |
| Telephone call | Speech-to-speech over SIP | Direct telephony integration |
| Auditable transcript before speech | Chained voice pipeline | Each intermediate stage can be reviewed or blocked |
| Strict JSON contract | Chained pipeline with a suitable text model | GPT-Realtime-2.1 does not support structured outputs |
Transport is not a performance footnote. It decides where credentials live, what telemetry is available, how reconnects behave, and who owns audio buffering. Browser clients should receive ephemeral authorization rather than a long-lived server key. Server and SIP deployments need a clear mapping from a call identifier to model events, tool traces, and usage.
A workload for interruption and recovery
OpenAI highlights improved alphanumeric recognition, silence and noise handling, interruption behavior, instruction following, and tool use. Turn those claims into a fixed evaluation suite. Use calls with a spelling-sensitive customer name, a mixed letter-number identifier, background audio, a long silence, a user barge-in halfway through an answer, a slow tool, a tool error, and a caller who corrects a prior fact.
For every trace, record whether the user goal was completed, whether the agent stopped when interrupted, which words were repeated after recovery, whether tool arguments matched the caller's latest statement, how stale tool results were handled, and why the call ended. Listen to failures while reading the event trace. A transcript alone hides timing; an audio file alone hides the tool state.
| Failure injection | Pass condition | Evidence |
|---|---|---|
| Barge-in during an answer | Playback stops and the new intent is used | Audio plus interruption events |
| Slow or failed tool | No fabricated result; recovery is explicit | Tool arguments, result, and spoken response |
| Noisy alphanumeric input | Critical identifier is confirmed before action | Captured value and confirmation turn |
| Correction after prior context | Latest user fact replaces stale state | State diff and subsequent tool call |
| Reconnect or handoff | No duplicated speech or unauthorized action | Session boundary and resumed state |
Reasoning effort is a latency-control decision
The Realtime guide recommends starting with low reasoning effort and tuning upward for the task. Apply that per route rather than globally. A greeting, menu choice, or simple lookup should not inherit the setting used for a policy exception. Keep the same workload when tuning so that lower latency is not purchased with missed constraints or weaker tool decisions.
Tool schemas should be narrow, and side-effecting actions need confirmation in the conversation design. Function calling is supported; structured outputs are not. Validate tool arguments in application code, expose only the minimum tools for the current stage, and require an explicit confirmation before payment, deletion, booking, or account changes.
Price the completed call by modality
OpenAI lists text input at $4, cached text input at $0.40, and text output at $24 per million tokens. Audio input is $32, cached audio input $0.40, and audio output $64. Image input is $5, with cached image input at $0.50. These are separate billing categories; a single blended token rate is misleading.
Capture usage from completed and failed calls, including retries and abandoned sessions. Report cost per completed task, not merely per minute or per prompt. The 128,000-token context window and 32,000-token maximum output provide capacity, not a retention policy: summarize stale turns, expire tool results, and remove sensitive audio or transcript data according to the product's policy.
When to avoid or downgrade it
Use a chained pipeline when the application must inspect text before speech, enforce a structured intermediate contract, or maintain deterministic stage-level approvals. Start with GPT-Realtime-2.1 mini for high-volume, bounded interactions and promote only the failure classes the full model fixes. Avoid any autonomous side effect that has no application-side validation, confirmation turn, and auditable tool trace.
| Category | Input / cached input | Output or capability |
|---|---|---|
| Text per 1M tokens | $4 / $0.40 | $24 output |
| Audio per 1M tokens | $32 / $0.40 | $64 output |
| Image per 1M tokens | $5 / $0.50 | Input only |
| Model interface | Text, audio, image input | Text/audio output; function calling; no structured outputs |
| Capacity | 128K context | 32K maximum output |