benchr
Mini is not simply “the cheaper one.” Its most useful role is as a routing baseline. Start the ordinary call there, observe the failures, and define an escalation rule that can be explained from the trace. Routing every call to the full model hides whether the premium solved anything.
Define the mini lane narrowly
Good starting workloads have bounded tools, short policy trees, reversible actions, and clear confirmation language: appointment lookup, order status, store information, intake, FAQ triage, and a handoff to a human. Mini still supports tool calls, but the application must validate arguments and gate side effects. Structured outputs are not supported, so do not use the spoken model response as an unchecked data contract.
Keep the transport choice separate from model size. WebRTC remains the fit for browser and mobile clients, WebSocket for server media, and SIP for telephony. A move from mini to full does not repair an unstable media path, leaked credential, broken reconnect, or missing confirmation step.
Route by observed failure, not by customer label
| Trace outcome on mini | Route decision | Reason |
|---|---|---|
| Intent and required fields are captured correctly | Stay on mini | The task is complete; a larger model adds no demonstrated value |
| Noisy identifier remains ambiguous after confirmation | Retry safely, then escalate or hand off | Do not act on an uncertain account or code |
| Tool choice or arguments fail on a complex branch | Escalate that branch to full | This is a model-capability failure class |
| Policy requires approval or strict intermediate text | Switch to a chained pipeline | Full realtime still does not add structured outputs |
| Transport, echo, or reconnect fails | Fix the application path | A larger model cannot repair media plumbing |
A routing policy should name the failure, allowed retry, destination, and privacy boundary. Do not escalate by vague labels such as “VIP caller” or “hard question.” The same caller may have a simple lookup followed by a regulated exception. Keep the simple turn on mini and route only the stage that needs a different model or a human.
Run a paired replay, not two unrelated demos
Create a shared call set with clean speech, background noise, an alphanumeric identifier, a user interruption, a slow tool, a failed tool, a correction, and a reconnect. Feed equivalent events and tools to mini and full. Score task completion, critical-field accuracy, interruption repair, tool arguments, unsupported claims, handoff quality, usage by modality, and total cost for successful calls.
Classify failures before reading the bill. If both models fail after a reconnect, the fault may be session state. If mini alone selects the wrong tool on a multi-condition route, that is a candidate for escalation. If both finish correctly, mini wins that lane unless a separate quality requirement says otherwise.
Mini's price card is still multi-modal
OpenAI lists text input at $0.60, cached text input at $0.06, and text output at $2.40 per million tokens. Audio input is $10, cached audio input $0.30, and audio output $20. Image input is $0.80, with cached image input at $0.08. These are distinct meters, and the call's mix determines the result.
Measure usage on retries, abandonments, safety refusals, and handoffs—not only completed calls. The 128,000-token context window and 32,000-token maximum output do not justify keeping a complete call forever. Summarize old turns, expire stale tool results, and remove sensitive media according to policy.
When to promote, chain, or hand off
Promote to GPT-Realtime-2.1 when mini repeatedly fails a specific complex-instruction, tool-selection, or recovery class that the full model passes. Use a chained voice architecture when an intermediate transcript or structured approval is mandatory. Hand off to a person when the user contests identity, policy, consent, or an irreversible action. The escalation design is part of the model decision, not a patch after launch.
| Category | Input / cached input | Output or capability |
|---|---|---|
| Text per 1M tokens | $0.60 / $0.06 | $2.40 output |
| Audio per 1M tokens | $10 / $0.30 | $20 output |
| Image per 1M tokens | $0.80 / $0.08 | Input only |
| Model interface | Text, audio, image input | Text/audio output; function calling; no structured outputs |
| Capacity | 128K context | 32K maximum output |