benchr
The name carries the deployment trap. “A4B” describes the parameters active during a token, not the amount of model you can leave on disk while inference runs. Google's official card says all 25.2B parameters must be loaded so the router can reach the experts it needs.
MoE buys compute efficiency, not a tiny footprint
Gemma 4 26B A4B has 128 routed experts plus one shared expert. Eight routed experts are active per token, for 3.8B active parameters. That can make generation more efficient than a similarly sized dense model, but every expert must remain available in memory. Plan capacity from the weight footprint, KV cache, runtime, and concurrency together.
A 16 GB GPU seeing “14.4 GB” is not a promise that a 256K session fits. Google's table is for loading weights and includes a stated 20% overhead, but it excludes the context KV cache and serving stack. Leave room or shorten the active context; otherwise the model can load and still fail on the first substantial request.
The sibling comparison explains the trade
| Field | 26B A4B | 31B dense |
|---|---|---|
| Architecture | 25.2B total, 3.8B active | 30.7B active dense model |
| Context | 256K | 256K |
| Q4_0 static load | 14.4 GB | 17.5 GB |
| LiveCodeBench v6 | 77.1% | 80.0% |
| GPQA Diamond | 82.3% | 84.3% |
| MRCR v2, 128K | 44.1% | 66.4% |
These are Google's own runs, not benchr tests. They suggest the 26B model gives up some measured quality, especially on the published long-context result, in exchange for a smaller static footprint and sparse compute. The right decision depends on whether that gap appears in your workload and whether the faster route improves throughput on your hardware.
Choose the 26B when serving shape matters
This model makes sense for a workstation or server where the 31B dense version is uncomfortable, or for a service that values throughput across bounded text-and-image tasks. Native function calling and system-role support make it a candidate for controlled agents. The 256K context also fits document sets, but long-context capacity should be earned with retrieval and chunking rather than filled automatically.
Use the instruction-tuned ID gemma-4-26b-a4b-it in AI Studio or the Gemini API, or download the official open weights. For local deployment, keep the model format and quantization in the test record. A Q4 result and a BF16 result are not interchangeable evidence.
Skip it when the sparse architecture adds no value
If you already have enough hardware for the 31B dense model and your priority is the strongest result in Google's published comparison, test the 31B first. If you need audio input, neither the 26B nor 31B is the right Gemma 4 branch; Google's card limits native audio to the smaller E2B, E4B, and 12B variants. And if you need a predictable hosted-token budget, wait for a model-specific Gemini API rate rather than inventing one.
Measure two capacities: resident and sustained
A model that loads is not automatically a model you can serve. The resident test asks whether the chosen weights, runtime, and one representative context fit without spilling or failing. The sustained test holds that setup under your expected mix of prompt lengths and simultaneous requests. Record memory after warm-up, not only at process start, and include the longest ordinary prompt rather than a synthetic empty request.
Keep the quantization file, runtime version, device split, batch policy, and context setting beside every result. Then repeat the same harness for 31B. The useful outcome is not a headline tokens-per-second number; it is the number of accepted tasks the machine completes before latency, memory, or output quality crosses your operating limit. That is where sparse routing either earns its complexity or does not.
A practical acceptance test
Run the same text-and-image tasks on 26B A4B and 31B with the same prompt, context, quantization class, and output cap. Measure task pass rate, tokens per second, peak memory, time to first token, and whether the 26B requires more retries. Publish those details internally. Without them, “faster” and “almost as capable” remain provider positioning rather than evidence about your system.