Gemma 4 31B IT: the dense quality-first choice

The decision is not “31B is the bigger number.” It is whether a dense, quality-biased model produces enough fewer errors on your documents, code, or agent steps to justify its higher resident memory and per-token compute. Google’s own sibling table gives useful signals, but only a workload holdout can settle that trade.

Dense means every layer pays rent

Gemma 4 31B has 30.7B parameters across 60 layers. Unlike the 26B A4B sibling, it does not route each token through a small subset of experts. That makes the execution path easier to reason about, but it also means capacity planning starts with the full dense model on every request. Quantization changes the weight footprint; it does not remove KV cache, runtime, batching, or concurrent-session overhead.

The strongest reason to choose 31B is the long tail

The published sibling results are not a universal ranking, but they show where the denser model’s margin grows. On the selected Google table below, the gap is modest on MMLU Pro and GPQA Diamond, wider on Tau2, and much wider on the 128K multi-needle retrieval test. If your work is mostly short extraction, those gains may not repay the footprint. If it combines long evidence, tool use, and multi-step judgment, the 31B candidate deserves the first test slot.

Selected Google-published Gemma 4 results; higher is better except OmniDocBench

Evaluation	31B dense	26B A4B	Decision signal
MMLU Pro	85.2	82.6	General knowledge
LiveCodeBench v6	80.0	77.1	Code generation
Tau2 average	76.9	68.2	Tool and agent tasks
MRCR v2, 8-needle 128K	66.4	44.1	Long-context retrieval
OmniDocBench ↓	0.131	0.149	Document parsing; lower is better

A deployment decision should begin with the constraint

Choose the route that matches the limiting constraint

Situation	Start with	Why	Validate before committing
One quality-critical workstation or server	31B Q4_0	Lowest official footprint for the denser model	Quality loss from quantization and peak memory at real context length
Long documents plus agent/tool steps	31B	Official sibling results favor it most on the long-context and Tau2 rows shown above	Your retrieval misses, tool failures, and end-to-end latency
High concurrency on fixed hardware	26B A4B	Lower published static-weight footprint and sparse active path	Requests per second at the same output quality threshold
Speech or audio understanding	Google lists text and image input, not audio	A native audio model or a speech-to-text pipeline

Test long context as evidence retrieval, not document stuffing

The published 128K multi-needle result makes long-context work the most important place to challenge the 31B model. Build a pack with facts spread across sections, believable distractors, and at least one contradiction that requires source preference. Require the answer to identify the supporting passage and to abstain when the pack is insufficient. Score evidence recovery separately from the final prose so a fluent answer cannot hide a retrieval miss.

Run the same pack at ordinary and high context lengths. If accuracy falls only when the prompt grows, the bottleneck may be retrieval or prompt construction rather than general model quality. Compare that failure curve with 26B before buying more memory. A larger window has value only when the model can recover the right evidence reliably and your serving stack can afford the cache.

Plan memory with a real prompt, not the empty model

Do not use maximum context as the acceptance target unless production needs it. The more useful test holds quality constant, then compares peak memory, first-token latency, total latency, and successful jobs per hour. That exposes whether 31B’s extra quality is operationally affordable.

Choose it when quality is scarce; avoid it when capacity is scarce