Gemma 4 Is Google's Best Open Model Yet (Here's How to Use It)

What Changed from Gemma 3#

The biggest change is the license. Gemma 3 shipped with a restrictive custom license that made lawyers nervous and kept the model out of production pipelines. Gemma 4 is Apache 2.0. That alone makes it a different product.

The context window jumped from 128K to 256K tokens. Native function calling and structured JSON output are built in, not bolted on. All four variants handle text and image input natively.

The two smallest models (E2B and E4B) also support audio input, which is new for the Gemma family. Language coverage expanded to 140+, and the community reports noticeably better performance on DevOps tasks and multilingual prompts compared to Gemma 3.

Google claims over 400 million downloads across all Gemma versions. The Apache 2.0 switch will probably accelerate that.

The Benchmarks That Matter#

I care about three benchmarks for coding and reasoning: AIME 2026, LiveCodeBench v6, and GPQA Diamond. Here is how the Gemma 4 31B Dense stacks up against Llama 4 Maverick and the broader field.

Gemma 4 wins every category. The LiveCodeBench gap is the most interesting one. Llama 4 Maverick scored 77.1% on the standard eval, but some independent benchmarks put it as low as 43.4% on coding tasks. That inconsistency is pushing users toward Gemma 4, and the Reddit threads reflect it.

The 26B MoE variant (which activates only 3.8B parameters per token) ranks #6 among open models on Arena AI. For a model that runs on 16GB VRAM, that is remarkable.

Pick the Right Model Size#

Not every task needs 31 billion parameters. Here is a quick guide to picking the right variant for your hardware and workload.

If you have a 16GB GPU, the 26B MoE is probably the right call. It is the model most people on r/LocalLLaMA are running, and for good reason.

Five Use Cases You Can Run in 5 Minutes#

1. Local code review. Point Gemma 4 at a diff and ask for bugs. The 26B MoE handles this well at 80+ tok/s, fast enough to feel interactive. If you are already running local models with Claude Code, swapping in Gemma 4 is a one-line change.

2. Multimodal document parsing. Feed it a screenshot of a UI and ask it to generate the HTML. All variants support image input, so even the E4B can do this on a laptop.

3. Structured data extraction. Native JSON output means you can skip the "please respond in JSON" prompt hack. Pass an invoice image, get clean structured data back.

4. Function calling agents. Built-in function calling support makes Gemma 4 a solid backbone for tool-using agents. Define your tools as JSON schema, and the model will generate the correct call format without few-shot examples.

5. Multilingual customer support. 140+ languages with improved multilingual performance. The Apache 2.0 license means you can deploy this in production without worrying about Google's usage restrictions.

Getting Started with Ollama and Transformers#

Two paths. If you want to be running in 30 seconds, use Ollama. If you need fine-grained control, use Transformers.

Ollama (fastest)#

# Edge model — runs on anything
ollama run gemma4:e2b

# Community favorite — 16GB VRAM
ollama run gemma4:26b-a4b

# Full power — 24GB VRAM
ollama run gemma4:31b

That's it. Day-one support shipped for Ollama, llama.cpp, vLLM, LM Studio, MLX, and Unsloth.

Transformers (multimodal)#

from transformers import AutoProcessor, AutoModelForImageTextToText
import torch

model_id = "google/gemma-4-26b-it"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
)

If you are working with less VRAM, 4-bit quantization cuts memory usage roughly in half:

from transformers import BitsAndBytesConfig

quant_config = BitsAndBytesConfig(
    load_in_4bit=True, bnb_4bit_quant_type="nf4"
)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, quantization_config=quant_config
)

Tip: The community-recommended sampling settings are temp 0.3, top-p 0.9, min-p 0.1, top-k 20. These work well across all four variants.

Where This Fits#

A week ago, Llama 4 was the default recommendation for local LLM work. Gemma 4 changes that calculus. Better coding benchmarks, Apache 2.0 licensing, and a MoE variant that runs on consumer hardware at 80+ tokens per second.

The 26B MoE is the model I would point most developers toward right now. It hits a rare combination of quality, speed, and accessibility. If you are building developer tooling on Windows or trying to cut your token costs, running Gemma 4 locally is one of the highest-impact changes you can make this month.

Google got this one right.

Sources: Google AI Blog, DeepMind, HuggingFace, Latent Space