Gemma is Google DeepMind’s family of open models built from Gemini research. Starting with Gemma 4 it ships under Apache 2.0, and every size — from the phone- and laptop-class E2B and E4B to the server-class 26B MoE and 31B — supports image input (the small models also take audio), thinking mode and function calling. This guide goes from model choice to local runs and multimodal processing.
Gemma Practical Examples
Examples of a lightweight inference server, summarization, sentiment analysis, code generation and Keras LoRA.
View practical examples →Gemma is Google DeepMind’s open model family, known for high quality at a given size and for running on small devices. Gemma 4, released in April 2026, moved to the Apache 2.0 license and added image input, thinking mode and function calling at every size.
| Generation | Highlights | License |
|---|---|---|
| Gemma 2 | 2B, 9B, 27B text models | Gemma Terms of Use |
| Gemma 3 / 3n | Image input and long context / 3n for on-device | Gemma Terms of Use |
| Gemma 4 | E2B, E4B, 12B, 26B A4B, 31B; images (small models also audio); thinking mode; function calling; up to 256K | Apache 2.0 |
To build an assistant that writes inspection reports from photos and voice memos with no internet connection, see the offline field-inspection assistant tutorial with Gemma 4.
| Model | Parameters | Context | Input | Ollama (4-bit) | Best for |
|---|---|---|---|---|---|
| E2B | 2.3B effective (5.1B total) | 128K | Text, image, audio | gemma4:e2b from about 4.6 GB | Phones and low-end devices, voice memos |
| E4B | 4.5B effective (8B total) | 128K | Text, image, audio | gemma4:e4b from about 6.6 GB | The default for laptops and edge devices |
| 12B | 11.95B | 256K | Text, image, audio | gemma4:12b from about 7.7 GB | Balance of quality and size |
| 26B A4B | 25.2B (3.8B active, MoE) | 256K | Text, image | gemma4:26b from about 16 GB | Servers that need fast responses |
| 31B | 30.7B (dense) | 256K | Text, image | gemma4:31b from about 19 GB | Highest quality |
The model card recommends temperature=1.0, top_p=0.95, top_k=64. For tasks where consistent answers matter, such as classification and extraction, evaluate with a lower temperature.
Gemma’s license differs by generation, so check it for the version you use.
| Version | License | What it means |
|---|---|---|
| Gemma 4 | Apache 2.0 | Free commercial use, modification and redistribution; no user-count limit or separate prohibited-use terms (keep the license notice) |
| Gemma 3 and earlier | Gemma Terms of Use + prohibited use policy | Commercial use allowed, but the terms and prohibited-use policy must be passed on when you distribute derivative models |
Moving a Gemma 3-based service to Gemma 4 lightens license review. A fine-tuned Gemma 3 model you keep using stays under the old terms.
ollama pull gemma4:e4b # laptop default (from about 6.6 GB)
ollama pull gemma4:e2b # low-end and portable devices
ollama pull gemma4:26b # 24 GB GPU server
ollama pull gemma4:e4b-mlx # MLX build for Apple Silicon
ollama run gemma4:e4b "Explain the difference between Apache 2.0 and MIT in three lines"For images, put file paths in the message’s images.
from ollama import chat
resp = chat(
model="gemma4:e4b",
messages=[{
"role": "user",
"content": "What readings can you see on this gauge, and does anything look wrong?",
"images": ["./gauge.jpg"],
}],
)
print(resp.message.content)Gemma 4 can generate its reasoning before answering, and you can switch that on and off. In Ollama, control it with think; the reasoning arrives separately in message.thinking.
import time
from ollama import chat
question = "Pump vibration rose from a usual 2.1 mm/s to 7.8 mm/s while bearing temperature is normal. Likely causes and inspection order?"
for think in (False, True):
started = time.perf_counter()
resp = chat(model="gemma4:e4b", messages=[{"role": "user", "content": question}], think=think)
elapsed = time.perf_counter() - started
print(f"=== think={think} ({elapsed:.1f}s)")
if resp.message.thinking:
print("[thinking]", resp.message.thinking[:300], "...")
print("[answer]", resp.message.content[:400])In Transformers use apply_chat_template(..., enable_thinking=True); with a raw prompt add the <|think|> token to the system prompt. Thinking slows responses, so turn it on only for hard requests.
The Ollama Python library builds the tool schema from a Python function’s type hints and docstring when you pass the function in tools.
from ollama import chat
def lookup_equipment(asset_id: str) -> dict:
"""Look up equipment type, location and last inspection date by asset ID.
Args:
asset_id: Asset ID (e.g. PUMP-07)
"""
return {"asset_id": asset_id, "type": "centrifugal pump", "location": "Building B, level B2", "last_inspection": "2026-08-14"}
TOOLS = {"lookup_equipment": lookup_equipment}
messages = [{"role": "user", "content": "Where is PUMP-07 and when was it last inspected?"}]
resp = chat(model="gemma4:e4b", messages=messages, tools=[lookup_equipment])
messages.append(resp.message)
for call in resp.message.tool_calls or []:
result = TOOLS[call.function.name](**call.function.arguments) # Ollama gives arguments as a dict
messages.append({"role": "tool", "tool_name": call.function.name, "content": str(result)})
final = chat(model="gemma4:e4b", messages=messages)
print(final.message.content)from typing import Literal
from ollama import chat
from pydantic import BaseModel
class Finding(BaseModel):
component: str
issue: str
severity: Literal["ok", "watch", "action", "stop"]
resp = chat(
model="gemma4:e4b",
messages=[{"role": "user", "content": "Rust on a pipe joint and a small puddle of water on the floor"}],
format=Finding.model_json_schema(),
options={"temperature": 0},
)
print(Finding.model_validate_json(resp.message.content))For input types Ollama does not support, or finer control, use Hugging Face Transformers. Gemma 4 handles text, images and audio with a single AutoModelForMultimodalLM. Download repositories such as google/gemma-4-E4B-it after accepting the license.
uv add -U transformers torch acceleratefrom transformers import AutoModelForMultimodalLM, AutoProcessor
MODEL_ID = "google/gemma-4-E4B-it"
processor = AutoProcessor.from_pretrained(MODEL_ID)
model = AutoModelForMultimodalLM.from_pretrained(MODEL_ID, dtype="auto", device_map="auto")
messages = [{
"role": "user",
"content": [
# Images go before the text, audio after it (audio up to 30 seconds)
{"type": "image", "url": "./gauge.jpg"},
{"type": "text", "text": "Use the gauge photo and the voice memo together to summarize the inspection result"},
{"type": "audio", "audio": "./memo.wav"},
],
}]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
return_dict=True,
return_tensors="pt",
add_generation_prompt=True,
enable_thinking=False,
).to(model.device)
input_len = inputs["input_ids"].shape[-1]
outputs = model.generate(**inputs, max_new_tokens=512)
response = processor.decode(outputs[0][input_len:], skip_special_tokens=False)
print(processor.parse_response(response, prefix=inputs["input_ids"]))Only E2B, E4B and 12B accept audio. To run the same code on 26B A4B or 31B, drop the audio item and, if needed, pass a transcript produced by a smaller model.
| Model | Use |
|---|---|
| EmbeddingGemma | Text embeddings for RAG and search; small enough to search documents on device |
| Gemma 3n | The on-device model before Gemma 4; for maintaining existing apps |
Fine-tuning follows the same TRL flow as the data preparation and QLoRA training steps in the Llama tutorial; for data with images, prepare it with an image/images column.
| Symptom | Cause | Fix |
|---|---|---|
| Transformers cannot find the model class | An old Transformers that predates Gemma 4 | Install the latest with uv add -U transformers |
| Audio input is ignored or errors | Audio sent to 26B A4B / 31B, or longer than 30 seconds | Use E2B, E4B or 12B and split audio into 30-second pieces |
| Reasoning text mixed into the answer | Using raw output as is | Ollama: message.thinking; Transformers: parse_response |
| Answers vary a lot between runs | Using the recommended temperature 1.0 as is | Evaluate classification and extraction with a lower temperature |
| Slow on Apple Silicon | Using the standard build | Use MLX tags such as gemma4:e4b-mlx |
Gemma 4 is released under Apache 2.0, so commercial use, modification and redistribution are free. Gemma 3 and earlier follow Google’s Gemma Terms of Use and prohibited use policy, so check those terms separately if you still use an older version.
E stands for effective parameters. E4B has about 8B parameters including embeddings, but only about 4.5B are used in computation, so it runs at 4B-class speed and is designed for devices such as phones and laptops.
E2B, E4B and 12B support audio input natively; 26B A4B and 31B accept text and images only. In Transformers you add a message item of type audio, and each audio input can be up to 30 seconds long.
In Ollama, pass think=True or False to chat, and read the reasoning separately from message.thinking. In Transformers, use the enable_thinking argument of apply_chat_template; with a raw prompt, add the <|think|> token to the system prompt.
26B A4B is an MoE model that computes only about 3.8B parameters per token, so it responds much faster than 31B; 31B is a dense model that is slower but highest in quality. Both are about 16-20 GB in Ollama’s 4-bit builds and run on a 24 GB GPU, so start with 26B A4B if latency matters and 31B if quality matters, and evaluate.