Skip to main content
AIDevOps
  • Learn
  • Learning Paths
  • Practice
  • Open Source
  • Books
  • Engineering

    AI DevOpsThe full map of building and operating AI servicesLLMOpsLLM deployment · evaluation · observabilityHands-on ProjectsBuild AI Agent projects

    Knowledge

    DocsTechnical documentationBlogEngineering articlesPloggerDevelopment log feed

    Validate

    Certification3-level skills certification · coming soon
AI Models
LlamaTutorial|MistralTutorial|GemmaTutorial|DeepSeekTutorial|QwenTutorial
🧠 AI Core
AI Intro & RoadmapML FundamentalsLLM Fundamentals|Python AIC++|PyTorchTensorFlowJAX
🤖 AI Applied Development
Applied AI Intro & RoadmapHugging FaceLangChainLlamaIndexLLMOps|LangGraphMCPMulti-AgentAgent Evaluation
🧠 AI Agent Development
Finance AI AgentLLM API ServerStock Investing AgentAIOps AI AgentEducation AI AgentCoding AI Agent
🌱 Spring Cloud
Spring Intro & RoadmapSpring Cloud GatewaySpring BootJava|Spring AISpring SecuritySpring BatchSpring JPA
🐳 DevOps
DevOps Intro & RoadmapLinuxDockerCI/CD|Kubernetes BasicsK8s AdvancedPrometheusGrafana
🧱 Infrastructure
Infrastructure Intro & RoadmapNginxRedis
☁️ Cloud
Cloud Intro & RoadmapAWSGCPAzureNCPCloudflare
🎨 Frontend
Frontend Intro & RoadmapJavaScriptTypeScript|ReactNext.js|VueNuxt|Electron
📱 Mobile
Mobile Intro & RoadmapKotlinAndroidFlutter
⚙️ Backend
Backend Intro & RoadmapPython BasicsFastAPIDjangoFlask|CGoGinNode.js
💾 Database
DB Intro & RoadmapCore SQLOracleMySQLPostgreSQL|MongoDBVector DB
🧪 Testing
k6JMeternGrinder
AIDevOps

Engineering AI. From Code to Production.
An engineering learning platform for building and operating AI and AI Agents

Learn

  • All Guides
  • Learning Paths
  • AI Tutorial
  • Practice
  • Books

Resources

  • AI DevOps
  • LLMOps
  • Hands-on Projects
  • Docs
  • Blog
  • Plogger
  • Open Source
  • Certification (coming soon)

Start Here

  • AI Core Roadmap
  • AI Applied Development Roadmap
  • Spring Cloud Roadmap
  • DevOps Roadmap
  • Infrastructure Roadmap

 

  • Cloud Roadmap
  • Frontend Roadmap
  • Mobile Roadmap
  • Backend Roadmap
  • Database Roadmap
© 2026 AIDevOps. All rights reserved.
Terms of ServicePrivacy PolicySitemaptestforge.kr
On-Device & Multimodal Open Model Guide

💎 Gemma Complete Guide

Visitors

Gemma is Google DeepMind’s family of open models built from Gemini research. Starting with Gemma 4 it ships under Apache 2.0, and every size — from the phone- and laptop-class E2B and E4B to the server-class 26B MoE and 31B — supports image input (the small models also take audio), thinking mode and function calling. This guide goes from model choice to local runs and multimodal processing.

← Back to AIDevOps home
💎

Gemma Practical Examples

Examples of a lightweight inference server, summarization, sentiment analysis, code generation and Keras LoRA.

View practical examples →

Contents

0 / 11
  1. Gemma at a glance
  2. Gemma 4 lineup & how to choose
  3. License change
  4. Running locally with Ollama
  5. Thinking mode
  6. Function calling
  7. Structured output
  8. Image & audio with Transformers
  9. Specialized models
  10. Troubleshooting
  11. FAQ
Contents 11 sections
  1. Gemma at a glance
  2. Gemma 4 lineup & how to choose
  3. License change
  4. Running locally with Ollama
  5. Thinking mode
  6. Function calling
  7. Structured output
  8. Image & audio with Transformers
  9. Specialized models
  10. Troubleshooting
  11. FAQ

Gemma at a glance

Gemma is Google DeepMind’s open model family, known for high quality at a given size and for running on small devices. Gemma 4, released in April 2026, moved to the Apache 2.0 license and added image input, thinking mode and function calling at every size.

GenerationHighlightsLicense
Gemma 22B, 9B, 27B text modelsGemma Terms of Use
Gemma 3 / 3nImage input and long context / 3n for on-deviceGemma Terms of Use
Gemma 4E2B, E4B, 12B, 26B A4B, 31B; images (small models also audio); thinking mode; function calling; up to 256KApache 2.0
💡

To build an assistant that writes inspection reports from photos and voice memos with no internet connection, see the offline field-inspection assistant tutorial with Gemma 4.

Gemma 4 lineup & how to choose

ModelParametersContextInputOllama (4-bit)Best for
E2B2.3B effective (5.1B total)128KText, image, audiogemma4:e2b from about 4.6 GBPhones and low-end devices, voice memos
E4B4.5B effective (8B total)128KText, image, audiogemma4:e4b from about 6.6 GBThe default for laptops and edge devices
12B11.95B256KText, image, audiogemma4:12b from about 7.7 GBBalance of quality and size
26B A4B25.2B (3.8B active, MoE)256KText, imagegemma4:26b from about 16 GBServers that need fast responses
31B30.7B (dense)256KText, imagegemma4:31b from about 19 GBHighest quality

The model card recommends temperature=1.0, top_p=0.95, top_k=64. For tasks where consistent answers matter, such as classification and extraction, evaluate with a lower temperature.

License change

Gemma’s license differs by generation, so check it for the version you use.

VersionLicenseWhat it means
Gemma 4Apache 2.0Free commercial use, modification and redistribution; no user-count limit or separate prohibited-use terms (keep the license notice)
Gemma 3 and earlierGemma Terms of Use + prohibited use policyCommercial use allowed, but the terms and prohibited-use policy must be passed on when you distribute derivative models
ℹ️

Moving a Gemma 3-based service to Gemma 4 lightens license review. A fine-tuned Gemma 3 model you keep using stays under the old terms.

Running locally with Ollama

BASH
ollama pull gemma4:e4b       # laptop default (from about 6.6 GB)
ollama pull gemma4:e2b       # low-end and portable devices
ollama pull gemma4:26b       # 24 GB GPU server
ollama pull gemma4:e4b-mlx   # MLX build for Apple Silicon

ollama run gemma4:e4b "Explain the difference between Apache 2.0 and MIT in three lines"

For images, put file paths in the message’s images.

gemma_vision.pyPYTHON
from ollama import chat

resp = chat(
    model="gemma4:e4b",
    messages=[{
        "role": "user",
        "content": "What readings can you see on this gauge, and does anything look wrong?",
        "images": ["./gauge.jpg"],
    }],
)
print(resp.message.content)

Thinking mode

Gemma 4 can generate its reasoning before answering, and you can switch that on and off. In Ollama, control it with think; the reasoning arrives separately in message.thinking.

gemma_thinking.pyPYTHON
import time
from ollama import chat

question = "Pump vibration rose from a usual 2.1 mm/s to 7.8 mm/s while bearing temperature is normal. Likely causes and inspection order?"

for think in (False, True):
    started = time.perf_counter()
    resp = chat(model="gemma4:e4b", messages=[{"role": "user", "content": question}], think=think)
    elapsed = time.perf_counter() - started
    print(f"=== think={think} ({elapsed:.1f}s)")
    if resp.message.thinking:
        print("[thinking]", resp.message.thinking[:300], "...")
    print("[answer]", resp.message.content[:400])
💡

In Transformers use apply_chat_template(..., enable_thinking=True); with a raw prompt add the <|think|> token to the system prompt. Thinking slows responses, so turn it on only for hard requests.

Function calling

The Ollama Python library builds the tool schema from a Python function’s type hints and docstring when you pass the function in tools.

gemma_tools.pyPYTHON
from ollama import chat

def lookup_equipment(asset_id: str) -> dict:
    """Look up equipment type, location and last inspection date by asset ID.

    Args:
      asset_id: Asset ID (e.g. PUMP-07)
    """
    return {"asset_id": asset_id, "type": "centrifugal pump", "location": "Building B, level B2", "last_inspection": "2026-08-14"}

TOOLS = {"lookup_equipment": lookup_equipment}
messages = [{"role": "user", "content": "Where is PUMP-07 and when was it last inspected?"}]

resp = chat(model="gemma4:e4b", messages=messages, tools=[lookup_equipment])
messages.append(resp.message)

for call in resp.message.tool_calls or []:
    result = TOOLS[call.function.name](**call.function.arguments)   # Ollama gives arguments as a dict
    messages.append({"role": "tool", "tool_name": call.function.name, "content": str(result)})

final = chat(model="gemma4:e4b", messages=messages)
print(final.message.content)

Structured output

gemma_structured.pyPYTHON
from typing import Literal
from ollama import chat
from pydantic import BaseModel

class Finding(BaseModel):
    component: str
    issue: str
    severity: Literal["ok", "watch", "action", "stop"]

resp = chat(
    model="gemma4:e4b",
    messages=[{"role": "user", "content": "Rust on a pipe joint and a small puddle of water on the floor"}],
    format=Finding.model_json_schema(),
    options={"temperature": 0},
)
print(Finding.model_validate_json(resp.message.content))

Image and audio with Transformers

For input types Ollama does not support, or finer control, use Hugging Face Transformers. Gemma 4 handles text, images and audio with a single AutoModelForMultimodalLM. Download repositories such as google/gemma-4-E4B-it after accepting the license.

BASH
uv add -U transformers torch accelerate
gemma_audio.pyPYTHON
from transformers import AutoModelForMultimodalLM, AutoProcessor

MODEL_ID = "google/gemma-4-E4B-it"
processor = AutoProcessor.from_pretrained(MODEL_ID)
model = AutoModelForMultimodalLM.from_pretrained(MODEL_ID, dtype="auto", device_map="auto")

messages = [{
    "role": "user",
    "content": [
        # Images go before the text, audio after it (audio up to 30 seconds)
        {"type": "image", "url": "./gauge.jpg"},
        {"type": "text", "text": "Use the gauge photo and the voice memo together to summarize the inspection result"},
        {"type": "audio", "audio": "./memo.wav"},
    ],
}]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
    add_generation_prompt=True,
    enable_thinking=False,
).to(model.device)
input_len = inputs["input_ids"].shape[-1]

outputs = model.generate(**inputs, max_new_tokens=512)
response = processor.decode(outputs[0][input_len:], skip_special_tokens=False)
print(processor.parse_response(response, prefix=inputs["input_ids"]))
ℹ️

Only E2B, E4B and 12B accept audio. To run the same code on 26B A4B or 31B, drop the audio item and, if needed, pass a transcript produced by a smaller model.

Specialized models

ModelUse
EmbeddingGemmaText embeddings for RAG and search; small enough to search documents on device
Gemma 3nThe on-device model before Gemma 4; for maintaining existing apps

Fine-tuning follows the same TRL flow as the data preparation and QLoRA training steps in the Llama tutorial; for data with images, prepare it with an image/images column.

Troubleshooting

SymptomCauseFix
Transformers cannot find the model classAn old Transformers that predates Gemma 4Install the latest with uv add -U transformers
Audio input is ignored or errorsAudio sent to 26B A4B / 31B, or longer than 30 secondsUse E2B, E4B or 12B and split audio into 30-second pieces
Reasoning text mixed into the answerUsing raw output as isOllama: message.thinking; Transformers: parse_response
Answers vary a lot between runsUsing the recommended temperature 1.0 as isEvaluate classification and extraction with a lower temperature
Slow on Apple SiliconUsing the standard buildUse MLX tags such as gemma4:e4b-mlx

Frequently asked questions

Can I use Gemma 4 commercially for free?

Gemma 4 is released under Apache 2.0, so commercial use, modification and redistribution are free. Gemma 3 and earlier follow Google’s Gemma Terms of Use and prohibited use policy, so check those terms separately if you still use an older version.

What does the "E" in Gemma 4 E2B and E4B mean?

E stands for effective parameters. E4B has about 8B parameters including embeddings, but only about 4.5B are used in computation, so it runs at 4B-class speed and is designed for devices such as phones and laptops.

Which Gemma 4 models accept audio input?

E2B, E4B and 12B support audio input natively; 26B A4B and 31B accept text and images only. In Transformers you add a message item of type audio, and each audio input can be up to 30 seconds long.

How do I turn Gemma 4’s thinking mode on and off?

In Ollama, pass think=True or False to chat, and read the reasoning separately from message.thinking. In Transformers, use the enable_thinking argument of apply_chat_template; with a raw prompt, add the <|think|> token to the system prompt.

Should I pick 26B A4B or 31B?

26B A4B is an MoE model that computes only about 3.8B parameters per token, so it responds much faster than 31B; 31B is a dense model that is slower but highest in quality. Both are about 16-20 GB in Ollama’s 4-bit builds and run on a 24 GB GPU, so start with 26B A4B if latency matters and 31B if quality matters, and evaluate.