Skip to main content
AIDevOps
  • Learn
  • Learning Paths
  • Practice
  • Open Source
  • Books
  • Engineering

    AI DevOpsThe full map of building and operating AI servicesLLMOpsLLM deployment Β· evaluation Β· observabilityHands-on ProjectsBuild AI Agent projects

    Knowledge

    DocsTechnical documentationBlogEngineering articlesPloggerDevelopment log feed

    Validate

    Certification3-level skills certification Β· coming soon
AI Models
LlamaTutorial|MistralTutorial|GemmaTutorial|DeepSeekTutorial|QwenTutorial
🧠 AI Core
AI Intro & RoadmapML FundamentalsLLM Fundamentals|Python AIC++|PyTorchTensorFlowJAX
πŸ€– AI Applied Development
Applied AI Intro & RoadmapHugging FaceLangChainLlamaIndexLLMOps|LangGraphMCPMulti-AgentAgent Evaluation
🧠 AI Agent Development
Finance AI AgentLLM API ServerStock Investing AgentAIOps AI AgentEducation AI AgentCoding AI Agent
🌱 Spring Cloud
Spring Intro & RoadmapSpring Cloud GatewaySpring BootJava|Spring AISpring SecuritySpring BatchSpring JPA
🐳 DevOps
DevOps Intro & RoadmapLinuxDockerCI/CD|Kubernetes BasicsK8s AdvancedPrometheusGrafana
🧱 Infrastructure
Infrastructure Intro & RoadmapNginxRedis
☁️ Cloud
Cloud Intro & RoadmapAWSGCPAzureNCPCloudflare
🎨 Frontend
Frontend Intro & RoadmapJavaScriptTypeScript|ReactNext.js|VueNuxt|Electron
πŸ“± Mobile
Mobile Intro & RoadmapKotlinAndroidFlutter
βš™οΈ Backend
Backend Intro & RoadmapPython BasicsFastAPIDjangoFlask|CGoGinNode.js
πŸ’Ύ Database
DB Intro & RoadmapCore SQLOracleMySQLPostgreSQL|MongoDBVector DB
πŸ§ͺ Testing
k6JMeternGrinder
AIDevOps

Engineering AI. From Code to Production.
An engineering learning platform for building and operating AI and AI Agents

Learn

  • All Guides
  • Learning Paths
  • AI Tutorial
  • Practice
  • Books

Resources

  • AI DevOps
  • LLMOps
  • Hands-on Projects
  • Docs
  • Blog
  • Plogger
  • Open Source
  • Certification (coming soon)

Start Here

  • AI Core Roadmap
  • AI Applied Development Roadmap
  • Spring Cloud Roadmap
  • DevOps Roadmap
  • Infrastructure Roadmap

Β 

  • Cloud Roadmap
  • Frontend Roadmap
  • Mobile Roadmap
  • Backend Roadmap
  • Database Roadmap
Β© 2026 AIDevOps. All rights reserved.
Terms of ServicePrivacy PolicySitemaptestforge.kr
Apache 2.0 Open-Weight LLM Guide

🌊 Mistral Complete Guide

Visitors

France’s Mistral AI releases most of its models under Apache 2.0, which makes it the most commercially permissive open-weight LLM family. From the edge-sized Ministral 3 to Mistral Small 4, which merges reasoning, coding and vision into one model, and the large Large 3 MoE, this guide covers model choice, local runs, the API, function calling, OCR and serving in the order you would use them.

← Back to AIDevOps home
🌊

Mistral Practical Examples

Examples of Function Calling, the ReAct pattern, RAG, and text classification and translation APIs.

View practical examples β†’

Contents

0 / 13
  1. Mistral at a glance
  2. Model lineup & how to choose
  3. Three license types
  4. Hardware sizing
  5. Running locally with Ollama
  6. Mistral API & SDK 3.0
  7. Reasoning mode (reasoning_effort)
  8. Function Calling
  9. Structured output
  10. Mistral OCR
  11. vLLM serving
  12. Troubleshooting
  13. FAQ
Contents 13 sections
  1. Mistral at a glance
  2. Model lineup & how to choose
  3. Three license types
  4. Hardware sizing
  5. Running locally with Ollama
  6. Mistral API & SDK 3.0
  7. Reasoning mode (reasoning_effort)
  8. Function Calling
  9. Structured output
  10. Mistral OCR
  11. vLLM serving
  12. Troubleshooting
  13. FAQ

Mistral at a glance

Mistral AI is the French AI company behind Mistral 7B and Mixtral. Today it releases most models β€” from small edge models to MoE models with hundreds of billions of parameters β€” under Apache 2.0, and offers the same models through a paid API (Mistral AI Studio). Its biggest strength is how easy it is to start locally and scale to the API, or start on the API and move local.

WhenKey modelsWhat changed
December 2025Mistral Large 3, Ministral 3A 675B MoE flagship and 3B/8B/14B edge models, all Apache 2.0 with image input
March 2026Mistral Small 4Reasoning (Magistral), vision (Pixtral) and coding (Devstral) merged into one model with a reasoning toggle
April 2026Mistral Medium 3.5A 128B dense multimodal model tuned for agents and coding, under a Modified MIT license
ℹ️

This guide is based on Mistral’s official documentation as of October 2026. Older models such as Mistral 7B, Mixtral, Mistral Small 3.2 and Magistral are being retired from the API, so pick from the lineup below for new projects.

πŸ’‘

To build one end to end β€” API key, structured output, function calling, reasoning mode, OCR, moving to local, evaluation and serving β€” see the Mistral customer-support agent tutorial.

Model lineup & how to choose

Small 4 and Large 3 are MoE models that compute only some experts per token. Speed follows active parameters; the memory you need follows total parameters.

ModelParametersContextInputLicenseBest for
Ministral 3 (3B, 8B, 14B)3B / 8B / 14B256KText + imageApache 2.0Local and edge devices, classification and extraction, light agents
Mistral Small 4119B (about 6B active; 4 of 128 experts)256KText + imageApache 2.0General, reasoning and coding in one model; the default API choice
Mistral Medium 3.5128B (dense)256KText + imageModified MITServices where agent and coding quality matter
Mistral Large 3675B (41B active)256KText + imageApache 2.0Top-quality general multimodal work
CodestralNot disclosedβ€”CodePremier (API only)IDE code completion (FIM)
Mistral OCRNot disclosedβ€”Documents, imagesPremier (API only)Converting PDFs and scans to Markdown
Shieldstral3Bβ€”Text + imageApache 2.0Input/output safety classification (guardrails)

Mistral model names end with a version date (for example mistral-small-2603 = the March 2026 release). Use dated names in production code, and aliases such as mistral-small-latest, which always point to the newest release, for experiments.

Three license types

LicenseModelsWhat it means
Apache 2.0Ministral 3, Small 4, Large 3, Shieldstral, Voxtral SmallFree commercial use, modification and redistribution; no user-count limit; keep the license notice
Modified MITMistral Medium 3.5Commercial use allowed, but MIT with added exceptions β€” read the license in the model repository
PremierCodestral, Mistral OCR, Mistral EmbedNo public weights; paid use under the Mistral API terms
πŸ’‘

The practical difference of Apache 2.0 Mistral models is the absence of conditions like Llama’s 700-million-user threshold or attribution requirements. If your organization has a strict license review, they are worth evaluating first.

Hardware sizing

ModelFootprintRealistic setup
Ministral 3 3BOllama about 3.0 GBCPU laptop, 8 GB GPU
Ministral 3 8BOllama about 6.0 GB; about 12 GB in FP88-12 GB GPU
Ministral 3 14BOllama about 9.1 GB12-16 GB GPU
Mistral Medium 3.5Ollama about 80 GBAn H100 80 GB class GPU or several GPUs
Mistral Small 4Official minimum: 4Γ— H100 (HGX) or 2Γ— H200A GPU server β€” usually used through the API
Mistral Large 3675B weightsMulti-GPU nodes β€” Ollama only offers a cloud tag

Running locally with Ollama

Start local experiments with Ministral 3. Every size supports a 256K context and image input.

BASH
ollama pull ministral-3          # default 8B (about 6 GB)
ollama pull ministral-3:3b       # lightweight (about 3 GB)
ollama pull ministral-3:14b      # higher quality (about 9 GB)
ollama pull mistral-medium-3.5   # 128B (about 80 GB, needs a large GPU)

ollama run ministral-3 "Summarize the key conditions of the Apache 2.0 license in three lines"
ministral_local.pyPYTHON
from openai import OpenAI

# Ollama's OpenAI-compatible endpoint β€” same message format as the Mistral API
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

resp = client.chat.completions.create(
    model="ministral-3",
    messages=[
        {"role": "system", "content": "You are a DevOps engineer who answers concisely."},
        {"role": "user", "content": "Compare blue-green and canary deployments in a table"},
    ],
    temperature=0.1,   # Ministral 3 model card: below 0.1 for production use
)
print(resp.choices[0].message.content)
ℹ️

Ollama runs with a default context sized to your GPU memory (4K below 24 GiB). To use the 256K window, raise OLLAMA_CONTEXT_LENGTH or num_ctx, as described in the Llama guide’s context section.

Mistral API & SDK 3.0

Create an API key in Mistral AI Studio (console.mistral.ai) and install the Python SDK.

BASH
uv add "mistralai>=3"
export MISTRAL_API_KEY="your_key"
mistral_api.pyPYTHON
import os
from mistralai.client import Mistral   # SDK 3.0: "from mistralai import Mistral" no longer works

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])

# ── Basic call ───────────────────────────────────
resp = client.chat.complete(
    model="mistral-small-latest",
    messages=[{"role": "user", "content": "Why does PostgreSQL need VACUUM?"}],
    temperature=0.2,
)
print(resp.choices[0].message.content)
print(resp.usage)   # prompt_tokens / completion_tokens

# ── Streaming ────────────────────────────────────
for event in client.chat.stream(
    model="ministral-3-8b-2512",
    messages=[{"role": "user", "content": "Explain the three kinds of Kubernetes probes"}],
):
    delta = event.data.choices[0].delta.content
    if isinstance(delta, str):
        print(delta, end="", flush=True)
print()

Reasoning mode (reasoning_effort)

Mistral Small 4 and Medium 3.5 switch reasoning on and off within one model. Send easy requests with "none" for speed and cost, and only hard ones β€” calculations, planning, debugging β€” with "high".

reasoning.pyPYTHON
import os
from mistralai.client import Mistral
from mistralai.client.models import TextChunk, ThinkChunk

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])

resp = client.chat.complete(
    model="mistral-small-latest",
    messages=[{"role": "user", "content": "How many pods do we need to serve 1,200 requests per second at p95 200 ms? State your assumptions."}],
    reasoning_effort="high",          # "none" answers right away with no thinking
)

content = resp.choices[0].message.content
# With reasoning on, content is a list of chunks instead of a string
if isinstance(content, str):
    print(content)
else:
    for chunk in content:
        if isinstance(chunk, ThinkChunk):
            thinking = "".join(t.text for t in chunk.thinking if isinstance(t, TextChunk))
            print("[thinking]", thinking[:300], "...")
        elif isinstance(chunk, TextChunk):
            print("[answer]", chunk.text)
⚠️

Use the thinking text for debugging only; do not show it to end users as is. It uses many tokens, so track reasoning usage and cost separately per request type.

Function Calling

Mistral models take tool schemas and generate the function to call with its arguments. Use tool_choice to force a tool ("any") or let the model decide ("auto").

function_calling.pyPYTHON
import json
import os
from mistralai.client import Mistral

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
MODEL = "mistral-small-latest"

def get_order_status(order_id: str) -> dict:
    """Look up an order (call your order DB/API in a real implementation)"""
    return {"order_id": order_id, "status": "in transit", "eta": "2026-10-07"}

TOOLS = {"get_order_status": get_order_status}
TOOL_SCHEMAS = [{
    "type": "function",
    "function": {
        "name": "get_order_status",
        "description": "Get the status and estimated delivery date of an order",
        "parameters": {
            "type": "object",
            "properties": {"order_id": {"type": "string", "description": "Order number (e.g. A-1024)"}},
            "required": ["order_id"],
        },
    },
}]

messages = [{"role": "user", "content": "When will order A-1024 arrive?"}]

resp = client.chat.complete(model=MODEL, messages=messages, tools=TOOL_SCHEMAS, tool_choice="auto")
msg = resp.choices[0].message
messages.append(msg)

for tc in msg.tool_calls or []:
    raw = tc.function.arguments          # the Mistral SDK returns a dict or a JSON string
    args = raw if isinstance(raw, dict) else json.loads(raw)
    result = TOOLS[tc.function.name](**args)
    messages.append({
        "role": "tool",
        "name": tc.function.name,
        "content": json.dumps(result),
        "tool_call_id": tc.id,
    })

final = client.chat.complete(model=MODEL, messages=messages)
print(final.choices[0].message.content)

Structured output

Pass a Pydantic model to chat.parse to get JSON that matches the schema, ready to use as an object. It is the basic tool for classification and extraction.

structured.pyPYTHON
import os
from typing import Literal
from mistralai.client import Mistral
from pydantic import BaseModel

class Ticket(BaseModel):
    category: Literal["billing", "delivery", "account", "bug", "other"]
    urgency: Literal["low", "medium", "high"]
    summary: str

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])

resp = client.chat.parse(
    model="ministral-3-8b-2512",
    messages=[
        {"role": "system", "content": "Classify customer inquiries."},
        {"role": "user", "content": "I was charged twice. If it isn't refunded today, I'm blocking my card."},
    ],
    response_format=Ticket,
    temperature=0,
)
ticket = resp.choices[0].message.parsed
print(ticket.category, ticket.urgency, ticket.summary)

Mistral OCR

Mistral OCR turns PDFs and images into per-page Markdown, preserving tables and headings. It is especially useful for cleaning documents before RAG.

ocr.pyPYTHON
import os
from mistralai.client import Mistral

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])

result = client.ocr.process(
    model="mistral-ocr-latest",
    document={"type": "document_url", "document_url": "https://arxiv.org/pdf/2310.06825"},
)
for page in result.pages[:2]:
    print(f"--- page {page.index} ---")
    print(page.markdown[:500])
ℹ️

For local files, upload them with client.files.upload and pass the signed URL, or pass images as base64 data URLs. Check your company policy before sending documents containing personal data to an external API.

vLLM serving

Mistral models also ship weights, config and tokenizer in Mistral’s own format, so pass the format flags to vLLM (vLLM 0.12 or later).

BASH
vllm serve mistralai/Ministral-3-8B-Instruct-2512 \
  --tokenizer_mode mistral --config_format mistral --load_format mistral \
  --enable-auto-tool-choice --tool-call-parser mistral \
  --max-model-len 32768
vllm_client.pyPYTHON
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")
resp = client.chat.completions.create(
    model="mistralai/Ministral-3-8B-Instruct-2512",
    messages=[{"role": "user", "content": "Three Redis cache invalidation strategies"}],
    temperature=0.1,
)
print(resp.choices[0].message.content)
πŸ’‘

Without --max-model-len, vLLM tries to reserve KV cache for the default 262,144 tokens and can run out of memory. Start with the length you actually need. Monitoring and load testing work the same as in the Llama guide’s vLLM section.

Troubleshooting

SymptomCauseFix
ImportError: cannot import name 'Mistral'The client moved in SDK 3.0from mistralai.client import Mistral
content is a list, not a stringReasoning is on, so thinking/text chunks are returnedParse by chunk type, or reasoning_effort="none"
Old model names return 404 or retirement errorsRetired models such as Mistral Small 3.2 and MagistralUse dated names from the current lineup or -latest aliases
429 Too Many RequestsPer-minute request/token limit for your planExponential backoff, batching and caching, a higher limit
vLLM fails to loadMissing Mistral format flags or an old vLLM--tokenizer_mode/--config_format/--load_format mistral, vLLM 0.12+
Local answers vary a lotTemperature is highKeep it below 0.1 for production use

Frequently asked questions

Can I use Mistral models commercially for free?

Models released under Apache 2.0 β€” Ministral 3, Mistral Small 4 and Mistral Large 3 β€” can be used commercially, modified and redistributed freely, with no user-count limits. Mistral Medium 3.5 uses a Modified MIT license, so check its exceptions, and Premier models such as Codestral and Mistral OCR have no public weights and are only available through the API.

Should I use Ministral 3 or Mistral Small 4?

To run locally on a laptop or a single consumer GPU, Ministral 3 (3B, 8B, 14B) is the realistic choice. Mistral Small 4 is fast because only about 6B parameters are active, but its full weights are 119B and need several H100s, so most teams use it through the Mistral API rather than serving it themselves.

How do I turn on Mistral’s reasoning mode?

Mistral Small 4 and Medium 3.5 can switch reasoning on and off within one model. Pass reasoning_effort="high" and the model produces a thinking chunk before the answer; "none" answers right away. With reasoning on, the response content becomes a list of thinking and text chunks, so your parsing code must handle that.

I get an ImportError on from mistralai import Mistral.

Since version 3.0 of the mistralai Python SDK, the client lives in mistralai.client. Change the import to from mistralai.client import Mistral. Response chunk types such as ThinkChunk and TextChunk are imported from mistralai.client.models.

Can I run Mistral OCR locally?

Mistral OCR is a Premier model with no public weights, so it is API-only. To read documents locally you can send page images to Ministral 3, which accepts image input, but table and layout preservation is weaker than with the dedicated OCR model.