Skip to main content
AIDevOps
  • Learn
  • Learning Paths
  • Practice
  • Open Source
  • Books
  • Engineering

    AI DevOpsThe full map of building and operating AI servicesLLMOpsLLM deployment Β· evaluation Β· observabilityHands-on ProjectsBuild AI Agent projects

    Knowledge

    DocsTechnical documentationBlogEngineering articlesPloggerDevelopment log feed

    Validate

    Certification3-level skills certification Β· coming soon
AI Models
LlamaTutorial|MistralTutorial|GemmaTutorial|DeepSeekTutorial|QwenTutorial
🧠 AI Core
AI Intro & RoadmapML FundamentalsLLM Fundamentals|Python AIC++|PyTorchTensorFlowJAX
πŸ€– AI Applied Development
Applied AI Intro & RoadmapHugging FaceLangChainLlamaIndexLLMOps|LangGraphMCPMulti-AgentAgent Evaluation
🧠 AI Agent Development
Finance AI AgentLLM API ServerStock Investing AgentAIOps AI AgentEducation AI AgentCoding AI Agent
🌱 Spring Cloud
Spring Intro & RoadmapSpring Cloud GatewaySpring BootJava|Spring AISpring SecuritySpring BatchSpring JPA
🐳 DevOps
DevOps Intro & RoadmapLinuxDockerCI/CD|Kubernetes BasicsK8s AdvancedPrometheusGrafana
🧱 Infrastructure
Infrastructure Intro & RoadmapNginxRedis
☁️ Cloud
Cloud Intro & RoadmapAWSGCPAzureNCPCloudflare
🎨 Frontend
Frontend Intro & RoadmapJavaScriptTypeScript|ReactNext.js|VueNuxt|Electron
πŸ“± Mobile
Mobile Intro & RoadmapKotlinAndroidFlutter
βš™οΈ Backend
Backend Intro & RoadmapPython BasicsFastAPIDjangoFlask|CGoGinNode.js
πŸ’Ύ Database
DB Intro & RoadmapCore SQLOracleMySQLPostgreSQL|MongoDBVector DB
πŸ§ͺ Testing
k6JMeternGrinder
AIDevOps

Engineering AI. From Code to Production.
An engineering learning platform for building and operating AI and AI Agents

Learn

  • All Guides
  • Learning Paths
  • AI Tutorial
  • Practice
  • Books

Resources

  • AI DevOps
  • LLMOps
  • Hands-on Projects
  • Docs
  • Blog
  • Plogger
  • Open Source
  • Certification (coming soon)

Start Here

  • AI Core Roadmap
  • AI Applied Development Roadmap
  • Spring Cloud Roadmap
  • DevOps Roadmap
  • Infrastructure Roadmap

Β 

  • Cloud Roadmap
  • Frontend Roadmap
  • Mobile Roadmap
  • Backend Roadmap
  • Database Roadmap
Β© 2026 AIDevOps. All rights reserved.
Terms of ServicePrivacy PolicySitemaptestforge.kr
  1. Home
  2. Learn
  3. AI Core
  4. Python AI
Guide to AI development with Python

🐍 Python AI Guide

Visitors

Build LLM applications and AI Agents with Python. Laid out as a practical flow: asynchronous LLM calls, structured output with Pydantic, the ReAct loop, a FastAPI streaming server, and Agent testing.

  • Beginner
  • Updated Oct 1, 2026
  • ~6 min read
  • 11 sections
  • 8 code examples

Learning Paths that include this guide

This guide is one step in the paths below. Study it together with the steps before and after.

  • AI Foundations β†’
  • AI Agent Engineer β†’
  • AI Application Engineer β†’
LLM applicationsAI Agent developmentStructured outputFastAPI streaming servers

Related frameworks & environments

🐍Python Basicsβ†’πŸ”₯PyTorchβ†’TFTensorFlowβ†’HFHugging Faceβ†’LCLangChainβ†’LILlamaIndexβ†’

Contents

0 / 14
  1. How to use this guide
  2. Architecture diagrams
  3. Map of the AI ecosystem
  4. Environment setup (uv)
  5. Async LLM calls
  6. Pydantic v2 structured output
  7. Implementing the Agent loop yourself
  8. Concurrency & batch processing
  9. FastAPI Agent server
  10. Agent testing
  11. Python AI design
  12. Operating standards
  13. Verification strategy
  14. Frequently asked questions
Contents 14 sections
  1. How to use this guide
  2. Architecture diagrams
  3. Map of the AI ecosystem
  4. Environment setup (uv)
  5. Async LLM calls
  6. Pydantic v2 structured output
  7. Implementing the Agent loop yourself
  8. Concurrency & batch processing
  9. FastAPI Agent server
  10. Agent testing
  11. Python AI design
  12. Operating standards
  13. Verification strategy
  14. Frequently asked questions

How to use this guide

How to read it

Understanding Python AI as a real-world workflow

Build LLM applications and AI Agents with Python. Laid out as a practical flow: asynchronous LLM calls, structured output with Pydantic, the ReAct loop, a FastAPI streaming server, and Agent testing. Rather than listing concepts, this guide is organized so you can follow the content in the order you have to make decisions on a real project.

Key perspective

AI / LLM systems

Look beyond the model and the prompt: connect the data flow, evaluation, and post-deployment operating metrics in one view.

LLM applicationsAI Agent developmentStructured outputFastAPI streaming servers

Architecture diagrams

To make what you read stick, it helps to capture the flow as a picture first. The two diagrams below are reference maps you can keep coming back to while learning Python AI.

Learning flow

Rendering diagram…

Architecture view

Rendering diagram…

Map of the AI ecosystem

When you first open Python AI, the big picture matters more than individual commands. This section starts with the problems the concepts ahead were created to solve.

Python is the de facto standard language for AI/ML development. Knowing what to use at which layer is the starting point of Agent design.
LayerLibrariesRole
Model executiontransformers, vLLM, OllamaLLM loading Β· inference Β· serving
Agent orchestrationLangChain, LlamaIndex, smolagentsTool calls Β· memory Β· routing
Structured outputPydantic v2, instructor, outlinesSchema validation Β· forced JSON
Vector searchChromaDB, Pinecone, pgvectorRAG embedding store
API serverFastAPI, LangServeStreaming endpoints
Evaluation & tracingRAGAS, LangSmith, deepevalQuality measurement Β· tracing

Environment setup (uv)

Here you look at Environment setup (uv) alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

uv is a package manager 10-100x faster than pip. AI projects have complex dependencies, so always use a separate virtual environment per project and guarantee reproducibility with a lock file.
BASH
# Install uv (macOS/Linux)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Initialize an AI Agent project
uv init my-agent && cd my-agent

# Add the core packages (pyproject.toml is updated automatically)
uv add langchain langchain-openai langchain-community
uv add pydantic fastapi uvicorn chromadb
uv add instructor ragas langsmith

# Reproducible environment β€” exact versions pinned in the lock file
uv run python main.py          # activates the venv automatically
uv sync                        # sync to the lock file

Async LLM calls

Here you look at Async LLM calls alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

An LLM call takes hundreds of milliseconds to several seconds. Calling several models at once with asyncio can cut total response time sharply. Always design production Agents around async.
async_llm.pyPYTHON
import asyncio
from openai import AsyncOpenAI

client = AsyncOpenAI()

# ── Basic async call ──────────────────────────────
async def call_llm(prompt: str, model: str = "gpt-4o-mini") -> str:
    resp = await client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    return resp.choices[0].message.content

# ── Streaming (live token-by-token output) ───────
async def stream_llm(prompt: str):
    async with client.chat.completions.stream(
        model="gpt-4o",
        messages=[{"role": "user", "content": prompt}],
    ) as stream:
        async for chunk in stream:
            delta = chunk.choices[0].delta.content
            if delta:
                print(delta, end="", flush=True)

# ── Call several prompts in parallel ─────────────
async def batch_call(prompts: list[str]) -> list[str]:
    return await asyncio.gather(*[call_llm(p) for p in prompts])

async def main():
    prompts = ["What is Python?", "What is Go?", "What is Rust?"]
    # Serial: one after another (~3s)  vs  parallel: at the same time (~1s)
    serial   = [await call_llm(p) for p in prompts]
    parallel = await batch_call(prompts)          # about 3x faster

asyncio.run(main())

Pydantic v2 structured output

Here you look at Pydantic v2 structured output alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

Turning the free text of an LLM into a validated data structure is the key to building a reliable Agent. The instructor library uses OpenAI function calling to extract Pydantic models automatically.
BASH
uv add instructor
structured_output.pyPYTHON
import instructor
from openai import OpenAI
from pydantic import BaseModel, Field
from enum import Enum

client = instructor.from_openai(OpenAI())

class Priority(str, Enum):
    HIGH   = "high"
    MEDIUM = "medium"
    LOW    = "low"

class Task(BaseModel):
    title:    str        = Field(description="Task title")
    priority: Priority   = Field(description="Priority")
    deadline: str | None = Field(description="Deadline YYYY-MM-DD")
    subtasks: list[str]  = Field(default_factory=list)

class ProjectPlan(BaseModel):
    goal:        str        = Field(description="Project goal")
    tasks:       list[Task] = Field(description="Task list")
    total_weeks: int        = Field(description="Estimated duration (weeks)")

# The LLM returns a validated Pydantic object instead of free text
plan = client.chat.completions.create(
    model="gpt-4o",
    response_model=ProjectPlan,
    messages=[{"role": "user",
               "content": "Plan the launch of an AI chatbot service within 3 months"}]
)

for task in plan.tasks:
    print(f"[{task.priority.upper()}] {task.title} β€” due: {task.deadline}")
    for sub in task.subtasks:
        print(f"  β€’ {sub}")

Implementing the Agent loop yourself

Here you look at Implementing the Agent loop yourself alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

Implement the ReAct loop yourself with OpenAI function calling alone, without LangChain. Once you understand how the framework works inside, you can customize even complex Agents freely.
agent_loop.pyPYTHON
import json, math
from openai import OpenAI
from typing import Callable

client = OpenAI()

# ── Tool definitions ──────────────────────────────
def search_web(query: str) -> str:
    """Search the web for up-to-date information."""
    return f"[Search results] {query}: relevant recent information..."   # real: Tavily API

def calculate(expression: str) -> str:
    """Evaluate a math expression safely."""
    try:
        allowed = {k: v for k, v in math.__dict__.items() if not k.startswith("_")}
        return str(eval(expression, {"__builtins__": {}}, allowed))
    except Exception as e:
        return f"Error: {e}"

TOOLS: dict[str, Callable] = {"search_web": search_web, "calculate": calculate}
SCHEMAS = [
    {"type": "function", "function": {
        "name": "search_web", "description": "Web search",
        "parameters": {"type": "object",
                       "properties": {"query": {"type": "string"}},
                       "required": ["query"]}}},
    {"type": "function", "function": {
        "name": "calculate", "description": "Evaluate an expression",
        "parameters": {"type": "object",
                       "properties": {"expression": {"type": "string"}},
                       "required": ["expression"]}}},
]

# ── ReAct loop ────────────────────────────────────
def run_agent(query: str, max_steps: int = 5) -> str:
    messages = [{"role": "user", "content": query}]
    for _ in range(max_steps):
        resp = client.chat.completions.create(
            model="gpt-4o", messages=messages, tools=SCHEMAS
        )
        msg = resp.choices[0].message
        messages.append(msg)
        if not msg.tool_calls:          # no tools β†’ final answer
            return msg.content
        for tc in msg.tool_calls:       # run the tools
            args   = json.loads(tc.function.arguments)
            result = TOOLS[tc.function.name](**args)
            print(f"  [Tool] {tc.function.name}({args}) β†’ {result[:60]}")
            messages.append({"role": "tool", "tool_call_id": tc.id,
                              "content": result})
    return "Maximum steps exceeded"

print(run_agent("What is the current Apple share price multiplied by 2.5?"))

Concurrency & batch processing

Here you look at Concurrency & batch processing alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

In production you have to process hundreds to thousands of documents. Limit the number of concurrent calls with asyncio.Semaphore to avoid API rate limit errors while keeping throughput at its maximum.
batch_processor.pyPYTHON
import asyncio
from openai import AsyncOpenAI

client = AsyncOpenAI()

async def summarize_one(sem: asyncio.Semaphore, doc: str, idx: int) -> dict:
    async with sem:   # limit to at most N concurrent calls
        try:
            resp = await client.chat.completions.create(
                model="gpt-4o-mini",
                messages=[{"role": "user", "content": f"Summarize:\n{doc}"}],
                max_tokens=200,
            )
            return {"idx": idx, "summary": resp.choices[0].message.content,
                    "ok": True}
        except Exception as e:
            return {"idx": idx, "error": str(e), "ok": False}

async def batch_summarize(docs: list[str], concurrency: int = 10) -> list[dict]:
    sem     = asyncio.Semaphore(concurrency)
    results = await asyncio.gather(
        *[summarize_one(sem, doc, i) for i, doc in enumerate(docs)]
    )
    return sorted(results, key=lambda x: x["idx"])

# Process 1,000 documents ~10x faster than serially
docs    = [f"Document {i} content..." for i in range(100)]
results = asyncio.run(batch_summarize(docs))
print(f"Done: {sum(r['ok'] for r in results)}, "
      f"failed: {sum(not r['ok'] for r in results)}")

FastAPI Agent server

Here you look at FastAPI Agent server alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

Serving an Agent with FastAPI gives you SSE streaming responses, automatic Pydantic validation, and automatic OpenAPI docs right away. The frontend can receive tokens in real time with `EventSource`.
agent_server.pyPYTHON
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
from openai import AsyncOpenAI
import json

app    = FastAPI(title="AI Agent API")
client = AsyncOpenAI()

class ChatRequest(BaseModel):
    message:    str
    model:      str = "gpt-4o"
    session_id: str = "default"

# ── SSE streaming endpoint ───────────────────────
@app.post("/chat/stream")
async def chat_stream(req: ChatRequest):
    async def generate():
        async with client.chat.completions.stream(
            model=req.model,
            messages=[{"role": "user", "content": req.message}],
        ) as stream:
            async for chunk in stream:
                delta = chunk.choices[0].delta.content
                if delta:
                    yield f"data: {json.dumps({'token': delta})}\n\n"
        yield "data: [DONE]\n\n"

    return StreamingResponse(generate(), media_type="text/event-stream",
                             headers={"Cache-Control": "no-cache"})

# ── Structured output endpoint ────────────────────
class AnalysisResult(BaseModel):
    sentiment:  str
    key_points: list[str]
    confidence: float

@app.post("/analyze", response_model=AnalysisResult)
async def analyze(req: ChatRequest):
    import instructor
    inst = instructor.from_openai(AsyncOpenAI(), mode=instructor.Mode.JSON)
    return await inst.chat.completions.create(
        model=req.model, response_model=AnalysisResult,
        messages=[{"role": "user", "content": f"Analyze:\n{req.message}"}]
    )

# Run: uvicorn agent_server:app --reload --port 8000

Agent testing

Here you look at Agent testing alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

Agent tests replace LLM calls with mocks and verify the tool execution path and the validity of structured output. For non-deterministic LLM output, measure quality quantitatively with RAGAS and deepeval.
test_agent.pyPYTHON
import pytest
from unittest.mock import patch, MagicMock
from pydantic import BaseModel, ValidationError

from agent_loop import calculate

# ── Tool unit tests ───────────────────────────────
def test_calculate_basic():
    assert calculate("2 + 3") == "5"
    assert calculate("10 ** 2") == "100"
    assert float(calculate("math.sqrt(2)")) == pytest.approx(1.414, rel=1e-3)

def test_calculate_blocks_dangerous():
    assert "Error" in calculate("__import__('os').system('ls')")

# ── Mocked LLM call tests ─────────────────────────
def test_agent_calls_tool_then_answers():
    tool_call = MagicMock()
    tool_call.function.name      = "calculate"
    tool_call.function.arguments = '{"expression": "2 + 2"}'
    tool_call.id                 = "call_1"

    msg1 = MagicMock(); msg1.tool_calls = [tool_call]; msg1.content = None
    msg2 = MagicMock(); msg2.tool_calls = None;        msg2.content = "It is 4"

    r1 = MagicMock(); r1.choices[0].message = msg1
    r2 = MagicMock(); r2.choices[0].message = msg2

    with patch("agent_loop.client.chat.completions.create",
               side_effect=[r1, r2]):
        from agent_loop import run_agent
        assert "4" in run_agent("What is 2 plus 2?")

# ── Structured output schema validation ───────────
class AgentOut(BaseModel):
    answer:     str
    sources:    list[str]
    confidence: float

def test_valid_schema():
    out = AgentOut(answer="result", sources=["doc.pdf"], confidence=0.9)
    assert 0.0 <= out.confidence <= 1.0

def test_rejects_out_of_range():
    with pytest.raises(ValidationError):
        AgentOut(answer="", sources=[], confidence=1.5)

Python AI practical design

Python AI practical design is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.

The core of the Python guide is the flow of AI Agents and API servers, so move beyond script-centered code to a structure that separates config, client, prompt, tool, and evaluator. The bar for intermediate level and above is an Agent loop with explicit state transitions and failure handling, and input/output contracts pinned down with Pydantic models.
Decision pointQuestion to askPractical guideline
BoundariesWhich parts of the Python AI code are most likely to change?Separate input/output, configuration, external integrations, and the core rules.
StateWhere is state created and where does it go away?Make the owner and lifecycle of state visible in the code.
FailureWhat does the caller receive on failure?Decide the timeout, fallback, and error contract first.

Python AI operating standards

This section covers Python AI operating standards from a practical perspective. Instead of memorizing the concept, focus on the situations in which you would reach for it.

In operation, the number of async calls, model timeouts, token usage, the retry policy, and cache key design decide cost and latency. Do not run batch jobs and the real-time API without limit in the same event loop; cap throughput with a queue and a semaphore.

Tip

  • Pydantic schema versioning
  • LLM timeout/retry budget
  • prompt regression set
  • token/cost metric

Python AI verification strategy

Python AI verification strategy is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.

Function-level tests alone are not enough for Agent quality. Manage a golden prompt set, samples of failed responses, the structured output schema, and hallucination criteria as test assets.
Quality axisHow to verifyDefinition of done
CorrectnessAutomate the normal/failure cases.The core scenarios pass reproducibly.
Regression preventionWhen fixing a bug, leave the same case behind as a test.The same failure is not deployed again.
OperabilityCheck the logs, metrics, and alerts.There is a path for tracing the cause when a problem occurs.

Python AI: frequently asked questions

Why is Python used for AI development?

The major libraries, examples, and paper implementations (PyTorch, TensorFlow, Hugging Face, LangChain) are mostly provided in Python. The heavy computation runs in C++ and GPU code underneath, so Python's speed is rarely the bottleneck. Even when the service is written in another language, experiments and training are usually done in Python.

Can I do AI development without a GPU?

Yes. Building applications that call model APIs needs no GPU, and inference with small models or classical machine learning runs fine on a CPU. You need a GPU to train large models or to run fast local inference, and then renting a cloud GPU or using a free notebook environment is an option.

When should I use a Jupyter notebook and when a Python script?

Notebooks are convenient for exploring data and experimenting while seeing results immediately. Code that runs repeatedly, needs tests, and gets deployed belongs in scripts and modules. Notebook results can depend on the order cells were run, and notebooks are hard to review and version.

← Previous guideLLM FundamentalsNext guide β†’C++