Build LLM applications and AI Agents with Python. Laid out as a practical flow: asynchronous LLM calls, structured output with Pydantic, the ReAct loop, a FastAPI streaming server, and Agent testing.
Beginner
Updated Oct 1, 2026
~6 min read
11 sections
8 code examples
Learning Paths that include this guide
This guide is one step in the paths below. Study it together with the steps before and after.
Build LLM applications and AI Agents with Python. Laid out as a practical flow: asynchronous LLM calls, structured output with Pydantic, the ReAct loop, a FastAPI streaming server, and Agent testing. Rather than listing concepts, this guide is organized so you can follow the content in the order you have to make decisions on a real project.
Key perspective
AI / LLM systems
Look beyond the model and the prompt: connect the data flow, evaluation, and post-deployment operating metrics in one view.
To make what you read stick, it helps to capture the flow as a picture first. The two diagrams below are reference maps you can keep coming back to while learning Python AI.
Learning flow
Rendering diagramβ¦
Architecture view
Rendering diagramβ¦
Map of the AI ecosystem
When you first open Python AI, the big picture matters more than individual commands. This section starts with the problems the concepts ahead were created to solve.
Python is the de facto standard language for AI/ML development. Knowing what to use at which layer is the starting point of Agent design.
Layer
Libraries
Role
Model execution
transformers, vLLM, Ollama
LLM loading Β· inference Β· serving
Agent orchestration
LangChain, LlamaIndex, smolagents
Tool calls Β· memory Β· routing
Structured output
Pydantic v2, instructor, outlines
Schema validation Β· forced JSON
Vector search
ChromaDB, Pinecone, pgvector
RAG embedding store
API server
FastAPI, LangServe
Streaming endpoints
Evaluation & tracing
RAGAS, LangSmith, deepeval
Quality measurement Β· tracing
Environment setup (uv)
Here you look at Environment setup (uv) alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
uv is a package manager 10-100x faster than pip. AI projects have complex dependencies, so always use a separate virtual environment per project and guarantee reproducibility with a lock file.
BASH
# Install uv (macOS/Linux)curl -LsSf https://astral.sh/uv/install.sh | sh# Initialize an AI Agent projectuv init my-agent && cd my-agent# Add the core packages (pyproject.toml is updated automatically)uv add langchain langchain-openai langchain-communityuv add pydantic fastapi uvicorn chromadbuv add instructor ragas langsmith# Reproducible environment β exact versions pinned in the lock fileuv run python main.py # activates the venv automaticallyuv sync # sync to the lock file
Async LLM calls
Here you look at Async LLM calls alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
An LLM call takes hundreds of milliseconds to several seconds. Calling several models at once with asyncio can cut total response time sharply. Always design production Agents around async.
async_llm.pyPYTHON
import asynciofrom openai import AsyncOpenAIclient = AsyncOpenAI()# ββ Basic async call ββββββββββββββββββββββββββββββasync def call_llm(prompt: str, model: str = "gpt-4o-mini") -> str: resp = await client.chat.completions.create( model=model, messages=[{"role": "user", "content": prompt}], ) return resp.choices[0].message.content# ββ Streaming (live token-by-token output) βββββββasync def stream_llm(prompt: str): async with client.chat.completions.stream( model="gpt-4o", messages=[{"role": "user", "content": prompt}], ) as stream: async for chunk in stream: delta = chunk.choices[0].delta.content if delta: print(delta, end="", flush=True)# ββ Call several prompts in parallel βββββββββββββasync def batch_call(prompts: list[str]) -> list[str]: return await asyncio.gather(*[call_llm(p) for p in prompts])async def main(): prompts = ["What is Python?", "What is Go?", "What is Rust?"] # Serial: one after another (~3s) vs parallel: at the same time (~1s) serial = [await call_llm(p) for p in prompts] parallel = await batch_call(prompts) # about 3x fasterasyncio.run(main())
Pydantic v2 structured output
Here you look at Pydantic v2 structured output alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Turning the free text of an LLM into a validated data structure is the key to building a reliable Agent. The instructor library uses OpenAI function calling to extract Pydantic models automatically.
BASH
uv add instructor
structured_output.pyPYTHON
import instructorfrom openai import OpenAIfrom pydantic import BaseModel, Fieldfrom enum import Enumclient = instructor.from_openai(OpenAI())class Priority(str, Enum): HIGH = "high" MEDIUM = "medium" LOW = "low"class Task(BaseModel): title: str = Field(description="Task title") priority: Priority = Field(description="Priority") deadline: str | None = Field(description="Deadline YYYY-MM-DD") subtasks: list[str] = Field(default_factory=list)class ProjectPlan(BaseModel): goal: str = Field(description="Project goal") tasks: list[Task] = Field(description="Task list") total_weeks: int = Field(description="Estimated duration (weeks)")# The LLM returns a validated Pydantic object instead of free textplan = client.chat.completions.create( model="gpt-4o", response_model=ProjectPlan, messages=[{"role": "user", "content": "Plan the launch of an AI chatbot service within 3 months"}])for task in plan.tasks: print(f"[{task.priority.upper()}] {task.title} β due: {task.deadline}") for sub in task.subtasks: print(f" β’ {sub}")
Implementing the Agent loop yourself
Here you look at Implementing the Agent loop yourself alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Implement the ReAct loop yourself with OpenAI function calling alone, without LangChain. Once you understand how the framework works inside, you can customize even complex Agents freely.
agent_loop.pyPYTHON
import json, mathfrom openai import OpenAIfrom typing import Callableclient = OpenAI()# ββ Tool definitions ββββββββββββββββββββββββββββββdef search_web(query: str) -> str: """Search the web for up-to-date information.""" return f"[Search results] {query}: relevant recent information..." # real: Tavily APIdef calculate(expression: str) -> str: """Evaluate a math expression safely.""" try: allowed = {k: v for k, v in math.__dict__.items() if not k.startswith("_")} return str(eval(expression, {"__builtins__": {}}, allowed)) except Exception as e: return f"Error: {e}"TOOLS: dict[str, Callable] = {"search_web": search_web, "calculate": calculate}SCHEMAS = [ {"type": "function", "function": { "name": "search_web", "description": "Web search", "parameters": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]}}}, {"type": "function", "function": { "name": "calculate", "description": "Evaluate an expression", "parameters": {"type": "object", "properties": {"expression": {"type": "string"}}, "required": ["expression"]}}},]# ββ ReAct loop ββββββββββββββββββββββββββββββββββββdef run_agent(query: str, max_steps: int = 5) -> str: messages = [{"role": "user", "content": query}] for _ in range(max_steps): resp = client.chat.completions.create( model="gpt-4o", messages=messages, tools=SCHEMAS ) msg = resp.choices[0].message messages.append(msg) if not msg.tool_calls: # no tools β final answer return msg.content for tc in msg.tool_calls: # run the tools args = json.loads(tc.function.arguments) result = TOOLS[tc.function.name](**args) print(f" [Tool] {tc.function.name}({args}) β {result[:60]}") messages.append({"role": "tool", "tool_call_id": tc.id, "content": result}) return "Maximum steps exceeded"print(run_agent("What is the current Apple share price multiplied by 2.5?"))
Concurrency & batch processing
Here you look at Concurrency & batch processing alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
In production you have to process hundreds to thousands of documents. Limit the number of concurrent calls with asyncio.Semaphore to avoid API rate limit errors while keeping throughput at its maximum.
batch_processor.pyPYTHON
import asynciofrom openai import AsyncOpenAIclient = AsyncOpenAI()async def summarize_one(sem: asyncio.Semaphore, doc: str, idx: int) -> dict: async with sem: # limit to at most N concurrent calls try: resp = await client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": f"Summarize:\n{doc}"}], max_tokens=200, ) return {"idx": idx, "summary": resp.choices[0].message.content, "ok": True} except Exception as e: return {"idx": idx, "error": str(e), "ok": False}async def batch_summarize(docs: list[str], concurrency: int = 10) -> list[dict]: sem = asyncio.Semaphore(concurrency) results = await asyncio.gather( *[summarize_one(sem, doc, i) for i, doc in enumerate(docs)] ) return sorted(results, key=lambda x: x["idx"])# Process 1,000 documents ~10x faster than seriallydocs = [f"Document {i} content..." for i in range(100)]results = asyncio.run(batch_summarize(docs))print(f"Done: {sum(r['ok'] for r in results)}, " f"failed: {sum(not r['ok'] for r in results)}")
FastAPI Agent server
Here you look at FastAPI Agent server alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Serving an Agent with FastAPI gives you SSE streaming responses, automatic Pydantic validation, and automatic OpenAPI docs right away. The frontend can receive tokens in real time with `EventSource`.
Here you look at Agent testing alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Agent tests replace LLM calls with mocks and verify the tool execution path and the validity of structured output. For non-deterministic LLM output, measure quality quantitatively with RAGAS and deepeval.
Python AI practical design is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.
The core of the Python guide is the flow of AI Agents and API servers, so move beyond script-centered code to a structure that separates config, client, prompt, tool, and evaluator. The bar for intermediate level and above is an Agent loop with explicit state transitions and failure handling, and input/output contracts pinned down with Pydantic models.
Decision point
Question to ask
Practical guideline
Boundaries
Which parts of the Python AI code are most likely to change?
Separate input/output, configuration, external integrations, and the core rules.
State
Where is state created and where does it go away?
Make the owner and lifecycle of state visible in the code.
Failure
What does the caller receive on failure?
Decide the timeout, fallback, and error contract first.
Python AI operating standards
This section covers Python AI operating standards from a practical perspective. Instead of memorizing the concept, focus on the situations in which you would reach for it.
In operation, the number of async calls, model timeouts, token usage, the retry policy, and cache key design decide cost and latency. Do not run batch jobs and the real-time API without limit in the same event loop; cap throughput with a queue and a semaphore.
Python AI verification strategy
Python AI verification strategy is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.
Function-level tests alone are not enough for Agent quality. Manage a golden prompt set, samples of failed responses, the structured output schema, and hallucination criteria as test assets.
Quality axis
How to verify
Definition of done
Correctness
Automate the normal/failure cases.
The core scenarios pass reproducibly.
Regression prevention
When fixing a bug, leave the same case behind as a test.
The same failure is not deployed again.
Operability
Check the logs, metrics, and alerts.
There is a path for tracing the cause when a problem occurs.
Python AI: frequently asked questions
Why is Python used for AI development?
The major libraries, examples, and paper implementations (PyTorch, TensorFlow, Hugging Face, LangChain) are mostly provided in Python. The heavy computation runs in C++ and GPU code underneath, so Python's speed is rarely the bottleneck. Even when the service is written in another language, experiments and training are usually done in Python.
Can I do AI development without a GPU?
Yes. Building applications that call model APIs needs no GPU, and inference with small models or classical machine learning runs fine on a CPU. You need a GPU to train large models or to run fast local inference, and then renting a cloud GPU or using a free notebook environment is an option.
When should I use a Jupyter notebook and when a Python script?
Notebooks are convenient for exploring data and experimenting while seeing results immediately. Code that runs repeatedly, needs tests, and gets deployed belongs in scripts and modules. Notebook results can depend on the order cells were run, and notebooks are hard to review and version.