Skip to main content
AIDevOps
  • Learn
  • Learning Paths
  • Practice
  • Open Source
  • Books
  • Engineering

    AI DevOpsThe full map of building and operating AI servicesLLMOpsLLM deployment Β· evaluation Β· observabilityHands-on ProjectsBuild AI Agent projects

    Knowledge

    DocsTechnical documentationBlogEngineering articlesPloggerDevelopment log feed

    Validate

    Certification3-level skills certification Β· coming soon
AI Models
LlamaTutorial|MistralTutorial|GemmaTutorial|DeepSeekTutorial|QwenTutorial
πŸ€– AI Applied Development
Applied AI Intro & RoadmapHugging FaceLangChainLlamaIndexLLMOps|LangGraphMCPMulti-AgentAgent Evaluation
🧠 AI Core
AI Intro & RoadmapML FundamentalsLLM Fundamentals|Python AIC++|PyTorchTensorFlowJAX
🧠 AI Agent Development
Finance AI AgentLLM API ServerStock Investing AgentAIOps AI AgentEducation AI AgentCoding AI Agent
🌱 Spring Cloud
Spring Intro & RoadmapSpring Cloud GatewaySpring BootJava|Spring AISpring SecuritySpring BatchSpring JPA
🐳 DevOps
DevOps Intro & RoadmapLinuxDockerCI/CD|Kubernetes BasicsK8s AdvancedPrometheusGrafana
🧱 Infrastructure
Infrastructure Intro & RoadmapNginxRedis
☁️ Cloud
Cloud Intro & RoadmapAWSGCPAzureNCPCloudflare
🎨 Frontend
Frontend Intro & RoadmapJavaScriptTypeScript|ReactNext.js|VueNuxt|Electron
πŸ“± Mobile
Mobile Intro & RoadmapKotlinAndroidFlutter
βš™οΈ Backend
Backend Intro & RoadmapPython BasicsFastAPIDjangoFlask|CGoGinNode.js
πŸ’Ύ Database
DB Intro & RoadmapCore SQLOracleMySQLPostgreSQL|MongoDBVector DB
πŸ§ͺ Testing
k6JMeternGrinder
AIDevOps

Engineering AI. From Code to Production.
An engineering learning platform for building and operating AI and AI Agents

Learn

  • All Guides
  • Learning Paths
  • AI Tutorial
  • Practice
  • Books

Resources

  • AI DevOps
  • LLMOps
  • Hands-on Projects
  • Docs
  • Blog
  • Plogger
  • Open Source
  • Certification (coming soon)

Start Here

  • AI Core Roadmap
  • AI Applied Development Roadmap
  • Spring Cloud Roadmap
  • DevOps Roadmap
  • Infrastructure Roadmap

Β 

  • Cloud Roadmap
  • Frontend Roadmap
  • Mobile Roadmap
  • Backend Roadmap
  • Database Roadmap
Β© 2026 AIDevOps. All rights reserved.
Terms of ServicePrivacy PolicySitemaptestforge.kr
  1. Home
  2. Learn
  3. AI Applied Development
  4. LangChain
Guide to the AI Agent development framework

LC LangChain Complete Guide

Visitors

Build a ReAct Agent, multi-agent systems, and advanced RAG pipelines with LangChain LCEL. Covers the whole journey of AI Agent development, through production serving, LangSmith observability, and RAGAS evaluation.

  • Intermediate
  • Updated Oct 1, 2026
  • ~14 min read
  • 12 sections
  • 9 code examples
  • Web IDE lab included
LC

LangChain Web IDE

Run code in the browser with nothing to install and learn through step-by-step examples.

Open the Web IDE β†’
ReAct Agent developmentMulti-agent systemsRAG pipelinesDeploying LLM apps to production

Related frameworks & environments

🐍Python AIβ†’HFHugging Faceβ†’LILlamaIndexβ†’

Contents

0 / 15
  1. How to use this guide
  2. Architecture diagrams
  3. LCEL architecture
  4. Designing custom tools
  5. ReAct agent
  6. Structured output
  7. Memory systems
  8. Multi-agent
  9. Streaming
  10. Advanced RAG
  11. Evaluation & observability
  12. LangChain design
  13. Operating standards
  14. Verification strategy
  15. Frequently asked questions
Contents 15 sections
  1. How to use this guide
  2. Architecture diagrams
  3. LCEL architecture
  4. Designing custom tools
  5. ReAct agent
  6. Structured output
  7. Memory systems
  8. Multi-agent
  9. Streaming
  10. Advanced RAG
  11. Evaluation & observability
  12. LangChain design
  13. Operating standards
  14. Verification strategy
  15. Frequently asked questions

How to use this guide

How to read it

Understanding LangChain as a real-world workflow

Build a ReAct Agent, multi-agent systems, and advanced RAG pipelines with LangChain LCEL. Covers the whole journey of AI Agent development, through production serving, LangSmith observability, and RAGAS evaluation. Rather than listing concepts, this guide is organized so you can follow the content in the order you have to make decisions on a real project.

Key perspective

AI / LLM systems

Look beyond the model and the prompt: connect the data flow, evaluation, and post-deployment operating metrics in one view.

ReAct Agent developmentMulti-agent systemsRAG pipelinesDeploying LLM apps to production

Architecture diagrams

To make what you read stick, it helps to capture the flow as a picture first. The two diagrams below are reference maps you can keep coming back to while learning LangChain.

Learning flow

Rendering diagram…

Architecture view

Rendering diagram…

LCEL architecture

When you first open LangChain, the big picture matters more than individual commands. This section starts with the problems the concepts ahead were created to solve.

The LangChain Expression Language (LCEL) composes components with the | pipe operator. RunnableParallel runs in parallel, RunnablePassthrough passes values through transparently, and RunnableLambda inserts arbitrary functions. LCEL chains support streaming, batching, and async alike, which makes them a good fit for production serving.
Rendering diagram…
lcel_arch.pyPYTHON
from operator import itemgetter
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnableParallel, RunnablePassthrough, RunnableLambda
from langchain_openai import ChatOpenAI

model = ChatOpenAI(model="gpt-4o-mini", temperature=0)
parser = StrOutputParser()

# ── 1. Basic pipe composition ────────────────────────────────────
prompt = ChatPromptTemplate.from_template("Explain {topic} in one sentence.")
basic_chain = prompt | model | parser

# ── 2. RunnableParallel: run two chains at once ──────────────────
summary_prompt   = ChatPromptTemplate.from_template("Summarize the following in 3 lines: {text}")
sentiment_prompt = ChatPromptTemplate.from_template("Analyze the sentiment of the following: {text}")

parallel_chain = RunnableParallel(
    summary=summary_prompt | model | parser,
    sentiment=sentiment_prompt | model | parser,
)

# ── 3. The RunnablePassthrough + assign pattern ─────────────────
# Keep the original input while adding a new key
augmented_chain = RunnablePassthrough.assign(
    summary=itemgetter("text") | summary_prompt | model | parser
)

# ── 4. RunnableLambda: custom preprocessing ─────────────────────
def preprocess(inputs: dict) -> dict:
    """Normalize the input text"""
    return {"text": inputs["text"].strip().lower()}

full_chain = (
    RunnableLambda(preprocess)
    | RunnableParallel(
        summary=summary_prompt | model | parser,
        sentiment=sentiment_prompt | model | parser,
    )
)

# ── 5. Run ───────────────────────────────────────────────────────
if __name__ == "__main__":
    print(basic_chain.invoke({"topic": "LCEL"}))

    result = parallel_chain.invoke({"text": "LangChain is an AI development framework."})
    print("Summary:", result["summary"])
    print("Sentiment:", result["sentiment"])

    # Batch run (several inputs at once)
    topics = [{"topic": "RAG"}, {"topic": "Agent"}, {"topic": "Embedding"}]
    for r in basic_chain.batch(topics):
        print(r)
ComponentRole
RunnableParallelRuns several chains at once and merges the results into a dict.
RunnablePassthroughPasses the input on to the next step unchanged.
RunnableLambdaWraps an ordinary Python function as a Runnable.
itemgetter / assignUsed in the middle of a chain to extract a specific key and add values.

Designing custom tools

Here you look at Designing custom tools alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

The @tool decorator turns an ordinary function into an Agent tool. A StructuredTool, which validates its input with a Pydantic schema, handles complex arguments safely. The tool description (docstring) is what the LLM bases its choice of tool on, so it has to be written clearly.
custom_tools.pyPYTHON
import json
import sqlite3
from typing import Optional
from pydantic import BaseModel, Field
from langchain_core.tools import tool, StructuredTool

# ── 1. The @tool decorator: the simplest way ────────────────────
@tool
def web_search(query: str) -> str:
    """Searches the internet for up-to-date information. Use it when real-time data or recent news is needed."""
    # Real implementation: use from tavily import TavilyClient or similar
    return f"Search results for '{query}': [search API response]"

@tool
def python_repl(code: str) -> str:
    """Runs Python code and returns the result. Use it for calculation, data processing, and checking code."""
    import io, contextlib
    buf = io.StringIO()
    try:
        with contextlib.redirect_stdout(buf):
            exec(code, {})  # use a sandboxed environment in production
        return buf.getvalue() or "Done (no output)"
    except Exception as e:
        return f"Error: {e}"

@tool
def read_file(path: str) -> str:
    """Reads a local file and returns its contents. Use it to analyze documents or check configuration files."""
    try:
        with open(path, "r", encoding="utf-8") as f:
            return f.read()
    except FileNotFoundError:
        return f"File not found: {path}"

# ── 2. StructuredTool: validate input with a Pydantic schema ─────
class DBQueryInput(BaseModel):
    sql: str = Field(description="The SELECT SQL query to run")
    db_path: str = Field(default="./data.db", description="Path to the SQLite file")
    limit: Optional[int] = Field(default=10, description="Maximum number of rows to return")

def run_db_query(sql: str, db_path: str = "./data.db", limit: int = 10) -> str:
    """Runs a SELECT query against a SQLite database."""
    if not sql.strip().upper().startswith("SELECT"):
        return "Error: only SELECT queries are allowed."
    try:
        conn = sqlite3.connect(db_path)
        cursor = conn.execute(sql)
        rows = cursor.fetchmany(limit)
        columns = [d[0] for d in cursor.description]
        conn.close()
        return json.dumps({"columns": columns, "rows": rows}, ensure_ascii=False)
    except Exception as e:
        return f"DB error: {e}"

db_query_tool = StructuredTool.from_function(
    func=run_db_query,
    name="db_query",
    description="Looks up information in the database. Use it for statistics and record searches.",
    args_schema=DBQueryInput,
)

# ── 3. List the tools ────────────────────────────────────────────
tools = [web_search, python_repl, read_file, db_query_tool]
for t in tools:
    print(f"[{t.name}] {t.description[:60]}...")

ReAct agent

Here you look at ReAct agent alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

In the ReAct (Reasoning + Acting) pattern, the model solves complex problems step by step by repeating a Thought β†’ Action β†’ Observation loop. The create_react_agent + AgentExecutor combination is the standard ReAct implementation in LangChain. Monitor the internal reasoning with verbose=True.
Rendering diagram…
react_agent.pyPYTHON
import os
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
from langchain_core.messages import HumanMessage, AIMessage
from langchain import hub
from langchain.agents import create_react_agent, AgentExecutor

os.environ["OPENAI_API_KEY"] = "sk-..."

# ── Tool definitions ─────────────────────────────────────────────
@tool
def calculator(expression: str) -> str:
    """Performs math calculations. Examples: '2 ** 10', 'sum([1,2,3])'"""
    try:
        return str(eval(expression, {"__builtins__": {}}, {}))
    except Exception as e:
        return f"Calculation error: {e}"

@tool
def web_search(query: str) -> str:
    """Searches for up-to-date information. Use it when real-time data, weather, or news is needed."""
    return f"Search results: the latest information about '{query}'."

@tool
def get_weather(city: str) -> str:
    """Looks up the current weather for a given city."""
    return f"Current weather in {city}: sunny, 22Β°C, humidity 55%"

tools = [calculator, web_search, get_weather]

# ── Create the ReAct agent ───────────────────────────────────────
model = ChatOpenAI(model="gpt-4o", temperature=0)
react_prompt = hub.pull("hwchase17/react")  # enforces the Thought/Action/Observation format

agent = create_react_agent(llm=model, tools=tools, prompt=react_prompt)
agent_executor = AgentExecutor(
    agent=agent, tools=tools,
    verbose=True,          # print the Thought/Action/Observation log
    max_iterations=10,     # prevent infinite loops
    handle_parsing_errors=True,
)

# ── Example run ──────────────────────────────────────────────────
if __name__ == "__main__":
    # Multi-step reasoning: weather lookup + unit conversion
    result = agent_executor.invoke({
        "input": "Tell me the current weather in Seoul and convert the temperature to Fahrenheit."
    })
    print("\nFinal answer:", result["output"])

    # Conversational agent (with history)
    chat_history = []
    queries = ["What is 2 to the power of 32?", "Work out how many GB that number is."]

    for query in queries:
        result = agent_executor.invoke({"input": query, "chat_history": chat_history})
        chat_history.extend([
            HumanMessage(content=query),
            AIMessage(content=result["output"]),
        ])
        print(f"Q: {query}\nA: {result['output']}\n")

Structured output

Here you look at Structured output alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

Forcing LLM responses into JSON or a Pydantic model lets downstream code handle them safely with no parsing failures. with_structured_output() uses function calling internally to guarantee the schema. TypedDict is useful when there are many optional fields.
structured_output.pyPYTHON
from typing import Optional, List
from typing_extensions import TypedDict
from pydantic import BaseModel, Field
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import PydanticOutputParser

model = ChatOpenAI(model="gpt-4o-mini", temperature=0)

# ── 1. Structured output with a Pydantic model ──────────────────
class Product(BaseModel):
    name: str = Field(description="Product name")
    price: float = Field(description="Price (USD)")
    category: str = Field(description="Category")
    in_stock: bool = Field(description="Whether it is in stock")
    tags: List[str] = Field(default_factory=list, description="List of tags")
    description: Optional[str] = Field(default=None, description="Product description")

structured_model = model.with_structured_output(Product)
prompt = ChatPromptTemplate.from_template("Extract the product information from the following text:\n\n{text}")
extraction_chain = prompt | structured_model

product = extraction_chain.invoke({
    "text": "New wireless earbuds AX-100, priced at 89 USD, in stock. Noise cancelling, Bluetooth"
})
print(f"Product: {product.name}, price: {product.price:,} USD")
print(f"Tags: {product.tags}")

# ── 2. TypedDict schema ──────────────────────────────────────────
class AnalysisResult(TypedDict):
    sentiment: str       # positive / negative / neutral
    score: float         # 0.0 ~ 1.0
    keywords: List[str]  # key keywords
    summary: str         # summary

typed_model = model.with_structured_output(AnalysisResult)
analysis_chain = ChatPromptTemplate.from_template("Analyze the following review:\n{review}") | typed_model

result = analysis_chain.invoke({"review": "Shipping was fast and the product quality is good. Great value for the price!"})
print(f"Sentiment: {result['sentiment']}, score: {result['score']}")

# ── 3. PydanticOutputParser (for models without function calling) ──
class ResearchReport(BaseModel):
    title: str
    key_findings: List[str]
    conclusion: str
    confidence: float = Field(ge=0, le=1)

pydantic_parser = PydanticOutputParser(pydantic_object=ResearchReport)
format_instructions = pydantic_parser.get_format_instructions()

report_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are an expert at writing research reports.\n{format_instructions}"),
    ("user", "Write a research report on {topic}."),
])
report = (report_prompt | model | pydantic_parser).invoke({
    "topic": "the state of enterprise adoption of large language models (LLMs)",
    "format_instructions": format_instructions,
})
print(f"Report: {report.title}")
print(f"Key findings: {report.key_findings}")

Memory systems

Here you look at Memory systems alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

How conversation history is managed decides how well an agent understands context. Use ConversationBufferWindowMemory for short conversations and VectorStoreRetrieverMemory when long-term memory is needed. The current LCEL approach injects history into the chain with RunnableWithMessageHistory.
memory_system.pyPYTHON
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_community.chat_message_histories import ChatMessageHistory
from langchain.memory import ConversationBufferWindowMemory, VectorStoreRetrieverMemory
from langchain_community.vectorstores import Chroma

model = ChatOpenAI(model="gpt-4o-mini", temperature=0.7)

# ── 1. ConversationBufferWindowMemory: keep only the last N turns ─
window_memory = ConversationBufferWindowMemory(
    k=5,                   # keep only the last 5 turns
    return_messages=True,  # return Message objects
    memory_key="chat_history",
)

# ── 2. VectorStoreRetrieverMemory: retrieve only relevant memories ─
vectorstore = Chroma(
    collection_name="agent_memory",
    embedding_function=OpenAIEmbeddings(),
    persist_directory="./chroma_memory",
)
vector_memory = VectorStoreRetrieverMemory(
    retriever=vectorstore.as_retriever(search_kwargs={"k": 3}),
    memory_key="relevant_history",
    return_messages=False,
)
vector_memory.save_context({"input": "My name is Alex Kim."}, {"output": "Hello!"})
vector_memory.save_context({"input": "I am a Python developer."}, {"output": "So you are a Python developer!"})

# ── 3. RunnableWithMessageHistory (recommended pattern) ──────────
session_store: dict = {}

def get_session_history(session_id: str) -> ChatMessageHistory:
    """Gets the conversation history for a session ID, creating it if it does not exist."""
    if session_id not in session_store:
        session_store[session_id] = ChatMessageHistory()
    return session_store[session_id]

chat_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a friendly AI assistant. Remember the earlier conversation."),
    MessagesPlaceholder(variable_name="history"),  # where the history is inserted
    ("human", "{input}"),
])

chain_with_history = RunnableWithMessageHistory(
    chat_prompt | model,
    get_session_history,
    input_messages_key="input",
    history_messages_key="history",
)

# ── Example run ──────────────────────────────────────────────────
if __name__ == "__main__":
    config = {"configurable": {"session_id": "user_001"}}

    r1 = chain_with_history.invoke({"input": "Hi! I live in Seattle."}, config=config)
    print("AI:", r1.content)

    # Check that it remembers the earlier context
    r2 = chain_with_history.invoke({"input": "Where did I say I live?"}, config=config)
    print("AI:", r2.content)  # "You said you live in Seattle."

    history = get_session_history("user_001")
    print(f"\nConversation history ({len(history.messages)} messages):")
    for msg in history.messages:
        role = "User" if msg.type == "human" else "AI"
        print(f"  [{role}] {msg.content[:50]}...")

Multi-agent

Here you look at Multi-agent alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

The Supervisor/Worker pattern delegates complex work to specialized agents. The Supervisor analyzes the input and routes it to the right Worker, and each Worker has its own set of tools and a specialist prompt. Used together with LangGraph, state management and flow control become even more powerful.
Rendering diagram…
multi_agent.pyPYTHON
import json
from typing import Literal
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain.agents import create_react_agent, AgentExecutor
from langchain import hub

model = ChatOpenAI(model="gpt-4o", temperature=0)
react_prompt = hub.pull("hwchase17/react")

# ── Worker tool definitions ───────────────────────────────────────
@tool
def analyze_data(data: str) -> str:
    """Analyzes data and provides statistics and insights."""
    return f"Analysis result: found a mean of 85.3 and a standard deviation of 12.1 in {data}."

@tool
def write_report(topic: str, findings: str) -> str:
    """Writes a report based on the analysis results."""
    return f"# {topic} report\n\n## Key findings\n{findings}\n\n## Conclusion\nAnalysis complete."

@tool
def search_web(query: str) -> str:
    """Searches the web for up-to-date information."""
    return f"Search results for '{query}': gathered the latest relevant information."

@tool
def execute_code(code: str) -> str:
    """Runs Python code."""
    import io, contextlib
    buf = io.StringIO()
    with contextlib.redirect_stdout(buf):
        exec(code, {})
    return buf.getvalue() or "Done"

# ── Specialized Worker agents ────────────────────────────────────
def make_executor(tools_list: list) -> AgentExecutor:
    agent = create_react_agent(llm=model, tools=tools_list, prompt=react_prompt)
    return AgentExecutor(
        agent=agent, tools=tools_list,
        verbose=True, max_iterations=5, handle_parsing_errors=True,
    )

data_analyst  = make_executor([analyze_data, execute_code])
report_writer = make_executor([write_report, search_web])

# ── Supervisor: routing logic ────────────────────────────────────
supervisor_prompt = ChatPromptTemplate.from_messages([
    ("system", """You are an AI team manager. Analyze the user request and route it to the right owner.

Owners:
- data_analyst: work that needs data analysis, calculation, or code execution
- report_writer: work that needs report writing or information search
- finish: the work is complete or you can answer directly

Return JSON only: {{\"next\": \"data_analyst\" | \"report_writer\" | \"finish\", \"reason\": \"why\"}}"""),
    ("human", "Task: {task}\nPrevious result: {previous_result}"),
])
supervisor_chain = supervisor_prompt | model | StrOutputParser()

def supervisor_route(task: str) -> Literal["data_analyst", "report_writer", "finish"]:
    result = supervisor_chain.invoke({"task": task, "previous_result": ""})
    try:
        return json.loads(result).get("next", "finish")
    except Exception:
        return "finish"

# ── Multi-agent orchestrator ────────────────────────────────────
def run_multi_agent(task: str) -> str:
    print(f"\n{'='*60}\nTask: {task}\n{'='*60}")
    results = []
    for step in range(5):
        next_agent = supervisor_route(task)
        print(f"\n[Step {step + 1}] Supervisor β†’ {next_agent}")
        if next_agent == "finish":
            break
        elif next_agent == "data_analyst":
            r = data_analyst.invoke({"input": task})
            results.append(f"Data analysis: {r['output']}")
        elif next_agent == "report_writer":
            r = report_writer.invoke({"input": task})
            results.append(f"Report: {r['output']}")
    return "\n".join(results)

if __name__ == "__main__":
    print(run_multi_agent("Analyze last quarter's sales data and write a report for the executive team."))

Streaming

Here you look at Streaming alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

Streaming improves the user experience by cutting the wait until the first token of an LLM response (TTFT). astream() emits tokens in real time as an async generator. Combining FastAPI with SSE (Server-Sent Events) makes streaming easy to apply to a web service.
streaming_server.pyPYTHON
import asyncio
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

# ── 1. Streaming to the terminal ─────────────────────────────────
def streaming_to_terminal():
    """Prints each token to the terminal as soon as it is generated"""
    m = ChatOpenAI(model="gpt-4o-mini", streaming=True,
                   callbacks=[StreamingStdOutCallbackHandler()])
    m.invoke("Explain the future of AI Agents in about 100 words.")

# ── 2. astream() β€” async streaming ──────────────────────────────
async def async_streaming():
    model = ChatOpenAI(model="gpt-4o-mini", temperature=0.7)
    chain = ChatPromptTemplate.from_template("Explain {topic}.") | model | StrOutputParser()
    full = ""
    async for chunk in chain.astream({"topic": "quantum computing"}):
        print(chunk, end="", flush=True)
        full += chunk
    print(f"\n\nGenerated {len(full)} characters in total")

# ── 3. FastAPI + SSE streaming server ────────────────────────────
app = FastAPI(title="LangChain Streaming API")
model = ChatOpenAI(model="gpt-4o-mini", temperature=0.7)
chain = (
    ChatPromptTemplate.from_template("System: expert AI assistant\n\nUser: {question}")
    | model | StrOutputParser()
)

async def token_generator(question: str):
    """Streams tokens in SSE format."""
    try:
        async for chunk in chain.astream({"question": question}):
            yield f"data: {chunk}\n\n"
        yield "data: [DONE]\n\n"
    except Exception as e:
        yield f"data: [ERROR] {e}\n\n"

@app.get("/stream")
async def stream_response(question: str = "What is AI?"):
    """Streams the LLM response over SSE."""
    return StreamingResponse(
        token_generator(question),
        media_type="text/event-stream",
        headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
    )

@app.get("/stream/json")
async def stream_json(question: str = "What is AI?"):
    """Streams in JSON Lines format."""
    import json

    async def gen():
        async for chunk in chain.astream({"question": question}):
            yield json.dumps({"delta": chunk}, ensure_ascii=False) + "\n"

    return StreamingResponse(gen(), media_type="application/x-ndjson")

if __name__ == "__main__":
    import uvicorn
    # asyncio.run(async_streaming())
    uvicorn.run(app, host="0.0.0.0", port=8000)
    # Open: http://localhost:8000/stream?question=Explain RAG

Advanced RAG

Here you look at Advanced RAG alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

Advanced techniques that raise retrieval quality beyond basic RAG. Hybrid Search (BM25 + Vector) combines keyword and semantic search, and the Cohere Reranker reorders results by relevance. HyDE improves embedding quality by generating a hypothetical document, and Multi-query reduces missed documents by asking the question from several angles.
advanced_rag.pyPYTHON
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain_community.vectorstores import Chroma
from langchain_community.retrievers import BM25Retriever
from langchain.retrievers import EnsembleRetriever, MultiQueryRetriever
from langchain.retrievers.document_compressors import CohereRerank
from langchain.retrievers import ContextualCompressionRetriever
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import TextLoader

# ── Load and chunk the documents ─────────────────────────────────
documents = TextLoader("knowledge_base.txt", encoding="utf-8").load()
chunks = RecursiveCharacterTextSplitter(
    chunk_size=500, chunk_overlap=50,
    separators=["\n\n", "\n", ".", " "],
).split_documents(documents)

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
model = ChatOpenAI(model="gpt-4o-mini", temperature=0)

# ── 1. Hybrid Search: BM25 + Vector ensemble ────────────────────
bm25_retriever = BM25Retriever.from_documents(chunks)
bm25_retriever.k = 5

vectorstore = Chroma.from_documents(chunks, embeddings)
vector_retriever = vectorstore.as_retriever(search_kwargs={"k": 5})

# Combine with weights of 40% BM25 + 60% Vector
ensemble_retriever = EnsembleRetriever(
    retrievers=[bm25_retriever, vector_retriever],
    weights=[0.4, 0.6],
)

# ── 2. Cohere Reranker: reorder the search results ──────────────
# pip install cohere
reranking_retriever = ContextualCompressionRetriever(
    base_compressor=CohereRerank(model="rerank-multilingual-v3.0", top_n=3),
    base_retriever=ensemble_retriever,
)

# ── 3. HyDE: hypothetical document embeddings ────────────────────
hyde_chain = (
    ChatPromptTemplate.from_template(
        "Write a hypothetical document that answers the following question:\n\nQuestion: {question}\n\nDocument:"
    )
    | model | StrOutputParser()
    | (lambda text: vectorstore.similarity_search(text, k=4))
)

# ── 4. Multi-query Retriever ─────────────────────────────────────
# The LLM generates 3 different questions from the original to search more widely
multi_query_retriever = MultiQueryRetriever.from_llm(
    retriever=vectorstore.as_retriever(search_kwargs={"k": 4}),
    llm=model,
)

# ── 5. The finished advanced RAG chain ───────────────────────────
rag_prompt = ChatPromptTemplate.from_messages([
    ("system", """You are an AI that provides accurate information.
Answer using only the given context. If something is not in the context, say you do not know.

Context:
{context}"""),
    ("human", "{question}"),
])

def format_docs(docs) -> str:
    return "\n\n---\n\n".join(
        f"[Source: {doc.metadata.get('source', 'unknown')}]\n{doc.page_content}"
        for doc in docs
    )

# The full Hybrid + Rerank + RAG pipeline
advanced_rag_chain = (
    {"context": reranking_retriever | format_docs, "question": RunnablePassthrough()}
    | rag_prompt | model | StrOutputParser()
)

if __name__ == "__main__":
    question = "How do you manage memory in LangChain?"
    print("Answer:", advanced_rag_chain.invoke(question))

    import logging
    logging.getLogger("langchain.retrievers.multi_query").setLevel(logging.INFO)
    docs = multi_query_retriever.invoke(question)
    print(f"\nDocuments retrieved by multi-query: {len(docs)}")

Evaluation & observability

Here you look at Evaluation & observability alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

LangSmith is the official observability platform for LangChain. It traces every chain run so you can debug it. RAGAS is a framework that evaluates RAG pipeline quality automatically, providing key metrics such as faithfulness, answer_relevancy, and context_recall.
eval_observability.pyPYTHON
import os
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import JsonOutputParser

# ── 1. LangSmith tracing setup ───────────────────────────────────
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_API_KEY"] = "ls__your_api_key"
os.environ["LANGCHAIN_PROJECT"] = "production-rag-v2"
# Once set, every chain run is recorded in LangSmith automatically

model = ChatOpenAI(model="gpt-4o-mini", temperature=0)

# ── 2. Evaluate RAG quality with RAGAS ───────────────────────────
# pip install ragas
from ragas import evaluate
from ragas.metrics import (
    faithfulness,      # is the answer faithful to the context (guards against hallucination)
    answer_relevancy,  # is the answer relevant to the question
    context_recall,    # does the context contain the information needed
    context_precision, # does the context contain only relevant information
)
from datasets import Dataset

# Build the evaluation dataset from QA pairs reviewed by domain experts
eval_dataset = Dataset.from_dict({
    "question": [
        "What is LCEL in LangChain?",
        "Explain how a ReAct agent works.",
        "What is the difference between RAG and fine-tuning?",
    ],
    "answer": [
        "LCEL is the LangChain Expression Language, a way of composing components with the | operator.",
        "ReAct is a pattern that combines Reasoning and Acting, repeating Thought→Action→Observation.",
        "RAG answers by retrieving external knowledge, while fine-tuning updates the model weights directly.",
    ],
    "contexts": [
        ["LCEL (LangChain Expression Language) connects LLM components using the pipe operator (|)."],
        ["The ReAct pattern combines Reasoning and Acting so that an agent solves problems step by step."],
        ["RAG is Retrieval-Augmented Generation, which retrieves external documents and adds them to the LLM input."],
    ],
    "ground_truth": [
        "LCEL is the LangChain expression language that composes components with the pipe operator.",
        "In ReAct, the agent reasons and acts in a Thought-Action-Observation loop.",
        "RAG is retrieval-based generation and fine-tuning trains the model parameters.",
    ],
})

results = evaluate(
    dataset=eval_dataset,
    metrics=[faithfulness, answer_relevancy, context_recall, context_precision],
    llm=model,
    embeddings=OpenAIEmbeddings(),
)

print("\n=== RAGAS evaluation results ===")
df = results.to_pandas()
print(df[["question", "faithfulness", "answer_relevancy", "context_recall"]].to_string())
print(f"\nMean Faithfulness:     {df['faithfulness'].mean():.3f}")
print(f"Mean Answer Relevancy: {df['answer_relevancy'].mean():.3f}")
print(f"Mean Context Recall:   {df['context_recall'].mean():.3f}")

# ── 3. A custom LLM-as-Judge evaluation loop ────────────────────
eval_chain = (
    ChatPromptTemplate.from_messages([
        ("system", """You are an evaluator of AI answer quality.
Compare the question, the reference answer, and the AI answer, score it from 0 to 10, and return JSON.
Format: {{\"score\": number, \"reason\": \"why\"}}"""),
        ("human", "Question: {question}\nReference answer: {reference}\nAI answer: {prediction}"),
    ])
    | model | JsonOutputParser()
)

def evaluate_answer(question: str, reference: str, prediction: str) -> dict:
    """Automatically scores answer quality on a 0-10 scale."""
    return eval_chain.invoke({
        "question": question, "reference": reference, "prediction": prediction,
    })

if __name__ == "__main__":
    score = evaluate_answer(
        question="What is LCEL?",
        reference="The LangChain expression language that composes components with the pipe operator",
        prediction="LCEL is the way LangChain builds pipelines.",
    )
    print(f"\nScore: {score['score']}/10")
    print(f"Reason: {score['reason']}")

LangChain practical design

LangChain practical design is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.

With LangChain, rather than chaining things at length, divide the contracts of prompt, retriever, tool, memory, and output parser clearly. An LCEL flow has to be broken into small observable units to be debuggable.
Decision pointQuestion to askPractical guideline
BoundariesWhich parts of the LangChain code are most likely to change?Separate input/output, configuration, external integrations, and the core rules.
StateWhere is state created and where does it go away?Make the owner and lifecycle of state visible in the code.
FailureWhat does the caller receive on failure?Decide the timeout, fallback, and error contract first.

LangChain operating standards

This section covers LangChain operating standards from a practical perspective. Instead of memorizing the concept, focus on the situations in which you would reach for it.

In operation, tool timeouts, retriever latency, LLM fallback, trace sampling, and the token budget matter. Turn on tracing such as LangSmith and track the failure rate per chain.

Tip

  • LCEL unit boundary
  • tool timeout
  • LangSmith trace
  • RAGAS eval set

LangChain verification strategy

LangChain verification strategy is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.

For RAG/Agent quality, evaluate answer relevancy, faithfulness, and tool call correctness separately.
Quality axisHow to verifyDefinition of done
CorrectnessAutomate the normal/failure cases.The core scenarios pass reproducibly.
Regression preventionWhen fixing a bug, leave the same case behind as a test.The same failure is not deployed again.
OperabilityCheck the logs, metrics, and alerts.There is a path for tracing the cause when a problem occurs.

LangChain: frequently asked questions

Do I have to use LangChain?

No. A simple feature that calls a model once or twice is fine with the provider SDK, and the code is simpler. LangChain's common interfaces and integrations help when you switch among models and vector databases, or build flows that combine retrieval, tool calls, and memory.

What is the difference between LangChain and LlamaIndex?

LangChain is a general framework for composing model calls, tools, and memory into many kinds of flows, while LlamaIndex focuses on RAG: indexing documents and retrieving them for answers. When document question answering is the core, LlamaIndex is quicker; when tool calls and multi-step workflows are the core, LangChain often fits better. The two can also be used together.

How are LangChain and LangGraph related?

LangGraph is a separate library from the same team that defines an agent's flow as a stateful graph. Use it for flows that are not a straight line: branching, loops, saving intermediate state, and human approval steps. A chain that runs in a fixed order needs only LangChain; add LangGraph when you build an agent that decides its own next action.

Why does old example code stop working after a LangChain upgrade?

LangChain's package layout and import paths have changed several times. The core interfaces, integrations with external services, and community modules were split into separate packages, so imports from older articles often no longer work. When following an example, check when it was written and which version you have installed, then fix the imports against the current official documentation.

← Previous guideHugging FaceNext guide β†’LlamaIndex