Build a ReAct Agent, multi-agent systems, and advanced RAG pipelines with LangChain LCEL. Covers the whole journey of AI Agent development, through production serving, LangSmith observability, and RAGAS evaluation.
Intermediate
Updated Oct 1, 2026
~14 min read
12 sections
9 code examples
Web IDE lab included
LC
LangChain Web IDE
Run code in the browser with nothing to install and learn through step-by-step examples.
Build a ReAct Agent, multi-agent systems, and advanced RAG pipelines with LangChain LCEL. Covers the whole journey of AI Agent development, through production serving, LangSmith observability, and RAGAS evaluation. Rather than listing concepts, this guide is organized so you can follow the content in the order you have to make decisions on a real project.
Key perspective
AI / LLM systems
Look beyond the model and the prompt: connect the data flow, evaluation, and post-deployment operating metrics in one view.
ReAct Agent developmentMulti-agent systemsRAG pipelinesDeploying LLM apps to production
Architecture diagrams
To make what you read stick, it helps to capture the flow as a picture first. The two diagrams below are reference maps you can keep coming back to while learning LangChain.
Learning flow
Rendering diagramβ¦
Architecture view
Rendering diagramβ¦
LCEL architecture
When you first open LangChain, the big picture matters more than individual commands. This section starts with the problems the concepts ahead were created to solve.
The LangChain Expression Language (LCEL) composes components with the | pipe operator. RunnableParallel runs in parallel, RunnablePassthrough passes values through transparently, and RunnableLambda inserts arbitrary functions. LCEL chains support streaming, batching, and async alike, which makes them a good fit for production serving.
Rendering diagramβ¦
lcel_arch.pyPYTHON
from operator import itemgetterfrom langchain_core.prompts import ChatPromptTemplatefrom langchain_core.output_parsers import StrOutputParserfrom langchain_core.runnables import RunnableParallel, RunnablePassthrough, RunnableLambdafrom langchain_openai import ChatOpenAImodel = ChatOpenAI(model="gpt-4o-mini", temperature=0)parser = StrOutputParser()# ββ 1. Basic pipe composition ββββββββββββββββββββββββββββββββββββprompt = ChatPromptTemplate.from_template("Explain {topic} in one sentence.")basic_chain = prompt | model | parser# ββ 2. RunnableParallel: run two chains at once ββββββββββββββββββsummary_prompt = ChatPromptTemplate.from_template("Summarize the following in 3 lines: {text}")sentiment_prompt = ChatPromptTemplate.from_template("Analyze the sentiment of the following: {text}")parallel_chain = RunnableParallel( summary=summary_prompt | model | parser, sentiment=sentiment_prompt | model | parser,)# ββ 3. The RunnablePassthrough + assign pattern βββββββββββββββββ# Keep the original input while adding a new keyaugmented_chain = RunnablePassthrough.assign( summary=itemgetter("text") | summary_prompt | model | parser)# ββ 4. RunnableLambda: custom preprocessing βββββββββββββββββββββdef preprocess(inputs: dict) -> dict: """Normalize the input text""" return {"text": inputs["text"].strip().lower()}full_chain = ( RunnableLambda(preprocess) | RunnableParallel( summary=summary_prompt | model | parser, sentiment=sentiment_prompt | model | parser, ))# ββ 5. Run βββββββββββββββββββββββββββββββββββββββββββββββββββββββif __name__ == "__main__": print(basic_chain.invoke({"topic": "LCEL"})) result = parallel_chain.invoke({"text": "LangChain is an AI development framework."}) print("Summary:", result["summary"]) print("Sentiment:", result["sentiment"]) # Batch run (several inputs at once) topics = [{"topic": "RAG"}, {"topic": "Agent"}, {"topic": "Embedding"}] for r in basic_chain.batch(topics): print(r)
Component
Role
RunnableParallel
Runs several chains at once and merges the results into a dict.
RunnablePassthrough
Passes the input on to the next step unchanged.
RunnableLambda
Wraps an ordinary Python function as a Runnable.
itemgetter / assign
Used in the middle of a chain to extract a specific key and add values.
Designing custom tools
Here you look at Designing custom tools alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
The @tool decorator turns an ordinary function into an Agent tool. A StructuredTool, which validates its input with a Pydantic schema, handles complex arguments safely. The tool description (docstring) is what the LLM bases its choice of tool on, so it has to be written clearly.
custom_tools.pyPYTHON
import jsonimport sqlite3from typing import Optionalfrom pydantic import BaseModel, Fieldfrom langchain_core.tools import tool, StructuredTool# ββ 1. The @tool decorator: the simplest way ββββββββββββββββββββ@tooldef web_search(query: str) -> str: """Searches the internet for up-to-date information. Use it when real-time data or recent news is needed.""" # Real implementation: use from tavily import TavilyClient or similar return f"Search results for '{query}': [search API response]"@tooldef python_repl(code: str) -> str: """Runs Python code and returns the result. Use it for calculation, data processing, and checking code.""" import io, contextlib buf = io.StringIO() try: with contextlib.redirect_stdout(buf): exec(code, {}) # use a sandboxed environment in production return buf.getvalue() or "Done (no output)" except Exception as e: return f"Error: {e}"@tooldef read_file(path: str) -> str: """Reads a local file and returns its contents. Use it to analyze documents or check configuration files.""" try: with open(path, "r", encoding="utf-8") as f: return f.read() except FileNotFoundError: return f"File not found: {path}"# ββ 2. StructuredTool: validate input with a Pydantic schema βββββclass DBQueryInput(BaseModel): sql: str = Field(description="The SELECT SQL query to run") db_path: str = Field(default="./data.db", description="Path to the SQLite file") limit: Optional[int] = Field(default=10, description="Maximum number of rows to return")def run_db_query(sql: str, db_path: str = "./data.db", limit: int = 10) -> str: """Runs a SELECT query against a SQLite database.""" if not sql.strip().upper().startswith("SELECT"): return "Error: only SELECT queries are allowed." try: conn = sqlite3.connect(db_path) cursor = conn.execute(sql) rows = cursor.fetchmany(limit) columns = [d[0] for d in cursor.description] conn.close() return json.dumps({"columns": columns, "rows": rows}, ensure_ascii=False) except Exception as e: return f"DB error: {e}"db_query_tool = StructuredTool.from_function( func=run_db_query, name="db_query", description="Looks up information in the database. Use it for statistics and record searches.", args_schema=DBQueryInput,)# ββ 3. List the tools ββββββββββββββββββββββββββββββββββββββββββββtools = [web_search, python_repl, read_file, db_query_tool]for t in tools: print(f"[{t.name}] {t.description[:60]}...")
ReAct agent
Here you look at ReAct agent alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
In the ReAct (Reasoning + Acting) pattern, the model solves complex problems step by step by repeating a Thought β Action β Observation loop. The create_react_agent + AgentExecutor combination is the standard ReAct implementation in LangChain. Monitor the internal reasoning with verbose=True.
Rendering diagramβ¦
react_agent.pyPYTHON
import osfrom langchain_openai import ChatOpenAIfrom langchain_core.tools import toolfrom langchain_core.messages import HumanMessage, AIMessagefrom langchain import hubfrom langchain.agents import create_react_agent, AgentExecutoros.environ["OPENAI_API_KEY"] = "sk-..."# ββ Tool definitions βββββββββββββββββββββββββββββββββββββββββββββ@tooldef calculator(expression: str) -> str: """Performs math calculations. Examples: '2 ** 10', 'sum([1,2,3])'""" try: return str(eval(expression, {"__builtins__": {}}, {})) except Exception as e: return f"Calculation error: {e}"@tooldef web_search(query: str) -> str: """Searches for up-to-date information. Use it when real-time data, weather, or news is needed.""" return f"Search results: the latest information about '{query}'."@tooldef get_weather(city: str) -> str: """Looks up the current weather for a given city.""" return f"Current weather in {city}: sunny, 22Β°C, humidity 55%"tools = [calculator, web_search, get_weather]# ββ Create the ReAct agent βββββββββββββββββββββββββββββββββββββββmodel = ChatOpenAI(model="gpt-4o", temperature=0)react_prompt = hub.pull("hwchase17/react") # enforces the Thought/Action/Observation formatagent = create_react_agent(llm=model, tools=tools, prompt=react_prompt)agent_executor = AgentExecutor( agent=agent, tools=tools, verbose=True, # print the Thought/Action/Observation log max_iterations=10, # prevent infinite loops handle_parsing_errors=True,)# ββ Example run ββββββββββββββββββββββββββββββββββββββββββββββββββif __name__ == "__main__": # Multi-step reasoning: weather lookup + unit conversion result = agent_executor.invoke({ "input": "Tell me the current weather in Seoul and convert the temperature to Fahrenheit." }) print("\nFinal answer:", result["output"]) # Conversational agent (with history) chat_history = [] queries = ["What is 2 to the power of 32?", "Work out how many GB that number is."] for query in queries: result = agent_executor.invoke({"input": query, "chat_history": chat_history}) chat_history.extend([ HumanMessage(content=query), AIMessage(content=result["output"]), ]) print(f"Q: {query}\nA: {result['output']}\n")
Structured output
Here you look at Structured output alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Forcing LLM responses into JSON or a Pydantic model lets downstream code handle them safely with no parsing failures. with_structured_output() uses function calling internally to guarantee the schema. TypedDict is useful when there are many optional fields.
structured_output.pyPYTHON
from typing import Optional, Listfrom typing_extensions import TypedDictfrom pydantic import BaseModel, Fieldfrom langchain_openai import ChatOpenAIfrom langchain_core.prompts import ChatPromptTemplatefrom langchain_core.output_parsers import PydanticOutputParsermodel = ChatOpenAI(model="gpt-4o-mini", temperature=0)# ββ 1. Structured output with a Pydantic model ββββββββββββββββββclass Product(BaseModel): name: str = Field(description="Product name") price: float = Field(description="Price (USD)") category: str = Field(description="Category") in_stock: bool = Field(description="Whether it is in stock") tags: List[str] = Field(default_factory=list, description="List of tags") description: Optional[str] = Field(default=None, description="Product description")structured_model = model.with_structured_output(Product)prompt = ChatPromptTemplate.from_template("Extract the product information from the following text:\n\n{text}")extraction_chain = prompt | structured_modelproduct = extraction_chain.invoke({ "text": "New wireless earbuds AX-100, priced at 89 USD, in stock. Noise cancelling, Bluetooth"})print(f"Product: {product.name}, price: {product.price:,} USD")print(f"Tags: {product.tags}")# ββ 2. TypedDict schema ββββββββββββββββββββββββββββββββββββββββββclass AnalysisResult(TypedDict): sentiment: str # positive / negative / neutral score: float # 0.0 ~ 1.0 keywords: List[str] # key keywords summary: str # summarytyped_model = model.with_structured_output(AnalysisResult)analysis_chain = ChatPromptTemplate.from_template("Analyze the following review:\n{review}") | typed_modelresult = analysis_chain.invoke({"review": "Shipping was fast and the product quality is good. Great value for the price!"})print(f"Sentiment: {result['sentiment']}, score: {result['score']}")# ββ 3. PydanticOutputParser (for models without function calling) ββclass ResearchReport(BaseModel): title: str key_findings: List[str] conclusion: str confidence: float = Field(ge=0, le=1)pydantic_parser = PydanticOutputParser(pydantic_object=ResearchReport)format_instructions = pydantic_parser.get_format_instructions()report_prompt = ChatPromptTemplate.from_messages([ ("system", "You are an expert at writing research reports.\n{format_instructions}"), ("user", "Write a research report on {topic}."),])report = (report_prompt | model | pydantic_parser).invoke({ "topic": "the state of enterprise adoption of large language models (LLMs)", "format_instructions": format_instructions,})print(f"Report: {report.title}")print(f"Key findings: {report.key_findings}")
Memory systems
Here you look at Memory systems alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
How conversation history is managed decides how well an agent understands context. Use ConversationBufferWindowMemory for short conversations and VectorStoreRetrieverMemory when long-term memory is needed. The current LCEL approach injects history into the chain with RunnableWithMessageHistory.
memory_system.pyPYTHON
from langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholderfrom langchain_core.runnables.history import RunnableWithMessageHistoryfrom langchain_community.chat_message_histories import ChatMessageHistoryfrom langchain.memory import ConversationBufferWindowMemory, VectorStoreRetrieverMemoryfrom langchain_community.vectorstores import Chromamodel = ChatOpenAI(model="gpt-4o-mini", temperature=0.7)# ββ 1. ConversationBufferWindowMemory: keep only the last N turns βwindow_memory = ConversationBufferWindowMemory( k=5, # keep only the last 5 turns return_messages=True, # return Message objects memory_key="chat_history",)# ββ 2. VectorStoreRetrieverMemory: retrieve only relevant memories βvectorstore = Chroma( collection_name="agent_memory", embedding_function=OpenAIEmbeddings(), persist_directory="./chroma_memory",)vector_memory = VectorStoreRetrieverMemory( retriever=vectorstore.as_retriever(search_kwargs={"k": 3}), memory_key="relevant_history", return_messages=False,)vector_memory.save_context({"input": "My name is Alex Kim."}, {"output": "Hello!"})vector_memory.save_context({"input": "I am a Python developer."}, {"output": "So you are a Python developer!"})# ββ 3. RunnableWithMessageHistory (recommended pattern) ββββββββββsession_store: dict = {}def get_session_history(session_id: str) -> ChatMessageHistory: """Gets the conversation history for a session ID, creating it if it does not exist.""" if session_id not in session_store: session_store[session_id] = ChatMessageHistory() return session_store[session_id]chat_prompt = ChatPromptTemplate.from_messages([ ("system", "You are a friendly AI assistant. Remember the earlier conversation."), MessagesPlaceholder(variable_name="history"), # where the history is inserted ("human", "{input}"),])chain_with_history = RunnableWithMessageHistory( chat_prompt | model, get_session_history, input_messages_key="input", history_messages_key="history",)# ββ Example run ββββββββββββββββββββββββββββββββββββββββββββββββββif __name__ == "__main__": config = {"configurable": {"session_id": "user_001"}} r1 = chain_with_history.invoke({"input": "Hi! I live in Seattle."}, config=config) print("AI:", r1.content) # Check that it remembers the earlier context r2 = chain_with_history.invoke({"input": "Where did I say I live?"}, config=config) print("AI:", r2.content) # "You said you live in Seattle." history = get_session_history("user_001") print(f"\nConversation history ({len(history.messages)} messages):") for msg in history.messages: role = "User" if msg.type == "human" else "AI" print(f" [{role}] {msg.content[:50]}...")
Multi-agent
Here you look at Multi-agent alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
The Supervisor/Worker pattern delegates complex work to specialized agents. The Supervisor analyzes the input and routes it to the right Worker, and each Worker has its own set of tools and a specialist prompt. Used together with LangGraph, state management and flow control become even more powerful.
Rendering diagramβ¦
multi_agent.pyPYTHON
import jsonfrom typing import Literalfrom langchain_openai import ChatOpenAIfrom langchain_core.tools import toolfrom langchain_core.prompts import ChatPromptTemplatefrom langchain_core.output_parsers import StrOutputParserfrom langchain.agents import create_react_agent, AgentExecutorfrom langchain import hubmodel = ChatOpenAI(model="gpt-4o", temperature=0)react_prompt = hub.pull("hwchase17/react")# ββ Worker tool definitions βββββββββββββββββββββββββββββββββββββββ@tooldef analyze_data(data: str) -> str: """Analyzes data and provides statistics and insights.""" return f"Analysis result: found a mean of 85.3 and a standard deviation of 12.1 in {data}."@tooldef write_report(topic: str, findings: str) -> str: """Writes a report based on the analysis results.""" return f"# {topic} report\n\n## Key findings\n{findings}\n\n## Conclusion\nAnalysis complete."@tooldef search_web(query: str) -> str: """Searches the web for up-to-date information.""" return f"Search results for '{query}': gathered the latest relevant information."@tooldef execute_code(code: str) -> str: """Runs Python code.""" import io, contextlib buf = io.StringIO() with contextlib.redirect_stdout(buf): exec(code, {}) return buf.getvalue() or "Done"# ββ Specialized Worker agents ββββββββββββββββββββββββββββββββββββdef make_executor(tools_list: list) -> AgentExecutor: agent = create_react_agent(llm=model, tools=tools_list, prompt=react_prompt) return AgentExecutor( agent=agent, tools=tools_list, verbose=True, max_iterations=5, handle_parsing_errors=True, )data_analyst = make_executor([analyze_data, execute_code])report_writer = make_executor([write_report, search_web])# ββ Supervisor: routing logic ββββββββββββββββββββββββββββββββββββsupervisor_prompt = ChatPromptTemplate.from_messages([ ("system", """You are an AI team manager. Analyze the user request and route it to the right owner.Owners:- data_analyst: work that needs data analysis, calculation, or code execution- report_writer: work that needs report writing or information search- finish: the work is complete or you can answer directlyReturn JSON only: {{\"next\": \"data_analyst\" | \"report_writer\" | \"finish\", \"reason\": \"why\"}}"""), ("human", "Task: {task}\nPrevious result: {previous_result}"),])supervisor_chain = supervisor_prompt | model | StrOutputParser()def supervisor_route(task: str) -> Literal["data_analyst", "report_writer", "finish"]: result = supervisor_chain.invoke({"task": task, "previous_result": ""}) try: return json.loads(result).get("next", "finish") except Exception: return "finish"# ββ Multi-agent orchestrator ββββββββββββββββββββββββββββββββββββdef run_multi_agent(task: str) -> str: print(f"\n{'='*60}\nTask: {task}\n{'='*60}") results = [] for step in range(5): next_agent = supervisor_route(task) print(f"\n[Step {step + 1}] Supervisor β {next_agent}") if next_agent == "finish": break elif next_agent == "data_analyst": r = data_analyst.invoke({"input": task}) results.append(f"Data analysis: {r['output']}") elif next_agent == "report_writer": r = report_writer.invoke({"input": task}) results.append(f"Report: {r['output']}") return "\n".join(results)if __name__ == "__main__": print(run_multi_agent("Analyze last quarter's sales data and write a report for the executive team."))
Streaming
Here you look at Streaming alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Streaming improves the user experience by cutting the wait until the first token of an LLM response (TTFT). astream() emits tokens in real time as an async generator. Combining FastAPI with SSE (Server-Sent Events) makes streaming easy to apply to a web service.
streaming_server.pyPYTHON
import asynciofrom fastapi import FastAPIfrom fastapi.responses import StreamingResponsefrom langchain_openai import ChatOpenAIfrom langchain_core.prompts import ChatPromptTemplatefrom langchain_core.output_parsers import StrOutputParserfrom langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler# ββ 1. Streaming to the terminal βββββββββββββββββββββββββββββββββdef streaming_to_terminal(): """Prints each token to the terminal as soon as it is generated""" m = ChatOpenAI(model="gpt-4o-mini", streaming=True, callbacks=[StreamingStdOutCallbackHandler()]) m.invoke("Explain the future of AI Agents in about 100 words.")# ββ 2. astream() β async streaming ββββββββββββββββββββββββββββββasync def async_streaming(): model = ChatOpenAI(model="gpt-4o-mini", temperature=0.7) chain = ChatPromptTemplate.from_template("Explain {topic}.") | model | StrOutputParser() full = "" async for chunk in chain.astream({"topic": "quantum computing"}): print(chunk, end="", flush=True) full += chunk print(f"\n\nGenerated {len(full)} characters in total")# ββ 3. FastAPI + SSE streaming server ββββββββββββββββββββββββββββapp = FastAPI(title="LangChain Streaming API")model = ChatOpenAI(model="gpt-4o-mini", temperature=0.7)chain = ( ChatPromptTemplate.from_template("System: expert AI assistant\n\nUser: {question}") | model | StrOutputParser())async def token_generator(question: str): """Streams tokens in SSE format.""" try: async for chunk in chain.astream({"question": question}): yield f"data: {chunk}\n\n" yield "data: [DONE]\n\n" except Exception as e: yield f"data: [ERROR] {e}\n\n"@app.get("/stream")async def stream_response(question: str = "What is AI?"): """Streams the LLM response over SSE.""" return StreamingResponse( token_generator(question), media_type="text/event-stream", headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"}, )@app.get("/stream/json")async def stream_json(question: str = "What is AI?"): """Streams in JSON Lines format.""" import json async def gen(): async for chunk in chain.astream({"question": question}): yield json.dumps({"delta": chunk}, ensure_ascii=False) + "\n" return StreamingResponse(gen(), media_type="application/x-ndjson")if __name__ == "__main__": import uvicorn # asyncio.run(async_streaming()) uvicorn.run(app, host="0.0.0.0", port=8000) # Open: http://localhost:8000/stream?question=Explain RAG
Advanced RAG
Here you look at Advanced RAG alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Advanced techniques that raise retrieval quality beyond basic RAG. Hybrid Search (BM25 + Vector) combines keyword and semantic search, and the Cohere Reranker reorders results by relevance. HyDE improves embedding quality by generating a hypothetical document, and Multi-query reduces missed documents by asking the question from several angles.
advanced_rag.pyPYTHON
from langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom langchain_core.prompts import ChatPromptTemplatefrom langchain_core.output_parsers import StrOutputParserfrom langchain_core.runnables import RunnablePassthroughfrom langchain_community.vectorstores import Chromafrom langchain_community.retrievers import BM25Retrieverfrom langchain.retrievers import EnsembleRetriever, MultiQueryRetrieverfrom langchain.retrievers.document_compressors import CohereRerankfrom langchain.retrievers import ContextualCompressionRetrieverfrom langchain.text_splitter import RecursiveCharacterTextSplitterfrom langchain_community.document_loaders import TextLoader# ββ Load and chunk the documents βββββββββββββββββββββββββββββββββdocuments = TextLoader("knowledge_base.txt", encoding="utf-8").load()chunks = RecursiveCharacterTextSplitter( chunk_size=500, chunk_overlap=50, separators=["\n\n", "\n", ".", " "],).split_documents(documents)embeddings = OpenAIEmbeddings(model="text-embedding-3-small")model = ChatOpenAI(model="gpt-4o-mini", temperature=0)# ββ 1. Hybrid Search: BM25 + Vector ensemble ββββββββββββββββββββbm25_retriever = BM25Retriever.from_documents(chunks)bm25_retriever.k = 5vectorstore = Chroma.from_documents(chunks, embeddings)vector_retriever = vectorstore.as_retriever(search_kwargs={"k": 5})# Combine with weights of 40% BM25 + 60% Vectorensemble_retriever = EnsembleRetriever( retrievers=[bm25_retriever, vector_retriever], weights=[0.4, 0.6],)# ββ 2. Cohere Reranker: reorder the search results ββββββββββββββ# pip install coherereranking_retriever = ContextualCompressionRetriever( base_compressor=CohereRerank(model="rerank-multilingual-v3.0", top_n=3), base_retriever=ensemble_retriever,)# ββ 3. HyDE: hypothetical document embeddings ββββββββββββββββββββhyde_chain = ( ChatPromptTemplate.from_template( "Write a hypothetical document that answers the following question:\n\nQuestion: {question}\n\nDocument:" ) | model | StrOutputParser() | (lambda text: vectorstore.similarity_search(text, k=4)))# ββ 4. Multi-query Retriever βββββββββββββββββββββββββββββββββββββ# The LLM generates 3 different questions from the original to search more widelymulti_query_retriever = MultiQueryRetriever.from_llm( retriever=vectorstore.as_retriever(search_kwargs={"k": 4}), llm=model,)# ββ 5. The finished advanced RAG chain βββββββββββββββββββββββββββrag_prompt = ChatPromptTemplate.from_messages([ ("system", """You are an AI that provides accurate information.Answer using only the given context. If something is not in the context, say you do not know.Context:{context}"""), ("human", "{question}"),])def format_docs(docs) -> str: return "\n\n---\n\n".join( f"[Source: {doc.metadata.get('source', 'unknown')}]\n{doc.page_content}" for doc in docs )# The full Hybrid + Rerank + RAG pipelineadvanced_rag_chain = ( {"context": reranking_retriever | format_docs, "question": RunnablePassthrough()} | rag_prompt | model | StrOutputParser())if __name__ == "__main__": question = "How do you manage memory in LangChain?" print("Answer:", advanced_rag_chain.invoke(question)) import logging logging.getLogger("langchain.retrievers.multi_query").setLevel(logging.INFO) docs = multi_query_retriever.invoke(question) print(f"\nDocuments retrieved by multi-query: {len(docs)}")
Evaluation & observability
Here you look at Evaluation & observability alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
LangSmith is the official observability platform for LangChain. It traces every chain run so you can debug it. RAGAS is a framework that evaluates RAG pipeline quality automatically, providing key metrics such as faithfulness, answer_relevancy, and context_recall.
eval_observability.pyPYTHON
import osfrom langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom langchain_core.prompts import ChatPromptTemplatefrom langchain_core.output_parsers import JsonOutputParser# ββ 1. LangSmith tracing setup βββββββββββββββββββββββββββββββββββos.environ["LANGCHAIN_TRACING_V2"] = "true"os.environ["LANGCHAIN_API_KEY"] = "ls__your_api_key"os.environ["LANGCHAIN_PROJECT"] = "production-rag-v2"# Once set, every chain run is recorded in LangSmith automaticallymodel = ChatOpenAI(model="gpt-4o-mini", temperature=0)# ββ 2. Evaluate RAG quality with RAGAS βββββββββββββββββββββββββββ# pip install ragasfrom ragas import evaluatefrom ragas.metrics import ( faithfulness, # is the answer faithful to the context (guards against hallucination) answer_relevancy, # is the answer relevant to the question context_recall, # does the context contain the information needed context_precision, # does the context contain only relevant information)from datasets import Dataset# Build the evaluation dataset from QA pairs reviewed by domain expertseval_dataset = Dataset.from_dict({ "question": [ "What is LCEL in LangChain?", "Explain how a ReAct agent works.", "What is the difference between RAG and fine-tuning?", ], "answer": [ "LCEL is the LangChain Expression Language, a way of composing components with the | operator.", "ReAct is a pattern that combines Reasoning and Acting, repeating ThoughtβActionβObservation.", "RAG answers by retrieving external knowledge, while fine-tuning updates the model weights directly.", ], "contexts": [ ["LCEL (LangChain Expression Language) connects LLM components using the pipe operator (|)."], ["The ReAct pattern combines Reasoning and Acting so that an agent solves problems step by step."], ["RAG is Retrieval-Augmented Generation, which retrieves external documents and adds them to the LLM input."], ], "ground_truth": [ "LCEL is the LangChain expression language that composes components with the pipe operator.", "In ReAct, the agent reasons and acts in a Thought-Action-Observation loop.", "RAG is retrieval-based generation and fine-tuning trains the model parameters.", ],})results = evaluate( dataset=eval_dataset, metrics=[faithfulness, answer_relevancy, context_recall, context_precision], llm=model, embeddings=OpenAIEmbeddings(),)print("\n=== RAGAS evaluation results ===")df = results.to_pandas()print(df[["question", "faithfulness", "answer_relevancy", "context_recall"]].to_string())print(f"\nMean Faithfulness: {df['faithfulness'].mean():.3f}")print(f"Mean Answer Relevancy: {df['answer_relevancy'].mean():.3f}")print(f"Mean Context Recall: {df['context_recall'].mean():.3f}")# ββ 3. A custom LLM-as-Judge evaluation loop ββββββββββββββββββββeval_chain = ( ChatPromptTemplate.from_messages([ ("system", """You are an evaluator of AI answer quality.Compare the question, the reference answer, and the AI answer, score it from 0 to 10, and return JSON.Format: {{\"score\": number, \"reason\": \"why\"}}"""), ("human", "Question: {question}\nReference answer: {reference}\nAI answer: {prediction}"), ]) | model | JsonOutputParser())def evaluate_answer(question: str, reference: str, prediction: str) -> dict: """Automatically scores answer quality on a 0-10 scale.""" return eval_chain.invoke({ "question": question, "reference": reference, "prediction": prediction, })if __name__ == "__main__": score = evaluate_answer( question="What is LCEL?", reference="The LangChain expression language that composes components with the pipe operator", prediction="LCEL is the way LangChain builds pipelines.", ) print(f"\nScore: {score['score']}/10") print(f"Reason: {score['reason']}")
LangChain practical design
LangChain practical design is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.
With LangChain, rather than chaining things at length, divide the contracts of prompt, retriever, tool, memory, and output parser clearly. An LCEL flow has to be broken into small observable units to be debuggable.
Decision point
Question to ask
Practical guideline
Boundaries
Which parts of the LangChain code are most likely to change?
Separate input/output, configuration, external integrations, and the core rules.
State
Where is state created and where does it go away?
Make the owner and lifecycle of state visible in the code.
Failure
What does the caller receive on failure?
Decide the timeout, fallback, and error contract first.
LangChain operating standards
This section covers LangChain operating standards from a practical perspective. Instead of memorizing the concept, focus on the situations in which you would reach for it.
In operation, tool timeouts, retriever latency, LLM fallback, trace sampling, and the token budget matter. Turn on tracing such as LangSmith and track the failure rate per chain.
LangChain verification strategy
LangChain verification strategy is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.
For RAG/Agent quality, evaluate answer relevancy, faithfulness, and tool call correctness separately.
Quality axis
How to verify
Definition of done
Correctness
Automate the normal/failure cases.
The core scenarios pass reproducibly.
Regression prevention
When fixing a bug, leave the same case behind as a test.
The same failure is not deployed again.
Operability
Check the logs, metrics, and alerts.
There is a path for tracing the cause when a problem occurs.
LangChain: frequently asked questions
Do I have to use LangChain?
No. A simple feature that calls a model once or twice is fine with the provider SDK, and the code is simpler. LangChain's common interfaces and integrations help when you switch among models and vector databases, or build flows that combine retrieval, tool calls, and memory.
What is the difference between LangChain and LlamaIndex?
LangChain is a general framework for composing model calls, tools, and memory into many kinds of flows, while LlamaIndex focuses on RAG: indexing documents and retrieving them for answers. When document question answering is the core, LlamaIndex is quicker; when tool calls and multi-step workflows are the core, LangChain often fits better. The two can also be used together.
How are LangChain and LangGraph related?
LangGraph is a separate library from the same team that defines an agent's flow as a stateful graph. Use it for flows that are not a straight line: branching, loops, saving intermediate state, and human approval steps. A chain that runs in a fixed order needs only LangChain; add LangGraph when you build an agent that decides its own next action.
Why does old example code stop working after a LangChain upgrade?
LangChain's package layout and import paths have changed several times. The core interfaces, integrations with external services, and community modules were split into separate packages, so imports from older articles often no longer work. When following an example, check when it was written and which version you have installed, then fix the imports against the current official documentation.