Build advanced RAG pipelines, a ReAct Agent, sub-query decomposition, and hybrid search with LlamaIndex. Covers the whole journey of data-centric AI Agents, including ChromaDB and Pinecone integration, RAGAS evaluation, and production deployment.
Intermediate
Updated Oct 1, 2026
~6 min read
11 sections
8 code examples
Web IDE lab included
LI
LlamaIndex Web IDE
Run code in the browser with nothing to install and learn through step-by-step examples.
Build advanced RAG pipelines, a ReAct Agent, sub-query decomposition, and hybrid search with LlamaIndex. Covers the whole journey of data-centric AI Agents, including ChromaDB and Pinecone integration, RAGAS evaluation, and production deployment. Rather than listing concepts, this guide is organized so you can follow the content in the order you have to make decisions on a real project.
Key perspective
AI / LLM systems
Look beyond the model and the prompt: connect the data flow, evaluation, and post-deployment operating metrics in one view.
To make what you read stick, it helps to capture the flow as a picture first. The two diagrams below are reference maps you can keep coming back to while learning LlamaIndex.
Learning flow
Rendering diagramβ¦
Architecture view
Rendering diagramβ¦
Core architecture
When you first open LlamaIndex, the big picture matters more than individual commands. This section starts with the problems the concepts ahead were created to solve.
LlamaIndex abstracts the whole path of data ingestion β indexing β retrieval β generation. VectorStoreIndex is optimized for embedding-based semantic search and SummaryIndex for summarizing whole documents. Setting the LLM and embedding model globally through the Settings singleton saves repeating parameters in every module.
PYTHON
from llama_index.core import Settingsfrom llama_index.llms.openai import OpenAIfrom llama_index.embeddings.openai import OpenAIEmbedding# Global settings β every module picks them up automaticallySettings.llm = OpenAI(model="gpt-4o", temperature=0.1)Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")Settings.chunk_size = 512 # maximum tokens per nodeSettings.chunk_overlap = 50 # overlap between chunks (keeps context continuous)
Index type
Traits
Good for
VectorStoreIndex
Search by embedding similarity
Semantic search, FAQ, knowledge bases
SummaryIndex
Processes every node in order
Whole-document summaries, long reports
KeywordTableIndex
BM25 keyword matching
Exact term search, legal/medical
KnowledgeGraphIndex
Traverses graph relationships
Entity links, multi-hop questions
Advanced indexing strategies
Here you look at Advanced indexing strategies alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Plain fixed-size chunking cuts sentences apart and lowers retrieval quality. Sentence Window indexes by sentence while bundling the surrounding sentences as a window to keep the context. Hierarchical (Auto-Merging) retrieves leaf nodes and returns their parent nodes, providing both a summary and the detail. In practice, an ensemble of the two strategies gives the highest RAGAS scores.
PYTHON
from llama_index.core.node_parser import ( SentenceWindowNodeParser, HierarchicalNodeParser, get_leaf_nodes,)from llama_index.core import VectorStoreIndex, StorageContextfrom llama_index.core.indices.postprocessor import ( MetadataReplacementPostProcessor, SentenceTransformerRerank,)# --- Sentence Window indexing ---window_parser = SentenceWindowNodeParser.from_defaults( window_size=3, # 3 sentences before and after as the window window_metadata_key="window", original_text_metadata_key="original_text",)nodes = window_parser.get_nodes_from_documents(documents)window_index = VectorStoreIndex(nodes)# Postprocessor that swaps the text for its window at query timewindow_query_engine = window_index.as_query_engine( node_postprocessors=[ MetadataReplacementPostProcessor(target_metadata_key="window"), SentenceTransformerRerank(top_n=3, model="BAAI/bge-reranker-base"), ])
Hybrid search & reranking
Here you look at Hybrid search & reranking alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Semantic (vector) search is good at finding documents with similar meaning but misses exact keywords, and BM25 (keyword) is the opposite. QueryFusionRetriever merges the two result sets with RRF (Reciprocal Rank Fusion) to get the strengths of both. Applying cross-encoder reranking with CohereRerank afterward greatly improves top-k quality.
PYTHON
from llama_index.retrievers.bm25 import BM25Retrieverfrom llama_index.core.retrievers import QueryFusionRetrieverfrom llama_index.postprocessor.cohere_rerank import CohereRerank# Create the vector retriever and the BM25 retrievervector_retriever = index.as_retriever(similarity_top_k=10)bm25_retriever = BM25Retriever.from_defaults(nodes=nodes, similarity_top_k=10)# Fuse the two result sets with RRF + raise recall with 4 query variationshybrid_retriever = QueryFusionRetriever( [vector_retriever, bm25_retriever], similarity_top_k=10, num_queries=4, # search with 4 variations of the query mode="reciprocal_rerank",)# Pick the final top 3 with the Cohere cross-encoderreranker = CohereRerank(api_key="co_...", top_n=3)from llama_index.core.query_engine import RetrieverQueryEnginequery_engine = RetrieverQueryEngine( retriever=hybrid_retriever, node_postprocessors=[reranker],)
ReAct Agent
Here you look at ReAct Agent alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
A ReAct (Reasoning + Acting) Agent solves complex multi-step questions by repeating a think β act β observe loop. Register several knowledge sources as tools with QueryEngineTool and the agent picks the tool that fits the situation. Always set max_iterations to prevent infinite loops.
PYTHON
from llama_index.core.agent import ReActAgentfrom llama_index.core.tools import QueryEngineTool, ToolMetadata# Wrap the knowledge source as a toolknowledge_tool = QueryEngineTool( query_engine=query_engine, metadata=ToolMetadata( name="knowledge_base", description="Searches internal company documents, policies, and procedures. Enter a specific question.", ),)agent = ReActAgent.from_tools( [knowledge_tool], verbose=True, # print the reasoning (essential for debugging) max_iterations=10, # prevent infinite loops)# The agent searches β analyzes β generates the answer by itselfresponse = agent.chat("Analyze the quarterly sales trend and find the outliers")print(response)
Event-driven workflows
Here you look at Event-driven workflows alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
A Workflow models a complex RAG pipeline explicitly as an async event graph. Each step takes an input event type and returns an output event, which expresses parallel execution and conditional branching naturally. It is easier to debug and test than the older chain style, and each step can be observed.
PYTHON
from llama_index.core.workflow import ( Workflow, StartEvent, StopEvent, step, Event)from llama_index.core import VectorStoreIndexclass RetrievalEvent(Event): """Event that passes the retrieval results to the next step""" nodes: list query: strclass RAGWorkflow(Workflow): @step async def retrieve(self, ev: StartEvent) -> RetrievalEvent: """Step 1: retrieve relevant documents for the query""" retriever = index.as_retriever(similarity_top_k=5) nodes = await retriever.aretrieve(ev.query) return RetrievalEvent(nodes=nodes, query=ev.query) @step async def generate(self, ev: RetrievalEvent) -> StopEvent: """Step 2: generate the LLM answer from the retrieved documents""" context = "\n".join([n.get_content() for n in ev.nodes]) response = await Settings.llm.acomplete( f"Context:\n{context}\n\nQuestion: {ev.query}" ) return StopEvent(result=str(response))# Run the workflowwf = RAGWorkflow(timeout=60)result = await wf.run(query="Summarize the key points")
Sub-question decomposition
Here you look at Sub-question decomposition alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
A complex question is hard to answer with a single retrieval. With SubQuestionQueryEngine, the LLM decomposes the question into several sub-questions, queries the right tool (data source) for each in parallel, and merges the results. It is very effective for analytical questions that span several DBs or document groups.
PYTHON
from llama_index.core.query_engine import SubQuestionQueryEnginefrom llama_index.core.tools import QueryEngineTool, ToolMetadata# Register each data source as a tooltools = [ QueryEngineTool( query_engine=sales_engine, metadata=ToolMetadata( name="sales_2024", description="Monthly and quarterly sales data for 2024. Includes amount, quantity, and region.", ), ), QueryEngineTool( query_engine=hr_engine, metadata=ToolMetadata( name="hr_data", description="HR data. Includes hire date, department, job level, and performance reviews.", ), ),]# The LLM generates the sub-questions automatically and queries in parallelengine = SubQuestionQueryEngine.from_defaults( query_engine_tools=tools, verbose=True, # show the generated sub-questions)# Compound question β sub-question decomposition β merged answerresponse = engine.query("How did the new hires of 2024 perform, and how much did they contribute to sales?")print(response)
Vector DB integration
Here you look at Vector DB integration alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
An in-memory index disappears when the server restarts. In production, use ChromaDB (local/self-hosted) or Pinecone (fully managed) as the persistent vector store. StorageContext connects a LlamaIndex index to an external vector DB transparently, and reusing an existing collection cuts the cost of re-indexing.
PYTHON
import chromadbfrom llama_index.vector_stores.chroma import ChromaVectorStorefrom llama_index.core import StorageContext, VectorStoreIndex# Create a persistent ChromaDB clientchroma_client = chromadb.PersistentClient(path="./chroma_db")collection = chroma_client.get_or_create_collection( name="company_docs", metadata={"hnsw:space": "cosine"}, # use cosine similarity)# Wrap it as a LlamaIndex vector storevector_store = ChromaVectorStore(chroma_collection=collection)storage_context = StorageContext.from_defaults(vector_store=vector_store)# First-time indexing (after a restart, load with from_vector_store instead)index = VectorStoreIndex.from_documents( documents, storage_context=storage_context, show_progress=True,)# Reuse the existing collection (no re-indexing needed)index = VectorStoreIndex.from_vector_store(vector_store)
RAGAS evaluation
Here you look at RAGAS evaluation alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
The quality of a RAG system has to be measured in numbers, not by feel. RAGAS quantifies the weak points of a RAG pipeline with three metrics: Faithfulness (hallucination), Answer Relevancy, and Context Recall (how much of the important information was retrieved). Integrate it into CI/CD to detect quality regressions automatically when code changes.
PYTHON
from ragas import evaluatefrom ragas.metrics import faithfulness, answer_relevancy, context_recallfrom datasets import Dataset# Build the evaluation dataset β at least 50 question-answer pairs recommendedeval_dataset = Dataset.from_dict({ "question": questions, # list of test questions "answer": answers, # answers from the RAG system "contexts": contexts, # retrieved contexts (a list of lists) "ground_truth": ground_truths, # reference answers})result = evaluate( eval_dataset, metrics=[ faithfulness, # is the answer grounded in the context (hallucination detection) answer_relevancy, # is the answer relevant to the question context_recall, # does the context contain what the answer needs ],)df = result.to_pandas()print(df[["question", "faithfulness", "answer_relevancy", "context_recall"]])# Aim for faithfulness > 0.9 and answer_relevancy > 0.85
LlamaIndex practical design
LlamaIndex practical design is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.
With LlamaIndex, document chunking, metadata, index type, and retriever strategy determine most of the quality. Separate the ingestion pipeline from the query pipeline so reindexing and tuning are easy.
Decision point
Question to ask
Practical guideline
Boundaries
Which parts of the LlamaIndex code are most likely to change?
Separate input/output, configuration, external integrations, and the core rules.
State
Where is state created and where does it go away?
Make the owner and lifecycle of state visible in the code.
Failure
What does the caller receive on failure?
Decide the timeout, fallback, and error contract first.
LlamaIndex operating standards
This section covers LlamaIndex operating standards from a practical perspective. Instead of memorizing the concept, focus on the situations in which you would reach for it.
In operation, look at embedding cost, the index refresh interval, vector DB latency, top-k tuning, and reranker cost together.
LlamaIndex verification strategy
LlamaIndex verification strategy is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.
Evaluate retrieval quality on context recall/precision and citation accuracy.
Quality axis
How to verify
Definition of done
Correctness
Automate the normal/failure cases.
The core scenarios pass reproducibly.
Regression prevention
When fixing a bug, leave the same case behind as a test.
The same failure is not deployed again.
Operability
Check the logs, metrics, and alerts.
There is a path for tracing the cause when a problem occurs.
LlamaIndex: frequently asked questions
When should I use LlamaIndex?
Use it to build RAG, where an LLM answers by consulting your own data such as internal documents, PDFs, or databases. It provides modules for reading and splitting documents, embedding and indexing them, and retrieving the parts that match a question to put in the prompt.
How do I choose the chunk size for splitting documents?
There is no single answer; it depends on the documents and the kinds of questions. Chunks that are too small cut off the context an answer needs, and chunks that are too large mix in unrelated content and lower retrieval accuracy. Pick a few sizes and overlap values, then compare retrieval hit rates on an evaluation set of question and expected-document pairs.
When an answer is wrong, how do I tell whether retrieval or generation is at fault?
Look at the retrieved chunks first. If the document containing the answer is not among the results, the problem is in indexing or retrieval; if it was retrieved and the answer is still wrong, the problem is the prompt or the model. Evaluating the two stages separately tells you which one to fix.