Skip to main content
AIDevOps
  • Learn
  • Learning Paths
  • Practice
  • Open Source
  • Books
  • Engineering

    AI DevOpsThe full map of building and operating AI servicesLLMOpsLLM deployment Β· evaluation Β· observabilityHands-on ProjectsBuild AI Agent projects

    Knowledge

    DocsTechnical documentationBlogEngineering articlesPloggerDevelopment log feed

    Validate

    Certification3-level skills certification Β· coming soon
AI Models
LlamaTutorial|MistralTutorial|GemmaTutorial|DeepSeekTutorial|QwenTutorial
πŸ€– AI Applied Development
Applied AI Intro & RoadmapHugging FaceLangChainLlamaIndexLLMOps|LangGraphMCPMulti-AgentAgent Evaluation
🧠 AI Core
AI Intro & RoadmapML FundamentalsLLM Fundamentals|Python AIC++|PyTorchTensorFlowJAX
🧠 AI Agent Development
Finance AI AgentLLM API ServerStock Investing AgentAIOps AI AgentEducation AI AgentCoding AI Agent
🌱 Spring Cloud
Spring Intro & RoadmapSpring Cloud GatewaySpring BootJava|Spring AISpring SecuritySpring BatchSpring JPA
🐳 DevOps
DevOps Intro & RoadmapLinuxDockerCI/CD|Kubernetes BasicsK8s AdvancedPrometheusGrafana
🧱 Infrastructure
Infrastructure Intro & RoadmapNginxRedis
☁️ Cloud
Cloud Intro & RoadmapAWSGCPAzureNCPCloudflare
🎨 Frontend
Frontend Intro & RoadmapJavaScriptTypeScript|ReactNext.js|VueNuxt|Electron
πŸ“± Mobile
Mobile Intro & RoadmapKotlinAndroidFlutter
βš™οΈ Backend
Backend Intro & RoadmapPython BasicsFastAPIDjangoFlask|CGoGinNode.js
πŸ’Ύ Database
DB Intro & RoadmapCore SQLOracleMySQLPostgreSQL|MongoDBVector DB
πŸ§ͺ Testing
k6JMeternGrinder
AIDevOps

Engineering AI. From Code to Production.
An engineering learning platform for building and operating AI and AI Agents

Learn

  • All Guides
  • Learning Paths
  • AI Tutorial
  • Practice
  • Books

Resources

  • AI DevOps
  • LLMOps
  • Hands-on Projects
  • Docs
  • Blog
  • Plogger
  • Open Source
  • Certification (coming soon)

Start Here

  • AI Core Roadmap
  • AI Applied Development Roadmap
  • Spring Cloud Roadmap
  • DevOps Roadmap
  • Infrastructure Roadmap

Β 

  • Cloud Roadmap
  • Frontend Roadmap
  • Mobile Roadmap
  • Backend Roadmap
  • Database Roadmap
Β© 2026 AIDevOps. All rights reserved.
Terms of ServicePrivacy PolicySitemaptestforge.kr
  1. Home
  2. Learn
  3. AI Applied Development
  4. LlamaIndex
Guide to data pipelines for RAG & AI Agents

LI LlamaIndex Complete Guide

Visitors

Build advanced RAG pipelines, a ReAct Agent, sub-query decomposition, and hybrid search with LlamaIndex. Covers the whole journey of data-centric AI Agents, including ChromaDB and Pinecone integration, RAGAS evaluation, and production deployment.

  • Intermediate
  • Updated Oct 1, 2026
  • ~6 min read
  • 11 sections
  • 8 code examples
  • Web IDE lab included
LI

LlamaIndex Web IDE

Run code in the browser with nothing to install and learn through step-by-step examples.

Open the Web IDE β†’
Advanced RAG systemsReAct AgentSub-query decompositionMultiple vector DBs

Related frameworks & environments

🐍Python AIβ†’HFHugging Faceβ†’LCLangChainβ†’

Contents

0 / 14
  1. How to use this guide
  2. Architecture diagrams
  3. Core architecture
  4. Advanced indexing strategies
  5. Hybrid search & reranking
  6. ReAct Agent
  7. Workflows
  8. Sub-question decomposition
  9. Vector DB integration
  10. RAGAS evaluation
  11. LlamaIndex design
  12. Operating standards
  13. Verification strategy
  14. Frequently asked questions
Contents 14 sections
  1. How to use this guide
  2. Architecture diagrams
  3. Core architecture
  4. Advanced indexing strategies
  5. Hybrid search & reranking
  6. ReAct Agent
  7. Workflows
  8. Sub-question decomposition
  9. Vector DB integration
  10. RAGAS evaluation
  11. LlamaIndex design
  12. Operating standards
  13. Verification strategy
  14. Frequently asked questions

How to use this guide

How to read it

Understanding LlamaIndex as a real-world workflow

Build advanced RAG pipelines, a ReAct Agent, sub-query decomposition, and hybrid search with LlamaIndex. Covers the whole journey of data-centric AI Agents, including ChromaDB and Pinecone integration, RAGAS evaluation, and production deployment. Rather than listing concepts, this guide is organized so you can follow the content in the order you have to make decisions on a real project.

Key perspective

AI / LLM systems

Look beyond the model and the prompt: connect the data flow, evaluation, and post-deployment operating metrics in one view.

Advanced RAG systemsReAct AgentSub-query decompositionMultiple vector DBs

Architecture diagrams

To make what you read stick, it helps to capture the flow as a picture first. The two diagrams below are reference maps you can keep coming back to while learning LlamaIndex.

Learning flow

Rendering diagram…

Architecture view

Rendering diagram…

Core architecture

When you first open LlamaIndex, the big picture matters more than individual commands. This section starts with the problems the concepts ahead were created to solve.

LlamaIndex abstracts the whole path of data ingestion β†’ indexing β†’ retrieval β†’ generation. VectorStoreIndex is optimized for embedding-based semantic search and SummaryIndex for summarizing whole documents. Setting the LLM and embedding model globally through the Settings singleton saves repeating parameters in every module.
PYTHON
from llama_index.core import Settings
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding

# Global settings β€” every module picks them up automatically
Settings.llm = OpenAI(model="gpt-4o", temperature=0.1)
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")
Settings.chunk_size = 512       # maximum tokens per node
Settings.chunk_overlap = 50     # overlap between chunks (keeps context continuous)
Index typeTraitsGood for
VectorStoreIndexSearch by embedding similaritySemantic search, FAQ, knowledge bases
SummaryIndexProcesses every node in orderWhole-document summaries, long reports
KeywordTableIndexBM25 keyword matchingExact term search, legal/medical
KnowledgeGraphIndexTraverses graph relationshipsEntity links, multi-hop questions

Advanced indexing strategies

Here you look at Advanced indexing strategies alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

Plain fixed-size chunking cuts sentences apart and lowers retrieval quality. Sentence Window indexes by sentence while bundling the surrounding sentences as a window to keep the context. Hierarchical (Auto-Merging) retrieves leaf nodes and returns their parent nodes, providing both a summary and the detail. In practice, an ensemble of the two strategies gives the highest RAGAS scores.
PYTHON
from llama_index.core.node_parser import (
    SentenceWindowNodeParser,
    HierarchicalNodeParser,
    get_leaf_nodes,
)
from llama_index.core import VectorStoreIndex, StorageContext
from llama_index.core.indices.postprocessor import (
    MetadataReplacementPostProcessor,
    SentenceTransformerRerank,
)

# --- Sentence Window indexing ---
window_parser = SentenceWindowNodeParser.from_defaults(
    window_size=3,                              # 3 sentences before and after as the window
    window_metadata_key="window",
    original_text_metadata_key="original_text",
)
nodes = window_parser.get_nodes_from_documents(documents)
window_index = VectorStoreIndex(nodes)

# Postprocessor that swaps the text for its window at query time
window_query_engine = window_index.as_query_engine(
    node_postprocessors=[
        MetadataReplacementPostProcessor(target_metadata_key="window"),
        SentenceTransformerRerank(top_n=3, model="BAAI/bge-reranker-base"),
    ]
)

Hybrid search & reranking

Here you look at Hybrid search & reranking alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

Semantic (vector) search is good at finding documents with similar meaning but misses exact keywords, and BM25 (keyword) is the opposite. QueryFusionRetriever merges the two result sets with RRF (Reciprocal Rank Fusion) to get the strengths of both. Applying cross-encoder reranking with CohereRerank afterward greatly improves top-k quality.
PYTHON
from llama_index.retrievers.bm25 import BM25Retriever
from llama_index.core.retrievers import QueryFusionRetriever
from llama_index.postprocessor.cohere_rerank import CohereRerank

# Create the vector retriever and the BM25 retriever
vector_retriever = index.as_retriever(similarity_top_k=10)
bm25_retriever   = BM25Retriever.from_defaults(nodes=nodes, similarity_top_k=10)

# Fuse the two result sets with RRF + raise recall with 4 query variations
hybrid_retriever = QueryFusionRetriever(
    [vector_retriever, bm25_retriever],
    similarity_top_k=10,
    num_queries=4,          # search with 4 variations of the query
    mode="reciprocal_rerank",
)

# Pick the final top 3 with the Cohere cross-encoder
reranker = CohereRerank(api_key="co_...", top_n=3)

from llama_index.core.query_engine import RetrieverQueryEngine
query_engine = RetrieverQueryEngine(
    retriever=hybrid_retriever,
    node_postprocessors=[reranker],
)

ReAct Agent

Here you look at ReAct Agent alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

A ReAct (Reasoning + Acting) Agent solves complex multi-step questions by repeating a think β†’ act β†’ observe loop. Register several knowledge sources as tools with QueryEngineTool and the agent picks the tool that fits the situation. Always set max_iterations to prevent infinite loops.
PYTHON
from llama_index.core.agent import ReActAgent
from llama_index.core.tools import QueryEngineTool, ToolMetadata

# Wrap the knowledge source as a tool
knowledge_tool = QueryEngineTool(
    query_engine=query_engine,
    metadata=ToolMetadata(
        name="knowledge_base",
        description="Searches internal company documents, policies, and procedures. Enter a specific question.",
    ),
)

agent = ReActAgent.from_tools(
    [knowledge_tool],
    verbose=True,       # print the reasoning (essential for debugging)
    max_iterations=10,  # prevent infinite loops
)

# The agent searches β†’ analyzes β†’ generates the answer by itself
response = agent.chat("Analyze the quarterly sales trend and find the outliers")
print(response)

Event-driven workflows

Here you look at Event-driven workflows alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

A Workflow models a complex RAG pipeline explicitly as an async event graph. Each step takes an input event type and returns an output event, which expresses parallel execution and conditional branching naturally. It is easier to debug and test than the older chain style, and each step can be observed.
PYTHON
from llama_index.core.workflow import (
    Workflow, StartEvent, StopEvent, step, Event
)
from llama_index.core import VectorStoreIndex

class RetrievalEvent(Event):
    """Event that passes the retrieval results to the next step"""
    nodes: list
    query: str

class RAGWorkflow(Workflow):
    @step
    async def retrieve(self, ev: StartEvent) -> RetrievalEvent:
        """Step 1: retrieve relevant documents for the query"""
        retriever = index.as_retriever(similarity_top_k=5)
        nodes = await retriever.aretrieve(ev.query)
        return RetrievalEvent(nodes=nodes, query=ev.query)

    @step
    async def generate(self, ev: RetrievalEvent) -> StopEvent:
        """Step 2: generate the LLM answer from the retrieved documents"""
        context = "\n".join([n.get_content() for n in ev.nodes])
        response = await Settings.llm.acomplete(
            f"Context:\n{context}\n\nQuestion: {ev.query}"
        )
        return StopEvent(result=str(response))

# Run the workflow
wf = RAGWorkflow(timeout=60)
result = await wf.run(query="Summarize the key points")

Sub-question decomposition

Here you look at Sub-question decomposition alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

A complex question is hard to answer with a single retrieval. With SubQuestionQueryEngine, the LLM decomposes the question into several sub-questions, queries the right tool (data source) for each in parallel, and merges the results. It is very effective for analytical questions that span several DBs or document groups.
PYTHON
from llama_index.core.query_engine import SubQuestionQueryEngine
from llama_index.core.tools import QueryEngineTool, ToolMetadata

# Register each data source as a tool
tools = [
    QueryEngineTool(
        query_engine=sales_engine,
        metadata=ToolMetadata(
            name="sales_2024",
            description="Monthly and quarterly sales data for 2024. Includes amount, quantity, and region.",
        ),
    ),
    QueryEngineTool(
        query_engine=hr_engine,
        metadata=ToolMetadata(
            name="hr_data",
            description="HR data. Includes hire date, department, job level, and performance reviews.",
        ),
    ),
]

# The LLM generates the sub-questions automatically and queries in parallel
engine = SubQuestionQueryEngine.from_defaults(
    query_engine_tools=tools,
    verbose=True,  # show the generated sub-questions
)

# Compound question β†’ sub-question decomposition β†’ merged answer
response = engine.query("How did the new hires of 2024 perform, and how much did they contribute to sales?")
print(response)

Vector DB integration

Here you look at Vector DB integration alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

An in-memory index disappears when the server restarts. In production, use ChromaDB (local/self-hosted) or Pinecone (fully managed) as the persistent vector store. StorageContext connects a LlamaIndex index to an external vector DB transparently, and reusing an existing collection cuts the cost of re-indexing.
PYTHON
import chromadb
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.core import StorageContext, VectorStoreIndex

# Create a persistent ChromaDB client
chroma_client = chromadb.PersistentClient(path="./chroma_db")
collection = chroma_client.get_or_create_collection(
    name="company_docs",
    metadata={"hnsw:space": "cosine"},  # use cosine similarity
)

# Wrap it as a LlamaIndex vector store
vector_store = ChromaVectorStore(chroma_collection=collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)

# First-time indexing (after a restart, load with from_vector_store instead)
index = VectorStoreIndex.from_documents(
    documents,
    storage_context=storage_context,
    show_progress=True,
)

# Reuse the existing collection (no re-indexing needed)
index = VectorStoreIndex.from_vector_store(vector_store)

RAGAS evaluation

Here you look at RAGAS evaluation alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

The quality of a RAG system has to be measured in numbers, not by feel. RAGAS quantifies the weak points of a RAG pipeline with three metrics: Faithfulness (hallucination), Answer Relevancy, and Context Recall (how much of the important information was retrieved). Integrate it into CI/CD to detect quality regressions automatically when code changes.
PYTHON
from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevancy, context_recall
from datasets import Dataset

# Build the evaluation dataset β€” at least 50 question-answer pairs recommended
eval_dataset = Dataset.from_dict({
    "question":     questions,      # list of test questions
    "answer":       answers,        # answers from the RAG system
    "contexts":     contexts,       # retrieved contexts (a list of lists)
    "ground_truth": ground_truths,  # reference answers
})

result = evaluate(
    eval_dataset,
    metrics=[
        faithfulness,       # is the answer grounded in the context (hallucination detection)
        answer_relevancy,   # is the answer relevant to the question
        context_recall,     # does the context contain what the answer needs
    ],
)

df = result.to_pandas()
print(df[["question", "faithfulness", "answer_relevancy", "context_recall"]])
# Aim for faithfulness > 0.9 and answer_relevancy > 0.85

Tip

  • If faithfulness is low, reduce the chunk size or add reranking.
  • If context_recall is low, raise similarity_top_k or introduce hybrid search.

LlamaIndex practical design

LlamaIndex practical design is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.

With LlamaIndex, document chunking, metadata, index type, and retriever strategy determine most of the quality. Separate the ingestion pipeline from the query pipeline so reindexing and tuning are easy.
Decision pointQuestion to askPractical guideline
BoundariesWhich parts of the LlamaIndex code are most likely to change?Separate input/output, configuration, external integrations, and the core rules.
StateWhere is state created and where does it go away?Make the owner and lifecycle of state visible in the code.
FailureWhat does the caller receive on failure?Decide the timeout, fallback, and error contract first.

LlamaIndex operating standards

This section covers LlamaIndex operating standards from a practical perspective. Instead of memorizing the concept, focus on the situations in which you would reach for it.

In operation, look at embedding cost, the index refresh interval, vector DB latency, top-k tuning, and reranker cost together.

Tip

  • chunking strategy
  • metadata schema
  • index refresh
  • citation validation

LlamaIndex verification strategy

LlamaIndex verification strategy is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.

Evaluate retrieval quality on context recall/precision and citation accuracy.
Quality axisHow to verifyDefinition of done
CorrectnessAutomate the normal/failure cases.The core scenarios pass reproducibly.
Regression preventionWhen fixing a bug, leave the same case behind as a test.The same failure is not deployed again.
OperabilityCheck the logs, metrics, and alerts.There is a path for tracing the cause when a problem occurs.

LlamaIndex: frequently asked questions

When should I use LlamaIndex?

Use it to build RAG, where an LLM answers by consulting your own data such as internal documents, PDFs, or databases. It provides modules for reading and splitting documents, embedding and indexing them, and retrieving the parts that match a question to put in the prompt.

How do I choose the chunk size for splitting documents?

There is no single answer; it depends on the documents and the kinds of questions. Chunks that are too small cut off the context an answer needs, and chunks that are too large mix in unrelated content and lower retrieval accuracy. Pick a few sizes and overlap values, then compare retrieval hit rates on an evaluation set of question and expected-document pairs.

When an answer is wrong, how do I tell whether retrieval or generation is at fault?

Look at the retrieved chunks first. If the document containing the answer is not among the results, the problem is in indexing or retrieval; if it was retrieved and the answer is still wrong, the problem is the prompt or the model. Evaluating the two stages separately tells you which one to fix.

← Previous guideLangChainNext guide β†’LangGraph