Skip to main content
AIDevOps
  • Learn
  • Learning Paths
  • Practice
  • Open Source
  • Books
  • Engineering

    AI DevOpsThe full map of building and operating AI servicesLLMOpsLLM deployment Β· evaluation Β· observabilityHands-on ProjectsBuild AI Agent projects

    Knowledge

    DocsTechnical documentationBlogEngineering articlesPloggerDevelopment log feed

    Validate

    Certification3-level skills certification Β· coming soon
AI Models
LlamaTutorial|MistralTutorial|GemmaTutorial|DeepSeekTutorial|QwenTutorial
πŸ€– AI Applied Development
Applied AI Intro & RoadmapHugging FaceLangChainLlamaIndexLLMOps|LangGraphMCPMulti-AgentAgent Evaluation
🧠 AI Core
AI Intro & RoadmapML FundamentalsLLM Fundamentals|Python AIC++|PyTorchTensorFlowJAX
🧠 AI Agent Development
Finance AI AgentLLM API ServerStock Investing AgentAIOps AI AgentEducation AI AgentCoding AI Agent
🌱 Spring Cloud
Spring Intro & RoadmapSpring Cloud GatewaySpring BootJava|Spring AISpring SecuritySpring BatchSpring JPA
🐳 DevOps
DevOps Intro & RoadmapLinuxDockerCI/CD|Kubernetes BasicsK8s AdvancedPrometheusGrafana
🧱 Infrastructure
Infrastructure Intro & RoadmapNginxRedis
☁️ Cloud
Cloud Intro & RoadmapAWSGCPAzureNCPCloudflare
🎨 Frontend
Frontend Intro & RoadmapJavaScriptTypeScript|ReactNext.js|VueNuxt|Electron
πŸ“± Mobile
Mobile Intro & RoadmapKotlinAndroidFlutter
βš™οΈ Backend
Backend Intro & RoadmapPython BasicsFastAPIDjangoFlask|CGoGinNode.js
πŸ’Ύ Database
DB Intro & RoadmapCore SQLOracleMySQLPostgreSQL|MongoDBVector DB
πŸ§ͺ Testing
k6JMeternGrinder
AIDevOps

Engineering AI. From Code to Production.
An engineering learning platform for building and operating AI and AI Agents

Learn

  • All Guides
  • Learning Paths
  • AI Tutorial
  • Practice
  • Books

Resources

  • AI DevOps
  • LLMOps
  • Hands-on Projects
  • Docs
  • Blog
  • Plogger
  • Open Source
  • Certification (coming soon)

Start Here

  • AI Core Roadmap
  • AI Applied Development Roadmap
  • Spring Cloud Roadmap
  • DevOps Roadmap
  • Infrastructure Roadmap

Β 

  • Cloud Roadmap
  • Frontend Roadmap
  • Mobile Roadmap
  • Backend Roadmap
  • Database Roadmap
Β© 2026 AIDevOps. All rights reserved.
Terms of ServicePrivacy PolicySitemaptestforge.kr
  1. Home
  2. Learn
  3. AI Applied Development
  4. Applied AI Intro & Roadmap
The first guide for newcomers to AI service development

🧭 Applied AI Intro & Roadmap Complete Guide

Visitors

A starting point for anyone moving from "trying out" LLMs to "building a service" with them. It lays out the full structure of an LLM application, when to use RAG, Agents, or fine-tuning, the framework landscape, and evaluation and operations β€” a roadmap for the order in which to read the guides in the AI Applied Development category.

  • Beginner
  • Updated Oct 1, 2026
  • ~9 min read
  • 12 sections
  • 1 code examples

Learning Paths that include this guide

This guide is one step in the paths below. Study it together with the steps before and after.

  • AI Agent Engineer β†’
  • AI Application Engineer β†’
Understand LLM app structureChoose between RAG, Agent, and fine-tuningFramework landscapeLearning order for AI servicesEvaluation and operations basics

Related frameworks & environments

HFHugging Faceβ†’LCLangChainβ†’LILlamaIndexβ†’πŸ•ΈοΈLangGraphβ†’πŸ”ŒMCPβ†’πŸ“ŠAgent Evaluationβ†’

Contents

0 / 15
  1. How to use this guide
  2. Architecture diagrams
  3. The big picture of LLM apps
  4. 4 implementation patterns
  5. What to use when
  6. Framework landscape
  7. Your first RAG in 30 lines
  8. Service architecture
  9. Before you start
  10. Step-by-step roadmap
  11. 10-week study plan
  12. Project ideas
  13. Common mistakes
  14. Key terms
  15. Frequently asked questions
Contents 15 sections
  1. How to use this guide
  2. Architecture diagrams
  3. The big picture of LLM apps
  4. 4 implementation patterns
  5. What to use when
  6. Framework landscape
  7. Your first RAG in 30 lines
  8. Service architecture
  9. Before you start
  10. Step-by-step roadmap
  11. 10-week study plan
  12. Project ideas
  13. Common mistakes
  14. Key terms
  15. Frequently asked questions

How to use this guide

How to read it

Understanding Applied AI Intro & Roadmap as a real-world workflow

A starting point for anyone moving from "trying out" LLMs to "building a service" with them. It lays out the full structure of an LLM application, when to use RAG, Agents, or fine-tuning, the framework landscape, and evaluation and operations β€” a roadmap for the order in which to read the guides in the AI Applied Development category. Rather than listing concepts, this guide is organized so you can follow the content in the order you have to make decisions on a real project.

Key perspective

AI / LLM systems

Look beyond the model and the prompt: connect the data flow, evaluation, and post-deployment operating metrics in one view.

Understand LLM app structureChoose between RAG, Agent, and fine-tuningFramework landscapeLearning order for AI servicesEvaluation and operations basics

Architecture diagrams

To make what you read stick, it helps to capture the flow as a picture first. The two diagrams below are reference maps you can keep coming back to while learning Applied AI Intro & Roadmap.

Learning flow

Rendering diagram…

Architecture view

Rendering diagram…

The big picture of an LLM application

When you first open Applied AI Intro & Roadmap, the big picture matters more than individual commands. This section starts with the problems the concepts ahead were created to solve.

Asking a question in the ChatGPT window and putting an LLM into a service are completely different jobs. In a service you have to connect your own data (internal documents, DBs), integrate with external systems (search, payments, email), and continuously manage answer quality, cost, and speed.

So an LLM app is not a single "model call" line but a pipeline: input handling β†’ context building (retrieval) β†’ model call β†’ tool execution β†’ output validation β†’ logging and evaluation. Each guide in the AI Applied Development category covers one part of that pipeline.
ComponentRoleGuide
ModelThe engine for understanding and generating text (API or open source)Hugging Face
Chain / pipelineThe flow that connects prompts, models, and parsersLangChain
RetrievalFinds your documents relevant to the question and supplies them as contextLlamaIndex, Vector DB
Agent / workflowA stateful flow with tool calls, branching, and loopsLangGraph, Multi-Agent
Tool connection standardThe specification for connecting LLMs to external tools and dataMCP
Evaluation Β· operationsQuality measurement, cost and latency monitoring, deploymentAgent Evaluation, LLMOps

Tip

If the principles behind LLMs themselves (Transformers, tokens, embeddings) are unfamiliar, read the "AI Intro & Roadmap" guide in the AI Core category and LLM Fundamentals first.

The 4 implementation patterns of LLM services

The 4 implementation patterns of LLM services is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.

Most LLM services are one of the four patterns below, or a combination of them. Complexity and cost grow as you go down the list, so the working rule is to start with the simplest pattern and move up one level only when it falls short.
PatternStructureGood fitExample
β‘  Prompt + APISystem prompt + user input β†’ modelSummarization, translation, and classification where general knowledge is enoughReview sentiment analysis, drafting emails
β‘‘ RAGQuestion β†’ document search β†’ search results + question β†’ modelAnswers based on your own data or fresh informationCompany policy Q&A, product manual chatbot
β‘’ Agent (Tool Use)The model decides on a tool call β†’ runs it β†’ judges again from the result (loop)Multi-step tasks, operating external systemsBooking a meeting, querying data and writing a report
β‘£ Fine-tuningFurther train the model itself on your dataA fixed format, tone, or domain vocabulary mattersGenerating reports in a set format, classifying in-house terms

What to use when: a decision table

What to use when: a decision table is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.

The question newcomers ask most is "Should I do RAG or should I fine-tune?" The key is the cause of the problem. If the problem is what the model does not know, use RAG; if it is how it behaves, use prompting or fine-tuning; if it is what it cannot do, use an Agent (tools).
SymptomCauseTry first
It does not know internal information / gets recent facts wrongMissing knowledgeRAG
The answer format is different every timeUnclear instructionsBetter prompts + structured output (JSON schema)
It needs to calculate, look something up, or run somethingMissing capabilityAgent + Tool Use
Prompting alone cannot pin down the tone or formatBehavior patternFine-tuning (last resort)
Answers are too slow or too expensiveModel size, number of callsSmaller models, caching, fewer call steps

Tip

Fine-tuning is an inefficient way to add knowledge, because it needs retraining every time the documents change. Knowledge goes in RAG, style goes in fine-tuning β€” remembering that split alone avoids most design mistakes.

Framework landscape: which tool solves which problem

Framework landscape: which tool solves which problem is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.

LLM frameworks change quickly, but the division of roles is fairly stable. You do not need to learn every tool up front: one chaining tool + one vector DB + one evaluation method is enough to build a first service.
ToolCore roleStrengthWhen to choose it
LangChainGeneral-purpose chains that assemble prompts, models, and toolsThe widest integration ecosystemConnecting many models and tools quickly
LlamaIndexSpecialized in document loading, indexing, and retrievalConcise RAG pipeline setupDocument-based Q&A is the core
LangGraphState-graph-based Agent workflowsControl over branching, loops, and human approvalMulti-step Agents, long-running tasks
MCPStandard protocol connecting LLMs ↔ tools/dataBuild a tool once and reuse it across AI clientsStandardizing tools to offer them to several apps
Hugging FaceOpen-source model hub + training and inference librariesBreadth of model choice, fine-tuningSelf-hosting or fine-tuning is needed
Vector DBStores embeddings and runs similarity searchpgvector (reuses your existing DB), Chroma (simple), Pinecone (managed)The retrieval layer of RAG

Your first RAG in 30 lines: the skeleton of document Q&A

Here you look at Your first RAG in 30 lines: the skeleton of document Q&A alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.

The core of RAG is just three steps. β‘  Split documents into small pieces and store them as embeddings, β‘‘ retrieve the pieces most similar to the question, then β‘’ put the retrieved results into the prompt to generate the answer. The code below shows that flow as-is, with no framework. In the end, frameworks are tools that make this process more convenient.
mini_rag.pyPYTHON
# pip install openai numpy
import numpy as np
from openai import OpenAI

client = OpenAI()
docs = [
    "Annual leave is 15 days after one year of service, plus 1 more day every 2 years.",
    "Remote work is allowed up to 2 days a week and needs prior approval from the team lead.",
    "Family event leave is 5 days for your own wedding and 1 day for the wedding of a child.",
]

def embed(texts):
    res = client.embeddings.create(model="text-embedding-3-small", input=texts)
    return np.array([d.embedding for d in res.data])

doc_vecs = embed(docs)                                   # β‘  store the document embeddings

def ask(question, k=2):
    q = embed([question])[0]
    scores = doc_vecs @ q                                # β‘‘ cosine similarity (normalized vectors)
    context = "\n".join(docs[i] for i in scores.argsort()[::-1][:k])
    res = client.chat.completions.create(                # β‘’ answer from the retrieved results
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "Answer only from the documents below, and say you do not know if the answer is not there.\n\n" + context},
            {"role": "user", "content": question},
        ],
    )
    return res.choices[0].message.content

print(ask("How many remote work days do I get?"))

Tip

  • The single line "if it is not in the documents, say you do not know" cuts hallucination sharply. Half of RAG quality comes from retrieval and the other half from prompt rules like this.
  • In a real service, chunk size, the number of results retrieved (k), and source citation decide the quality. The LlamaIndex and LangChain guides cover this in detail.

AI service architecture: what production looks like

AI service architecture: what production looks like is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.

The difference between a demo and production is not the model but the surrounding systems. A real service needs the elements below before it can be delivered to users reliably.
LayerWhat is neededRelated guides
API serverStreaming responses, authentication, rate limiting, timeoutsFastAPI, Python AI
Data pipelineBatch jobs that collect, split, and re-embed documentsLlamaIndex
CacheCaching answers to identical questions to cut cost and latencyRedis
GuardrailsInput validation, prompt injection defense, output filteringAgent Evaluation
ObservabilityTracking prompts, tokens, latency, and cost per requestLLMOps
DeploymentContainerization, autoscaling, model version managementDocker, Kubernetes

Before you start

Before you start is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.

Applied AI development is closer to "backend development" than to "AI research". API design, async programming, data handling, and deployment skills are used far more often than the math inside the model.
AreaLevel neededHow to check
PythonFunctions, classes, async/await, type hintsWrite code that calls 3 APIs concurrently
HTTP / REST APIRequests, responses, status codes, JSONImplement CRUD endpoints in FastAPI
LLM basicsTokens, context, embeddings, temperatureCan summarize the LLM Fundamentals guide
Working with dataJSON, text processing, basic SQLExtract text from PDFs and CSVs
Git Β· DockerVersion control, running containersBuild your app as a Docker image

Step-by-step roadmap: from the first API call to production

Step-by-step roadmap: from the first API call to production is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.

Each step builds on the previous one. Move on once you pass the done check for a step. Follow the links to the detailed guide for each topic.
StepGoalWhat to learnGuidesDone check
0. BasicsPython and LLM conceptsAsync Python, token and embedding conceptsPython AI, LLM FundamentalsGet structured JSON output from an LLM API
1. Working with modelsChoosing models and running them locallyModel hub, tokenizers, inference pipelinesHugging FaceRun text classification with an open-source model
2. ChainsComposing prompt pipelinesPrompt templates, output parsers, chain compositionLangChainImplement a 3-stage summarize→classify→translate chain
3. RAGAI that answers from your documentsLoaders, chunking, embeddings, vector search, rerankingLlamaIndex, Vector DBA document Q&A chatbot that cites its sources
4. AgentAI that uses toolsTool Use, state graphs, human approval stepsLangGraphAn Agent that uses search and calculator tools
5. Tool standardizationA reusable tool serverMCP servers and clients, resources, promptsMCPImplement an MCP server that queries an internal DB
6. Multi-agentCollaboration through divided rolesSupervisor, handoffs, parallel executionMulti-AgentA 3-Agent research, write, and review pipeline
7. Evaluation & operationsA service you can trustEvaluation datasets, LLM-as-judge, tracing, cost managementAgent Evaluation, LLMOpsWire regression evaluation into CI

Tip

Once you finish step 3 (RAG), you can contribute to most enterprise AI projects. Agents and multi-agent systems only work properly when the RAG underneath is solid.

10-week study plan: 1-2 hours a day

10-week study plan: 1-2 hours a day is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.

The goal is to leave behind one working deliverable every week. After 10 weeks you have a RAG and Agent service, complete with evaluation metrics, in your portfolio.
WeekTopicDeliverable
Week 1LLM API, structured output, streamingA review analysis CLI that answers in JSON
Week 2Running Hugging Face modelsAn open-source model classifier + a comparison table against an API model
Week 3Composing LangChain chainsA multi-stage document summarization chain
Weeks 4-5RAG: chunking, embeddings, vector DB, rerankingDocument Q&A with source citations (FastAPI + vector DB)
Week 6LangGraph AgentA research Agent that uses 3 tools
Week 7MCP serverAn MCP server that serves your own data
Week 8Multi-agentA 2-Agent writer-reviewer pipeline
Week 9EvaluationA 30-question evaluation set + an automatic grading script
Week 10Deployment and observabilityDocker deployment + a token and latency dashboard

Project ideas by difficulty

Project ideas by difficulty is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.

Picking a tool you actually want to use every day raises the odds that you finish it. More concrete step-by-step labs are collected on the Hands-on Projects page.
DifficultyProjectKey techniques
β˜…β˜†β˜†Meeting notes summarizer + action item extractorStructured output, prompt design
β˜…β˜…β˜†Company policy and manual Q&A chatbotRAG, source citations, vector DB
β˜…β˜…β˜†GitHub issue auto-classification and labeling botClassification chain, webhook integration
β˜…β˜…β˜…Data analysis Agent (generate SQL β†’ run β†’ chart)Tool Use, LangGraph, guardrails
β˜…β˜…β˜…Multi-agent customer inquiry handlingSupervisor, handoffs, evaluation pipeline

Common mistakes in applied AI development

Common mistakes in applied AI development is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.

A demo can be built in a day, but a service you can trust is built from evaluation and guardrails.
MistakeResultDo this instead
Designing with Agents and multi-agent from the startComplexity that cannot be debuggedGrow in order: simple prompt β†’ RAG β†’ Agent
Editing prompts without evaluationFixing one place breaks anotherCompare against a fixed evaluation set on every change
Leaving chunking at the defaultsWorse retrieval β†’ off-target answersSplit by document structure (headings, paragraphs) + inspect retrieval results yourself
Using the largest model for every requestCost and latency blow upRoute to models by task difficulty, cache
Granting tools broad permissionsData leaks or misbehavior under prompt injectionLeast privilege, human approval for risky actions
Using LLM output without validationParsing errors, wrong values savedSchema validation (Pydantic) + retries

Key terms

Key terms is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.

TermOne-line explanation
ChunkingSplitting documents into small searchable pieces for RAG
RerankingRe-sorting first-pass retrieval results with a more precise model to raise accuracy
Hybrid searchCombining keyword search (BM25) with vector search
Structured outputForcing model responses into a set format such as a JSON schema
Tool Use / Function CallingThe model decides which function to call and with which arguments, and returns that
ReActAn Agent pattern that alternates reasoning (Reason) and acting (Act)
GuardrailA mechanism that inspects input and output to block risky or wrong responses
Prompt injectionAn attack that bypasses the original rules of the model with instructions hidden in the input
LLM-as-a-JudgeUsing an LLM as the grader to evaluate the answer quality of another LLM
TracingRecording, step by step, the prompts, tool calls, and tokens a single request went through

Tip

The next step is the Hugging Face guide. Load and run a model yourself to get a feel for the LLM app from its bottom layer up.

Applied AI Intro & Roadmap: frequently asked questions

What should I learn first for AI application development?

Start by calling a model API and reading the response, then widen out in this order: prompt design, RAG that consults your own documents, agents that call tools, and quality evaluation. Adopt a framework after you understand the structure through direct calls, so you know what it does for you.

What is the difference between RAG and an agent?

RAG finds documents related to the question and puts them in the prompt, while an agent lets the model decide which tools to use and carries out several steps. RAG has a fixed flow and is predictable; an agent is flexible but costs more and can fail in more ways. Retrieval can also be one of the tools an agent calls.

What do I need before putting an AI feature into production?

You need logging of inputs, outputs, token usage, and latency; a cost ceiling; retries and a fallback path for model outages and rate limits; and an evaluation set that runs on every change. Prepare input and output checks for personal data and prompt injection as well.

Next guide β†’Hugging Face