Skip to main content
AIDevOps
  • Learn
  • Learning Paths
  • Practice
  • Open Source
  • Books
  • Engineering

    AI DevOpsThe full map of building and operating AI servicesLLMOpsLLM deployment · evaluation · observabilityHands-on ProjectsBuild AI Agent projects

    Knowledge

    DocsTechnical documentationBlogEngineering articlesPloggerDevelopment log feed

    Validate

    Certification3-level skills certification · coming soon
AI Models
LlamaTutorial|MistralTutorial|GemmaTutorial|DeepSeekTutorial|QwenTutorial
🧠 AI Agent Development
Finance AI AgentLLM API ServerStock Investing AgentAIOps AI AgentEducation AI AgentCoding AI Agent
🤖 AI Applied Development
Applied AI Intro & RoadmapHugging FaceLangChainLlamaIndexLLMOps|LangGraphMCPMulti-AgentAgent Evaluation
🧠 AI Core
AI Intro & RoadmapML FundamentalsLLM Fundamentals|Python AIC++|PyTorchTensorFlowJAX
🌱 Spring Cloud
Spring Intro & RoadmapSpring Cloud GatewaySpring BootJava|Spring AISpring SecuritySpring BatchSpring JPA
🐳 DevOps
DevOps Intro & RoadmapLinuxDockerCI/CD|Kubernetes BasicsK8s AdvancedPrometheusGrafana
🧱 Infrastructure
Infrastructure Intro & RoadmapNginxRedis
☁️ Cloud
Cloud Intro & RoadmapAWSGCPAzureNCPCloudflare
🎨 Frontend
Frontend Intro & RoadmapJavaScriptTypeScript|ReactNext.js|VueNuxt|Electron
📱 Mobile
Mobile Intro & RoadmapKotlinAndroidFlutter
⚙️ Backend
Backend Intro & RoadmapPython BasicsFastAPIDjangoFlask|CGoGinNode.js
💾 Database
DB Intro & RoadmapCore SQLOracleMySQLPostgreSQL|MongoDBVector DB
🧪 Testing
k6JMeternGrinder
AIDevOps

Engineering AI. From Code to Production.
An engineering learning platform for building and operating AI and AI Agents

Learn

  • All Guides
  • Learning Paths
  • AI Tutorial
  • Practice
  • Books

Resources

  • AI DevOps
  • LLMOps
  • Hands-on Projects
  • Docs
  • Blog
  • Plogger
  • Open Source
  • Certification (coming soon)

Start Here

  • AI Core Roadmap
  • AI Applied Development Roadmap
  • Spring Cloud Roadmap
  • DevOps Roadmap
  • Infrastructure Roadmap

 

  • Cloud Roadmap
  • Frontend Roadmap
  • Mobile Roadmap
  • Backend Roadmap
  • Database Roadmap
© 2026 AIDevOps. All rights reserved.
Terms of ServicePrivacy PolicySitemaptestforge.kr
Education AI Agent

EDU EduLab Tutor Agent AI Agent Project

Visitors

An education Agent that connects learning diagnosis, tailored question generation, and explanatory feedback. Provides a personal learning path and feedback based on the learner solution history, error patterns, and curriculum documents.

← Back to projects
EDU

Contents

0 / 15
  1. Technical guide
  2. Development environment
  3. AI model selection
  4. Model install and run
  5. Training and tuning
  6. Vector DB setup
  7. RAG development
  8. System integration
  9. Validation and operations
  10. Model selection by domain
  11. Overall architecture
  12. Project ABC
  13. Data and tools
  14. Milestones/deliverables
  15. Tech stack
Contents 15 sections
  1. Technical guide
  2. Development environment
  3. AI model selection
  4. Model install and run
  5. Training and tuning
  6. Vector DB setup
  7. RAG development
  8. System integration
  9. Validation and operations
  10. Model selection by domain
  11. Overall architecture
  12. Project ABC
  13. Data and tools
  14. Milestones/deliverables
  15. Tech stack

Technical Blueprint

Hands-on AI Agent Development Checklist

Executable development documents come before explanations of the idea. An AI Agent only moves from demo to real service once local execution, the API contract, RAG data, tool calls, the validation routine, and the operational fallback are fixed first.

1. Development environment and run baseline

  • Put `npm install`, `docker compose up`, `npm run build`, and the health check command in order at the top of the README.
  • Fix the default development environment: Node.js 22 LTS, Python 3.11 or later, Git, Docker Desktop, and VS Code or Cursor.
  • Start with Next.js for the frontend, FastAPI or a Cloudflare Worker for the Agent API, and either Chroma or pgvector for the Vector DB.
  • In `.env.example`, keep the model name, API key, DB URL, vector collection, timeout, and observability endpoint under their real variable names.
  • From the first commit, treat the build, API unit tests, representative curl calls, and a k6 smoke test as the pass criteria.

2. Model and cost criteria

  • First classify the task as one of: conversation, document Q&A, code generation, data analysis, or tool calling.
  • Compare candidate production models not on quality alone but also on monthly cost, p95 latency, JSON schema compliance rate, and how well they can fall back on failure.
  • For local-first projects, write down the actual model names to install and their expected sizes, such as `llama3.1:8b`, `llama3.2:3b`, `gemma2:9b`, and `nomic-embed-text`.
  • When also using API models, add a provider adapter so you can switch through `MODEL_PROVIDER`, `MODEL_NAME`, and `MODEL_TIMEOUT_SECONDS`.
  • Evaluate models against a golden set of 30-50 representative questions and tabulate accuracy, groundedness, hallucination rate, latency, and failure types.

3. API contract and local execution

  • Four endpoints are enough to fix at first: `/health`, `/v1/chat`, `/v1/rag/query`, and `/v1/agent/run`.
  • Every response returns `answer`, `citations`, `model`, `latency_ms`, `trace_id`, and `warnings` in the same shape.
  • Design it so the browser never calls the model provider or the laptop API directly, only the server-side API facade.
  • Run local models with Ollama or Docker Compose, and include the model list call and a first chat completion curl in the docs.
  • Do not hard-code timeout, retry, max tokens, temperature, or streaming; move them into environment variables.

4. Prompts and response policy

  • Keep the system prompt short and fixed: the role, forbidden behavior, how to answer when evidence is thin, and the conditions for using tools.
  • Early on, improve the prompt template, RAG document quality, tool schema, and JSON response format before reaching for fine-tuning.
  • Include examples of forbidden answers, insufficient-evidence answers, and answers that need user confirmation in the golden set.
  • Manage per-task output not as free text but as a schema such as `summary`, `evidence`, `next_actions`, `confidence`, and `needs_review`.
  • When a prompt changes, compare the golden set pass rate and failure types against the previous version.

5. Building the RAG data

  • Store the source documents, chunks, embeddings, metadata, and ingestion version separately, and put the re-index command in the README.
  • Start with 10 documents and inspect the chunk size, overlap, metadata, and search results by eye.
  • The search API takes explicit `query`, `top_k`, `filters`, and `min_score` and returns the source id, title, page, score, and chunk text.
  • Make citations mandatory in RAG answers, and fall back to an “insufficient evidence” response when there is no citation.
  • Decide in advance how to detect document changes, index incrementally, tombstone deleted documents, and re-index when the embedding version changes.

6. Implementing the Agent workflow

  • Split the workflow into intent classification, deciding whether retrieval is needed, tool calls, answer generation, verification, and fallback, in that order.
  • If you use LangGraph, document the input/output schema of each node and the next node on failure.
  • Start the tools the Agent can call as read-only, and put write actions behind a human approval step.
  • Attach a timeout, retry, rate limit, audit log, and user permission check to every tool call by default.
  • Return the final answer with evidence, next actions, and whether review is needed as separate fields so the UI can render it as-is.

7. System integration and security

  • The Agent does not query the DB directly; it calls only restricted tool APIs such as `stock_search`, `portfolio_summary`, `log_search`, and `policy_check`.
  • Give external APIs and internal DB adapters permissions, rate limits, timeouts, retries, and audit logs by default.
  • Clearly divide where the API key, JWT, session id, user id, and request id are handled between the cloud and the local API.
  • Do not leave sensitive data as-is in prompts and logs; mask it or move it to a separate secure store.
  • Link the user request, tool calls, model response, and final answer by trace id and use it for incident analysis and quality improvement.

8. Validation and moving to production

  • Measure correctness, groundedness, policy violations, and output format compliance with the golden set, and compare results before and after each change.
  • The minimum bar before production is the health check, smoke test, k6 p95 latency, API error rate, RAG hit rate, and fallback rate.
  • In high-risk areas such as finance, education, and operations automation, always separate LLM judgment from deterministic rule checks.
  • Observe latency, token usage, tool failures, and retrieval hit rate with OpenTelemetry, Prometheus, Grafana, or at least structured logs.
  • Start the first release as a read-only Agent with a human-approval workflow, and widen the scope of automatic execution after enough validation.

Model Selection

AI model selection and install/usage by domain

So that you can start a POC right away on a Education project, this section lays out the recommended API models, local models, embedding models, and install/usage guidelines.

Recommended API models

GPT-4.1 mini or Gemini 2.5 Flash

Local/on-premises alternatives

Gemma 2/3 family or Llama 3.1/3.3 8B Instruct

Embedding models

text-embedding-3-small or bge-m3

Chosen models and Docker install baseline

Choose Gemma 3 12B Instruct or Llama 3.1 8B Instruct as the default local model for the education AI Agent. Friendly explanations and fast responses matter, and grading is left to a separate checker.

Start teacher demos and offline learning environments on Ollama Docker. Add an adapter so that only hard explanations are routed to a higher-tier API model.

Recommended server spec

  • Development/demo: 8-core CPU, 32GB RAM, GPU with 12-16GB VRAM. Validate an 8B-12B Q4 model together with bge-m3 embeddings.
  • School/institution POC: one GPU with 24GB VRAM, 64GB RAM, SSD of 100GB or more. Test multiple learner sessions and question generation.
  • For a low-spec laptop demo, scale down to a 4B-8B model and compare question generation quality separately against the golden set.

Capacity and storage

  • About 5-9GB for an 8B-12B Q4 model, about 2-3GB for bge-m3, and at least an 80GB SSD recommended including the curriculum/question vector index.
  • Store student submissions and assessment records in separate DB tables, and pass only minimal information into the model prompt.
  • Manage the question bank with the original text, answer, explanation, difficulty, unit metadata, and embedding version together.

Docker install commands

  • Ollama: `docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama`
  • Model download: `docker exec -it ollama ollama pull gemma3:12b` or `docker exec -it ollama ollama pull llama3.1:8b`
  • Embeddings: after `docker exec -it ollama ollama pull bge-m3`, set `EMBEDDING_MODEL=bge-m3` in the app.

Post-install verification

  • Evaluate the answer leak rate and difficulty fit over 20 learning diagnoses, 30 generated explanations, and 30 generated hints.
  • Compare grading results against the deterministic checker, and evaluate the LLM only on the quality of its feedback text.
  • Inspect sample logs to make sure student personal data does not remain in prompts or trace logs.

Why these were chosen

  • An education Agent needs friendly explanations, difficulty adjustment, and curriculum-based evidence, so a fast and stable mid-size model is the right fit.
  • A deterministic checker suits grading and score calculation better than the model; keep the model focused on generating explanations and feedback.
  • Student data and assessment records are sensitive, so you need to be able to choose a local model or a structure that sends minimal data.

Install and configuration

  • For the API setup, start with `LLM_MODEL=gpt-4.1-mini` and `EMBEDDING_MODEL=text-embedding-3-small`, and route only hard explanations to a higher-tier model.
  • For the local setup, run a Gemma/Llama 8B-class model with Ollama to build the teacher demo and offline learning environment.
  • Store curriculum documents and the question bank across PostgreSQL and the Vector DB with unit, concept tag, difficulty, and answer/explanation metadata.

How to use them

  • `curriculum_retriever` finds the relevant concepts, `quiz_generator` creates candidate questions, and `answer_checker` handles grading.
  • Structure the model output as `diagnosis`, `hint`, `explanation`, `next_questions`, and `learning_path`.
  • Start per-student personalization with session/profile-based recommendations, and store long-term memory only after explicit consent.

Cautions

  • Do not send sensitive student assessment records to an external model as-is.
  • Pin the grading logic with unit tests so a wrong answer is never graded as correct.

Reference Architecture

Overall architecture design

An AI Agent is easier to change and validate when the UI, orchestration, tool integration, data retrieval, and evaluation/operations layers are separated.

Presentation Layer

  • Next.js UI
  • Agent run status
  • Evidence/source display
  • User feedback collection

Agent Orchestration

  • Prompt policy
  • Tool routing
  • Memory/session
  • Structured output

Tool & Integration

  • Business API adapter
  • DB query tool
  • Document search tool
  • Policy check tool

Data & Retrieval

  • PostgreSQL
  • pgvector
  • Document chunking
  • embedding/reranking

Evaluation & Ops

  • Golden set
  • Playwright
  • k6
  • OpenTelemetry/Prometheus

Project ABC

The ABCs of running the project for real

A is architecture design, B is the AI Agent implementation, and C is the validation criteria. Following this order makes it easy to grow from a demo into an Agent you can operate.

A. Architecture

  • Split curriculum search, question generation, answer grading, and learning path recommendation into independent tools.
  • The Agent runs as a flow: confirm the learning goal, ask diagnostic questions, present problems, explain, and recommend the next assignment.
  • Grading uses the deterministic checker first, and the LLM focuses on generating explanations and feedback.

B. Build

  • Keep the AI Agent scope small and fixed: one subject, one unit, 30 questions.
  • Store the answer, explanation, difficulty, and concept tags as required metadata on each question.
  • The learning screen consists of the diagnosis result, recommended questions, submitted answers, explanatory feedback, and a link to the next lesson.

C. Check

  • Pin grading and score calculation with unit tests, and validate LLM output against a schema.
  • Verify with sample learner profiles that difficulty recommendations do not skew toward questions that are too easy or too hard.
  • Block learner personal data, sensitive assessment records, and wording meant for external sharing with policy tests.

Project Spec

Data and Agent tools

Data

  • Curriculum documents
  • Question bank
  • Solution history
  • Error notebook

Agent Tools

  • curriculum_retriever
  • quiz_generator
  • answer_checker
  • learning_path_planner

AI Agent scope

  • Diagnose the learning level
  • Generate 5 tailored questions
  • Explain the cause of wrong answers
  • Recommend the next learning path

Validation criteria

  • Answer grading accuracy
  • Difficulty fit
  • Rate of explanations with evidence
  • Minimal personal data collection

Execution

Milestones and deliverables

Milestones

  • Design unit/question data
  • Implement the grading tool
  • Connect the question generation Agent
  • Learning feedback UI
  • Validate difficulty/answers

Deliverables

  • Tutor Agent UI
  • Question bank schema
  • Answer grading API
  • Learning path recommendation logic
  • Education quality evaluation set

Stack

Tech stack

Next.jsFastAPIPydanticPostgreSQLpgvectorPlaywright