Contents 15 sections
- Technical guide
- Development environment
- AI model selection
- Model install and run
- Training and tuning
- Vector DB setup
- RAG development
- System integration
- Validation and operations
- Model selection by domain
- Overall architecture
- Project ABC
- Data and tools
- Milestones/deliverables
- Tech stack
Technical Blueprint
Hands-on AI Agent Development Checklist
Executable development documents come before explanations of the idea. An AI Agent only moves from demo to real service once local execution, the API contract, RAG data, tool calls, the validation routine, and the operational fallback are fixed first.
1. Development environment and run baseline
- Put `npm install`, `docker compose up`, `npm run build`, and the health check command in order at the top of the README.
- Fix the default development environment: Node.js 22 LTS, Python 3.11 or later, Git, Docker Desktop, and VS Code or Cursor.
- Start with Next.js for the frontend, FastAPI or a Cloudflare Worker for the Agent API, and either Chroma or pgvector for the Vector DB.
- In `.env.example`, keep the model name, API key, DB URL, vector collection, timeout, and observability endpoint under their real variable names.
- From the first commit, treat the build, API unit tests, representative curl calls, and a k6 smoke test as the pass criteria.
2. Model and cost criteria
- First classify the task as one of: conversation, document Q&A, code generation, data analysis, or tool calling.
- Compare candidate production models not on quality alone but also on monthly cost, p95 latency, JSON schema compliance rate, and how well they can fall back on failure.
- For local-first projects, write down the actual model names to install and their expected sizes, such as `llama3.1:8b`, `llama3.2:3b`, `gemma2:9b`, and `nomic-embed-text`.
- When also using API models, add a provider adapter so you can switch through `MODEL_PROVIDER`, `MODEL_NAME`, and `MODEL_TIMEOUT_SECONDS`.
- Evaluate models against a golden set of 30-50 representative questions and tabulate accuracy, groundedness, hallucination rate, latency, and failure types.
3. API contract and local execution
- Four endpoints are enough to fix at first: `/health`, `/v1/chat`, `/v1/rag/query`, and `/v1/agent/run`.
- Every response returns `answer`, `citations`, `model`, `latency_ms`, `trace_id`, and `warnings` in the same shape.
- Design it so the browser never calls the model provider or the laptop API directly, only the server-side API facade.
- Run local models with Ollama or Docker Compose, and include the model list call and a first chat completion curl in the docs.
- Do not hard-code timeout, retry, max tokens, temperature, or streaming; move them into environment variables.
4. Prompts and response policy
- Keep the system prompt short and fixed: the role, forbidden behavior, how to answer when evidence is thin, and the conditions for using tools.
- Early on, improve the prompt template, RAG document quality, tool schema, and JSON response format before reaching for fine-tuning.
- Include examples of forbidden answers, insufficient-evidence answers, and answers that need user confirmation in the golden set.
- Manage per-task output not as free text but as a schema such as `summary`, `evidence`, `next_actions`, `confidence`, and `needs_review`.
- When a prompt changes, compare the golden set pass rate and failure types against the previous version.
5. Building the RAG data
- Store the source documents, chunks, embeddings, metadata, and ingestion version separately, and put the re-index command in the README.
- Start with 10 documents and inspect the chunk size, overlap, metadata, and search results by eye.
- The search API takes explicit `query`, `top_k`, `filters`, and `min_score` and returns the source id, title, page, score, and chunk text.
- Make citations mandatory in RAG answers, and fall back to an “insufficient evidence” response when there is no citation.
- Decide in advance how to detect document changes, index incrementally, tombstone deleted documents, and re-index when the embedding version changes.
6. Implementing the Agent workflow
- Split the workflow into intent classification, deciding whether retrieval is needed, tool calls, answer generation, verification, and fallback, in that order.
- If you use LangGraph, document the input/output schema of each node and the next node on failure.
- Start the tools the Agent can call as read-only, and put write actions behind a human approval step.
- Attach a timeout, retry, rate limit, audit log, and user permission check to every tool call by default.
- Return the final answer with evidence, next actions, and whether review is needed as separate fields so the UI can render it as-is.
7. System integration and security
- The Agent does not query the DB directly; it calls only restricted tool APIs such as `stock_search`, `portfolio_summary`, `log_search`, and `policy_check`.
- Give external APIs and internal DB adapters permissions, rate limits, timeouts, retries, and audit logs by default.
- Clearly divide where the API key, JWT, session id, user id, and request id are handled between the cloud and the local API.
- Do not leave sensitive data as-is in prompts and logs; mask it or move it to a separate secure store.
- Link the user request, tool calls, model response, and final answer by trace id and use it for incident analysis and quality improvement.
8. Validation and moving to production
- Measure correctness, groundedness, policy violations, and output format compliance with the golden set, and compare results before and after each change.
- The minimum bar before production is the health check, smoke test, k6 p95 latency, API error rate, RAG hit rate, and fallback rate.
- In high-risk areas such as finance, education, and operations automation, always separate LLM judgment from deterministic rule checks.
- Observe latency, token usage, tool failures, and retrieval hit rate with OpenTelemetry, Prometheus, Grafana, or at least structured logs.
- Start the first release as a read-only Agent with a human-approval workflow, and widen the scope of automatic execution after enough validation.
Model Selection
AI model selection and install/usage by domain
So that you can start a POC right away on a Coding project, this section lays out the recommended API models, local models, embedding models, and install/usage guidelines.
Recommended API models
GPT-4.1 or a Claude/Gemini Pro family model that is strong at code
Local/on-premises alternatives
Qwen2.5-Coder 7B, Qwen3 4B, Llama 3.2 3B
Embedding models
text-embedding-3-large or an embedding model specialized for code search
Chosen models and Docker install baseline
On a laptop with a 1TB SSD and 16GB of RAM, choose Qwen2.5-Coder 7B as the default local model for the Coding AI Agent. Evaluate Qwen3 4B alongside it for general tool calling and short task Agents, and keep Llama 3.2 3B as the very-low-spec fallback.
Start laptop development with a local Ollama install or Docker in CPU mode. The model does not edit files directly; changes go through a patch planner and an approval workflow.
Recommended server spec
- Laptop development: with a 1TB SSD, 16GB RAM, and CPU only, a 3B-7B Q4 model is recommended. Responses can slow down when the browser, IDE, DB, and Docker are all running at once.
- Recommended free space: 3-6GB for models, 1-2GB for embeddings, 5-20GB for the code index and vector DB, and 30GB or more of headroom for Docker and caches.
- For a large monorepo, do not load the whole repository at once; index only the relevant directories and start with a 4K-8K token context.
Capacity and storage
- Plan on roughly 4-5GB for the Qwen2.5-Coder 7B Q4 model, about 2-3GB for Qwen3 4B, and about 2GB for Llama 3.2 3B.
- Count on around 1GB for a local embedding model such as nomic-embed-text, and allocate 5-20GB for the pgvector/Chroma index depending on repository size.
- Store the repository source, vector index, test run results, and patch drafts separately, and keep secret scan results as an audit log.
Docker install commands
- Ollama Docker CPU: `docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama`
- Coding model: `docker exec -it ollama ollama pull qwen2.5-coder:7b`
- General Agent/embeddings: `docker exec -it ollama ollama pull qwen3:4b` and `docker exec -it ollama ollama pull nomic-embed-text`
- App settings: `LLM_BASE_URL=http://localhost:11434/v1`, `CODE_MODEL=qwen2.5-coder:7b`, `AGENT_MODEL=qwen3:4b`, `EMBEDDING_MODEL=nomic-embed-text`
Post-install verification
- Check the installed models with `curl http://localhost:11434/api/tags`, and index only 20-50 files from one small repository first.
- Evaluate root-cause accuracy and change-plan quality over 5 sample issues, 5 failing test logs, and 5 PR diffs.
- Do not mark a PR draft as complete until `npm run build`, the unit tests, and the Playwright smoke test pass.
- Block secret exposure, deletion of risky files, excessive change scope, and dependency lockfile changes with policy tests.
Why these were chosen
- For a Coding Agent, understanding code structure, multi-file reasoning, analyzing why tests fail, and stable diff generation matter more than natural-language answers.
- Secure accuracy in the initial POC with a high-performance API model, and evaluate coder-family local models separately if there are in-house code security requirements.
- Code changes should not be applied automatically by the model alone; separating the plan, patch, test, and review steps reduces regression and security risk.
Install and configuration
- For the API setup, split the analysis/edit/summary models in `.env`, as in `LLM_MODEL=gpt-4.1`, `CODE_FAST_MODEL=gpt-4.1-mini`, `EMBEDDING_MODEL=text-embedding-3-large`.
- For the local setup, run a coder model with vLLM or Ollama and restrict repository file access to the `repo_search` and `code_reader` tools only.
- Code indexing stores the file path, language, symbols, import graph, and test file mapping as metadata, and chunks are split per function/class.
How to use them
- `repo_search` finds the related files, `code_reader` reads only the files needed, and the model first presents the change plan and the risky files.
- `patch_planner` produces candidate edits, `test_runner` runs the related tests, and the failure logs are passed back to the model.
- PR review generates structured comments on correctness, security, performance, test coverage, and maintainability.
Cautions
- Apply a secret scanner and file exclusion rules first so that secret keys, auth tokens, and customer data never end up in the prompt.
- Large automated changes, dependency upgrades, and running migration scripts go ahead only after human approval.
Reference Architecture
Overall architecture design
An AI Agent is easier to change and validate when the UI, orchestration, tool integration, data retrieval, and evaluation/operations layers are separated.
Presentation Layer
- Next.js UI
- Agent run status
- Evidence/source display
- User feedback collection
Agent Orchestration
- Prompt policy
- Tool routing
- Memory/session
- Structured output
Tool & Integration
- Business API adapter
- DB query tool
- Document search tool
- Policy check tool
Data & Retrieval
- PostgreSQL
- pgvector
- Document chunking
- embedding/reranking
Evaluation & Ops
- Golden set
- Playwright
- k6
- OpenTelemetry/Prometheus
Project ABC
The ABCs of running the project for real
A is architecture design, B is the AI Agent implementation, and C is the validation criteria. Following this order makes it easy to grow from a demo into an Agent you can operate.
A. Architecture
- Split repository search, code reading, test execution, patch planning, and PR review into independent tools.
- The Agent works in this order: understand the issue, search the impact, plan the change, draft the patch, analyze the test results, draft the review.
- Start code access with allowlisted paths and a read-only default mode, and put an approval step in front of actual file edits.
B. Build
- Prepare one sample repository, 10 issues, and 10 failing test logs to build the AI Agent evaluation set.
- Build a function/class-level index with Tree-sitter or per-language parsers, and store the mapping to related test files.
- The first UI consists of panels for the issue input, related file list, change plan, test results, and PR review draft.
C. Check
- After a change, run `npm run build`, the unit tests, and the Playwright smoke test to check for regressions.
- Block secret exposure, deletion of risky files, and excessive change scope with policy tests.
- Treat issue resolution rate, test pass rate, review comment validity, and mean time to fix as the core metrics.
Project Spec
Data and Agent tools
Data
- Git repository
- Issue/PR descriptions
- Test failure logs
- Coding convention documents
Agent Tools
repo_searchcode_readertest_runnerpatch_plannerpr_review
AI Agent scope
- Summarize the issue and analyze its impact
- Find the related files/functions
- Generate a change plan
- Draft test cases and a PR review
Validation criteria
- Build success rate
- Blocking of test regressions
- Blocking of insecure patterns
- Review comment accuracy
Execution
Milestones and deliverables
Milestones
- Build the sample repository/issue set
- Create the code search index
- Wire up Agent tool calls
- Integrate test execution
- Build the PR review UI
Deliverables
- CodeOps Agent UI
- Code search API
- Test execution adapter
- PR review template
- Coding Agent evaluation set
Stack
Tech stack
Next.jsFastAPILangGraphTree-sitterGitHub APIPlaywright