From the big picture of what machine learning is to learning types, core algorithms, neural networks, metrics, overfitting, hyperparameter tuning, and MLOps β the ML fundamentals to get solid before learning LLMs, covered from start to finish in a way that is easy but deep.
Beginner
Updated Oct 1, 2026
~11 min read
12 sections
5 code examples
Learning Paths that include this guide
This guide is one step in the paths below. Study it together with the steps before and after.
Organize machine learning conceptsNeural network basicsUnderstand the PyTorch training loopGetting started as an AI engineerModel evaluation and tuning
Understanding ML Fundamentals as a real-world workflow
From the big picture of what machine learning is to learning types, core algorithms, neural networks, metrics, overfitting, hyperparameter tuning, and MLOps β the ML fundamentals to get solid before learning LLMs, covered from start to finish in a way that is easy but deep. Rather than listing concepts, this guide is organized so you can follow the content in the order you have to make decisions on a real project.
Key perspective
AI / LLM systems
Look beyond the model and the prompt: connect the data flow, evaluation, and post-deployment operating metrics in one view.
Organize machine learning conceptsNeural network basicsUnderstand the PyTorch training loopGetting started as an AI engineerModel evaluation and tuning
Architecture diagrams
To make what you read stick, it helps to capture the flow as a picture first. The two diagrams below are reference maps you can keep coming back to while learning ML Fundamentals.
Learning flow
Rendering diagramβ¦
Architecture view
Rendering diagramβ¦
The big picture: how AI, ML, DL, and LLMs relate
When you first open ML Fundamentals, the big picture matters more than individual commands. This section starts with the problems the concepts ahead were created to solve.
We start by sorting out the terms. Artificial intelligence (AI) is the broadest goal: "build systems that judge like people". Machine learning (ML) is one way of reaching that goal, an approach in which rules are found from data instead of being hand-coded one by one. Deep learning (DL) is a subfield of ML that stacks many layers of neural networks (deep) to learn complex patterns. An LLM (large language model) is, within deep learning, a model that handles language by training on vast text with the Transformer architecture. In other words, AI β ML β DL β LLM, each more specific than the last.
Layer
Definition
Examples
AI (artificial intelligence)
The whole range of systems that perceive, judge, and act like people
Chess engines, voice assistants, recommender systems
ML (machine learning)
The subfield of AI that learns rules from data
Spam filters, credit scoring models
DL (deep learning)
The subfield of ML that learns complex patterns with multi-layer neural networks
Image classification (CNN), speech recognition
LLM (large language model)
A branch of DL that learns language on the Transformer architecture
GPT, Claude, LLaMA
The three kinds of learning (+1)
The three kinds of learning (+1) is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.
Machine learning divides by "what is treated as the answer during training" into three kinds, plus a fourth that has become especially important in the LLM era. Working out first which of these a problem falls under lets you pick the right tool much faster.
Kind
Answer data
Typical problems
Examples
Supervised learning
Needs input-answer pairs
Classification, regression
Spam classification, stock price prediction
Unsupervised learning
No answers
Clustering, dimensionality reduction
Customer segments, anomaly detection
Reinforcement learning
A reward signal
Sequential decision making
RLHF, game AI, robot control
Self-supervised learning
The data itself is the answer
Representation learning
LLM pre-training (next-token prediction)
The 6-step ML pipeline: how data becomes a model
Here you look at The 6-step ML pipeline: how data becomes a model alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
In practice, building a model is far broader than "picking an algorithm". Knowing the 6 steps below means that, whatever project you meet, you never lose track of which step you are in.
pipeline_skeleton.pyPYTHON
import pandas as pdfrom sklearn.model_selection import train_test_splitfrom sklearn.preprocessing import StandardScalerfrom sklearn.linear_model import LogisticRegressionfrom sklearn.metrics import classification_report# 1-2. Load the datadf = pd.read_csv("customers.csv")X, y = df.drop(columns=["churn"]), df["churn"]# 3. Split the data first! (preprocessing parameters are learned from train only)X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, stratify=y, random_state=42)scaler = StandardScaler().fit(X_train) # fit on train onlyX_train_s = scaler.transform(X_train)X_test_s = scaler.transform(X_test) # test is only transformed# 4. Start with a baseline modelmodel = LogisticRegression(max_iter=1000)model.fit(X_train_s, y_train)# 5. Evaluatey_pred = model.predict(X_test_s)print(classification_report(y_test, y_pred))
Step
What you do
Common trap
1. Problem definition
Decide whether it is classification/regression/clustering and set the success criterion (metric) first
Aiming only at "get it right" with no metric
2. Data collection
Gather data relevant to the problem
Labeling criteria differ from person to person, so data quality is uneven
3. Preprocessing & feature engineering
Handle missing values, normalize, encode categoricals, create derived features
Data leakage, where test data information seeps into preprocessing
4. Model training
Start from a baseline and move gradually to more complex models
Jumping straight to a complex deep learning model with no validation
5. Evaluation
Measure performance with the right metric, analyze the error cases
Judging by accuracy alone (dangerous with class imbalance)
6. Deployment & monitoring
Ship it to production and keep watching for performance decay
Ignoring drift as the data distribution changes after deployment
Map of core algorithms
Map of core algorithms is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.
The classic algorithms used since before deep learning are still powerful today as baseline models and on tabular data. Drawing a map of what exists before jumping into neural networks makes it easier to pick the tool that fits the problem.
Algorithm
Type
One-line principle
Traits
Linear regression
Supervised (regression)
Predicts a continuous value as a weighted sum of the inputs
Easy to interpret, fast, try first as a baseline
Logistic regression
Supervised (classification)
Turns the weighted sum into a 0-1 probability with a sigmoid
The standard baseline for binary classification
Decision tree / random forest
Supervised
Splits the data by asking questions repeatedly, like pruning
Strong on tabular data, little preprocessing needed
Gradient boosting (XGBoost and others)
Supervised
Each new tree corrects the errors of the previous ones
The model family that regularly wins Kaggle competitions
k-NN
Supervised
Predicts by majority vote/average of the k nearest neighbors
Simple and distance-based, slow on large data
SVM
Supervised
Finds the boundary that separates two classes with the widest margin
Strong on small-to-medium high-dimensional data
k-means
Unsupervised
Forms clusters by moving the centroids repeatedly
Distance-based clustering, the number of clusters k is set in advance
PCA
Unsupervised
Rebuilds the axes along the directions of greatest variance
Compresses high-dimensional data to low dimensions (dimensionality reduction)
Neural network basics
Here you look at Neural network basics alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
A neural network is perceptrons (one linear function + an activation function) stacked in layers. The output of each layer becomes the input of the next, and at the end the error against the answer is passed back to the earlier layers by backpropagation to update the weights. Put simply, it is a process of turning the input into ever more abstract representations through several stages of filters β for an image, "lines β shapes β parts β the whole object".
Squashes the output to 0-1, suffers from vanishing gradients
The last layer of binary classification
ReLU
Negatives become 0, positives pass through β fast to compute with fewer vanishing gradients
The default choice for hidden layers
GELU
A smoothed form of ReLU
Transformer family models
Softmax
Turns several values into a probability distribution that sums to 1
The last layer of multi-class classification
CNN Β· RNN at a glance: deep learning before the Transformer
CNN Β· RNN at a glance: deep learning before the Transformer is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.
Before the Transformer took over, deep learning used dedicated architectures matched to the kind of data. They are still valid choices for image and time-series problems, so it is worth knowing at least the concepts.
Architecture
Key idea
Main use
Limits
CNN (convolutional neural network)
Small filters slide over the image to detect local patterns (lines, edges)
Image classification, object detection
Relatively weak at capturing long-range relationships
RNN / LSTM
Uses the hidden state of the previous time step as input to the next, reflecting order
Time-series forecasting, early machine translation
Forgets the beginning as sentences get long, and is slow because it processes sequentially
Transformer
Self-Attention compares every position with every other at once β capturing relationships directly however far apart
Language models (LLMs), modern vision models (ViT)
Compute grows with the square of the input length
Loss functions and optimization
Here you look at Loss functions and optimization alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
A loss function expresses as a number how far the prediction of the model is from the answer. An optimizer is the algorithm that moves the weights little by little in the direction that minimizes that loss. Imagine standing on a mountain in fog β feeling the slope (gradient) under your feet and taking one step at a time (the size of the learning rate) in the steepest downhill direction is gradient descent.
train_loop.pyPYTHON
import torch.nn as nnimport torch.optim as optimmodel = SimpleMLP(in_dim=10, hidden_dim=32, out_dim=1)criterion = nn.MSELoss()optimizer = optim.AdamW(model.parameters(), lr=1e-3)for epoch in range(100): optimizer.zero_grad() # reset the previous gradients y_pred = model(x_train) loss = criterion(y_pred, y_train) loss.backward() # compute gradients by backpropagation optimizer.step() # update the weights along the gradient if epoch % 20 == 0: print(f"epoch {epoch}: loss={loss.item():.4f}")
Category
Common choice
Use
Regression loss
MSE (mean squared error)
Predicting continuous values (price, temperature, and so on)
Classification loss
Cross-Entropy
Class classification, next-token prediction in LLMs
Optimizer
SGD
Simple and light, sensitive to learning rate tuning
Optimizer
Adam / AdamW
The default choice for most deep learning and LLM training
How a model is graded: evaluation metrics
Here you look at How a model is graded: evaluation metrics alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
"Is a model with 95% accuracy a good model?" The answer is "it depends". On data where only 1% of transactions are fraudulent, answering "always normal" already gives 99% accuracy. You only see real performance when you pick the metric that fits the problem type.
Share of all cases it got right β trustworthy only when the classes are balanced
Classification
Precision
Share of predicted "positives" that are truly positive β matters when false positives are costly
Classification
Recall
Share of true positives the model found β matters when misses are unacceptable (disease, fraud)
Classification
F1 Score
The harmonic mean of Precision and Recall β the balance of the two in one number
Classification
ROC-AUC
Overall classification performance across thresholds (closer to 1 is better)
Regression
MAE / RMSE
Average difference between predicted and actual values (RMSE is more sensitive to large errors)
Regression
RΒ² (coefficient of determination)
How much of the variance in the data the model explains (closer to 1 is better)
Overfitting and regularization
Overfitting and regularization is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.
Overfitting is when a model memorizes even the noise in the training data and then performs worse on new data. Training loss keeps falling while validation loss starts to rise again: that is the signal. It is also explained by the bias-variance tradeoff β a model that is too simple has high bias and underfits, and a model that is too complex has high variance and overfits.
Technique
How it works
Train/Val/Test split
Detects the onset of overfitting early with validation data
K-Fold cross-validation (K-Fold CV)
Splits the data into K parts and rotates which one validates, making the evaluation more reliable
Dropout
Randomly switches off some neurons during training to prevent reliance on specific neurons
L1/L2 regularization
Penalizes weight magnitude to keep the model simple
Early Stopping
Stops training early when validation loss no longer improves
Data augmentation
Transforms and expands the training data to improve generalization
Hyperparameter tuning
Here you look at Hyperparameter tuning alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Unlike weights, which are set automatically by training, hyperparameters are settings a person has to choose in advance (learning rate, hidden layer size, batch size, number of trees, and so on). Finding good hyperparameters alone can change performance a great deal.
Lays out every possible combination on a grid and searches exhaustively
When the search space is small
Random Search
Searches by sampling combinations at random
More efficient than Grid Search, the default choice in most cases
Bayesian Optimization
Picks the next point to try intelligently based on previous results
When training is expensive and trials must be spent carefully
A real-world ML workflow
This section covers A real-world ML workflow from a practical perspective. Instead of memorizing the concept, focus on the situations in which you would reach for it.
In practice, the data pipeline and validation strategy often decide the outcome more than the model architecture does. Use the order below as your basic checklist.
Overview of deployment and MLOps
Overview of deployment and MLOps is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.
Training a model and operating it reliably in production are entirely different problems. MLOps is the practice that systematizes model deployment, monitoring, and retraining.
Concept
Description
Model serving
Wrapping a trained model as an API that answers real-time prediction requests
Data drift
The distribution of production input data moving away from what it was at training time
Model drift
Model performance decaying gradually over time because of drift and similar causes
A/B testing / shadow deployment
Applying a new model to a slice of traffic, or comparing results without affecting real use
Versioning
Versioning data, code, and model artifacts together for reproducibility
ML Fundamentals: frequently asked questions
What is the difference between supervised and unsupervised learning?
Supervised learning trains on data that has both inputs and correct answers, then predicts the answer for new inputs. Spam classification and price prediction are examples. Unsupervised learning finds structure in data without answers; clustering customers into similar groups and dimensionality reduction are typical.
What is overfitting and how do I prevent it?
Overfitting is when a model memorizes accidental patterns in the training data, so it does well on the training data and worse on new data. You detect it by checking performance on validation data that was not used for training, and reduce it with more data, a simpler model, regularization, or early stopping.
Why split data into training, validation, and test sets?
The training set fits the model, the validation set is for choosing hyperparameters and models, and the test set estimates real performance once at the end. If you adjust the model while looking at the test set, the model adapts to it and the measured performance looks better than it is. That is why the test set stays untouched until the end.