Complete guide to Linux system architecture & server operations
🐧 Linux Complete Guide
From the moment Linux boots to the moment a service handles a request — kernel and user space structure, the boot process, the filesystem hierarchy (FHS), the permission model, the process lifecycle, systemd, the network stack, storage and memory, logging, security hardening, performance diagnosis, and the kernel foundations of containers (namespaces and cgroups) — a complete guide, with diagrams, to understanding Linux as one system.
Beginner
Updated Oct 1, 2026
~15 min read
16 sections
15 code examples
Web IDE lab included
🐧
Linux Web IDE
Run code in the browser with nothing to install and learn through step-by-step examples.
From the moment Linux boots to the moment a service handles a request — kernel and user space structure, the boot process, the filesystem hierarchy (FHS), the permission model, the process lifecycle, systemd, the network stack, storage and memory, logging, security hardening, performance diagnosis, and the kernel foundations of containers (namespaces and cgroups) — a complete guide, with diagrams, to understanding Linux as one system. Rather than listing concepts, this guide is organized so you can follow the content in the order you have to make decisions on a real project.
Key perspective
Infrastructure / operations
Rather than memorizing install commands, understand the order in which traffic, runtime, observability, and incident response connect.
Understanding system architecturesystemd & process operationsDesigning network and storage layoutsSecurity hardening & performance troubleshooting
Architecture diagrams
To make what you read stick, it helps to capture the flow as a picture first. The two diagrams below are reference maps you can keep coming back to while learning Linux.
Learning flow
Rendering diagram…
Architecture view
Rendering diagram…
Architecture & the boot process
When you first open Linux, the big picture matters more than individual commands. This section starts with the problems the concepts ahead were created to solve.
Linux is divided into the kernel, which controls the hardware directly, and user space, which sits on top and reaches the hardware only through the system calls the kernel provides. When an application opens a file or creates a socket, it is always the kernel that actually operates the disk or NIC, and the application just gets the result back through a system call. Once you understand this boundary, you start to see "which part of the kernel a given command touches", and it becomes much easier to see how the process, filesystem, and network sections that follow connect to each other.
Rendering diagram…
Rendering diagram…
Component
Role
Firmware (BIOS/UEFI)
Runs first when power comes on, does the initial hardware checks, and finds the bootloader
Bootloader (GRUB)
Chooses which kernel to boot with which options — handles multi-boot and passing kernel parameters
Kernel
Arbitrates every hardware resource: process scheduling, memory management, filesystems, networking, and more
init (PID 1)
The first user space process the kernel runs — the ancestor of every later process
Filesystem hierarchy (FHS)
Here you look at Filesystem hierarchy (FHS) alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Whatever the distribution, Linux follows the FHS (Filesystem Hierarchy Standard), a common rule that "this file belongs under this directory". Thanks to this rule, the sense of paths you learn on Ubuntu carries over almost unchanged to CentOS or Alpine, and because configuration files, logs, and executables do not get mixed up, automation scripts can be reused regardless of distribution.
Rendering diagram…
Directory
Purpose
Practical tip
/etc
Daemon and system configuration files
Before changing configuration, always back up to .bak or version it with Git
/var/log
Application and system logs
A logrotate configuration is essential to avoid a full disk
/proc
Exposes running processes and kernel parameters as files
The state of a specific process can be read directly from /proc/[PID]/status
/sys
Information on devices and drivers the kernel has detected
Kernel parameters can be adjusted at runtime by changing sysfs values
/opt
Standalone software that does not follow the standard FHS paths
The proper place for manually installed software not managed by the package manager
Permission & ownership model
Here you look at Permission & ownership model alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Every file in Linux has read (r), write (w), and execute (x) permissions for each of three subjects: the owner, the owning group, and everyone else (other). When a process tries to access a file, the kernel applies only the one of these three that the process "matches first" — the point often missed in practice is that if you are the owner, only the owner permissions apply, however broad the group and other permissions are.
Rendering diagram…
BASH
# Change the owner/groupchown deploy:deploy /var/www/myappchgrp developers shared-folder/# Change permissions — numeric (absolute) notationchmod 755 script.sh # rwxr-xr-xchmod 644 config.yml # rw-r--r--chmod -R 750 /var/www/myapp # apply recursively to the whole directory# umask, which decides the default permissions of new filesumask # check the current value (e.g. 0022)umask 0027 # files created from now on block group write and all access by others by default# When fine-grained access control is needed — where standard rwx is not enoughsetfacl -m u:alice:rx /var/www/myappgetfacl /var/www/myapp
Symbol
Number
Meaning (file)
Meaning (directory)
r
4
Read the file contents
List the files in the directory (ls)
w
2
Modify the file contents
Create and delete files in the directory
x
1
Run the file as a program
Enter the directory (cd)
setuid (4000)
-
Runs with the privileges of the owner (e.g. passwd)
Not applicable
setgid (2000)
-
Runs with the privileges of the owning group
New files in the directory inherit the same group
sticky bit (1000)
-
Not applicable
Only you can delete your own files (e.g. /tmp)
Essential commands
Here you look at Essential commands alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
A good share of server incidents is decided by how quickly you look at the logs and disk usage. Get into the habit of first finding old or large files with find/du and filtering logs in real time with grep/awk/sed, and the time to find the cause drops sharply.
BASH
# Files & directoriesls -lah # detailed listingfind / -name "*.log" -mtime +7 # logs older than 7 daysdu -sh /var/log/* | sort -hr # sort by sizetail -f /var/log/nginx/access.log# Text processinggrep -r "ERROR" /var/log --include="*.log"awk '{print $1}' access.log | sort | uniq -c | sort -rnsed -i 's/old/new/g' config.txt# Apply a command to many files at oncefind . -name "*.tmp" | xargs rm -f
Process management & lifecycle
Here you look at Process management & lifecycle alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Every process is born when an already running parent process clones itself with fork() and the clone is replaced by the program to actually run with exec(). So every process has a PID (its own number) and a PPID (the number of its parent), and following this lineage eventually leads to PID 1 (systemd). Checking which process holds a port and how much CPU/memory it uses is the first step of incident response.
Rendering diagram…
BASH
# Monitoringtop / htopps aux | grep nginxps -o pid,ppid,cmd -p 1234 # check the PID/PPID lineagelsof -i :8080 # process using the port# Sending signalskill -TERM 1234 # request a graceful exit (default)kill -9 1234 # force kill (SIGKILL)pkill -f "node server.js"# Adjusting prioritynice -n 10 ./batch-job.sh # start with low priorityrenice -n 5 -p 1234 # change the priority of a running process
Signal
Default action
Use
SIGHUP (1)
Terminate (daemons often redefine it as reload)
Terminal hangup notice — nginx and others reload their configuration on this signal
SIGINT (2)
Terminate
Ctrl+C — the user interrupts the foreground process
SIGTERM (15)
Terminate (the process can intercept it and clean up)
The default signal of kill — a request to "exit gracefully"
SIGKILL (9)
Immediate forced termination (cannot be intercepted)
The last resort when a process does not respond
SIGSTOP (19)
Pause (cannot be intercepted)
Stops a process, similar to Ctrl+Z
systemd & service management
Here you look at systemd & service management alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
systemd runs as PID 1 and manages the whole boot process and the service lifecycle afterward. Each service is declared as a "unit", and once the unit file states "what must be ready before it starts (After)" and "which target it belongs to (WantedBy)", systemd computes the dependency graph and boots as fast as it can, in parallel.
# Apply after writing the unit filesystemctl daemon-reload# Register for automatic start at boot + run nowsystemctl enable --now myapp# Check the status & logssystemctl status myappjournalctl -u myapp -f --since "1 hour ago"# Run in the background (for temporary debugging — unsuitable for production services)nohup ./server &tmux new -s work
Directive
Meaning
After=
Units that must start before this one (this unit still tries to start if they fail)
Requires=
Units this unit depends on — if a dependency fails, this unit stops too
Restart=on-failure
Restart automatically on abnormal exit
WantedBy=multi-user.target
Registers it to start along with this target (enable creates a symbolic link)
Network stack & firewall
Here you look at Network stack & firewall alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
When a service does not respond, first check whether the port is really open and whether a firewall is blocking the traffic. A packet entering the kernel passes through several netfilter chains in order, and iptables is the tool for attaching rules to each chain to accept (ACCEPT), drop (DROP), or translate (NAT) packets.
Rendering diagram…
BASH
# Check connectivityss -tlnp # list listening portscurl -v https://api.example.comnc -zv host 443 # test a port connection# iptables (firewall) — matched in order from the topiptables -A INPUT -p tcp --dport 80 -j ACCEPTiptables -A INPUT -p tcp --dport 443 -j ACCEPTiptables -A INPUT -p tcp --dport 22 -j ACCEPTiptables -A INPUT -j DROP # block everything not explicitly listed (add only after allowing SSH)# Traffic analysistcpdump -i eth0 port 80 -w capture.pcap
Chain
When the packet passes through
PREROUTING
After the packet arrives, before the routing decision (used for DNAT in port forwarding)
INPUT
Packets destined for this host itself
FORWARD
Packets passing through this host to another host (when acting as a router or NAT gateway)
OUTPUT
Packets generated on this host and going out
POSTROUTING
Just before the packet actually leaves (used for SNAT/MASQUERADE)
Storage (LVM) & memory/swap
Here you look at Storage (LVM) & memory/swap alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
A physical disk is divided into partitions, and layering LVM (Logical Volume Manager) on top lets you pool several disks into one storage pool and grow volume sizes flexibly even while mounted. Memory, too, is split between the "virtual memory" a process sees and the actual "physical memory (RAM)", so the kernel responds automatically when RAM runs short.
Rendering diagram…
Rendering diagram…
BASH
# Disk & mount overviewlsblk # view the block device treedf -h # usage of mounted filesystems# LVM — from creating the PV/VG/LV to online expansionpvcreate /dev/sdbvgcreate data-vg /dev/sdblvcreate -L 50G -n data-lv data-vgmkfs.ext4 /dev/data-vg/data-lvmount /dev/data-vg/data-lv /data# Expand capacity with no service interruptionlvextend -L +20G /dev/data-vg/data-lvresize2fs /dev/data-vg/data-lv# Check memory & swapfree -hswapon --showvmstat 1 5 # 5 samples at 1-second intervals — si/so columns that stay non-zero mean heavy swap use
free -h field
Meaning
available
Memory a new process can really use right away (including reclaimable cache) — judge free memory by this value
buff/cache
Cache the kernel uses temporarily to speed up file I/O — reclaimed at once when needed, so do not mistake it for "in use"
swap used
The total pages pushed out to disk for lack of physical memory — steady growth is a sign to add memory
Logging system (journald)
Here you look at Logging system (journald) alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
On systemd-based distributions, the standard output of services (stdout/stderr) and kernel messages all collect in journald and are stored in a binary format. Unlike text log files, journald is structured with metadata such as the unit name, priority, and timestamp, so instead of grep you can pull out exactly the logs you want with the filter options of journalctl.
Rendering diagram…
BASH
# Follow the logs of one specific service in real timejournalctl -u myapp -f# The last hour, ERROR and above onlyjournalctl -u myapp --since "1 hour ago" -p err# Kernel messages since bootjournalctl -k -b# Set a cap on the disk space journald uses# /etc/systemd/journald.conf# SystemMaxUse=500M
Here you look at Security hardening alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
A Linux server in its default installed state often has root password login open and unnecessary services running. Locking down just three things properly — SSH, sudo, and the firewall — stops most brute force attacks from automated scanners, and blocking repeated failures automatically with a tool such as fail2ban means nobody has to watch the logs all the time.
/etc/ssh/sshd_config (excerpt)TEXT
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
AllowUsers deploy admin
Port 22
# Grant least privilege per command in sudoersecho 'deploy ALL=(ALL) NOPASSWD: /usr/bin/systemctl restart myapp' \ > /etc/sudoers.d/deploy-myappvisudo -c # validate the syntax before applying# Check the SELinux status (RHEL family)getenforcesestatus
Item
Risk in the default state
Recommended action
SSH root login
Anyone who knows the password can log in directly as root
PermitRootLogin no + allow key-based authentication only
SSH password authentication
Exposed to brute-force attacks
PasswordAuthentication no, switch to public key authentication
sudo privileges
A user can run every command with root privileges
Grant only the needed commands with least privilege in /etc/sudoers.d/
Repeated login failures
Automated bots can keep trying indefinitely
Use fail2ban to block the IP automatically after a set number of failures
Limits of the DAC permission model
Once root is breached, unlimited access to the whole system
Further restrict the reach of each process with SELinux/AppArmor (MAC)
Performance diagnosis & troubleshooting
Here you look at Performance diagnosis & troubleshooting alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
The single symptom "the server is slow" does not lead straight to a cause. A systematic approach — checking the three resources CPU, memory, and disk I/O in turn to narrow down where the bottleneck is — is far faster than checking this and that by feel.
Rendering diagram…
BASH
uptime # check the load averagevmstat 1 5 # 5 samples at 1-second intervals — watch the r and si/so columnsiostat -x 1 5 # %util and await per disksar -n DEV 1 5 # network interface trafficmpstat -P ALL 1 5 # CPU usage per core (reveals a single core stuck at 100%)
Tool
Key values to watch
Interpretation
uptime
load average (1/5/15 minutes)
Consistently above the CPU core count signals a CPU shortage
vmstat 1
r (run queue), si/so (swap in/out)
r above the core count or non-zero si/so mean CPU or memory pressure respectively
iostat -x 1
%util, await
%util near 100% with a high await means the disk is the bottleneck
sar -n DEV 1
rxkB/s, txkB/s
Shows whether the NIC bandwidth is saturated
Kernel foundations of containers
Here you look at Kernel foundations of containers alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Docker and Kubernetes may look like magic that creates containers, but they are really a combination of two Linux kernel features — namespaces isolate the PIDs, network, and mount list a process sees, and cgroups limit the total CPU, memory, and I/O a process (group) can use. Understanding these two makes it clear what docker run --memory and Kubernetes resource limits actually configure under the hood.
Rendering diagram…
BASH
# Run a shell in new PID/network/mount namespaces — experience container isolation firsthandunshare --pid --net --mount --fork /bin/bashps aux # inside this shell, only it is visible, near PID 1# Set a memory cap on a process group directly with cgroup v2mkdir /sys/fs/cgroup/mygroupecho $$ > /sys/fs/cgroup/mygroup/cgroup.procs # move the current shell into this groupecho 100M > /sys/fs/cgroup/mygroup/memory.max # 100MB memory cap for the whole group# docker run --memory=100m does the same thing internally:# it puts the container process in a cgroup and sets memory.max.
Shell scripting
Here you look at Shell scripting alongside real code. Rather than copying the example as-is, read it with an eye on the inputs, the outputs, and the parts most likely to change.
Repetitive deploy and backup tasks should be turned into shell scripts, to remove the room for mistakes that comes with running them by hand. set -euo pipefail stops the script the moment an error occurs, preventing the commands after a failed step from carrying on in a bad state.
Linux practical design is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.
The key to Linux operations is understanding the process, file, network, and permission models rather than memorizing commands.
Decision point
Question to ask
Practical guideline
Boundaries
Which parts of the Linux code are most likely to change?
Separate input/output, configuration, external integrations, and the core rules.
State
Where is state created and where does it go away?
Make the owner and lifecycle of state visible in the code.
Failure
What does the caller receive on failure?
Decide the timeout, fallback, and error contract first.
Linux operating standards
This section covers Linux operating standards from a practical perspective. Instead of memorizing the concept, focus on the situations in which you would reach for it.
Look at CPU load, memory pressure, disk I/O, network sockets, and systemd service state together.
Linux verification strategy
Linux verification strategy is a point where the options diverge. Using the table to compare what each approach is for and how it differs in operation makes later decisions much easier.
Operational work should include a runbook, rollback commands, a dry-run, and an audit log.
Quality axis
How to verify
Definition of done
Correctness
Automate the normal/failure cases.
The core scenarios pass reproducibly.
Regression prevention
When fixing a bug, leave the same case behind as a test.
The same failure is not deployed again.
Operability
Check the logs, metrics, and alerts.
There is a path for tracing the cause when a problem occurs.
Linux: frequently asked questions
Do developers really need to know Linux?
Most servers, containers, and CI/CD pipelines run on Linux, so it pays to know the basic commands and layout. When a deployed application misbehaves, you end up logging in to a server and looking at processes, logs, disks, and the network. Working with files and directories, permissions, processes, and logs covers most situations.
How do I fix a "Permission denied" error?
First check the owner and permissions of the file with ls -l. If a script will not run, give it execute permission with chmod +x filename. If the file belongs to another user, change the owner with chown, or run the command with sudo for system files. Opening everything with chmod 777 creates security problems, so grant only the permissions that are needed.
How do I find out which process is using a port?
ss -ltnp or lsof -i :PORT shows the process and PID listening on the port. When you see "address already in use", find the process this way and, if it is not needed, stop it with kill PID. You need sudo to see details of processes owned by other users.
What should I check when the disk is full?
Use df -h to see which partition is full, then walk down the large directories with du -sh directory/*. On servers the cause is often log files, Docker images and volumes, or package caches. Rotate logs with logrotate, and for Docker check usage with docker system df and remove resources that are no longer used.