Complete guide to Python basics + intermediate
Python Basics Guide
From installing Python through syntax, data structures, comprehensions/generators, decorators, context managers, advanced type hints, concurrency (threading/multiprocessing/asyncio), package management (uv), and testing — everything from first steps to a practical intermediate level, in one place.
Contents 16 sections
- Installation & runtime environment
- Variables and basic syntax
- Data structures
- Comprehensions & generators
- Functions and type hints
- Decorators
- Modules and packages
- File I/O
- Context managers & resource management
- Exception handling
- Classes and object orientation
- Advanced type hints
- Using the standard library
- Concurrency basics (threading/multiprocessing/asyncio)
- Advanced package management (uv)
- Testing and project structure
Installation & runtime environment
With Python, the norm is a separate virtual environment per project rather than using the interpreter installed system-wide. At first, installing from python.org and using venv is enough, and once you start handling several projects it is better to manage dependencies and lock files with uv.
BASH
# Check the version
python --version
# Create a project
mkdir python-basic && cd python-basic
python -m venv .venv
# macOS/Linux
source .venv/bin/activate
# Windows PowerShell
.venv\Scripts\Activate.ps1
# Install packages
python -m pip install --upgrade pip
python -m pip install requests pytest- Use a separate virtual environment for each project.
- Naming your script python.py can clash with standard modules.
- In practice, leave a reproducible environment behind with pyproject.toml and a lock file.
Variables and basic syntax
Python delimits blocks by indentation, and variables do not need a declared type. In intermediate and more advanced code, though, it is good to use type hints as well, to improve IDE autocompletion, reviews, and test quality.
basics.pyPYTHON
name = "AI DevOps"
score = 87
active = True
if score >= 90:
grade = "A"
elif score >= 80:
grade = "B"
else:
grade = "C"
for index in range(3):
print(index, name, grade)
def format_user(name: str, score: int) -> str:
return f"{name}: {score} points"
print(format_user(name, score))| Concept | Example | Practical guideline |
|---|---|---|
| Strings | `str`, f-string | Use f-strings by default for building strings. |
| Numbers | `int`, `float` | For money calculations, consider Decimal instead of float. |
| Conditionals | `if / elif / else` | Move complex conditions into functions. |
| Loops | `for`, `while` | Use enumerate when you need the index. |
Data structures
Productivity in Python comes from handling list, tuple, dict, and set naturally. Deciding first whether the data needs order, whether duplicates are allowed, and whether key-based lookup is needed leads you to the right data structure.
collections_demo.pyPYTHON
users = [
{"id": 1, "name": "Ada", "role": "admin"},
{"id": 2, "name": "Linus", "role": "developer"},
{"id": 3, "name": "Grace", "role": "developer"},
]
names = [user["name"] for user in users]
developers = [u for u in users if u["role"] == "developer"]
user_by_id = {user["id"]: user for user in users}
roles = {user["role"] for user in users}
print(names)
print(user_by_id[2]["name"])
print(sorted(roles))| Data structure | Traits | Typical uses |
|---|---|---|
| list | Ordered, duplicates allowed | Lists, sorting, filtering |
| tuple | Immutable | Coordinates, fixed return values |
| dict | Key-value lookup | Settings, indexes, JSON-shaped data |
| set | Removes duplicates | Tags, permissions, intersection/difference |
Comprehensions & generators
A comprehension is syntax that compresses a loop into one line, and a generator produces results lazily, only as many as needed, instead of loading them all into memory at once (lazy evaluation). When handling large volumes of data, using a generator instead of a list cuts memory use considerably.
generators_demo.pyPYTHON
# list/dict/set comprehensions — build the whole thing in memory immediately
squares = [n * n for n in range(10)]
even_squares = {n: n * n for n in range(10) if n % 2 == 0}
# Generator function — returns one value at each yield and pauses
def read_large_file(path: str):
with open(path, encoding="utf-8") as f:
for line in f:
yield line.strip()
# Generator expression — using () instead of [] makes it lazy
total = sum(n * n for n in range(1_000_000)) # sums on the fly without building a list
# Handling infinite/large streams with itertools
from itertools import islice
def counter():
n = 0
while True:
yield n
n += 1
first_five = list(islice(counter(), 5))
print(first_five)| Kind | When evaluated | Memory | Re-iteration |
|---|---|---|---|
| List comprehension `[...]` | Builds everything immediately | As much as the whole data | Any number of times |
| Generator expression `(...)` | Produces one value each time one is requested | Only one item at a time | One-shot (cannot be reused once exhausted) |
- A generator cannot be iterated again once exhausted — if you need to iterate several times, convert it to a list or call the generator function again.
- When processing a large file, do not read it all with file.readlines(); the file object itself is already an iterator that returns one line at a time lazily, so iterate with `for line in f:`.
Functions and type hints
The function is the most important unit of separation in Python code. Stating the input and output types lets even a small script grow into a maintainable module.
functions.pyPYTHON
from dataclasses import dataclass
@dataclass
class Order:
id: int
amount: float
paid: bool = False
def total_paid(orders: list[Order]) -> float:
return sum(order.amount for order in orders if order.paid)
def normalize_name(value: str) -> str:
return value.strip().title()
orders = [Order(1, 12000, True), Order(2, 8000, False)]
print(total_paid(orders))- Keep functions small so each does one thing.
- Do not put a mutable object such as an empty list directly in a default argument.
- If the return value is complex, consider a dataclass or Pydantic model rather than a dict.
Decorators
A decorator is syntax that wraps a function to slot in common logic (logging, caching, permission checks, retries) before and after it runs. Do not forget to preserve the name and docstring of the original function with `functools.wraps`.
decorators.pyPYTHON
import time
import functools
def retry(times: int = 3):
"""A decorator that takes arguments is a factory that wraps the function one level further."""
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
last_error: Exception | None = None
for attempt in range(1, times + 1):
try:
return func(*args, **kwargs)
except Exception as exc:
last_error = exc
print(f"[retry {attempt}/{times}] {func.__name__} failed: {exc}")
raise last_error
return wrapper
return decorator
@retry(times=3)
def call_flaky_api() -> str:
if time.time() % 2 < 1:
raise ConnectionError("Temporary network error")
return "ok"
# Built-in decorators
class Circle:
def __init__(self, radius: float) -> None:
self._radius = radius
@property
def area(self) -> float:
return 3.14159 * self._radius ** 2
@staticmethod
def unit() -> "Circle":
return Circle(1)
@functools.lru_cache(maxsize=None)
def fibonacci(n: int) -> int:
return n if n < 2 else fibonacci(n - 1) + fibonacci(n - 2)| Decorator | Use |
|---|---|
| `@property` | Makes a method accessible like an attribute (getter) |
| `@staticmethod` | A function with no self that only borrows the class namespace |
| `@classmethod` | Receives the class itself (cls) as the first argument — alternative constructors and so on |
| `@functools.lru_cache` | Caches the results of calls with the same arguments to avoid recomputation |
| `@functools.wraps` | Preserves the metadata of the original function inside a custom decorator |
- If you catch an exception inside a decorator, do not just log it and swallow it silently; always raise it again after handling, or express the failure explicitly.
- A decorator that takes arguments ends up wrapping the function three levels deep (factory → decorator → wrapper) — confusing at first, but learn it using the retry example above as a template.
Modules and packages
When a single file grows large, split it into modules by feature. Clear import paths make testing and deployment easier, and it is a good habit to put the execution entry point under `if __name__ == "__main__"`.
project layoutTEXT
python-basic/
pyproject.toml
src/
app/
__init__.py
calculator.py
cli.py
tests/
test_calculator.pysrc/app/calculator.pyPYTHON
def add(a: int, b: int) -> int:
return a + bsrc/app/cli.pyPYTHON
from app.calculator import add
def main() -> None:
print(add(2, 3))
if __name__ == "__main__":
main()File I/O
When working with files you need to think about encoding, paths, and exception handling together. Default to UTF-8 for text, and build paths with pathlib to reduce differences between operating systems.
files.pyPYTHON
from pathlib import Path
import json
data_dir = Path("data")
data_dir.mkdir(exist_ok=True)
users = [{"id": 1, "name": "Ada"}, {"id": 2, "name": "Linus"}]
path = data_dir / "users.json"
path.write_text(json.dumps(users, ensure_ascii=False, indent=2), encoding="utf-8")
loaded = json.loads(path.read_text(encoding="utf-8"))
for user in loaded:
print(user["id"], user["name"])Context managers & resource management
The `with` statement safely releases resources that must be cleaned up, such as files, network connections, and locks, even when an exception occurs. You can implement `__enter__`/`__exit__` yourself, or create one simply from a single generator function with `contextlib.contextmanager`.
context_managers.pyPYTHON
import time
from contextlib import contextmanager
# 1) Class-based context manager
class Timer:
def __enter__(self):
self._start = time.perf_counter()
return self
def __exit__(self, exc_type, exc_value, traceback):
elapsed = time.perf_counter() - self._start
print(f"Elapsed: {elapsed:.3f}s")
return False # do not swallow the exception; let it propagate
with Timer():
time.sleep(0.1)
# 2) Generator-based context manager — the same job in far less code
@contextmanager
def timer():
start = time.perf_counter()
try:
yield
finally:
print(f"Elapsed: {time.perf_counter() - start:.3f}s")
with timer():
time.sleep(0.1)
# 3) Cleaning up several resources at once
with open("in.txt", encoding="utf-8") as src, open("out.txt", "w", encoding="utf-8") as dst:
dst.write(src.read().upper())- If `__exit__` returns True, the exception that occurred is silently suppressed (swallowed) — unless that is what you intend, always return False or None.
- For "exceptions that are fine to ignore", stating the intent in code, as in `contextlib.suppress(FileNotFoundError)`, reads better than `try/except: pass`.
Exception handling
Exception handling is not a tool for hiding errors but a device for expressing failure clearly. An except that is too broad hides problems, so catch only the exceptions you can anticipate and pass a meaningful message to the caller.
errors.pyPYTHON
from pathlib import Path
def read_config(path: str) -> dict[str, str]:
file = Path(path)
if not file.exists():
raise FileNotFoundError(f"config not found: {path}")
result: dict[str, str] = {}
for line in file.read_text(encoding="utf-8").splitlines():
if not line.strip() or line.startswith("#"):
continue
key, value = line.split("=", 1)
result[key.strip()] = value.strip()
return result
try:
config = read_config("app.env")
except FileNotFoundError as exc:
print(f"Configuration error: {exc}")- If you catch an exception, record the cause in a log or message.
- Do not silently swallow unrecoverable errors; raise them again.
- Give external API calls a timeout and a retry policy.
Classes and object orientation
In Python, use a class when state and behavior need to be bundled together. A dataclass is enough for plain data, and favoring composition and small objects over complex inheritance is better for maintainability.
oop.pyPYTHON
from dataclasses import dataclass
@dataclass
class Account:
owner: str
balance: int = 0
def deposit(self, amount: int) -> None:
if amount <= 0:
raise ValueError("amount must be positive")
self.balance += amount
def withdraw(self, amount: int) -> None:
if amount > self.balance:
raise ValueError("insufficient balance")
self.balance -= amount
account = Account("Ada")
account.deposit(10000)
account.withdraw(3000)
print(account.balance)Advanced type hints
Beyond the simple type hints covered in the basic syntax, the tools of the `typing` module let you express more precise contracts. Hooking a static type checker such as mypy or pyright into CI catches type errors ahead of time, without running the code.
typing_advanced.pyPYTHON
from typing import TypedDict, Protocol, Literal
# State that a value may be absent (in Python 3.10+, str | None instead of Optional[str])
def find_user(user_id: int) -> str | None:
return "Ada" if user_id == 1 else None
# One of several types
def parse_amount(value: int | str) -> float:
return float(value)
# A dict type with a fixed structure, like JSON
class UserPayload(TypedDict):
id: int
name: str
role: Literal["admin", "developer", "viewer"]
def greet(user: UserPayload) -> str:
return f"Welcome, {user['name']} ({user['role']})"
# Structural typing: "having this method is enough", with no inheritance
class SupportsTotal(Protocol):
def total(self) -> float: ...
def print_total(item: SupportsTotal) -> None:
print(f"Total: {item.total()}")| Tool | Use |
|---|---|
| `X | None` (`Optional[X]`) | States that a value may be absent |
| `A | B` (`Union[A, B]`) | Allows one of several types |
| `TypedDict` | States the key and value types of a dict, like a JSON struct |
| `Protocol` | Structural typing that only requires the needed methods, with no inheritance |
| `Literal["a", "b"]` | Restricts to one of a fixed set of values |
- In Python 3.10+ codebases, the `str | None` and `int | str` notation is more widely used than `Optional[str]` and `Union[int, str]`.
- Type hints are not enforced at runtime — at boundaries where values really have to be validated, such as external input (API requests, files), use a runtime validation library such as Pydantic as well.
Using the standard library
The Python standard library is the basic toolkit for automation and data processing. Knowing just datetime, pathlib, json, csv, argparse, and logging is enough to build small operational scripts reliably.
report.pyPYTHON
import argparse
import csv
import logging
from pathlib import Path
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
parser = argparse.ArgumentParser()
parser.add_argument("--file", default="orders.csv")
args = parser.parse_args()
path = Path(args.file)
total = 0
with path.open(encoding="utf-8", newline="") as f:
for row in csv.DictReader(f):
total += int(row["amount"])
logging.info("total amount: %s", total)Concurrency basics (threading/multiprocessing/asyncio)
Because of the GIL (Global Interpreter Lock), Python cannot have several threads executing Python bytecode at the same time within one process. So the tool has to be chosen according to "what is being waited on right now" — asyncio or threading for work that waits on I/O, and multiprocessing for CPU-heavy computation.
concurrency_demo.pyPYTHON
import asyncio
import time
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
# I/O bound: wait in several threads at once
def download(url: str) -> str:
time.sleep(0.5) # simulates waiting on the network
return f"downloaded: {url}"
with ThreadPoolExecutor(max_workers=4) as pool:
results = list(pool.map(download, ["a.com", "b.com", "c.com"]))
# CPU bound: true parallel computation across several processes
def heavy_square(n: int) -> int:
return sum(i * i for i in range(n))
with ProcessPoolExecutor(max_workers=4) as pool:
results = list(pool.map(heavy_square, [10_000_000] * 4))
# asyncio: handle a great many waits at once in a single thread
async def fetch(session_id: int) -> str:
await asyncio.sleep(0.5) # must be the asyncio sleep — time.sleep stalls the event loop
return f"session {session_id} done"
async def main() -> None:
results = await asyncio.gather(*(fetch(i) for i in range(10)))
print(results)
asyncio.run(main())| Situation | Suitable tool | Why |
|---|---|---|
| Making many network requests at once (I/O bound) | asyncio | Waits on thousands at once in a single thread, with the lowest context switching cost |
| File/DB I/O with existing synchronous libraries | threading | The GIL is released while waiting on I/O, so several threads can wait at the same time |
| Image processing, numeric computation (CPU bound) | multiprocessing | Each process has its own interpreter and GIL, giving true parallel execution |
- Using multiprocessing for I/O bound work only adds process creation overhead for no gain — first work out "is this task waiting, or computing?".
- Using `time.sleep` or synchronous requests calls as-is inside asyncio code stalls the whole event loop — always use the asynchronous versions, such as `asyncio.sleep` and the async client of `httpx`.
Advanced package management (uv)
If you started with just venv + pip, then as the project grows you come to need reproducible dependency locking and a standardized build. `pyproject.toml` is now the standard configuration file replacing `setup.py`, and `uv` builds on it to unify dependency installation, virtual environments, and lock management in a single, very fast CLI.
pyproject.tomlTOML
[project]
name = "python-basic"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"requests>=2.32",
]
[dependency-groups]
dev = ["pytest>=8.0"]uv commandsBASH
# Initialize a project (creates the venv + pyproject.toml automatically)
uv init python-basic && cd python-basic
# Add a dependency (installs + updates pyproject.toml + uv.lock together)
uv add requests
uv add --dev pytest
# Reproduce exactly the same environment from the lock
uv sync
# Run directly in the locked environment without activating the venv yourself
uv run python -m app.cli
uv run pytest| Tool | Characteristics |
|---|---|
| pip + requirements.txt | The simplest, but with no proper lock file (imitated with pip freeze) |
| pip-tools | Generates a lock file from requirements.in → requirements.txt |
| Poetry | pyproject.toml based; integrates the lock file with build and publishing |
| uv | Written in Rust and very fast; unifies venv, install, and lock in one CLI — spreading quickly as the new standard |
- Always commit the lock file (uv.lock, poetry.lock) to git — most "it works on my machine" problems come from differences in unlocked dependency versions.
- When starting a new project, starting straight with uv is recommended unless you have a specific reason not to — installs are far faster than pip+venv, and lock management is solved in the same step.
Testing and project structure
The fastest way to raise your fundamentals to a practical level is to write tests alongside the code. Pinning down calculation logic, parsing logic, and exception cases with tests makes refactoring easier, and once you know fixtures and mocks you can safely verify code with external dependencies too.
tests/test_calculator.pyPYTHON
import pytest
from app.calculator import add
def test_add():
assert add(2, 3) == 5
@pytest.mark.parametrize("a,b,expected", [
(1, 2, 3),
(-1, 1, 0),
(0, 0, 0),
])
def test_add_cases(a: int, b: int, expected: int):
assert add(a, b) == expectedtests/test_client.pyPYTHON
import pytest
from app.weather import WeatherClient
@pytest.fixture
def client() -> WeatherClient:
# A reusable piece of setup created fresh for each test
return WeatherClient(api_key="test-key")
def test_parses_temperature(client: WeatherClient, monkeypatch: pytest.MonkeyPatch):
def fake_get(url: str) -> dict:
return {"temp": 21.5}
# Replace the real network call with a fake function
monkeypatch.setattr(client, "_get", fake_get)
assert client.current_temperature("Seoul") == 21.5| Concept | Role |
|---|---|
| `fixture` | Creates the setup each test needs (DB connections, objects) and handles the cleanup too |
| `monkeypatch` / `unittest.mock` | Replaces hard-to-control dependencies such as external APIs, time, and environment variables with fakes |
| `pytest.mark.parametrize` | Verifies the same logic repeatedly with several input values |
| `pytest --cov=app` | Shows the lines of code the tests do not cover, through coverage.py integration |
- Collect test files under tests/, and when fixing a bug, first add a test that reproduces it so the same problem does not come back.
- Tests that call a real network or DB are slow and flaky — in unit tests replace external dependencies with monkeypatch/mock, and split real integration checks into separate integration tests.
- Building the habit of function-level testing, fixtures and mocks included, before moving on to FastAPI or Django makes the TestClient of those frameworks much easier to understand.