Skip to content

sumitsingh4411/devops

BranchesTags

Repository files navigation

🚀 The DevOps Engineer Roadmap

Beginner → Advanced · a complete, written course

Every topic below is a full lesson you read right here — not a pile of links. Work through the stages in order and go from zero to a hireable DevOps engineer.

Stages Lessons Resources Certs

Read on the live site

Important

New here? Start at Stage 1 and go in order — each stage builds on the ones before it. Prefer a polished, interactive version with progress tracking? Read it on the website →

🧭 How to read this

Each lesson ends with a > Takeaway callout. After the lesson, a collapsible Read more holds curated links, tagged by type:

Tag Meaning Tag Meaning
✅ FREE Free 🧪 LAB Hands-on lab
💳 PAID Paid course 📘 BOOK Book
🎓 CERT Certification 💬 COMM Community

Difficulty: 🟢 Beginner   🔵 Intermediate   🟣 Advanced

🗺️ The path — what comes after what

Four tracks, sixteen stages. Each stage lists the stages it builds on — those prerequisites are real. If you get stuck, the gap is almost always in a prerequisite.

🧱 Track 1 · Foundations

Start here. Every tool later runs on Linux, talks over a network, is automated by scripts, and lives in Git. Skipping this track is why people get stuck at Kubernetes.

# Stage Level Time Finish first
01 🧭 Foundations & the DevOps mindset 🟢 1–2 weeks —
02 🐧 Linux & operating systems 🟢 3–4 weeks Foundations & the DevOps mindset
03 🌐 Networking & protocols 🟢 2–3 weeks Linux & operating systems
04 🐍 Programming & scripting 🟢 4–6 weeks Linux & operating systems
05 🌿 Git & version control 🟢 2 weeks Foundations & the DevOps mindset

⚙️ Track 2 · Core DevOps

The daily job: build and test automatically, ship in containers, run them on Kubernetes, deliver safely, and operate a cloud.

# Stage Level Time Finish first
06 ⚙️ Build, test & Continuous Integration 🔵 3–4 weeks Git & version control, Programming & scripting
07 🐳 Containers & Docker 🔵 2–3 weeks Linux & operating systems, Programming & scripting
08 ☸️ Kubernetes & orchestration 🔵 6–8 weeks Containers & Docker, Networking & protocols
09 🚀 Continuous Delivery, GitOps & releases 🔵 2–3 weeks Build, test & Continuous Integration, Kubernetes & orchestration
10 ☁️ Cloud platforms 🔵 6–8 weeks Networking & protocols, Linux & operating systems

🚀 Track 3 · Advanced

What turns a junior into a senior: codified infrastructure, real observability, security, reliability, and platforms other teams build on.

# Stage Level Time Finish first
11 🏗️ Infrastructure as Code 🟣 4–5 weeks Cloud platforms, Programming & scripting
12 📈 Observability & monitoring 🟣 3–4 weeks Kubernetes & orchestration, Cloud platforms
13 🔒 Security & DevSecOps 🟣 3–4 weeks Cloud platforms, Build, test & Continuous Integration
14 🛡️ Reliability & SRE 🟣 3–4 weeks Observability & monitoring
15 🧱 Platform engineering & scale 🟣 Ongoing Kubernetes & orchestration, Infrastructure as Code, Observability & monitoring

🎯 Track 4 · Career

Turn the skills into an offer: a capstone portfolio, interview technique, and the certifications that actually pay off.

# Stage Level Time Finish first
16 🎯 Career: get hired 🟡 Ongoing —

🧭 Stage 1 · Foundations & the DevOps mindset

What DevOps actually is before you touch a tool

🟢 Beginner  ·  ⏱️ 1–2 weeks  ·  📚 5 lessons  ·  🧩 Foundations

DevOps is a culture and a set of practices, not a job title you install. Understand the 'why' — and how to work through this roadmap — so every tool later has a place to hang.

▶ Open Stage 01 on the live site →

What is DevOps (and what it is not)

The CALMS model, the DORA metrics, the infinity loop, and why the wall between Dev and Ops had to come down.

DevOps is not a tool you install or a title you hire — it is a way of working that removes the wall between the people who build software (Dev) and the people who run it (Ops). Before DevOps, developers threw code "over the wall" to operations, who were then blamed when it broke in production. DevOps says the same team owns an application from commit to production and beyond.

A useful mental model is CALMS:

  • Culture — shared ownership, blameless learning from failure.
  • Automation — if you do something twice, script it.
  • Lean — small batches, fast feedback, less work-in-progress.
  • Measurement — you can't improve what you don't measure.
  • Sharing — knowledge and tools flow across teams.

The industry measures DevOps performance with four DORA metrics: deployment frequency, lead time for changes, change failure rate, and mean time to recovery. Elite teams deploy many times a day and recover from incidents in under an hour — the two are not in tension, because automation and small changes make both safer.

Tip

Takeaway — every tool you'll learn next (Git, Docker, Kubernetes, Terraform) exists to serve one of the CALMS pillars. Keep asking "which problem does this solve?" and the tooling stops feeling random.

📚 Read more — go deeper (3)

How to use this roadmap (what comes after what)

The four tracks, why the order matters, how long it takes, and the mistakes that stall most beginners.

Most people fail at DevOps not from lack of ability but from learning things in the wrong order — jumping to Kubernetes before they can use a terminal, then wondering why nothing makes sense. This roadmap fixes that. Here is the sequence, and why each thing comes after the last.

The four tracks, in order:

  1. Foundations (stages 1–5) — Linux, networking, scripting, Git. Every later tool runs on Linux, talks over a network, is automated by scripts, and is stored in Git. Skipping this is why people get stuck later.
  2. Core DevOps (stages 6–10) — CI, containers, Kubernetes, delivery, and Infrastructure as Code. This is the daily job.
  3. Advanced (stages 11–15) — cloud, observability, security, reliability, platform. This is what turns a junior into a senior.
  4. Career (stage 16) — portfolio, interviews, certifications.

How to actually work through it:

  • Go in order. Each stage lists its prerequisites; they're real, not decorative.
  • Build, don't just read. Every stage has projects. A topic you haven't typed out isn't learned.
  • Tick topics off as you go — the progress bar is your build status.
  • Expect ~6–12 months part-time to reach job-ready (stages 1–11). That's normal.

Tip

Takeaway — depth beats breadth. Finish a stage, build its project, then move on. The order exists because each stage is the foundation of the next.

📚 Read more — go deeper (1)

SDLC, Agile & where DevOps fits

Waterfall vs Agile, Scrum and Kanban, sprints and backlogs — the process you'll actually work inside.

DevOps doesn't exist in a vacuum — it's how modern software teams deliver. To sit in one, you need the vocabulary of how software gets built.

The SDLC (Software Development Life Cycle) is the loop every product runs: plan → design → build → test → deploy → operate → learn, then repeat. The old Waterfall model ran each phase once, in sequence, over months — so mistakes surfaced only at the end, when they were most expensive.

Agile replaced it with short iterations: build a small slice, ship it, get feedback, adjust. Scrum is the most common flavour — work is split into sprints (usually two weeks), tracked on a backlog of user stories, with a daily stand-up and a retrospective to improve the process. Kanban is the lighter alternative: a continuous board of work with limits on how much is in progress.

Where does DevOps fit? Agile made development iterative, but releases were still slow and manual — a bottleneck at the end. DevOps extends that same fast-feedback principle across the deploy and operate half of the loop. Agile without DevOps just means you build fast and still ship slowly.

Tip

Takeaway — Agile shortened the loop for building; DevOps shortens it for releasing and running. You'll live in a sprint, a backlog, and a stand-up — knowing the language makes you a teammate, not just a tool operator.

📚 Read more — go deeper (2)

SRE, DevOps & platform engineering

How Google's SRE implements DevOps with hard numbers — SLIs, SLOs and error budgets — and where platform engineering fits.

If DevOps is the philosophy, Site Reliability Engineering (SRE) is Google's concrete implementation of it — "what happens when you ask a software engineer to design an operations team." SRE gives DevOps hard numbers.

The core SRE ideas you must know:

  • SLI (Service Level Indicator): a measurement of behaviour, e.g. "the percentage of requests served in under 300 ms."
  • SLO (Service Level Objective): the target for an SLI, e.g. "99.9% of requests succeed over 30 days."
  • Error budget: 100% − SLO. If your SLO is 99.9%, you have a 0.1% budget to "spend" on failures. Budget left over? Ship faster. Budget exhausted? Freeze features and fix reliability.

The error budget is the genius part: it turns "how reliable should we be?" from an argument into a shared, data-driven decision.

Platform engineering is the newest evolution: instead of every team reinventing pipelines and infrastructure, a platform team builds an Internal Developer Platform (IDP) with "golden paths" so product teams self-serve safely.

Tip

Takeaway — DevOps is the culture, SRE is the engineering discipline that measures it, and platform engineering packages it so others can reuse it.

📚 Read more — go deeper (2)

How the internet & a web app work end-to-end

Client → DNS → load balancer → server → database. The request path is the map you'll spend your career operating.

You will spend your career operating the request path, so burn it into memory. When a user opens https://shop.example.com:

  1. DNS resolves the name to an IP address (like a phone book).
  2. The browser opens a TCP connection to that IP on port 443, then negotiates TLS for encryption.
  3. The request usually hits a load balancer first, which forwards it to one of many application servers.
  4. The app server runs your code, often querying a database or cache, and returns an HTTP response.
  5. Static assets (images, JS) may come from a CDN near the user.
Browser → DNS → Load Balancer → App Server → Database
                     │              │
                    CDN           Cache

Every layer here is a place where things break and a place you'll add monitoring. "The site is slow" almost always resolves to one of these hops: slow DNS, an overloaded server, a missing cache, a slow query.

Tip

Takeaway — when something is broken, don't guess — walk the request path hop by hop. The fix is nearly always at a specific stage, not "somewhere in the cloud."

📚 Read more — go deeper (2)

🛠️ Build it — practise this stage

  • Write a one-page README explaining, in your own words, what happens when you type a URL and press Enter.
  • Map a web app you use daily into the DevOps loop: plan → code → build → test → release → deploy → operate → monitor.

🐧 Stage 2 · Linux & operating systems

The OS every server, container and pipeline runs on

🟢 Beginner  ·  ⏱️ 3–4 weeks  ·  📚 6 lessons  ·  🧩 Foundations

✅ Finish first: Foundations & the DevOps mindset

You will live in a Linux shell. Get fluent with the filesystem, permissions, processes, packages, services, SSH and scheduling — it pays off in every later stage.

▶ Open Stage 02 on the live site →

The command line & filesystem

Navigation, pipes and redirection, and the text triad (grep/sed/awk). Small tools, composed.

Servers have no mouse. Your entire job runs through a shell — almost always Bash on Linux. Fluency here is the single biggest early productivity multiplier.

Start with navigation and inspection: pwd (where am I), ls -la (list, including hidden), cd, cat, less, find, and tree. Everything in Linux is a file — even devices and processes — so these commands go a very long way.

The real power is composition with pipes (|) and redirection (>, >>, <). Each tool does one thing; you chain them:

# find the 5 largest files under /var/log
du -ah /var/log | sort -rh | head -5

# count errors in a log
grep -i error app.log | wc -l

Learn the "text triad": grep (search), sed (stream edit), awk (field processing). And use man <command> or tldr <command> the moment you're unsure — reading the manual is the skill.

Tip

Takeaway — don't memorise flags; memorise the pattern of "small tools + pipes." Once grep … | sort | uniq -c | sort -rn feels natural, you can answer almost any question about a system from the terminal.

📚 Read more — go deeper (3)

Filesystem hierarchy, disks & mounts

Where everything lives (/etc, /var/log), how disks are mounted, and how to find what filled the disk.

Linux organises everything under a single root, / — there are no drive letters. Learn the Filesystem Hierarchy Standard, because it tells you where to look:

  • /etc — configuration files (this is where you'll spend a lot of time).
  • /var — variable data: logs (/var/log), caches, spools.
  • /home — user home directories. /root — root's home.
  • /usr/bin, /bin — installed programs. /tmp — scratch space, wiped on reboot.
  • /proc, /sys — virtual filesystems exposing kernel and process state.

Storage is mounted into that tree: a disk or partition is attached at a mount point (e.g. /data). Inspect with:

df -h          # free space per filesystem — check this when "disk full"
du -sh /var/*  # what is actually eating the space
lsblk          # block devices and partitions
mount | column -t

Permanent mounts are declared in /etc/fstab. LVM (Logical Volume Manager) adds a layer that lets you resize and span volumes across disks without repartitioning — common on servers.

"Disk full" is one of the most frequent production incidents, and it's usually runaway logs in /var/log. df -h then du -sh narrows it in seconds.

Tip

Takeaway — know /etc, /var/log, and how to find what's filling a disk. Those three cover most day-one server problems.

📚 Read more — go deeper (2)

Users, permissions & processes

chmod/chown, sudo and least privilege, ps/top, signals and exit codes.

Linux is multi-user by design, and its security model is refreshingly simple once it clicks.

Permissions apply to three classes — user (owner), group, and others — each with read (4), write (2), and execute (1) bits. That's why chmod 750 script.sh means owner=7 (rwx), group=5 (r-x), others=0 (none). Use ls -l to read them and chown user:group file to change ownership.

Users and privilege: you avoid logging in as root. Instead, an ordinary user runs specific commands as root via sudo. This gives you an audit trail and limits blast radius — the foundation of least privilege.

Processes: every running program is a process with a PID. Inspect with ps aux, watch live with top/htop, and control with signals: kill -TERM asks a process to stop gracefully, kill -9 (SIGKILL) forces it. Background a job with &, and understand exit codes — 0 means success, anything else is failure (this is what pipelines check).

Tip

Takeaway — most "permission denied" and "why is the server on fire" problems come down to three questions — who owns it, what can they do, and which process is doing it.

📚 Read more — go deeper (2)

Package managers, services & systemd

apt/dnf, managing daemons with systemctl, unit files, and reading logs with journalctl.

Real servers install software from package managers, not by downloading binaries by hand. On Debian/Ubuntu that's apt (apt install nginx); on RHEL/Fedora it's dnf/yum. Packages handle dependencies, upgrades, and removal cleanly, and you can pin versions for reproducibility.

Long-running programs (web servers, databases, your app) run as services managed by systemd, the init system on modern Linux. The key command is systemctl:

systemctl status nginx     # is it running? why did it crash?
systemctl start|stop|restart nginx
systemctl enable nginx     # start automatically on boot

A service is defined by a small unit file (/etc/systemd/system/myapp.service) that says what to run, which user to run as, and how to restart on failure (Restart=always). Writing one for your own app is a rite of passage.

Logs go to the journal: journalctl -u nginx -f follows a service's logs live — this is where you look first when something won't start.

Tip

Takeaway — systemctl status and journalctl -u <service> are the two commands you'll run more than any others when a service misbehaves. Learn them cold.

📚 Read more — go deeper (2)

SSH & surviving in vim

Key-based login, ~/.ssh/config, port forwarding, hardening sshd — plus enough vim to edit any config file.

Two skills you'll use every single day on a server.

SSH is how you reach remote machines securely. Stop using passwords — use key pairs: you keep a private key, the server holds your public key.

ssh-keygen -t ed25519            # generate a modern key pair
ssh-copy-id user@server          # install your public key on the server
ssh user@server                  # log in, no password

Configure shortcuts in ~/.ssh/config so ssh prod just works. Learn two power moves: scp/rsync to copy files, and port forwarding (ssh -L 8080:localhost:80 server) to reach a service that isn't exposed publicly. On the server side, harden sshd: disable root login and password authentication entirely.

A terminal editor is unavoidable — there's no VS Code on a remote box. Vim is everywhere, so learn survival mode: i to insert, Esc to leave insert, :w write, :q quit, :wq both, :q! quit discarding changes, /text to search, dd delete a line. That's genuinely enough to edit a config file. (nano is friendlier if it's installed, but vim is always installed.)

Tip

Takeaway — SSH keys plus survival vim mean you can reach any server and fix any config file. These are the hands you'll work with for the rest of your career.

📚 Read more — go deeper (2)

Scheduling with cron & systemd timers

Crontab syntax, the environment traps, logging output, and the modern systemd timer alternative.

Plenty of ops work is just "run this thing on a schedule": backups, cleanups, report generation, certificate renewal.

Cron is the classic scheduler. Each line in a crontab (crontab -e) is a schedule plus a command, with five time fields — minute, hour, day-of-month, month, day-of-week:

# ┌ min ┌ hour ┌ dom ┌ mon ┌ dow
  0     3      *     *     *    /usr/local/bin/backup.sh
# → run backup.sh every day at 03:00
  */15  *      *     *     *    /usr/local/bin/healthcheck.sh
# → every 15 minutes

The classic traps: cron runs with a minimal environment (your PATH isn't what you think — use absolute paths), and it emails output nobody reads, so always redirect output to a log file (>> /var/log/backup.log 2>&1) or you'll be blind when it silently fails.

systemd timers are the modern alternative: more verbose, but they give you real logging via journalctl, dependency handling, and the ability to catch up on missed runs after a reboot.

In Kubernetes, the same idea is a CronJob — the concept follows you up the stack.

Tip

Takeaway — schedule with cron or systemd timers, always log the output, and always use absolute paths. A silent failing cron job is a classic way to discover your backups haven't run for six months.

📚 Read more — go deeper (2)

Note

🎓 Certifications to target after this stage

🛠️ Build it — practise this stage

  • Spin up a Linux VM and harden it: non-root user, key-only SSH, firewall, automatic security updates.
  • Write a backup script, schedule it with cron, log its output, and verify it actually ran.

🌐 Stage 3 · Networking & protocols

Nothing works until packets reach the right place

🟢 Beginner  ·  ⏱️ 2–3 weeks  ·  📚 5 lessons  ·  🧩 Foundations

✅ Finish first: Linux & operating systems

DNS, HTTP/S, TCP/IP, proxies and load balancers — plus the troubleshooting ladder that turns 'the site is down' into a specific, fixable cause.

▶ Open Stage 03 on the live site →

TCP/IP, the OSI model & ports

IP addressing, subnets and CIDR, TCP vs UDP, and the port model that lets one host run many services.

Networking sounds academic until an app "can't connect" and you're the one who has to find out why. The OSI model is the shared vocabulary; in practice you care about a few layers.

Every machine has an IP address. Subnets and CIDR notation (10.0.1.0/24) define ranges of addresses that can talk to each other directly — this is the backbone of every cloud network (VPCs). The /24 means the first 24 bits are the network, leaving 256 addresses.

On top of IP sit two transport protocols:

  • TCP — reliable, ordered, connection-based. Used by HTTP, SSH, databases. It does a "three-way handshake" (SYN, SYN-ACK, ACK) before sending data.
  • UDP — fire-and-forget, no guarantees, low latency. Used by DNS, video, gaming.

Ports let one machine run many services: HTTP is 80, HTTPS 443, SSH 22, PostgreSQL 5432. An address like 10.0.1.5:443 means "port 443 on that host."

Tip

Takeaway — when connectivity fails, ask in order — is the host reachable (ping), is the port open (nc -vz host 443), and is a firewall in the way? That sequence solves most network tickets.

📚 Read more — go deeper (2)

DNS, HTTP/HTTPS & TLS

Record types and TTLs, HTTP methods and status codes, and how TLS certificates keep it all private.

DNS turns names into IP addresses. The records you'll touch constantly:

  • A / AAAA — name → IPv4 / IPv6 address.
  • CNAME — name → another name (an alias).
  • MX — mail servers; TXT — arbitrary text, used for domain verification and SPF/DKIM.

Changes take time to propagate because records are cached for their TTL. Debug with dig example.com or nslookup.

HTTP is the language of the web — a simple request/response text protocol. A request has a method (GET, POST, PUT, DELETE), a path, headers, and an optional body. A response has a status code you must know by heart:

  • 2xx success · 3xx redirect · 4xx you (the client) messed up (404 not found, 401 unauthorised) · 5xx the server messed up (500, 502 bad gateway, 503 unavailable).

HTTPS is just HTTP inside TLS encryption. TLS uses certificates to prove identity and to set up an encrypted channel, so nobody between the user and server can read the traffic. Expired certificates are a classic outage — automate their renewal (Let's Encrypt).

Tip

Takeaway — curl -v https://site shows you the DNS, TLS handshake, request, and response headers all at once. It's the most useful debugging command on the web.

📚 Read more — go deeper (3)

Load balancing & firewalls

L4 vs L7 balancing, health checks, default-deny firewalls and security groups.

One server is a single point of failure and a scaling ceiling. Load balancers solve both by spreading incoming requests across many identical servers and quietly removing any that fail a health check.

There are two flavours you'll hear about:

  • Layer 4 (transport): balances by IP/port, fast and protocol-agnostic.
  • Layer 7 (application): understands HTTP, so it can route by URL path or hostname (/api → API servers, everything else → the web app). This is what tools like NGINX, HAProxy, and cloud ALBs do.

A reverse proxy sits in front of your app and can also terminate TLS, cache responses, compress output, and rate-limit abusers. NGINX is the classic example and worth configuring once by hand.

Firewalls (and their cloud cousin, security groups) decide which traffic is allowed in and out. The rule of thumb is default-deny: block everything, then open only the specific ports you need (443 to the world, 22 only from your office/VPN).

Tip

Takeaway — in production, users never hit your app directly — they hit a load balancer / reverse proxy that gives you scaling, TLS, and a security chokepoint for free.

📚 Read more — go deeper (1)

Reverse proxies, CDNs & caching

Forward vs reverse proxies, what a CDN actually does, and the caching headers that make (or break) a site.

Between your user and your server sit several layers whose job is speed and control. Knowing which is which matters when things break.

A forward proxy sits in front of clients (an outbound gateway for a corporate network). A reverse proxy sits in front of servers — this is the one you'll operate. It receives every request and decides what to do: route it, terminate TLS, compress, cache, rate-limit, or block it. NGINX, HAProxy, Envoy, and Traefik all do this.

A CDN (Cloudflare, CloudFront, Fastly) is a reverse proxy replicated to hundreds of locations worldwide. It serves cached content from a PoP physically near the user, which slashes latency and absorbs traffic spikes and DDoS attacks before they reach you.

The key concept is caching, governed by HTTP headers: Cache-Control: max-age=3600 says "reusable for an hour." Static assets (images, JS, CSS) should be cached hard and given hashed filenames so a new deploy busts the cache automatically. Dynamic, user-specific responses must not be cached — leaking one user's page to another is a classic, embarrassing outage.

Tip

Takeaway — a reverse proxy gives you TLS, routing, and a chokepoint for security; a CDN gives you global speed and shielding. Cache static content aggressively, never cache personalised responses.

📚 Read more — go deeper (2)

Network troubleshooting toolkit

dig, ping, nc, curl, ss, tcpdump, mtr — and the exact order to use them so you stop guessing.

Knowing networking theory is useless if you can't diagnose. This is the toolkit, and — more importantly — the order you use it in. Work up the stack:

  1. Is DNS resolving? dig example.com +short — do you get the IP you expect?
  2. Is the host reachable? ping 10.0.1.5 (note: many hosts block ICMP, so a failed ping isn't proof of death).
  3. Is the port open? nc -vz 10.0.1.5 443 or nmap -p 443 host. This distinguishes "server down" from "firewall blocking."
  4. Is the app answering correctly? curl -v https://host/health — shows DNS, TLS, headers, and the response in one shot.
  5. Is the service even listening? On the box itself: ss -tulpn lists listening sockets and which process owns them.
  6. What's actually on the wire? tcpdump -i any port 443 when you truly need to see packets.

Also learn traceroute (where does the path die?) and mtr (continuous traceroute — brilliant for intermittent loss).

The discipline is refusing to guess. Each command eliminates one layer, so you converge on the truth in minutes rather than restarting things and hoping.

Tip

Takeaway — DNS → host → port → app → listener. Walk that ladder in order and you'll out-debug engineers with twice your experience.

📚 Read more — go deeper (2)

🛠️ Build it — practise this stage

  • Trace a request end-to-end with dig, curl -v, traceroute and tcpdump; document every hop.
  • Put NGINX in front of two app servers as a reverse proxy + load balancer with health checks and TLS.

🐍 Stage 4 · Programming & scripting

Automation is just code you write for machines to run

🟢 Beginner  ·  ⏱️ 4–6 weeks  ·  📚 6 lessons  ·  🧩 Foundations

✅ Finish first: Linux & operating systems

Bash for glue, Python for real automation, the data formats every tool speaks, regex for text, APIs to drive everything — and a taste of Go, the language of the cloud-native stack.

▶ Open Stage 04 on the live site →

Bash scripting

Safe scripts with set -euo pipefail, quoting, exit codes, and knowing when to stop using Bash.

Bash is the glue of DevOps: bootstrapping servers, wiring CI steps, and automating the boring stuff. You don't need to be a Bash wizard, but you must write scripts you can trust.

Start every serious script with a safety header:

#!/usr/bin/env bash
set -euo pipefail   # exit on error, unset vars, and failed pipes

That one line prevents a huge class of silent failures. From there the building blocks are variables (name="prod"), conditionals (if [[ -f "$file" ]]; then …), loops (for f in *.log; do …; done), and functions.

Two habits separate pros from beginners: quote your variables ("$var", always — spaces and empty values will bite you) and check exit codes instead of assuming success. Run every script through ShellCheck, which catches these bugs automatically.

backup() {
  local src="$1" dest="$2"
  tar -czf "$dest/backup-$(date +%F).tgz" "$src" && echo "backed up $src"
}

Tip

Takeaway — Bash is perfect for gluing commands together, but the moment you're doing real data structures or logic, reach for Python. Knowing when to stop using Bash is itself a skill.

📚 Read more — go deeper (2)

Python for automation

Enough Python to call APIs, parse JSON, and glue systems together — plus venvs and pinned dependencies.

When a task outgrows Bash — calling APIs, parsing JSON, real error handling — Python is the DevOps default. It's readable, ships everywhere, and has a library for every cloud and tool.

You only need a practical slice to be effective: variables and types, lists/dicts, for loops, functions, try/except for errors, and reading/writing files. The killer feature is the ecosystem:

  • requests to call REST APIs.
  • json / PyYAML to parse config and API responses.
  • Cloud SDKs like boto3 (AWS) to automate infrastructure.
import requests
r = requests.get("https://api.github.com/repos/kubernetes/kubernetes")
r.raise_for_status()
print(r.json()["stargazers_count"], "stars")

Two professional habits: always use a virtual environment (python -m venv .venv) so projects don't pollute each other, and pin dependencies in requirements.txt for reproducible runs.

Tip

Takeaway — Bash for gluing commands, Python for logic and APIs. With requests + a cloud SDK you can automate almost any operational task, from tagging resources to generating reports.

📚 Read more — go deeper (2)

YAML, JSON & jq

Every manifest and pipeline is YAML; every API speaks JSON. Master indentation and jq.

Configuration in DevOps is data, and that data is almost always YAML or JSON. Kubernetes manifests, CI pipelines, Docker Compose, Ansible — all YAML. API responses and Terraform state — JSON.

YAML is designed for humans: indentation defines structure (spaces only, never tabs), key: value for maps, and - for list items.

service:
  name: web
  replicas: 3
  ports:
    - 80
    - 443

The number-one YAML mistake is indentation — a single wrong space breaks the whole file. Validate with yamllint before shipping.

JSON is the machine-friendly cousin: stricter, no comments, great for APIs. Learn jq to slice JSON from the command line — it's indispensable:

curl -s api/data | jq '.items[] | .name'

Go is worth a taste later, not now: Docker, Kubernetes, and Terraform are all written in it, so reading Go helps you understand your tools and contribute to them.

Tip

Takeaway — master YAML indentation and jq, and you can configure and inspect nearly every tool in the stack.

📚 Read more — go deeper (2)

Regular expressions

The dozen symbols that let you extract anything from any log, alert rule or CI filter.

Regular expressions are the pattern language for text — and DevOps is drowning in text: logs, configs, alert rules, CI filters, monitoring queries. A little regex goes an extremely long way.

The building blocks:

  • . any character · \d digit · \w word char · \s whitespace
  • * zero or more · + one or more · ? optional · {2,4} a range
  • ^ start of line · $ end of line
  • [abc] any of these · [^abc] none of these
  • (…) a capture group — the part you want to extract
  • | alternation (or)
# find IPs in a log
grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}' access.log | sort | uniq -c | sort -rn

# find failed logins
grep -E 'Failed password.*from ([0-9.]+)' /var/log/auth.log

Regex shows up far beyond grep: log-parsing rules, Prometheus label matchers, Git ignore/branch filters, CI path triggers, and validation rules everywhere.

The professional's warning: regex is greedy by default (.* grabs as much as possible — use .*? to be lazy), and an over-clever regex is unmaintainable. Test them interactively (regex101.com) rather than guessing in production.

Tip

Takeaway — learn the dozen symbols above and you can extract anything from any log. It's the highest-leverage hour you'll spend on text processing.

📚 Read more — go deeper (2)

REST APIs, auth & webhooks

Methods, status codes, bearer tokens, rate limits, pagination and webhooks — how you drive every tool in the stack.

Almost everything you automate is driven through an API. Terraform, kubectl, cloud CLIs, and your monitoring dashboards are all just clients making HTTP calls. Understanding APIs turns you from a tool user into a tool builder.

REST is the dominant style: resources at URLs, manipulated with HTTP methods.

  • GET /users — list · GET /users/42 — fetch one
  • POST /users — create · PUT/PATCH /users/42 — update · DELETE /users/42 — remove

Responses carry a status code (200, 201, 404, 500) and usually a JSON body. Requests carry headers, including authentication.

Authentication is where beginners stumble. The common patterns: an API key header, a Bearer token (Authorization: Bearer <token>), or OAuth2 where you exchange credentials for a short-lived token. Tokens expire — good automation handles refresh and never hard-codes secrets.

curl -s -H "Authorization: Bearer $TOKEN" \
     https://api.example.com/v1/deployments | jq '.items[].status'

Also expect rate limits (back off and retry, don't hammer), pagination (results come in pages — follow next), and webhooks, the inverse pattern where the service calls you when an event happens (this is how CI triggers on a Git push).

Tip

Takeaway — curl + jq + a token lets you drive nearly any system in the stack. Every tool you'll learn is ultimately a friendly wrapper over an HTTP API.

📚 Read more — go deeper (2)

A taste of Go

Docker, Kubernetes and Terraform are all written in Go. Read it to understand your tools; write it for fast single binaries.

Docker, Kubernetes, Terraform, Prometheus — the entire cloud-native stack is written in Go. You don't need to be a Go developer, but reading it lets you understand your tools, debug them, and eventually contribute.

Go was designed for exactly this world: it compiles to a single static binary with no runtime to install (which is why Go programs ship so beautifully in tiny containers), it's fast, and concurrency is built in.

package main

import "fmt"

func main() {
    services := []string{"api", "web", "worker"}
    for _, s := range services {
        fmt.Printf("deploying %s\n", s)
    }
}

The Go-isms to recognise: explicit error handling (functions return a value and an error; if err != nil { … } is everywhere), goroutines (go doWork() runs concurrently — lightweight threads) and channels for passing data between them. There are no exceptions and no classes; it's deliberately small and boring, which is a virtue at scale.

Where it pays off for you: writing a fast CLI tool, a custom Prometheus exporter, or a Kubernetes operator.

Tip

Takeaway — learn enough Go to read the source of the tools you operate — that alone is a superpower when docs are thin. Write Go when you need a single fast binary; keep using Python for glue.

📚 Read more — go deeper (2)

🛠️ Build it — practise this stage

  • Write a Bash script that backs up a directory, rotates old backups, and exits non-zero on failure. Pass ShellCheck cleanly.
  • Write a Python tool that calls a REST API with a token, handles pagination and rate limits, and writes a report.

🌿 Stage 5 · Git & version control

The system of record for code and, soon, infrastructure

🟢 Beginner  ·  ⏱️ 2 weeks  ·  📚 4 lessons  ·  🧩 Foundations

✅ Finish first: Foundations & the DevOps mindset

Git is the backbone of CI/CD and GitOps. Master the model, the collaboration workflow, the rescue commands, and how versions and releases are automated.

▶ Open Stage 05 on the live site →

Git fundamentals

The three areas, commits as snapshots, branches as pointers, merge vs rebase, and recovering from mistakes.

Git is the system of record for everything you do — code, pipelines, and soon infrastructure itself. Understanding its model, not just memorising commands, is what stops the panic.

Git has three areas: your working directory (files you edit), the staging area (changes you've marked with git add), and the repository (permanent snapshots created by git commit). A commit is an immutable snapshot with a parent, which is how history forms a chain.

The daily loop:

git status              # what changed?
git add file.py         # stage it
git commit -m "message" # snapshot it
git push                # send to the remote

Branches are just cheap, movable pointers to a commit — creating one is instant. You merge to combine histories (keeps both lines) or rebase to replay your commits on top of another branch (keeps history linear). When two people change the same lines, you get a conflict; Git marks it and you choose the result.

The confidence booster: almost nothing is truly lost. git reflog remembers where you were, so you can recover from most mistakes.

Tip

Takeaway — picture the three areas and branches-as-pointers, and Git stops being magic. Commit small and often — small commits are easy to review, revert, and understand.

📚 Read more — go deeper (2)

Collaboration & branching strategies

Pull requests and review, trunk-based vs GitFlow, branch protection, and conventional commits.

Git alone is version control; workflows are how a team uses it without stepping on each other. This is where GitHub and GitLab come in.

The unit of collaboration is the Pull Request (PR) / Merge Request: you push a branch, open a PR, and teammates review the diff before it merges. Good PRs are small, have a clear description, and pass automated checks (tests, linting) before a human even looks.

Two dominant branching strategies:

  • Trunk-based development: everyone merges tiny changes into main frequently, behind feature flags if needed. Favoured by high-performing teams because it minimises painful merges.
  • GitFlow: long-lived develop, release, and feature branches. More ceremony; better suited to versioned software with scheduled releases.

Protect main with branch protection rules: require reviews and green CI before merge, and forbid force-pushes. Adopt Conventional Commits (feat:, fix:, chore:) so history is readable and changelogs can be generated automatically.

Tip

Takeaway — the PR is where quality is enforced — through review and automation. Master the flow of branch → PR → review → merge, because every later stage (CI/CD, GitOps) is built on top of it.

📚 Read more — go deeper (2)

Advanced Git: rebase, cherry-pick, bisect, reflog

Curate history, hotfix a single commit, binary-search for the bug, and undo almost any disaster.

Once the basics are automatic, a handful of advanced Git commands make you the person who rescues the repository.

  • git rebase -i HEAD~5 — interactive rebase. Squash messy work-in-progress commits into one clean commit, reword messages, drop mistakes. Clean history is a gift to reviewers. Rule: rebase your own unpushed branch, never shared history.
  • git cherry-pick <sha> — copy one specific commit onto another branch. The standard way to hotfix production without shipping everything else on main.
  • git bisect — binary-search your history for the commit that introduced a bug. Mark a known-good and known-bad commit, and Git walks you through the middle each time. It finds the culprit in ~log₂(n) steps — magic on a big repo.
  • git stash — park uncommitted changes to switch context fast.
  • git reflog — the safety net. It records every position HEAD has been at, so even a "lost" commit after a bad reset is recoverable. Almost nothing in Git is truly gone.
  • Hooks (.git/hooks, or pre-commit framework) run checks automatically — lint, format, secret-scan — before a bad commit ever exists.

Tip

Takeaway — rebase to curate history, cherry-pick to hotfix, bisect to hunt bugs, reflog to undo disasters. These four turn Git from a save button into a genuine engineering instrument.

📚 Read more — go deeper (2)

SemVer, tags & automated releases

MAJOR.MINOR.PATCH as a contract, immutable tags, changelogs — and why you never deploy latest.

"Which version is in production?" must have a precise answer. That's what versioning and release management give you.

Semantic Versioning (SemVer) is the standard: MAJOR.MINOR.PATCH.

  • PATCH (1.4.2 → 1.4.3) — backwards-compatible bug fix.
  • MINOR (1.4.2 → 1.5.0) — new functionality, still backwards-compatible.
  • MAJOR (1.4.2 → 2.0.0) — a breaking change. Consumers must act.

That contract lets other teams upgrade safely, and it lets tools auto-update patches without fear.

In Git, a release is a tag (git tag -a v1.5.0 -m "..."), an immutable pointer to a commit. Your CI builds artifacts from tags, so a version maps to exactly one build.

Automate it: with Conventional Commits (feat:, fix:, feat!: for breaking), tools like semantic-release read your commit messages, work out the next version, tag it, generate a CHANGELOG, and publish — no human deciding version numbers.

Never deploy latest. A container tagged latest is unreproducible and unrollbackable; deploy app:1.5.0 and you always know exactly what's running and what to roll back to.

Tip

Takeaway — SemVer communicates risk, tags make versions immutable, and Conventional Commits let you automate the whole release. "What's in prod?" should be answerable in one second.

📚 Read more — go deeper (3)

🛠️ Build it — practise this stage

  • Contribute a fix to an open-source project end-to-end: fork, branch, PR, address review, merge.
  • Set up a repo with branch protection, required checks, conventional commits, and automated SemVer releases.

⚙️ Stage 6 · Build, test & Continuous Integration

Every commit is built, tested and proven — automatically

🔵 Intermediate  ·  ⏱️ 3–4 weeks  ·  📚 6 lessons  ·  🧩 Core DevOps

✅ Finish first: Git & version control, Programming & scripting

CI turns 'it works on my machine' into 'it provably works, every time'. Learn pipelines, a real test strategy, the major CI systems, and how artifacts are versioned and stored.

▶ Open Stage 06 on the live site →

CI/CD concepts

Pipelines, stages, jobs, runners, artifacts, caching, secrets and environments — the vocabulary of every CI system.

Continuous Integration (CI) means every commit is automatically built and tested, so integration problems surface in minutes, not at the end of a release. Continuous Delivery/Deployment (CD) extends that: every change that passes automatically becomes a release candidate (delivery) or goes straight to production (deployment).

A pipeline is the automation that runs on each push. It's made of stages (build → test → deploy) containing jobs that execute on runners (worker machines). Jobs produce artifacts (a compiled binary, a Docker image) that later stages consume.

Concepts that make pipelines fast and safe:

  • Caching dependencies so builds don't reinstall everything each time.
  • Secrets injected securely at run time — never hard-coded.
  • Environments (dev, staging, prod) with approvals gating production.

For releasing without downtime, learn two patterns: blue/green (run two identical environments, switch traffic instantly) and canary (send 5% of traffic to the new version, watch the metrics, then ramp up).

Tip

Takeaway — a pipeline turns "it works on my machine" into "it provably works, every time." The goal is a green commit that flows to production with no manual steps — and an instant rollback when a metric goes red.

📚 Read more — go deeper (2)

Test strategy & quality gates

The test pyramid, what to run in CI, coverage as a signal not a target, and why flaky tests are a real bug.

CI is only as valuable as the tests it runs. A pipeline that runs no meaningful tests is just an expensive way to compile code.

The test pyramid tells you where to invest:

  • Unit tests (the wide base) — test one function in isolation. Milliseconds, thousands of them. Catch most bugs, cheapest to maintain.
  • Integration tests (the middle) — test components together: your app and a real database. Slower, fewer.
  • End-to-end tests (the narrow top) — drive the whole system like a user. Slowest, most brittle — have a handful covering critical journeys only.

Invert this pyramid (mostly E2E) and your pipeline becomes slow and flaky, and people start ignoring failures — the death of CI.

Other checks belong in the pipeline too: linting and formatting, type checks, security/dependency scans, and code coverage (useful as a signal, dangerous as a target — 100% coverage of trivial code proves nothing).

Enforce this with quality gates: the PR cannot merge unless everything is green. And treat a flaky test as a real bug — one test that fails randomly teaches the whole team to ignore red builds.

Tip

Takeaway — many fast unit tests, some integration tests, few E2E tests — all gating the merge. Fast and trustworthy beats exhaustive and ignored.

📚 Read more — go deeper (1)

GitHub Actions

Workflows, events, jobs and steps, matrix builds, secrets and reusable workflows. The easiest place to start.

GitHub Actions is the easiest place to learn CI/CD because it lives right next to your code. A workflow is a YAML file in .github/workflows/ that runs on events (a push, a PR, a schedule).

name: CI
on: [push]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20 }
      - run: npm ci
      - run: npm test

The pieces: a job runs on a fresh runner; steps run in order; uses: pulls a reusable action from the Marketplace (checkout, language setup, cloud login), while run: executes shell commands.

Level up with matrix builds (test across several versions at once), secrets (${{ secrets.TOKEN }}) for credentials, and reusable workflows so teams don't copy-paste pipelines. Actions can also build and push Docker images and deploy to your cloud — a complete path from commit to production.

Tip

Takeaway — start by adding a workflow that runs your tests on every PR. That single file gives you the core of CI, and everything else (build, scan, deploy) is just more steps.

📚 Read more — go deeper (2)

GitLab CI

Stages, jobs, runners, artifacts, cache, rules and manual production gates in .gitlab-ci.yml.

GitLab CI is the other CI system you'll meet constantly, and it's fully integrated into GitLab — repo, CI, registry, and issues in one product. Pipelines live in .gitlab-ci.yml at the repo root.

stages: [build, test, deploy]

build-job:
  stage: build
  image: node:20
  script:
    - npm ci
    - npm run build
  artifacts:
    paths: [dist/]

test-job:
  stage: test
  script: npm test

deploy-prod:
  stage: deploy
  script: ./deploy.sh
  environment: production
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
      when: manual        # require a human click for prod

The model: stages run in sequence, jobs inside a stage run in parallel, and jobs execute on runners (which you can host yourself). Jobs pass files forward as artifacts, speed up with cache, and are conditionally included with rules. environment: gives you deployment tracking and one-click rollback in the UI.

The concepts map almost one-to-one onto GitHub Actions — stages/jobs/steps, artifacts, caching, secrets, manual approvals. Learn one deeply and you can read any of them.

Tip

Takeaway — GitLab CI's stage → job → runner model is the same shape as every other CI system. Once you see that, switching CI tools is a syntax change, not a re-learn.

📚 Read more — go deeper (1)

Jenkins & pipeline-as-code

The enterprise veteran: controllers, agents, stages, plugins — and why you always use a Jenkinsfile.

Jenkins is the veteran of CI — self-hosted, endlessly pluggable, and still running the pipelines of a huge share of enterprises. Modern SaaS CI is nicer to use, but Jenkins is everywhere in real jobs, so knowing it makes you employable in a large installed base.

You define builds as pipeline as code in a Jenkinsfile committed to your repo:

pipeline {
  agent any
  stages {
    stage('Build') { steps { sh 'make build' } }
    stage('Test')  { steps { sh 'make test' } }
    stage('Deploy') {
      when { branch 'main' }
      steps { sh './deploy.sh' }
    }
  }
}

The model: a controller schedules work onto agents (worker nodes), pipelines are made of stages and steps, and thousands of plugins integrate everything from Docker to Slack. That plugin ecosystem is both its superpower and its weakness — plugins are a maintenance and security burden, so keep them few and updated.

Practical advice: never configure jobs by clicking in the UI ("freestyle" jobs) — they can't be reviewed or version-controlled. Always use a Jenkinsfile, so your pipeline lives in Git like everything else.

Tip

Takeaway — Jenkins teaches you CI fundamentals that transfer everywhere — agents, stages, artifacts, and pipeline-as-code. Learn the Jenkinsfile model, and you'll be productive on any CI system.

📚 Read more — go deeper (1)

Artifacts & registries

Build once, deploy everywhere: immutable versioned artifacts, container registries, and retention policies.

An artifact is the thing your build produces — a JAR, a binary, a Python wheel, a container image, a Helm chart. The golden rule of delivery: build once, deploy everywhere.

Why it matters: if you rebuild the app separately for staging and for production, you have not tested what you shipped — the two builds can differ (a dependency moved, a base image updated). Instead, build a single immutable, versioned artifact, and promote that exact artifact through each environment.

Artifacts live in a registry/repository:

  • Container images — Docker Hub, GitHub Container Registry, AWS ECR, Harbor.
  • Language packages — npm, PyPI, Maven — usually proxied through Nexus or Artifactory, which also caches public packages (faster builds, and you survive an upstream outage).

Good practice: tag artifacts immutably with a version or Git SHA (app:1.5.0, app:git-a1b2c3d), never overwrite a published tag, scan images on push, sign them so you can prove provenance, and set retention policies — registries grow without limit and get expensive.

Tip

Takeaway — one build → one versioned artifact → promoted through dev, staging, prod. If your pipeline rebuilds per environment, you're shipping something you never tested.

📚 Read more — go deeper (2)

🛠️ Build it — practise this stage

  • Build a pipeline that lints, tests, scans, builds a container image, and pushes it to a registry on every push to main.
  • Add a staging deploy plus a manual approval gate before production, and make the version a Git tag.

🐳 Stage 7 · Containers & Docker

Package once, run the same everywhere

🔵 Intermediate  ·  ⏱️ 2–3 weeks  ·  📚 4 lessons  ·  🧩 Core DevOps

✅ Finish first: Linux & operating systems, Programming & scripting

Containers ended 'it works on my machine'. Learn images and Dockerfiles, what a container actually is under the hood, multi-service local stacks, and how to harden images.

▶ Open Stage 07 on the live site →

Docker fundamentals

Images vs containers, the Dockerfile, layer caching, volumes and ports, and the everyday commands.

Containers ended "it works on my machine." A container packages your app with its exact dependencies so it runs identically on a laptop, in CI, and in production. Unlike a virtual machine, it shares the host's kernel, so it starts in milliseconds and is tiny.

Two concepts to keep straight:

  • An image is the immutable blueprint (your app + libraries + runtime).
  • A container is a running instance of an image.

You build an image from a Dockerfile, a recipe read top to bottom:

FROM node:20-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
CMD ["node", "server.js"]

Each instruction creates a cached layer, so ordering matters: copy package.json and install before copying source, and Docker reuses the dependency layer whenever only your code changes — builds go from minutes to seconds.

Everyday commands: docker build -t app ., docker run -p 8080:3000 app, docker ps, docker logs, docker exec -it <id> sh to poke inside a running container.

Tip

Takeaway — a container is "your app + everything it needs, frozen." Master the Dockerfile and layer caching first — it's the foundation Kubernetes builds on.

📚 Read more — go deeper (3)

How containers actually work

Namespaces, cgroups and the union filesystem — why a container is just a Linux process, and why that matters.

A container feels like a lightweight VM, but it isn't one — and knowing the difference explains almost every weird container behaviour you'll hit.

A VM virtualises hardware and runs a whole guest kernel. A container is just a normal Linux process on the host, isolated by three kernel features:

  • Namespaces — what the process can see. Separate namespaces for PID (its own process tree, so your app thinks it's PID 1), network (its own interfaces and ports), mount (its own filesystem view), user, and more.
  • cgroups (control groups) — what the process can use. CPU, memory, and I/O limits. This is exactly what Kubernetes resource requests/limits configure.
  • Union filesystem (OverlayFS) — layered, copy-on-write images. Layers are shared between containers, which is why images are small and start instantly.

The consequences matter in practice: containers share the host kernel (so a Linux container can't run on a Windows kernel without a VM, and a kernel exploit is a host compromise — this is why container security matters). And since your app runs as PID 1, it must handle SIGTERM to shut down gracefully, or Kubernetes will hard-kill it.

Tip

Takeaway — a container is a process with namespaces (isolation), cgroups (limits), and a layered filesystem. Nothing magic — which is exactly why it's fast, and why the kernel is your shared trust boundary.

📚 Read more — go deeper (2)

Compose & multi-service stacks

Run an app plus a database plus a cache with one command; push to registries with meaningful tags.

Real apps aren't one container — they're an app plus a database plus a cache. Docker Compose describes the whole set in one docker-compose.yml and starts it with a single command.

services:
  web:
    build: .
    ports: ["8080:3000"]
    depends_on: [db]
  db:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: secret

docker compose up brings the entire stack to life on a shared network where services reach each other by name (db:5432). This is how you run realistic local environments.

To share images you push them to a registry — Docker Hub, GitHub Container Registry, or a cloud one (ECR/GCR). Tag meaningfully (app:1.4.2, not just latest) so deployments are reproducible.

Two production must-dos:

  • Multi-stage builds: compile in a fat "builder" image, then copy only the finished artifact into a tiny runtime image — smaller and more secure.
  • Scan images (Trivy) and run as a non-root user; never bake secrets into layers.

Tip

Takeaway — Compose is your realistic local stack; registries are how images travel to production. Small, scanned, well-tagged images are the mark of someone who's done this before.

📚 Read more — go deeper (2)

Container & image security

Minimal/distroless bases, non-root users, no secrets in layers, and scanning that fails the build.

A container is not a security boundary by default — it shares the host kernel. Hardening images is a core DevOps responsibility.

Build a minimal image. Every package you don't ship is a vulnerability you don't have. Use multi-stage builds to leave compilers and dev dependencies behind, and prefer a small base — Alpine, or distroless images which contain your app and nothing else (no shell, no package manager, so an attacker who gets in has no tools).

Never run as root. By default containers run as root, and root in the container can be root on the host if it escapes. Always:

RUN adduser -D appuser
USER appuser

Never bake secrets into images. A secret in any layer is permanent and visible with docker history, even if a later layer deletes it. Inject secrets at runtime.

Scan everything, continuously. Trivy (or Grype/Snyk) finds known CVEs in your base image and dependencies. Wire it into CI and fail the build on critical findings. Note that a clean scan today can be vulnerable tomorrow — rescan running images, don't just scan at build.

Then: pin base image versions (not latest), drop Linux capabilities, mount the root filesystem read-only, and sign images so you can verify what you deploy.

Tip

Takeaway — minimal base, non-root user, no secrets in layers, scanned in CI and at rest. These four steps eliminate the overwhelming majority of container risk.

📚 Read more — go deeper (3)

Note

🎓 Certifications to target after this stage

🛠️ Build it — practise this stage

  • Containerise an app with a multi-stage Dockerfile, non-root user, and a final image under 100MB.
  • Run app + database + cache with Docker Compose, then scan the image with Trivy and fix every critical CVE.

☸️ Stage 8 · Kubernetes & orchestration

Run containers at scale, self-healing and declarative

🔵 Intermediate  ·  ⏱️ 6–8 weeks  ·  📚 7 lessons  ·  🧩 Core DevOps

✅ Finish first: Containers & Docker, Networking & protocols

The centre of gravity of modern DevOps — and the biggest stage here. Architecture, every workload type, config and networking, autoscaling, Helm, debugging, and extending the API with operators.

▶ Open Stage 08 on the live site →

Architecture & core objects

Control plane vs nodes, Pods → ReplicaSets → Deployments → Services, and the reconciliation loop behind it all.

Kubernetes (K8s) runs containers across a fleet of machines and keeps them healthy without you babysitting them. It's declarative: you describe the desired state, and Kubernetes continuously works to make reality match — the reconciliation loop at the heart of everything.

A cluster has two halves:

  • Control plane — the brain: the API server (front door), etcd (the database of desired state), the scheduler (places workloads), and controllers (drive reality toward desired state).
  • Nodes — the workers that actually run your containers.

The core objects, from small to large:

  • Pod — one or more containers that share a network/storage; the smallest deployable unit.
  • ReplicaSet — keeps N copies of a Pod running.
  • Deployment — manages ReplicaSets and gives you rolling updates and rollbacks.
  • Service — a stable network address and load balancer for a set of Pods (Pods are ephemeral; Services are not).

You interact via kubectl: kubectl get pods, kubectl describe, kubectl apply -f app.yaml, kubectl logs.

Tip

Takeaway — you declare what you want in YAML; Kubernetes figures out how and self-heals when things die. Internalise the reconciliation loop and the rest of Kubernetes falls into place.

📚 Read more — go deeper (2)

Workload types: Deployment, StatefulSet, DaemonSet, Job

Pick the right one: stateless vs stateful, per-node agents, run-once tasks and CronJobs. Plus probes and init containers.

Deployment is not the only way to run something. Kubernetes has a workload type for each shape of job, and picking the wrong one is a classic beginner mistake.

  • Deployment — for stateless apps (web servers, APIs). Pods are interchangeable; any one can be killed and replaced. Gives rolling updates and rollbacks. This is your default.
  • StatefulSet — for stateful apps (databases, Kafka, Elasticsearch). Pods get a stable identity (db-0, db-1) that survives restarts, stable storage attached to each, and ordered startup/shutdown. A database in a Deployment will corrupt itself; use a StatefulSet.
  • DaemonSet — runs exactly one Pod on every node. Perfect for node-level agents: log collectors (Fluent Bit), monitoring (node-exporter), network plugins. Add a node, it gets the Pod automatically.
  • Job — runs a Pod to completion and stops. For migrations, batch processing, one-off tasks. It retries on failure.
  • CronJob — a Job on a schedule (same cron syntax you already know). For nightly backups and reports.

Supporting these: init containers run to completion before your app container starts (wait for a database, run a migration), and liveness/readiness probes tell Kubernetes whether a Pod is alive and whether it's ready for traffic.

Tip

Takeaway — stateless → Deployment. Stateful → StatefulSet. Per-node agent → DaemonSet. Run-once → Job. Scheduled → CronJob. Choosing correctly prevents entire categories of production pain.

📚 Read more — go deeper (1)

Config, networking & storage

ConfigMaps and Secrets, Services and Ingress, PersistentVolumes, namespaces and RBAC.

Once Pods run, three practical concerns decide whether your app actually works in Kubernetes.

Configuration: never bake config into images. Inject it with ConfigMaps (plain settings) and Secrets (passwords, tokens — base64-encoded, and ideally encrypted at rest). Your Pods read them as environment variables or mounted files, so the same image runs in dev and prod with different config.

Networking: every Pod gets an IP, but Pods die and get replaced, so you never talk to them directly. A Service gives a stable virtual IP and DNS name that load-balances across healthy Pods. To expose HTTP to the outside world, an Ingress (or the newer Gateway API) routes external traffic by hostname/path to the right Service — one entry point, TLS termination included.

Storage: containers are ephemeral, so for databases you need PersistentVolumes and PersistentVolumeClaims, which attach durable disks that survive Pod restarts.

Finally, namespaces partition a cluster for different teams/environments, and RBAC controls who can do what — essential once more than one person shares a cluster.

Tip

Takeaway — ConfigMaps/Secrets for config, Services + Ingress for traffic, PersistentVolumes for data. These four turn "a running Pod" into "a real, configurable, reachable application."

📚 Read more — go deeper (2)

Resources, limits & autoscaling

Requests vs limits (and OOMKilled), then HPA for pods, VPA for right-sizing, and the Cluster Autoscaler for nodes.

Kubernetes can scale for you — but only if you tell it what your Pods need. That starts with resource requests and limits:

resources:
  requests: { cpu: "100m", memory: "128Mi" }   # what the scheduler reserves
  limits:   { cpu: "500m", memory: "512Mi" }   # the hard ceiling

Requests decide where a Pod is scheduled (the node must have that much free). Limits cap usage — exceed the memory limit and your container is OOMKilled; exceed CPU and it's throttled. Set no requests and the scheduler is flying blind, which causes most "random" cluster instability.

Then three autoscalers, operating at different levels:

  • HPA (Horizontal Pod Autoscaler) — adds/removes Pod replicas based on CPU, memory, or custom metrics (e.g. requests per second). This is the one you'll use most: handle more traffic by running more copies.
  • VPA (Vertical Pod Autoscaler) — adjusts the requests/limits of a Pod, i.e. right-sizes it. Useful for tuning, and it conflicts with HPA on the same metric — don't run both on CPU.
  • Cluster Autoscaler — adds/removes nodes when Pods can't be scheduled for lack of capacity. HPA is useless without it once the cluster is full.

Tip

Takeaway — always set requests and limits — they're the foundation. Then HPA scales Pods, Cluster Autoscaler scales nodes. Together they turn a traffic spike into a bigger bill instead of an outage.

📚 Read more — go deeper (2)

Helm & managed Kubernetes

Templated, versioned releases with one-command rollback — and letting a cloud run your control plane.

Writing raw YAML for every environment gets repetitive fast — you end up copy-pasting near-identical manifests for dev, staging, and prod. Helm is the package manager for Kubernetes that fixes this.

A Helm chart is a templated, versioned bundle of manifests. You keep one template and supply a values.yaml per environment:

helm install myapp ./chart -f values-prod.yaml
helm upgrade myapp ./chart --set replicaCount=5
helm rollback myapp 1     # instant rollback to a previous release

Charts make deployments reusable, parameterised, and versioned — and there are public charts for common software (Postgres, Redis, Prometheus) so you don't reinvent them.

Managed Kubernetes is the other half of being productive. Running your own control plane is hard and thankless, so use a cloud offering — EKS (AWS), GKE (Google), or AKS (Azure). They operate the control plane, handle upgrades, and integrate with cloud load balancers and storage, leaving you to focus on your workloads.

Tip

Takeaway — Helm gives you reusable, versioned deployments with one-command rollback; managed Kubernetes removes the operational burden of the control plane. In real jobs you'll almost always use both.

📚 Read more — go deeper (2)

Debugging Kubernetes

get → describe → logs → exec, and the five errors you'll actually meet: ImagePullBackOff, CrashLoopBackOff, Pending, OOMKilled, empty endpoints.

Kubernetes fails in a small number of recognisable ways. Learn the loop and the top errors and you can debug almost anything.

The loop — always in this order:

kubectl get pods                 # what state is it in?
kubectl describe pod <name>      # WHY — read the Events at the bottom
kubectl logs <name>              # what did the app itself say?
kubectl logs <name> --previous   # logs from the crashed instance
kubectl exec -it <name> -- sh    # go inside and look around
kubectl get events --sort-by=.metadata.creationTimestamp

describe is the one beginners skip and experts read first — the Events section usually states the problem in plain English.

The errors you will actually see:

  • ImagePullBackOff / ErrImagePull — Kubernetes can't fetch the image. Wrong name/tag, or missing registry credentials (imagePullSecrets).
  • CrashLoopBackOff — the container starts, dies, and restarts in a loop. It's an app problem: read logs --previous. Usually a bad config, a missing env var, or an unreachable dependency.
  • Pending — the scheduler can't place it: not enough CPU/memory on any node, or an unsatisfiable node selector / taint.
  • OOMKilled — it exceeded its memory limit. Raise the limit or fix the leak.
  • Service returns nothing — the Service's selector doesn't match the Pod's labels. Check with kubectl get endpoints <svc>: empty endpoints means the selector is wrong.

Tip

Takeaway — get → describe → logs → exec. Read the Events. Ninety percent of Kubernetes debugging is knowing those five errors and that one sequence.

📚 Read more — go deeper (2)

CRDs & operators

Teach Kubernetes new object types and encode operational expertise into a controller that never sleeps.

Kubernetes' real power isn't the built-in objects — it's that you can teach it new ones. This is how the ecosystem (Prometheus, cert-manager, database operators) extends the platform.

Two pieces:

  • A CRD (Custom Resource Definition) adds a new object type to the Kubernetes API. Define a Database kind, and kubectl get databases starts working as if it were built in.
  • An Operator is a controller that watches those objects and makes reality match them. It's the same reconciliation loop Kubernetes uses internally, but carrying your operational knowledge.

The insight: an operator encodes what a human expert would do. A Postgres operator doesn't just start a container — it provisions storage, configures replication, takes scheduled backups, handles failover, and performs safe version upgrades. You declare:

apiVersion: acid.zalan.do/v1
kind: postgresql
spec:
  numberOfInstances: 3
  version: "16"

…and the operator does everything a DBA would, continuously.

You'll use operators long before you write one (install them via Helm). When you do write one, frameworks like Kubebuilder or the Operator SDK generate the scaffolding in Go.

Tip

Takeaway — CRDs extend the API; operators automate the expertise. This is the pattern behind almost every serious piece of software running on Kubernetes — and the ceiling of what "automation" means.

📚 Read more — go deeper (3)

Note

🎓 Certifications to target after this stage

🛠️ Build it — practise this stage

  • Deploy a multi-service app to kind/minikube with Deployments, a StatefulSet database, Services, Ingress and probes.
  • Package it as a Helm chart with per-environment values, add an HPA, then break it on purpose and debug it from kubectl describe alone.

🚀 Stage 9 · Continuous Delivery, GitOps & releases

Ship to production safely, and roll back in seconds

🔵 Intermediate  ·  ⏱️ 2–3 weeks  ·  📚 3 lessons  ·  🧩 Core DevOps

✅ Finish first: Build, test & Continuous Integration, Kubernetes & orchestration

CI proves the code is good; CD gets it to users without breaking them. GitOps makes Git the source of truth, and release strategies make deploys boring.

▶ Open Stage 09 on the live site →

GitOps & Argo CD

Stop pushing to clusters. Let an in-cluster agent pull from Git, self-heal drift, and make rollback a git revert.

Traditional CD pushes: your pipeline holds cluster credentials and runs kubectl apply. GitOps inverts it — an agent inside the cluster continuously pulls the desired state from Git and reconciles reality to match.

The four principles (OpenGitOps):

  1. The system is declarative — everything is described as data, not scripts.
  2. Desired state is versioned in Git — one auditable source of truth.
  3. Changes are pulled automatically by an agent.
  4. The agent continuously reconciles — drift is detected and corrected.

Why this is better:

  • Git is the source of truth. To know what's in production, read the repo.
  • Rollback is git revert. No special tooling, no panic.
  • No cluster credentials in CI. The agent pulls, so your pipeline never holds production keys — a huge security win.
  • Drift self-heals. If someone hand-edits the cluster at 2 a.m., it's reverted automatically.

Argo CD is the leading tool (Flux is the other). You point it at a repo of manifests/Helm charts, and its UI shows every app as Synced/OutOfSync and Healthy/Degraded.

The standard layout is two repos: application source code in one, deployment manifests in another. CI builds and pushes an image, then bumps the image tag in the config repo — and Argo CD deploys it.

Tip

Takeaway — stop pushing to clusters; let clusters pull from Git. You get audit, rollback, drift correction, and no production credentials in CI — for free.

📚 Read more — go deeper (3)

Rolling, blue/green, canary & feature flags

Pick a strategy by the risk you're carrying — and decouple deploy from release with feature flags.

Getting new code into production without breaking users is its own engineering discipline. The strategy you choose trades cost against risk.

  • Recreate — stop the old, start the new. Simple, but causes downtime. Rarely acceptable.
  • Rolling update — replace instances gradually (Kubernetes' default). No downtime, but during the rollout both versions serve traffic — so your changes must be backwards-compatible.
  • Blue/green — run two complete environments. Blue is live; deploy to green, test it privately, then flip the load balancer. Rollback is flipping back — instant. Costs double the infrastructure during the switch.
  • Canary — release to a small slice of traffic (1% → 5% → 25% → 100%), watching error rates and latency at each step. Automatically roll back if metrics degrade. The safest option for high-traffic systems; tools like Argo Rollouts and Flagger automate the ramp.

Underpinning all of them: feature flags. Deploy the code dark (shipped but switched off), then enable it for internal users, then 1% of customers. This decouples deploy from release — the single biggest reliability upgrade a team can make, because a bad feature is switched off in seconds without a redeploy.

And every strategy needs backwards-compatible database migrations (expand → migrate → contract), or rollback becomes impossible.

Tip

Takeaway — rolling for routine changes, blue/green for instant rollback, canary for risky ones — and feature flags so releasing is a switch, not a deploy.

📚 Read more — go deeper (3)

Environments & artifact promotion

Dev → staging → prod with one artifact and many configs, approval gates, and ephemeral per-PR environments.

Code should not leap from a laptop to production. It walks through environments, each one a rehearsal that's closer to the real thing.

The usual ladder:

  • Development — fast, disposable, broken often. Where you build.
  • Staging / pre-production — as close to production as you can afford (same infra, same config shape, ideally anonymised production-like data). The last honest rehearsal.
  • Production — real users, real money.

The rule that makes this meaningful: promote the same artifact. The exact image tested in staging is the one deployed to production — you never rebuild. What changes between environments is only configuration (via env vars, ConfigMaps, or Helm values files), never the code.

build once → app:1.5.0
   ↓ deploy to dev      (values-dev.yaml)
   ↓ deploy to staging  (values-staging.yaml)
   ↓ approval gate 🔒
   ↓ deploy to prod     (values-prod.yaml)

Guard production with an approval gate and keep secrets per-environment and separately scoped — a staging credential must never open production.

Increasingly teams also use ephemeral environments: a temporary environment spun up per pull request, torn down on merge. Reviewers get a live URL to click, which catches what a diff never will.

Tip

Takeaway — one artifact, many configs, promoted up a ladder with a gate before production. If staging and production differ in anything but config, staging is lying to you.

📚 Read more — go deeper (1)

🛠️ Build it — practise this stage

  • Set up Argo CD to deploy your app from a config repo; change an image tag by commit and watch it sync.
  • Implement a canary release with automated rollback when the error rate rises, and put one feature behind a flag.

☁️ Stage 10 · Cloud platforms

Where the infrastructure actually lives

🔵 Intermediate  ·  ⏱️ 6–8 weeks  ·  📚 6 lessons  ·  🧩 Core DevOps

✅ Finish first: Networking & protocols, Linux & operating systems

Go deep on one provider (AWS is the safe default). Compute, storage, IAM, VPC design, serverless and data services — then map the concepts to GCP and Azure.

▶ Open Stage 10 on the live site →

Core cloud building blocks

EC2, S3, VPC, RDS, Lambda — the five primitives everything else composes from. Plus billing alarms on day one.

The cloud is where your infrastructure actually lives. Go deep on one provider — AWS is the safe default given its market share — and the concepts transfer to the others.

The primitives everything else is built from:

  • Compute — EC2: virtual servers you rent by the second. (Plus serverless Lambda: run code with no server to manage.)
  • Storage — S3: effectively infinite object storage for files, backups, and static sites.
  • Networking — VPC: your private, isolated network — subnets, route tables, and security groups.
  • Database — RDS: managed SQL databases (Postgres, MySQL) so you don't operate the engine yourself.
  • Identity — IAM: who can do what. This is the one to learn properly — most cloud breaches are misconfigured permissions.

The golden rule everywhere is least privilege: grant the minimum permissions needed, use roles instead of long-lived keys, and turn on a billing alarm on day one so a mistake doesn't become a shocking invoice. Frame every design against the Well-Architected Framework (reliability, security, cost, performance, operations).

Tip

Takeaway — compute, storage, network, database, identity — every cloud is a variation on these five. Nail IAM and cost controls early; they're where beginners get burned.

📚 Read more — go deeper (2)

IAM & cloud identity

Users, roles, policies and least privilege. Most cloud breaches are misconfigured permissions — learn this properly.

Most cloud breaches are not clever exploits — they're misconfigured permissions. IAM is the single highest-stakes thing you'll configure, and it's worth genuine study.

The building blocks (AWS names; every cloud has equivalents):

  • Users — humans. Groups — collections of users.
  • Roles — a set of permissions that can be assumed temporarily, by a person, a service, or an EC2 instance/Pod.
  • Policies — JSON documents granting or denying specific actions on specific resources.

The rules that keep you safe:

  • Least privilege. Grant the minimum action on the minimum resource. Start from nothing and add; never start from * and trim. "Action": "*" on "Resource": "*" is how companies end up in the news.
  • Roles over long-lived keys. A static access key that leaks into a Git repo is a permanent breach. Roles issue short-lived, auto-rotating credentials. Give applications a role (IAM Roles for Service Accounts on EKS, Workload Identity on GKE), never a key file.
  • MFA everywhere, and never use the root account for daily work — lock it away.
  • Audit. Turn on CloudTrail (every API call is logged), and use access analysers to find unused permissions and prune them.

Tip

Takeaway — identity is the security perimeter in the cloud. Least privilege, temporary role-based credentials, no long-lived keys, and full audit logging. Get IAM right and most other risks shrink dramatically.

📚 Read more — go deeper (1)

VPC design & cloud networking

Public vs private subnets, NAT, security groups, multi-AZ, peering and PrivateLink — the backbone diagram of production.

Cloud networking is where the theory from Stage 3 becomes a design you're responsible for. Get it wrong and you're either exposed to the internet or unable to talk to your own database.

A VPC is your private, isolated network in the cloud, defined by a CIDR block (e.g. 10.0.0.0/16). Inside it you carve subnets, and the fundamental split is:

  • Public subnet — has a route to an Internet Gateway. Put only internet-facing things here: load balancers, bastion hosts.
  • Private subnet — no direct route in from the internet. Put everything else here: app servers, and above all databases. Private resources reach out (for updates) through a NAT Gateway.

That two-tier pattern is the default for a reason: your database should be unreachable from the internet by construction, not by a firewall rule someone might loosen.

Traffic is controlled by security groups (stateful, attached to resources — the main tool) and NACLs (stateless, at the subnet level). Deploy across multiple Availability Zones so one datacentre failure doesn't take you down.

To connect networks: VPC peering (simple, one-to-one), Transit Gateway (a hub for many VPCs), and PrivateLink/VPC endpoints — which let you reach cloud services (like S3) without traversing the public internet at all.

Tip

Takeaway — public subnet for load balancers, private subnet for apps and databases, NAT for outbound, multi-AZ for resilience. That single diagram is the backbone of nearly every production cloud architecture.

📚 Read more — go deeper (1)

Serverless & event-driven

Lambda and Cloud Run, event triggers, cold starts and lock-in — and when serverless genuinely wins.

Serverless doesn't mean no servers — it means you don't manage them. You hand the provider a function or a container, and it handles provisioning, scaling, and patching. You pay only for what you actually execute.

The core service is Functions as a Service — AWS Lambda, Google Cloud Functions, Azure Functions. Your code runs in response to an event: an HTTP request via an API Gateway, a file landing in S3, a message on a queue, a cron schedule.

def handler(event, context):
    bucket = event["Records"][0]["s3"]["bucket"]["name"]
    key    = event["Records"][0]["s3"]["object"]["key"]
    make_thumbnail(bucket, key)
    return {"statusCode": 200}

Where serverless wins: spiky or unpredictable traffic (it scales from zero to thousands automatically), event-driven glue, scheduled tasks, and anything where paying for an idle server is silly.

Where it hurts: cold starts (an idle function takes time to boot — bad for latency-critical paths), execution time limits (~15 min), vendor lock-in, and the fact that a large serverless system becomes a distributed system that is genuinely hard to debug and test locally. At sustained high volume it can also cost more than a plain server.

The middle ground is serverless containers — AWS Fargate, Google Cloud Run — where you ship a normal container image and still don't manage nodes. This is often the sweet spot.

Tip

Takeaway — serverless for spiky, event-driven, and glue work; containers for steady, long-running services. Use cold starts and cost curves — not fashion — to decide.

📚 Read more — go deeper (2)

Databases, caching & queues

Managed SQL vs NoSQL, Redis in front, and queues/streams (SQS, Kafka) so failures buffer instead of cascading.

Applications are stateless and disposable; data is neither. Choosing and operating the right data store is where a lot of real engineering happens.

Databases

  • Relational (SQL) — Postgres, MySQL, via managed services like RDS/Cloud SQL. Strong consistency, transactions, joins. This should be your default; "boring" Postgres solves the overwhelming majority of problems.
  • NoSQL — DynamoDB, MongoDB, Cassandra. Enormous scale and flexible schemas, but you must design around your access patterns up front and give up joins and (often) strong consistency.

Whatever you pick, use the managed version. Running your own production database — backups, replication, failover, patching — is a full-time job you don't want.

Caching — Redis/Memcached (ElastiCache) sit in front of the database and serve hot data from memory. A cache turns a 200 ms query into a 1 ms lookup, and is the cheapest performance win available. The two hard parts are, famously, cache invalidation and naming things.

Queues and streams decouple services so a spike or a failure doesn't cascade:

  • Queues — SQS, RabbitMQ. One message, one consumer, processed once. Perfect for background jobs.
  • Streams — Kafka, Kinesis. An append-only log many consumers can read independently and replay. The backbone of event-driven architectures.

Tip

Takeaway — managed Postgres until proven otherwise, Redis in front of it, and a queue between services so failures buffer instead of cascading. And test that you can actually restore your backups.

📚 Read more — go deeper (2)

GCP, Azure & multi-cloud

The same primitives, different names. A translation table, and an honest take on multi-cloud hype.

Once AWS clicks, the other big clouds are the same ideas with different names — and being able to move between them makes you far more hireable.

A quick translation table:

Concept AWS Google Cloud Azure
Virtual server EC2 Compute Engine Virtual Machines
Object storage S3 Cloud Storage Blob Storage
Managed K8s EKS GKE AKS
Serverless Lambda Cloud Functions Functions
Identity IAM IAM Entra ID

GCP is loved for its networking and Kubernetes heritage (Google created K8s). Azure dominates enterprises already invested in Microsoft. You don't need to master all three — skim GCP and Azure enough to map concepts and read their docs.

"Multi-cloud" (running across providers) is sometimes reality and often hype — it adds real complexity, so it should be a deliberate choice, not a default. More common and pragmatic is picking one primary cloud and avoiding needless lock-in by preferring open tools (Kubernetes, Terraform, Postgres) over proprietary ones where it matters.

Tip

Takeaway — learn one cloud deeply, then map the vocabulary to the others. The primitives are universal; only the names and consoles change.

📚 Read more — go deeper (2)

Note

🎓 Certifications to target after this stage

🛠️ Build it — practise this stage

  • Design and build a two-tier VPC: load balancer in a public subnet, app and database private, across two AZs.
  • Deploy a containerised app to a managed service (ECS/Cloud Run) with least-privilege IAM roles and a billing alarm.

🏗️ Stage 11 · Infrastructure as Code

Provision infrastructure the way you write software

🟣 Advanced  ·  ⏱️ 4–5 weeks  ·  📚 6 lessons  ·  🧩 Advanced

✅ Finish first: Cloud platforms, Programming & scripting

Stop clicking in consoles. Declare infrastructure in code, review it in a PR, and apply it reproducibly — with the state, module, policy and testing discipline that real teams need.

▶ Open Stage 11 on the live site →

Terraform fundamentals

Providers, resources, variables, state, and the plan/apply workflow that makes infrastructure reviewable.

Clicking around a cloud console doesn't scale and can't be reviewed or repeated. Infrastructure as Code (IaC) with Terraform lets you declare your infrastructure in files, version it in Git, and apply it reproducibly across environments.

You write HCL describing the resources you want:

resource "aws_instance" "web" {
  ami           = "ami-0abc123"
  instance_type = "t3.micro"
  tags = { Name = "web-server" }
}

The workflow is three commands: terraform init (download providers), terraform plan (preview exactly what will change — read this every time), and terraform apply (make it real). To tear it all down: terraform destroy.

Terraform tracks what it manages in a state file. On any real team you store this remotely (e.g. an S3 bucket with locking) so state is shared and two people can't apply at once. As your config grows, factor it into reusable modules — a network module, a database module — that you call with different inputs per environment.

Tip

Takeaway — plan before apply, always — it's the diff between what exists and what you're asking for. IaC means your infrastructure is reviewable, repeatable, and recoverable, just like application code.

📚 Read more — go deeper (2)

Terraform at scale

Remote state with locking, reusable modules, per-environment state, drift, and reading the plan before it deletes prod.

Terraform on a real team looks nothing like the tutorial. Four things separate a toy config from production infrastructure.

Remote state with locking. The state file maps your code to real resources. Locally, it's a landmine — two engineers applying at once will corrupt it. Store it in an S3 bucket with DynamoDB locking (or Terraform Cloud), so state is shared, versioned, and only one apply runs at a time. State also contains secrets in plaintext — encrypt it and lock down access.

Modules. Stop copy-pasting. A module is a reusable component with inputs and outputs — a network module, a service module — called once per environment with different variables. This is the difference between 3,000 lines of duplication and 300 lines of intent.

Environment separation. Keep dev, staging, and prod in separate state files so a mistake in dev can never destroy prod. Do this with separate directories (clear, explicit — usually the better choice) or workspaces (one config, many states).

Drift and safety. Someone will click in the console. terraform plan detects that drift. Protect critical resources with prevent_destroy, and read every plan — the line that matters is Plan: 0 to add, 0 to change, 1 to destroy, and that destroy might be your database.

Tip

Takeaway — remote locked state, modules for reuse, separate state per environment, and read the plan every single time. That's the difference between IaC that scales and IaC that deletes production.

📚 Read more — go deeper (3)

Configuration management with Ansible

Agentless config over SSH: inventories, playbooks, roles — and idempotency that makes re-running safe.

Terraform provisions infrastructure (create the servers); Ansible configures what's on it (install packages, edit config, deploy apps). They're complementary.

Ansible's big win is being agentless — it connects over plain SSH, so there's nothing to install on the targets. You describe the desired state in a playbook (YAML) and Ansible makes it so:

- hosts: webservers
  become: true
  tasks:
    - name: Ensure nginx is installed
      apt: { name: nginx, state: present }
    - name: Ensure nginx is running
      service: { name: nginx, state: started, enabled: true }

An inventory lists your hosts (grouped, e.g. webservers, dbservers), and roles package reusable sets of tasks so you can share and compose configuration.

The critical property is idempotency: running a playbook twice changes nothing the second time, because tasks declare a desired state ("nginx present"), not imperative steps ("run apt install"). That's what makes it safe to re-run against production.

Tip

Takeaway — use Terraform to build the servers, Ansible to configure them — and lean on idempotency so applying the same playbook is always safe.

📚 Read more — go deeper (1)

Immutable infrastructure & Packer

Bake golden images and replace servers instead of patching them. Cattle, not pets.

There are two philosophies for managing servers. In the mutable model you patch and tweak running servers over time — which slowly produces "snowflake" servers no two of which are alike, and nobody remembers how they got that way.

The modern approach is immutable infrastructure: you never modify a running server. To change anything, you build a brand-new machine image and replace the old one wholesale. This kills configuration drift, makes rollbacks trivial (boot the previous image), and guarantees every server is identical.

Packer (by HashiCorp) is the tool for baking these "golden images." You define a template that starts from a base OS, runs your provisioning (often an Ansible playbook), and outputs a ready-to-boot image — an AWS AMI, a Docker image, etc. Your Terraform then launches instances from that image.

The pattern ties the whole stack together:

Packer (bake image) → Terraform (launch it) → replace, don't patch

Tip

Takeaway — treat servers as disposable, identical, and replaceable — "cattle, not pets." Immutable images plus IaC give you fleets you can rebuild from scratch at any time, which is exactly what you want at 3 a.m. during an incident.

📚 Read more — go deeper (1)

OpenTofu, Pulumi, CDK & Crossplane

The IaC landscape and how to choose deliberately — real languages, cloud-native tools, and Kubernetes-driven infra.

Terraform is the default, not the only option. Knowing the landscape lets you choose deliberately — and read someone else's stack.

  • OpenTofu — the open-source fork of Terraform, created after HashiCorp changed its licence. Drop-in compatible, and increasingly the neutral default.
  • Pulumi — infrastructure in real programming languages (TypeScript, Python, Go). You get loops, conditionals, classes, and unit tests naturally, rather than fighting HCL's limits. The trade-off: full language power invites full language complexity.
  • AWS CDK — define AWS infra in code that synthesises to CloudFormation. Excellent if you're all-in on AWS; useless anywhere else.
  • CloudFormation / ARM / Bicep — the clouds' own native tools. Deeply integrated and always support new services on day one, but they lock you in and are typically more verbose.
  • Crossplane — manages cloud infrastructure from inside Kubernetes, using CRDs. Provision a database with kubectl apply. Powerful for platform teams already living in Kubernetes.

How to choose: declarative HCL (Terraform/OpenTofu) is the safe, hireable default — the biggest ecosystem and the most jobs. Reach for Pulumi when your infra logic is genuinely complex, CDK when you're AWS-only and your team are developers, and Crossplane when you're building a Kubernetes-native platform.

Tip

Takeaway — learn Terraform/OpenTofu properly first — the concepts (declarative state, plan/apply, modules) transfer to every alternative. The tool is a detail; the model is the skill.

📚 Read more — go deeper (3)

Testing & policy for infrastructure

tflint, checkov and tfsec on every PR; OPA/Conftest for rules that must never break; Terratest for real verification.

Infrastructure code can delete a database. It deserves the same rigour as application code — tested, linted, and governed before it applies.

Static analysis catches problems without touching the cloud:

  • terraform validate and tflint — syntax and provider-specific mistakes.
  • checkov, tfsec, Trivy — security misconfigurations: a public S3 bucket, an unencrypted volume, a security group open to 0.0.0.0/0. Run these on every PR.

Policy as code enforces the rules that must never be broken. OPA/Conftest (or Sentinel) let you write them as code and fail the pipeline:

"All resources must have an owner tag." "No security group may allow 0.0.0.0/0 on port 22." "Production databases must have deletion protection enabled."

This turns tribal knowledge and code-review vigilance into an automated gate that never gets tired or distracted.

Testing goes further: Terratest (Go) actually provisions infrastructure in a sandbox account, asserts it behaves (the endpoint responds, the bucket is private), and destroys it. Slower, but it's the only way to verify a module genuinely works.

The pipeline shape: validate → lint → security scan → policy check → plan → human review → apply.

Tip

Takeaway — never let apply be the first thing that checks your infrastructure. Lint, scan, and policy-gate every PR — the cost of catching a public S3 bucket in CI versus in the news is not comparable.

📚 Read more — go deeper (3)

Note

🎓 Certifications to target after this stage

🛠️ Build it — practise this stage

  • Provision your whole cloud environment in Terraform with remote locked state and reusable modules; destroy and rebuild it from scratch.
  • Add checkov + an OPA policy ('no 0.0.0.0/0 on port 22', 'all resources tagged') to CI and make a violating PR fail.

📈 Stage 12 · Observability & monitoring

You can't operate what you can't see

🟣 Advanced  ·  ⏱️ 3–4 weeks  ·  📚 5 lessons  ·  🧩 Advanced

✅ Finish first: Kubernetes & orchestration, Cloud platforms

Metrics, logs and traces — then the hard parts: defining reliability with SLOs, alerting on symptoms rather than noise, and profiling down to the offending line of code.

▶ Open Stage 12 on the live site →

Metrics & dashboards (Prometheus + Grafana)

Scraping, PromQL, the RED and USE methods, and dashboards that answer real questions.

You can't operate what you can't see. Observability rests on three pillars — metrics, logs, and traces — and metrics are where you start.

A metric is a number over time: request rate, error rate, latency, CPU, memory. The de-facto open-source stack is Prometheus (collects and stores metrics) plus Grafana (visualises them).

Prometheus works by scraping — it periodically pulls a /metrics endpoint that your apps expose. You then query with PromQL:

# 95th-percentile request latency over 5 minutes
histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m]))

Grafana turns those queries into dashboards and, crucially, alerts — fire a notification when error rate crosses a threshold or latency spikes. A good starting framework is the RED method for services (Rate, Errors, Duration) and USE for resources (Utilization, Saturation, Errors).

The discipline: alert on symptoms users feel (high latency, errors), not on every low-level cause, or you'll drown in noise and start ignoring pages.

Tip

Takeaway — instrument request rate, errors, and latency first; visualise them in Grafana; alert only on what hurts users. That covers 80% of real-world monitoring value.

📚 Read more — go deeper (2)

Logs, traces & OpenTelemetry

Structured logs, distributed tracing across services, and OTel as the vendor-neutral standard for all three pillars.

Metrics tell you that something is wrong; logs and traces tell you what and where.

Logs are timestamped event records. On one server you tail a file; across a fleet you ship them to a central system — the ELK stack (Elasticsearch, Logstash, Kibana) or Grafana Loki — so you can search everything in one place. Emit structured logs (JSON with fields) rather than free text, so they're queryable, and include a request ID to tie events together.

Traces follow a single request as it hops across many services — essential in microservices, where one user click might touch ten backends. A trace shows the whole journey and how long each hop took, so you can pinpoint which service caused the slowdown. Jaeger and Tempo visualise them.

OpenTelemetry (OTel) is the unifying standard: one vendor-neutral set of libraries and agents to emit metrics, logs, and traces. Instrument with OTel once and you can send data to any backend, avoiding lock-in.

Tip

Takeaway — metrics to detect, traces to locate, logs to explain — the three pillars together. Standardise on OpenTelemetry so you instrument once and stay free to switch tools.

📚 Read more — go deeper (2)

SLIs, SLOs & error budgets

Turn 'reliable' into a number, and let the error budget decide when to ship features vs fix reliability.

"Reliable" is meaningless until you make it a number. That's what SLIs, SLOs, and error budgets do — they turn reliability into an engineering decision instead of an argument.

  • An SLI is a measured indicator of health: e.g. the proportion of requests that are both successful and faster than 300 ms.
  • An SLO is your target for that SLI over a window: e.g. 99.9% over 30 days. Note that 99.9% still allows ~43 minutes of "bad" time a month — 100% is the wrong target; it's infinitely expensive.
  • The error budget is what's left: 100% − SLO. That budget is a currency you get to spend.

The mechanism is elegant: if you have budget remaining, you can take risks and ship features fast. If you've burned it, you stop shipping and invest in reliability until you're back in the black. It aligns product and ops around one shared number, and it tells you when to page a human — when you're burning budget fast enough to blow the SLO.

Tip

Takeaway — don't chase 100%. Pick an SLO that matches what users actually need, measure it honestly, and let the error budget decide when to push features versus fix reliability.

📚 Read more — go deeper (1)

Alerting, runbooks & on-call

Page on symptoms not causes, burn-rate alerts, runbooks on every alert, and an on-call rotation that's actually humane.

Monitoring that pages you for everything is worse than no monitoring — because you learn to ignore it. Alert fatigue is the number-one cause of missed real incidents.

Alert on symptoms, not causes. A page should mean users are being hurt right now: error rate is up, latency is up, the queue is backing up beyond recovery. "CPU is at 90%" is not an alert — if CPU is high and users are fine, nothing is wrong. Put causes on dashboards; put symptoms on pagers.

Every page must be actionable and urgent. The test: if the human can't do anything about it, or it can wait until Monday, it is not a page — make it a ticket or a dashboard. A good alert links to a runbook: what this alert means, how to confirm it, what to check, how to mitigate.

Tie alerts to SLOs: rather than static thresholds, alert on error-budget burn rate — "we're consuming the month's budget in hours." That fires for the things that genuinely threaten your objective, and stays quiet for harmless blips.

On-call must be humane and sustainable: a fair rotation, a real handover, compensation, and — critically — the authority and time to fix the things that woke you. If the same alert fires every week and nobody's allowed to fix the cause, the rotation is broken.

Tip

Takeaway — page on user-facing symptoms, link every alert to a runbook, alert on error-budget burn, and treat repeat pages as bugs. An on-call that sleeps is a system that works.

📚 Read more — go deeper (2)

APM, profiling & finding the bottleneck

Read a flame graph, and learn the usual suspects — starting with the N+1 query that's slowing half the internet.

Metrics say "checkout is slow." Traces say "it's the payment service." Profiling says "it's this function, on line 42, allocating memory in a loop." That last mile is where the fix lives.

APM (Application Performance Monitoring) — Datadog, New Relic, Grafana Pyroscope, Elastic APM — instruments your app and shows request throughput, error rates, slow transactions, and a breakdown of where the time actually went (database, external call, CPU).

Learn to read a flame graph: the x-axis is time spent (wider = more time), and the y-axis is the call stack. The widest bar is your bottleneck. It's the fastest way to find a performance problem in an unfamiliar codebase.

The usual culprits, in the order you should suspect them:

  1. The N+1 query — code that runs one database query per row in a loop. It's fast with 10 rows in dev and catastrophic with 10,000 in production. This is the most common performance bug in the world.
  2. A missing database index — a full table scan on a big table.
  3. A missing cache on a hot, expensive path.
  4. Blocking I/O on the request thread.
  5. Memory leaks and excessive garbage collection.

Continuous profiling runs in production at low overhead, so you can look at yesterday's incident rather than trying to reproduce it.

Tip

Takeaway — metrics detect, traces localise, profiles pinpoint. And check for the N+1 query first — it's the answer far more often than it has any right to be.

📚 Read more — go deeper (2)

🛠️ Build it — practise this stage

  • Instrument an app with Prometheus metrics + OTel traces, build a RED dashboard, and alert on error-budget burn.
  • Write an SLO with a runbook, then deliberately break the app and follow your own runbook to recovery.

🔒 Stage 13 · Security & DevSecOps

Shift security left — automate it, don't bolt it on

🟣 Advanced  ·  ⏱️ 3–4 weeks  ·  📚 4 lessons  ·  🧩 Advanced

✅ Finish first: Cloud platforms, Build, test & Continuous Integration

Security becomes everyone's job and moves into the pipeline: secrets that never touch Git, scanning that fails builds, a supply chain you can prove, and compliance as a by-product of how you work.

▶ Open Stage 13 on the live site →

Security in the pipeline

SAST, DAST, dependency and image scanning, policy as code — automated gates instead of a human bottleneck.

DevSecOps means security is everyone's job and it "shifts left" — built into the pipeline from the first commit, not bolted on before release. The goal is to catch problems automatically, early, and cheaply.

The practices you'll add to a pipeline:

  • Secrets management: never commit passwords or keys. Store them in a vault (HashiCorp Vault, cloud secret managers) and inject at run time. Scan the repo for accidental leaks.
  • Scanning: SAST analyses your source, DAST tests the running app, and dependency/image scanners (Trivy, Snyk) find known vulnerabilities (CVEs) in your libraries and containers.
  • Policy as code: tools like OPA let you write rules ("no public S3 buckets", "images must be signed") that the pipeline enforces automatically.
  • Supply-chain security: generate an SBOM (a bill of materials of everything in your build) and sign artifacts (Sigstore) so you can prove what you shipped. SLSA is the framework here.

Underneath it all is least privilege — the same principle from IAM, applied everywhere.

Tip

Takeaway — wire secrets scanning, dependency/image scanning, and policy checks into CI so security is automatic and continuous. Security that depends on humans remembering will eventually fail.

📚 Read more — go deeper (3)

Secrets management

Never in Git. Vault and cloud secret managers, runtime injection, rotation, and short-lived dynamic credentials.

A secret in Git is a breach. Once committed it's in the history, on every clone, and bots scan public repos for keys within seconds of a push. Secrets management is the discipline of never being in that position.

The rules:

  • Never commit secrets. Enforce it mechanically with pre-commit hooks and scanners (gitleaks, trufflehog) in CI — humans will forget.
  • Inject at runtime, via environment variables or mounted files, never baked into an image or a config file.
  • Rotate regularly, and automatically if you can. A secret that can't be rotated quickly is a secret you can't respond to.
  • Scope tightly. A staging credential must not open production.

The tools:

  • HashiCorp Vault — the heavyweight. Central storage, fine-grained policies, full audit log, and its killer feature: dynamic secrets — Vault generates a unique, short-lived database credential per application on demand and revokes it after use. A leaked credential expires by itself.
  • Cloud secret managers — AWS Secrets Manager, GCP Secret Manager, Azure Key Vault. Simpler, and integrate directly with IAM roles.
  • Kubernetes: a plain Secret is only base64-encoded — not encrypted. Enable encryption at rest, and use the External Secrets Operator to sync from a real vault, or Sealed Secrets if you must store an encrypted secret in Git safely.

If a secret leaks: rotate it first, investigate second. Deleting the commit does not help — assume it's already harvested.

Tip

Takeaway — scan to prevent, inject at runtime, rotate automatically, and prefer short-lived dynamic credentials. The best secret is one that's already expired by the time an attacker finds it.

📚 Read more — go deeper (3)

Software supply-chain security

SBOMs, pinned digests, signed artifacts and provenance — because you didn't write most of the code you ship.

You didn't write most of the code you ship. A typical application is a thin layer of your logic on top of hundreds of third-party dependencies and a base image you didn't build. The software supply chain is your attack surface.

The threats are real and recent: typosquatting (a malicious package named one character off a popular one), dependency confusion (a public package shadowing your internal one), compromised maintainers pushing a backdoored update, and build system compromise (SolarWinds — the source was clean, the build was poisoned).

The defences, in order of value:

  • SBOM (Software Bill of Materials) — a machine-readable inventory of every component in your artifact (SPDX/CycloneDX; generate with Syft or Trivy). When the next Log4Shell lands, an SBOM answers "are we affected, and where?" in minutes instead of a frantic week.
  • Pin and verify dependencies. Use lockfiles; pin base images by digest (@sha256:…), not a mutable tag.
  • Sign your artifacts. Sigstore/cosign signs images and generates provenance — a verifiable statement of what was built, from which commit, by which pipeline. Then have your cluster refuse to run unsigned images (admission policy).
  • Harden the build. The build system holds the keys to everything; treat it as production. SLSA is the framework that grades exactly this.

Tip

Takeaway — know what's in your artifact (SBOM), pin it, sign it, and verify signatures at deploy. Trusting an unverified image is trusting every person who ever touched it.

📚 Read more — go deeper (3)

Compliance & governance as code

SOC 2, ISO 27001, GDPR — and how reviewed commits, IAM policies, audit logs and tested backups become the evidence.

At some point a customer, auditor, or regulator asks: prove your systems are secure. If that proof is a person manually collecting screenshots for six weeks, you've lost. Compliance as code turns it into an artifact of how you already work.

The frameworks you'll hear named: SOC 2 (the common one for SaaS — security, availability, confidentiality), ISO 27001 (international infosec standard), GDPR (EU personal data), HIPAA (US health data), PCI-DSS (card payments). They differ in detail but demand the same underlying things.

Nearly every control reduces to evidence you can already produce automatically:

  • Access control — least-privilege IAM, MFA, reviewed regularly.
  • Change management — every production change is a reviewed, approved, traceable commit. (Your Git history + PR approvals is this evidence — which is why GitOps makes auditors happy.)
  • Audit logging — CloudTrail and friends, retained and tamper-resistant.
  • Encryption — in transit (TLS) and at rest.
  • Backups and DR — proven by actual restore tests.
  • Vulnerability management — scanning in CI, with documented remediation SLAs.

Enforce it continuously with policy as code (OPA, cloud config rules) so a non-compliant resource is blocked at creation, and use drift detection so the environment can't silently fall out of compliance between audits.

Tip

Takeaway — don't "prepare for the audit" — build a system whose normal operation produces the evidence. Reviewed commits, IAM policies, audit logs, and tested backups are 80% of any framework.

📚 Read more — go deeper (2)

Note

🎓 Certifications to target after this stage

🛠️ Build it — practise this stage

  • Add secret scanning, dependency scanning and image scanning to CI — and make a PR with a hard-coded key fail.
  • Sign your container images with cosign, generate an SBOM, and make the cluster refuse unsigned images.

🛡️ Stage 14 · Reliability & SRE

Failure is inevitable — recovery is engineering

🟣 Advanced  ·  ⏱️ 3–4 weeks  ·  📚 4 lessons  ·  🧩 Advanced

✅ Finish first: Observability & monitoring

Incidents, blameless postmortems, deliberately breaking your own systems, backups you've actually restored, and knowing where the capacity cliff is before launch day.

▶ Open Stage 14 on the live site →

Incidents & blameless postmortems

Detect, respond, communicate, learn. Design for failure with timeouts, retries and circuit breakers.

Reliability engineering is what separates "it deployed" from "it stays up." Failure is inevitable, so you engineer to absorb and recover from it rather than pretending it won't happen.

Incident response is a discipline: detect fast (good alerting), have a clear on-call rotation, assign an incident commander for big ones, communicate status, and — most importantly — write a blameless postmortem afterward. Blameless means you fix the system and process, not punish the person; people are honest only when they're not afraid, and honesty is how you actually learn.

To find weaknesses before users do, practise chaos engineering: deliberately inject failure (kill a server, add latency, cut a dependency) in a controlled way and verify the system copes. Netflix's Chaos Monkey made this famous. It sounds reckless; done well it's the opposite — it turns "we think we're resilient" into evidence.

Design for failure with timeouts, retries with backoff, circuit breakers, and graceful degradation, so one broken dependency doesn't cascade into a full outage.

Tip

Takeaway — you will have incidents — optimise for fast recovery (MTTR) and honest, blameless learning. A team that runs game days and writes good postmortems gets more reliable every quarter.

📚 Read more — go deeper (2)

Chaos engineering

Hypothesis, small blast radius, measure, learn. Find the missing timeout before it finds you at 3 a.m.

You do not know your system is resilient until you've broken it on purpose. Chaos engineering is the practice of injecting failure deliberately, in a controlled way, to find weaknesses before they find you at 3 a.m.

It is not "randomly break production." It's a scientific method:

  1. Define steady state — a measurable signal of healthy (e.g. checkout success rate > 99%).
  2. Hypothesise: "If one payment-service Pod is killed, the success rate will not change."
  3. Inject the smallest failure — kill one Pod. Start in staging; limit the blast radius in production to a tiny slice of traffic.
  4. Measure. Did steady state hold?
  5. Learn and fix. If it broke, you've found a real bug cheaply, on your terms, during working hours.

Netflix's Chaos Monkey (which kills random instances continuously) made the idea famous; Litmus, Chaos Mesh, and AWS FIS are modern tools. Typical experiments: kill instances, add network latency, drop a dependency, fill a disk, exhaust CPU, simulate an entire AZ failure.

The findings are always the same shape and always valuable: a missing timeout, a retry storm, a hidden single point of failure, a health check that lies, an alert that never fired.

A gentler on-ramp is a game day — a scheduled, human-run exercise where the team simulates an outage and practises the response.

Tip

Takeaway — always have a hypothesis, start small, limit blast radius, and be able to stop instantly. Failure is going to happen — chaos engineering just means you schedule it, with a rollback plan and coffee.

📚 Read more — go deeper (2)

Backups & disaster recovery

RPO/RTO, the 3-2-1 rule, immutable off-site copies — and the restore rehearsal that proves it all works.

Everything above is worthless if you lose the data. Backups are the last line of defence — and an untested backup is not a backup.

Two numbers define your strategy, and the business must set them:

  • RPO (Recovery Point Objective) — how much data can you afford to lose? Hourly backups mean an RPO of one hour.
  • RTO (Recovery Time Objective) — how long can you afford to be down? This drives how much standby infrastructure you pay for.

The classic rule is 3-2-1: three copies, on two different media, one off-site — and, in the ransomware era, one immutable/offline copy that a compromised admin account cannot delete. Automated backups that your production credentials can erase will be erased by an attacker.

Test the restore. On a schedule. The graveyard of companies is full of teams whose backups had been silently failing for months, or who discovered the restore takes 40 hours. A backup you have never restored is a hope, not a plan.

Disaster recovery scales the same idea to a whole region, and the tiers cost accordingly:

  • Backup & restore — cheapest, slowest (hours–days).
  • Pilot light — core systems (like a replicated database) always running, the rest started on demand.
  • Warm standby — a scaled-down full copy, ready to scale up (minutes).
  • Multi-site active/active — full capacity in two regions, near-zero RTO, roughly double the cost.

Write the runbook and rehearse the failover — during an actual disaster is the worst time to read your own documentation.

Tip

Takeaway — define RPO/RTO with the business, automate backups, keep one immutable off-site copy, and rehearse the restore. The only backup that counts is the one you've successfully restored.

📚 Read more — go deeper (2)

Load testing & capacity planning

Load, stress, soak and spike tests; find the bottleneck, keep real headroom, and fix the query before buying hardware.

"Will it survive launch day?" is a question you should be able to answer with data, not vibes.

Load testing is how you get that data — and there are distinct kinds, each answering a different question:

  • Load test — expected peak traffic. Does it hold up?
  • Stress test — push past the limit. Where does it break, and does it fail gracefully (shedding load, returning 503s) or catastrophically (falling over and corrupting things)?
  • Soak test — sustained load for hours. This is how you catch memory leaks and connection-pool exhaustion, which look fine for ten minutes and die overnight.
  • Spike test — instant 10× traffic. Can your autoscaler react in time? (Usually not as fast as you hope — scaling takes minutes; a spike takes seconds. This is why you pre-scale for known events.)

Tools: k6, Gatling, Locust, JMeter. Run them in CI against staging so a performance regression is caught in a PR, not by users.

Capacity planning turns the results into decisions: measure your headroom (how much spare capacity you run with), find the bottleneck (CPU, memory, database connections, IOPS, a third-party API's rate limit), and model growth. A well-run service typically targets ~50% utilisation at peak so it can absorb a surge and lose a node without drama.

Then remember: performance is a feature, and it is also a cost. The cheapest capacity is code that does less work — an index, a cache, a fixed N+1 query is often worth more than doubling the servers.

Tip

Takeaway — load, stress, soak, and spike tests tell you where the cliff is. Keep real headroom, know your bottleneck, and fix the query before you buy the hardware.

📚 Read more — go deeper (2)

🛠️ Build it — practise this stage

  • Run a game day: kill a Pod, add latency, fill a disk. Write a blameless postmortem for what you found.
  • Back up a production-like database, then actually restore it into a fresh environment and time the whole thing.

🧱 Stage 15 · Platform engineering & scale

Build the paved road other teams drive on

🟣 Advanced  ·  ⏱️ Ongoing  ·  📚 3 lessons  ·  🧩 Advanced

✅ Finish first: Kubernetes & orchestration, Infrastructure as Code, Observability & monitoring

The senior frontier: internal developer platforms with golden paths, service meshes for zero-trust networking, and FinOps so the cloud bill is engineered rather than endured.

▶ Open Stage 15 on the live site →

Platform engineering & internal developer platforms

Golden paths and self-service so product teams ship safely without becoming Kubernetes experts.

The frontier of DevOps is making all of this reusable so every team doesn't rebuild it. Three advanced areas define the senior end of the field.

Platform engineering builds an Internal Developer Platform — self-service "golden paths" (templated pipelines, infrastructure, and guardrails) so product developers ship safely without becoming Kubernetes experts. The measure of success is developer experience: how fast can a new service go from idea to production?

Service mesh (Istio, Linkerd) adds a networking layer between your services, handling mutual TLS, retries, traffic splitting, and rich telemetry without changing app code. It shines in large microservice estates — and it's genuine complexity, so adopt it only when you have the scale to justify it.

FinOps brings financial accountability to the cloud: tagging resources, watching spend, rightsizing, and using savings plans, so the cloud bill is a managed engineering concern rather than a quarterly shock. At scale, cost is a reliability and design constraint.

Tip

Takeaway — senior DevOps is about leverage — platforms that multiply other teams, meshes that standardise service communication, and FinOps that keeps it affordable. Add these when scale demands them, not before.

📚 Read more — go deeper (2)

Service mesh

mTLS, retries, canaries and uniform telemetry at the network layer — and an honest look at the complexity cost.

In a large microservice estate, every service needs the same boring things: retries, timeouts, mutual TLS, traffic splitting, and telemetry. Implementing that in every service, in every language, is a nightmare of duplicated, inconsistent code.

A service mesh moves it out of your application entirely. A sidecar proxy (usually Envoy) is injected next to every Pod, and all traffic flows through it. Your app just calls http://billing — the mesh handles everything else.

  • Control plane — where you configure policy (Istio, Linkerd).
  • Data plane — the proxies that actually move the traffic.

What you get without changing a line of application code:

  • mTLS everywhere — every service-to-service call is encrypted and mutually authenticated. This is the foundation of zero-trust networking, and it's the single biggest reason to adopt a mesh.
  • Traffic management — canary releases, weighted traffic splits, mirroring.
  • Resilience — retries, timeouts, and circuit breakers, consistently applied.
  • Observability — uniform metrics, logs, and traces for every call, for free.

The honest counterweight: a mesh is serious operational complexity, adds a network hop and latency, and gives you a new critical system that can itself break. It solves problems you only have at scale. Linkerd is markedly simpler than Istio if you decide you need one.

Tip

Takeaway — a mesh gives you mTLS, resilience, and uniform observability for free at the network layer — at the price of real complexity. Adopt it when the number of services makes doing this per-app worse than running a mesh. Not before.

📚 Read more — go deeper (2)

FinOps & cloud cost engineering

Tag everything, kill idle resources, rightsize, commit for the baseline, use spot. Cost is a non-functional requirement.

In the cloud, every engineering decision is a purchasing decision — and engineers, not finance, make them. FinOps is the practice of making cost a visible, owned engineering metric instead of a quarterly surprise.

Get visibility first. You cannot manage what you cannot attribute. Enforce a tagging policy (owner, environment, service) via policy-as-code, so every dollar maps to a team and a service. Untagged resources are orphans nobody will ever turn off.

Then the big levers, roughly in order of payoff:

  • Turn off what nobody uses. Idle dev/staging environments running nights and weekends, unattached storage volumes, old snapshots, forgotten load balancers. This is free money, and it's always there.
  • Rightsizing. Most instances are dramatically over-provisioned — a machine at 5% CPU is 95% waste. Use the metrics you already collect.
  • Commit for discounts. Once your baseline is predictable, Savings Plans / Reserved Instances cut 30–70% off compute for a 1–3 year commitment.
  • Spot instances — up to ~90% off for interruptible workloads (CI runners, batch jobs, stateless workers). Enormous, underused win.
  • Architecture. Data egress and cross-AZ traffic are silent budget killers. A missing cache means you're paying per query, forever.

Make it continuous: budgets and anomaly alerts so a runaway job is caught in hours, and showback dashboards so each team sees its own spend.

Tip

Takeaway — tag everything, kill idle resources, rightsize, commit for the baseline, spot for the interruptible. Cost is a non-functional requirement like latency — and the engineer who reduces the bill by 40% is very, very visible.

📚 Read more — go deeper (2)

🛠️ Build it — practise this stage

  • Build a golden path: a template repo that gives a new service CI, a Helm chart, dashboards and alerts on day one.
  • Audit a cloud account: tag everything, find idle resources, rightsize, and report the monthly savings.

🎯 Stage 16 · Career: get hired

Turn the skills into an offer

🟡 Any  ·  ⏱️ Ongoing  ·  📚 4 lessons  ·  🧩 Career

Do this alongside everything else. A capstone that proves you can build and run systems, interview technique that shows judgement, and the certifications that actually pay off.

▶ Open Stage 16 on the live site →

The DevOps career, end to end

What employers actually screen for, how projects beat certificates, and staying current once you're in.

Learning the tools is half the job; turning it into a career is the other half. The single most persuasive thing you can show is projects, not certificates.

Build a portfolio that proves the whole loop end-to-end. A strong capstone: an app that a CI pipeline builds and scans, Terraform provisions the infrastructure, Helm deploys it to Kubernetes, and Prometheus/Grafana monitors it — all in a public repo with a clear README explaining your decisions and trade-offs. One project like that beats a wall of course completions.

For interviews, practise out loud: devops-exercises has 2,600+ real questions, and you should be able to whiteboard the request path, debug a broken pipeline, and explain why you'd choose a tool, not just how to use it. Certifications (AWS, CKA, Terraform Associate) help pass HR filters and give structure — treat them as a bonus on top of projects.

Finally, stay plugged in: follow the CNCF ecosystem, read incident writeups, and keep one project running in production so you keep hitting real problems.

Tip

Takeaway — projects prove ability, certs open doors, and community keeps you current. Ship something real, write about it, and keep the loop turning — that's a DevOps career.

📚 Read more — go deeper (2)

Building a portfolio that gets interviews

One end-to-end capstone, an architecture diagram, and an honest write-up of the trade-offs you made.

Nobody hires a DevOps engineer for a certificate. They hire for evidence you can build and run systems. A portfolio is that evidence, and it's the highest-leverage thing you can do to get hired.

Build one serious capstone rather than five toy demos. The goal is a project that touches the whole loop, because that's the job:

A containerised app → CI pipeline (test, scan, build image, push to a registry) → infrastructure provisioned with Terraform → deployed to Kubernetes via Helm and Argo CD (GitOps) → monitored with Prometheus/Grafana → with an SLO, an alert, and a documented runbook.

That single project demonstrates every stage of this roadmap.

Then make it legible. An impressive repo nobody understands is worth nothing. Your README must have:

  • An architecture diagram (one picture, immediately).
  • What it does and how to run it in one command.
  • The decisions and trade-offs — why Terraform over CDK, why canary over blue/green, what you'd do differently at 100× scale. This section is what separates you from someone who followed a tutorial, and it's exactly what interviewers probe.
  • What broke and how you debugged it. Real engineers find this credible; polished perfection reads as copied.

Write it up in a blog post or a detailed README. Then keep one thing actually running in production (even a small one, on a cheap VPS) — real uptime, real certificates expiring, real 3 a.m. lessons.

Tip

Takeaway — one end-to-end capstone, documented with diagrams and honest trade-offs, beats a wall of course certificates. Build it, break it, write about it.

📚 Read more — go deeper (2)

Interview preparation

Fundamentals, troubleshooting out loud, and system design — they're hiring judgement, not trivia.

DevOps interviews test three distinct things. Prepare for each deliberately, because they fail you in different ways.

1. Fundamentals ("do you actually understand this?"). Expect: What happens when you type a URL and press Enter? Explain the difference between a container and a VM. What is a Kubernetes Service and why can't I just use the Pod's IP? TCP vs UDP. How does DNS resolution work? These reward the foundations in stages 1–5 — which is precisely why skipping them costs you jobs.

2. Troubleshooting ("are you useful at 3 a.m.?"). This is the one that decides the offer. You'll get a scenario — the site is returning 502s, what do you do? — and they are watching your method, not your answer. Be systematic and say it out loud: check the symptom, walk the request path (DNS → LB → app → DB), form a hypothesis, state which command you'd run and what result would confirm or eliminate it. Candidates who guess randomly fail; candidates who narrow methodically pass even without the right answer.

3. Design ("can you build something real?"). Design a CI/CD pipeline for a team of 30. How would you make this service highly available across regions? How do you do zero-downtime deploys with a database migration? Talk about trade-offs, not just tools — cost vs reliability, complexity vs need. Saying "I wouldn't use a service mesh here, it's not worth the complexity at this scale" is a strong signal.

Also expect live tasks (write a Dockerfile, fix a broken YAML, a Bash/Python script) and behavioural questions — have a real story about an incident you handled and what you changed afterwards.

Tip

Takeaway — they're hiring judgement, not trivia. Practise narrating your debugging out loud, always reason from the request path, and always name the trade-off.

📚 Read more — go deeper (2)

Which certifications, in what order

The efficient trio — AWS SAA, CKA, Terraform Associate — and how to study for them by building.

Certifications don't make you an engineer — projects do. But they get you past HR filters, give you a structured syllabus when you're lost, and some (the hands-on ones) genuinely teach you. Here's what's worth your money, in order.

The recommended sequence:

  1. AWS Certified Cloud Practitioner (optional, ~2 weeks) — broad, easy, good if you're brand new to cloud or coming from a non-technical background. Skip it if you're already comfortable.
  2. AWS Solutions Architect – Associate (~2 months) — the highest-ROI cloud certificate in the market. Widely recognised, and it forces you to learn the whole service landscape properly.
  3. CKA — Certified Kubernetes Administrator (~2 months) — the one that most changes your CV. It is entirely hands-on: a live cluster, real tasks, a time limit. You cannot pass by memorising, which is exactly why employers respect it. Practise on killer.sh, and learn to move fast in kubectl and the official docs (which you're allowed to use).
  4. HashiCorp Terraform Associate (~3 weeks) — cheap, quick, and directly relevant to the daily job.

Then specialise, if it matches your target role: CKS (Kubernetes security — requires an active CKA), AWS DevOps Engineer – Professional (advanced, pipeline- and reliability-heavy), or the Azure AZ-104/AZ-400 track if you're aiming at Microsoft-heavy enterprises.

How to actually use them: study by building, not by watching. Use the exam objectives as a checklist against real hands-on work, and let the certificate be a by-product of skills you genuinely acquired. A cert with no project behind it collapses in the first troubleshooting question.

Tip

Takeaway — SAA + CKA + Terraform Associate is the strongest, most efficient trio for a DevOps role. Get them because you learned the material, not instead of it.

📚 Read more — go deeper (3)

🛠️ Build it — practise this stage

  • Ship the capstone: app → CI (test/scan/build) → Terraform infra → Helm + Argo CD to Kubernetes → Prometheus/Grafana + an SLO.
  • Write it up with an architecture diagram, the trade-offs you chose, and what broke while you built it.

🎉 That's the whole path

DevOps is a practice, not a finish line. Turn your capstone into a portfolio case study, keep one project running in production, and give back to the community.

▶ Study it on the website →

Grounded in these excellent open resources:

Warning

Certification details and pricing change over time — always verify on each provider's official site.

🛠️ For developers — run or edit this project

This roadmap is a Next.js site (dark terminal-themed docs, one page per stage, progress tracking, an animated roadmap). This README is generated from the site's content — so the file you're reading and the live site can never drift apart.

npm install
npm run dev      # → http://localhost:3000
npm run build    # static export → ./out  (deploy anywhere)

To change the content, edit the source — never this file:

To change… Edit…
A lesson (the teaching content) content/lessons*.md
Stages, topics, prerequisites, links, certs, projects lib/roadmap-data.ts 
npm run gen:md   # regenerates this README.md from the source

Lessons are split across content/lessons.md, lessons-core.md and lessons-advanced.md (any content/lessons*.md file is loaded). The website reads the same source. gen:md fails loudly if a topic has no lesson, so ids can't silently drift.

Stack: Next.js 14 (App Router, static export) · React 18 · TypeScript · Tailwind CSS · marked.

This README is generated from lib/roadmap-data.ts + content/lessons*.md via npm run gen:md.

About

🚀 A free, complete DevOps course — 16 stages of full written lessons from beginner to advanced: Linux, CI/CD, Docker, Kubernetes, Terraform, cloud, SRE. Read on the site or in ROADMAP.md.

devops.nextjoblist.com

Topics

docker kubernetes aws devops cloud tutorial terraform nextjs ci-cd sre study-guide observability devsecops free-course tailwindcss gitops platform-engineering devops-roadmap devops-engineer learning-roadmap

Resources

Readme
Activity

Stars

2 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages