Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/workflows/verify-kubernetes-blue-green-deployment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: Verify kubernetes-blue-green-deployment

on:
push:
branches: ['main']
paths:
- 'projects/kubernetes-blue-green-deployment/**'
pull_request:
branches: ['main']
paths:
- 'projects/kubernetes-blue-green-deployment/**'
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Deploy blue/green to kind, cut over, and verify zero downtime
run: |
cd projects/kubernetes-blue-green-deployment
chmod +x demo_project.sh; ls -la
./demo_project.sh
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ This is the **third** repo of my DevOps trio repositories: [**tungbq/devops-basi
| 21 | Deploy a static website to AWS S3 | [#6](https://github.com/tungbq/devops-project/issues/6) | `AWS` `S3` `Static Website` | 🚧 Planned |
| 22 | Azure Monitoring & Dashboard (APIM + Container Apps) | [azure-monitoring-dashboard](./projects/azure-monitoring-dashboard/) | `Azure` `APIM` `Container Apps` `Monitoring` `Observability` | ✔️ Done |
| 23 | CI/CD Best Practices — GitHub Actions to Azure Container Apps | [cicd-best-practices-container-apps](./projects/cicd-best-practices-container-apps/) | `CI/CD` `GitHub Actions` `Azure` `Container Apps` `OIDC` `Terraform` | ✔️ Done |
| 24 | Kubernetes Blue-Green Deployment | [kubernetes-blue-green-deployment](./projects/kubernetes-blue-green-deployment/) | `Kubernetes` `Deployment Strategy` `Zero Downtime` | ✔️ Done |

### Explore our upcoming projects by visiting [this link](https://github.com/tungbq/devops-project/issues?q=is%3Aissue+is%3Aopen+label%3Aproject) ⏩

Expand Down
64 changes: 64 additions & 0 deletions projects/kubernetes-blue-green-deployment/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Project: Kubernetes Blue-Green Deployment

This project demonstrates the blue-green deployment strategy on Kubernetes: two full versions of an app run side by side, and switching live traffic between them is a single `Service` selector patch — no pod restart, no rolling update, and rollback is the exact same patch in reverse.

## Overview

### Introduction

- Tech stack: `kubernetes`, `docker`
- To get the basic concept of Kubernetes, you could visit: [**devops-basics/k8s**](https://github.com/tungbq/devops-basics/blob/main/topics/k8s/README.md)
- The demo runs entirely on a local `kind` cluster — no cloud account needed

### Prerequisite

- You have `docker` and `kind` installed on your machine
- `kubectl` configured against your cluster
- Basic knowledge about Kubernetes (`Deployment`, `Service`, label selectors)

### Why blue-green, not a rolling update?

A Kubernetes rolling update replaces pods gradually — for a window of time, both old and new versions are live *and* you can't instantly go back to 100% old without another rollout. Blue-green keeps both versions fully deployed simultaneously:

- **Cutover is atomic and instant** — one `kubectl patch service`, not a gradual pod-by-pod replacement
- **Rollback is equally instant** — the same patch, in reverse, no need to "roll forward" to an old image again
- **The new version can be smoke-tested with real traffic before going live** — via a second, "preview" Service that always points at green, while the main Service still serves 100% blue
- **Trade-off, stated plainly**: you run 2x the pod count while both versions coexist, and this demo doesn't cover stateful workloads (a shared database between blue and green needs its own compatibility story — out of scope here)

## 1-Deploy blue (the current live version)

- `kubectl apply -f manifests/deployment-blue.yaml`
- `kubectl apply -f manifests/service.yaml` — the `demo-app` Service, selecting `version: blue`
- Verify: `kubectl run curltest --image=curlimages/curl --restart=Never --command -- sleep 3600` then `kubectl exec curltest -- curl -s demo-app` → `Hello from BLUE (v1)`

## 2-Deploy green alongside, without touching live traffic

- `kubectl apply -f manifests/deployment-green.yaml` — a second, independent `Deployment`, labeled `version: green`
- `kubectl apply -f manifests/service-preview.yaml` — `demo-app-preview`, a Service that *always* points at green, for smoke-testing
- `kubectl exec curltest -- curl -s demo-app` still returns `Hello from BLUE (v1)` — green existing changes nothing about live traffic
- `kubectl exec curltest -- curl -s demo-app-preview` returns `Hello from GREEN (v2)` — green is verified healthy and correct *before* it ever sees real traffic

## 3-Cut over

- `kubectl patch service demo-app -p '{"spec":{"selector":{"app":"demo-app","version":"green"}}}'`
- `kubectl exec curltest -- curl -s demo-app` now returns `Hello from GREEN (v2)` — immediately, with zero pods created/restarted/deleted

## 4-Verify zero downtime across the cutover

- Fire a burst of requests, patching the selector partway through the burst
- Every single request gets a `200`, whether it landed before or after the cutover — see `demo_project.sh` step 5

## 5-Roll back

- `kubectl patch service demo-app -p '{"spec":{"selector":{"app":"demo-app","version":"blue"}}}'`
- Traffic is back on blue immediately — this is the entire rollback procedure, no redeploy needed

## 6-Bonus

All of the above, scripted end-to-end (cluster → blue → green → preview → cutover → zero-downtime proof → rollback), is in [demo_project.sh](./demo_project.sh).

## Related link

- https://kubernetes.io/docs/concepts/services-networking/service/
- https://martinfowler.com/bliki/BlueGreenDeployment.html
- https://github.com/tungbq/devops-basics/blob/main/topics/k8s/README.md
128 changes: 128 additions & 0 deletions projects/kubernetes-blue-green-deployment/demo_project.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
#!/usr/bin/env bash
# Kubernetes Blue-Green Deployment Demo — deploys two versions of a tiny
# app side by side, cuts live traffic over from one to the other by
# patching a single Service selector (no pod restarts, no rolling update),
# proves zero requests are dropped during the cutover, then rolls back
# instantly the same way. Used both for local hands-on runs and by
# .github/workflows/verify-kubernetes-blue-green-deployment.yml.
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
MANIFESTS_DIR="$SCRIPT_DIR/manifests"
CLUSTER_NAME="bluegreen-demo"
KIND_VERSION="v0.30.0"
KUBECTL_VERSION="v1.34.0"

install_if_missing() {
local bin="$1" url="$2"
if command -v "$bin" >/dev/null 2>&1; then
return
fi
echo "Installing $bin..."
arch="$(uname -m)"
case "$arch" in
x86_64) arch="amd64" ;;
aarch64 | arm64) arch="arm64" ;;
*)
echo "Unsupported architecture: $arch" >&2
exit 1
;;
esac
curl -sL "${url//ARCH/$arch}" -o "/tmp/$bin"
chmod +x "/tmp/$bin"
sudo mv "/tmp/$bin" "/usr/local/bin/$bin"
}

install_if_missing kind "https://kind.sigs.k8s.io/dl/${KIND_VERSION}/kind-linux-ARCH"
install_if_missing kubectl "https://dl.k8s.io/release/${KUBECTL_VERSION}/bin/linux/ARCH/kubectl"

echo "=============================="
echo "1. Ensuring a local cluster exists"
echo "=============================="
if kubectl config get-contexts "kind-${CLUSTER_NAME}" >/dev/null 2>&1; then
echo "Reusing existing kind cluster: ${CLUSTER_NAME}"
kubectl config use-context "kind-${CLUSTER_NAME}"
else
echo "Creating kind cluster: ${CLUSTER_NAME}"
kind create cluster --name "${CLUSTER_NAME}"
fi

cleanup() {
kubectl delete pod curltest --ignore-not-found --force --grace-period=0 >/dev/null 2>&1 || true
}
trap cleanup EXIT

echo ""
echo "=============================="
echo "2. Deploying BLUE (the current live version) and its Service"
echo "=============================="
kubectl apply -f "$MANIFESTS_DIR/deployment-blue.yaml"
kubectl apply -f "$MANIFESTS_DIR/service.yaml"
kubectl wait --for=condition=available --timeout=120s deployment/app-blue

kubectl run curltest --image=curlimages/curl --restart=Never --command -- sleep 3600
kubectl wait --for=condition=Ready --timeout=60s pod/curltest

echo ""
echo "Live traffic right now:"
kubectl exec curltest -- curl -s demo-app

echo ""
echo "=============================="
echo "3. Deploying GREEN (the new version) alongside — it receives zero"
echo " live traffic yet, since the Service selector still says 'blue'"
echo "=============================="
kubectl apply -f "$MANIFESTS_DIR/deployment-green.yaml"
kubectl wait --for=condition=available --timeout=120s deployment/app-green
kubectl apply -f "$MANIFESTS_DIR/service-preview.yaml"

echo ""
echo "Live traffic is still unaffected by green existing:"
kubectl exec curltest -- curl -s demo-app

echo ""
echo "Smoke-testing green directly via the preview Service (retrying — a"
echo "freshly created Service can take a moment for cluster DNS to resolve):"
for _ in $(seq 1 10); do
if kubectl exec curltest -- curl -sf demo-app-preview; then break; fi
sleep 2
done
echo ""

echo ""
echo "=============================="
echo "4. Cutover — patch the live Service's selector, nothing else"
echo "=============================="
kubectl patch service demo-app -p '{"spec":{"selector":{"app":"demo-app","version":"green"}}}'
echo "Live traffic immediately after the patch:"
kubectl exec curltest -- curl -s demo-app

echo ""
echo "=============================="
echo "5. Proving zero downtime — 20 rapid requests spanning the moment of cutover"
echo "=============================="
kubectl patch service demo-app -p '{"spec":{"selector":{"app":"demo-app","version":"blue"}}}' >/dev/null
codes=""
for i in $(seq 1 20); do
if [ "$i" -eq 10 ]; then
kubectl patch service demo-app -p '{"spec":{"selector":{"app":"demo-app","version":"green"}}}' >/dev/null
fi
codes="$codes $(kubectl exec curltest -- curl -s -o /dev/null -w '%{http_code}' demo-app)"
done
echo "HTTP status codes for all 20 requests (cutover happened mid-loop):$codes"

echo ""
echo "=============================="
echo "6. Rollback — the exact same patch, in reverse, is the entire rollback"
echo "=============================="
kubectl patch service demo-app -p '{"spec":{"selector":{"app":"demo-app","version":"blue"}}}'
echo "Live traffic after rollback:"
kubectl exec curltest -- curl -s demo-app

echo ""
echo "==> Done! Cutover and rollback were both a single Service patch —"
echo " no pod was ever created, restarted, or deleted to move traffic."
echo ""
echo "==> Cleanup:"
echo " kubectl delete -f manifests/"
echo " kind delete cluster --name ${CLUSTER_NAME}"
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: app-blue
labels:
app: demo-app
version: blue
spec:
replicas: 2
selector:
matchLabels:
app: demo-app
version: blue
template:
metadata:
labels:
app: demo-app
version: blue
spec:
containers:
- name: app
image: hashicorp/http-echo:1.0
args:
- -text=Hello from BLUE (v1)
- -listen=:8080
ports:
- containerPort: 8080
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: app-green
labels:
app: demo-app
version: green
spec:
replicas: 2
selector:
matchLabels:
app: demo-app
version: green
template:
metadata:
labels:
app: demo-app
version: green
spec:
containers:
- name: app
image: hashicorp/http-echo:1.0
args:
- -text=Hello from GREEN (v2)
- -listen=:8080
ports:
- containerPort: 8080
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
apiVersion: v1
kind: Service
metadata:
name: demo-app-preview
spec:
# Always points at green, regardless of what's live — lets you smoke-test
# the new version before flipping the real demo-app Service's selector.
selector:
app: demo-app
version: green
ports:
- port: 80
targetPort: 8080
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
apiVersion: v1
kind: Service
metadata:
name: demo-app
spec:
# This selector's `version` value is exactly what "cutover" and
# "rollback" mean in this project — patching it is the entire
# deployment strategy. No pods are ever restarted to switch traffic.
selector:
app: demo-app
version: blue
ports:
- port: 80
targetPort: 8080