-
Notifications
You must be signed in to change notification settings - Fork 4.7k
tests: add simple e2e test for guestbook #595
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
justinsb
wants to merge
1
commit into
kubernetes:master
Choose a base branch
from
justinsb:e2e_test_for_guestbook
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+263
−0
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -43,3 +43,6 @@ cscope.* | |
|
|
||
| /bazel-* | ||
| *.pyc | ||
|
|
||
| # Build artifacts | ||
| .build/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| #!/usr/bin/env bash | ||
| # | ||
| # Entry point for running e2e tests with kind in prow. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| CLUSTER_NAME="${CLUSTER_NAME:-e2e}" | ||
|
|
||
| REPO_ROOT=$(git rev-parse --show-toplevel) | ||
| cd "${REPO_ROOT}" | ||
|
|
||
| BUILD_DIR="${REPO_ROOT}/.build" | ||
| BIN_DIR="${BUILD_DIR}/bin" | ||
| mkdir -p "${BIN_DIR}" | ||
|
|
||
| # Download kind | ||
| KIND_VERSION="v0.32.0" | ||
| KIND_URL="https://github.com/kubernetes-sigs/kind/releases/download/${KIND_VERSION}/kind-linux-amd64" | ||
| curl -Lo "${BIN_DIR}/kind" "${KIND_URL}" | ||
| chmod +x "${BIN_DIR}/kind" | ||
|
|
||
| # Make sure kind is in the path | ||
| export PATH="${BIN_DIR}:${PATH}" | ||
|
|
||
| require() { | ||
| command -v "$1" >/dev/null 2>&1 || { echo "Required command not found: $1"; exit 1; } | ||
| } | ||
| require docker | ||
| require kind | ||
| require kubectl | ||
|
|
||
|
|
||
| # Create a kind cluster | ||
| echo "Creating kind cluster '${CLUSTER_NAME}'" | ||
| kind create cluster --name "${CLUSTER_NAME}" --wait 120s | ||
|
|
||
| # Delete the kind cluster when we're done (usually) | ||
| cleanup() { | ||
| echo "Deleting kind cluster '${CLUSTER_NAME}'" | ||
| kind delete cluster --name "${CLUSTER_NAME}" >/dev/null 2>&1 || true | ||
| } | ||
| trap cleanup EXIT | ||
|
|
||
|
|
||
| # Run the per-app e2e tests | ||
| echo "Running e2e tests" | ||
| web/guestbook/e2e/test-kind | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| # Guestbook end-to-end smoke test | ||
|
|
||
| `test` is a self-contained smoke test that verifies the guestbook example | ||
| actually works. | ||
|
|
||
| It assumes a kind cluster has already been created and is called `e2e`; this can be changed with the `CLUSTER_NAME` environment variable. | ||
|
|
||
| It: | ||
|
|
||
| 1. builds the images from source, | ||
| 2. loads those images into the cluster, | ||
| 3. deploys the guestbook manifests, | ||
| 4. waits for everything to become ready, and | ||
| 5. POSTs a message through the frontend and reads it back. | ||
|
|
||
| Because the read is served by the redis *replica*, a successful round trip | ||
| also proves redis replication is working. | ||
|
|
||
| ## Running it locally | ||
|
|
||
| You should create a kind cluster first with `kind create cluster --name e2e`, then run | ||
|
|
||
| ```console | ||
| $ web/guestbook/e2e/test-kind | ||
| ``` | ||
|
|
||
| ## CI | ||
|
|
||
| Prow runs this test via the main `tests/e2e/e2e-kind` entry point. | ||
|
|
||
| ## Known issue: the redis-master image | ||
|
|
||
| The manifests reference `registry.k8s.io/redis:e2e` for the master, but that | ||
| image is no longer pullable by modern container runtimes (it ships an ancient | ||
| Docker v1 schema manifest). The test overrides it with `redis:3.2.9` — the same | ||
| version the redis-replica image is built from — so the example runs. Updating | ||
| the manifest itself is a separate follow-up. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,176 @@ | ||
| #!/usr/bin/env bash | ||
| # | ||
| # End-to-end smoke test for the PHP / Redis guestbook example. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
|
|
||
| REPO_ROOT=$(git rev-parse --show-toplevel) | ||
| GUESTBOOK_DIR="${REPO_ROOT}/web/guestbook" | ||
| cd "${GUESTBOOK_DIR}" | ||
|
|
||
|
|
||
| CLUSTER_NAME="${CLUSTER_NAME:-e2e}" | ||
| REDIS_MASTER_IMAGE="${REDIS_MASTER_IMAGE:-redis:3.2.9}" | ||
|
|
||
| # Local port used for the kubectl port-forward to the frontend service. | ||
| FRONTEND_LOCAL_PORT="${FRONTEND_LOCAL_PORT:-18080}" | ||
|
|
||
| PORT_FORWARD_PID="" | ||
|
|
||
| log() { echo -e "\n\033[1;34m==> $*\033[0m"; } | ||
| err() { echo -e "\033[1;31m$*\033[0m" >&2; } | ||
|
|
||
| # Read the frontend / replica image names straight out of the manifests so the | ||
| # images we build and load always match what actually gets deployed, even if | ||
| # someone bumps the tags later. | ||
| image_from_manifest() { | ||
| # $1 = manifest file, $2 = substring identifying the image | ||
| awk -v pat="$2" '$1=="image:" && $2 ~ pat {print $2; exit}' "$1" | ||
| } | ||
|
|
||
| FRONTEND_IMAGE="$(image_from_manifest "${GUESTBOOK_DIR}/frontend-deployment.yaml" gb-frontend)" | ||
| REDIS_REPLICA_IMAGE="$(image_from_manifest "${GUESTBOOK_DIR}/redis-replica-deployment.yaml" gb-redisslave)" | ||
|
|
||
| if [[ -z "${FRONTEND_IMAGE}" || -z "${REDIS_REPLICA_IMAGE}" ]]; then | ||
| err "Could not determine image names from the manifests." | ||
| exit 1 | ||
| fi | ||
|
|
||
| dump_diagnostics() { | ||
| err "Test failed - dumping cluster state for debugging:" | ||
| kubectl get pods -o wide || true | ||
| kubectl get events --sort-by=.lastTimestamp | tail -n 30 || true | ||
| for d in redis-master redis-replica frontend; do | ||
| echo "--- describe deployment/${d} ---" | ||
| kubectl describe deployment "${d}" || true | ||
| echo "--- logs deployment/${d} ---" | ||
| kubectl logs "deployment/${d}" --tail=40 || true | ||
| done | ||
| } | ||
|
|
||
| cleanup() { | ||
| local ec=$? | ||
| if [[ -n "${PORT_FORWARD_PID}" ]]; then | ||
| kill "${PORT_FORWARD_PID}" 2>/dev/null || true | ||
| fi | ||
| if [[ ${ec} -ne 0 ]]; then | ||
| dump_diagnostics || true | ||
| fi | ||
| exit "${ec}" | ||
| } | ||
| trap cleanup EXIT | ||
|
|
||
|
|
||
| # --------------------------------------------------------------------------- | ||
| log "Guestbook e2e configuration" | ||
| echo " kind cluster name : ${CLUSTER_NAME}" | ||
| echo " frontend image : ${FRONTEND_IMAGE}" | ||
| echo " redis-replica image: ${REDIS_REPLICA_IMAGE}" | ||
| echo " redis-master image : ${REDIS_MASTER_IMAGE}" | ||
|
|
||
|
|
||
| # --------------------------------------------------------------------------- | ||
| log "Building images from source" | ||
| docker build -t "${FRONTEND_IMAGE}" "${GUESTBOOK_DIR}/php-redis" | ||
| docker build -t "${REDIS_REPLICA_IMAGE}" "${GUESTBOOK_DIR}/redis-slave" | ||
|
|
||
| log "Pulling redis master image (${REDIS_MASTER_IMAGE})" | ||
| docker pull "${REDIS_MASTER_IMAGE}" | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| log "Loading images into the kind cluster" | ||
| kind load docker-image --name "${CLUSTER_NAME}" \ | ||
| "${FRONTEND_IMAGE}" "${REDIS_REPLICA_IMAGE}" "${REDIS_MASTER_IMAGE}" | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| log "Deploying the guestbook" | ||
| kubectl apply \ | ||
| -f "${GUESTBOOK_DIR}/redis-master-deployment.yaml" \ | ||
| -f "${GUESTBOOK_DIR}/redis-master-service.yaml" \ | ||
| -f "${GUESTBOOK_DIR}/redis-replica-deployment.yaml" \ | ||
| -f "${GUESTBOOK_DIR}/redis-replica-service.yaml" \ | ||
| -f "${GUESTBOOK_DIR}/frontend-deployment.yaml" \ | ||
| -f "${GUESTBOOK_DIR}/frontend-service.yaml" | ||
|
|
||
| # Override the unpullable registry.k8s.io/redis:e2e master image. See the note | ||
| # at the top of this file. | ||
| log "Overriding redis-master image with ${REDIS_MASTER_IMAGE}" | ||
| kubectl set image deployment/redis-master "master=${REDIS_MASTER_IMAGE}" | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| log "Waiting for deployments to become ready" | ||
| for d in redis-master redis-replica frontend; do | ||
| kubectl rollout status "deployment/${d}" --timeout=180s | ||
| done | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| log "Waiting for the redis replica to sync from the master" | ||
| replication_up() { | ||
| local replica | ||
| replica="$(kubectl get pod -l role=replica -o jsonpath='{.items[0].metadata.name}' 2>/dev/null || true)" | ||
| [[ -n "${replica}" ]] || return 1 | ||
| kubectl exec "${replica}" -- redis-cli info replication 2>/dev/null \ | ||
| | grep -q 'master_link_status:up' | ||
| } | ||
| for _ in $(seq 1 30); do | ||
| if replication_up; then | ||
| echo " replica is in sync with the master" | ||
| break | ||
| fi | ||
| sleep 2 | ||
| done | ||
| if ! replication_up; then | ||
| err "redis replica never reached master_link_status:up" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| log "Port-forwarding the frontend service to localhost:${FRONTEND_LOCAL_PORT}" | ||
| kubectl port-forward svc/frontend "${FRONTEND_LOCAL_PORT}:80" >/dev/null 2>&1 & | ||
| PORT_FORWARD_PID=$! | ||
|
|
||
| base_url="http://localhost:${FRONTEND_LOCAL_PORT}" | ||
|
|
||
| # Wait until the port-forward is actually serving. | ||
| ready="" | ||
| for _ in $(seq 1 30); do | ||
| if curl -fsS -o /dev/null "${base_url}/guestbook.php?cmd=get&key=messages" 2>/dev/null; then | ||
| ready=1 | ||
| break | ||
| fi | ||
| sleep 1 | ||
| done | ||
| if [[ -z "${ready}" ]]; then | ||
| err "frontend did not become reachable via port-forward" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # The actual smoke test: write a message, then read it back. | ||
| message="smoketest-$$-${RANDOM}" | ||
|
|
||
| log "POSTing a message (cmd=set) via the frontend" | ||
| set_resp="$(curl -fsS "${base_url}/guestbook.php?cmd=set&key=messages&value=${message}")" | ||
| echo " response: ${set_resp}" | ||
| if [[ "${set_resp}" != *'"message": "Updated"'* ]]; then | ||
| err "unexpected response to set: ${set_resp}" | ||
| exit 1 | ||
| fi | ||
|
|
||
| log "Reading the message back (cmd=get) via the frontend" | ||
| get_resp="" | ||
| for _ in $(seq 1 15); do | ||
| get_resp="$(curl -fsS "${base_url}/guestbook.php?cmd=get&key=messages" || true)" | ||
| if [[ "${get_resp}" == *"${message}"* ]]; then | ||
| break | ||
| fi | ||
| sleep 1 | ||
| done | ||
| echo " response: ${get_resp}" | ||
| if [[ "${get_resp}" != *"${message}"* ]]; then | ||
| err "message '${message}' was not returned by the guestbook (got: ${get_resp})" | ||
| exit 1 | ||
| fi | ||
|
|
||
| log "SUCCESS: the guestbook stored and returned the message." |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You need to change the script to match what we have in https://github.com/kubernetes/test-infra/blob/06525d81d9d588c8064d4ecd0483b03d3497fc81/config/jobs/kubernetes/examples/examples-presubmits.yaml#L22 which is
test/e2e/e2e-kind