Skip to content

fix: add strict deb version match in VHD containerd content test#8960

Open
abigailliang-aks-sig-node wants to merge 5 commits into
mainfrom
fix/strict-containerd-version-content-test
Open

fix: add strict deb version match in VHD containerd content test#8960
abigailliang-aks-sig-node wants to merge 5 commits into
mainfrom
fix/strict-containerd-version-content-test

Conversation

@abigailliang-aks-sig-node

@abigailliang-aks-sig-node abigailliang-aks-sig-node commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Add strict full-version matching in the VHD content test for containerd and cri-tools, catching version drift at VHD build time rather than in e2e.

Problem

The VHD content test (testContainerd/testCriCtl) previously only compared major.minor.patch (stripping the hotfix suffix via cut -d- -f1). This meant:

  • components.json says 2.3.2-ubuntu24.04u1
  • VHD build installs 2.3.2-ubuntu24.04u2 (due to apt wildcard moby-containerd=2.3.2*)
  • Content test passes (only checks 2.3.2 == 2.3.2)
  • E2e validator fails (exact match of full version string)

Changes

  • Add assertPackageVersion helper: uses dpkg-query (deb) or rpm (RPM-based) to compare the full installed version against components.json
  • Resolve package name dynamically from components.json renovateTag via getPackageJSON — handles moby-containerd (Ubuntu), containerd2 (Azure Linux 3.0), etc.
  • Fail fast when package name cannot be resolved from renovateTag (instead of silently skipping the strict check)
  • Fix err() call signatures to match the two-argument format ($1=test name, $2=message)
  • Rename debPackageNameinstalledPackageName since the check now covers both deb and rpm

Follow-up items

  1. ShellSpec unit test: A ShellSpec test for the package name extraction logic (getPackageJSON + renovateTag parsing) was attempted but deferred. cse_helpers.sh has system dependencies (/etc/*-release, etc.) that make it difficult to source in the shellspec Docker environment. Extracting individual functions via sed is fragile. A proper fix would require refactoring getPackageJSON and its helpers into a standalone sourceable module.
  2. ARM64 support: testPackagesInstalled currently skips entirely on ARM64 (line 209), meaning ARM64 VHDs have no package version validation. Opening this up to ARM64 would require verifying that all packages in components.json have correct ARM64 entries and that the download/install paths work for arm64.

Test plan

  • Verified locally that getPackageJSON correctly resolves package names for all OS variants via test_pkg_name_extraction.sh:
    ✓ containerd on Ubuntu 24.04: got 'moby-containerd'
    ✓ containerd on Ubuntu 22.04: got 'moby-containerd'
    ✓ containerd on Ubuntu 20.04: got 'moby-containerd'
    ✓ containerd on Azure Linux 3.0: got 'containerd2'
    ✓ containerd on Mariner 2.0: got 'moby-containerd'
    ✓ cri-tools on Ubuntu 24.04: got 'kubernetes-cri-tools'
    ✓ cri-tools on Ubuntu 22.04: got 'kubernetes-cri-tools'
    ✓ cri-tools on Azure Linux 3.0: got 'kubernetes-cri-tools'
    Results: 8 passed, 0 failed
    
  • VHD build with matching components.json version passes content test
  • VHD build with mismatched version fails at content test (not at e2e)
    https://github.com/Azure/AgentBaker/pull/8960/checks?check_run_id=87739564057
    https://msazure.visualstudio.com/CloudNativeCompute/_build/results?buildId=172577610&view=results

This comment was marked as duplicate.

Copilot AI review requested due to automatic review settings July 15, 2026 21:29
@abigailliang-aks-sig-node
abigailliang-aks-sig-node force-pushed the fix/strict-containerd-version-content-test branch from 6af5478 to f541322 Compare July 15, 2026 21:29

This comment was marked as duplicate.

Copilot AI review requested due to automatic review settings July 16, 2026 00:57
Copilot AI review requested due to automatic review settings July 16, 2026 20:28

This comment was marked as duplicate.

Copilot AI review requested due to automatic review settings July 16, 2026 20:35

This comment was marked as off-topic.

The VHD content test previously only validated major.minor.patch
for containerd and crictl (via binary --version), ignoring the
hotfix suffix (e.g. u1 vs u2). This allowed mismatches between
components.json and the actual installed package to go undetected
until e2e.

Changes:
- Add assertPackageVersion helper that uses dpkg-query (deb) or
  rpm (RPM-based) to compare the full installed version against
  components.json
- Resolve the real package name dynamically from components.json
  renovateTag via getPackageJSON (handles moby-containerd on Ubuntu,
  containerd2 on Azure Linux 3.0, etc.)
- Fail fast when package name cannot be resolved from renovateTag
- Fix err() call signatures to match the two-argument format
- Add local test script for package name extraction verification
…atches mismatch

DO NOT MERGE - this commit will be reverted after CI confirms the content test
fails at VHD build time (not at e2e).
@abigailliang-aks-sig-node
abigailliang-aks-sig-node force-pushed the fix/strict-containerd-version-content-test branch from 9079e42 to 6c4c257 Compare July 16, 2026 20:45
@github-actions github-actions Bot added the components This pull request updates cached components on Linux or Windows VHDs label Jul 16, 2026

This comment was marked as duplicate.

Copilot AI review requested due to automatic review settings July 16, 2026 20:46
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

This comment was marked as duplicate.

Copilot AI review requested due to automatic review settings July 16, 2026 20:52

This comment was marked as duplicate.

Copilot AI review requested due to automatic review settings July 16, 2026 22:39
@Azure Azure deleted a comment from aks-node-assistant Bot Jul 16, 2026

This comment was marked as off-topic.

@@ -0,0 +1,96 @@
#!/bin/bash

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This script file is not called or referenced by anyone, right?

@abigailliang-aks-sig-node abigailliang-aks-sig-node Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this a local test file starting as test_....

I originally planned to add a ShellSpec test, but ces_hepler.sh does not work well in the ShellSpec Docker environment. As a temporary solution, I’m keeping this as a local test script for manual validation.

And one of the follow-up is to figure out a way to unit test vhdbuilder/packer/test/linux-vhd-content-test.sh in ci

@abigailliang-aks-sig-node
abigailliang-aks-sig-node enabled auto-merge (squash) July 17, 2026 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

components This pull request updates cached components on Linux or Windows VHDs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants