Skip to content

Split NuGet publishing into ordered buckets with feed-availability gates - #810

Draft
marcpopMSFT with Copilot wants to merge 12 commits into
engfrom
copilot/split-package-publishing-buckets
Draft

Split NuGet publishing into ordered buckets with feed-availability gates#810
marcpopMSFT with Copilot wants to merge 12 commits into
engfrom
copilot/split-package-publishing-buckets

Conversation

Copilot AI commented May 11, 2026

Copy link
Copy Markdown
Contributor

Package publishing needed strict ordering to avoid downstream breakage: publish workload packs first, then manifest packages, then the workload set package (Microsoft.NET.Workloads.*). The pipeline now enforces this order and blocks progression until prior bucket artifacts are discoverable via feed APIs.

  • Publish orchestration in official.yml

    • Replaced single bulk publish with three explicit publishes per feed target (AzDO and NuGet.org):
      1. packs
      2. manifests
      3. workloadSet
    • Added wait steps between buckets so manifests only publish after packs are visible, and workload set only publishes after manifests are visible.
  • Package bucketing script

    • Added eng/split-package-publish-buckets.ps1.
    • Splits downloaded .nupkg artifacts into:
      • publishBuckets/packs
      • publishBuckets/manifests
      • publishBuckets/workloadSet
    • Classification is based on nuspec package ID (case-insensitive), with workload-set/MSI distinction handled explicitly.
  • Feed availability polling script

    • Added eng/wait-for-package-availability.ps1.
    • Polls feed V3 PackageBaseAddress for package/version visibility sequentially (one package at a time), matching the desired low-pressure API-check pattern.
    • Supports configurable polling interval and max attempts.
# Publish sequence per feed target
- task: 1ES.PublishNuget@1   # packs
- powershell: eng/wait-for-package-availability.ps1 -packagesPath '.../publishBuckets/packs' -feedIndexUrl '...'
- task: 1ES.PublishNuget@1   # manifests
- powershell: eng/wait-for-package-availability.ps1 -packagesPath '.../publishBuckets/manifests' -feedIndexUrl '...'
- task: 1ES.PublishNuget@1   # workloadSet

Copilot AI changed the title [WIP] Split package publishing into separate buckets Split NuGet publishing into ordered buckets with feed-availability gates May 11, 2026
Copilot AI requested a review from marcpopMSFT May 11, 2026 23:49
marcpopMSFT and others added 6 commits May 12, 2026 11:14
Deployment jobs don't automatically checkout source code, so the
eng/split-package-publish-buckets.ps1 and eng/wait-for-package-availability.ps1
scripts were not available. Add checkout: self with fetchDepth: 1 to make
the scripts accessible during the publish stage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1ES release jobs block checkout, so eng/ scripts aren't available on
disk in the Publish stage. Copy the two publish scripts into the
artifacts/publishScripts directory during the Build stage so they're
downloaded alongside the packages in the Publish stage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Arcade template publishes the Artifacts pipeline artifact from
Build.ArtifactStagingDirectory, not System.DefaultWorkingDirectory.
The source repo is checked out to 'source-branch' subdirectory, and
build outputs are gathered to the staging directory before publishing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…sets

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@marcpopMSFT

Copy link
Copy Markdown
Member

Tested locally publishing to azdo. Will test out with the preview 6 release to nuget.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Split package publishing into separate buckets

2 participants