Skip to content

Remove public PSGallery in isolated builds to fix CFSClean2 violations#1608

Closed
givinalis wants to merge 6 commits into
mainfrom
givinalis/cfs-remove-psgallery
Closed

Remove public PSGallery in isolated builds to fix CFSClean2 violations#1608
givinalis wants to merge 6 commits into
mainfrom
givinalis/cfs-remove-psgallery

Conversation

@givinalis

Copy link
Copy Markdown
Collaborator

Summary

Resolves the 3 residual CFSClean2 violations to www.powershellgallery.com observed in the network-isolated 1ES PR/CI builds (e.g. build 172485127, "Stop Network Isolation" step). Follow-up to #1607.

Root cause

All 3 violations are Task name: PowerShell, Process path: C:\Program Files\PowerShell\7\pwsh.exewww.powershellgallery.com. They are not from the PSScriptAnalyzer@1 task.

Even though every Install-Module / Publish-Module in the build already targets the CFS feed (or the build's local gallery) via -Repository, the default PSGallery repository is still registered on the agent. PowerShellGet contacts it while:

  • resolving/refreshing package sources, and
  • validating RequiredModules dependencies during Publish-Module (the Microsoft.Graph.* compat modules declare Microsoft.Graph.Authentication as a required module — see build/Publish-LocalCompatModule.ps1).

Those probes are the leak. The build still succeeds because CFSClean2 is currently audit/warn (no blocked connections), but the connections are flagged.

Fix

  • Add build/Remove-PublicPSGallery.ps1 — unregisters PSGallery from both PowerShellGet (Unregister-PSRepository) and PSResourceGet (Unregister-PSResourceRepository). Idempotent, and a no-op unless DEPENDENCY_PS_REPO points at a private feed, so local development keeps PSGallery.
  • Run it right after NuGetAuthenticate@1 (inside the existing if ne(CfsFeedUrl,'') guard) in both generation templates:
    • .azure-pipelines/generation-templates/generate_adapter-1es.yml (PR + CI)
    • .azure-pipelines/generation-templates/generate_adapter.yml (integration tests)

Repository un-registration persists on the agent for the rest of the job, so every subsequent PowerShell step (installs and publishes) resolves modules exclusively from the feed and the local gallery — leaving no route to the public gallery.

Why this is safe

  • Every Install-Module / Publish-Module in build/ already passes an explicit -Repository (feed or local gallery); none rely on an implicit PSGallery.
  • Microsoft.Graph.Authentication (the publish-time dependency) resolves from the local gallery (published first for Entra) or the feed, so Publish-Module validation still succeeds without PSGallery.
  • The release pipeline (1es-entra-powershell-release.yml) only republishes the prebuilt .nupkg to the public gallery in a release job and does not install modules under isolation, so it is unaffected.

Pipelines covered

  • PR: 1es-entra-powershell-pr.yml
  • CI: 1es-entra-powershell-ci-build.yml
  • Integration: integration-tests.yml

Expected result

CFSClean2 = COMPLIANT (0 violations), matching CFSClean / CFSClean3 / Default Deny.

Under network isolation, PowerShellGet still probed the default PSGallery source while resolving package sources and Publish-Module RequiredModules dependencies, even though every install/publish targeted the CFS feed. These probes were the 3 residual CFSClean2 violations to www.powershellgallery.com.

Add build/Remove-PublicPSGallery.ps1 and run it (guarded by CfsFeedUrl) right after NuGetAuthenticate in both generation templates. It unregisters PSGallery from PowerShellGet and PSResourceGet so the feed and local gallery are the only sources for the rest of the job. Local development is unaffected: it is a no-op unless DEPENDENCY_PS_REPO points at a private feed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@learn-build-service-prod

Copy link
Copy Markdown

Learn Build status updates of commit 21a9e69:

❌ Validation status: errors

Please follow instructions here which may help to resolve issue.

File Status Preview URL Details
❌Error Details

  • Line 0, Column 0: [Error: PSMD2Yaml_FileLoadFailed] Failed to load file: C:/LocalRun/W/lkhb-s/module/mapping/monikerMapping.json. PackageRoot, ReferenceTocUrl, and ConceptualTocUrl are required for every moniker. PackageRoot should be a valid relative path to docset root.

For more details, please refer to the build report.

Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.

…lery removal

Unregistering PSGallery from PowerShellGet/PSResourceGet did not reduce the 3 residual CFSClean2 violations, so this instruments the isolation-prep step to dump the full package-source inventory (PowerShellGet, PackageManagement, PSResourceGet, and NuGet.config) before and after cleanup, and also unregisters any powershellgallery-backed PackageManagement source. This pinpoints which subsystem still reaches www.powershellgallery.com on the isolated agent.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@learn-build-service-prod

Copy link
Copy Markdown

Learn Build status updates of commit 5794df3:

❌ Validation status: errors

Please follow instructions here which may help to resolve issue.

File Status Preview URL Details
❌Error Details

  • Line 0, Column 0: [Error: PSMD2Yaml_FileLoadFailed] Failed to load file: C:/LocalRun/W/k2bv-s/module/mapping/monikerMapping.json. PackageRoot, ReferenceTocUrl, and ConceptualTocUrl are required for every moniker. PackageRoot should be a valid relative path to docset root.

For more details, please refer to the build report.

Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.

… violations

An instrumented run proved that after unregistering PSGallery from PowerShellGet, PackageManagement and PSResourceGet, pwsh still opened 3 connections to www.powershellgallery.com. Those originate from the hard-coded default gallery endpoint in the PowerShellGet/PSResourceGet stack and cannot be removed by repository management. Since every dependency installs from the private CFS feed, this null-routes www.powershellgallery.com in the agent hosts file so the host resolves locally to 0.0.0.0, eliminating the CFSClean2 violations across PR, CI and release pipelines.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@learn-build-service-prod

Copy link
Copy Markdown

Learn Build status updates of commit 745c6f5:

❌ Validation status: errors

Please follow instructions here which may help to resolve issue.

File Status Preview URL Details
❌Error Details

  • Line 0, Column 0: [Error: PSMD2Yaml_FileLoadFailed] Failed to load file: C:/LocalRun/W/pmcq-s/module/mapping/monikerMapping.json. PackageRoot, ReferenceTocUrl, and ConceptualTocUrl are required for every moniker. PackageRoot should be a valid relative path to docset root.

For more details, please refer to the build report.

Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.

…connections

The hosts null-route did not help because 1ES netiso manages the hosts file (HostsFileStabilizer) and records the DNS query name regardless of local resolution. Replace it with DNS-Client Operational logging: Remove-PublicPSGallery.ps1 enables/clears the log right after isolation starts, and a new Report-GalleryDns.ps1 step at the end of the PR/CI templates prints each www.powershellgallery.com query (name, UTC/local time, PID) so it can be correlated against the pipeline timeline to pinpoint the offending step.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@learn-build-service-prod

Copy link
Copy Markdown

Learn Build status updates of commit c14fa4f:

❌ Validation status: errors

Please follow instructions here which may help to resolve issue.

File Status Preview URL Details
❌Error Details

  • Line 0, Column 0: [Error: PSMD2Yaml_FileLoadFailed] Failed to load file: C:/LocalRun/W/jauy-s/module/mapping/monikerMapping.json. PackageRoot, ReferenceTocUrl, and ConceptualTocUrl are required for every moniker. PackageRoot should be a valid relative path to docset root.

For more details, please refer to the build report.

Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.

DNS-Client Operational logging captured nothing because the 1ES netiso
HostsFileStabilizer pre-populates www.powershellgallery.com in the hosts
file: the name resolves with no DNS query while the blocked TCP connection
still trips CFSClean2. Switch attribution to Windows Filtering Platform
connection auditing (Security event 5157/5152), which records the process,
PID, destination address/port and timestamp of the blocked egress so it can
be correlated against the pipeline timeline.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@learn-build-service-prod

Copy link
Copy Markdown

Learn Build status updates of commit cadd9a2:

❌ Validation status: errors

Please follow instructions here which may help to resolve issue.

File Status Preview URL Details
❌Error Details

  • Line 0, Column 0: [Error: PSMD2Yaml_FileLoadFailed] Failed to load file: C:/LocalRun/W/wetx-s/module/mapping/monikerMapping.json. PackageRoot, ReferenceTocUrl, and ConceptualTocUrl are required for every moniker. PackageRoot should be a valid relative path to docset root.

For more details, please refer to the build report.

Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.

The first WFP run found no 5157/5152 events: netiso runs in audit mode, so the
gallery connection is permitted (event 5156) but still reported. Read 5156 too,
filter destination port at the XPath layer to avoid materialising every permit
event, print the effective audit setting and a grouped destination summary that
surfaces the public-gallery front-door IP with per-connection timestamps.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@learn-build-service-prod

Copy link
Copy Markdown

Learn Build status updates of commit 685cde2:

❌ Validation status: errors

Please follow instructions here which may help to resolve issue.

File Status Preview URL Details
❌Error Details

  • Line 0, Column 0: [Error: PSMD2Yaml_FileLoadFailed] Failed to load file: C:/LocalRun/W/m5yh-s/module/mapping/monikerMapping.json. PackageRoot, ReferenceTocUrl, and ConceptualTocUrl are required for every moniker. PackageRoot should be a valid relative path to docset root.

For more details, please refer to the build report.

Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.

@givinalis

Copy link
Copy Markdown
Collaborator Author

Closing without merging.

Why

Investigation with kernel-level Windows Filtering Platform (WFP) connection auditing (Security event 5156) on the isolated 1ES agent proved the 3 residual CFSClean2 violations to www.powershellgallery.com are not produced by this repo's build.

Every audited connection to the PowerShell Gallery front-door IPs (150.171.109.114 / 150.171.109.115, port 443) originated from shavavulnscan.exe (host Defender / SHAVA vulnerability scanner baked into the 1ES hosted-VM image, PID 1592) — never from pwsh.exe. The build's pwsh.exe only connected to the CFS feed IP (150.171.73.16) and Azure AD.

The CFSClean2 report mislabels the offending process as pwsh.exe / task PowerShell; the WFP kernel layer identifies the true process. These connections fire on the scanner's own schedule (observed at 08:48:28 during 1ES auto-injected Guardian SDL scans and 08:57:41 during tests), independent of any pipeline step.

Conclusion: the residual 3 violations are host-agent noise from the 1ES VM image and cannot be eliminated by any change in this repository. The real remediation (routing all module installs through the CFS-backed feed, which reduced violations 204 -> 3) is already merged in #1607.

Next step is a 1ES network-isolation baseline/waiver request citing the WFP evidence above, not a code change here. Evidence build: 173277588.

@givinalis givinalis closed this Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant