Pauldorsch/reduce linux scan time

[pauld-msft]

3 Improvements to Linux Scanning:

1. The larger boost to perf of the 2, removing the Temp directory binding from the Syft scans
2. Ensuring that we only pull each image one time
3. Ensuring that we only run Syft with the same parameters one time

Testing the de-duplication:

dotnet run --project src/Microsoft.ComponentDetection/Microsoft.ComponentDetection.csproj scan --SourceDirectory C:\src\temp --DockerImagesToScan "alpine:3.18,ubuntu:22.04,test-with-base:latest,node:18-bullseye" --LogLevel Debug
Many images, and since there are 2 Linux scanner happening at the same time, we hit the logic that keeps us from trying to pull the same one more than once:

we can also confirm that we only see the syft scan happening once per image sha:

Testing the performance:

dotnet run --project src/Microsoft.ComponentDetection/Microsoft.ComponentDetection.csproj scan --SourceDirectory C:\src\temp --DockerImagesToScan "node:18-bullseye"

Before, ran in over 400s for node:18-bullseye image (and we also see 2 containers starting up to syft scan this single image):

After, ran in under 60s for node:18-bullseye image (and we only see a single container start up to run the syft scan):

[Copilot]

Pull request overview

This PR aims to reduce Linux container scanning time by deduplicating expensive Docker/Syft operations across concurrent callers, so the same image/scope work isn’t repeated unnecessarily within a single process run.

Changes:

• Added a static cache in LinuxScanner to share in-flight (and completed) Syft runs between callers with identical (source, scope, binds).
• Added a static cache in DockerService to share in-flight (and completed) image pulls between callers for the same image.
• Added extra debug logging around base image resolution and Docker operations, and reset the Syft cache in tests for isolation.

Show a summary per file

File	Description
test/Microsoft.ComponentDetection.Detectors.Tests/LinuxScannerTests.cs	Resets the new Syft run cache for test isolation.
src/Microsoft.ComponentDetection.Detectors/linux/LinuxScanner.cs	Adds Syft run deduplication cache and exposes a test-only reset method.
src/Microsoft.ComponentDetection.Detectors/linux/LinuxContainerDetector.cs	Adds debug logging for resolved base image details.
src/Microsoft.ComponentDetection.Common/DockerService.cs	Adds image pull deduplication cache and adjusts logging/binds.

Copilot's findings

• Files reviewed: 4/4 changed files
• Comments generated: 5

[github-actions]

👋 Hi! It looks like you modified some files in the Detectors folder.
You may need to bump the detector versions if any of the following scenarios apply:

• The detector detects more or fewer components than before
• The detector generates different parent/child graph relationships than before
• The detector generates different devDependencies values than before

If none of the above scenarios apply, feel free to ignore this comment 🙂

[Copilot]

Copilot's findings

• Files reviewed: 5/5 changed files
• Comments generated: 5

[Copilot]

Copilot's findings

• Files reviewed: 5/5 changed files
• Comments generated: 4

[Copilot]

Copilot's findings

• Files reviewed: 5/5 changed files
• Comments generated: 1

[pauld-msft]

👋 Hi! It looks like you modified some files in the Detectors folder. You may need to bump the detector versions if any of the following scenarios apply:

• The detector detects more or fewer components than before
• The detector generates different parent/child graph relationships than before
• The detector generates different devDependencies values than before

If none of the above scenarios apply, feel free to ignore this comment 🙂

None of these apply, the Detectors return the exact same results