Skip to content

AST-188321 Added Linguist Extensions - #1616

Merged
cx-anand-nandeshwar merged 7 commits into
mainfrom
feature/AST-188321_linguist_extension
Nov 19, 2026
Merged

cx-anand-nandeshwar merged 7 commits into
mainfrom
feature/AST-188321_linguist_extension

Conversation

@cx-anand-nandeshwar

@cx-anand-nandeshwar cx-anand-nandeshwar commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

By submitting this pull request, you agree to the terms within the Checkmarx Code of Conduct. Please review the contributing guidelines for guidance on creating high-quality pull requests.

Description

  • Added additional default filters pattern for Programming and Markup linguist filenames and extensions (https://github.com/github-linguist/linguist/blob/main/lib/linguist/languages.yml)
  • Added 100 MB file check before including in the zip for all the files excluding SCA files
  • Added unit and integration test cases for additional filter patterns
  • Removed duplicates patterns from internal/params/filters.go;
    • .html — 3 occurrences
    • .java — 2 occurrences
    • .vb — 2 occurrences
    • .config — 2 occurrences
    • .cls — 2 occurrences
    • .h — 2 occurrences
    • .js — 2 occurrences
    • .htm — 2 occurrences
    • .kt — 2 occurrences

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

Related Issues

Link any related issues or tickets.

Checklist

  • I have performed a self-review of my code
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)
  • Any dependent changes have been merged and published in downstream modules
  • I have updated the CLI help for new/changed functionality in this PR (if applicable)
  • All active GitHub checks for tests, formatting, and security are passing
  • The correct base branch is being used

Screenshots (if applicable)

Add screenshots to help explain your changes.

Additional Notes

  • Linguist extension files will be sent for the scan only when if LLM based scanning is enabled

cx-anand-nandeshwar and others added 2 commits November 17, 2026 20:06
…nning settings

- Add the programming and markup Linguist extensions and filenames to the
  default include filter (BaseIncludeFilters = SAST + Linguist patterns).
- Use the Linguist patterns only when Allow AI Usage
  (scan.config.ai.allowAiUsage) is on and both LLM-based settings
  (scan.config.sast.findingsAnalysis, scan.config.sast.extendedAnalysis) are
  on at tenant level, or else at project level; otherwise use the existing
  SAST patterns. Fall back to the SAST patterns when the settings cannot be
  read, and skip the settings calls for git sources and --skip-default-filter.
- Add ProjectsWrapper.GetConfiguration (GET api/configuration/project).
- Add unit tests and a Linguist integration test that follows the tenant's
  LLM setting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cx-anurag-dalke

Copy link
Copy Markdown
Contributor

Review: Linguist extensions / default filter size limit

Builds pass, new unit tests pass at the PR head, and go vet shows only pre-existing warnings outside the diff. Findings below are ordered by severity; line numbers refer to the PR head.

Correctness

  1. Silent empty SAST scan — internal/commands/scan.go:1707
    When every file matching the default filter exceeds the new 100MB limit, isDirEmpty reports the tree as empty and compressFolder uploads only the .container-scan placeholder. A repo whose only source is a large generated SQL file or JS bundle now completes with 0 findings and exit code 0, where it was previously uploaded and scanned. TestCompressFolder_AllFilesOversizedAddsPlaceholder locks this in as expected behavior. Suggest failing (or at least warning loudly) when the size limit removes every candidate file.

  2. Leaked response body — internal/wrappers/projects-http.go:119
    The deferred resp.Body.Close() in GetConfiguration is guarded by err == nil, but err is reassigned by decoder.Decode(...) below it, so on a malformed body the response is never closed. The same guard is safe in UpdateConfiguration only because nothing there reassigns err.

  3. No escape from the size limit — internal/commands/scan.go:1819
    A file the user names via --file-include is still dropped if it also matches a default pattern, because exceedsDefaultFilterSizeLimit only checks membership in the default list. The only workaround is --skip-default-filter, which disables all filtering. This change is not gated on the LLM setting, so it affects every existing SAST pipeline and should be coordinated with the plugin ecosystem.

  4. Shell history files uploaded — internal/params/filters_linguist.go:1156
    .bash_history and .rapp.history are added to the default include filter. These routinely hold tokens, passwords, and internal hostnames, and would be zipped and uploaded whenever LLM scanning is on, without the user opting in. The comment says this is "mainly useful for secret detection", which is not the purpose of the SAST source filter.

Design / efficiency

  1. Wasted tenant-config call on --no-scan — internal/commands/scan.go:2910
    The --no-scan branch calls getScanDefaultIncludeFilters (GET tenant configuration, with retries), but getUploadURLFromSource returns before compressFolder when --no-scan is set, so the result is never used. Passing nil here avoids an extra round trip.

  2. Dead tenant fallback — internal/commands/llm_settings.go:60
    GetConfiguration returns the project's effective configuration including tenant-inherited entries (distinguished by OriginLevel, which is ignored). The project value is therefore always present and the tenant extendedAnalysis fallback only runs on API failure. The doc comment ("project setting overriding the tenant setting") misdescribes what runs. Either check OriginLevel or drop the layered fallback.

  3. Duplicated manifest list — internal/params/filters.go:14
    ScaFilePatterns hand-copies the 27 manifest names already encoded in ossrealtime.IsSupportedManifestFile; TestScaFilePatterns_CoverOssRealtimeManifests exists only to keep the two in sync. It also carries redundant entries (requirement*.txt alongside *requirement*.txt). isScaFile could call ossrealtime.IsSupportedManifestFile plus a short list of lock/resolver/jar patterns.

  4. Duplicated tenant-setting helpers — internal/commands/llm_settings.go:89
    tenantSettingsMap / isSettingTrue re-implement what GetTenantConfigurationResponses / GetTenantConfiguration / GetTenantConfigurationBool already do in internal/commands/chat-sast.go:167-195.

Minor

  1. *.md in the Linguist list — internal/params/filters_linguist.go:909
    Linguist classifies Markdown as prose, not programming or markup, so this contradicts the file's stated source and pulls every README/CHANGELOG/docs tree into the upload. Remove it or document it as a deliberate exception.

  2. Global mutable mock state — internal/wrappers/mock/projects-mock.go:13
    The mock's GetConfiguration reads a package-level ProjectConfiguration slice, so any test that sets it and forgets to reset leaks into later tests. The PR's own tests avoid this by defining projectSettingsWrapper, which suggests a struct field is the form actually needed.

Items 1, 2 and 4 are the ones I'd want resolved before merge.

🤖 Generated with Claude Code

Comment thread internal/commands/scan.go Resolved
Comment thread internal/commands/scan.go Resolved
Comment thread internal/wrappers/mock/projects-mock.go Resolved
Comment thread internal/wrappers/projects-http.go Resolved
Comment thread internal/commands/scan.go Resolved
Comment thread internal/commands/llm_settings.go Resolved
Comment thread internal/params/filters.go Resolved
Comment thread internal/commands/scan.go Resolved
Comment thread internal/params/filters_linguist.go Resolved
Comment thread internal/wrappers/scans-http.go Resolved
…paration documentation (golangci-lint)

- Resolved Leaked response body — internal/wrappers/projects-http.go:119
- Ignored indirect-dependency trivy vulnerability

@cx-anurag-dalke cx-anurag-dalke left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@cx-anand-nandeshwar
cx-anand-nandeshwar merged commit f4a4da1 into main Nov 19, 2026
40 of 47 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants