diff --git a/CHANGELOG.md b/CHANGELOG.md index 470706c..b5ee9a0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,56 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] +### Added +- `--version` CLI flag. (CE-445) +- `--socket-org` CLI flag, the command-line equivalent of the `socket_org` + action input and the `SOCKET_ORG` environment variable. The API key remains + environment-only. (CE-445) +- GitHub Action inputs `verbose`, `console_tabular_enabled` and + `console_json_enabled`, delivered as `INPUT_VERBOSE`, + `INPUT_CONSOLE_TABULAR_ENABLED` and `INPUT_CONSOLE_JSON_ENABLED` and honored + from the environment the same way as the matching CLI flags. (CE-445) +- GitHub Action inputs `jira_url` and `jira_project`, matching the names used in + the documentation; `server` and `project` remain as aliases. Also added + `ms_sentinel_shared_key` (alias of `ms_sentinel_key`), + `opengrep_notification_method` and `trufflehog_notification_method` + (`notification_method` remains as an alias). (CE-445) +- `docs/parameters.md` gains a **Name Mapping** section listing every setting as + CLI flag, GitHub Action input, environment variable and JSON key, generated + from `connectors.yaml`, `notifications.yaml` and `action.yml`. A new test + keeps `action.yml` and the parameter declarations in step. (CE-445) +- `scripts/check_release_docs.py` now also checks that action references use an + exact release tag and that the bundled scanner versions quoted in the guides + match the Dockerfile pins; `--write` updates both. (CE-445) +- Documentation for the `-heavy` image variant and for when the standard image + is the right choice. (CE-445) + +### Removed +- The `workspace` and `GITHUB_API_URL` GitHub Action inputs. Neither had an + effect: the action always scans `GITHUB_WORKSPACE`, and `GITHUB_API_URL` is + provided by the runner. Workflows that still set them receive an + "Unexpected input" warning and otherwise behave as before. (CE-445) +- `docs/alert-quality-improvement-plan.md`, a draft working document from a + hackathon branch. The plan itself is now tracked in Linear as CE-447. (CE-445) + +### Fixed +- The Sentinel and Sumo Logic notifiers now read `ms_sentinel_workspace_id`, + `ms_sentinel_key` and `sumologic_endpoint` from CLI flags, action inputs and + dashboard configuration, in addition to the `MS_SENTINEL_*` and + `SUMO_LOGIC_HTTP_SOURCE_URL` environment variables. (CE-445) +- Documentation consistency pass across the GitHub Action, Docker and local + installation guides (CE-445). CLI examples use the flag names that + `socket-basics --help` prints. Docker examples keep the facts file inside the + workspace so the dashboard upload succeeds, and show the environment variables + needed for PR comments outside GitHub Actions. The GitHub Action guide reflects + the bundled Trivy scanner, lists only declared inputs, and passes discovered + Dockerfiles through in the auto-discovery example. JSON configuration examples + use the keys the loader reads, the S3 variable names and `--config` precedence + match the code, GitLab and Jenkins examples override the image entrypoint, + pre-commit hook examples use the published image name, and the installation + guide states the Python 3.10 requirement and the npm install path for the + Socket CLI. New guidance covers large repositories and facts-file size. + ## [3.1.0] - 2026-09-02 ### Added diff --git a/README.md b/README.md index 3327b92..3d4131c 100644 --- a/README.md +++ b/README.md @@ -97,7 +97,7 @@ Socket Basics can also run locally or in other CI/CD environments: **Flexible Configuration:** - Configure via CLI flags, environment variables, JSON files, or the Socket Dashboard - Auto-enablement for container scanning when images or Dockerfiles are specified -- Support for both standard and GitHub Actions `INPUT_*` environment variables +- Every setting is reachable as an `INPUT_*` environment variable (the GitHub Action input name, upper-cased); credentials and notifier endpoints also accept plain names such as `SOCKET_ORG` and `SLACK_WEBHOOK_URL` ## 🎨 Enhanced PR Comments @@ -129,9 +129,9 @@ All configuration can be managed through: 1. **Socket Dashboard** (Enterprise) β€” Centralized policy management 2. **CLI Arguments** β€” Direct command-line flags 3. **Environment Variables** β€” Standard or `INPUT_*` prefixed for GitHub Actions -4. **JSON Configuration File** β€” Structured configuration (see `socket_config_example.json`) +4. **JSON Configuration File** β€” Structured configuration passed with `--config` (see [Configuration File](docs/parameters.md#configuration-file)) -See [Parameters Reference](docs/parameters.md) for the full list of CLI options and environment variables. +See [Parameters Reference](docs/parameters.md) for the full list of CLI options and environment variables, including the [name mapping](docs/parameters.md#name-mapping) between CLI flags, GitHub Action inputs, environment variables and JSON keys. The names differ between interfaces (for example `--python` on the CLI is `python_sast_enabled` in the action and `INPUT_PYTHON_SAST_ENABLED` in the environment), and the CLI rejects unknown flags. #### Integration Environment Variables @@ -184,21 +184,25 @@ docker pull ghcr.io/socketdev/socket-basics:3.1.0 # Run scan docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.1.0 \ --workspace /workspace \ - --python-sast-enabled \ - --secret-scanning-enabled \ + --python \ + --secrets \ --console-tabular-enabled ``` The pre-built image is versioned and intended to be pinned exactly. Avoid floating tags like `:latest` in CI. +Use the standard image. The registries also carry a `-heavy` variant that exists for one specific deployment constraint and adds nothing to Socket Basics itself; see [Image variants](docs/local-install-docker.md#image-variants) before choosing it. + πŸ“– **[View Docker Installation Guide](docs/local-install-docker.md)** ### CLI ```bash -socket-basics --python --secrets --containers --verbose +socket-basics --python --secrets --dockerfiles Dockerfile --verbose ``` +The API key is read from the environment (`SOCKET_SECURITY_API_KEY`; there is deliberately no flag for it) and the organization from `SOCKET_ORG` or `--socket-org`. The same flag names work inside the Docker image. The [name mapping table](docs/parameters.md#name-mapping) shows how every CLI flag corresponds to a GitHub Action input and an environment variable. + πŸ“– **[View Local Installation Guide](docs/local-installation.md)** ## πŸ”§ Requirements @@ -259,8 +263,10 @@ Add new connectors by: **Connector fails to load:** - Verify `module_path` and `class` in `socket_basics/connectors.yaml` -**Socket API errors:** +**Socket API errors / results missing from the dashboard:** - Ensure `SOCKET_SECURITY_API_KEY` and `SOCKET_ORG` are set correctly +- `No Socket organization configured` in the log means nothing was uploaded: set `SOCKET_ORG`, or use a key with the `socket-basics` scope so the organization can be discovered +- `Need at least one file to be uploaded` means the facts file was written outside the scanned workspace; keep `--output` inside it (see [Save Results to File](docs/local-install-docker.md#save-results-to-file)) - Verify your Socket Enterprise subscription is active - If you see `Insufficient permissions`, confirm your API token has the scopes required for your configuration mode (see [Required API Token Scopes](#required-api-token-scopes)) @@ -348,13 +354,13 @@ The `app_tests/` directory contains deliberately vulnerable applications (git su ```bash # Scan a vulnerable Node.js app socket-basics --workspace app_tests/juice-shop \ - --javascript-sast-enabled \ - --secret-scanning-enabled + --javascript \ + --secrets # Scan a vulnerable Python app socket-basics --workspace app_tests/pygoat \ - --python-sast-enabled \ - --secret-scanning-enabled + --python \ + --secrets # Compare results against known vulnerabilities # (Manual verification of findings) diff --git a/action.yml b/action.yml index d200ae2..ef02dcb 100644 --- a/action.yml +++ b/action.yml @@ -8,7 +8,10 @@ runs: env: # Core GitHub variables (these are automatically available, but we explicitly pass GITHUB_TOKEN) GITHUB_TOKEN: ${{ inputs.github_token }} - INPUT_WORKSPACE: ${{ inputs.workspace }} + # Output and logging (same behavior as the --verbose / --console-*-enabled CLI flags) + INPUT_VERBOSE: ${{ inputs.verbose }} + INPUT_CONSOLE_TABULAR_ENABLED: ${{ inputs.console_tabular_enabled }} + INPUT_CONSOLE_JSON_ENABLED: ${{ inputs.console_json_enabled }} # Scan scope INPUT_CHANGED_FILES: ${{ inputs.changed_files }} INPUT_SCAN_ALL: ${{ inputs.scan_all }} @@ -50,13 +53,14 @@ runs: INPUT_JAVA_SAST_ENABLED: ${{ inputs.java_sast_enabled }} INPUT_JIRA_API_TOKEN: ${{ inputs.jira_api_token }} INPUT_JIRA_EMAIL: ${{ inputs.jira_email }} - INPUT_JIRA_PROJECT: ${{ inputs.project }} - INPUT_JIRA_URL: ${{ inputs.server }} + INPUT_JIRA_PROJECT: ${{ inputs.jira_project || inputs.project }} + INPUT_JIRA_URL: ${{ inputs.jira_url || inputs.server }} INPUT_KOTLIN_DISABLED_RULES: ${{ inputs.kotlin_disabled_rules }} INPUT_KOTLIN_ENABLED_RULES: ${{ inputs.kotlin_enabled_rules }} INPUT_KOTLIN_SAST_ENABLED: ${{ inputs.kotlin_sast_enabled }} INPUT_MSTEAMS_WEBHOOK_URL: ${{ inputs.msteams_webhook_url }} - INPUT_MS_SENTINEL_KEY: ${{ inputs.ms_sentinel_key }} + INPUT_MS_SENTINEL_KEY: ${{ inputs.ms_sentinel_key || inputs.ms_sentinel_shared_key }} + INPUT_MS_SENTINEL_SHARED_KEY: ${{ inputs.ms_sentinel_shared_key || inputs.ms_sentinel_key }} INPUT_MS_SENTINEL_WORKSPACE_ID: ${{ inputs.ms_sentinel_workspace_id }} INPUT_PHP_DISABLED_RULES: ${{ inputs.php_disabled_rules }} INPUT_PHP_ENABLED_RULES: ${{ inputs.php_enabled_rules }} @@ -86,7 +90,8 @@ runs: INPUT_TRIVY_NOTIFICATION_METHOD: ${{ inputs.trivy_notification_method }} INPUT_TRIVY_VULN_ENABLED: ${{ inputs.trivy_vuln_enabled }} INPUT_TRUFFLEHOG_EXCLUDE_DIR: ${{ inputs.trufflehog_exclude_dir }} - INPUT_TRUFFLEHOG_NOTIFICATION_METHOD: ${{ inputs.notification_method }} + INPUT_TRUFFLEHOG_NOTIFICATION_METHOD: ${{ inputs.trufflehog_notification_method || inputs.notification_method }} + INPUT_OPENGREP_NOTIFICATION_METHOD: ${{ inputs.opengrep_notification_method }} INPUT_TRUFFLEHOG_SHOW_UNVERIFIED: ${{ inputs.trufflehog_show_unverified }} INPUT_USE_CUSTOM_SAST_RULES: ${{ inputs.use_custom_sast_rules }} INPUT_WEBHOOK_URL: ${{ inputs.webhook_url }} @@ -106,10 +111,6 @@ runs: INPUT_PR_LABEL_LOW: ${{ inputs.pr_label_low }} inputs: - workspace: - description: "Workspace directory to scan (defaults to GITHUB_WORKSPACE)" - required: false - default: "" changed_files: description: >- Diff-only mode: scope every scanner (SAST/OpenGrep, secrets, containers) @@ -151,10 +152,18 @@ inputs: description: "GitHub token for PR comments and API access" required: false default: "" - GITHUB_API_URL: - description: "Github Api Url" + verbose: + description: "Enable verbose (DEBUG) logging in the step log. Same as the --verbose CLI flag." required: false - default: "" + default: "false" + console_tabular_enabled: + description: "Print consolidated findings as tables in the step log. Same as the --console-tabular-enabled CLI flag." + required: false + default: "false" + console_json_enabled: + description: "Print consolidated findings as JSON in the step log. Same as the --console-json-enabled CLI flag." + required: false + default: "false" all_languages_enabled: description: "Enable SAST for all supported languages" required: false @@ -288,11 +297,19 @@ inputs: required: false default: "" jira_api_token: - description: "Jira Api Token" + description: "Jira API token (also reads JIRA_API_TOKEN or INPUT_JIRA_API_TOKEN)" required: false default: "" jira_email: - description: "Jira Email" + description: "Jira account email (also reads JIRA_EMAIL or INPUT_JIRA_EMAIL)" + required: false + default: "" + jira_url: + description: "Jira base URL, e.g. https://your-org.atlassian.net (also reads JIRA_URL or INPUT_JIRA_URL)" + required: false + default: "" + jira_project: + description: "Jira project key, e.g. SEC (also reads JIRA_PROJECT or INPUT_JIRA_PROJECT)" required: false default: "" kotlin_disabled_rules: @@ -308,11 +325,15 @@ inputs: required: false default: "false" ms_sentinel_key: - description: "Ms Sentinel Key" + description: "Microsoft Sentinel shared key (also reads MS_SENTINEL_SHARED_KEY or INPUT_MS_SENTINEL_SHARED_KEY)" + required: false + default: "" + ms_sentinel_shared_key: + description: "Alias for ms_sentinel_key, matching the MS_SENTINEL_SHARED_KEY environment variable name" required: false default: "" ms_sentinel_workspace_id: - description: "Ms Sentinel Workspace Id" + description: "Microsoft Sentinel workspace ID (also reads MS_SENTINEL_WORKSPACE_ID or INPUT_MS_SENTINEL_WORKSPACE_ID)" required: false default: "" msteams_webhook_url: @@ -320,7 +341,15 @@ inputs: required: false default: "" notification_method: - description: "Notification method for TruffleHog (e.g., console, slack)" + description: "Deprecated alias for trufflehog_notification_method; kept for existing workflows" + required: false + default: "" + trufflehog_notification_method: + description: "Notification method for TruffleHog secret findings (e.g., console, slack). Same as --trufflehog-notify." + required: false + default: "" + opengrep_notification_method: + description: "Notification method for OpenGrep SAST findings (e.g., console, slack). Same as --opengrep-notify." required: false default: "" php_disabled_rules: @@ -336,7 +365,7 @@ inputs: required: false default: "false" project: - description: "Project" + description: "Deprecated alias for jira_project; kept for existing workflows" required: false default: "" python_disabled_rules: @@ -392,7 +421,7 @@ inputs: required: false default: "false" server: - description: "Server" + description: "Deprecated alias for jira_url; kept for existing workflows" required: false default: "" slack_webhook_url: diff --git a/docs/alert-quality-improvement-plan.md b/docs/alert-quality-improvement-plan.md deleted file mode 100644 index 54d97b1..0000000 --- a/docs/alert-quality-improvement-plan.md +++ /dev/null @@ -1,599 +0,0 @@ -# Alert Quality Improvement Plan - -**Status:** Draft -**Date:** 2026-02-05 -**Branch:** hackathon -**Prerequisite:** Alert enrichment quick wins (commit `cfcee2f`) β€” CWE/OWASP extraction, auto-generated references, and `detailedReport` markdown are already in place. - ---- - -## Problem Statement - -Customers see alerts with descriptions like *"Generic ad-hoc alert, uploaded by user or produced by system diagnostics"* and a raw rule ID like `python-sql-injection` as the title. This happens because: - -1. **Alert `type` is always `'generic'`** β€” the Socket Dashboard renders a generic fallback when it doesn't recognize the alert type -2. **Rule messages are sparse** β€” ~60% of our 499 rules have terse one-liner messages with no remediation context -3. **No human-readable vulnerability name** β€” customers see `python-sql-injection`, not "SQL Injection" -4. **Missing metadata** β€” `vulnerability_class`, `likelihood`, `impact`, OWASP (81% of rules lack it), `references` (99.8% lack it), and `fix` (99.6% lack it) are absent from rule definitions - -The enrichment work in `cfcee2f` extracts everything the rules *already provide*, but the rules themselves need to provide more. - ---- - -## Phase Overview - -| Phase | Scope | Effort | Files Changed | Impact | -|-------|-------|--------|---------------|--------| -| **1** | CWE lookup table + connector improvements | Small | 1-2 files | Alerts with a CWE (498/499) get a human-readable title and description | -| **2** | Rule metadata enrichment (all 499 rules) | Medium | 15 YAML files | OWASP, references, vulnerability_class on every alert | -| **3** | Rule message rewrite | Large | 15 YAML files | Every alert explains What/Why/How | -| **4** | Dataflow traces + advanced enrichment | Medium | 1-2 files | Taint-mode alerts show source-to-sink flow | - ---- - -## Phase 1: CWE Lookup Table + Connector Improvements - -**Goal:** Every alert with a CWE (498 of 499 rules) gets a human-readable vulnerability name and description, sourced from a CWE lookup table, with zero rule file changes. The one rule without a CWE (`js-react-missing-key`) will fall back to the raw rule message. - -**Files to change:** -- `socket_basics/core/connector/opengrep/__init__.py` (alert construction block) -- New: `socket_basics/core/connector/opengrep/cwe_catalog.py` (lookup table) - -### 1.1 Create CWE Catalog Lookup Table - -A Python dict mapping CWE IDs to human-readable names and descriptions. 498 of our 499 rules reference one of 90 unique CWEs. The top 20 CWEs by rule count cover 68% of rules (339/498): - -```python -CWE_CATALOG = { - "CWE-327": { - "name": "Broken or Risky Cryptographic Algorithm", - "description": "The code uses a cryptographic algorithm that is known to be weak or insufficient. This may allow attackers to decrypt sensitive data or bypass integrity checks.", - "category": "Cryptographic Weakness", - }, - "CWE-89": { - "name": "SQL Injection", - "description": "User-supplied input is included in a SQL query without proper sanitization, potentially allowing attackers to read, modify, or delete database contents.", - "category": "Injection Vulnerability", - }, - "CWE-798": { - "name": "Hard-coded Credentials", - "description": "Credentials such as passwords, API keys, or cryptographic keys are embedded directly in source code, making them easily discoverable if the code is exposed.", - "category": "Authentication Weakness", - }, - # ... remaining 87 CWEs ... -} -``` - -Full catalog of all 90 CWEs is provided in [Appendix A](#appendix-a-cwe-catalog). - -### 1.2 Use CWE Catalog in Alert Construction - -In the alert construction block (after the existing enrichment code), add: - -```python -from .cwe_catalog import CWE_CATALOG - -# After extracting _cwe from metadata: -_cwe_info = CWE_CATALOG.get(_cwe, {}) - -# Add human-readable fields to props -if _cwe_info: - alert['props']['vulnerabilityName'] = _cwe_info.get('name', '') - alert['props']['vulnerabilityCategory'] = _cwe_info.get('category', '') - # Use CWE description as fallback when rule message is sparse - if len(message) < 60: # sparse message threshold - alert['props']['enrichedDescription'] = _cwe_info.get('description', '') -``` - -### 1.3 Extract Additional Metadata Fields - -Extract fields that some rules already provide but the connector currently ignores: - -```python -# Already extracted: cwe, owasp, subcategory, fix, references, confidence -# Add these: -_vulnerability_class = _metadata.get('vulnerability_class', '') -_likelihood = _metadata.get('likelihood', '') -_impact = _metadata.get('impact', '') -_technology = _metadata.get('technology', '') -_framework = _metadata.get('framework', '') - -if _vulnerability_class: - alert['props']['vulnerabilityClass'] = _vulnerability_class -if _likelihood: - alert['props']['likelihood'] = _likelihood -if _impact: - alert['props']['impact'] = _impact -if _technology: - alert['props']['technology'] = _technology -if _framework: - alert['props']['framework'] = _framework -``` - -### 1.4 Improve `detailedReport` Markdown - -Incorporate CWE catalog data into the markdown report: - -```markdown -## SQL Injection - -**Description:** SQL injection vulnerability detected. User-controlled data flows into -SQL query without proper sanitization. Use parameterized queries with placeholders -(?, %s) to prevent SQL injection. - -**Location:** `app/models/user.py` (line 42) - -```python -cursor.execute("SELECT * FROM users WHERE id = " + user_id) -``` - -**Severity:** critical | **Confidence:** high - -**What is CWE-89?** User-supplied input is included in a SQL query without proper -sanitization, potentially allowing attackers to read, modify, or delete database contents. - -**References:** [CWE-89](https://cwe.mitre.org/data/definitions/89.html) | [OWASP Top 10 A03:2021](https://owasp.org/Top10/A03/) -``` - -The "What is CWE-X?" section is pulled from the CWE catalog and provides context even when the rule message is sparse. - -### 1.5 Acceptance Criteria - -- [ ] Every alert with a CWE has `vulnerabilityName` in props (e.g., "SQL Injection") -- [ ] Every alert with a CWE has `vulnerabilityCategory` in props (e.g., "Injection Vulnerability") -- [ ] Sparse messages (< 60 chars) get `enrichedDescription` from CWE catalog -- [ ] `detailedReport` includes CWE explainer section -- [ ] All 90 CWEs in our rules are covered in the catalog -- [ ] No changes to rule YAML files - ---- - -## Phase 2: Rule Metadata Enrichment - -**Goal:** Add `vulnerability_class`, OWASP mappings, and `references` to all 499 rules across 15 YAML files. - -**Files to change:** All files in `socket_basics/rules/*.yml` - -### 2.1 Add `vulnerability_class` to All Rules - -Map each rule to one of 20 standardized vulnerability class names derived from the existing `subcategory` values and CWE associations: - -| vulnerability_class | Maps From subcategory | Associated CWEs | -|---|---|---| -| Injection Vulnerability | `injection`, `process` | CWE-78, CWE-89, CWE-90, CWE-94, CWE-95, CWE-943 | -| Cross-Site Scripting (XSS) | `xss` | CWE-79 | -| Cryptographic Weakness | `crypto` | CWE-208, CWE-295, CWE-310, CWE-319, CWE-326, CWE-327, CWE-338 | -| Authentication Weakness | `authentication` | CWE-287, CWE-347, CWE-384, CWE-521, CWE-798, CWE-916 | -| Access Control Violation | `access-control` | CWE-22, CWE-601, CWE-639, CWE-862, CWE-863 | -| Security Misconfiguration | `configuration`, `proxy` | CWE-16, CWE-200, CWE-209, CWE-489, CWE-614, CWE-693, CWE-732 | -| Insecure Deserialization | `integrity` | CWE-502 | -| Sensitive Data Exposure | `logging` | CWE-312, CWE-522, CWE-532 | -| Server-Side Request Forgery | `ssrf` | CWE-918 | -| Unrestricted File Upload | `upload` | CWE-434 | -| Insecure File Operation | `file-operations` | CWE-73, CWE-377 | -| XML External Entity (XXE) | β€” | CWE-611 | -| Denial of Service | `dos` | CWE-400, CWE-1333, CWE-409 | -| Improper Error Handling | `error-handling`, `async` | CWE-396, CWE-703, CWE-755 | -| Type Safety Violation | `type-safety` | CWE-697, CWE-704 | -| Insecure Design | `design` | CWE-20, CWE-307, CWE-330 | -| Memory Safety Violation | `deprecated` (for C/C++) | CWE-119, CWE-120, CWE-131, CWE-190, CWE-415, CWE-416, CWE-476 | -| Template Injection | β€” | CWE-1336 | -| Prototype Pollution | β€” | CWE-1321 | -| Unsafe Reflection | β€” | CWE-470 | - -For rules that currently lack `subcategory`, derive `vulnerability_class` from the CWE using the table above. - -Example rule change: - -```yaml -# Before -- id: java-sql-injection - message: "SQL injection vulnerability detected..." - metadata: - category: security - cwe: CWE-89 - confidence: high - -# After -- id: java-sql-injection - message: "SQL injection vulnerability detected..." - metadata: - category: security - cwe: CWE-89 - confidence: high - subcategory: injection - vulnerability_class: Injection Vulnerability - owasp: "A03:2021" -``` - -### 2.2 Add OWASP Mappings - -Currently 93/499 rules (19%) have OWASP. Target: 100% of applicable rules. - -CWE-to-OWASP mapping table for bulk application: - -| OWASP Category | CWEs | -|---|---| -| A01:2021 (Broken Access Control) | CWE-22, CWE-73, CWE-601, CWE-639, CWE-862, CWE-863, CWE-918 | -| A02:2021 (Cryptographic Failures) | CWE-208, CWE-259, CWE-295, CWE-310, CWE-312, CWE-319, CWE-322, CWE-326, CWE-327, CWE-338, CWE-522, CWE-798, CWE-916 | -| A03:2021 (Injection) | CWE-74, CWE-78, CWE-79, CWE-89, CWE-90, CWE-91, CWE-94, CWE-95, CWE-117, CWE-134, CWE-611, CWE-943, CWE-1321, CWE-1336 | -| A04:2021 (Insecure Design) | CWE-20, CWE-307, CWE-330, CWE-362, CWE-367 | -| A05:2021 (Security Misconfiguration) | CWE-16, CWE-200, CWE-209, CWE-276, CWE-489, CWE-614, CWE-693, CWE-732, CWE-942 | -| A06:2021 (Vulnerable Components) | CWE-477, CWE-1104 | -| A07:2021 (Auth Failures) | CWE-287, CWE-347, CWE-384, CWE-521 | -| A08:2021 (Data Integrity Failures) | CWE-345, CWE-353, CWE-434, CWE-494, CWE-502 | -| A09:2021 (Logging Failures) | CWE-532, CWE-778 | -| A10:2021 (SSRF) | CWE-918 | - -Not all CWEs have a natural OWASP mapping (e.g., CWE-190 Integer Overflow, CWE-416 Use After Free). Memory safety and low-level CWEs (~30 rules, primarily in `c_cpp.yml`) should be left without OWASP rather than forcing an inaccurate mapping. - -### 2.3 Add `subcategory` to Rules Missing It - -Currently 103/499 rules (21%) have `subcategory`. The remaining 396 rules need it. - -Derivation: Use the CWEβ†’subcategory mapping from Phase 2.1 in reverse. For each rule, look up its CWE and assign the corresponding subcategory. - -### 2.4 Add `references` to Rules - -Currently 1/499 rules has explicit `references`. While the connector auto-generates CWE/OWASP URLs, adding references directly to rules enables: -- Framework-specific documentation links -- Language-specific remediation guides -- More targeted reference URLs than generic CWE pages - -Priority: Add references to all rules with `framework` metadata (53 rules) first, linking to the framework's security documentation. - -Example: - -```yaml -- id: js-react-dangerous-html - metadata: - framework: react - references: - - https://react.dev/reference/react-dom/components/common#dangerously-setting-the-inner-html - - https://cheatsheetseries.owasp.org/cheatsheets/Cross_Site_Scripting_Prevention_Cheat_Sheet.html -``` - -### 2.5 Acceptance Criteria - -- [ ] All 499 rules have `subcategory` in metadata -- [ ] All 499 rules have `vulnerability_class` in metadata -- [ ] All applicable rules (~470) have `owasp` in metadata -- [ ] All 53 framework-specific rules have `references` with framework docs -- [ ] All existing tests pass (if any) - ---- - -## Phase 3: Rule Message Rewrite - -**Goal:** Every rule message follows the "What/Why/How" pattern so customers immediately understand the finding and what to do about it. - -**Files to change:** All files in `socket_basics/rules/*.yml` - -### 3.1 Message Format Standard - -Every rule `message` should follow this three-part structure: - -``` -{What is wrong}. {Why it matters}. {How to fix it}. -``` - -**Example β€” sparse message (before):** -```yaml -message: "Route handler missing authentication/authorization check" -``` - -**Example β€” improved message (after):** -```yaml -message: >- - Route handler is missing an authentication or authorization check. Without - access control, any user can invoke this endpoint and access or modify - protected resources. Add an authentication decorator (e.g., @login_required) - or middleware check before processing the request. -``` - -### 3.2 Message Length Guidelines - -| Severity | Target Length | Rationale | -|----------|-------------|-----------| -| Critical | 2-4 sentences (150-300 chars) | Urgent, needs clear remediation | -| High | 2-3 sentences (120-250 chars) | Important, needs fix guidance | -| Medium | 1-3 sentences (80-200 chars) | Informational with context | -| Low | 1-2 sentences (60-150 chars) | Awareness, minimal action | - -### 3.3 Prioritization - -Rewrite messages in this order: - -1. **Critical severity rules** (most customer-visible, highest urgency) β€” 95 rules -2. **High severity rules with sparse messages** β€” 184 rules -3. **Medium severity rules** β€” 159 rules -4. **Low severity rules** β€” 61 rules - -Within each severity tier, prioritize by CWE frequency (rules for CWE-89, CWE-78, CWE-79 first since they fire most often). - -### 3.4 Add `fix` Metadata - -Currently 2/499 rules have `fix`. The `fix` field provides a concise remediation instruction that appears separately from the message in the `detailedReport`. - -Target: Add `fix` to all critical and high severity rules (279 rules). - -Example: - -```yaml -- id: python-sql-injection - metadata: - fix: >- - Use parameterized queries with placeholders (cursor.execute("SELECT * FROM - users WHERE id = %s", (user_id,))). For ORMs like SQLAlchemy or Django, - use the query builder API instead of raw SQL. -``` - -The `fix` field should be **language-specific** and **actionable** β€” not a repeat of the message, but a concrete code-level instruction. - -### 3.5 Acceptance Criteria - -- [ ] All critical rules have 2-4 sentence messages with What/Why/How -- [ ] All high rules have 2-3 sentence messages with What/Why/How -- [ ] All critical + high rules have `fix` metadata -- [ ] No rule has a message shorter than 60 characters -- [ ] All existing tests pass - ---- - -## Phase 4: Dataflow Traces + Advanced Enrichment - -**Goal:** For taint-mode rules (SQL injection, command injection, XSS, etc.), show the data flow path from source to sink in the alert. - -**Files to change:** -- `socket_basics/core/connector/opengrep/__init__.py` (CLI invocation + alert construction) - -### 4.1 Enable `--dataflow-traces` Flag - -Add the flag to the OpenGrep CLI invocation: - -```python -# In the command construction (around line 162-174): -cmd = ['opengrep', '--json', '--dataflow-traces', '--output', out_file] -``` - -This causes OpenGrep to include `extra.dataflow_trace` in results for taint-mode rules, with structure: - -```json -{ - "extra": { - "dataflow_trace": { - "taint_source": { - "location": { "path": "...", "start": {...}, "end": {...} }, - "content": "request.args.get('id')" - }, - "intermediate_vars": [ - { - "location": { "path": "...", "start": {...}, "end": {...} }, - "content": "user_id = request.args.get('id')" - } - ], - "taint_sink": { - "location": { "path": "...", "start": {...}, "end": {...} }, - "content": "cursor.execute(query + user_id)" - } - } - } -} -``` - -### 4.2 Extract and Format Dataflow Trace - -In the alert construction block, after existing enrichment: - -```python -_dataflow = (r.get('extra') or {}).get('dataflow_trace', {}) -if _dataflow: - _source = _dataflow.get('taint_source', {}) - _sink = _dataflow.get('taint_sink', {}) - _intermediates = _dataflow.get('intermediate_vars', []) - - alert['props']['dataflowTrace'] = { - 'source': { - 'content': _source.get('content', ''), - 'location': _format_location(_source.get('location', {})), - }, - 'sink': { - 'content': _sink.get('content', ''), - 'location': _format_location(_sink.get('location', {})), - }, - 'intermediates': [ - { - 'content': v.get('content', ''), - 'location': _format_location(v.get('location', {})), - } - for v in _intermediates - ], - } -``` - -### 4.3 Add Dataflow to `detailedReport` - -For alerts with a dataflow trace, append a "Data Flow" section to the markdown: - -```markdown -### Data Flow - -1. **Source** (`app/routes.py:12`): - ```python - user_id = request.args.get('id') - ``` - -2. **Intermediate** (`app/routes.py:15`): - ```python - query = "SELECT * FROM users WHERE id = " + user_id - ``` - -3. **Sink** (`app/routes.py:16`): - ```python - cursor.execute(query) - ``` -``` - -This turns an abstract "SQL injection detected" into a concrete story: *here is where the user input enters, here is where it flows, and here is where it reaches the dangerous operation*. - -### 4.4 Verify Performance Impact - -The `--dataflow-traces` flag adds overhead to OpenGrep's analysis. Measure: -- Scan time on `app_tests/python` (baseline vs. with flag) -- Scan time on a larger real-world codebase -- Output JSON size increase - -If overhead is >20% scan time increase, make the flag configurable via a config parameter (e.g., `opengrep_dataflow_traces: true/false`). - -### 4.5 Acceptance Criteria - -- [ ] `--dataflow-traces` is passed to OpenGrep -- [ ] Taint-mode alerts include `dataflowTrace` in props -- [ ] `detailedReport` includes "Data Flow" section for taint alerts -- [ ] Performance impact measured and documented -- [ ] Flag is configurable if overhead is significant - ---- - -## Appendix A: CWE Catalog - -Complete lookup table for all 90 CWEs referenced in our rules, sorted by frequency. - -| CWE | Name | Customer Description | Category | Rules | -|-----|------|---------------------|----------|-------| -| CWE-327 | Broken or Risky Cryptographic Algorithm | The code uses a cryptographic algorithm that is known to be weak or insufficient. This may allow attackers to decrypt sensitive data or bypass integrity checks. | Cryptographic Weakness | 46 | -| CWE-89 | SQL Injection | User-supplied input is included in a SQL query without proper sanitization, potentially allowing attackers to read, modify, or delete database contents. | Injection Vulnerability | 32 | -| CWE-798 | Hard-coded Credentials | Credentials such as passwords, API keys, or cryptographic keys are embedded directly in source code, making them easily discoverable if the code is exposed. | Authentication Weakness | 32 | -| CWE-79 | Cross-Site Scripting (XSS) | User-supplied data is rendered in a web page without proper escaping, potentially allowing attackers to inject malicious scripts that execute in other users' browsers. | Injection Vulnerability | 21 | -| CWE-94 | Code Injection | User-controlled input is passed to a code evaluation function, potentially allowing attackers to execute arbitrary code on the server. | Injection Vulnerability | 20 | -| CWE-78 | OS Command Injection | User-supplied input is incorporated into an operating system command without proper sanitization, potentially allowing attackers to execute arbitrary system commands. | Injection Vulnerability | 19 | -| CWE-295 | Improper Certificate Validation | The application does not properly validate TLS/SSL certificates, which could allow attackers to intercept encrypted communications via man-in-the-middle attacks. | Cryptographic Weakness | 18 | -| CWE-22 | Path Traversal | User input is used to construct a file path without proper validation, potentially allowing attackers to access files outside the intended directory. | Access Control Violation | 18 | -| CWE-502 | Insecure Deserialization | The application deserializes data from an untrusted source without validation, which can lead to remote code execution or denial of service. | Insecure Deserialization | 17 | -| CWE-338 | Weak PRNG | The code uses a non-cryptographic random number generator for security-sensitive operations, producing predictable values that an attacker could guess. | Cryptographic Weakness | 17 | -| CWE-319 | Cleartext Transmission | Sensitive data is transmitted over an unencrypted channel, allowing network attackers to intercept and read the information. | Cryptographic Weakness | 16 | -| CWE-532 | Sensitive Information in Logs | Sensitive data such as passwords or tokens is written to log files, where it may be accessible to unauthorized parties. | Sensitive Data Exposure | 14 | -| CWE-601 | Open Redirect | The application redirects users to a URL from user input without validation, which can be exploited for phishing. | Access Control Violation | 13 | -| CWE-489 | Active Debug Code | Debug code or development-only features are left enabled in production, potentially exposing sensitive information. | Security Misconfiguration | 10 | -| CWE-862 | Missing Authorization | The application does not perform authorization checks before granting access to a resource, allowing unauthorized actions. | Access Control Violation | 9 | -| CWE-434 | Unrestricted File Upload | The application allows file uploads without validating type or content, potentially enabling upload of malicious code. | Unrestricted File Upload | 9 | -| CWE-732 | Incorrect Permission Assignment | Resources are created with overly permissive access rights, potentially exposing them to unauthorized access. | Security Misconfiguration | 7 | -| CWE-614 | Sensitive Cookie Without 'Secure' Flag | A sensitive cookie may be transmitted over unencrypted HTTP connections, making it interceptable by attackers. | Security Misconfiguration | 7 | -| CWE-352 | Cross-Site Request Forgery (CSRF) | The application does not verify that requests originated from its own interface, allowing forged requests from malicious sites. | Access Control Violation | 7 | -| CWE-347 | Improper Cryptographic Signature Verification | The application does not properly verify digital signatures, potentially allowing attackers to tamper with data. | Cryptographic Weakness | 7 | -| CWE-90 | LDAP Injection | User-supplied input is included in an LDAP query without sanitization, potentially allowing attackers to modify query logic. | Injection Vulnerability | 6 | -| CWE-703 | Improper Exception Handling | The application does not properly handle errors, which may lead to unexpected behavior or information disclosure. | Improper Error Handling | 6 | -| CWE-611 | XML External Entity (XXE) | The application parses XML that can reference external entities, potentially allowing attackers to read files or perform SSRF. | Injection Vulnerability | 6 | -| CWE-200 | Information Exposure | The application exposes sensitive information such as internal paths or configuration details to unauthorized users. | Security Misconfiguration | 6 | -| CWE-20 | Improper Input Validation | The application does not sufficiently validate user input, potentially allowing malformed data to trigger vulnerabilities. | Insecure Design | 5 | -| CWE-943 | NoSQL Injection | User-supplied input is included in a NoSQL query without sanitization, potentially allowing attackers to manipulate query logic. | Injection Vulnerability | 4 | -| CWE-755 | Improper Exception Handling | The application fails to properly handle unexpected situations, potentially leading to crashes or exploitable behavior. | Improper Error Handling | 4 | -| CWE-400 | Uncontrolled Resource Consumption | The application does not limit resource usage, making it vulnerable to denial-of-service attacks. | Denial of Service | 4 | -| CWE-377 | Insecure Temporary File | Temporary files are created insecurely, potentially allowing attackers to read or replace them. | Insecure File Operation | 4 | -| CWE-16 | Insecure Configuration | The application uses an insecure configuration that may weaken its security posture. | Security Misconfiguration | 4 | -| CWE-918 | Server-Side Request Forgery (SSRF) | The application fetches a remote resource using a user-controlled URL, allowing attackers to make requests to unintended destinations. | Server-Side Request Forgery | 3 | -| CWE-778 | Insufficient Logging | The application does not adequately log security events, making incident detection difficult. | Sensitive Data Exposure | 3 | -| CWE-639 | Insecure Direct Object Reference (IDOR) | A user-supplied identifier is used to look up resources without authorization checks, enabling unauthorized access. | Access Control Violation | 3 | -| CWE-521 | Weak Password Requirements | The application does not enforce strong password policies, making accounts vulnerable to brute-force attacks. | Authentication Weakness | 3 | -| CWE-416 | Use After Free | The application accesses memory after it has been freed, which can lead to crashes or code execution. | Memory Safety Violation | 3 | -| CWE-415 | Double Free | The application frees memory more than once, which can corrupt memory and allow code execution. | Memory Safety Violation | 3 | -| CWE-367 | TOCTOU Race Condition | A resource is checked and then used in separate operations, creating a window for attacker manipulation. | Insecure Design | 3 | -| CWE-326 | Inadequate Encryption Strength | Encryption uses an insufficient key length, making brute-force decryption feasible. | Cryptographic Weakness | 3 | -| CWE-310 | Cryptographic Issues | The application contains a general cryptographic weakness that may undermine data protection. | Cryptographic Weakness | 3 | -| CWE-287 | Improper Authentication | The application does not properly verify user identity, potentially allowing unauthorized access. | Authentication Weakness | 3 | -| CWE-250 | Execution with Unnecessary Privileges | The application runs with more permissions than required, increasing exploit impact. | Security Misconfiguration | 3 | -| CWE-209 | Error Message Information Leak | Error messages include sensitive details that could help an attacker plan further attacks. | Security Misconfiguration | 3 | -| CWE-190 | Integer Overflow | An arithmetic operation exceeds the integer range, potentially leading to buffer overflows or logic errors. | Memory Safety Violation | 3 | -| CWE-134 | Format String Vulnerability | User-supplied input is used as a format string, potentially allowing attackers to read or write memory. | Injection Vulnerability | 3 | -| CWE-1333 | ReDoS (Regular Expression Denial of Service) | A regular expression can be exploited with crafted input to cause catastrophic backtracking. | Denial of Service | 3 | -| CWE-120 | Buffer Overflow | Data is copied into a fixed-size buffer without checking input length, potentially enabling code execution. | Memory Safety Violation | 3 | -| CWE-119 | Buffer Overrun | Operations read or write beyond memory boundaries, which can cause crashes or enable code execution. | Memory Safety Violation | 3 | -| CWE-74 | Injection | User-supplied input is passed to a downstream interpreter without sanitization. | Injection Vulnerability | 2 | -| CWE-704 | Incorrect Type Conversion | An unsafe type conversion may lead to data truncation or memory corruption. | Type Safety Violation | 2 | -| CWE-697 | Incorrect Comparison | A flawed comparison can lead to logic bypasses or security check circumvention. | Type Safety Violation | 2 | -| CWE-693 | Protection Mechanism Failure | A security mechanism is absent or bypassable, reducing the application's security. | Security Misconfiguration | 2 | -| CWE-494 | Download Without Integrity Check | Code or updates are downloaded without verifying integrity, allowing supply of malicious code. | Insecure Deserialization | 2 | -| CWE-409 | Decompression Bomb | Compressed data is processed without size limits, potentially causing resource exhaustion. | Denial of Service | 2 | -| CWE-401 | Memory Leak | Allocated memory is never released, potentially leading to resource exhaustion. | Memory Safety Violation | 2 | -| CWE-384 | Session Fixation | Session identifiers are not regenerated after authentication, enabling session hijacking. | Authentication Weakness | 2 | -| CWE-362 | Race Condition | Shared resources are accessed without synchronization, potentially leading to data corruption. | Insecure Design | 2 | -| CWE-353 | Missing Integrity Check | Data is accepted without verifying integrity, allowing in-transit tampering. | Insecure Deserialization | 2 | -| CWE-322 | Key Exchange Without Authentication | Cryptographic key exchange lacks entity authentication, enabling man-in-the-middle attacks. | Cryptographic Weakness | 2 | -| CWE-312 | Cleartext Storage of Sensitive Information | Sensitive data is stored in plaintext, readable by anyone with storage access. | Sensitive Data Exposure | 2 | -| CWE-307 | Excessive Authentication Attempts | Failed login attempts are not limited, enabling brute-force attacks. | Insecure Design | 2 | -| CWE-276 | Incorrect Default Permissions | Resources are created with overly permissive defaults. | Security Misconfiguration | 2 | -| CWE-248 | Uncaught Exception | Unhandled exceptions may cause crashes or information disclosure. | Improper Error Handling | 2 | -| CWE-1336 | Template Injection | User input in template expressions can enable server-side code execution. | Template Injection | 2 | -| CWE-98 | Remote File Inclusion | User input controls which file is loaded, potentially enabling remote code execution. | Injection Vulnerability | 1 | -| CWE-95 | Eval Injection | User-supplied input is passed to eval(), allowing arbitrary code execution. | Injection Vulnerability | 1 | -| CWE-942 | Permissive CORS Policy | Overly permissive CORS allows malicious sites to access sensitive data. | Security Misconfiguration | 1 | -| CWE-926 | Improper Android Component Export | An Android component is exported without access restrictions. | Security Misconfiguration | 1 | -| CWE-916 | Weak Password Hashing | Passwords are hashed with a fast or weak algorithm, making cracking feasible. | Authentication Weakness | 1 | -| CWE-915 | Mass Assignment | Users can set arbitrary object attributes through input binding. | Access Control Violation | 1 | -| CWE-91 | XML/XPath Injection | User input in XML or XPath queries can alter query logic. | Injection Vulnerability | 1 | -| CWE-88 | Argument Injection | User input is passed as command arguments without delimiter neutralization. | Injection Vulnerability | 1 | -| CWE-863 | Incorrect Authorization | Authorization checks are implemented incorrectly, allowing unauthorized access. | Access Control Violation | 1 | -| CWE-73 | External File Path Control | User input determines which file to access, enabling arbitrary file operations. | Insecure File Operation | 1 | -| CWE-667 | Improper Locking | Lock mismanagement may lead to deadlocks or race conditions. | Insecure Design | 1 | -| CWE-522 | Insufficiently Protected Credentials | Credentials are stored or transmitted with inadequate protection. | Sensitive Data Exposure | 1 | -| CWE-479 | Signal Handler Safety | A signal handler calls a non-reentrant function, causing undefined behavior. | Memory Safety Violation | 1 | -| CWE-477 | Obsolete Function | A deprecated function with known weaknesses is used. | Security Misconfiguration | 1 | -| CWE-476 | NULL Pointer Dereference | A NULL pointer is used, causing a crash or undefined behavior. | Memory Safety Violation | 1 | -| CWE-470 | Unsafe Reflection | User input selects classes dynamically, enabling arbitrary code execution. | Injection Vulnerability | 1 | -| CWE-396 | Generic Exception Catch | Catching broad exceptions may mask security-relevant failures. | Improper Error Handling | 1 | -| CWE-345 | Insufficient Data Authenticity | Data is accepted without verifying its source or integrity. | Insecure Deserialization | 1 | -| CWE-330 | Insufficient Randomness | Random values are not unpredictable enough for their security context. | Insecure Design | 1 | -| CWE-259 | Hard-coded Password | A password is embedded in source code, easily discoverable and unchangeable. | Authentication Weakness | 1 | -| CWE-242 | Inherently Dangerous Function | An inherently unsafe function is called that cannot be used securely. | Memory Safety Violation | 1 | -| CWE-208 | Timing Side Channel | Timing differences in responses may allow attackers to extract secrets. | Cryptographic Weakness | 1 | -| CWE-1321 | Prototype Pollution | User input modifies JavaScript object prototypes, altering application logic. | Prototype Pollution | 1 | -| CWE-131 | Incorrect Buffer Size Calculation | Buffer size miscalculation can lead to overflow and code execution. | Memory Safety Violation | 1 | -| CWE-117 | Log Injection | Unsanitized input in logs allows forged entries or malicious content. | Sensitive Data Exposure | 1 | -| CWE-1104 | Unmaintained Third-Party Components | A dependency is no longer maintained, leaving vulnerabilities unpatched. | Security Misconfiguration | 1 | -| CWE-1059 | Insufficient Documentation | Inadequate code documentation makes security issues harder to find and fix. | Security Misconfiguration | 1 | - -## Appendix B: Subcategory-to-Vulnerability-Class Mapping - -For rules that already have `subcategory`, this table drives `vulnerability_class` assignment: - -| subcategory | vulnerability_class | Rule Count | -|---|---|---| -| injection | Injection Vulnerability | 21 | -| crypto | Cryptographic Weakness | 16 | -| configuration | Security Misconfiguration | 11 | -| access-control | Access Control Violation | 9 | -| authentication | Authentication Weakness | 8 | -| error-handling | Improper Error Handling | 5 | -| integrity | Insecure Deserialization | 5 | -| design | Insecure Design | 5 | -| logging | Sensitive Data Exposure | 4 | -| deprecated | Memory Safety Violation | 3 | -| dos | Denial of Service | 3 | -| file-operations | Insecure File Operation | 2 | -| ssrf | Server-Side Request Forgery | 2 | -| type-safety | Type Safety Violation | 2 | -| upload | Unrestricted File Upload | 2 | -| xss | Cross-Site Scripting (XSS) | 1 | -| process | Injection Vulnerability | 1 | -| async | Improper Error Handling | 1 | -| proxy | Security Misconfiguration | 1 | -| performance | *(exclude β€” not security)* | 1 | - -## Appendix C: Framework Metadata - -53 rules have `framework` metadata across 32 unique frameworks. These should receive framework-specific `references` URLs in Phase 2.4. - -| Framework | Rules | Documentation Link (for references) | -|---|---|---| -| phoenix | 8 | https://hexdocs.pm/phoenix/ | -| play | 4 | https://www.playframework.com/documentation/ | -| rails | 4 | https://guides.rubyonrails.org/security.html | -| aspnet | 2 | https://learn.microsoft.com/en-us/aspnet/core/security/ | -| express | 2 | https://expressjs.com/en/advanced/best-practice-security.html | -| jpa | 2 | https://docs.oracle.com/javaee/7/tutorial/persistence-intro.htm | -| react | 2 | https://react.dev/reference/react-dom/ | -| spring | 2 | https://docs.spring.io/spring-security/reference/ | -| otp | 2 | https://www.erlang.org/doc/design_principles/ | -| cowboy | 2 | https://ninenines.eu/docs/en/cowboy/ | -| coredata | 2 | https://developer.apple.com/documentation/coredata | -| *(22 others)* | 1 each | *(framework-specific docs)* | diff --git a/docs/github-action.md b/docs/github-action.md index db7165e..975df05 100644 --- a/docs/github-action.md +++ b/docs/github-action.md @@ -187,18 +187,31 @@ Socket Basics requires the following permissions to post PR comments and create ```yaml permissions: - issues: write # Create and update issues for findings + issues: write # Severity labels are managed through the issues API contents: read # Read repository contents - pull-requests: write # Post comments on pull requests + pull-requests: write # Post and update the findings comment ``` Include these in your workflow's `jobs..permissions` section. ### Required Inputs -**`github_token`** (required) +**`github_token`** (required for PR comments and labels) - GitHub token for posting PR comments and API access - Use `${{ secrets.GITHUB_TOKEN }}` (automatically provided) +- Also set `GITHUB_PR_NUMBER` in the step `env:` as shown in the Quick Start. On + `pull_request` events the number is also read from the event payload, so the + env entry is a belt-and-braces default that keeps the same workflow working on + `issue_comment` and other triggers. + +**`socket_security_api_key`** (required to upload results and load dashboard configuration) +- Without it the scan still runs and comments on the PR, but nothing reaches the Socket dashboard. + +**`socket_org`** (optional) +- Discovered from the API key when the key has the `socket-basics` scope. Set it + explicitly when configuring the scan from `with:` inputs instead of the dashboard. +- Outside the action the same value is `SOCKET_ORG` or `--socket-org`. See the + [name mapping](parameters.md#name-mapping) for the equivalents of every input. ### Common Scanning Options @@ -233,22 +246,20 @@ Include these in your workflow's `jobs..permissions` section. - uses: SocketDev/socket-basics@v3.1.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} - # The supported pre-built GitHub Action path currently ships without - # Trivy while we evaluate the safest way to bundle it with Basics again. - # Use a native install if you need container scanning today. - # See docs/local-installation.md#trivy-container-scanning. + # Listing images or Dockerfiles auto-enables the matching Trivy scan. + # Images must be reachable from the job: public, pullable after an earlier + # `docker login` step, or built earlier in the same job. + container_images: 'nginx:1.27.4,ghcr.io/your-org/api:1.4.2' + dockerfiles: 'Dockerfile,docker/Dockerfile.prod' + # Optional: Trivy vulnerability scanning for supported language ecosystems + trivy_vuln_enabled: 'true' ``` > [!NOTE] -> The supported pre-built GitHub Action and Docker image paths currently ship -> _without_ Trivy while we evaluate the safest way to bundle it with Basics -> again. -> If you need container or Dockerfile scanning today, use the -> [native installation path](local-installation.md). See -> [Trivy (Container Scanning)](local-installation.md#trivy-container-scanning) -> for the current version guidance and install options, and review the upstream -> install path and artifacts carefully before adopting that path in production -> CI. +> Trivy is bundled in the pre-built action image (a Socket-built distribution, +> rebuilt from unmodified upstream source and pinned by digest), so these inputs +> need no extra setup. Only a native install needs the version guidance in +> [Trivy (Container Scanning)](local-installation.md#trivy-container-scanning). **Socket Tier 1 Reachability:** ```yaml @@ -551,10 +562,9 @@ jobs: socket_org: ${{ secrets.SOCKET_ORG }} socket_security_api_key: ${{ secrets.SOCKET_SECURITY_API_KEY }} - # Enable multiple languages + # Enable multiple languages (javascript_sast_enabled covers TypeScript) python_sast_enabled: 'true' javascript_sast_enabled: 'true' - typescript_sast_enabled: 'true' go_sast_enabled: 'true' # Security scans @@ -618,10 +628,7 @@ jobs: > [!NOTE] > The pre-built GitHub Action bundles Trivy (a Socket-built distribution, > rebuilt from unmodified upstream source and pinned by digest), so the -> container-scanning inputs below work out of the box. The standalone-install -> example that follows is only for running Trivy independently of Socket -> Basics; if you do that, never use versions `0.69.4`–`0.69.6` β€” see -> [Local Installation](local-installation.md#trivy-container-scanning). +> container-scanning inputs below work out of the box with no Trivy install step. ```yaml name: Container Security @@ -646,20 +653,18 @@ jobs: - name: Build Docker Image run: docker build -t myapp:${{ github.sha }} . - - - name: Install pinned Trivy - run: | - # Pin explicitly; never use 0.69.4–0.69.6 (see the Trivy section in - # docs/local-installation.md). Keep in step with the version bundled - # in the Socket Basics image (TRIVY_VERSION in the Dockerfile). - TRIVY_VERSION=0.73.0 - curl -fsSL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh \ - | sh -s -- -b /usr/local/bin "v${TRIVY_VERSION}" - - - name: Scan Container - run: | - trivy image --exit-code 1 --severity HIGH,CRITICAL "myapp:${{ github.sha }}" - trivy config --exit-code 1 --severity HIGH,CRITICAL Dockerfile + + - name: Run Socket Basics (image + Dockerfile scan) + uses: SocketDev/socket-basics@v3.1.0 + env: + GITHUB_PR_NUMBER: ${{ github.event.pull_request.number }} + with: + github_token: ${{ secrets.GITHUB_TOKEN }} + socket_security_api_key: ${{ secrets.SOCKET_SECURITY_API_KEY }} + # The image built above is visible to the action through the runner's + # Docker daemon; registry images must be pullable from the job. + container_images: 'myapp:${{ github.sha }}' + dockerfiles: 'Dockerfile' ``` ### Dockerfile Auto-Discovery @@ -718,6 +723,7 @@ jobs: github_token: ${{ secrets.GITHUB_TOKEN }} # Discovered Dockerfiles feed Trivy-backed misconfiguration scanning, # which is bundled in the pre-built action image. + dockerfiles: ${{ needs.discover-dockerfiles.outputs.dockerfiles }} verbose: 'true' ``` @@ -735,9 +741,9 @@ jobs: 3. **Scan job** receives discovered paths via job output and skips if none found -**Customizing discovery patterns:** +**Customizing discovery patterns** (edit the `find` expression in the discovery step): -```yaml +```bash # Only scan production Dockerfiles -type f -name 'Dockerfile.prod' -print @@ -822,23 +828,35 @@ Notes: See [`action.yml`](../action.yml) for the complete list of inputs. +Every input has a CLI flag and environment-variable equivalent; the +[name mapping](parameters.md#name-mapping) lists them side by side. + **Core Configuration:** - `socket_org` β€” Socket organization slug (Enterprise) - `socket_security_api_key` β€” Socket Security API key (Enterprise) -- `github_token` β€” GitHub token (required) -- `verbose` β€” Enable verbose logging -- `console_tabular_enabled` β€” Tabular console output -- `console_json_enabled` β€” JSON console output +- `github_token` β€” GitHub token (required for PR comments and labels) +- `verbose` β€” Enable verbose logging in the step log +- `console_tabular_enabled` β€” Tabular console output in the step log +- `console_json_enabled` β€” JSON console output in the step log +- The action always scans `GITHUB_WORKSPACE` (there is no `workspace` input); + narrow the scope with the inputs below. + +**Scan Scope:** +- `changed_files` β€” Diff-only mode (`auto`, `pr`, `current-commit`, a commit hash, or a file list) +- `scan_files` β€” Explicit comma-separated file list +- `scan_all` β€” Fail-open fallback when a `changed_files` scope cannot be resolved **SAST Languages:** - `all_languages_enabled` β€” Enable all languages -- `python_sast_enabled`, `javascript_sast_enabled`, `typescript_sast_enabled` +- `python_sast_enabled`, `javascript_sast_enabled` (covers TypeScript) - `go_sast_enabled`, `golang_sast_enabled` - `java_sast_enabled`, `php_sast_enabled`, `ruby_sast_enabled` - `csharp_sast_enabled`, `dotnet_sast_enabled` - `c_sast_enabled`, `cpp_sast_enabled` - `kotlin_sast_enabled`, `scala_sast_enabled`, `swift_sast_enabled` -- `rust_sast_enabled`, `elixir_sast_enabled` +- `rust_sast_enabled`, `elixir_sast_enabled`, `erlang_sast_enabled` +- `all_rules_enabled` β€” Run every bundled rule for the enabled languages (disables the per-language allowlists) +- `opengrep_notification_method` β€” Route SAST findings to one notifier (e.g. `console`, `slack`) **Rule Configuration (per language):** - `_enabled_rules` β€” Comma-separated rules to enable @@ -849,15 +867,20 @@ See [`action.yml`](../action.yml) for the complete list of inputs. **Security Scanning:** - `secret_scanning_enabled` β€” Enable secret scanning +- `disable_all_secrets` β€” Turn every secret-scanning feature off - `trufflehog_exclude_dir` β€” Directories to exclude - `trufflehog_show_unverified` β€” Show unverified secrets +- `trufflehog_notification_method` β€” Route secret findings to one notifier (`notification_method` is a deprecated alias) - `socket_tier_1_enabled` β€” Socket Tier 1 reachability +- `socket_additional_params` β€” Extra arguments for `socket scan reach` -**Container Scanning (configuration surface):** -- `container_images` β€” Comma-separated images to scan -- `dockerfiles` β€” Comma-separated Dockerfiles to scan +**Container Scanning:** +- `container_images` β€” Comma-separated images to scan (auto-enables image scanning) +- `dockerfiles` β€” Comma-separated Dockerfiles to scan (auto-enables Dockerfile scanning) - `trivy_disabled_rules` β€” Trivy rules to disable -- `trivy_vuln_enabled` β€” Enable vulnerability scanning +- `trivy_image_scanning_disabled` β€” Disable image scanning +- `trivy_vuln_enabled` β€” Enable vulnerability scanning for supported language ecosystems +- `trivy_notification_method` β€” Route container findings to one notifier > [!NOTE] > Container scanning is backed by Trivy, bundled in the pre-built GitHub @@ -866,18 +889,27 @@ See [`action.yml`](../action.yml) for the complete list of inputs. **Notifications (Enterprise Required):** - `slack_webhook_url` β€” Slack webhook -- `jira_url`, `jira_email`, `jira_api_token`, `jira_project` β€” Jira config +- `jira_url`, `jira_email`, `jira_api_token`, `jira_project` β€” Jira config (`server` and `project` are deprecated aliases) - `msteams_webhook_url` β€” MS Teams webhook - `webhook_url` β€” Generic webhook -- `ms_sentinel_workspace_id`, `ms_sentinel_shared_key` β€” MS Sentinel +- `ms_sentinel_workspace_id`, `ms_sentinel_shared_key` (alias `ms_sentinel_key`) β€” MS Sentinel - `sumologic_endpoint` β€” Sumo Logic -**Storage:** -- `s3_enabled`, `s3_bucket`, `s3_access_key`, `s3_secret_key` β€” S3 upload +**PR Comments and Labels:** +- `pr_comment_enabled`, `pr_comment_collapse_all`, `pr_labels_enabled`, `pr_label_` and the + other `pr_comment_*` inputs β€” see the [PR Comment Guide](github-pr-comment-guide.md) + +**Storage (environment variables, not inputs):** +S3 upload is configured through the step `env:` block, not `with:`: +`SOCKET_S3_ENABLED`, `SOCKET_S3_BUCKET`, `SOCKET_S3_ACCESS_KEY`, `SOCKET_S3_SECRET_KEY`, +and optionally `SOCKET_S3_REGION` and `SOCKET_S3_ENDPOINT`. See +[S3 Upload Configuration](parameters.md#s3-upload-configuration). ### Environment Variables -All inputs support both standard and `INPUT_` prefixed environment variables: +Every `with:` input is delivered to the container as an `INPUT_` +environment variable (the input name upper-cased), so the two forms below are +equivalent and either can be used in the step `env:` block: ```yaml env: @@ -887,6 +919,12 @@ env: SOCKET_SECURITY_API_KEY: ${{ secrets.SOCKET_SECURITY_API_KEY }} ``` +Plain (un-prefixed) names are accepted only for credentials, notifier endpoints +and a few GitHub context values (`SOCKET_ORG`, `SOCKET_SECURITY_API_KEY`, +`GITHUB_TOKEN`, `SLACK_WEBHOOK_URL`, `JIRA_URL`, ...). A scanner toggle such as +`PYTHON_SAST_ENABLED` without the `INPUT_` prefix is ignored. The full list is +in the [name mapping](parameters.md#name-mapping). + ## Troubleshooting ### Action Not Finding Files @@ -911,7 +949,35 @@ steps: permissions: contents: read pull-requests: write + issues: write # only needed for severity labels +``` +3. Make sure the run knows which PR it is on. Pass the number explicitly, as the + Quick Start does, so it works on every trigger: +```yaml +env: + GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} ``` +4. Check that `pr_comment_enabled` has not been set to `'false'` (in the + workflow or in the Socket dashboard configuration). The scan then runs and + uploads results but deliberately posts nothing. +5. Nothing is posted when there are no findings above the reporting threshold. + Look for `GithubPRNotifier` lines in the step log to see what the notifier + decided and why. + +### Results Not Showing in the Socket Dashboard + +**Problem:** The action is green but no full scan appears in the dashboard. + +**Solutions:** +1. Provide `socket_security_api_key`. Without it the scan runs locally only. +2. Check the log for `No Socket organization configured`. Either set `socket_org` + or use an API key with the `socket-basics` scope so the organization can be + discovered from the key. +3. Check the token scopes: `full-scans` is required to upload; + `socket-basics` is required to load dashboard configuration. `Insufficient + permissions` in the log means a scope is missing. +4. `pr_comment_enabled: 'false'` does **not** affect the upload; results still + reach the dashboard. ### Container Scanning Fails @@ -948,17 +1014,29 @@ permissions: **Additional signal:** If the configured path does not exist under the workspace, Socket Basics logs a warning and does not fall back to rule-only matching. -### High Memory Usage +### Large Repositories and Monorepos -**Problem:** Action runs out of memory. +**Problem:** The action is slow, runs out of memory, posts very large PR +comments, or produces a huge `.socket.facts.json`. + +`all_languages_enabled` over a whole monorepo runs every SAST rule set against +every file on every PR. Independently of findings, the SAST connector also +records **one component per non-gitignored file in the workspace**, so the +facts file grows with the file count, and repositories that commit media, +vendored code or generated assets produce very large uploads. **Solutions:** -1. Exclude large directories: -```yaml -trufflehog_exclude_dir: 'node_modules,vendor,dist,.git' -``` -2. Scan specific languages instead of `all_languages_enabled` -3. Use self-hosted runner with more resources +1. Scope PR runs to the diff. `changed_files: 'auto'` (with `fetch-depth: 0`) + makes each PR report only its own changes and is the single biggest win. +2. Enable only the languages the repository actually uses instead of + `all_languages_enabled`. +3. Keep generated and vendored trees out of git or list them in `.gitignore`; + ignored files are excluded from the inventory. `trufflehog_exclude_dir` + affects secret scanning only. +4. For multi-project monorepos, run one job per project with `scan_files` / + `changed_files`, or use the Docker image with a per-project `--workspace` (see + [Local Docker Installation](local-install-docker.md#large-repositories-and-monorepos)). +5. Use a self-hosted runner with more memory only after the above. ### Rate Limiting @@ -979,4 +1057,4 @@ with: **Next Steps:** - [Pre-Commit Hook Setup](pre-commit-hook.md) β€” Catch issues before commit - [Local Installation](local-installation.md) β€” Run scans from your terminal -- [Configuration Guide](configuration.md) β€” Detailed configuration options +- [Parameters Reference](parameters.md) β€” Every CLI flag, action input and environment variable, with the mapping between them diff --git a/docs/local-install-docker.md b/docs/local-install-docker.md index a0e54e7..3e52258 100644 --- a/docs/local-install-docker.md +++ b/docs/local-install-docker.md @@ -18,7 +18,8 @@ Run Socket Basics locally using Docker without installing security tools on your # 1. Pull a pinned release from GHCR (no build step required) docker pull ghcr.io/socketdev/socket-basics:3.1.0 -# 2. Create .env file with your credentials +# 2. Create .env file with your credentials (the API key is environment-only; +# the organization can also be passed per run with --socket-org) cat > .env << 'EOF' SOCKET_SECURITY_API_KEY=your-api-key-here SOCKET_ORG=your-org-slug @@ -48,8 +49,9 @@ inspect exactly what's inside: docker inspect ghcr.io/socketdev/socket-basics:3.1.0 \ | jq '.[0].Config.Labels' # { -# "com.socket.trufflehog-version": "3.93.8", -# "com.socket.opengrep-version": "v1.16.5", +# "com.socket.trivy-version": "0.73.0", +# "com.socket.trufflehog-version": "3.96.0", +# "com.socket.opengrep-version": "v1.26.0", # "org.opencontainers.image.version": "3.1.0", # ... # } @@ -70,20 +72,47 @@ docker inspect ghcr.io/socketdev/socket-basics:3.1.0 \ | Docker Hub | `docker.io/socketdev/socket-basics:` | | GHCR (app tests) | `ghcr.io/socketdev/socket-basics-app-tests:` | +### Image Variants + +| Tag | Contents | Use it when | +|-----|----------|-------------| +| `socket-basics:` | Socket Basics with every bundled scanner (OpenGrep, TruffleHog, Socket-built Trivy) and the Socket npm CLI for Tier 1 reachability | **Always, unless Socket has told you otherwise.** This is the image the GitHub Action runs and the one every example in these docs uses. | +| `socket-basics:-heavy` | The standard image plus a pinned copy of the Socket **Python** CLI (`socketcli`) | Only when a pipeline must run the Python Socket CLI and Socket Basics from one container and cannot pull a second image. | + +The heavy variant was built for a single customer deployment with that +constraint. It adds no scanners, no features and no extra findings to Socket +Basics; it is larger, pulls more slowly and carries a second CLI to keep +patched. The larger image does not produce a more thorough scan. If you are not +certain you need `socketcli` inside the same container, use the standard image. + +The heavy image also has a different entrypoint. Its first argument selects the +tool: `socketcli` runs the Python CLI; `socket-basics`, or any other argument, +runs Socket Basics: + +```bash +docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.1.0-heavy socketcli --help +docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.1.0-heavy --workspace /workspace --python +``` + +`latest` and `latest-heavy` are floating aliases published for onboarding +convenience. Pin an exact version (or digest) in anything automated. + ### Pinning in CI/CD -**GitHub Actions** β€” pin to the exact version and only bump when you're ready: +**GitHub Actions** β€” prefer the [action itself](github-action.md), which adds PR +comments and labels. If you must run the image directly, pin the exact version: ```yaml - name: Security scan run: | docker run --rm \ -v "$GITHUB_WORKSPACE:/workspace" \ - -e SOCKET_SECURITY_API_KEY=${{ secrets.SOCKET_API_KEY }} \ + -e SOCKET_SECURITY_API_KEY=${{ secrets.SOCKET_SECURITY_API_KEY }} \ -e SOCKET_ORG=${{ secrets.SOCKET_ORG }} \ ghcr.io/socketdev/socket-basics:3.1.0 \ --workspace /workspace \ - --all-languages \ + --python \ + --javascript \ --secrets \ --console-tabular-enabled ``` @@ -92,12 +121,15 @@ docker inspect ghcr.io/socketdev/socket-basics:3.1.0 \ ```yaml security-scan: - image: ghcr.io/socketdev/socket-basics:3.1.0 + image: + name: ghcr.io/socketdev/socket-basics:3.1.0 + entrypoint: [""] # GitLab needs a shell; the image's entrypoint is socket-basics stage: test script: - socket-basics - --workspace /builds/$CI_PROJECT_PATH - --all-languages + --workspace "$CI_PROJECT_DIR" + --python + --javascript --secrets --console-tabular-enabled variables: @@ -172,8 +204,8 @@ The image pins the bundled tools to specific versions. You can override them at ```bash docker build \ - --build-arg TRUFFLEHOG_VERSION=3.93.8 \ - --build-arg OPENGREP_VERSION=v1.16.5 \ + --build-arg TRUFFLEHOG_VERSION=3.96.0 \ + --build-arg OPENGREP_VERSION=v1.26.0 \ -t socket-basics:3.1.0 . ``` @@ -186,11 +218,14 @@ tests image, build from the `app_tests` directory and use the same build args. ### Verify Installation ```bash -# Check that all tools are available in the container -docker run --rm socket-basics:3.1.0 socket-basics --version -docker run --rm socket-basics:3.1.0 socket --version -docker run --rm socket-basics:3.1.0 opengrep --version -docker run --rm socket-basics:3.1.0 trufflehog --version +# The image's entrypoint is `socket-basics`, so its own flags need no prefix +docker run --rm socket-basics:3.1.0 --version + +# Other bundled tools need --entrypoint +docker run --rm --entrypoint socket socket-basics:3.1.0 --version +docker run --rm --entrypoint opengrep socket-basics:3.1.0 --version +docker run --rm --entrypoint trufflehog socket-basics:3.1.0 --version +docker run --rm --entrypoint trivy socket-basics:3.1.0 --version ``` ### Smoke Test @@ -268,12 +303,17 @@ Create a `.env` file in your project (add to `.gitignore`): ```bash # .env -# Socket Configuration (Required for Enterprise features) +# Socket Configuration (required to upload results and load dashboard config). +# The API key is environment-only. The organization can also be passed as +# --socket-org; --repo names the repository, not the organization. SOCKET_SECURITY_API_KEY=scrt_your_api_key_here SOCKET_ORG=your-organization-slug -# GitHub Integration (for PR comments) +# GitHub Integration (PR comments on a GitHub repository; see +# "Posting PR Comments from a Docker Run" below) GITHUB_TOKEN=ghp_your_github_token +GITHUB_REPOSITORY=owner/repo +GITHUB_PR_NUMBER=123 # Notification Integrations (Enterprise) SLACK_WEBHOOK_URL=https://hooks.slack.com/services/T00/B00/XXXX @@ -290,9 +330,9 @@ MS_SENTINEL_WORKSPACE_ID=your-workspace-id MS_SENTINEL_SHARED_KEY=your-shared-key SUMOLOGIC_ENDPOINT=https://endpoint.sumologic.com/... -# Scanning Options -CONSOLE_TABULAR_ENABLED=true -VERBOSE=false +# Scanning Options (INPUT_* names mirror the CLI flags; see parameters.md#name-mapping) +INPUT_CONSOLE_TABULAR_ENABLED=true +INPUT_VERBOSE=false ``` **Run with .env file:** @@ -362,6 +402,45 @@ docker run --rm \ --python ``` +### Posting PR Comments from a Docker Run + +The GitHub Action wires this up for you. From a plain `docker run` (Jenkins, +CircleCI, a laptop) Socket Basics needs three things to comment on a GitHub +pull request: + +| Variable | Purpose | If omitted | +|----------|---------|------------| +| `GITHUB_TOKEN` | Token with `pull-requests: write` (and `issues: write` for labels) | The PR notifier stays off | +| `GITHUB_REPOSITORY` | `owner/repo` of the pull request | Discovered from the `origin` remote of the mounted checkout | +| `GITHUB_PR_NUMBER` | The pull request number | Looked up through the GitHub API by branch name; nothing is posted if the branch has no open PR or the branch cannot be discovered | + +Set all three for deterministic behavior: + +```bash +docker run --rm \ + -v "$PWD:/workspace" \ + -e SOCKET_SECURITY_API_KEY \ + -e SOCKET_ORG \ + -e GITHUB_TOKEN \ + -e GITHUB_REPOSITORY=owner/repo \ + -e GITHUB_PR_NUMBER=123 \ + ghcr.io/socketdev/socket-basics:3.1.0 \ + --workspace /workspace \ + --python --javascript --secrets +``` + +Notes: +- `-e NAME` with no value forwards the variable from your shell. +- To report only the PR's own changes, add `-e GITHUB_BASE_REF=main` (with the + base branch fetched in the checkout) and `--changed-files pr`. Without a base + ref, `--changed-files auto` falls back to *staged* changes, which are empty + in CI, so nothing would be scanned. `--changed-files current-commit` or an + explicit file list are the alternatives. +- The PR notifier reads `GITHUB_PR_NUMBER`; `--pull-request` only sets the + number recorded on the full scan, so use the environment variable. +- Uploading to the dashboard still needs `SOCKET_SECURITY_API_KEY` and + `SOCKET_ORG`; PR comments and the dashboard are independent. + ## Advanced Usage ### Container Scanning Status @@ -374,24 +453,31 @@ docker run --rm \ ### Save Results to File -Mount a volume to save scan results: +The facts file defaults to `.socket.facts.json` inside the workspace, so with +`-v "$PWD:/workspace"` it already lands in your project directory. To use a +different name or subdirectory, keep it **inside the workspace**: ```bash -# Create results directory mkdir -p ./scan-results -# Run scan and save output docker run --rm \ -v "$PWD:/workspace" \ - -v "$PWD/scan-results:/results" \ --env-file .env \ socket-basics:3.1.0 \ --workspace /workspace \ --python \ --secrets \ - --output /results/scan-results.json + --output /workspace/scan-results/scan-results.json ``` +> [!WARNING] +> Do not write the facts file outside the workspace, for example to a second +> `-v .../results:/results` mount with `--output /results/...`. The dashboard +> upload uses the workspace as the base path of the upload, so a file outside +> it is discarded and the run logs `Need at least one file to be uploaded`. +> The scan looks green but nothing reaches the Socket dashboard. Add +> `scan-results/` to `.gitignore` if you keep results in the repository. + ### Interactive Mode Run the container interactively for debugging: @@ -420,7 +506,7 @@ cat > socket-config.json << 'EOF' { "python_sast_enabled": true, "javascript_sast_enabled": true, - "secrets_enabled": true, + "secret_scanning_enabled": true, "console_tabular_enabled": true, "trufflehog_exclude_dir": "node_modules,vendor,dist" } @@ -463,6 +549,23 @@ for PROJECT in "${PROJECTS[@]}"; do done ``` +### Large Repositories and Monorepos + +`--all-languages` over an entire monorepo is the slowest and noisiest way to run +Socket Basics, and the facts file grows with the repository rather than with the +findings: the SAST connector records one component for every non-gitignored file +in the workspace, so a media-heavy or vendored tree yields a very large +`.socket.facts.json` and upload. + +- Scope to what changed: `--changed-files auto` (staged changes locally), + `--changed-files pr` with `GITHUB_BASE_REF` set in CI, or `--changed-files `. +- Scope to a project: mount and scan one project at a time + (`-v "$PWD/services/api:/workspace"`), or pass an explicit `--scan-files` list. +- Enable only the languages the code uses; `--all-languages` runs every rule set. +- Keep generated, vendored and media files out of git or in `.gitignore`; + ignored files are excluded from the inventory. `--exclude-dir` affects secret + scanning only. + ### CI/CD Integration > **Using GitHub Actions?** Socket Basics has first-class GitHub Actions support with automatic PR comments, labels, and more β€” no Docker setup needed. See the [Quick Start](../README.md#-quick-start---github-actions) or the [GitHub Actions Guide](github-action.md). @@ -479,8 +582,10 @@ pipeline { stage('Security Scan') { steps { script { + // --entrypoint='' is required: Jenkins runs `cat` to keep the + // container alive, and the image's entrypoint is socket-basics. docker.image('ghcr.io/socketdev/socket-basics:3.1.0').inside( - "-v ${WORKSPACE}:/workspace --env-file .env" + "--entrypoint='' -v ${WORKSPACE}:/workspace --env-file .env" ) { sh ''' socket-basics \ @@ -501,12 +606,15 @@ pipeline { ```yaml security-scan: - image: ghcr.io/socketdev/socket-basics:3.1.0 + image: + name: ghcr.io/socketdev/socket-basics:3.1.0 + entrypoint: [""] # GitLab needs a shell; the image's entrypoint is socket-basics stage: test script: - socket-basics - --workspace /builds/$CI_PROJECT_PATH - --all-languages + --workspace "$CI_PROJECT_DIR" + --python + --javascript --secrets --console-tabular-enabled variables: @@ -522,19 +630,15 @@ security-scan: **Solutions:** -1. Run as current user: +1. Fix ownership after the scan. The image runs as root (OpenGrep lives under + `/root`, so `--user` breaks SAST), which means files it writes into the + mount are root-owned: ```bash - docker run --rm \ - -v "$PWD:/workspace" \ - --user "$(id -u):$(id -g)" \ - socket-basics:3.1.0 \ - --workspace /workspace + sudo chown -R "$USER:$USER" .socket.facts.json scan-results ``` -2. Fix ownership after scan: - ```bash - sudo chown -R $USER:$USER ./scan-results - ``` +2. Or point `--output` at a workspace subdirectory you pre-create, and `chown` + only that directory afterwards. ### Volume Mount Not Working @@ -549,12 +653,13 @@ security-scan: socket-basics:3.1.0 ``` -2. Verify mount: +2. Verify mount (the entrypoint is `socket-basics`, so override it to run `ls`): ```bash docker run --rm \ -v "$PWD:/workspace" \ + --entrypoint ls \ socket-basics:3.1.0 \ - ls -la /workspace + -la /workspace ``` ### Environment Variables Not Loaded @@ -640,17 +745,39 @@ security-scan: --output /workspace/results.json # Save to mounted directory ``` -2. Use separate results volume: +2. Keep the file inside the workspace. A separate `/results` volume breaks the + dashboard upload (see [Save Results to File](#save-results-to-file)): ```bash - mkdir -p ./results + mkdir -p ./scan-results docker run --rm \ -v "$PWD:/workspace" \ - -v "$PWD/results:/results" \ socket-basics:3.1.0 \ --workspace /workspace \ - --output /results/scan.json + --output /workspace/scan-results/scan.json ``` +### Results Missing From the Dashboard + +**Problem:** The scan finishes but no full scan appears in the Socket dashboard. + +**Solutions:** + +1. `Need at least one file to be uploaded` in the log: the facts file was + written outside the workspace. Use a path under `/workspace` for `--output`. +2. `No Socket organization configured` in the log: set `SOCKET_ORG`, or use an + API key with the `socket-basics` scope so the organization can be discovered. +3. `Socket API key not detected - running in free plan mode`: the key did not + reach the container. Check the `.env` file or `-e` flags and the variable + name (`SOCKET_SECURITY_API_KEY`). +4. `Insufficient permissions`: the key is missing the `full-scans` scope (upload) + or the `socket-basics` scope (dashboard configuration). + +### PR Comment Not Posted + +See [Posting PR Comments from a Docker Run](#posting-pr-comments-from-a-docker-run): +`GITHUB_TOKEN`, `GITHUB_REPOSITORY` and `GITHUB_PR_NUMBER` must all reach the +container, and `GithubPRNotifier` lines in the log say what was decided. + ## Shell Aliases Add these to your `~/.bashrc` or `~/.zshrc` for quick access: @@ -682,13 +809,15 @@ sb-all ## Best Practices 1. **Use pre-built images** β€” Pull `ghcr.io/socketdev/socket-basics:` instead of building locally -2. **Pin to a specific version** β€” Avoid `:latest` in production CI; pin to `3.1.0` and upgrade deliberately -3. **Use Dependabot** β€” Reference the image in your Dockerfile/Compose to get automatic upgrade PRs -4. **Inspect baked-in labels** β€” Run `docker inspect | jq '.[0].Config.Labels'` to verify tool versions -5. **Use .env files** β€” Keep credentials out of command history -6. **Add .env to .gitignore** β€” Never commit secrets -7. **Mount minimal volumes** β€” Only mount what you need to scan -8. **Resource limits** β€” Set CPU/memory limits for long-running scans +2. **Use the standard image** β€” `-heavy` exists for one deployment constraint (see [Image Variants](#image-variants)); it adds nothing to Socket Basics +3. **Pin to a specific version** β€” Avoid `:latest` in production CI; pin to `3.1.0` and upgrade deliberately +4. **Use Dependabot** β€” Reference the image in your Dockerfile/Compose to get automatic upgrade PRs +5. **Inspect baked-in labels** β€” Run `docker inspect | jq '.[0].Config.Labels'` to verify tool versions +6. **Use .env files** β€” Keep credentials out of command history +7. **Add .env to .gitignore** β€” Never commit secrets +8. **Mount minimal volumes** β€” Only mount what you need to scan +9. **Keep `--output` inside the workspace** β€” Anything else is not uploaded to the dashboard +10. **Resource limits** β€” Set CPU/memory limits for long-running scans ## Example: Complete Workflow @@ -700,12 +829,12 @@ set -e # Configuration PROJECT_DIR="$(pwd)" -RESULTS_DIR="./scan-results" -IMAGE_NAME="socket-basics:3.1.0" +RESULTS_DIR="scan-results" # relative to the project: it must stay inside the workspace +IMAGE_NAME="ghcr.io/socketdev/socket-basics:3.1.0" ENV_FILE=".env" -# Create results directory -mkdir -p "$RESULTS_DIR" +# Create results directory (add it to .gitignore) +mkdir -p "$PROJECT_DIR/$RESULTS_DIR" # Verify .env exists if [ ! -f "$ENV_FILE" ]; then @@ -713,7 +842,7 @@ if [ ! -f "$ENV_FILE" ]; then cat > "$ENV_FILE" << 'EOF' SOCKET_SECURITY_API_KEY=your-api-key-here SOCKET_ORG=your-org-slug -CONSOLE_TABULAR_ENABLED=true +INPUT_CONSOLE_TABULAR_ENABLED=true EOF echo "⚠️ Please edit .env with your credentials" exit 1 @@ -721,10 +850,10 @@ fi echo "πŸ” Starting security scan..." -# Run comprehensive scan +# Run the scan. Enable the languages the project uses rather than +# --all-languages on a large monorepo (see "Large Repositories and Monorepos"). docker run --rm \ -v "$PROJECT_DIR:/workspace" \ - -v "$RESULTS_DIR:/results" \ --env-file "$ENV_FILE" \ "$IMAGE_NAME" \ --workspace /workspace \ @@ -732,7 +861,7 @@ docker run --rm \ --secrets \ --socket-tier1 \ --console-tabular-enabled \ - --output /results/scan-$(date +%Y%m%d-%H%M%S).json + --output "/workspace/$RESULTS_DIR/scan-$(date +%Y%m%d-%H%M%S).json" echo "βœ… Scan complete! Results saved to $RESULTS_DIR" ``` diff --git a/docs/local-installation.md b/docs/local-installation.md index 5a5c0b7..7534d4e 100644 --- a/docs/local-installation.md +++ b/docs/local-installation.md @@ -23,8 +23,11 @@ git clone https://github.com/SocketDev/socket-basics.git cd socket-basics pip install -e . -# Install pinned security tools -brew install socket trivy trufflehog +# Install the Socket CLI (npm package; there is no Homebrew formula) +npm install -g socket + +# Install security tools +brew install trivy trufflehog # Install OpenGrep (SAST scanning) curl -fsSL https://raw.githubusercontent.com/opengrep/opengrep/main/install.sh | bash @@ -50,7 +53,7 @@ For detailed installation instructions, continue reading below. ### Required Software -**Python 3.8 or higher:** +**Python 3.10 or higher:** ```bash # Check Python version @@ -274,23 +277,26 @@ OpenGrep works with the bundled Socket Basics SAST rules. No additional configur # macOS/Linux with Homebrew: brew install trufflehog -# Using Docker (alternative; pin explicitly): -docker pull trufflesecurity/trufflehog:v3.93.8 +# Using Docker (alternative; pin explicitly β€” Docker Hub tags have no "v" prefix): +docker pull trufflesecurity/trufflehog:3.96.0 # Manual installation (Linux): -wget https://github.com/trufflesecurity/trufflehog/releases/download/v3.93.8/trufflehog_3.93.8_linux_amd64.tar.gz -tar -xzf trufflehog_3.93.8_linux_amd64.tar.gz +wget https://github.com/trufflesecurity/trufflehog/releases/download/v3.96.0/trufflehog_3.96.0_linux_amd64.tar.gz +tar -xzf trufflehog_3.96.0_linux_amd64.tar.gz sudo mv trufflehog /usr/local/bin/ # Manual installation (macOS): -wget https://github.com/trufflesecurity/trufflehog/releases/download/v3.93.8/trufflehog_3.93.8_darwin_arm64.tar.gz -tar -xzf trufflehog_3.93.8_darwin_arm64.tar.gz +wget https://github.com/trufflesecurity/trufflehog/releases/download/v3.96.0/trufflehog_3.96.0_darwin_arm64.tar.gz +tar -xzf trufflehog_3.96.0_darwin_arm64.tar.gz sudo mv trufflehog /usr/local/bin/ # Verify installation trufflehog --version ``` +The versions above match the TruffleHog pinned in the Socket Basics image +(`TRUFFLEHOG_VERSION` in the [Dockerfile](../Dockerfile)). + **Documentation:** https://github.com/trufflesecurity/trufflehog ## Verification @@ -307,8 +313,8 @@ socket-basics --version # View help socket-basics --help -# Test basic scan (dry run) -socket-basics --python-sast-enabled --verbose +# Run a first scan of the current directory with verbose logging +socket-basics --python --verbose ``` ### Test Individual Tools @@ -405,6 +411,18 @@ chmod +x check-installation.sh ## Configuration +### Socket Credentials + +The API key is an **environment variable only**; there is no flag for it, so it +never lands in shell history. The organization comes from `SOCKET_ORG`, or from +`--socket-org` when you want to set it per run. `--repo owner/repo` names the +repository recorded on the scan and is not the organization. + +| Setting | CLI flag | Environment variable | Also accepted | +|---------|----------|----------------------|---------------| +| Organization | `--socket-org` | `SOCKET_ORG` | `SOCKET_ORG_SLUG`, `INPUT_SOCKET_ORG` | +| API key | β€” | `SOCKET_SECURITY_API_KEY` | `SOCKET_SECURITY_API_TOKEN`, `SOCKET_API_KEY`, `INPUT_SOCKET_SECURITY_API_KEY` | + ### Environment Variables Create `.env` file in your project (add to `.gitignore`): @@ -414,8 +432,12 @@ Create `.env` file in your project (add to `.gitignore`): SOCKET_ORG=your-org-slug SOCKET_SECURITY_API_KEY=your-socket-api-key -# GitHub Integration (for PR comments) +# GitHub Integration (for PR comments). The repository and PR number are +# discovered from git and the GitHub API when omitted; set them explicitly +# for deterministic commenting outside GitHub Actions. GITHUB_TOKEN=your-github-token +GITHUB_REPOSITORY=owner/repo +GITHUB_PR_NUMBER=123 # Notification Integrations (Enterprise) SLACK_WEBHOOK_URL=https://hooks.slack.com/services/... @@ -424,10 +446,9 @@ JIRA_EMAIL=you@example.com JIRA_API_TOKEN=your-jira-token JIRA_PROJECT=SEC -# Scanning Options -INPUT_CONSOLE_ENABLED=true -INPUT_VERBOSE=false +# Scanning Options (INPUT_* names mirror the CLI flags; see parameters.md#name-mapping) INPUT_CONSOLE_TABULAR_ENABLED=true +INPUT_VERBOSE=false ``` Load environment variables: @@ -440,7 +461,7 @@ source .env export $(cat .env | grep -v '^#' | xargs) # Option 3: Run with env prefix -env $(cat .env | grep -v '^#' | xargs) socket-basics --python-sast-enabled +env $(cat .env | grep -v '^#' | xargs) socket-basics --python ``` ### Configuration File @@ -456,7 +477,7 @@ Create `.socket-basics.json`: "console_tabular_enabled": true, "verbose": false, "trufflehog_exclude_dir": "node_modules,vendor,dist,.git", - "python_disabled_rules": "unused-import,line-too-long", + "python_disabled_rules": "python-bare-except,python-insecure-temp-file", "socket_tier_1_enabled": false } ``` @@ -474,10 +495,10 @@ Add to your `~/.bashrc` or `~/.zshrc`: ```bash # Quick security scans alias sb='socket-basics' -alias sb-quick='socket-basics --secret-scanning-enabled --console-tabular-enabled' -alias sb-python='socket-basics --python-sast-enabled --secret-scanning-enabled --console-tabular-enabled' -alias sb-js='socket-basics --javascript-sast-enabled --secret-scanning-enabled --console-tabular-enabled' -alias sb-full='socket-basics --all-languages-enabled --secret-scanning-enabled --socket-tier-1-enabled --console-tabular-enabled' +alias sb-quick='socket-basics --secrets --console-tabular-enabled' +alias sb-python='socket-basics --python --secrets --console-tabular-enabled' +alias sb-js='socket-basics --javascript --secrets --console-tabular-enabled' +alias sb-full='socket-basics --all-languages --secrets --socket-tier1 --console-tabular-enabled' # With venv activation alias sb-activate='source .venv/bin/activate && socket-basics' @@ -498,74 +519,85 @@ source ~/.bashrc # or source ~/.zshrc source .venv/bin/activate # Quick secret scan -socket-basics --secret-scanning-enabled +socket-basics --secrets # Python SAST + secrets -socket-basics --python-sast-enabled --secret-scanning-enabled +socket-basics --python --secrets # JavaScript/TypeScript SAST + secrets -socket-basics --javascript-sast-enabled --typescript-sast-enabled --secret-scanning-enabled +socket-basics --javascript --secrets # All languages -socket-basics --all-languages-enabled --secret-scanning-enabled +socket-basics --all-languages --secrets ``` ### Advanced Scans ```bash -# With Socket Tier 1 reachability +# With Socket Tier 1 reachability (SOCKET_SECURITY_API_KEY comes from the +# environment; the organization from SOCKET_ORG or --socket-org) socket-basics \ - --python-sast-enabled \ - --secret-scanning-enabled \ - --socket-tier-1-enabled \ - --socket-org your-org + --python \ + --secrets \ + --socket-tier1 \ + --socket-org your-org-slug # Container scanning socket-basics \ - --container-images nginx:1.27.4,redis:7.4 \ + --images nginx:1.27.4,redis:7.4 \ --dockerfiles Dockerfile,docker/Dockerfile.prod # Scan specific workspace socket-basics \ --workspace /path/to/project \ - --python-sast-enabled \ - --secret-scanning-enabled + --python \ + --secrets # Custom output file socket-basics \ - --python-sast-enabled \ + --python \ --output ./security-results.json ``` ### With Enterprise Features +Notifiers switch on when their endpoint is present, either as an environment +variable (`SLACK_WEBHOOK_URL`, `JIRA_URL`, ...) or as a CLI flag +(`--slack-webhook-url`, `--jira-url`, ...). Results upload to the dashboard +whenever `SOCKET_SECURITY_API_KEY` and `SOCKET_ORG` are set. + ```bash -# Load environment variables +# Load credentials and notifier endpoints source .env -# Scan with Slack notifications +# Scan with Slack notifications (uses SLACK_WEBHOOK_URL from .env) socket-basics \ - --python-sast-enabled \ - --secret-scanning-enabled \ - --socket-org $SOCKET_ORG \ + --python \ + --secrets \ --console-tabular-enabled -# Scan with Jira ticket creation +# Scan with Jira ticket creation, passing the endpoint explicitly socket-basics \ - --all-languages-enabled \ - --secret-scanning-enabled \ - --socket-org $SOCKET_ORG \ + --all-languages \ + --secrets \ + --jira-url "https://your-org.atlassian.net" \ + --jira-project SEC \ --console-tabular-enabled # Full enterprise scan socket-basics \ - --all-languages-enabled \ - --secret-scanning-enabled \ - --socket-tier-1-enabled \ - --socket-org $SOCKET_ORG \ + --all-languages \ + --secrets \ + --socket-tier1 \ --verbose ``` +> [!TIP] +> `--all-languages` over a large monorepo is slow and noisy, and the facts file +> records every non-gitignored file in the workspace. Prefer the languages the +> project uses, and `--changed-files auto` for pre-commit or PR runs. See +> [Large repositories and monorepos](local-install-docker.md#large-repositories-and-monorepos). + ### CI/CD Integration > **Using GitHub Actions?** Socket Basics has first-class GitHub Actions support with automatic PR comments, labels, and more β€” no local installation needed. See the [Quick Start](../README.md#-quick-start---github-actions) or the [GitHub Actions Guide](github-action.md). @@ -582,11 +614,11 @@ brew install fswatch sudo apt install inotify-tools # Watch and scan on changes (macOS) -fswatch -o . | xargs -n1 -I{} socket-basics --python-sast-enabled --secret-scanning-enabled +fswatch -o . | xargs -n1 -I{} socket-basics --python --secrets # Watch and scan on changes (Linux) while inotifywait -r -e modify .; do - socket-basics --python-sast-enabled --secret-scanning-enabled + socket-basics --python --secrets done ``` @@ -647,14 +679,14 @@ ls -la /path/to/project 1. Exclude unnecessary directories: ```bash socket-basics \ - --python-sast-enabled \ - --trufflehog-exclude-dir "node_modules,vendor,dist,.git" + --python \ + --exclude-dir "node_modules,vendor,dist,.git" ``` 2. Scan specific languages only: ```bash - # Instead of --all-languages-enabled - socket-basics --python-sast-enabled --javascript-sast-enabled + # Instead of --all-languages + socket-basics --python --javascript ``` 3. Use faster storage (SSD vs HDD) @@ -731,4 +763,4 @@ eval "$(/opt/homebrew/bin/brew shellenv)" **Next Steps:** - [GitHub Actions Integration](github-action.md) β€” Automate in CI/CD - [Pre-Commit Hook Setup](pre-commit-hook.md) β€” Catch issues before commit -- [Configuration Guide](configuration.md) β€” Detailed configuration options +- [Parameters Reference](parameters.md) β€” Every CLI flag, action input and environment variable, with the mapping between them diff --git a/docs/parameters.md b/docs/parameters.md index f40ec6b..4c8e314 100644 --- a/docs/parameters.md +++ b/docs/parameters.md @@ -4,6 +4,7 @@ Complete reference for all CLI options and environment variables supported by So ## Table of Contents +- [Name Mapping](#name-mapping) β€” CLI flag ↔ action input ↔ environment variable ↔ JSON key - [Core Options](#core-options) - [Language Scanning](#language-scanning) - [Secret Scanning](#secret-scanning) @@ -13,10 +14,151 @@ Complete reference for all CLI options and environment variables supported by So - [Environment Variables](#environment-variables) - [Configuration File](#configuration-file) +## Name Mapping + +The same setting is spelled differently on each interface, and the CLI is +strict: an unknown flag such as `--python-sast-enabled` exits with +`unrecognized arguments`. Use this table to translate between them. Every +GitHub Action input is delivered to the container as the environment variable +`INPUT_` (the input name upper-cased); a JSON key is the name used in a +`--config` file and in Socket dashboard configuration. + +### Core, scope and credentials + +| CLI flag | GitHub Action input | Environment variable | JSON key | Description | +|---|---|---|---|---| +| `--workspace` | β€”1 | `GITHUB_WORKSPACE` | `workspace` | Directory to scan. Defaults to the current directory (`GITHUB_WORKSPACE` in Actions). | +| `--output` | β€” | `OUTPUT_DIR` (directory only) | β€” | Facts file name, default `.socket.facts.json`. Must stay inside the workspace when uploading to the dashboard. | +| `--config` | β€” | β€” | β€” | Path to a JSON configuration file. | +| `--changed-files` | `changed_files` | `INPUT_CHANGED_FILES` | `changed_files` | Diff-only scope: `auto`, `pr`, `current-commit`, a commit hash or a file list. | +| `--scan-files` | `scan_files` | `INPUT_SCAN_FILES` | `scan_files` | Explicit comma-separated file list. | +| β€” | `scan_all` | `INPUT_SCAN_ALL` | `scan_all` | Fail-open fallback when a `changed_files` scope cannot be resolved. | +| `--verbose`, `-v` | `verbose` | `INPUT_VERBOSE` | `verbose` | DEBUG logging. | +| `--console-tabular-enabled` | `console_tabular_enabled` | `INPUT_CONSOLE_TABULAR_ENABLED` | `console_tabular_enabled` | Print consolidated findings as tables. | +| `--console-json-enabled` | `console_json_enabled` | `INPUT_CONSOLE_JSON_ENABLED` | `console_json_enabled` | Print consolidated findings as JSON. | +| `--socket-org` | `socket_org` | `SOCKET_ORG` (also `SOCKET_ORG_SLUG`, `INPUT_SOCKET_ORG`) | `socket_org` | Socket organization slug. Not the same thing as `--repo`. | +| β€”2 | `socket_security_api_key` | `SOCKET_SECURITY_API_KEY` (also `SOCKET_SECURITY_API_TOKEN`, `SOCKET_API_KEY`, `INPUT_SOCKET_SECURITY_API_KEY`) | `socket_api_key`3 | Socket API key (`full-scans` scope to upload, `socket-basics` scope to load dashboard config). | +| `--repo` | β€” | `GITHUB_REPOSITORY` | `repo` | `owner/repo` recorded on the scan and used to find the PR. Discovered from git when omitted. | +| `--branch` | β€” | `GITHUB_HEAD_REF`, `GITHUB_REF_NAME` | `branch` | Branch recorded on the scan. Discovered from git when omitted. | +| `--pull-request` | β€” | `GITHUB_PR_NUMBER` (also `INPUT_PR_NUMBER`) | β€” | PR number. The PR notifier reads the environment variable. | +| `--default-branch` | β€” | `SOCKET_DEFAULT_BRANCH` | β€” | Mark the scan as the repository default branch. | +| `--enable-s3-upload` | β€” | `SOCKET_S3_ENABLED` | β€” | Upload the facts file to S3 (see [S3 Upload Configuration](#s3-upload-configuration)). | + +1 The action has no `workspace` input; it always scans `GITHUB_WORKSPACE`. Narrow the scope with `changed_files` / `scan_files` instead. +2 No CLI flag exists for the API key, so it never lands in shell history. Set the environment variable. +3 A JSON `socket_api_key` is used to upload results but not to load dashboard configuration; prefer the environment variable. + +### SAST languages (OpenGrep) + +| CLI flag | GitHub Action input | Environment variable | JSON key | Description | +|---|---|---|---|---| +| `--python` | `python_sast_enabled` | `INPUT_PYTHON_SAST_ENABLED` | `python_sast_enabled` | Enable Python SAST scanning | +| `--javascript` | `javascript_sast_enabled` | `INPUT_JAVASCRIPT_SAST_ENABLED` | `javascript_sast_enabled` | Enable JavaScript/TypeScript SAST scanning | +| `--go` | `go_sast_enabled` | `INPUT_GO_SAST_ENABLED` | `go_sast_enabled` | Enable Go SAST scanning | +| `--golang` | `golang_sast_enabled` | `INPUT_GOLANG_SAST_ENABLED` | `golang_sast_enabled` | Enable Golang SAST scanning | +| `--java` | `java_sast_enabled` | `INPUT_JAVA_SAST_ENABLED` | `java_sast_enabled` | Enable Java SAST scanning | +| `--php` | `php_sast_enabled` | `INPUT_PHP_SAST_ENABLED` | `php_sast_enabled` | Enable PHP SAST scanning | +| `--ruby` | `ruby_sast_enabled` | `INPUT_RUBY_SAST_ENABLED` | `ruby_sast_enabled` | Enable Ruby SAST scanning | +| `--csharp` | `csharp_sast_enabled` | `INPUT_CSHARP_SAST_ENABLED` | `csharp_sast_enabled` | Enable C# SAST scanning | +| `--dotnet` | `dotnet_sast_enabled` | `INPUT_DOTNET_SAST_ENABLED` | `dotnet_sast_enabled` | Enable .NET SAST scanning | +| `--c` | `c_sast_enabled` | `INPUT_C_SAST_ENABLED` | `c_sast_enabled` | Enable C SAST scanning | +| `--cpp` | `cpp_sast_enabled` | `INPUT_CPP_SAST_ENABLED` | `cpp_sast_enabled` | Enable C++ SAST scanning | +| `--kotlin` | `kotlin_sast_enabled` | `INPUT_KOTLIN_SAST_ENABLED` | `kotlin_sast_enabled` | Enable Kotlin SAST scanning | +| `--scala` | `scala_sast_enabled` | `INPUT_SCALA_SAST_ENABLED` | `scala_sast_enabled` | Enable Scala SAST scanning | +| `--swift` | `swift_sast_enabled` | `INPUT_SWIFT_SAST_ENABLED` | `swift_sast_enabled` | Enable Swift SAST scanning | +| `--rust` | `rust_sast_enabled` | `INPUT_RUST_SAST_ENABLED` | `rust_sast_enabled` | Enable Rust SAST scanning | +| `--elixir` | `elixir_sast_enabled` | `INPUT_ELIXIR_SAST_ENABLED` | `elixir_sast_enabled` | Enable Elixir SAST scanning | +| `--erlang` | `erlang_sast_enabled` | `INPUT_ERLANG_SAST_ENABLED` | `erlang_sast_enabled` | Enable Erlang SAST scanning | + +`--javascript` / `javascript_sast_enabled` covers TypeScript; there is no separate TypeScript setting. + +### SAST rules and options + +| CLI flag | GitHub Action input | Environment variable | JSON key | Description | +|---|---|---|---|---| +| `--all-languages` | `all_languages_enabled` | `INPUT_ALL_LANGUAGES_ENABLED` | `all_languages_enabled` | Enable SAST for all supported languages | +| `--all-rules` | `all_rules_enabled` | `INPUT_ALL_RULES_ENABLED` | `all_rules_enabled` | Run all bundled SAST rules regardless of language filters | +| `--opengrep-notify` | `opengrep_notification_method` | `INPUT_OPENGREP_NOTIFICATION_METHOD` | `notification_method` | Notification method for OpenGrep (e.g., console, slack) | +| `--use-custom-sast-rules` | `use_custom_sast_rules` | `INPUT_USE_CUSTOM_SAST_RULES` | `use_custom_sast_rules` | Use custom SAST rules instead of bundled rules (falls back to bundled rules for languages without custom rules) | +| `--custom-sast-rule-path` | `custom_sast_rule_path` | `INPUT_CUSTOM_SAST_RULE_PATH` | `custom_sast_rule_path` | Relative path to custom SAST rules directory (relative to workspace if set, otherwise cwd) | +| `--sast-ignore-overrides` | `sast_ignore_overrides` | `INPUT_SAST_IGNORE_OVERRIDES` | `sast_ignore_overrides` | Comma-separated list of SAST ignore overrides in rule_id or rule_id:path format | +| `---enabled-rules` | `_enabled_rules` | `INPUT__ENABLED_RULES` | `_enabled_rules` | Comma-separated allowlist of rules for one language (defaults to the high-confidence set). | +| `---disabled-rules` | `_disabled_rules` | `INPUT__DISABLED_RULES` | `_disabled_rules` | Comma-separated rules to disable for one language. | + +`` is one of: `c`, `cpp`, `csharp`, `dotnet`, `elixir`, `go`, `java`, `javascript`, `kotlin`, `php`, `python`, `ruby`, `rust`, `scala`, `swift`. + +### Secret scanning (TruffleHog) + +| CLI flag | GitHub Action input | Environment variable | JSON key | Description | +|---|---|---|---|---| +| `--secrets` | `secret_scanning_enabled` | `INPUT_SECRET_SCANNING_ENABLED` | `secret_scanning_enabled` | Enable secret scanning | +| `--disable-secrets` | `disable_all_secrets` | `INPUT_DISABLE_ALL_SECRETS` | `disable_all_secrets` | Disable all secret scanning features | +| `--exclude-dir` | `trufflehog_exclude_dir` | `INPUT_TRUFFLEHOG_EXCLUDE_DIR` | `trufflehog_exclude_dir` | Comma-separated literal directory/file names or glob patterns to exclude from secret scanning beneath the workspace root; matching is case-sensitive | +| `--trufflehog-notify` | `trufflehog_notification_method` (alias `notification_method`) | `INPUT_TRUFFLEHOG_NOTIFICATION_METHOD` | `notification_method` | Notification method for TruffleHog (e.g., console, slack) | +| `--show-unverified` | `trufflehog_show_unverified` | `INPUT_TRUFFLEHOG_SHOW_UNVERIFIED` | `trufflehog_show_unverified` | Show unverified secrets in TruffleHog results | + +### Socket Tier 1 reachability + +| CLI flag | GitHub Action input | Environment variable | JSON key | Description | +|---|---|---|---|---| +| `--socket-tier1` | `socket_tier_1_enabled` | `SOCKET_TIER_1_ENABLED` | `socket_tier_1_enabled` | Enable Socket Tier 1 reachability analysis | +| `--socket-additional-params` | `socket_additional_params` | `SOCKET_ADDITIONAL_PARAMS` | `socket_additional_params` | Additional CLI params for 'socket scan reach' (comma or space separated). Also reads SOCKET_ADDITIONAL_PARAMS | + +Note the environment variable names here have no `INPUT_` prefix. + +### Container scanning (Trivy) + +| CLI flag | GitHub Action input | Environment variable | JSON key | Description | +|---|---|---|---|---| +| `--images` | `container_images` | `INPUT_CONTAINER_IMAGES_TO_SCAN` | `container_images` | Comma-separated list of container images to scan (auto-enables image scanning) | +| `--dockerfiles` | `dockerfiles` | `INPUT_DOCKERFILES` | `dockerfiles` | Comma-separated list of Dockerfiles to scan (auto-enables Dockerfile scanning) | +| `--trivy-notify` | `trivy_notification_method` | `INPUT_TRIVY_NOTIFICATION_METHOD` | `trivy_notification_method` | Notification method for Trivy (e.g., console, slack) | +| `--trivy-disabled-rules` | `trivy_disabled_rules` | `INPUT_TRIVY_DISABLED_RULES` | `trivy_disabled_rules` | Comma-separated list of Trivy rules to disable | +| `--trivy-image-scanning-disabled` | `trivy_image_scanning_disabled` | `INPUT_TRIVY_IMAGE_SCANNING_DISABLED` | `trivy_image_scanning_disabled` | Disable Trivy image scanning | +| `--trivy-vuln-enabled` | `trivy_vuln_enabled` | `INPUT_TRIVY_VULN_ENABLED` | `trivy_vuln_enabled` | Enable Trivy vulnerability scanning for all supported language ecosystems | + +### Notifications + +A notifier turns on when its endpoint or token is present from any source. + +| CLI flag | GitHub Action input | Environment variable | JSON key | Description | +|---|---|---|---|---| +| `--slack-webhook-url` | `slack_webhook_url` | `INPUT_SLACK_WEBHOOK_URL` (also `SLACK_WEBHOOK_URL`) | `slack_webhook_url` | Slack webhook URL (also reads SLACK_WEBHOOK_URL or INPUT_SLACK_WEBHOOK_URL) | +| `--webhook-url` | `webhook_url` | `INPUT_WEBHOOK_URL` (also `WEBHOOK_URL`) | `webhook_url` | Generic webhook URL for WebhookNotifier | +| `--msteams-webhook-url` | `msteams_webhook_url` | `INPUT_MSTEAMS_WEBHOOK_URL` (also `MSTEAMS_WEBHOOK_URL`) | `msteams_webhook_url` | MS Teams incoming webhook URL (also reads MSTEAMS_WEBHOOK_URL or INPUT_MSTEAMS_WEBHOOK_URL) | +| `--jira-url` | `jira_url` (alias `server`) | `INPUT_JIRA_URL` (also `JIRA_URL`) | `jira_url` | Jira base URL (turns the Jira notifier on) | +| `--jira-project` | `jira_project` (alias `project`) | `INPUT_JIRA_PROJECT` (also `JIRA_PROJECT`) | `jira_project` | Jira project key | +| `--jira-email` | `jira_email` | `INPUT_JIRA_EMAIL` (also `JIRA_EMAIL`) | `jira_email` | Jira account email | +| `--jira-api-token` | `jira_api_token` | `INPUT_JIRA_API_TOKEN` (also `JIRA_API_TOKEN`) | `jira_api_token` | Jira API token | +| `--sumologic-endpoint` | `sumologic_endpoint` | `INPUT_SUMOLOGIC_ENDPOINT` (also `SUMOLOGIC_ENDPOINT`, `SUMO_LOGIC_HTTP_SOURCE_URL`) | `sumologic_endpoint` | Sumo Logic HTTP source URL | +| `--ms-sentinel-workspace-id` | `ms_sentinel_workspace_id` | `INPUT_MS_SENTINEL_WORKSPACE_ID` (also `MS_SENTINEL_WORKSPACE_ID`) | `ms_sentinel_workspace_id` | Microsoft Sentinel workspace ID | +| `--ms-sentinel-key` | `ms_sentinel_key` (alias `ms_sentinel_shared_key`) | `INPUT_MS_SENTINEL_KEY` (also `MS_SENTINEL_SHARED_KEY`, `INPUT_MS_SENTINEL_SHARED_KEY`) | `ms_sentinel_key` | Microsoft Sentinel shared key | + +### GitHub PR comments and labels + +| CLI flag | GitHub Action input | Environment variable | JSON key | Description | +|---|---|---|---|---| +| `--github-token` | `github_token` | `GITHUB_TOKEN` (also `INPUT_GITHUB_TOKEN`) | `github_token` | GitHub token (turns the PR notifier on) | +| `--github-api-url` | β€” | `GITHUB_API_URL` | `GITHUB_API_URL` | GitHub API base URL; set automatically by GitHub Actions | +| `--pr-comment` | `pr_comment_enabled` | `INPUT_PR_COMMENT_ENABLED` | `pr_comment_enabled` | Post/update the findings comment on the PR (scanning and dashboard upload are unaffected) | +| `--pr-comment-links` | `pr_comment_links_enabled` | `INPUT_PR_COMMENT_LINKS_ENABLED` | `pr_comment_links_enabled` | Enable clickable file/line links in PR comments | +| `--pr-comment-collapse` | `pr_comment_collapse_enabled` | `INPUT_PR_COMMENT_COLLAPSE_ENABLED` | `pr_comment_collapse_enabled` | Enable collapsible sections in PR comments | +| `--pr-comment-collapse-non-critical` | `pr_comment_collapse_non_critical` | `INPUT_PR_COMMENT_COLLAPSE_NON_CRITICAL` | `pr_comment_collapse_non_critical` | Auto-collapse non-critical findings (critical stays expanded) | +| `--pr-comment-collapse-all` | `pr_comment_collapse_all` | `INPUT_PR_COMMENT_COLLAPSE_ALL` | `pr_comment_collapse_all` | Collapse the SAST and Socket Tier 1 sections, critical findings included | +| `--pr-comment-code-fencing` | `pr_comment_code_fencing_enabled` | `INPUT_PR_COMMENT_CODE_FENCING_ENABLED` | `pr_comment_code_fencing_enabled` | Enable language-aware code fencing for trace output | +| `--pr-comment-show-rules` | `pr_comment_show_rule_names` | `INPUT_PR_COMMENT_SHOW_RULE_NAMES` | `pr_comment_show_rule_names` | Show explicit rule names for each finding | +| `--pr-labels` | `pr_labels_enabled` | `INPUT_PR_LABELS_ENABLED` | `pr_labels_enabled` | Add severity-based labels to PRs | +| `--pr-label-critical` | `pr_label_critical` | `INPUT_PR_LABEL_CRITICAL` | `pr_label_critical` | Label name for critical severity findings | +| `--pr-label-high` | `pr_label_high` | `INPUT_PR_LABEL_HIGH` | `pr_label_high` | Label name for high severity findings | +| `--pr-label-medium` | `pr_label_medium` | `INPUT_PR_LABEL_MEDIUM` | `pr_label_medium` | Label name for medium severity findings | +| `--pr-label-low` | `pr_label_low` | `INPUT_PR_LABEL_LOW` | `pr_label_low` | Label name for low severity findings | + +Boolean PR options that default to `true` have a `--no-` form on the CLI (for example `--no-pr-comment`). See the [PR Comment Guide](github-pr-comment-guide.md). + ## Core Options ### `--config CONFIG` -Path to JSON configuration file. JSON config is merged with environment variables (environment takes precedence). +Path to JSON configuration file. JSON values override environment variables, and CLI flags override both (see [Configuration Precedence](#configuration-precedence)). **Example:** ```bash @@ -24,10 +166,15 @@ socket-basics --config /path/to/config.json ``` ### `--output OUTPUT` -Output file name for scan results. +Output file name for scan results. A relative name is written inside the workspace. **Default:** `.socket.facts.json` +When results are uploaded to the Socket dashboard the file **must be inside the +scanned workspace**: the upload uses the workspace as its base path, and a file +elsewhere is discarded with `Need at least one file to be uploaded`. In Docker +that means a path under `/workspace`, not a separate results mount. + **Example:** ```bash socket-basics --output scan-results.json @@ -44,7 +191,10 @@ socket-basics --workspace /path/to/project ``` ### `--repo REPO` -Repository name (use when workspace is not a git repo). +Repository name in `owner/repo` form (use when the workspace is not a git repo). +This is the repository recorded on the full scan and used to look up the pull +request. It is **not** the Socket organization: that comes from `--socket-org` +or `SOCKET_ORG`. **Example:** ```bash @@ -158,7 +308,7 @@ workflow and pass it in yourself: run: echo "ref=$(gh pr view ${{ github.event.issue.number }} --json baseRefName -q .baseRefName)" >> "$GITHUB_OUTPUT" env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} -- uses: SocketDev/socket-basics@v2 +- uses: SocketDev/socket-basics@v3.1.0 env: GITHUB_BASE_REF: ${{ steps.prbase.outputs.ref }} with: @@ -245,6 +395,11 @@ Enable SAST for all supported languages. socket-basics --all-languages ``` +On a large repository this is the slowest and noisiest option, and the facts +file records every non-gitignored file in the workspace regardless of findings. +Prefer the languages the code uses, and `--changed-files` for PR and pre-commit +runs. See [Large Repositories and Monorepos](local-install-docker.md#large-repositories-and-monorepos). + ### `--all-rules` Run all bundled SAST rules regardless of language filters. @@ -457,6 +612,29 @@ socket-basics --trivy-vuln-enabled ## Socket Integration +### Socket organization and API key + +The organization can be set per run with `--socket-org` or through the +environment. The API key is **environment only**; there is deliberately no flag, +so the key never lands in shell history or CI logs. + +| Setting | CLI flag | Environment variable | Also accepted | +|---------|----------|----------------------|---------------| +| Organization slug | `--socket-org` | `SOCKET_ORG` | `SOCKET_ORG_SLUG`, `INPUT_SOCKET_ORG`, `socket_org` in a JSON file | +| API key | β€” | `SOCKET_SECURITY_API_KEY` | `SOCKET_SECURITY_API_TOKEN`, `SOCKET_API_KEY`, `INPUT_SOCKET_SECURITY_API_KEY`, `INPUT_SOCKET_API_KEY` | + +**Example:** +```bash +export SOCKET_SECURITY_API_KEY="scrt_..." +socket-basics --socket-org your-org-slug --python --secrets +``` + +The key needs the `full-scans` scope to upload results and the `socket-basics` +scope to load dashboard configuration. With the latter the organization is +discovered from the key; otherwise set `SOCKET_ORG` explicitly or the run logs +`No Socket organization configured` and uploads nothing. `--repo` names the +repository, not the organization. + ### `--socket-tier1` Enable Socket Tier 1 reachability analysis for dependency scanning. @@ -596,8 +774,13 @@ socket-basics --msteams-webhook-url "https://outlook.office.com/webhook/..." | Variable | Aliases | Description | |----------|---------|-------------| | `GITHUB_TOKEN` | `INPUT_GITHUB_TOKEN` | GitHub token for API access and PR comments | -| `GITHUB_REPOSITORY` | `INPUT_GITHUB_REPOSITORY` | Repository name (owner/repo) | -| `GITHUB_PR_NUMBER` | `INPUT_PR_NUMBER` | Pull request number | +| `GITHUB_REPOSITORY` | `INPUT_GITHUB_REPOSITORY` | Repository name (owner/repo); discovered from the git remote when unset | +| `GITHUB_PR_NUMBER` | `INPUT_PR_NUMBER` | Pull request number; looked up by branch through the API when unset | + +GitHub Actions sets everything below automatically. Outside Actions (a plain +`docker run` or a native install) set the first three yourself to post PR +comments deterministically; see +[Posting PR Comments from a Docker Run](local-install-docker.md#posting-pr-comments-from-a-docker-run). | `GITHUB_WORKSPACE` | - | Workspace directory (auto-set in GitHub Actions) | | `GITHUB_ACTOR` | - | GitHub username who triggered the action | | `GITHUB_HEAD_REF` | - | Source branch for pull request | @@ -609,21 +792,30 @@ socket-basics --msteams-webhook-url "https://outlook.office.com/webhook/..." | Variable | Description | |----------|-------------| -| `OUTPUT_DIR` | Directory for output files (default: current directory) | -| `INPUT_SCAN_ALL` | Set to 'true' to scan all files | +| `OUTPUT_DIR` | Directory for output files (default: current directory, or the `--workspace` when one is given) | +| `INPUT_CHANGED_FILES` | Diff-only scope; same values as `--changed-files` | | `INPUT_SCAN_FILES` | Comma-separated list of files to scan | +| `INPUT_SCAN_ALL` | `'true'` widens to a full scan when a `changed_files` scope cannot be resolved (fail-open) | | `INPUT_CONSOLE_TABULAR_ENABLED` | Enable tabular console output | +| `INPUT_CONSOLE_JSON_ENABLED` | Enable JSON console output | | `INPUT_VERBOSE` | Enable verbose logging | +Scanner and notifier settings use the `INPUT_` form, e.g. +`INPUT_PYTHON_SAST_ENABLED=true`; see the [Name Mapping](#name-mapping). + ### S3 Upload Configuration | Variable | Description | |----------|-------------| -| `SOCKET_S3_ENABLED` | Set to 'true', '1', or 'yes' to enable S3 upload | -| `SOCKET_S3_BUCKET` | S3 bucket name | -| `SOCKET_S3_REGION` | S3 bucket region | -| `SOCKET_S3_ACCESS_KEY_ID` | AWS access key ID | -| `SOCKET_S3_SECRET_ACCESS_KEY` | AWS secret access key | +| `SOCKET_S3_ENABLED` | Set to 'true', '1', or 'yes' to enable S3 upload (or pass `--enable-s3-upload`) | +| `SOCKET_S3_BUCKET` | S3 bucket name (required) | +| `SOCKET_S3_ACCESS_KEY` | AWS access key ID (required) | +| `SOCKET_S3_SECRET_KEY` | AWS secret access key (required) | +| `SOCKET_S3_REGION` | Bucket region (default `us-east-1`) | +| `SOCKET_S3_ENDPOINT` | Custom S3-compatible endpoint URL (optional) | + +These are environment variables only; the GitHub Action has no `s3_*` inputs, so +set them in the step `env:` block. ### Notification Configuration @@ -644,36 +836,40 @@ You can provide configuration via a JSON file using `--config`: ### Example Configuration File +Keys are the **JSON key** column of the [Name Mapping](#name-mapping). Keep +credentials in the environment rather than in the file. + ```json { "workspace": "/path/to/project", - "output": "security-scan.json", "console_tabular_enabled": true, "verbose": false, - + "python_sast_enabled": true, "javascript_sast_enabled": true, + "go_sast_enabled": true, "use_custom_sast_rules": true, "custom_sast_rule_path": ".socket/rules", - "go_sast_enabled": true, "sast_ignore_overrides": "js-sql-injection:index.js", - - "secrets_enabled": true, + + "secret_scanning_enabled": true, "trufflehog_exclude_dir": "node_modules,vendor,dist,.git", - "show_unverified": false, - + "trufflehog_show_unverified": false, + "socket_tier_1_enabled": true, "socket_org": "your-org-slug", - "socket_api_key": "scrt_your_api_key", - - "images": "nginx:1.27.4,redis:7.4", + + "container_images": "nginx:1.27.4,redis:7.4", + "dockerfiles": "Dockerfile", "trivy_vuln_enabled": true, - - "slack_webhook_url": "https://hooks.slack.com/services/T00/B00/XXXX", - "github_token": "ghp_your_token" + + "slack_webhook_url": "https://hooks.slack.com/services/T00/B00/XXXX" } ``` +The output file name is a CLI-only option (`--output`); set +`SOCKET_SECURITY_API_KEY` and `GITHUB_TOKEN` in the environment. + ### Configuration Precedence Configuration is merged in the following order (later sources override earlier ones): diff --git a/docs/pre-commit-hook.md b/docs/pre-commit-hook.md index 1965954..1475013 100644 --- a/docs/pre-commit-hook.md +++ b/docs/pre-commit-hook.md @@ -20,6 +20,12 @@ Catch security issues before they're committed to your repository using Socket B Both methods integrate with Git's pre-commit hook system to automatically scan your code before each commit. +The CLI flags are the same in both: `--python`, `--javascript`, `--secrets`, +`--all-languages`, `--socket-tier1`, `--exclude-dir`. These are the flag names +`socket-basics --help` prints; the GitHub Action uses different names +(`python_sast_enabled`, ...) and the CLI rejects those. See the +[name mapping](parameters.md#name-mapping). + ## Docker Installation (Recommended) Best for: Teams wanting consistent environments without installing security tools locally. @@ -44,7 +50,6 @@ Create `.git/hooks/pre-commit` in your project: ```bash #!/bin/bash -set -e echo "πŸ” Running Socket Basics security scan..." @@ -59,12 +64,11 @@ fi # Run Socket Basics in Docker docker run --rm \ -v "$PWD:/workspace" \ - -e INPUT_CONSOLE_ENABLED=true \ - socket-basics \ + ghcr.io/socketdev/socket-basics:3.1.0 \ --workspace /workspace \ - --python-sast-enabled \ - --javascript-sast-enabled \ - --secret-scanning-enabled \ + --python \ + --javascript \ + --secrets \ --console-tabular-enabled EXIT_CODE=$? @@ -94,38 +98,34 @@ git commit -m "Test commit" ### Docker Pre-Commit Configuration -**Scan only changed files:** +**Scan only staged files:** + +`--changed-files auto` scopes every scanner to the staged changes when there is +no pull-request context, which is exactly what a pre-commit hook wants. Git runs +inside the container against the mounted `.git` directory, so nothing else is +needed. ```bash #!/bin/bash -set -e echo "πŸ” Running Socket Basics security scan on staged files..." -STAGED_FILES=$(git diff --cached --name-only --diff-filter=ACMR) - -if [ -z "$STAGED_FILES" ]; then +if [ -z "$(git diff --cached --name-only --diff-filter=ACMR)" ]; then echo "No files to scan" exit 0 fi -# Create temporary file list -TEMP_FILE=$(mktemp) -echo "$STAGED_FILES" > "$TEMP_FILE" - -# Run scan only on staged files +# Scope the scan to the staged changes docker run --rm \ -v "$PWD:/workspace" \ - -v "$TEMP_FILE:/tmp/scan-files.txt" \ - -e INPUT_CONSOLE_ENABLED=true \ - socket-basics \ + ghcr.io/socketdev/socket-basics:3.1.0 \ --workspace /workspace \ - --python-sast-enabled \ - --secret-scanning-enabled \ + --changed-files auto \ + --python \ + --secrets \ --console-tabular-enabled EXIT_CODE=$? -rm "$TEMP_FILE" if [ $EXIT_CODE -ne 0 ]; then echo "❌ Security issues found! Please fix before committing." @@ -140,7 +140,6 @@ exit 0 ```bash #!/bin/bash -set -e echo "πŸ” Running Socket Basics security scan..." @@ -151,16 +150,15 @@ fi docker run --rm \ -v "$PWD:/workspace" \ - -e INPUT_CONSOLE_ENABLED=true \ -e SOCKET_ORG="$SOCKET_ORG" \ -e SOCKET_SECURITY_API_KEY="$SOCKET_SECURITY_API_KEY" \ - -e INPUT_SLACK_WEBHOOK_URL="$SLACK_WEBHOOK_URL" \ - socket-basics \ + -e SLACK_WEBHOOK_URL="$SLACK_WEBHOOK_URL" \ + ghcr.io/socketdev/socket-basics:3.1.0 \ --workspace /workspace \ - --python-sast-enabled \ - --javascript-sast-enabled \ - --secret-scanning-enabled \ - --socket-tier-1-enabled \ + --python \ + --javascript \ + --secrets \ + --socket-tier1 \ --console-tabular-enabled EXIT_CODE=$? @@ -216,7 +214,6 @@ Create `.git/hooks/pre-commit`: ```bash #!/bin/bash -set -e echo "πŸ” Running Socket Basics security scan..." @@ -227,9 +224,9 @@ fi # Run Socket Basics socket-basics \ - --python-sast-enabled \ - --javascript-sast-enabled \ - --secret-scanning-enabled \ + --python \ + --javascript \ + --secrets \ --console-tabular-enabled EXIT_CODE=$? @@ -262,7 +259,6 @@ git commit -m "Test commit" ```bash #!/bin/bash -set -e echo "πŸ” Quick security check..." @@ -271,7 +267,7 @@ if [ -d ".venv" ]; then fi socket-basics \ - --secret-scanning-enabled \ + --secrets \ --console-tabular-enabled if [ $? -ne 0 ]; then @@ -287,7 +283,6 @@ exit 0 ```bash #!/bin/bash -set -e echo "πŸ” Running comprehensive security scan..." @@ -301,9 +296,9 @@ if [ -f .env ]; then fi socket-basics \ - --all-languages-enabled \ - --secret-scanning-enabled \ - --socket-tier-1-enabled \ + --all-languages \ + --secrets \ + --socket-tier1 \ --console-tabular-enabled \ --verbose @@ -325,7 +320,7 @@ exit 0 **Fast (< 10 seconds):** ```bash -socket-basics --secret-scanning-enabled +socket-basics --secrets ``` - Only scans for leaked secrets - Best for quick feedback during development @@ -333,8 +328,8 @@ socket-basics --secret-scanning-enabled **Balanced (30-60 seconds):** ```bash socket-basics \ - --python-sast-enabled \ - --secret-scanning-enabled + --python \ + --secrets ``` - Language-specific SAST + secrets - Good balance of speed and coverage @@ -342,9 +337,9 @@ socket-basics \ **Comprehensive (2-5 minutes):** ```bash socket-basics \ - --all-languages-enabled \ - --secret-scanning-enabled \ - --socket-tier-1-enabled + --all-languages \ + --secrets \ + --socket-tier1 ``` - All security features enabled - Best for final checks or CI/CD @@ -355,7 +350,6 @@ Only scan relevant languages based on file extensions: ```bash #!/bin/bash -set -e STAGED_FILES=$(git diff --cached --name-only) @@ -363,21 +357,21 @@ SCAN_ARGS="" # Check for Python files if echo "$STAGED_FILES" | grep -q "\.py$"; then - SCAN_ARGS="$SCAN_ARGS --python-sast-enabled" + SCAN_ARGS="$SCAN_ARGS --python" fi # Check for JavaScript/TypeScript files if echo "$STAGED_FILES" | grep -qE "\.(js|ts|jsx|tsx)$"; then - SCAN_ARGS="$SCAN_ARGS --javascript-sast-enabled --typescript-sast-enabled" + SCAN_ARGS="$SCAN_ARGS --javascript" fi # Check for Go files if echo "$STAGED_FILES" | grep -q "\.go$"; then - SCAN_ARGS="$SCAN_ARGS --go-sast-enabled" + SCAN_ARGS="$SCAN_ARGS --go" fi # Always scan for secrets -SCAN_ARGS="$SCAN_ARGS --secret-scanning-enabled" +SCAN_ARGS="$SCAN_ARGS --secrets" if [ -z "$SCAN_ARGS" ]; then echo "No scannable files in commit" @@ -407,8 +401,8 @@ SOCKET_SECURITY_API_KEY=your-api-key # Notification webhooks (optional, Enterprise) SLACK_WEBHOOK_URL=https://hooks.slack.com/services/... -# Scanning options -INPUT_CONSOLE_ENABLED=true +# Scanning options (INPUT_* names mirror the CLI flags; see docs/parameters.md#name-mapping) +INPUT_CONSOLE_TABULAR_ENABLED=true INPUT_VERBOSE=false ``` @@ -423,7 +417,7 @@ Create `.socket-basics.json` in your project root: "secret_scanning_enabled": true, "console_tabular_enabled": true, "trufflehog_exclude_dir": "node_modules,vendor,dist", - "python_disabled_rules": "unused-import" + "python_disabled_rules": "python-bare-except" } ``` @@ -448,13 +442,12 @@ Make the hook non-blocking but still show warnings: ```bash #!/bin/bash -set -e echo "πŸ” Running Socket Basics security scan..." socket-basics \ - --python-sast-enabled \ - --secret-scanning-enabled \ + --python \ + --secrets \ --console-tabular-enabled EXIT_CODE=$? @@ -476,11 +469,10 @@ Only fail on high/critical issues: ```bash #!/bin/bash -set -e OUTPUT=$(socket-basics \ - --python-sast-enabled \ - --secret-scanning-enabled \ + --python \ + --secrets \ --console-json-enabled 2>&1) echo "$OUTPUT" @@ -512,7 +504,7 @@ repos: hooks: - id: socket-basics name: Socket Basics Security Scan - entry: docker run --rm -v "$PWD:/workspace" socket-basics --workspace /workspace --python-sast-enabled --secret-scanning-enabled + entry: docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.1.0 --workspace /workspace --changed-files auto --python --secrets language: system pass_filenames: false ``` @@ -550,7 +542,7 @@ pre-commit install 1. Scan only changed files (see conditional scanning above) 2. Reduce scan scope: ```bash - socket-basics --secret-scanning-enabled # Fast + socket-basics --secrets # Fast ``` 3. Use warning-only mode for local commits 4. Run comprehensive scans only in CI/CD @@ -568,7 +560,7 @@ pre-commit install ```bash /path/to/.venv/bin/socket-basics ``` -3. Install globally: `pip install --user socket-basics` +3. Use the Docker hook instead. Socket Basics is not published to PyPI, so there is no global `pip install`. ### False Positives @@ -578,14 +570,14 @@ pre-commit install 1. Disable specific rules: ```bash socket-basics \ - --python-sast-enabled \ + --python \ --python-disabled-rules "rule-id-1,rule-id-2" ``` 2. Exclude directories: ```bash socket-basics \ - --secret-scanning-enabled \ - --trufflehog-exclude-dir "test,fixtures,samples" + --secrets \ + --exclude-dir "test,fixtures,samples" ``` 3. Use configuration file with exceptions @@ -603,4 +595,4 @@ pre-commit install **Next Steps:** - [GitHub Actions Integration](github-action.md) β€” Automated CI/CD scanning - [Local Installation](local-installation.md) β€” Install security tools natively -- [Configuration Guide](configuration.md) β€” Detailed configuration options +- [Parameters Reference](parameters.md) β€” Every CLI flag, action input and environment variable, with the mapping between them diff --git a/scripts/check_release_docs.py b/scripts/check_release_docs.py index e49a22c..c6f37fb 100644 --- a/scripts/check_release_docs.py +++ b/scripts/check_release_docs.py @@ -1,5 +1,14 @@ #!/usr/bin/env python3 -"""Check or update current Socket Basics release references in documentation.""" +"""Check or update version references in documentation. + +Two kinds of reference are kept in sync: + +* the current Socket Basics release (``@v3.1.0``, ``socket-basics:3.1.0`` ...), + compared against the canonical ``pyproject.toml`` version, and +* the bundled scanner versions the guides quote (TruffleHog, OpenGrep, Trivy), + compared against the ``ARG`` pins in the ``Dockerfile`` so "match the version + bundled in the image" advice stays true. +""" from __future__ import annotations @@ -12,12 +21,14 @@ REPO_ROOT = Path(__file__).resolve().parent.parent PYPROJECT_PATH = REPO_ROOT / "pyproject.toml" +DOCKERFILE_PATH = REPO_ROOT / "Dockerfile" DOC_PATHS = (REPO_ROOT / "README.md", *sorted((REPO_ROOT / "docs").rglob("*.md"))) SEMVER = r"\d+\.\d+\.\d+" # Match only references that describe the current Socket Basics release. Other -# versions in these guides (scanner versions, vulnerable Trivy versions, pinned -# third-party Actions, and so on) must remain independent. +# versions in these guides (vulnerable Trivy versions, pinned third-party +# Actions, and so on) must remain independent. Bundled scanner versions are +# checked separately against the Dockerfile pins via TOOL_REFERENCE_PATTERNS. REFERENCE_PATTERNS = ( re.compile(rf"SocketDev/socket-basics@v?(?P{SEMVER})"), re.compile(rf"SocketDev/socket-basics@\s+# v(?P{SEMVER})"), @@ -33,8 +44,37 @@ re.compile(rf"exact version such as `(?P{SEMVER})`"), re.compile(rf'org\.opencontainers\.image\.version": "(?P{SEMVER})"'), re.compile(rf"pin to `(?P{SEMVER})` and upgrade"), + # The action never publishes a floating major tag, so a `@v2`-style + # reference is stale by definition and is rewritten to the exact release. + re.compile(r"SocketDev/socket-basics@v(?P\d+)(?![\d.])"), ) +# Bundled scanner pins: Dockerfile ARG name per tool, and the doc patterns that +# quote that tool's version. Values are compared without any leading "v". +TOOL_PINS = { + "trufflehog": "TRUFFLEHOG_VERSION", + "opengrep": "OPENGREP_VERSION", + "trivy": "TRIVY_VERSION", +} +TOOL_REFERENCE_PATTERNS = { + "trufflehog": ( + re.compile(rf"trufflesecurity/trufflehog:v?(?P{SEMVER})"), + re.compile(rf"trufflehog/releases/download/v(?P{SEMVER})/"), + re.compile(rf"trufflehog_(?P{SEMVER})_"), + re.compile(rf'com\.socket\.trufflehog-version": "(?P{SEMVER})"'), + re.compile(rf"TRUFFLEHOG_VERSION=(?P{SEMVER})"), + ), + "opengrep": ( + re.compile(rf'com\.socket\.opengrep-version": "v(?P{SEMVER})"'), + re.compile(rf"OPENGREP_VERSION=v(?P{SEMVER})"), + ), + "trivy": ( + re.compile(rf"aquasec/trivy:(?P{SEMVER})"), + re.compile(rf'com\.socket\.trivy-version": "(?P{SEMVER})"'), + re.compile(rf"TRIVY_VERSION=(?P{SEMVER})"), + ), +} + @dataclass(frozen=True) class Reference: @@ -47,9 +87,31 @@ def read_canonical_version() -> str: return tomllib.loads(PYPROJECT_PATH.read_text())["project"]["version"] -def find_references(content: str) -> list[Reference]: +def read_tool_pins() -> dict[str, str]: + """Return {tool: version} from the Dockerfile ARG pins, without any "v".""" + content = DOCKERFILE_PATH.read_text() + pins: dict[str, str] = {} + for tool, arg in TOOL_PINS.items(): + match = re.search(rf"^ARG {arg}=v?(?P{SEMVER})\b", content, re.MULTILINE) + if not match: + raise ValueError(f"Dockerfile has no 'ARG {arg}=' pin") + pins[tool] = match.group("version") + return pins + + +def _targets(version: str) -> list[tuple[str, str, tuple[re.Pattern[str], ...]]]: + """Every (label, expected version, patterns) the docs must agree with.""" + targets = [("socket-basics", version, REFERENCE_PATTERNS)] + for tool, pinned in read_tool_pins().items(): + targets.append((tool, pinned, TOOL_REFERENCE_PATTERNS[tool])) + return targets + + +def find_references( + content: str, patterns: tuple[re.Pattern[str], ...] = REFERENCE_PATTERNS +) -> list[Reference]: references: set[Reference] = set() - for pattern in REFERENCE_PATTERNS: + for pattern in patterns: for match in pattern.finditer(content): references.add( Reference( @@ -61,8 +123,10 @@ def find_references(content: str) -> list[Reference]: return sorted(references, key=lambda reference: reference.start) -def render_content(content: str, version: str) -> tuple[str, int]: - references = find_references(content) +def render_content( + content: str, version: str, patterns: tuple[re.Pattern[str], ...] = REFERENCE_PATTERNS +) -> tuple[str, int]: + references = find_references(content, patterns) updated = content changed = 0 for reference in reversed(references): @@ -75,24 +139,31 @@ def render_content(content: str, version: str) -> tuple[str, int]: def check_docs(version: str) -> list[str]: mismatches: list[str] = [] + targets = _targets(version) for path in DOC_PATHS: content = path.read_text() - for reference in find_references(content): - if reference.version == version: - continue - line = content.count("\n", 0, reference.start) + 1 - mismatches.append( - f"{path.relative_to(REPO_ROOT)}:{line} references " - f"{reference.version}; expected {version}" - ) + for label, expected, patterns in targets: + for reference in find_references(content, patterns): + if reference.version == expected: + continue + line = content.count("\n", 0, reference.start) + 1 + mismatches.append( + f"{path.relative_to(REPO_ROOT)}:{line} references " + f"{label} {reference.version}; expected {expected}" + ) return mismatches def write_docs(version: str, dry_run: bool) -> int: total = 0 + targets = _targets(version) for path in DOC_PATHS: - content = path.read_text() - updated, changed = render_content(content, version) + original = path.read_text() + updated = original + changed = 0 + for _label, expected, patterns in targets: + updated, count = render_content(updated, expected, patterns) + changed += count total += changed if changed and not dry_run: path.write_text(updated) @@ -131,18 +202,18 @@ def main() -> int: if args.write: changed = write_docs(version, args.dry_run) action = "Would update" if args.dry_run else "Updated" - print(f"{action} {changed} current-release reference(s) to {version}") + print(f"{action} {changed} version reference(s) (release {version} + Dockerfile tool pins)") return 0 mismatches = check_docs(version) if mismatches: - print(f"Current-release documentation is out of sync with {version}:") + print(f"Documentation version references are out of sync (release {version}, Dockerfile tool pins):") for mismatch in mismatches: print(f" - {mismatch}") print("Run: python3 scripts/check_release_docs.py --write") return 1 - print(f"Current-release documentation is in sync: {version}") + print(f"Documentation version references are in sync: release {version}, tool pins {read_tool_pins()}") return 0 diff --git a/socket_basics/core/config.py b/socket_basics/core/config.py index 1dc2d9b..2128956 100644 --- a/socket_basics/core/config.py +++ b/socket_basics/core/config.py @@ -829,6 +829,14 @@ def load_config_from_env() -> Dict[str, Any]: 'scan_all': os.getenv('INPUT_SCAN_ALL', 'false').lower() == 'true', 'scan_files': os.getenv('INPUT_SCAN_FILES', ''), 'changed_files': os.getenv('INPUT_CHANGED_FILES', ''), + + # Output and logging toggles. These mirror the --verbose, + # --console-tabular-enabled and --console-json-enabled CLI flags so the + # GitHub Action inputs of the same name (and INPUT_* entries in a .env + # file) behave identically to the flags. + 'verbose': coerce_bool(os.getenv('INPUT_VERBOSE', ''), False), + 'console_tabular_enabled': coerce_bool(os.getenv('INPUT_CONSOLE_TABULAR_ENABLED', ''), False), + 'console_json_enabled': coerce_bool(os.getenv('INPUT_CONSOLE_JSON_ENABLED', ''), False), # Core Socket API configuration (top-level, like workspace) 'socket_org': ( @@ -953,9 +961,13 @@ def load_config_from_env() -> Dict[str, Any]: return config -def load_socket_basics_config() -> Dict[str, Any] | None: +def load_socket_basics_config(org_slug_override: str | None = None) -> Dict[str, Any] | None: """Load Socket Basics configuration from Socket API if organization has enterprise plan - + + Args: + org_slug_override: Organization slug given on the command line + (``--socket-org``). Takes precedence over the environment. + Returns: Socket Basics configuration dictionary if available, None otherwise """ @@ -987,7 +999,7 @@ def load_socket_basics_config() -> Dict[str, Any] | None: logger.info("Socket API key detected - attempting to load dashboard configuration") # Support both direct env vars and GitHub Actions INPUT_ prefixed vars - org_slug = ( + org_slug = org_slug_override or ( os.environ.get('SOCKET_ORG_SLUG') or os.environ.get('SOCKET_ORG') or os.environ.get('INPUT_SOCKET_ORG') @@ -1130,6 +1142,12 @@ def load_explicit_env_config() -> Dict[str, Any]: config['changed_files'] = os.environ['INPUT_CHANGED_FILES'] if 'INPUT_OPENGREP_RULES_DIR' in os.environ: config['opengrep_rules_dir'] = os.environ['INPUT_OPENGREP_RULES_DIR'] + if 'INPUT_VERBOSE' in os.environ: + config['verbose'] = coerce_bool(os.environ['INPUT_VERBOSE'], False) + if 'INPUT_CONSOLE_TABULAR_ENABLED' in os.environ: + config['console_tabular_enabled'] = coerce_bool(os.environ['INPUT_CONSOLE_TABULAR_ENABLED'], False) + if 'INPUT_CONSOLE_JSON_ENABLED' in os.environ: + config['console_json_enabled'] = coerce_bool(os.environ['INPUT_CONSOLE_JSON_ENABLED'], False) # Dynamically load connector parameters from YAML configuration - only if explicitly set try: @@ -1342,7 +1360,10 @@ def normalize_api_config(api_config: Dict[str, Any]) -> Dict[str, Any]: return normalized -def merge_json_and_env_config(json_config: Dict[str, Any] | None = None) -> Dict[str, Any]: +def merge_json_and_env_config( + json_config: Dict[str, Any] | None = None, + socket_org: str | None = None, +) -> Dict[str, Any]: """Merge JSON configuration with environment variables Priority order (highest to lowest): @@ -1352,6 +1373,9 @@ def merge_json_and_env_config(json_config: Dict[str, Any] | None = None) -> Dict Args: json_config: Optional dictionary from JSON config file + socket_org: Organization slug from ``--socket-org``. Applied before the + dashboard lookup so the right organization's configuration loads, + and re-applied by the caller after the merge so it wins overall. Returns: Merged configuration dictionary @@ -1360,13 +1384,15 @@ def merge_json_and_env_config(json_config: Dict[str, Any] | None = None) -> Dict # Start with environment defaults (lowest priority) config = load_config_from_env() + if socket_org: + config['socket_org'] = socket_org logger.info("Configuration sources: environment defaults loaded") # Override with Socket Basics API config if no explicit JSON config provided # API config takes precedence over environment defaults if not json_config: logger.debug(" No JSON config provided, attempting to load Socket Basics API config") - socket_basics_config = load_socket_basics_config() + socket_basics_config = load_socket_basics_config(org_slug_override=socket_org) logger.debug(f" Socket Basics API config result: {socket_basics_config is not None}") if socket_basics_config: # Normalize camelCase API keys to snake_case internal format @@ -1564,12 +1590,20 @@ def add_dynamic_cli_args(parser: argparse.ArgumentParser): def parse_cli_args(): """Parse command line arguments and return argument parser""" - parser = argparse.ArgumentParser(description='Socket Security Basics - Dynamic security scanning') + parser = argparse.ArgumentParser(prog='socket-basics', description='Socket Security Basics - Dynamic security scanning') + try: + from ..version import __version__ as _socket_basics_version + except Exception: # pragma: no cover - version module is always shipped + _socket_basics_version = 'unknown' + parser.add_argument('--version', action='version', version=f'%(prog)s {_socket_basics_version}') parser.add_argument('--config', type=str, help='Path to JSON configuration file. JSON config is merged with environment variables (JSON takes precedence)') parser.add_argument('--output', type=str, default='.socket.facts.json', help='Output file name (default: .socket.facts.json)') parser.add_argument('--workspace', type=str, help='Workspace directory to scan') + parser.add_argument('--socket-org', type=str, default=None, + help='Socket organization slug (overrides SOCKET_ORG). The API key has no flag; ' + 'set SOCKET_SECURITY_API_KEY in the environment.') parser.add_argument('--repo', type=str, help='Repository name (use when workspace is not a git repo)') parser.add_argument('--branch', type=str, help='Branch name (use when workspace is not a git repo)') parser.add_argument('--default-branch', action='store_true', help='Explicitly mark this as the default branch (sets make_default_branch=true and set_as_pending_head=true)') @@ -1590,19 +1624,23 @@ def parse_cli_args(): def create_config_from_args(args) -> Config: """Create configuration object from parsed CLI arguments""" + cli_socket_org = (getattr(args, 'socket_org', None) or '').strip() or None + # Load base config from environment or JSON file if args.config: try: json_config = load_config_from_json(args.config) - config_dict = merge_json_and_env_config(json_config) + config_dict = merge_json_and_env_config(json_config, socket_org=cli_socket_org) except (FileNotFoundError, json.JSONDecodeError, ValueError) as e: logger = logging.getLogger(__name__) logger.error("Failed to load JSON config from %s: %s", args.config, e) raise SystemExit(f"Error loading configuration file: {e}") else: - config_dict = merge_json_and_env_config() + config_dict = merge_json_and_env_config(socket_org=cli_socket_org) # Override config with CLI args + if cli_socket_org: + config_dict['socket_org'] = cli_socket_org if args.workspace: config_dict['workspace'] = args.workspace # When workspace is explicitly set, default output_dir to workspace unless OUTPUT_DIR env var is set diff --git a/socket_basics/core/notification/ms_sentinel_notifier.py b/socket_basics/core/notification/ms_sentinel_notifier.py index 4f0d508..827343b 100644 --- a/socket_basics/core/notification/ms_sentinel_notifier.py +++ b/socket_basics/core/notification/ms_sentinel_notifier.py @@ -21,12 +21,18 @@ class MSSentinelNotifier(BaseNotifier): def __init__(self, params: Dict[str, Any] | None = None): super().__init__(params or {}) # MS Sentinel configuration from params, env variables, or app config + # `ms_sentinel_workspace_id` / `ms_sentinel_key` are the parameter names + # declared in notifications.yaml, so they are what the --ms-sentinel-* + # CLI flags, the action inputs and dashboard config deliver. self.workspace_id = ( self.config.get('workspace_id') or + self.config.get('ms_sentinel_workspace_id') or get_ms_sentinel_workspace_id() ) self.shared_key = ( self.config.get('shared_key') or + self.config.get('ms_sentinel_key') or + self.config.get('ms_sentinel_shared_key') or get_ms_sentinel_shared_key() ) self.collector_url = ( diff --git a/socket_basics/core/notification/sumologic_notifier.py b/socket_basics/core/notification/sumologic_notifier.py index d1f4c60..9da1895 100644 --- a/socket_basics/core/notification/sumologic_notifier.py +++ b/socket_basics/core/notification/sumologic_notifier.py @@ -2,7 +2,7 @@ import logging from socket_basics.core.notification.base import BaseNotifier -from socket_basics.core.config import get_sumologic_http_source_url +from socket_basics.core.config import get_sumologic_endpoint, get_sumologic_http_source_url logger = logging.getLogger(__name__) @@ -17,10 +17,15 @@ class SumoLogicNotifier(BaseNotifier): def __init__(self, params: Dict[str, Any] | None = None): super().__init__(params or {}) - # SumoLogic HTTP source URL from params, env variable, or app config + # SumoLogic HTTP source URL from params, env variable, or app config. + # `sumologic_endpoint` is the parameter name declared in + # notifications.yaml, so it is what --sumologic-endpoint, the + # `sumologic_endpoint` action input and dashboard config deliver. self.http_source_url = ( self.config.get('http_source_url') or - get_sumologic_http_source_url() + self.config.get('sumologic_endpoint') or + get_sumologic_http_source_url() or + get_sumologic_endpoint() ) def notify(self, facts: Dict[str, Any]) -> None: diff --git a/socket_basics/socket_basics.py b/socket_basics/socket_basics.py index 75c2b0c..4cb034a 100644 --- a/socket_basics/socket_basics.py +++ b/socket_basics/socket_basics.py @@ -432,6 +432,13 @@ def main(): # Create configuration from CLI args config = create_config_from_args(args) + + # `verbose` can also arrive via INPUT_VERBOSE (the GitHub Action `verbose` + # input), a --config JSON file or the Socket dashboard, none of which + # argparse sees. Raise the log level once the merged config exists so those + # sources behave exactly like --verbose. + if not getattr(args, 'verbose', False) and config.get('verbose'): + logging.getLogger().setLevel(logging.DEBUG) # Create scanner and run scanner = SecurityScanner(config) diff --git a/tests/test_action_inputs_consistency.py b/tests/test_action_inputs_consistency.py new file mode 100644 index 0000000..b587246 --- /dev/null +++ b/tests/test_action_inputs_consistency.py @@ -0,0 +1,69 @@ +"""action.yml, connectors.yaml and notifications.yaml must describe one interface. + +Every parameter that the scanners or notifiers read from an ``INPUT_*`` +environment variable has to be produced by the action's ``runs.env`` block, and +every ``inputs.`` the env block references has to be a declared input. +Otherwise a documented action input silently does nothing. +""" + +import re +from pathlib import Path + +import yaml + +ROOT = Path(__file__).resolve().parent.parent + + +def _action() -> dict: + return yaml.safe_load((ROOT / "action.yml").read_text()) + + +def _parameter_env_vars() -> set[str]: + env_vars: set[str] = set() + for filename, top_key in (("connectors.yaml", "connectors"), ("notifications.yaml", "notifiers")): + data = yaml.safe_load((ROOT / "socket_basics" / filename).read_text()) + for cfg in (data.get(top_key) or {}).values(): + for param in cfg.get("parameters") or []: + if param.get("env_variable"): + env_vars.add(param["env_variable"]) + return env_vars + + +def test_every_parameter_env_var_is_set_by_the_action() -> None: + action_env = set(_action()["runs"]["env"]) + missing = _parameter_env_vars() - action_env + # GITHUB_API_URL is a runner-provided default variable; action.yml cannot + # override it (runner env wins), so it is intentionally not mapped. + missing.discard("GITHUB_API_URL") + assert missing == set(), f"parameters with no action env mapping: {sorted(missing)}" + + +def test_every_env_mapping_references_a_declared_input() -> None: + action = _action() + inputs = set(action["inputs"]) + undeclared = [ + (env_key, name) + for env_key, expr in action["runs"]["env"].items() + for name in re.findall(r"inputs\.([A-Za-z0-9_]+)", str(expr)) + if name not in inputs + ] + assert undeclared == [] + + +def test_input_names_used_in_the_docs_are_declared() -> None: + """Names the guides tell users to put under ``with:``.""" + inputs = set(_action()["inputs"]) + documented = { + "verbose", + "console_tabular_enabled", + "console_json_enabled", + "jira_url", + "jira_project", + "ms_sentinel_shared_key", + "opengrep_notification_method", + "trufflehog_notification_method", + "changed_files", + "scan_all", + "scan_files", + } + assert documented <= inputs, f"undeclared: {sorted(documented - inputs)}" diff --git a/tests/test_config_custom_sast.py b/tests/test_config_custom_sast.py index 868c0c2..e3c266b 100644 --- a/tests/test_config_custom_sast.py +++ b/tests/test_config_custom_sast.py @@ -31,7 +31,7 @@ def test_merge_json_and_env_config_api_overrides_env_custom_sast(monkeypatch): monkeypatch.setattr( config_module, "load_socket_basics_config", - lambda: {"useCustomSastRules": False, "customSastRulePath": "dashboard/rules"}, + lambda **_kwargs: {"useCustomSastRules": False, "customSastRulePath": "dashboard/rules"}, ) merged = merge_json_and_env_config() diff --git a/tests/test_env_output_flags.py b/tests/test_env_output_flags.py new file mode 100644 index 0000000..98570f5 --- /dev/null +++ b/tests/test_env_output_flags.py @@ -0,0 +1,63 @@ +"""INPUT_VERBOSE / INPUT_CONSOLE_*_ENABLED and --version. + +The GitHub Action inputs ``verbose``, ``console_tabular_enabled`` and +``console_json_enabled`` arrive as ``INPUT_*`` environment variables and must +behave like the matching CLI flags. +""" + +import pytest + +from socket_basics.core.config import load_config_from_env, load_explicit_env_config, parse_cli_args +from socket_basics.version import __version__ + +OUTPUT_ENV_VARS = ("INPUT_VERBOSE", "INPUT_CONSOLE_TABULAR_ENABLED", "INPUT_CONSOLE_JSON_ENABLED") + + +@pytest.fixture +def clean_env(monkeypatch): + for name in OUTPUT_ENV_VARS: + monkeypatch.delenv(name, raising=False) + return monkeypatch + + +def test_output_flags_default_off(clean_env): + config = load_config_from_env() + assert config["verbose"] is False + assert config["console_tabular_enabled"] is False + assert config["console_json_enabled"] is False + + explicit = load_explicit_env_config() + assert not any(key in explicit for key in ("verbose", "console_tabular_enabled", "console_json_enabled")) + + +def test_output_flags_read_from_input_env(clean_env): + clean_env.setenv("INPUT_VERBOSE", "true") + clean_env.setenv("INPUT_CONSOLE_TABULAR_ENABLED", "True") + clean_env.setenv("INPUT_CONSOLE_JSON_ENABLED", "false") + + config = load_config_from_env() + assert config["verbose"] is True + assert config["console_tabular_enabled"] is True + assert config["console_json_enabled"] is False + + explicit = load_explicit_env_config() + assert explicit["verbose"] is True + assert explicit["console_tabular_enabled"] is True + assert explicit["console_json_enabled"] is False + + +def test_empty_action_input_does_not_turn_flags_on(clean_env): + """An unset workflow input is forwarded as an empty string, not as off.""" + for name in OUTPUT_ENV_VARS: + clean_env.setenv(name, "") + config = load_config_from_env() + assert config["verbose"] is False + assert config["console_tabular_enabled"] is False + assert config["console_json_enabled"] is False + + +def test_version_flag_prints_version_and_exits(capsys): + with pytest.raises(SystemExit) as exc: + parse_cli_args().parse_args(["--version"]) + assert exc.value.code == 0 + assert capsys.readouterr().out.strip() == f"socket-basics {__version__}" diff --git a/tests/test_ms_sentinel_notifier_params.py b/tests/test_ms_sentinel_notifier_params.py new file mode 100644 index 0000000..1fce210 --- /dev/null +++ b/tests/test_ms_sentinel_notifier_params.py @@ -0,0 +1,79 @@ +import pytest + +from socket_basics.core.notification.manager import NotificationManager +from socket_basics.core.notification.ms_sentinel_notifier import MSSentinelNotifier + +SENTINEL_ENV_VARS = ( + "MS_SENTINEL_WORKSPACE_ID", + "INPUT_MS_SENTINEL_WORKSPACE_ID", + "MS_SENTINEL_SHARED_KEY", + "INPUT_MS_SENTINEL_SHARED_KEY", + "INPUT_MS_SENTINEL_KEY", +) + + +@pytest.fixture +def clean_env(monkeypatch): + for name in SENTINEL_ENV_VARS: + monkeypatch.delenv(name, raising=False) + return monkeypatch + + +def _base_cfg(): + return { + "notifiers": { + "ms_sentinel": { + "module_path": "socket_basics.core.notification.ms_sentinel_notifier", + "class": "MSSentinelNotifier", + "parameters": [ + {"name": "ms_sentinel_workspace_id", "env_variable": "INPUT_MS_SENTINEL_WORKSPACE_ID", "type": "str"}, + {"name": "ms_sentinel_key", "env_variable": "INPUT_MS_SENTINEL_KEY", "type": "str"}, + ], + } + } + } + + +def test_sentinel_reads_notifications_yaml_param_names(clean_env): + """--ms-sentinel-workspace-id / --ms-sentinel-key and the action inputs deliver these names.""" + n = MSSentinelNotifier({"ms_sentinel_workspace_id": "ws-123", "ms_sentinel_key": "shared"}) + assert n.workspace_id == "ws-123" + assert n.shared_key == "shared" + + +def test_sentinel_accepts_shared_key_alias(clean_env): + n = MSSentinelNotifier({"ms_sentinel_workspace_id": "ws-123", "ms_sentinel_shared_key": "shared"}) + assert n.shared_key == "shared" + + +def test_sentinel_falls_back_to_env(clean_env): + clean_env.setenv("MS_SENTINEL_WORKSPACE_ID", "ws-env") + clean_env.setenv("INPUT_MS_SENTINEL_SHARED_KEY", "key-env") + n = MSSentinelNotifier({}) + assert n.workspace_id == "ws-env" + assert n.shared_key == "key-env" + + +def test_sentinel_enabled_via_action_env_vars(clean_env): + """The action maps ms_sentinel_workspace_id / ms_sentinel_key to these INPUT_ vars.""" + clean_env.setenv("INPUT_MS_SENTINEL_WORKSPACE_ID", "ws-action") + clean_env.setenv("INPUT_MS_SENTINEL_KEY", "key-action") + + nm = NotificationManager(_base_cfg(), app_config={}) + nm.load_from_config() + + sentinel = next(n for n in nm.notifiers if getattr(n, "name", "") == "ms_sentinel") + assert sentinel.workspace_id == "ws-action" + assert sentinel.shared_key == "key-action" + + +def test_sentinel_enabled_via_app_config(clean_env): + nm = NotificationManager( + _base_cfg(), + app_config={"ms_sentinel_workspace_id": "ws-cli", "ms_sentinel_key": "key-cli"}, + ) + nm.load_from_config() + + sentinel = next(n for n in nm.notifiers if getattr(n, "name", "") == "ms_sentinel") + assert sentinel.workspace_id == "ws-cli" + assert sentinel.shared_key == "key-cli" diff --git a/tests/test_release_docs.py b/tests/test_release_docs.py index d9985ee..6053075 100644 --- a/tests/test_release_docs.py +++ b/tests/test_release_docs.py @@ -41,6 +41,46 @@ def test_render_content_updates_only_socket_basics_release_references() -> None: assert "TRIVY_VERSION=0.73.0" in rendered +def test_floating_major_tag_is_treated_as_stale() -> None: + """The action publishes no `@v2`-style tag, so such references must be rewritten.""" + content = "- uses: SocketDev/socket-basics@v2\n # prose about `@v2` floating tags stays\n" + + rendered, changed = check_release_docs.render_content(content, "3.1.0") + + assert changed == 1 + assert "uses: SocketDev/socket-basics@v3.1.0" in rendered + assert "prose about `@v2` floating tags stays" in rendered + # an exact release is matched once, not also as a floating major + assert len(check_release_docs.find_references("SocketDev/socket-basics@v3.1.0")) == 1 + + +def test_tool_pins_come_from_dockerfile_and_docs_match() -> None: + pins = check_release_docs.read_tool_pins() + + assert set(pins) == {"trufflehog", "opengrep", "trivy"} + assert all(not v.startswith("v") for v in pins.values()) + assert check_release_docs.check_docs(check_release_docs.read_canonical_version()) == [] + + +def test_render_tool_references_only_touches_that_tool() -> None: + content = """\ +docker pull trufflesecurity/trufflehog:3.93.8 +wget .../trufflehog/releases/download/v3.93.8/trufflehog_3.93.8_linux_amd64.tar.gz +"com.socket.opengrep-version": "v1.16.5" +uses: SocketDev/socket-basics@v3.1.0 +""" + + rendered, changed = check_release_docs.render_content( + content, "3.96.0", check_release_docs.TOOL_REFERENCE_PATTERNS["trufflehog"] + ) + + assert changed == 3 + assert "trufflesecurity/trufflehog:3.96.0" in rendered + assert "download/v3.96.0/trufflehog_3.96.0_linux_amd64" in rendered + assert '"com.socket.opengrep-version": "v1.16.5"' in rendered + assert "SocketDev/socket-basics@v3.1.0" in rendered + + def test_canonical_version_matches_pyproject() -> None: pyproject = tomllib.loads(check_release_docs.PYPROJECT_PATH.read_text()) diff --git a/tests/test_socket_org_flag.py b/tests/test_socket_org_flag.py new file mode 100644 index 0000000..e65eeed --- /dev/null +++ b/tests/test_socket_org_flag.py @@ -0,0 +1,50 @@ +"""--socket-org: the CLI equivalent of the socket_org action input / SOCKET_ORG.""" + +import pytest + +from socket_basics.core.config import merge_json_and_env_config, parse_cli_args + +ORG_ENV_VARS = ("SOCKET_ORG", "SOCKET_ORG_SLUG", "INPUT_SOCKET_ORG") +KEY_ENV_VARS = ( + "SOCKET_SECURITY_API_KEY", + "SOCKET_SECURITY_API_TOKEN", + "SOCKET_API_KEY", + "INPUT_SOCKET_SECURITY_API_KEY", + "INPUT_SOCKET_API_KEY", +) + + +@pytest.fixture +def clean_env(monkeypatch): + # No API key, so the dashboard lookup stays offline (free-plan path). + for name in ORG_ENV_VARS + KEY_ENV_VARS: + monkeypatch.delenv(name, raising=False) + return monkeypatch + + +def test_socket_org_flag_parses(): + args = parse_cli_args().parse_args(["--socket-org", "acme"]) + assert args.socket_org == "acme" + + +def test_socket_org_flag_absent_is_none(): + assert parse_cli_args().parse_args([]).socket_org is None + + +def test_cli_org_overrides_environment(clean_env): + clean_env.setenv("SOCKET_ORG", "from-env") + config = merge_json_and_env_config(socket_org="from-cli") + assert config["socket_org"] == "from-cli" + + +def test_environment_org_used_without_flag(clean_env): + clean_env.setenv("SOCKET_ORG", "from-env") + assert merge_json_and_env_config()["socket_org"] == "from-env" + + +def test_cli_org_overrides_json_config(clean_env): + config = merge_json_and_env_config({"socket_org": "from-json"}, socket_org="from-cli") + # merge applies JSON over env; the CLI layer re-applies the flag afterwards, + # so here we only assert the flag reached the pre-merge config and JSON won + # the merge step as documented. + assert config["socket_org"] == "from-json" diff --git a/tests/test_sumologic_notifier_params.py b/tests/test_sumologic_notifier_params.py new file mode 100644 index 0000000..3d09db5 --- /dev/null +++ b/tests/test_sumologic_notifier_params.py @@ -0,0 +1,65 @@ +import pytest + +from socket_basics.core.notification.manager import NotificationManager +from socket_basics.core.notification.sumologic_notifier import SumoLogicNotifier + +SUMO_ENV_VARS = ( + "SUMOLOGIC_ENDPOINT", + "INPUT_SUMOLOGIC_ENDPOINT", + "SUMO_LOGIC_HTTP_SOURCE_URL", + "INPUT_SUMO_LOGIC_HTTP_SOURCE_URL", +) + + +@pytest.fixture +def clean_env(monkeypatch): + for name in SUMO_ENV_VARS: + monkeypatch.delenv(name, raising=False) + return monkeypatch + + +def _base_cfg(): + return { + "notifiers": { + "sumologic": { + "module_path": "socket_basics.core.notification.sumologic_notifier", + "class": "SumoLogicNotifier", + "parameters": [ + {"name": "sumologic_endpoint", "env_variable": "INPUT_SUMOLOGIC_ENDPOINT", "type": "str"}, + ], + } + } + } + + +def test_sumologic_reads_notifications_yaml_param_name(clean_env): + """--sumologic-endpoint and the sumologic_endpoint action input deliver this name.""" + n = SumoLogicNotifier({"sumologic_endpoint": "https://endpoint.sumologic.com/receiver/v1/http/abc"}) + assert n.http_source_url == "https://endpoint.sumologic.com/receiver/v1/http/abc" + + +def test_sumologic_falls_back_to_either_env_name(clean_env): + clean_env.setenv("SUMOLOGIC_ENDPOINT", "https://endpoint.sumologic.com/env") + assert SumoLogicNotifier({}).http_source_url == "https://endpoint.sumologic.com/env" + + clean_env.delenv("SUMOLOGIC_ENDPOINT") + clean_env.setenv("SUMO_LOGIC_HTTP_SOURCE_URL", "https://endpoint.sumologic.com/legacy") + assert SumoLogicNotifier({}).http_source_url == "https://endpoint.sumologic.com/legacy" + + +def test_sumologic_enabled_via_action_env_var(clean_env): + clean_env.setenv("INPUT_SUMOLOGIC_ENDPOINT", "https://endpoint.sumologic.com/action") + + nm = NotificationManager(_base_cfg(), app_config={}) + nm.load_from_config() + + sumo = next(n for n in nm.notifiers if getattr(n, "name", "") == "sumologic") + assert sumo.http_source_url == "https://endpoint.sumologic.com/action" + + +def test_sumologic_enabled_via_app_config(clean_env): + nm = NotificationManager(_base_cfg(), app_config={"sumologic_endpoint": "https://endpoint.sumologic.com/cli"}) + nm.load_from_config() + + sumo = next(n for n in nm.notifiers if getattr(n, "name", "") == "sumologic") + assert sumo.http_source_url == "https://endpoint.sumologic.com/cli"