feat(findings): mark vendored code with its origin, apart from its context

docs/usage.md

@@ -120,7 +120,7 @@ variable is not treated that way yet.
 
 | Rule | Reached sink |
 |---|---|
-| `command-injection` | Shell or process execution (`os.system`, `subprocess` with a string, …), judged on what the input controls in the command. A string a shell runs, a string naming the program, the program itself, or an operand the program runs as a command (after the host for `ssh`, the program of `sudo`, `env`, `xargs`, `timeout`…, any argument of a batch file) is `high`. The script an interpreter runs (`bash run.sh`, `python tool.py`, its first operand when no option such as `-c` gives the code), or the module of `python -m`, is `high` and kept for review: execution is possible, and its exploitability depends on the file the input chooses. The arguments after a script or code are the script's own: options of unknown meaning, reported `medium` (a hotspot after an option of the script), refuted behind a constant prefix. Otherwise an element of an argument list run without a shell can only be read as an option: `medium`, or `high` when a documented option of the program runs a command it chooses (`git push --receive-pack=…`, `ssh -o ProxyCommand=…`). A value a documented option takes is judged by what the option does with it, whatever its prefix: run it (`sh -c`, `cmd /c`, `python -c`: `high`), keep it as data (`git commit -m`: refuted). After an option the rule does not know, the finding stays a hotspot to review. A constant prefix that keeps a standalone input from being an option (`f"v{version}"`), or an operand after `--` for a program that honours it, is refuted. Every case keeps the rule id `command-injection`, so suppressions and baselines match whatever the case. When the rule reads the command element by element and establishes what the input is there, the finding's `injection` metadata (in the JSON report, and as the only property of its SARIF result) says so: `command` when the input runs or chooses what runs (a shell string, the program, an operand run as a command, the value of an option that runs it); `option` when it can only reach the program as an option or as the value of an option not established to run it, and the message is then titled `Option injection`. When that cannot be established (an option whose effect is to review, an unknown program or element before the input, a command that is not a list or tuple display, a sink such as `os.system` the rule does not read element by element), the metadata is absent and the title stays `Command injection`. |
+| `command-injection` | Shell or process execution (`os.system`, `subprocess` with a string, …), judged on what the input controls in the command. A string a shell runs, a string naming the program, the program itself, or an operand the program runs as a command (after the host for `ssh`, the program of `sudo`, `env`, `xargs`, `timeout`…, any argument of a batch file) is `high`. The script an interpreter runs (`bash run.sh`, `python tool.py`, its first operand when no option such as `-c` gives the code), or the module of `python -m`, is `high` and kept for review: execution is possible, and its exploitability depends on the file the input chooses. The arguments after a script or code are the script's own: options of unknown meaning, reported `medium` (a hotspot after an option of the script), refuted behind a constant prefix. Otherwise an element of an argument list run without a shell can only be read as an option: `medium`, or `high` when a documented option of the program runs a command it chooses (`git push --receive-pack=…`, `ssh -o ProxyCommand=…`). A value a documented option takes is judged by what the option does with it, whatever its prefix: run it (`sh -c`, `cmd /c`, `python -c`: `high`), keep it as data (`git commit -m`: refuted). After an option the rule does not know, the finding stays a hotspot to review. A constant prefix that keeps a standalone input from being an option (`f"v{version}"`), or an operand after `--` for a program that honours it, is refuted. Every case keeps the rule id `command-injection`, so suppressions and baselines match whatever the case. When the rule reads the command element by element and establishes what the input is there, the finding's `injection` metadata (in the JSON report, and as a property of its SARIF result) says so: `command` when the input runs or chooses what runs (a shell string, the program, an operand run as a command, the value of an option that runs it); `option` when it can only reach the program as an option or as the value of an option not established to run it, and the message is then titled `Option injection`. When that cannot be established (an option whose effect is to review, an unknown program or element before the input, a command that is not a list or tuple display, a sink such as `os.system` the rule does not read element by element), the metadata is absent and the title stays `Command injection`. |
 | `code-injection` | The code `eval` or `exec` runs; the globals and locals passed with it are data. `dangerous-eval` reports every such call, this rule the ones attacker input reaches. Also the module `importlib.import_module`, `__import__` or `runpy.run_module` imports, and the file `runpy.run_path` runs, by position or by keyword (`name`, `mod_name`, `path_name`); the package of a relative name, the globals and the run name are data. An import runs the top-level code of a module the input chooses among those that can be found, a script path the file it chooses: execution is possible, and how far it goes depends on what the input can reach, so these findings are titled `Untrusted module import` and `Untrusted script execution`. A constant package prefix (`"app.plugins." + name`) confines an import to that package: a hotspot. The environment and the command line reach these sinks as they reach `eval`. |
 | `sql-injection` | A database statement (`cursor.execute`, SQLAlchemy `text`, Django `raw`, …). Parameters of a parameterised query are not statements. |
 | `nosql-injection` | A NoSQL query, such as a MongoDB filter, given attacker input that may hold a structure: a JSON body giving `{"$ne": null}` matches any value. The input that may hold a structure is JSON, the raw bodies a `json.loads` decodes, files, WebSocket messages and standard input. Text cannot hold an operator: form fields, query parameters, headers, cookies, URL parameters, the command line, the environment, and an entry-point parameter annotated with scalars (`str`, `int`, `Optional[str]`, `Annotated[str, Query()]`, `list[str]`, …), which FastAPI validates. A Django, REST framework, aiohttp or tornado view reads a request object: its text attributes (`GET`, `POST`, `headers`, `query_params`, `query`, `match_info`, `arguments`, …) and the URL parameters after it are text, its body, files and `data` are not. Limits: a helper function the request is passed to reads it as a whole; the keys of a dict are not followed, so a string concatenated into `$where` is not reported. The analyzer ships no NoSQL sink: model plugins declare them. |
@@ -356,6 +356,25 @@ in the file (`"/tool/app/api_token"`). The SARIF log gives such a result no `reg
 with the pointer under the location's `properties`, and the text report shows the file
 alone (`settings.toml: high hardcoded-credential: …`).
 
+A finding in test code carries `context: test` in its metadata, in the JSON report and as
+a property of its SARIF result. Test code is a file under a `test`, `tests`, `testing`,
+`fixtures` or `__tests__` directory of the project, or a file named `test_*.py`,
+`*_test.py` or `conftest.py`, judged on its path from the root of the report, not on
+where the project sits. For a file checked alone, that root is the file's own directory,
+as the report gives it, so a `test_*.py` module is test code either way, while a test
+directory above the file is seen only when its directory is checked. The context
+changes no verdict: it is there for baselines, `--fail-on`
+wrappers, readers and refiner plugins, which see it. A rule may still give its own
+context, as `hardcoded-credential` does with `example` for template files, and the
+SARIF result carries that context too.
+
+A finding in vendored code, under a `third_party`, `third-party`, `vendor` or `vendored`
+directory, carries `origin: vendored`, in the JSON report and as a property of its SARIF
+result. The origin is told apart from the context: the tests of a vendored project
+carry both `context: test` and `origin: vendored`. Vendored code is still analysed and
+its findings reported, so that a flow from the project through a vendored module is
+not lost; the origin changes no verdict either.
+
 `root` is the directory the paths are relative to. A file's `status` is `analysed`,
 `ambiguous` (analysed, but another file has its module name, see above), `syntax-error`
 or `unreadable`; only `analysed` files count in `files_analysed`.

src/coretrace_python/cli.py

@@ -303,7 +303,7 @@ def main(argv: list[str] | None = None) -> int:
                         encoding="utf-8",
                     )
             else:
-                file_analysis = engine.analyze_file(SourceManager().load_file(args.path), plugin_roots)
+                file_analysis = engine.analyze_file(SourceManager().load_file(args.path), plugin_roots, root)
                 findings, coverage = file_analysis.findings, file_analysis.coverage
                 suppressed = file_analysis.suppressed
                 components = ()

src/coretrace_python/engine.py

@@ -72,6 +72,7 @@
     Finding,
     Severity,
 )
+from coretrace_python.findings.context import code_context, code_origin
 from coretrace_python.findings.refutation import RefutationAnalysis
 from coretrace_python.findings.suppressions import partition
 from coretrace_python.frontend import HIRBuildError, ParseError, build_hir
@@ -326,8 +327,11 @@ def check(source: SourceFile, plugin_roots: Sequence[Path]) -> tuple[Finding, ..
     return analyze_file(source, plugin_roots).findings
 
 
-def analyze_file(source: SourceFile, plugin_roots: Sequence[Path]) -> FileAnalysis:
-    """One file's findings and coverage, with every plugin found under ``plugin_roots``."""
+def analyze_file(source: SourceFile, plugin_roots: Sequence[Path], root: Path | None = None) -> FileAnalysis:
+    """One file's findings and coverage, with every plugin found under ``plugin_roots``.
+    The context of its findings is judged on its path from ``root``, the root of its
+    report: by default its own directory, as the command line reports a file checked
+    alone. A source that is not a file is judged on its name as given."""
 
     manager = _register_all(build_hir(source))
     registry = load_plugins(plugin_roots, manager)
@@ -339,6 +343,11 @@ def analyze_file(source: SourceFile, plugin_roots: Sequence[Path]) -> FileAnalys
     manager.provide(ProjectSummaries, SummaryIndex())
     manager.provide(RegisteredRoutes, _routes_of(manager))
     findings, supported = _check_module(manager, tuple(loaded.plugin for loaded in registry))
+    if source.path is None:
+        path: str | None = Path(str(source.source_id)).as_posix()
+    else:
+        path = _path_from(source.path, source.path.resolve().parent if root is None else root)
+    findings = tuple(_situated(f, path if f.span.source_id == source.source_id else None) for f in findings)
     functions = len(analyzable_functions(manager.module))
     coverage = Coverage((FileCoverage(str(source.source_id), "analysed", functions, len(supported)),))
     kept, suppressed = partition(findings, lambda sid: source.text if sid == source.source_id else None)
@@ -603,6 +612,8 @@ def analyze_project(
     for plugin in all_plugins:
         if isinstance(plugin, ProjectPlugin):
             findings.extend(plugin.analyze_project(context))
+    # The context comes first, so that a refiner can read it.
+    findings = [_situated(finding, _path_from(finding.span.source_id, root)) for finding in findings]
     for plugin in all_plugins:
         if isinstance(plugin, ProjectPlugin):
             findings = list(apply_refinement(plugin, findings, plugin.refine(context, tuple(findings))))
@@ -687,6 +698,34 @@ def _located(path: Path, root: Path) -> str:
         return str(path)
 
 
+def _path_from(source_id: object, root: Path) -> str | None:
+    """The path of the file ``source_id`` names from ``root``, the root of the report;
+    ``None`` for a file outside it."""
+
+    try:
+        return Path(str(source_id)).resolve().relative_to(root.resolve()).as_posix()
+    except ValueError:
+        return None
+
+
+def _situated(finding: Finding, path: str | None) -> Finding:
+    """``finding`` with the context (``code_context``) and the origin (``code_origin``) of
+    its code in its metadata, judged on ``path``, its file's path from the root of the
+    report. A context the rule gave the finding stays: it is the rule's (#244). The origin
+    is told apart from the context, so vendored test code is both (#243)."""
+
+    if path is None:
+        return finding
+    added: dict[str, str] = {}
+    context = code_context(path)
+    if context is not None and "context" not in finding.metadata:
+        added["context"] = context
+    origin = code_origin(path)
+    if origin is not None:
+        added["origin"] = origin
+    return replace(finding, metadata={**finding.metadata, **added}) if added else finding
+
+
 def _sourced(finding: Finding, origins: Origins) -> Finding:
     """``finding`` with the plugin or advisory file its advisory comes from, when it
     cites one."""

src/coretrace_python/findings/context.py

@@ -0,0 +1,44 @@
+"""Where the code of a finding lives, by its path in the project, as two separate facts:
+its context, test code a project runs to check itself and does not ship (#244), and its
+origin, vendored code, someone else's kept in the project (#243). A vendored project's
+tests are both. Both are shown in the finding's metadata; what they change, if
+anything, is the rule's decision, or a refiner plugin's."""
+
+from __future__ import annotations
+
+import re
+from pathlib import PurePosixPath
+
+TEST = "test"
+VENDORED = "vendored"
+
+_VENDORED_DIRECTORIES = frozenset({"third_party", "third-party", "vendor", "vendored"})
+
+_TEST_DIRECTORIES = frozenset({"test", "tests", "testing", "fixtures", "__tests__"})
+_TEST_FILE = re.compile(r"^(test_.*|.*_test|conftest)\.py$")
+
+
+def is_test_code(path: str) -> bool:
+    """Whether ``path`` is test code or a fixture: under a test directory, or named like
+    a test module."""
+
+    parts = PurePosixPath(path).parts
+    return bool(parts) and (
+        any(part in _TEST_DIRECTORIES for part in parts[:-1]) or _TEST_FILE.match(parts[-1]) is not None
+    )
+
+
+def code_context(path: str) -> str | None:
+    """The context a finding at ``path`` carries: ``"test"`` for test code, ``None`` for
+    anything else. ``path`` is relative to the project root, so where the project sits
+    does not count."""
+
+    return TEST if is_test_code(path) else None
+
+
+def code_origin(path: str) -> str | None:
+    """The origin a finding at ``path`` carries: ``"vendored"`` under a ``third_party``,
+    ``third-party``, ``vendor`` or ``vendored`` directory of the project, ``None`` for the
+    project's own code."""
+
+    return VENDORED if any(part in _VENDORED_DIRECTORIES for part in PurePosixPath(path).parts[:-1]) else None

src/coretrace_python/plugins/secrets.py

@@ -29,6 +29,7 @@
 from typing import ClassVar
 
 from coretrace_python.findings import Confidence, Finding, Severity
+from coretrace_python.findings.context import is_test_code
 from coretrace_python.hir import nodes
 from coretrace_python.hir.visitors import Node, children
 from coretrace_python.interprocedural import discover_files
@@ -100,19 +101,16 @@ def looks_like_digest(value: str, name: str | None) -> bool:
     return name is not None and any(word in name.lower() for word in _DIGEST_WORDS)
 
 
-_TEST_DIRECTORIES = frozenset({"test", "tests", "testing", "fixtures", "__tests__"})
-_TEST_FILE = re.compile(r"^(test_.*|.*_test|conftest)\.py$")
 _EXAMPLE_SUFFIXES = (".example", ".sample", ".dist", ".tpl")
 
 
 def credential_context(path: str) -> str | None:
-    """``"test"`` for test code and fixtures, ``"example"`` for template configuration
-    files (``.env.example``), ``None`` for anything else."""
+    """``"test"`` for test code and fixtures (``is_test_code``), ``"example"`` for
+    template configuration files (``.env.example``), ``None`` for anything else."""
 
-    parts = Path(path).parts
-    if any(part in _TEST_DIRECTORIES for part in parts[:-1]) or _TEST_FILE.match(parts[-1]):
+    if is_test_code(Path(path).as_posix()):
         return "test"
-    if parts[-1].endswith(_EXAMPLE_SUFFIXES):
+    if Path(path).name.endswith(_EXAMPLE_SUFFIXES):
         return "example"
     return None
 

src/coretrace_python/reporters/sarif.py

@@ -19,9 +19,12 @@
     Severity.LOW: "note",
     Severity.INFO: "note",
 }
-# The kinds of injection a finding may carry, the only metadata a result exports: other
-# metadata, such as evidence, may quote what the analysed code holds.
+# The kinds of injection, the contexts and the origins of code a finding may carry, the
+# only metadata a result exports: other metadata, such as evidence, may quote what the
+# analysed code holds.
 _INJECTIONS = frozenset({"option", "command"})
+_CONTEXTS = frozenset({"test", "example"})
+_ORIGINS = frozenset({"vendored"})
 
 
 def _artifact(report: Report, path: str) -> dict[str, str]:
@@ -56,9 +59,18 @@ def _result(
         "message": {"text": finding.message},
         "locations": [location],
     }
+    properties: dict[str, str] = {}
     injection = finding.metadata.get("injection")
     if injection in _INJECTIONS:
-        result["properties"] = {"injection": injection}
+        properties["injection"] = injection
+    context = finding.metadata.get("context")
+    if context in _CONTEXTS:
+        properties["context"] = context
+    origin = finding.metadata.get("origin")
+    if origin in _ORIGINS:
+        properties["origin"] = origin
+    if properties:
+        result["properties"] = properties
     if suppressed:
         result["suppressions"] = [{"kind": "inSource"}]
     if baseline_state is not None: