Description
Problem
hardcoded-credential lowers the confidence of a credential found in test code, and records context: test. It decides that with credential_context(str(span.source_id)) (plugins/secrets.py). The source id is the file's absolute path, so every directory above the project counts. A project that sits under a directory named test, tests, testing, fixtures or __tests__ (a CI workspace, a monorepo, a checkout under ~/tests/) has every credential reported at low confidence, as test code.
Reproduction
Checked on main (501030a), bundled plugins only. The same file in two projects that differ only by where they sit:
tests/project/app/settings.py
work/project/app/settings.py
PASSWORD = "Zx81kQpLw0RtY7vB"| Project root | Confidence | metadata.context |
|---|---|---|
…/tests/project |
low | test |
…/work/project |
medium | absent |
Cause
A module plugin sees only the absolute source id: PluginContext does not expose the project root, so the rule cannot compute the file's path in the project.
The engine-level context of #244 (PR #251) is judged on the path relative to the root. It keeps a context a rule already gave, so the wrong test stays visible there too.
Expected
Test code is judged on the path relative to the project root, for this rule as for the engine-level context. In the reproduction above, both projects report the credential at medium confidence, without context. A credential in tests/test_settings.py or a .env.example keeps its current behaviour.
Directions to discuss
- Give module plugins the file's path relative to the project root, for instance through
PluginContext, and have the secret detectors classify that path withis_test_code(findings: test code is recognised only by hardcoded-credential #244). - Or let the rule leave the test context to the engine and lower the confidence from it, keeping
exampleits own.
The first keeps the decision in the rule. The second removes the duplicate classification but moves a verdict decision out of the rule. Either changes verdicts for projects located under such a directory: that is the fix.
Acceptance criteria
- A test with the project root under a
testsdirectory: the credential at medium confidence, no context. - Unchanged behaviour for credentials in test files and template files of the project, pinned by tests.
Activity