Skip to content

hardcoded-credential: test code is judged on the absolute path, so a project under a tests directory has every credential at low confidence #253

Closed

Description

@SizzleUnrlsd

Problem

hardcoded-credential lowers the confidence of a credential found in test code, and records context: test. It decides that with credential_context(str(span.source_id)) (plugins/secrets.py). The source id is the file's absolute path, so every directory above the project counts. A project that sits under a directory named test, tests, testing, fixtures or __tests__ (a CI workspace, a monorepo, a checkout under ~/tests/) has every credential reported at low confidence, as test code.

Reproduction

Checked on main (501030a), bundled plugins only. The same file in two projects that differ only by where they sit:

tests/project/app/settings.py
work/project/app/settings.py
PASSWORD = "Zx81kQpLw0RtY7vB"
Project root Confidence metadata.context
…/tests/project low test
…/work/project medium absent

Cause

A module plugin sees only the absolute source id: PluginContext does not expose the project root, so the rule cannot compute the file's path in the project.

The engine-level context of #244 (PR #251) is judged on the path relative to the root. It keeps a context a rule already gave, so the wrong test stays visible there too.

Expected

Test code is judged on the path relative to the project root, for this rule as for the engine-level context. In the reproduction above, both projects report the credential at medium confidence, without context. A credential in tests/test_settings.py or a .env.example keeps its current behaviour.

Directions to discuss

  • Give module plugins the file's path relative to the project root, for instance through PluginContext, and have the secret detectors classify that path with is_test_code (findings: test code is recognised only by hardcoded-credential #244).
  • Or let the rule leave the test context to the engine and lower the confidence from it, keeping example its own.

The first keeps the decision in the rule. The second removes the duplicate classification but moves a verdict decision out of the rule. Either changes verdicts for projects located under such a directory: that is the fix.

Acceptance criteria

  • A test with the project root under a tests directory: the credential at medium confidence, no context.
  • Unchanged behaviour for credentials in test files and template files of the project, pinned by tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions