Skip to content

work-items: two conformance assertions are vacuous — "capabilities names a provider" can never fail, including for an empty provider #3485

Description

@kyle-sexton

Found by the repo-wide /code-tidying:batch-simplify sweep (group G66). Reported, not fixed — the sweep may not weaken or drop a conformance case, and both strengthening and removing these changes behavior.

What happens

Two conformance assertions build their haystack by prefixing the literal needle, so the containment check passes for every possible value of the variable under test — including empty.

plugins/work-items/tools/work-item-tracker/conformance/run-conformance.sh:124

assert_contains "capabilities names a provider" "provider=$PROVIDER" "provider="

The haystack is provider= followed by $PROVIDER; the needle is provider=. It matches unconditionally.

plugins/work-items/tools/work-item-tracker/conformance/e2e-probe.sh:117

assert_contains "lease comment id numeric" "n$LEASE_CID" "n"

Same shape: the haystack is n followed by $LEASE_CID, the needle is n.

Why it matters

run-conformance.sh:124 is the more serious of the two. It is the only check that the capabilities manifest names a provider at all, and it is blind — an adapter whose manifest emitted an empty provider would pass this conformance case. The suite reports coverage it does not have, and the %d cases summary counts a case that cannot fail.

e2e-probe.sh:117 is backstopped: the real numeric check is the [[ "$LEASE_CID" =~ ^[0-9]+$ ]] block immediately below it at lines 118-122. So that one is a redundant always-pass rather than a coverage hole, though it still inflates the case count.

Suggested direction

For run-conformance.sh:124, the assertion presumably meant to check that $PROVIDER is non-empty and appears in the capabilities output — assert against the capabilities output itself rather than against a string the test just built. For e2e-probe.sh:117, decide whether it should become a real assertion or be folded into the regex check below it that already does the work.

Either change alters the suite's case numbering and its %d cases summary line, which is why this is a report rather than a sweep edit.

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions