Fix pytest parsing of parametrized node IDs containing spaces

[abhinavgautam01]

Summary

Pytest node IDs containing spaces were dropped from verbose output or truncated in short summaries. This could omit tests from F2P/P2P discovery and incorrectly penalize passing tests during grading.

Update both the generation-time parser and standalone runtime verifier to preserve complete parameter IDs while separating pytest statuses, progress indicators and diagnostic messages. Preserve folded skip handling and last-write-wins behavior.

Add 72 regression cases covering spaces, repeated whitespace, embedded dashes and status words, brackets, skip reasons and distinct parameter identities. Integration tests exercise actual pytest output, F2P/P2P discovery and standalone verifier grading.

Fixes #90.

Validation

• Python 3.12, 3.13, and 3.14: 788 passed, 5 skipped on each.
• ruff check . and ruff format --check . passed.
• Source distribution and wheel builds passed.
• Clean wheel installation and CLI version check passed.

[KNambiarDJsc]

Thanks for this. It's the more complete of the two fixes for #90. I checked it against real pytest 9.0.3 output rather than hand-written lines. I recorded each test's outcome through a pytest_runtest_logreport hook as ground truth, then parsed the terminal log. The IDs covered spaces, -, status words, and unmatched [ / ].

With -v, -v -rA, -vv -rA and console_output_style=classic, both parsers get 17/17. The canonical parser and the standalone verifier always agree. The test that runs the shipped verifier.py in a subprocess is a great addition.

Two things I'd fix before merge:

1. Regression: console_output_style = count / times. The verbose regex is $-anchored and only allows a [ NN%] suffix. With the count and times styles, pytest ends the line differently, so every verbose line stops matching. main still parses these for ordinary names:

test_cases.py::test_pass[plain] PASSED        [ 8/17]
test_cases.py::test_pass[plain] PASSED        415.6us

In those styles only the FAILED short-summary lines survive, and every PASSED / SKIPPED test disappears (7/17 correct). validate_pr only records F2P when the post-fix status is PASSED, so any repo with one of these styles in its pytest config would produce no candidates.

2. Quadratic backtracking on long non-test lines. .+?(?:\[.*?\])? is tried on every log line, including assertion diffs. On a single line of bracket-heavy text (E assert [0, [x]], [1, [x]], ...), parse_pytest took:

line length	time
5k chars	0.036s
24k chars	0.61s
50k chars	2.5s

That's roughly 4× per doubling. -vv diffs can be that long, generation parses logs up to 5 MB, and the verifier runs under the task's timeout_sec.

A small change fixes both. Accept pytest's other progress suffixes (_pytest/terminal.py: [ 1/17], [ 17 / 17 ], format_node_duration). Keep main's :: / .py heuristic, but check it on the first token before running the regex. Node IDs keep :: in the first token even when the parameter contains spaces. Same edit in _pr_runtime_verifier.py:

_PROGRESS = (
    r"\[\s*\d+%\]|\[\s*\d+\s*/\s*\d+\s*\]"  # percent / count styles
    r"|\d+(?:\.\d+)?(?:us|ms|s)|\d+m \d+s|\d+h \d+m"  # times style
)
_VERBOSE_RE = re.compile(
    r"^(?P<name>.+?(?:\[.*?\])?)\s+(?P<status>PASSED|FAILED|SKIPPED|ERROR)"
    rf"(?:\s+\(.*\))?(?:\s+(?:{_PROGRESS}))?$"
)
...
        # --- format (1): verbose progress (NAME first, STATUS after) ---
        head = line.split(None, 1)[0]
        if "::" not in head and not head.endswith(".py"):
            continue
        m = _VERBOSE_RE.match(line)

With that change:

• Real-pytest runs: 17/17 in all six output styles.
• 50k-char line: 0.000s.
• 100k-line realistic log: 0.25s (0.41s before).
• Your 72 tests plus test_log_parsers*, test_pr_runtime_verifier and test_grading: all 127 pass.

It would be worth adding count / times cases to test_real_pytest_logs_preserve_oracle_and_standalone_grading.

Non-blocking: a parameter containing ] - , e.g. test_x[a] - b], is inherently ambiguous in - summary lines. It yields an extra test_x[a] key, but the verbose line still records the right ID, so I don't think it needs handling.

[abhinavgautam01]

Thanks @KNambiarDJsc for the detailed review, confirmed and fixed both findings in both parsers. Added count/times suffix coverage, real-pytest checks against report-hook outcomes across output styles and verbosity levels and a large assertion-diff regression test. Tests pass on Python 3.12–3.14; lint and formatting are clean. Left the non-blocking ] - summary ambiguity unchanged as suggested.

[KNambiarDJsc]

Thanks for the quick turnaround. I re-ran my checks on ff9cedc, and everything from the review is resolved:

• Real pytest 9.0.3. I compared against pytest_runtest_logreport outcomes under -v, -v -rA, -vv -rA, and the classic, count and times styles. All 19/19 are correct in each, and the canonical parser matches the standalone verifier every time. The IDs covered spaces, -, status words, unmatched [, and :: inside the failure message.
• Performance. A 50k-char bracket-heavy line now takes 0.000s (it was 2.5s). A 448k-char line full of status words takes 0.000s, and a 100k-line log takes 0.25s.
• Durations. 2m 5s and 1h 3m durations (tests ≥ 60s in times style) parse correctly.
• CI-equivalent run. On Linux with py3.12, ruff check, ruff format --check and the full suite pass (816 passed). The Fix spaced pytest node ID parsing #92 test file also passes against this branch (88/88).

LGTM from my side. Happy for a maintainer to take it from here.