Add optional local structural code-map provider for MAP research (closes #310)

[azalio]

Summary

• Implements mapify_cli.code_map: Python AST-backed deterministic structural discovery — no external binaries, MCP servers, or network access.
• Adds mapify code-map query CLI command that scans Python files and emits ResearchEvidence-compatible JSON.
• 37 fixture-only tests (VC1–VC10), no live model calls.

Files changed

• src/mapify_cli/code_map.py — new module: CodeSymbol, CodeMapIndex, CodeMapQueryResult dataclasses; _SymbolVisitor (AST NodeVisitor); build_python_ast_index() with unsafe-path symlink guard; query_code_map() with ranked results; default_index_path() helper.
• tests/test_code_map.py — 37 tests covering symbol extraction (classes, functions, methods), search ranking (exact→prefix→contains), malformed/unsafe file handling, ResearchEvidence compatibility via parse_research_locations, max_results cap, and CLI command.
• src/mapify_cli/__init__.py — mapify code-map query command with --repo-root, --max-results, --out flags.

First-slice scope

• Python AST fallback only — no tree-sitter, no multi-language support
• No persistent index (always fresh scan)
• No MCP/CodeGraph integration
• No cloud indexing or source upload
• No route recognition

ResearchEvidence compatibility

Output from as_research_evidence() passes directly into parse_research_locations() — the existing research-eval pipeline accepts it unchanged.

Test plan

• uv run pytest tests/test_code_map.py -v → 37 passed
• uv run python -m pyright src/mapify_cli/code_map.py tests/test_code_map.py src/mapify_cli/__init__.py → 0 errors
• make check → 3427 passed, 4 skipped

Closes #310

---

Generated by Claude Code

Summary by CodeRabbit

• New Features

  • Added new CLI commands to compare research runs, inspect repository code structure, and generate JSON reports.
  • Added optional output files and non-zero exit codes when checks fail.

• Bug Fixes

  • Improved handling for missing, malformed, or out-of-scope repository paths during code scanning.
  • Added stricter validation for comparison results, including quality and regression limits.

• Tests

  • Added end-to-end coverage for the new CLI commands and report formats.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: a62e28ea-b314-4e4a-962b-bf5979b1af5c

📥 Commits

Reviewing files that changed from the base of the PR and between d6091d5 and 657447b.

📒 Files selected for processing (7)

• src/mapify_cli/__init__.py
• src/mapify_cli/code_map.py
• src/mapify_cli/minimality_eval.py
• src/mapify_cli/research_eval_compare.py
• tests/test_code_map.py
• tests/test_minimality_eval.py
• tests/test_research_eval_compare.py

---

📝 Walkthrough

Walkthrough

This PR adds three new optional subsystems: a Python AST-based structural code-map provider queryable via a new mapify code-map query CLI command; a minimality A/B evaluation harness (mapify_cli.minimality_eval); and a research discovery ROI comparison module (mapify_cli.research_eval_compare) exposed via mapify research-eval compare. Each subsystem includes a dedicated test suite.

Changes

Structural Code-Map Provider

Layer / File(s)	Summary
Data model and AST indexing src/mapify_cli/code_map.py	Adds CodeSymbol, CodeMapIndex, CodeMapQueryResult dataclasses and an AST visitor plus build_python_ast_index() to safely index classes/functions/methods under a repo root.
Query API src/mapify_cli/code_map.py	Adds query_code_map() returning status/confidence/search stats/relevant locations and default_index_path().
CLI wiring src/mapify_cli/__init__.py	Registers code-map Typer sub-app and code-map query command converting results to ResearchEvidence with configurable exit codes.
Tests tests/test_code_map.py	Covers indexing, search ranking, query statuses, evidence conversion, path helpers, and CLI behavior.

Minimality A/B Eval Harness

Layer / File(s)	Summary
Doctrine block and data model src/mapify_cli/minimality_eval.py	Adds doctrine constants, build_doctrine_block(), and MiniEvalTask/MiniEvalArmResult/MiniEvalTaskResult/MiniEvalReport dataclasses.
Fixture corpus and arms src/mapify_cli/minimality_eval.py	Defines deterministic fixture tasks and DEFAULT_ARMS/FULL_ARMS.
Scoring, report, and CLI src/mapify_cli/minimality_eval.py	Implements scoring/evaluation logic, run_minimality_eval(), default_run_path(), and an argparse-based _main() CLI.
Tests tests/test_minimality_eval.py	Covers doctrine output, contamination detection, LOC counts, irreducible-task warnings, report persistence, and serializability.

Research Discovery ROI Comparison

Layer / File(s)	Summary
Metrics and report data model src/mapify_cli/research_eval_compare.py	Adds DiscoveryMetrics, ArmScore, CompareReport dataclasses plus exploration-cost helper computations.
Comparison orchestration and I/O src/mapify_cli/research_eval_compare.py	Implements compare_research_runs(), compare_research_files(), default_compare_path(), and in-file fixtures.
CLI wiring src/mapify_cli/__init__.py	Adds research-eval compare command validating inputs and applying pass/fail exit codes.
Tests tests/test_research_eval_compare.py	Covers quality floors, stale regression, warnings, discovery metrics, schema, and CLI behavior.

Estimated code review effort: 4 (Complex) | ~75 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CLI as code-map query CLI
  participant QueryAPI as query_code_map
  participant Indexer as build_python_ast_index
  CLI->>QueryAPI: query(query, repo_root, max_results)
  QueryAPI->>Indexer: build index from repo_root
  Indexer-->>QueryAPI: CodeMapIndex(symbols)
  QueryAPI->>QueryAPI: search symbols, compute confidence
  QueryAPI-->>CLI: CodeMapQueryResult(status, relevant_locations)
  CLI->>CLI: print/write ResearchEvidence, set exit code

Loading

sequenceDiagram
  participant CLI as research-eval compare CLI
  participant FileIO as compare_research_files
  participant Compare as compare_research_runs
  CLI->>FileIO: read baseline/treatment/expected paths
  FileIO->>Compare: compare_research_runs(outputs, expected, thresholds)
  Compare->>Compare: score arms, check quality floors and stale regression
  Compare-->>FileIO: CompareReport(warnings, failures, deltas)
  FileIO-->>CLI: CompareReport
  CLI->>CLI: print JSON, write --out, exit 1 if not passed

Loading

Possibly related issues

• [CodeGraph] Add structural-discovery ROI eval for MAP research #311: Implements the structural-discovery ROI eval and code-map-backed ResearchEvidence comparison work this PR builds toward.

Poem

A rabbit hopped through AST trees,
found every def and class with ease,
compared two arms, baseline and new,
counted stale paths, warned a few,
now code-maps hum and evals score —
🐇 thump thump, let's ship some more!

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/compassionate-cerf-5tugql

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}