test: spec compliance test validating constants.py against table.csv

[sumanjeet0012]

Closes #41.

What was changed:

• Added multiformats/multicodec as a git submodule in multicodec-spec/.
• Created a new test file tests/test_spec.py that validates CODECS, CODE_TABLE, and NAME_TABLE against the table.csv file from the authoritative upstream spec.
• Updated multicodec/constants.py and multicodec/code_table.py to match the latest upstream table.csv.
• Configured .pre-commit-config.yaml and pyproject.toml to appropriately exclude the submodule from linting.
• Updated the Makefile to exclude the submodule and docs/conf.py (which previously failed due to import location) during make fix.

Why the change was needed:

• The existing codebase lacked a compliance test, meaning the codecs registry could silently drift out of sync with the upstream spec. In fact, running the new test immediately caught missing codecs (mldsa-44-pub etc) that had been added to upstream but not updated in py-multicodec.

How it was verified:

• Validated via make pr, which runs the ruff formatter/linter, mypy typechecks, and the full test suite.
• The new test_spec_table_completeness test passes successfully.

[acul71]

AI PR Review: #51 — test: spec compliance test validating constants.py against table.csv

Reviewer: acul71 (maintainer)
PR: #51
Author: sumanjeet0012
Date: 2026-09-07

---

1. Summary of Changes

This PR addresses Closes #41: there was no automated check that CODECS / NAME_TABLE / CODE_TABLE match the authoritative multicodec table.csv.

Changes:

• Adds git submodule multicodec-spec → https://github.com/multiformats/multicodec
• Adds tests/test_spec.py completeness + consistency checks (skip tag == "none")
• Updates multicodec/constants.py and multicodec/code_table.py to include missing upstream codecs (e.g. mldsa-*), bringing registry to 649 entries on this branch
• CI (tox.yml): actions/checkout with submodules: true
• Excludes submodule from pre-commit / ruff / make fix

Modules/files: .gitmodules, .github/workflows/tox.yml, .pre-commit-config.yaml, Makefile, pyproject.toml, multicodec-spec, multicodec/constants.py, multicodec/code_table.py, tests/test_spec.py
Breaking changes: None expected (additive codec registry expansion)

---

2. Branch Sync Status and Merge Conflicts

Branch Sync Status

• Status: Ahead of origin/master
• Details: 0 1 → 0 behind, 1 ahead

Merge Conflict Analysis

✅ No merge conflicts detected. The PR branch can be merged cleanly into origin/master.

---

3. Strengths

• Matches go-multicodec’s “validate against CSV” approach
• Immediately useful: caught missing codecs vs upstream
• CI correctly checks out submodules on Linux and Windows jobs
• Lint exclusions for the vendored subtree are appropriate
• Local validation: lint, typecheck, 736 tests passed (including test_spec), docs OK; GitHub CI green

---

4. Issues Found

Critical

• File: newsfragments/ (missing)
• Issue: Missing mandatory newsfragment for issue #41
• Suggestion: Add newsfragments/41.feature.rst (or 41.misc.rst) noting the spec compliance test and codec table sync with upstream table.csv

Major

• File: CONTRIBUTING.rst / README.rst

• Issue: Contributors cloning without --recurse-submodules will fail test_spec / local pytest when the submodule is empty

• Suggestion: Document git clone --recurse-submodules and git submodule update --init in contributing docs

• File: tests/test_spec.py

• Line(s): 14–43

• Issue: Validates name/code bidirectional consistency but does not assert CSV tag / status / description fields (if those are represented elsewhere) or pin/document the expected submodule commit update process

• Suggestion: Acceptable v1 of the test per issue proposal; add a short note in CONTRIBUTING on how to bump the submodule when refreshing the table (and re-run tools/update-table.py if applicable)

• File: pyproject.toml

• Line(s): [tool.ruff.lint] ignore = ["RUF003"]

• Issue: New global ruff ignore appears unrelated to the submodule exclude; may weaken lint for the whole project

• Suggestion: Prefer per-file ignore or justify why RUF003 must be ignored project-wide; keep extend-exclude = ["multicodec-spec"] only if that was the real goal

Minor

• File: tests/test_spec.py
• Issue: Uses os.path joins; pathlib would match modern style
• Suggestion: Optional cleanup

---

5. Security Review

• Risk: Submodule pulls third-party content into the tree; CI/runtime only reads table.csv as data
• Impact: Low (trusted multiformats org URL; no code execution from CSV)
• Mitigation: Keep submodule URL pinned to multiformats/multicodec; continue excluding it from lint/format tools

No unsafe I/O beyond reading the CSV path under the repo.

---

6. Documentation and Examples

Missing contributor guidance for submodule init (see Major). Newsfragment should describe user-visible codec table additions if new public codec names/codes ship with this PR.

---

7. Newsfragment Requirement

⚠️ BLOCKER

• Severity: CRITICAL / BLOCKER
• Issue: No newsfragments/41.*.rst; PR correctly references Closes #41
• Suggestion: Add fragment covering (1) spec compliance test and/or (2) newly registered codecs for users
• Action Required: Newsfragment before approval

---

8. Tests and Validation

Linting (make lint)

• Exit 0 — all hooks passed

Type Checking (make typecheck)

• Exit 0

Test Execution (make test)

• Exit 0 — 736 passed, including tests/test_spec.py

Documentation Build (make docs-ci)

• Exit 0

GitHub CI

• SUCCESS with submodule checkout

---

9. Recommendations for Improvement

1. Add newsfragments/41.*.rst (blocker)
2. Document submodule clone/update for contributors
3. Revisit global RUF003 ignore — scope it or remove
4. Optionally document submodule bump + table regeneration workflow

---

10. Questions for the Author

1. Was [tool.ruff.lint] ignore = ["RUF003"] required for the submodule, or accidental?
2. Should newly added codecs also get a dedicated user-facing newsfragment list, or is a single “synced to upstream table” note enough?
3. Is the pinned submodule commit (833c740…) intentional as the compatibility baseline?

---

11. Overall Assessment

• Quality Rating: Good
• Security Impact: Low
• Merge Readiness: Needs fixes (newsfragment; docs for submodule; clarify ruff ignore)
• Confidence: High

Verdict: REQUEST_CHANGES — valuable compliance infrastructure and table sync; blocked on newsfragment and a few process/docs cleanups.