Add JOSS paper draft

.github/workflows/paper.yml

@@ -0,0 +1,61 @@
+name: Paper
+
+# JOSS paper preview: compiles paper/paper.md to PDF whenever paper content changes, so a
+# broken front matter, malformed bibtex, or unresolvable citation fails here instead of at
+# submission. The paper's structural gates (citation resolution, word budgets, front
+# matter) live in tests/test_paper.py and run in ci.yml's python-test job on every PR;
+# this workflow deliberately adds only the compiled artifact, path-filtered so untouched
+# PRs pay nothing.
+#
+# Compilation uses the official JOSS paperdraft resources from the archived (frozen
+# 2024-04-16) openjournals/whedon repository. The prebuilt openjournals/whedon Docker Hub
+# image is gone, so the image is built per run; the Dockerfile needs three patches,
+# identical to the Makefile's paper-image target (keep the two in sync):
+#   1. fully-qualify the pandoc/latex base image registry,
+#   2. pin tlmgr to the frozen TeXLive 2021 historic repo (the live tlnet now serves a
+#      newer TeXLive, so the stock install commands silently install nothing),
+#   3. strip the vestigial biblatex block from the JOSS latex template (frozen polyglossia
+#      forbids the template's load order; pandoc citeproc renders references anyway).
+on:
+  pull_request:
+    paths:
+      - "paper/**"
+      - ".github/workflows/paper.yml"
+  push:
+    branches:
+      - main
+    paths:
+      - "paper/**"
+      - ".github/workflows/paper.yml"
+
+permissions:
+  contents: read
+
+concurrency:
+  group: paper-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  paper:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v7
+      - name: Clone and patch JOSS paperdraft sources
+        run: |
+          git clone --depth 1 https://github.com/openjournals/whedon.git /tmp/whedon
+          sed -i 's|^FROM pandoc/latex|FROM docker.io/pandoc/latex|' /tmp/whedon/paperdraft.Dockerfile
+          sed -i '/^RUN tlmgr update --self/i RUN tlmgr option repository https://ftp.math.utah.edu/pub/tex/historic/systems/texlive/2021/tlnet-final' /tmp/whedon/paperdraft.Dockerfile
+          perl -0pi -e 's/\\usepackage\[\n  backend=biber,\n%[^\n]*\n%[^\n]*\n\]\{biblatex\}\n\\bibliography\{[^}]*\}\n//; s/\\renewcommand\{\\bibfont\}\{[^{}]*\}\n//' /tmp/whedon/resources/joss/latex.template
+      - name: Build paperdraft image
+        run: docker build -f /tmp/whedon/paperdraft.Dockerfile -t paperdraft /tmp/whedon
+      # paperdraft writes paper.pdf next to paper.md (fail-if-warnings: on), so an
+      # unresolvable citation or latex warning fails the job.
+      - name: Compile the JOSS paper PDF (test build, fails on warnings)
+        run: docker run --rm -v "$PWD/paper:/paper" -w /paper paperdraft paper.md
+      - name: Upload PDF preview
+        uses: actions/upload-artifact@v4
+        with:
+          name: paper
+          path: paper/paper.pdf
+          retention-days: 14

.gitignore

@@ -34,4 +34,8 @@
 *.opencode/
 # mkdocs build output
 site/
+# JOSS paper build output: paper.md/paper.bib/INSTRUCTIONS.md are tracked, the PDF is generated
+paper/*.pdf
+
 .ralph/
+.paperdraft/

.markdownlint-cli2.jsonc

@@ -6,7 +6,8 @@
 // Scope deliberately includes top-level *.md, docs/, and .github/ templates, and excludes
 // machine-generated/internal trees: .git, node_modules, .venv, site (mkdocs output),
 // target (Rust build), __pycache__, .ralph (agent state), plans/ + PLAN.md (agent-generated
-// planning artifacts), tests/ (fixture data), .tablassert (runtime artifacts).
+// planning artifacts), .paperdraft (local clone of the JOSS paperdraft compiler sources),
+// tests/ (fixture data), .tablassert (runtime artifacts).
 {
   "config": {
     "default": true,
@@ -35,6 +36,18 @@
   },
   "customRules": ["./.markdownlint/no-unicode-chars.cjs"],
   "globs": ["**/*.md"],
+  // The JOSS paper template puts every required section (Summary, Statement of need,
+  // ...) at heading level 1 below the YAML front-matter title; multiple H1s are the
+  // expected format there, not a structure error.
+  "overrides": [
+    {
+      "filter": ["paper/paper.md"],
+      "config": {
+        "MD025": false
+      },
+      "combine": "merge"
+    }
+  ],
   "ignores": [
     "**/.git/**",
     "**/node_modules/**",
@@ -45,6 +58,7 @@
     ".ralph/**",
     "plans/**",
     "PLAN.md",
+    ".paperdraft/**",
     "tests/**",
     ".tablassert/**"
   ]

.pre-commit-config.yaml

@@ -62,6 +62,17 @@ repos:
           - html-validate@11.16.4
           - html-validate-markdown@6.0.0
         pass_filenames: false
+      # JOSS paper integrity gate (front matter, citation resolution, word budgets).
+      # Scoped to one stdlib-only test file so it stays fast enough for the pre-commit
+      # stage, unlike the full suite (see the two-stage design comment above); the
+      # regular CI suite also runs it in python-test. markdownlint-cli2 above already
+      # lints paper/*.md via its **/*.md globs, so no paper scope is added there.
+      - id: paper-checks
+        name: paper-checks
+        entry: uv run pytest tests/test_paper.py -q -p no:cacheprovider
+        language: system
+        files: ^(paper/|tests/test_paper\.py$)
+        pass_filenames: false
       - id: pyright
         name: pyright
         entry: uv run pyright

CITATION.cff

@@ -12,16 +12,16 @@ authors:
     email: sgoetz@isbscience.org
     affiliation: Institute for Systems Biology
     orcid: "https://orcid.org/0009-0007-8373-2352"
-  - given-names: "Gwênlyn"
-    family-names: Glusman
-    email: gglusman@isbscience.org
-    affiliation: Institute for Systems Biology
-    orcid: "https://orcid.org/0000-0001-8060-5955"
   - given-names: Jared C.
     family-names: Roach
     email: jroach@isbscience.org
     affiliation: Institute for Systems Biology
     orcid: "https://orcid.org/0000-0003-4524-0107"
+  - given-names: "Gwênlyn"
+    family-names: Glusman
+    email: gglusman@isbscience.org
+    affiliation: Institute for Systems Biology
+    orcid: "https://orcid.org/0000-0001-8060-5955"
 references:
   - type: article
     title: "MicrobiomeKG: bridging microbiome research and host health through knowledge graphs"

CONTRIBUTING.md

@@ -59,6 +59,10 @@ The `Makefile` is intentionally small and mirrors the underlying commands:
 | `make fmt-check` | Ruff format check and `cargo fmt --check` |
 | `make typecheck` | Pyright |
 | `make check` | Lint, format check, typecheck, Python tests, Rust tests, and clippy with `-D warnings` |
+| `make paper-check` | JOSS paper gate: front matter, citation resolution, word budgets |
+| `make paper-image` | Build the JOSS paperdraft compiler image (local Docker layer cache) |
+| `make paper-pdf` | Compile `paper/paper.pdf` from `paper/paper.md` with Docker |
+| `make paper` | JOSS paper gate plus PDF compile |
 | `make clean` | Local build/test/doc artifacts |
 
 ## Project layout

Makefile

@@ -1,4 +1,4 @@
-.PHONY: setup build dev test test-rust lint fmt fmt-check typecheck check clean
+.PHONY: setup build dev test test-rust lint fmt fmt-check typecheck check clean paper paper-check paper-pdf paper-image
 
 # Install Python dev dependencies, CLI/QC/log extras, and the editable Rust extension.
 setup:
@@ -45,4 +45,40 @@ check: lint fmt-check typecheck test test-rust
 
 # Remove local build, test, documentation, and compiled-extension artifacts.
 clean:
-	rm -rf build dist site htmlcov .coverage .pytest_cache .ruff_cache rust/target src/tablassert/*.so
+	rm -rf build dist site htmlcov .coverage .pytest_cache .ruff_cache rust/target src/tablassert/*.so .paperdraft paper/paper.pdf
+
+# Validate the JOSS paper draft: front matter, citation resolution, word budgets.
+paper-check:
+	uv run pytest tests/test_paper.py -q
+
+# Clone + patch the JOSS paperdraft sources once; the stamp file keeps the patches
+# single-application across repeated make runs.
+.paperdraft/.patched:
+	git clone --depth 1 https://github.com/openjournals/whedon.git .paperdraft/whedon
+	# Qualify the base image registry so podman-based runtimes (no unqualified-search
+	# registries) resolve it exactly like Docker does, and pin tlmgr to the frozen
+	# TeXLive 2021 historic repo: the base image (pandoc/latex:2.11.2 ships TeXLive 2021)
+	# points tlmgr at the live tlnet, which now serves a newer TeXLive, so the stock
+	# tlmgr install calls silently install nothing.
+	sed -i 's|^FROM pandoc/latex|FROM docker.io/pandoc/latex|' .paperdraft/whedon/paperdraft.Dockerfile
+	sed -i '/^RUN tlmgr update --self/i RUN tlmgr option repository https://ftp.math.utah.edu/pub/tex/historic/systems/texlive/2021/tlnet-final' .paperdraft/whedon/paperdraft.Dockerfile
+	# The JOSS template loads biblatex before pandoc's polyglossia and the frozen
+	# TeXLive 2021 polyglossia forbids that order; pandoc citeproc renders the
+	# bibliography anyway, so the biblatex block is vestigial and gets stripped.
+	perl -0pi -e 's/\\usepackage\[\n  backend=biber,\n%[^\n]*\n%[^\n]*\n\]\{biblatex\}\n\\bibliography\{[^}]*\}\n//; s/\\renewcommand\{\\bibfont\}\{[^{}]*\}\n//' .paperdraft/whedon/resources/joss/latex.template
+	touch .paperdraft/.patched
+
+# Build the JOSS paperdraft compiler image from the openjournals/whedon resources.
+# whedon is archived (frozen 2024-04-16) but its paperdraft.Dockerfile and LaTeX templates
+# are still the official JOSS compilation path; the old openjournals/whedon Docker Hub
+# image is gone, so the image is built locally. Docker layer cache makes this a no-op
+# after the first build.
+paper-image: .paperdraft/.patched
+	docker build -f .paperdraft/whedon/paperdraft.Dockerfile -t paperdraft .paperdraft/whedon
+
+# Compile paper/paper.md to paper/paper.pdf with the JOSS paperdraft image.
+paper-pdf: paper-image
+	docker run --rm -v $(PWD)/paper:/paper -w /paper paperdraft paper.md
+
+# Full JOSS paper gate: validate the draft, then compile the PDF.
+paper: paper-check paper-pdf

llms.txt

@@ -38,7 +38,7 @@ Current CLI behavior queries an embedded redb database whose primary file is `fu
 - [Project Metadata](pyproject.toml): dependencies, tooling config, and CLI entrypoint. Optional extras: `cli`, `rt`, `aria2`, `qc`, and `log`.
 - [Tests](tests/): pytest coverage and fixtures.
 - [MkDocs Configuration](mkdocs.yml): docs navigation and published pages.
-- [GitHub Workflows](.github/workflows/): `ci.yml` runs lint, format, type, test, and rust checks; `docs.yml` builds and deploys MkDocs; `pipy.yml` checks the project version, builds sdist and wheel artifacts, and publishes to PyPI; `tag-version.yml` tags changed project versions.
+- [GitHub Workflows](.github/workflows/): `ci.yml` runs lint, format, type, test, and rust checks; `docs.yml` builds and deploys MkDocs; `pipy.yml` checks the project version, builds sdist and wheel artifacts, and publishes to PyPI; `tag-version.yml` tags changed project versions; `paper.yml` tests the JOSS paper compile and uploads the PDF artifact.
 - [Changelog](CHANGELOG.md): release history and version notes.
 
 ## Implementation Map

paper/INSTRUCTIONS.md

@@ -0,0 +1,88 @@
+# JOSS Paper Instructions
+
+This directory holds the draft JOSS (Journal of Open Source Software) paper for Tablassert.
+`paper.md` is the manuscript, `paper.bib` is its bibliography, and the compiled `paper.pdf`
+is generated, never committed. See the [JOSS author guidelines](https://joss.readthedocs.io/en/latest/submitting.html)
+before making major changes.
+
+## What JOSS reviewers check
+
+JOSS reviews the software as much as the text. Tablassert already satisfies the entry
+requirements, so the review focuses on the paper itself:
+
+- A clear **statement of need**: what problem Tablassert solves and for whom.
+- **State of the field**: how the prevailing practice (hand-written per-source Translator
+  ingests) falls short, what alternatives exist (Koza, KGX, hosted name resolution), and the
+  build-versus-contribute justification for a new package.
+- **Software design** and a **research impact statement**: the design trade-offs and the
+  realized uses (MicrobiomeKG, DAKP, KRAKEN, pending Translator integration).
+- An **AI usage disclosure** naming any generative tools used for the software,
+  documentation, or manuscript, with versions and how outputs were verified.
+- An **archive**: the tagged release gets a Zenodo DOI at acceptance; keep
+  `CITATION.cff` and this paper's author list in sync.
+- The paper's body stays between 750 and 1750 words excluding references; the abstract
+  under 250. The `paper-check` gate enforces both locally.
+
+JOSS rules change; re-check the
+ [paper format page](https://joss.readthedocs.io/en/latest/paper.html) before submission.
+
+## File map
+
+- `paper.md`: manuscript in JOSS (pandoc) format. YAML front matter carries title,
+  authors, ORCIDs, tags, and the `bibliography: paper.bib` key that citeproc loads;
+  the body uses the sections JOSS expects (Summary, Statement of need, State of the
+  field, Software design, Research impact statement, AI usage disclosure,
+  Acknowledgements, References). Author `affiliation:` values are INDICES into the
+  separate `affiliations:` list (the JOSS convention); putting an institution string
+  there silently collapses the rendered author block to the last author.
+- `paper.bib`: BibTeX entries. Every `[@key]` citation in `paper.md` must resolve to an
+  entry here, and every entry must be cited; `paper-check` enforces both directions.
+  Note: pandoc's bibliography parser treats `%` comment lines loosely, so keep `]`, `@`,
+  and other entry-like punctuation out of comments.
+- `figures/`: figure images referenced from `paper.md` (create when adding the figure).
+
+## Workflow
+
+1. Draft or edit `paper.md`. Cite with `[@key]`, for example KGX [@kgx] renders as
+   "(Mungall et al. 2022)". Multi-cite as `[@a; @b]`.
+2. Verify the matching `paper.bib` entry: title, authors, year, DOI. Entries seeded with
+   a `% TODO: verify` comment are unverified; check them against the publisher page or
+   <https://doi.org> before submission.
+3. Add a figure by dropping the image into `figures/` and referencing it from its own
+   paragraph with `![Caption text](figures/fig1.png)`; JOSS turns a captioned standalone
+   image into a numbered figure. There is no figure in the current draft.
+4. Run the local gates before committing:
+
+```bash
+make paper-check   # front matter, citations, word budgets (also a pre-commit hook)
+make paper-pdf     # compile paper/paper.pdf with Docker (requires Docker)
+make paper         # both
+```
+
+`make paper-pdf` builds the official JOSS paperdraft compiler image from the archived
+openjournals/whedon resources (the prebuilt Docker Hub image is gone); the first build
+takes a few minutes and later builds hit the Docker layer cache. The existing
+markdownlint hooks cover this directory automatically; keep prose ASCII (no unicode
+dashes or arrows) as everywhere else in the repository.
+
+## CI
+
+`.github/workflows/paper.yml` runs only when files under `paper/` change on a pull
+request: it compiles the PDF from the JOSS paperdraft resources and uploads `paper.pdf`
+as a workflow artifact so reviewers can preview the compiled paper without Docker. The
+pytest gate runs in the regular CI suite on every pull request.
+
+## Submitting
+
+When the draft is ready:
+
+1. Resolve every `TODO(collaborators)` marker in `paper.md` (AI usage disclosure,
+   Acknowledgements) and verify every remaining `paper.bib` entry against its publisher
+   page or <https://doi.org> before submission.
+2. Confirm `make paper-check` and `make paper-pdf` both pass.
+3. The submitting author creates an account at <https://joss.theoj.org>, then submits
+   through the JOSS site, pointing at this repository and branch. The editor assigns
+   two reviewers; JOSS compiles `paper.md` automatically during review (the bot stack
+   formerly called Whedon, now Buffy, does the compilation).
+4. Address review comments here, on this branch, until acceptance; JOSS archives the
+   accepted version and mints the DOI.