Mint DOIs for notebook groups from the image workflow

[bendichter]

Every notebook group is now published as a verified container image, and the blog post makes the case for the collection as a venue for executable research artifacts. What is missing is a citable identifier for each artifact. This issue proposes minting a DOI per notebook group on Zenodo, automatically, from the same workflow that publishes the image.

What Gets a DOI

One concept DOI per notebook group (the unit that already maps to one image), with a new version DOI whenever the group changes. The image is recorded inside each version as its executable form rather than getting its own DOI, since the notebook is the intellectual object and the image is its frozen environment.

Authorship Comes from an Opt-In CITATION.cff

A DOI needs creators, ideally with ORCIDs, and nothing structured exists per notebook today (a survey found zero authorship metadata in any notebook or README). Proposal: a CITATION.cff in the group directory, alongside requirements.in, carrying authors with ORCIDs, a title, the license, and the paper DOI under references. Only groups with one get a DOI, so nothing is minted without real authorship. This also gives GitHub's "Cite this repository" button something to show.

When Minting Happens, and When It Does Not

The step runs at the end of the merge job, only when a manifest was actually published. Versioning is keyed to the full build hash, which covers the notebooks, pins, helper fetches, the CITATION.cff, and the Dockerfile, so each DOI version corresponds to exactly one published image digest. A change to a group's own content mints a new version automatically. Rebuilds caused by tooling changes (Dockerfile, base image, system libraries) rebuild every group, and those mint new versions only when the run is dispatched with an explicit mint flag, so an environment bump is a deliberate release rather than a side effect. Old image digests are never deleted, so earlier DOI versions keep resolving to exactly what they described. If the newest Zenodo version already carries the current build hash, the step exits. Existing records are found by searching Zenodo for a keyword of the form dandi-notebooks:<group>, so no state is written back to the repo.

What a Record Contains

The verified container image itself, deposited as an OCI image archive (one file holding both the amd64 and arm64 images, 3 to 4 GB, within Zenodo's 50 GB per-record limit), so the executable form persists with the record and does not depend on the GitHub Container Registry staying up. Alongside it: the notebooks, requirements.in, README.md, CITATION.cff, the resolved pin list, and an image.json recording the registry reference, manifest digest, tags, and commit, so the deposited archive can be matched to what the registry served. Related identifiers link the dandiset's published DOI (DANDI exposes 10.48324/dandi.<id>/<version> for published versions; drafts get the URL instead), the paper DOI, and the GitHub tree at the commit. The index page would show a DOI badge next to the Colab and docker badges, looked up the same way it checks GHCR today.

Account, License, and Rollout

A DANDI-owned Zenodo account provides the token as a repository secret. The whole flow is validated against sandbox.zenodo.org first (create, no-op rerun, new version on change) before the production token is added. Default record license is Apache-2.0 to match the repository, overridable per group in CITATION.cff. Pilot on one group, then invite contributors to add CITATION.cff files.

Open Questions

1. Depositing the image archive means roughly 3 to 4 GB per version across a few dozen groups. Zenodo allows it, but it is worth a word with them about expected volume before the first production mint.
2. Is there a DANDI Zenodo community these should be collected under?
3. For existing notebooks, who supplies authorship: the contributors who wrote them, the dataset authors, or both? The dataset's DANDI contributors (with ORCIDs) are machine-readable and could be offered as a default for review.
4. Relation types for the dandiset and paper links: isSupplementTo seems right for both, but opinions welcome.

[bendichter]

Thoughts, @CodyCBakerPhD, @yarikoptic , @asmacdo ?

[CodyCBakerPhD]

Probably better off as an official design doc IMO, too many details to hash under an issue

[bendichter]

@CodyCBakerPhD really? I worry about it dragging as a design doc. Could we try discussing here and move to a design doc if the discussion becomes too involved?

[satra]

i would add a constraint that a notebook doi should only exist if the underlying dataset has a doi. if source data is not persistent, the notebook should not receive a doi. also is the role of the container a purely replication enterprise, or a container that can be used to do variant analysis? the two are different concepts and can have different preservation requirements.

[bendichter]

I would add a constraint that a notebook doi should only exist if the underlying dataset has a doi

Sounds good to me. Otherwise, that would be a glaring missing piece in reproducibility and persistence.

also is the role of the container a purely replication enterprise...

By "replication" do you mean purely reproducing the figures of a paper? (We can call that replication even if its done by the same group, which I imagine will happen a lot). Yes, that's the main use-case I was picturing. Sort of a new take on eLife's Executable Research Articles. Though I don't see why not create something new. The system would allow it. If reproducing the figures of a paper, we can add a field for explicitly linking to that paper. That field would just be left blank otherwise. Are there different considerations for a novel notebook that doesn't map to a paper?

[bendichter]

A few facts on the account question, since it came up. Zenodo has no organizational accounts: its documentation is explicit that community members "login on Zenodo using their personal account," and organizations are represented by communities instead. I also checked for existing DANDI presence: there is no DANDI community on Zenodo today, and the DANDI-related records that exist (Neurosift, the open-data notebooks paper, and others) were each uploaded under an individual's account.

Zenodo separates the people credited for a record (the creators listed in its metadata, who get the citation) from the account that uploaded it (the owner, who can edit it). The notebook authors are always the creators, whoever owns the record, so the question is only about administration. With that in mind I propose the following.

A dandi community as the organizational layer. Communities are what Zenodo intends for "projects, institutions, domains, and conferences," with owner, manager, and curator roles, so several of us can hold management rights and the community persists regardless of who uploaded any given record. It also gives the existing DANDI-related records a home, and every notebook record would be submitted to it at creation.

A maintainer's account as the uploader, with shared management. The minting token belongs to one maintainer's personal account (I can be that person to start). The script then shares each record with the other maintainers with management rights as part of creation, so if the token holder ever steps back, the others already control every record and only the token needs rotating. This is Zenodo's own answer to continuity: shared management rather than shared credentials.

Authors get editing rights to their own records. CITATION.cff carries author emails, so the script can also share each record with its authors, giving those who have Zenodo accounts the ability to edit their own record without us holding anyone's credentials.

The alternative is a functional account tied to dandibot. It would work in practice and would keep any individual's name off the uploader line, but it sits outside Zenodo's documented model and could collide with policy or login requirements later. I would keep it as the fallback if the group prefers no personal account in the loop. Opinions welcome, particularly on the maintainer-versus-bot choice.

[bendichter]

Is Byte-Exact Preservation in Zenodo a Requirement?

Everything the record needs is already there without the image archive: the Dockerfile, resolved pins, commit, and image.json digest describe the environment well enough to rebuild it, and since we never delete old GHCR digests, each DOI version keeps pointing at the exact bytes it describes. The archive adds byte-exact persistence only if GHCR itself disappears, which feels like insurance against a narrow failure mode.

It is also expensive insurance, because docker builds are not bit-reproducible, so nothing deduplicates across versions: every mint uploads the full 3 to 4 GB again, even when the trigger was a one-line notebook edit. Across a few dozen groups that is on the order of a few hundred GB per year, which dominates our Zenodo footprint (open question 1).

So the question to settle is whether byte-exact preservation is a requirement or a nicety. If it is a nicety, I would start with records containing the notebooks, metadata, pins, and image.json, and attach the archive as a new version only when someone actually asks, at which point open question 1 largely resolves itself. If it is a requirement, the volumes still sit comfortably under every published Zenodo limit, which are all per record, so all this owes them is a brief fair-usage heads-up before scaling past the pilot.