Skip to content

Fix make mongodb failing to start from a clean checkout — Closes #113 - #116

Draft
conradbzura wants to merge 1 commit into
masterfrom
113-fix-make-mongodb
Draft

Fix make mongodb failing to start from a clean checkout — Closes #113#116
conradbzura wants to merge 1 commit into
masterfrom
113-fix-make-mongodb

Conversation

@conradbzura

Copy link
Copy Markdown
Collaborator

Summary

Make make mongodb produce a working database from a clean checkout. Two independent defects blocked it, and either one alone was fatal — the container either failed to build or exited 1 seconds after starting.

The first is a latent build failure: database/ is gitignored, so COPY database/ /data/database/ fails outright on a fresh clone. This has been broken since 2fc98f8 removed the committed 4DN dump without updating the Dockerfile, and went unnoticed because a stray .DS_Store kept the directory alive on developer machines. Move the dump from a build-time COPY to a run-time read-only mount, which fixes the build and is the better shape regardless — the image no longer varies with what a developer has on disk, and obtaining a dump later costs a container restart rather than a rebuild.

The second is the reported symptom: ensureIndex in the index bootstrap calls getIndexes(), which raises NamespaceNotFound rather than returning an empty list when the collection has never been written. Treat that specific error as "no existing index to conflict with" and fall through to createIndex, which creates the collection.

Deliberately not changed: set -e in the startup script, and mongorestore. set -e surfaced a real bug rather than causing one, and a failed restore of a dump that is present should stop the container rather than leave a half-loaded database looking healthy. mongorestore was never at fault — it handled the dump-less directory correctly, logging 0 document(s) restored and exiting 0.

Closes #113

Proposed changes

Stop baking the dump into the image

Replace COPY database/ /data/database/ with RUN mkdir -p /data/database, and mount database/ read-only at run time from the mongodb Makefile target. The directory is still created inside the image so mongorestore always has a valid target — it exits 1 on a nonexistent path, which under set -e would take the container down — and so running the image with no mount at all degrades to an empty database rather than a crash. The Makefile creates database/ locally so the mount source always exists and the drop-in point stays visible.

Guard the restore on a dump actually being present

Branch on the directory being non-empty. An absent dump is the normal state of a clean checkout, so report it as such rather than leaving the reader to interpret mongorestore's 0 document(s) restored trace, which reads like a failure.

Survive a namespace that does not exist

Wrap getIndexes() in a try/catch that swallows only NamespaceNotFound (code 26) and rethrows everything else, so a genuine server error still aborts rather than silently producing a half-indexed database. The IndexOptionsConflict drop-and-recreate path is untouched and unreachable in the empty case — a collection with no indexes cannot produce a name match.

Only ensureIndex was affected. Every bare createIndex call in the file is immune because createIndex creates the namespace implicitly, which is why jobs — the sole ensureIndex caller — was the single collection that aborted the run.

Correct the README

The Docker Startup block claimed step 1 restores sample data and that POST /sync was optional. Both are false: the database starts empty and /sync is how it gets data. The Makefile Targets table repeated the same claim and is corrected to match, so the two places state one fact in one voice.

Test cases

No automated tests accompany this change, and the reason is worth stating rather than glossing: the defects live in a Dockerfile and a mongosh bootstrap script, neither of which the Python suite executes. tests/test_indexes.py pins the operational index specs in lockstep between scripts/create-indexes.js and src/cfdb/indexes.py, but it parses the specs rather than running the script against a server — which is precisely why a NamespaceNotFound raised at run time was invisible to a green suite. Closing that gap needs a container-backed integration test and is larger than this fix.

Verification was performed by hand against real containers:

# Scenario Given When Then Coverage target
1 Baseline reproduction A build context with no database/ directory The pre-fix Dockerfile.mongodb is built Build fails with "/database": not found Defect (b) is real
2 Clean checkout build The same context, no database/ The fixed Dockerfile is built Build succeeds Defect (b) is fixed
3 Empty database startup No dump present make mongodb runs Container stays up, logs No dump at /data/database, then All indexes created successfully Defect (a) is fixed
4 Index completeness A container started with no dump cfdb.jobs indexes are listed All six are present, including workflow_key_active_unique and terminal_ttl The ensureIndex path completes
5 Dump present Two seeded docs dumped with mongodump --gzip into database/ make mongodb runs Restore reports 2 document(s) restored, indexes still apply, docs are queryable The mount path works

The Python suite was confirmed unchanged at 1112 passed, 91 deselected against origin/master, and tests/test_indexes.py passes 29/29. Ruff reports two findings, both pre-existing on master and in files this PR does not touch.

`make mongodb` could not bring up a working database from a fresh clone.
Two independent defects, either fatal on its own.

`ensureIndex` in scripts/create-indexes.js called `getIndexes()` to look
for a conflicting index, which throws NamespaceNotFound (code 26) on a
collection that has never been written to — the normal state of a
database with no dump restored. `jobs` is the only collection routed
through the helper, so it was the single index that aborted the run, and
`set -e` in the startup script took the container down with it. Treat
only NamespaceNotFound as "no index to conflict with" and fall through to
`createIndex`, which creates the namespace. Any other server error still
propagates, and the IndexOptionsConflict drop-and-recreate path is
untouched: a collection with no indexes cannot produce a name match.

Dockerfile.mongodb copied `database/` into the image, but that directory
is gitignored (the 4DN dump was removed in 2fc98f8) so on a clean clone
it does not exist and the build failed outright with `"/database": not
found`. It only appeared to work where a stray file such as `.DS_Store`
kept the directory alive. Create the directory in the image instead and
mount the dump read-only at run time from the Makefile, guarding the
restore on the mount being non-empty. The image no longer varies with
what a developer happens to have on disk, and obtaining a dump later is
a container restart rather than a rebuild.

`set -e` is deliberately kept: it surfaced a real bug rather than causing
one, and a failed restore of a dump that IS present should stop the
container rather than leave a half-loaded database looking healthy.

README's Docker Startup section claimed step 1 restores sample data and
that `POST /sync` was optional. Both were false — correct them and record
`database/` as the drop-in point for an optional dump.

Closes #113
@conradbzura conradbzura self-assigned this Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix make mongodb failing to start from a clean checkout

1 participant