Repository navigation
build: make the workspace crates packageable for crates.io - #927
Open
LeeroyHannigan wants to merge 1 commit into
Open
LeeroyHannigan wants to merge 1 commit into
LeeroyHannigan wants to merge 1 commit into
Conversation
LeeroyHannigan
requested review from
amrith,
c33howard,
eliseong,
hitesh96db,
jcshepherd,
navareamol,
pdf-amzn and
robinnsc
as code owners
November 10, 2026 11:31
All 14 workspace crates now pass `cargo package`, the step before a publish to crates.io. This change publishes nothing. - Each internal entry of `[workspace.dependencies]` requires the workspace version exactly, as in `version = "=0.1.14"`. The crates change their APIs between patch releases. An exact pin stops a fresh install from mixing crates of two releases. - Internal dev-dependencies use a path and no version, so cargo removes them from the published manifests. Packaging then no longer stops at the dev-dependency cycle between extenddb-engine and extenddb-storage-sqlite. - The workspace uses `resolver = "3"`. On Rust 1.88, our minimum version, resolver 2 picked uuid 1.29.2 for an install without `--locked`. That version needs Rust 1.89. Resolver 3 picks uuid 1.26.1, and the build passes. - Each crate has a description, a repository and a homepage. The extenddb crate also has a README for its crates.io page, keywords and categories. - Each crate directory links LICENSE and NOTICE to the root files. This change regenerates the four license notices, and they list the workspace crates. - Cargo.lock moves spin to 0.9.9 and chacha20 to 0.10.2. crates.io lists the old versions as yanked. - cdrs-tokio stays on 9.0.x and cassandra-protocol on 4.0.x. cassandra-protocol 4.1.0 uses num-bigint 0.5 for its public BigInt, and the Cassandra backend then fails to compile. To hold the bound, extenddb-storage-cassandra now depends on cassandra-protocol directly. - crates/bin/build.rs reads the packaged commit from .cargo_vcs_info.json. A build from crates.io then reports its commit instead of "unknown". - The new devtools/check-crate-packages checks the pins, the dev-dependencies, the license files and the metadata. It also checks the field rules and size limits that crates.io applies at upload. A new `package` job in the Test workflow runs it, and the required `test` gate needs that job. - The backup-store README and config.rs drop one S3 server from their examples. - ADR-0029 records the decision. CONTRIBUTING.md describes the dependency rules, and packaging/README.md adds the pins to the release bump.
LeeroyHannigan
force-pushed
the
build/crates-io-packaging
branch
from
November 10, 2026 13:55
586e3c7 to
0ce13cf
Compare
5 of 8 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
This change makes all 14 workspace crates pass
cargo package, the step before a publish to crates.io. It publishes nothing.[workspace.dependencies]requires the workspace version exactly:version = "=0.1.14".{ workspace = true }now use a path and no version. Cargo removes them from the published manifests.resolver = "3".extenddbcrate also gets a README for its crates.io page, keywords and categories.LICENSEandNOTICEto the root files. I regenerated the four license notices, and they now list the workspace crates under the text of our LICENSE file.Cargo.lockmovesspin0.9.8 to 0.9.9 andchacha200.10.1 to 0.10.2. crates.io lists both old versions as yanked.config.rsdrop one S3 server from their examples.cdrs-tokiostays on 9.0.x andcassandra-protocolon 4.0.x. cassandra-protocol 4.1.0 moved its publicBigIntto num-bigint 0.5, and the Cassandra backend then fails to compile. Without the bounds, a fresh resolution, such ascargo installwithout--locked, picks 4.1.0.crates/bin/build.rsreads the packaged commit from.cargo_vcs_info.json. A build from crates.io then reports its commit instead ofunknown.devtools/check-crate-packageschecks the pins, the dev-dependencies, the license files and the metadata. It also checks the field rules and size limits that crates.io applies at upload. It reads the workspace throughcargo metadata. Its tests are indevtools/check_crate_packages_test.py.packagejob, and the requiredtestgate needs it.packaging/README.mdadds the pins to the release bump.Why
Because girls just wanna have fun!!
On main,
cargo package --workspacestops at the first crate with an internal dependency:With a version on each internal dependency, it stops at the dev-dependency cycle between
extenddb-engineandextenddb-storage-sqlite:crates.io accepts the
extenddbbinary only after the 13 crates it depends on, so all 14 must package. The crates change their APIs between patch releases. An exact pin stops a fresh install from mixing crates of two releases.Without
--locked,cargo installresolves every dependency again. On Rust 1.88, our minimum version, that choseuuid1.29.2, which needs Rust 1.89, and the build failed. Withresolver = "3", cargo choseuuid1.26.1 and the build passed.Testing done
cargo package --workspace --no-verify --lockedwrote the 14 packages in 5 s, anddevtools/check-crate-packagesprintedok.cargo package --workspace --lockedbuilt all 14 packages from their tarballs in 305 s with 12 jobs. It passed again on the final tree. The packagedextenddbbuild recorded the full commit SHA.cargo metadata --lockedgave 530 nodes before and after. Only thespinandchacha20nodes, the edges of their 3 dependents, and the new edge fromextenddb-storage-cassandratocassandra-protocoldiffer.Cargo.toml,Cargo.lockandcrates/, as the build stage ofDockerfile.devdoes. In that copy theLICENSEandNOTICElinks point at nothing. Thesqlite,dev-modebuild passed on Rust 1.88, and cargo gave no warning.devtools/generate-software-license-notices --checkfor each backend anddevtools/generate-dev-license-notices --checkpass.python3 -m unittest discover -s devtools -p '*_test.py'runs 35 tests, and all pass. Each of 48 one-line changes to the check script makes a test fail.cargo update -p cdrs-tokiomovedcassandra-protocolto 4.1.0, andcargo check -p extenddb-storage-cassandrafailed with E0308 onBigInt. With the bounds, a fresh lock on Rust 1.88 chose 4.0.1, and the check passed. The bounds change one line ofCargo.lock, the new dependency edge.run-testsjobs run the workspace tests on Rust 1.88.0 and on stable, and both pass. The Clippy workflow runs clippy with-D warningson every crate and target of the workspace, and it passes.-W clippy::pedantic, clippy reports nothing on the lines that this change touches. It still reports warnings in code outside this change.cargo fmt --all -- --checkpasses..cargo_vcs_info.jsonreader ran on Rust 1.88. It reads clean and dirty files in pretty and compact form, and it returnsNonefor malformed input.Checklist
cargo test --workspace)cargo fmt --check)cargo clippy -- -W clippy::pedantic)Storagetrait, auth model, on-diskformat, or public CLI surface, an RFC has been accepted or is linked
below. Otherwise, an ADR captures the decision (link below).
ADR / RFC: ADR-0029,
docs/adr/0029-crates-io-packaging.mdin this PRBy submitting this pull request, I confirm that my contribution is made under
the terms of the Apache License 2.0 and I agree to the Developer Certificate of
Origin (DCO). See CONTRIBUTING.md for details.