diff --git a/README.md b/README.md index 3b71bb2..057314a 100644 --- a/README.md +++ b/README.md @@ -181,7 +181,7 @@ hand. **What if I lose my key?** If no remaining device has access, the data is gone. There is no recovery service by design. Keep an offline backup of at least one administrator key and -its `admin-state.json` pin (under `~/.local/state/git-e2ee//`). +its `continuity.json` floor (under `~/.local/state/git-e2ee//`). **Does revoking a device hide old history from it?** No. It keeps whatever it could already decrypt. It gets no keys for anything diff --git a/docs/benchmarks.md b/docs/benchmarks.md index 7ea44e2..0727502 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -540,8 +540,11 @@ quadratic in aggregate metadata: each immutable policy generation repeats a larger device list and each manifest repeats a larger envelope list. This does not multiply content packs. -Push timing also depends on the client path. The direct diagnostic -`git-e2ee push` has no clone-local continuity pin, so it deliberately validates +Push timing also depends on the client path. (The direct diagnostic +`git-e2ee push`/`fetch` mode measured here no longer exists: every push and +fetch now goes through the continuity floor, and the benchmark scripts use the +native helper only. The numbers below are kept as recorded.) The direct +`git-e2ee push` had no clone-local continuity pin, so it deliberately validated the complete 100-generation chain on every push. It took 3.12--3.54 seconds in this run. Normal Git usage goes through `git-remote-e2ee`; set `BENCH_E2EE_PUSH_MODE=native` to benchmark that path. Its first push after 99 diff --git a/docs/design.md b/docs/design.md index 552d1f1..89526db 100644 --- a/docs/design.md +++ b/docs/design.md @@ -130,18 +130,22 @@ administrative CAS is never automatically rebased. - Packs are decrypted segment by segment directly into `git index-pack` oldest-first. - After import, every advertised ref must resolve to a complete local Git - object graph before the continuity floor or remote-tracking refs move. -- Authentication alone never advances the continuity floor. The floor advances - after a successful import and connectivity check, after the client's own - publication wins compare-and-swap, or during listing when every advertised - ref already resolves to a complete local object graph. The last case matters - because Git skips the helper's fetch command when nothing needs to be - transferred, as after a membership change, a deletion, or a tag pointing at - an existing object; without it, such a state would never be pinned and a - later replay of the previous state would be accepted. -- Per-repository/remote client-state updates hold an advisory file lock across - validation and read-modify-write. The state replacement remains durable and - atomic, and its manifest and policy generations never move backward. + object graph before the checkout's import progress or remote-tracking refs + move. +- One continuity floor per repository root per machine, + `$XDG_STATE_HOME/git-e2ee//continuity.json`, is the only continuity + authority. Listing, fetch, push and dry-run push, `init`, `device-*`, + `verify`, and `recover` take its exclusive lock, require the authenticated + chain to pass through it, and advance it after success, so concurrent + processes on one machine serialize. Listing advances it on authentication + and continuity alone; Git omits the fetch command when nothing needs to be + transferred (after a membership change, a deletion, or a tag pointing at an + existing object), so a floor that waited for fetch would never pin such a + state. +- Each checkout's import progress (`.git/git-remote-e2ee//progress.json`) + records the imported generation, imported pack IDs, and verified ref + frontier. It is a cache under its own lock, taken after the floor's, and is + never consulted for continuity. - The helper advertises `refs/heads/*` and `refs/tags/*`; every other ref namespace is rejected before publication. - Branch updates require fast-forward ancestry unless force is explicit. Tags @@ -256,25 +260,22 @@ motivates future garbage collection. ## Continuity and limits -Returning clients pin repository root, manifest ID/generation, and policy -generation. They reject rollback, non-descendant state, policy rollback, and -root substitution. A signed manifest seen by list is not yet a pin: helper -state advances after all required packs are imported and every advertised ref -passes the complete-object-graph check, or after the client's own publication -successfully wins compare-and-swap. A missing or unusable pack therefore cannot -raise the continuity floor. - -Each local repository/remote pair has one stable lock file beside its client -state. Helpers hold the exclusive lock while checking the current floor and -performing an import or publication update. State writes merge monotonically -and use the existing flushed temporary-file, rename, and parent-directory -sync path. Administrator pins use the same serialized, monotonic state update -rules after initialization or a successful policy transition. - -State files written by earlier builds are not lowered automatically. Their -head pin may have come from listing, but it is indistinguishable from a valid -pin made after the client's own publication; lowering it could weaken rollback -detection. +The machine floor pins repository root, manifest ID/generation, and policy +generation. It rejects rollback, non-descendant state, policy rollback, and +root substitution. The library takes the state directory as an argument +(`EncryptedRepository::new(storage, key, state_dir)`); only the binaries read +`XDG_STATE_HOME`. There is no code path that publishes or fetches without the +floor. + +A single machine-wide floor closes a gap that per-checkout pins left open: a +revocation seen only by `device-revoke` used to protect administrative +commands but not a checkout that had not fetched it, so a replayed +pre-revocation state could receive a new push readable by the revoked device. + +The floor file is replaced with the flushed temporary-file, rename, and +parent-directory sync path. Its stable `continuity.json.lock` sibling is held +from validation through advancement. Floor writes are monotonic: a lower +generation is ignored and a different head at the same generation is a fork. Storage alone still cannot prevent: @@ -389,8 +390,8 @@ require the current tip to be a single-parent child of it. `git-e2ee recover` takes a target (a Git remote name or a storage URL), plus `--key` and `--repo`. The repository defaults to `.`; the continuity floor -is read from `.git/git-remote-e2ee//state.json`, where the root -comes from the key file. Without `--publish`, the command +is this machine's `$XDG_STATE_HOME/git-e2ee//continuity.json`, +where the root comes from the key file. Without `--publish`, the command only reports. Publishing an older offer requires `--discard-newer`. Directory publication requires `--accept-stale-floor` and prints the warning above. diff --git a/docs/durability.md b/docs/durability.md index 8d66d4b..9eb828e 100644 --- a/docs/durability.md +++ b/docs/durability.md @@ -6,7 +6,7 @@ Windows and virtual-power-cut results below were verified on 2026-09-23. ## Publication sequence -Client continuity state (`state.json`), filesystem `HEAD`, and immutable +The continuity floor (`continuity.json`), filesystem `HEAD`, and immutable objects use the same flush rules: 1. Create any missing directory that will hold the published name, then flush @@ -19,7 +19,7 @@ objects use the same flush rules: 2. Write a temporary file and `File::sync_all` it. Return the error if that fails. 3. Publish the name. - - `state.json` and `HEAD` use `rename` in the destination directory and + - `continuity.json` and `HEAD` use `rename` in the destination directory and replace an existing file. - Immutable objects use `hard_link` from `.staging/` to `packs//`. Those are different directories on the same @@ -33,9 +33,9 @@ objects use the same flush rules: `FilesystemStorage::initialize` creates `packs` and `manifests` with the same directory helper, so the storage root (the preexisting ancestor) is flushed when those directories are new. A later object shard flushes the new -`` directory and `packs`, not the storage root again. Client state -creates `.git/git-remote-e2ee/` the same way and stops at `.git` when -that directory already exists. +`` directory and `packs`, not the storage root again. The floor +creates `$XDG_STATE_HOME/git-e2ee/` the same way and stops at +the deepest directory that already exists. The carrier-Git backend does not use this helper. Its compare-and-swap is the fast-forward push of the carrier branch (`refs/heads/main` by default). @@ -199,7 +199,7 @@ above, and panics otherwise. Ops are isolated inside the fixture: -- `client_state` → `\client\state.json` via `persist_client_state` +- `client_state` → `\client\durability-fixture\continuity.json` via the continuity-floor writer - `head` → `\head\HEAD` via `FilesystemStorage::compare_and_swap_head` - `object` → `\object-store\objects\\` via `put_object_if_absent` diff --git a/docs/guide.md b/docs/guide.md index 0949e32..74201ce 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -44,20 +44,22 @@ repository is its administrator. The key path is stored only in local Git configuration. The key file is never written to the encrypted remote and must never be committed. -The CLI keeps an administrative continuity pin per repository root at -`$XDG_STATE_HOME/git-e2ee//admin-state.json` (by default under -`~/.local/state`). `device-add`, `device-revoke`, and `device-list` all check -the storage head against this pin and then advance it, so storage cannot show -an older membership list, for example one that hides a revocation, once you -have seen a newer one. Moving or renaming the key file keeps the pin. Back up -that file together with the key. - -Inside a Git repository, the continuity pin is likewise kept per repository -root at `.git/git-remote-e2ee//state.json`, not per Git remote -name. Renaming a remote or adding a second remote for the same encrypted -repository keeps rollback protection. Every remote for one repository root -shares one floor, so a mirror that lags behind the newest state you have seen -is reported as rolled back. +Each machine keeps one continuity floor per repository root at +`$XDG_STATE_HOME/git-e2ee//continuity.json` (by default under +`~/.local/state`). `git fetch`, `git push`, `init`, `device-add`, +`device-revoke`, `device-list`, `verify`, and `recover` all check the storage +head against this floor and then advance it, so once anything on this machine +has seen a state, storage cannot show an older one. For example, after +`device-revoke` no checkout on this machine will push onto a replayed +pre-revocation state, even one that never fetched the revocation. Moving or +renaming the key file, renaming a Git remote, or adding a second remote or +clone for the same repository keeps the floor. Every remote for one +repository root shares it, so a mirror that lags behind the newest state you +have seen is reported as rolled back. Back up that file together with the key. + +Each clone also records its own import progress under +`.git/git-remote-e2ee//`. That is a cache of work already +done; deleting it only causes a full re-import. ## Git host as storage (carrier-Git backend) @@ -237,11 +239,11 @@ cd /work/my-clone git-e2ee recover origin ``` -The local floor is read from -`/.git/git-remote-e2ee//state.json`; use a repository -that previously fetched or pushed. The remote name does not matter. A fresh clone has no floor, -so recovery reports that freshness and fork identity are unverified and will -not publish. +The local floor is this machine's +`$XDG_STATE_HOME/git-e2ee//continuity.json`, so run recovery +on a machine that previously used the repository. A machine that never did has +no floor, so recovery reports that freshness and fork identity are unverified +and will not publish. Review the offered manifest IDs, generations, verified signers, replays, and any fork report. Recovery is allowed only when the current head is demonstrably diff --git a/docs/spec.md b/docs/spec.md index 345d5e3..180324e 100644 --- a/docs/spec.md +++ b/docs/spec.md @@ -432,23 +432,26 @@ refs, pinning the new head, or publishing a successor. A signed ref advance whose required pack delta is absent is invalid even when every cryptographic check succeeds. -Authenticating a manifest alone MUST NOT advance the continuity floor. A -helper client's head, generation, repository root, and policy-generation floor -advances only when the complete-ref connectivity check succeeds for that -manifest's refs, either after pack import or, during listing, because every -advertised ref is already connected locally; or after that client successfully -publishes its own successor with compare-and-swap. A client MUST NOT rely on -receiving a fetch command to pin a state, because Git omits it when no objects -need to be transferred. A failed or incomplete -import leaves the previous floor intact, so an authenticated but unusable -publication cannot make a later valid successor look like rollback. - -Client-state updates for one repository and remote MUST serialize their -read-modify-write operation across helper processes. The merge MUST preserve -the greatest recorded manifest and policy generations and MUST reject a -same-generation head-ID conflict. The durable state-file replacement remains -atomic; the lock is held while an import or publication checks and advances -the floor. +Each machine (per OS user) keeps one continuity floor per repository root: +repository root, manifest ID and generation, and policy generation. Every +client operation that reads or publishes storage state (listing, fetch, push +and its dry run, initialization, membership listing and changes, +verification, and recovery) MUST hold that floor's exclusive lock, require the +authenticated storage chain to pass through the floor, and, after it succeeds, +advance the floor to the head it accepted or published. The floor never moves +backward; a different head at the floor's generation is a fork. Because every +operation shares the floor, a rollback that hides a state seen by any +operation on the machine, including a revocation published by +`device-revoke`, is rejected by every later operation, whichever checkout runs +it. Holding the lock across validation and publication serializes processes +on one machine. + +The floor records what storage has shown, not what a checkout has imported. +Import progress (imported generation, imported pack IDs, and the verified +frontier below) is kept per checkout and never substitutes for the floor. A +signed head whose packs are missing still advances the floor once it +authenticates and descends from it; import progress stays where it was until +a fetch succeeds. A client MAY satisfy that requirement incrementally. After a successful full or incremental connectivity check, it records the exact checked ref tips as a @@ -464,10 +467,10 @@ Imported pack IDs do not prove that their objects remain in the local object database; deleting a remote-tracking ref and pruning removes them. When the incremental check fails, the client MUST re-import every pack of the authenticated chain and run a full connectivity walk before failing. The -re-import does not move the floor; only a successful full check advances it. +recorded import progress moves only after a successful check. -A missing frontier in a v5 client state requires a full connectivity walk. A -client state from another wire version is not accepted. The incremental rule assumes the trusted +A missing frontier requires a full connectivity walk. Local state in an +unknown format is not accepted. The incremental rule assumes the trusted client's existing object database has not been corrupted outside this protocol; explicit full verification remains available to check local storage. @@ -561,14 +564,13 @@ Unreachable losing objects require later GC. ## 11. Continuity and invitations -Returning clients pin repository root, manifest ID and generation, and policy -generation. The policy ID is not stored separately: the pinned manifest ID -binds it through the manifest's sealed header. Helper continuity pins advance -only after the complete-ref connectivity check succeeds (after import, or -during listing when the advertised refs are already connected locally) or -after a successful own publication, never from authentication alone. They reject rollback, a non-descendant manifest, policy -rollback/sideways movement, root substitution, and generation gaps. Local -client-state updates are serialized and monotonic across concurrent helpers. +Each machine pins, per repository root, the root, manifest ID and generation, +and policy generation (section 9). The policy ID is not stored separately: +the pinned manifest ID binds it through the manifest's sealed header. The +floor advances after any successful validation or publication on the machine. +It rejects rollback, a non-descendant manifest, policy rollback/sideways +movement, root substitution, and generation gaps. Floor updates are serialized +and monotonic across concurrent processes. A fresh device should receive an administrator-authorized invitation checkpoint through an authenticated channel containing: diff --git a/scripts/benchmark-comparison.sh b/scripts/benchmark-comparison.sh index 80496d5..66e99ed 100755 --- a/scripts/benchmark-comparison.sh +++ b/scripts/benchmark-comparison.sh @@ -9,8 +9,7 @@ Compares a local bare Git remote with the git-remote-e2ee filesystem backend. The source repository is cloned into an isolated temporary directory and is never modified. Set BENCH_TOTAL_READERS to an integer of at least 2 to exercise recipient scaling (default: 2). Set BENCH_KEEP_WORK=1 to retain the temporary -directory. Set BENCH_E2EE_PUSH_MODE=native to measure pushes through the Git -remote helper instead of the direct diagnostic CLI (default: direct). +directory. e2ee pushes and fetches run through the Git remote helper. EOF } @@ -21,15 +20,10 @@ fi source_repo=$1 total_readers=${BENCH_TOTAL_READERS:-2} -e2ee_push_mode=${BENCH_E2EE_PUSH_MODE:-direct} if [[ ! $total_readers =~ ^[0-9]+$ ]] || (( total_readers < 2 )); then echo "BENCH_TOTAL_READERS must be an integer of at least 2" >&2 exit 2 fi -if [[ $e2ee_push_mode != direct && $e2ee_push_mode != native ]]; then - echo "BENCH_E2EE_PUSH_MODE must be direct or native" >&2 - exit 2 -fi project_root=$(cd "$(dirname "$0")/.." && pwd -P) results_dir=${2:-$project_root/bench-results/comparison-$(date -u +%Y%m%dT%H%M%SZ)} source_repo=$(cd "$source_repo" && pwd -P) @@ -65,10 +59,11 @@ if [[ ${BENCH_SKIP_BUILD:-0} != 1 ]]; then cargo build --release --bins --manifest-path "$project_root/Cargo.toml" fi git_e2ee="$project_root/target/release/git-e2ee" -[[ -x $git_e2ee ]] || { - echo "release binary not found: $git_e2ee" >&2 +[[ -x $git_e2ee && -x $project_root/target/release/git-remote-e2ee ]] || { + echo "release binaries not found under $project_root/target/release" >&2 exit 1 } +export PATH="$project_root/target/release:$PATH" work_parent=${TMPDIR:-/tmp} work=$(mktemp -d "$work_parent/git-remote-e2ee-compare.XXXXXX") @@ -258,19 +253,13 @@ git -C "$work/e2ee-new-client" config maintenance.auto false "$git_e2ee" keygen --output "$work/repository.key.json" >/dev/null (cd "$work" && "$git_e2ee" init "e2ee::dir:$work/e2ee-storage" \ --key "$work/repository.key.json" >/dev/null) -if [[ $e2ee_push_mode == native ]]; then - git -C "$work/source" remote add private "e2ee::dir:$work/e2ee-storage" - git -C "$work/source" config remote.private.e2ee-key "$work/repository.key.json" - e2ee_push_command=( - env "PATH=$project_root/target/release:$PATH" - git -C "$work/source" push --quiet private "$source_ref:$source_ref" - ) -else - e2ee_push_command=( - "$git_e2ee" push --storage "$work/e2ee-storage" - --key "$work/repository.key.json" --repo "$work/source" --ref "$source_ref" - ) -fi +git -C "$work/source" remote add private "e2ee::dir:$work/e2ee-storage" +git -C "$work/source" config remote.private.e2ee-key "$work/repository.key.json" +e2ee_push_command=(git -C "$work/source" push --quiet private "$source_ref:$source_ref") +for client in e2ee-client e2ee-new-client; do + git -C "$work/$client" remote add benchmark "e2ee::dir:$work/e2ee-storage" +done +git -C "$work/e2ee-client" config remote.benchmark.e2ee-key "$work/repository.key.json" run_phase initial git push "$work/raw.git" "$work/source" \ git -C "$work/source" push --quiet "$work/raw.git" "$source_ref:$source_ref" @@ -280,9 +269,7 @@ run_phase initial git fetch "$work/raw.git" "$work/raw-client" \ git -C "$work/raw-client" fetch --quiet "$work/raw.git" \ "$source_ref:refs/remotes/benchmark/$source_branch" run_phase initial e2ee fetch "$work/e2ee-storage" "$work/e2ee-client" \ - "$git_e2ee" fetch --storage "$work/e2ee-storage" \ - --key "$work/repository.key.json" --repo "$work/e2ee-client" \ - --remote-name benchmark + git -C "$work/e2ee-client" fetch --quiet benchmark repository_root=$(sed -n 's/^[[:space:]]*"repository_root": "\([^"]*\)",$/\1/p' \ "$work/repository.key.json") @@ -314,11 +301,10 @@ if (( total_readers > 2 )); then fi object_snapshot "$work/e2ee-storage" >"$work/packs-after-device-add.sha256" cmp "$work/packs-before-device-add.sha256" "$work/packs-after-device-add.sha256" +git -C "$work/e2ee-new-client" config remote.benchmark.e2ee-key "$active_reader_key" run_phase add_user e2ee fetch-new-user "$work/e2ee-storage" \ "$work/e2ee-new-client" \ - "$git_e2ee" fetch --storage "$work/e2ee-storage" \ - --key "$active_reader_key" --repo "$work/e2ee-new-client" \ - --remote-name benchmark + git -C "$work/e2ee-new-client" fetch --quiet benchmark initial_head=$(git -C "$work/source" rev-parse "$source_ref") new_user_head=$(git -C "$work/e2ee-new-client" \ rev-parse "refs/remotes/benchmark/$source_branch") @@ -334,9 +320,7 @@ for scenario in tiny_change add_10mib modify_10mib_one_byte add_1000_small; do git -C "$work/raw-client" fetch --quiet "$work/raw.git" \ "$source_ref:refs/remotes/benchmark/$source_branch" run_phase "$scenario" e2ee fetch "$work/e2ee-storage" "$work/e2ee-new-client" \ - "$git_e2ee" fetch --storage "$work/e2ee-storage" \ - --key "$active_reader_key" --repo "$work/e2ee-new-client" \ - --remote-name benchmark + git -C "$work/e2ee-new-client" fetch --quiet benchmark done expected=$(git -C "$work/source" rev-parse "$source_ref") @@ -347,7 +331,7 @@ e2ee_received=$(git -C "$work/e2ee-new-client" \ [[ $e2ee_received == "$expected" ]] git -C "$work/raw-client" fsck --full --no-dangling >/dev/null git -C "$work/e2ee-new-client" fsck --full --no-dangling >/dev/null -"$git_e2ee" verify --storage "$work/e2ee-storage" \ +"$git_e2ee" verify "e2ee::dir:$work/e2ee-storage" \ --key "$work/repository.key.json" >/dev/null source_commits=$(git -C "$source_repo" rev-list --count HEAD) @@ -368,7 +352,6 @@ cat >"$results_dir/summary.json" <&2 +[[ -x "$git_e2ee" && -x "$project_root/target/release/git-remote-e2ee" ]] || { + echo "release binaries not found under $project_root/target/release" >&2 exit 1 } +export PATH="$project_root/target/release:$PATH" case $(uname -s) in Darwin) time_style=bsd ;; @@ -134,19 +135,19 @@ git -C "$work/source" remote remove origin mkdir -p "$work/storage" "$git_e2ee" keygen --output "$work/repository.key.json" >/dev/null (cd "$work" && "$git_e2ee" init "e2ee::dir:$work/storage" --key "$work/repository.key.json" >/dev/null) +git -C "$work/source" remote add private "e2ee::dir:$work/storage" +git -C "$work/source" config remote.private.e2ee-key "$work/repository.key.json" -run_phase initial_push "$git_e2ee" push \ - --storage "$work/storage" --key "$work/repository.key.json" \ - --repo "$work/source" --ref "$source_ref" +run_phase initial_push git -C "$work/source" push --quiet private "$source_ref:$source_ref" storage_after_initial=$(storage_bytes) storage_logical_after_initial=$(tree_logical_bytes "$work/storage") pack_logical_after_initial=$(tree_logical_bytes "$work/storage/packs") initial_pack_files=$(find "$work/storage/packs" -type f | wc -l | tr -d ' ') git init -q "$work/destination" -run_phase fresh_fetch "$git_e2ee" fetch \ - --storage "$work/storage" --key "$work/repository.key.json" \ - --repo "$work/destination" --remote-name benchmark +git -C "$work/destination" remote add benchmark "e2ee::dir:$work/storage" +git -C "$work/destination" config remote.benchmark.e2ee-key "$work/repository.key.json" +run_phase fresh_fetch git -C "$work/destination" fetch --quiet benchmark run_phase verify "$git_e2ee" verify \ "e2ee::dir:$work/storage" --key "$work/repository.key.json" @@ -164,12 +165,8 @@ new_head=$(printf 'git-remote-e2ee benchmark increment\n' | \ commit-tree "$tree" -p "$old_head") git -C "$work/source" update-ref "$source_ref" "$new_head" "$old_head" -run_phase incremental_push "$git_e2ee" push \ - --storage "$work/storage" --key "$work/repository.key.json" \ - --repo "$work/source" --ref "$source_ref" -run_phase returning_fetch "$git_e2ee" fetch \ - --storage "$work/storage" --key "$work/repository.key.json" \ - --repo "$work/destination" --remote-name benchmark +run_phase incremental_push git -C "$work/source" push --quiet private "$source_ref:$source_ref" +run_phase returning_fetch git -C "$work/destination" fetch --quiet benchmark destination_ref="refs/remotes/benchmark/$source_branch" [[ $(git -C "$work/destination" rev-parse "$destination_ref") == "$new_head" ]] diff --git a/src/bin/git-remote-e2ee.rs b/src/bin/git-remote-e2ee.rs index a8de70b..ba61851 100644 --- a/src/bin/git-remote-e2ee.rs +++ b/src/bin/git-remote-e2ee.rs @@ -3,8 +3,9 @@ use std::path::{Path, PathBuf}; use std::process::Command; use anyhow::{Context, Result, bail}; +use git_remote_e2ee::continuity::default_state_dir; use git_remote_e2ee::crypto::KeyFile; -use git_remote_e2ee::repository::{EncryptedRepository, PushRejection}; +use git_remote_e2ee::repository::{EncryptedRepository, PushRejection, ReadOnlyDevice, RefChange}; use git_remote_e2ee::storage::{CasConflict, FilesystemStorage, GitStorage, Storage}; use git_remote_e2ee::url::StorageUrl; @@ -27,13 +28,14 @@ fn run() -> Result<()> { let key = KeyFile::read(&key_path)?; let storage = StorageUrl::parse(&url)?; persist_absolute_url(&remote_name, &url, &storage)?; + let state_dir = default_state_dir()?; match storage { StorageUrl::Directory(path) => run_protocol( - EncryptedRepository::new(FilesystemStorage::new(path), key), + EncryptedRepository::new(FilesystemStorage::new(path), key, state_dir), remote_name, ), StorageUrl::Git(remote) => run_protocol( - EncryptedRepository::new(GitStorage::open(&remote)?, key), + EncryptedRepository::new(GitStorage::open(&remote)?, key, state_dir), remote_name, ), } @@ -148,33 +150,27 @@ fn process_push_batch( let (source, destination) = spec .split_once(':') .context("push refspec must contain ':'")?; - let result = if source.is_empty() && dry_run { - repository - .validate_delete_ref_for_remote(local_repository, remote_name, destination) - .map(|_| String::new()) - } else if source.is_empty() { - repository.delete_ref_for_remote(local_repository, remote_name, destination) - } else if dry_run { - repository - .validate_push_update_for_remote( - local_repository, - remote_name, - source, - destination, - force, - ) - .map(|_| String::new()) + let change = if source.is_empty() { + RefChange::Delete } else { - repository.push_update_for_remote( - local_repository, - remote_name, - source, - destination, - force, - ) + RefChange::Update { source, force } + }; + let result = if dry_run { + repository.check_push(local_repository, destination, change) + } else { + repository + .push(local_repository, destination, change) + .map(drop) }; match result { - Ok(_) => writeln!(output, "ok {destination}")?, + Ok(()) => writeln!(output, "ok {destination}")?, + Err(error) if error.is::() => { + let message = format!( + "this device is read-only for {remote_name}; ask an administrator for the write role" + ); + writeln!(output, "error {destination} {message}")?; + eprintln!("e2ee: {destination}: {message}"); + } Err(error) => { writeln!(output, "error {destination} {}", push_status(&error))?; eprintln!("e2ee: {destination}: {}", single_line(&error)); diff --git a/src/continuity.rs b/src/continuity.rs new file mode 100644 index 0000000..b3ffc3b --- /dev/null +++ b/src/continuity.rs @@ -0,0 +1,298 @@ +//! Local state: the machine's continuity floor for each repository root, and +//! the atomic, locked JSON files that hold it and per-checkout progress. + +use std::fs::{self, File, OpenOptions}; +use std::io::{self, Write}; +use std::path::{Path, PathBuf}; + +use anyhow::{Context, Result, bail}; +use fs2::FileExt; +use rand::RngCore; +use rand::rngs::OsRng; +use serde::de::DeserializeOwned; +use serde::{Deserialize, Serialize}; + +use crate::manifest::Manifest; + +const FLOOR_FORMAT: u32 = 1; + +/// `$XDG_STATE_HOME/git-e2ee`, or `~/.local/state/git-e2ee`. Only binaries +/// call this; the library takes its state directory explicitly. +pub fn default_state_dir() -> Result { + let state_home = match std::env::var_os("XDG_STATE_HOME") { + Some(path) if !path.is_empty() => PathBuf::from(path), + _ => PathBuf::from( + std::env::var_os("HOME").context("HOME is unset and XDG_STATE_HOME is unset")?, + ) + .join(".local/state"), + }; + Ok(state_home.join("git-e2ee")) +} + +/// The newest authenticated head this machine has accepted for one +/// repository root. Storage must stay on a chain through it. +#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub(crate) struct ContinuityFloor { + format_version: u32, + pub repository_root: String, + pub head_id: String, + pub generation: u64, + pub policy_generation: u64, +} + +impl ContinuityFloor { + pub fn at(head_id: &str, manifest: &Manifest) -> Self { + Self { + format_version: FLOOR_FORMAT, + repository_root: manifest.repository_root.clone(), + head_id: head_id.to_owned(), + generation: manifest.generation, + policy_generation: manifest.policy_generation, + } + } +} + +/// Exclusive hold on one root's floor. Every operation that reads or +/// publishes storage state holds it from validation through advancement, so +/// processes on this machine serialize. +pub(crate) struct FloorGuard { + path: PathBuf, + _lock: FileLock, +} + +impl FloorGuard { + pub fn acquire(state_dir: &Path, repository_root: &str) -> Result { + let directory = state_dir.join(repository_root); + crate::persist::create_dir_all_durable(&directory) + .with_context(|| format!("create {}", directory.display()))?; + let path = directory.join("continuity.json"); + let lock = FileLock::acquire(&path)?; + Ok(Self { path, _lock: lock }) + } + + pub fn read(&self) -> Result> { + let floor: Option = read_json(&self.path)?; + if let Some(floor) = &floor + && floor.format_version != FLOOR_FORMAT + { + bail!( + "unsupported continuity floor format {}", + floor.format_version + ) + } + Ok(floor) + } + + /// Moves the floor to `next` unless it already stands at a later + /// generation. A different head at the same generation is a fork. + pub fn advance(&self, next: ContinuityFloor) -> Result<()> { + if let Some(previous) = self.read()? { + if previous.repository_root != next.repository_root { + bail!("remote repository root changed") + } + if previous.generation > next.generation { + return Ok(()); + } + if previous.generation == next.generation && previous.head_id != next.head_id { + bail!("remote manifest history forked from the locally pinned head") + } + if next.policy_generation < previous.policy_generation { + bail!("remote policy rolled back") + } + if previous == next { + return Ok(()); + } + } + write_json(&self.path, &next) + } +} + +/// An exclusive advisory lock on a stable sibling of `path`, which writers +/// replace atomically. +pub(crate) struct FileLock { + _file: File, +} + +impl FileLock { + pub fn acquire(path: &Path) -> Result { + let lock_path = path.with_extension("json.lock"); + let file = OpenOptions::new() + .read(true) + .write(true) + .create(true) + .truncate(false) + .open(&lock_path) + .with_context(|| format!("open lock {}", lock_path.display()))?; + FileExt::lock_exclusive(&file).with_context(|| format!("lock {}", path.display()))?; + Ok(Self { _file: file }) + } +} + +pub(crate) fn read_json(path: &Path) -> Result> { + match fs::read(path) { + Ok(contents) => Ok(Some( + serde_json::from_slice(&contents) + .with_context(|| format!("parse {}", path.display()))?, + )), + Err(error) if error.kind() == io::ErrorKind::NotFound => Ok(None), + Err(error) => Err(error).with_context(|| format!("read {}", path.display())), + } +} + +pub(crate) fn write_json(path: &Path, value: &T) -> Result<()> { + let mut random = [0_u8; 8]; + OsRng.fill_bytes(&mut random); + let temporary = path.with_extension(format!("json.{}.tmp", hex::encode(random))); + let mut file = OpenOptions::new() + .write(true) + .create_new(true) + .open(&temporary)?; + file.write_all(&serde_json::to_vec_pretty(value)?)?; + file.sync_all()?; + crate::persist::durability_checkpoint(crate::persist::STAGE_AFTER_FILE_FLUSH); + fs::rename(&temporary, path)?; + crate::persist::durability_checkpoint(crate::persist::STAGE_AFTER_NAME_PUBLISH); + let parent = match path.parent() { + Some(parent) if parent.as_os_str().is_empty() => Path::new("."), + Some(parent) => parent, + None => bail!("state path has no parent"), + }; + crate::persist::sync_directory(parent) + .with_context(|| format!("sync directory {}", parent.display()))?; + Ok(()) +} + +#[cfg(test)] +pub(crate) fn testing_write_floor( + state_dir: &Path, + repository_root: &str, + head_id: &str, + generation: u64, +) -> Result { + let guard = FloorGuard::acquire(state_dir, repository_root)?; + guard.advance(ContinuityFloor { + format_version: FLOOR_FORMAT, + repository_root: repository_root.to_owned(), + head_id: head_id.to_owned(), + generation, + policy_generation: 1, + })?; + Ok(guard.path.clone()) +} + +#[cfg(test)] +mod tests { + use std::env; + use std::fs; + use std::path::Path; + use std::process::{Child, Command}; + use std::thread; + use std::time::{Duration, Instant}; + + use super::{FloorGuard, testing_write_floor}; + + const ROOT: &str = "concurrency-fixture"; + + #[test] + fn concurrent_floor_process_writes_never_lower_or_corrupt_the_floor() { + let temporary = tempfile::tempdir().unwrap(); + let state_dir = temporary.path().join("state"); + testing_write_floor(&state_dir, ROOT, "head-10", 10).unwrap(); + let release = temporary.path().join("release-old-writer"); + let ready = temporary.path().join("old-writer-ready"); + let executable = env::current_exe().unwrap(); + + let older = Command::new(&executable) + .args([ + "--exact", + "continuity::tests::floor_process_write_worker", + "--nocapture", + ]) + .env("E2EE_TEST_FLOOR_STATE_DIR", &state_dir) + .env("E2EE_TEST_FLOOR_GENERATION", "11") + .env("E2EE_TEST_FLOOR_READY", &ready) + .env("E2EE_TEST_FLOOR_RELEASE", &release) + .spawn() + .unwrap(); + let deadline = Instant::now() + Duration::from_secs(10); + while !ready.exists() && Instant::now() < deadline { + thread::sleep(Duration::from_millis(10)); + } + assert!(ready.exists(), "older writer did not reach its wait point"); + + let newer = launch_floor_writer(&executable, &state_dir, 12); + let newer_output = newer.wait_with_output().unwrap(); + assert!( + newer_output.status.success(), + "newer writer failed: {}", + String::from_utf8_lossy(&newer_output.stderr) + ); + fs::write(&release, "continue").unwrap(); + + let older_output = older.wait_with_output().unwrap(); + assert!( + older_output.status.success(), + "older writer failed: {}", + String::from_utf8_lossy(&older_output.stderr) + ); + let floor = FloorGuard::acquire(&state_dir, ROOT) + .unwrap() + .read() + .unwrap() + .unwrap(); + assert_eq!(floor.generation, 12); + assert_eq!(floor.head_id, "head-12"); + } + + #[test] + fn floor_process_write_worker() { + let Ok(state_dir) = env::var("E2EE_TEST_FLOOR_STATE_DIR") else { + return; + }; + let generation = env::var("E2EE_TEST_FLOOR_GENERATION") + .unwrap() + .parse::() + .unwrap(); + if let (Ok(ready), Ok(release)) = ( + env::var("E2EE_TEST_FLOOR_READY"), + env::var("E2EE_TEST_FLOOR_RELEASE"), + ) { + fs::write(&ready, "ready").unwrap(); + let release = Path::new(&release); + let deadline = Instant::now() + Duration::from_secs(10); + while !release.exists() && Instant::now() < deadline { + thread::sleep(Duration::from_millis(10)); + } + assert!(release.exists(), "parent did not release the older writer"); + } + testing_write_floor( + Path::new(&state_dir), + ROOT, + &format!("head-{generation}"), + generation, + ) + .unwrap(); + } + + fn launch_floor_writer(executable: &Path, state_dir: &Path, generation: u64) -> Child { + Command::new(executable) + .args([ + "--exact", + "continuity::tests::floor_process_write_worker", + "--nocapture", + ]) + .env("E2EE_TEST_FLOOR_STATE_DIR", state_dir) + .env("E2EE_TEST_FLOOR_GENERATION", generation.to_string()) + .spawn() + .unwrap() + } + + #[test] + fn a_different_head_at_the_floor_generation_is_a_fork() { + let temporary = tempfile::tempdir().unwrap(); + testing_write_floor(temporary.path(), ROOT, "head-a", 3).unwrap(); + let error = testing_write_floor(temporary.path(), ROOT, "head-b", 3).unwrap_err(); + assert!(format!("{error:#}").contains("forked"), "{error:#}"); + } +} diff --git a/src/lib.rs b/src/lib.rs index 2cc2ffd..dd312a7 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -1,3 +1,4 @@ +pub mod continuity; pub mod crypto; pub mod error; pub mod git; diff --git a/src/main.rs b/src/main.rs index 27dcaa5..268eca3 100644 --- a/src/main.rs +++ b/src/main.rs @@ -1,14 +1,13 @@ -use std::path::{Path, PathBuf}; +use std::path::PathBuf; use std::process::Command as Process; use anyhow::{Context, Result, bail}; use clap::{Parser, Subcommand, ValueEnum}; +use git_remote_e2ee::continuity::default_state_dir; use git_remote_e2ee::crypto::{KeyFile, PublicDevice}; use git_remote_e2ee::policy::{DeviceRoles, validate_device_label}; -use git_remote_e2ee::repository::{ - EncryptedRepository, RecoveryOptions, RecoveryReport, admin_state_path, -}; +use git_remote_e2ee::repository::{EncryptedRepository, RecoveryOptions, RecoveryReport}; use git_remote_e2ee::storage::{FilesystemStorage, GitStorage}; use git_remote_e2ee::url::StorageUrl; @@ -116,32 +115,6 @@ enum Command { #[arg(long, requires = "publish")] accept_stale_floor: bool, }, - /// Push without the remote helper; kept for benchmarks - #[command(hide = true)] - Push { - #[arg(long)] - storage: PathBuf, - #[arg(long)] - key: PathBuf, - #[arg(long, default_value = ".")] - repo: PathBuf, - #[arg(long = "ref")] - reference: String, - #[arg(long)] - force: bool, - }, - /// Fetch without the remote helper; kept for benchmarks - #[command(hide = true)] - Fetch { - #[arg(long)] - storage: PathBuf, - #[arg(long)] - key: PathBuf, - #[arg(long, default_value = ".")] - repo: PathBuf, - #[arg(long, default_value = "origin")] - remote_name: String, - }, } struct Target { @@ -185,13 +158,16 @@ macro_rules! with_repository { ($target:expr, |$repository:ident| $body:expr) => {{ let target: &Target = $target; let key = KeyFile::read(&target.key)?; + let state_dir = default_state_dir()?; match &target.storage { StorageUrl::Directory(path) => { - let $repository = EncryptedRepository::new(FilesystemStorage::new(path), key); + let $repository = + EncryptedRepository::new(FilesystemStorage::new(path), key, state_dir); $body } StorageUrl::Git(remote) => { - let $repository = EncryptedRepository::new(GitStorage::open(remote)?, key); + let $repository = + EncryptedRepository::new(GitStorage::open(remote)?, key, state_dir); $body } } @@ -265,10 +241,8 @@ fn main() -> Result<()> { storage, key: std::path::absolute(key)?, }; - let pin = admin_pin_path(&target.key)?; with_repository!(&target, |repository| { repository.initialize(&label)?; - repository.pin_admin_state(&pin)?; }); Ok(target) })(); @@ -310,12 +284,11 @@ fn main() -> Result<()> { key, } => { let target = Target::resolve(&target, key)?; - let pin = admin_pin_path(&target.key)?; let public = PublicDevice::read(&device)?; let device_id = public.device_id.clone(); let roles = DeviceRoles::from(role); let (manifest, policy) = with_repository!(&target, |repository| { - repository.add_device(public, &label, roles, &pin)? + repository.add_device(public, &label, roles)? }); println!("added {device_id} ({label}); manifest {manifest}; policy {policy}"); } @@ -325,17 +298,14 @@ fn main() -> Result<()> { key, } => { let target = Target::resolve(&target, key)?; - let pin = admin_pin_path(&target.key)?; - let (manifest, policy) = with_repository!(&target, |repository| { - repository.revoke_device(&device_id, &pin)? - }); + let (manifest, policy) = with_repository!(&target, |repository| repository + .revoke_device(&device_id)?); println!("revoked {device_id}; manifest {manifest}; policy {policy}"); } Command::DeviceList { target, key } => { let target = Target::resolve(&target, key)?; - let pin = admin_pin_path(&target.key)?; let own_id = KeyFile::read(&target.key)?.device_id()?; - let devices = with_repository!(&target, |repository| repository.list_devices(&pin)?); + let devices = with_repository!(&target, |repository| repository.list_devices()?); for device in devices { println!( "{}{} {} role={} status={}", @@ -371,28 +341,6 @@ fn main() -> Result<()> { with_repository!(&target, |repository| repository.recover(&repo, options)?); print_recovery_report(&report); } - Command::Push { - storage, - key, - repo, - reference, - force, - } => { - let repository = - EncryptedRepository::new(FilesystemStorage::new(storage), KeyFile::read(&key)?); - println!("{}", repository.push_ref(&repo, &reference, force)?); - } - Command::Fetch { - storage, - key, - repo, - remote_name, - } => { - let repository = - EncryptedRepository::new(FilesystemStorage::new(storage), KeyFile::read(&key)?); - let manifest = repository.fetch_into(&repo, &remote_name)?; - println!("fetched generation {}", manifest.generation); - } } Ok(()) } @@ -422,10 +370,6 @@ fn default_owner_label() -> String { .unwrap_or_else(|| "owner".to_owned()) } -fn admin_pin_path(key: &Path) -> Result { - admin_state_path(&KeyFile::read(key)?.repository_root) -} - fn inside_git_repository() -> Result { Ok(Process::new("git") .args(["rev-parse", "--git-dir"]) diff --git a/src/persist.rs b/src/persist.rs index 52390be..a6b497e 100644 --- a/src/persist.rs +++ b/src/persist.rs @@ -212,8 +212,8 @@ mod tests { STAGE_AFTER_ACKNOWLEDGE, STAGE_AFTER_FILE_FLUSH, STAGE_AFTER_NAME_PUBLISH, create_dir_all_durable, set_durability_hook, sync_directory, }; + use crate::continuity::testing_write_floor; use crate::crypto::object_id; - use crate::repository::testing_write_client_state; use crate::storage::{FilesystemStorage, ObjectKind, Storage}; const CLIENT_V1: &[u8] = b"e2ee-durability-client-v1"; @@ -273,7 +273,9 @@ mod tests { } fn client_path(fixture: &Path) -> PathBuf { - op_dir(fixture, "client_state").join("state.json") + op_dir(fixture, "client_state") + .join(ROOT_LABEL) + .join("continuity.json") } fn object_file(root: &Path, id: &str) -> PathBuf { @@ -305,11 +307,7 @@ mod tests { .get("generation") .and_then(|item| item.as_u64()) .ok_or("client state is missing generation")?; - let imported = value - .get("imported_generation") - .and_then(|item| item.as_u64()) - .ok_or("client state is missing imported_generation")?; - if imported != generation || root != ROOT_LABEL { + if root != ROOT_LABEL { return Err("client state fields are not a complete baseline or successor".into()); } let (v1, v2) = client_ids(); @@ -380,7 +378,7 @@ mod tests { let dir = op_dir(fixture, op); reset_dir(&dir); let (v1, _) = client_ids(); - testing_write_client_state(&dir.join("state.json"), &v1, 1).unwrap(); + testing_write_floor(&dir, ROOT_LABEL, &v1, 1).unwrap(); } "head" => { let dir = op_dir(fixture, op); @@ -418,7 +416,7 @@ mod tests { match op { "client_state" => { let (_, v2) = client_ids(); - testing_write_client_state(&client_path(fixture), &v2, 2).unwrap(); + testing_write_floor(&op_dir(fixture, op), ROOT_LABEL, &v2, 2).unwrap(); } "head" => { let (v1, v2) = head_ids(); @@ -617,8 +615,9 @@ mod tests { let path = client_path(fixture.path()); let second = object_id(b"e2ee-durability-client-v2"); let third = object_id(b"e2ee-durability-client-v3"); - testing_write_client_state(&path, &second, 2).unwrap(); - testing_write_client_state(&path, &third, 3).unwrap(); + let state_dir = op_dir(fixture.path(), "client_state"); + testing_write_floor(&state_dir, ROOT_LABEL, &second, 2).unwrap(); + testing_write_floor(&state_dir, ROOT_LABEL, &third, 3).unwrap(); let value: serde_json::Value = serde_json::from_slice(&fs::read(&path).unwrap()).unwrap(); assert_eq!(value["head_id"], third); assert_eq!(value["generation"], 3); diff --git a/src/repository.rs b/src/repository.rs index a0e5985..bb6a374 100644 --- a/src/repository.rs +++ b/src/repository.rs @@ -1,15 +1,13 @@ use std::collections::{BTreeMap, BTreeSet, HashSet}; -use std::fs::{self, File, OpenOptions}; -use std::io::{self, Write}; +use std::fs; +use std::io; use std::path::{Path, PathBuf}; use anyhow::{Context, Result, bail}; -use fs2::FileExt; -use rand::RngCore; -use rand::rngs::OsRng; use serde::{Deserialize, Serialize}; use sha2::{Digest, Sha256}; +use crate::continuity::{ContinuityFloor, FileLock, FloorGuard, read_json, write_json}; use crate::crypto::{ KeyFile, PublicDevice, SecretKey, SubkeyKind, derive_subkey, object_id, open_pack_stream, random_key, seal_pack_stream, @@ -59,6 +57,11 @@ impl PushRejection { } } +/// The device may read but not publish. +#[derive(Debug, thiserror::Error)] +#[error("this device is read-only; ask an administrator for the write role")] +pub struct ReadOnlyDevice; + #[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "snake_case")] pub enum RecoveryClass { @@ -116,26 +119,53 @@ pub struct RecoveryOptions<'a> { pub accept_stale_floor: bool, } +/// This checkout's import progress for one repository root. It records work +/// already done here; continuity is the machine floor's job. #[derive(Clone, Debug, Default, Serialize, Deserialize)] -struct ClientState { - #[serde(default)] - format_version: u64, - #[serde(default)] +#[serde(deny_unknown_fields)] +struct CheckoutProgress { + format_version: u32, packs: Vec, - // Continuity floor: imported and connectivity-verified, or our own CAS winner. - head_id: Option, - generation: Option, // The last generation whose pack deltas and advertised refs we imported and checked. - #[serde(default)] imported_generation: Option, - #[serde(default)] - policy_generation: Option, - #[serde(default)] - repository_root: Option, - #[serde(default)] verified_refs: BTreeMap, } +const CHECKOUT_PROGRESS_FORMAT: u32 = 1; + +/// A locked view of `.git/git-remote-e2ee//` in one checkout. Always +/// acquired after the machine floor. +struct Checkout { + directory: PathBuf, + _lock: FileLock, +} + +impl Checkout { + fn progress_path(&self) -> PathBuf { + self.directory.join("progress.json") + } + + fn read(&self) -> Result { + let Some(progress) = read_json::(&self.progress_path())? else { + return Ok(CheckoutProgress::default()); + }; + if progress.format_version != CHECKOUT_PROGRESS_FORMAT { + bail!( + "unsupported checkout progress format {}", + progress.format_version + ) + } + Ok(progress) + } + + fn write(&self, mut progress: CheckoutProgress) -> Result<()> { + progress.format_version = CHECKOUT_PROGRESS_FORMAT; + progress.packs.sort(); + progress.packs.dedup(); + write_json(&self.progress_path(), &progress) + } +} + #[derive(Clone)] struct OpenedState { id: String, @@ -151,8 +181,11 @@ struct CommitHead<'a> { observation: Option<&'a HeadObservation>, } -struct ClientStateLock { - _file: File, +/// What a push does to one destination ref. +#[derive(Clone, Copy, Debug)] +pub enum RefChange<'a> { + Update { source: &'a str, force: bool }, + Delete, } struct RecoveryBudget { @@ -188,48 +221,86 @@ impl RecoveryBudget { } } -impl ClientStateLock { - fn acquire(path: &Path) -> Result { - // Lock a stable sibling: persist_client_state atomically replaces the state file. - let lock_path = path.with_extension("json.lock"); - let file = OpenOptions::new() - .read(true) - .write(true) - .create(true) - .truncate(false) - .open(&lock_path) - .with_context(|| format!("open client state lock {}", lock_path.display()))?; - FileExt::lock_exclusive(&file) - .with_context(|| format!("lock client state {}", path.display()))?; - Ok(Self { _file: file }) - } -} - pub struct EncryptedRepository { storage: S, key: KeyFile, + state_dir: PathBuf, } impl EncryptedRepository { - pub fn new(storage: S, key: KeyFile) -> Self { - Self { storage, key } + /// `state_dir` holds this machine's continuity floors, one per repository + /// root; binaries pass `continuity::default_state_dir()`. + pub fn new(storage: S, key: KeyFile, state_dir: impl Into) -> Self { + Self { + storage, + key, + state_dir: state_dir.into(), + } + } + + fn lock_floor(&self) -> Result { + FloorGuard::acquire(&self.state_dir, &self.key.repository_root) + } + + /// Continuity progress is keyed by repository root rather than by Git + /// remote name, so `git remote rename` or a second remote for the same + /// encrypted repository shares it. + fn open_checkout(&self, repo: &Path) -> Result { + git::ensure_repository(repo)?; + let directory = git_state_directory(repo)? + .join("git-remote-e2ee") + .join(&self.key.repository_root); + crate::persist::create_dir_all_durable(&directory) + .with_context(|| format!("create {}", directory.display()))?; + let lock = FileLock::acquire(&directory.join("progress.json"))?; + Ok(Checkout { + directory, + _lock: lock, + }) + } + + /// Authenticates the storage chain and requires it to pass through the + /// machine floor. Another process on this machine may have advanced the + /// floor after this storage view was opened, so a view that fails the + /// floor is refreshed once before the failure counts as a rollback. + fn pinned_chain(&self, floor: &FloorGuard) -> Result> { + let floor = floor.read()?; + let chain = self.current_chain()?; + match validate_pinned_head(&chain, floor.as_ref()) { + Ok(()) => Ok(chain), + Err(error) if !self.storage.refresh()? => Err(error), + Err(_) => { + let chain = self.current_chain()?; + validate_pinned_head(&chain, floor.as_ref())?; + Ok(chain) + } + } } pub fn initialize(&self, owner_label: &str) -> Result { + let floor = self.lock_floor()?; if self.storage.read_head()?.is_some() { bail!("encrypted repository is already initialized") } + if floor.read()?.is_some() { + bail!( + "this machine already pins repository root {}; refusing to initialize other storage for it", + self.key.repository_root + ) + } let (policy, policy_bytes) = PolicyState::genesis(&self.key, owner_label)?; let generation_key = random_key(); let genesis = Manifest::genesis(self.key.repository_root.clone(), &policy, policy_bytes); - self.commit_manifest( + let id = self.commit_manifest( None, &policy, None, &generation_key, ManifestAuthorization::PolicyTransition, - genesis, - ) + genesis.clone(), + )?; + floor.advance(ContinuityFloor::at(&id, &genesis))?; + Ok(id) } pub fn add_device( @@ -237,142 +308,101 @@ impl EncryptedRepository { public: PublicDevice, label: &str, roles: DeviceRoles, - pin_path: &Path, ) -> Result<(String, String)> { - let _pin_lock = ClientStateLock::acquire(pin_path)?; - let chain = self.current_chain()?; - let current = chain.first().context("manifest chain is empty")?.clone(); - self.validate_pinned_head(&chain, &read_client_state(pin_path)?)?; - self.require_admin(¤t.policy)?; - if current.policy.device(&public.device_id).is_some() { - bail!("device already exists in policy") - } - let mut devices = current.policy.body.devices.clone(); - devices.push(DeviceRecord { - public, - label: label.to_owned(), - roles, - revoked_at: None, - }); - let (next_policy, policy_bytes) = - PolicyState::successor(¤t.policy, devices, &self.key)?; - let next_key = random_key(); - let next = transition_manifest(¤t, &next_policy, policy_bytes, &next_key)?; - let result = - self.publish_policy_transition(¤t, &next_policy, &next_key, next.clone())?; - write_client_state_locked( - pin_path, - &HashSet::new(), - &result.0, - &next, - &BTreeMap::new(), - )?; - Ok(result) + self.change_policy(|current| { + if current.policy.device(&public.device_id).is_some() { + bail!("device already exists in policy") + } + let mut devices = current.policy.body.devices.clone(); + devices.push(DeviceRecord { + public, + label: label.to_owned(), + roles, + revoked_at: None, + }); + Ok(devices) + }) + } + + pub fn revoke_device(&self, device_id: &str) -> Result<(String, String)> { + self.change_policy(|current| { + let mut devices = current.policy.body.devices.clone(); + let target = devices + .iter_mut() + .find(|device| device.public.device_id == device_id && device.active()) + .context("device is not active in the current policy")?; + target.revoked_at = Some( + current + .policy + .body + .generation + .checked_add(1) + .context("policy generation overflow")?, + ); + if !devices + .iter() + .any(|device| device.active() && device.roles.reader) + { + bail!("cannot revoke the last active reader") + } + if !devices + .iter() + .any(|device| device.active() && device.roles.administrator) + { + bail!("cannot revoke the last active administrator") + } + Ok(devices) + }) } - pub fn revoke_device(&self, device_id: &str, pin_path: &Path) -> Result<(String, String)> { - let _pin_lock = ClientStateLock::acquire(pin_path)?; - let chain = self.current_chain()?; - let current = chain.first().context("manifest chain is empty")?.clone(); - self.validate_pinned_head(&chain, &read_client_state(pin_path)?)?; + fn change_policy( + &self, + next_devices: impl FnOnce(&OpenedState) -> Result>, + ) -> Result<(String, String)> { + let floor = self.lock_floor()?; + let chain = self.pinned_chain(&floor)?; + let current = chain.first().context("manifest chain is empty")?; self.require_admin(¤t.policy)?; - let mut devices = current.policy.body.devices.clone(); - let target = devices - .iter_mut() - .find(|device| device.public.device_id == device_id && device.active()) - .context("device is not active in the current policy")?; - target.revoked_at = Some( - current - .policy - .body - .generation - .checked_add(1) - .context("policy generation overflow")?, - ); - if !devices - .iter() - .any(|device| device.active() && device.roles.reader) - { - bail!("cannot revoke the last active reader") - } - if !devices - .iter() - .any(|device| device.active() && device.roles.administrator) - { - bail!("cannot revoke the last active administrator") - } + let devices = next_devices(current)?; let (next_policy, policy_bytes) = PolicyState::successor(¤t.policy, devices, &self.key)?; let next_key = random_key(); - let next = transition_manifest(¤t, &next_policy, policy_bytes, &next_key)?; - let result = - self.publish_policy_transition(¤t, &next_policy, &next_key, next.clone())?; - write_client_state_locked( - pin_path, - &HashSet::new(), - &result.0, - &next, - &BTreeMap::new(), + let next = transition_manifest(current, &next_policy, policy_bytes, &next_key)?; + next_policy.validate_successor(¤t.policy)?; + let manifest_id = self.commit_manifest( + Some(current), + &next_policy, + Some(¤t.policy), + &next_key, + ManifestAuthorization::PolicyTransition, + next.clone(), )?; - Ok(result) - } - - /// Continuity state is keyed by repository root rather than by Git remote - /// name, so `git remote rename` or a second remote for the same encrypted - /// repository cannot silently start from an empty pin. - fn client_state_path(&self, repo: &Path) -> Result { - let directory = git_state_directory(repo)? - .join("git-remote-e2ee") - .join(&self.key.repository_root); - crate::persist::create_dir_all_durable(&directory) - .with_context(|| format!("create {}", directory.display()))?; - Ok(directory.join("state.json")) - } - - pub fn pin_admin_state(&self, pin_path: &Path) -> Result<()> { - self.advance_admin_pin(pin_path).map(drop) + floor.advance(ContinuityFloor::at(&manifest_id, &next))?; + Ok((manifest_id, next_policy.id)) } - /// Returns the current membership only after the head passes the local - /// continuity pin, so storage cannot hide a revocation by serving an - /// older, validly signed policy. - pub fn list_devices(&self, pin_path: &Path) -> Result> { - Ok(self.advance_admin_pin(pin_path)?.policy.body.devices) + /// Returns the current membership only after the head passes the machine + /// floor, so storage cannot hide a revocation by serving an older, + /// validly signed policy. + pub fn list_devices(&self) -> Result> { + Ok(self.pinned_head()?.policy.body.devices) } - fn advance_admin_pin(&self, pin_path: &Path) -> Result { - let _pin_lock = ClientStateLock::acquire(pin_path)?; - let chain = self.current_chain()?; - let current = chain.first().context("manifest chain is empty")?; - let previous = read_client_state(pin_path)?; - self.validate_pinned_head(&chain, &previous)?; - write_client_state_locked( - pin_path, - &HashSet::new(), - ¤t.id, - ¤t.manifest, - &BTreeMap::new(), - )?; - Ok(current.clone()) + /// The authenticated storage head, after it passes the machine floor. + pub fn current_manifest(&self) -> Result<(String, Manifest)> { + let head = self.pinned_head()?; + Ok((head.id, head.manifest)) } - fn publish_policy_transition( - &self, - current: &OpenedState, - policy: &PolicyState, - generation_key: &[u8; 32], - manifest: Manifest, - ) -> Result<(String, String)> { - policy.validate_successor(¤t.policy)?; - let manifest_id = self.commit_manifest( - Some(current), - policy, - Some(¤t.policy), - generation_key, - ManifestAuthorization::PolicyTransition, - manifest, - )?; - Ok((manifest_id, policy.id.clone())) + fn pinned_head(&self) -> Result { + let floor = self.lock_floor()?; + let current = self + .pinned_chain(&floor)? + .into_iter() + .next() + .context("manifest chain is empty")?; + floor.advance(ContinuityFloor::at(¤t.id, ¤t.manifest))?; + Ok(current) } fn require_admin(&self, policy: &PolicyState) -> Result<()> { @@ -382,82 +412,22 @@ impl EncryptedRepository { Ok(()) } - pub fn push_ref(&self, repo: &Path, reference: &str, force: bool) -> Result { - self.push_update(repo, reference, reference, force) - } - - pub fn push_update( - &self, - repo: &Path, - source_ref: &str, - destination_ref: &str, - force: bool, - ) -> Result { - self.push_update_inner(repo, None, source_ref, destination_ref, force) - } - - pub fn push_update_for_remote( + /// Publishes one ref change as the next generation and returns the + /// manifest ID that holds it. + pub fn push( &self, repo: &Path, - remote_name: &str, - source_ref: &str, destination_ref: &str, - force: bool, + change: RefChange<'_>, ) -> Result { - self.push_update_inner(repo, Some(remote_name), source_ref, destination_ref, force) - } - - pub fn delete_ref(&self, repo: &Path, destination_ref: &str) -> Result { - self.change_ref_inner(repo, None, None, destination_ref, false) - } - - pub fn delete_ref_for_remote( - &self, - repo: &Path, - remote_name: &str, - destination_ref: &str, - ) -> Result { - self.change_ref_inner(repo, Some(remote_name), None, destination_ref, false) - } - - fn push_update_inner( - &self, - repo: &Path, - remote_name: Option<&str>, - source_ref: &str, - destination_ref: &str, - force: bool, - ) -> Result { - self.change_ref_inner(repo, remote_name, Some(source_ref), destination_ref, force) - } - - fn change_ref_inner( - &self, - repo: &Path, - remote_name: Option<&str>, - source_ref: Option<&str>, - destination_ref: &str, - force: bool, - ) -> Result { - let state_path = remote_name - .map(|_| self.client_state_path(repo)) - .transpose()?; - let _state_lock = state_path - .as_deref() - .map(ClientStateLock::acquire) - .transpose()?; - let (current, next_object, non_fast_forward) = self.preflight_ref_change( - repo, - remote_name, - state_path.as_deref(), - source_ref, - destination_ref, - force, - )?; + let floor = self.lock_floor()?; + let (current, next_object, non_fast_forward) = + self.preflight_ref_change(repo, &floor, destination_ref, change)?; if next_object .as_ref() .is_some_and(|next| current.manifest.refs.get(destination_ref) == Some(next)) { + floor.advance(ContinuityFloor::at(¤t.id, ¤t.manifest))?; return Ok(current.id); } @@ -545,102 +515,54 @@ impl EncryptedRepository { ManifestAuthorization::Writer, next.clone(), )?; - if let Some(state_path) = state_path.as_deref() { - self.pin_manifest_locked( - state_path, - &next_id, - &next, - Some((repo, ¤t.manifest.refs)), - )?; + floor.advance(ContinuityFloor::at(&next_id, &next))?; + let checkout = self.open_checkout(repo)?; + let mut progress = checkout.read()?; + if progress.verified_refs == current.manifest.refs && git::has_complete_object_graph(repo) { + // The completed local pack walk covers new reachability only beyond + // refs already verified by this checkout. Shallow and promisor repos + // can omit history, so they must not advance this frontier. + progress.verified_refs = next.refs.clone(); + checkout.write(progress)?; } Ok(next_id) } - pub fn validate_push_update( - &self, - repo: &Path, - source_ref: &str, - destination_ref: &str, - force: bool, - ) -> Result<()> { - self.preflight_ref_change(repo, None, None, Some(source_ref), destination_ref, force)?; - Ok(()) - } - - pub fn validate_push_update_for_remote( - &self, - repo: &Path, - remote_name: &str, - source_ref: &str, - destination_ref: &str, - force: bool, - ) -> Result<()> { - let state_path = self.client_state_path(repo)?; - let _state_lock = ClientStateLock::acquire(&state_path)?; - self.preflight_ref_change( - repo, - Some(remote_name), - Some(&state_path), - Some(source_ref), - destination_ref, - force, - )?; - Ok(()) - } - - pub fn validate_delete_ref_for_remote( + /// Runs every check `push` would, without publishing (`git push --dry-run`). + pub fn check_push( &self, repo: &Path, - remote_name: &str, destination_ref: &str, + change: RefChange<'_>, ) -> Result<()> { - let state_path = self.client_state_path(repo)?; - let _state_lock = ClientStateLock::acquire(&state_path)?; - self.preflight_ref_change( - repo, - Some(remote_name), - Some(&state_path), - None, - destination_ref, - false, - )?; - Ok(()) + let floor = self.lock_floor()?; + let (current, _, _) = self.preflight_ref_change(repo, &floor, destination_ref, change)?; + floor.advance(ContinuityFloor::at(¤t.id, ¤t.manifest)) } fn preflight_ref_change( &self, repo: &Path, - remote_name: Option<&str>, - state_path: Option<&Path>, - source_ref: Option<&str>, + floor: &FloorGuard, destination_ref: &str, - force: bool, + change: RefChange<'_>, ) -> Result<(OpenedState, Option, bool)> { git::ensure_repository(repo)?; let ref_kind = git::validate_inner_ref(repo, destination_ref)?; - let chain = self.current_chain()?; + let chain = self.pinned_chain(floor)?; let current = chain.first().context("manifest chain is empty")?.clone(); let device_id = self.key.device_id()?; if !current.policy.is_active_writer(&device_id) { if current.policy.is_active_reader(&device_id) { - if let Some(remote_name) = remote_name { - bail!( - "this device is read-only for {remote_name}; ask an administrator for the write role" - ) - } - bail!("this device is read-only; ask an administrator for the write role") + bail!(ReadOnlyDevice) } bail!("device is not an active writer") } - if remote_name.is_some() { - let state_path = state_path.context("remote client state path is missing")?; - self.validate_remote_continuity(state_path, &chain)?; - } for reference in current.manifest.refs.keys() { git::validate_inner_ref(repo, reference)?; } let old_object = current.manifest.refs.get(destination_ref); - let Some(source_ref) = source_ref else { + let RefChange::Update { source, force } = change else { if old_object.is_none() { bail!("remote ref {destination_ref} does not exist") } @@ -652,7 +574,7 @@ impl EncryptedRepository { return Ok((current, None, false)); }; - let next_object = git::resolve_ref(repo, source_ref)?; + let next_object = git::resolve_ref(repo, source)?; if old_object == Some(&next_object) { return Ok((current, Some(next_object), false)); } @@ -678,11 +600,10 @@ impl EncryptedRepository { Ok((current, Some(next_object), non_fast_forward)) } + /// Imports the current generation and points `refs/remotes//*` + /// and tags at its refs, as `git fetch` through the helper would. pub fn fetch_into(&self, repo: &Path, remote_name: &str) -> Result { - git::ensure_repository(repo)?; - let state_path = self.client_state_path(repo)?; - let _state_lock = ClientStateLock::acquire(&state_path)?; - let manifest = self.import_packs_locked(repo, &state_path)?; + let manifest = self.import_packs(repo)?; for (reference, object) in &manifest.refs { match git::validate_inner_ref(repo, reference)? { git::InnerRefKind::Branch => { @@ -696,77 +617,64 @@ impl EncryptedRepository { } pub fn observe_manifest(&self, repo: &Path) -> Result { - git::ensure_repository(repo)?; - let state_path = self.client_state_path(repo)?; - let _state_lock = ClientStateLock::acquire(&state_path)?; - let chain = self.current_chain()?; - let current = chain.first().context("manifest chain is empty")?.clone(); + let floor = self.lock_floor()?; + let checkout = self.open_checkout(repo)?; + let chain = self.pinned_chain(&floor)?; + let current = chain.first().context("manifest chain is empty")?; for reference in current.manifest.refs.keys() { git::validate_inner_ref(repo, reference)?; } - let state = read_client_state(&state_path)?; - self.validate_pinned_head(&chain, &state)?; + floor.advance(ContinuityFloor::at(¤t.id, ¤t.manifest))?; // Git skips the helper's fetch command when every advertised object is // already local, for example after a membership change, a deletion, or - // a tag that points at an existing object. Advance the floor here only + // a tag that points at an existing object. Record the import here only // when the advertised refs are fully connected locally, which is the - // same check a fetch would finish with; authentication alone never - // moves it. Earlier packs are not needed once these refs are - // connected, because later packs are built against these tips. - if git::ensure_refs_connected_since(repo, ¤t.manifest.refs, &state.verified_refs) + // same check a fetch would finish with. Earlier packs are not needed + // once these refs are connected, because later packs are built + // against these tips. + let mut progress = checkout.read()?; + if git::ensure_refs_connected_since(repo, ¤t.manifest.refs, &progress.verified_refs) .is_ok() { - let packs: HashSet = state.packs.iter().cloned().collect(); - write_client_state_locked( - &state_path, - &packs, - ¤t.id, - ¤t.manifest, - ¤t.manifest.refs, - )?; + progress.imported_generation = Some(current.manifest.generation); + progress.verified_refs = current.manifest.refs.clone(); + checkout.write(progress)?; } Ok(current.manifest.clone()) } pub fn import_packs(&self, repo: &Path) -> Result { - git::ensure_repository(repo)?; - let state_path = self.client_state_path(repo)?; - let _state_lock = ClientStateLock::acquire(&state_path)?; - self.import_packs_locked(repo, &state_path) - } - - fn import_packs_locked(&self, repo: &Path, state_path: &Path) -> Result { - let chain = self.current_chain()?; - let current = chain.first().context("manifest chain is empty")?.clone(); + let floor = self.lock_floor()?; + let checkout = self.open_checkout(repo)?; + let chain = self.pinned_chain(&floor)?; + let current = chain.first().context("manifest chain is empty")?; for reference in current.manifest.refs.keys() { git::validate_inner_ref(repo, reference)?; } - let state = read_client_state(state_path)?; - self.validate_pinned_head(&chain, &state)?; - - let mut imported: HashSet = state.packs.into_iter().collect(); - self.import_chain_packs(repo, &chain, state.imported_generation, &mut imported)?; + let progress = checkout.read()?; + let mut imported: HashSet = progress.packs.into_iter().collect(); + self.import_chain_packs(repo, &chain, progress.imported_generation, &mut imported)?; let connectivity_timer = trace::Span::new("inner_ref_connectivity_check"); - if git::ensure_refs_connected_since(repo, ¤t.manifest.refs, &state.verified_refs) + if git::ensure_refs_connected_since(repo, ¤t.manifest.refs, &progress.verified_refs) .is_err() { // Imported pack IDs are cache state, not proof that their objects // are still local: deleting a remote-tracking ref and pruning // removes them. Re-import the whole authenticated chain and check - // the full graph; the pinned floor is unchanged until that passes. + // the full graph; recorded progress is unchanged until that passes. imported.clear(); self.import_chain_packs(repo, &chain, None, &mut imported)?; git::ensure_refs_connected_since(repo, ¤t.manifest.refs, &BTreeMap::new())?; } drop(connectivity_timer); - write_client_state_locked( - state_path, - &imported, - ¤t.id, - ¤t.manifest, - ¤t.manifest.refs, - )?; - Ok(current.manifest) + floor.advance(ContinuityFloor::at(¤t.id, ¤t.manifest))?; + checkout.write(CheckoutProgress { + format_version: CHECKOUT_PROGRESS_FORMAT, + packs: imported.into_iter().collect(), + imported_generation: Some(current.manifest.generation), + verified_refs: current.manifest.refs.clone(), + })?; + Ok(current.manifest.clone()) } /// Imports, oldest first, the packs of every chain entry newer than @@ -819,66 +727,11 @@ impl EncryptedRepository { Ok(()) } - fn validate_remote_continuity(&self, state_path: &Path, chain: &[OpenedState]) -> Result<()> { - let state = read_client_state(state_path)?; - self.validate_pinned_head(chain, &state) - } - - fn pin_manifest_locked( - &self, - state_path: &Path, - head_id: &str, - manifest: &Manifest, - published_from: Option<(&Path, &BTreeMap)>, - ) -> Result<()> { - let mut state = read_client_state(state_path)?; - if let Some((repo, previous_refs)) = published_from - && state.verified_refs == *previous_refs - && git::has_complete_object_graph(repo) - { - // The completed local pack walk covers new reachability only beyond - // refs already verified by this client. Shallow and promisor repos - // can omit history, so they must not advance this frontier. - state.verified_refs = manifest.refs.clone(); - } - write_published_client_state_locked(state_path, state, head_id, manifest) - } - - fn validate_pinned_head(&self, chain: &[OpenedState], state: &ClientState) -> Result<()> { - let current = chain.first().context("manifest chain is empty")?; - if let Some(root) = &state.repository_root - && root != ¤t.manifest.repository_root - { - bail!("remote repository root changed") - } - if let Some(policy_generation) = state.policy_generation - && current.manifest.policy_generation < policy_generation - { - bail!("remote policy rolled back") - } - let (Some(pinned_id), Some(pinned_generation)) = - (state.head_id.as_deref(), state.generation) - else { - return Ok(()); - }; - if current.manifest.generation < pinned_generation { - bail!( - "remote manifest rolled back from generation {pinned_generation} to {}", - current.manifest.generation - ) - } - let pinned = chain - .iter() - .find(|entry| entry.manifest.generation == pinned_generation) - .context("remote manifest chain ended before pinned generation")?; - if pinned.id != pinned_id { - bail!("remote manifest history forked from the locally pinned head") - } - Ok(()) - } - + /// Checks the whole manifest chain against the machine floor and opens + /// every pack. pub fn verify(&self) -> Result { - let chain = self.current_chain()?; + let floor = self.lock_floor()?; + let chain = self.pinned_chain(&floor)?; let current = chain.first().context("manifest chain is empty")?; let mut count = 0_u64; for entry in chain.iter().rev() { @@ -909,35 +762,26 @@ impl EncryptedRepository { if count != current.manifest.total_pack_count { bail!("manifest delta log pack count mismatch") } + floor.advance(ContinuityFloor::at(¤t.id, ¤t.manifest))?; Ok(current.manifest.clone()) } - pub fn current_manifest(&self) -> Result<(String, Manifest)> { - let current = self - .current_chain()? - .into_iter() - .next() - .context("manifest chain is empty")?; - Ok((current.id, current.manifest)) - } - pub fn recover(&self, repo: &Path, options: RecoveryOptions<'_>) -> Result { - git::ensure_repository(repo)?; - let state_path = self.client_state_path(repo)?; - let _state_lock = ClientStateLock::acquire(&state_path)?; - let state = read_client_state(&state_path)?; + let floor_guard = self.lock_floor()?; + let checkout = self.open_checkout(repo)?; + let local_floor = floor_guard.read()?; let mut report = RecoveryReport { classification: RecoveryClass::Unavailable, reason: "could not read the storage head".to_owned(), head_id: None, claimed_signer: None, - floor_id: state.head_id.clone(), - floor_generation: state.generation, + floor_id: local_floor.as_ref().map(|floor| floor.head_id.clone()), + floor_generation: local_floor.as_ref().map(|floor| floor.generation), candidates: Vec::new(), default_base: None, replays: Vec::new(), conflict: false, - freshness_unverified: state.head_id.is_none() || state.generation.is_none(), + freshness_unverified: local_floor.is_none(), warning: None, blocked_reason: None, published_manifest: None, @@ -974,8 +818,8 @@ impl EncryptedRepository { match current_chain { Ok(chain) => { report.claimed_signer = chain.first().map(|entry| entry.verified_signer.clone()); - if state.head_id.is_some() && state.generation.is_some() { - match self.validate_pinned_head(&chain, &state) { + if local_floor.is_some() { + match validate_pinned_head(&chain, local_floor.as_ref()) { Ok(()) => match self.check_chain_ciphertext_presence(&chain) { Ok(()) => { report.classification = RecoveryClass::Valid; @@ -1036,7 +880,7 @@ impl EncryptedRepository { )) }); record_rejected_head( - &state_path, + &checkout, &evidence_id, report.classification, &report.reason, @@ -1065,8 +909,7 @@ impl EncryptedRepository { return Ok(report); } - let (Some(floor_id), Some(floor_generation)) = (state.head_id.as_deref(), state.generation) - else { + let Some(pinned) = local_floor.as_ref() else { report.freshness_unverified = true; report.blocked_reason = Some( "this repository has no authenticated continuity floor; freshness and fork identity are unverified".to_owned(), @@ -1074,6 +917,8 @@ impl EncryptedRepository { return Ok(report); }; + let floor_id = pinned.head_id.as_str(); + let floor_generation = pinned.generation; let history = self .storage .recovery_history(MAX_RECOVERY_OUTER_COMMITS, MAX_RECOVERY_MANIFEST_BYTES)?; @@ -1158,23 +1003,17 @@ impl EncryptedRepository { Some(floor) => floor, None => { report.blocked_reason = Some( - "client state floor does not match its authenticated manifest generation" + "continuity floor does not match its authenticated manifest generation" .to_owned(), ); return Ok(report); } }; - if state - .repository_root - .as_deref() - .is_some_and(|root| root != floor.manifest.repository_root) - || state - .policy_generation - .is_some_and(|generation| generation > floor.manifest.policy_generation) + if pinned.repository_root != floor.manifest.repository_root + || pinned.policy_generation > floor.manifest.policy_generation { - report.blocked_reason = Some( - "client state floor metadata does not match the authenticated floor".to_owned(), - ); + report.blocked_reason = + Some("continuity floor metadata does not match the authenticated floor".to_owned()); return Ok(report); } let floor_candidate = RecoveryCandidate { @@ -1389,7 +1228,7 @@ impl EncryptedRepository { ManifestAuthorization::Writer, next.clone(), )?; - self.pin_manifest_locked(&state_path, &recovered_id, &next, None)?; + floor_guard.advance(ContinuityFloor::at(&recovered_id, &next))?; report.published_manifest = Some(recovered_id); report.blocked_reason = None; Ok(report) @@ -1800,16 +1639,41 @@ fn proven_invalid(error: anyhow::Error) -> anyhow::Error { } } +fn validate_pinned_head(chain: &[OpenedState], floor: Option<&ContinuityFloor>) -> Result<()> { + let current = chain.first().context("manifest chain is empty")?; + let Some(floor) = floor else { + return Ok(()); + }; + if floor.repository_root != current.manifest.repository_root { + bail!("remote repository root changed") + } + if current.manifest.policy_generation < floor.policy_generation { + bail!("remote policy rolled back") + } + if current.manifest.generation < floor.generation { + bail!( + "remote manifest rolled back from generation {} to {}", + floor.generation, + current.manifest.generation + ) + } + let pinned = chain + .iter() + .find(|entry| entry.manifest.generation == floor.generation) + .context("remote manifest chain ended before pinned generation")?; + if pinned.id != floor.head_id { + bail!("remote manifest history forked from the locally pinned head") + } + Ok(()) +} + fn record_rejected_head( - state_path: &Path, + checkout: &Checkout, head_id: &str, classification: RecoveryClass, reason: &str, ) -> Result<()> { - let directory = state_path - .parent() - .context("client state path has no parent")?; - let path = directory.join("rejected-heads.json"); + let path = checkout.directory.join("rejected-heads.json"); let mut log = match fs::read(&path) { Ok(bytes) => serde_json::from_slice::(&bytes).unwrap_or_default(), Err(error) if error.kind() == io::ErrorKind::NotFound => RejectedHeadLog::default(), @@ -1827,18 +1691,7 @@ fn record_rejected_head( let remove = log.entries.len() - MAX_REJECTED_HEADS; log.entries.drain(..remove); } - let mut random = [0_u8; 8]; - OsRng.fill_bytes(&mut random); - let temporary = directory.join(format!(".rejected-heads-{}.tmp", hex::encode(random))); - let mut file = OpenOptions::new() - .write(true) - .create_new(true) - .open(&temporary)?; - file.write_all(&serde_json::to_vec_pretty(&log)?)?; - file.sync_all()?; - fs::rename(&temporary, &path)?; - crate::persist::sync_directory(directory)?; - Ok(()) + write_json(&path, &log) } fn git_state_directory(repo: &Path) -> Result { @@ -1853,324 +1706,15 @@ fn git_state_directory(repo: &Path) -> Result { Ok(String::from_utf8(output.stdout)?.trim().into()) } -/// Where an administrator's continuity pin lives on this machine. It is keyed -/// by repository root, not by key-file path, so moving or renaming the key -/// keeps the pin. -pub fn admin_state_path(repository_root: &str) -> Result { - let state_home = match std::env::var_os("XDG_STATE_HOME") { - Some(path) if !path.is_empty() => PathBuf::from(path), - _ => PathBuf::from( - std::env::var_os("HOME").context("HOME is unset and XDG_STATE_HOME is unset")?, - ) - .join(".local/state"), - }; - let directory = state_home.join("git-e2ee").join(repository_root); - crate::persist::create_dir_all_durable(&directory) - .with_context(|| format!("create {}", directory.display()))?; - Ok(directory.join("admin-state.json")) -} - -fn read_client_state(path: &Path) -> Result { - match fs::read(path) { - Ok(contents) => { - let state: ClientState = - serde_json::from_slice(&contents).context("parse git-remote-e2ee client state")?; - if state.format_version != 5 { - bail!( - "unsupported git-remote-e2ee client state format {}", - state.format_version - ) - } - Ok(state) - } - Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(ClientState::default()), - Err(error) => Err(error.into()), - } -} - -fn write_client_state_locked( - path: &Path, - values: &HashSet, - head_id: &str, - manifest: &Manifest, - verified_refs: &BTreeMap, -) -> Result<()> { - let mut values: Vec<_> = values.iter().cloned().collect(); - values.sort(); - let state = ClientState { - format_version: 5, - packs: values, - head_id: Some(head_id.to_owned()), - generation: Some(manifest.generation), - imported_generation: Some(manifest.generation), - policy_generation: Some(manifest.policy_generation), - repository_root: Some(manifest.repository_root.clone()), - verified_refs: verified_refs.clone(), - }; - persist_client_state_locked(path, &state) -} - -fn write_published_client_state_locked( - path: &Path, - previous: ClientState, - head_id: &str, - manifest: &Manifest, -) -> Result<()> { - let state = ClientState { - format_version: 5, - packs: previous.packs, - head_id: Some(head_id.to_owned()), - generation: Some(manifest.generation), - imported_generation: previous.imported_generation, - policy_generation: Some(manifest.policy_generation), - repository_root: Some(manifest.repository_root.clone()), - verified_refs: previous.verified_refs, - }; - persist_client_state_locked(path, &state) -} - -fn persist_client_state_locked(path: &Path, state: &ClientState) -> Result<()> { - let previous = read_client_state(path)?; - let state = merge_client_state(previous, state.clone())?; - persist_client_state(path, &state) -} - -fn merge_client_state(previous: ClientState, mut next: ClientState) -> Result { - if previous.format_version != 5 { - return Ok(next); - } - if next.generation.is_some() - && (next.head_id.is_none() - || next.policy_generation.is_none() - || next.repository_root.is_none()) - { - bail!("client state update is missing a continuity pin field") - } - if let (Some(old_root), Some(new_root)) = (&previous.repository_root, &next.repository_root) - && old_root != new_root - { - bail!("remote repository root changed") - } - if previous.generation.is_some() && next.generation.is_none() { - return Ok(previous); - } - - if let (Some(old_generation), Some(new_generation)) = (previous.generation, next.generation) { - if old_generation > new_generation { - return Ok(previous); - } - if old_generation == new_generation { - if let (Some(old_id), Some(new_id)) = (&previous.head_id, &next.head_id) - && old_id != new_id - { - bail!("remote manifest history forked from the locally pinned head") - } - if let (Some(old_policy), Some(new_policy)) = - (previous.policy_generation, next.policy_generation) - && old_policy != new_policy - { - bail!("remote policy pin changed within the same manifest generation") - } - } - } - if let (Some(old_policy), Some(new_policy)) = - (previous.policy_generation, next.policy_generation) - && new_policy < old_policy - { - bail!("remote policy rolled back") - } - - if previous.repository_root.is_some() { - next.repository_root = previous.repository_root.clone(); - } - if next.head_id.is_none() { - next.head_id = previous.head_id.clone(); - } - if previous.policy_generation.is_some() { - next.policy_generation = previous.policy_generation.max(next.policy_generation); - } - if previous - .imported_generation - .is_some_and(|old| next.imported_generation.is_none_or(|new| new < old)) - { - next.imported_generation = previous.imported_generation; - next.verified_refs = previous.verified_refs; - } - - let mut packs = previous.packs; - packs.extend(next.packs); - packs.sort(); - packs.dedup(); - next.packs = packs; - Ok(next) -} - -fn persist_client_state(path: &Path, state: &ClientState) -> Result<()> { - let mut random = [0_u8; 8]; - OsRng.fill_bytes(&mut random); - let temporary = path.with_extension(format!("json.{}.tmp", hex::encode(random))); - let mut file = OpenOptions::new() - .write(true) - .create_new(true) - .open(&temporary)?; - file.write_all(&serde_json::to_vec_pretty(state)?)?; - file.sync_all()?; - crate::persist::durability_checkpoint(crate::persist::STAGE_AFTER_FILE_FLUSH); - fs::rename(&temporary, path)?; - crate::persist::durability_checkpoint(crate::persist::STAGE_AFTER_NAME_PUBLISH); - let parent = match path.parent() { - Some(parent) if parent.as_os_str().is_empty() => Path::new("."), - Some(parent) => parent, - None => bail!("client state path has no parent"), - }; - crate::persist::sync_directory(parent) - .with_context(|| format!("sync directory {}", parent.display()))?; - Ok(()) -} - -#[cfg(test)] -pub(crate) fn testing_write_client_state( - path: &Path, - head_id: &str, - generation: u64, -) -> Result<()> { - if let Some(parent) = path - .parent() - .filter(|parent| !parent.as_os_str().is_empty()) - { - crate::persist::create_dir_all_durable(parent)?; - } - let state = ClientState { - format_version: 5, - packs: Vec::new(), - head_id: Some(head_id.to_owned()), - generation: Some(generation), - imported_generation: Some(generation), - policy_generation: Some(1), - repository_root: Some("durability-fixture".to_owned()), - verified_refs: BTreeMap::new(), - }; - let _state_lock = ClientStateLock::acquire(path)?; - persist_client_state_locked(path, &state) -} - #[cfg(test)] mod tests { - use std::env; - use std::fs; - use std::path::Path; - use std::process::{Child, Command}; - use std::thread; - use std::time::{Duration, Instant}; - use std::io; use super::{ - ClientState, ForeignRepository, InvalidState, MissingStorageHead, NotActiveReader, - ObjectUnavailable, RecoveryClass, Unsupported, classify_recovery_error, read_client_state, - testing_write_client_state, + ForeignRepository, InvalidState, MissingStorageHead, NotActiveReader, ObjectUnavailable, + RecoveryClass, Unsupported, classify_recovery_error, }; - #[test] - fn concurrent_client_state_process_writes_never_lower_or_corrupt_the_floor() { - let temporary = tempfile::tempdir().unwrap(); - let state_path = temporary.path().join("state.json"); - testing_write_client_state(&state_path, "head-10", 10).unwrap(); - let release = temporary.path().join("release-old-writer"); - let ready = temporary.path().join("old-writer-ready"); - let executable = env::current_exe().unwrap(); - - let older = Command::new(&executable) - .args([ - "--exact", - "repository::tests::client_state_process_write_worker", - "--nocapture", - ]) - .env("E2EE_TEST_CLIENT_STATE_PATH", &state_path) - .env("E2EE_TEST_CLIENT_STATE_GENERATION", "11") - .env("E2EE_TEST_CLIENT_STATE_READY", &ready) - .env("E2EE_TEST_CLIENT_STATE_RELEASE", &release) - .spawn() - .unwrap(); - let deadline = Instant::now() + Duration::from_secs(10); - while !ready.exists() && Instant::now() < deadline { - thread::sleep(Duration::from_millis(10)); - } - assert!(ready.exists(), "older writer did not reach its wait point"); - - let newer = launch_client_state_writer(&executable, &state_path, 12); - let newer_output = newer.wait_with_output().unwrap(); - assert!( - newer_output.status.success(), - "newer writer failed: {}", - String::from_utf8_lossy(&newer_output.stderr) - ); - fs::write(&release, "continue").unwrap(); - - let older_output = older.wait_with_output().unwrap(); - assert!( - older_output.status.success(), - "older writer failed: {}", - String::from_utf8_lossy(&older_output.stderr) - ); - let state = read_client_state(&state_path).unwrap(); - assert_eq!(state.generation, Some(12)); - assert_eq!(state.head_id.as_deref(), Some("head-12")); - } - - #[test] - fn client_state_process_write_worker() { - let Ok(path) = env::var("E2EE_TEST_CLIENT_STATE_PATH") else { - return; - }; - let generation = env::var("E2EE_TEST_CLIENT_STATE_GENERATION") - .unwrap() - .parse::() - .unwrap(); - if let (Ok(ready), Ok(release)) = ( - env::var("E2EE_TEST_CLIENT_STATE_READY"), - env::var("E2EE_TEST_CLIENT_STATE_RELEASE"), - ) { - fs::write(&ready, "ready").unwrap(); - let release = Path::new(&release); - let deadline = Instant::now() + Duration::from_secs(10); - while !release.exists() && Instant::now() < deadline { - thread::sleep(Duration::from_millis(10)); - } - assert!(release.exists(), "parent did not release the older writer"); - } - testing_write_client_state(Path::new(&path), &format!("head-{generation}"), generation) - .unwrap(); - } - - fn launch_client_state_writer(executable: &Path, state_path: &Path, generation: u64) -> Child { - Command::new(executable) - .args([ - "--exact", - "repository::tests::client_state_process_write_worker", - "--nocapture", - ]) - .env("E2EE_TEST_CLIENT_STATE_PATH", state_path) - .env("E2EE_TEST_CLIENT_STATE_GENERATION", generation.to_string()) - .spawn() - .unwrap() - } - - #[test] - fn client_state_without_verified_frontier_requires_a_full_check() { - let legacy = br#"{ - "format_version": 4, - "packs": ["pack-id"], - "head_id": "head-id", - "generation": 7, - "imported_generation": 7, - "policy_generation": 2, - "repository_root": "repository-root" - }"#; - let state: ClientState = serde_json::from_slice(legacy).unwrap(); - assert!(state.verified_refs.is_empty()); - } - #[test] fn recovery_classification_ignores_storage_controlled_message_text() { for text in [ diff --git a/src/storage.rs b/src/storage.rs index 8ae3fad..c6f7c1c 100644 --- a/src/storage.rs +++ b/src/storage.rs @@ -93,6 +93,13 @@ pub trait Storage { } fn read_head(&self) -> Result>; + + /// Replaces a cached view of storage with the current one and reports + /// whether the head may have changed. Backends that read live need not. + fn refresh(&self) -> Result { + Ok(false) + } + fn compare_and_swap_head(&self, expected: Option<&str>, next: &str) -> Result<()>; fn observe_head(&self) -> Result { @@ -1378,6 +1385,21 @@ impl Storage for GitStorage { self.local_head() } + fn refresh(&self) -> Result { + let mut state = self + .state + .lock() + .map_err(|_| anyhow::anyhow!("carrier state lock poisoned"))?; + let mut cache = CarrierCache::lock(&self.remote, &self.branch)?; + let remote_tip = cache.refresh(&self.remote, &self.branch)?; + if remote_tip == state.base_commit { + return Ok(false); + } + self.restore_checkout(&remote_tip)?; + state.base_commit = remote_tip; + Ok(true) + } + fn compare_and_swap_head(&self, expected: Option<&str>, next: &str) -> Result<()> { validate_id(next)?; let mut state = self diff --git a/tests/anonymous_recipients.rs b/tests/anonymous_recipients.rs index 7c046e1..0ad15ac 100644 --- a/tests/anonymous_recipients.rs +++ b/tests/anonymous_recipients.rs @@ -65,10 +65,8 @@ fn run_metadata_scenario( scratch: &Path, ) { let owner = KeyFile::generate(); - let pin = scratch.join("owner.admin-state.json"); - let repository = EncryptedRepository::new(make_storage(), owner.clone()); + let repository = EncryptedRepository::new(make_storage(), owner.clone(), scratch.join("state")); repository.initialize("owner").unwrap(); - repository.pin_admin_state(&pin).unwrap(); let mut devices = vec![owner.public_device().unwrap()]; assert_eq!( stored_header(make_storage()).generation_key_envelopes.len(), @@ -86,7 +84,6 @@ fn run_metadata_scenario( reader.public_device().unwrap(), &format!("phone-{index}"), DeviceRoles::collaborator(), - &pin, ) .unwrap(); devices.push(reader.public_device().unwrap()); @@ -100,7 +97,7 @@ fn run_metadata_scenario( } repository - .revoke_device(&readers[0].device_id().unwrap(), &pin) + .revoke_device(&readers[0].device_id().unwrap()) .unwrap(); assert_eq!( stored_header(make_storage()).generation_key_envelopes.len(), diff --git a/tests/carrier_git.rs b/tests/carrier_git.rs index 8305f86..75e80a3 100644 --- a/tests/carrier_git.rs +++ b/tests/carrier_git.rs @@ -1,3 +1,5 @@ +mod support; + use std::collections::BTreeMap; use std::env; use std::fs; @@ -13,7 +15,9 @@ use git_remote_e2ee::manifest::{ unwrap_generation_key, wrap_predecessor_key, }; use git_remote_e2ee::policy::{DeviceRoles, PolicyState}; -use git_remote_e2ee::repository::{EncryptedRepository, RecoveryClass, RecoveryOptions}; +use git_remote_e2ee::repository::{ + EncryptedRepository, PushRejection, RecoveryClass, RecoveryOptions, RefChange, +}; use git_remote_e2ee::storage::{CasConflict, GitStorage, ObjectKind, Storage}; static CACHE_ENV_LOCK: Mutex<()> = Mutex::new(()); @@ -98,6 +102,7 @@ fn init_carrier(root: &Path, carrier: &Path, key: &KeyFile) { EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), key.clone(), + support::state_dir(), ) .initialize("owner") .unwrap(); @@ -208,16 +213,9 @@ fn add_helper_to_path(command: &mut Command) { let helper = Path::new(env!("CARGO_BIN_EXE_git-remote-e2ee")); let mut paths = vec![helper.parent().unwrap().to_path_buf()]; paths.extend(env::split_paths(&env::var_os("PATH").unwrap_or_default())); - command.env("PATH", env::join_paths(paths).unwrap()); -} - -fn state_dir(repo: &Path) -> std::path::PathBuf { - let entries: Vec<_> = fs::read_dir(repo.join(".git/git-remote-e2ee")) - .unwrap() - .map(|entry| entry.unwrap().path()) - .collect(); - assert_eq!(entries.len(), 1, "{entries:?}"); - entries.into_iter().next().unwrap() + command + .env("PATH", env::join_paths(paths).unwrap()) + .env("XDG_STATE_HOME", support::state_home()); } #[test] @@ -382,10 +380,13 @@ fn carrier_git_supports_tags_and_ref_deletion() { .status .success() ); - let manifest = - EncryptedRepository::new(GitStorage::open(carrier.to_str().unwrap()).unwrap(), key) - .verify() - .unwrap(); + let manifest = EncryptedRepository::new( + GitStorage::open(carrier.to_str().unwrap()).unwrap(), + key, + support::state_dir(), + ) + .verify() + .unwrap(); assert!(!manifest.refs.contains_key("refs/heads/doomed")); assert!(!manifest.refs.contains_key("refs/tags/v-carrier")); } @@ -404,25 +405,42 @@ fn carrier_git_allows_exactly_one_concurrent_writer() { init_carrier(temporary.path(), &carrier, &key); // Open both carrier checkouts before either push. They therefore race from - // the same outer commit and exercise receive-pack's ref lock/FF check. + // the same outer commit and exercise receive-pack's ref lock/FF check. Each + // writer is its own machine; one machine serializes on its floor lock. let repository_a = EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), key.clone(), + temporary.path().join("machine-a"), ); let repository_b = EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), key.clone(), + temporary.path().join("machine-b"), ); let barrier = Arc::new(Barrier::new(2)); let barrier_a = barrier.clone(); let first = thread::spawn(move || { barrier_a.wait(); - repository_a.push_update(&source_a, "refs/heads/main", "refs/heads/writer-a", false) + repository_a.push( + &source_a, + "refs/heads/writer-a", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) }); let barrier_b = barrier.clone(); let second = thread::spawn(move || { barrier_b.wait(); - repository_b.push_update(&source_b, "refs/heads/main", "refs/heads/writer-b", false) + repository_b.push( + &source_b, + "refs/heads/writer-b", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) }); let results = [first.join().unwrap(), second.join().unwrap()]; @@ -436,7 +454,11 @@ fn carrier_git_allows_exactly_one_concurrent_writer() { "unexpected losing error: {loser:#}" ); - let fresh = EncryptedRepository::new(GitStorage::open(carrier.to_str().unwrap()).unwrap(), key); + let fresh = EncryptedRepository::new( + GitStorage::open(carrier.to_str().unwrap()).unwrap(), + key, + support::state_dir(), + ); let manifest = fresh.verify().unwrap(); assert_eq!(manifest.generation, 1); assert_eq!(manifest.refs.len(), 1); @@ -456,6 +478,62 @@ fn carrier_git_allows_exactly_one_concurrent_writer() { ); } +#[test] +fn stale_carrier_views_on_one_machine_refresh_instead_of_reporting_rollback() { + let temporary = tempfile::tempdir().unwrap(); + let _cache = isolated_cache(&temporary); + let carrier = temporary.path().join("carrier.git"); + let source_a = temporary.path().join("source-a"); + let source_b = temporary.path().join("source-b"); + let fetched = temporary.path().join("fetched"); + init_source(&source_a, "writer a"); + init_source(&source_b, "writer b"); + git( + temporary.path(), + &["init", "-q", fetched.to_str().unwrap()], + false, + ); + let key = KeyFile::generate(); + init_carrier(temporary.path(), &carrier, &key); + let open = || { + EncryptedRepository::new( + GitStorage::open(carrier.to_str().unwrap()).unwrap(), + key.clone(), + support::state_dir(), + ) + }; + let update = RefChange::Update { + source: "refs/heads/main", + force: false, + }; + + let stale_fetch = open(); + let stale_push_main = open(); + let stale_push_new = open(); + open().push(&source_a, "refs/heads/main", update).unwrap(); + assert_eq!(support::read_floor(&key.repository_root)["generation"], 1); + + let manifest = stale_fetch.fetch_into(&fetched, "private").unwrap(); + assert_eq!(manifest.generation, 1); + + let error = stale_push_main + .push(&source_b, "refs/heads/main", update) + .unwrap_err(); + let rejected_as_fetch_first = error + .downcast_ref::() + .is_some_and(|rejection| rejection.git_status() == "fetch first") + || error.downcast_ref::().is_some(); + assert!(rejected_as_fetch_first, "{error:#}"); + + stale_push_new + .push(&source_b, "refs/heads/writer-b", update) + .unwrap(); + let manifest = open().verify().unwrap(); + assert_eq!(manifest.generation, 2); + assert!(manifest.refs.contains_key("refs/heads/main")); + assert!(manifest.refs.contains_key("refs/heads/writer-b")); +} + #[test] fn carrier_cache_fetches_only_new_objects() { let temporary = tempfile::tempdir().unwrap(); @@ -809,8 +887,16 @@ fn carrier_git_detects_tampered_ciphertext_chunk() { EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), key.clone(), + support::state_dir(), + ) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, ) - .push_ref(&source, "refs/heads/main", false) .unwrap(); git( @@ -840,7 +926,11 @@ fn carrier_git_detects_tampered_ciphertext_chunk() { git(&attacker, &["commit", "-q", "-m", "tamper chunk"], false); git(&attacker, &["push", "-q", "origin", "main"], false); - let fresh = EncryptedRepository::new(GitStorage::open(carrier.to_str().unwrap()).unwrap(), key); + let fresh = EncryptedRepository::new( + GitStorage::open(carrier.to_str().unwrap()).unwrap(), + key, + support::state_dir(), + ); let error = fresh.verify().unwrap_err(); assert!( error.to_string().contains("invalid pack stream magic"), @@ -863,7 +953,6 @@ fn carrier_recovery_chooses_legitimate_descendant_and_restores_deleted_ciphertex let owner = KeyFile::generate(); let reader = KeyFile::generate_for_repository(owner.repository_root.clone()).unwrap(); - let admin_pin = temporary.path().join("owner.admin-state.json"); git( temporary.path(), &["init", "--bare", "-q", carrier.to_str().unwrap()], @@ -872,15 +961,14 @@ fn carrier_recovery_chooses_legitimate_descendant_and_restores_deleted_ciphertex let repository = EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), owner.clone(), + support::state_dir(), ); repository.initialize("owner").unwrap(); - repository.pin_admin_state(&admin_pin).unwrap(); let (reader_policy_manifest, _) = repository .add_device( reader.public_device().unwrap(), "device-1", DeviceRoles::reader(), - &admin_pin, ) .unwrap(); let policy_storage = GitStorage::open(carrier.to_str().unwrap()).unwrap(); @@ -900,20 +988,40 @@ fn carrier_recovery_chooses_legitimate_descendant_and_restores_deleted_ciphertex .unwrap(); let pushed_object = git(&source, &["rev-parse", "HEAD"], false); repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); let (floor_id, floor_manifest) = repository.current_manifest().unwrap(); let pack_id = floor_manifest.new_packs[0].id.clone(); - repository.fetch_into(&returning, "private").unwrap(); + // The returning client is another machine; the writer's floor is not its floor. + let returning_machine = temporary.path().join("returning-machine"); + EncryptedRepository::new( + GitStorage::open(carrier.to_str().unwrap()).unwrap(), + owner.clone(), + &returning_machine, + ) + .fetch_into(&returning, "private") + .unwrap(); let floor_generation: u64 = serde_json::from_slice::( - &fs::read(state_dir(&returning).join("state.json")).unwrap(), + &fs::read( + returning_machine + .join(&owner.repository_root) + .join("continuity.json"), + ) + .unwrap(), ) .unwrap()["generation"] .as_u64() .unwrap(); assert_eq!(floor_manifest.generation, floor_generation); repository - .revoke_device(&reader.device_id().unwrap(), &admin_pin) + .revoke_device(&reader.device_id().unwrap()) .unwrap(); let (legitimate_id, legitimate) = repository.current_manifest().unwrap(); assert_eq!(legitimate.refs["refs/heads/main"], pushed_object); @@ -954,7 +1062,7 @@ fn carrier_recovery_chooses_legitimate_descendant_and_restores_deleted_ciphertex .unwrap(); let recovery_storage = GitStorage::open(carrier.to_str().unwrap()).unwrap(); - let recoverer = EncryptedRepository::new(recovery_storage, owner.clone()); + let recoverer = EncryptedRepository::new(recovery_storage, owner.clone(), &returning_machine); let report = recoverer .recover( &returning, @@ -1067,19 +1175,17 @@ fn carrier_recovery_offers_hidden_legitimate_content_descendant() { let owner = KeyFile::generate(); let reader = KeyFile::generate_for_repository(owner.repository_root.clone()).unwrap(); - let admin_pin = temporary.path().join("owner.admin-state.json"); let repository = EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), owner.clone(), + support::state_dir(), ); repository.initialize("owner").unwrap(); - repository.pin_admin_state(&admin_pin).unwrap(); let (floor_id, _) = repository .add_device( reader.public_device().unwrap(), "device-2", DeviceRoles::reader(), - &admin_pin, ) .unwrap(); let floor_storage = GitStorage::open(carrier.to_str().unwrap()).unwrap(); @@ -1097,11 +1203,26 @@ fn carrier_recovery_offers_hidden_legitimate_content_descendant() { .unwrap(), ) .unwrap(); - repository.fetch_into(&returning, "private").unwrap(); + // The returning client is another machine; the writer's floor is not its floor. + let returning_machine = temporary.path().join("returning-machine"); + EncryptedRepository::new( + GitStorage::open(carrier.to_str().unwrap()).unwrap(), + owner.clone(), + &returning_machine, + ) + .fetch_into(&returning, "private") + .unwrap(); let pushed_object = git(&source, &["rev-parse", "HEAD"], false); repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); let (legitimate_id, legitimate) = repository.current_manifest().unwrap(); assert_eq!(legitimate.refs["refs/heads/main"], pushed_object); @@ -1136,8 +1257,11 @@ fn carrier_recovery_offers_hidden_legitimate_content_descendant() { .compare_and_swap_head(Some(&floor_id), &junk_id) .unwrap(); - let recoverer = - EncryptedRepository::new(GitStorage::open(carrier.to_str().unwrap()).unwrap(), owner); + let recoverer = EncryptedRepository::new( + GitStorage::open(carrier.to_str().unwrap()).unwrap(), + owner, + &returning_machine, + ); let report = recoverer .recover( &returning, @@ -1220,19 +1344,17 @@ fn carrier_recovery_refuses_history_with_a_merge_hiding_a_legitimate_state() { let owner = KeyFile::generate(); let reader = KeyFile::generate_for_repository(owner.repository_root.clone()).unwrap(); - let admin_pin = temporary.path().join("owner.admin-state.json"); let repository = EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), owner.clone(), + support::state_dir(), ); repository.initialize("owner").unwrap(); - repository.pin_admin_state(&admin_pin).unwrap(); let (floor_id, _) = repository .add_device( reader.public_device().unwrap(), "device-3", DeviceRoles::reader(), - &admin_pin, ) .unwrap(); let floor_storage = GitStorage::open(carrier.to_str().unwrap()).unwrap(); @@ -1252,7 +1374,14 @@ fn carrier_recovery_refuses_history_with_a_merge_hiding_a_legitimate_state() { .unwrap(); repository.fetch_into(&returning, "private").unwrap(); repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); // Put a replay of the floor on the first-parent path and the legitimate @@ -1299,8 +1428,11 @@ fn carrier_recovery_refuses_history_with_a_merge_hiding_a_legitimate_state() { .compare_and_swap_head(Some(&floor_id), &junk_id) .unwrap(); - let recoverer = - EncryptedRepository::new(GitStorage::open(carrier.to_str().unwrap()).unwrap(), owner); + let recoverer = EncryptedRepository::new( + GitStorage::open(carrier.to_str().unwrap()).unwrap(), + owner, + support::state_dir(), + ); let error = recoverer .recover( &returning, @@ -1335,19 +1467,17 @@ fn carrier_recovery_refuses_conflicting_authenticated_descendants() { let owner = KeyFile::generate(); let reader = KeyFile::generate_for_repository(owner.repository_root.clone()).unwrap(); - let admin_pin = temporary.path().join("owner.admin-state.json"); let repository = EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), owner.clone(), + support::state_dir(), ); repository.initialize("owner").unwrap(); - repository.pin_admin_state(&admin_pin).unwrap(); let (floor_id, _) = repository .add_device( reader.public_device().unwrap(), "device-4", DeviceRoles::reader(), - &admin_pin, ) .unwrap(); repository.fetch_into(&returning, "private").unwrap(); @@ -1397,7 +1527,7 @@ fn carrier_recovery_refuses_conflicting_authenticated_descendants() { .unwrap(); let recovery_storage = GitStorage::open(carrier.to_str().unwrap()).unwrap(); - let report = EncryptedRepository::new(recovery_storage, owner) + let report = EncryptedRepository::new(recovery_storage, owner, support::state_dir()) .recover( &returning, RecoveryOptions { @@ -1451,19 +1581,17 @@ fn carrier_recovery_loses_cas_without_retrying_when_another_writer_publishes() { let owner = KeyFile::generate(); let reader = KeyFile::generate_for_repository(owner.repository_root.clone()).unwrap(); - let admin_pin = temporary.path().join("owner.admin-state.json"); let repository = EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), owner.clone(), + support::state_dir(), ); repository.initialize("owner").unwrap(); - repository.pin_admin_state(&admin_pin).unwrap(); let (floor_id, _) = repository .add_device( reader.public_device().unwrap(), "device-5", DeviceRoles::reader(), - &admin_pin, ) .unwrap(); repository.fetch_into(&returning, "private").unwrap(); @@ -1489,7 +1617,8 @@ fn carrier_recovery_loses_cas_without_retrying_when_another_writer_publishes() { .compare_and_swap_head(Some(&floor_id), &junk) .unwrap(); let stale_storage = GitStorage::open(carrier.to_str().unwrap()).unwrap(); - let stale_recoverer = EncryptedRepository::new(stale_storage, owner.clone()); + let stale_recoverer = + EncryptedRepository::new(stale_storage, owner.clone(), support::state_dir()); let offered = stale_recoverer .recover( &returning, @@ -1528,8 +1657,11 @@ fn carrier_recovery_loses_cas_without_retrying_when_another_writer_publishes() { ) .unwrap_err(); assert!(error.to_string().contains("concurrently")); - let latest = - EncryptedRepository::new(GitStorage::open(carrier.to_str().unwrap()).unwrap(), owner); + let latest = EncryptedRepository::new( + GitStorage::open(carrier.to_str().unwrap()).unwrap(), + owner, + support::state_dir(), + ); assert_eq!(latest.current_manifest().unwrap().0, winner); } @@ -1543,11 +1675,11 @@ fn carrier_refs(carrier: &Path) -> String { fn helper_command(cwd: &Path, home: &Path) -> Command { let mut command = Command::new("git"); + add_helper_to_path(&mut command); command .current_dir(cwd) .env("XDG_STATE_HOME", home.join("state")) .env("XDG_CONFIG_HOME", home.join("config")); - add_helper_to_path(&mut command); command } @@ -1674,6 +1806,7 @@ fn carrier_format_mismatch_names_the_found_format() { let repository = EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), key.clone(), + support::state_dir(), ); format!("{:#}", repository.current_manifest().unwrap_err()) }; @@ -1717,9 +1850,13 @@ fn carrier_branch_fragment_selects_the_branch_through_the_helper() { false, ); let location = format!("{}#vault", carrier.display()); - EncryptedRepository::new(GitStorage::open(&location).unwrap(), key.clone()) - .initialize("owner") - .unwrap(); + EncryptedRepository::new( + GitStorage::open(&location).unwrap(), + key.clone(), + support::state_dir(), + ) + .initialize("owner") + .unwrap(); let url = format!("e2ee::{location}"); push_through_helper(&source, &url, &key_path, temporary.path()); @@ -1773,6 +1910,7 @@ fn carrier_init_refuses_an_existing_branch_and_another_branch_works() { let refused = EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), key.clone(), + support::state_dir(), ) .initialize("owner") .unwrap_err(); @@ -1786,7 +1924,11 @@ fn carrier_init_refuses_an_existing_branch_and_another_branch_works() { assert_eq!(carrier_refs(&carrier), refs_before); let location = format!("{}#vault", carrier.display()); - let encrypted = EncryptedRepository::new(GitStorage::open(&location).unwrap(), key); + let encrypted = EncryptedRepository::new( + GitStorage::open(&location).unwrap(), + key, + support::state_dir(), + ); let caches = fs::read_dir(temporary.path().join("carrier-cache")) .unwrap() .filter(|entry| entry.as_ref().unwrap().path().is_dir()) @@ -1989,10 +2131,13 @@ fn failed_ref_in_a_helper_push_batch_is_not_published_by_the_next_ref() { assert!(status(failed).starts_with('!'), "{stdout}"); assert!(!status(published).starts_with('!'), "{stdout}"); - let manifest = - EncryptedRepository::new(GitStorage::open(carrier.to_str().unwrap()).unwrap(), key) - .verify() - .unwrap(); + let manifest = EncryptedRepository::new( + GitStorage::open(carrier.to_str().unwrap()).unwrap(), + key, + support::state_dir(), + ) + .verify() + .unwrap(); let names: Vec<_> = manifest.refs.keys().map(String::as_str).collect(); assert!(names.contains(&published), "{names:?}"); assert!( @@ -2025,9 +2170,17 @@ fn carrier_recovery_classifies_missing_head_manifest_as_unavailable() { let repository = EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), owner.clone(), + support::state_dir(), ); repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); repository.fetch_into(&returning, "private").unwrap(); @@ -2036,9 +2189,17 @@ fn carrier_recovery_classifies_missing_head_manifest_as_unavailable() { let repository = EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), owner.clone(), + support::state_dir(), ); repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); let (head_id, _) = repository.current_manifest().unwrap(); @@ -2055,6 +2216,7 @@ fn carrier_recovery_classifies_missing_head_manifest_as_unavailable() { EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), owner.clone(), + support::state_dir(), ) .recover( &returning, @@ -2098,9 +2260,17 @@ fn carrier_recovery_refuses_to_skip_a_historical_head_with_missing_objects() { let repository = EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), owner.clone(), + support::state_dir(), ); repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); repository.fetch_into(&returning, "private").unwrap(); let (floor_id, floor) = repository.current_manifest().unwrap(); @@ -2138,6 +2308,7 @@ fn carrier_recovery_refuses_to_skip_a_historical_head_with_missing_objects() { let report = EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), owner.clone(), + support::state_dir(), ) .recover( &returning, @@ -2175,9 +2346,17 @@ fn carrier_history_reads_skip_corrupt_copies_for_an_older_valid_one() { let repository = EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), owner.clone(), + support::state_dir(), ); repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); let (head_id, head) = repository.current_manifest().unwrap(); let pack_id = head.new_packs[0].id.clone(); @@ -2244,30 +2423,29 @@ fn recover_junk_head_after_later_writer_push(recover_as_owner: bool) { let owner = KeyFile::generate(); let writer = KeyFile::generate_for_repository(owner.repository_root.clone()).unwrap(); - let admin_pin = temporary.path().join("owner.admin-state.json"); init_carrier(temporary.path(), &carrier, &owner); let open = |key: &KeyFile| { EncryptedRepository::new( GitStorage::open(carrier.to_str().unwrap()).unwrap(), key.clone(), + support::state_dir(), ) }; - open(&owner).pin_admin_state(&admin_pin).unwrap(); open(&owner) .add_device( writer.public_device().unwrap(), "writer", DeviceRoles::collaborator(), - &admin_pin, ) .unwrap(); open(&writer) - .push_update_for_remote( + .push( &writer_checkout, - "origin", "refs/heads/main", - "refs/heads/main", - false, + RefChange::Update { + source: "refs/heads/main", + force: false, + }, ) .unwrap(); open(&owner).fetch_into(&owner_checkout, "origin").unwrap(); diff --git a/tests/cli_roles.rs b/tests/cli_roles.rs index f272309..ac22957 100644 --- a/tests/cli_roles.rs +++ b/tests/cli_roles.rs @@ -143,14 +143,14 @@ fn init_accepts_relative_storage_and_key_paths() { .path() .join("state/git-e2ee") .join(&owner.repository_root) - .join("admin-state.json") + .join("continuity.json") .is_file(), - "init must pin the administrator state under XDG_STATE_HOME" + "init must pin the continuity floor under XDG_STATE_HOME" ); } #[test] -fn admin_pin_survives_moving_the_key() { +fn continuity_floor_survives_moving_the_key() { let temporary = tempfile::tempdir().unwrap(); let storage = temporary.path().join("storage"); let snapshot = temporary.path().join("storage-snapshot"); diff --git a/tests/continuity_floor.rs b/tests/continuity_floor.rs new file mode 100644 index 0000000..d87217e --- /dev/null +++ b/tests/continuity_floor.rs @@ -0,0 +1,161 @@ +use std::env; +use std::fs; +use std::path::{Path, PathBuf}; +use std::process::{Command, Output}; + +use git_remote_e2ee::crypto::KeyFile; + +struct Machine { + root: tempfile::TempDir, +} + +impl Machine { + fn new() -> Self { + Self { + root: tempfile::tempdir().unwrap(), + } + } + + fn path(&self, name: &str) -> PathBuf { + self.root.path().join(name) + } + + fn command(&self, program: &str, cwd: &Path, args: &[&str]) -> Output { + let helper = Path::new(env!("CARGO_BIN_EXE_git-remote-e2ee")); + let mut paths = vec![helper.parent().unwrap().to_path_buf()]; + paths.extend(env::split_paths(&env::var_os("PATH").unwrap_or_default())); + Command::new(program) + .current_dir(cwd) + .args(args) + .env("PATH", env::join_paths(paths).unwrap()) + .env("XDG_STATE_HOME", self.path("state")) + .env("XDG_CONFIG_HOME", self.path("config")) + .env("GIT_REMOTE_E2EE_CACHE_DIR", self.path("cache")) + .output() + .unwrap() + } + + fn cli(&self, cwd: &Path, args: &[&str]) -> Output { + self.command(env!("CARGO_BIN_EXE_git-e2ee"), cwd, args) + } + + fn git(&self, cwd: &Path, args: &[&str]) -> Output { + self.command("git", cwd, args) + } +} + +fn succeed(action: &str, output: Output) -> String { + assert!( + output.status.success(), + "{action} failed: {}", + String::from_utf8_lossy(&output.stderr) + ); + String::from_utf8_lossy(&output.stdout).into_owned() +} + +fn commit(machine: &Machine, repo: &Path, text: &str) { + fs::write(repo.join("note.md"), text).unwrap(); + succeed("git add", machine.git(repo, &["add", "note.md"])); + succeed( + "git commit", + machine.git(repo, &["commit", "-q", "-m", text]), + ); +} + +fn copy_tree(source: &Path, destination: &Path) { + let copied = Command::new("cp") + .arg("-R") + .arg(source) + .arg(destination) + .status() + .unwrap(); + assert!(copied.success()); +} + +/// A device-revoke on this machine must protect every later push from this +/// machine, even from a checkout that never fetched the revocation. Before +/// one machine floor existed, the checkout pin still accepted the replayed +/// pre-revocation storage and the push re-published content to the revoked +/// reader. +#[test] +fn push_after_rollback_of_a_revocation_seen_only_by_device_revoke_is_rejected() { + let machine = Machine::new(); + let clone = machine.path("clone"); + let storage = machine.path("storage"); + let snapshot = machine.path("snapshot"); + fs::create_dir_all(&clone).unwrap(); + succeed( + "git init", + machine.git(&clone, &["init", "-q", "-b", "main"]), + ); + succeed("config", machine.git(&clone, &["config", "user.name", "T"])); + succeed( + "config", + machine.git(&clone, &["config", "user.email", "t@example.invalid"]), + ); + let owner_path = machine.path("owner.key.json"); + let url = format!("e2ee::dir:{}", storage.display()); + succeed( + "git-e2ee init", + machine.cli( + &clone, + &["init", &url, "--key", owner_path.to_str().unwrap()], + ), + ); + commit(&machine, &clone, "v1"); + succeed( + "push v1", + machine.git(&clone, &["push", "-q", "origin", "main"]), + ); + + let owner = KeyFile::read(&owner_path).unwrap(); + let reader = KeyFile::generate_for_repository(owner.repository_root.clone()).unwrap(); + let reader_public = machine.path("reader.public.json"); + reader + .public_device() + .unwrap() + .write_new(&reader_public) + .unwrap(); + succeed( + "device-add", + machine.cli( + &clone, + &[ + "device-add", + "origin", + reader_public.to_str().unwrap(), + "--label", + "reader", + "--role", + "read", + ], + ), + ); + succeed("fetch after add", machine.git(&clone, &["fetch", "origin"])); + + copy_tree(&storage, &snapshot); + succeed( + "device-revoke", + machine.cli( + &clone, + &["device-revoke", "origin", &reader.device_id().unwrap()], + ), + ); + fs::remove_dir_all(&storage).unwrap(); + fs::rename(&snapshot, &storage).unwrap(); + let replayed_head = fs::read(storage.join("HEAD")).unwrap(); + + commit(&machine, &clone, "post-revocation secret"); + let pushed = machine.git(&clone, &["push", "origin", "main"]); + let stderr = String::from_utf8_lossy(&pushed.stderr); + assert!( + !pushed.status.success(), + "push over the replayed pre-revocation state succeeded: {stderr}" + ); + assert!(stderr.contains("rolled back"), "{stderr}"); + assert_eq!( + fs::read(storage.join("HEAD")).unwrap(), + replayed_head, + "the rejected push must not publish a generation" + ); +} diff --git a/tests/end_to_end.rs b/tests/end_to_end.rs index fc8ad18..1c9549f 100644 --- a/tests/end_to_end.rs +++ b/tests/end_to_end.rs @@ -1,3 +1,5 @@ +mod support; + use std::collections::BTreeMap; use std::fs; use std::io::Read; @@ -15,7 +17,7 @@ use git_remote_e2ee::manifest::{ unwrap_generation_key, verify_manifest, wrap_predecessor_key, }; use git_remote_e2ee::policy::{DeviceRoles, PolicyState}; -use git_remote_e2ee::repository::{EncryptedRepository, RecoveryClass, RecoveryOptions}; +use git_remote_e2ee::repository::{EncryptedRepository, RecoveryClass, RecoveryOptions, RefChange}; use git_remote_e2ee::storage::{ FilesystemStorage, ObjectKind, ObjectStage, RecoveryCommit, RecoveryHistory, Storage, }; @@ -126,7 +128,7 @@ fn floor_with_malformed_head( initialize_git(&repo); let key = KeyFile::generate(); let storage = FilesystemStorage::new(temporary.join("storage")); - let repository = EncryptedRepository::new(storage.clone(), key.clone()); + let repository = EncryptedRepository::new(storage.clone(), key.clone(), support::state_dir()); repository.initialize("owner").unwrap(); repository.fetch_into(&repo, "encrypted").unwrap(); let floor = storage.read_head().unwrap().unwrap(); @@ -193,6 +195,7 @@ fn recovery_budget_exhaustion_inside_candidate_discovery_is_fatal() { manifests: BTreeMap::from([(oversized_id, oversized)]), }, key, + support::state_dir(), ); let error = publish_recovery(&recovery, &repo).unwrap_err(); @@ -204,7 +207,7 @@ fn recovery_budget_exhaustion_inside_candidate_discovery_is_fatal() { storage.read_head().unwrap().as_deref(), Some(junk_id.as_str()) ); - assert_eq!(client_state(&repo)["head_id"], floor); + assert_eq!(machine_floor(&repo)["head_id"], floor); } #[test] @@ -212,9 +215,13 @@ fn recovery_refuses_to_skip_a_discontinuous_historical_head() { let temporary = tempfile::tempdir().unwrap(); let (key, storage, repo, floor, junk_id) = floor_with_malformed_head(temporary.path()); let foreign_storage = FilesystemStorage::new(temporary.path().join("foreign")); - EncryptedRepository::new(foreign_storage.clone(), KeyFile::generate()) - .initialize("foreign") - .unwrap(); + EncryptedRepository::new( + foreign_storage.clone(), + KeyFile::generate(), + support::state_dir(), + ) + .initialize("foreign") + .unwrap(); let foreign_id = foreign_storage.read_head().unwrap().unwrap(); let foreign = foreign_storage .get_object(ObjectKind::Manifest, &foreign_id) @@ -226,6 +233,7 @@ fn recovery_refuses_to_skip_a_discontinuous_historical_head() { manifests: BTreeMap::from([(foreign_id.clone(), foreign)]), }, key, + support::state_dir(), ); let report = publish_recovery(&recovery, &repo).unwrap(); @@ -242,7 +250,7 @@ fn recovery_refuses_to_skip_a_discontinuous_historical_head() { storage.read_head().unwrap().as_deref(), Some(junk_id.as_str()) ); - assert_eq!(client_state(&repo)["head_id"], floor); + assert_eq!(machine_floor(&repo)["head_id"], floor); } #[test] @@ -267,7 +275,11 @@ fn recover_refuses_a_head_with_a_future_manifest_schema() { .compare_and_swap_head(Some(&junk_id), &future_id) .unwrap(); - let report = publish_recovery(&EncryptedRepository::new(storage.clone(), key), &repo).unwrap(); + let report = publish_recovery( + &EncryptedRepository::new(storage.clone(), key, support::state_dir()), + &repo, + ) + .unwrap(); assert_eq!( report.classification, RecoveryClass::Unsupported, @@ -279,7 +291,7 @@ fn recover_refuses_a_head_with_a_future_manifest_schema() { storage.read_head().unwrap().as_deref(), Some(future_id.as_str()) ); - assert_eq!(client_state(&repo)["head_id"], floor); + assert_eq!(machine_floor(&repo)["head_id"], floor); } impl Storage for BarrierStorage { @@ -337,8 +349,14 @@ fn commit(repo: &Path, contents: &str, message: &str) -> String { git(repo, &["rev-parse", "HEAD"]) } -fn client_state(repo: &Path) -> serde_json::Value { - let path = state_dir(repo).join("state.json"); +/// This machine's continuity floor for the repository `repo` is a checkout of. +fn machine_floor(repo: &Path) -> serde_json::Value { + let directory = state_dir(repo); + support::read_floor(directory.file_name().unwrap().to_str().unwrap()) +} + +fn progress(repo: &Path) -> serde_json::Value { + let path = state_dir(repo).join("progress.json"); serde_json::from_slice(&fs::read(path).unwrap()).unwrap() } @@ -389,18 +407,33 @@ fn shallow_push_is_rejected_before_publication_and_unshallow_recovers() { "true" ); let key = KeyFile::generate(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&storage), key); + let encrypted = + EncryptedRepository::new(FilesystemStorage::new(&storage), key, support::state_dir()); encrypted.initialize("owner").unwrap(); let head_before = fs::read(storage.join("HEAD")).unwrap(); let error = encrypted - .push_ref(&shallow, "refs/heads/main", false) + .push( + &shallow, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap_err(); assert!(error.to_string().contains("shallow"), "{error:#}"); assert_eq!(fs::read(storage.join("HEAD")).unwrap(), head_before); assert!(encrypted.verify().unwrap().refs.is_empty()); git(&shallow, &["fetch", "--unshallow"]); encrypted - .push_ref(&shallow, "refs/heads/main", false) + .push( + &shallow, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); initialize_git(&destination); encrypted.fetch_into(&destination, "encrypted").unwrap(); @@ -424,12 +457,23 @@ fn pushes_incrementally_and_fetches_into_another_repository() { initialize_git(&destination); let key = KeyFile::generate(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key.clone()); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key.clone(), + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); let first = commit(&source, "first\n", "first"); encrypted - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); let first_state = encrypted.verify().unwrap(); assert_eq!(first_state.generation, 1); @@ -437,7 +481,14 @@ fn pushes_incrementally_and_fetches_into_another_repository() { let second = commit(&source, "second\n", "second"); encrypted - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); let second_state = encrypted.verify().unwrap(); assert_eq!(second_state.generation, 2); @@ -460,7 +511,7 @@ fn pushes_incrementally_and_fetches_into_another_repository() { } #[test] -fn missing_pack_observation_does_not_advance_the_floor() { +fn missing_pack_observation_does_not_advance_import_progress() { let temporary = tempfile::tempdir().unwrap(); let source = temporary.path().join("source"); let destination = temporary.path().join("destination"); @@ -469,30 +520,36 @@ fn missing_pack_observation_does_not_advance_the_floor() { initialize_git(&destination); let key = KeyFile::generate(); - let repository = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key); + let repository = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ); repository.initialize("owner").unwrap(); commit(&source, "first\n", "first"); repository - .push_update_for_remote( + .push( &source, - "encrypted", - "refs/heads/main", "refs/heads/main", - false, + RefChange::Update { + source: "refs/heads/main", + force: false, + }, ) .unwrap(); repository.fetch_into(&destination, "encrypted").unwrap(); - assert_eq!(client_state(&destination)["generation"], 1); - assert_eq!(client_state(&destination)["imported_generation"], 1); + assert_eq!(machine_floor(&destination)["generation"], 1); + assert_eq!(progress(&destination)["imported_generation"], 1); commit(&source, "second\n", "second"); repository - .push_update_for_remote( + .push( &source, - "encrypted", - "refs/heads/main", "refs/heads/main", - false, + RefChange::Update { + source: "refs/heads/main", + force: false, + }, ) .unwrap(); let (_, manifest) = repository.current_manifest().unwrap(); @@ -503,20 +560,19 @@ fn missing_pack_observation_does_not_advance_the_floor() { let listed = repository.observe_manifest(&destination).unwrap(); assert_eq!(listed.generation, 2); - assert_eq!(client_state(&destination)["generation"], 1); + assert_eq!(progress(&destination)["imported_generation"], 1); let error = repository .fetch_into(&destination, "encrypted") .unwrap_err() .to_string(); assert!(error.contains(&pack_id), "unexpected error: {error}"); - assert_eq!(client_state(&destination)["generation"], 1); + assert_eq!(progress(&destination)["imported_generation"], 1); write_stored_object(&remote_path, "packs", &pack_id, &saved_pack); let fetched = repository.fetch_into(&destination, "encrypted").unwrap(); assert_eq!(fetched.generation, 2); - let state = client_state(&destination); - assert_eq!(state["generation"], 2); - assert_eq!(state["imported_generation"], 2); + assert_eq!(machine_floor(&destination)["generation"], 2); + assert_eq!(progress(&destination)["imported_generation"], 2); } #[test] @@ -527,24 +583,30 @@ fn successful_own_publication_advances_the_client_floor() { initialize_git(&source); let key = KeyFile::generate(); - let repository = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key); + let repository = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ); repository.initialize("owner").unwrap(); commit(&source, "published\n", "published"); let head = repository - .push_update_for_remote( + .push( &source, - "encrypted", - "refs/heads/main", "refs/heads/main", - false, + RefChange::Update { + source: "refs/heads/main", + force: false, + }, ) .unwrap(); - let state_path = state_dir(&source).join("state.json"); - let state: serde_json::Value = serde_json::from_slice(&fs::read(state_path).unwrap()).unwrap(); - assert_eq!(state["head_id"], head); - assert_eq!(state["generation"], 1); - assert_eq!(state["imported_generation"], serde_json::Value::Null); + assert_eq!(machine_floor(&source)["head_id"], head); + assert_eq!(machine_floor(&source)["generation"], 1); + assert_eq!( + progress(&source)["imported_generation"], + serde_json::Value::Null + ); } #[test] @@ -556,19 +618,34 @@ fn returning_client_fetches_multiple_offline_generations_from_deltas() { initialize_git(&source); initialize_git(&destination); let key = KeyFile::generate(); - let repository = EncryptedRepository::new(FilesystemStorage::new(&remote), key); + let repository = + EncryptedRepository::new(FilesystemStorage::new(&remote), key, support::state_dir()); repository.initialize("owner").unwrap(); commit(&source, "one\n", "one"); repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); repository.fetch_into(&destination, "e2ee").unwrap(); for value in ["two\n", "three\n", "four\n"] { commit(&source, value, value.trim()); repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); } let latest = repository.fetch_into(&destination, "e2ee").unwrap(); @@ -588,11 +665,22 @@ fn returning_client_rejects_signed_ref_advance_without_its_pack() { initialize_git(&source); initialize_git(&destination); let key = KeyFile::generate(); - let repository = EncryptedRepository::new(FilesystemStorage::new(&remote), key.clone()); + let repository = EncryptedRepository::new( + FilesystemStorage::new(&remote), + key.clone(), + support::state_dir(), + ); repository.initialize("owner").unwrap(); commit(&source, "valid\n", "valid"); repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); repository.fetch_into(&destination, "e2ee").unwrap(); @@ -656,8 +744,8 @@ fn returning_client_rejects_signed_ref_advance_without_its_pack() { error.contains("does not resolve") || error.contains("missing Git objects"), "unexpected connectivity error: {error}" ); - assert_eq!(client_state(&destination)["generation"], 1); - assert_eq!(client_state(&destination)["imported_generation"], 1); + assert_eq!(machine_floor(&destination)["generation"], 1); + assert_eq!(progress(&destination)["imported_generation"], 1); } #[test] @@ -670,15 +758,23 @@ fn missing_verified_frontier_tip_forces_full_reverification() { initialize_git(&destination); let key = KeyFile::generate(); - let repository = EncryptedRepository::new(FilesystemStorage::new(&remote), key); + let repository = + EncryptedRepository::new(FilesystemStorage::new(&remote), key, support::state_dir()); repository.initialize("owner").unwrap(); commit(&source, "valid\n", "valid"); repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); repository.fetch_into(&destination, "e2ee").unwrap(); - let state_path = state_dir(&destination).join("state.json"); + let state_path = state_dir(&destination).join("progress.json"); let mut state: serde_json::Value = serde_json::from_slice(&fs::read(&state_path).unwrap()).unwrap(); state["verified_refs"]["refs/heads/main"] = serde_json::Value::String("1".repeat(40)); @@ -732,39 +828,39 @@ fn local_publication_advances_verified_frontier_before_observation() { initialize_git(&source); let key = KeyFile::generate(); - let repository = EncryptedRepository::new(FilesystemStorage::new(&remote), key); + let repository = + EncryptedRepository::new(FilesystemStorage::new(&remote), key, support::state_dir()); repository.initialize("owner").unwrap(); let first = commit(&source, "first\n", "first"); repository - .push_update_for_remote( + .push( &source, - "origin", "refs/heads/main", - "refs/heads/main", - false, + RefChange::Update { + source: "refs/heads/main", + force: false, + }, ) .unwrap(); - assert_eq!( - client_state(&source)["verified_refs"]["refs/heads/main"], - first - ); + assert_eq!(progress(&source)["verified_refs"]["refs/heads/main"], first); let observed = repository.observe_manifest(&source).unwrap(); assert_eq!(observed.refs.get("refs/heads/main"), Some(&first)); let second = commit(&source, "second\n", "second"); repository - .push_update_for_remote( + .push( &source, - "origin", - "refs/heads/main", "refs/heads/main", - false, + RefChange::Update { + source: "refs/heads/main", + force: false, + }, ) .unwrap(); assert_eq!( - client_state(&source)["verified_refs"]["refs/heads/main"], + progress(&source)["verified_refs"]["refs/heads/main"], second ); } @@ -787,21 +883,46 @@ fn rejects_non_fast_forward_update() { initialize_git(&source); let key = KeyFile::generate(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); let first = commit(&source, "first\n", "first"); encrypted - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); commit(&source, "second\n", "second"); encrypted - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); git(&source, &["reset", "--hard", &first]); commit(&source, "diverged\n", "diverged"); let error = encrypted - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap_err(); assert!(error.to_string().contains("non-fast-forward")); assert_eq!(encrypted.verify().unwrap().generation, 2); @@ -817,20 +938,45 @@ fn explicit_force_can_publish_a_rollback() { initialize_git(&destination); let key = KeyFile::generate(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); let first = commit(&source, "first\n", "first"); encrypted - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); commit(&source, "second\n", "second"); encrypted - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); git(&source, &["reset", "--hard", &first]); encrypted - .push_ref(&source, "refs/heads/main", true) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: true, + }, + ) .unwrap(); assert_eq!(encrypted.verify().unwrap().refs["refs/heads/main"], first); @@ -849,11 +995,22 @@ fn detects_tampered_pack_ciphertext() { initialize_git(&source); let key = KeyFile::generate(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); commit(&source, "secret\n", "secret"); encrypted - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); let manifest = encrypted.verify().unwrap(); let id = &manifest.new_packs[0].id; @@ -889,10 +1046,21 @@ fn late_stream_authentication_failure_does_not_advance_refs_or_pins() { git(&source, &["commit", "-q", "-m", "large"]); let key = KeyFile::generate(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); encrypted - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); let manifest = encrypted.current_manifest().unwrap().1; let id = &manifest.new_packs[0].id; @@ -904,7 +1072,7 @@ fn late_stream_authentication_failure_does_not_advance_refs_or_pins() { fs::write(path, bytes).unwrap(); assert!(encrypted.fetch_into(&destination, "encrypted").is_err()); - assert!(!state_dir(&destination).join("state.json").exists()); + assert!(!state_dir(&destination).join("progress.json").exists()); let output = Command::new("git") .arg("-C") .arg(&destination) @@ -924,15 +1092,33 @@ fn rejects_storage_head_rollback_after_fetch_pins_history() { initialize_git(&destination); let key = KeyFile::generate(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); commit(&source, "first\n", "first"); let first_head = encrypted - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); commit(&source, "second\n", "second"); encrypted - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); encrypted.fetch_into(&destination, "encrypted").unwrap(); @@ -952,17 +1138,41 @@ fn rejects_same_generation_manifest_fork_after_fetch_pins_history() { initialize_git(&destination); let key = KeyFile::generate(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key.clone()); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key.clone(), + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); commit(&source, "first\n", "first"); encrypted - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); encrypted.fetch_into(&destination, "encrypted").unwrap(); - let fork = EncryptedRepository::new(FilesystemStorage::new(&fork_path), key); + let fork = EncryptedRepository::new( + FilesystemStorage::new(&fork_path), + key, + temporary.path().join("forker-machine"), + ); fork.initialize("owner").unwrap(); - let fork_head = fork.push_ref(&source, "refs/heads/main", false).unwrap(); + let fork_head = fork + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) + .unwrap(); copy_tree(&fork_path.join("manifests"), &remote_path.join("manifests")); fs::write(remote_path.join("HEAD"), format!("{fork_head}\n")).unwrap(); @@ -983,28 +1193,31 @@ fn recover_replaces_reader_signed_junk_from_the_directory_floor() { let owner = KeyFile::generate(); let reader = KeyFile::generate_for_repository(owner.repository_root.clone()).unwrap(); - let admin_pin = temporary.path().join("owner.admin-state.json"); let storage = FilesystemStorage::new(&storage_path); - let repository = EncryptedRepository::new(storage.clone(), owner.clone()); + let repository = EncryptedRepository::new(storage.clone(), owner.clone(), support::state_dir()); repository.initialize("owner").unwrap(); - repository.pin_admin_state(&admin_pin).unwrap(); commit(&source, "recovery keeps this content\n", "base"); repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); repository .add_device( reader.public_device().unwrap(), "device-1", DeviceRoles::reader(), - &admin_pin, ) .unwrap(); repository.fetch_into(&returning, "encrypted").unwrap(); let (floor_id, floor_manifest) = repository.current_manifest().unwrap(); let floor_generation = floor_manifest.generation; - let floor_state = client_state(&returning); - assert_eq!(floor_state["head_id"], floor_id); + assert_eq!(machine_floor(&returning)["head_id"], floor_id); let encrypted = storage.get_object(ObjectKind::Manifest, &floor_id).unwrap(); let header = peek_manifest_header(&encrypted).unwrap(); @@ -1052,7 +1265,7 @@ fn recover_replaces_reader_signed_junk_from_the_directory_floor() { .compare_and_swap_head(Some(&floor_id), &junk_id) .unwrap(); - let recoverer = EncryptedRepository::new(storage.clone(), owner.clone()); + let recoverer = EncryptedRepository::new(storage.clone(), owner.clone(), support::state_dir()); let blocked = recoverer .recover( &returning, @@ -1080,7 +1293,7 @@ fn recover_replaces_reader_signed_junk_from_the_directory_floor() { .unwrap() .contains("--accept-stale-floor") ); - assert_eq!(client_state(&returning)["head_id"], floor_id); + assert_eq!(machine_floor(&returning)["head_id"], floor_id); let recovered = recoverer .recover( @@ -1099,7 +1312,7 @@ fn recover_replaces_reader_signed_junk_from_the_directory_floor() { assert_eq!(current.previous.as_deref(), Some(floor_id.as_str())); assert_eq!(current.generation, floor_generation + 1); assert_ne!(current_id, junk_id); - assert_eq!(client_state(&returning)["head_id"], current_id); + assert_eq!(machine_floor(&returning)["head_id"], current_id); let rejected = fs::read(state_dir(&returning).join("rejected-heads.json")).unwrap(); assert!(String::from_utf8_lossy(&rejected).contains(&junk_id)); @@ -1109,8 +1322,8 @@ fn recover_replaces_reader_signed_junk_from_the_directory_floor() { git(&fresh, &["show", "refs/remotes/encrypted/main:note.md"]), "recovery keeps this content" ); - assert_eq!(client_state(&returning)["head_id"], current_id); - assert_eq!(client_state(&fresh)["head_id"], current_id); + assert_eq!(machine_floor(&returning)["head_id"], current_id); + assert_eq!(machine_floor(&fresh)["head_id"], current_id); } #[test] @@ -1121,32 +1334,31 @@ fn recover_refuses_a_revoked_devices_unverifiable_head() { initialize_git(&reader_repo); let owner = KeyFile::generate(); let reader = KeyFile::generate_for_repository(owner.repository_root.clone()).unwrap(); - let admin_pin = temporary.path().join("owner.admin-state.json"); let storage = FilesystemStorage::new(&storage_path); - let owner_repository = EncryptedRepository::new(storage.clone(), owner); + let owner_repository = EncryptedRepository::new(storage.clone(), owner, support::state_dir()); owner_repository.initialize("owner").unwrap(); - owner_repository.pin_admin_state(&admin_pin).unwrap(); owner_repository .add_device( reader.public_device().unwrap(), "device-2", DeviceRoles::reader(), - &admin_pin, ) .unwrap(); - owner_repository + let reader_machine = temporary.path().join("reader-machine"); + let reader_floor = reader_machine + .join(&reader.repository_root) + .join("continuity.json"); + let reader_repository = + EncryptedRepository::new(storage.clone(), reader.clone(), &reader_machine); + reader_repository .fetch_into(&reader_repo, "encrypted") .unwrap(); - let old_floor = client_state(&reader_repo)["head_id"] - .as_str() - .unwrap() - .to_owned(); + let old_floor = fs::read(&reader_floor).unwrap(); owner_repository - .revoke_device(&reader.device_id().unwrap(), &admin_pin) + .revoke_device(&reader.device_id().unwrap()) .unwrap(); let revoked_head = storage.read_head().unwrap().unwrap(); - let reader_repository = EncryptedRepository::new(storage.clone(), reader); let report = reader_repository .recover( &reader_repo, @@ -1164,7 +1376,7 @@ fn recover_refuses_a_revoked_devices_unverifiable_head() { storage.read_head().unwrap().as_deref(), Some(revoked_head.as_str()) ); - assert_eq!(client_state(&reader_repo)["head_id"], old_floor); + assert_eq!(fs::read(&reader_floor).unwrap(), old_floor); assert!(!state_dir(&reader_repo).join("rejected-heads.json").exists()); } @@ -1177,7 +1389,8 @@ fn recover_refuses_unknown_formats_and_missing_floor_objects() { initialize_git(&unknown_repo); let key = KeyFile::generate(); let unknown_storage = FilesystemStorage::new(&unknown_storage_path); - let unknown_repository = EncryptedRepository::new(unknown_storage.clone(), key.clone()); + let unknown_repository = + EncryptedRepository::new(unknown_storage.clone(), key.clone(), support::state_dir()); unknown_repository.initialize("owner").unwrap(); unknown_repository .fetch_into(&unknown_repo, "encrypted") @@ -1207,7 +1420,7 @@ fn recover_refuses_unknown_formats_and_missing_floor_objects() { unknown_storage .compare_and_swap_head(Some(&floor_id), &unsupported_id) .unwrap(); - let unsupported = EncryptedRepository::new(unknown_storage.clone(), key) + let unsupported = EncryptedRepository::new(unknown_storage.clone(), key, support::state_dir()) .recover( &unknown_repo, RecoveryOptions { @@ -1229,13 +1442,24 @@ fn recover_refuses_unknown_formats_and_missing_floor_objects() { initialize_git(&missing_repo); let missing_key = KeyFile::generate(); let missing_storage = FilesystemStorage::new(&missing_storage_path); - let missing_repository = EncryptedRepository::new(missing_storage.clone(), missing_key.clone()); + let missing_repository = EncryptedRepository::new( + missing_storage.clone(), + missing_key.clone(), + support::state_dir(), + ); missing_repository.initialize("owner").unwrap(); let source = temporary.path().join("missing-source"); initialize_git(&source); commit(&source, "required floor object\n", "required"); missing_repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); missing_repository .fetch_into(&missing_repo, "encrypted") @@ -1270,17 +1494,18 @@ fn recover_refuses_unknown_formats_and_missing_floor_objects() { missing_storage .compare_and_swap_head(Some(&floor_id), &invalid_id) .unwrap(); - let refused = EncryptedRepository::new(missing_storage.clone(), missing_key) - .recover( - &missing_repo, - RecoveryOptions { - publish: true, - base: None, - discard_newer: false, - accept_stale_floor: true, - }, - ) - .unwrap(); + let refused = + EncryptedRepository::new(missing_storage.clone(), missing_key, support::state_dir()) + .recover( + &missing_repo, + RecoveryOptions { + publish: true, + base: None, + discard_newer: false, + accept_stale_floor: true, + }, + ) + .unwrap(); assert_eq!(refused.classification, RecoveryClass::Invalid); assert!( refused @@ -1293,7 +1518,7 @@ fn recover_refuses_unknown_formats_and_missing_floor_objects() { missing_storage.read_head().unwrap().as_deref(), Some(invalid_id.as_str()) ); - assert_eq!(client_state(&missing_repo)["head_id"], floor_id); + assert_eq!(machine_floor(&missing_repo)["head_id"], floor_id); } #[test] @@ -1306,15 +1531,29 @@ fn recover_never_overrides_authenticated_rollback() { initialize_git(&returning); let key = KeyFile::generate(); let storage = FilesystemStorage::new(&storage_path); - let repository = EncryptedRepository::new(storage.clone(), key); + let repository = EncryptedRepository::new(storage.clone(), key, support::state_dir()); repository.initialize("owner").unwrap(); commit(&source, "one\n", "one"); let first = repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); commit(&source, "two\n", "two"); repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); repository.fetch_into(&returning, "encrypted").unwrap(); storage @@ -1348,7 +1587,7 @@ fn recovery_discovery_budget_exhaustion_fails_closed() { initialize_git(&repo); let key = KeyFile::generate(); let inner = FilesystemStorage::new(&storage_path); - let repository = EncryptedRepository::new(inner.clone(), key.clone()); + let repository = EncryptedRepository::new(inner.clone(), key.clone(), support::state_dir()); repository.initialize("owner").unwrap(); repository.fetch_into(&repo, "encrypted").unwrap(); let floor = inner.read_head().unwrap().unwrap(); @@ -1366,6 +1605,7 @@ fn recovery_discovery_budget_exhaustion_fails_closed() { inner: inner.clone(), }, key, + support::state_dir(), ); let error = recovery .recover( @@ -1383,7 +1623,7 @@ fn recovery_discovery_budget_exhaustion_fails_closed() { inner.read_head().unwrap().as_deref(), Some(invalid_id.as_str()) ); - assert_eq!(client_state(&repo)["head_id"], floor); + assert_eq!(machine_floor(&repo)["head_id"], floor); } #[test] @@ -1396,11 +1636,22 @@ fn writer_can_add_branch_without_having_other_remote_branch_objects() { initialize_git(&second_writer); let key = KeyFile::generate(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); let main = commit(&first_writer, "main\n", "main"); encrypted - .push_ref(&first_writer, "refs/heads/main", false) + .push( + &first_writer, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); encrypted.fetch_into(&second_writer, "encrypted").unwrap(); git( @@ -1411,17 +1662,25 @@ fn writer_can_add_branch_without_having_other_remote_branch_objects() { git(&first_writer, &["switch", "-q", "-c", "feature"]); let feature = commit(&first_writer, "feature\n", "feature"); encrypted - .push_ref(&first_writer, "refs/heads/feature", false) + .push( + &first_writer, + "refs/heads/feature", + RefChange::Update { + source: "refs/heads/feature", + force: false, + }, + ) .unwrap(); let other = commit(&second_writer, "other\n", "other"); encrypted - .push_update_for_remote( + .push( &second_writer, - "encrypted", - "refs/heads/other", "refs/heads/other", - false, + RefChange::Update { + source: "refs/heads/other", + force: false, + }, ) .unwrap(); @@ -1456,15 +1715,33 @@ fn deletion_and_tag_publication_race_with_exactly_one_cas_winner() { let remote = temporary.path().join("remote"); initialize_git(&source); let key = KeyFile::generate(); - let seed_repository = EncryptedRepository::new(FilesystemStorage::new(&remote), key.clone()); + let seed_repository = EncryptedRepository::new( + FilesystemStorage::new(&remote), + key.clone(), + support::state_dir(), + ); seed_repository.initialize("owner").unwrap(); let main = commit(&source, "main\n", "main"); seed_repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); git(&source, &["branch", "doomed", &main]); seed_repository - .push_ref(&source, "refs/heads/doomed", false) + .push( + &source, + "refs/heads/doomed", + RefChange::Update { + source: "refs/heads/doomed", + force: false, + }, + ) .unwrap(); git(&source, &["tag", "race-tag", &main]); let before = seed_repository.verify().unwrap(); @@ -1475,13 +1752,26 @@ fn deletion_and_tag_publication_race_with_exactly_one_cas_winner() { reads: Arc::new(AtomicUsize::new(0)), barrier: Arc::new(Barrier::new(2)), }; - let deleter = EncryptedRepository::new(storage.clone(), key.clone()); - let tagger = EncryptedRepository::new(storage, key); + // Two machines: one machine's floor lock would serialize the racers. + let deleter = EncryptedRepository::new( + storage.clone(), + key.clone(), + temporary.path().join("deleter-machine"), + ); + let tagger = EncryptedRepository::new(storage, key, temporary.path().join("tagger-machine")); let deletion_repo = source.clone(); let tag_repo = source.clone(); - let deletion = thread::spawn(move || deleter.delete_ref(&deletion_repo, "refs/heads/doomed")); + let deletion = + thread::spawn(move || deleter.push(&deletion_repo, "refs/heads/doomed", RefChange::Delete)); let tag_push = thread::spawn(move || { - tagger.push_update(&tag_repo, "refs/tags/race-tag", "refs/tags/race-tag", false) + tagger.push( + &tag_repo, + "refs/tags/race-tag", + RefChange::Update { + source: "refs/tags/race-tag", + force: false, + }, + ) }); let deletion = deletion.join().unwrap(); let tag_push = tag_push.join().unwrap(); @@ -1517,11 +1807,22 @@ fn rejects_unsupported_ref_namespaces_consistently() { initialize_git(&source); let key = KeyFile::generate(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); commit(&source, "tagged\n", "tagged"); let error = encrypted - .push_update(&source, "refs/heads/main", "refs/notes/review", false) + .push( + &source, + "refs/notes/review", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap_err(); assert!(error.to_string().contains("unsupported ref namespace")); assert_eq!(encrypted.verify().unwrap().generation, 0); @@ -1531,6 +1832,7 @@ fn git_e2ee(workdir: &Path, args: &[impl AsRef]) -> std::proces let output = Command::new(env!("CARGO_BIN_EXE_git-e2ee")) .current_dir(workdir) .env("XDG_STATE_HOME", workdir.join("state")) + .env("XDG_CONFIG_HOME", workdir.join("config")) .args(args) .output() .unwrap(); @@ -1616,12 +1918,22 @@ fn directory_root_holds_exactly_the_v6_layout() { let source = temporary.path().join("source"); let remote_path = temporary.path().join("remote"); initialize_git(&source); - let encrypted = - EncryptedRepository::new(FilesystemStorage::new(&remote_path), KeyFile::generate()); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + KeyFile::generate(), + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); commit(&source, "layout\n", "layout"); encrypted - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); // HEAD.lock and .staging are local coordination files of this backend. @@ -1654,12 +1966,22 @@ fn directory_format_mismatch_names_the_found_format() { let source = temporary.path().join("source"); let remote_path = temporary.path().join("remote"); initialize_git(&source); - let encrypted = - EncryptedRepository::new(FilesystemStorage::new(&remote_path), KeyFile::generate()); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + KeyFile::generate(), + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); commit(&source, "format\n", "format"); encrypted - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); fs::write(remote_path.join("FORMAT"), "git-remote-e2ee storage 7\n").unwrap(); diff --git a/tests/git_compatibility.rs b/tests/git_compatibility.rs index 8588106..ca43157 100644 --- a/tests/git_compatibility.rs +++ b/tests/git_compatibility.rs @@ -2,6 +2,8 @@ //! Git upstream's `t/t5801-remote-helpers.sh`. This is an independent Rust //! implementation, not a copy of Git's GPL test code. +mod support; + use std::env; use std::fs; use std::path::Path; @@ -15,7 +17,9 @@ fn add_helper_to_path(command: &mut Command) { let helper = Path::new(env!("CARGO_BIN_EXE_git-remote-e2ee")); let mut paths = vec![helper.parent().unwrap().to_path_buf()]; paths.extend(env::split_paths(&env::var_os("PATH").unwrap_or_default())); - command.env("PATH", env::join_paths(paths).unwrap()); + command + .env("PATH", env::join_paths(paths).unwrap()) + .env("XDG_STATE_HOME", support::state_home()); } fn git_output(repo: &Path, args: &[&str], with_helper: bool) -> Output { @@ -62,9 +66,13 @@ fn setup_remote(root: &Path) -> (std::path::PathBuf, std::path::PathBuf, KeyFile let key_path = root.join("repository.key.json"); let key = KeyFile::generate(); key.write_new(&key_path).unwrap(); - EncryptedRepository::new(FilesystemStorage::new(&storage), key.clone()) - .initialize("owner") - .unwrap(); + EncryptedRepository::new( + FilesystemStorage::new(&storage), + key.clone(), + support::state_dir(), + ) + .initialize("owner") + .unwrap(); (storage, key_path, key) } @@ -166,7 +174,11 @@ fn upstream_style_push_refspecs_and_force_work() { git(&source, &["push", "private", "named:renamed"], true); git(&source, &["push", "private", "HEAD:from-head"], true); - let repository = EncryptedRepository::new(FilesystemStorage::new(&storage), key.clone()); + let repository = EncryptedRepository::new( + FilesystemStorage::new(&storage), + key.clone(), + support::state_dir(), + ); let manifest = repository.verify().unwrap(); for reference in ["named", "renamed", "from-head"] { assert_eq!(manifest.refs[&format!("refs/heads/{reference}")], named); @@ -256,7 +268,8 @@ fn lightweight_annotated_and_followed_tags_round_trip_through_helper() { ); git(&source, &["push", "--follow-tags", "private", "main"], true); - let repository = EncryptedRepository::new(FilesystemStorage::new(&storage), key); + let repository = + EncryptedRepository::new(FilesystemStorage::new(&storage), key, support::state_dir()); let manifest = repository.verify().unwrap(); assert_eq!(manifest.refs["refs/tags/v1.0"], first); assert_eq!( @@ -303,7 +316,8 @@ fn moving_an_existing_tag_requires_force() { let second = commit_file(&source, "second", "second"); git(&source, &["push", "private", "main"], true); git(&source, &["tag", "-f", "v1.0", &second], false); - let repository = EncryptedRepository::new(FilesystemStorage::new(&storage), key); + let repository = + EncryptedRepository::new(FilesystemStorage::new(&storage), key, support::state_dir()); let before = repository.verify().unwrap(); let rejected = git_output(&source, &["push", "private", "v1.0"], true); assert!(!rejected.status.success()); @@ -337,7 +351,8 @@ fn deletions_publish_without_packs_and_prune_on_returning_and_fresh_clones() { git(&source, &["push", "private", "v-delete"], true); clone_remote(&returning, &storage, &key_path); - let repository = EncryptedRepository::new(FilesystemStorage::new(&storage), key); + let repository = + EncryptedRepository::new(FilesystemStorage::new(&storage), key, support::state_dir()); let before = repository.verify().unwrap(); let default_delete = git_output(&source, &["push", "--delete", "private", "main"], true); assert!(!default_delete.status.success()); @@ -423,7 +438,8 @@ fn unsupported_upstream_operations_fail_without_publication() { let head = commit_file(&source, "main", "main"); git(&source, &["push", "private", "main"], true); - let repository = EncryptedRepository::new(FilesystemStorage::new(&storage), key); + let repository = + EncryptedRepository::new(FilesystemStorage::new(&storage), key, support::state_dir()); let generation = repository.verify().unwrap().generation; let delete = git_output(&source, &["push", "private", ":main"], true); @@ -525,7 +541,8 @@ fn stale_push_is_reported_as_an_ordinary_git_rejection() { ); assert!(!stderr.contains("remote rejected"), "{stderr}"); assert!(stderr.contains("git pull"), "{stderr}"); - let repository = EncryptedRepository::new(FilesystemStorage::new(&storage), key); + let repository = + EncryptedRepository::new(FilesystemStorage::new(&storage), key, support::state_dir()); assert_eq!( repository.verify().unwrap().refs["refs/heads/main"], published diff --git a/tests/multi_device.rs b/tests/multi_device.rs index fbdeaa8..268ee40 100644 --- a/tests/multi_device.rs +++ b/tests/multi_device.rs @@ -4,7 +4,7 @@ use std::process::Command; use git_remote_e2ee::crypto::KeyFile; use git_remote_e2ee::policy::DeviceRoles; -use git_remote_e2ee::repository::EncryptedRepository; +use git_remote_e2ee::repository::{EncryptedRepository, RefChange}; use git_remote_e2ee::storage::FilesystemStorage; fn git(repo: &Path, args: &[&str]) -> String { @@ -23,15 +23,6 @@ fn git(repo: &Path, args: &[&str]) -> String { String::from_utf8(output.stdout).unwrap().trim().to_owned() } -fn state_dir(repo: &Path) -> std::path::PathBuf { - let entries: Vec<_> = fs::read_dir(repo.join(".git/git-remote-e2ee")) - .unwrap() - .map(|entry| entry.unwrap().path()) - .collect(); - assert_eq!(entries.len(), 1, "{entries:?}"); - entries.into_iter().next().unwrap() -} - fn initialize_git(repo: &Path) { fs::create_dir_all(repo).unwrap(); git(repo, &["init", "-q", "-b", "main"]); @@ -55,29 +46,41 @@ fn two_devices_can_decrypt_and_push_independently() { let remote = temp.path().join("remote"); let source_a = temp.path().join("a"); let source_b = temp.path().join("b"); - let admin_pin = temp.path().join("a-admin-state.json"); initialize_git(&source_a); initialize_git(&source_b); let key_a = KeyFile::generate(); let key_b = KeyFile::generate_for_repository(key_a.repository_root.clone()).unwrap(); - let repo_a = EncryptedRepository::new(FilesystemStorage::new(&remote), key_a.clone()); + let repo_a = EncryptedRepository::new( + FilesystemStorage::new(&remote), + key_a.clone(), + temp.path().join("a-machine"), + ); repo_a.initialize("owner").unwrap(); - repo_a.pin_admin_state(&admin_pin).unwrap(); let first = commit(&source_a, "from a\n", "from a"); repo_a - .push_ref(&source_a, "refs/heads/main", false) + .push( + &source_a, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); repo_a .add_device( key_b.public_device().unwrap(), "device-1", DeviceRoles::collaborator(), - &admin_pin, ) .unwrap(); - let repo_b = EncryptedRepository::new(FilesystemStorage::new(&remote), key_b.clone()); + let repo_b = EncryptedRepository::new( + FilesystemStorage::new(&remote), + key_b.clone(), + temp.path().join("b-machine"), + ); repo_b.fetch_into(&source_b, "e2ee").unwrap(); assert_eq!( git(&source_b, &["rev-parse", "refs/remotes/e2ee/main"]), @@ -89,7 +92,14 @@ fn two_devices_can_decrypt_and_push_independently() { ); let second = commit(&source_b, "from b\n", "from b"); repo_b - .push_ref(&source_b, "refs/heads/main", false) + .push( + &source_b, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); let destination = temp.path().join("destination"); @@ -107,31 +117,54 @@ fn revoked_reader_cannot_reach_later_history_but_keeps_prior_snapshot() { let remote = temp.path().join("remote"); let source = temp.path().join("source"); let client_b = temp.path().join("client-b"); - let admin_pin = temp.path().join("a-admin-state.json"); initialize_git(&source); initialize_git(&client_b); let key_a = KeyFile::generate(); let key_b = KeyFile::generate_for_repository(key_a.repository_root.clone()).unwrap(); let key_c = KeyFile::generate_for_repository(key_a.repository_root.clone()).unwrap(); - let repo_a = EncryptedRepository::new(FilesystemStorage::new(&remote), key_a.clone()); + let repo_a = EncryptedRepository::new( + FilesystemStorage::new(&remote), + key_a.clone(), + temp.path().join("a-machine"), + ); repo_a.initialize("owner").unwrap(); - repo_a.pin_admin_state(&admin_pin).unwrap(); let before = commit(&source, "before revoke\n", "before revoke"); - repo_a.push_ref(&source, "refs/heads/main", false).unwrap(); + repo_a + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) + .unwrap(); let before_tip = commit(&source, "second pack\n", "second pack"); - repo_a.push_ref(&source, "refs/heads/main", false).unwrap(); + repo_a + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) + .unwrap(); assert_eq!(count_files(&remote.join("packs")), 2); repo_a .add_device( key_b.public_device().unwrap(), "device-2", DeviceRoles::collaborator(), - &admin_pin, ) .unwrap(); assert_eq!(count_files(&remote.join("packs")), 2); - let repo_b = EncryptedRepository::new(FilesystemStorage::new(&remote), key_b.clone()); + let repo_b = EncryptedRepository::new( + FilesystemStorage::new(&remote), + key_b.clone(), + temp.path().join("b-machine"), + ); repo_b.fetch_into(&client_b, "e2ee").unwrap(); assert_eq!( git(&client_b, &["show", "refs/remotes/e2ee/main:note.md"]), @@ -145,14 +178,21 @@ fn revoked_reader_cannot_reach_later_history_but_keeps_prior_snapshot() { copy_directory(&remote, &before_revoke); let old_objects = snapshot_files(&remote.join("packs")); - repo_a - .revoke_device(&key_b.device_id().unwrap(), &admin_pin) - .unwrap(); + repo_a.revoke_device(&key_b.device_id().unwrap()).unwrap(); assert_eq!(count_files(&remote.join("packs")), 2); assert!(repo_b.current_manifest().is_err()); let after = commit(&source, "after revoke secret\n", "after revoke"); - repo_a.push_ref(&source, "refs/heads/main", false).unwrap(); + repo_a + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) + .unwrap(); assert_eq!(count_files(&remote.join("packs")), 3); for (path, contents) in old_objects { assert_eq!(fs::read(path).unwrap(), contents); @@ -174,12 +214,13 @@ fn revoked_reader_cannot_reach_later_history_but_keeps_prior_snapshot() { commit(&client_b, "revoked writer\n", "revoked writer"); let push_error = repo_b - .push_update_for_remote( + .push( &client_b, - "e2ee", - "refs/heads/main", "refs/heads/main", - false, + RefChange::Update { + source: "refs/heads/main", + force: false, + }, ) .unwrap_err() .to_string(); @@ -191,8 +232,11 @@ fn revoked_reader_cannot_reach_later_history_but_keeps_prior_snapshot() { let snapshot_destination = temp.path().join("snapshot-destination"); initialize_git(&snapshot_destination); - let old_repo_b = - EncryptedRepository::new(FilesystemStorage::new(&before_revoke), key_b.clone()); + let old_repo_b = EncryptedRepository::new( + FilesystemStorage::new(&before_revoke), + key_b.clone(), + temp.path().join("b-machine"), + ); old_repo_b .fetch_into(&snapshot_destination, "snapshot") .unwrap(); @@ -216,13 +260,16 @@ fn revoked_reader_cannot_reach_later_history_but_keeps_prior_snapshot() { key_c.public_device().unwrap(), "device-3", DeviceRoles::collaborator(), - &admin_pin, ) .unwrap(); assert_eq!(count_files(&remote.join("packs")), 3); let destination = temp.path().join("destination"); initialize_git(&destination); - let repo_c = EncryptedRepository::new(FilesystemStorage::new(&remote), key_c); + let repo_c = EncryptedRepository::new( + FilesystemStorage::new(&remote), + key_c, + temp.path().join("c-machine"), + ); let manifest = repo_c.fetch_into(&destination, "e2ee").unwrap(); repo_c.verify().unwrap(); assert_eq!(manifest.total_pack_count, 3); @@ -242,20 +289,34 @@ fn connected_membership_observation_pins_and_rejects_replay() { let remote = temp.path().join("remote"); let source = temp.path().join("source"); let client = temp.path().join("client"); - let admin_pin = temp.path().join("admin-state.json"); initialize_git(&source); initialize_git(&client); let key_a = KeyFile::generate(); let key_b = KeyFile::generate_for_repository(key_a.repository_root.clone()).unwrap(); - let repository = EncryptedRepository::new(FilesystemStorage::new(&remote), key_a); + let root = key_a.repository_root.clone(); + let repository = EncryptedRepository::new( + FilesystemStorage::new(&remote), + key_a.clone(), + temp.path().join("a-machine"), + ); + // The client is another machine, so only its own operations move its floor. + let client_machine = temp.path().join("client-machine"); + let client_view = + EncryptedRepository::new(FilesystemStorage::new(&remote), key_a, &client_machine); repository.initialize("owner").unwrap(); - repository.pin_admin_state(&admin_pin).unwrap(); commit(&source, "one\n", "one"); repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); - repository.fetch_into(&client, "e2ee").unwrap(); + client_view.fetch_into(&client, "e2ee").unwrap(); let before_membership = fs::read_to_string(remote.join("HEAD")).unwrap(); // The membership change keeps every ref, so the client already holds a @@ -266,19 +327,18 @@ fn connected_membership_observation_pins_and_rejects_replay() { key_b.public_device().unwrap(), "device-4", DeviceRoles::collaborator(), - &admin_pin, ) .unwrap(); - let observed = repository.observe_manifest(&client).unwrap(); + let observed = client_view.observe_manifest(&client).unwrap(); assert_eq!(observed.generation, 2); - let state_path = state_dir(&client).join("state.json"); + let state_path = client_machine.join(&root).join("continuity.json"); let state: serde_json::Value = serde_json::from_slice(&fs::read(&state_path).unwrap()).unwrap(); assert_eq!(state["generation"], 2); assert_eq!(state["policy_generation"], 1); let after_membership = fs::read_to_string(remote.join("HEAD")).unwrap(); fs::write(remote.join("HEAD"), &before_membership).unwrap(); - let error = repository.observe_manifest(&client).unwrap_err(); + let error = client_view.observe_manifest(&client).unwrap_err(); assert!(format!("{error:#}").contains("rolled back"), "{error:#}"); let state: serde_json::Value = serde_json::from_slice(&fs::read(&state_path).unwrap()).unwrap(); assert_eq!(state["generation"], 2); @@ -286,9 +346,16 @@ fn connected_membership_observation_pins_and_rejects_replay() { fs::write(remote.join("HEAD"), after_membership).unwrap(); commit(&source, "two\n", "two"); repository - .push_ref(&source, "refs/heads/main", false) + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) .unwrap(); - repository.fetch_into(&client, "e2ee").unwrap(); + client_view.fetch_into(&client, "e2ee").unwrap(); assert_eq!( git(&client, &["show", "refs/remotes/e2ee/main:note.md"]), "two" @@ -299,19 +366,20 @@ fn connected_membership_observation_pins_and_rejects_replay() { fn administrative_pin_rejects_policy_rollback() { let temp = tempfile::tempdir().unwrap(); let remote = temp.path().join("remote"); - let pin = temp.path().join("admin-state.json"); let owner = KeyFile::generate(); let first = KeyFile::generate_for_repository(owner.repository_root.clone()).unwrap(); let second = KeyFile::generate_for_repository(owner.repository_root.clone()).unwrap(); - let repository = EncryptedRepository::new(FilesystemStorage::new(&remote), owner); + let repository = EncryptedRepository::new( + FilesystemStorage::new(&remote), + owner, + temp.path().join("owner-machine"), + ); let genesis = repository.initialize("owner").unwrap(); - repository.pin_admin_state(&pin).unwrap(); repository .add_device( first.public_device().unwrap(), "device-5", DeviceRoles::collaborator(), - &pin, ) .unwrap(); @@ -321,8 +389,7 @@ fn administrative_pin_rejects_policy_rollback() { .add_device( second.public_device().unwrap(), "device-6", - DeviceRoles::collaborator(), - &pin, + DeviceRoles::collaborator() ) .unwrap_err() .to_string() @@ -337,27 +404,30 @@ fn non_admin_device_cannot_change_policy() { let key_a = KeyFile::generate(); let key_b = KeyFile::generate_for_repository(key_a.repository_root.clone()).unwrap(); let key_c = KeyFile::generate_for_repository(key_a.repository_root.clone()).unwrap(); - let admin_pin = temp.path().join("a-admin-state.json"); - let non_admin_pin = temp.path().join("b-admin-state.json"); - let repo_a = EncryptedRepository::new(FilesystemStorage::new(&remote), key_a); + let repo_a = EncryptedRepository::new( + FilesystemStorage::new(&remote), + key_a, + temp.path().join("a-machine"), + ); repo_a.initialize("owner").unwrap(); - repo_a.pin_admin_state(&admin_pin).unwrap(); repo_a .add_device( key_b.public_device().unwrap(), "device-7", DeviceRoles::collaborator(), - &admin_pin, ) .unwrap(); - let repo_b = EncryptedRepository::new(FilesystemStorage::new(&remote), key_b); + let repo_b = EncryptedRepository::new( + FilesystemStorage::new(&remote), + key_b, + temp.path().join("b-machine"), + ); assert!( repo_b .add_device( key_c.public_device().unwrap(), "device-8", - DeviceRoles::collaborator(), - &non_admin_pin, + DeviceRoles::collaborator() ) .unwrap_err() .to_string() @@ -373,28 +443,32 @@ fn read_only_device_cannot_add_or_revoke_devices() { let reader_key = KeyFile::generate_for_repository(owner_key.repository_root.clone()).unwrap(); let candidate_key = KeyFile::generate_for_repository(owner_key.repository_root.clone()).unwrap(); - let admin_pin = temp.path().join("owner-admin-state.json"); - let owner = EncryptedRepository::new(FilesystemStorage::new(&remote), owner_key); + let owner = EncryptedRepository::new( + FilesystemStorage::new(&remote), + owner_key, + temp.path().join("owner-machine"), + ); owner.initialize("owner").unwrap(); - owner.pin_admin_state(&admin_pin).unwrap(); owner .add_device( reader_key.public_device().unwrap(), "device-9", DeviceRoles::reader(), - &admin_pin, ) .unwrap(); - let reader = EncryptedRepository::new(FilesystemStorage::new(&remote), reader_key.clone()); + let reader = EncryptedRepository::new( + FilesystemStorage::new(&remote), + reader_key.clone(), + temp.path().join("reader-machine"), + ); let before = snapshot_files(&remote); assert!( reader .add_device( candidate_key.public_device().unwrap(), "device-10", - DeviceRoles::collaborator(), - &admin_pin, + DeviceRoles::collaborator() ) .unwrap_err() .to_string() @@ -402,7 +476,7 @@ fn read_only_device_cannot_add_or_revoke_devices() { ); assert!( reader - .revoke_device(&reader_key.device_id().unwrap(), &admin_pin) + .revoke_device(&reader_key.device_id().unwrap()) .unwrap_err() .to_string() .contains("not an active administrator") diff --git a/tests/read_only_helper.rs b/tests/read_only_helper.rs index 344a198..cf24efb 100644 --- a/tests/read_only_helper.rs +++ b/tests/read_only_helper.rs @@ -1,3 +1,5 @@ +mod support; + use std::env; use std::fs; use std::path::{Path, PathBuf}; @@ -36,7 +38,9 @@ fn add_helper_to_path(command: &mut Command) { let helper = Path::new(env!("CARGO_BIN_EXE_git-remote-e2ee")); let mut paths = vec![helper.parent().unwrap().to_path_buf()]; paths.extend(env::split_paths(&env::var_os("PATH").unwrap_or_default())); - command.env("PATH", env::join_paths(paths).unwrap()); + command + .env("PATH", env::join_paths(paths).unwrap()) + .env("XDG_STATE_HOME", support::state_home()); } fn initialize_git(repo: &Path) { @@ -106,16 +110,13 @@ fn exercise_read_only_helper( owner_key.write_new(&owner_key_path).unwrap(); reader_key.write_new(&reader_key_path).unwrap(); - let owner = EncryptedRepository::new(storage, owner_key); + let owner = EncryptedRepository::new(storage, owner_key, support::state_dir()); owner.initialize("owner").unwrap(); - let admin_pin = temporary.join("owner.admin-state.json"); - owner.pin_admin_state(&admin_pin).unwrap(); owner .add_device( reader_key.public_device().unwrap(), "device-1", DeviceRoles::reader(), - &admin_pin, ) .unwrap(); @@ -262,16 +263,13 @@ fn read_only_device_cannot_publish_recovery() { let owner = KeyFile::generate(); let reader = KeyFile::generate_for_repository(owner.repository_root.clone()).unwrap(); let storage = FilesystemStorage::new(&storage_path); - let owner_repository = EncryptedRepository::new(storage.clone(), owner); + let owner_repository = EncryptedRepository::new(storage.clone(), owner, support::state_dir()); owner_repository.initialize("owner").unwrap(); - let admin_pin = temporary.path().join("owner.admin-state.json"); - owner_repository.pin_admin_state(&admin_pin).unwrap(); owner_repository .add_device( reader.public_device().unwrap(), "device-2", DeviceRoles::reader(), - &admin_pin, ) .unwrap(); owner_repository.fetch_into(&reader_repo, "origin").unwrap(); @@ -285,7 +283,7 @@ fn read_only_device_cannot_publish_recovery() { .compare_and_swap_head(Some(&floor_id), &invalid_id) .unwrap(); - let read_only = EncryptedRepository::new(storage.clone(), reader); + let read_only = EncryptedRepository::new(storage.clone(), reader, support::state_dir()); let report = read_only .recover( &reader_repo, diff --git a/tests/remote_helper.rs b/tests/remote_helper.rs index cf287d2..01441f1 100644 --- a/tests/remote_helper.rs +++ b/tests/remote_helper.rs @@ -1,3 +1,5 @@ +mod support; + use std::env; use std::fs; use std::io::Write; @@ -6,7 +8,7 @@ use std::process::{Command, Output, Stdio}; use git_remote_e2ee::crypto::KeyFile; use git_remote_e2ee::policy::DeviceRoles; -use git_remote_e2ee::repository::EncryptedRepository; +use git_remote_e2ee::repository::{EncryptedRepository, RefChange}; use git_remote_e2ee::storage::FilesystemStorage; fn git_output(repo: &Path, args: &[&str], with_helper: bool) -> Output { @@ -16,7 +18,9 @@ fn git_output(repo: &Path, args: &[&str], with_helper: bool) -> Output { let helper = Path::new(env!("CARGO_BIN_EXE_git-remote-e2ee")); let mut paths = vec![helper.parent().unwrap().to_path_buf()]; paths.extend(env::split_paths(&env::var_os("PATH").unwrap_or_default())); - command.env("PATH", env::join_paths(paths).unwrap()); + command + .env("PATH", env::join_paths(paths).unwrap()) + .env("XDG_STATE_HOME", support::state_home()); } command.output().unwrap() } @@ -89,9 +93,13 @@ fn native_git_push_and_fetch_use_the_remote_helper() { let key = KeyFile::generate(); key.write_new(&key_path).unwrap(); - EncryptedRepository::new(FilesystemStorage::new(&remote_path), key) - .initialize("owner") - .unwrap(); + EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ) + .initialize("owner") + .unwrap(); fs::write(source.join("note.md"), "through native Git\n").unwrap(); git(&source, &["add", "note.md"], false); @@ -154,7 +162,11 @@ fn concurrent_helper_processes_preserve_the_client_state_floor() { let key = KeyFile::generate(); key.write_new(&key_path).unwrap(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); configure_remote(&source, &remote_path, &key_path); configure_remote(&destination, &remote_path, &key_path); @@ -184,6 +196,7 @@ fn concurrent_helper_processes_preserve_the_client_state_floor() { .arg("private") .arg(remote_url) .current_dir(&destination) + .env("XDG_STATE_HOME", support::state_home()) .stdin(Stdio::piped()) .stdout(Stdio::piped()) .stderr(Stdio::piped()) @@ -225,11 +238,16 @@ fn concurrent_helper_processes_preserve_the_client_state_floor() { } assert_eq!(current_successes, 2); - let state_path = state_dir(&destination).join("state.json"); - let state: serde_json::Value = serde_json::from_slice(&fs::read(state_path).unwrap()).unwrap(); - assert_eq!(state["generation"], expected_manifest.generation); - assert_eq!(state["imported_generation"], expected_manifest.generation); - assert_eq!(state["head_id"], expected_head); + let progress_path = state_dir(&destination).join("progress.json"); + let progress: serde_json::Value = + serde_json::from_slice(&fs::read(progress_path).unwrap()).unwrap(); + assert_eq!( + progress["imported_generation"], + expected_manifest.generation + ); + let floor = support::read_floor(&expected_manifest.repository_root); + assert_eq!(floor["generation"], expected_manifest.generation); + assert_eq!(floor["head_id"], expected_head); } #[test] @@ -242,7 +260,11 @@ fn native_git_dry_run_does_not_mutate_remote() { let key = KeyFile::generate(); key.write_new(&key_path).unwrap(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); fs::write(source.join("note.md"), "dry run only\n").unwrap(); @@ -284,7 +306,11 @@ fn native_git_fetch_rejects_rollback_even_when_objects_are_local() { let key = KeyFile::generate(); key.write_new(&key_path).unwrap(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); configure_remote(&source, &remote_path, &key_path); configure_remote(&destination, &remote_path, &key_path); @@ -339,7 +365,11 @@ fn native_git_push_rejects_storage_rollback_before_creating_a_fork() { let key = KeyFile::generate(); key.write_new(&key_path).unwrap(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); configure_remote(&source, &remote_path, &key_path); @@ -365,7 +395,10 @@ fn native_git_push_rejects_storage_rollback_before_creating_a_fork() { "stderr={}", String::from_utf8_lossy(&output.stderr) ); - assert_eq!(encrypted.current_manifest().unwrap().0, first_remote_head); + assert_eq!( + fs::read_to_string(remote_path.join("HEAD")).unwrap().trim(), + first_remote_head + ); } #[test] @@ -381,7 +414,11 @@ fn native_git_fetch_rejects_same_generation_manifest_fork() { let key = KeyFile::generate(); key.write_new(&key_path).unwrap(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key.clone()); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key.clone(), + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); configure_remote(&source, &remote_path, &key_path); configure_remote(&destination, &remote_path, &key_path); @@ -392,9 +429,22 @@ fn native_git_fetch_rejects_same_generation_manifest_fork() { git(&source, &["push", "private", "main"], true); git(&destination, &["fetch", "private"], true); - let fork = EncryptedRepository::new(FilesystemStorage::new(&fork_path), key); + let fork = EncryptedRepository::new( + FilesystemStorage::new(&fork_path), + key, + temporary.path().join("forker-machine"), + ); fork.initialize("owner").unwrap(); - let fork_head = fork.push_ref(&source, "refs/heads/main", false).unwrap(); + let fork_head = fork + .push( + &source, + "refs/heads/main", + RefChange::Update { + source: "refs/heads/main", + force: false, + }, + ) + .unwrap(); copy_tree(&fork_path.join("manifests"), &remote_path.join("manifests")); fs::write(remote_path.join("HEAD"), format!("{fork_head}\n")).unwrap(); @@ -414,18 +464,18 @@ fn native_git_fetch_pins_membership_changes_that_transfer_no_objects() { let destination = temporary.path().join("destination"); let remote_path = temporary.path().join("remote"); let key_path = temporary.path().join("repository.key.json"); - let admin_pin = temporary - .path() - .join("repository.key.json.admin-state.json"); initialize_git(&source); initialize_git(&destination); let key = KeyFile::generate(); key.write_new(&key_path).unwrap(); let extra = KeyFile::generate_for_repository(key.repository_root.clone()).unwrap(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); - encrypted.pin_admin_state(&admin_pin).unwrap(); configure_remote(&source, &remote_path, &key_path); configure_remote(&destination, &remote_path, &key_path); @@ -443,7 +493,6 @@ fn native_git_fetch_pins_membership_changes_that_transfer_no_objects() { extra.public_device().unwrap(), "device-1", DeviceRoles::reader(), - &admin_pin, ) .unwrap(); git(&destination, &["fetch", "private"], true); @@ -477,7 +526,11 @@ fn native_git_fetch_pins_branch_deletions_that_transfer_no_objects() { let key = KeyFile::generate(); key.write_new(&key_path).unwrap(); - let encrypted = EncryptedRepository::new(FilesystemStorage::new(&remote_path), key); + let encrypted = EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ); encrypted.initialize("owner").unwrap(); configure_remote(&source, &remote_path, &key_path); configure_remote(&destination, &remote_path, &key_path); @@ -517,9 +570,13 @@ fn native_git_fetch_reimports_packs_after_local_objects_are_pruned() { initialize_git(&destination); let key = KeyFile::generate(); key.write_new(&key_path).unwrap(); - EncryptedRepository::new(FilesystemStorage::new(&remote_path), key) - .initialize("owner") - .unwrap(); + EncryptedRepository::new( + FilesystemStorage::new(&remote_path), + key, + support::state_dir(), + ) + .initialize("owner") + .unwrap(); configure_remote(&source, &remote_path, &key_path); configure_remote(&destination, &remote_path, &key_path); diff --git a/tests/support/mod.rs b/tests/support/mod.rs new file mode 100644 index 0000000..901566e --- /dev/null +++ b/tests/support/mod.rs @@ -0,0 +1,34 @@ +//! Local state shared by one test binary. Continuity floors are keyed by +//! repository root and every test generates its own root, so tests in one +//! binary act like one machine without sharing floors. + +#![allow(dead_code)] + +use std::path::{Path, PathBuf}; +use std::sync::OnceLock; + +/// The `XDG_STATE_HOME` that Git commands running the helper must see. +pub fn state_home() -> &'static Path { + static HOME: OnceLock = OnceLock::new(); + HOME.get_or_init(|| { + tempfile::Builder::new() + .prefix("state-home") + .tempdir_in(env!("CARGO_TARGET_TMPDIR")) + .unwrap() + .keep() + }) +} + +/// The state directory the library takes: `$XDG_STATE_HOME/git-e2ee`. +pub fn state_dir() -> PathBuf { + state_home().join("git-e2ee") +} + +/// This machine's continuity floor file for `repository_root`. +pub fn floor_path(repository_root: &str) -> PathBuf { + state_dir().join(repository_root).join("continuity.json") +} + +pub fn read_floor(repository_root: &str) -> serde_json::Value { + serde_json::from_slice(&std::fs::read(floor_path(repository_root)).unwrap()).unwrap() +}