Skip to content

fix: key the import file cache on the source repo revision - #11112

Draft
adarshsm wants to merge 1 commit into
treeverse:mainfrom
adarshsm:fix/11111-import-fs-cache-rev
Draft

adarshsm wants to merge 1 commit into
treeverse:mainfrom
adarshsm:fix/11111-import-fs-cache-rev

Conversation

@adarshsm

@adarshsm adarshsm commented Oct 1, 2026

Copy link
Copy Markdown

Fixes #11111

Bug

For a --no-download repo import, dvc pull stores the fetched files under .dvc/cache/fs/<protocol>/<token>. The token is built in _load_storage_from_import (dvc/repo/index.py) as tokenize(dep.fs_path, meta_token). A RepoDependency records no meta, so meta_token is the same constant for every revision and every source repo. That causes two problems:

  1. Stale data after update (the reported case). import --no-download --rev v1 → pull → update --no-download --rev main → pull checks out the v1 data. .dvc/cache/fs already has an entry under the same token, so the new rev_lock is never fetched.
  2. Data from the wrong repo. dvc import --no-download ../srcA data -o a plus dvc import --no-download ../srcB data -o b, then pulling both, gives b/ the contents of srcA. Both imports land on the same token.

I reproduced both with the CLI on current main. I only saw this with directory imports. A single-file import with the same steps already picked up the new content.

Fix

For repo imports, build the token from the source repo url and rev_lock instead of the empty dep meta. Every source revision now gets its own cache entry. Other imports (import-url, version-aware) are unchanged.

Entries written under the old token are just never looked up again. Nothing reads them, but they aren't cleaned up either.

Tests

Two new tests in tests/func/test_import.py:

  • test_pull_import_no_download_after_update: covers the scenario from the issue.
  • test_pull_import_no_download_same_path_from_two_repos: covers the cross-repo collision.

Both fail on main and pass with this change. tests/func/test_import.py, test_update.py, test_repo_index.py, test_data_cloud.py, test_import_url.py, test_external_repo.py, test_checkout.py, test_status.py and tests/func/api pass locally: 278 passed. ruff check and ruff format --check are clean.

  • ❗ I have followed the Contributing to DVC checklist.

  • 📖 If this PR requires documentation updates, I have created a separate PR (or issue, at least) in dvc.org and linked it here. (No docs change needed.)

🤖 Generated with Claude Code

Partial (--no-download) repo imports keep their pulled files under
.dvc/cache/fs/<protocol>/<token>, where the token came from the dep path
and the dep meta. Repo dependencies record no meta, so the token was the
same for every revision and every source repo with that path:

- after `update --no-download` moved rev_lock, `pull` checked out the
  previous revision's copy
- importing the same path from two repos gave both outputs the first
  repo's data

Use the repo url and rev_lock as the token instead.

Fixes treeverse#11111

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-project-automation github-project-automation Bot moved this to Backlog in DVC Oct 1, 2026
@codecov

codecov Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.98%. Comparing base (2431ec6) to head (2071ebb).
⚠️ Report is 213 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #11112      +/-   ##
==========================================
+ Coverage   90.68%   90.98%   +0.30%     
==========================================
  Files         504      505       +1     
  Lines       39795    41163    +1368     
  Branches     3141     3264     +123     
==========================================
+ Hits        36087    37452    +1365     
- Misses       3042     3072      +30     
+ Partials      666      639      -27     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pull checks out the previous version after update --no-download moves an import's rev_lock (.dvc/cache/fs is not keyed by revision)

1 participant