Support nested task dataset layouts when pushing to Hub - #97
Merged
adithya-s-k merged 1 commit intoSep 15, 2026
Merged
adithya-s-k merged 1 commit into
adithya-s-k merged 1 commit into
Conversation
KNambiarDJsc
left a comment
Contributor
There was a problem hiding this comment.
I reviewed and tested 92b5138:
- CI-equivalent run on Linux: ruff is clean, and the full suite passes on py3.12 and py3.14 (719 passed).
- End-to-end on a real nested dataset. I copied three tasks from the published
AdithyaSK/repo2rlenv-pr-diffdataset in itstasks/<id>/layout. I also added a rootREADME.mdandregistry.json, ashuggingface-cli downloadleaves them. Then I ranpush_to_hubwithHfApimocked.- On
mainit raisesRuntimeError: no Harbor tasks found. - With this PR, all 3 tasks are staged under
tasks/, and the generatedmanifest.jsonhas the same 3 rows. - Nothing ends up double-nested as
tasks/tasks/. - The push runs through image preparation without errors; these tasks ship
environment/Dockerfiles.
- On
LGTM.
One non-blocking nit: hub.py now imports the private _list_task_dirs from registry.integration. If both modules are meant to share discovery, dropping the underscore would make that dependency explicit.
adithya-s-k
approved these changes
Sep 15, 2026
Collaborator
|
thanks @karatarassul4-max, merged! appreciate the nested task layout support and the duplicate ID checks. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #96
Summary
_list_task_dirs()inpush_to_hub()so Hub staging and image preparation share the same discovery logictasks/<id>/task.tomllayoutpush_to_hub()metadata discoveryValidation
uv run pytest -q tests/registry/test_integration.py tests/test_hub_metadata_read.py→ 34 passeduv run ruff check src/repo2rlenv/registry/integration.py src/repo2rlenv/hub.py tests/registry/test_integration.py tests/test_hub_metadata_read.py→ passeduv run ruff format --check src/repo2rlenv/registry/integration.py src/repo2rlenv/hub.py tests/registry/test_integration.py tests/test_hub_metadata_read.py→ 4 files already formattedgit diff --check→ cleanuv run pytest -qon Windows: 712 passed, 6 skipped, 7 platform-specific failures related to executable bits, path separators, and Windows local-path parsing; none are in files changed by this PR