Repository navigation
infra: add GitHub Actions extraction workflow and CI - #22
Merged
Merged
Conversation
extract.yml — permanent workflow dispatched by the Cloudflare Worker
for every extraction job. Accepts run_id, source_repo, source_path,
target_repo, target_branch, and target_path as inputs. Secrets are
named GE_SRC_{run_id} and GE_TGT_{run_id} so concurrent runs never
collide. Supports any HTTPS-accessible repo (GitHub or GitLab).
ci.yml — runs on every PR and push to main: install, lint, build for
both workspaces. Steps use continue-on-error until workspaces are
fully scaffolded.
Closes #2
BFG mirror-clones the source repo and deletes all top-level dirs not on the source path prefix — fast blob-level operation that strips the bulk of irrelevant history before any path rewriting happens. git-filter-repo then runs on the already-lean working clone to do the subdirectory extraction and rename, which is much faster now that the repo contains only the relevant subtree. BFG requires a mirror clone and a gc pass after deletion. git-filter-repo handles the precise --subdirectory-filter + --to-subdirectory-filter rename that BFG cannot do.
Collaborator
Author
BFG strips irrelevant top-level dirs from the mirror clone (fast, blob-level). The path rename and deeper-level filtering are handled entirely with git fast-export piped through awk into git fast-import — native git, no extra tools. awk rewrites M/D lines whose path matches source_path (or is prefixed by source_path/) and drops all others, eliminating sibling dirs at deeper levels that BFG cannot safely remove by name.
fast-export, fast-import and git push all work on bare repos. No need to clone /tmp/source.git to /tmp/source.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2
What
Adds the two GitHub Actions workflows that live permanently in the repo.
extract.yml— the extraction runnerDispatched by the Cloudflare Worker via
workflow_dispatch. Never modified at runtime — only secrets change per run.Inputs:
run_id,source_repo,source_path,target_repo,target_branch,target_pathSteps:
pip install git-filter-repohttps://oauth2:${GE_SRC_<run_id>}@<host>— works for GitHub and GitLabgit filter-repo --subdirectory-filter <source_path> --to-subdirectory-filter <target_path> --forcehttps://oauth2:${GE_TGT_<run_id>}@<host>Secret names are unique per
run_idso concurrent extractions never collide.ci.yml— PR build checkRuns on every PR and push to
main. Installs from root, then lints and builds each workspace. Steps usecontinue-on-error: trueuntil workspaces are fully scaffolded (will be tightened in #19).Test plan
extract.ymlappears in Actions tab after merge