CFTL-589 GitHub OAuth as a third git authentication method - #32
CFTL-589 GitHub OAuth as a third git authentication method#32Oscar-XXII wants to merge 16 commits into
Conversation
Cloning a private repository so far required the customer to create and rotate a credential by hand, and a classic PAT grants read and write across every private repository the user can reach. A GitHub App user access token is limited to Contents: Read-only on the repositories selected when the app is installed, so nothing has to be created or rotated manually. The token arrives outside "parameters", in the authorization section of the configuration, and is handed to git through a GIT_ASKPASS helper. That keeps it out of the command line, out of the cloned repository's .git/config and out of the environment of the executed user script. The authorization section is also stripped from the config.json written for that script. Besides the user's own token it carries the shared application secret of the Keboola GitHub App, which no user code may be able to read. The existing none/pat/ssh branches are untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Repository selection happens when the GitHub App is installed, not when the component is authorized, and the two are independent flows on GitHub's side. Authorizing without installing first yields a valid token that can see no repositories, and the job then fails with a bare "repository not found". Spelling the order out makes that failure avoidable rather than merely explainable after the fact. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Typing the repository URL by hand works, but it defers every mistake to the
first job run: a repository the app cannot see fails as "not found" only once
the clone is attempted. Listing what the installation actually exposes moves
that feedback into the configuration form, and an empty list is precisely the
signal that the app was authorized but never installed.
The list comes from /user/installations followed by
/user/installations/{id}/repositories, because a user access token carries no
installation id of its own. Both endpoints are paginated and one user may see
several installations, so the results are aggregated.
The picker writes a clone URL into git.repository. The free-text git.url stays
in place for the other authentication methods, because a schema field cannot be
a dropdown and a text input at the same time; GitConfiguration.repository_url
resolves which of the two applies.
Calls go through urllib to avoid adding an HTTP dependency for two endpoints.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…nnot fill in The free-text url field is hidden for auth: oauth, where the repository is picked from a dropdown into git.repository instead, but it stayed listed in git.required. That leaves an OAuth configuration demanding a field the form gives no way to fill in. Only auth stays required. An empty repository is still caught at runtime, now with wording that fits a dropdown rather than asking for a URL. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tion The installation dialog defaults to a choice, not to a safe default, and the onboarding text so far only said that repositories are picked during installation. Picking "All repositories" grants Contents: Read-only across the whole account or organisation with a token that does not expire — the same over-scoped long-lived credential that motivated moving away from personal access tokens. Nothing on the Keboola side can narrow it afterwards, so the guidance has to arrive before the user clicks. Also records the app's client ID and the page where a user can review or revoke the access they granted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The app slug is keboola-custom-python-read; it is derived from the app name and is not the client ID, so it could not be inferred from what the OAuth registration already recorded. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Job 53340777 cloned its repository over OAuth and then failed in `uv sync` with "could not read Username for https://github.com": uv shells out to git to fetch a private dependency, and that git had no credentials at all. The PAT path is unaffected because it leaves a ~/.netrc behind, which uv picks up on its own — but a file in the home directory is also readable by the executed user script, which is why the OAuth path does not write one. The credentials now travel to the dependency installation explicitly, through a new optional env argument on SubprocessRunner.run. The executed script keeps inheriting the untouched process environment, so the token still does not reach it. GitHandler now holds only the git-specific overrides and resolves the full environment when a subprocess is started, rather than snapshotting os.environ in the constructor. The snapshot predates the virtual environment selection, so handing it to `uv sync` would have pointed the install at the wrong environment. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The app is now "Keboola Custom Python", which moves its slug from keboola-custom-python-read to keboola-custom-python. The old slug returns 404 — GitHub does not redirect a renamed app — so the previous link would simply have been dead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Review: GitHub OAuth as a third git auth method. No blockers — the token handling (askpass helper + stripping Backward-compat: the change is additive — a new
Happy to discuss any of these. |
A read timeout surfaces as a bare TimeoutError. It is an OSError but not a URLError, so it escaped both handlers and ended the job as an internal error — and listRepositories already runs close to the sync action limit against a large organisation, so this is a likely path rather than a theoretical one. Broadening the existing handler alone would not have been enough: TimeoutError carries no "reason" attribute, so the message formatting has to change with it. URLError keeps its own branch to preserve the more readable reason it does carry. Also from review: the OAuth repository field is labelled "Repository URL" like its free-text counterpart, since both hold the same clone URL, and the setup section no longer promises two steps before listing three. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Went through all six. Current disposition — some of these moved after later changes, so this is the up-to-date view rather than what I first replied:
Thanks for #4 and #6 in particular — both went further than the finding as written. |
Bold move, @claude! 🤖
Is it possible to add a check for the URL field not being empty when the correct auth value is selected? I am not sure that is even possible in the RJSF, but can you check the docs please?
Nope, it's confusing to have the same value (repo URL) in two differently named variables, Oh, I get it now, once loaded via the sync action, the field is populated with the repo names! Too bad the URL is then saved anyway, so when users pick the repo, save the configuration and then open it again, they will see the URL instead of the pretty name, which can only be triggered by clicking the sync action button again. As for me, I'd still use
No, please don't skip the reordering. Currently, when users pick the repo's visibility & auth, the whole forms jumps up and down based on the option selected, that is a terrible and unfriendly UX. Yes, the current OAuth order is the right sequence, so please change the original field to that order too. Existing users will deal with that, no biggie, they probably won't be able to tell to order themselves anyway.
Yes, the repo vocabulary all drifted towards AI slop, but no problem, we will deal with that later. :-P |
Review raised that dropping url from git.required lost the pre-save check. It can be restored conditionally: allOf + if/then keyed on auth, which is the pattern the team's own advanced schema reference documents, and which the UI's sanitizer already anticipates — if/then/else branches are its one exemption from the automatic additionalProperties: false. This is not the dependencies + oneOf pattern the UI guidance forbids; visibility stays on options.dependencies and if/then carries validation only. required alone would not have been enough. The form writes an empty string for a field that was touched and cleared, and required only checks presence, so minLength: 1 goes in the same branch. Verified against the eight relevant shapes with the empty-enum placeholders stripped the way the UI strips them. Closes a gap wider than the one reported: an OAuth configuration could also be saved with no repository selected at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Superseded. Claimed conditional |
…l stable Two review points that turn out to answer each other. The field is renamed repository -> url_oauth. It holds the same thing as url, a clone URL, and differs only by the authentication that produced it, so it reads better as a variant of the shipped key than as a second name for one concept. The dropdown now uses the clone URL as both value and label. Options are only fetched when the user runs the sync action, so a saved configuration reopened without running it again had no label for the stored value and fell back to showing the raw URL — the pretty owner/repo name appeared only after clicking the button a second time. Using one string for both makes what is displayed the same before and after a reload, and keeps url and url_oauth looking alike in the form, which is the consistency the rename is after. The cost is a dropdown of URLs rather than short owner/repo names. The prettier label only becomes viable once the list can autoload, which is blocked on the truncated-response problem. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Superseded. Described the |
The form had two fields holding the same thing: a free-text url above the authentication switch for none/pat/ssh, and a url_oauth dropdown below it for oauth. That split existed only because I concluded a field cannot be both a text input and a dropdown. options.creatable makes it one field: the list is loaded on demand for OAuth, and any authentication method can type or paste a clone URL. The shipped url key is the one that survives, so nothing migrates. Three things fall away with the split: the naming problem of one value under two keys, the allOf + if/then conditional required (a single always-visible field is simply required, with minLength: 1 for the empty-string case), and the two-key autoload on the branch list. The empty-repository message no longer branches on the authentication method either, because there is one field to talk about. Caveat to validate before merge: options.creatable combined with options.async is not documented. If the two do not compose, the field would stop accepting typed input and every non-OAuth configuration would be unable to set a repository. The schema needs a pass in the Ctrl+D sandbox — type a URL, save, reopen — before this ships. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…thods Now that one field serves every authentication method, its List Repositories button is offered to all of them — options.async belongs to the field, and options.dependencies hides the whole field rather than the button, so there is no way to show the button for OAuth alone without reinstating the two-field split. The button being visible is cosmetic; the answer it gave was not. A personal access token configuration was told "GitHub authorization is missing, please authorize the component", sending the user to fix an authorization they neither have nor need. The action now checks git.auth first and says the listing belongs to GitHub authorization, and that other methods enter the URL directly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three mismatches were only discovered when a job ran: a non-GitHub URL under GitHub authorization, an SSH URL under a personal access token, and an HTTPS URL under an SSH key — the last one silently sets up no credentials at all. Each is knowable the moment the URL is typed. The patterns mirror what source_git.py actually accepts, deliberately no stricter: https://github.com or www.github.com for oauth, any https for pat (self-hosted GitHub Enterprise included), git@ or ssh:// for ssh, and anything for a public repository. Verified against thirteen combinations. This does not help the List Repositories button, which stays visible for every method — the schema has no notion of a button being applicable. That remains the sync action's job. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Confirmed in the Ctrl+D sandbox — with the allOf block present, the GitHub OAuth option disappears from the authentication radio entirely. This is the known RJSF allOf merge failure (rjsf-team/react-jsonschema-form#2752, #3445), which reports exactly this symptom: "could not merge subschemas in allOf: Could not resolve values for path: properties.value.enum". The if clauses mention properties.auth, so the merge narrows the enum it was only meant to test against. The schema itself was always valid — the eight and thirteen case runs that justified this passed. They were run with the jsonschema library, which validates but does not render, and the renderer is where this breaks. Validating semantics is not evidence about the form. Conditional validation is therefore off the table in this UI, whatever the standard permits: url stays plainly required with minLength: 1, and the auth specific URL rules stay in source_git.py where they already have clear messages. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Folded into the summary at the end of the thread. The durable finding: |
Shortens the message the List Repositories button returns for the other authentication methods to the one we settled on. It stays an exception rather than a returned option: a sync action can only answer with value/label pairs, so a returned notice would be selectable and would end up saved as the repository URL. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Summary of where this landed, since the thread above wandered. The two-field design and the conditional-validation attempt are both gone. There is one One durable finding worth keeping: Blocking before merge: |
Why
Cloning a private repository required the customer to create and rotate a credential by hand, and a classic PAT grants read and write across every private repository the user can reach. Enterprise security reviews keep flagging it.
This adds a third
git.authoption: authorize a Keboola-owned GitHub App through the OAuth broker and get a user access token limited toContents: Read-onlyon the repositories chosen at installation. SSH and PAT are untouched, and no key changes for existing configurations.What it does
oauthappended togit.auth. Existing radio order is unchanged.git.urlnow serves every authentication method — a select withcreatable, moved below the auth switch. OAuth users load the list with List Repositories; everyone else types or pastes the URL. For non-OAuth the action answersSupported only for OAuth. Please insert the URL manually.parameters, in theauthorizationsection, and reaches git through aGIT_ASKPASShelper — never in the command line, in.git/config, or in the executed script's environment.authorizationis stripped from theconfig.jsonhanded to the user script. It carries the user's token and#appSecret, the shared secret of the Keboola GitHub App, and this component runs arbitrary user code. The security-critical change in this PR; covered by a test.listRepositoriesreads/user/installationsthen/user/installations/{id}/repositories— a user token carries no installation id of its own. Paginated, aggregated across installations.uv syncshells out to git with no credentials of its own;SubprocessRunner.runtakes an optionalenvso the dependency install gets them. The executed script still inherits the untouched environment.UserException.38 tests, flake8 clean.
Verified
Job 53340777 cloned over OAuth against real GitHub — broker token, askpass helper, clone. It then failed in
uv sync, which is the bug fixed here.Not verified
options.creatablecombined withoptions.asyncis undocumented. If they do not compose, the field stops accepting typed input and no non-OAuth configuration can set a repository. Needs a Ctrl+D pass before merge.listRepositoriesreturns a truncated response against an org with ~1000 repositories; response-size cap vs sync-action timeout not yet distinguished.Reviewer note
source_git.pyreads larger than it is:self.git_cfg.urlcollapsed into one resolvedself.repo_url, and the environment became a call-time overlay rather than a constructor snapshot.🤖 Generated with Claude Code