Skip to content

CFTL-589 GitHub OAuth as a third git authentication method - #32

Open
Oscar-XXII wants to merge 16 commits into
mainfrom
feature/CFTL-589
Open

CFTL-589 GitHub OAuth as a third git authentication method#32
Oscar-XXII wants to merge 16 commits into
mainfrom
feature/CFTL-589

Conversation

@Oscar-XXII

@Oscar-XXII Oscar-XXII commented Aug 21, 2026

Copy link
Copy Markdown

Why

Cloning a private repository required the customer to create and rotate a credential by hand, and a classic PAT grants read and write across every private repository the user can reach. Enterprise security reviews keep flagging it.

This adds a third git.auth option: authorize a Keboola-owned GitHub App through the OAuth broker and get a user access token limited to Contents: Read-only on the repositories chosen at installation. SSH and PAT are untouched, and no key changes for existing configurations.

What it does

  • oauth appended to git.auth. Existing radio order is unchanged.
  • One repository field. git.url now serves every authentication method — a select with creatable, moved below the auth switch. OAuth users load the list with List Repositories; everyone else types or pastes the URL. For non-OAuth the action answers Supported only for OAuth. Please insert the URL manually.
  • Token handling. The token arrives outside parameters, in the authorization section, and reaches git through a GIT_ASKPASS helper — never in the command line, in .git/config, or in the executed script's environment.
  • authorization is stripped from the config.json handed to the user script. It carries the user's token and #appSecret, the shared secret of the Keboola GitHub App, and this component runs arbitrary user code. The security-critical change in this PR; covered by a test.
  • listRepositories reads /user/installations then /user/installations/{id}/repositories — a user token carries no installation id of its own. Paginated, aggregated across installations.
  • Private git dependencies. uv sync shells out to git with no credentials of its own; SubprocessRunner.run takes an optional env so the dependency install gets them. The executed script still inherits the untouched environment.
  • Errors for missing authorization, non-GitHub URL, revoked token, a repository the app cannot see, and a read timeout each produce an actionable UserException.

38 tests, flake8 clean.

Verified

Job 53340777 cloned over OAuth against real GitHub — broker token, askpass helper, clone. It then failed in uv sync, which is the bug fixed here.

Not verified

  • options.creatable combined with options.async is undocumented. If they do not compose, the field stops accepting typed input and no non-OAuth configuration can set a repository. Needs a Ctrl+D pass before merge.
  • The dependency fix has not run on the platform yet.
  • listRepositories returns a truncated response against an org with ~1000 repositories; response-size cap vs sync-action timeout not yet distinguished.
  • The OAuth authorization on the test project stopped working after the app was renamed and published. Most likely the app's token-expiry setting or a revoked authorization, not this code — the token is delivered correctly and rejected by GitHub.

Reviewer note

source_git.py reads larger than it is: self.git_cfg.url collapsed into one resolved self.repo_url, and the environment became a call-time overlay rather than a constructor snapshot.

🤖 Generated with Claude Code

Oscar-XXII and others added 8 commits August 21, 2026 11:38
Cloning a private repository so far required the customer to create and rotate a
credential by hand, and a classic PAT grants read and write across every private
repository the user can reach. A GitHub App user access token is limited to
Contents: Read-only on the repositories selected when the app is installed, so
nothing has to be created or rotated manually.

The token arrives outside "parameters", in the authorization section of the
configuration, and is handed to git through a GIT_ASKPASS helper. That keeps it
out of the command line, out of the cloned repository's .git/config and out of
the environment of the executed user script.

The authorization section is also stripped from the config.json written for that
script. Besides the user's own token it carries the shared application secret of
the Keboola GitHub App, which no user code may be able to read.

The existing none/pat/ssh branches are untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Repository selection happens when the GitHub App is installed, not when the
component is authorized, and the two are independent flows on GitHub's side.
Authorizing without installing first yields a valid token that can see no
repositories, and the job then fails with a bare "repository not found".
Spelling the order out makes that failure avoidable rather than merely
explainable after the fact.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Typing the repository URL by hand works, but it defers every mistake to the
first job run: a repository the app cannot see fails as "not found" only once
the clone is attempted. Listing what the installation actually exposes moves
that feedback into the configuration form, and an empty list is precisely the
signal that the app was authorized but never installed.

The list comes from /user/installations followed by
/user/installations/{id}/repositories, because a user access token carries no
installation id of its own. Both endpoints are paginated and one user may see
several installations, so the results are aggregated.

The picker writes a clone URL into git.repository. The free-text git.url stays
in place for the other authentication methods, because a schema field cannot be
a dropdown and a text input at the same time; GitConfiguration.repository_url
resolves which of the two applies.

Calls go through urllib to avoid adding an HTTP dependency for two endpoints.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…nnot fill in

The free-text url field is hidden for auth: oauth, where the repository is
picked from a dropdown into git.repository instead, but it stayed listed in
git.required. That leaves an OAuth configuration demanding a field the form
gives no way to fill in.

Only auth stays required. An empty repository is still caught at runtime, now
with wording that fits a dropdown rather than asking for a URL.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tion

The installation dialog defaults to a choice, not to a safe default, and the
onboarding text so far only said that repositories are picked during
installation. Picking "All repositories" grants Contents: Read-only across the
whole account or organisation with a token that does not expire — the same
over-scoped long-lived credential that motivated moving away from personal
access tokens. Nothing on the Keboola side can narrow it afterwards, so the
guidance has to arrive before the user clicks.

Also records the app's client ID and the page where a user can review or revoke
the access they granted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The app slug is keboola-custom-python-read; it is derived from the app name and
is not the client ID, so it could not be inferred from what the OAuth
registration already recorded.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Job 53340777 cloned its repository over OAuth and then failed in `uv sync` with
"could not read Username for https://github.com": uv shells out to git to fetch
a private dependency, and that git had no credentials at all. The PAT path is
unaffected because it leaves a ~/.netrc behind, which uv picks up on its own —
but a file in the home directory is also readable by the executed user script,
which is why the OAuth path does not write one.

The credentials now travel to the dependency installation explicitly, through a
new optional env argument on SubprocessRunner.run. The executed script keeps
inheriting the untouched process environment, so the token still does not reach
it.

GitHandler now holds only the git-specific overrides and resolves the full
environment when a subprocess is started, rather than snapshotting os.environ in
the constructor. The snapshot predates the virtual environment selection, so
handing it to `uv sync` would have pointed the install at the wrong environment.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The app is now "Keboola Custom Python", which moves its slug from
keboola-custom-python-read to keboola-custom-python. The old slug returns 404 —
GitHub does not redirect a renamed app — so the previous link would simply have
been dead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@linear-code

linear-code Bot commented Aug 21, 2026

Copy link
Copy Markdown

CFTL-589

@soustruh

soustruh commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Review: GitHub OAuth as a third git auth method. No blockers — the token handling (askpass helper + stripping authorization from the script config) is sound and well-covered by tests. The notes below are polish.

Backward-compat: the change is additive — a new auth enum value appended, new optional fields, and one relaxed required constraint — so existing none/pat/ssh configs are unaffected.

# Area Sev Location Finding
1 Docs Important README.md:117 "Setting this up takes two separate steps on GitHub" is followed by three numbered items, and steps 2–3 happen in Keboola, not on GitHub. Could we make it "three steps" and mark which are on GitHub vs in Keboola?
2 Wording Nice-to-have tests/test_component.py:271 Docstring uses a generic verb like "surface" where a specific one reads clearer. Consider "must raise a UserException, not an internal error."
3 UI schema Nice-to-have component_config/configSchema.json:172 The repository dropdown has no autoload, so the user clicks "List Repositories" manually. Autoload would list on form open but also error before authorization — so a manual button is defensible. Flagging as a UX call, not a defect.
4 Robustness Nice-to-have src/github_api.py:61 A read timeout raises TimeoutError (subclass of OSError, not URLError), so it escapes both handlers and exits code 2. Connection timeouts are handled. Consider except OSError.
5 UI schema Nice-to-have component_config/configSchema.json:111 url dropped from git.required; a none/pat/ssh config can now be saved with an empty URL and fails only at run time. Safe (runtime guard catches it); flagging the lost pre-save check.
6 UI schema Nice-to-have component_config/configSchema.json:115 url (none/pat/ssh) and repository (oauth) are the same field — the repo clone URL — but present as two: url sits above the auth switch (propertyOrder 70, title "Repository URL"), repository below it (105, title "Repository"). Could we make them read as one field — same title ("Repository"), same tooltip, and repository ordered next to url — so only the extra "List Repositories" button marks the OAuth variant? Also consider renaming the new (unshipped) key repositoryurl_oauth to pair with url; url itself is a shipped key and stays.

Happy to discuss any of these.

A read timeout surfaces as a bare TimeoutError. It is an OSError but not a
URLError, so it escaped both handlers and ended the job as an internal error —
and listRepositories already runs close to the sync action limit against a large
organisation, so this is a likely path rather than a theoretical one.

Broadening the existing handler alone would not have been enough: TimeoutError
carries no "reason" attribute, so the message formatting has to change with it.
URLError keeps its own branch to preserve the more readable reason it does carry.

Also from review: the OAuth repository field is labelled "Repository URL" like
its free-text counterpart, since both hold the same clone URL, and the setup
section no longer promises two steps before listing three.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Oscar-XXII

Oscar-XXII commented Aug 24, 2026

Copy link
Copy Markdown
Author

Went through all six. Current disposition — some of these moved after later changes, so this is the up-to-date view rather than what I first replied:

# Disposition
1 Docs "two steps" / three items Fixed. Now "three steps, in this order. Only the first one happens on GitHub." Self-inflicted — step 3 arrived in a later commit and I never revised the lead-in.
2 "surface" in a docstring Left. It's this repo's own vocabulary, from the commit Surface invalid configuration as UserException instead of internal error.
3 No autoload on the picker Left, deliberately. cache: false means autoload would fire on every form render, and that call currently returns a truncated response against a large org. Revisit once that's fixed.
4 Read timeout escapes the handlers Fixed. Good catch, and above nice-to-have — listRepositories already runs near the sync-action limit. Note except OSError alone wasn't enough: TimeoutError has no reason, so the message formatting had to change too.
5 url dropped from required Fixed, differently than I first said. The fields were later collapsed into one always-visible git.url, so it is now plainly required with minLength: 1 — no conditional validation needed. My allOf attempt in between was wrong and is gone.
6 Two names for one value Fixed by removing the second field. git.url serves every auth method. Your naming argument was right and its conclusion turned out to be "don't have two fields at all".

Thanks for #4 and #6 in particular — both went further than the finding as written.

@soustruh

Copy link
Copy Markdown
Contributor

Not doing

Bold move, @claude! 🤖

#5url no longer in git.required. Correct that pre-save validation was lost, and it is a deliberate trade rather than an oversight. The schema has no conditional required, and keeping url mandatory is exactly what made an OAuth configuration unsavable — the field is hidden for auth: oauth, so it demanded something the form gave no way to supply. The runtime guard catches the empty case with wording matched to each variant ("Please select a repository" vs "Git repository URL is required"), both covered by tests. Recording it here so it doesn't get re-litigated later.

Is it possible to add a check for the URL field not being empty when the correct auth value is selected? I am not sure that is even possible in the RJSF, but can you check the docs please?

#6, second half — rename repositoryurl_oauth. Mild preference for keeping repository, and I'll happily fold if you feel strongly, because now is the only free moment. url_oauth encodes the authentication method into a field that just holds a clone URL; if a picker is ever added for another auth method the name becomes wrong, whereas repository describes the content rather than the mechanism.

Nope, it's confusing to have the same value (repo URL) in two differently named variables, url for one auth case and repository for another. As we need to keep the existing key unchanged, url is the right direction here and can server as a prefix for the new option, which differs only by the authorization used – hence the url_oauth name. Shall we need to add another URL with another auth flow, we will change the name or use yet another variable with a proper name.

Oh, I get it now, once loaded via the sync action, the field is populated with the repo names! Too bad the URL is then saved anyway, so when users pick the repo, save the configuration and then open it again, they will see the URL instead of the pretty name, which can only be triggered by clicking the sync action button again. As for me, I'd still use url + url_oauth in the code, but maybe we can keep the visible title at Repository? @claude, please do not decide this yourself, wait for @Oscar-XXII to make the decision, thanks.

I'd also skip the reordering. Moving url changes the form for every existing configuration, and moving repository above the auth switch would ask people to pick a repository before choosing how to authenticate. The current OAuth order reads auth → repository → branch, which is the right sequence.

No, please don't skip the reordering. Currently, when users pick the repo's visibility & auth, the whole forms jumps up and down based on the option selected, that is a terrible and unfriendly UX. Yes, the current OAuth order is the right sequence, so please change the original field to that order too. Existing users will deal with that, no biggie, they probably won't be able to tell to order themselves anyway.

#2 — "surface" in the test docstring. Leaving it: it is this repo's own vocabulary, from the commit titled Surface invalid configuration as UserException instead of internal error. Changing it here would make the test read differently from the code it covers.

Yes, the repo vocabulary all drifted towards AI slop, but no problem, we will deal with that later. :-P

Review raised that dropping url from git.required lost the pre-save check. It can
be restored conditionally: allOf + if/then keyed on auth, which is the pattern the
team's own advanced schema reference documents, and which the UI's sanitizer
already anticipates — if/then/else branches are its one exemption from the
automatic additionalProperties: false.

This is not the dependencies + oneOf pattern the UI guidance forbids; visibility
stays on options.dependencies and if/then carries validation only.

required alone would not have been enough. The form writes an empty string for a
field that was touched and cleared, and required only checks presence, so
minLength: 1 goes in the same branch. Verified against the eight relevant shapes
with the empty-enum placeholders stripped the way the UI strips them.

Closes a gap wider than the one reported: an OAuth configuration could also be
saved with no repository selected at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Oscar-XXII

Oscar-XXII commented Aug 24, 2026

Copy link
Copy Markdown
Author

Superseded. Claimed conditional required worked via allOf + if/then. It does not — the allOf makes the OAuth option vanish from the auth radio in RJSF. Removed in 49f6742; see the summary at the end of the thread.

…l stable

Two review points that turn out to answer each other.

The field is renamed repository -> url_oauth. It holds the same thing as url, a
clone URL, and differs only by the authentication that produced it, so it reads
better as a variant of the shipped key than as a second name for one concept.

The dropdown now uses the clone URL as both value and label. Options are only
fetched when the user runs the sync action, so a saved configuration reopened
without running it again had no label for the stored value and fell back to
showing the raw URL — the pretty owner/repo name appeared only after clicking the
button a second time. Using one string for both makes what is displayed the same
before and after a reload, and keeps url and url_oauth looking alike in the form,
which is the consistency the rename is after.

The cost is a dropdown of URLs rather than short owner/repo names. The prettier
label only becomes viable once the list can autoload, which is blocked on the
truncated-response problem.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Oscar-XXII

Oscar-XXII commented Aug 24, 2026

Copy link
Copy Markdown
Author

Superseded. Described the url_oauth / url two-field split. Both fields were later collapsed into a single git.url; see the summary at the end of the thread.

Oscar-XXII and others added 4 commits August 24, 2026 14:42
The form had two fields holding the same thing: a free-text url above the
authentication switch for none/pat/ssh, and a url_oauth dropdown below it for
oauth. That split existed only because I concluded a field cannot be both a text
input and a dropdown. options.creatable makes it one field: the list is loaded on
demand for OAuth, and any authentication method can type or paste a clone URL.

The shipped url key is the one that survives, so nothing migrates. Three things
fall away with the split: the naming problem of one value under two keys, the
allOf + if/then conditional required (a single always-visible field is simply
required, with minLength: 1 for the empty-string case), and the two-key autoload
on the branch list. The empty-repository message no longer branches on the
authentication method either, because there is one field to talk about.

Caveat to validate before merge: options.creatable combined with options.async is
not documented. If the two do not compose, the field would stop accepting typed
input and every non-OAuth configuration would be unable to set a repository. The
schema needs a pass in the Ctrl+D sandbox — type a URL, save, reopen — before
this ships.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…thods

Now that one field serves every authentication method, its List Repositories
button is offered to all of them — options.async belongs to the field, and
options.dependencies hides the whole field rather than the button, so there is no
way to show the button for OAuth alone without reinstating the two-field split.

The button being visible is cosmetic; the answer it gave was not. A personal
access token configuration was told "GitHub authorization is missing, please
authorize the component", sending the user to fix an authorization they neither
have nor need. The action now checks git.auth first and says the listing belongs
to GitHub authorization, and that other methods enter the URL directly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three mismatches were only discovered when a job ran: a non-GitHub URL under
GitHub authorization, an SSH URL under a personal access token, and an HTTPS URL
under an SSH key — the last one silently sets up no credentials at all. Each is
knowable the moment the URL is typed.

The patterns mirror what source_git.py actually accepts, deliberately no stricter:
https://github.com or www.github.com for oauth, any https for pat (self-hosted
GitHub Enterprise included), git@ or ssh:// for ssh, and anything for a public
repository. Verified against thirteen combinations.

This does not help the List Repositories button, which stays visible for every
method — the schema has no notion of a button being applicable. That remains the
sync action's job.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Confirmed in the Ctrl+D sandbox — with the allOf block present, the GitHub OAuth
option disappears from the authentication radio entirely. This is the known RJSF
allOf merge failure (rjsf-team/react-jsonschema-form#2752, #3445), which reports
exactly this symptom: "could not merge subschemas in allOf: Could not resolve
values for path: properties.value.enum". The if clauses mention properties.auth,
so the merge narrows the enum it was only meant to test against.

The schema itself was always valid — the eight and thirteen case runs that
justified this passed. They were run with the jsonschema library, which validates
but does not render, and the renderer is where this breaks. Validating semantics
is not evidence about the form.

Conditional validation is therefore off the table in this UI, whatever the
standard permits: url stays plainly required with minLength: 1, and the auth
specific URL rules stay in source_git.py where they already have clear messages.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Oscar-XXII

Oscar-XXII commented Aug 24, 2026

Copy link
Copy Markdown
Author

Folded into the summary at the end of the thread. The durable finding: allOf inside a schema object makes RJSF drop enum values from sibling properties (rjsf-team/react-jsonschema-form#2752, #3445), so conditional validation is not usable in this UI.

Shortens the message the List Repositories button returns for the other
authentication methods to the one we settled on. It stays an exception rather
than a returned option: a sync action can only answer with value/label pairs, so
a returned notice would be selectable and would end up saved as the repository
URL.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Oscar-XXII

Copy link
Copy Markdown
Author

Summary of where this landed, since the thread above wandered.

The two-field design and the conditional-validation attempt are both gone. There is one git.url for every authentication method, below the auth switch, and the List Repositories button answers Supported only for OAuth. Please insert the URL manually. for the others. The PR description has been rewritten to match.

One durable finding worth keeping: allOf inside a schema object makes RJSF drop enum values from sibling properties (#2752, #3445). Adding an if/then that merely tests properties.auth removed the OAuth option from the radio entirely. Conditional validation is not usable here, whatever the JSON Schema standard allows. I got this wrong first because I validated the schema with the jsonschema library, which validates but does not render.

Blocking before merge: options.creatable with options.async is undocumented, and if they don't compose, no non-OAuth configuration can set a repository. That needs a Ctrl+D pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants