Skip to content

Commit e6404d9

Browse files
feat(bots): local install-bot-engine + bot-prelude composites (PAT-free)
External consumers can't `uses:` the internal engine's actions cross-repo, so install the engine via a LOCAL composite that pip-installs it over HTTPS with a short-lived, engine-scoped GitHub App token (no stored PAT), routed through the internal JFrog mirror. bot-prelude wraps the shared per-workflow setup (mint bot token + engine-scoped token + Node + install) and centralizes the engine pin in one place (engine-ref default, pinned to databricks-bot-engine@3963f4b). Signed-off-by: Eric Wang <e.wang@databricks.com> Co-authored-by: Isaac Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
1 parent 614dd91 commit e6404d9

2 files changed

Lines changed: 216 additions & 0 deletions

File tree

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Shared prelude for the bot workflows — the steps identical across all four
2+
# (reviewer, reviewer-followup, engineer, engineer-followup):
3+
# 1. mint the bot's App installation token (this repo) — for PR/issue posting
4+
# 2. mint a SECOND token scoped to the engine repo (contents:read) — PAT-free
5+
# auth for the private-engine install
6+
# 3. set up Node (the Claude Code CLI the SDK spawns)
7+
# 4. install the pinned engine + Claude SDK/CLI via the local install-bot-engine
8+
# composite, authenticated by the engine-scoped token
9+
#
10+
# LOCAL composite (`uses: ./…`) — resolves against this checked-out repo, so it
11+
# works where a cross-repo `uses:` of the engine does NOT. Call it AFTER checkout
12+
# (a composite can't run the checkout that loads it) and AFTER Python/Poetry setup
13+
# (the engine `pip install` needs the interpreter present). What stays inline in
14+
# each workflow: checkout (differs per flow), Python-vs-Poetry (reviewer needs
15+
# only Python; engineer needs the connector's deps via setup-poetry), and the
16+
# run/publish tail.
17+
name: Bot prelude (tokens + Node + engine install)
18+
description: Mint the bot + engine-scoped App tokens, set up Node, and install the pinned engine (PAT-free). Shared by all four bot workflows.
19+
20+
inputs:
21+
app-id:
22+
description: 'Bot App id (review-bot or engineer-bot) — mints the this-repo token used to post.'
23+
required: true
24+
private-key:
25+
description: 'Bot App private key (PEM) for the same App.'
26+
required: true
27+
engine-ref:
28+
# SINGLE SOURCE OF TRUTH for the engine pin. All four bot workflows route
29+
# through this composite, so the SHA lives here once (overridable per-caller)
30+
# instead of being duplicated — and drifting — across the callers. Bump this
31+
# value to move every bot to a new engine commit; never @main.
32+
description: 'Engine commit SHA (full 40-char) to install.'
33+
required: false
34+
default: '3963f4be76be3b4ac0382ca0d7b6bf9d051929e6'
35+
engine-repo:
36+
description: 'owner/name of the engine repo.'
37+
required: false
38+
default: 'databricks/databricks-bot-engine'
39+
40+
outputs:
41+
token:
42+
description: 'The minted bot App installation token for THIS repo (post reviews / push fix branches / comment).'
43+
value: ${{ steps.app-token.outputs.token }}
44+
45+
runs:
46+
using: composite
47+
steps:
48+
# This-repo token: the identity the bot acts as (post reviews, push the fix
49+
# branch, comment). Exposed as the `token` output for the caller's run steps.
50+
- name: Mint bot App token
51+
id: app-token
52+
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
53+
with:
54+
app-id: ${{ inputs.app-id }}
55+
private-key: ${{ inputs.private-key }}
56+
57+
# Split `engine-repo` (owner/name) so the token mint below is scoped to the
58+
# SAME repo `install-bot-engine` clones. Deriving it (rather than hardcoding
59+
# databricks/databricks-bot-engine) keeps the overridable `engine-repo` input
60+
# honest: a fork/renamed engine gets a correctly-scoped token instead of an
61+
# auth failure against the wrong repo.
62+
- name: Derive engine repo owner/name
63+
id: engine-scope
64+
shell: bash
65+
env:
66+
ENGINE_REPO: ${{ inputs.engine-repo }}
67+
run: |
68+
echo "owner=${ENGINE_REPO%%/*}" >> "$GITHUB_OUTPUT"
69+
echo "repo=${ENGINE_REPO##*/}" >> "$GITHUB_OUTPUT"
70+
71+
# Engine-scoped token: a SECOND, per-run token scoped to ONLY the engine repo
72+
# (contents:read), used to `pip install` the private engine with no stored
73+
# PAT. Requires the same App installed on the engine repo with contents:read.
74+
- name: Mint engine-scoped install token
75+
id: engine-token
76+
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
77+
with:
78+
app-id: ${{ inputs.app-id }}
79+
private-key: ${{ inputs.private-key }}
80+
owner: ${{ steps.engine-scope.outputs.owner }}
81+
repositories: ${{ steps.engine-scope.outputs.repo }}
82+
permission-contents: read
83+
84+
- name: Setup Node (for the Claude Code CLI the SDK spawns)
85+
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
86+
with:
87+
node-version: '20'
88+
89+
- name: Install engine + Claude SDK/CLI
90+
uses: ./.github/actions/install-bot-engine
91+
with:
92+
engine-ref: ${{ inputs.engine-ref }}
93+
engine-repo: ${{ inputs.engine-repo }}
94+
engine-token: ${{ steps.engine-token.outputs.token }}
Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
# Install databricks-bot-engine (pinned REF) + the Claude Agent SDK/CLI.
2+
#
3+
# WHY THIS IS A LOCAL COMPOSITE (not `uses: databricks/databricks-bot-engine/...`):
4+
# databricks-sql-python cannot reference the engine's actions/reusable workflows
5+
# cross-repo — GitHub fails with "Unable to resolve action ..., not found" (the
6+
# internal engine repo's Actions aren't consumable from an external repo; adbc's
7+
# workflows note the same: "a cross-repo `uses:` of the hub's internal composite
8+
# action does NOT work"). So we do NOT `uses:` engine actions. Instead this local
9+
# action `pip install`s the engine over HTTPS with a token — a plain clone, which
10+
# an external repo CAN do — inlining what the engine's own install-bot-engine
11+
# action does in REF mode. Keep in sync with that action at the pinned engine SHA.
12+
#
13+
# Auth is PAT-FREE: the caller passes a short-lived, engine-scoped GitHub App
14+
# installation token as `engine-token` (minted from the bot App creds). It slots
15+
# into the same git auth path a PAT would; nothing stored/long-lived.
16+
name: Install bot engine (local)
17+
description: pip-install the pinned databricks-bot-engine + Claude Agent SDK/CLI, PAT-free (engine-scoped App token).
18+
19+
inputs:
20+
engine-ref:
21+
description: 'Engine commit SHA (full 40-char) to install.'
22+
required: true
23+
engine-repo:
24+
description: 'owner/name of the engine repo.'
25+
required: false
26+
default: 'databricks/databricks-bot-engine'
27+
engine-token:
28+
description: 'Short-lived engine-scoped GitHub App installation token (contents:read). Authenticates the private-engine clone; no stored PAT.'
29+
required: true
30+
use-jfrog:
31+
description: 'Route pip/npm through the internal JFrog mirror (egress-blocked runners). Requires id-token: write on the job.'
32+
required: false
33+
default: 'true'
34+
sdk-version:
35+
description: 'claude-agent-sdk version. Keep in sync with the engine at the pinned SHA.'
36+
required: false
37+
default: '0.2.102'
38+
cli-version:
39+
description: '@anthropic-ai/claude-code version. Keep in sync with the engine at the pinned SHA.'
40+
required: false
41+
default: '2.1.61'
42+
43+
runs:
44+
using: composite
45+
steps:
46+
- name: Configure JFrog mirror (pip + npm)
47+
if: inputs.use-jfrog == 'true'
48+
shell: bash
49+
run: |
50+
set -euo pipefail
51+
# Keyless: mint a GitHub OIDC token, exchange it for a short-lived JFrog
52+
# access token (the protected runner group is egress-blocked from
53+
# pypi.org / registry.npmjs.org). Requires the job to grant id-token: write.
54+
ID_TOKEN=$(curl -sLS \
55+
-H "User-Agent: actions/oidc-client" \
56+
-H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
57+
"${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=jfrog-github" | jq .value | tr -d '"')
58+
echo "::add-mask::${ID_TOKEN}"
59+
# Fail fast HERE if the OIDC mint failed (jq prints `null` for a missing
60+
# .value; a request error yields empty), so the diagnostic points at the
61+
# real cause instead of surfacing later as a misleading "could not extract
62+
# JFrog access token" from the exchange below.
63+
if [ -z "$ID_TOKEN" ] || [ "$ID_TOKEN" = "null" ]; then
64+
echo "::error::Could not mint GitHub OIDC token (is id-token:write granted?)"; exit 1
65+
fi
66+
ACCESS_TOKEN=$(curl -sLS -XPOST -H "Content-Type: application/json" \
67+
"https://databricks.jfrog.io/access/api/v1/oidc/token" \
68+
-d "{\"grant_type\": \"urn:ietf:params:oauth:grant-type:token-exchange\", \"subject_token_type\":\"urn:ietf:params:oauth:token-type:id_token\", \"subject_token\": \"${ID_TOKEN}\", \"provider_name\": \"github-actions\"}" | jq .access_token | tr -d '"')
69+
echo "::add-mask::${ACCESS_TOKEN}"
70+
if [ -z "$ACCESS_TOKEN" ] || [ "$ACCESS_TOKEN" = "null" ]; then
71+
echo "::error::Could not extract JFrog access token (is id-token:write granted?)"; exit 1
72+
fi
73+
echo "PIP_INDEX_URL=https://gha-service-account:${ACCESS_TOKEN}@databricks.jfrog.io/artifactory/api/pypi/db-pypi/simple" >> "$GITHUB_ENV"
74+
cat > ~/.npmrc << EOF
75+
registry=https://databricks.jfrog.io/artifactory/api/npm/db-npm/
76+
//databricks.jfrog.io/artifactory/api/npm/db-npm/:_authToken=${ACCESS_TOKEN}
77+
always-auth=true
78+
EOF
79+
echo "JFrog mirror configured for pip + npm"
80+
81+
- name: Install engine (pinned REF) + Claude SDK/CLI
82+
shell: bash
83+
env:
84+
ENGINE_REF: ${{ inputs.engine-ref }}
85+
ENGINE_REPO: ${{ inputs.engine-repo }}
86+
ENGINE_TOKEN: ${{ inputs.engine-token }}
87+
SDK_VERSION: ${{ inputs.sdk-version }}
88+
CLI_VERSION: ${{ inputs.cli-version }}
89+
run: |
90+
set -euo pipefail
91+
# Supply-chain: require a full 40-char commit SHA (immutable pin). `case`
92+
# globs match the whole string, so an embedded newline is rejected too.
93+
case "$ENGINE_REF" in
94+
*[!0-9a-f]* | "")
95+
echo "::error::engine-ref must be a full 40-char commit SHA, got '$ENGINE_REF'"; exit 1 ;;
96+
esac
97+
if [ "${#ENGINE_REF}" -ne 40 ]; then
98+
echo "::error::engine-ref must be a full 40-char commit SHA, got '$ENGINE_REF'"; exit 1
99+
fi
100+
if [ -z "$ENGINE_TOKEN" ]; then
101+
echo "::error::engine-token is required (engine repo is private)"; exit 1
102+
fi
103+
# Supply the token via a git extraheader, NOT in the URL handed to pip:
104+
# pip echoes the requirement spec (incl. any in-URL credential) in normal
105+
# and error output where Actions masking is not guaranteed. The header
106+
# lives only in a JOB-LOCAL git config for this job. Mask the token + its
107+
# base64 form defensively.
108+
echo "::add-mask::${ENGINE_TOKEN}"
109+
BASIC=$(printf 'x-access-token:%s' "$ENGINE_TOKEN" | base64 | tr -d '\n')
110+
echo "::add-mask::${BASIC}"
111+
export GIT_CONFIG_GLOBAL="${RUNNER_TEMP:-/tmp}/bot-engine-gitconfig-$$"
112+
: > "$GIT_CONFIG_GLOBAL"
113+
trap 'git config --global --unset-all "http.https://github.com/.extraheader" || true; rm -f "$GIT_CONFIG_GLOBAL"' EXIT
114+
git config --global "http.https://github.com/.extraheader" "Authorization: Basic ${BASIC}"
115+
python -m pip install --upgrade pip
116+
# REF mode resolves the published engine's deps unpinned + installs the
117+
# SDK/CLI by version (a consumer can't see the engine's hash-locked
118+
# requirements). SDK_VERSION/CLI_VERSION track the engine at ENGINE_REF.
119+
pip install "databricks-bot-engine @ git+https://github.com/${ENGINE_REPO}@${ENGINE_REF}"
120+
pip install "claude-agent-sdk==${SDK_VERSION}"
121+
npm install -g "@anthropic-ai/claude-code@${CLI_VERSION}"
122+
echo "$(npm prefix -g)/bin" >> "$GITHUB_PATH"

0 commit comments

Comments
 (0)