diff --git a/AGENTS.md b/AGENTS.md index ce34569..ed0dae1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -69,7 +69,7 @@ for the full product spec and `docs/build-plan.md` for the delivery order. repo. It emits an ordinary UDF around `AI_COMPLETE` anyway, so we write that ourselves. - **Backend**: Cloud Run — `/api/generate`, plus serving `/{owner}/{repo}` and - `/{owner}/{repo}/card.svg`. It renders the SVG from Snowflake's card payload and + `/{owner}/{repo}/card.png`. It renders the SVG from Snowflake's card payload and writes it to the bucket. It computes no analysis of its own. - **Cache of record**: a public GCS bucket. The card's existence in the bucket _is_ the ready state. @@ -105,8 +105,9 @@ rescaffold; if you find a reference to either, it is stale — fix it. - A repo with no real story says so. Sparse histories get an honest template card, not manufactured drama. - Cost guards are mandatory: cap commits per repo, cap daily live generations, cache - failed states, reject private/missing/oversized repos, and keep gallery cards - pre-generated. + failed states, reject private/missing repos, and keep the landing-page example cards + pre-generated. An oversized history is windowed to its newest slice, not rejected — + and the card says so rather than passing a slice off as the whole life. ## Documentation diff --git a/Dockerfile b/Dockerfile index a09a8d7..e644121 100644 --- a/Dockerfile +++ b/Dockerfile @@ -23,6 +23,9 @@ ENV NODE_ENV=production COPY --from=deps /app/node_modules ./node_modules COPY --from=build /app/dist ./dist COPY package.json ./ +# The image ships no fonts, and the card names its families by hand. Without these the PNG +# rasterizes with the type missing — and the type is the card. +COPY assets ./assets USER node EXPOSE 8080 diff --git a/LICENSE b/LICENSE index 771b212..22dc0c7 100644 --- a/LICENSE +++ b/LICENSE @@ -4,6 +4,12 @@ Copyright (c) 2026 Ashley Childress +Required Notice: Copyright Ashley Childress (https://anchildress1.dev) + +Licensor: Ashley Childress + +Software: Commit Chronicles + ## Acceptance In order to get any license under these terms, you must agree diff --git a/Makefile b/Makefile index 79b02e6..4381e21 100644 --- a/Makefile +++ b/Makefile @@ -65,7 +65,7 @@ gcp-bootstrap: ## Create the GCP resources deploy.sh expects (idempotent, one-of ./scripts/gcp-bootstrap.sh deploy: ## Build the image and deploy to Cloud Run - set -a; [ -f .env ] && . ./.env; set +a; ./deploy.sh + ./deploy.sh clean: ## Remove build output rm -rf dist coverage playwright-report test-results .vite diff --git a/README.md b/README.md index 47a82d3..b4bff9e 100644 --- a/README.md +++ b/README.md @@ -1,66 +1,339 @@ # Commit Chronicles -A contribution graph tells you that work happened. It never tells you what happened. +A contribution graph tells you that work happened. It never tells you _what_ happened. -Paste a public GitHub repo. Snowflake reads its commit history, finds the one story -actually hiding in it — the project that went dark for 107 days and came back at 3:32am, -the repo built entirely after midnight whose last commit landed at 3:53 and never got -another — and renders it as a card you can drop into a README. +Paste a public GitHub repo. **Snowflake fetches its own commit history, finds the one story hiding in it with plain SQL, and narrates that single thread with Cortex.** You get a card you can drop into a README. -Plain SQL finds the story. Cortex narrates it, and picks the color it deserves. +[![CI](https://github.com/anchildress1/commit-chronicles/actions/workflows/ci.yml/badge.svg)](https://github.com/anchildress1/commit-chronicles/actions/workflows/ci.yml) [![License: PolyForm Shield 1.0.0](https://img.shields.io/badge/license-PolyForm%20Shield%201.0.0-blue)](LICENSE) [![Conventional Commits](https://img.shields.io/badge/commits-conventional-fe5196)](https://www.conventionalcommits.org/) -```md -[![Commit Chronicles](https://storage.googleapis.com/commit-chronicles-cards/cards/owner/repo/card.svg)](https://commitchronicles.dev/owner/repo) +[![Snowflake Cortex](https://img.shields.io/badge/Snowflake-Cortex-29B5E8?logo=snowflake&logoColor=white)](#the-snowflake-case) [![Claude](https://img.shields.io/badge/Claude-D97757?logo=claude&logoColor=white)](https://claude.com/claude-code) [![Codex](https://img.shields.io/badge/Codex-000000?logo=openai&logoColor=white)](https://openai.com/codex) [![Copilot](https://img.shields.io/badge/Copilot-000000?logo=githubcopilot&logoColor=white)](https://github.com/features/copilot) + +[![Created](https://img.shields.io/github/created-at/anchildress1/commit-chronicles?label=repo%20created&color=6ab5f5)](https://github.com/anchildress1/commit-chronicles/commits) [![Last commit](https://img.shields.io/github/last-commit/anchildress1/commit-chronicles?color=d3e85a)](https://github.com/anchildress1/commit-chronicles/commits) [![Release](https://img.shields.io/github/v/release/anchildress1/commit-chronicles?color=e8a04a)](https://github.com/anchildress1/commit-chronicles/releases) [![Release date](https://img.shields.io/github/release-date/anchildress1/commit-chronicles?label=released&color=e8a04a)](https://github.com/anchildress1/commit-chronicles/releases) + +![Commit Chronicles](https://repository-images.githubusercontent.com/1296802597/3d5d4e9b-0ea6-4ff1-b0a9-80e1947d7be3) + +**Live:** [commitchronicles.anchildress1.dev](https://commitchronicles.anchildress1.dev) · **Prize target:** Best Use of Snowflake + +> **Judging scope: `v1.0.0` is the challenge submission.** It was cut for the DEV Weekend Challenge deadline and is the only thing that should be judged. The repo was created on 10 Jul 2026 and everything in the entry was built inside the challenge window — the badges above are the receipt. Any commit or release after `v1.0.0` is post-deadline work and is not part of the entry. + +Two real repos, two storylines the detector found unaided — a **binge** and a **nocturne**. Every dot, timestamp, and quoted commit is real: + +[![save-the-sun](https://storage.googleapis.com/commit-chronicles-cards/cards/anchildress1/save-the-sun/card.png)](https://commitchronicles.anchildress1.dev/anchildress1/save-the-sun) + +[![carbon-trace](https://storage.googleapis.com/commit-chronicles-cards/cards/anchildress1/carbon-trace/card.png)](https://commitchronicles.anchildress1.dev/anchildress1/carbon-trace) + +And a third, which is the one that proves the other two. `forem/forem` has 463 commits and **no story** — a team repo with continuous CI has no personal arc to find. It says so, in grey, and Cortex is never called: + +[![forem/forem](https://storage.googleapis.com/commit-chronicles-cards/cards/forem/forem/card.png)](https://commitchronicles.anchildress1.dev/forem/forem) + +A product that cannot return nothing will invent something. This one returns nothing. + +--- + +## Table of Contents + +- [About](#about) +- [Features](#features) +- [Tech Stack](#tech-stack) +- [Architecture](#architecture) +- [The Snowflake case](#the-snowflake-case) +- [Project Structure](#project-structure) +- [Getting Started](#getting-started) +- [Configuration](#configuration) +- [Security](#security) +- [How to Contribute](#how-to-contribute) +- [What's Next](#whats-next) +- [License](#license) +- [Author](#author) + +--- + +## About + +Buried in a repo's commit history there is usually exactly one story worth telling. A project that went dark for 107 days and came back at 3:32am. A repo built entirely after midnight whose last commit landed at 3:53 and never got another. A week where every commit was a revert. + +Commit Chronicles finds that story with SQL, narrates it with Cortex, and renders it as a 1200×630 SVG sized for a README and a social preview. + +The scope is **one repository**, not a whole profile. A year-in-review across a profile turns to mush. A repo has a clean arc: commits start, cluster, pause, restart, or stop. + +Two rules hold the product together: + +- **Cortex interprets the shape. It never invents the facts.** Every timestamp, count, gap, and quoted message on the card is real and derived from ingested commits. Reading the arc is the product; asserting the author's motivation is not. +- **A repo with no real story says so.** Sparse histories get an honest grey template card, not manufactured drama. + +--- + +## Features + +| Feature | What it does | +| ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | +| **Repo-first flow** | Enter a public `owner/repo`, submit once. Generation is keyed by the slug. | +| **Durable generation** | Work runs on a Cloud Tasks worker request, so closing the tab doesn't kill the job. Come back later to `/{owner}/{repo}` and re-attach. | +| **Six-storyline SQL detector** | `relapse`, `nocturne`, `binge`, `collapse`, `fight`, `resurrection` — scored deterministically, exactly one winner, plus a `none` fallback. | +| **One Cortex call** | The winning thread's evidence only, never the whole history. Schema-constrained output. | +| **Cortex picks the palette** | The accent hex is a reading of the arc, chosen from five muted neons, with the reason recorded on the card row. | +| **Copyable README embed** | The card is a public bucket object. Hotlink it from anywhere. | +| **Cost guards by construction** | Daily cap, in-flight dedupe, cached failures, queue-level concurrency ceiling, ingest cap. | + +--- + +## Tech Stack + +| Layer | Choice | +| -------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| **Data + AI engine** | **Snowflake** — external access integration, plain-SQL detector views, `AI_COMPLETE` via a hand-written UDF (`claude-sonnet-4-5`) | +| **Deploy tooling** | `snow` CLI — every warehouse object is SQL in this repo | +| **Backend** | Node 24 (ESM), TypeScript strict, [Hono](https://hono.dev) | +| **Frontend** | React 19 + Vite (path-driven SPA, no router library) | +| **Compute** | Cloud Run (scale-to-zero, request-billed) | +| **Queue** | Cloud Tasks (OIDC-signed worker callbacks) | +| **Cache of record** | Public GCS bucket | +| **Tests** | Vitest + Testing Library, Playwright for E2E | + +--- + +## Architecture + +Snowflake does the work. Cloud Run guards the request, calls one stored procedure, turns the returned payload into an SVG, and writes it to the bucket. **It computes no analysis of its own.** + +```mermaid +flowchart TD + accTitle: Commit Chronicles architecture + accDescr: The SPA posts to Cloud Run, which claims quota and enqueues a Cloud Tasks job. The worker calls one Snowflake procedure. Inside the warehouse, a Python stored procedure reaches api.github.com through an external access integration to ingest commits; plain SQL views score six storylines and pick one winner; only that winner's evidence is selected and passed to Cortex, whose output is validated in SQL before a JSON card payload is returned. Cloud Run renders that payload to SVG and writes it to a public GCS bucket, which serves the card to READMEs. + + GH[("GitHub
api.github.com")] + + SPA["SPA"] + API["Cloud Run
POST /api/generate"] + Q["Cloud Tasks
max 2 concurrent"] + W["Cloud Run worker
POST /internal/generate"] + + subgraph SF["Snowflake — CALL READ_REPO"] + direction TB + EAI["External Access Integration
GITHUB_API_ACCESS"] + ING["ingest
INGEST_REPO_COMMITS, cap 500"] + DET["detector.sql
gaps, streaks, night hours"] + SCORE["storyline scoring
six candidates, one winner"] + EV["evidence selection
CARD_EVIDENCE, the winner only"] + CTX["Cortex
CHRONICLE_CARD, one AI_COMPLETE"] + VAL["validation
hex, labels, kicker, in SQL"] + JSON["JSON card payload"] + + EAI --> ING --> DET --> SCORE --> EV --> CTX --> VAL --> JSON + SCORE -.->|"storyline none, Cortex skipped"| JSON + end + + REN["Cloud Run
renders the SVG"] + GCS[("Public GCS bucket
the cache of record")] + OUT["SVG
README embed, social preview"] + + SPA -->|"repo slug"| API + API -->|"guard, claim, enqueue"| Q + Q -->|"survives the tab closing"| W + W --> SF + GH <-.->|"the warehouse gets its own data"| EAI + JSON --> REN + REN -->|"only writer"| GCS + GCS --> OUT + GCS -->|"GET /api/state"| SPA +``` + +The `none` path is not an error branch — it is the honest answer for a history with no arc, and it skips Cortex entirely. + +**The card's existence in the bucket _is_ the ready state.** There is no Firestore, no status column, no separate database. `readState` checks for `card.json`; if it's there, the repo is ready. + +### The generation path + +```mermaid +sequenceDiagram + accTitle: Generation sequence + accDescr: The browser reads state first and only posts to generate on a miss. Cloud Run claims a quota slot and a generating marker, then enqueues a Cloud Tasks job. The worker calls Snowflake, renders the SVG, and writes it to the bucket. The browser polls state until ready. + + actor U as Browser + participant R as Cloud Run + participant B as GCS bucket + participant Q as Cloud Tasks + participant S as Snowflake + + U->>R: GET /api/state + R->>B: does card.json exist? + B-->>R: no + R-->>U: unknown + + U->>R: POST /api/generate + R->>B: claim daily quota + R->>B: claim generating, create-only + Note over R,B: two racing requests cannot
both start a Cortex call + R->>Q: enqueue, OIDC-signed + R-->>U: 202 generating + + Q->>R: POST /internal/generate + R->>S: CALL READ_REPO + S->>S: ingest if cold, detect, narrate + S-->>R: card payload + + R->>R: render SVG + R->>B: card.png, then card.json, then clear state + Note over R,B: card.json last: a crash leaves it
retryable, never ready-with-no-card + + loop every 2.5s + U->>R: GET /api/state + end + R-->>U: ready + cardUrl +``` + +**The queue is a cost decision.** Cloud Tasks calls back _into_ the service, so the pipeline runs inside a request — CPU is billed only while it works and the service still scales to zero. Detaching the work instead would need `--no-cpu-throttling`, which bills the container for sitting there doing nothing. + +--- + +## The Snowflake case + +This is the part that matters for the prize. **The warehouse is the editor, not a bucket the LLM reads from.** + +### 1. Snowflake reaches out and gets its own data + +An `EXTERNAL ACCESS INTEGRATION` lets a Python stored procedure call `api.github.com` from _inside_ the warehouse. There is no ingestion service, no ETL job, no Cloud Function shovelling JSON. + +| Object | Type | Job | +| ---------------------------------- | ----------------------------- | ------------------------------------------------------------------------------------- | +| `GITHUB_API_RULE` | `NETWORK RULE` (EGRESS) | Let the warehouse out to `api.github.com` | +| `GITHUB_TOKEN` | `SECRET` | The GitHub token, created out-of-band ([setup docs](docs/snowflake-setup.md)) | +| `GITHUB_API_ACCESS` | `EXTERNAL ACCESS INTEGRATION` | Binds the rule to the secret | +| `INGEST_REPO_COMMITS(owner, repo)` | `PROCEDURE` (Python) | Paginates the REST Commits API into `COMMITS`, classifies bot/AI-assisted rows in SQL | + +Ingest caps at **500 commits** by default (hard cap 2000). A longer history sets `windowed`, and the card says so out loud — `last 500 commits · quiet since Feb 25` — because reporting a slice as the repo's whole life is simply false. + +### 2. Plain SQL finds the story — no LLM, no cost + +`detector.sql` is 15 views and zero AI calls. It scores every candidate storyline and keeps exactly one. + +```mermaid +flowchart LR + accTitle: The SQL detector + accDescr: COMMITS_CLEAN feeds six storyline views, each gated on a minimum commit count. Their scores union into STORYLINE_SCORES, and REPO_STORYLINE picks the single highest score, breaking ties by drama rank. CARD_EVIDENCE then selects only the winning thread's commit lines for Cortex. + + C["COMMITS_CLEAN
merges and bots dropped"] + + subgraph SC["STORY views — scored, gated, free"] + direction TB + R["relapse
gap 30d or more"] + N["nocturne
half the commits at night"] + B["binge
streak of 7d or more"] + X["collapse
a spike, then 90d silent"] + F["fight
4 reverts inside 7d"] + S["resurrection
a relapse that shipped"] + end + + W["REPO_STORYLINE
highest score, ties by drama"] + E["CARD_EVIDENCE
the winner's lines only"] + AI["CHRONICLE_CARD
the one Cortex call"] + NONE["none
grey template card
Cortex never runs"] + + C --> SC --> W + W -->|"a winner"| E --> AI + W -->|"nothing clears its floor"| NONE +``` + +Every storyline gates on `MIN_COMMITS = 15` so bot noise can't win, and scoring is deterministic — the same repo always yields the same story. Thresholds live in one `DETECTOR_CONFIG` view: + +```sql +CREATE OR REPLACE VIEW DETECTOR_CONFIG AS SELECT + 15 AS MIN_COMMITS, 30 AS RELAPSE_MIN_GAP_DAYS, + 90 AS ABANDONED_AFTER_DAYS, 7 AS BINGE_MIN_STREAK_DAYS, + 4 AS FIGHT_MIN_COMMITS, 22 AS NIGHT_START_HOUR, + 5 AS NIGHT_END_HOUR, 25 AS EVIDENCE_SHARE_PCT, + 20 AS EVIDENCE_MIN_LINES, 140 AS EVIDENCE_MAX_LINES; +``` + +The winner is picked with one window function: + +```sql +QUALIFY ROW_NUMBER() OVER ( + PARTITION BY f.REPO_OWNER, f.REPO_NAME + ORDER BY s.SCORE DESC NULLS LAST, s.DRAMA_RANK +) = 1 ``` -## How it works 🔧 +**Surveying a whole history produces a report. Picking one story produces an argument.** + +### 3. Cortex narrates _only_ the winning thread + +`CHRONICLE_CARD` is a hand-written SQL UDF wrapping `AI_COMPLETE` — one schema-constrained call. Model `claude-sonnet-4-5`, `temperature 0.4`, `max_tokens 2048`. + +It is fed `CARD_EVIDENCE`: the winning thread's commit lines, budgeted at **25% of the history, floored at 20 and capped at 140** — plus the first 5 and last 8 regardless, because the opening and the ending are the shape of the story. Never the whole repo. Squash-merge bodies are exploded into individual lines first, so work hidden inside a merge is still visible. + +The selection is deterministic: lines are ranked by distance from the pivot and tie-broken on `SHA` and `PART`, so the same repo hands Cortex the same evidence in the same order every time. + +The response schema constrains exactly nine keys: + +```json +{ + "kicker": "the death of a side project", + "headline_upright": "Born in daylight. Last touched at", + "headline_accent": "3:53 in the morning", + "headline_trail": ".", + "label_first": "it begins", + "label_pivot": "", + "label_last": "", + "accent": "#e8a04a", + "accent_reason": "amber, for a repo that ran hot and went out" +} +``` + +That is the entire surface area of the writing. **Everything else on the card is composed by the renderer from facts** — the commit count, the status verb, the anchor timestamps, the void-panel gap, the caption. Cortex is never taught to produce a number. + +The output is then _verified in SQL_ before it is stored. A bad accent hex, a digit smuggled into a poetic label, or a kicker that just echoes the storyline name gets rejected with `cortex_rejected` — the card is not written. + +### 4. Cheap by construction + +Detection costs nothing. The LLM sees ~20–140 lines, not twenty thousand. `CHRONICLES_WH` is an XSMALL that auto-suspends after 60 seconds with a 300-second statement timeout. Every Cortex query ID is stored on the card row for cost audit. + +> **Not built with Cortex AI Function Studio — deliberately.** The Studio registers functions through `SNOWFLAKE.CORTEX.CREATE_AI_FUNCTION`, which Snowflake documents as internal and subject to change without notice, and its entry points are a Snowsight wizard and the Cortex Code CLI — neither leaves the function in this repo. It emits an ordinary UDF around `AI_COMPLETE` anyway, so we wrote that ourselves. You can read it. + +--- + +## Project Structure ```text -SPA → POST /api/generate → Cloud Tasks → /internal/generate - │ - ▼ - Snowflake: CALL READ_REPO(owner, repo) - ├─ ingest external access → api.github.com - ├─ detector plain SQL → score storylines → pick ONE - └─ cortex narrate that one thread + pick the accent - │ - card payload (JSON) - ▼ - Cloud Run renders the SVG → public GCS bucket +snowflake/ # the app. Every object is SQL, deployed with `snow`. + schema.sql # warehouse, tables, COMMITS_CLEAN, PIPELINE_VERSION + ingest_pipeline.sql # network rule, external access integration, ingest proc + detector.sql # 15 views: facts, gaps, the six storylines, the winner, the evidence + ai_functions.sql # CHRONICLE_CARD — the AI_COMPLETE wrapper + read_repo.sql # READ_REPO — the single entry point Cloud Run calls +src/ + server/ + app.ts # routes + generate.ts # claim → enqueue → run → render → write + bucket.ts # GCS: card.png, card.json, state.json, quota counters + queue.ts # Cloud Tasks + OIDC verification (inline fallback for laptops) + snowflake.ts # the driver wrapper + rerender.ts # CLI: redraw stored cards, no Cortex spend + card/ # SVG renderer — layout, text fitting, formatting + client/ # React SPA: Landing, Loading, Result, Failed + shared/ # slug parsing, error taxonomy (shared by both sides) +docs/ # the spec, the build plan, Snowflake account bootstrap +scripts/ # gcp-bootstrap.sh — one-off GCP resources ``` -- **Snowflake is the app.** It fetches its own data through an external access integration, - finds the story with window functions, and shows Cortex only the winning thread — a - quarter of the material, capped, never the whole history. Squash-merge bodies are split - into line items first, so the work inside a merge is visible. -- **Cloud Run computes nothing.** It guards the request, calls one stored procedure, turns - the returned payload into an SVG, and writes it to the bucket. -- **The bucket is the cache of record.** A card's existence in it _is_ the ready state. - A cached page never re-runs Cortex. -- **Every colour on the card is Cortex's.** A repo that died and one that came back and - shipped must not wear the same accent. +--- -Generation runs through a queue so it survives you closing the tab, and so the service can -still scale to zero — see the architecture section of -[`docs/initial-design-spec.md`](docs/initial-design-spec.md) for why that is a cost -decision rather than a plumbing one. +## Getting Started -## Development +Requires **Node ≥ 24** and, for the warehouse half, the [`snow` CLI](https://docs.snowflake.com/en/developer-guide/snowflake-cli/index). ```bash make install # deps + git hooks +cp .env.example .env make dev # API on :8080, SPA on :5273 make ai-checks # format, lint, typecheck, test, build — the full gate ``` -Without a Cloud Tasks queue configured, generation runs in-process, so a laptop needs no -queue to work. +Without a Cloud Tasks queue configured, generation runs in-process. **A laptop needs no queue.** -Copy `.env.example` to `.env` and fill it in — see -[`docs/snowflake-setup.md`](docs/snowflake-setup.md) for how to mint the Snowflake PAT and -the GitHub token. +See [`docs/snowflake-setup.md`](docs/snowflake-setup.md) for minting the Snowflake PAT (it's role-locked at creation — that trips everyone once) and the GitHub token. -## Deploying +### Deploying ```bash make snowflake-deploy # every warehouse object, in dependency order @@ -68,16 +341,68 @@ make gcp-bootstrap # bucket, image repo, service accounts, secret, queue (o make deploy # build the image, deploy to Cloud Run, prune to 3 revisions ``` -`SNOWFLAKE_PAT` lives in Secret Manager and is mounted at run time. `.env` is for local -development only; nothing in it is baked into an image. +`SNOWFLAKE_PAT` lives in Secret Manager and is mounted at run time. `.env` is local-only; nothing in it is baked into an image. + +--- + +## Configuration + +| Variable | Required | Default | Purpose | +| ------------------------ | -------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `CARD_BUCKET` | **yes** | — | The public GCS bucket. Cards, state, quota counters. | +| `SNOWFLAKE_ACCOUNT` | **yes** | — | Account identifier | +| `SNOWFLAKE_USER` | **yes** | — | | +| `SNOWFLAKE_PAT` | **yes** | — | Programmatic access token; role-locked at creation | +| `SNOWFLAKE_WAREHOUSE` | no | `CHRONICLES_WH` | | +| `SNOWFLAKE_DATABASE` | no | `CHRONICLES` | | +| `SNOWFLAKE_SCHEMA` | no | `RAW` | | +| `SNOWFLAKE_ROLE` | no | `ACCOUNTADMIN` | | +| `PORT` | no | `8080` | | +| `PUBLIC_ORIGIN` | no | `https://commitchronicles.anchildress1.dev` | Used to build the page URL in the README embed | +| `DAILY_GENERATION_CAP` | no | `100` | Hard ceiling on live generations per day | +| `GENERATING_TTL_SECONDS` | no | `600` | After this, an in-flight job is presumed dead and re-admitted | +| `TASKS_QUEUE` | no | — | **All-or-nothing.** Set it and `GOOGLE_CLOUD_PROJECT`, `TASKS_LOCATION`, `WORKER_URL`, `TASKS_INVOKER_SA` all become required. Unset ⇒ in-process generation. | +| `GITHUB_TOKEN` | no | — | Only used to mint the Snowflake `SECRET`. **Cloud Run never calls GitHub.** | -## Docs +--- -- [`docs/initial-design-spec.md`](docs/initial-design-spec.md) — the product spec, the card - contract, and the split between what Cortex writes and what the renderer composes. -- [`docs/build-plan.md`](docs/build-plan.md) — delivery order. -- [`docs/snowflake-setup.md`](docs/snowflake-setup.md) — account bootstrap. +## Security + +- **Cloud Run is the only writer to the bucket.** Client writes are forbidden. The bucket is public-read because hotlinked cards _are_ the product. +- **`/internal/generate` spends Cortex credits, so it verifies OIDC** — the task's token is checked against the invoker service account's email before anything is billed. No token, no work. +- **Path handling denies by default.** `parseSlug` rejects `..`, enforces GitHub's own owner/repo grammar, and runs on both the client and the server. An unparseable slug never reaches the warehouse. +- **Cortex's accent hex is re-verified before it reaches the SVG** (`safeAccent`), even though the response schema already constrains it. The card is public; the hex is untrusted output. +- **Secrets never land in the repo.** `SNOWFLAKE_PAT` is mounted from Secret Manager; the GitHub token exists only as a Snowflake `SECRET` object. `.env` is local-only and git-ignored. +- **Abuse controls:** daily generation cap counted in the bucket (so it holds across instances), create-only claim so two racing requests can't both bill a Cortex call, cached failures for terminal errors, and a queue `max-concurrent-dispatches=2` ceiling on the warehouse spend rate. + +--- + +## How to Contribute + +- Branch and PR always. Nothing lands directly on `main`. +- [Conventional Commits](https://www.conventionalcommits.org/), GPG-signed, one logical change per commit. +- AI-authored commits carry a `Generated-by:` footer naming the model that wrote the diff, plus a human `Signed-off-by:`. Commitlint enforces both. +- `make ai-checks` is the gate. Warnings are errors. +- New components and utilities ship with positive, negative, and edge-case tests. + +--- + +## What's Next + +- **Embedded font subset.** GitHub proxies README images through camo, so webfonts don't load in the card. The fix is a base64-embedded Didone subset; today the card falls back through a serif stack. +- **A gallery route.** Cut for the deadline — the landing page ships three example chips instead. +- **Auto-regeneration of stale cards.** `STALE_CARDS` already reports which cards were written by a pipeline version that no longer exists. It reports rather than acts, because acting costs a Cortex call each. + +--- ## License -[Polyform Shield 1.0.0](LICENSE) +[PolyForm Shield 1.0.0](LICENSE). + +Read it, learn from it, fork it, run it yourself. The one thing you can't do is sell it as a competing product. Poking at the SQL is fine; relabelling this as your own commit-storytelling product is not. + +--- + +## Author + +**Ashley Childress** — [@anchildress1](https://github.com/anchildress1) diff --git a/assets/fonts/BodoniModa-0.ttf b/assets/fonts/BodoniModa-0.ttf new file mode 100644 index 0000000..8a7b7d6 Binary files /dev/null and b/assets/fonts/BodoniModa-0.ttf differ diff --git a/assets/fonts/BodoniModa-1.ttf b/assets/fonts/BodoniModa-1.ttf new file mode 100644 index 0000000..e0f38ca Binary files /dev/null and b/assets/fonts/BodoniModa-1.ttf differ diff --git a/assets/fonts/BodoniModa-2.ttf b/assets/fonts/BodoniModa-2.ttf new file mode 100644 index 0000000..788a538 Binary files /dev/null and b/assets/fonts/BodoniModa-2.ttf differ diff --git a/assets/fonts/BodoniModa-3.ttf b/assets/fonts/BodoniModa-3.ttf new file mode 100644 index 0000000..7d2cb49 Binary files /dev/null and b/assets/fonts/BodoniModa-3.ttf differ diff --git a/assets/fonts/BodoniModa-4.ttf b/assets/fonts/BodoniModa-4.ttf new file mode 100644 index 0000000..bc1a234 Binary files /dev/null and b/assets/fonts/BodoniModa-4.ttf differ diff --git a/assets/fonts/HankenGrotesk-0.ttf b/assets/fonts/HankenGrotesk-0.ttf new file mode 100644 index 0000000..a7cbaa1 Binary files /dev/null and b/assets/fonts/HankenGrotesk-0.ttf differ diff --git a/assets/fonts/HankenGrotesk-1.ttf b/assets/fonts/HankenGrotesk-1.ttf new file mode 100644 index 0000000..ad9c7c7 Binary files /dev/null and b/assets/fonts/HankenGrotesk-1.ttf differ diff --git a/assets/fonts/HankenGrotesk-2.ttf b/assets/fonts/HankenGrotesk-2.ttf new file mode 100644 index 0000000..e6e24ae Binary files /dev/null and b/assets/fonts/HankenGrotesk-2.ttf differ diff --git a/assets/fonts/SpaceMono-0.ttf b/assets/fonts/SpaceMono-0.ttf new file mode 100644 index 0000000..edc2425 Binary files /dev/null and b/assets/fonts/SpaceMono-0.ttf differ diff --git a/assets/fonts/SpaceMono-1.ttf b/assets/fonts/SpaceMono-1.ttf new file mode 100644 index 0000000..51cd703 Binary files /dev/null and b/assets/fonts/SpaceMono-1.ttf differ diff --git a/deploy.sh b/deploy.sh index 365ed63..ef02acc 100755 --- a/deploy.sh +++ b/deploy.sh @@ -5,11 +5,29 @@ # The project resources this expects (bucket, Artifact Registry repo, service accounts, # Secret Manager entry, Cloud Tasks queue) are created once by `make gcp-bootstrap`. # -# Secrets never appear here. SNOWFLAKE_PAT is mounted from Secret Manager at run time; -# .env is for local development only and is not read by this script. +# Config comes from .env; SNOWFLAKE_PAT comes from Secret Manager and is mounted at run +# time. No secret value is ever passed on a gcloud command line, where it would land in +# shell history and the audit log. set -euo pipefail +# Read here rather than in the caller, so the script behaves the same run bare or run by +# make. Named keys only — sourcing .env would pull SNOWFLAKE_PAT and GITHUB_TOKEN into the +# environment of a process that shells out to gcloud, which is exactly what Secret Manager +# exists to prevent. An already-exported value wins, so CI can override with no file present. +# +# Every key here is optional at *this* layer: a missing one falls through to the defaults +# below, or to the ${VAR:?} guards that make the genuinely-required ones fatal. Hence the +# `|| true` — under `set -e`, a grep that matches nothing would otherwise kill the deploy. +CONFIG_KEYS=(SNOWFLAKE_ACCOUNT SNOWFLAKE_USER CARD_BUCKET PUBLIC_ORIGIN DAILY_GENERATION_CAP) +if [[ -f .env ]]; then + for key in "${CONFIG_KEYS[@]}"; do + [[ -n ${!key:-} ]] && continue + value="$(grep -E "^${key}=" .env | tail -1 | cut -d= -f2- || true)" + [[ -n ${value} ]] && export "${key}=${value}" + done +fi + PROJECT="${PROJECT:-anchildress1}" REGION="${REGION:-us-east1}" SERVICE="${SERVICE:-commit-chronicles}" diff --git a/docs/build-plan.md b/docs/build-plan.md index 5670c3a..5b12f76 100644 --- a/docs/build-plan.md +++ b/docs/build-plan.md @@ -1,136 +1,49 @@ # Commit Chronicles — Build Plan -## TL;DR +Delivery order and what became of it. The product contract — architecture, the card, the detector, the Cortex call, the renderer/Cortex ownership split — lives in [`initial-design-spec.md`](initial-design-spec.md) and is not repeated here. -- **What:** a shareable OG card that tells the one real story hiding in a developer's commit history. Drops into a README, embeds in a post. -- **Stack:** Snowflake does the work. A public **GCS bucket** holds the card. **Cloud Run** is a thin display shell — it serves the embeddable page and pushes the button. -- **Built with:** `snow` CLI (SQL objects scripted in-repo) + Snowflake MCP (so Claude writes and runs the DDL). - **Prize:** Best Use of Snowflake — the app _is_ Snowflake. - **Due:** Mon Jul 13, 06:59 UTC. - -## Architecture - -``` -Cloud Run (static SPA + API, no analysis) - │ POST /api/generate → guard, claim, enqueue - ▼ -Cloud Tasks - │ OIDC POST /internal/generate - ▼ -Cloud Run worker ────────────────────────────────────────┐ - ▼ - Snowflake - │ ├─ ingest proc external access → api.github.com → COMMITS - │ ├─ detector plain SQL: gaps, streaks, hours → pick ONE thread - │ └─ cortex narrates it + picks the palette (hex + mood) - │ │ - │ ◄── card payload (JSON) ────────────────────────────┘ - ▼ -Cloud Run renders the SVG → writes it to the bucket - ▼ -public GCS bucket ──► README · dev.to embed -``` - -Snowflake gets its own data, finds the story, and reads it. Cloud Run serves the page and -API, queues durable work, calls one proc in the worker request, and turns the payload into -an SVG. **The card in the bucket is the state** — if the file exists, it is ready. The -browser polls `/api/state/{owner}/{repo}`; it never calls the bucket for state. - -**Rendering is Cloud Run's job, not Snowflake's.** Templating an SVG proves nothing about a warehouse, and doing it in-warehouse would put a `STORAGE INTEGRATION`, an external stage, and a service-account IAM grant on the critical path for zero narrative gain. Cloud Run has to exist anyway; it writes to GCS with ordinary credentials. - -## Tooling - -- **Snowflake MCP (self-hosted, `Snowflake-Labs/mcp`)** — Cortex plus object management and SQL orchestration. This is what lets Claude create the network rule, secret, integration, and procedures. -- **Snowflake connector (directory)** — Cortex Agents / Cortex Search. Retrieval only; good for querying, not for DDL. -- **`snow` CLI** — every object lives as SQL in the repo and deploys with one command. Reproducible, reviewable, and it's what the writeup screenshots. - -## Snowflake objects (the whole app) - -| object | job | status | -| --------------------------------------------------- | ---------------------------------------------------------- | ----------- | -| `GITHUB_API_RULE` (EGRESS, `api.github.com`) | let the warehouse out | ✅ deployed | -| `GITHUB_TOKEN` (`SECRET`) | GitHub token | ✅ deployed | -| `GITHUB_API_ACCESS` (`EXTERNAL ACCESS INTEGRATION`) | binds rule + secret | ✅ deployed | -| `COMMITS` | owner, repo, sha, subject, body, authored_at, bot/AI flags | ✅ deployed | -| `PROC INGEST_REPO_COMMITS(owner, repo)` | Python + external access → REST Commits API → `COMMITS` | ✅ deployed | -| `COMMITS_CLEAN` (view) | drops merges + bots, derives date/hour parts | ✅ deployed | -| detector views | gaps, streaks, night share, drama scores → one winner | ✅ scripted | -| `CARD_EVIDENCE` (view) | the winning thread's ~20 commits — all Cortex ever sees | ✅ scripted | -| `PROC READ_REPO(owner, repo)` | ingest if cold → detector → Cortex → card payload JSON | ✅ deployed | +- **Deploy:** `make snowflake-deploy` — every object is SQL in the repo. An object created by clicking in a UI does not exist. + +## Snowflake objects + +| object | status | +| --------------------------------------------------- | ---------- | +| `GITHUB_API_RULE` (EGRESS, `api.github.com`) | ✅ shipped | +| `GITHUB_TOKEN` (`SECRET`) — created out-of-band | ✅ shipped | +| `GITHUB_API_ACCESS` (`EXTERNAL ACCESS INTEGRATION`) | ✅ shipped | +| `COMMITS` · `REPO_INGEST` · `INGEST_STAGE` | ✅ shipped | +| `PROC INGEST_REPO_COMMITS(owner, repo)` | ✅ shipped | +| `COMMITS_CLEAN` · `DETECTOR_CONFIG` (views) | ✅ shipped | +| detector views (15) → `REPO_STORYLINE` | ✅ shipped | +| `COMMIT_LINES` · `CARD_EVIDENCE` (views) | ✅ shipped | +| `CHRONICLE_CARD` (UDF) | ✅ shipped | +| `CARD_PLOT` (view) · `CARDS` | ✅ shipped | +| `PIPELINE_VERSION` · `STALE_CARDS` (views) | ✅ shipped | +| `CARD_PAYLOAD` (view) · `REFRESH_CARD_DATA()` | ✅ shipped | +| `PROC READ_REPO(owner, repo)` | ✅ shipped | No `STORAGE INTEGRATION` and no external stage — Cloud Run owns the bucket. -## Stage 1 — the detector (plain SQL, no LLM, free) - -Score candidate threads, pick the single most dramatic **true** one: - -- **relapse** — `LAG` over commit dates: silent ≥N days, then resumed. _(rai-lint: 107 days dark, back at 3:32am)_ -- **nocturne** — share of commits after 22:00. _(my-hermantic-agent: 85%)_ -- **binge** — longest consecutive-day streak, or the heaviest single night. -- **collapse** — a spike, then permanent silence. _(the 3:53am ending)_ -- **fight** — a revert/hotfix cluster (regex; no AI needed). -- **resurrection** — dead, returned, **and shipped a release**. - -Floors (minimum real commits) keep bots and noise from winning. Deterministic — same handle, same story. - -## Stage 2 — Cortex (one small call) - -Feed **only the winning thread's evidence** — ~10–20 real commit messages plus the computed facts. It returns: - -```json -{ - "kicker": "the death of a side project", - "headline_upright": "Born in daylight. Last touched at", - "headline_accent": "3:53 in the morning", - "headline_trail": ".", - "label_first": "it begins", - "label_pivot": "", - "label_last": "", - "accent": "#e8a04a", - "accent_reason": "amber, for a thing that burned out" -} -``` - -Nine keys, all the writing on the card. The italic run sits inside sentence 2 — `headline_upright` is upright, `headline_accent` is italic and in the accent colour, `headline_trail` is upright again. Empty labels for the anchors this storyline doesn't use (a collapse has no separate pivot; a non-active repo doesn't get a poetic label on the last commit). - -**Cortex picks the palette.** One `accent` hex paints every accent-coloured element on the card — kicker slug, italic headline fragment, last-commit dot, arrow, void-panel rule, attribution bullet. A project that died and one that shipped must not wear the same colour. - -**Renderer owns the facts.** Kicker slug prefix, header meta (`59 COMMITS · QUIET SINCE FEB 25`), first/last-commit anchor prefixes, void-panel text, caption, author handle — all composed from `FACTS`, `STATUS`, `PIVOT_AT`, and the `PLOT` array. See the Ownership section in `docs/initial-design-spec.md` for the full split. - -## Stage 3 — the card - -Cloud Run templates an SVG (1200×630) from the Cortex JSON plus the plotted commits, and writes it to the public GCS bucket. - -**Design:** - -- Didone headline in three slots — upright, italic accent fragment, upright · mono kicker naming the genre · status label (`abandoned` / `shipped 1.0.1`). -- **The arc is the card:** beeswarm scatter, date across and hour down, night at the bottom. Daylight commits hollow, night commits solid. The dead stretch is a **void panel** you look through. The final commit is one accent dot. -- Poetic tails pin to the anchors the storyline uses. The headline **interprets** — that's the point. - -## Cost - -Detection is free SQL. Cortex sees ~20 commits, not thousands. Cloud Run scales to zero. Storage is a bucket. Ballpark: **lunch money, once.** - -## Timeline +## Timeline (as delivered) -- **Fri night — THE GATE.** MCP + `snow` CLI connected. Prove two things: external access can reach `api.github.com` from inside a proc, and Cortex AISQL runs in your region. Nothing else matters until both are green. -- **Sat AM** — `ingest_commits` → `COMMITS_RAW` for one handle. -- **Sat PM** — detector SQL. Confirm it picks `my-hermantic-agent` (nocturne/collapse) and `rai-lint` (resurrection) unaided. -- **Sun AM** — Cortex call (kicker, headline, anchor labels, accent) via `READ_REPO`; Cloud Run renders the SVG → GCS. -- **Sun PM** — Cloud Run shell + `/api/generate`; confirm the card renders in a real README and the page embeds in a dev.to draft. -- **Sun night** — writeup: the detector SQL, the Cortex call, the `snow` deploy. Cards are the demo. +- **Fri night — THE GATE.** Two things proven before anything else was allowed to matter: external access can reach `api.github.com` from inside a proc, and Cortex AISQL runs in the region. +- **Sat AM** — ingest → `COMMITS`. +- **Sat PM** — detector SQL. It picked the intended storylines unaided. +- **Sun AM** — Cortex call via `READ_REPO`; Cloud Run renders the SVG → GCS. +- **Sun PM** — Cloud Run shell + `/api/generate`; card confirmed rendering in a real README. +- **Sun night** — writeup. - **Mon early** — submit with buffer. ## Open items -- **Font rendering in the SVG** — GitHub proxies README images through camo, so webfonts won't load. Fix is a base64-embedded subset. Deferred by decision. -- **Anonymous serving** — Snowflake will not answer an anonymous HTTP request (SPCS "public" endpoints are RBAC-gated; a browser gets a login page). The public GCS bucket is the answer. -- **Async generation** — Cloud Tasks owns the durable worker request. The page polls - `/api/state`, so closing the tab does not affect the job. +- ~~**Font rendering**~~ **Closed.** The card is rasterized to PNG with the fonts vendored, so nothing depends on a proxy fetching a webfont. dev.to forced this — its image proxy will not serve an SVG. +- **Stale cards** — `STALE_CARDS` reports which cards a dead pipeline version wrote; it does not act, because acting costs a Cortex call each. `make cards-rerender` redraws stored cards for free — only a prompt change needs real spend. -## Cut first if behind +## Cut -1. Extra story types (ship `relapse` + `nocturne` only). -2. The gallery page. +1. **The gallery page.** Three example chips on the landing page cover the same need. +2. Story types beyond the six — none were needed. -**Never cut:** external-access ingest · the detector SQL · the Cortex call including the palette pick · the SVG card · the writeup showing the SQL. +**Never cut, and wasn't:** external-access ingest · the detector SQL · the Cortex call including the palette pick · the SVG card. diff --git a/docs/initial-design-spec.md b/docs/initial-design-spec.md index d440319..860dc5d 100644 --- a/docs/initial-design-spec.md +++ b/docs/initial-design-spec.md @@ -2,6 +2,8 @@ _DEV Weekend Challenge: Passion Edition. Prize target: **Best use of Snowflake**. Due Mon Jul 13, 6:59 AM UTC._ +This document describes the shipped system. Where the original design was cut or changed, it says so rather than describing a product that does not exist. + ## Thesis A contribution graph tells you that work happened. It never tells you what happened. Buried in a repo's commit history is usually exactly one story worth telling — a project that went dark for 107 days and came back at 3:32am, a repo built entirely after midnight whose last commit landed at 3:53 and never got another, a week where every commit was a revert. @@ -16,23 +18,25 @@ Scope is **one repository**, not a whole profile. A profile year-in-review turns - A repo entry flow for public GitHub repos (`owner/repo` or a URL). - A durable generation state: `generating`, `ready`, `failed`. -- A generated SVG card sized for README and social previews. +- A generated PNG card sized for README and social previews. **PNG, not SVG**: dev.to proxies remote images and will not serve an SVG, so an SVG-only card does not exist on the platform this was built for. The renderer still composes SVG; it is rasterized before it leaves the service. - A copyable Markdown embed. - A cached public page at `/{owner}/{repo}` and a card served directly from the public GCS bucket. -- A gallery of pre-generated cards for judge-safe demo coverage. +- Three pre-generated example repos on the landing page, for judge-safe demo coverage. Two carry a storyline; `forem/forem` deliberately carries none — the honest-negative case is part of the demo, not an omission from it. + +**Cut:** the gallery route. Three example chips on the landing page cover the same need — a judge who wants a card without waiting for one — at a fraction of the surface area. ## The card 1200×630 SVG, readable as a standalone artifact. - **Product mark** — `Commit Chronicles`. -- **Kicker** naming the genre — `the death of a side project`, `the one that came back`. +- **Kicker** naming the genre — `the death of a side project`, `the one that came back` — prefixed with the repo slug. - **Headline** — Didone serif, in three slots: upright, then an italic accent-colored fragment, then upright again. The italic run may start mid-sentence. - **The arc** — a beeswarm scatter: date across, hour of day down, rotated so night sits at the bottom of the frame. Daylight commits render hollow, night commits solid. Long quiet stretches render as a **void panel** you look straight through. The last commit is a single accent dot. - **Anchor labels** — a poetic tail on the first commit, the pivot, and the last commit. Only the anchors this storyline actually uses get one. -- **Status** — an observed label: `abandoned` (no commits in N days), `shipped 1.0.1`, `active`. +- **Status** — an observed label: `abandoned`, `dormant`, `active`. - **Counts** — commits, span. -- **Attribution** — `read by Snowflake Cortex`. +- **Attribution** — `Read by Snowflake Cortex`. If the card would work equally well as a bar chart, it has failed. @@ -48,123 +52,249 @@ If the card would work equally well as a bar chart, it has failed. ## User flow -1. User enters a repo. -2. Client normalizes the slug; Cloud Run checks the bucket for an existing card. -3. `ready` (the card object exists) → render the cached card immediately. -4. Missing or stale → client calls `POST /api/generate` **once**. -5. Cloud Run marks the job `generating`, then invokes the Snowflake pipeline. -6. The user can wait or leave. -7. Snowflake returns the card payload; Cloud Run renders the SVG and writes it to the - bucket (or records `failed`). -8. Returning to `/{owner}/{repo}` finds the existing card and renders it. -9. User copies the card image or the README embed. +Read state before spending anything. The expensive path is entered only on a genuine miss. + +```mermaid +flowchart TD + accTitle: User flow + accDescr: The user enters a repo and the client validates the slug locally. It then reads job state before doing anything else. A ready card renders straight from the bucket. A terminally failed repo stops without retrying. Only a genuine miss posts to generate, which claims quota and enqueues durable work. The user may leave; polling only drives the display, never the job. + + ENTER["User enters owner/repo"] + SLUG{"valid slug?"} + REJECT["rejected in the browser
no request made"] + STATE["GET /api/state"] + + READY{"card in the bucket?"} + RENDER["render the cached card
Snowflake is never called"] + COPY["copy the image URL
or the README embed"] + + DEAD{"failed, and terminal?"} + STOP["show the failure
no retry offered"] + + GEN["POST /api/generate
once"] + QUOTA{"daily quota left?"} + CAP["429, budget spent for today"] + CLAIM["claim generating, enqueue"] + + LEAVE["user may close the tab"] + WORK["Cloud Tasks, Snowflake, render, write"] + POLL["poll every 2.5s"] + + ENTER --> SLUG + SLUG -->|"no"| REJECT + SLUG -->|"yes"| STATE --> READY + READY -->|"yes"| RENDER --> COPY + READY -->|"no"| DEAD + DEAD -->|"yes"| STOP + DEAD -->|"no"| GEN --> QUOTA + QUOTA -->|"no"| CAP + QUOTA -->|"yes"| CLAIM + CLAIM --> WORK + CLAIM --> POLL + POLL -->|"still generating"| POLL + POLL -->|"settled"| READY + CLAIM -.-> LEAVE + LEAVE -.->|"return later to the same URL"| STATE + WORK -->|"writes the card, or a failed marker"| READY +``` + +**Polling drives the display, not the job.** The client gives up _showing_ progress after five minutes; the work is unaffected. The browser tab must not be required for generation to complete — if it is, the app is a loading spinner wearing a trench coat. + +### Job state + +State is not a column in a database. **The card's existence in the bucket _is_ the ready state.** -The browser tab must not be required for generation to complete. If it is, the app is a loading spinner wearing a trench coat. +```mermaid +stateDiagram-v2 + accTitle: Job state + accDescr: A repo starts unknown. Claiming a quota slot and a generating marker moves it to generating. Success writes the card and makes it ready. Failure writes a failed marker, which is terminal for bad-repo errors and retryable for pipeline and Cortex errors. A generating marker older than its TTL is presumed dead and re-admitted. + + [*] --> unknown: no objects in the bucket + unknown --> generating: quota claimed, create-only marker written + generating --> ready: card.png then card.json written, state cleared + generating --> failed: state.json records the error code + generating --> generating: marker past its TTL, presumed dead, re-admitted + failed --> generating: retryable code only + failed --> [*]: terminal code, cached so it cannot be retried into a bill + ready --> [*]: served from the bucket forever +``` + +`readState` checks for `card.json` first, so a card always outranks a stale `state.json` beside it. The accent hex rides on the object's custom metadata, so polling never downloads the payload. + +**Terminal error codes** (cached, no retry offered): `repo_not_found`, `repo_private`, `repo_empty`, `invalid_repo_slug`, `no_commits`. +**Retryable:** `cortex_empty`, `cortex_rejected`, `pipeline_error`. ## Product surface (ready state) -- Repo address, normalized as `commitchronicles.dev/{owner}/{repo}`. -- Large card preview. -- Primary action: **Copy card image**. +- Repo address, normalized as `commitchronicles.anchildress1.dev/{owner}/{repo}`. +- Large card preview, loaded straight from the bucket. +- Primary action: **Copy image URL**. - Secondary action: **Copy README embed**. ```md - [![Commit Chronicles](https://storage.googleapis.com/commit-chronicles-cards/cards/{owner}/{repo}/card.svg)](https://commitchronicles.dev/{owner}/{repo}) + [![Commit Chronicles](https://storage.googleapis.com/commit-chronicles-cards/cards/{owner}/{repo}/card.png)](https://commitchronicles.anchildress1.dev/{owner}/{repo}) ``` - Link to read another repo. +The whole page takes its accent colour from the card's — Cortex picks the palette for the site, too. + ## Data source The GitHub REST **Commits API** — `/repos/{owner}/{repo}/commits`. Public repos only. **Do not use the Activity Events API.** GitHub stripped commit summaries and counts from `PushEvent` payloads on 7 Oct 2025, which also guts every GH-Archive mirror of it. Commit text now survives only in the main REST API. -Ingest: commit message, SHA, authored timestamp, author login when public. Cap commits per repo. Filter obvious bot noise before anything expensive runs. +Ingest: commit message, SHA, authored timestamp, author login when public. **Cap: 500 commits** per repo by default, hard-capped at 2000. Bot and AI-assisted commits are classified at ingest and filtered before anything expensive runs. + +A history longer than the cap sets `windowed`, and the card admits it — the header reads `last 500 commits · quiet since Feb 25` rather than presenting a slice as the repo's whole life. The Snowflake Marketplace archive is not used in V1. This product is about one submitted repo and its Cortex reading, not whole-population percentile math. ## Architecture -```text -Cloud Run (static SPA + /api/generate — no analysis logic) - │ POST /api/generate → guards, quota, mark `generating`, enqueue - ▼ -Cloud Tasks (commit-chronicles-gen) - │ POST /internal/generate (OIDC-signed) - ▼ -Cloud Run worker → CALL READ_REPO(owner, repo) - ▼ -Snowflake - ├─ ingest proc external access → api.github.com → COMMITS - ├─ detector plain SQL → score storylines → pick ONE - └─ cortex narrate that thread + choose the palette - │ - │ ◄── card payload (JSON) - ▼ -Cloud Run templates the SVG → writes gs://…/{owner}/{repo}/card.svg - ▼ -Public GCS bucket (serving cards and storing state; the card's existence is ready) +```mermaid +flowchart TD + accTitle: Commit Chronicles architecture + accDescr: Cloud Run guards the generate request, claims quota, and enqueues to Cloud Tasks. Cloud Tasks calls back into Cloud Run with an OIDC token. That worker calls one Snowflake procedure, which ingests from GitHub through an external access integration, detects a storyline in SQL, and narrates it with Cortex. Cloud Run renders the returned payload to SVG and writes it to a public GCS bucket, which serves the card directly to READMEs. + + SPA["SPA
no analysis logic"] + API["Cloud Run
POST /api/generate
guards, quota, claim, enqueue"] + Q["Cloud Tasks
max 2 concurrent dispatches"] + W["Cloud Run worker
POST /internal/generate
OIDC-verified"] + + subgraph SF["Snowflake — CALL READ_REPO"] + direction TB + ING["ingest
external access integration"] + DET["detector
plain SQL, pick ONE"] + CTX["Cortex
narrate it, choose the palette"] + ING --> DET --> CTX + end + + GH[("api.github.com")] + REN["Cloud Run
templates the SVG"] + GCS[("Public GCS bucket
the card's existence
is the ready state")] + + SPA --> API --> Q --> W --> SF + ING <-.-> GH + SF -->|"card payload"| REN + REN -->|"only writer"| GCS + GCS -->|"GET /api/state"| SPA ``` -**Generation goes through a queue, and that is a cost decision.** The work has to outlive -the browser tab. Detaching it from the request that started it would need Cloud Run's -`--no-cpu-throttling`, which bills instance time instead of request time — you pay for the -container to sit there doing nothing. Cloud Tasks calls back into the service, so the -pipeline runs _inside_ a request: CPU is billed only while it is working, the service still -scales to zero, and closing the tab has no effect on a job that is no longer attached to -the tab's connection. The queue's `max-concurrent-dispatches` is the real ceiling on the -Cortex spend rate — at most two repos can be in the warehouse at once, whatever the front -page is doing. +Three decisions worth defending: -`/internal/generate` is on a public service, so it verifies the task's OIDC token against -the invoker service account before it spends anything. No token, no work. +**The queue is a cost decision.** The work must outlive the browser tab. Detaching it from its originating request would need `--no-cpu-throttling`, which bills instance time instead of request time — you pay for the container to sit idle. Cloud Tasks calls back into the service, so the pipeline runs _inside_ a request: CPU is billed only while it works, and the service still scales to zero. `max-concurrent-dispatches=2` is the real ceiling on Cortex spend — at most two repos in the warehouse at once, whatever the front page is doing. -- **Snowflake** reaches GitHub itself via an external access integration, finds the story in SQL, and narrates it with Cortex. The ingest layer is a stored procedure, not a service. -- **Cloud Run** owns the routes, calls one Snowflake proc, and turns the returned payload into an SVG. It fetches no commit data and computes no analysis. -- **The GCS bucket** is the cache of record and serves cards directly. Cloud Run serves the - SPA and `/api/state`; a cached render never calls Snowflake or GitHub. +**Rendering lives in Cloud Run, not Snowflake.** Templating an SVG proves nothing about a warehouse, and doing it in-warehouse would drag a `STORAGE INTEGRATION`, an external stage, and a Snowflake-minted IAM grant onto the critical path to buy nothing. The Snowflake case rests on the ingest, the detector, and the Cortex call. -**Rendering lives in Cloud Run, not Snowflake.** Templating an SVG string is a chore, not a demonstration of a data warehouse — and doing it in-warehouse would drag a `STORAGE INTEGRATION`, an external stage, and a Snowflake-minted service-account IAM grant onto the critical path to buy nothing. The Snowflake case rests on the ingest, the detector, and the Cortex call. Cloud Run writes to the bucket with ordinary GCP credentials. +**The bucket serves the card because Snowflake can't.** SPCS "public" endpoints are RBAC-gated and hand an anonymous browser a login page. -Snowflake cannot serve an anonymous HTTP request — SPCS "public" endpoints are RBAC-gated and hand a browser a login page. The public GCS bucket serves the card; Cloud Run serves the page and API. +`/internal/generate` is on a public service and spends Cortex credits, so it verifies the task's OIDC token against the invoker service account first. No token, no work. -## Snowflake objects +### Routes + +| Route | Guard | Job | +| ------------------------------- | ----- | -------------------------------------------------------------------- | +| `GET /healthz` | — | Liveness | +| `POST /api/generate` | — | Claim and enqueue. `202` accepted, `200` attach, `429` quota spent | +| `POST /internal/generate` | OIDC | The worker. Calls Snowflake, renders, writes. **Spends the credit.** | +| `POST /internal/rerender` | OIDC | Redraws one stored card with the current renderer. No Cortex spend. | +| `GET /api/state/{owner}/{repo}` | — | Job state. `no-store`. The only thing the client polls. | +| `GET /` · `GET /{owner}/{repo}` | — | The SPA shell | + +There is deliberately **no card route**. The bucket is public and serves the SVG itself. + +### Bucket layout -| object | job | -| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | -| `GITHUB_API_RULE` (EGRESS, `api.github.com`) | let the warehouse out | -| `GITHUB_TOKEN` (`SECRET`) | GitHub token | -| `GITHUB_API_ACCESS` (`EXTERNAL ACCESS INTEGRATION`) | binds rule + secret | -| `COMMITS` | owner, repo, sha, subject, body, authored_at, bot/AI flags | -| `COMMITS_CLEAN` (view) | drops merges and bots; derives date + hour parts | -| `PROC INGEST_REPO_COMMITS(owner, repo)` | Python + external access → Commits API → `COMMITS` | -| detector views | gaps, streaks, night share, storyline scores → `REPO_STORYLINE` | -| `CARD_EVIDENCE` (view) | the winning thread's ~20 commits — the only thing Cortex sees | -| `CHRONICLE_CARD` (UDF) | hand-written wrapper around `AI_COMPLETE`; one structured call → the whole card | -| `CARDS` | the generated card payloads, plus the Cortex query id for cost audit | -| `PROC READ_REPO(owner, repo)` | the one entry point: ingests on a cold repo, then detector → `CHRONICLE_CARD` → structured card payload | +| Object | Cache-Control | Job | +| --------------------------------- | ---------------------- | ---------------------------------------------------- | +| `cards/{owner}/{repo}/card.png` | `public, max-age=3600` | The product | +| `cards/{owner}/{repo}/card.json` | `public, max-age=300` | The payload. **Its existence is the ready state.** | +| `cards/{owner}/{repo}/state.json` | `no-store` | `generating` or `failed`. Deleted on success. | +| `meta/quota/{YYYY-MM-DD}.json` | `no-store` | The daily counter, so the cap holds across instances | + +Writes are ordered `card.png` → `card.json` → clear `state.json`. A crash between the first two leaves the repo retryable rather than ready-with-no-card. + +## Snowflake objects Every object is SQL in the repo, deployed with the `snow` CLI. An object created by clicking in a UI does not exist. +| object | job | +| --------------------------------------------------- | ------------------------------------------------------------------------------------------ | +| `CHRONICLES_WH` | XSMALL, auto-suspend 60s, 300s statement timeout | +| `GITHUB_API_RULE` (EGRESS, `api.github.com`) | let the warehouse out | +| `GITHUB_TOKEN` (`SECRET`) | GitHub token. Created out-of-band — see `docs/snowflake-setup.md`, never in a tracked file | +| `GITHUB_API_ACCESS` (`EXTERNAL ACCESS INTEGRATION`) | binds rule + secret | +| `INGEST_REPO_COMMITS(owner, repo)` | Python proc + external access → Commits API → `COMMITS` | +| `INGEST_STAGE` | transient staging table; an owner's-rights proc cannot create a temp table | +| `COMMITS` | owner, repo, sha, subject, body, authored_at, bot/AI flags | +| `REPO_INGEST` | per-repo ingest metadata: `WINDOWED`, `COMMIT_CAP`, `INGESTED_AT` | +| `COMMITS_CLEAN` (view) | drops merges and bots; derives date + hour parts | +| `DETECTOR_CONFIG` (view) | every threshold the detector uses, in one place | +| detector views | facts, gaps, six `STORY_*` views, scores → `REPO_STORYLINE` | +| `COMMIT_LINES` (view) | explodes squash-merge bodies into line items; drops trailers and stub lines | +| `CARD_EVIDENCE` (view) | the winning thread's lines — the only thing Cortex sees | +| `CHRONICLE_CARD` (UDF) | hand-written wrapper around `AI_COMPLETE`; one structured call → the whole card | +| `CARD_PLOT` (view) | the scatter array. Never sent to a model | +| `CARDS` | the generated card payloads, plus the Cortex query id for cost audit | +| `PIPELINE_VERSION` (view) | MD5 of the deployed prompt + evidence DDL. Moves on its own when either changes | +| `STALE_CARDS` (view) | cards written by a pipeline version that no longer exists. **Reports; does not act** | +| `CARD_PAYLOAD` (view) | an existing card, in the shape `READ_REPO` returns. The zero-cost read path | +| `REFRESH_CARD_DATA()` (proc) | re-derives `FACTS` and `PLOT` without re-billing Cortex | +| `READ_REPO(owner, repo)` (proc) | the one entry point: ingest if cold → detector → `CHRONICLE_CARD` → verify → payload | + ## The detector (plain SQL, no LLM) This is the core of the product and the core of the Snowflake case. Score every candidate storyline deterministically; keep the highest. Surveying a repo's whole history produces a report. Picking one story produces an argument. -| storyline | signal | -| ---------------- | ---------------------------------------------------------------------- | -| **relapse** | `LAG` over commit dates — quiet ≥ N days, then resumed | -| **nocturne** | share of commits after 22:00 | -| **binge** | longest consecutive-day streak, or the heaviest single night | -| **collapse** | a spike, then permanent silence | -| **fight** | a cluster of reverts/hotfixes in a short window (regex — no AI needed) | -| **resurrection** | quiet, resumed, and shipped a release | +Every storyline gates on `MIN_COMMITS = 15`, so bot noise cannot win. + +| storyline | signal | gate | +| ---------------- | ------------------------------------------------------ | -------------------------------------------------------------- | +| **relapse** | `LAG` over commit dates — quiet, then resumed | gap ≥ 30 days | +| **nocturne** | share of commits in the night window (22:00–04:59 UTC) | ≥ 50% of commits | +| **binge** | longest consecutive-active-day streak | streak ≥ 7 days | +| **collapse** | a spike, then permanent silence | ≥ 90 days silent **and** ≥ 15% of commits in the final 30 days | +| **fight** | revert/hotfix cluster (regex on the subject — no AI) | ≥ 4 such commits in a 7-day window | +| **resurrection** | a relapse that also shipped a release after the pivot | inherits relapse's gate, `+15` score | +| **none** | nothing cleared its floor | the fallback | + +Thresholds live in one view, so tuning the detector is a one-line diff: + +```sql +CREATE OR REPLACE VIEW DETECTOR_CONFIG AS SELECT + 15 AS MIN_COMMITS, 30 AS RELAPSE_MIN_GAP_DAYS, + 90 AS ABANDONED_AFTER_DAYS, 7 AS BINGE_MIN_STREAK_DAYS, + 4 AS FIGHT_MIN_COMMITS, 22 AS NIGHT_START_HOUR, + 5 AS NIGHT_END_HOUR, 25 AS EVIDENCE_SHARE_PCT, + 20 AS EVIDENCE_MIN_LINES, 140 AS EVIDENCE_MAX_LINES; +``` + +The winner is one window function — highest score, ties broken by how dramatic the storyline is (`resurrection` > `collapse` > `relapse` > `nocturne` > `fight` > `binge`): + +```sql +QUALIFY ROW_NUMBER() OVER ( + PARTITION BY f.REPO_OWNER, f.REPO_NAME + ORDER BY s.SCORE DESC NULLS LAST, s.DRAMA_RANK +) = 1 +``` + +Scoring is deterministic: the same repo always yields the same story. -Apply floors — a storyline needs a minimum number of real commits so bot noise can't win. Scoring is deterministic: the same repo always yields the same story. +**`none` never reaches Cortex.** It writes a hard-coded grey template card (`#6b7280`, `MODEL = 'none'`) and costs nothing. A history with nothing to say gets told so. Cheap, explainable, and it means Cortex is only ever pointed at the part that matters. ## Cortex -One call, fed **only the winning storyline's evidence**: ~10–20 real commit messages plus the computed facts. Never the whole history — that's how you buy an expensive, unfocused paragraph. +One call, fed **only the winning storyline's evidence**. Never the whole history — that's how you buy an expensive, unfocused paragraph. + +**The evidence budget is measured in lines, not commits**: 25% of the winning thread's commit lines, floored at 20 and capped at 140, plus the first 5 and last 8 lines regardless. `COMMIT_LINES` explodes squash-merge bodies into separate items first, so work buried inside a merge is still visible; trailers and stub lines are dropped. + +Model: `claude-sonnet-4-5` via `AI_COMPLETE`, `temperature 0.4`, `max_tokens 2048`. + +**Temperature stays at 0.4.** Determinism is the detector's job. At 0 every repo gets the same phrasing. + +The response schema constrains exactly nine keys: ```json { @@ -180,88 +310,111 @@ One call, fed **only the winning storyline's evidence**: ~10–20 real commit me } ``` -Nine keys. That is the whole surface area of the writing on the card. +Nine keys. That is the whole surface area of the writing on the card — **not** the card payload, which is a 22-key camelCase object wrapping these nine alongside the facts, the plot, the storyline, the query id, and the pipeline version. **The italic run is a fragment, not a clause.** The renderer sets `headline_upright` upright, `headline_accent` italic and in the accent colour, `headline_trail` upright again. The italic can start mid-sentence — that's the design. Punctuation may live in `accent` or `trail`; put it where the typography looks right. **Cortex chooses the palette.** One `accent` hex paints every accent-coloured element on the card: kicker slug, italic headline fragment, last-commit dot, arrow, void-panel rule, attribution bullet. It is a reading of the arc, not a brand constant: a repo that went quiet and a repo that came back and shipped must not wear the same colour. -The palette is **muted neons** — bright but not fluorescent, saturated but not raw. Anchor hues on the spectrum: - -| hex | family | reads as | -| --------- | ----------- | ------------------------ | -| `#e8a04a` | muted amber | burn, heat, ember | -| `#e56b5a` | muted coral | conflict, alarm, warning | -| `#d3e85a` | muted lime | return, growth, life | -| `#7fe4c5` | muted mint | cool, dawn, calm | -| `#6ab5f5` | muted sky | night, distance, quiet | +The palette is five muted neons. Cortex picks one, and the choice is a reading of the arc: -Cortex may drift slightly off these anchors — the constraint is the family, not the exact swatch. No greys, no browns, no fluorescents, no deep saturated primaries. `none`-storyline cards use grey (`#6b7280`) deliberately; that colour is reserved for "no story to tell". +| hex | family | reads as | +| --------- | ------ | ------------------------ | +| `#e8a04a` | amber | burn, heat, ember | +| `#e56b5a` | coral | conflict, alarm, warning | +| `#d3e85a` | lime | return, growth, life | +| `#7fe4c5` | mint | cool, dawn, calm | +| `#6ab5f5` | sky | night, distance, quiet | -**Not every storyline uses every anchor.** The detector picks the storyline first and only the anchors that storyline actually uses get labelled. `label_pivot` is empty for a collapse (no separate pivot from the last commit). `label_last` is empty unless the repo is still active — for abandoned or dormant repos the renderer prints `"last commit ·