From c6725f9eff39b0dca378d5f1ca5f4479d8e41290 Mon Sep 17 00:00:00 2001 From: Anthony Ettinger Date: Sun, 13 Sep 2026 04:32:56 +0000 Subject: [PATCH 1/2] gh-pulse: what moved on GitHub since yesterday, ranked, with traffic (0.33.0) Every repo the gh token can see (own account plus every org, forks excluded) is checked for movement since the previous run: stars, forks, commits, PRs, issues, releases, and the /graphs/traffic numbers (views, unique visitors, clones, referrers, popular content). Movers are ranked by a weighted score and reach every surface: the email (Resend, HTML with inline PNG charts rendered from SVG by @resvg/resvg-js), the terminal (`gh-pulse show`, an hqtui dashboard: click a row, the selected repo's 14-day views and clones, a History tab over every snapshot), the browser (`open`) and pipes (`text`, `json`). Pit alias `/pulse`. GitHub publishes traffic in UTC-day buckets one to two days late, so the window is the growth of each bucket since the previous snapshot rather than a clock: nothing is missed or double counted when a run is skipped. Every run writes a gzipped snapshot under ~/.local/share/gh-pulse; GitHub keeps nothing past 14 days, so those files are the long-run history. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_011gfb1vcQH5rf82HfbJvf1Z --- README.md | 50 ++ bin/gh-pulse.ts | 136 ++++++ package.json | 4 +- pnpm-lock.yaml | 144 ++++++ src/gh-pulse-tui.ts | 236 ++++++++++ src/gh-pulse.ts | 1031 +++++++++++++++++++++++++++++++++++++++++ src/registry.ts | 4 + test/gh-pulse.test.ts | 198 ++++++++ 8 files changed, 1802 insertions(+), 1 deletion(-) create mode 100755 bin/gh-pulse.ts create mode 100644 src/gh-pulse-tui.ts create mode 100644 src/gh-pulse.ts create mode 100644 test/gh-pulse.test.ts diff --git a/README.md b/README.md index 2ff5dd0..c45248f 100644 --- a/README.md +++ b/README.md @@ -465,6 +465,56 @@ What it will not do is as much of the design: - **A check that ran and failed is a result, not an obstacle.** Retrying until it passes is how a flaky suite becomes a green one that means nothing. +### `gh-pulse` + +What moved on GitHub since yesterday, ranked, with traffic. Every repo the +`gh` token can see (yours plus every org you belong to, forks excluded) is +checked for movement since the previous run: stars, forks, commits, pull +requests, issues, releases, and the traffic GitHub shows at `/graphs/traffic` +(views, unique visitors, clones, referrers, popular content). Repos with +movement are ranked by a weighted score; each comes with its traffic, the top +ones with a 14-day chart, and new or lost followers are named. + +```sh +gh-pulse # scan every repo, email the report, snapshot +gh-pulse --dry-run # scan and write the report, send nothing +gh-pulse show # the last report and the history, as a TUI +gh-pulse open # the last HTML report, in the browser +gh-pulse text # the last report as plain text +gh-pulse json # the last report as JSON +gh-pulse --repo profullstack/nixamp --dry-run # one repo, for a look +``` + +The same report reaches every surface: the email (HTML with inline charts, +through Resend), the terminal (`show`, built on hqtui: click a row, the +selected repo's 14-day views and clones, a History tab over every snapshot), +the browser (`open`), and pipes (`text`, `json`). The moshcode pit alias is +`/pulse`. + +GitHub publishes traffic in UTC-day buckets one to two days late, so "the last +24 hours" cannot be read off the clock. Each run instead counts the growth of +the 14-day buckets since the previous snapshot: exactly "since the last +email", and a skipped day widens the window rather than losing data. Every +run writes `~/.local/share/gh-pulse/snapshots/.json.gz` (`GH_PULSE_DATA` +moves it) with the raw per-repo counts and buckets. GitHub keeps nothing past +14 days, so those files are the long-run history, and what `show` reads. + +Scoring: star 5, fork 4, release 5, PR merged 3, PR or issue opened 2, commit +1 (capped at 25), unique visitor 1, unique cloner 1, clone 0.5, view 0.2. A +single clone or visitor in a day is treated as background noise; two unique +visitors or two unique cloners make a repo a mover on traffic alone. + +Mail: `RESEND_API_KEY` from the environment, else from +`~/.config/logicsrc/shell.env` (the cron case). The sender must sit on a +verified Resend domain (`GH_PULSE_FROM`, default `pulse@profullstack.com`); +`GH_PULSE_TO` or `--to` names the recipient, falling back to the committer +email. A crash sends a plain FAILED mail so a broken cron is never silent. A +daily entry: + +``` +5 13 * * * $HOME/.local/bin/gh-pulse >>$HOME/.local/share/gh-pulse/cron.log 2>&1 +``` + ### `gh-prs-fix-all` Looks at every open threatcrush-scan pull request and fixes the ones broken diff --git a/bin/gh-pulse.ts b/bin/gh-pulse.ts new file mode 100755 index 0000000..3ed5bb3 --- /dev/null +++ b/bin/gh-pulse.ts @@ -0,0 +1,136 @@ +#!/usr/bin/env node +/** + * gh-pulse — what moved on GitHub since yesterday, ranked, with traffic. + * + * gh-pulse scan every repo, email the report, snapshot + * gh-pulse --dry-run scan and write the report, send nothing + * gh-pulse show the last report and the history, as a TUI + * gh-pulse open the last HTML report, in the browser + * gh-pulse text the last report as plain text + * gh-pulse json the last report as JSON (what `show` reads) + * + * Options for a scan: + * --to a@b.c recipient (default GH_PULSE_TO, then the git email) + * --from "N " sender on a verified Resend domain (default GH_PULSE_FROM) + * --top N how many ranked repos get a chart and detail (12) + * --repo owner/name only this repo; repeatable; never moves the baseline + * --hours N window when there is no previous snapshot (24) + * + * Every repo the gh token can see (yours plus every org, forks excluded) is + * checked for movement since the previous run: stars, forks, commits, PRs, + * issues, releases, and the traffic GitHub shows at /graphs/traffic. Movers are + * ranked by a weighted score; each comes with its traffic, the top ones with a + * 14-day chart, and new or lost followers are named. + * + * GitHub publishes traffic in UTC-day buckets one to two days late, so the + * window is the growth of the buckets since the previous snapshot rather than + * a clock. Snapshots live under ~/.local/share/gh-pulse (GH_PULSE_DATA) and are + * the only long-run traffic record GitHub leaves you. + * + * Mail goes through Resend: RESEND_API_KEY from the environment, else from + * ~/.config/logicsrc/shell.env (the cron case). A crash sends a FAILED mail. + */ + +import { readFileSync } from 'node:fs'; + +import { UsageError, parseArgs } from '../src/args.ts'; +import { + DEFAULT_FROM, + dataDir, + defaultDeps, + gitEmail, + loadShellEnv, + openInBrowser, + outputPaths, + run, + sendFailure, +} from '../src/gh-pulse.ts'; +import { isMain } from '../src/is-main.ts'; + +export const USAGE = `Usage: + gh-pulse [--dry-run] [--to ADDR] [--from ADDR] [--top N] [--hours N] [--repo OWNER/NAME]... + gh-pulse show + gh-pulse open + gh-pulse text + gh-pulse json + gh-pulse --help`; + +export async function main(argv: readonly string[]): Promise { + const parsed = parseArgs(argv, { + boolean: ['--dry-run', '--help', '-h'], + string: ['--to', '--from', '--top', '--hours', '--repo'], + }); + if (parsed.flags.has('--help') || parsed.flags.has('-h')) { + process.stdout.write(`${USAGE}\n`); + return 0; + } + const dir = dataDir(); + const [verb, ...rest] = parsed.positional; + if (rest.length > 0) throw new UsageError(`unexpected argument: ${rest[0]}`); + + if (verb === 'show') { + const { showTui } = await import('../src/gh-pulse-tui.ts'); + return showTui(dir); + } + if (verb === 'open') { + const file = outputPaths(dir).html; + if (openInBrowser(file)) return 0; + process.stderr.write(`gh-pulse open: no opener found; the report is at ${file}\n`); + return 1; + } + if (verb === 'text' || verb === 'json') { + const file = verb === 'text' ? outputPaths(dir).text : outputPaths(dir).json; + try { + process.stdout.write(readFileSync(file, 'utf8')); + return 0; + } catch { + process.stderr.write(`gh-pulse ${verb}: no report yet in ${dir}. Run \`gh-pulse\` first.\n`); + return 1; + } + } + if (verb !== undefined) throw new UsageError(`unknown command: ${verb}`); + + const top = Number(parsed.values.get('--top') ?? 12); + const hours = Number(parsed.values.get('--hours') ?? 24); + if (!Number.isFinite(top) || top < 0) throw new UsageError('--top wants a non-negative number'); + if (!Number.isFinite(hours) || hours <= 0) throw new UsageError('--hours wants a positive number'); + + loadShellEnv(); + const to = parsed.values.get('--to') ?? process.env['GH_PULSE_TO'] ?? gitEmail(); + if (!to) throw new UsageError('no recipient: pass --to, set GH_PULSE_TO, or configure git user.email'); + const from = parsed.values.get('--from') ?? process.env['GH_PULSE_FROM'] ?? DEFAULT_FROM; + const dryRun = parsed.flags.has('--dry-run'); + // parseArgs keeps the last value of a repeated flag; --repo is the one flag + // people repeat, so it is gathered from argv directly. + const repos: string[] = []; + for (let i = 0; i < argv.length; i += 1) { + const a = argv[i]!; + if (a === '--repo' && argv[i + 1]) repos.push(argv[i + 1]!); + else if (a.startsWith('--repo=')) repos.push(a.slice('--repo='.length)); + } + + const deps = defaultDeps(); + try { + await run({ dryRun, to, from, top, hours, repos, dataDir: dir }, deps); + return 0; + } catch (error) { + process.stderr.write(`gh-pulse: ${(error as Error).stack ?? String(error)}\n`); + if (!dryRun) await sendFailure(error, to, from, deps.resendKey(), deps.fetch); + return 1; + } +} + +if (isMain(import.meta.url)) { + main(process.argv.slice(2)).then( + (code) => { process.exitCode = code; }, + (error: unknown) => { + if (error instanceof UsageError) { + process.stderr.write(`gh-pulse: ${error.message}\n${USAGE}\n`); + process.exitCode = 2; + return; + } + process.stderr.write(`gh-pulse: ${(error as Error).stack ?? String(error)}\n`); + process.exitCode = 1; + }, + ); +} diff --git a/package.json b/package.json index 1332238..73ce3fe 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@profullstack/cli-tools", - "version": "0.32.0", + "version": "0.33.0", "private": true, "description": "Local command-line tools, in TypeScript, exposed on PATH.", "type": "module", @@ -32,6 +32,8 @@ "sharp": "^0.35.3" }, "dependencies": { + "@profullstack/hqtui": "^0.5.1", + "@resvg/resvg-js": "^2.6.2", "axe-core": "^4.13.0", "imapflow": "^1.7.8", "mailparser": "^3.9.20", diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 59f24f5..eb5bc45 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -8,6 +8,12 @@ importers: .: dependencies: + '@profullstack/hqtui': + specifier: ^0.5.1 + version: 0.5.1 + '@resvg/resvg-js': + specifier: ^2.6.2 + version: 2.6.2 axe-core: specifier: ^4.13.0 version: 4.13.0 @@ -376,6 +382,91 @@ packages: '@pinojs/redact@0.4.0': resolution: {integrity: sha512-k2ENnmBugE/rzQfEcdWHcCY+/FM3VLzH9cYEsbdsoqrvzAKRhUZeRNhAZvB8OitQJ1TBed3yqWtdjzS6wJKBwg==} + '@profullstack/hqtui@0.5.1': + resolution: {integrity: sha512-TRj2CKCUhpXhhmsQ1DMIZb9OcU+XLm90BqhPPo0zk2OwyXNcQhlqBeNdxLqwTY/BlIWZpcTyQto458AQ7iCTFg==} + engines: {bun: '>=1.1', node: '>=22.6'} + hasBin: true + + '@resvg/resvg-js-android-arm-eabi@2.6.2': + resolution: {integrity: sha512-FrJibrAk6v29eabIPgcTUMPXiEz8ssrAk7TXxsiZzww9UTQ1Z5KAbFJs+Z0Ez+VZTYgnE5IQJqBcoSiMebtPHA==} + engines: {node: '>= 10'} + cpu: [arm] + os: [android] + + '@resvg/resvg-js-android-arm64@2.6.2': + resolution: {integrity: sha512-VcOKezEhm2VqzXpcIJoITuvUS/fcjIw5NA/w3tjzWyzmvoCdd+QXIqy3FBGulWdClvp4g+IfUemigrkLThSjAQ==} + engines: {node: '>= 10'} + cpu: [arm64] + os: [android] + + '@resvg/resvg-js-darwin-arm64@2.6.2': + resolution: {integrity: sha512-nmok2LnAd6nLUKI16aEB9ydMC6Lidiiq2m1nEBDR1LaaP7FGs4AJ90qDraxX+CWlVuRlvNjyYJTNv8qFjtL9+A==} + engines: {node: '>= 10'} + cpu: [arm64] + os: [darwin] + + '@resvg/resvg-js-darwin-x64@2.6.2': + resolution: {integrity: sha512-GInyZLjgWDfsVT6+SHxQVRwNzV0AuA1uqGsOAW+0th56J7Nh6bHHKXHBWzUrihxMetcFDmQMAX1tZ1fZDYSRsw==} + engines: {node: '>= 10'} + cpu: [x64] + os: [darwin] + + '@resvg/resvg-js-linux-arm-gnueabihf@2.6.2': + resolution: {integrity: sha512-YIV3u/R9zJbpqTTNwTZM5/ocWetDKGsro0SWp70eGEM9eV2MerWyBRZnQIgzU3YBnSBQ1RcxRZvY/UxwESfZIw==} + engines: {node: '>= 10'} + cpu: [arm] + os: [linux] + + '@resvg/resvg-js-linux-arm64-gnu@2.6.2': + resolution: {integrity: sha512-zc2BlJSim7YR4FZDQ8OUoJg5holYzdiYMeobb9pJuGDidGL9KZUv7SbiD4E8oZogtYY42UZEap7dqkkYuA91pg==} + engines: {node: '>= 10'} + cpu: [arm64] + os: [linux] + libc: [glibc] + + '@resvg/resvg-js-linux-arm64-musl@2.6.2': + resolution: {integrity: sha512-3h3dLPWNgSsD4lQBJPb4f+kvdOSJHa5PjTYVsWHxLUzH4IFTJUAnmuWpw4KqyQ3NA5QCyhw4TWgxk3jRkQxEKg==} + engines: {node: '>= 10'} + cpu: [arm64] + os: [linux] + libc: [musl] + + '@resvg/resvg-js-linux-x64-gnu@2.6.2': + resolution: {integrity: sha512-IVUe+ckIerA7xMZ50duAZzwf1U7khQe2E0QpUxu5MBJNao5RqC0zwV/Zm965vw6D3gGFUl7j4m+oJjubBVoftw==} + engines: {node: '>= 10'} + cpu: [x64] + os: [linux] + libc: [glibc] + + '@resvg/resvg-js-linux-x64-musl@2.6.2': + resolution: {integrity: sha512-UOf83vqTzoYQO9SZ0fPl2ZIFtNIz/Rr/y+7X8XRX1ZnBYsQ/tTb+cj9TE+KHOdmlTFBxhYzVkP2lRByCzqi4jQ==} + engines: {node: '>= 10'} + cpu: [x64] + os: [linux] + libc: [musl] + + '@resvg/resvg-js-win32-arm64-msvc@2.6.2': + resolution: {integrity: sha512-7C/RSgCa+7vqZ7qAbItfiaAWhyRSoD4l4BQAbVDqRRsRgY+S+hgS3in0Rxr7IorKUpGE69X48q6/nOAuTJQxeQ==} + engines: {node: '>= 10'} + cpu: [arm64] + os: [win32] + + '@resvg/resvg-js-win32-ia32-msvc@2.6.2': + resolution: {integrity: sha512-har4aPAlvjnLcil40AC77YDIk6loMawuJwFINEM7n0pZviwMkMvjb2W5ZirsNOZY4aDbo5tLx0wNMREp5Brk+w==} + engines: {node: '>= 10'} + cpu: [ia32] + os: [win32] + + '@resvg/resvg-js-win32-x64-msvc@2.6.2': + resolution: {integrity: sha512-ZXtYhtUr5SSaBrUDq7DiyjOFJqBVL/dOBN7N/qmi/pO0IgiWW/f/ue3nbvu9joWE5aAKDoIzy/CxsY0suwGosQ==} + engines: {node: '>= 10'} + cpu: [x64] + os: [win32] + + '@resvg/resvg-js@2.6.2': + resolution: {integrity: sha512-xBaJish5OeGmniDj9cW5PRa/PtmuVU3ziqrbr5xJj901ZDN4TosrVaNZpEiLZAxdfnhAe7uQ7QFWfjPe9d9K2Q==} + engines: {node: '>= 10'} + '@rolldown/binding-android-arm64@1.2.4': resolution: {integrity: sha512-jHC2cnyKz5xU2fhECtFl8OZ83cYNt13GZQD+0uMJ/X3o+ijmd56okHhTUwxVSHPx1IRVIJEZ1/1pPzeLCU6XKA==} engines: {node: ^20.19.0 || >=22.12.0} @@ -1183,6 +1274,59 @@ snapshots: '@pinojs/redact@0.4.0': {} + '@profullstack/hqtui@0.5.1': {} + + '@resvg/resvg-js-android-arm-eabi@2.6.2': + optional: true + + '@resvg/resvg-js-android-arm64@2.6.2': + optional: true + + '@resvg/resvg-js-darwin-arm64@2.6.2': + optional: true + + '@resvg/resvg-js-darwin-x64@2.6.2': + optional: true + + '@resvg/resvg-js-linux-arm-gnueabihf@2.6.2': + optional: true + + '@resvg/resvg-js-linux-arm64-gnu@2.6.2': + optional: true + + '@resvg/resvg-js-linux-arm64-musl@2.6.2': + optional: true + + '@resvg/resvg-js-linux-x64-gnu@2.6.2': + optional: true + + '@resvg/resvg-js-linux-x64-musl@2.6.2': + optional: true + + '@resvg/resvg-js-win32-arm64-msvc@2.6.2': + optional: true + + '@resvg/resvg-js-win32-ia32-msvc@2.6.2': + optional: true + + '@resvg/resvg-js-win32-x64-msvc@2.6.2': + optional: true + + '@resvg/resvg-js@2.6.2': + optionalDependencies: + '@resvg/resvg-js-android-arm-eabi': 2.6.2 + '@resvg/resvg-js-android-arm64': 2.6.2 + '@resvg/resvg-js-darwin-arm64': 2.6.2 + '@resvg/resvg-js-darwin-x64': 2.6.2 + '@resvg/resvg-js-linux-arm-gnueabihf': 2.6.2 + '@resvg/resvg-js-linux-arm64-gnu': 2.6.2 + '@resvg/resvg-js-linux-arm64-musl': 2.6.2 + '@resvg/resvg-js-linux-x64-gnu': 2.6.2 + '@resvg/resvg-js-linux-x64-musl': 2.6.2 + '@resvg/resvg-js-win32-arm64-msvc': 2.6.2 + '@resvg/resvg-js-win32-ia32-msvc': 2.6.2 + '@resvg/resvg-js-win32-x64-msvc': 2.6.2 + '@rolldown/binding-android-arm64@1.2.4': optional: true diff --git a/src/gh-pulse-tui.ts b/src/gh-pulse-tui.ts new file mode 100644 index 0000000..a231d4c --- /dev/null +++ b/src/gh-pulse-tui.ts @@ -0,0 +1,236 @@ +/** + * gh-pulse show — the last report and the history, in the terminal. + * + * Built on @profullstack/hqtui so the same data that goes out by email is + * readable over SSH without a mail client: the ranked movers on the left, the + * selected repo's 14-day traffic and detail on the right, and a History tab + * over every snapshot on disk (stars, followers, movers, traffic per run). + * + * Mouse: one click selects a row, the row under the pointer is lit, the wheel + * scrolls. Keys: up/down or j/k move, 1/2 or left/right switch tabs, o opens + * the HTML report in the browser, q quits. + */ + +import { createApp } from '@profullstack/hqtui'; + +import { + listSnapshots, + movementBits, + openInBrowser, + outputPaths, + readReport, + readSnapshot, + trafficBits, + type ReportJson, + type ReportMover, + type Snapshot, +} from './gh-pulse.ts'; + +export interface HistoryRow { + at: string; + repos: number; + stars: number; + followers: number; + movers: number; + views: number; + clones: number; +} + +/** One line per snapshot, oldest first. The movement rows are only stored for movers, which is what the counts are. */ +export function historyRows(snapshots: Snapshot[]): HistoryRow[] { + return snapshots.map((s) => { + const rows = Object.values(s.repos); + const moved = rows.filter((r) => r.movement); + return { + at: s.at, + repos: rows.length, + stars: rows.reduce((t, r) => t + r.stars, 0), + followers: s.followers.length, + movers: moved.length, + views: moved.reduce((t, r) => t + (r.movement?.views ?? 0), 0), + clones: moved.reduce((t, r) => t + (r.movement?.clones ?? 0), 0), + }; + }); +} + +export function loadHistory(dataDir: string, max = 120): HistoryRow[] { + const files = listSnapshots(dataDir).slice(-max); + const snaps: Snapshot[] = []; + for (const f of files) { + try { + snaps.push(readSnapshot(f)); + } catch { + // A truncated file (a run killed mid-write) is skipped, not fatal. + } + } + return historyRows(snaps); +} + +const fmtStamp = (iso: string): string => iso.replace('T', ' ').slice(0, 16); + +export async function showTui(dataDir: string): Promise { + const report: ReportJson | null = readReport(dataDir); + if (!report) { + process.stderr.write(`gh-pulse show: no report yet in ${dataDir}. Run \`gh-pulse\` (or \`gh-pulse --dry-run\`) first.\n`); + return 1; + } + const history = loadHistory(dataDir); + const movers = report.movers; + + let tab = 0; + let selected = 0; + let offset = 0; + let hovered = -1; + let hOffset = Math.max(0, history.length - 1); + let hSelected = Math.max(0, history.length - 1); + let hHovered = -1; + let notice = ''; + + const app = await createApp({ mouse: true, quitKeys: ['q', 'ctrl+c', 'escape'] }); + + const move = (delta: number): void => { + if (tab === 0) selected = Math.max(0, Math.min(movers.length - 1, selected + delta)); + else hSelected = Math.max(0, Math.min(history.length - 1, hSelected + delta)); + }; + + app.on('key', (e) => { + if (e.name === 'down' || e.name === 'j') move(1); + else if (e.name === 'up' || e.name === 'k') move(-1); + else if (e.name === 'pagedown') move(10); + else if (e.name === 'pageup') move(-10); + else if (e.name === 'home') move(-1e9); + else if (e.name === 'end') move(1e9); + else if (e.name === '1' || e.name === 'left') tab = 0; + else if (e.name === '2' || e.name === 'right') tab = 1; + else if (e.name === 'tab') tab = (tab + 1) % 2; + else if (e.name === 'o') { + notice = openInBrowser(outputPaths(dataDir).html) ? 'opened the HTML report in the browser' : `no opener found; the report is at ${outputPaths(dataDir).html}`; + } else return; + app.invalidate(); + }); + + app.on('mouse', (e) => { + if (e.action === 'move') { hovered = -1; hHovered = -1; app.invalidate(); } + }); + + const sel = (): ReportMover | undefined => movers[selected]; + + app.render(({ ui, theme, width, height }) => { + const bodyRows = Math.max(3, height - 8); + // The histogram draws one cell per value; repeat each day so fourteen days + // fill the detail pane instead of its left fifth. + const paneCols = Math.max(14, Math.floor((width - 3) / 2) - 4); + const perDay = Math.max(1, Math.floor(paneCols / 14)); + const widen = (values: number[]): number[] => values.flatMap((v) => Array(perDay).fill(v)); + if (selected < offset) offset = selected; + if (selected >= offset + bodyRows) offset = selected - bodyRows + 1; + if (hSelected < hOffset) hOffset = hSelected; + if (hSelected >= hOffset + bodyRows) hOffset = hSelected - bodyRows + 1; + + ui.column({ padding: 0 }, (col) => { + col.tabs({ tabs: ['Movers', 'History'], active: tab, size: 1, variant: 'underline', onSelect: (i) => { tab = i; app.invalidate(); } }); + col.row({ size: 4, gap: 2 }, (r) => { + r.keyValues([ + { label: 'Report', value: fmtStamp(report.at) }, + { label: 'Window since', value: fmtStamp(report.since) }, + { label: 'Moved', value: report.totals.repos ? `${movers.length} of ${report.totals.repos} repos` : String(movers.length) }, + { label: 'Followers', value: `${report.followers} (+${report.followersGained.length} / -${report.followersLost.length})` }, + ], { width: '1fr' }); + r.keyValues([ + { label: 'Stars', value: `${report.totals.stars}${report.totals.starsDelta === null ? '' : ` (${report.totals.starsDelta >= 0 ? '+' : ''}${report.totals.starsDelta})`}`, color: theme.accent }, + { label: 'Views', value: `${report.totals.views} / ${report.totals.uniques} unique` }, + { label: 'Clones', value: `${report.totals.clones} / ${report.totals.cloners} unique` }, + { label: 'Traffic', value: `${report.trafficLabel}, newest GitHub day ${report.newestTrafficDay}` }, + ], { width: '1fr' }); + }); + + if (tab === 0) { + col.grid({ columns: ['1fr', '1fr'], rows: ['1fr'], gap: 1 }, (grid) => { + grid.panel({ title: `Ranked by movement (${movers.length})`, footer: notice || '↑/↓ move · click selects · o opens HTML · 2 history · q quit' }, (p) => { + p.table({ + rows: movers, + selected, + hovered, + offset, + followSelection: true, + scrollbar: true, + zebra: true, + columns: [ + { key: 'rank', title: '#', width: 4, align: 'right', render: (_r, i) => String(i + 1) }, + { key: 'repo', title: 'Repo', color: theme.primary }, + { key: 'score', title: 'pts', width: 6, align: 'right', render: (r) => r.score.toFixed(0) }, + { key: 'bits', title: 'Movement', render: (r) => [...movementBits(r.movement), ...trafficBits(r.movement)].join(' · ') }, + ], + onSelectRow: (row) => { selected = offset + row; app.invalidate(); }, + onHoverRow: (row) => { hovered = row === null ? -1 : offset + row; app.invalidate(); }, + onScroll: (delta) => { move(delta); app.invalidate(); }, + }); + }); + const x = sel(); + grid.panel({ title: x ? x.repo : 'Nothing moved', subtitle: x ? `★ ${x.stars} · ⑂ ${x.forks} · ${x.score.toFixed(0)} pts` : '' }, (p) => { + if (!x) return; + const m = x.movement; + const v14 = x.views14d.reduce((t, d) => t + d.count, 0); + const c14 = x.clones14d.reduce((t, d) => t + d.count, 0); + p.text(movementBits(m).join(' · ') || 'traffic only', { fg: theme.muted, size: 1 }); + p.keyValues([ + { label: `Views (${report.trafficLabel})`, value: `${m.views} / ${m.uniques} unique`, color: theme.info }, + { label: `Clones (${report.trafficLabel})`, value: `${m.clones} / ${m.cloners} unique`, color: theme.warning }, + ], { size: 2 }); + p.text(`Views per day, ${x.views14d[0]?.day ?? ''} to ${report.newestTrafficDay} (${v14} in 14d, peak ${Math.max(0, ...x.views14d.map((d) => d.count))})`, { fg: theme.muted, size: 1 }); + p.histogram({ values: widen(x.views14d.map((d) => d.count)), color: theme.info, size: 5 }); + p.text(`Clones per day (${c14} in 14d, peak ${Math.max(0, ...x.clones14d.map((d) => d.count))})`, { fg: theme.muted, size: 1 }); + p.histogram({ values: widen(x.clones14d.map((d) => d.count)), color: theme.warning, size: 5 }); + const lines: string[] = []; + if (x.referrers.length) lines.push(`Referrers: ${x.referrers.slice(0, 5).map((q) => `${q.referrer} ${q.count}/${q.uniques}`).join(' · ')}`); + if (x.paths.length) lines.push(`Popular: ${x.paths.slice(0, 4).map((q) => `${q.path.replace(`/${x.repo}`, '') || '/'} ${q.count}/${q.uniques}`).join(' · ')}`); + if (m.newStargazers.length) lines.push(`Starred by ${m.newStargazers.slice(0, 10).join(', ')}`); + if (m.newForks.length) lines.push(`Forked by ${m.newForks.slice(0, 6).map((f) => f.split('/')[0]).join(', ')}`); + if (m.authors.length) lines.push(`Commits by ${m.authors.slice(0, 6).join(', ')}`); + for (const pr of m.mergedPrs.slice(0, 5)) lines.push(`merged #${pr.n} ${pr.t}`); + for (const pr of m.openedPrs.slice(0, 3)) lines.push(`opened #${pr.n} ${pr.t} (${pr.u})`); + for (const is of m.openedIssues.slice(0, 3)) lines.push(`issue #${is.n} ${is.t} (${is.u})`); + for (const rel of m.releaseList.slice(0, 5)) lines.push(`released ${rel.tag}`); + if (lines.length) p.text(lines.join('\n'), { wrap: true }); + p.spacer('fill'); + p.text(`${x.url}/graphs/traffic`, { fg: theme.muted, size: 1 }); + }); + }); + } else { + col.panel({ title: `History (${history.length} snapshot${history.length === 1 ? '' : 's'})`, footer: notice || 'one row per run · ↑/↓ move · 1 movers · q quit' }, (p) => { + const stars = history.map((h) => h.stars); + const followers = history.map((h) => h.followers); + p.sparkline({ values: stars, label: 'Stars', text: String(stars.at(-1) ?? 0), color: theme.accent, size: 1 }); + p.sparkline({ values: followers, label: 'Followers', text: String(followers.at(-1) ?? 0), color: theme.primary, size: 1 }); + p.sparkline({ values: history.map((h) => h.views), label: 'Views', text: String(history.at(-1)?.views ?? 0), color: theme.info, size: 1 }); + p.sparkline({ values: history.map((h) => h.clones), label: 'Clones', text: String(history.at(-1)?.clones ?? 0), color: theme.warning, size: 1 }); + p.spacer(1); + p.table({ + rows: history, + selected: hSelected, + hovered: hHovered, + offset: hOffset, + followSelection: true, + scrollbar: true, + zebra: true, + columns: [ + { key: 'at', title: 'Run (UTC)', width: 17, render: (r) => fmtStamp(r.at) }, + { key: 'repos', title: 'Repos', width: 6, align: 'right' }, + { key: 'movers', title: 'Moved', width: 6, align: 'right' }, + { key: 'stars', title: 'Stars', width: 7, align: 'right' }, + { key: 'followers', title: 'Followers', width: 10, align: 'right' }, + { key: 'views', title: 'Views', width: 7, align: 'right' }, + { key: 'clones', title: 'Clones', width: 7, align: 'right' }, + ], + onSelectRow: (row) => { hSelected = hOffset + row; app.invalidate(); }, + onHoverRow: (row) => { hHovered = row === null ? -1 : hOffset + row; app.invalidate(); }, + onScroll: (delta) => { move(delta); app.invalidate(); }, + }); + }); + } + }); + }); + + await app.start(); + return 0; +} diff --git a/src/gh-pulse.ts b/src/gh-pulse.ts new file mode 100644 index 0000000..249691a --- /dev/null +++ b/src/gh-pulse.ts @@ -0,0 +1,1031 @@ +/** + * gh-pulse — what moved on GitHub since yesterday, ranked, with traffic. + * + * Every repo the `gh` token can see (yours plus every org you belong to, + * forks excluded) is checked for movement since the previous run: stars, + * forks, commits, pull requests, issues, releases, and the traffic GitHub + * shows at /graphs/traffic (views, unique visitors, clones, referrers, + * popular content). Movers are ranked by a weighted score and emailed with + * their traffic; the top ones get a 14-day chart. + * + * WHY THE WINDOW IS A SNAPSHOT, NOT A CLOCK. GitHub publishes traffic in + * UTC-day buckets one to two days late: at 03:00 UTC the newest bucket with + * anything in it is usually the day before yesterday, and some repos carry + * empty buckets padded up to today. "The last 24 hours" read off the clock is + * therefore zero for almost everything. What can be measured exactly is the + * growth of each bucket since the previous run, so that is the window: it is + * "since the last email", and a skipped day widens it instead of losing data. + * Follower changes work the same way from a stored list of logins, which is + * also what lets new and lost followers be named. + * + * WHY THE SNAPSHOTS MATTER. GitHub keeps traffic for 14 days and nothing + * else. Every run writes the raw per-repo counts and buckets, gzipped and + * dated, under the data dir. Those files are the only long-run record, and + * they are what `gh-pulse show` and any downstream dataset read. + * + * Pure functions (scoring, windows, rendering) are exported for tests; the + * network and the filesystem enter only through `run()` and its `deps`. + */ + +import { execFileSync, spawnSync } from 'node:child_process'; +import { existsSync, mkdirSync, readdirSync, readFileSync, writeFileSync } from 'node:fs'; +import { createRequire } from 'node:module'; +import { homedir, hostname } from 'node:os'; +import { join } from 'node:path'; +import { gunzipSync, gzipSync } from 'node:zlib'; + +// ---------------------------------------------------------------- types + +export interface Bucket { + timestamp: string; + count: number; + uniques: number; +} + +export interface Day { + day: string; + count: number; + uniques: number; +} + +export interface Ref { + n: number; + t: string; + u: string; + url: string; +} + +export interface Movement { + stars: number; + forks: number; + commits: number; + authors: string[]; + prOpened: number; + prMerged: number; + prClosed: number; + issuesOpened: number; + issuesClosed: number; + releases: number; + views: number; + uniques: number; + clones: number; + cloners: number; + newStargazers: string[]; + newForks: string[]; + mergedPrs: Ref[]; + openedPrs: Ref[]; + openedIssues: Ref[]; + releaseList: { tag: string; url: string }[]; + score: number; + mover: boolean; +} + +export interface Referrer { + referrer: string; + count: number; + uniques: number; +} + +export interface PopularPath { + path: string; + title?: string; + count: number; + uniques: number; +} + +/** What GitHub says about a repo in the listing; the fields this tool reads. */ +export interface RepoInfo { + id: number; + full_name: string; + html_url: string; + private: boolean; + fork: boolean; + archived: boolean; + stargazers_count: number; + forks_count: number; + open_issues_count: number; + pushed_at: string; + updated_at: string; +} + +/** One repo's row in a snapshot file. */ +export interface RepoSnapshot { + id: number; + stars: number; + forks: number; + openIssues: number; + pushedAt: string; + private: boolean; + archived: boolean; + views: Bucket[] | null; + clones: Bucket[] | null; + referrers?: Referrer[]; + paths?: PopularPath[]; + movement?: Movement; +} + +export interface Snapshot { + at: string; + user: string; + followersCount: number; + followers: string[]; + repos: Record; +} + +export interface RepoState { + repo: RepoInfo; + views: Bucket[] | null; + clones: Bucket[] | null; + trafficOk: boolean; + m: Movement; + referrers?: Referrer[]; + paths?: PopularPath[]; + img?: string; +} + +export interface Portfolio { + views: number; + uniques: number; + clones: number; + cloners: number; + viewsByDay: Map; + clonesByDay: Map; +} + +export interface ReportContext { + login: string; + now: Date; + cutoff: Date; + useBaseline: boolean; + newestDay: string; + trafficLabel: string; + repoCount: number; + movers: RepoState[]; + detailed: RepoState[]; + followers: string[]; + followerMoves: { gained: string[]; lost: string[] }; + totalStars: number; + prevTotalStars: number | null; + port: Portfolio; + trafficBlind: number; + portfolioImg: string; +} + +export interface Image { + id: string; + buf: Buffer; +} + +export interface RunOptions { + dryRun: boolean; + to: string; + top: number; + repos: string[]; + hours: number; + dataDir: string; + from: string; +} + +export interface RunDeps { + fetch: typeof fetch; + now: () => Date; + token: () => string; + log: (line: string) => void; + resendKey: () => string | undefined; + render: (svg: string) => Buffer; +} + +export interface RunResult { + subject: string; + movers: number; + repos: number; + sent: boolean; + snapshot: string | null; + html: string; + calls: number; +} + +// ---------------------------------------------------------------- paths + +export function dataDir(env: NodeJS.ProcessEnv = process.env): string { + return env['GH_PULSE_DATA'] ?? join(homedir(), '.local', 'share', 'gh-pulse'); +} + +export const DEFAULT_FROM = 'GitHub Pulse '; + +/** + * cron gets no environment; the house vault export is the fallback. Only + * variables that are not already set are taken, so a shell wins over the file. + */ +export function loadShellEnv(env: NodeJS.ProcessEnv = process.env, file = join(homedir(), '.config', 'logicsrc', 'shell.env')): void { + if (!existsSync(file)) return; + for (const line of readFileSync(file, 'utf8').split('\n')) { + const m = /^(?:export\s+)?([A-Z0-9_]+)=(.*)$/.exec(line); + if (!m || env[m[1]!] !== undefined) continue; + env[m[1]!] = m[2]!.trim().replace(/^["']|["']$/g, ''); + } +} + +export function ghToken(): string { + try { + return execFileSync('gh', ['auth', 'token'], { encoding: 'utf8' }).trim(); + } catch { + throw new Error('gh is not logged in (gh auth token failed)'); + } +} + +export function gitEmail(): string { + try { + return execFileSync('git', ['config', '--get', 'user.email'], { encoding: 'utf8' }).trim(); + } catch { + return ''; + } +} + +// ---------------------------------------------------------------- scoring + +export const WEIGHTS = { + star: 5, + fork: 4, + unique: 1, + view: 0.2, + clone: 0.5, + cloner: 1, + commit: 1, + prOpened: 2, + prMerged: 3, + prClosed: 0.5, + issueOpened: 2, + issueClosed: 1, + release: 5, +} as const; + +export const COMMIT_CAP = 25; + +export function emptyMovement(): Movement { + return { + stars: 0, forks: 0, commits: 0, authors: [], prOpened: 0, prMerged: 0, prClosed: 0, + issuesOpened: 0, issuesClosed: 0, releases: 0, views: 0, uniques: 0, clones: 0, cloners: 0, + newStargazers: [], newForks: [], mergedPrs: [], openedPrs: [], openedIssues: [], releaseList: [], + score: 0, mover: false, + }; +} + +export function score(m: Movement): number { + const W = WEIGHTS; + return Math.max(0, m.stars) * W.star + Math.max(0, m.forks) * W.fork + m.uniques * W.unique + m.views * W.view + + m.clones * W.clone + m.cloners * W.cloner + Math.min(m.commits, COMMIT_CAP) * W.commit + + m.prOpened * W.prOpened + m.prMerged * W.prMerged + m.prClosed * W.prClosed + + m.issuesOpened * W.issueOpened + m.issuesClosed * W.issueClosed + m.releases * W.release; +} + +/** + * Movement worth a line. One clone a day is background noise across a long + * tail of repos (mirrors, CI), so traffic-only movement needs two unique + * visitors or two unique cloners; any event at all counts. + */ +export function isMover(m: Movement): boolean { + return m.stars !== 0 || m.forks !== 0 || m.commits > 0 || m.prOpened > 0 || m.prMerged > 0 || m.prClosed > 0 || + m.issuesOpened > 0 || m.issuesClosed > 0 || m.releases > 0 || m.uniques >= 2 || m.cloners >= 2; +} + +// ---------------------------------------------------------------- windows + +/** + * Growth of the 14-day traffic buckets since the previous snapshot. A bucket + * present in both counts only what it gained; a bucket new since last time + * counts in full; a bucket that rolled off is gone and irrelevant. Without a + * baseline, every bucket on or after `firstRunSince` counts. + */ +export function trafficDelta(cur: Bucket[] | null, prev: Bucket[] | null | undefined, firstRunSince: string): { count: number; uniques: number } { + if (!cur) return { count: 0, uniques: 0 }; + const pm = new Map((prev ?? []).map((b) => [b.timestamp, b])); + let count = 0; + let uniques = 0; + for (const b of cur) { + if (prev) { + const p = pm.get(b.timestamp); + count += Math.max(0, b.count - (p ? p.count : 0)); + uniques += Math.max(0, b.uniques - (p ? p.uniques : 0)); + } else if (b.timestamp.slice(0, 10) >= firstRunSince) { + count += b.count; + uniques += b.uniques; + } + } + return { count, uniques }; +} + +/** The newest UTC day on which any bucket recorded something, else `fallback`. */ +export function newestDay(all: Iterable, fallback: string): string { + let best = ''; + for (const buckets of all) { + for (const b of buckets ?? []) { + const day = b.timestamp.slice(0, 10); + if (b.count > 0 && day > best) best = day; + } + } + return best || fallback; +} + +/** Fourteen days ending on `endDay`, zero-filled, oldest first. */ +export function fourteenDays(buckets: Bucket[] | null, endDay: string): Day[] { + const m = new Map((buckets ?? []).map((b) => [b.timestamp.slice(0, 10), b])); + const end = Date.parse(`${endDay}T00:00:00Z`); + const out: Day[] = []; + for (let i = 13; i >= 0; i -= 1) { + const day = new Date(end - i * 86400000).toISOString().slice(0, 10); + const b = m.get(day); + out.push({ day, count: b ? b.count : 0, uniques: b ? b.uniques : 0 }); + } + return out; +} + +export const isoDay = (d: Date): string => d.toISOString().slice(0, 10); + +/** + * The previous snapshot is the baseline when it is plausibly "yesterday's": + * older than six hours (a re-run minutes later would report nothing) and + * younger than four days (after that the clock window is the honest one). + */ +export function baselineFor(prev: Snapshot | null, now: Date): Snapshot | null { + if (!prev) return null; + const ageH = (now.getTime() - Date.parse(prev.at)) / 3600000; + return ageH >= 6 && ageH <= 96 ? prev : null; +} + +// ---------------------------------------------------------------- snapshots + +export function snapshotsDir(dir: string): string { + return join(dir, 'snapshots'); +} + +export function readSnapshot(file: string): Snapshot { + return JSON.parse(gunzipSync(readFileSync(file)).toString('utf8')) as Snapshot; +} + +export function listSnapshots(dir: string): string[] { + const d = snapshotsDir(dir); + if (!existsSync(d)) return []; + return readdirSync(d).filter((f) => f.endsWith('.json.gz')).sort().map((f) => join(d, f)); +} + +export function latestSnapshot(dir: string, log: (s: string) => void = () => {}): Snapshot | null { + const files = listSnapshots(dir); + const file = files[files.length - 1]; + if (!file) return null; + try { + return readSnapshot(file); + } catch (error) { + log(`unreadable snapshot ${file}: ${(error as Error).message}`); + return null; + } +} + +export function writeSnapshot(dir: string, snap: Snapshot): string { + const d = snapshotsDir(dir); + mkdirSync(d, { recursive: true }); + const file = join(d, `${snap.at.slice(0, 13)}.json.gz`); + writeFileSync(file, gzipSync(JSON.stringify(snap))); + return file; +} + +// ---------------------------------------------------------------- github + +export class GitHub { + readonly calls = { made: 0, retries: 0 }; + remaining: number | null = null; + private readonly queue: (() => void)[] = []; + private active = 0; + // Explicit fields rather than constructor parameter properties: Node's type + // stripping accepts only syntax it can erase, and this is the one construct + // in the repo it refuses (see gh.ts). + private readonly token: string; + private readonly fetchImpl: typeof fetch; + private readonly concurrency: number; + + constructor(token: string, fetchImpl: typeof fetch, concurrency = 6) { + this.token = token; + this.fetchImpl = fetchImpl; + this.concurrency = concurrency; + } + + /** Run `fn` when a slot is free; six at a time keeps clear of the abuse limits. */ + limit(fn: () => Promise): Promise { + return new Promise((resolve, reject) => { + const start = () => { + this.active += 1; + fn().then(resolve, reject).finally(() => { + this.active -= 1; + this.queue.shift()?.(); + }); + }; + if (this.active < this.concurrency) start(); + else this.queue.push(start); + }); + } + + async get(pathOrUrl: string, options: { accept?: string; allow?: number[] } = {}): Promise<{ data: T | null; link: string; status: number }> { + const url = pathOrUrl.startsWith('http') ? pathOrUrl : `https://api.github.com${pathOrUrl}`; + for (let attempt = 0; ; attempt += 1) { + this.calls.made += 1; + const r = await this.fetchImpl(url, { + headers: { + authorization: `Bearer ${this.token}`, + accept: options.accept ?? 'application/vnd.github+json', + 'x-github-api-version': '2022-11-28', + 'user-agent': 'gh-pulse (cli-tools)', + }, + }); + const rem = r.headers.get('x-ratelimit-remaining'); + if (rem !== null) this.remaining = Number(rem); + if (r.ok) return { data: (await r.json()) as T, link: r.headers.get('link') ?? '', status: r.status }; + if (options.allow?.includes(r.status)) return { data: null, link: '', status: r.status }; + const retryable = r.status === 429 || r.status === 502 || r.status === 503 || + (r.status === 403 && (r.headers.get('retry-after') !== null || rem === '0')); + if (retryable && attempt < 4) { + this.calls.retries += 1; + let wait = Number(r.headers.get('retry-after') ?? 0) * 1000; + if (!wait && rem === '0') wait = Math.max(0, Number(r.headers.get('x-ratelimit-reset')) * 1000 - Date.now()) + 1000; + if (!wait) wait = 2000 * (attempt + 1); + await new Promise((s) => setTimeout(s, Math.min(wait, 120000))); + continue; + } + const body = await r.text().catch(() => ''); + throw new Error(`GitHub ${r.status} for ${url}: ${body.slice(0, 200)}`); + } + } + + async all(path: string, options: { accept?: string; allow?: number[]; max?: number } = {}): Promise { + const out: T[] = []; + let url: string | undefined = path; + for (let i = 0; url && i < (options.max ?? 20); i += 1) { + const { data, link } = await this.get(url, options); + if (!Array.isArray(data)) break; + out.push(...data); + url = nextLink(link); + } + return out; + } +} + +export const nextLink = (link: string): string | undefined => /<([^>]+)>;\s*rel="next"/.exec(link)?.[1]; +export const lastPage = (link: string): number => Number(/[?&]page=(\d+)>;\s*rel="last"/.exec(link)?.[1] ?? 1); + +// ---------------------------------------------------------------- charts + +/** + * Two single-hue bar charts side by side in one SVG: thin marks with rounded + * tops, a recessive grid, direct labels only on the peak and the latest day. + * Text is the chart's own; the email around it carries everything else. + */ +export function chartPair(a: ChartSeries, b: ChartSeries, W = 640, H = 150): string { + const half = W / 2; + return `` + + `${bars(a, 0, 0, half - 12, H)}${bars(b, half + 12, 0, half - 12, H)}`; +} + +export interface ChartSeries { + title: string; + color: string; + days: { day: string; count: number }[]; +} + +function bars({ title, color, days }: ChartSeries, x0: number, y0: number, w: number, h: number): string { + const padT = 38; + const padB = 20; + const padL = 8; + const padR = 8; + const max = Math.max(1, ...days.map((d) => d.count)); + const n = Math.max(1, days.length); + const gap = 2; + const bw = (w - padL - padR - gap * (n - 1)) / n; + const ph = h - padT - padB; + const baseY = y0 + padT + ph; + const total = days.reduce((t, d) => t + d.count, 0); + let s = `${esc(title)}`; + s += `${total} in 14d`; + for (const f of [0.5, 1]) { + const gy = (baseY - ph * f).toFixed(1); + s += ``; + } + s += ``; + let peak = 0; + days.forEach((d, i) => { if (d.count > (days[peak]?.count ?? 0)) peak = i; }); + days.forEach((d, i) => { + const bx = x0 + padL + i * (bw + gap); + const bh = d.count ? Math.max(2, (d.count / max) * ph) : 0; + if (bh) s += ``; + if (d.count && (i === peak || i === n - 1)) { + s += `${d.count}`; + } + if (i === 0 || i === n - 1 || i === 7) { + s += `${d.day.slice(5)}`; + } + }); + return s; +} + +function roundTop(x: number, y: number, w: number, h: number, r0: number): string { + const r = Math.min(r0, h); + return `M${x} ${y + h} V${y + r} Q${x} ${y} ${x + r} ${y} H${x + w - r} Q${x + w} ${y} ${x + w} ${y + r} V${y + h} Z`; +} + +// ---------------------------------------------------------------- text bits + +export function esc(s: unknown): string { + return String(s ?? '').replace(/[&<>"']/g, (c) => ({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' })[c]!); +} +export const signed = (n: number): string => (n > 0 ? `+${n}` : `${n}`); +export const plural = (n: number, w: string): string => `${n} ${w}${n === 1 ? '' : 's'}`; + +export function movementBits(m: Movement): string[] { + const b: string[] = []; + if (m.stars) b.push(`${signed(m.stars)} star${Math.abs(m.stars) === 1 ? '' : 's'}`); + if (m.forks) b.push(`${signed(m.forks)} fork${Math.abs(m.forks) === 1 ? '' : 's'}`); + if (m.commits) b.push(plural(m.commits, 'commit')); + if (m.prMerged) b.push(`${plural(m.prMerged, 'PR')} merged`); + if (m.prOpened) b.push(`${plural(m.prOpened, 'PR')} opened`); + if (m.prClosed) b.push(`${plural(m.prClosed, 'PR')} closed`); + if (m.issuesOpened) b.push(`${plural(m.issuesOpened, 'issue')} opened`); + if (m.issuesClosed) b.push(`${plural(m.issuesClosed, 'issue')} closed`); + if (m.releases) b.push(plural(m.releases, 'release')); + return b; +} + +export function trafficBits(m: Movement): string[] { + const b: string[] = []; + if (m.views) b.push(`${m.views} view${m.views === 1 ? '' : 's'} / ${m.uniques} unique`); + if (m.clones) b.push(`${m.clones} clone${m.clones === 1 ? '' : 's'} / ${m.cloners} unique`); + return b; +} + +export function subjectLine(now: Date, movers: number, starsDelta: number | null, views: number, clones: number): string { + return `GitHub pulse ${isoDay(now)}: ${movers} repos moved` + + (starsDelta ? `, ${signed(starsDelta)} stars` : '') + + `, ${views} views, ${clones} clones`; +} + +const fmtWhen = (d: Date): string => d.toUTCString().replace(/:\d\d GMT$/, ' UTC'); + +// ---------------------------------------------------------------- html + +export function renderHtml(c: ReportContext): string { + const { movers, detailed } = c; + const maxScore = Math.max(1, ...movers.map((x) => x.m.score)); + const windowH = Math.round((c.now.getTime() - c.cutoff.getTime()) / 3600000); + const link = (x: RepoState): string => + `${esc(x.repo.full_name)}` + + (x.repo.private ? ' private' : ''); + const tile = (label: string, value: string | number, sub: string): string => + `` + + `
${label}
` + + `
${value}
` + + (sub ? `
${sub}
` : '') + ''; + const n = (v: number): string => v.toLocaleString('en-US'); + const starDelta = c.prevTotalStars === null ? 'all repos' : `${signed(c.totalStars - c.prevTotalStars)} in window`; + const followerDelta = c.useBaseline + ? `${signed(c.followerMoves.gained.length - c.followerMoves.lost.length)} (${c.followerMoves.gained.length} new, ${c.followerMoves.lost.length} lost)` + : 'baseline captured'; + const userLink = (l: string): string => `${esc(l)}`; + const refLink = (p: Ref): string => `#${p.n} ${esc(p.t)}`; + + let h = `
+
+
GitHub pulse
+
${esc(c.login)} · ${fmtWhen(c.now)} · window ${windowH}h since ${fmtWhen(c.cutoff)}${c.useBaseline ? '' : ' (no previous snapshot; clock window)'}
+ +${tile('Repos moved', movers.length, `of ${c.repoCount} scanned`)} +${tile('Stars', n(c.totalStars), starDelta)} +${tile('Views', n(c.port.views), `${n(c.port.uniques)} unique visitors · ${esc(c.trafficLabel)}`)} +${tile('Clones', n(c.port.clones), `${n(c.port.cloners)} unique cloners · ${esc(c.trafficLabel)}`)} +
+
Followers: ${n(c.followers.length)} · ${followerDelta}${c.trafficBlind ? ` · traffic unavailable for ${c.trafficBlind} repos (no push access)` : ''}
+14-day views and clones across all repos +`; + + h += '
Ranked by movement
'; + if (movers.length === 0) h += '
Nothing moved in the window.
'; + h += ''; + movers.slice(0, 40).forEach((x, i) => { + const pct = Math.max(2, Math.round((x.m.score / maxScore) * 100)); + const bits = [...movementBits(x.m), ...trafficBits(x.m)].join(' · '); + h += `` + + ``; + }); + h += '
${i + 1}${link(x)}
${esc(bits)}
` + + `
 ${x.m.score.toFixed(0)} pts
'; + if (movers.length > 40) h += `
and ${movers.length - 40} more with smaller movement
`; + + if (detailed.length) h += `
Traffic for the top ${detailed.length}
`; + for (const x of detailed) { + const m = x.m; + const r = x.repo; + const v14 = fourteenDays(x.views, c.newestDay).reduce((t, d) => t + d.count, 0); + const c14 = fourteenDays(x.clones, c.newestDay).reduce((t, d) => t + d.count, 0); + h += `
+
${link(x)} ★ ${r.stargazers_count} · ⑂ ${r.forks_count} · ${m.score.toFixed(0)} pts
+
${esc(movementBits(m).join(' · ') || 'traffic only')}
`; + if (x.trafficOk) { + h += ` + + + +
${esc(c.trafficLabel)}14 days to ${esc(c.newestDay)}
Views / unique visitors${m.views} / ${m.uniques}${v14}
Clones / unique cloners${m.clones} / ${m.cloners}${c14}
`; + if (x.img) h += `14-day views and clones for ${esc(r.full_name)}`; + const cols: string[] = []; + if (x.referrers?.length) { + cols.push('
Referrers (14d)
' + + x.referrers.slice(0, 5).map((q) => `
${esc(q.referrer)} ${q.count} views · ${q.uniques} unique
`).join('')); + } + if (x.paths?.length) { + cols.push('
Popular content (14d)
' + + x.paths.slice(0, 5).map((q) => `
${esc(q.path.replace(`/${r.full_name}`, '') || '/')} ${q.count} views · ${q.uniques} unique
`).join('')); + } + if (cols.length) h += `${cols.map((col) => ``).join('')}
${col}
`; + } else { + h += '
Traffic not available (needs push access).
'; + } + const extras: string[] = []; + if (m.newStargazers.length) extras.push(`Starred by ${m.newStargazers.slice(0, 12).map(userLink).join(', ')}${m.newStargazers.length > 12 ? ` and ${m.newStargazers.length - 12} more` : ''}`); + if (m.newForks.length) extras.push(`Forked by ${m.newForks.slice(0, 8).map((f) => esc(f.split('/')[0])).join(', ')}`); + if (m.authors.length) extras.push(`Commits by ${m.authors.slice(0, 6).map(esc).join(', ')}`); + if (m.mergedPrs.length) extras.push(`Merged: ${m.mergedPrs.slice(0, 5).map(refLink).join('; ')}`); + if (m.openedPrs.length) extras.push(`Opened PRs: ${m.openedPrs.slice(0, 5).map((p) => `${refLink(p)} (${esc(p.u)})`).join('; ')}`); + if (m.openedIssues.length) extras.push(`Opened issues: ${m.openedIssues.slice(0, 5).map((p) => `${refLink(p)} (${esc(p.u)})`).join('; ')}`); + if (m.releaseList.length) extras.push(`Released ${m.releaseList.map((p) => `${esc(p.tag)}`).join(', ')}`); + if (extras.length) h += `
${extras.map((e) => `
${e}
`).join('')}
`; + h += `
`; + } + + if (c.followerMoves.gained.length || c.followerMoves.lost.length) { + h += '
Followers
'; + if (c.followerMoves.gained.length) h += `
New: ${c.followerMoves.gained.slice(0, 30).map(userLink).join(', ')}
`; + if (c.followerMoves.lost.length) h += `
Unfollowed: ${c.followerMoves.lost.slice(0, 30).map(esc).join(', ')}
`; + h += '
'; + } + + h += `
Scoring: star 5 · fork 4 · release 5 · PR merged 3 · PR/issue opened 2 · commit 1 (capped ${COMMIT_CAP}) · unique visitor 1 · unique cloner 1 · clone 0.5 · view 0.2. Traffic is GitHub's own /graphs/traffic data, which GitHub publishes one to two days late; it is counted as growth of the 14-day buckets since the previous run, so nothing is missed or double counted. Generated by gh-pulse on ${esc(hostname())}.
+
`; + return h; +} + +export function renderText(c: ReportContext): string { + const out: string[] = []; + out.push(`GITHUB PULSE ${isoDay(c.now)} (${c.login})`); + out.push(`window since ${c.cutoff.toISOString()} · traffic: ${c.trafficLabel}, newest GitHub day ${c.newestDay}`); + out.push(''); + out.push(`${c.movers.length} of ${c.repoCount} repos moved · stars ${c.totalStars}${c.prevTotalStars === null ? '' : ` (${signed(c.totalStars - c.prevTotalStars)})`} · views ${c.port.views}/${c.port.uniques} unique · clones ${c.port.clones}/${c.port.cloners} unique · followers ${c.followers.length} (${signed(c.followerMoves.gained.length - c.followerMoves.lost.length)})`); + out.push(''); + c.movers.forEach((x, i) => { + out.push(`${String(i + 1).padStart(2)}. ${x.repo.full_name} ${x.m.score.toFixed(0)} pts`); + out.push(` ${[...movementBits(x.m), ...trafficBits(x.m)].join(' · ')}`); + }); + return `${out.join('\n')}\n`; +} + +/** The report as data: what `gh-pulse show` and any downstream dataset read. */ +export interface ReportJson { + at: string; + since: string; + newestTrafficDay: string; + trafficLabel: string; + user: string; + followers: number; + followersGained: string[]; + followersLost: string[]; + totals: { repos: number; stars: number; starsDelta: number | null; views: number; uniques: number; clones: number; cloners: number }; + movers: ReportMover[]; +} + +export interface ReportMover { + repo: string; + private: boolean; + url: string; + stars: number; + forks: number; + score: number; + movement: Movement; + views14d: Day[]; + clones14d: Day[]; + referrers: Referrer[]; + paths: PopularPath[]; +} + +export function reportJson(c: ReportContext): ReportJson { + return { + at: c.now.toISOString(), + since: c.cutoff.toISOString(), + newestTrafficDay: c.newestDay, + trafficLabel: c.trafficLabel, + user: c.login, + followers: c.followers.length, + followersGained: c.followerMoves.gained, + followersLost: c.followerMoves.lost, + totals: { + repos: c.repoCount, stars: c.totalStars, starsDelta: c.prevTotalStars === null ? null : c.totalStars - c.prevTotalStars, + views: c.port.views, uniques: c.port.uniques, clones: c.port.clones, cloners: c.port.cloners, + }, + movers: c.movers.map((x) => ({ + repo: x.repo.full_name, private: x.repo.private, url: x.repo.html_url, stars: x.repo.stargazers_count, forks: x.repo.forks_count, + score: Number(x.m.score.toFixed(2)), movement: x.m, + views14d: fourteenDays(x.views, c.newestDay), clones14d: fourteenDays(x.clones, c.newestDay), + referrers: x.referrers ?? [], paths: x.paths ?? [], + })), + }; +} + +// ---------------------------------------------------------------- mail + +export interface Mail { + to: string; + from: string; + subject: string; + html: string; + text: string; + images: Image[]; +} + +/** The Resend request body. Inline images ride as attachments with a content_id, referenced as cid: in the HTML. */ +export function resendBody(mail: Mail): Record { + return { + from: mail.from, + to: mail.to.split(',').map((s) => s.trim()).filter(Boolean), + subject: mail.subject, + html: mail.html, + text: mail.text, + attachments: mail.images.map((i) => ({ filename: `${i.id}.png`, content: i.buf.toString('base64'), content_type: 'image/png', content_id: i.id })), + }; +} + +export async function sendResend(mail: Mail, key: string, fetchImpl: typeof fetch): Promise { + const r = await fetchImpl('https://api.resend.com/emails', { + method: 'POST', + headers: { authorization: `Bearer ${key}`, 'content-type': 'application/json' }, + body: JSON.stringify(resendBody(mail)), + }); + const j = (await r.json().catch(() => ({}))) as { id?: string }; + if (!r.ok || !j.id) throw new Error(`Resend ${r.status}: ${JSON.stringify(j).slice(0, 300)}`); + return j.id; +} + +/** A crash has to reach a human too, or the first anyone knows is that the reports quietly stopped. */ +export async function sendFailure(error: unknown, to: string, from: string, key: string | undefined, fetchImpl: typeof fetch): Promise { + if (!key) return; + try { + await fetchImpl('https://api.resend.com/emails', { + method: 'POST', + headers: { authorization: `Bearer ${key}`, 'content-type': 'application/json' }, + body: JSON.stringify({ + from, to: [to], subject: 'GitHub pulse FAILED', + text: `The daily GitHub pulse could not be produced on ${hostname()}.\n\n${(error as Error).stack ?? String(error)}\n`, + }), + }); + } catch { + // Nothing further to do: the failure is already on stderr. + } +} + +// ---------------------------------------------------------------- the run + +interface StarEntry { starred_at: string; user: { login: string } } +interface ForkEntry { full_name: string; created_at: string } +interface CommitEntry { author?: { login?: string } | null; commit?: { author?: { name?: string } } } +interface PullEntry { number: number; title: string; created_at: string; merged_at: string | null; closed_at: string | null; user?: { login?: string }; html_url: string } +interface IssueEntry { number: number; title: string; created_at: string; closed_at: string | null; user?: { login?: string }; html_url: string; pull_request?: unknown } +interface ReleaseEntry { tag_name: string; published_at: string | null; html_url: string } + +export const defaultDeps = (): RunDeps => ({ + fetch: globalThis.fetch, + now: () => new Date(), + token: ghToken, + log: (line) => process.stderr.write(`gh-pulse: ${line}\n`), + resendKey: () => process.env['RESEND_API_KEY'], + render: svgToPng, +}); + +/** + * Rasterise with @resvg/resvg-js at 2x for retina mail clients. The native + * module is CommonJS and loaded through createRequire on first use, so the TUI + * and the tests never pay for it. + */ +export function svgToPng(svg: string): Buffer { + const { Resvg } = createRequire(import.meta.url)('@resvg/resvg-js') as typeof import('@resvg/resvg-js'); + return new Resvg(svg, { fitTo: { mode: 'zoom', value: 2 }, font: { loadSystemFonts: true, defaultFontFamily: 'DejaVu Sans' } }).render().asPng(); +} + +export async function run(opt: RunOptions, deps: RunDeps = defaultDeps()): Promise { + const gh = new GitHub(deps.token(), deps.fetch); + const now = deps.now(); + const prev = latestSnapshot(opt.dataDir, deps.log); + const base = baselineFor(prev, now); + const cutoff = base ? new Date(base.at) : new Date(now.getTime() - opt.hours * 3600000); + const cutoffMs = cutoff.getTime(); + const cutoffIso = cutoff.toISOString(); + const prevRepos = new Map(Object.entries(base?.repos ?? {})); + + const me = (await gh.get<{ login: string; followers: number }>('/user')).data!; + deps.log(`${me.login}, window since ${cutoffIso}${base ? ' (previous snapshot)' : ' (clock)'}`); + + // 1. every repo the token can see, minus forks + let repos = await gh.all('/user/repos?affiliation=owner,organization_member&per_page=100&sort=pushed', { max: 30 }); + repos = repos.filter((r) => !r.fork); + if (opt.repos.length) repos = repos.filter((r) => opt.repos.includes(r.full_name)); + deps.log(`${repos.length} repos`); + + // 2. traffic for all of them: the bulk of the calls + const R = new Map(); + await Promise.all(repos.map((r) => gh.limit(async () => { + const [v, c] = await Promise.all([ + gh.get<{ views: Bucket[] }>(`/repos/${r.full_name}/traffic/views`, { allow: [403, 404] }), + gh.get<{ clones: Bucket[] }>(`/repos/${r.full_name}/traffic/clones`, { allow: [403, 404] }), + ]); + R.set(r.full_name, { repo: r, views: v.data?.views ?? null, clones: c.data?.clones ?? null, trafficOk: !!(v.data && c.data), m: emptyMovement() }); + }))); + + // 3. movement per repo + const newest = newestDay([...R.values()].flatMap((x) => [x.views, x.clones]), isoDay(now)); + const firstRunSince = isoDay(new Date(now.getTime() - 2 * 86400000)); + const needStars = new Set(); + const needForks = new Set(); + const needEvents = new Set(); + for (const [name, x] of R) { + const r = x.repo; + const p = prevRepos.get(name); + const m = x.m; + m.stars = p ? r.stargazers_count - p.stars : 0; + m.forks = p ? r.forks_count - p.forks : 0; + const dv = trafficDelta(x.views, p ? p.views : null, firstRunSince); + const dc = trafficDelta(x.clones, p ? p.clones : null, firstRunSince); + m.views = dv.count; m.uniques = dv.uniques; m.clones = dc.count; m.cloners = dc.uniques; + if (Date.parse(r.pushed_at) >= cutoffMs || Date.parse(r.updated_at) >= cutoffMs || (p && r.open_issues_count !== p.openIssues)) needEvents.add(name); + if (p ? m.stars > 0 : r.stargazers_count > 0) needStars.add(name); + if (p ? m.forks > 0 : r.forks_count > 0) needForks.add(name); + } + + // 4. detail for candidates + await Promise.all([...R.values()].map((x) => gh.limit(async () => { + const n = x.repo.full_name; + const m = x.m; + const hasBaseline = prevRepos.has(n); + if (needStars.has(n)) { + // Newest stargazers live on the LAST page. + const first = await gh.get(`/repos/${n}/stargazers?per_page=100`, { accept: 'application/vnd.github.star+json', allow: [404] }); + let page = first.data ?? []; + const last = lastPage(first.link); + if (last > 1) page = (await gh.get(`/repos/${n}/stargazers?per_page=100&page=${last}`, { accept: 'application/vnd.github.star+json' })).data ?? []; + const recent = page.filter((s) => Date.parse(s.starred_at) >= cutoffMs); + m.newStargazers = recent.map((s) => s.user.login).reverse(); + if (!hasBaseline) m.stars = recent.length; + } + if (needForks.has(n)) { + const f = await gh.get(`/repos/${n}/forks?sort=newest&per_page=30`, { allow: [404] }); + const recent = (f.data ?? []).filter((k) => Date.parse(k.created_at) >= cutoffMs); + m.newForks = recent.map((k) => k.full_name); + if (!hasBaseline) m.forks = recent.length; + } + if (needEvents.has(n)) { + const [commits, pulls, issues, releases] = await Promise.all([ + gh.all(`/repos/${n}/commits?since=${cutoffIso}&per_page=100`, { max: 3, allow: [409, 404] }), + gh.get(`/repos/${n}/pulls?state=all&sort=updated&direction=desc&per_page=60`, { allow: [404] }), + gh.all(`/repos/${n}/issues?state=all&since=${cutoffIso}&sort=updated&per_page=100`, { max: 2, allow: [404] }), + gh.get(`/repos/${n}/releases?per_page=10`, { allow: [404] }), + ]); + m.commits = commits.length; + m.authors = [...new Set(commits.map((c) => c.author?.login ?? c.commit?.author?.name).filter((a): a is string => !!a))]; + for (const pr of pulls.data ?? []) { + const ref: Ref = { n: pr.number, t: pr.title, u: pr.user?.login ?? '', url: pr.html_url }; + if (Date.parse(pr.created_at) >= cutoffMs) { m.prOpened += 1; m.openedPrs.push(ref); } + if (pr.merged_at && Date.parse(pr.merged_at) >= cutoffMs) { m.prMerged += 1; m.mergedPrs.push(ref); } + else if (pr.closed_at && Date.parse(pr.closed_at) >= cutoffMs) m.prClosed += 1; + } + for (const is of issues) { + if (is.pull_request) continue; + if (Date.parse(is.created_at) >= cutoffMs) { m.issuesOpened += 1; m.openedIssues.push({ n: is.number, t: is.title, u: is.user?.login ?? '', url: is.html_url }); } + if (is.closed_at && Date.parse(is.closed_at) >= cutoffMs) m.issuesClosed += 1; + } + for (const rel of releases.data ?? []) { + if (rel.published_at && Date.parse(rel.published_at) >= cutoffMs) { m.releases += 1; m.releaseList.push({ tag: rel.tag_name, url: rel.html_url }); } + } + } + m.score = score(m); + m.mover = isMover(m); + }))); + + // 5. referrers + popular paths, only where something moved + const movers = [...R.values()].filter((x) => x.m.mover).sort((a, b) => b.m.score - a.m.score); + await Promise.all(movers.map((x) => gh.limit(async () => { + if (!x.trafficOk) return; + const n = x.repo.full_name; + const [ref, paths] = await Promise.all([ + gh.get(`/repos/${n}/traffic/popular/referrers`, { allow: [403, 404] }), + gh.get(`/repos/${n}/traffic/popular/paths`, { allow: [403, 404] }), + ]); + x.referrers = ref.data ?? []; + x.paths = paths.data ?? []; + }))); + + // 6. followers, by name + const followers = (await gh.all<{ login: string }>('/user/followers?per_page=100', { max: 60 })).map((u) => u.login).sort(); + const prevFollowers = new Set(base?.followers ?? []); + const followerMoves = base + ? { gained: followers.filter((l) => !prevFollowers.has(l)), lost: [...prevFollowers].filter((l) => !followers.includes(l)) } + : { gained: [], lost: [] }; + + // 7. portfolio totals + const totalStars = repos.reduce((s, r) => s + r.stargazers_count, 0); + const prevTotalStars = base ? Object.values(base.repos).reduce((s, r) => s + r.stars, 0) : null; + const port: Portfolio = { views: 0, uniques: 0, clones: 0, cloners: 0, viewsByDay: new Map(), clonesByDay: new Map() }; + for (const x of R.values()) { + port.views += x.m.views; port.uniques += x.m.uniques; port.clones += x.m.clones; port.cloners += x.m.cloners; + for (const b of fourteenDays(x.views, newest)) port.viewsByDay.set(b.day, (port.viewsByDay.get(b.day) ?? 0) + b.count); + for (const b of fourteenDays(x.clones, newest)) port.clonesByDay.set(b.day, (port.clonesByDay.get(b.day) ?? 0) + b.count); + } + const trafficBlind = [...R.values()].filter((x) => !x.trafficOk).length; + + // 8. charts + const images: Image[] = []; + const addImage = (id: string, svg: string): string => { images.push({ id, buf: deps.render(svg) }); return `cid:${id}`; }; + const portDays = [...port.viewsByDay.keys()].sort(); + const portfolioImg = addImage('portfolio', chartPair( + { title: 'Views per day, all repos', color: '#2a78d6', days: portDays.map((d) => ({ day: d, count: port.viewsByDay.get(d) ?? 0 })) }, + { title: 'Clones per day, all repos', color: '#eb6834', days: portDays.map((d) => ({ day: d, count: port.clonesByDay.get(d) ?? 0 })) }, + 640, 170)); + const detailed = movers.slice(0, opt.top); + for (const x of detailed) { + if (!x.trafficOk) continue; + x.img = addImage(`r${x.repo.id}`, chartPair( + { title: 'Views', color: '#2a78d6', days: fourteenDays(x.views, newest) }, + { title: 'Clones', color: '#eb6834', days: fourteenDays(x.clones, newest) }, 640, 120)); + } + + // 9. render + const ctx: ReportContext = { + login: me.login, now, cutoff, useBaseline: !!base, newestDay: newest, + trafficLabel: base ? 'new since last run' : `GitHub days from ${firstRunSince}`, + repoCount: repos.length, movers, detailed, followers, followerMoves, totalStars, prevTotalStars, port, trafficBlind, portfolioImg, + }; + const html = renderHtml(ctx); + const text = renderText(ctx); + const out = join(opt.dataDir, 'out'); + mkdirSync(join(out, 'charts'), { recursive: true }); + writeFileSync(join(out, 'latest.html'), html.replace(/cid:([\w-]+)/g, (_, id: string) => `data:image/png;base64,${images.find((i) => i.id === id)?.buf.toString('base64') ?? ''}`)); + writeFileSync(join(out, 'latest.txt'), text); + writeFileSync(join(out, 'latest.json'), `${JSON.stringify(reportJson(ctx), null, 2)}\n`); + for (const i of images) writeFileSync(join(out, 'charts', `${i.id}.png`), i.buf); + + const subject = subjectLine(now, movers.length, prevTotalStars === null ? null : totalStars - prevTotalStars, port.views, port.clones); + + // 10. send + let sent = false; + if (opt.dryRun) { + deps.log(`dry run, wrote ${join(out, 'latest.html')} (subject: ${subject})`); + } else { + const key = deps.resendKey(); + if (!key) throw new Error('RESEND_API_KEY is not set (and not in ~/.config/logicsrc/shell.env)'); + await sendResend({ to: opt.to, from: opt.from, subject, html, text, images }, key, deps.fetch); + sent = true; + deps.log(`sent to ${opt.to}: ${subject}`); + } + + // 11. snapshot: the history. A dry run or a partial scan must not move the baseline. + let snapshotFile: string | null = null; + if (!opt.dryRun && opt.repos.length === 0) { + const snap: Snapshot = { at: now.toISOString(), user: me.login, followersCount: me.followers, followers, repos: {} }; + for (const [n, x] of R) { + const r = x.repo; + const row: RepoSnapshot = { + id: r.id, stars: r.stargazers_count, forks: r.forks_count, openIssues: r.open_issues_count, pushedAt: r.pushed_at, + private: r.private, archived: r.archived, views: x.views, clones: x.clones, + }; + if (x.referrers) row.referrers = x.referrers; + if (x.paths) row.paths = x.paths; + if (x.m.mover) row.movement = x.m; + snap.repos[n] = row; + } + snapshotFile = writeSnapshot(opt.dataDir, snap); + deps.log(`snapshot ${snapshotFile}`); + } + deps.log(`${gh.calls.made} API calls (${gh.calls.retries} retries), ${gh.remaining} remaining this hour`); + return { subject, movers: movers.length, repos: repos.length, sent, snapshot: snapshotFile, html, calls: gh.calls.made }; +} + +/** Where the last report's files are, for `show`, `open` and `--json`. */ +export function outputPaths(dir: string): { html: string; text: string; json: string } { + const out = join(dir, 'out'); + return { html: join(out, 'latest.html'), text: join(out, 'latest.txt'), json: join(out, 'latest.json') }; +} + +export function readReport(dir: string): ReportJson | null { + const p = outputPaths(dir).json; + if (!existsSync(p)) return null; + return JSON.parse(readFileSync(p, 'utf8')) as ReportJson; +} + +/** Open the last HTML report in the desktop browser, whichever opener this box has. */ +export function openInBrowser(file: string): boolean { + for (const opener of ['xdg-open', 'open', 'wslview']) { + const r = spawnSync(opener, [file], { stdio: 'ignore' }); + if (r.status === 0) return true; + } + return false; +} diff --git a/src/registry.ts b/src/registry.ts index 01dda5c..0b466d7 100644 --- a/src/registry.ts +++ b/src/registry.ts @@ -49,6 +49,7 @@ const SUMMARIES: Record = { 'gh-prs': 'Every open PR across the owners you name', 'gh-prs-fix-all': 'Repair the open scan PRs that are broken because of us', 'gh-prs-merge': 'Squash-merge the PRs that are genuinely ready', + 'gh-pulse': 'What moved on GitHub since yesterday, ranked, with traffic: by email, TUI or JSON', hqtui: 'Every vital of this box, in the terminal: sockets, HTTP, sessions, services', porkbun: 'Read and change DNS at Porkbun, and un-park a domain', shorten: 'Mint a short link on the pit, and follow it from /f/', @@ -193,6 +194,9 @@ export const PIT_ALIASES: Record = { // after `| domainfree`, so `free-names` had to become a command first. names: 'free-names', prs: 'gh-prs', + // `pulse` opens the terminal view of the last report; the scan itself is + // cron's job and stays the long spelling. + pulse: 'gh-pulse show', speak: 'tts', // The short word for `sysupdate`, and the reason that command is not itself // called `update`: `cli-tools update` already means "move this checkout", diff --git a/test/gh-pulse.test.ts b/test/gh-pulse.test.ts new file mode 100644 index 0000000..a3f58bd --- /dev/null +++ b/test/gh-pulse.test.ts @@ -0,0 +1,198 @@ +import { describe, expect, it } from 'vitest'; + +import { + baselineFor, + chartPair, + emptyMovement, + fourteenDays, + isMover, + lastPage, + movementBits, + newestDay, + nextLink, + renderHtml, + resendBody, + score, + subjectLine, + trafficBits, + trafficDelta, + type Bucket, + type Movement, + type ReportContext, + type RepoState, + type Snapshot, +} from '../src/gh-pulse.ts'; +import { historyRows } from '../src/gh-pulse-tui.ts'; + +const b = (day: string, count: number, uniques = count): Bucket => ({ timestamp: `${day}T00:00:00Z`, count, uniques }); +const moved = (patch: Partial): Movement => ({ ...emptyMovement(), ...patch }); + +describe('trafficDelta', () => { + it('counts only the growth of buckets present in the baseline, and new buckets in full', () => { + const prev = [b('2026-09-10', 5, 2), b('2026-09-11', 20, 9)]; + const cur = [b('2026-09-11', 29, 17), b('2026-09-12', 7, 3)]; + expect(trafficDelta(cur, prev, '2026-09-11')).toEqual({ count: 9 + 7, uniques: 8 + 3 }); + }); + + it('never goes negative when GitHub revises a bucket down', () => { + expect(trafficDelta([b('2026-09-11', 3)], [b('2026-09-11', 5)], '2026-09-11')).toEqual({ count: 0, uniques: 0 }); + }); + + it('without a baseline takes every bucket on or after the first-run day', () => { + const cur = [b('2026-09-09', 100), b('2026-09-11', 4), b('2026-09-12', 6)]; + expect(trafficDelta(cur, null, '2026-09-11')).toEqual({ count: 10, uniques: 10 }); + }); + + it('is zero when the repo has no traffic access', () => { + expect(trafficDelta(null, null, '2026-09-11')).toEqual({ count: 0, uniques: 0 }); + }); +}); + +describe('newestDay', () => { + it('ignores the empty buckets GitHub pads up to today', () => { + const padded = [b('2026-09-11', 12), b('2026-09-12', 0), b('2026-09-13', 0)]; + const other = [b('2026-09-10', 1)]; + expect(newestDay([padded, other, null], '2026-09-13')).toBe('2026-09-11'); + }); + + it('falls back when nothing has any traffic', () => { + expect(newestDay([[b('2026-09-12', 0)], null], '2026-09-13')).toBe('2026-09-13'); + }); +}); + +describe('fourteenDays', () => { + it('zero-fills fourteen days ending on the given day, oldest first', () => { + const days = fourteenDays([b('2026-09-11', 29, 17)], '2026-09-11'); + expect(days).toHaveLength(14); + expect(days[0]).toEqual({ day: '2026-08-29', count: 0, uniques: 0 }); + expect(days[13]).toEqual({ day: '2026-09-11', count: 29, uniques: 17 }); + }); + + it('leaves out buckets after the end day', () => { + const days = fourteenDays([b('2026-09-12', 99)], '2026-09-11'); + expect(days.every((d) => d.count === 0)).toBe(true); + }); +}); + +describe('scoring', () => { + it('weights stars above commits and caps commits', () => { + expect(score(moved({ stars: 2 }))).toBe(10); + expect(score(moved({ commits: 100 }))).toBe(25); + expect(score(moved({ prMerged: 1, prOpened: 1 }))).toBe(5); + expect(score(moved({ views: 10, uniques: 4, clones: 2, cloners: 1 }))).toBeCloseTo(2 + 4 + 1 + 1); + }); + + it('does not reward lost stars but still reports them', () => { + const m = moved({ stars: -1 }); + expect(score(m)).toBe(0); + expect(isMover(m)).toBe(true); + expect(movementBits(m)).toEqual(['-1 star']); + }); + + it('treats one clone or one visitor as background noise', () => { + expect(isMover(moved({ clones: 1, cloners: 1 }))).toBe(false); + expect(isMover(moved({ views: 3, uniques: 1 }))).toBe(false); + expect(isMover(moved({ views: 3, uniques: 2 }))).toBe(true); + expect(isMover(moved({ clones: 5, cloners: 2 }))).toBe(true); + expect(isMover(moved({ releases: 1 }))).toBe(true); + }); +}); + +describe('text bits', () => { + it('pluralises and signs', () => { + const m = moved({ stars: 3, forks: 1, commits: 1, prMerged: 2, releases: 1, views: 1, uniques: 1, clones: 2, cloners: 2 }); + expect(movementBits(m)).toEqual(['+3 stars', '+1 fork', '1 commit', '2 PRs merged', '1 release']); + expect(trafficBits(m)).toEqual(['1 view / 1 unique', '2 clones / 2 unique']); + }); + + it('writes the subject with the star delta only when there is one', () => { + const now = new Date('2026-09-13T13:05:00Z'); + expect(subjectLine(now, 12, null, 368, 1362)).toBe('GitHub pulse 2026-09-13: 12 repos moved, 368 views, 1362 clones'); + expect(subjectLine(now, 12, 4, 1, 2)).toBe('GitHub pulse 2026-09-13: 12 repos moved, +4 stars, 1 views, 2 clones'); + }); +}); + +describe('baselineFor', () => { + const snap = (at: string): Snapshot => ({ at, user: 'u', followersCount: 0, followers: [], repos: {} }); + const now = new Date('2026-09-14T13:05:00Z'); + + it('uses yesterday\'s snapshot, not one from minutes ago or last week', () => { + expect(baselineFor(snap('2026-09-13T13:05:00Z'), now)?.at).toBe('2026-09-13T13:05:00Z'); + expect(baselineFor(snap('2026-09-14T12:00:00Z'), now)).toBeNull(); + expect(baselineFor(snap('2026-09-01T12:00:00Z'), now)).toBeNull(); + expect(baselineFor(null, now)).toBeNull(); + }); +}); + +describe('link headers', () => { + it('finds the next and last pages', () => { + const link = '; rel="next", ; rel="last"'; + expect(nextLink(link)).toBe('https://api.github.com/x?page=2'); + expect(lastPage(link)).toBe(7); + expect(nextLink('')).toBeUndefined(); + expect(lastPage('')).toBe(1); + }); +}); + +describe('chartPair', () => { + it('draws one bar per day with data and labels the peak and the latest day', () => { + const days = fourteenDays([b('2026-09-05', 10), b('2026-09-11', 4)], '2026-09-11'); + const svg = chartPair({ title: 'Views', color: '#2a78d6', days }, { title: 'Clones', color: '#eb6834', days: [] }); + expect((svg.match(/10<'); + expect(svg).toContain('>4<'); + expect(svg).toContain('14 in 14d'); + expect(svg).toContain('0 in 14d'); + }); +}); + +describe('resendBody', () => { + it('sends inline images as attachments with a content id', () => { + const body = resendBody({ to: 'a@b.c, d@e.f', from: 'P ', subject: 's', html: '', text: 't', images: [{ id: 'portfolio', buf: Buffer.from('png') }] }); + expect(body['to']).toEqual(['a@b.c', 'd@e.f']); + expect(body['attachments']).toEqual([{ filename: 'portfolio.png', content: Buffer.from('png').toString('base64'), content_type: 'image/png', content_id: 'portfolio' }]); + }); +}); + +describe('renderHtml', () => { + const repo = (name: string, m: Partial): RepoState => ({ + repo: { id: 1, full_name: name, html_url: `https://github.com/${name}`, private: false, fork: false, archived: false, stargazers_count: 10, forks_count: 2, open_issues_count: 0, pushed_at: '', updated_at: '' }, + views: [b('2026-09-11', 29, 17)], clones: [b('2026-09-11', 279, 79)], trafficOk: true, + m: { ...moved(m), score: score(moved(m)), mover: true }, + referrers: [{ referrer: 'news.ycombinator.com', count: 12, uniques: 9 }], + paths: [{ path: `/${name}/blob/main/README.md`, count: 5, uniques: 4 }], + img: 'cid:r1', + }); + + it('ranks movers, embeds the charts by cid, and names the new stargazers', () => { + const movers = [repo('profullstack/nixamp', { stars: 1, newStargazers: ['octocat'], views: 29, uniques: 17, clones: 279, cloners: 79 }), repo('profullstack/scripts', { commits: 2 })]; + const ctx: ReportContext = { + login: 'ralyodio', now: new Date('2026-09-13T03:05:00Z'), cutoff: new Date('2026-09-12T03:05:00Z'), useBaseline: false, + newestDay: '2026-09-11', trafficLabel: 'GitHub days from 2026-09-11', repoCount: 349, movers, detailed: movers.slice(0, 1), + followers: ['a', 'b'], followerMoves: { gained: [], lost: [] }, totalStars: 2294, prevTotalStars: null, + port: { views: 29, uniques: 17, clones: 279, cloners: 79, viewsByDay: new Map(), clonesByDay: new Map() }, trafficBlind: 0, portfolioImg: 'cid:portfolio', + }; + const html = renderHtml(ctx); + expect(html.indexOf('profullstack/nixamp')).toBeLessThan(html.indexOf('profullstack/scripts')); + expect(html).toContain('src="cid:portfolio"'); + expect(html).toContain('src="cid:r1"'); + expect(html).toContain('Starred by { + it('sums stars over every repo but traffic only over the movers that were stored', () => { + const snap: Snapshot = { + at: '2026-09-13T03:05:36Z', user: 'u', followersCount: 2, followers: ['a', 'b'], + repos: { + 'o/a': { id: 1, stars: 5, forks: 0, openIssues: 0, pushedAt: '', private: false, archived: false, views: null, clones: null, movement: moved({ views: 3, clones: 4, mover: true }) }, + 'o/b': { id: 2, stars: 7, forks: 0, openIssues: 0, pushedAt: '', private: false, archived: false, views: null, clones: null }, + }, + }; + expect(historyRows([snap])).toEqual([{ at: snap.at, repos: 2, stars: 12, followers: 2, movers: 1, views: 3, clones: 4 }]); + }); +}); From 204d253c6cb319ea13d57c3f39722873db0380a5 Mon Sep 17 00:00:00 2001 From: Anthony Ettinger Date: Sun, 13 Sep 2026 04:33:34 +0000 Subject: [PATCH 2/2] pnpm: allow @profullstack/hqtui 0.5.1 under the release-age policy pnpm added the exclusion itself when the dependency was installed; without it a frozen-lockfile install on the runner refuses the version the lockfile pins. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_011gfb1vcQH5rf82HfbJvf1Z --- pnpm-workspace.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index ad0ee26..df8d7d3 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -7,3 +7,5 @@ # makes on purpose. allowBuilds: esbuild: true +minimumReleaseAgeExclude: + - '@profullstack/hqtui@0.5.1'