Foundry script fixes, consolidation, image + CI cleanups - #1945
Merged
Conversation
Three confirmed issues across the six script/foundry-*.ts scripts: - parseJson fence-truncation: classify/fix/qa used an unanchored ```-fence regex that mis-extracts and truncates JSON when a string value contains a literal code fence (e.g. fix.ts quoting a post's code block verbatim in a find/replace pair). engagement/interlinks had no fence handling at all. All now use drafts.ts's anchored parseJson, which tries whole-string JSON first and only matches a fence anchored to the string ends. - Missing backoff in foundry-drafts.ts: its chat() did a single fetch with no retry, while every sibling backs off on 429/5xx. Ported the qa.ts-style network + 429/5xx backoff honoring Retry-After. - Inconsistent loadCorpus() filtering: drafts.ts did not skip redirect_to stubs, so a post could process in drafts but skip in qa/engagement/interlinks. Aligned to skip published:false, archived, and redirect_to. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The six script/foundry-*.ts tools all talk to the same Azure AI Foundry
surface and had independently reimplemented (and drifted on) the transport,
JSON parsing, CLI parsing, corpus loading, and concurrency. Extract that
shared surface into script/lib/foundry.ts and import it everywhere:
- createFoundryClient({endpoint,apiKey,model,dryRun}) → {chat, usage}, with a
single network + 429/5xx backoff (honoring Retry-After) and a live usage
counter, replacing six divergent chat() implementations.
- parseJson (the anchored, fence-safe version), pool, hasFlag/flagVal,
chatEndpoint, isReasoningModel, estCost.
- loadPosts(root, {skipEmpty}) — single source of truth for the
published/archived/redirect_to live-post filter; each script maps the base
shape to its own richer Post.
Behavior is preserved for every script's default model (verified by a
request-body audit across all call sites). Three intentional normalizations,
each only reachable via a non-default --model override:
- fix/classify now send max_completion_tokens (not max_tokens) for
non-reasoning models, matching the other four scripts and the modern Azure
param.
- drafts' reasoning-model completion floor rises from 8000 to 16000, matching
qa/fix (default model gpt-4.1 is unaffected).
- fix/classify now honor Retry-After; engagement/interlinks go from 6 retries
to 5. failOnLength stays on only for qa/fix/classify.
Net ~570 lines of duplication removed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 30 legacy images under public/wp-content/uploads (referenced by old posts as raw <img> tags) were serving uncompressed. Re-encoded in place with Sharp — JPEG via mozjpeg q80, PNG via palette quantization q90 — stripping metadata and keeping each result only where it shrank. Dimensions are unchanged, so the existing remote-image-dimensions sizing is unaffected. 44% smaller overall (540K saved). Quality spot-checked on the highest-risk files (a text-heavy JPEG diagram, a dense multi-color logo PNG, and an anti-aliased text screenshot) — visually identical to the originals. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
build-and-deploy.yml ran `npm run build` → `wrangler deploy` on every push to main with no test gate: ci.yml (astro check, vitest, lint) runs as a separate workflow on the same push and cannot block the deploy job, and `needs:` only works within a single workflow file. Add `npm run check` and `npm run test:vitest` as steps right after install — before the expensive build/Chromium steps — so a type error or failing test fails the job and aborts before wrangler deploy. In-job checks (not a workflow_run trigger) are the right fit here: ci.yml is path-filtered and this workflow is not, so a public-only change (e.g. an image update) still builds and deploys while remaining gated on tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add intFlag() to script/lib/foundry.ts and use it for every numeric flag (--top, --concurrency, --candidates, --max-iters) across the foundry scripts. Previously `parseInt(flagVal(...), 10)` on a bad value (e.g. `--concurrency abc`) silently yielded NaN and flowed into slice()/pool(); now it exits with a clear "must be a positive integer" error. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| function assembleMarkdown(title: string, description: string, body: string): string { | ||
| body = stripEchoedFrontmatter(body); | ||
| // Deterministic, valid front matter — the model supplies prose, not YAML. | ||
| const esc = (s: string) => `"${s.replace(/"/g, '\\"')}"`; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Codebase-improvement sweep from an open-ended review, scoped to four verified, self-contained wins. Each is an independent, revertable commit.
1. Foundry script correctness bugs (
efa7497)classify/fix/qaused an unanchored ```-fence regex that mis-extracts and truncates JSON when a string value contains a literal code fence (e.g.fix.tsquoting a post's code block in a find/replace pair). `engagement`/`interlinks` had no fence handling. All now use the anchored, fence-safe parser.foundry-drafts.ts(single fetch, no retry) — ported the qa-style network + 429/5xx backoff honoringRetry-After.loadCorpus()filtering —draftsdidn't skipredirect_tostubs; now aligned across all scripts.2. Consolidate plumbing into
script/lib/foundry.ts(bac613e)Extracted the shared client (
createFoundryClientwith unified backoff + live usage counter),parseJson,pool, CLI helpers,loadPosts(single source of truth for the published/archived/redirect_to filter), andestCost. ~570 lines of duplication removed. Behavior preserved for every script's default model — verified by a request-body audit across all call sites. Three intentional normalizations, each only reachable via a non-default--modeloverride (documented in the commit).3. Recompress legacy images (
e5c8c76)30 raw images under
public/wp-content/uploadsre-encoded in place with Sharp (mozjpeg q80 / PNG palette q90, metadata stripped, kept only where smaller). 1.3M → 776K (−44%), dimensions unchanged. Quality spot-checked on the highest-risk files.4. Gate deploy on tests (
ddc0cd6)build-and-deploy.ymldeployed on every push with no test gate (ci.ymlruns in a separate workflow and can't block it;needs:only works within one file). Addednpm run check+npm run test:vitestas fail-fast steps beforewrangler deploy. In-job (notworkflow_run) becauseci.ymlis path-filtered and this workflow isn't, so public-only changes still deploy while staying gated.Plus a small hardening (
3672ff2): anintFlagvalidator so bad numeric flags (--concurrency abc) error cleanly instead of passingNaN.Verification
npm run check: 0 errors ·npm run test:vitest: 1919/1919 passnoUnusedLocals)🤖 Generated with Claude Code