Repository navigation
ci: run TechEngine's collectors by hand and verify in final-check - #381
Merged
Merged
Conversation
collect.yml (workflow_dispatch only, on develop) calls TechEngine's ingest, refresh and verify-promote workflows, pinned by SHA. Each task keeps one open PR into develop on a fixed collect/* branch, updated in place, from github-actions[bot]. Nothing is scheduled and nothing posts comments; the collector validates before it pushes and writes the result into the PR body. final-check also runs the engine's Tier 0-3 report on the records a PR changes and puts it in the one TechEngineBot comment, with mentions and cross-references neutralised. pr-checks no longer skips the old bot branch names. TechEngine moves from 5577b3f to 3c50cb8: crawlers identify as TechEngine-verify and anti-bot challenges no longer count as alive (TechEngine #127), and photos and logos must be Wikimedia Commons files, with image credits served for every photo category (TechEngine #124, ADR-018). Refs #297
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
The user asked to keep collection and verification, with as few comments as possible: either a PR generator, or one final verification comment and nothing more. This PR does both.
Collection becomes a manual PR generator.
collect.yml:workflow_dispatchonly and runs ondeveloponly. It calls one of TechEngine's reusable workflows, pinned by SHA:ingest(category, limit, include_drafts),refresh(sleep), orpromote(apply, max_urls, max_wikidata, max_crossref).develop, on a fixed branch:collect/ingest-<category>,collect/refreshorcollect/promote. Every run force-pushes that branch and edits the PR body in place. If nothing changed, no PR is opened.GITHUB_TOKEN. No personal or bot-account token is involved, nothing runs on a schedule, and nothing posts comments.app.validateandintegrity_check.py --strictbefore it pushes. The result and the Tier 0 report go in the PR body, which is capped below GitHub's limit and scrubbed so it can't mention anyone.dataandenhancement, the Massive milestone, andCloses #295/#296.Verification goes into the one final-check comment. When a maintainer adds
final-check, the check job now also runspython -m app.verify pr --base <base>:TechEngine-verifyUser-Agent.The report goes into TechEngineBot's single comment under a
<details>block.@is removed, and a zero-width space breaks#123,GH-123and links, so the comment can't mention anyone or cross-reference an issue. It is informational: the pass/fail verdict still comes from the build.Engine pin. TechEngine moves from
5577b3fto3c50cb8(TechEngine main):TechEngine-verify/0.1 (+https://github.com/GetTechAPI/TechEngine)instead of a Chrome User-Agent. An anti-bot challenge counts as "unknown" and can no longer promote a record. The DuckDuckGo and Bing scraping is gone. ingest, refresh and verify-promote became the reusable PR generators above, and the five old bot workflows were deleted.imageslist must be Wikimedia Commons files. The API servesimage_license/image_attributionfor tablets, watches, PDAs, laptops and monitors.openapi.jsondeclares the data license, CC BY-SA 4.0.Also: pr-checks no longer skips the old bot branch names (
refresh/,ingest/,verify/promote-,dump-refresh/). The README describescollect.ymland the verification in final-check.Closes #297
Before the first run
Opening a PR with
GITHUB_TOKENneeds Settings → Actions → General → "Allow GitHub Actions to create and approve pull requests". I'll turn it on when this merges. The default token stays read-only, and nothing here approves PRs.Checks
3c50cb8, the engine validator passes over all of develop.python -m app.verify pr --base <sha>locally on the new engine. It exits 0 and prints the Tier 0-3 markdown.@,#12,GH-7and URLs are broken up.3c50cb8are byte-identical to the version I reviewed in fix: honest crawler User-Agent; collection as manual PR generators TechEngine#127.Checklist
develop(mainonly moves through release PRs)projectscope, and Copilot is out of quota, so it was reviewed by hand.