feat(ingest): weekly crawler that opens PRs to TechAPI with new SKUs #2

Purpose

Long-running TechEngine tracker for finding missing TechAPI records, drafting import candidates, requesting PR validation, and keeping TechAPI data PRs reviewable.

This issue is the TechEngine-side companion to GetTechAPI/TechAPI#1. It should stay open while the dataset rebuild continues. TechAPI PRs may use Closes GetTechAPI/TechAPI#1 for Development linking, but this TechEngine issue tracks the automation and validation work behind those PRs.

Current Status

• State: active TechEngine automation / validation tracker
• Companion TechAPI tracker: TechAPI dataset roadmap and status: all categories (1989-2026) TechAPI#1
• Current TechAPI data PR: data(smartphone): import 5000 PhoneDB raw variants TechAPI#25
• Current active import branch: data/import-staging
• Current active batch: +5,045 smartphone records from Kaggle / PhoneDB-derived sources
• TechEngine role: validate the PR, summarize changed data, report dataset stats, and warn about low verified coverage without blocking bulk imports
• Import policy: new bulk records should remain verified: false until manual audit or follow-up verification confirms them

Latest Dataset Snapshot From PR #25

TechEngine Responsibilities

• Maintain coverage reports that show missing CPU, GPU, smartphone, SoC, and brand records
• Generate or support import batches where source coverage is useful
• Validate TechAPI PRs from reusable branches such as data/import-staging and feat/site
• Post two useful PR comments:
  • Changed-data review: what changed, examples, source/verified counts, and heuristic warnings
  • Validation stats: totals, verified coverage, warning callouts, and key command output
• Keep validation warnings actionable without failing expected bulk-import cases
• Add site-build verification only when site/ files changed
• Keep project metadata, priority, labels, milestones, assignees, and issue linkage filled in automatically where possible

Validation Contract

Data PR validation should include:

Recent Linked Work

Remaining Work

• Keep coverage reports useful by reducing obvious table-artifact slugs and source noise
• Expand automated candidate generation beyond CPU/GPU into smartphone and SoC sources where structured data is reliable
• Continue improving PR comments so reviewers can see what changed without opening thousands of files
• Make low verified coverage warnings clear and category-specific
• Keep issue and project metadata synchronized for TechAPI and TechEngine PRs
• Preserve weekly coverage workflows unless a separate daily/PR workflow is explicitly added
• Avoid closing this tracker until the TechAPI dataset rebuild and supporting automation are mature

Operational Notes

• Assignees: @Seungpyo1007 and @TechEngineBot
• Labels: enhancement
• Milestone: Daily automation
• Project: TechEngine work
• Priority: High
• Start date: 2026-05-29
• Target date: 2026-09-30

TechEngineBot should comment on relevant PRs and update linked tracker issues whenever validation or metadata automation runs.