Skip to content

data(verify): promote 761 CPUs, 6 GPUs and 5 SoCs with a live cited source - #311

Merged
Seungpyo1007 merged 1 commit into
developfrom
verify/promote-local-20260927
Sep 27, 2026
Merged

Seungpyo1007 merged 1 commit into
developfrom
verify/promote-local-20260927

Conversation

@Seungpyo1007

Copy link
Copy Markdown
Member

What & why

772 verified promotions (CPU 761, GPU 6, SoC 5) for records that were green but never had their own citation checked. Two check-urls bugs caused this, both now fixed: the frontier starved lower-ranked records (GetTechAPI/TechEngine#104), and dedupe ignored the ?cpu= query and cached #fragment citations only once (GetTechAPI/TechEngine#105). Closes #296 (link only).

Produced by app.verify check-urls --category cpu gpu soc and promote --apply with the fixed TechEngine code. Each data change flips only verified. The ledger and URL cache are updated as in the automated runs.

cpubenchmark.net returns 404 for unknown CPU names and near-misses (Opteron 2219, Athlon II X2 266, Core i7-7501U all 404), so a live page confirms the exact SKU.

Not promoted: 415 SoC citations to phonedb.net returned connection errors (the host blocks automated requests).

Checklist

  • The PR targets develop
  • python -m app.validate passes locally
  • Only verified changes in data files

…ource

These records were green but never had their own citation checked (check-urls
frontier and dedupe bugs, fixed in GetTechAPI/TechEngine#104 and #105). Each now
has a live Tier-1/Tier-2 citation; cpubenchmark.net returns 404 for unknown or
near-miss CPU names, so a live page confirms the exact SKU.

Refs #296
@TechEngineBot

Copy link
Copy Markdown
Member

🔎 Data verification — Tier 0 (offline existence/trust)

Scored by app.verify; posted by TechEngineBot. Informational only — the structural gate (app.validate) is separate and authoritative for merge.

12/12 categories ? 191,814 records ? diff base develop@a1a509d

Changed records in this PR

772 record(s) assessed.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 772
    "Yellow" : 0
    "Red" : 0
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
soc 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 5 5 0 0 100.0%
gpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 6 6 0 0 100.0%
cpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 761 761 0 0 100.0%
All 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 772 772 0 0 100.0%

Full-dataset baseline

191814 record(s) assessed.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 32564
    "Yellow" : 158195
    "Red" : 1055
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
brand 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨 207 122 85 0 58.9%
soc 🟩🟩🟨🟨🟨🟨🟨🟥🟥🟥🟥🟥 2117 284 837 996 13.4%
smartphone 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 93407 18003 75350 54 19.3%
tablet 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 3459 292 3166 1 8.4%
watch 🟩🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟥 447 94 350 3 21.0%
pda 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨 140 83 57 0 59.3%
gpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨 2076 1529 547 0 73.7%
cpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟥 4507 4211 295 1 93.4%
laptop 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 1995 107 1888 0 5.4%
monitor 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 882 0 882 0 0.0%
software 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 42493 216 42277 0 0.5%
website 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 40084 7623 32461 0 19.0%
All 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 191814 32564 158195 1055 17.0%

Hard violations (forced red):

Count Check
1 !boost_ge_base

green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to verified runs in the scheduled verify-network workflow.

@Seungpyo1007
Seungpyo1007 merged commit c6cf3b6 into develop Sep 27, 2026
5 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in TechAPI-Project Sep 27, 2026
@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine change review: PASS

12/12 categories ? 191,814 records ? diff base develop@a1a509d

Check Result
python -m app.validate PASS
Changed dump JSON and manifest/index counts PASS
New hard integrity anomalies vs PR base PASS

Changed data

Category Added Modified Deleted Added verified Added unverified Added Kaggle-sourced
brand 0 0 0 0 0 0
soc 0 5 0 0 0 0
smartphone 0 0 0 0 0 0
tablet 0 0 0 0 0 0
watch 0 0 0 0 0 0
pda 0 0 0 0 0 0
gpu 0 6 0 0 0 0
cpu 0 761 0 0 0 0
laptop 0 0 0 0 0 0
monitor 0 0 0 0 0 0
software 0 0 0 0 0 0
website 0 0 0 0 0 0

Changed record examples

soc modified

  • soc/hisilicon/2014/hisilicon-kirin-910t.json - HiSilicon Kirin 910T
  • soc/huawei/2020/kirin-9000e.json - Kirin 9000E
  • soc/mediatek/2021/dimensity-900.json - Dimensity 900
  • soc/mediatek/2021/dimensity-920.json - Dimensity 920
  • soc/mediatek/2022/mediatek-dimensity-8000-mt6895.json - MediaTek Dimensity 8000 MT6895

gpu modified

  • gpu/amd/2016/consumer/radeon-rx-470d.json - Radeon RX 470D
  • gpu/amd/2016/enterprise/radeon-instinct-mi8.json - Radeon Instinct MI8
  • gpu/amd/2017/enterprise/radeon-instinct-mi25.json - Radeon Instinct MI25
  • gpu/amd/2018/enterprise/radeon-instinct-mi50.json - Radeon Instinct MI50
  • gpu/nvidia/2006/enterprise/quadro-nvs-440-pcie-x1.json - Quadro NVS 440 PCIe x1
  • gpu/nvidia/2006/enterprise/quadro-nvs-440-pcie-x16.json - Quadro NVS 440 PCIe x16

cpu modified

  • cpu/amd/2009/enterprise/amd-opteron-2218.json - AMD Opteron 2218
  • cpu/amd/2009/enterprise/amd-opteron-2220-se.json - AMD Opteron 2220 SE
  • cpu/amd/2009/enterprise/amd-opteron-2220.json - AMD Opteron 2220
  • cpu/amd/2009/enterprise/amd-opteron-2356.json - AMD Opteron 2356
  • cpu/amd/2009/enterprise/amd-opteron-2378.json - AMD Opteron 2378
  • cpu/amd/2009/enterprise/amd-opteron-2384.json - AMD Opteron 2384
  • cpu/amd/2009/enterprise/amd-opteron-2431.json - AMD Opteron 2431
  • cpu/amd/2009/enterprise/amd-opteron-280.json - AMD Opteron 280
  • cpu/amd/2009/enterprise/amd-opteron-285.json - AMD Opteron 285
  • cpu/amd/2009/enterprise/amd-opteron-290.json - AMD Opteron 290
  • cpu/amd/2010/consumer/amd-athlon-5200-dual-core.json - AMD Athlon 5200 Dual-Core
  • cpu/amd/2010/consumer/amd-athlon-64-x2-dual-core-3400-plus.json - AMD Athlon 64 X2 Dual Core 3400+
  • cpu/amd/2010/consumer/amd-athlon-dual-core-4850b.json - AMD Athlon Dual Core 4850B
  • cpu/amd/2010/consumer/amd-athlon-ii-170u.json - AMD Athlon II 170u
  • cpu/amd/2010/consumer/amd-athlon-ii-x2-210e.json - AMD Athlon II X2 210e
  • ... 746 more

Heuristic review

  • Heuristic warnings: none found.

@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine validation stats: PASS

12/12 categories ? 191,814 records ? diff base develop@a1a509d

Data summary

Category Total Verified Unverified Missing verified Tracked Verified % of tracked
brand 207 123 84 0 207 59.4%
soc 2117 277 1840 0 2117 13.1%
smartphone 93407 17576 75831 0 93407 18.8%
tablet 3459 288 3171 0 3459 8.3%
watch 447 94 353 0 447 21.0%
pda 140 74 66 0 140 52.9%
gpu 2076 1490 586 0 2076 71.8%
cpu 4507 4210 297 0 4507 93.4%
laptop 1995 105 1890 0 1995 5.3%
monitor 882 0 882 0 882 0.0%
software 42493 216 42277 0 42493 0.5%
website 40084 7585 32499 0 40084 18.9%
all 191814 32038 159776 0 191814 16.7%

Warning

Tracked verified coverage is below 50% for monitor 0.0% (0/882), software 0.5% (216/42493), laptop 5.3% (105/1995), tablet 8.3% (288/3459), soc 13.1% (277/2117), all 16.7% (32038/191814), smartphone 18.8% (17576/93407), website 18.9% (7585/40084), and 1 more.
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

  • The integrity gate blocks only hard anomalies introduced by this PR relative to its merge base.
  • Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
  • Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py (PR head compared with PR base)
loaded CPU=4507 GPU=2076
Integrity section Flagged lines
structural 18
CPU name/tier consistency (desktop mainstream only) 0
CPU single>multi (cinebench/geekbench — should be multi>=single) 0
CPU era-vs-score outliers 8
CPU cross-source ratio outliers (possible wrong-variant) 131
GPU cross-source ratio outliers + sanity 51

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data Dataset changes enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Data accuracy: corrections, duplicates and dates (workstream)

2 participants