Skip to content

fix(data): backfill Commons photos for 7 Wikipedia-cited smartphones - #317

Merged
Seungpyo1007 merged 2 commits into
developfrom
fix/smartphone-wikipedia-images
Sep 28, 2026
Merged

Seungpyo1007 merged 2 commits into
developfrom
fix/smartphone-wikipedia-images

Conversation

@Seungpyo1007

Copy link
Copy Markdown
Member

What & why

Ran the existing Wikipedia image backfill tool (app/verify/wikipedia_image_backfill.py) against every smartphone record that cites a Wikipedia article and still lacks image_url.

Of 432 eligible records, only 7 had a freely licensed, model-matching Commons photo in their article's infobox:

Device License
HTC Butterfly CC-BY-SA-3.0
Samsung Galaxy S WiFi 5.0 (3 variants) CC-BY-SA-3.0
Samsung Galaxy A57 5G CC-BY-SA-4.0
Toshiba/Fujitsu Windows Phone IS12T CC-BY-SA-4.0
Vivo Y31d CC0-1.0

The rest were rejected by the tool's existing filters: logo/promo/group images, non-free licenses, or no infobox image at all — nothing was guessed or force-matched.

Closes #296

Last commit is the static dump refresh (7 pages changed, matching the 7 records — confirms the dump stayed deterministic).

Ran the existing Wikipedia image backfill tool against every smartphone
record that cites a Wikipedia article and still lacks image_url. Of 432
eligible records, only 7 had a freely licensed, model-matching Commons
photo in their article infobox; the rest were rejected as logo/promo
images, group shots, non-free licenses, or had no infobox image at all.

Closes #296
@github-actions github-actions Bot added bug Something isn't working data Dataset changes enhancement New feature or request labels Sep 28, 2026
@TechEngineBot

Copy link
Copy Markdown
Member

🔎 Data verification — Tier 0 (offline existence/trust)

Scored by app.verify; posted by TechEngineBot. Informational only — the structural gate (app.validate) is separate and authoritative for merge.

12/12 categories ? 191,814 records ? diff base develop@48f5b11

Changed records in this PR

7 record(s) assessed.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 5
    "Yellow" : 2
    "Red" : 0
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
smartphone 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨 7 5 2 0 71.4%
All 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨 7 5 2 0 71.4%

Full-dataset baseline

191814 record(s) assessed.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 32764
    "Yellow" : 157995
    "Red" : 1055
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
brand 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨 207 122 85 0 58.9%
soc 🟩🟩🟨🟨🟨🟨🟨🟥🟥🟥🟥🟥 2117 284 837 996 13.4%
smartphone 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 93407 18003 75350 54 19.3%
tablet 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 3459 292 3166 1 8.4%
watch 🟩🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟥 447 94 350 3 21.0%
pda 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨 140 83 57 0 59.3%
gpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨 2076 1529 547 0 73.7%
cpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟥 4507 4211 295 1 93.4%
laptop 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 1995 107 1888 0 5.4%
monitor 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 882 0 882 0 0.0%
software 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 42493 366 42127 0 0.9%
website 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 40084 7673 32411 0 19.1%
All 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 191814 32764 157995 1055 17.1%

Hard violations (forced red):

Count Check
1 !boost_ge_base

green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to verified runs in the scheduled verify-network workflow.

@Seungpyo1007
Seungpyo1007 merged commit 2c2c152 into develop Sep 28, 2026
5 checks passed
@Seungpyo1007
Seungpyo1007 deleted the fix/smartphone-wikipedia-images branch September 28, 2026 06:34
@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine change review: PASS

12/12 categories ? 191,814 records ? diff base develop@48f5b11

Check Result
python -m app.validate PASS
Changed dump JSON and manifest/index counts PASS
New hard integrity anomalies vs PR base PASS

Changed data

Category Added Modified Deleted Added verified Added unverified Added Kaggle-sourced
brand 0 0 0 0 0 0
soc 0 0 0 0 0 0
smartphone 0 7 0 0 0 0
tablet 0 0 0 0 0 0
watch 0 0 0 0 0 0
pda 0 0 0 0 0 0
gpu 0 0 0 0 0 0
cpu 0 0 0 0 0 0
laptop 0 0 0 0 0 0
monitor 0 0 0 0 0 0
software 0 0 0 0 0 0
website 0 0 0 0 0 0

Changed record examples

smartphone modified

  • smartphone/htc/2012/butterfly.json - Butterfly
  • smartphone/samsung/2011/galaxy-s-wifi-5-0.json - Galaxy S WiFi 5.0
  • smartphone/samsung/2011/galaxy-s-wifi-5-0/samsung-galaxy-s-wifi-5-0-allphones-7365-16gb-16gb.json - Samsung Galaxy S WiFi 5.0
  • smartphone/samsung/2011/galaxy-s-wifi-5-0/samsung-galaxy-s-wifi-5-0-beridzeg45-589-16gb.json - Samsung Galaxy S WiFi 5.0
  • smartphone/samsung/2026/galaxy-a57-5g.json - Galaxy A57 5G
  • smartphone/toshiba/2011/windows-phone-is12t.json - Windows Phone IS12T
  • smartphone/vivo/2026/y31d.json - Y31d

Heuristic review

  • Heuristic warnings: none found.

@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine validation stats: PASS

12/12 categories ? 191,814 records ? diff base develop@48f5b11

Data summary

Category Total Verified Unverified Missing verified Tracked Verified % of tracked
brand 207 123 84 0 207 59.4%
soc 2117 280 1837 0 2117 13.2%
smartphone 93407 17581 75826 0 93407 18.8%
tablet 3459 288 3171 0 3459 8.3%
watch 447 94 353 0 447 21.0%
pda 140 74 66 0 140 52.9%
gpu 2076 1493 583 0 2076 71.9%
cpu 4507 4211 296 0 4507 93.4%
laptop 1995 105 1890 0 1995 5.3%
monitor 882 0 882 0 882 0.0%
software 42493 358 42135 0 42493 0.8%
website 40084 7635 32449 0 40084 19.0%
all 191814 32242 159572 0 191814 16.8%

Warning

Tracked verified coverage is below 50% for monitor 0.0% (0/882), software 0.8% (358/42493), laptop 5.3% (105/1995), tablet 8.3% (288/3459), soc 13.2% (280/2117), all 16.8% (32242/191814), smartphone 18.8% (17581/93407), website 19.0% (7635/40084), and 1 more.
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

  • The integrity gate blocks only hard anomalies introduced by this PR relative to its merge base.
  • Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
  • Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py (PR head compared with PR base)
loaded CPU=4507 GPU=2076
Integrity section Flagged lines
structural 18
CPU name/tier consistency (desktop mainstream only) 0
CPU single>multi (cinebench/geekbench — should be multi>=single) 0
CPU era-vs-score outliers 8
CPU cross-source ratio outliers (possible wrong-variant) 131
GPU cross-source ratio outliers + sanity 51

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working data Dataset changes enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Data accuracy: corrections, duplicates and dates (workstream)

2 participants