Repository navigation
release: copyright fixes, Google Play device catalog, duplicate merges (2026-10-07) - #367
Open
Seungpyo1007 wants to merge 125 commits into
Open
Seungpyo1007 wants to merge 125 commits into
Seungpyo1007 wants to merge 125 commits into
Conversation
Wikipedia lists cores only for these 23 SKUs. Each thread count is the Total Threads figure on that SKU's Intel specification page.
Adds the 23 new CPU pages. Other pages stay byte-for-byte the same; these SKUs have no benchmark scores, so CPU percentiles do not move.
Ran the existing Wikipedia image backfill tool against every smartphone record that cites a Wikipedia article and still lacks image_url. Of 432 eligible records, only 7 had a freely licensed, model-matching Commons photo in their article infobox; the rest were rejected as logo/promo images, group shots, non-free licenses, or had no infobox image at all. Closes #296
Auto-promotions from the verification layer (green+live-source or crossref-confirm). Each flip is verified:false->true only; see data/_verify/ledger.jsonl. Refs #1
The strict integrity gate (integrity_check.py --strict) has been hard-blocking weekly-refresh for at least the last 5 runs: 18 Intel Atom CPUs were each imported twice under two slugs, once without an "intel-" prefix (Wikipedia- sourced) and once with it (a later Kaggle-dataset import). These were not harmless duplicates - the two sides disagreed on release_date, threads and other fields for most pairs. Cross-checked every pair against https://en.wikipedia.org/wiki/List_of_Intel_Atom_processors: - release_date: the "atom-X" (Wikipedia) record matches the table almost exactly, often to the day. The "intel-atom-X" record's dates are frequently wrong, in two cases (Z3530, Z3560) off by a full 8 years. - threads: Wikipedia lists Hyper-Threading support for 17 of these 18 SKUs (only D2500 does not). The "intel-atom-X" record already encoded threads = 2x cores; the "atom-X" record wrongly had threads = cores for 16 of them. - clock/tdp/cache/socket/process_node: spot-checked the same way; kept whichever side matched Wikipedia (mostly the "atom-X" side, e.g. Z3560's "intel-atom-z3560" record had clock_ghz=1.0 vs Wikipedia's stated 1.83). Each merged record keeps the union of both sides' source_urls and survives under the shorter "atom-X" slug/path. `integrity_check.py --strict` now reports 0 hard anomalies. Closes #296
Correct release dates, one clock, and two TDPs where the Wikipedia rows for that exact record agree, and move changed-year files to the matching directory.
The Northwood-128 row lists 52.8 W. The CPU field is an integer, and 53 W is that value already stored.
Country is not sourced for these makers, so they live under brand/unknown/ (ADR-016). Refs #295
Identity from Google Play supported_devices.csv; RAM/SoC/GPU/screen/SDK from the Play Console device catalog where it lists the device. No release dates (unknown by design, ADR-016). Refs #295
Identity from Google Play supported_devices.csv; RAM/SoC/GPU/screen/SDK from the Play Console device catalog where it lists the device. No release dates (unknown by design, ADR-016). Refs #295
Identity from Google Play supported_devices.csv; RAM/SoC/GPU/screen/SDK from the Play Console device catalog where it lists the device. No release dates (unknown by design, ADR-016). Refs #295
Identity from Google Play supported_devices.csv; RAM/SoC/GPU/screen/SDK from the Play Console device catalog where it lists the device. No release dates (unknown by design, ADR-016). Refs #295
…es (2003–2017) Host: cdn2.gsmarena.com. GSMArena is citation-correction only (ADR-013). These URLs hotlinked product photos from a third-party site we hold no license for, and the site rendered them on public device pages. Only Wikimedia Commons files (with image_license/image_attribution) are kept; the nulled records stay eligible for a later Commons photo backfill. Refs #296
…msung +42 (2009–2025) Host: www.mobiledokan.com. These URLs hotlinked product photos from a third-party site we hold no license for, and the site rendered them on public device pages. Only Wikimedia Commons files (with image_license/image_attribution) are kept; the nulled records stay eligible for a later Commons photo backfill. Refs #296
…msung/huawei +50 (2003–2025) Host: aitoolbuzz.com, which republishes photos copied from other sites. These URLs hotlinked product photos from a third-party site we hold no license for, and the site rendered them on public device pages. Only Wikimedia Commons files (with image_license/image_attribution) are kept; the nulled records stay eligible for a later Commons photo backfill. Refs #296
…r–huawei (1994–2025) Host: aitoolbuzz.com, which republishes photos copied from other sites. These URLs hotlinked product photos from a third-party site we hold no license for, and the site rendered them on public device pages. Only Wikimedia Commons files (with image_license/image_attribution) are kept; the nulled records stay eligible for a later Commons photo backfill. Refs #296
…ate–o2 (1995–2025) Host: aitoolbuzz.com, which republishes photos copied from other sites. These URLs hotlinked product photos from a third-party site we hold no license for, and the site rendered them on public device pages. Only Wikimedia Commons files (with image_license/image_attribution) are kept; the nulled records stay eligible for a later Commons photo backfill. Refs #296
…plus–sonim (1997–2025) Host: aitoolbuzz.com, which republishes photos copied from other sites. These URLs hotlinked product photos from a third-party site we hold no license for, and the site rendered them on public device pages. Only Wikimedia Commons files (with image_license/image_attribution) are kept; the nulled records stay eligible for a later Commons photo backfill. Refs #296
…y–zte (1997–2026) Host: aitoolbuzz.com, which republishes photos copied from other sites. These URLs hotlinked product photos from a third-party site we hold no license for, and the site rendered them on public device pages. Only Wikimedia Commons files (with image_license/image_attribution) are kept; the nulled records stay eligible for a later Commons photo backfill. Refs #296
REDMAGIC (global.redmagic.gg), StarFive (starfivetech.com) and Ulefone (cdn.shopify.com) linked their logos on company and shop CDN hosts we hold no license for. Wikidata lists no logo (P154) for any of them, and the Commons files for REDMAGIC and Ulefone are recent "own work" uploads of the company marks, so they are not used either. Every remaining logo_url is a Wikimedia Commons file. Refs #296
image_url and logo_url now point only at Wikimedia Commons files. State that those files are not covered by the dataset's CC BY-SA 4.0 grant and how to credit them. Refs #296
data/_verify/cache/wikipedia_html/ held 325 full Wikipedia article pages (61 MB), several unrelated to any device. TechEngine's wikipedia_smartphone_backfill writes this page cache during apply runs, and it was swept into 0bb442a by accident. Untrack it and ignore the cache directory so it stays local. Refs #296
The Commons file Imgi-17-samsung-galaxy-tab-s11-ultra-galaxy-ai-scaled.webp names its source as "Samsung Website" and its author as Samsung; the CC0 tag is the uploader's, not Samsung's grant. Remove the three image fields the backfill inserted and record a bad_license decision in its cache so a later run does not put the file back. Refs #296
Mirror of the engine rule (TechEngine ADR-017): reject any image_url outside upload.wikimedia.org/wikipedia/commons at any depth, require image_license and image_attribution next to a photo, and accept only Commons files for logo_url. The data in this PR already satisfies it. Refs #296
… page Device cards rendered any image_url, which put hotlinked third-party product photos on the public site. Render only upload.wikimedia.org/wikipedia/commons files, and add a 'Photo: Wikimedia Commons' credit linking to the file page, which lists the author and license (CC BY-SA allows attribution by link). Refs #19
Smartphone pages already carry image_attribution and image_license, so the credit reads 'Photo: <author>, <license> · Wikimedia Commons' and still links to the file page; records without those fields keep the plain Commons credit. Refs #19
Member
Author
🔎 Data verification — Tier 0 (offline existence/trust)Scored by 13/13 categories ? 213,649 records ? diff base main@d3aac35 Changed records in this PR76603 record(s) assessed. %%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
title Verification bands — all records
"Green" : 14325
"Yellow" : 62224
"Red" : 54
Full-dataset baselineAs of 2026-10-06T12:12:50Z (daily
green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to |
Member
Author
TechEngine change review: PASS13/13 categories ? 213,649 records ? diff base main@d3aac35
Changed data
Changed record examplesbrand added
brand modified
soc modified
smartphone added
smartphone modified
smartphone deleted
tablet modified
tablet deleted
watch modified
watch deleted
pda modified
pda deleted
gpu added
gpu modified
cpu added
cpu modified
cpu deleted
laptop modified
laptop deleted
monitor modified
monitor deleted
website modified
device_catalog added
Heuristic review
Changed site
Modified site files
|
Member
Author
TechEngine validation stats: PASS13/13 categories ? 213,649 records ? diff base main@d3aac35
Data summary
Warning Tracked verified coverage is below 50% for device_catalog 0.0% (0/20366), software 0.8% (358/42493), brand 4.1% (123/3022), laptop 6.6% (129/1966), tablet 8.9% (293/3305), monitor 9.1% (21/231), soc 13.4% (284/2117), all 18.3% (39048/213649), and 3 more. Validation notes
Key output:
Homepage build: |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 2026-10-07
Everything merged into
developsince the 2026-09-27 release (#315).Copyright fixes
data/LICENSE.md, and validates thatimage_url/logo_urlonly link Wikimedia Commons files.Google Play identity data
device_catalogentries and the device makers they reference (ADR-016).Accuracy
New records and citations
CI
develop.site/public/v1was regenerated bydump-developafter #365.Closes #1