Skip to content

data: add Google Play model numbers and codenames to 31.9k device records - #359

Merged
Seungpyo1007 merged 11 commits into
developfrom
data/play-model-ids
Oct 6, 2026
Merged

Seungpyo1007 merged 11 commits into
developfrom
data/play-model-ids

Conversation

@Seungpyo1007

@Seungpyo1007 Seungpyo1007 commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Summary

Adds Google Play identifiers to existing device records: 31,903 files across smartphone / tablet / watch get model_numbers and codenames from Google Play's public supported_devices.csv.

  • Identifiers only. No spec values change. Existing ids are kept and unioned, and the Play CSV is appended to source_urls.
  • The first commit mirrors TechEngine feat(game): stream validation for game metadata #121 into app/validate.py: the new id fields, optional battery/weight/brand country, and device_catalog.

How matches were accepted

The staged matcher went through three sampled audit rounds:

  • model-number match (0.99)
  • codename match (0.97)
  • exact normalised name (0.95)
  • brand-checked Play Console join with SoC equivalence
  • family/qualifier guard

Only two groups are applied here:

  • auto-accept (4,272 groups), audited at 97%;
  • review rows promoted by a strict rule (1,538 groups): code-like token, no SoC conflict, token present in every matched path.

Everything else (major-brand new devices, conflicts, possible duplicates, remaining review rows) stays in a hold queue and is not part of this PR.

Policy: TechEngine docs/SPEC.md §9.2 + ADR-016 (TechEngine #122).

Refs #296

Closes #295

…ight/brand country and device_catalog

Mirrors TechEngine #121 into the self-validate subset so data PRs using the new fields pass.

Refs #296
…names for samsung

6087 records get model_numbers/codenames from Google Play supported_devices.csv (ids only; no spec value changes).
Brands: samsung

Refs #296
…names for xiaomi

3223 records get model_numbers/codenames from Google Play supported_devices.csv (ids only; no spec value changes).
Brands: xiaomi

Refs #296
…for motorola

2372 records get model_numbers/codenames from Google Play supported_devices.csv (ids only; no spec value changes).
Brands: motorola

Refs #296
…names for oppo

2372 records get model_numbers/codenames from Google Play supported_devices.csv (ids only; no spec value changes).
Brands: oppo

Refs #296
…names for huawei, zte

3015 records get model_numbers/codenames from Google Play supported_devices.csv (ids only; no spec value changes).
Brands: huawei, zte

Refs #296
…names for sony, htc, lg

2895 records get model_numbers/codenames from Google Play supported_devices.csv (ids only; no spec value changes).
Brands: sony, htc, lg

Refs #296
…names for alcatel, vivo, nokia, oneplus

3028 records get model_numbers/codenames from Google Play supported_devices.csv (ids only; no spec value changes).
Brands: alcatel, vivo, nokia, oneplus

Refs #296
…for infinix, lenovo, blu, ulefone and 2 more brands

3022 records get model_numbers/codenames from Google Play supported_devices.csv (ids only; no spec value changes).
Brands: infinix, lenovo, blu, ulefone, asus, micromax

Refs #296
…names for tcl, allview, tecno, realme and 5 more brands

3046 records get model_numbers/codenames from Google Play supported_devices.csv (ids only; no spec value changes).
Brands: tcl, allview, tecno, realme, blackview, panasonic, google, doogee, sharp

Refs #296
…for wiko, acer, archos, meizu and 52 more brands

2843 records get model_numbers/codenames from Google Play supported_devices.csv (ids only; no spec value changes).
Brands: wiko, acer, archos, meizu, honor, lava, bq, gionee, haier, kyocera, vodafone, itel, gigabyte, energizer, coolpad, pantech, caterpillar, hisense, hp, verykool, qmobile, blackberry, benq, intex, philips, yu, yezz, dell, sony-ericsson, hmd, fujitsu, razer, t-mobile, unihertz, iqoo, sonim, nothing, nec, spice, prestigio, thuraya, vertu, orange, leeco, karbonn, toshiba, maxwest, palm, poco, xolo, nvidia, fairphone, i-mobile, intel, nubia, redmagic

Refs #296
@Seungpyo1007 Seungpyo1007 added enhancement New feature or request data Dataset changes labels Oct 6, 2026
@github-actions github-actions Bot added the app Validator or application code changes label Oct 6, 2026
@Seungpyo1007

Copy link
Copy Markdown
Member Author

🔎 Data verification — Tier 0 (offline existence/trust)

Scored by app.verify; posted by TechEngineBot. Informational only — the structural gate (app.validate) is separate and authoritative for merge.

13/13 categories ? 190,468 records ? diff base develop@379aa46

Changed records in this PR

31903 record(s) assessed.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 7669
    "Yellow" : 24227
    "Red" : 7
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
smartphone 🟩🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟥 30727 7538 23182 7 24.5%
tablet 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 1122 116 1006 0 10.3%
watch 🟩🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨 54 15 39 0 27.8%
All 🟩🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟥 31903 7669 24227 7 24.0%

Full-dataset baseline

As of 2026-09-30T06:34:03Z (daily verify-status aggregate).

category total green yellow red
brand 207 122 85 0
soc 2117 284 837 996
smartphone 92824 18216 74555 53
tablet 3305 293 3011 1
watch 418 96 320 2
pda 112 80 32 0
gpu 2083 1536 547 0
cpu 4628 4340 287 1
laptop 1966 126 1840 0
monitor 231 3 228 0
software 42493 366 42127 0
website 40084 13420 26664 0
all 190468 38882 150533 1053

green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to verified runs in the scheduled verify-network workflow.

@Seungpyo1007
Seungpyo1007 merged commit 491a89a into develop Oct 6, 2026
5 checks passed
@Seungpyo1007

Copy link
Copy Markdown
Member Author

TechEngine change review: PASS

13/13 categories ? 190,468 records ? diff base develop@379aa46

Check Result
python -m app.validate PASS
Changed dump JSON and manifest/index counts PASS
New hard integrity anomalies vs PR base PASS

Changed data

Category Added Modified Deleted Added verified Added unverified Added Kaggle-sourced
brand 0 0 0 0 0 0
soc 0 0 0 0 0 0
smartphone 0 30727 0 0 0 0
tablet 0 1122 0 0 0 0
watch 0 54 0 0 0 0
pda 0 0 0 0 0 0
gpu 0 0 0 0 0 0
cpu 0 0 0 0 0 0
laptop 0 0 0 0 0 0
monitor 0 0 0 0 0 0
software 0 0 0 0 0 0
website 0 0 0 0 0 0
device_catalog 0 0 0 0 0 0

Changed record examples

smartphone modified

  • smartphone/acer/2010/liquid-mt.json - Liquid mt
  • smartphone/acer/2010/liquid-mt/acer-liquid-mt.json - Acer Liquid mt
  • smartphone/acer/2010/stream.json - Stream
  • smartphone/acer/2010/stream/acer-stream.json - Acer Stream
  • smartphone/acer/2011/acer-betouch-e210.json - Acer beTouch E210
  • smartphone/acer/2011/acer-iconia-smart-s300.json - Acer Iconia Smart S300
  • smartphone/acer/2011/betouch-e210.json - beTouch E210
  • smartphone/acer/2011/iconia-tab-a100.json - Iconia Tab A100
  • smartphone/acer/2011/iconia-tab-a101.json - Iconia Tab A101
  • smartphone/acer/2011/iconia-tab-a500.json - Iconia Tab A500
  • smartphone/acer/2011/iconia-tab-a501.json - Iconia Tab A501
  • smartphone/acer/2012/acer-liquid-c-at390.json - Acer Liquid C AT390
  • smartphone/acer/2012/acer-liquid-gallant-duo-e350-liquid-c-ak330s.json - Acer Liquid Gallant Duo E350 / Liquid C AK330s
  • smartphone/acer/2012/cloudmobile-s500.json - Acer CloudMobile S500
  • smartphone/acer/2012/cloudmobile-s500/acer-cloudmobile-s500-aitoolbuzz-16-1gb-8gb-3g.json - Acer CloudMobile S500
  • ... 30712 more

tablet modified

  • tablet/acer/2011/iconia-tab-a100/acer-iconia-tab-a100-aitoolbuzz-31-1gb-8gb-2g.json - Acer Iconia Tab A100
  • tablet/acer/2011/iconia-tab-a100/acer-iconia-tab-a100-scrapegsma-1774.json - Acer Iconia Tab A100
  • tablet/acer/2011/iconia-tab-a100/acer-iconia-tab-a100.json - Acer Iconia Tab A100
  • tablet/acer/2011/iconia-tab-a101/acer-iconia-tab-a101-aitoolbuzz-32-1gb-8gb-3g.json - Acer Iconia Tab A101
  • tablet/acer/2011/iconia-tab-a101/acer-iconia-tab-a101-scrapegsma-1773.json - Acer Iconia Tab A101
  • tablet/acer/2011/iconia-tab-a101/acer-iconia-tab-a101.json - Acer Iconia Tab A101
  • tablet/acer/2011/iconia-tab-a500/acer-iconia-tab-a500-1gb-1gb.json - Acer Iconia Tab A500
  • tablet/acer/2011/iconia-tab-a500/acer-iconia-tab-a500-aitoolbuzz-41-1gb-16gb-2g.json - Acer Iconia Tab A500
  • tablet/acer/2011/iconia-tab-a500/acer-iconia-tab-a500-scrapegsma-1772.json - Acer Iconia Tab A500
  • tablet/acer/2011/iconia-tab-a500/iconia-tab-a500.json - Iconia Tab A500
  • tablet/acer/2011/iconia-tab-a501/acer-iconia-tab-a501-1gb-1gb-3g.json - Acer Iconia Tab A501
  • tablet/acer/2011/iconia-tab-a501/acer-iconia-tab-a501-aitoolbuzz-42-1gb-16gb-3g.json - Acer Iconia Tab A501
  • tablet/acer/2011/iconia-tab-a501/acer-iconia-tab-a501-scrapegsma-1770.json - Acer Iconia Tab A501
  • tablet/acer/2011/iconia-tab-a501/iconia-tab-a501.json - Iconia Tab A501
  • tablet/acer/2012/iconia-tab-a110/acer-iconia-tab-a110-aitoolbuzz-33-1gb-8gb-2g.json - Acer Iconia Tab A110
  • ... 1107 more

watch modified

  • watch/google/2022/pixel-watch/google-pixel-watch-2gb-32gb-4g-lte.json - Google Pixel Watch
  • watch/google/2022/pixel-watch/google-pixel-watch-aitoolbuzz-2753-2gb-32gb-4g-lte.json - Google Pixel Watch
  • watch/google/2024/pixel-watch-3/google-pixel-watch-3-2gb-32gb-4g-lte.json - Google Pixel Watch 3
  • watch/google/2024/pixel-watch-3/google-pixel-watch-3-aitoolbuzz-2755-2gb-32gb-4g-lte.json - Google Pixel Watch 3
  • watch/lg/2015/watch-urbane-2nd-edition-lte/lg-watch-urbane-2nd-edition-lte-0-75gb-4gb-4g-lte.json - LG Watch Urbane 2nd Edition LTE
  • watch/lg/2015/watch-urbane-2nd-edition-lte/lg-watch-urbane-2nd-edition-lte-scrapegsma-4996.json - LG Watch Urbane 2nd Edition LTE
  • watch/lg/2015/watch-urbane-2nd-edition-lte/lg-watch-urbane-2nd-edition-lte.json - Lg Watch Urbane 2nd Edition LTE
  • watch/lg/2015/watch-urbane-2nd-edition/watch-urbane-2nd-edition-lte.json - Watch Urbane 2nd Edition LTE
  • watch/lg/2016/watch-urbane-2nd-edition-lte/lg-watch-urbane-2nd-edition-lte-aitoolbuzz-5427-0-75gb-4gb-4g-lte.json - LG Watch Urbane 2nd Edition LTE
  • watch/lg/2017/watch-sport/lg-watch-sport-aitoolbuzz-5425-0-75gb-4gb-4g-lte.json - LG Watch Sport
  • watch/lg/2017/watch-sport/watch-sport.json - Watch Sport
  • watch/oneplus/2024/watch-2-esim/oneplus-watch-2-esim-2gb-32gb-4g-lte.json - OnePlus Watch 2 (eSIM)
  • watch/oneplus/2024/watch-2/oneplus-watch-2-2gb-32gb-4g-lte.json - OnePlus Watch 2
  • watch/oneplus/2024/watch-2/oneplus-watch-2-aitoolbuzz-7460-2gb-32gb-4g-lte.json - OnePlus Watch 2
  • watch/oneplus/2024/watch-2/oneplus-watch-2-aitoolbuzz-7461-2gb-32gb-4g-lte.json - OnePlus Watch 2 (eSIM)
  • ... 39 more

Heuristic review

  • Heuristic warnings: none found.

@Seungpyo1007

Copy link
Copy Markdown
Member Author

TechEngine validation stats: PASS

13/13 categories ? 190,468 records ? diff base develop@379aa46

Data summary

Category Total Verified Unverified Missing verified Tracked Verified % of tracked
brand 207 123 84 0 207 59.4%
soc 2117 284 1833 0 2117 13.4%
smartphone 92824 17235 75589 0 92824 18.6%
tablet 3305 293 3012 0 3305 8.9%
watch 418 98 320 0 418 23.4%
pda 112 68 44 0 112 60.7%
gpu 2083 1496 587 0 2083 71.8%
cpu 4628 4222 406 0 4628 91.2%
laptop 1966 129 1837 0 1966 6.6%
monitor 231 21 210 0 231 9.1%
software 42493 358 42135 0 42493 0.8%
website 40084 14721 25363 0 40084 36.7%
device_catalog 0 0 0 0 0 n/a
all 190468 39048 151420 0 190468 20.5%

Warning

Tracked verified coverage is below 50% for software 0.8% (358/42493), laptop 6.6% (129/1966), tablet 8.9% (293/3305), monitor 9.1% (21/231), soc 13.4% (284/2117), smartphone 18.6% (17235/92824), all 20.5% (39048/190468), watch 23.4% (98/418), and 1 more.
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

  • The integrity gate blocks only hard anomalies introduced by this PR relative to its merge base.
  • Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
  • Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py (PR head compared with PR base, changed records only)
loaded CPU=0 GPU=0
Integrity section Flagged lines
structural 0
CPU name/tier consistency (desktop mainstream only) 0
CPU single>multi (cinebench/geekbench — should be multi>=single) 0
CPU era-vs-score outliers 0
CPU cross-source ratio outliers (possible wrong-variant) 0
GPU cross-source ratio outliers + sanity 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

app Validator or application code changes data Dataset changes enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Data collection: new records (workstream)

2 participants