Skip to content

data: add 11.8k Google Play device_catalog entries and 1.9k device makers - #360

Merged
Seungpyo1007 merged 6 commits into
developfrom
data/play-device-catalog
Oct 6, 2026
Merged

Seungpyo1007 merged 6 commits into
developfrom
data/play-device-catalog

Conversation

@Seungpyo1007

@Seungpyo1007 Seungpyo1007 commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Summary

Expands the dataset with devices from Google Play that are not yet in it:

  • 1,881 device makers at data/brand/unknown/<slug>.json. Country is not sourced, so it is left unset (never guessed). categories: ["smartphone-oem"], verified: false.
  • 11,783 device_catalog entries at data/device_catalog/<brand>/<slug>.json: 6,517 phones, 5,223 tablets and 43 watches.
    • Identity (model_numbers, codenames, merged marketing_names) comes from Google Play supported_devices.csv.
    • RAM, SoC (FK where it resolves; soc_raw kept), GPU, resolution, density and Android SDK range come from the Play Console device catalog.
    • Multi-valued Console fields are left null.
    • No release dates. A catalog entry's date is unknown by design (ADR-016); release_year is only ever set from a model code.

Scope

Only white-label / long-tail makers that have a Phone, Tablet or Wearable Console row are included, with TV/STB SoCs excluded. In the round-3 audit this population was 98% genuinely new mobile devices. New devices from major brands (70% in the audit) are held back for review and are not in this PR.

Schema: TechEngine #121. Policy: TechEngine docs/SPEC.md §9.2 + ADR-016 (#122).

Closes #295

@Seungpyo1007

Copy link
Copy Markdown
Member Author

🔎 Data verification — Tier 0 (offline existence/trust)

Scored by app.verify; posted by TechEngineBot. Informational only — the structural gate (app.validate) is separate and authoritative for merge.

13/13 categories ? 204,132 records ? diff base develop@491a89a

Changed records in this PR

13664 record(s) assessed.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 0
    "Yellow" : 13664
    "Red" : 0
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
brand 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 1881 0 1881 0 0.0%
device_catalog 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 11783 0 11783 0 0.0%
All 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 13664 0 13664 0 0.0%

Full-dataset baseline

As of 2026-10-06T01:56:23Z (daily verify-status aggregate).

category total green yellow red
brand 207 122 85 0
soc 2117 284 837 996
smartphone 92824 18216 74555 53
tablet 3305 293 3011 1
watch 418 96 320 2
pda 112 80 32 0
gpu 2083 1536 547 0
cpu 4628 4340 287 1
laptop 1966 126 1840 0
monitor 231 3 228 0
software 42493 366 42127 0
website 40084 13420 26664 0
device_catalog 0 0 0 0
all 190468 38882 150533 1053

green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to verified runs in the scheduled verify-network workflow.

Country is not sourced for these makers, so they live under brand/unknown/ (ADR-016).

Refs #295
Identity from Google Play supported_devices.csv; RAM/SoC/GPU/screen/SDK from the Play Console device catalog.
No release dates (unknown by design, ADR-016).

Refs #295
Identity from Google Play supported_devices.csv; RAM/SoC/GPU/screen/SDK from the Play Console device catalog.
No release dates (unknown by design, ADR-016).

Refs #295
Identity from Google Play supported_devices.csv; RAM/SoC/GPU/screen/SDK from the Play Console device catalog.
No release dates (unknown by design, ADR-016).

Refs #295
Identity from Google Play supported_devices.csv; RAM/SoC/GPU/screen/SDK from the Play Console device catalog.
No release dates (unknown by design, ADR-016).

Refs #295
Identity from Google Play supported_devices.csv; RAM/SoC/GPU/screen/SDK from the Play Console device catalog.
No release dates (unknown by design, ADR-016).

Refs #295
@Seungpyo1007
Seungpyo1007 force-pushed the data/play-device-catalog branch from fcbb0af to c4eb13b Compare October 6, 2026 02:22
@Seungpyo1007
Seungpyo1007 merged commit fd43d53 into develop Oct 6, 2026
5 checks passed
@Seungpyo1007

Copy link
Copy Markdown
Member Author

TechEngine change review: PASS

13/13 categories ? 204,132 records ? diff base develop@491a89a

Check Result
python -m app.validate PASS
Changed dump JSON and manifest/index counts PASS
New hard integrity anomalies vs PR base PASS

Changed data

Category Added Modified Deleted Added verified Added unverified Added Kaggle-sourced
brand 1881 0 0 0 1881 0
soc 0 0 0 0 0 0
smartphone 0 0 0 0 0 0
tablet 0 0 0 0 0 0
watch 0 0 0 0 0 0
pda 0 0 0 0 0 0
gpu 0 0 0 0 0 0
cpu 0 0 0 0 0 0
laptop 0 0 0 0 0 0
monitor 0 0 0 0 0 0
software 0 0 0 0 0 0
website 0 0 0 0 0 0
device_catalog 11783 0 0 0 11783 0

Changed record examples

brand added

  • brand/unknown/1-1.json - 1&1
  • brand/unknown/10-or.json - 10.or
  • brand/unknown/3atech.json - 3ATech
  • brand/unknown/3nstar.json - 3nStar
  • brand/unknown/3rtablet.json - 3Rtablet
  • brand/unknown/8849.json - 8849
  • brand/unknown/a1.json - A1
  • brand/unknown/aamra-we.json - Aamra WE
  • brand/unknown/aauw.json - AAUW
  • brand/unknown/abctech.json - ABCTECH
  • brand/unknown/abil.json - ABIL
  • brand/unknown/abxylute.json - abxylute
  • brand/unknown/accent.json - Accent
  • brand/unknown/acd-computers.json - ACD Computers
  • brand/unknown/ace-france.json - ACE France
  • ... 1866 more

device_catalog added

  • device_catalog/1-1/1-1-aeop525.json - AEOP525
  • device_catalog/10-or/10-or-10or-g2.json - 10or_G2
  • device_catalog/10-or/10-or-d.json - D
  • device_catalog/10-or/10-or-e.json - E
  • device_catalog/10-or/10-or-g.json - G
  • device_catalog/10-or/10-or-g2.json - G2
  • device_catalog/3atech/3atech-t10-8128.json - T10-8128
  • device_catalog/3nstar/3nstar-nustar65.json - Nustar65
  • device_catalog/3rtablet/3rtablet-vt-7ge.json - VT-7GE
  • device_catalog/8849/8849-a101.json - A101
  • device_catalog/8849/8849-ob-a98.json - OB-A98
  • device_catalog/8849/8849-ob-a99.json - OB-A99
  • device_catalog/8849/8849-ob-p08.json - OB-P08
  • device_catalog/8849/8849-ob-p10.json - OB-P10
  • device_catalog/8849/8849-ob-p12.json - OB-P12
  • ... 11768 more

Heuristic review

  • Added records by manufacturer/brand: hisense: 154, prestigio: 124, symphony: 111, bqru: 108, blu: 105, sky-devices: 104, cubot: 103, mobicell: 98

  • Added records by source class: other: 13664

  • Heuristic warnings: 40 total; showing first 20.

    • brand: brand/unknown/lzu-lzu.json: repeated adjacent word in name
    • brand: brand/unknown/unbluable.json: unbalanced parentheses in name
    • device_catalog: device_catalog/adoc/adoc-d18.json: double spaces in name
    • device_catalog: device_catalog/ag-mobile/ag-mobile-agm-h1.json: double spaces in name
    • device_catalog: device_catalog/allview/allview-viva-h1003-lte.json: double spaces in name
    • device_catalog: device_catalog/blu/blu-c6-2020.json: double spaces in name
    • device_catalog: device_catalog/condor/condor-allure-m1.json: double spaces in name
    • device_catalog: device_catalog/cubot/cubot-max-2.json: double spaces in name
    • device_catalog: device_catalog/cubot/cubot-tab-60.json: double spaces in name
    • device_catalog: device_catalog/hisense/hisense-agm-x2.json: double spaces in name
    • device_catalog: device_catalog/hisense/hisense-f23-plus.json: double spaces in name
    • device_catalog: device_catalog/hisense/hisense-hi-2.json: double spaces in name
    • device_catalog: device_catalog/hisense/hisense-infinity-f17.json: double spaces in name
    • device_catalog: device_catalog/hisense/hisense-infinity-h12.json: double spaces in name
    • device_catalog: device_catalog/hisense/hisense-iron-2.json: double spaces in name
    • device_catalog: device_catalog/hisense/hisense-startrail-9.json: double spaces in name
    • device_catalog: device_catalog/hisense/hisense-u601s-pro.json: double spaces in name
    • device_catalog: device_catalog/ihunt/ihunt-s22-ultra.json: double spaces in name
    • device_catalog: device_catalog/iris/iris-vox-4s.json: double spaces in name
    • device_catalog: device_catalog/iris/iris-vox-energy.json: double spaces in name

@Seungpyo1007

Copy link
Copy Markdown
Member Author

TechEngine validation stats: PASS

13/13 categories ? 204,132 records ? diff base develop@491a89a

Data summary

Category Total Verified Unverified Missing verified Tracked Verified % of tracked
brand 2088 123 1965 0 2088 5.9%
soc 2117 284 1833 0 2117 13.4%
smartphone 92824 17235 75589 0 92824 18.6%
tablet 3305 293 3012 0 3305 8.9%
watch 418 98 320 0 418 23.4%
pda 112 68 44 0 112 60.7%
gpu 2083 1496 587 0 2083 71.8%
cpu 4628 4222 406 0 4628 91.2%
laptop 1966 129 1837 0 1966 6.6%
monitor 231 21 210 0 231 9.1%
software 42493 358 42135 0 42493 0.8%
website 40084 14721 25363 0 40084 36.7%
device_catalog 11783 0 11783 0 11783 0.0%
all 204132 39048 165084 0 204132 19.1%

Warning

Tracked verified coverage is below 50% for device_catalog 0.0% (0/11783), software 0.8% (358/42493), brand 5.9% (123/2088), laptop 6.6% (129/1966), tablet 8.9% (293/3305), monitor 9.1% (21/231), soc 13.4% (284/2117), smartphone 18.6% (17235/92824), and 3 more.
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

  • The integrity gate blocks only hard anomalies introduced by this PR relative to its merge base.
  • Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
  • Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py (PR head compared with PR base, changed records only)
loaded CPU=0 GPU=0
Integrity section Flagged lines
structural 0
CPU name/tier consistency (desktop mainstream only) 0
CPU single>multi (cinebench/geekbench — should be multi>=single) 0
CPU era-vs-score outliers 0
CPU cross-source ratio outliers (possible wrong-variant) 0
GPU cross-source ratio outliers + sanity 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data Dataset changes enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Data collection: new records (workstream)

2 participants