Repository navigation
TechAPI dataset roadmap and status: all categories (1989-2026) #1
Description
Purpose
Roadmap and status board for the TechAPI dataset: every category from 1989 to 2026, published as a static JSON API at https://gettechapi.github.io/TechAPI/.
This issue stays open. The body is the current status, updated each release. Comments are one summary per release. Day-to-day work is tracked in the workstream issues below and on Project #2.
How work is tracked (since 2026-09-27)
| Workstream | Issue | Which PRs link there |
|---|---|---|
| Roadmap & releases | #1 (this issue) | develop → main release PRs and the batched dump PRs |
| New records | #295 | feat(data…): new CPUs/GPUs/SoCs/devices, benchmark backfill, new sources |
| Data accuracy | #296 | fix(data…): corrections, duplicate merges, dates, placeholder cleanup |
| Engine, bot & automation | #297 | ci/app PRs in TechAPI; TechEngine PRs reference GetTechAPI/TechAPI#297 |
| Site | #19 | Homepage and site PRs |
pr-metadata adds the matching Closes #N line automatically. Auto-close is disabled, so trackers stay open. Project #2 has a Workstream field with the same split.
Current status (release #315, 2026-09-27)
| Category | Records | Verified | Verified % | Notes |
|---|---|---|---|---|
| smartphone | 93,407 | 17,581 | 18.8% | 39,622 models; Wikipedia/Wikidata matching confirms only ~1.5% of Tier-3-only models |
| software | 42,493 | 358 | 0.8% | Domain scoring since TechEngine #103; most records lack dates/developers even on Wikidata |
| website | 40,084 | 7,635 | 19.0% | Domain scoring + Wikidata liveness since TechEngine #103 |
| cpu | 4,507 | 4,211 | 93.4% | |
| tablet | 3,459 | 288 | 8.3% | |
| soc | 2,117 | 280 | 13.2% | 996 red = placeholder imports with no failing check (weak source only) |
| gpu | 2,076 | 1,493 | 71.9% | |
| laptop | 1,995 | 105 | 5.3% | Domain scoring since TechEngine #102 |
| monitor | 882 | 0 | 0.0% | Domain scoring since TechEngine #102; no T1/T2 sources yet |
| watch | 447 | 94 | 21.0% | |
| brand | 207 | 123 | 59.4% | |
| pda | 140 | 74 | 52.9% | |
| devices (excl. software, website) | 109,237 | 24,249 | 22.2% | Comparable to the pre-09-27 figure (20.8%) |
| all 12 categories | 191,814 | 32,242 | 16.8% |
Two figures are shown because software and website (82,577 records) entered the denominator on 2026-09-27. Their verification is a same-source check: the cited Wikidata item still exists and the record passes domain rules. Most software records are too sparse for green, even after filling from Wikidata. Verified records have only grown.
Goals
- Verification: 30% of records in the scored categories (smartphone, tablet, watch, pda, cpu, gpu, soc, brand); laptop and monitor scoring is done (TechEngine data: add laptop category — 970 laptops (first non-mobile class) #102).
- Coverage: keep new releases (2025-2026 phones, SoCs, CPUs, GPUs, laptops, watches) within one release of launch, when a citable source exists.
- Accuracy: no known impossible values (storage-as-RAM, placeholder dates, estimate-only benchmarks); duplicate-free slugs.
- Automation: every PR is checked on all 12 categories; weekly refresh and ingest run green.
Recent releases
| Release | Date | Main content |
|---|---|---|
| #315 | 2026-09-27 | 8,839 verified promotions (first software), Wikidata-filled software/website fields, check-urls fixes |
| #308 | 2026-09-27 | Dump with 312 accuracy fixes (storage, battery, monitor fields, SoC years), watch SoC links |
| #304 | 2026-09-27 | Wearable SoCs, Galaxy Watch9 / Ultra 2, Pixel Watch 4, laptop/monitor scoring, 1,306 promotions |
| #292 | 2026-09-26 | Galaxy Tab S11 / iPad Pro M5, 13 Surface laptops, benchmark backfill (~1,900 records), 182 SoC dates, GPU duplicate merge |
| #284 | 2026-09-25 | 2025-26 watches and phones, 83 Xeon thread counts, RAM and duplicate fixes |
| #274 | 2026-09-25 | 470 new CPUs/GPUs/SoCs/MacBooks, Intel and GPU cross-check fixes |
| #252 | 2026-09-25 | SoC dedup, Commons images, tablet citations, 146 verified promotions |
| #249 | 2026-09-25 | TechPicks image fixes, base_model_slug fill |
Data rules (summary)
- Every record cites
source_urls. Nothing is invented; if no source states a value, it stays null. - Bulk imports land as
verified: false. The verify layer promotes a record only when a live T1/T2 source agrees. - Sources: Wikipedia, Wikidata, manufacturer pages and CC datasets are fine. GSMArena and PhoneDB may only be used to correct existing values (ADR-013). Namuwiki is banned. DeviceSpecifications is paused.
- PRs target
develop. Before each release,developgets one batched dump (site/public/v1);mainonly moves through release PRs.
History
This issue began on 2026-05-28 as the "Massive dataset rebuild" tracker for 8 categories (51,783 records at PR #36). The original description is preserved in the comment that announced this restructure.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsIn Progress
Activity
chore(scripts): widen validator bounds and refresh seed loader
feat(app): expand GPU model and schema for richer hardware specs
feat(data/brand): restructure into country folders and expand catalog
feat(data/cpu): expand AMD lineup from K5 to Zen 6 with segment split
feat(data/cpu): expand Intel lineup from i486 to Panther Lake with se…
feat(data/cpu): add IBM POWER, z mainframe, PowerPC, Cell, Motorola G…
feat(data/cpu): add historic and niche vendors across 11 manufacturers
feat(data/gpu): rebuild discrete GPU catalog across 10 vendors with s…
feat(data/smartphone): rebuild smartphone catalog across major brands
feat(data/soc): rebuild SoC catalog covering Apple, Qualcomm, MediaTe…
docs: update README and data/README for expanded dataset and new layout
Seungpyo1007 commentedon May 28, 2026
Rolled out in 11 commits on
main4dc4b579708ee37a9df9ede9b32cb08e2cf97816705cd4b5b2a434bffc7bcd154306c8f64a74a(The site homepage update was already on
mainasef15e00 feat(site): add interactive API playground and redesign homepage, so the rebase deduplicated it.)Result
python -m scripts.validatePASS across brand / cpu / gpu / smartphone / socbrand/<country>/<slug>.json+cpu/<mfr>/<year>/<consumer|enterprise>/<slug>.jsonchore(scripts): widen validator bounds and refresh seed loader
feat(app): expand GPU model and schema for richer hardware specs
601 remaining items
github-actions commentedon Sep 26, 2026
Linked PR update: Add 13 Microsoft Surface laptop models (2015-2023) from Wikipedia #286
data/surface-laptopsdata,enhancementChanged Data
Changed Areas
Notes
Closes #...for GitHub Development linking.github-actions commentedon Sep 26, 2026
Linked PR update: Fix 65 SoC release dates from Wikipedia launch tables #287
fix/soc-release-datesbug,data,enhancementChanged Data
Changed Areas
Notes
Closes #...for GitHub Development linking.github-actions commentedon Sep 26, 2026
Linked PR update: Backfill benchmark scores for ~1,000 CPUs and ~900 GPUs #288
data/benchmarks-new-recordsdata,enhancementChanged Data
Changed Areas
Notes
Closes #...for GitHub Development linking.github-actions commentedon Sep 26, 2026
Linked PR update: Fix 117 MediaTek, Exynos and Kirin SoC release dates #289
fix/soc-release-dates-2bug,data,enhancementChanged Data
Changed Areas
Notes
Closes #...for GitHub Development linking.github-actions commentedon Sep 26, 2026
Linked PR update: fix(data): resolve GPU conflicts against manufacturer specifications #290
fix/gpu-conflicts-manufacturerbug,data,enhancementChanged Data
Changed Areas
Notes
Closes #...for GitHub Development linking.github-actions commentedon Sep 26, 2026
Linked PR update: chore(dump): refresh static API for 2026-09-26 data #291
chore/dump-refresh-20260926data,enhancementChanged Data
Changed Areas
Notes
Closes #...for GitHub Development linking.github-actions commentedon Sep 26, 2026
Linked PR update: release: tablets, Surface laptops, benchmark backfill, SoC dates (2026-09-26) #292
developdata,enhancementChanged Data
Changed Areas
Notes
Closes #...for GitHub Development linking.Seungpyo1007 commentedon Sep 26, 2026
2026-09-26 progress
New records (collection)
Corrections (accuracy)
YYYY-01-01placeholders running 1-2 years late.Release
tablets/samsung-galaxy-tab-s11200,laptops/microsoft-surface-laptop-go-3200,gpus/radeon-instinct-mi300x404 (merged duplicate, expected)github-actions commentedon Sep 26, 2026
Linked PR update: fix(data): correct storage-as-RAM on 12 older devices from sourced specs #293
fix/ram-storage-sourcedbug,data,enhancementChanged Data
Changed Areas
Notes
Closes #...for GitHub Development linking.github-actions commentedon Sep 26, 2026
Linked PR update: ci: send the PR base to TechEngine checks and stop tracking scores.jsonl #294
ci/bot-full-coverage-relayci,data,enhancementChanged Data
Changed Areas
Notes
Closes #...for GitHub Development linking.Seungpyo1007 commentedon Sep 26, 2026
2026-09-27: PR/issue/automation audit and fixes
Audit: TechAPI PRs #173–#292, all TechEngine PRs and issues, Project #2 across the org, and TechEngineBot coverage.
TechEngineBot coverage (TechEngine #95 + TechAPI #294)
12/12 categories · 191,797 records · diff base develop@<sha>.data/_verify/state/scores.jsonl(43.5 MB generated cache) is no longer tracked.Rules and process
main(the cause of chore(data): weekly refresh 2026-09-21 #195); it now targetsdevelop.Data
Seungpyo1007 commentedon Sep 26, 2026
Tracking restructured (2026-09-27)
This issue is now the roadmap and status board. The body above is rewritten and will be updated each release; comments here are one summary per release. Ongoing work moves to the workstream issues:
Existing PR links and comments stay as they are.
pr-metadatawill route new PRs to the matching issue.Original description (2026-05-28, last edited before PR #36)
Purpose
Long-running tracker for the TechAPI dataset rebuild across
brand,cpu,gpu,smartphone,soc,tablet,watch, andpda.This issue intentionally stays open while bulk imports, validation hardening, public dump refreshes, and manual verification work continue. PRs may use
Closes #1so GitHub Development links the PR back to this tracker. Repository auto-close is disabled for this workflow, so merging linked PRs should not finish this issue.Current Status
data/import-stagingverified: falseuntil manual audit or TechEngine verification confirms recordsLatest Data Snapshot
Recent PR History
Sources Currently Used
Validation Policy
Every data PR should include TechEngineBot comments for:
python -m app.validate,python integrity_check.py TechAPI/data --strict, and site build when site files changeLow verified coverage is allowed for bulk import PRs, but should be called out as a follow-up warning instead of failing validation.
Remaining Work
data/import-stagingbefore each push and keep commits split by source, brand, era, or categoryverifiedflags so thebrandcategory no longer has missing verification metadatav1/index.jsonand category dumps refreshed after each data batchOperational Notes
data,enhancementTechEngineBot should add or update a tracking comment on this issue whenever a linked data PR is opened or synchronized.
Latest linked PR: #36
Seungpyo1007 commentedon Sep 26, 2026
Release 2026-09-27 ① (#304): deployed and live.
Seungpyo1007 commentedon Sep 27, 2026
Release 2026-09-27 ② (#308): 1,306 verified promotions published, and the static API now carries the #300–#302 and #306 accuracy fixes. Verified: 21.4% of device categories and 12.2% of all 12 categories (table in the body).
Seungpyo1007 commentedon Sep 27, 2026
Release 2026-09-27 ③ (#315): 8,839 verified promotions (websites 7,635, the first 358 software, 762 CPUs), Wikidata-filled software/website fields, and check-urls fixes. Verified: 22.2% of device categories and 16.8% of all 12 categories (table in the body).