Add 41 catalog datasets (141 total)

[mattfaltyn]

Summary

• Adds 41 YAML datasets so the catalog reaches 141 operational sources, covering patents, vital stats, mortgages, trade, OECD/ONS/StatCan, CO2/sea ice, caselaw, lobbying, and non-cyber technology.
• Substitutes four recommended sources that fail inclusion rules: USPTO Open Data Portal (PatentsView shut down), GovInfo USCOURTS (CourtListener REST needs a commercial agreement; CAP is frozen), CDC/NCI USCS (SEER DUA/SEER*Stat), and NTSB aviation accidents (OpenSky terms ban commercial/operational API use). Skips RxNorm (UMLS license). Scopes Treasury Fiscal Data to Debt to the Penny because auctions already exist.
• Updates the theme-registry test and README count, adds 11 keyless GET provider contracts, raises the all-guides e2e timeout, and adds exact URL-validation exceptions only for bot-protected Senate LDA, Smithsonian GVP, and UN Security Council pages.

Test plan

• npm run validate-datasets:offline — 141 datasets
• npm test and npm run lint
• npm run validate-datasets — live source/license identity
• Confirm new provider contracts: NSF awards, Treasury debt-to-the-penny, NOAA GML CO2, PubChem, DailyMed, openFDA device, OpenFEMA NFIP, RIPEstat, NCHS mortality, NDBC buoys, ONS
• Spot-check discovery still pins NWS first and that a new guide (e.g. noaa-gml-co2) renders
• Confirm substitutions are present and the excluded originals are not: no PatentsView, CourtListener/CAP, SEER, OpenSky, or RxNorm YAML files

Made with Cursor

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
foundation_data	Ready	Preview	Aug 18, 2026 10:44am