Add 24 catalog datasets spanning crime, companies, occupations, and global weather.

README.md

@@ -147,7 +147,7 @@ No application code changes required.
 
 ## Catalog
 
-The catalog currently contains 76 operational datasets spanning natural
+The catalog currently contains 100 operational datasets spanning natural
 hazards, weather, water, climate, flood risk, drought, space weather, global
 disaster alerts, public health, clinical research, cybersecurity, package
 graphs, legislation, sanctions screening, government spending and procurement,
@@ -157,7 +157,9 @@ petroleum inventories, prediction markets, geospatial analysis, places,
 broadband, bridges, EV charging, consumer finance, education, K-12 directories,
 housing prices, rents, food and product recalls, elections, European statistics,
 live transit feeds, aviation, provider directories, drinking water, preprints,
-pageviews, agriculture, and tropical cyclones. The YAML files in
+pageviews, agriculture, tropical cyclones, crime, companies, occupations,
+global forecasts, live OSM, food products, nursing homes, LEI, and euro-area
+statistics. The YAML files in
 [`data/datasets`](data/datasets) are the source of truth for the current list.
 
 ## Deploy to Vercel

data/datasets/cdc-social-vulnerability-index.yaml

@@ -0,0 +1,88 @@
+id: cdc-social-vulnerability-index
+name: CDC ATSDR Social Vulnerability Index
+description: >
+  Census-tract and county social-vulnerability rankings for building emergency
+  planning, outreach, and hazard-equity tools.
+theme: Health, Food & Safety
+url: https://www.atsdr.cdc.gov/place-health/php/svi/svi-data-documentation-download.html
+access_type:
+  - download
+  - api
+api_key_required: false
+free_to_access: true
+size_gb_min: 0.01
+size_gb_max: 0.5
+formats:
+  - CSV
+  - JSON
+license: U.S. Government public data / federal copyright guidance
+license_url: https://www.usa.gov/government-copyright
+url_checks:
+  source_marker: Social Vulnerability Index
+  license_marker: federal government materials
+domains:
+  - Community Health
+  - Emergency Management
+  - Demographics
+data_types:
+  - Tabular
+  - Geospatial
+  - Index Scores
+tasks:
+  - Risk Assessment
+  - Geographic Analysis
+  - Emergency Planning
+difficulty: intermediate
+geography:
+  - United States
+temporal_coverage: 2000-2022 SVI releases
+update_frequency: occasional
+provider: Agency for Toxic Substances and Disease Registry
+source_type: government
+last_verified: 2026-08-18
+getting_started:
+  overview: >
+    SVI ranks communities on socioeconomic, household, minority-status, and
+    housing themes derived from ACS. Start with 2022 county ranks for the
+    United States file. Percentiles are relative within one vintage, so do not
+    compare 2020 and 2022 ranks as a time series.
+  prerequisites:
+    - Python 3.10 or newer
+    - A notebook environment such as Jupyter or Google Colab
+    - An internet connection
+  access_steps:
+    - Open the SVI data-download page and read the comparison-over-time warning.
+    - Query a bounded page of 2022 county ranks from the public feature service.
+    - Keep FIPS codes, county names, and the overall RPL_THEMES percentile.
+  python:
+    packages:
+      - pandas
+      - requests
+    code: |
+      import pandas as pd
+      import requests
+
+      response = requests.get(
+          "https://onemap.cdc.gov/onemapservices/rest/services/SVI/"
+          "CDC_ATSDR_Social_Vulnerability_Index_2022_USA/FeatureServer/1/query",
+          params={
+              "where": "1=1",
+              "outFields": "ST,STATE,STCNTY,COUNTY,RPL_THEMES,E_TOTPOP",
+              "returnGeometry": "false",
+              "resultRecordCount": 50,
+              "f": "json",
+          },
+          timeout=30,
+      )
+      response.raise_for_status()
+      counties = pd.DataFrame(
+          [feature["attributes"] for feature in response.json()["features"]]
+      )
+      print(counties.head())
+  first_project:
+    title: Rank counties by 2022 overall SVI
+    goal: Test whether SVI percentiles can flag counties for emergency-planning outreach.
+    steps:
+      - Keep state, county, FIPS, population, and RPL_THEMES.
+      - Sort by overall percentile and report how many ranks are missing.
+      - Explain that percentiles are relative within 2022 and should not be subtracted from an earlier SVI vintage.

data/datasets/census-county-business-patterns.yaml

@@ -0,0 +1,86 @@
+id: census-county-business-patterns
+name: Census County Business Patterns
+description: >
+  Annual establishment, employment, and payroll counts by industry and county
+  for building local industry-mix and site-selection tools.
+theme: Markets & Economics
+url: https://www.census.gov/programs-surveys/cbp.html
+access_type:
+  - api
+  - download
+api_key_required: true
+free_to_access: true
+size_gb_min: 0.001
+size_gb_max: 2
+formats:
+  - JSON
+  - CSV
+license: U.S. Census Bureau API Terms of Service
+license_url: https://www.census.gov/data/developers/about/terms-of-service.html
+url_checks:
+  source_marker: County Business Patterns
+  license_marker: Terms of Service Agreement
+domains:
+  - Local Economics
+  - Business
+  - Labor Economics
+data_types:
+  - Tabular
+  - Survey Estimates
+tasks:
+  - Market Sizing
+  - Site Selection
+  - Industry Analysis
+difficulty: intermediate
+geography:
+  - United States
+temporal_coverage: 1986-present
+update_frequency: annual
+provider: U.S. Census Bureau
+source_type: government
+last_verified: 2026-08-18
+getting_started:
+  overview: >
+    County Business Patterns counts establishments with paid employees by
+    NAICS and geography. Start with one state and the all-industry NAICS code.
+    Employment is a March 12 snapshot, payroll can be suppressed, and CBP
+    excludes most government and self-employed activity.
+  prerequisites:
+    - Python 3.10 or newer
+    - A notebook environment such as Jupyter or Google Colab
+    - A free Census Data API key saved in the CENSUS_API_KEY environment variable
+  access_steps:
+    - Request a free Census Data API key and open the CBP program page.
+    - Review the current year variables for establishments, employment, and payroll.
+    - Request all-industry county totals for one state.
+  python:
+    packages:
+      - pandas
+      - requests
+    code: |
+      import os
+      import pandas as pd
+      import requests
+
+      response = requests.get(
+          "https://api.census.gov/data/2023/cbp",
+          params={
+              "get": "NAME,ESTAB,EMP,PAYANN,NAICS2017_LABEL",
+              "for": "county:*",
+              "in": "state:08",
+              "NAICS2017": "00",
+              "key": os.environ["CENSUS_API_KEY"],
+          },
+          timeout=30,
+      )
+      response.raise_for_status()
+      rows = response.json()
+      counties = pd.DataFrame(rows[1:], columns=rows[0])
+      print(counties.head())
+  first_project:
+    title: Rank one state's counties by establishment counts
+    goal: Test whether CBP can describe which counties hold the most paid employers.
+    steps:
+      - Keep county name, establishments, employment, and annual payroll as numbers.
+      - Rank counties by establishments and flag suppressed payroll cells.
+      - Explain that March employment, NAICS suppression, and the exclusion of most government work limit a local industry story.

data/datasets/cftc-commitment-of-traders.yaml

@@ -0,0 +1,88 @@
+id: cftc-commitment-of-traders
+name: CFTC Commitments of Traders
+description: >
+  Weekly futures positioning by trader category for building commodities,
+  rates, and speculative-position monitors.
+theme: Markets & Economics
+url: https://www.cftc.gov/MarketReports/CommitmentsofTraders/index.htm
+access_type:
+  - api
+  - download
+api_key_required: false
+free_to_access: true
+size_gb_min: 0
+size_gb_max: 0.5
+formats:
+  - JSON
+  - CSV
+license: U.S. Government public data / federal copyright guidance
+license_url: https://www.usa.gov/government-copyright
+url_checks:
+  source_marker: Commitments of Traders
+  license_marker: federal government materials
+domains:
+  - Commodities
+  - Finance
+  - Capital Markets
+data_types:
+  - Time Series
+  - Tabular
+tasks:
+  - Market Monitoring
+  - Positioning Analysis
+  - Trend Analysis
+difficulty: intermediate
+geography:
+  - United States
+temporal_coverage: 1986-present weekly reports
+update_frequency: weekly
+provider: Commodity Futures Trading Commission
+source_type: government
+last_verified: 2026-08-18
+getting_started:
+  overview: >
+    The COT reports break down open interest for futures markets with enough
+    large traders. Start with one contract in the legacy combined dataset.
+    Categories are regulatory, not strategy labels, and Tuesday positions are
+    published later in the week.
+  prerequisites:
+    - Python 3.10 or newer
+    - A notebook environment such as Jupyter or Google Colab
+    - An internet connection
+  access_steps:
+    - Open the Commitments of Traders page and the public reporting environment.
+    - Request a bounded page of legacy combined records for one market.
+    - Keep report date, market name, and commercial versus noncommercial long/short.
+  python:
+    packages:
+      - pandas
+      - requests
+    code: |
+      import pandas as pd
+      import requests
+
+      response = requests.get(
+          "https://publicreporting.cftc.gov/resource/jun7-fc8e.json",
+          params={
+              "$limit": 20,
+              "$order": "report_date_as_yyyy_mm_dd DESC",
+              "contract_market_name": "GOLD",
+          },
+          timeout=30,
+      )
+      response.raise_for_status()
+      cot = pd.DataFrame(response.json())
+      print(cot[[
+          "report_date_as_yyyy_mm_dd",
+          "contract_market_name",
+          "open_interest_all",
+          "noncomm_positions_long_all",
+          "noncomm_positions_short_all",
+      ]].head())
+  first_project:
+    title: Track speculative positioning in one futures market
+    goal: Test whether weekly COT records can power a bounded positioning monitor.
+    steps:
+      - Keep report date, market name, open interest, and noncommercial long/short.
+      - Compute net noncommercial positioning and note the publication lag from Tuesday.
+      - Explain that trader categories are CFTC reporting classes, not a forecast of price direction.

data/datasets/cms-nursing-homes.yaml

@@ -0,0 +1,92 @@
+id: cms-nursing-homes
+name: CMS Nursing Home Compare
+description: >
+  Medicare-certified nursing home ratings, staffing, and inspection records
+  for building local long-term-care comparison tools.
+theme: Health, Food & Safety
+url: https://data.cms.gov/provider-data/api/1/metastore/schemas/dataset/items/4pq5-n9py
+access_type:
+  - api
+  - download
+api_key_required: false
+free_to_access: true
+size_gb_min: 0
+size_gb_max: 0.2
+formats:
+  - JSON
+  - CSV
+license: CMS public-data principles
+license_url: https://www.cms.gov/data-research/cms-data/cms-data-principles-and-operating-norms
+url_checks:
+  source_marker: Provider Information
+  license_marker: CMS data is a public good
+domains:
+  - Health Care
+  - Health Services
+  - Long-Term Care
+data_types:
+  - Tabular
+  - Geospatial
+tasks:
+  - Provider Comparison
+  - Geographic Analysis
+  - Quality Monitoring
+difficulty: beginner
+geography:
+  - United States
+temporal_coverage: current Medicare-certified nursing homes
+update_frequency: monthly
+provider: Centers for Medicare & Medicaid Services
+source_type: government
+last_verified: 2026-08-18
+getting_started:
+  overview: >
+    Nursing Home Provider Information lists certified facilities with Five-Star
+    ratings, staffing, and inspection dates. Start with one state and the
+    stable dataset identifier. Ratings can be missing, inspections lag events,
+    and a star rating is not a clinical recommendation.
+  prerequisites:
+    - Python 3.10 or newer
+    - A notebook environment such as Jupyter or Google Colab
+    - An internet connection
+  access_steps:
+    - Open the Provider Information dataset and read the data dictionary.
+    - Query the stable dataset identifier for a bounded page of facilities.
+    - Preserve CMS Certification Number values as text.
+  python:
+    packages:
+      - pandas
+      - requests
+    code: |
+      import pandas as pd
+      import requests
+
+      response = requests.get(
+          "https://data.cms.gov/provider-data/api/1/datastore/query/4pq5-n9py/0",
+          params={
+              "limit": 20,
+              "conditions[0][property]": "state",
+              "conditions[0][value]": "CA",
+              "conditions[0][operator]": "=",
+          },
+          timeout=30,
+      )
+      response.raise_for_status()
+      homes = pd.DataFrame(response.json()["results"])
+      print(
+          homes[
+              [
+                  "cms_certification_number_ccn",
+                  "provider_name",
+                  "state",
+                  "overall_rating",
+              ]
+          ].head(20)
+      )
+  first_project:
+    title: Compare nursing homes within one state
+    goal: Summarize Five-Star ratings and staffing for facilities in one state.
+    steps:
+      - Filter records by state while preserving CMS Certification Numbers as text.
+      - Summarize overall ratings and report missingness before ranking facilities.
+      - Explain why inspection lag, missing ratings, and resident mix limit a quality comparison.