Skip to content

feat(configs): add EU/Nordic English newsroom feeds - #315

Merged
gildesmarais merged 3 commits into
masterfrom
feat/eu-nordic-configs
Aug 17, 2026
Merged

feat(configs): add EU/Nordic English newsroom feeds#315
gildesmarais merged 3 commits into
masterfrom
feat/eu-nordic-configs

Conversation

@gildesmarais

Copy link
Copy Markdown
Member

Summary

  • Add 20 English newsroom/jobs configs for NL/FI/DK/SE/NO plus pan-EU EURES, skipping first-party RSS surfaces.
  • Tighten asml.com/press-releases.yml (year-anchored item hrefs, span title, Europe/Amsterdam) and register four JS-rendered feeds for Botasaurus fetch.
  • Drop ui.se/news.yml after sense-check: the English news list only had two cards.

What changed

  • New Faraday configs: diis.dk/research.yml, equinor.com/news.yml, eures.europa.eu/news.yml, ikea.com/newsroom.yml, lego.com/press-releases.yml, maersk.com/press-releases.yml, nbim.no/press-releases.yml, nokia.com/newsroom.yml, novonordisk.com/latest-news.yml, nupi.no/news.yml, telenor.com/newsroom.yml, tennet.eu/news.yml, tno.nl/news.yml, tudelft.nl/news.yml, tue.nl/news.yml, vattenfall.com/newsroom.yml.
  • New Botasaurus configs (strategy: botasaurus): aalto.fi/news.yml, academictransfer.com/jobs.yml, chalmers.se/news.yml, visitfinland.com/press-releases.yml.
  • spec/support/botasaurus_fetch_configs.rb — sorted registration of those four Botasaurus paths.
  • lib/html2rss/configs/asml.com/press-releases.yml — item selector now requires /en/news/press-releases/20, title via span.related-content-card-title, timezone Europe/Amsterdam.

Why

Curate stable English newsroom lists for EU/Nordic orgs that do not already ship useful first-party RSS. Faraday is the default; Botasaurus only where static fetch is empty or JS-gated. ui.se was not worth shipping at two English items.

Risk

  • Selector drift on corporate newsrooms (card class / path fragment changes).
  • Botasaurus-backed feeds (aalto.fi, academictransfer.com, chalmers.se, visitfinland.com) need BOTASAURUS_SCRAPER_URL in the fetch lane.
  • Locale/path dependence on /en/ or English newsroom URLs.

Review map

  1. spec/support/botasaurus_fetch_configs.rb — confirm only the four JS configs are added; Visit Finland / AcademicTransfer stay registered.
  2. lib/html2rss/configs/academictransfer.com/jobs.yml and lib/html2rss/configs/visitfinland.com/press-releases.yml — Botasaurus wait selectors and wait_timeout_seconds ≤ 20.
  3. lib/html2rss/configs/aalto.fi/news.yml and lib/html2rss/configs/chalmers.se/news.yml — remaining Botasaurus newsrooms.
  4. lib/html2rss/configs/asml.com/press-releases.yml — existing-feed tightening (year-anchored hrefs).
  5. Faraday cluster (tennet.eu, tudelft.nl, tue.nl, tno.nl, energy/consumer newsrooms, eures.europa.eu) — minimal items/title/url, enhance: false where set.

Validation

  • make validate — exit 0 (144 configs).
  • make test — exit 0 (1460 examples, 0 failures; fetch excluded).
  • Focused fetch not run for this deletion/PR open (optional; ui.se was dropped).

Test plan

  • Spot-check RSS from a Faraday config (html2rss feed lib/html2rss/configs/tennet.eu/news.yml) for non-zero items and no nav leakage.
  • Spot-check one Botasaurus config with BOTASAURUS_SCRAPER_URL=http://localhost:4010.
  • Confirm ui.se is absent from lib/html2rss/configs/ and from botasaurus_fetch_configs.rb.
  • Confirm Visit Finland and AcademicTransfer remain in the Botasaurus registry.

Curate Faraday/Botasaurus configs for NL/FI/DK/SE/NO plus pan-EU jobs, skipping first-party RSS surfaces. Drop ui.se after sense-check (only two items).
Turn enhance on for ten EU/Nordic feeds whose leftover card text is a real dek; leave it off where extractors pick CTAs, labels, or date-only chrome.
@gildesmarais
gildesmarais merged commit 90f4322 into master Aug 17, 2026
5 of 7 checks passed
@gildesmarais
gildesmarais deleted the feat/eu-nordic-configs branch August 17, 2026 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant