feat(configs): add EU/Nordic English newsroom feeds - #315
Merged
Conversation
Curate Faraday/Botasaurus configs for NL/FI/DK/SE/NO plus pan-EU jobs, skipping first-party RSS surfaces. Drop ui.se after sense-check (only two items).
Turn enhance on for ten EU/Nordic feeds whose leftover card text is a real dek; leave it off where extractors pick CTAs, labels, or date-only chrome.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
asml.com/press-releases.yml(year-anchored item hrefs,spantitle,Europe/Amsterdam) and register four JS-rendered feeds for Botasaurus fetch.ui.se/news.ymlafter sense-check: the English news list only had two cards.What changed
diis.dk/research.yml,equinor.com/news.yml,eures.europa.eu/news.yml,ikea.com/newsroom.yml,lego.com/press-releases.yml,maersk.com/press-releases.yml,nbim.no/press-releases.yml,nokia.com/newsroom.yml,novonordisk.com/latest-news.yml,nupi.no/news.yml,telenor.com/newsroom.yml,tennet.eu/news.yml,tno.nl/news.yml,tudelft.nl/news.yml,tue.nl/news.yml,vattenfall.com/newsroom.yml.strategy: botasaurus):aalto.fi/news.yml,academictransfer.com/jobs.yml,chalmers.se/news.yml,visitfinland.com/press-releases.yml.spec/support/botasaurus_fetch_configs.rb— sorted registration of those four Botasaurus paths.lib/html2rss/configs/asml.com/press-releases.yml— item selector now requires/en/news/press-releases/20, title viaspan.related-content-card-title, timezoneEurope/Amsterdam.Why
Curate stable English newsroom lists for EU/Nordic orgs that do not already ship useful first-party RSS. Faraday is the default; Botasaurus only where static fetch is empty or JS-gated.
ui.sewas not worth shipping at two English items.Risk
aalto.fi,academictransfer.com,chalmers.se,visitfinland.com) needBOTASAURUS_SCRAPER_URLin the fetch lane./en/or English newsroom URLs.Review map
spec/support/botasaurus_fetch_configs.rb— confirm only the four JS configs are added; Visit Finland / AcademicTransfer stay registered.lib/html2rss/configs/academictransfer.com/jobs.ymlandlib/html2rss/configs/visitfinland.com/press-releases.yml— Botasaurus wait selectors andwait_timeout_seconds≤ 20.lib/html2rss/configs/aalto.fi/news.ymlandlib/html2rss/configs/chalmers.se/news.yml— remaining Botasaurus newsrooms.lib/html2rss/configs/asml.com/press-releases.yml— existing-feed tightening (year-anchored hrefs).tennet.eu,tudelft.nl,tue.nl,tno.nl, energy/consumer newsrooms,eures.europa.eu) — minimalitems/title/url,enhance: falsewhere set.Validation
make validate— exit 0 (144 configs).make test— exit 0 (1460 examples, 0 failures; fetch excluded).ui.sewas dropped).Test plan
html2rss feed lib/html2rss/configs/tennet.eu/news.yml) for non-zero items and no nav leakage.BOTASAURUS_SCRAPER_URL=http://localhost:4010.ui.seis absent fromlib/html2rss/configs/and frombotasaurus_fetch_configs.rb.