Repository navigation
Stop storing 'Unknown' as a venue city: derive it from the address - #3732
Open
jakebromberg wants to merge 2 commits into
Open
jakebromberg wants to merge 2 commits into
jakebromberg wants to merge 2 commits into
Conversation
…S#3731)
Clients print venue.city verbatim, so the placeholder reached listeners as a city (iOS showed "Unknown" under "Cat's Cradle Presents at 107 Brewer Lane").
The RHP scraper's unseeded ensureVenue path now takes the city from the JSON-LD address ("107 Brewer Lane, Carrboro, North Carolina" -> Carrboro), or '' when the address names none. On conflict it writes only city, and only while the row still holds a placeholder ('Unknown' or ''), so existing placeholder rows heal on the next run and a hand-corrected city is never overwritten. The triangle-shows ETL's missing-city fallback is '' instead of 'Unknown'.
…e city, and always clear a legacy 'Unknown' (BS#3731)
cityFromAddress picked the part before the last comma, so a trailing country stored the state ('NC') and a two-part address stored the street. Because the writer never corrects a city it has filled in, a wrong guess stuck, and a long one overflowed varchar(64) and dropped the venue's concerts. It now returns the canonical spelling of the right-most part that names a known city (the Etix city list, now KNOWN_CITIES), or null.
The unseeded conflict predicate is now venues.city IN ('Unknown', '') AND venues.city <> city, so a legacy 'Unknown' becomes '' even when the address names no city. Also fixes the stale created docstring and the DO NOTHING test name.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3731
Why
Clients print
venue.cityverbatim, and the backend stored the literal string'Unknown'as a city. The iOS Box Office ticket showed Unknown under "Cat's Cradle Presents at 107 Brewer Lane" (venue 2964).What changed
jobs/venue-events-scraper:ensureVenue's unseeded path derives the city from the JSON-LD address with the newcityFromAddress("107 Brewer Lane, Carrboro, North Carolina"→Carrboro), and stores''when the address names none. On conflict it now writes onlycity, and only while the row still holds a placeholder ('Unknown'or'') and the address yields one. Every other column keeps the existing never-overwrite rule, and a hand-corrected city survives. Venue 2964 is lifted to Carrboro on the next nightly run, with no manual SQL.jobs/triangle-shows-etl: the on-demand venue's missing-city fallback is'', not'Unknown'.rhp-venues.tscomment describe the new placeholder row.venues.cityisvarchar(64) NOT NULL, and''satisfies it, so no migration is needed.createdon the unseeded path now comes fromxmax = 0, like the seeded path, because the conflict branch can update.Not included
Rows the triangle-shows ETL already stored as
'Unknown'stay as they are until their source sends a city. Clearing them is a one-time productionUPDATE, which needs separate approval (see #3731).Testing
cityFromAddressover live address shapes and rejections; unseededensureVenuepins the inserted city, the conflictset, and the placeholder-onlysetWhere; the ETL pins''for a null, empty, or blank city.tscfor both jobs, andtypecheck:tests(baseline matches).