feat(weekly-rotation): instrument plays, shares and notifications for usage analytics - #14605
Merged
Merged
Conversation
…play state - Cap the Priority/General tab backfill at 5 pages per tab visit (web and mobile) so a near-empty tab doesn't page through the whole chat history. - Raise the chat row swipe activation offsets to 30pt so diagonal vertical scrolls stay with the list. - OTA banner: stop an in-flight poll from rescheduling after stop/restart. - Weekly Rotation: don't treat a shared mix as your own while its handle is loading or not found, and key playback state per mix owner so your own mix and a shared one don't share the header play button. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Shorten comments to plain statements, fix stale api references in the weekly rotation period helper, and align the weekly rotation notification body with the push copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… usage analytics - Add PlaybackSource/FavoriteSource/RepostSource WEEKLY_ROTATION and use them on the web page and mobile screen lineups and header play. - Passive plays from a mix carry weeklyRotationOwnerId, derived from the queue source. - Weekly Rotation banner, page view and play-all events carry period, isOwnMix and ownerUserId; page view adds status and isSignedIn and now fires for empty mixes and fetch errors. - Share modal opens record Modal Opened (name Share) with kind and id. Mobile copy-link and X shares for weeklyRotation record Share and Share to Twitter; Share gains a channel property and Chat Entry Point a kind. - Mobile notification tile tap records Notifications: Clicked Tile, and push opens record the payload type and id. - Web banner CTA no longer fires the click event twice (the button click also bubbled to the card). - Move the period helper and queue source into common. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
…analytics # Conflicts: # packages/mobile/src/screens/weekly-rotation-screen/WeeklyRotationScreen.tsx # packages/web/src/pages/weekly-rotation-page/WeeklyRotationPage.tsx # packages/web/src/utils/weeklyRotationPeriod.test.ts # packages/web/src/utils/weeklyRotationPeriod.ts
…analytics # Conflicts: # packages/web/src/components/weekly-rotation/WeeklyRotationBanner.tsx
dylanjeffers
added a commit
that referenced
this pull request
Sep 24, 2026
…14606) The Amplitude proxy (gain2.audius.co, metrics-worker) caps us at 1.95M events a month, and it has been hitting that in about 4 days. This PR removes most of the events the clients send: anything that's a duplicate, automatic, from a bot, or not used by any recent chart. What's left, apart from the core events, goes out from 10% of devices. ## Volume before (Amplitude stored totals, Aug 31 to Sep 4, 5 days, project 265616) | # | Event | Stored | Status after this PR | |---|---|---:|---| | 1 | Session Start | 118,654 | **removed**, replaced by a `client` user property | | 2 | Page View | 44,992 | kept (core; also used by sign-up charts) | | 3 | Playback: Play | 38,170 | kept (core) | | 4 | Search: Search | 23,636 | **removed** | | 5 | Embed: Open Player | 23,015 | **removed** | | 6 | Favorite | 20,708 | kept (core) | | 7 | Buffering Time | 18,473 | **removed** | | 8 | Repost | 17,618 | kept (core) | | 9 | Explore: Section View | 11,997 | **removed** | | 10 | Follow | 9,718 | kept (core) | | 11 | Playback: Pause | 8,047 | kept (core) | | 12 | Link Click | 7,340 | **removed** | | 13 | session_end | 7,279 | **removed** (SDK automatic tracking off) | | 14 | Share | 5,821 | kept (core) | | 15 | Track Upload: Track Uploading | 4,094 | **removed** | | 16 | Message Unfurl: Track | 3,966 | **removed** | | 17 | [Amplitude] Form Started / Submitted | 3,798 | **removed** (SDK automatic tracking off) | | 18 | Create Account: Select Genre | 3,786 | kept (core, sign-up) | | 19 | Feed: Change view | 3,298 | sampled at 10% (Chronological Feed dashboard) | | 20 | Comments: Load More Comments | 3,222 | **removed** | Web plus embed is 314k of that, iOS 66k and Android 64k. Some events are sent but never stored, so they don't show up above: `[Amplitude] Page Viewed`, `session_start` (both deleted in the project) and `$identify`. ## What changed ### 1. SDK automatic events off, and bots skipped - **Web `defaultTracking`:** it now names every option. Any option left out defaults to on (`isTrackingEnabled` in analytics-client-common), so `{ sessions: true }` had also been sending page views, form events and file downloads. Attribution stays on. - **Bots:** crawler and `navigator.webdriver` traffic gets no analytics on web or embed. On Aug 1-3, one Singapore "Chrome 145" crawler sent 371k Session Starts from 347k new devices. - **Identify dedupe:** identify only goes out when the traits change, on web and mobile, and is resent weekly. ### 2. Session Start removed everywhere The platform it carried (`Android Web`, `iOS Web`, `MacOS`, `Windows`, `Linux`, `Desktop Web`, `iOS App`, `Android App`, `Embed`) is now a **`client` user property**. It's set through identify only when it changes for a device. Amplitude already computes sessions from session ids. ### 3. Events removed (call site, enum entry and type) The rule: an event is removed if it isn't core and isn't referenced by an Amplitude chart, dashboard or cohort that was viewed or edited in the last 12 months. I listed all 642 charts, dashboards, cohorts and notebooks in project 265616, then read the definitions of the 33 charts and 7 dashboards used since 2025-09-24. None of the 11 cohorts or 3 notebooks had been used in that window. - **Session and diagnostics:** Session Start, App Error, App Unexpected Error, Buffering Time, Not Found Page, Chat Websocket Error, Modal Opened/Closed - **Search and navigation:** Search: Search, Tag Search, Result Select, Tag Click, Link Click, Explore: Section View, Section Click, Trending: Change view, Profile Page: Tab Click, Sort, the social and website link clicks, Shown Artist Recommendations, Track Page: Play More By This Artist, Visualizer: Open/Close - **Embed:** Open Player, Player Error, Open modal - **Comments (all interaction events):** create, update, delete, like, report, mute, pin, reply, mention, timestamp and link clicks, load more/new, sort, drawer open/close, overflow menus, notification toggles, Comments History, Recent Comments - **Chat:** Create Chat, Chat Blast CTA/Create/Message Viewed, Send Message: Failure, Delete Chat, Set Chat Category, Block User, Change Inbox Settings, Send Message Reaction, Message Unfurl: Track/Playlist, Chat Entry Point (see note) - **Playlists and library:** Playlist: Add To Playlist, Open/Start/Complete Create, Make Public, Open Edit From Sidebar, Play; Playlist Library: Reorder; Folder: Open/Submit/Cancel Edit - **Tracks:** Track Download (all 6), Track Edit (all 3), Collection Edit (both), Track Replace (all 3), Track Upload: Track Uploading, View Track page, the follow and token gated upload variants; Follow/Token Gated unlocks; Stem: Complete Upload, Delete; Delete; Remix: New Remix, CoSign, CoSign Indicator, Hide; Artist Pick: Select Track - **Remix contests:** Create, Update, Delete, Pick Winners Open, Finalize Winners, View, Enter, View Submissions - **Settings and account:** Change Theme, Resend Account Recovery, Complete Change Password, Log Out, Deactivate Account (all 4), Developer Apps (all 8), Authorized Apps (both), Manager Mode (all 5), Export Private Key (all 5), Notifications: Toggle Setting, Browser Push Notification, Account Health: Upload Cover Photo, Upload Profile Picture, Download Desktop - **Wallet and rewards:** Connect Wallet (all 5), Create User Bank (both), Rewards Claim: Opened/Request/Success, Rewards Claim All: Failure - **Banners and prompts:** Banner Artist Coins/Yak Coin/Trading Volume/TOS clicks, Rate CTA (all 3) - **Mobile offline mode:** everything except Offline Play (the migration and download lifecycle events) The analytics-only `source` props on SearchTag, ExternalLink and UserGeneratedText (`linkSource`) are removed too. ### 4. Sampling - **Core allowlist:** `CORE_ANALYTICS_EVENTS` in `@audius/common` (`models/AnalyticsSampling.ts`), plus the mobile share-channel events. - **What's core:** Playback Play/Pause, Listen, Page View, and the sign-up, sign-in, create-account and OAuth flows. Also purchase, buy, withdraw, USDC, Stripe, coin swaps and Launchpad; the upload open/start/complete/success/failure funnel; Share and share channels (including the notification and first-upload Twitter shares); Favorite, Repost and Follow and their undos; notification opens and clicks (including push and comment notifications); chat sends (Send Message: Success, Chat Blast: Message Sent); Report User: Chat (the only record of an abuse report); Rewards Claim All request/success; Embed: Copy; and all Weekly Rotation events. - **Everything else:** sent only from devices where an FNV-1a hash of the Amplitude device id falls in the bottom 10%. It carries `sampleRate: 0.1`, so counts scale back up by dividing by `sampleRate`. What's still sampled after step 3 is only chart-backed: Play Queue (all 7), Feed: Change view, Error Page, Account Health: Meter Full, Folder: Delete, the Playlist Library folder moves, Offline Play, and Chat Entry Point. - **Where it runs:** once, in the shared track path: web `services/analytics/amplitude.ts` and mobile `services/analytics.ts`. Embed now only sends core events. ## Why the proxy counts about 5.5x what Amplitude stores - **Confirmed:** the proxy counted events that Amplitude drops without billing: session_start and [Amplitude] Page Viewed. It also counted `$identify` on every account load. Together that's roughly 50-70k a day. - **Inferred:** the other ~330k a day. The proxy counts a batch before forwarding it, so every SDK resend counts again (after 400/413/429/5xx responses, network errors, and the localStorage queue being replayed on the next load). Amplitude removes duplicates by `insert_id`, so it bills only one copy. AudiusProject/metrics-worker#1 gives back events Amplitude doesn't accept, and now also drops the removed event types that older clients still send. ## Estimate against the 1.95M cap | | Per day now | Per day after | Per month after | |---|---:|---:|---:| | Amplitude stored (billed) | ~100k | ~35k (core 35.0k + sampled 0.1k) | **~1.05M** | | Proxy count, if the gap is retries of rejected batches (fixed by worker#1) | ~488k | ~34k events + ≤8k identify | **~1.3M** | | Proxy count, if the gap is duplicates Amplitude accepts with a 200 | ~488k | ~34k × 4.8 + 8k | ~5.2M | - **Billed volume ends up under 1.95M either way**, with about 45% headroom. Kept core events stored 175k over the 5 days. The sampled events stored 4.7k and will keep 10% of that. - **The proxy count ends up under 1.95M only if** the unexplained gap is retries of rejected batches. After deploying metrics-worker#1, compare the counter against Amplitude's daily totals for a few days to find out. - **Rollout timing:** web doesn't auto-deploy (Production gate), and mobile needs an OTA dispatch. Until those ship, metrics-worker#1's denylist is what stops old clients from spending the budget. ## Kept even though it's low value - **Undo actions:** Unfavorite, Undo Repost and Unfollow. They're the inverse of core actions and about 1k in 5 days. - **Upload USDC-gated variants and USDC unlock events:** grouped with purchase. Tiny volume. - **Launchpad form-change events:** part of the coin flow and nearly zero volume. - **Chat Entry Point:** still fired from ShareModal and ShareDrawer, which #14605 is changing. The enum and type stay so that merges cleanly. It's sampled on the client and dropped by the proxy. Remove it after #14605 lands. - **A few enum entries and types stay** next to lines #14605 changes, even though nothing fires them anymore: Modal Opened/Closed, Explore: Section Click, Browser Push Notification, Chat Websocket Error, Message Unfurl, Playlist: Play, Stem: Complete Upload, Export Private Key link. They can be deleted after #14605 merges. ## Behavior changes to review - **Sign-out:** Log Out used to run through the analytics callback. It's now called directly on desktop web and in the mobile web SignOutModal, still before the redirect. - **Modal sagas:** `store/ui/modals/sagas.ts` now returns no sagas. `trackModalOpened`/`trackModalClosed` still get dispatched and now do nothing. - **Wallet-connect errors:** the catch blocks in the mobile wallet-connect sagas only sent an error event, so they're now empty. - **Mobile Chat screen:** the blast "viewed" AsyncStorage bookkeeping was only for the removed event, so it's gone. ## Tests - **Typecheck:** `tsc` passes for common, web and mobile. The local sdk and spl dist were rebuilt, because they were stale. - **Lint:** eslint on all touched files has no errors. Two `exhaustive-deps` warnings in the remix-contest host were already there. - **Web vitest:** 33 files and 177 tests pass. That includes `services/analytics/amplitude.test.ts`: automatic events off, client property set once per device, core events always sent, about 10% of devices kept for non-core events with `sampleRate`, identify dedupe, and bots skipped. - **Common vitest:** 172 of 179 pass. The 7 failures are in `store/upload/selectors.test.ts`, which fails the same way on an untouched checkout. - **Embed jest:** passes. - **Mobile jest:** can't run in this worktree, because react-native can't be resolved from the copied node_modules. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #14602 (
fix/review-cleanup-sep). Merge that first; this PR's base will then retarget tomain.Instruments Weekly Rotation so reach, click-through, plays, shares, notification opens and retention can be measured in Amplitude. No existing events are renamed.
Events and properties
Common
PlaybackSource,FavoriteSourceandRepostSourceeach gainWEEKLY_ROTATION = 'weekly rotation'.Playback: PlaygainsweeklyRotationOwnerId. It is set on passive (auto-advance and previous) plays whose queue entry came from a mix, and is parsed from theWEEKLY_ROTATION_TRACKS:<ownerId>queue source in the shared playback saga, so it works on both web and mobile.Weekly Rotation: Banner View,Banner Click,Page ViewandPlay Allgainperiod(e.g.2026-38),isOwnMixandownerUserId. Banner events are alwaysisOwnMix: true, with the viewer as owner.Weekly Rotation: Page Viewgainsstatus(success|empty|error) andisSignedIn. It now fires once the fetch settles, so empty mixes and errors are counted. Signed-out visitors on/explore/weekly-rotation/:handleare counted too.Modal Openednow fires withname: 'Share'whenever the share modal or drawer opens (from the shared share-modal saga, any content type). It carriessource(the ShareSource),kind(e.g.weeklyRotation) and the entity id (userIdfor a mix).Sharegains an optionalchannel(copyLink|shareSheet).Chat Entry Pointgainskind(what was shared) whensource: 'share'.Notifications: Open Push Notificationgainstypeandidfrom the push payload. This applies to every push type.@audius/common/utils(getWeeklyRotationPeriod,formatWeeklyRotationPeriod), along withgetWeeklyRotationQueueSourceandgetWeeklyRotationOwnerIdFromQueueSource. The web SSR still imports it by path, and the file has no dependencies.Web
source: 'weekly rotation'. Before this they used'playlist page'or'track tile'.'weekly rotation'.TrackTableLineuptakesfavoriteSourceandrepostSource, and History keeps its default.Sharewithchannel: 'copyLink'. DM share recordsChat Entry Pointwithkind.Banner Clickwas recorded (and navigation ran) twice. Only the card handles the click now.Mobile
Playback: PlayandPlayback: Pausewithsource: 'weekly rotation'. The header play button also records them. Before this, mobile lineup taps recorded no Playback events:TrackLineupacceptedplaybackSourcebut ignored it. It now records Playback events only when that prop is passed, and only this screen passes it, so other screens are unchanged.'weekly rotation'.TrackLineupandTrackTiletakefavoriteSourceandrepostSource, defaulting toTILE.Share(channel: 'copyLink'), the share sheet recordsShare(channel: 'shareSheet'), and X recordsShare to Twitter. DM share recordsChat Entry Pointwithkind.Notifications: Clicked Tilewithkind: 'WeeklyRotation', matching web.Not covered
overflowon both platforms. Adding a source there means threading it through the table,OverflowMenuButtonandTrackMenuon web, and through the overflow drawer state on mobile.Playlist Addhas no source property at all. Row and tile buttons are covered.How to measure (Amplitude)
Weekly Rotation: Banner View, uniques, grouped bysurfaceandsourceWeekly Rotation: Banner View→Weekly Rotation: Banner Click, samesurfaceWeekly Rotation: Page Viewgrouped byisOwnMix; filterstatus = successfor non-empty viewsWeekly Rotation: Page ViewwhereisOwnMix = false, grouped byisSignedIn(signed-out = new-visitor landings)Weekly Rotation: Page Viewgrouped bystatusPlayback: Playwheresource = weekly rotationORweeklyRotationOwnerIdis set; formulaTOTALS / UNIQUES. Split own vs shared by comparingweeklyRotationOwnerIdto the user, or usePlay All'sisOwnMixPage View→Weekly Rotation: Play AllFavoriteorRepostwheresource = weekly rotationModal Openedwherename = Shareandkind = weeklyRotationSharewherekind = weeklyRotation, grouped bychannel(web copy link, mobile copy link or share sheet);Share to Twitterwherekind = weeklyRotation;Chat Entry Pointwheresource = shareandkind = weeklyRotationNotifications: Open Push Notificationwheretype = WeeklyRotation(idincludes theweekly_rotation:<period>:<user>group). Sends come from theweekly rotation push processedlog line added in AudiusProject/pedalboard#98 (summobileSentpergroupIdperiod)Notifications: Clicked Tilewherekind = WeeklyRotationWeekly Rotation: Page View(orPlay All) with weekly buckets, or group byperiodto line up with the Wednesday rolloverTesting
tscpasses for common, web and mobile.eslintpasses on the touched files.vitest: new tests for the common period and queue-source helpers and for the share-modal saga'sModal Opened. The web OG URL test and tracks-table tests pass./explore/weekly-rotation/dylansigned out and reading Amplitude's unsent-event queue:Weekly Rotation: Page Viewwas recorded with{isOwnMix: false, isSignedIn: false, ownerUserId: "18269", period: "2026-39", status: "success", trackCount: 30}.Weekly Rotation: Play AllandPlayback: Playwithsource: "weekly rotation". Pressing Next recordedPlayback: Playwithsource: "passive", weeklyRotationOwnerId: "18269".Modal Openedwith{name: "Share", kind: "weeklyRotation", source: "page", userId: 18269}. Copy link recordedSharewithchannel: "copyLink".🤖 Generated with Claude Code