Skip to content

fix(upcoming): let a single request run the daily sync - #563

Open
Dx-Alz-xD wants to merge 1 commit into
CodeChefVIT:stagingfrom
Dx-Alz-xD:krish/upcoming-sync-race
Open

Dx-Alz-xD wants to merge 1 commit into
CodeChefVIT:stagingfrom
Dx-Alz-xD:krish/upcoming-sync-race

Conversation

@Dx-Alz-xD

Copy link
Copy Markdown

📌 Purpose

Requests that arrive together right after the cutoff could all see an old lastSyncedDate and all rebuild the upcoming list. That left duplicate subjects, or briefly an empty list (404) between the delete and the insert.


Corresponding issue: closes #556


🖼️ Showcase

20 simultaneous requests to /api/upcoming-papers on a local copy (3 runs each)

  • staging: 32 48, 144 and 304 subject documents==> some requests got 404
  • this branch: always 16 documents, every request 200

🔧 Changes

-the day is claimed with an atomic findOneAndUpdate on lastSyncedDate; only the request that wins rebuilds the list

  • new subjects are inserted first, then the old ones are deleted, so the list is never empty mid-update

➕ Additional Notes

  • The very first sync (no UpcomingSlot document yet) still uses the upsert as before

two requests right after the cutoff could both see an old
lastSyncedDate and both rebuild the list, leaving duplicate subjects or
an empty list between the delete and the insert. claim the day with an
atomic update first, and insert the new subjects before deleting the
old ones

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Upcoming papers sync can run twice at once and duplicate or empty the list

1 participant