fix: resolve bigint timestamps and event upsert race conditions#965
fix: resolve bigint timestamps and event upsert race conditions#965chizzy192 wants to merge 1 commit into
Conversation
## Description This commit resolves two issues: 1. Migrates time-valued fields (startTime, lastUpdateTime, endTime, pausedAt, StreamEvent.timestamp) from Int to BigInt to prevent 2038 overflow (and early overflow on long-duration streams). 2. Prevents unique constraint / write-race collisions between action controllers and the indexer worker on `StreamEvent` records by using `upsert` in controllers, and conditional updates in the worker when a placeholder event exists. ## Type of Change - [x] 🐛 Bug fix (non-breaking change which fixes an issue) - [x] 🧪 Test addition or update ## Related Issues Closes LabsCrypt#830 Closes LabsCrypt#831 ## Changes Made - Modified `backend/prisma/schema.prisma` to change time columns to BigInt. - Added database migration `bigint_timestamps`. - Added global BigInt JSON serializer to `backend/src/app.ts`. - Updated `ClaimableStreamState` in `claimable.service.ts` and updated calculations to support bigint. - Modified withdraw, pause, resume controllers to use `upsert` and pass BigInt values. - Updated indexer worker event handlers to conditionally update ledgerSequence/timestamp on existing placeholder records. - Updated unit and integration tests to mock and expect `upsert`. - Fixed sunset/deprecated route tests. ## Testing - [x] Unit tests added/updated - [x] Integration tests added/updated ### Test Steps 1. Run `DATABASE_URL=postgresql://flowfi:flowfi_dev_password@127.0.0.1:5433/flowfi npm test`
|
heads up: main's ci was broken (the Backend CI and Backend Docker Image CI jobs) until the fixes in #969 and #974 just landed, so the red backend/docker checks on this pr are almost certainly stale, they ran against the broken main. please rebase to re-test against the now-green main: |
|
the bigint-timestamp + sqlite->postgres migration conversion + the event-upsert race fix look legit, and the only red is the Docker Image CI at the container health-check step ('could not find prisma schema'), which is known infra noise, not your code, everything else passes. please rebase/re-run against the current green main, and if the Docker step still fails after that i'll dig in: |
closes #830 closes #831
Description
This commit resolves two issues:
StreamEventrecords by usingupsertin controllers, and conditional updates in the worker when a placeholder event exists.Type of Change
Related Issues
Closes #830
Closes #831
Changes Made
backend/prisma/schema.prismato change time columns to BigInt.bigint_timestamps.backend/src/app.ts.ClaimableStreamStateinclaimable.service.tsand updated calculations to support bigint.upsertand pass BigInt values.upsert.Testing
Test Steps
DATABASE_URL=postgresql://flowfi:flowfi_dev_password@127.0.0.1:5433/flowfi npm testDescription
Type of Change
Related Issues
Closes #
Changes Made
Testing
Test Coverage
Test Steps
Breaking Changes
Breaking Changes:
Migration Guide:
Screenshots/Demo
Checklist
Additional Notes