Skip to content

ENG-55756 Fix data race in CloneForSummary#86

Open
and-bezzir wants to merge 1 commit into
masterfrom
ENG-55756-fix-data-race
Open

ENG-55756 Fix data race in CloneForSummary#86
and-bezzir wants to merge 1 commit into
masterfrom
ENG-55756-fix-data-race

Conversation

@and-bezzir

@and-bezzir and-bezzir commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes a data race in view.(*Statelet).CloneForSummary that caused a SIGSEGV (nil pointer dereference) panic in datly-platform-production.

During a read, queryInBatches shares one *Statelet between two concurrent goroutines:

  • the summary goroutine — querySummaryCloneForSummary, which reads Statelet.Filters
  • the object-query goroutine — queryObjectsBuild, which grows Statelet.Filters via AppendFilters (service/reader/sql.go)

AppendFilters mutates the slice under filtersMu, but CloneForSummary read it without the lock. When append reallocated the backing array, the clone could observe a torn slice header (new len + stale/nil ptr) and copy off a nil pointer, faulting at addr=0x0.

Observed in prod:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0]
view.(*Statelet).CloneForSummary               view/state.go:183
service/reader.(*Service).querySummary         service/reader/service.go:280
service/reader.(*Service).queryInBatches.func1 service/reader/service.go:435

Changes

  • view/state.go — acquire filtersMu around the Filters read in CloneForSummary. The mutex itself is still not copied.
  • view/state_race_test.go — race tests driving the real CloneForSummary (read) and AppendFilters (write) on a shared Statelet:
    • TestStatelet_SummaryClone_Fixed_NoRace — passes under -race.
    • TestStatelet_SummaryClone_Unsafe_ReproducesRace — reproduces the original race (skipped by default; gated behind REPRODUCE_STATELET_RACE=1).

Test plan

  • go test -race -run TestStatelet_SummaryClone_Fixed_NoRace ./view/ passes
  • REPRODUCE_STATELET_RACE=1 go test -race -run TestStatelet_SummaryClone_Unsafe_ReproducesRace ./view/ reports the data race (read in clone vs write in AppendFilters)
  • go vet ./view/ clean

@and-bezzir and-bezzir requested a review from adranwit June 25, 2026 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant