Bug: fluid.bufstats~ can crash when mixing threads

[rconstanzo]

Please tell us what you were doing! You can include code and files by drag and dropping them into the text area.

It seems that fluid.bufstats~ can insta-crash when it is getting messages from mixed threads, specifically receiving weights in it's right-most inlet in the high-priority thread, while the object is in @blocking 1 and then also receiving buffers upstream from the high-priority thread.

So I think the steps are:

1. have fluid.bufstats~ @blocking 1
2. send weights buffer to right inlet from high-priority thread
3. send buffer into left inlet from high-priority thread

Sometimes this gives an Invalid weights error message (as mentioned below) but other times it will just crash, presumably when read/writing from unsafe memory.

This is happening in the patch outlined in this thread:
https://discourse.flucoma.org/t/creating-a-filter-from-spectral-moments-fluid-spectralshape/

Specifically this version of the patch:
https://discourse.flucoma.org/uploads/short-url/nkoDHX3alGOt8e2sudrKHQsmq3J.zip

The offending abstraction is dk.spectralshapeframe which has three instances of fluid.stats~ in it, and only one of which was having it's @blocking mode updated automatically (I missed the connection when building the patch).

If you run the patch for any amount of time you will see the error messages, and eventually a crash.

Here's a crash report:

bufstats crash.zip

What was the expected result?

Complaining about threading, which it did do as well:

fluid.bufstats~: Invalid weights

But sometimes it would not complain and instead crash.

What was the actual result?

An insta-crash.

What operating system were you using?

Mac

Operating system version

macOS 15.2

FluCoMa Version

1.0.8+sha.0a9327e.core.sha.7dfcc220

[weefuzzy]

Sorry, never saw this when you filed it. It's either some horrible threading mystery, or maybe the inputs for the weights buffer aren't being deferred as they should be. Will investigate

[tremblap]

as far as I could make sense of it, it was the same strange race condition that @AlexHarker was sorting in summer 2023 in a PR not finished yet because it implied deep long testing...

[rconstanzo]

Yeah, I guess there's some threading thing not being accounted for in @blocking 1 based on different threads arriving at each input. I imagine the right-most inlet is being treated differently here and there are less safety checks?

And/or it's a bigger issue like @tremblap is pointing towards.

[AlexHarker]

@rconstanzo - can you confirm that your description is correct that both inputs are being made a high priority?

[rconstanzo]

It looks like the loudness descriptors coming into the right inlet were in fact low priority, so it was just the left inlet of fluid.bufstats~.

[AlexHarker]

That would be a more likely candidate for a threading bug. Can I suggest that you to try moving all messages to high priority and testing to see if the crash (or warning) occur?

The crash log is not super informative about the crash - it seems to be crashing in more generic code, which is sometimes a sign of memory stamping (which may or may not be threading related), but it's also a bad instruction error (which is less usual). At present I'd say it is possibly an issue of threading, but more evidence would be helpful.

[rconstanzo]

Do you mean specifically the right inlet (vs the left)? And while in @blocking 1?

If I move everything to high-priority (with @blocking 2) then it doesn't crash. And prior to this I had seen the error when threading was incorrect, but never a crash.

[AlexHarker]

I would have both inputs at the same priority (might be worth testing both) and in the blocking mode in which you experienced the crash. Testing the original patch with override off might also be informative.

[rconstanzo]

I started off making a post that I wasn't able to get any error messages or crashes then got a single error message:
fluid.bufstats~: already processing

And after letting the patch run for a few minutes it did an insta-crash.

I don't know if the version of Max I have installed now is a bit more robust to some of this, but I want to say the patch as I ran it back when I made the original post was spamming error messages to the console at a rate of a few a minute at least.

Thread 0 Crashed:: CrBrowserMain Dispatch queue: com.apple.main-thread
0   fluid.bufstats~               	       0x318f6f82c fluid::client::Result fluid::client::bufstats::BufferStatsClient::process<float>(fluid::client::FluidContext&) + 88
1   fluid.bufstats~               	       0x318f6f580 fluid::client::NRTThreadingAdaptor<fluid::client::ClientWrapper<fluid::client::bufstats::BufferStatsClient>>::ThreadedTask::process(std::__1::promise<void>) + 68
2   fluid.bufstats~               	       0x318f6f360 fluid::client::NRTThreadingAdaptor<fluid::client::ClientWrapper<fluid::client::bufstats::BufferStatsClient>>::ThreadedTask::ThreadedTask(std::__1::shared_ptr<fluid::client::ClientWrapper<fluid::client::bufstats::BufferStatsClient>>, fluid::client::NRTThreadingAdaptor<fluid::client::ClientWrapper<fluid::client::bufstats::BufferStatsClient>>::NRTJob&, bool) + 424
3   fluid.bufstats~               	       0x318f7f1f0 fluid::client::NRTThreadingAdaptor<fluid::client::ClientWrapper<fluid::client::bufstats::BufferStatsClient>>::process() + 352
4   fluid.bufstats~               	       0x318f7efd0 fluid::client::impl::NonRealTime<fluid::client::FluidMaxWrapper<fluid::client::NRTThreadingAdaptor<fluid::client::ClientWrapper<fluid::client::bufstats::BufferStatsClient>>>>::process() + 96

crash.zip

This is with high-priority messages going into both inlets of fluid.bufstats~ @blocking 2, with overdrive ON.

Letting in run with overdrive OFF for a while but got no crash or error message. If I check the thread coming into the offending patch, with overdrive OFF, it is coming in in low-priority, which I guess makes sense.

I wonder if there's multiple things going on. A generic threading one with high priority messages going into a low priority fluid.bufstats~ and something where descriptor inlet and weights inlets coming in as different threads are also triggering some (related but different) funny business.

[weefuzzy]

If it's sad in @blocking 2 (YOLO mode), then I suspect it's some awful interaction with a buffer operation that Max will always defer (like resizing).

[rconstanzo]

Well in this case the crash is happening from fluid.bufoudness~ @blocking 2 -> fluid.bufstats~ @blocking 1.

Critically, there's a single lap of @blocking 1 with everything so the required buffer~s are made, but I made a mistake patching and didn't switch over some, leading to this high->low priority funny business.