Publish and subscribe from any Julia task - #413
Open
tsaubergine wants to merge 3 commits into
Open
Conversation
Only Main could reach the layers the C++ side owns; every other task threw an AssertionError. Forward those publications to the task that owns the application over the interthread channel it already has, which is what a C++ thread's InterProcessForwarder does with its inner interthread transporter. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016nsf4QMNTfUxkyWh4xGaok
Subscriptions on the layers the C++ side owns were registered from whichever task called subscribe(), and every message came back to Main regardless of who had asked for it. Register them on the task that owns the application, and remember which task each subscription belongs to so its messages are delivered there. The callback registry was a single Function per (layer, scheme, type, group), so two tasks subscribing to the same thing kept only the last one, and it was written without a lock. Key it by task as well, behind the same lock as the delivery map. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016nsf4QMNTfUxkyWh4xGaok
A group the generated C++ has no case for hits GOBY_JULIA_FAIL, which logs at glog's die level and terminates the application. The docstring claimed the constant stops the message going nowhere; nothing was going nowhere. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016nsf4QMNTfUxkyWh4xGaok
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Finishes the Julia pub/sub story:
Goby.publish()andGoby.subscribe()now work from any task, on every layer, rather than only fromMain.Two commits, publish then subscribe.
Publish
check_and_publishthrew anAssertionErrornaming the task for any layer other thanINTERTHREADunless it was called onMain. It now forwards those publications to the task that owns the C++ application over the interthread channel that task already has — which is what a C++ thread'sInterProcessForwarderdoes with its inner interthread transporter.publish_forward_interprocessis renamedpublish_forward_cxx: the layer is passed through untouched, soINTERMODULEworks the same way.Subscribe
Subscriptions on the layers the C++ side owns were registered from whichever task called
subscribe(), and every message came back toMainregardless of who had asked for it (theTODOinreceive_forward_interprocess).check_and_cxx_subscribe()hands the registration to the application-owning task as a:cxx_subscribechannel message, soGoby.cxx_subscribeis always called on the task that owns the application.cxx_subscriber_tasksrecords which task asked for each(layer, scheme, type, group), andreceive_forward_cxxdelivers there. Anything unregistered still goes toMain, as before.This also fixes a latent bug in
interprocess_callbacks: the leaf was a singleFunctionper key and was written without a lock, so two tasks subscribing to the same group kept only the last-registered callback. It is now keyed by task as well, behind the same lock as the delivery map, andreceive_dereferencedruns only the callbacks belonging to the current task.Known cost
An interprocess publication from a task waits up to one
cxx_channel_check_frequencytick (default 10 Hz) before it goes out. That is documented in the README so an application with latency-sensitive traffic knows to raise it.Tests
src/test/julia/test_app.jlgains testsets covering both registries directly — two tasks subscribing to one group, a task replacing its own callback, and the default-to-Mainrouting. Moving an actual message needs a portal, so that part is covered by the example rather than here. 29 assertions, passing.Verified end-to-end against a live
gobydwith the companion goby3-examples change (GobySoft/goby3-examples#48): reports flow publisher-task -> interthread -> subscriber-task -> interprocess, reaching both a separate subscriber process and the publishing application's own task subscription, exactly once each.Docs
New "Multi-Threaded Julia Usage" section in
share/goby/Goby.jl/README.md, and a note indoc250_languages.mdthat Julia — unlike Python — supports multi-threaded applications.Generated by Claude Code