Skip to content

feat(node): activate dynamic plugins#366

Open
bbednarski9 wants to merge 8 commits into
NVIDIA:mainfrom
bbednarski9:bbednarski/node-dynamic-plugin-host
Open

feat(node): activate dynamic plugins#366
bbednarski9 wants to merge 8 commits into
NVIDIA:mainfrom
bbednarski9:bbednarski/node-dynamic-plugin-host

Conversation

@bbednarski9

@bbednarski9 bbednarski9 commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Overview

This adds an owned Node binding for explicit native and worker dynamic-plugin activation. It is stacked on #364; once that PR lands, this diff reduces to the Node binding and its tests.

Static-only harness-native applications keep using plugin.initialize() / plugin.clear(). activateDynamicPlugins(...) requires at least one dynamic specification; its base config may also contain static components, which initialize before components appended by the dynamic plugins.

  • I confirm this contribution is my own work, or I have the right to submit it under this project's license.
  • I searched existing issues and open pull requests, and this does not duplicate existing work.

Details

  • Add activateDynamicPlugins(config, specs) with camelCase native and worker specifications.
  • Return an owned activation exposing report, active, and async close().
  • Share one detached close operation across repeated and concurrent callers; each caller receives the same cached result.
  • Run cleanup off the JavaScript thread and retain defensive finalization when callers omit explicit close.
  • Preserve existing static plugin.initialize() and plugin.clear() behavior. Empty dynamic activation is rejected without claiming process ownership, so static initialization remains available.
  • Keep complete core validation, manifest, load, ownership, and teardown diagnostics in rejected promises.
  • Add neutral native and worker fixture coverage for mixed static/dynamic configuration, managed LLM/tool callbacks, callback absence after close, partial-load rollback, conflicts, repeated/concurrent close, cancellation-safe cleanup, and garbage collection.

Where should the reviewer start?

Start with crates/node/src/api/mod.rs for N-API ownership and cleanup, then crates/node/plugin.d.ts for the public contract. The lifecycle cases are in crates/node/tests/dynamic_plugin_tests.mjs.

Validation

  • npm run build-debug
  • Dynamic-plugin Node tests — 8 passed
  • cargo fmt --all -- --check
  • cargo clippy -p nemo-relay-node --all-targets -- -D warnings
  • Changed-file Prettier checks passed
  • git diff --check
  • The earlier full Node suite passed (252 tests)

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

Relates to #364.

Summary by CodeRabbit

  • New Features

    • Added explicit dynamic plugin activation for Node.js, including validation reports and deterministic close() teardown.
    • Added support for managing native and worker plugin lifecycles with safe ownership and cleanup.
    • Improved plugin registration handling to prevent conflicts between built-in and dynamically loaded plugins.
  • Bug Fixes

    • Strengthened rollback and teardown behavior when plugin initialization, callbacks, or worker shutdowns fail.
    • Ensured concurrent and cancelled operations complete safely without leaving stale plugin state.

@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

An error occurred during the review process. Please try again later.

Walkthrough

Changes

The plugin system now supports owned dynamic activation with serialized mutations, registration-aware rollback, guarded native and worker teardown, and explicit lifecycle handles. CLI startup uses static or dynamic activation modes, while Node.js exposes dynamic activation and asynchronous close APIs with integration coverage.

Plugin activation lifecycle

Layer / File(s) Summary
Registry ownership and transactional mutation
crates/core/src/plugin.rs, crates/core/src/plugins/*, crates/core/tests/unit/plugin_tests.rs
Plugin registrations track ownership and IDs; mutations run through a serialized executor with checked rollback, panic handling, and incomplete-teardown state. Builtin registration paths and related tests use the new ownership rules.
Dynamic host and runtime teardown
crates/core/src/plugin/dynamic/*, crates/core/tests/integration/*, crates/core/tests/fixtures/*, crates/core/Cargo.toml
PluginHostActivation loads explicit native or worker specs, owns host state, and performs checked teardown. Native and worker runtimes record registration IDs; worker shutdown is guarded and reports process, RPC, and cleanup failures.
CLI server host integration
crates/cli/src/server.rs, crates/cli/tests/coverage/server_tests.rs
Server startup selects static or dynamic activation through initialize_plugin_host, and teardown clears the corresponding activation mode.
Node dynamic activation API
crates/node/plugin.d.ts, crates/node/plugin.js, crates/node/src/api/mod.rs, crates/node/tests/dynamic_plugin_tests.mjs
Node bindings expose dynamic activation specs, activation reports, active state, and asynchronous idempotent close behavior, including finalizer and concurrent-close coverage.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant NodeClient
  participant NodeBinding
  participant PluginHostActivation
  participant PluginRegistry
  participant NativeOrWorkerRuntime

  NodeClient->>NodeBinding: activateDynamicPlugins(config, specs)
  NodeBinding->>PluginHostActivation: activate(config, specs)
  PluginHostActivation->>PluginRegistry: acquire host lease and initialize callbacks
  PluginHostActivation->>NativeOrWorkerRuntime: load and register dynamic plugins
  NativeOrWorkerRuntime-->>PluginHostActivation: activation handles and report
  PluginHostActivation-->>NodeBinding: owned activation
  NodeBinding-->>NodeClient: activation report and close()
  NodeClient->>NodeBinding: close()
  NodeBinding->>PluginHostActivation: clear()
  PluginHostActivation->>PluginRegistry: clear callbacks and configuration
  PluginHostActivation->>NativeOrWorkerRuntime: deregister and shut down
  NativeOrWorkerRuntime-->>PluginHostActivation: teardown outcome
  PluginHostActivation-->>NodeBinding: completion
  NodeBinding-->>NodeClient: close resolved
Loading
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added size:XL PR is extra large Feature a new feature lang:js PR changes/introduces Javascript/Typescript code lang:rust PR changes/introduces Rust code labels Jul 6, 2026
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

@AteebNoOne AteebNoOne left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together! I took a look at the JavaScript bindings and left a quick suggestion on the JSDoc typing to help with intellisense. Looking forward to seeing this land!

Comment thread crates/node/plugin.js Outdated
Comment thread crates/node/plugin.js
* @returns {object|null|undefined} The last activation report, if one exists.
* @remarks This returns an empty value until `initialize` succeeds at least
* once in the current process.
*/

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ince deterministic teardown is required for these dynamic plugins, should we implement Symbol.asyncDispose on the returned activation object under the hood? Doing so would allow Node developers to use the modern await using syntax for guaranteed cleanup, even if the runtime throws an error before close() is explicitly called.

@willkill07 willkill07 added this to the 0.6 milestone Jul 6, 2026
@bbednarski9 bbednarski9 force-pushed the bbednarski/node-dynamic-plugin-host branch from a4ecdcc to baa5cd8 Compare July 7, 2026 00:32
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown

License Diff

Compared against origin/main.

Lockfile license changes

Lockfile License Changes

Rust

Added

  • None

Removed

  • None

Updated/Changed

  • None

Node

Added

  • None

Removed

  • None

Updated/Changed

  • None

Python

Added

  • None

Removed

  • None

Updated/Changed

  • None
Status output
[license-diff] selected languages: rust, node, python
[license-diff] generating current inventory
[license-diff] current: generating Rust inventory
[license-diff] current: Rust inventory complete (378 packages)
[license-diff] current: generating Node inventory
[license-diff] current: Node inventory complete (363 packages)
[license-diff] current: generating Python inventory
[license-diff] current: Python inventory complete (105 packages)
[license-diff] current inventory complete
[license-diff] checking out base ref origin/main into a temporary worktree
[license-diff] base: generating Rust inventory
[license-diff] base: Rust inventory complete (378 packages)
[license-diff] base: generating Node inventory
[license-diff] base: Node inventory complete (363 packages)
[license-diff] base: generating Python inventory
[license-diff] base: Python inventory complete (105 packages)
[license-diff] base inventory complete
[license-diff] removing temporary base worktree
[license-diff] comparing inventories
[license-diff] rendering Markdown output
[license-diff] done

@bbednarski9 bbednarski9 force-pushed the bbednarski/node-dynamic-plugin-host branch from baa5cd8 to bce713b Compare July 7, 2026 14:17
@bbednarski9 bbednarski9 force-pushed the bbednarski/node-dynamic-plugin-host branch from bce713b to 0931dc0 Compare July 7, 2026 19:55
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
@bbednarski9 bbednarski9 force-pushed the bbednarski/node-dynamic-plugin-host branch from 0931dc0 to 7f9ac76 Compare July 13, 2026 16:36
@bbednarski9 bbednarski9 marked this pull request as ready for review July 13, 2026 16:49
@bbednarski9 bbednarski9 requested a review from a team as a code owner July 13, 2026 16:49

@mnajafian-nv mnajafian-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is in good shape overall. I left one Node-surface comment on the public activation-handle contract.

Comment thread crates/node/plugin.d.ts
export interface DynamicPluginActivation {
/** Validation report produced by the successful activation. */
readonly report: ConfigReport;
/** Whether this object still owns the dynamic plugin host. */

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’d tighten this public wording to match the behavior the tests already lock in. active becomes false as soon as close() starts, before the returned promise resolves and before teardown is necessarily complete. Framing this as 'teardown has not started for this handle' or calling out that false does not by itself mean a replacement activation can succeed yet would make the Node contract easier to use correctly.

@zhongxuanwang-nv zhongxuanwang-nv left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think overall this is good. Thanks Bryan!

Comment thread crates/node/plugin.js
Comment on lines +91 to +94
function activateDynamicPlugins(config, specs) {
return lib.activateDynamicPlugins(config, specs);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the same thing that happened in PR-365, that here they might skip the base plugins.toml file

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Feature a new feature lang:js PR changes/introduces Javascript/Typescript code lang:rust PR changes/introduces Rust code size:XL PR is extra large

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants