Skip to content

fix(orchestrator): verify remote agent placement - #392

Merged
miyaontherelay merged 4 commits into
mainfrom
fix/agent-node-discovery
Aug 26, 2026
Merged

fix(orchestrator): verify remote agent placement#392
miyaontherelay merged 4 commits into
mainfrom
fix/agent-node-discovery

Conversation

@miyaontherelay

Copy link
Copy Markdown
Contributor

Summary

  • discover live fleet nodes with the required spawn capability and pin every remote spawn to a verified node
  • refuse remote dispatch before durable lifecycle creation when no eligible node exists
  • require bounded broker registration after spawn, release an unregistered worker, and atomically roll back its durable claim/slot
  • reject hostless or offline-host roster adoption and fence rollback against lifecycle lease takeover

This is the upstream complement to #391: this PR stops creating hostless/unregistered agents; #391 bounds and hands off failures already in downstream release.

Red → green

Before implementation, the production-shaped remote suite failed all three cases: Received node "self" instead of expected "sf-mini"; no-eligible dispatch resolved instead of rejecting; and unregistered dispatch resolved instead of rejecting.

After implementation:

Test Files  1 passed (1)
Tests  3 passed | 652 skipped (655)
Duration  7.72s

Additional validation:

  • npm run build
  • npm run featuremap:check
  • src/fleet/relay-fleet-client.test.ts + src/state/file-state-store.test.ts: 122/122 pass
  • focused cross-file selection: 5/5 pass

The bounded full suite exceeded 15 minutes under host load. Its unrelated timing failures were baselined in a detached unmodified origin/main worktree: main reproduced the load-harness timing failure and five 5-second worktree timeouts. No assertion was weakened or skipped.

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ed2cae77-b2e6-48d0-95e9-572ad7ce44f5


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

@coderabbitai review

Requested for exact head 220988027754845b01963147b7bde608dcf49001.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 issues found across 9 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/orchestrator/factory.ts">

<violation number="1" location="src/orchestrator/factory.ts:10694">
P1: `#rollbackUnregisteredRemoteDispatch` clears the durable dispatch-lifecycle claim and abandons the batch record, but it never persists a reaper handoff for agents that were already spawned earlier in this same dispatch (unlike the generic failure path a few lines below, which calls `#persistDispatchFailureReaperHandoff` first). If an implementer spawn already succeeded before the reviewer spawn throws `FleetPlacementUnavailableError`/`RemoteAgentRegistrationTimeoutError`, that already-live remote agent is left running with no lifecycle row and no handoff record, so nothing will ever release it. Persist a failure-reaper handoff for `spawnedForReaperHandoff`/`record.agents` (or explicitly release each of them) before clearing the claim and abandoning the batch entry.</violation>
</file>

<file name="src/fleet/relay-fleet-client.test.ts">

<violation number="1" location="src/fleet/relay-fleet-client.test.ts:837">
P2: Case 2 returns false from the node-mismatch `find`, not from the live/offline-host gate, so it never exercises the new `node.live && capabilities` rejection that this PR is meant to guard. To actually test offline-host refusal, report an agent on node 'beta' and query it there, e.g. add a third agentRow `{ name: 'ar-offline-host-impl', status: 'online', node: 'beta' }` and assert `isAgentRegistered({ name: 'ar-offline-host-impl', node: 'beta', capability: 'spawn:codex' })` rejects because beta.live is false.</violation>
</file>

<file name="src/orchestrator/factory.test.ts">

<violation number="1" location="src/orchestrator/factory.test.ts:1108">
P2: This test cannot detect the regression it targets. When the orchestrator fails to pin a node and spawns with node 'self' (the pre-fix 'Received node self' behavior), the fake's spawn override falls back to `result.node`, which the base RemoteLifecycleFleetClient.spawn hardcodes to 'sf-mini', so `fleet.spawns` still reports 'sf-mini' and the assertion passes. Return the requested node directly (preserving 'self') so the node-pinning assertion actually fails when pinning regresses.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread src/orchestrator/factory.ts
Comment thread src/fleet/relay-fleet-client.test.ts
Comment thread src/orchestrator/factory.test.ts Outdated
@github-actions

Copy link
Copy Markdown
Contributor

@coderabbitai review

Requested for exact head a09a1d097b3d6debfabefe21662749085d5c80de.

Session-Id: 01a03e13-5b3b-7473-a480-53403c8df263

Session-Id: 01a03e13-5b3b-7473-a480-53403c8df263
Session-Id: 01a03e13-5b3b-7473-a480-53403c8df263

Session-Id: 01a03e13-5b3b-7473-a480-53403c8df263
@miyaontherelay
miyaontherelay force-pushed the fix/agent-node-discovery branch from a09a1d0 to a4b2984 Compare August 26, 2026 13:42
@github-actions

Copy link
Copy Markdown
Contributor

@coderabbitai review

Requested for exact head a4b2984922de63683e2b7e5c153a30db3ecd0a2d.

Session-Id: 01a03e13-5b3b-7473-a480-53403c8df263
@github-actions

Copy link
Copy Markdown
Contributor

@coderabbitai review

Requested for exact head a912e03e652b0383f60c52199fb7d56277217f5d.

Session-Id: 01a03e13-5b3b-7473-a480-53403c8df263
@github-actions

Copy link
Copy Markdown
Contributor

@coderabbitai review

Requested for exact head 9bce29ab9c87ac9270507c6d373c404dd872232c.

@miyaontherelay
miyaontherelay merged commit 85c704c into main Aug 26, 2026
12 of 13 checks passed
@miyaontherelay
miyaontherelay deleted the fix/agent-node-discovery branch August 26, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant