fix(run): conform the cloud NDJSON stream to the documented event contract

[mattmillerai]

ELI-5

comfy run --json promises a fixed stream of events: "I converted your workflow", "here's the graph I'm about to send", "the server has it", then a final result line. The local run kept that promise. The cloud run (--where cloud) quietly did something else — it never said "converted", only showed the graph under --print-prompt, and said "queued" before it had actually sent anything, with the wrong fields. So a script that trusted the docs got a different stream depending on where the job ran, and "queued" on cloud didn't mean the server had the job. This makes the cloud stream match what the docs say, and documents the handful of differences that genuinely can't match.

What changed

comfy_cli/command/run/__init__.py (execute_cloud) — cloud now conforms to docs/json-output.md:

• emits converted after a client-side UI→API conversion (it emitted none)
• emits prompt_preview unconditionally, not only under --print-prompt
• emits queued after a successful submit, carrying the documented prompt_id / client_id / validation_warnings / nodes (plus base_url). A failed submit, or one the server rejects with node_errors, now emits no queued at all — the pre-submit event was the defect the ticket singles out
• drops the --wait announcement that reused executing (a per-node event type) with {workflow, base_url} and no node field
• --print-prompt returns instead of raise typer.Exit(0), matching local. Corrected in review (thanks @bigcat88): an earlier revision of this PR claimed cmdline.py's else: branch tracked execution_success on a normal return just as its except typer.Exit branch did for Exit(0). That was wrong — Python skips a try's else: clause when the suite leaves via return, and cmdline.py returned early on the cloud branch, so the except typer.Exit handler was the only thing that ever reached the tracking call. Dropping the Exit(0) therefore dropped the event. Fixed at the call site: the cloud branch is now an if/else so both targets fall off the end of the try suite. That also closes a pre-existing gap — no cloud run emitted execution_success before this, not just --print-prompt. Exit codes were never affected
• the async envelope carries watcher_spawned, like local
• prompt_rejected details.node_errors is now the documented array-of-records, not the server's id-keyed dict. This is a bigger fix than it reads, and it lands on the local path too, because _node_errors_to_list is the shared helper: previously any non-dict error value was silently dropped, so a rejection could report "rejected 1 node(s)" next to an empty node_errors array ({"1": "missing input"} and {"2": ["a", "b"]} both yielded []). Worse, a bare-string error item raised AttributeError: 'str' object has no attribute 'get' out of the hint builder — aborting with a traceback and no envelope at all, breaking the exactly-one-envelope guarantee. The shape checks close both. The map key now also wins over a server-supplied node_id, so the id you see is the one you keyed on

comfy_cli/command/run/execution.py: workflow_manifest() / node_title() lifted to module level; WorkflowExecution delegates to them. Both targets now build queued.nodes from one function, so the manifest shape can't diverge on one side again.

docs/json-output.md: a new Per-target differences section listing every remaining difference (no per-node events from a cloud run, host/port vs base_url, local-only cached_node_ids/executed_node_ids, cloud-only warnings, cloud-only error codes, the prompt_rejected details difference), plus a cloud stream example and fixes to the success-envelope table.

comfy_cli/schemas/run{,_event}.json: these are published as this contract's machine-readable half via comfy --json discover, and they rejected real output on both targets — converted, prompt_preview and login_url were missing from the event enum, and data.status had no "preview" value despite --print-prompt emitting it. Corrected, with per-target annotations.

Judgment calls

• Conform the code and extend the doc, rather than either/or. The ticket allows either, but neither alone is sufficient: the four documented events could be made true on cloud, while "no per-node events on a cloud run" can't be, so it has to be written down.
• Pipelines deliberately left unmerged — explicitly out of scope per the ticket. Every change here is local to the cloud function plus a helper extraction.
• queued.base_url kept as an extra field so cloud consumers don't lose the target they used to get from the old pre-submit event. Additive optional fields are non-breaking under the doc's own stability rules.
• Adjacent fixes I chose to include (all in the same contract, all one-liners): the schema corrections above, and a transient_auth row in the error-code table — it is registered in error_codes.py and emitted by execution_errors.classify() on both paths, but was undocumented.
• Ticket item 5 ("local and cloud report differently on the same terminal states") is addressed by documentation, not behaviour change. The differences are structural — cached_node_ids/executed_node_ids are derived from a per-node event stream the cloud has none of; cloud warnings come from a partial-execution guard with no local equivalent. Unifying them would require merging the pipelines, which the ticket rules out.

Self-review notes

• Riskiest line: the now-unconditional renderer.event("prompt_preview", prompt=parsed_workflow) on cloud, because it puts the graph in every cloud stream. Verified it carries no credentials: Client.submit_prompt builds extra_data (where auth_token_comfy_org / api_key_comfy_org ride) at send time from a local dict and never mutates the workflow, so the previewed graph is credential-free — same guarantee the local path's prompt_preview documents. renderer.event() is a no-op outside stream mode, so pretty and single-envelope JSON output are byte-identical.
• Negative-claim falsification. The diff makes one capability-denying claim: "a cloud run emits no per-node events." I went looking rather than asserting it, and the search corrected me twice. (1) comfy jobs watch --where cloud (jobs.py:_cloud_watch) does emit progress — coarse state transitions plus output events — so the doc now redirects the reader there instead of dead-ending, and state was added to the event enum since discovery.py maps comfy jobs watch to that same schema. (2) My first draft claimed execution_error "cannot occur" on cloud; execute_cloud calls the same execution_errors.classify() the local path does, so it can — that claim is removed and transient_auth documented. Also caught in review: my own new nodes schema constraint would have rejected jobs watch's execution_cached.nodes (an array of strings), now widened.
• Verification: full pytest green (3764 passed, 37 skipped) and ruff check / ruff format --diff clean under the CI-pinned ruff 0.15.15. New tests pin the cloud stream shape (async, --wait, --print-prompt, UI-converted), assert no queued on a failed or rejected submit, assert local/cloud manifest parity, and validate both real streams against the published schemas so this drift can't silently return.

[coderabbitai]

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 51 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ad88c9cb-8926-4520-ad91-c44148eb4b90

📥 Commits

Reviewing files that changed from the base of the PR and between 5eaa964 and bb5aa83.

📒 Files selected for processing (1)

• comfy_cli/schemas/run.json

📝 Walkthrough

Walkthrough

The run command now aligns local and cloud execution paths. It adds shared workflow manifests, consistent preview and queue events, normalized node errors, expanded JSON schemas, updated documentation, and lifecycle and NDJSON contract tests.

Changes

Run execution and output contracts

Layer / File(s)	Summary
Shared node manifest helpers comfy_cli/command/run/execution.py, comfy_cli/command/run/__init__.py	Shared helpers resolve node titles and build workflow manifests. The run module re-exports workflow_manifest.
Cloud preview, submission, and results comfy_cli/command/run/__init__.py, comfy_cli/command/run/loader.py, tests/comfy_cli/command/test_run_json.py, tests/comfy_cli/test_run_execution_lifecycle.py	Cloud runs emit conversion and prompt-preview events, emit queued after successful submission, preserve malformed node errors, and report watcher startup. Tests cover event ordering, rejection paths, wait mode, and lifecycle results.
Local and cloud target lifecycle comfy_cli/cmdline.py, tests/comfy_cli/test_run_execution_lifecycle.py	Local runs resolve host and port settings before execution. Cloud runs avoid local endpoint resolution and reach shared success or error tracking.
Run output schemas and documentation comfy_cli/schemas/run.json, comfy_cli/schemas/run_event.json, docs/json-output.md, tests/comfy_cli/command/test_run_cli.py, tests/comfy_cli/command/test_run_json.py	Schemas and documentation define preview status, target-specific fields, event payloads, node shapes, cloud output semantics, and normalised error records. Tests validate the published contracts.

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant CloudExecution
  participant CloudAPI
  participant Watcher
  CLI->>CloudExecution: convert workflow
  CloudExecution-->>CLI: converted and prompt_preview
  CloudExecution->>CloudAPI: submit prompt
  CloudAPI-->>CloudExecution: prompt metadata and warnings
  CloudExecution-->>CLI: queued with workflow manifest
  CloudExecution->>Watcher: start detached watcher
  Watcher-->>CLI: watcher_spawned status

Loading

🚥 Pre-merge checks | ✅ 2

✅ Passed checks (2 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch matt/be-6039-run-json-cloud-event-contract

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch matt/be-6039-run-json-cloud-event-contract

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

🔍 Cursor Review — Consolidated panel

Triggered by @mattmillerai.

Found 7 finding(s).

Severity	Count
🟡 Medium	3
🟢 Low	3
⚪ Nit	1

Panel: 8/8 reviewers contributed findings.

[mattmillerai]

🤖 The reviews loop filed Linear follow-up ticket(s) for review thread(s) deferred as out of scope for this PR:

• BE-6071 — Emit run's queued event after the job state file is persisted, on both targets — filed as agent-spike (premise unverified)

The following carry agent-spike instead of agent-ok because their reachability claim was not backed by evidence (BE-5378) — the claim is investigated before any code is written, and "the premise does not hold" is a valid, successful outcome:

• Emit run's queued event after the job state file is persisted, on both targets — no reachability block in the proposal

[bigcat88]

Requesting changes on one line. Everything else here holds up — I ran the whole thing against the real Comfy Cloud staging environment rather than fixtures, and the contract really was as broken as you describe.

Blocker: --print-prompt --where cloud stops emitting execution_success

The description says:

--print-prompt returns instead of raise typer.Exit(0) … cmdline.py's else: branch tracks execution_success on a normal return exactly as its except typer.Exit branch does for Exit(0), so telemetry and exit code are unchanged

The exit code is unchanged. The telemetry isn't. cmdline.py:1004 is a bare return immediately after the execute_cloud(...) call — inside the same try whose else: (line 1043) does the tracking — and Python skips an else: clause when the try suite leaves via return. The only thing reaching that tracking call on the cloud branch was the except typer.Exit handler, which is exactly what this hunk removes.

Measured on the wire, with a MITM'd telemetry endpoint capturing the real Mixpanel and PostHog uploads, running comfy run --where cloud --print-prompt against staging:

	Mixpanel	PostHog
main	run, execution_success	cli:execution_start, cli:execution_success
this branch	run	cli:execution_start

Two adjacent facts that argue for fixing it properly rather than reverting the hunk:

• The local path really does track it, so the asymmetry this hunk set out to remove is the one it creates. execute()'s print-prompt branch also returns (run/__init__.py:272) — but run_inner.execute(...) is the last statement of the try suite with no return after it, so control falls through to else:.
• No cloud run emits execution_success today either. Same return at 1004. I measured a successful --no-wait cloud submit on main: run + cli:execution_start, no success event. So cloud success telemetry has been silently missing all along, and this hunk removes the one case that still worked by accident.

Fix that closes both:

if decision.target is where_module.WhereTarget.CLOUD:
    where_module.cloud_preflight_or_exit()
    run_inner.execute_cloud(...)
else:
    from comfy_cli.host_port import parse_host_port_arg, resolve_host_port
    ...
    run_inner.execute(...)
# try suite ends normally on both targets -> else: fires

If you'd rather keep this PR tightly scoped, reverting line 820 to raise typer.Exit(code=0) restores the status quo — but it leaves the pre-existing cloud gap in place, and I'd rather see that one go.

Everything else — verified live against staging

Every stream shape below was run against https://stagingcloud.comfy.org on both main and this branch.

The contract really was broken

Real async cloud submit (--no-wait), same workflow (SD1.5 txt2img), same server:

• main: queued {workflow, base_url} — emitted before the POST, with no prompt_id, no client_id, no nodes, no validation_warnings. No prompt_preview anywhere. Envelope missing watcher_spawned.
• this branch: prompt_preview → queued {prompt_id, client_id, validation_warnings: [], nodes: [7 records, titles included], base_url} → envelope(ok, status="queued", watcher_spawned=true).

And a live --wait run (a real job, executed and returned): prompt_preview → queued → envelope(ok, status="completed") carrying outputs (1), outputs_by_node, outputs_by_item, warnings: [], base_url, state_file, elapsed_seconds — no per-node events and no cached_node_ids/executed_node_ids, exactly as your per-target table says. Schema-valid on all three lines.

The schemas really did reject real output

Validating the captured staging streams against the repo's own published schemas:

• main's output vs main's schemas: 2 failures — 'prompt_preview' is not one of [...] and 'preview' is not one of ['queued','completed','cancelled']. Exactly the two you name.
• this branch's output vs this branch's schemas: 0 failures, across --print-prompt and async.

converted

Fed tests/comfy_cli/fixtures/sd15_ui_workflow.json to the cloud path: branch emits converted {node_count: 7} before prompt_preview; main emits nothing.

_node_errors_to_list is a bigger fix than the description claims

It's the shared helper, so this changes the local path too. Probed both versions directly:

input	main	this branch
{"1": "missing input"}	[] — silently dropped	[{"errors": ["missing input"], "node_id": "1"}]
{"2": ["a", "b"]}	[] — dropped	[{"errors": ["a", "b"], "node_id": "2"}]
record carrying its own node_id: "99" under key "4"	node_id: "99" — server wins	node_id: "4" — map key wins

So on main a rejection could report "rejected 1 node(s)" alongside an empty node_errors array. And while running the non-vacuity pass I hit the other half: on main a bare-string error item raises AttributeError: 'str' object has no attribute 'get' out of the hint builder at __init__.py:878 — aborting with a traceback and no envelope at all, breaking the exactly-one-envelope guarantee. Your shape-checking closes that. Both are worth saying out loud in the description; right now the comment undersells them.

Credential safety of the unconditional prompt_preview

Confirmed by reading comfy_client.submit_prompt: extra_data — where auth_token_comfy_org / api_key_comfy_org ride — is built inside payload() at send time and attached beside "prompt"; the workflow dict is never mutated. Your riskiest-line note holds.

Tests and suite

With the three run/* sources and both schemas reverted to main, 13 of 18 new tests fail — non-vacuous. Full pytest . on your branch merged with current main: 3781 passed, 31 skipped. ruff check and ruff format --diff clean at the CI-pinned 0.15.15.

Follow-up nit (pre-existing, not yours)

The CQL pre-flight failure envelope comes out with where: null on the cloud path — I hit it with an unknown ckpt_name — while the per-target table this PR adds says envelope where is "cloud". Cause is preflight.py:106, which calls renderer.error(...) without where=; that file isn't in this diff. Out of scope here, but since this PR is the one documenting the field, worth a ticket.

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@comfy_cli/cmdline.py`:
- Around line 1012-1015: Update the port-selection condition in the host parsing
flow around parse_host_port_arg to check whether port is None rather than using
its truthiness. Preserve an explicitly supplied port value of 0, while still
applying parsed_port when no --port value was provided.

In `@comfy_cli/schemas/run.json`:
- Line 10: Add a documented prompt property to the run.json response schema,
defining it as an object to represent the workflow graph returned by
--print-prompt, while preserving the existing status enum and permissive
additional-property behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b9aa1e97-b438-4495-95da-ad9b10a2f38a

📥 Commits

Reviewing files that changed from the base of the PR and between 5704b55 and 5eaa964.

📒 Files selected for processing (10)

• comfy_cli/cmdline.py
• comfy_cli/command/run/__init__.py
• comfy_cli/command/run/execution.py
• comfy_cli/command/run/loader.py
• comfy_cli/schemas/run.json
• comfy_cli/schemas/run_event.json
• docs/json-output.md
• tests/comfy_cli/command/test_run_cli.py
• tests/comfy_cli/command/test_run_json.py
• tests/comfy_cli/test_run_execution_lifecycle.py

[mattmillerai]

🤖 The reviews loop filed Linear follow-up ticket(s) for review thread(s) deferred as out of scope for this PR:

• BE-6272 — Populate where on JSON error envelopes (Renderer.error always emits where: null) — filed as agent-spike (premise unverified)

The following carry agent-spike instead of agent-ok because their reachability claim was not backed by evidence (BE-5378) — the claim is investigated before any code is written, and "the premise does not hold" is a valid, successful outcome:

• Populate where on JSON error envelopes (Renderer.error always emits where: null) — no reachability block in the proposal