[ZEPPELIN-6659] Render streaming interpreter output in Angular New UI - #5460
[ZEPPELIN-6659] Render streaming interpreter output in Angular New UI#5460miinhho wants to merge 5 commits into
Conversation
voidmatcha
left a comment
There was a problem hiding this comment.
The overall approach makes sense, but I think two items still need to be addressed before this is ready to merge.
In personalized mode, UPDATE is no longer broadcast, but it still mutates the shared paragraph output buffer. A later checkpoint can therefore persist user-specific output as the shared result. Please skip ownerless incremental updates before mutating the shared buffer, and add a regression test proving that the output is not visible from another user's paragraph after checkpointing. Full owner-aware streaming support can remain tracked in ZEPPELIN-6704.
For reference, the related output-ordering and client-state handling changes are available in this branch:
https://github.com/voidmatcha/zeppelin/tree/fix/pr5460-output-ordering
Separately, PR #5456 for ZEPPELIN-6660 has now been merged. Please rebase this PR onto the latest master and add the streaming-output scenario to the notebook parity registry, as listed in this PR's remaining TODO.
What is this PR for?
The Angular New UI declares
PARAGRAPH_APPEND_OUTPUTandPARAGRAPH_UPDATE_OUTPUTin its WebSocket operation enum but does not consume either message. Interpreter output emitted while a paragraph is runningis therefore discarded until the terminal
PARAGRAPHsnapshot arrives.This PR adds the missing consumer based on the server's actual wire contract. It declares the receive payloads in the SDK, handles both operations in
ParagraphBase, and introducesParagraphOutputStateto fold UPDATE, APPEND, and terminal snapshots into the current paragraph result. APPEND data is accumulated by result index, APPEND received before a typed UPDATE is held until its type is known, and the terminalPARAGRAPHsnapshot becomes authoritative so late frames cannot duplicate or overwrite the final result.The PR also fixes a server-side ordering ambiguity. APPEND events were buffered for up to 100 ms by
AppendOutputRunner, while UPDATE events bypassed that queue and could overtake an earlier APPEND. A clientcannot distinguish that delayed APPEND from one genuinely produced after the UPDATE. APPEND and UPDATE now share the same queue, with UPDATE acting as an ordering boundary: preceding APPEND chunks are flushed
before the UPDATE, and subsequent APPEND chunks remain after it.
A focused, versioned capture records the callback-order evidence and observed WebSocket sequences with paragraph streaming enabled and disabled. Reducer tests replay the capture and cover delayed-frame permutations. The client remains defensive against late frames for compatibility with older servers.
The change is limited to the standard paragraph result-rendering path. Application output rendering is unchanged.
What type of PR is it?
Bug Fix
Todos
What is the Jira issue?
The reusable WebSocket fixture infrastructure tracked by ZEPPELIN-6665 and ZEPPELIN-6671 is not available yet, so this PR includes the focused capture and ordering replay required by ZEPPELIN-6659.
The notebook parity registry is being introduced by ZEPPELIN-6660 in #5456. After that PR lands, this change will register streaming output as a separate result scenario. The scenario will require the first chunk to be visible while the paragraph is RUNNING, later chunks to accumulate rather than replace earlier output, and the terminal result to contain every chunk exactly once.
How should this be tested?
./mvnw test -pl zeppelin-server -Dtest=AppendOutputRunnerTestcd zeppelin-web-angular npm run test:shell -- src/app/core/paragraph-base/paragraph-output-state.spec.ts projects/zeppelin-sdk/src/interfaces/message-data-type-map.interface.spec.tsThe browser test runs a shell paragraph that prints three chunks separated by 3 and 5 seconds. It verifies that
firstis visible while the paragraph is RUNNING,secondis appended while the paragraphremains RUNNING, and the FINISHED result contains
first,second, andthird.The capture replay additionally verifies coalesced APPEND chunks, APPEND before a typed UPDATE, UPDATE overtaking a queued APPEND, APPEND after the terminal
PARAGRAPH, and terminal fallback whenzeppelin.websocket.paragraph_status_progress.enable=false.Questions: