The provenance marker disappears when a model's answer ends with a markdown table. Both packages (R and Python) are affected. Nothing errors, so the answer simply reaches the reader unclassified.
I ran across this while testing the Python UI and noticed the "verified icon" was missing sometimes. I was able to narrow it down to only happening in the case when the answer ends with a table. The agent returns the provenance marker, but it's not getting rendered properly by the UI. Compare this result, where the agent added a text summary line after the table:
With this one, where I told it not to:
shinychat attaches an aside to the markdown block it sits in. Both packages append <shiny-aside> as a trailing chunk with no block of its own (pkg-r/R/commons.R:409, pkg-py/src/commons/_agent.py:287). After a paragraph it lands inline and renders. After a table there is no block to attach to, the aside never reaches the DOM, and a small stray artifact of an empty <tr> row is left at the bottom of the table.
I confirmed this behavior in both R and Python.
Agent-written detail: how this was isolated
Four runs against Bedrock Sonnet, one boundary at a time.
The agent emits the aside correctly. Streaming outside any UI yields a final chunk of <shiny-aside label="Verified answer" icon="lib/commons-chat-<version>/figs/trusted-icon.svg">…, with collect_appended_tags() returning [Tag.A]. Every icon file returns 200 from the served bundle. The markup and the icon URL are correct, so the loss is at the browser.
Answer ending in a paragraph. Pill renders, label Verified answer, trusted-icon.svg in the DOM, last block P.
Answer ending in a table. Last block TABLE, no pill, and the string shiny-aside is not in the DOM at all, so the markdown renderer consumed it rather than leaving it as inert markup. In R the same run shows "Ran a trusted calculation" in the trace, so the tag was A and the markup was produced.
One run showed no pill for a different and correct reason. No data tool ran in that exchange, so derive_provenance_tag() returned None, which is the documented "no pill at all" case. Worth noting so it is not mistaken for this bug.
Both client bundles carry the same aside-parsing regex even though R ships shinychat 0.4.0.9000 and Python ships 0.7.0, which is why one root cause produces the same symptom in both.
shinychat documents an aside as "a small pill that appears at the end of the paragraph or list item it's attached to", created by "an inline <shiny-aside> tag anywhere in a block's markdown". The packages do not give it a block.
A likely fix is to emit the aside as its own block when the answer does not end in an attachable one, rather than always inline. That changes marker placement, which is why it belongs in a shared fixture.
Tracked locally in kata as g6vz.
The provenance marker disappears when a model's answer ends with a markdown table. Both packages (R and Python) are affected. Nothing errors, so the answer simply reaches the reader unclassified.
I ran across this while testing the Python UI and noticed the "verified icon" was missing sometimes. I was able to narrow it down to only happening in the case when the answer ends with a table. The agent returns the provenance marker, but it's not getting rendered properly by the UI. Compare this result, where the agent added a text summary line after the table:
With this one, where I told it not to:
shinychat attaches an aside to the markdown block it sits in. Both packages append
<shiny-aside>as a trailing chunk with no block of its own (pkg-r/R/commons.R:409,pkg-py/src/commons/_agent.py:287). After a paragraph it lands inline and renders. After a table there is no block to attach to, the aside never reaches the DOM, and a small stray artifact of an empty<tr>row is left at the bottom of the table.I confirmed this behavior in both R and Python.
Agent-written detail: how this was isolated
Four runs against Bedrock Sonnet, one boundary at a time.
The agent emits the aside correctly. Streaming outside any UI yields a final chunk of
<shiny-aside label="Verified answer" icon="lib/commons-chat-<version>/figs/trusted-icon.svg">…, withcollect_appended_tags()returning[Tag.A]. Every icon file returns 200 from the served bundle. The markup and the icon URL are correct, so the loss is at the browser.Answer ending in a paragraph. Pill renders, label
Verified answer,trusted-icon.svgin the DOM, last blockP.Answer ending in a table. Last block
TABLE, no pill, and the stringshiny-asideis not in the DOM at all, so the markdown renderer consumed it rather than leaving it as inert markup. In R the same run shows "Ran a trusted calculation" in the trace, so the tag was A and the markup was produced.One run showed no pill for a different and correct reason. No data tool ran in that exchange, so
derive_provenance_tag()returnedNone, which is the documented "no pill at all" case. Worth noting so it is not mistaken for this bug.Both client bundles carry the same aside-parsing regex even though R ships shinychat 0.4.0.9000 and Python ships 0.7.0, which is why one root cause produces the same symptom in both.
shinychat documents an aside as "a small pill that appears at the end of the paragraph or list item it's attached to", created by "an inline
<shiny-aside>tag anywhere in a block's markdown". The packages do not give it a block.A likely fix is to emit the aside as its own block when the answer does not end in an attachable one, rather than always inline. That changes marker placement, which is why it belongs in a shared fixture.