Repository navigation
Keep tool calls on the E2EE stream and in message history - #1
Merged
Merged
Conversation
Venice function calls were dropped because the stream parser skipped events without delta.content and encrypt rebuilt every message as role plus content. Yield decrypted content, tool-call deltas, and finish_reason, and keep tool metadata in the clear while encrypting non-empty bodies, including tool results. Co-authored-by: guil <guil@axlabs.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Agents such as Hermes never ran tools on E2EE models. The model wrote a short preamble into
delta.content, then emitted the call ondelta.tool_callsand finished withfinish_reason: "tool_calls".decryptSSEStreamskipped any event whosedelta.contentwas missing, so the call disappeared and the caller closed the turn asstop. The next request then showed the model looking at its own cut-off text.encryptalso rebuilt every message as{ role, content }and encrypted every role. An assistant turn that is only a tool call (content: nullor"") became ciphertext, which makes the TEE treat that turn as text. Tool results were not kept as confidential bodies next to their call id.After this change:
content, unchangedtool_callsdeltas, and the realfinish_reason(tool_calls,stop, orlength), in upstream order.toolmessages keeptool_calls,tool_call_id, andname.content: nullstaysnull. An empty string is not encrypted for those roles.contentis encrypted for every role, including tool-result bodies.userandsystemcontent is still always ciphertext, including an empty string, because Venice rejects plaintext in those roles.decryptStreamno longer yields bare strings. This is a breaking change. The package is still0.2.0in this PR; publishing is a localnpm run release -- minor(0.2.0→0.3.0). There is no npm token in this environment and CI does not publish, so that release is a follow-up.preparebuildsdist/so the unpublished commit can be installed from git until then.Tests
npm test— 42 passed, 3 skipped (live integration tests; noVENICE_API_KEY)npm run build—tscsucceeded