You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ai-sandbox-local-process: exec() decodes stdout/stderr per chunk and corrupts multi-byte UTF-8 #2153
@tanstack/ai-sandbox-local-process@1.0.0, @tanstack/ai-sandbox@1.3.0 (also main at 1888233)
Framework/Library version
Node.js v26.13.0 on Linux x86_64 (server-side sandbox, no UI framework)
Describe the bug and the steps to reproduce it
LocalProcessHandle.exec() (packages/ai-sandbox-local-process/src/handle.ts, lines 887-890) collects command output per data event with stdout += d.toString('utf8') and stderr += d.toString('utf8'). Whenever a multi-byte UTF-8 sequence straddles two pipe reads, both halves decode into U+FFFD replacement characters.
The same file already documents this issue and uses a streaming TextDecoder (decodeStream, lines 139-161) for spawnProcess(), while exec() still decodes each Buffer chunk independently.
Steps to reproduce:
npm i @tanstack/ai-sandbox-local-process@1.0.0 @tanstack/ai-sandbox@1.3.0
node repro.mjs
// repro.mjsimport{localProcessSandbox}from"@tanstack/ai-sandbox-local-process";consthandle=awaitlocalProcessSandbox().create({});consttext="日本語のテキスト\n".repeat(4000);awaithandle.fs.write("notes.txt",text);const{ stdout }=awaithandle.process.exec("cat notes.txt");console.log("same as file:",stdout===text);console.log("U+FFFD count:",[...stdout].filter((c)=>c==="\ufffd").length);awaithandle.destroy();
Actual output:
same as file: false
U+FFFD count: 2
Expected output:
same as file: true
U+FFFD count: 0
Collecting Buffer chunks and decoding once on close with Buffer.concat(...).toString('utf8') (or using a streaming TextDecoder for stdout and stderr) avoids splitting multi-byte sequences.
Your Minimal, Reproducible Example - (Sandbox Highly Recommended)
Do you intend to try to help solve this bug with your own PR?
No, because I do not have time to dig into it
Terms & Code of Conduct
I agree to follow this project's Code of Conduct
I understand that if my bug cannot be reliable reproduced in a debuggable environment, it will probably not be fixed and this issue may even be closed.
I've fixed this in #2162. exec() now calls setEncoding('utf8') on stdout and stderr, so Node's StringDecoder keeps a split multi-byte character until the rest arrives, the same way spawn() already does. A test covering split € on both streams is included.
Activity
mohamadgarmabi commentedon Nov 16, 2026
I've fixed this in #2162.
exec()now callssetEncoding('utf8')on stdout and stderr, so Node's StringDecoder keeps a split multi-byte character until the rest arrives, the same wayspawn()already does. A test covering split€on both streams is included.