Skip to content

fix: preserve UTF-8 across stdout chunks - #345

Open
dreamyt wants to merge 1 commit into
agentclientprotocol:mainfrom
dreamyt:codex/fix-utf8-jsonrpc-reader
Open

fix: preserve UTF-8 across stdout chunks#345
dreamyt wants to merge 1 commit into
agentclientprotocol:mainfrom
dreamyt:codex/fix-utf8-jsonrpc-reader

Conversation

@dreamyt

@dreamyt dreamyt commented Jul 30, 2026

Copy link
Copy Markdown

Summary

  • decode Codex app-server stdout with StringDecoder so UTF-8 characters survive arbitrary Node stream chunk boundaries
  • add a regression test that splits a three-byte Chinese character across three chunks

Problem

createJSONRPCReader called Buffer#toString() independently for each stdout chunk. If a multibyte UTF-8 character crossed a chunk boundary, Node emitted replacement characters. The resulting JSON remained parseable but its text was corrupted, which could also defeat downstream content-based history deduplication and surface duplicate messages.

Testing

  • npx vitest run src/__tests__/StdUtils.test.ts
  • npm run typecheck
  • npm test
  • npm run build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant