Skip to content

fix: stop dropping host-to-device messages containing emoji - #505

Open
V3RON wants to merge 1 commit into
mainfrom
fix/host-to-device-message-encoding
Open

V3RON wants to merge 1 commit into
mainfrom
fix/host-to-device-message-encoding

Conversation

@V3RON

@V3RON V3RON commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Description

A message the app was sent containing an emoji — or any other character outside the Basic Multilingual Plane — never reached the app. Editing a stored value that contains one in the Storage, MMKV, or SQLite panels reported success and did nothing.

All three hosts that talk to a device (RozeniteBindingsModel.sendMessage in the embedded DevTools shell, the agent session's sendDomainMessage, and the standalone app's device connection) build the same Runtime.evaluate expression: serialize the payload, JSON.stringify it again to turn it into a JS string literal, and interpolate that into source text for the device's dispatcher to parse. JSON.stringify escapes quotes, backslashes and control characters — and leaves every non-ASCII code unit raw. Hermes compiles that source text from UTF-8 and refuses a raw astral-plane code unit with Invalid UTF-8 code point, so the evaluation never ran. No call site looked at the response, so a message the device never received was indistinguishable from one it did.

The fix escapes every non-ASCII code unit of the finished expression as a \uXXXX sequence, which makes the injected source text pure ASCII — the form Hermes always compiles. The device's own parser turns the escape back into the original code unit, so the payload it reconstructs is byte-identical to the one that was serialized.

A message the device refuses is now reported instead of dropped: the two hosts whose sends are fire-and-forget log it, and the agent session rejects, so a failed agent-session-ready handshake is retried by bootstrap rather than recorded as a ready session.

Related Issue

Closes #407

Context

Why escaping rather than Runtime.callFunctionOn. #408 switched the send path to callFunctionOn with a by-value argument and was closed because the transport change measured as no win — the second JSON.stringify costs ~70µs on a 52KB payload, on the host, never on the device's JS thread. The defect is in the payload text, not the transport, so the fix belongs there. The double JSON.stringify also stays: it is the wire contract, since the device's dispatcher is handed a JSON string and parses it. #408's two preserved review findings are encoded here instead of being re-litigated:

  • The frontend's generated invoke_* methods never reject, so the runtime host inspects response.exceptionDetails rather than attaching a .catch(), which would be dead code. Catching a protocol-level failure there would mean adding getError() to the hand-written rn-devtools-frontend-api.d.ts; left out deliberately, and the other two hosts already reject on a protocol error in their sendCommand.
  • A sanitizer running over JSON.stringify output sees \uXXXX escape text, not raw code units. That is why this escaping happens on the finished source text, after the second JSON.stringify — and why it cannot happen before it: escaped earlier, the next stringify escapes the backslash too and the device is handed \uD83C as six characters of text.

Why one copy per host rather than a shared helper. @rozenite/app does not depend on @rozenite/runtime, and the only package all three share, @rozenite/tools, publishes its browser-safe surface as the ./integration subpath precisely because its index pulls in node:fs and node:path. Sharing four lines would therefore mean a third subpath export plus a second rollupTypes entry, for a helper small enough to read at the call site. These hosts already each own a private copy of this exact protocol shape — RUNTIME_GLOBAL, the double stringify, the dispatcher-wait poll, the binding handshake, with device-connection.ts describing itself as a port of session.ts — so the escaping joins them: one identical copy each, a comment naming the other two, and a guard in each host's own suite pinning the expression it emits. Drift between them fails a test. Promoting the three copies into a shared subpath later is a mechanical move, and cheaper done once there is a fourth host to justify it.

Scope, and what is deliberately not here.

  • Worst case an expression grows by six bytes per non-ASCII code unit (twelve per astral character). Host→device traffic is user-initiated and sparse, and there is no perf claim attached to this change.
  • A payload containing a lone surrogate still does not get through. It becomes \uD83D escape text, which survives escaping untouched and is what the device's JSON parser rejects in the first place. Fixing that means a toWellFormed() replacer on the payload — a separate decision with a host-support requirement, and a rarer failure than the one reported here.
  • Lynx needs no change: the rspeedy bridge forwards host→device frames untouched, so it inherits the fix from the hosts.

Testing

Automated, from the repository root after git fetch origin main:

  • pnpm checks:affected — 98 tasks green (typecheck + lint across affected packages, plus oxfmt --check . over the repo).
  • pnpm test:affected — 61 tasks green. It needed TURBO_CONCURRENCY=2: at default concurrency three plugin release-bundle.test.ts benches, which drive a real Metro bundle, exceeded their 120s timeout on this machine. Each passes on its own in ~45–60s, none of them import anything this change touches, and they are green under limited concurrency.
  • pnpm release:plan — version plan present for @rozenite/app, @rozenite/middleware, @rozenite/runtime.
  • Two regression guards per host, six in total: one per host for the encoding, and one for the refusal path. Each encoding guard was confirmed to fail with the escaping removed, and each refusal guard to fail with the response check removed.

The guards assert two things about the expression a host emits, and the first is the important one: it matches /^[\x20-\x7E]+$/. Node and V8 accept the raw astral form that Hermes rejects, so replaying the expression alone would not have reproduced this bug — a pure-ASCII expression is the property that guarantees Hermes' UTF-8 decode has nothing to fail on. The second half replays the device side anyway: evaluate the expression against a fake dispatcher, JSON.parse the payload it is handed, and compare it to what went in, which is what keeps the escaping from quietly mangling data. The middleware suite additionally asserts that a refused agent-session-ready is retried and that start() does not resolve over it.

What was not run here: the Playground scenario below, and any device run at all. The Hermes behavior is reproduced from the issue's own raw-CDP script and from #408's device check of this same escape-based fix, not re-measured.

Manual scenario for a reviewer, with the Playground app and Metro running:

  1. Open the Rozenite DevTools URL, Storage panel, edit any value so it contains 🎉, and save.
  2. On main the write silently does not happen — the panel shows the old value again, and nothing is logged anywhere. On this branch the write lands, and the value reads back with the emoji intact.
  3. Repeat for a MMKV value, for a SQLite text cell, and for a value containing CJK text, since those three panels are where a host→device write carries user data.
  4. The refusal path is covered by the unit guards rather than by a manual step: whenever an evaluation is refused, the DevTools console, the standalone app's console, or the agent session now says so instead of treating the message as delivered.

Every host sends a domain message by interpolating it into a
`Runtime.evaluate` JS source string, which Hermes compiles from UTF-8 and
which `JSON.stringify` leaves non-ASCII. A payload with an astral-plane
code unit therefore reached the device as a raw surrogate pair, Hermes
refused to compile it, and the message vanished without a trace: editing a
stored value containing an emoji in the Storage, MMKV, or SQLite panels
looked like it worked.

Escape every non-ASCII code unit of the finished expression as a `\uXXXX`
sequence, which the device's own parser turns back into the original code
unit, so the payload it reconstructs is byte-identical. Applied after the
second `JSON.stringify`, the only position at which the escape text is not
itself re-escaped; the three hosts that speak this protocol each keep their
own copy, like the dispatcher-wait poll they already duplicate.

Also stop swallowing the failure: the two hosts whose sends are
fire-and-forget now report a device-refused evaluation, and the agent
session rejects, so `bootstrap` retries the handshake and an agent tool
call surfaces an error instead of an empty result.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: host-to-device messages containing emoji/astral characters are silently dropped

1 participant