Report browser-control CDP commands in the control telemetry category - #323
Draft
archandatta wants to merge 1 commit into
Conversation
api_call covers browser control issued through the kernel-images API, but a client driving the browser over the CDP proxy — Playwright, Puppeteer, an SDK — produced nothing at all, so a session whose agent works over CDP showed connect, disconnect and nothing in between. The proxy now emits cdp_command under control for the methods that drive the browser: input gestures, navigation, dialog handling, file selection and screenshots. Configuration and the DOM/Runtime traffic a client library issues on the caller's behalf stay out, and the phases that duplicate a gesture (mouseMoved, keyUp, char) are dropped so one action reads as one event. Payloads are shape only: method, session, event type, coordinates, button and the length of submitted text. Never the text, the key or the URL — control is captured by default, and on a login page those are the credentials. Frames are rejected by a single scan for the method name before anything is unmarshalled, so a large Runtime.callFunctionOn costs a scan rather than a parse. The parsed top-level method decides classification, so a nested "method" key cannot spoof one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
archandatta
force-pushed
the
archand/kernel-1870/cdp-control-telemetry
branch
from
August 4, 2026 13:20
b630069 to
0296e93
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #322 (base branch is that PR's branch, so the diff shown here is only this change). #322 makes
controlmean browser control for calls that arrive through the kernel-images API; this covers the other way a caller drives the browser. A client on the CDP proxy — Playwright, Puppeteer, our own SDKs — produced no control telemetry at all, so a session whose agent works over CDP showedcdp_connect,cdp_disconnect, and nothing in between.The proxy now emits
cdp_commandundercontrol:Input.dispatch{Mouse,Key,Touch,Drag}Event,Input.insertText,Input.synthesize{Scroll,Tap}Gesture,Page.navigate,Page.navigateToHistoryEntry,Page.reload,Page.captureScreenshot,Page.handleJavaScriptDialog,DOM.setFileInputFiles.Emulation.*,*.enable) and the DOM/Runtime traffic a client library issues on the caller's behalf.Runtime.evaluate/Runtime.callFunctionOnare deliberately out — every locator resolution is one, so including them would bury the gestures and put arbitrary submitted source into a default-on category.mouseMoved,keyUp,char. A Playwright click is three CDP frames and a keystroke is three; this makes one action read as one event. Humanized cursor paths also emitmouseMovedin bulk.Data captured
method,session_id,event_type,x,y,button,text_length,key. Shape only — never the submitted text or the navigation URL.keyis reported only when it is more than one rune, which makes it a named key (Enter,Tab,ArrowDown) rather than a character someone typed; that reads back usefully and cannot be a password character.controlis in the default capture set, so anything content-bearing here would land in a default-on category on a login page; URLs stay inpageand typed text stays ininteraction, both opt-in. A test asserts the payload for text, key, URL and file-path frames contains none of those values.Hot path
Every client frame hits the transform, so classification starts with one scan for the frame's method name and a map lookup; only an allowlisted method is unmarshalled. A large
Runtime.callFunctionOncosts the scan, not a parse. The parsed top-levelmethodis what decides the category, so a nested"method"key insideparamscannot spoof one.Follow-ups (not in this PR)
cdp_command. Pairing a command with its result byidneeds per-connection in-flight state with eviction; worth doing if latency-per-gesture turns out to matter.Testing
go vet ./...clean andgo test -racegreen across the non-e2e packages. New coverage: classification table (control vs configuration vs library bookkeeping vs command results vs upstream events vs nested-method spoof vs malformed), a payload-leakage test, and a proxy-level test asserting two client gestures produce two events while the echoed upstream frames produce none. e2e not run — needs Docker.CI
server-testonly triggers for PRs based onmain, so it does not run automatically while this is stacked on #322. Dispatched manually against this branch and it passed: https://github.com/kernel/kernel-images/actions/runs/30913709695. It will run on thepull_requesttrigger once #322 merges and this PR's base flips tomain.