feat(teleop): add latest-result pipelined retargeting#467
Conversation
|
📝 Docs preview is not auto-deployed for fork PRs. A maintainer with write access to |
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThis pull request introduces pipelined asynchronous retargeting execution for TeleopSession. A new AsyncRetargetRunner manages a background worker thread that executes retargeting steps with configurable deadline-based or immediate pacing, allowing the main application thread to submit work and retrieve the latest completed frame. The tensor snapshotting mechanism is enhanced to prefer device-specific copy operations (via Sequence DiagramsequenceDiagram
participant App as Application
participant TS as TeleopSession
participant Runner as AsyncRetargetRunner
participant Worker as Background Worker
App->>TS: step(external_inputs, ...)
TS->>TS: Determine execution mode (latch)
alt First Step (Pipelined)
TS->>TS: Seed: execute synchronously
TS->>Runner: publish_seed(frame)
Runner->>Runner: Store initial frame
TS->>App: Return seeded frame
else Subsequent Steps (Pipelined)
TS->>Runner: submit(StepRequest)
activate Runner
Runner->>Runner: Replace pending work (if pacing allows)
Runner->>Runner: Notify worker
deactivate Runner
par Background Execution
Worker->>Worker: Wait for pacing delay
Worker->>Worker: _execute_step_request()
Worker->>Worker: Snapshot outputs & context
Worker->>Runner: Publish completed frame
end
TS->>Runner: latest()
Runner->>TS: Return newest completed frame
TS->>App: Return frame (may be from previous submission)
else Synchronous
TS->>TS: _execute_step_request()
TS->>App: Return frame immediately
end
Note over TS,Runner: Mode latched on first step<br/>prevents mid-run switching
Note over Runner,Worker: Pacing delay allows<br/>request superseding
Note over App,TS: last_step_info tracks<br/>age, timing, deadlines
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/core/teleop_session_manager/python/teleop_session.py`:
- Around line 374-383: The bug: when snapshot_external_inputs is false
(sync/default path) we still forward unexpected nested per-leaf keys to
execute_pipeline because _filter_external_inputs is only applied inside the
snapshot_external_inputs branch; fix by applying _filter_external_inputs
regardless of snapshot flag before building request_external_inputs. Update the
logic around _validate_external_inputs, _filter_external_inputs,
snapshot_pipeline_inputs, and request_external_inputs so that external_inputs is
passed through _filter_external_inputs unconditionally (then
snapshot_pipeline_inputs only when snapshot_external_inputs is true) so both
sync and pipelined modes drop extra per-leaf keys consistently.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: ae084988-ad5f-485f-84dc-da676be73136
📒 Files selected for processing (12)
AGENTS.mddocs/source/getting_started/teleop_session.rstsrc/core/AGENTS.mdsrc/core/retargeting_engine/python/interface/tensor_group.pysrc/core/retargeting_engine_tests/python/test_tensor_group.pysrc/core/teleop_session_manager/CMakeLists.txtsrc/core/teleop_session_manager/python/__init__.pysrc/core/teleop_session_manager/python/async_retarget_runner.pysrc/core/teleop_session_manager/python/config.pysrc/core/teleop_session_manager/python/teleop_session.pysrc/core/teleop_session_manager_tests/python/test_session_reset.pysrc/core/teleop_session_manager_tests/python/test_teleop_session.py
Pipelined retargeting now submits the current request and returns the latest completed output after the seed frame. Exact current-frame sample/compute/return behavior stays in sync mode, while deadline-guarded pacing improves temporal alignment without current-frame barriers. Add the async runner, immediate/deadline pacing configs, docs, and tests for the simplified contract.
hougantc-nvda
force-pushed
the
hougantc/pipelined-retargeting
branch
from
52183e7 to
11607bd
Compare
* feat(teleop): add latest-result pipelined retargeting Pipelined retargeting now submits the current request and returns the latest completed output after the seed frame. Exact current-frame sample/compute/return behavior stays in sync mode, while deadline-guarded pacing improves temporal alignment without current-frame barriers. Add the async runner, immediate/deadline pacing configs, docs, and tests for the simplified contract. * Address coderabbit comment
2 participants
This PR adds opt-in pipelined retargeting for TeleopSession, allowing applications to submit the current frame’s retarget request while returning the latest completed result after an initial seed frame. The default synchronous mode preserves exact current-frame sample/compute/return behavior, while pipelined mode runs the normal step body on a single background worker and keeps returned outputs, last_context, and step metadata self-consistent for the completed frame.
It also adds execution and pacing configuration, including immediate and deadline-based pacing, plus last_step_info timing/age diagnostics for observing result freshness, dropped submissions, deadline misses, and worker failures. Snapshot support for TensorGroup values was added so external inputs and cached outputs can safely cross the async boundary, with docs and tests covering the new contract.
Summary by CodeRabbit
New Features
Bug Fixes
Documentation
Tests