Repository navigation
fix(verl): prevent stale/failed rollouts from breaking batch completion - #649
Open
Anvarbek Ziyodov (ziyodovanvar) wants to merge 2 commits into
Open
Anvarbek Ziyodov (ziyodovanvar) wants to merge 2 commits into
Anvarbek Ziyodov (ziyodovanvar) wants to merge 2 commits into
Conversation
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The grace-poll branch exits immediately after initialization, preventing successful retries from being collected once all IDs have results.
1 open issue
What changed in this PR
Addresses #355 on stable/v0.2.x by hardening VERL batch completion against stale results and failed rollouts.
Changes:
- Drains stale completions, rejects unknown IDs, and reconciles batch counts.
- Allows successful retries to replace failed placeholders and introduces bounded grace polling.
- Adds bookkeeping regression tests and lightweight VERL import stubs.
| File | Description |
|---|---|
| tests/verl/test_daemon_completed_rollouts.py | Tests failure detection, stale-ID removal, and retry replacement. |
| tests/verl/conftest.py | Provides a VERL stub for unit tests. |
| tests/verl/__init__.py | Marks the test directory as a package. |
| agentlightning/verl/daemon.py | Updates completion filtering, reconciliation, and retry handling. |
Review effort: Balanced
| if failed_grace_polls_remaining is None: | ||
| # A few extra polls (~15s) for in-flight retries to land. | ||
| failed_grace_polls_remaining = 3 | ||
| elif failed_grace_polls_remaining > 0: |
There was a problem hiding this comment.
Good catch — fixed.
The grace counter init used elif, so the first all-present-but-failed poll set the counter and then fell through to break without sleeping. It is now an independent if, and test_async_run_until_finished_grace_polls_when_only_failure_present covers fail-on-first / succeed-on-next.
Author
|
@microsoft-github-policy-service agree |
Anvarbek Ziyodov (ziyodovanvar)
force-pushed
the
fix/355-stale-completed-rollouts
branch
from
November 17, 2026 02:34
937836c to
df144e5
Compare
Reject leftover server results, allow successful retries to overwrite empty failed placeholders, and soften the completed==queued assert so training does not crash when counts drift (microsoft#355).
Anvarbek Ziyodov (ziyodovanvar)
force-pushed
the
fix/355-stale-completed-rollouts
branch
from
November 17, 2026 02:37
df144e5 to
a1513a0
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
stable/v0.2.x(AgentModeDaemonwas removed frommain).rollout_id, with a short grace poll window so in-flight retries can land.completed == queuedasserts with reconciliation that drops stale IDs and warns instead of crashing training.This is a fresh take on the closed #480 approach: failed rollouts can still satisfy the wait loop (so permanent backend failures do not hang forever), but they no longer permanently block a later successful retry, and leftover results from previous steps cannot inflate the completed count.
Test plan
pytest -q tests/verl/test_daemon_completed_rollouts.py(4 passed)Completed N/MwithN > Mafter vLLM 400s and confirm the assert no longer firesRelated issues
Made with Cursor