feat(rocm): show live progress for multi-gigabyte downloads - #347
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Fix premature 100% reporting and preserve known expected totals in progress callbacks before approval.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds live, resume-aware TTY download progress for large ROCm SDK and ComfyUI archives.
Changes:
- Adds streaming download progress callbacks.
- Introduces a shared, throttled progress spinner.
- Integrates progress into SDK and ComfyUI downloads.
File summaries
| File | Description |
|---|---|
crates/rocm-core/src/lib.rs |
Adds progress-aware downloads; callbacks should fall back to expected_len. |
apps/rocm/src/therock.rs |
Adds SDK tarball progress; PTY scenario coverage is missing. |
apps/rocm/src/serve_summary.rs |
Removes the relocated spinner implementation. |
apps/rocm/src/main.rs |
Uses the shared spinner for server startup. |
apps/rocm/src/comfyui.rs |
Adds archive progress; command-level PTY coverage is missing. |
apps/rocm/src/cli_progress.rs |
Implements shared progress rendering; percentage must not round incomplete transfers to 100%, and PTY behavior needs scenario coverage. |
Review details
Suppressed comments (4)
apps/rocm/src/cli_progress.rs:68
- This is user-observable terminal behavior, but the added tests call
Spinnerdirectly while stderr is normally non-TTY, sorender_currentreturns without proving the live repaint, TTY gating, or cleanup used by either install command. Add Gherkin coverage for the SDK tarball and ComfyUI download paths using the repository's existing PTY harness and a deterministic local download source; unit callback/formatter tests do not satisfy the required CLI scenario coverage.
pub(crate) fn set_progress(&mut self, prefix: &str, bytes: u64, total: Option<u64>) {
apps/rocm/src/comfyui.rs:1308
- The ComfyUI install now has new user-visible TTY progress, but its test invokes only the internal download wrapper and cannot detect missing rendering, TTY gating, or line cleanup. Add a Gherkin scenario using the existing PTY driver and a deterministic source-download fixture to exercise this command-level behavior, as required for observable CLI changes.
let download_result = download_file(
COMFYUI_SOURCE_ARCHIVE_URL,
&archive_path,
&mut |bytes, total| {
spinner.set_progress("Fetching ComfyUI source archive…", bytes, total);
},
);
apps/rocm/src/therock.rs:1103
- This adds user-visible terminal behavior to
rocm install sdk --format tarball, but the PR only unit-tests callback values and explicitly leaves the interactive path to a manual smoke check. Add a Gherkin scenario that drives this command through the existingtests/e2e-cucumber/tests/e2e/tui_driver.rsPTY harness and verifies an in-progress byte/percentage frame and cleanup; the repository contribution policy requires scenarios for observable CLI output.
let download_result = download_file(&artifact.url, &cache_path, &mut |bytes, total| {
spinner.set_progress(&download_label, bytes, total);
});
crates/rocm-core/src/lib.rs:474
- This per-chunk event has the same known-total loss as the initial event: downloads with
expected_lenbut no responseContent-LengthreportNonethroughout. Use the request's expected size as the fallback so callback semantics remain consistent for every event.
on_progress(written, total_len);
- Files reviewed: 6/6 changed files
- Comments generated: 2
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
rocm_core::download_file_streaming gains a progress-callback sibling reporting cumulative bytes (and total, when known) after every chunk, resume-aware so a retried attempt reports its true starting offset instead of 0. Extract the serve command's TTY-gated stderr spinner into a shared apps/rocm::cli_progress module and give it a throttled, byte-aware repaint mode with a high-water-mark clamp so a retry never visibly regresses the displayed progress. Wire the SDK tarball install path (therock.rs) and the ComfyUI source archive download (comfyui.rs) to drive that spinner, so an interactive terminal shows a live "X / Y (Z%)" status line while non-interactive output is unaffected. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
- format_download_progress rounded 99.5% up to a misleading "100%" while bytes were still outstanding; floor instead, reserving 100% for bytes >= total. - download_attempt's on_progress callback reported None whenever the server omitted Content-Length, even when the caller had already supplied expected_len (the same value already used for preflight disk-space and size checks). Fall back to it so progress reporting doesn't lose a total that's already known. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
fb2fb04 to
cbb856e
Compare
download_file_streaming_with_progress's doc comment claimed a resumed attempt always reports its true starting offset instead of 0, but that only holds for a confirmed 206 continuation. A restart that discards the partial file (the server ignored Range, or resumed at the wrong offset) truncates written back to 0 internally, and on_progress saw that raw value — a real regression the doc comment promised wouldn't happen. This was invisible in practice only because both current callers (therock.rs, comfyui.rs) apply their own high-water-mark clamp in Spinner. Any caller of the primitive without that clamp would show progress jumping backward. Wrap on_progress in download_file_streaming_with_progress with a high-water mark so the guarantee holds in rocm-core itself, for every caller, not just ones that add their own UI-side clamp. download_attempt keeps counting from 0 on a from-scratch restart internally; that's now explicitly documented as an implementation detail the wrapper absorbs. Adds a test exercising the restart-from-scratch path (a wrong-offset resume discarding the partial file) and asserting the callback sequence never decreases. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
cli_progress.rs: Spinner::set_progress's doc comment described the final-chunk always-repaints exception but not that the very first call also always repaints, since last_progress_paint starts unset. Note both explicitly. comfyui.rs: bind the progress label to a variable once instead of repeating the literal, matching therock.rs's existing pattern. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Two UX gaps in the download progress spinner, found while reviewing the progress-indication feature: - The serve spinner keeps animating throughout its wait because its poll loop ticks every iteration regardless of readiness. The download spinner only ticked once before the transfer started and otherwise relied entirely on on_progress firing from data arrival, so it froze solid during a stall (slow handshake, a mid-transfer network hiccup) with no indication the process hadn't hung. AnimatedSpinner fixes this with a background thread that ticks on a fixed interval independent of progress callbacks. - render_current never checked terminal width. Download labels (with byte counts and percentages) are long enough to wrap on a narrow terminal, and Clear(CurrentLine) can only erase the row the cursor ends up on after wrapping, leaving stale fragments behind on every repaint. truncate_to_width keeps every repaint confined to one row. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
There was a problem hiding this comment.
🟡 Changes recommended
Percentage precision and Unicode display-width handling must be corrected; PTY coverage is also requested.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 6/6 changed files
- Comments generated: 4
- Review effort level: Balanced
Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Addresses Copilot review comments on PR #347: - truncate_to_width measured char count, not terminal columns; a wide (e.g. CJK) glyph in a label could still overflow the row and leave stale spinner fragments. Truncate by Unicode display width instead. - format_download_progress computed the percentage via an f64 ratio, which loses precision for u64 values near u64::MAX and could report 100% while a byte was still outstanding. Use exact u128 arithmetic. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Signed-off-by: Jussi Elo <jussi.elo@amd.com>
|
🔴 Automated review · pr-review-watcher · 7565e19 SummaryAdds live byte-progress reporting to 🚫 Blocking (must fix before merge)None. Non-blocking
|
Clarify that the progress reporter's monotonic guarantee covers the byte count, not total (which can change across a retry); document that a from-scratch restart holds the count flat until it catches up. Rename a resume test to match what it actually guards, and note the sibling test that exercises the high-water-mark clamp. Narrow the spinner's "never stale" doc to the known-total case. Skip spawning the animated spinner's ticker thread when stderr isn't a TTY, since every repaint it would trigger is a no-op there. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
|
Addressed all 5 non-blocking items in d8d8f9d:
|
There was a problem hiding this comment.
🟡 Changes recommended
The spinner test can fail during normal interactive test runs because it depends on inherited stderr TTY state.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 7/8 changed files
- Comments generated: 1
- Review effort level: Balanced
The disabled-ticker test relied on the test process's real stderr not being a TTY, so it only passed because CI happens to redirect stderr. Run from an interactive terminal it would flip to failing. Force Spinner::enabled explicitly via a test-only constructor instead of probing the environment. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Summary
rocm_core::download_file_streaminggains a_with_progresssibling reporting cumulative bytes (and total, when known) after every chunk, resume-aware so a retried attempt reports its true starting offset instead of 0servecommand's TTY-gated stderr spinner into a sharedapps/rocm::cli_progressmodule; add a throttled, byte-aware repaint mode with a high-water-mark clamp so a retry never visibly regresses displayed progresstherock.rs) and the ComfyUI source archive download (comfyui.rs) to drive that spinner, so an interactive terminal shows a liveX / Y (Z%)status line while non-interactive/piped output is unaffectedTest plan
cargo fmt -p rocm -p rocm-corecargo clippy -p rocm -p rocm-core --all-targets— no warningscargo test -p rocm -p rocm-core— 557 + 326 tests passing, including new coverage for cumulative/resume-continuous progress reporting (rocm-core) and the progress high-water-mark clamp (cli_progress)rocm install sdk --format tarballagainst a real artifact to visually confirm the spinner renders and clears correctlye2e coverage gap (AGENTS.md §3): the spinner is TTY-gated
(
stderr().is_terminal()), so it cannot be observed by thenon-interactive
tests/e2e-cucumberharness — the same gap as thepre-existing
servespinner, which also has no feature-file coverage.Covered instead at the unit level: cumulative/resume-continuous progress
reporting in
rocm-core, and the repaint-throttle/high-water-mark clampin
cli_progress.