feat(rest): add ProgressEndpoint with download progress via ReadableStream#3863
feat(rest): add ProgressEndpoint with download progress via ReadableStream#3863
Conversation
…tream Adds a ProgressEndpoint subclass of RestEndpoint that supports onDownloadProgress callbacks using the ReadableStream API. - Overrides fetchResponse() to intercept the response body stream - Reads chunks via ReadableStream, calling onDownloadProgress per chunk - Reconstructs Response for parseResponse compatibility - Handles edge cases: Content-Encoding, mid-stream errors, abort signals - Gracefully degrades when response.body is unavailable (polyfills) Co-authored-by: natmaster <natmaster@gmail.com>
Tests cover:
- Progress callback with correct loaded/total/lengthComputable
- Missing Content-Length (lengthComputable: false)
- Content-Encoding present (lengthComputable: false)
- No-op when onDownloadProgress not set
- Graceful fallback when response.body is null
- Response.url preserved through reconstruction
- Mid-stream TypeError gets status 500
- Works with .extend()
- Works with resource({ Endpoint: ProgressEndpoint })
- 204 No Content handled correctly
- Text/non-JSON responses with progress
- Callback exceptions propagate
Co-authored-by: natmaster <natmaster@gmail.com>
Adds doc page for ProgressEndpoint with usage examples, DownloadProgress interface reference, graceful degradation notes, and inheritance guide. Adds to REST sidebar under Endpoint API. Co-authored-by: natmaster <natmaster@gmail.com>
🦋 Changeset detectedLatest commit: 5beebc3 The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #3863 +/- ##
==========================================
+ Coverage 98.08% 98.10% +0.01%
==========================================
Files 152 153 +1
Lines 2876 2902 +26
Branches 564 564
==========================================
+ Hits 2821 2847 +26
Misses 11 11
Partials 44 44 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Size Change: 0 B Total Size: 80.9 kB ℹ️ View Unchanged
|
Co-authored-by: natmaster <natmaster@gmail.com>
Co-authored-by: natmaster <natmaster@gmail.com>
… network-transform docs Co-authored-by: natmaster <natmaster@gmail.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit c7571d5. Configure here.
…gressEndpoint The catch block in fetchResponse tagged any TypeError with status=500, but the onProgress callback is invoked inside readWithProgress's try block. If the user's onDownloadProgress callback throws a TypeError (e.g., accessing a property of undefined), it gets incorrectly marked as a network error, causing errorPolicy to return 'soft' and triggering retry logic for what is actually a bug in user code. Move the TypeError tagging into readWithProgress, scoped only to the reader.read() call. Callback errors now propagate unmodified. Co-authored-by: Nathaniel Tucker <me@ntucker.me>

Motivation
Closes the feature gap versus axios's
onDownloadProgress/onUploadProgresscallbacks. Users of@data-client/restpreviously had no built-in way to track download progress for file downloads or large API responses.Solution
Adds a new
ProgressEndpointclass that extendsRestEndpointwith download progress tracking using the ReadableStream API. This keeps the coreRestEndpointuntouched -- users opt in by usingProgressEndpointdirectly or viaresource({ Endpoint: ProgressEndpoint }).Key design decisions:
ProgressEndpointoverridesfetchResponse()to intercept the response body stream betweensuper.fetchResponse()andparseResponse(), reading chunks viaReadableStreamand calling the progress callback per chunk.Responseis constructed from the collectedBlobsoparseResponse()works unchanged.response.urlis preserved viaObject.defineProperty.Content-Encoding(gzip/br) → setslengthComputable: falsesinceContent-Lengthreflects compressed sizeTypeErrorfromreader.read()→ tagged with.status = 500to matchfetchResponse's error handlingreader.releaseLock()infinallyfor cleanupwhatwg-fetchpolyfill (noresponse.body) → gracefully skips progress trackingUsage:
New exports from
@data-client/rest:ProgressEndpoint— the classDownloadProgress— the progress event typeBug fix:
TypeErrorfromonDownloadProgresscallback no longer tagged as network errorThe
catchblock infetchResponsepreviously tagged anyTypeErrorwithstatus = 500. Since theonProgresscallback is invoked insidereadWithProgress'stryblock, aTypeErrorfrom user code (e.g., accessing a property ofundefinedin the callback) would be incorrectly classified as a network error, causingerrorPolicyto return'soft'and triggering retry logic.Fix: Moved the
TypeErrortagging intoreadWithProgress, scoped only around thereader.read()call. Callback errors now propagate unmodified. Added a test to verify this behavior.Open questions
onUploadProgress) viaReadableStreambody +duplex: 'half'is left for a follow-up. TheonDownloadProgressnaming anticipates this.onDownloadProgressis not currently inRestEndpointOptions/RestInstanceBaseto avoid leaking into the genericOtype parameter. It can be passed via theProgressEndpointconstructor or.extend()(where the baseEndpointconstructor'sObject.assignpicks it up at runtime).