Skip to content

feat(rest): add ProgressEndpoint with download progress via ReadableStream#3863

Open
ntucker wants to merge 7 commits intomasterfrom
cursor/axios-features-7ab3
Open

feat(rest): add ProgressEndpoint with download progress via ReadableStream#3863
ntucker wants to merge 7 commits intomasterfrom
cursor/axios-features-7ab3

Conversation

@ntucker
Copy link
Copy Markdown
Collaborator

@ntucker ntucker commented Apr 3, 2026

Motivation

Closes the feature gap versus axios's onDownloadProgress / onUploadProgress callbacks. Users of @data-client/rest previously had no built-in way to track download progress for file downloads or large API responses.

Solution

Adds a new ProgressEndpoint class that extends RestEndpoint with download progress tracking using the ReadableStream API. This keeps the core RestEndpoint untouched -- users opt in by using ProgressEndpoint directly or via resource({ Endpoint: ProgressEndpoint }).

Key design decisions:

  • Subclass approach: ProgressEndpoint overrides fetchResponse() to intercept the response body stream between super.fetchResponse() and parseResponse(), reading chunks via ReadableStream and calling the progress callback per chunk.
  • Response reconstruction: After reading all chunks, a new Response is constructed from the collected Blob so parseResponse() works unchanged. response.url is preserved via Object.defineProperty.
  • Edge cases handled:
    • Content-Encoding (gzip/br) → sets lengthComputable: false since Content-Length reflects compressed size
    • Mid-stream TypeError from reader.read() → tagged with .status = 500 to match fetchResponse's error handling
    • Abort signals propagate automatically; reader.releaseLock() in finally for cleanup
    • whatwg-fetch polyfill (no response.body) → gracefully skips progress tracking
    • 204 No Content / null body → handled by truthiness check

Usage:

import { ProgressEndpoint, resource } from '@data-client/rest';

const FileResource = resource({
  path: '/files/:id',
  schema: FileEntity,
  Endpoint: ProgressEndpoint,
});

const downloadWithProgress = FileResource.get.extend({
  onDownloadProgress({ loaded, total, lengthComputable }) {
    setProgress(lengthComputable ? loaded / total : undefined);
  },
});

New exports from @data-client/rest:

  • ProgressEndpoint — the class
  • DownloadProgress — the progress event type

Bug fix: TypeError from onDownloadProgress callback no longer tagged as network error

The catch block in fetchResponse previously tagged any TypeError with status = 500. Since the onProgress callback is invoked inside readWithProgress's try block, a TypeError from user code (e.g., accessing a property of undefined in the callback) would be incorrectly classified as a network error, causing errorPolicy to return 'soft' and triggering retry logic.

Fix: Moved the TypeError tagging into readWithProgress, scoped only around the reader.read() call. Callback errors now propagate unmodified. Added a test to verify this behavior.

Open questions

  • Upload progress (onUploadProgress) via ReadableStream body + duplex: 'half' is left for a follow-up. The onDownloadProgress naming anticipates this.
  • onDownloadProgress is not currently in RestEndpointOptions/RestInstanceBase to avoid leaking into the generic O type parameter. It can be passed via the ProgressEndpoint constructor or .extend() (where the base Endpoint constructor's Object.assign picks it up at runtime).
Open in Web Open in Cursor 

cursoragent and others added 3 commits April 3, 2026 20:59
…tream

Adds a ProgressEndpoint subclass of RestEndpoint that supports
onDownloadProgress callbacks using the ReadableStream API.

- Overrides fetchResponse() to intercept the response body stream
- Reads chunks via ReadableStream, calling onDownloadProgress per chunk
- Reconstructs Response for parseResponse compatibility
- Handles edge cases: Content-Encoding, mid-stream errors, abort signals
- Gracefully degrades when response.body is unavailable (polyfills)

Co-authored-by: natmaster <natmaster@gmail.com>
Tests cover:
- Progress callback with correct loaded/total/lengthComputable
- Missing Content-Length (lengthComputable: false)
- Content-Encoding present (lengthComputable: false)
- No-op when onDownloadProgress not set
- Graceful fallback when response.body is null
- Response.url preserved through reconstruction
- Mid-stream TypeError gets status 500
- Works with .extend()
- Works with resource({ Endpoint: ProgressEndpoint })
- 204 No Content handled correctly
- Text/non-JSON responses with progress
- Callback exceptions propagate

Co-authored-by: natmaster <natmaster@gmail.com>
Adds doc page for ProgressEndpoint with usage examples, DownloadProgress
interface reference, graceful degradation notes, and inheritance guide.
Adds to REST sidebar under Endpoint API.

Co-authored-by: natmaster <natmaster@gmail.com>
@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Apr 3, 2026

🦋 Changeset detected

Latest commit: 5beebc3

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@data-client/rest Minor
example-benchmark-react Patch
test-bundlesize Patch
coinbase-lite Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel
Copy link
Copy Markdown

vercel bot commented Apr 3, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs-site Ignored Ignored Preview Apr 3, 2026 10:13pm

@ntucker ntucker marked this pull request as ready for review April 3, 2026 21:39
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 3, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.10%. Comparing base (17d938b) to head (5beebc3).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3863      +/-   ##
==========================================
+ Coverage   98.08%   98.10%   +0.01%     
==========================================
  Files         152      153       +1     
  Lines        2876     2902      +26     
  Branches      564      564              
==========================================
+ Hits         2821     2847      +26     
  Misses         11       11              
  Partials       44       44              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

Size Change: 0 B

Total Size: 80.9 kB

ℹ️ View Unchanged
Filename Size
examples/test-bundlesize/dist/App.js 3.25 kB
examples/test-bundlesize/dist/polyfill.js 307 B
examples/test-bundlesize/dist/rdcClient.js 10.4 kB
examples/test-bundlesize/dist/rdcEndpoint.js 6.39 kB
examples/test-bundlesize/dist/react.js 59.7 kB
examples/test-bundlesize/dist/webpack-runtime.js 938 B

compressed-size-action

cursoragent and others added 2 commits April 3, 2026 21:44
Co-authored-by: natmaster <natmaster@gmail.com>
Co-authored-by: natmaster <natmaster@gmail.com>
… network-transform docs

Co-authored-by: natmaster <natmaster@gmail.com>
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit c7571d5. Configure here.

…gressEndpoint

The catch block in fetchResponse tagged any TypeError with status=500,
but the onProgress callback is invoked inside readWithProgress's try
block. If the user's onDownloadProgress callback throws a TypeError
(e.g., accessing a property of undefined), it gets incorrectly marked
as a network error, causing errorPolicy to return 'soft' and triggering
retry logic for what is actually a bug in user code.

Move the TypeError tagging into readWithProgress, scoped only to the
reader.read() call. Callback errors now propagate unmodified.

Co-authored-by: Nathaniel Tucker <me@ntucker.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants