Skip to content

Default hourly() to poll_interval=1 and forward additional_kw to retrieve - #12

Draft
aklocker42 wants to merge 1 commit into
mainfrom
akl/hourly-poll-interval-default
Draft

aklocker42 wants to merge 1 commit into
mainfrom
akl/hourly-poll-interval-default

Conversation

@aklocker42

Copy link
Copy Markdown
Collaborator

Summary

  • hourly, monthly and yearly accepted additional_kw... but never forwarded it to retrieve, so poll_interval, max_wait and verbose could not be overridden. It is now forwarded.
  • hourly() gets an explicit poll_interval=1 keyword (documented). This matches CDSAPI.jl, which polls every 1 s.
  • monthly() and yearly() keep a default of 10. Their jobs are long, so a shorter interval only adds status calls. Any kwarg you pass now overrides the defaults.

Why

Relates to #4 (CDSAPI.jl vs this package). Hourly requests are small and usually finish within a minute. retrieve backs off from 1 s up to poll_interval, so with the old ceiling of 10 the client could sit on a finished job for several seconds before noticing.

Paired timings against CDSAPI.jl are dominated by CDS queue time, with the same request ranging from 13.9 s to 101.9 s. The effect is real but hard to see statistically:

Comparison (single-variable hourly requests) Result
Earlier 12-pair run at 1 s polling, this package vs CDSAPI.jl mean 40.4 s vs 47.2 s, this package faster in 7 of 12 pairs
6-pair run, poll_interval=10 vs 1 mean 50.7 s vs 47.7 s, 3 wins each

The earlier runs at the old default of 10 were slower than CDSAPI.jl: 40.6 s vs 23.2 s and 48.9 s vs 23.2 s.

Tests

  • New offline test "poll_interval plumbing" stubs retrieve and checks the defaults and overrides reach it for hourly (1), monthly (10) and yearly (10).
  • The stub redefines retrieve inside the module, so that testset has to stay last in runtests.jl.
  • Pkg.test(): 36 of 36 pass, including the live ERA5 download test.

🤖 Generated with Claude Code

…ieve

hourly/monthly/yearly accepted `additional_kw...` but never forwarded it, so
poll_interval, max_wait and verbose could not be overridden. Forward it, and
default hourly() to poll_interval=1 (matching CDSAPI.jl's 1 s polling) since
hourly requests are small; monthly/yearly keep 10.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant