gh-64862: Add the stop_exception parameter in iter() and aiter() - #156298
gh-64862: Add the stop_exception parameter in iter() and aiter()#156298serhiy-storchaka wants to merge 6 commits into
Conversation
The created iterator stops when the callable raises the specified exception. The second parameter of iter() is now named stop_value and can be passed as a keyword argument. aiter() now accepts the same stop_value and stop_exception parameters, calling an asynchronous callable and awaiting the result. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Documentation build overview
33 files changed ·
|
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Use StopIteration (StopAsyncIteration for aiter()) as the default instead of normalizing it to NULL, so that the check is a single PyErr_ExceptionMatches(). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
If the callable raises StopIteration (StopAsyncIteration in aiter()) which does not match stop_exception, the consumer would mistake it for the end of the iteration, or, in the asynchronous case, for the result of the await. Replace it with RuntimeError, as PEP 479 and PEP 525 do for generators. StopIteration is therefore no longer special: it stops the iteration only because it is the default stop_exception. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| /* Both are set to NULL when the iterator is exhausted */ | ||
| PyObject *it_callable; |
There was a problem hiding this comment.
“Both” doesn't make sense with 3 items. Should all be NULLed on exhaustion?
There was a problem hiding this comment.
It was related only to it_callable and it_sentinel. Reworded.
it_stop_exc is not NULLed intentionally. In case of reentrant __next__ call (usually a concurrent use) we can get an exception, after the iterator was exhausted. Without it_stop_exc we cannot distinguish a StopIteration which stops iteration from StopIteration which should be converted to RuntimeError.
it_callable and it_sentinel should be NULLed because they can keep large objects, but it_stop_exc is normally just a type or a tuple of types.
| PyObject_HEAD | ||
| PyObject *aw_iterator; /* the iterator which created this object */ | ||
| PyObject *aw_wrapped; /* the awaitable returned by the callable */ | ||
| char aw_closed; |
There was a problem hiding this comment.
Nitpick: we can use bool internally.
serhiy-storchaka
left a comment
serhiy-storchaka
left a comment
There was a problem hiding this comment.
Thank you for your review. Applied suggestions, answered questions.
| /* Both are set to NULL when the iterator is exhausted */ | ||
| PyObject *it_callable; |
There was a problem hiding this comment.
It was related only to it_callable and it_sentinel. Reworded.
it_stop_exc is not NULLed intentionally. In case of reentrant __next__ call (usually a concurrent use) we can get an exception, after the iterator was exhausted. Without it_stop_exc we cannot distinguish a StopIteration which stops iteration from StopIteration which should be converted to RuntimeError.
it_callable and it_sentinel should be NULLed because they can keep large objects, but it_stop_exc is normally just a type or a tuple of types.
Reword the aiter() documentation like the iter() one, avoid using the name "queue" for two different things in the example, describe every field of the iterator structs separately, add calliter_exhaust() for symmetry with acalliter_exhaust(), use bool and designated initializers in new code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
encukou
left a comment
encukou
left a comment
There was a problem hiding this comment.
Thanks! I found one more nitpick; otherwise this looks good to me!
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2 participants
iter()andaiter()now accept the keyword-only stop_exception parameter -- an exception class or a tuple of exception classes which ends the iteration:Many callables report exhaustion by raising an exception instead of returning a special value, so the sentinel form cannot be used with them at all.
aiter()also gained the callable form, which it did not have before: the callable is called and its result is awaited for every__anext__()(the callable is only called when the result of__anext__()is awaited).The second parameter of
iter()is now named stop_value and can be passed by keyword. It can be omitted if stop_exception is given.stop_exception=StopIteration(StopAsyncIterationforaiter()) and an empty tuple never change the behavior, so they are normalized to "no stop exception"; such an iterator is pickled exactly as before. For other casescallable_iteratornow has__setstate__(), because the stop exception and the absence of the sentinel cannot be expressed as arguments ofiter().