Skip to content

feat(streaming): add EventStreamer for iterating stream events - #1567

Draft
ajbozarth wants to merge 1 commit into
generative-computing:mainfrom
ajbozarth:feat/1440-events-streamer
Draft

feat(streaming): add EventStreamer for iterating stream events#1567
ajbozarth wants to merge 1 commit into
generative-computing:mainfrom
ajbozarth:feat/1440-events-streamer

Conversation

@ajbozarth

Copy link
Copy Markdown
Contributor

⚠️ Design POC — not finished, not for merge. This is a draft to evaluate the approach for consuming streaming events, following the discussion on #1543. Tests, typing checks, and docs are intentionally deferred (see below). Please review the design, not the completeness.

Issue

Follow-up to the design discussion on #1543 (now merged) — streaming event consumption ergonomics. No separate tracking issue yet — flagging for a maintainer to route.

Description

Adds an opt-in EventStreamer, returned by stream(..., as_events=True), that lets you iterate a stream's typed StreamEvent objects directly with async for — rather than the only route today, a process-global STREAMING_EVENT plugin hook demuxed on streaming_id.

It wraps a Streamer, drives it to completion on a background task, and delivers events through a queue. Because the queue's transport (put_nowait) is legal from teardown and during exception unwinding, the terminal CompletedEvent/ErrorEvent are delivered on every exit path — including an early break — which the alternative of yielding events from the driver generator cannot do (yield is illegal during GeneratorExit).

Additive and opt-in. The default stream() chunk-iterating contract is unchanged. Implementation: an optional per-stream event_queue is threaded through _emit_event; the STREAMING_EVENT hook still fires on all paths (telemetry unaffected). stream() is split into _stream() + a dispatching wrapper with typed @overloads on as_events.

Testing

  • Tests added to the respective file if code was changed
  • New code has 100% coverage if code was added
  • Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Deferred — this is a design-review draft. Not yet done: unit tests for EventStreamer, a test/typing/check_*.py for the overloads, a docs example, and two known edge cases (setup-failure event symmetry; external cancellation during setup). Verified locally: ruff format/check clean, mypy clean, existing test_streaming.py passes (37), and an offline harness exercises all four event-mode paths (natural completion, requirement-fail, error re-raise, early break with no task leak).

Attribution

  • AI coding assistants used

  • Component
  • Requirement
  • Sampling Strategy
  • Tool

stream(as_events=True) returns an EventStreamer that yields a stream's
typed StreamEvent objects via async-for, instead of the Streamer's str
chunks. It runs the stream to completion on a background task and
delivers events through a queue, so the terminal CompletedEvent and
ErrorEvent reach the consumer on every exit path — including an early
break — which iterating the driver's generator cannot guarantee.

Additive and opt-in: the default stream() chunk contract is unchanged.
Events are threaded to an optional per-stream queue in _emit_event; the
STREAMING_EVENT hook still fires on all paths, so telemetry is
unaffected. stream() is split into _stream() plus a dispatching wrapper
with typed overloads on as_events.

Tests, typing checks, and docs are deferred to follow-ups.

Assisted-by: Claude Code
Signed-off-by: Alex Bozarth <ajbozart@us.ibm.com>
@github-actions github-actions Bot added the enhancement New feature or request label Aug 18, 2026
@ajbozarth ajbozarth self-assigned this Aug 18, 2026
@ajbozarth ajbozarth added area/stdlib Core abstractions: Context, MOT, SamplingStrategy, formatters, serialization area/streaming Streaming chunks, events, per-chunk validation labels Aug 18, 2026
@ajbozarth

Copy link
Copy Markdown
Contributor Author

@jakelorocco here's my initial draft POC for a events iterator for streaming. This is the better of the two POCs I wrote and I'll leave this here for your review while I'm out. Once I get back next week we can sync and I'll continue work on this

@jakelorocco jakelorocco left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pretty much looks like exactly what I was looking for; thank you!

Comment on lines +368 to +381
try:
self._streamer = await _stream(
action,
backend,
ctx,
chunking=chunking,
requirements=requirements,
validation_backend=validation_backend,
event_queue=self._queue,
)
async for _ in self._streamer: # chunks discarded; events are surfaced
pass
finally:
self._queue.put_nowait(None)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are awaiting the underlying _stream function, the stream still progresses even when not directly awaiting on this event streamer, right?

Comment on lines +369 to +377
self._streamer = await _stream(
action,
backend,
ctx,
chunking=chunking,
requirements=requirements,
validation_backend=validation_backend,
event_queue=self._queue,
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it not an option to do the as_events boolean on the streamer itself and remove the need for discarding empty chunks and creating a new queue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/stdlib Core abstractions: Context, MOT, SamplingStrategy, formatters, serialization area/streaming Streaming chunks, events, per-chunk validation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants