Repository navigation
Python: fix(ag-ui): keep repeated single-chunk text deltas - #9339
Conversation
A duplicate delta equal to the whole accumulated text was always treated as a full-message replay and dropped. When the message only streamed one chunk so far, that same shape is a genuine repeated token (like the 2s in 22) and the text was silently lost. Only treat the duplicate as a replay when it carries cumulative annotations or the text already streamed in more than one chunk, and track the emitted delta count on FlowState. Fixes microsoft#9336 Signed-off-by: Manohar Paturi <186662190+ManoharPaturi@users.noreply.github.com>
GitHub Actions
Active
GitHub Actions
Active
agent-framework-automation
Bot
added
the
python
GitHub Actions
Active
|
/review |
GitHub Actions
Active
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
MAF Automated Review — Iteration 1
Result: Findings reported
Scope: full PR (2 commit(s)): fa37b1f7c2fe, 15b174520e51
Model: gpt-5.6-sol
Overview
The review found 1 verified inline finding(s).
Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
1 verified finding remained after source verification (1 medium) across 1 file. Details are attached to the affected lines below.
Affected areas: python/packages/ag-ui/agent_framework_ag_ui/_run_common.py
|
/review |
…duplicate The replay guard still treated any annotated duplicate equal to the accumulated text as a cumulative replay, dropping a genuine repeated token that happened to carry annotations. A replay re-sends the annotations already recorded on the message segment, so the guard now suppresses only when the duplicate's annotation set covers the recorded occurrences. A repeated delta with new annotations streams normally. Follow-up to the review on microsoft#9339. Signed-off-by: Manohar Paturi <186662190+ManoharPaturi@users.noreply.github.com>
GitHub Actions
Active
GitHub Actions
Active
Eduard van Valkenburg (eavanvalkenburg)
left a comment
Eduard van Valkenburg (eavanvalkenburg)
left a comment
There was a problem hiding this comment.
Manohar Paturi (@ManoharPaturi) The current replay heuristic still drops a genuine repeated annotated token when the second delta carries the same annotation payload as the first. For example, streaming "ha" with citation A and then another real "ha" with the same citation A should produce "haha". At python/packages/ag-ui/agent_framework_ag_ui/_run_common.py:678, _annotations_replay_recorded_occurrences returns true because the recorded annotation appears in the incoming set, so the second token is classified as a cumulative replay and never appended.
The added regression covers a repeated token with a different annotation URL, but not this same-annotation case. Please keep the one-prior-delta ambiguity policy consistent with the unannotated path, or defer the replay decision explicitly, and add a regression for two genuine identical annotated deltas.
…ted ones A duplicate equal to the whole accumulated text was still suppressed after a single prior chunk whenever its annotations re-sent the recorded occurrences, which dropped a genuine repeated token carrying the same citation. Suppression now needs more than one prior chunk for annotated duplicates too, matching the unannotated policy from the issue. The genuine path emits the delta with its annotations, spans rebased to the new chunk position, and the existing occurrence reconciliation still applies on the suppression path. Follow-up to the review on microsoft#9339. Signed-off-by: Manohar Paturi <186662190+ManoharPaturi@users.noreply.github.com>
GitHub Actions
Active
|
c33d7e4 takes the consistency option: a duplicate equal to the whole accumulated text is now suppressed only when the text already streamed in more than one chunk, annotated or not, so the exact case you described (a second real token carrying the same citation) streams and produces haha. The genuine path emits the delta with its annotations and the spans rebased to the new chunk position, while the multi-chunk suppression path keeps the occurrence reconciliation for re-sent citations. The two existing one-chunk suppression tests moved to the new expectations, the multi-chunk replay coverage is unchanged, and the identical-annotated-delta regression is included (fails before this change). Full ag-ui suite 1579 passed. |
Motivation & Context
While streaming through the ag-ui endpoint, a text delta is silently dropped when its text equals the whole accumulated text of the open message. Any answer that starts with a doubled digit or a repeated short token hits this. The model streams "2" then "2" for 22 and the client only ever sees 2. Issue #9336 has the repro.
The replay guard from #4194 could not tell a genuine repeated token from a provider replaying the full text, because after a single streamed chunk the two look identical on the wire.
Description & Review Guide
FlowStatenow counts the text deltas emitted for the open message, reset whereveraccumulated_textis reset. The duplicate check in_emit_textonly suppresses the text when the duplicate carries cumulative annotations or the text already streamed in more than one chunk. A plain duplicate after a single chunk now streams as a real delta._emit_textand the reset sites for the new counter.Related Issue
Fixes #9336
AI Assistance
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.