Skip to content

perf(flow-chat): reuse turn indexes and summaries during streaming - #3851

Merged
kev1n77 merged 3 commits into
openBitFun:mainfrom
kev1n77:fmy/perf
Nov 9, 2026
Merged

kev1n77 merged 3 commits into
openBitFun:mainfrom
kev1n77:fmy/perf

Conversation

@kev1n77

@kev1n77 kev1n77 commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

Share batched Turn ordinal lookups and reuse Turn summaries across streamed body updates. The reader-owned cache retains scalar identity metadata and invalidates on Turn, catalog, partial-tail, or rendered-history structure changes.

Type and Areas

Type: Performance bug fix.

Areas: Web UI / FlowChat main and BTW conversations.

Motivation / Impact

Long resident conversations rebuilt full ordinal maps for every Turn in two complete summaries on each streamed text update. This quadratic work blocked the main WebView thread and made both the main and side conversation difficult to scroll. The summaries now share a batched lookup and keep stable array identities during body-only updates. Legacy catalog storage fallbacks also use indexed lookups.

Verification

After merging upstream/main at d3ee3e5b1, the focused FlowChat suites passed 122 tests across 9 files. Frontend type checking, focused ESLint, capability generation checks, and diff whitespace checks also passed.

Commands
pnpm run type-check:web
pnpm --dir src/web-ui exec vitest run src/flow_chat/components/modern/flowChatTurnSummaries.test.ts src/flow_chat/utils/flowChatTurnIdentity.test.ts src/flow_chat/utils/flowChatTurnOrdinal.test.ts src/flow_chat/components/modern/ModernFlowChatContainer.history-state.test.tsx src/flow_chat/components/modern/ModernFlowChatContainer.history-restore.test.tsx src/flow_chat/components/modern/FlowChatTurnRail.test.tsx src/flow_chat/components/modern/useFlowChatFollowOutput.test.tsx src/flow_chat/components/btw/useBtwSessionState.test.tsx src/flow_chat/components/btw/useBtwPanelViewport.test.tsx
pnpm --dir src/web-ui exec eslint src/flow_chat/components/modern/flowChatTurnSummaries.ts src/flow_chat/components/modern/ModernFlowChatContainer.tsx src/flow_chat/utils/flowChatTurnIdentity.ts
pnpm run capabilities:check
git diff --check upstream/main...HEAD

Local native Windows Debug / Vite testing used the same synthetic 1200-Turn main + 400-Turn BTW history, native backend binary, viewport, and loopback SSE provider before and after the optimization. In the first matched unprofiled dual-stream samples:

Main-thread metric Before After
Main / BTW RAF interval median 260.5 / 266.6 ms 6.1 / 6.1 ms
Main / BTW RAF interval p99 321.3 / 363.7 ms 60.4 / 42.4 ms
Main / BTW wheel dispatch round trip median 502.7 / 504.8 ms 21.1 / 18.7 ms

All three optimized dual-stream repetitions showed the same improvement. 21 native state checks passed, covering reading anchors during generation, returning while generating or after completion, BTW collapse/return, separate drafts, and Monaco model/text/caret/selection/scroll/undo/redo/save retention. Static scrolling, 24 pane switches, and restart/reload comparisons were also exercised.

Reviewer Notes

Catalog ordinal precedence, sparse and deprecated storage indexes, provisional usage reports, partial-tail offsets, catalog-only history, and explicit history-window offsets retain their existing semantics. Scroll ownership, reading-anchor logic, editor lifecycle, and persisted shapes are unchanged.

Native measurements preceded the latest upstream/main merge; the merged tree was rechecked with the automated commands above. History was synthetic, with BTW history seeded into the renderer; live output used the native runtime/event chain with a local model fixture. Release builds and live remote scenarios were not tested. Occasional static long frames remain outside this change's scope.

Checklist

  • This PR is focused and does not include secrets, temporary prompts, generated scratch files, or unrelated artifacts.
  • Relevant verification is recorded above, or skipped checks are explained.
  • User-facing strings, docs, and locales are updated where applicable.

@kev1n77
kev1n77 merged commit 2c29b5f into openBitFun:main Nov 9, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant