Skip to content

fix(orchestrator/grok): Prevent spurious wake run after in-turn monitors#4218

Open
mwolson wants to merge 2 commits into
pingdotgg:t3code/codex-turn-mappingfrom
mwolson:fix/grok-v2.1
Open

fix(orchestrator/grok): Prevent spurious wake run after in-turn monitors#4218
mwolson wants to merge 2 commits into
pingdotgg:t3code/codex-turn-mappingfrom
mwolson:fix/grok-v2.1

Conversation

@mwolson

@mwolson mwolson commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Base

This branch is stacked on #4193 (fix/ctm-post-merge-ci) so CI's Test job
runs against aligned post-#3578 fixtures. The first commit
(test(orchestrator): Align post-merge CTM fixtures) belongs to #4193;
please review only the second commit here. Once #4193 lands in
codex-turn-mapping, this branch rebases back to a single commit above the
stack base.

Summary

  • Fixes a multiturn bug where the Grok adapter queued a spurious synthetic
    "Background task completed." continuation run while the original turn was
    still streaming, after the agent had already reported the monitor result
    in-turn.
  • Ensures injected <monitor-event> acknowledgement chatter for work already
    reported in-turn is never retained as wake evidence for later turns.
  • Adds four adapter tests covering the multiturn repro, mid-turn completion
    deferral, the legitimate unhandled-completion wake, and the
    settle-without-report hold interaction.

Problem and Fix

Problem and Why it Happened Fix
A x.ai/task_completed ext notification landing mid-turn reached applyLateBackgroundMutation, which had no notion of an active un-finalized root turn. With the running set empty and stale frames in wakeBuffer, it called offerContinuationRun and dispatched a "Background task completed." run (queue_after_active) while the original turn was still streaming its own report. Continuation offers are never made while a root turn is active and un-finalized. Completions that register mid-turn but are not handled in-turn are recorded in a new midTurnUnreportedCompletedTaskIds set, and finalizeTurn offers exactly one continuation after finalize when legitimate evidence remains (settled status completed, no running tasks). The arming also requires the prompt not to have settled, so the settled-held window still uses the pre-existing injected-report path.
After a turn that reported its monitor in-turn, the CLI's injected <monitor-event> turn hit the late-mutation path, which deleted the task id from the in-turn handled set. The subsequent injected ack chatter then failed the handled-chatter guard and was buffered as wake evidence, surviving into later turns (the buffer was intentionally not cleared at turn start). Late monitor-event mutations no longer erase in-turn handled marks, the handled-chatter guard runs before wake buffering, and stale wakeBuffer frames are cleared at non-continuation user-turn start. Frames cleared there cannot be legitimate: a genuinely unhandled completion is tracked by the mid-turn set and offered at finalize instead.

Defensive Fixes

Problem and Why it Happened Fix
In the late-mutation path, suppressPostSettleMonitorPrompt was set true and then unconditionally set false two lines later for terminal mutations on already-ended tasks, making the intent unreadable and the true branch dead. The suppress logic is rewritten as mutually exclusive running/terminal branches.
A stopped run's quarantine cleared the wake buffer and running set but would have left mid-turn unreported completions armed. quarantineStoppedRun also clears midTurnUnreportedCompletedTaskIds.
A completion armed pre-settle into midTurnUnreportedCompletedTaskIds survived the settle-hold: when the CLI's injected report streamed into the held turn, only pendingInjectedReport was cleared, so finalizeTurn still saw the armed id and offered a duplicate continuation right after the report projected (Bugbot round 1). When the injected report chunk streams, the delivered task ids are also removed from midTurnUnreportedCompletedTaskIds. If no report ever streams, the id survives and finalize offers exactly one wake, as before.
A midTurn-only continuation offer is made with an empty wakeBuffer (the buffer cannot grow while the root turn is active), so if the CLI's late frames had not arrived by the wake run's start, the empty drain finalized a blank continuation run immediately (Bugbot round 1). An empty-drain continuation now uses the same 3s deferred-finalize quiet window as a non-empty drain (when the flavor defers finalize for background work), letting late frames attach; flavors without the defer flag keep the immediate finalize so the turn cannot wedge.
finalizeTurn cleared midTurnUnreportedCompletedTaskIds unconditionally, but the offer gate requires the running set to be empty. A task armed pre-settle lost its mark when the turn finalized while a second task was still running, and the second task's bare end-notice frame is excluded from wake evidence, so no continuation was ever offered for either completion (Bugbot round 2). The marks are kept across finalize when the turn settled completed and background work is still running, so the post-finalize gate offers exactly once when the last task ends. Interrupted and failed turns still clear unconditionally, and non-continuation turn start clears too, so kept marks cannot wake after an interrupt or arm a later user turn.

Validation

  • vp check: pass (0 errors, 63 pre-existing warnings)
  • vp run typecheck: pass (all packages)
  • Focused suites (AcpAdapterV2.test.ts, GrokAdapterV2.test.ts): 98/98,
    including 9 new tests
  • Live desktop retest on a rebuilt AppImage containing this commit: the
    original two-turn repro produced no spurious wake and no replayed acks;
    legitimate post-settle monitor and detached-command scenarios each produced
    exactly one continuation; interrupt/steer/queue scenarios stayed clean

Note

Medium Risk
Changes orchestration turn finalization, continuation offers, and wake buffering in AcpAdapterV2—high interaction complexity with edge cases, though heavily tested. Minor test/fixture path updates are low risk.

Overview
Fixes Grok/ACP post-settle continuation so synthetic Background task completed. runs do not fire while a root turn is still active or after monitors were already reported in-turn.

AcpAdapterV2 adds midTurnUnreportedCompletedTaskIds to defer wake offers for unhandled mid-turn completions until finalizeTurn, blocks offerContinuationRun during an active un-finalized turn, and stops late monitor mutations from clearing in-turn handled marks. In-turn-handled agent/thought chatter is filtered before wakeBuffer retention; non-continuation user turns clear stale buffer residue. Injected report streaming clears matching mid-turn arms; empty-drain continuation turns use the deferred-finalize quiet window when enabled.

Adds extensive AcpAdapterV2.test.ts coverage for settle-hold, staggered monitors, multiturn ack chatter, and empty-drain continuations.

Also updates CLI config tests to expect userdata-v2 state paths and refreshes Claude replay fixtures (query.open tool lists) for post-merge alignment.

Reviewed by Cursor Bugbot for commit a62ed92. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Fix spurious wake runs after in-turn monitor completions in AcpAdapterV2

  • Introduces midTurnUnreportedCompletedTaskIds to track background tasks that reach a terminal state mid-turn without being reported in-turn, deferring wake offers until after turn finalization.
  • Fixes late terminal mutations from incorrectly unmarking in-turn handled tasks, which previously triggered duplicate Background task completed. continuations.
  • Filters agent/thought chatter frames from dirtying wakeBuffer, preventing spurious post-settle wake runs for already-handled background work.
  • Adds a quiet-window/deferred-finalize path for continuation turns with an empty wake buffer, allowing late frames to attach before finalizing.
  • Clears midTurnUnreportedCompletedTaskIds on interrupted/quarantined turns and when injected reports stream, preventing stale deferred wakes.

Macroscope summarized a62ed92.

@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 6ce7b571-743d-4ea8-b90a-69b8cfe29fb2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Jul 21, 2026
@mwolson
mwolson marked this pull request as ready for review July 21, 2026 01:51
Comment thread apps/server/src/orchestration-v2/Adapters/AcpAdapterV2.ts
Comment thread apps/server/src/orchestration-v2/Adapters/AcpAdapterV2.ts
Comment thread apps/server/src/orchestration-v2/Adapters/AcpAdapterV2.ts
@macroscopeapp

macroscopeapp Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces new state tracking (midTurnUnreportedCompletedTaskIds) and modifies multiple code paths in the ACP adapter that control when continuation runs are offered after background task completion. While framed as a bug fix with extensive tests, changes to orchestration logic that gates whether follow-up work happens warrant human review to verify the intended behavior.

You can customize Macroscope's approvability policy. Learn more.

Comment thread apps/server/src/orchestration-v2/Adapters/AcpAdapterV2.ts
Comment thread apps/server/src/orchestration-v2/Adapters/AcpAdapterV2.ts Outdated
Comment thread apps/server/src/orchestration-v2/Adapters/AcpAdapterV2.ts
Comment thread apps/server/src/orchestration-v2/Adapters/AcpAdapterV2.ts
yield* finalizeTurn(context, "completed", undefined, {
drainTrailingChunks: true,
});
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty-drain continuation can wedge

Medium Severity

When an empty-drain midTurn continuation starts with carryover live subagents, hasDeferredBackgroundWork is true so the new branch calls rearmDeferredFinalize, which no-ops under that same condition. Nothing later rearms from child-session subagent completion, so the continuation never finalizes. Previously empty drain finalized immediately, so this wedge is new.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit cd0348b. Configure here.

mwolson added 2 commits July 22, 2026 08:59
- Defer continuation offers for background completions that land while a
  root turn is active and un-finalized; finalize offers exactly one wake
  only when unhandled completed work remains
- Stop late monitor-event mutations from erasing in-turn handled marks, so
  injected-turn ack chatter is never retained as wake evidence
- Clear stale wake buffer frames at non-continuation user-turn start
- Make the late-mutation suppress logic's running/terminal branches
  mutually exclusive

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit a62ed92. Configure here.

(yield* Ref.get(runningBackgroundTaskIds)).size === 0
) {
yield* Ref.set(midTurnUnreportedCompletedTaskIds, new Set());
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mid-turn wake evidence cleared too early

Medium Severity

After a midTurn-only offer, finalizeTurn clears midTurnUnreportedCompletedTaskIds whenever the running set is empty, including right after offerContinuationRun. That path often has an empty wakeBuffer, so if continuation dispatch later fails and clears continuationRequested, no residual wake evidence remains to re-offer. A genuinely unhandled mid-turn completion can then disappear with hasPendingBackgroundWork already false.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit a62ed92. Configure here.

@mwolson mwolson changed the title fix(grok): Prevent spurious wake run after in-turn monitors fix(orchestrator/grok): Prevent spurious wake run after in-turn monitors Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L 100-499 changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant