Skip to content

refactor(server): project commands commit through EventSinkV2 - #13960

Merged
juliusmarminge merged 4 commits into
t3code/codex-turn-mappingfrom
v2/projects-in-v2
Sep 28, 2026
Merged

juliusmarminge merged 4 commits into
t3code/codex-turn-mappingfrom
v2/projects-in-v2

Conversation

@juliusmarminge

@juliusmarminge juliusmarminge commented Sep 27, 2026 •

Copy link
Copy Markdown
Member

Project commands were the last thing still running through the V1 OrchestrationEngine. That engine bootstraps the V1 projection pipeline at startup, which caused two real bugs: the first V2 boot wiped four project settings (Bug 1), and projects whose imported threads had been deleted in V2 could not be removed without force (Bug 2).

This is PR 1 of the "projects in V2, V1 deleted" stack against t3code/codex-turn-mapping.

What changed

  • orchestration-v2/ProjectCommands.ts: a pure planProjectCommand ported from V1 decider.ts:225-375. It keeps the V1 behavior: create forces defaultModelSelection: null, monograms are limited to 2 graphemes, new script ids must pass the script.<id>.run check, existence checks apply, and an active workspace root must be unique by exact string. project.delete emits only project.deleted; ProjectService already runs the thread cascade in V2. Rejections are typed as ProjectCommandInvariantError or ProjectWorkspaceConflictError.
  • orchestration-v2/ProjectStore.ts: a new ProjectStoreV2 service (Context.Service with an inline interface, constructor make). It runs SQL over projection_projects and exposes apply(event), get(id, {includeDeleted}), list({projectIds, includeDeleted}), findActiveByWorkspaceRoot(root), and unenriched getShell/listShells. Swapping the old ProjectionProjectRepository readers onto it is PR 2.
  • EventSinkV2.commitProjectCommand / commitRejectedProjectCommand: these use the same transaction shape as commitCommand. The receipt is inserted if absent, then appendProjectEvent, then ProjectStoreV2.apply, then the receipt is upserted; publishCommitted runs after commit. A reused command id commits nothing and returns its existing receipt. CommandReceiptStoreV2 now stores and reads project receipts (getProjectByCommandId).
  • OrchestrationEventStore.appendProjectEvent: writes the unchanged ApplicationProjectEvent shape (aggregate_kind='project') into orchestration_events. Project events are not added to OrchestrationV2DomainEvent.
  • ProjectService: normalizes the root, then runs lock → read → plan → commit. Commands are serialized per project, and a command that claims a workspace root also holds that root, so two projects cannot both claim it between read and commit. It also owns the delete cascade (unchanged) and enrichment. It contains no transaction, receipt, append or projection code. Client RPCs and errors are unchanged: ProjectOperationError('dispatch-project-command'), ProjectConflictError and ProjectNotEmptyError.
  • Deleted: the V1 OrchestrationEngine (Layers and Services), OrchestrationLayerLive and its uses in orchestration-v2/runtimeLayer.ts, the three t3_orchestration_command* metrics, and their section in docs/operations/observability.md. The trace jq example now uses the orchestration_v2.* span attributes that are actually emitted. The V1 pipeline and snapshot query remain built but inert; nothing bootstraps the pipeline any more. They are removed in PRs 3 and 4.

Bug 1: the first V2 boot reset four project settings

Migration 055's baseline project.created (ApplicationEventSource.ts:147-158) omits defaultThreadEnvMode, autoPull, faviconPath and projectIcon. It is appended after the V1 projects cursor, so on the first V2 boot ProjectionPipeline (:562-577) replays it and resets those columns to null/false.

  • Real data: I booted the base head on a VACUUM copy of a V1 DB from a local worktree whose project had default_thread_env_mode = 'worktree'. After the boot the row read null. The value survives only in that project's pre-baseline project.meta-updated event.
  • Test: the new ProjectSettingsUpgrade.integration.test.ts migrates to 54, seeds a project with auto-pull, env mode, favicon and icon, then boots the V2 runtime (which runs 055 and 056) on the same file. On the base head it fails with all four columns reset (auto_pull 1→0, default_thread_env_mode 'worktree'→null, favicon_path→null, project_icon_json→null). On this branch the row is unchanged.

Bug 2: non-force delete failed after the imported threads were deleted in V2

V1's project.delete "not empty" check read V1 projection_threads, which V2 never writes. The new deletion test imports a V1 thread, deletes it through V2 (planThreadDeletion + commitCommand, as thread.delete does), and then removes the project without force. On base this fails with Project '…' is not empty and cannot be deleted without force=true. On this branch it succeeds; the only emptiness check is the V2 shell check in ProjectService.

No repair migration

Only installs that already booted V2 (the Sept 21-25 preview builds and V2 dev servers, a handful of testers) had these four settings reset. Everyone upgrading from main or nightly (at migration 053/054) never ran 055, and with this PR nothing replays the 055 baseline, so their settings survive. Per the maintainer, the few affected testers can re-set the settings by hand rather than adding a one-off repair migration. V2 stays at migrations 055 and 056.

Heap

I booted the base and the branch on separate VACUUM copies of the live V1 DB (214 threads, 11,802 messages, 404k events). Each ran until the legacy import settled (214 threads, 8,307 messages), then I took three samples of forced-GC heapUsed:

heapUsed (3 samples)
base a3fbbe4531 193.3 / 189.0 / 189.0 MB
this branch 191.6 / 193.8 / 191.8 MB

The difference is within sample noise. Both copies were deleted afterwards.

Verification

  • vp exec tsc --noEmit -p . in apps/server, packages/contracts, packages/client-runtime, apps/web and apps/mobile: 0 error TS, 0 warning TS.
  • vp run knip:check: clean. This PR also removes isOrchestrationCommandRejection, which was only used by the engine.
  • vp lint on the touched files: clean, except 7 no-unused-vars warnings in observability/Metrics.ts that exist on base as well.
  • vp test run on 73 files: src/project/, src/persistence/, src/cli/, src/mcp/toolkits/project/, src/orchestration/Layers/, ProjectCommands, ProjectSettingsUpgrade, LegacyV1Cutover, LegacyV1ThreadImporter, runtimeLayer, DelegatedCompletionDelivery, FoundationPersistence, V1ImportBoundary, OrchestratorReplayFixtures, ProjectionMaintenance*, ws, server and serverRuntimeStartup. Result: 498/498 passed.
  • New tests, all on real SQLite:
    • ProjectService.test.ts covers same-command-id retry, rejected-receipt replay, workspace conflict, event + row + receipt atomicity (via a trigger-injected row failure), and a gated race between two projects claiming one root. With the workspace lock removed, the race test fails.
    • ProjectCommands.test.ts ports the project cases from decider.projectScripts and decider.projectThreadEnvMode.
    • Also: the upgrade test (fails on base), the Bug 2 deletion test (fails on base).
  • One existing test needed a change. runtimeLayer.test.ts > usage-limit recovery > manually resumes… (added in fix(server): preserve queued messages after usage limits #13877) writes an interrupted run next to a queued one without holding the queue. A real stop sends thread.turn.interrupt with holdQueue. Whether the terminal-run worker promotes the queued run before the test's resume depends on fiber scheduling. On base, a single added Effect.yieldNow makes both cases fail, and dropping the V1 engine layer changes that timing the same way. The test now writes the queue hold that a real stop produces, and passes on both base and the branch.
  • Not run: repo-wide checks and live provider runs (no provider behavior changed).

Left for PRs 2-6

  1. Move project reads and replay stats off ProjectionSnapshotQuery, and swap ProjectionProjectRepository for ProjectStoreV2.
  2. Move thread search and the git project lookup to V2 (Bug 3: GitManager.ts:723), then delete ProjectionSnapshotQuery and ProjectionPipeline.
  3. Delete the V1 decider, projector and V1-only repositories.
  4. Quarantine the V1 importer under orchestration-v2/legacy/.
  5. Retire the legacy orchestration schemas, keeping the wire format identical.

Model: Claude Opus 5.5 (Claude Code)

🤖 Generated with Claude Code


Devin Review

Project create, update and delete no longer go through the V1
OrchestrationEngine. ProjectService plans each command with the pure
planProjectCommand (ported from the V1 decider) under a per-project lock
and commits it through EventSinkV2, which appends the project event,
folds it into projection_projects through the new ProjectStoreV2 and
records the receipt in one transaction.

This removes the last live caller of the V1 engine, so its layer, its
three command metrics and their docs go too. With nothing bootstrapping
the V1 projection pipeline, the first V2 boot no longer resets
auto-pull, env mode, favicon and icon from migration 055's baseline
event, and a project whose imported threads were deleted in V2 can be
removed without force. Migration 057 restores those four settings on
installs that already booted V2, from the project's pre-baseline events.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL 1,000+ changed lines (additions + deletions). labels Sep 27, 2026
Comment thread apps/server/src/orchestration-v2/ProjectCommands.ts
Comment thread apps/server/src/orchestration-v2/runtimeLayer.ts Outdated
Comment thread apps/server/src/orchestration-v2/ProjectStore.ts Outdated
Comment thread apps/server/src/project/ProjectService.ts Outdated
Comment thread apps/server/src/project/ProjectService.ts Outdated
Comment thread apps/server/src/project/ProjectService.ts Outdated
new ProjectOperationError({ operation: "dispatch-project-command", projectId, cause });
const workspaceRoot = command.type === "project.delete" ? undefined : command.workspaceRoot;
const planAndCommit = Effect.gen(function* () {
const project = Option.getOrUndefined(yield* readRow(projectId, { includeDeleted: true }));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium project/ProjectService.ts:211

commit accepts project.meta.update and project.delete against a soft-deleted row, so a concurrent update or second delete reports success and appends another event instead of returning ProjectNotFoundError. Because this read uses includeDeleted: true after acquiring the project lock, the planner must treat deleted projects as absent for these command types (or reject them before planning).

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/server/src/project/ProjectService.ts around line 211:

`commit` accepts `project.meta.update` and `project.delete` against a soft-deleted row, so a concurrent update or second delete reports success and appends another event instead of returning `ProjectNotFoundError`. Because this read uses `includeDeleted: true` after acquiring the project lock, the planner must treat deleted projects as absent for these command types (or reject them before planning).

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ No successful main baseline artifact is available yet. This run establishes the initial measurement.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire — 4.9 KiB — 6.8 KiB ✅
Codex Thread snapshot wire — 3.7 KiB — 4.9 KiB ✅
Codex Live turn WebSocket wire — 1.1 KiB — 2.0 KiB ✅
Codex Live turn WebSocket decoded — 20.4 KiB — 29.3 KiB ✅
Codex Live turn messages — 1 — 8 ✅
Claude Total thread wire — 4.9 KiB — 6.8 KiB ✅
Claude Thread snapshot wire — 3.7 KiB — 4.9 KiB ✅
Claude Live turn WebSocket wire — 1.2 KiB — 2.0 KiB ✅
Claude Live turn WebSocket decoded — 20.8 KiB — 29.3 KiB ✅
Claude Live turn messages — 2 — 8 ✅

Baseline: unavailable · PR result: db65fe9 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 106.1 KiB
  • Claude decoded thread snapshot: 106.4 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@macroscopeapp

macroscopeapp Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR replaces the production project-command pipeline with a new event-sink, receipt, locking, and projection flow while retiring the prior orchestration engine and metrics. The broad runtime and startup behavior changes, including project deletion and persistence semantics, warrant human review.

Not approved because:

  • 1 blocking correctness issue found at or above your repo's Minimum Blocking Severity

Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more.

juliusmarminge and others added 2 commits September 27, 2026 14:08
Only V2 testers who already booted V2 lost those settings, and a handful of
testers can set them again. The upgrade path for everyone else is fixed by
no longer replaying the migration 055 baseline through the V1 pipeline.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A rejected project receipt now stores its typed rejection, so retrying a
command id after a workspace conflict returns ProjectConflictError even once
the root is free. A retried project.delete reaches its receipt instead of
failing as not found, and updates or new deletes against a deleted project
are rejected as not found instead of appending events.

Script-ID rejections no longer persist the raw, unbounded ID. ProjectStore
exports make, and its consumers import it as a namespace.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comment thread apps/server/src/orchestration-v2/ProjectStore.ts Outdated
Every ProjectStoreV2Error is built by mapError around a failed SQL effect,
so the cause is always present. Make it required so a future construction
site cannot drop it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@juliusmarminge
juliusmarminge merged commit 6e0c38e into t3code/codex-turn-mapping Sep 28, 2026
24 checks passed
@juliusmarminge
juliusmarminge deleted the v2/projects-in-v2 branch September 28, 2026 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL 1,000+ changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant