Repository navigation
test(launcher): no hook launch installs the ML set in-band (#747) - #763
Merged
Merged
Conversation
Issue #747: tests_py/hooks/test_entry.py hung in communicate() on the launcher subprocess (test_entry.py:98) until the 300 s pytest timeout. Cause, reproduced on the issue's base 02c3cf0 with a recording uv shim under the test's own hermetic env: scripts/launcher.py ran ensure_all_deps for mcp_server.hooks.session_start, i.e. a synchronous `uv pip install` of the ML set (70 packages, torch and sentence-transformers) inside the hook process, ignoring CORTEX_EMBEDDING_ZERO_DOWNLOAD. ADR-1104 (#749) moved that install to the detached launcher_ml job; on origin/main the same launch requests the base set (45 packages) only. Nothing pinned that: reverting the launcher to the in-band install failed no fast test, only test_entry.py by hanging. This test runs the real launcher.main() for every hook .claude-plugin/plugin.json wires, with the install seams recorded, and requires the base set alone, plus the background job for SessionStart. Against the reverted launcher it fails in 3 s (['base+ml'] == ['base']) instead of hanging for 300 s. Fixes #747 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VH9JM8qvKiJysP16BmAxAU
Owner
Author
|
ZETETIC-REVIEW: APPROVE Le test lance le vrai launcher.main() dans un processus enfant. La liste des hooks vient de plugin.json. Pas de timeout ni de sleep comme verdict. Il échoue si l'install ML revient dans un hook. Le correctif cité, #749 (8a4112f), est bien ancêtre de main. 13 passed en local. Merge après CI verte. |
cdeust
added a commit
that referenced
this pull request
Oct 16, 2026
…le stamps and registries fail loudly (ADR-1108) Merging origin/main brought tests_py/scripts/test_launcher_hook_deps.py (#763), which pins the ADR-1104 contract: every hook launch installs the base set. Under ADR-1108 no hook installs, so on the merged head it failed 13 cases and, because its child stubbed the install seams but not subprocess.Popen, each case walked into launcher_base_job.start and left a real detached `uv pip install` running into pytest's tmp dir (ten were found and stopped). The test now keeps its value, every hook the manifest wires, under the ADR-1108 contract with subprocess.run/Popen recorded: with the set missing a launch installs nothing, starts the one base job and exits as launcher_base_job.refuse decides; with it installed it dispatches the hook; SessionStart hands the ML set to the background job. The manifest check accepts the one Setup command. Catch-and-degrade sites this PR introduced, narrowed: - launcher_base_job.last_failure read an unreadable failure stamp as "no failure", which restarts a job on every hook call; it now names the unreadable stamp as the failure. - launcher_gc.in_use read a user registry it could not list as "no user", which makes a set a live server imports from removable; only a missing registry means no user, any other OSError stops the collection. - launcher_gc.doomed and launcher_deps_fs._sweep_dead: only a missing directory is "nothing there"; an unlistable one raises (GC) or is named on stderr (scratch sweep). PRIVACY.md item 1 said "one-time" and "the first launch": downloads happen once per locked dependency set, from the Setup hook or, when it did not run, the MCP server and one detached job, and ahead of an update when the local marketplace clone holds that version; nothing asks a git remote. ADR-1108 decisions 3 and 4 record the two fail-loud behaviours (mirror regenerated). launcher._ensure_runtime's postcondition named exit 1 for every hook; the gates and SessionStart exit 0 with a notice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BYyqzx4FVMHasVPZV3uBYq
cdeust
added a commit
that referenced
this pull request
Oct 16, 2026
…te call (ADR-1108, ADR-1107) (#759) * fix(hooks): no hook installs dependencies; the SessionStart curation sample stops spilling to disk (ADR-1106, ADR-1107) Dependencies. Every hook ran scripts/launcher.py, and every launch called ensure_deps: an install whenever the stamp missed (new uv.lock, new Python minor, empty deps/). Hook budgets are 5 to 30 s; the uv bootstrap alone is 29 s, the base set 28 to 109 s. A hook the host cancels mid-install leaves deps.tmp-<pid> behind: 64 of them (each holding only uv's .lock) in the owner's plugin data, 2026-10-10 to 2026-10-14. ADR-1104 answered by raising the SessionStart timeout 30 -> 300 s in both manifests. Now: a hook returns from the launcher only when launcher_deps.set_ready says the base set is stamped; otherwise it starts the one detached `launcher.py --install-base-job` (job lock, log beside deps/), prints "Cortex not ready: <cause>" with the log path to stderr and exits 1, having installed and downloaded nothing (no uv bootstrap, no CPython download). Only mcp_server (behind the stand-in), the two detached jobs and an explicit --install-deps install. SessionStart timeout is back to 30 s in the Claude and Codex manifests. Dead installs' deps.tmp-<pid> scratch is reclaimed at the next install. The install-e2e first-session check becomes "ready or names its cause"; the second session must still say "Cortex ready". Curation sample. _CURATION_SAMPLE_SQL sorted every live memory with its content as payload: external merge, 3 x ~128 MB spilled, 55.3 s cold / 13.4 s warm on 68,496 memories. It now picks the 500 ids on three narrow columns and reads only those rows whole: 146 / 157 ms on the same live store (read-only), identical 500 ids in identical order. No index: it would end heap-only updates on the recall path (ADR-1107). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QUZVpwGGs2CxwhiRdqgPTs * fix(install): dependencies install at Setup and live in per-lock sets; the Setup hook, init_plugin.py, pre-install and GC (ADR-1108) Owner decision 2026-10-15: no session starts with Cortex not ready and none waits on an install. Sets live in <data>/sets/<key>, key = uv.lock without the project's own version line (dependency set changed at 8 of 29 releases, not 28 of 37). A Setup hook (claude --init-only) installs base + semantic search + model; scripts/init_plugin.py shows progress and exits non-zero on failure (the host discards Setup output and exits 0). Hooks still never install; the residual exits fast and loud, the PreToolUse gates exit 2, a failed job is not retried per hook call, logs rotate. A detached job installs the next marketplace version's set and removes unreferenced sets. ADR-1106 draft renumbered: ADR-1108 replaces it (1106 belongs to #758). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QUZVpwGGs2CxwhiRdqgPTs * docs(adr): ADR-1104 decision 4 points forward to ADR-1108 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QUZVpwGGs2CxwhiRdqgPTs * fix(install): the legacy deps/ goes only when no old copy imports from it; a set's key covers the install selection and the pinned CPython; the not-ready message names ways out that work from any shell (ADR-1108) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QUZVpwGGs2CxwhiRdqgPTs * wip(install): gates open with a visible notice while Cortex installs; SessionStart and the stand-in announce the install; remote-watching pre-install of the next release's set (ADR-1108) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QUZVpwGGs2CxwhiRdqgPTs * feat(install): remote watcher tests, README and ADR-1108 for the open gates, the announced install and the remote pre-install Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QUZVpwGGs2CxwhiRdqgPTs * fix(ci): the Windows real-install check looks for the bootstrapped uv in <data>/uv-<version>/ (ADR-1108) The deps move into <data>/sets/<key>/ left uv in <data>/uv-<version>/{bin,Scripts}; the check still globbed deps.uv-*/bin and printed []. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QUZVpwGGs2CxwhiRdqgPTs * fix(ci): the [sqlite] parity check resolves the stamped set under <data>/sets/<key>/ itself (ADR-1108) The Windows real-install step passed "$CLAUDE_PLUGIN_DATA/deps", the layout before sets/. verify_sqlite_extra_parity.py now takes the plugin data directory and resolves the set with launcher_layout.project_deps_dir, so no workflow hardcodes where a set lives. Tests cover the set layout and prove a package present only in the legacy flat deps/ does not count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QUZVpwGGs2CxwhiRdqgPTs * fix(install): the maintain job reaches no remote; GC reports what it could not remove; Setup waits on its lock without a clock (ADR-1108) Owner decision 2026-10-16 on the REQUEST_CHANGES verdict of #759: the detached maintain job no longer asks the marketplace's git remote whether a release is out. scripts/launcher_remote.py (git ls-remote, a partial shallow sparse fetch, then the fetched tree's own --setup, and the next-remote.json record that kept its set alive) is removed with its tests; launcher_maintain loses Plan.check_remote, CORTEX_REMOTE_CHECK and the remote branch of run(); launcher_gc._keys loses the record's key. As a side effect a session start with nothing to do no longer starts a job (check_remote made every SessionStart plan non-empty). The local clone pre-install and the GC stay. launcher_gc.collect used rmtree(ignore_errors=True) and returned every doomed path as removed. It now removes path by path, returns only what went, prints each path that stayed with its cause and records them in <data>/gc-failed.json; the maintain job exits 1 then, and Setup says so. A recorded path does not make the next hook start a job (has_new_work): a new unreferenced set or a successful Setup collects again and rewrites the record. launcher_setup waited 1800 s on the Setup lock and recorded "failed" past it, while the holder was alive and installing (the kernel releases the lock when its holder exits). It now waits for as long as the holder runs (launcher_lock.held(path, None)). The --setup refusal test loses its subprocess timeout=60. The dead-scratch sweep names a scratch it cannot remove instead of ignoring the error. Docs: ADR-1108 decisions 2 and 4 and the residual (canonical wiki page, mirror regenerated); README drops the remote watcher sentence; the test_launcher_hooks_never_install.py source points to the real ADR file. Retired-Because: scripts/launcher_remote.py (Clone, clone_of, remote_head, fetch, discard, read_state, record, superseded, referenced_key, prepare) existed only to fetch the next release from the marketplace's git remote; the owner withdrew that network call (2026-10-16). Its two callers, launcher_maintain.run and launcher_gc._keys, are migrated in this commit; the other names the deletion gate matches (mcp_toplist_ranking.fetch, benchmarks.reranker_matrix.cache.prepare, set.discard) are different functions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(install): no hook installs, pinned for every wired hook; unreadable stamps and registries fail loudly (ADR-1108) Merging origin/main brought tests_py/scripts/test_launcher_hook_deps.py (#763), which pins the ADR-1104 contract: every hook launch installs the base set. Under ADR-1108 no hook installs, so on the merged head it failed 13 cases and, because its child stubbed the install seams but not subprocess.Popen, each case walked into launcher_base_job.start and left a real detached `uv pip install` running into pytest's tmp dir (ten were found and stopped). The test now keeps its value, every hook the manifest wires, under the ADR-1108 contract with subprocess.run/Popen recorded: with the set missing a launch installs nothing, starts the one base job and exits as launcher_base_job.refuse decides; with it installed it dispatches the hook; SessionStart hands the ML set to the background job. The manifest check accepts the one Setup command. Catch-and-degrade sites this PR introduced, narrowed: - launcher_base_job.last_failure read an unreadable failure stamp as "no failure", which restarts a job on every hook call; it now names the unreadable stamp as the failure. - launcher_gc.in_use read a user registry it could not list as "no user", which makes a set a live server imports from removable; only a missing registry means no user, any other OSError stops the collection. - launcher_gc.doomed and launcher_deps_fs._sweep_dead: only a missing directory is "nothing there"; an unlistable one raises (GC) or is named on stderr (scratch sweep). PRIVACY.md item 1 said "one-time" and "the first launch": downloads happen once per locked dependency set, from the Setup hook or, when it did not run, the MCP server and one detached job, and ahead of an update when the local marketplace clone holds that version; nothing asks a git remote. ADR-1108 decisions 3 and 4 record the two fail-loud behaviours (mirror regenerated). launcher._ensure_runtime's postcondition named exit 1 for every hook; the gates and SessionStart exit 0 with a notice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BYyqzx4FVMHasVPZV3uBYq * fix(install): a GC failure after Setup's ready is logged and recorded; every narrowed catch is pinned (ADR-1108) Round-2 review of #759 (REQUEST_CHANGES at c055f22): B1. Three narrowings had no test (mutants survived the launcher suite): launcher_gc.doomed reading an unlistable sets/ as empty, _sweep_dead dropping its listdir message, and _sweep_dead back to rmtree(ignore_errors=True). New tests: an unlistable sets/ makes collect and has_new_work raise PermissionError; an unlistable scratch parent and a scratch that cannot be removed are each named on stderr with the cause. B2. launcher_setup collected after recording ready, outside any guard: an OSError from collect escaped after run() restored the streams, so setup.log ended "ready in 0 s", nothing recorded the failure and init_plugin.py said "Cortex is ready". _collect now catches OSError while the output still goes to setup.log, prints it and records it as gc_error in <key>.setup.json (not gc-failed.json, whose write can be what failed); init_plugin.py prints it on stderr and still exits 0 because the set is ready. Any other exception leaves its traceback in setup.log before it propagates. B3. CHANGELOG [Unreleased] / Fixed names the change: no hook installs, claude --init-only after install and update, ADR-1104 decision 4 replaced, no remote check, the 64 orphaned scratch directories. Also: launcher_installed._read names a host record that exists but cannot be read on stderr (a missing one stays silent "no work"); plan() and the launcher_maintain docstring say the hook removes dead users' markers; PRIVACY item 1 counts the downloading jobs exactly (two hook-started jobs, plus the maintain job that runs the clone's Setup). ADR-1108: new host fact 8, measured on Claude Code 2.1.301: `claude plugin marketplace update` refreshes the clone without applying the update, so decision 4 is live after it, and the RESIDUAL no longer says the clone never holds the next uv.lock (mirror regenerated). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BYyqzx4FVMHasVPZV3uBYq * fix(install): a GC failure after Setup's ready names the path that failed (ADR-1108) Round 3 of #759, B4: the gc_error cause used repr(exc); the repr of an OSError drops its filename (PermissionError(13, 'Permission denied')), so gc_error, setup.log and init_plugin's stderr did not say which path failed. The cause is now "<Type>: <str(exc)>", which keeps "[Errno 13] Permission denied: '<data>/sets'". The test asserts that <data>/sets is in gc_error and in setup.log: red with the repr code, green now. ADR-1108 decision 4 and launcher_maintain.run's docstring now say which branch exits 1 on a path that cannot be removed: the job's own collection does; on the clone's Setup the job returns Setup's exit code, 0, with the cause in setup.log and gc-failed.json. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BYyqzx4FVMHasVPZV3uBYq --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #747:
tests_py/hooks/test_entry.py::test_entry_matches_launcher_on_benign_eventhung incommunicate()on the launcher subprocess (test_entry.py:98) until the 300 s pytest timeout, which aborted the full suite at about 61 %.Root cause (reproduced, not inferred)
At the issue's base
02c3cf00,scripts/launcher.pyranlauncher_deps.ensure_all_deps(deps_dir)whenever the module wasmcp_server.hooks.session_start. That is a synchronousuv pip installof the ML set (torch, sentence-transformers, flashrank, about 1 GB) inside the hook process. It ignoredCORTEX_EMBEDDING_ZERO_DOWNLOAD, which the test's hermetic env sets. When the deps directory's ML stamp did not name the currentuv.lock(the main checkout'sdeps/.cortex-deps-stamp-ml.jsonis still in the pre-ADR-1104 format:{"python", "pins"}, nolock/setkey), the SessionStart launch reinstalled the whole set before it ran the hook, and the test waited on that.It was not stdin, a lock or a detached grandchild holding the pipe. Every
Popenspawn site undermcp_server/hooksandscripts/launcher_ml.pypassesstdin=DEVNULLand sends stdout and stderr to a log orDEVNULL. Line 90 (mcp_server.hooks.entry) installs no dependencies (module docstring), so the launcher subprocess at line 98 was the only one that could block.I reproduced it by running
scripts/launcher.py mcp_server.hooks.session_startunder the test's ownhermetic_hook_env, with an emptyCLAUDE_PLUGIN_DATAand auvshim first onPATH. The shim passes--version/exportthrough to the real uv 0.11.3, records everypip installalong with the ML packages its requirement file names, and refuses the install:ADR-1104 (#749,
8a4112f2) already removed the in-band install: the ML set now goes to one detachedlauncher_mljob, and that job is gated by the zero-download opt-out. #749 did not close #747, and nothing pinned the fix. If someone put the in-band install back, no fast test would fail. Onlytest_entry.pywould catch it, by hanging for 300 s again.This change
tests_py/scripts/test_launcher_hook_deps.pyruns the reallauncher.main()in a child process for every hook that.claude-plugin/plugin.jsonwires. The module names are parsed from the manifest's hook commands, and every command must name one. The install seams (ensure_deps,ensure_all_deps,launcher_ml.start) andrunpy.run_moduleare replaced by recorders. The test requires:launcher_ml.startis called with the plugin-datadepsdir).The test has no
timeout=and no sleep.Evidence
The new test fails against the old behaviour. I put back the
02c3cf00branch (if install_deps or module == "mcp_server.hooks.session_start": ensure_all_deps) inscripts/launcher.py:With the launcher unchanged:
13 passed.Checks on the final head
66ce3a4e(rebased on7e4864d2):pytest tests_py/hooks/test_entry.py:19 passed in 14.80s, exit 0pytest tests_py/hooks/:882 passed, 2 skipped, 13 subtests passed in 133.52s, exit 0uvx ruff@0.16.6 check/format --check, andruff@0.16.10(the CI lint pin): all passed, 3649 files already formattedpython scripts/check_craftsmanship.py(with and without--base origin/main):Craftsmanship gate: OKpython scripts/check_no_deps_invariant.py: cleanI did not run the full suite locally (resources). CI runs it.
Not in this PR (adjacent)
test_entry.pystill runs the launcher againstCLAUDE_PLUGIN_ROOT/deps(the checkout'sdeps/), becauseCLAUDE_PLUGIN_DATAis unset. On a fresh checkout, or after auv.lockchange, the first launcher call does a networkeduv pip installof the base set (45 packages) inside the test. That is bounded and no longer a hang, but the test depends on the network and writes into the checkout.deps/.cortex-deps-stamp-ml.jsonis still in the pre-ADR-1104 format. This is harmless now (set_ready("ml")is False, so the background job is the one that reinstalls).🤖 Generated with Claude Code
https://claude.ai/code/session_01VH9JM8qvKiJysP16BmAxAU