fix(run): make type = "file" scripts executable via poetry run - #11092
akashchamp wants to merge 3 commits into
Conversation
`RunCommand.run_script()` unconditionally assumed that any dict-shaped script entry from `[tool.poetry.scripts]` was a console entry point and read `script["callable"]`, which doesn't exist for scripts declared with `type = "file"` (only `reference` and `type` are present there). Once such a file script was installed into the venv's `bin/` directory (as of the fix for python-poetry#10664), running it via `poetry run <name>` crashed with `KeyError: 'callable'` instead of executing the file. Run file-type scripts directly instead of trying to import them as `module:callable`. When the script has already been installed, this simply executes the resolved path found in `env.script_dirs`. When it hasn't been installed yet, fall back to running the script at its `reference` path relative to the project directory, mirroring how console scripts still work (with a warning) before `poetry install` has been run. Fixes python-poetry#11090
There was a problem hiding this comment.
Hey - I've found 1 issue
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="src/poetry/console/commands/run.py" line_range="88-89" />
<code_context>
else:
# If we reach this point, the script is not installed
self._warning_not_installed_script(args[0])
+ if is_file_script:
+ # File scripts have no importable ``module:callable`` fallback, so
+ # fall back to running the referenced file directly.
+ assert isinstance(script, dict)
+ reference = script.get("reference")
+ if reference:
+ args = [
+ str(self.poetry.file.path.parent / reference),
+ *args[1:],
+ ]
+
+ if is_file_script:
+ return self.env.execute(*args)
if isinstance(script, dict):
</code_context>
<issue_to_address>
**issue (bug_risk):** On Windows, file scripts are passed as an absolute path to `Env.execute()`, whose `_bin()` resolution appends `.exe` to any non-`.exe` command. The installed lookup also changes the name to `.cmd`, so the resulting command becomes `<script>.cmd.exe` and the file script fails to execute.
**Triggers:** When `poetry run` executes a `type = "file"` script on Windows.
**Suggested fix:** Execute the resolved file path without applying environment binary-name normalization, or update `Env.execute()` to preserve absolute paths and avoid adding `.exe` to `.cmd` or other script paths.
</issue_to_address>Sourcery assessment
Approval pending. 1 finding to address first.
Blocking findings: src/poetry/console/commands/run.py:89
… on Windows `RunCommand.run_script()`'s installed-script lookup unconditionally appended `.cmd` to the candidate path on Windows before checking whether it exists. Console entry point scripts do get a `.cmd` wrapper on Windows (see `EditableBuilder`), but `type = "file"` scripts are copied to the venv's script directory verbatim, with no suffix. As a result the lookup for a file script never found the actually-installed file, incorrectly reported it as "not installed", and fell back to re-running the (possibly stale) source file instead -- which, once handed to `Env.execute()`, failed on Windows because the resolved path has no recognized extension. Skip the `.cmd` normalization for file scripts so the lookup finds the real installed path. Also make `Env._bin()` a no-op when it is given an absolute path to a file that already exists, so a fully-resolved script path is never mangled by the bare-command-name normalization (e.g. by appending `.exe`) -- this is exercised for the first time now that `run_script()` can pass such a path to `Env.execute()`. Addresses a review comment from sourcery-ai on python-poetry#11092 pointing out this exact corruption, and is very likely the root cause of that PR's failing Windows pytest jobs (`tests/console/commands/test_run.py::test_run_file_script`). Added a regression test that forces the Windows lookup path on Linux CI (by patching `WINDOWS`) and asserts the installed script is found without the bogus "not installed" warning, plus a unit test on `Env._bin()` / `get_command_from_bin()` covering the absolute-path case directly.
… on Windows `RunCommand.run_script()`'s installed-script lookup unconditionally appended `.cmd` to the candidate path on Windows before checking whether it exists. Console entry point scripts do get a `.cmd` wrapper on Windows (see `EditableBuilder`), but `type = "file"` scripts are copied to the venv's script directory verbatim, with no suffix. As a result the lookup for a file script never found the actually-installed file, incorrectly reported it as "not installed", and fell back to re-running the (possibly stale) source file instead -- which, once handed to `Env.execute()`, failed on Windows because the resolved path has no recognized extension. Skip the `.cmd` normalization for file scripts so the lookup finds the real installed path. Also make `Env._bin()` a no-op when it is given an absolute path to a file that already exists, so a fully-resolved script path is never mangled by the bare-command-name normalization (e.g. by appending `.exe`) -- this is exercised for the first time now that `run_script()` can pass such a path to `Env.execute()`. Addresses a review comment from sourcery-ai on python-poetry#11092 pointing out this exact corruption, and is very likely the root cause of that PR's failing Windows pytest jobs (`tests/console/commands/test_run.py::test_run_file_script`). Added a regression test that forces the Windows lookup path on Linux CI (by patching `WINDOWS`) and asserts the installed script is found without the bogus "not installed" warning, plus a unit test on `Env._bin()` / `get_command_from_bin()` covering the absolute-path case directly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Investigated the still-red Windows Confirmed via the live job logs (e.g. Root cause: I checked whether I looked at how this project already solves this for console-entry-point scripts on Windows ( Rather than leave CI red or claim Windows execution support that doesn't exist, I skipped both Windows-affected tests ( Caveat: I don't have a Windows machine, so this is code-reading-and-log analysis, not something I ran on Windows myself. I verified the skip decorator only fires on real Windows ( |
The `.cmd`-normalization fix in b7567f0 made the installed-script lookup correctly find `type = "file"` scripts on Windows, but the live Windows CI job is still red with a different, more fundamental error: '...\venv\Scripts\file-script' is not recognized as an internal or external command, operable program or batch file. This is not a path-resolution bug. `Env.execute()` runs commands on Windows through `cmd.exe` (`shell=True`), and `type = "file"` scripts are copied into the venv's script directory verbatim, with no `.cmd`/ `.exe` wrapper generated for them (unlike console entry points, which get a generated `.cmd` wrapper from `EditableBuilder._add_scripts()`). `cmd.exe` has no notion of a POSIX `#!/bin/sh` shebang and refuses to execute an extension-less file at all, regardless of whether the path under it is correct. Both Windows-affected tests fail this way: - `test_run_file_script` actually installs and runs the fixture's `#!/bin/sh` file script end-to-end, so it hits this directly on real Windows. It is a new test added by this PR (c5eb299), not a pre-existing baseline failure. - `test_run_file_script_on_windows_finds_installed_script` mocks `WINDOWS = True` to exercise the Windows lookup path on POSIX. On real Windows the lookup itself now succeeds -- its stderr shows no "not installed" warning -- but the final `poetry run` still fails for the same cmd.exe/shebang reason. Making `type = "file"` scripts actually runnable on Windows would need shebang parsing and a generated wrapper (the same mechanism console scripts get), which is a separate, larger design change than the `KeyError: 'callable'` crash and `.cmd`-normalization bugs this PR fixes. Rather than leave CI red or claim Windows support that doesn't exist, skip both tests on real Windows with a reason that documents the limitation and points back to this PR's description.
c188ea1 to
939b8ef
Compare
Summary
Fixes #11090
poetry run <name>crashes withKeyError: 'callable'for any scriptdeclared in
[tool.poetry.scripts]withtype = "file", even after thescript has been correctly installed into the venv's
bin/directory(thanks to the fix for #10664).
RunCommand.run_script()treats every dict-shaped script entry as aconsole entry point and unconditionally does
script["callable"]. Filescripts only have
reference/typekeys — there is nocallablekey —so this always raises:
Changes
src/poetry/console/commands/run.py:run_script()now detectstype = "file"scripts up front and runs them directly (they'realready-executable files, not
module:callableimportables) instead offalling into the console-script branch:
env.script_dirs(the normalcase after
poetry install), the resolved path from that lookup isexecuted as-is.
at its
referencepath relative to the project directory, sobehavior is consistent with (and no worse than) the existing
not-installed fallback for console scripts.
tests/fixtures/scripts/: added afile-scriptentry(
type = "file", pointing at a smallbin/file-scriptshell script) tothe existing
scriptsfixture project used by the run-command tests.tests/console/commands/test_run.py: addedtest_run_file_script, which installs the fixture project into a realvirtualenv and asserts
poetry run file-scriptexecutes successfully(previously raised
KeyError: 'callable') and its output is produced.Known limitation (Windows)
type = "file"scripts still cannot actually be run viapoetry runonWindows, for a reason unrelated to the bugs fixed above.
Env.execute()runs commands on Windows through
cmd.exe(shell=True), and file scriptsare copied into the venv's script directory verbatim, with no
.cmd/.exewrapper generated for them the way console entry points get one from
EditableBuilder._add_scripts()(WINDOWS_CMD_TEMPLATE).cmd.exehas nonotion of a
#!/bin/shshebang and refuses to execute an extension-lessfile at all, independent of path correctness.
Making this work in general would require parsing each script's shebang
and hoping the named interpreter is available on the Windows machine,
which is a materially larger, separate change from the
KeyError: 'callable'crash this PR fixes.test_run_file_scriptandtest_run_file_script_on_windows_finds_installed_scriptare thereforeskipped on real Windows, with a reason explaining this. Details in the
PR comment thread.
Test plan
test_run_file_script, which reproduces the exact tracebackfrom Make file scripts executable via
poetry run#11090 against the unmodified code (KeyError: 'callable'atrun.py:73) and passes once the fix is applied.pytest tests/console/commands/test_run.py tests/masonry/builders/test_editable_builder.py— 34 passed, 1 skipped (Windows-only case).
pytest tests/console/commands/(full directory) — 863 passed, 6skipped; the only 2 failures in this directory
(
debug/test_info.py::test_debug_info_displays_complete_info,env/test_info.py::test_env_info_displays_complete_info) arepre-existing and unrelated to this change — they fail identically on
unmodified
mainin this environment because it has no unversionedpythonexecutable onPATH(onlypython3), which those twocommands shell out to directly.
mypy src/poetry/console/commands/run.py— no issues.ruff check/ruff format --checkon the changed files — clean.mainbyrunning the new test against the unmodified file, then confirmed the
fix resolves it.