docs(agents): name the commands AGENTS.md's rules require - #372
Conversation
AGENTS.md required behavior changes to be covered by Gherkin scenarios under tests/e2e-cucumber/, but the quality gate it prescribed could not run them: the e2e target sets `test = false`, so `cargo test --workspace --all-targets` skips every scenario silently and still reports success. The same setting excludes the harness from `cargo clippy --workspace --all-targets`. Both gaps are now named, with the commands that actually cover them (`cargo xtask e2e` and a second, package-scoped clippy invocation), matching what CI runs. The scenario rule now names `cargo xtask e2e` where the decision to write a scenario is made, with the caveat that `-n` selection replaces the harness's own filter rather than composing with it: a filtered run drops OS applicability, xfail expectations and lifecycle selection, so it is a quick loop and not the gate. docs/testing.md already gives that warning. Two more checks CI enforces hard - license headers and third-party notices - appeared in no contributor-facing document. Their pre-commit hooks fail differently when the tool is absent locally, and the file now says which is which: the notices hook passes `--if-available` and no-ops, so a green local run can silently omit it, while the license-header hook is a plain `language: system` hook and aborts the run outright. The gate lists `prek run --all-files` plus the direct CI forms (`hawkeye check` and `cargo xtask tpn --check`), so a locally green run means what CI means. CONTRIBUTING.md gains a pointer to AGENTS.md so contributors arriving from the setup instructions find the agent rules, and AGENTS.md points back for setup and commit conventions rather than restating them. Signed-off-by: Teemu Kuusisto <teemu.kuusisto@amd.com>
jussielo-amd
left a comment
There was a problem hiding this comment.
Reviewed. Docs-only change (AGENTS.md, CONTRIBUTING.md), no runtime/CLI-facing impact.
Correctness: All 11 new factual claims (e2e test = false quirk, exact CI clippy invocation, cargo xtask e2e, license-header hook aborting vs. third-party-notices hook silently no-oping, etc.) were independently verified against the repo — all accurate. Fixes the four gaps it describes; a few other "rule with no named command" spots remain elsewhere in AGENTS.md (§3/§13, §6, §8, §12) — same defect class, out of scope here, good follow-up candidate.
Note: you already flagged it yourself, but the new e2e-cucumber clippy line uses --locked (matching CI) while the adjacent pre-existing workspace clippy line still doesn't, even though CI's version has it too — worth aligning whenever convenient.
UX: minor flow nits only — the new §3 paragraph sits between the topic sentence and the bullets it explains (slightly breaks the antecedent), and the AGENTS.md cross-reference in CONTRIBUTING.md lands mid-setup-section without a transition. Neither is misleading, just placement polish.
No blocking issues, CI green. Approving.
AGENTS.mdstates rules but, in several places, never names the command thatsatisfies them. Someone following it exactly runs none of the behaviour tests it
mandates and fails CI checks with no way to learn from the file what to run.
What was wrong
AGENTS.mdrequires user-observable behaviour changes to be covered by aGherkin scenario under
tests/e2e-cucumber/features/. Its quality gateprescribed
cargo test --workspace --all-targets— but the e2e target setstest = false, socargo testandcargo nextestskip every scenariosilently and still report success. 17 feature files were reachable only via
cargo xtask e2e, which the file never mentioned.test = falseexcludes the harness from the prescribed clippyrun, so the e2e step and harness code went unlinted by the gate as written.
CI covers it with a second, package-scoped invocation; the file did not.
the licence-header check and the third-party-notices staleness check. Their
pre-commit hooks also fail differently when the underlying tool is absent
locally — the notices hook passes
--if-availableand no-ops, so a greenlocal run did not mean what CI means, while the licence-header hook is a
plain
language: systemhook and aborts the run outright.What changed
prek run --all-files, the package-scoped clippyinvocation for the e2e harness, and
cargo xtask e2e, alongside the threecommands it already had (kept verbatim).
cargo xtask e2eat the point where the decisionto write a scenario is made, including the filter form.
CI-equivalent commands and install hints, and the file states how each hook
behaves when its tool is missing — the notices hook no-ops, the licence-header
hook aborts — so the difference between a locally green run and a CI-green run
is explicit.
AGENTS.mdandCONTRIBUTING.mdnow cross-reference each other; previouslyneither mentioned the other.
Commands were sourced from the tool configuration and the CI workflow rather
than copied between documents, since for several tools the content did not
exist in either document.
Test plan
Documentation-only change; no behaviour is added, so no regression test applies.
Every command added to the file was verified by executing it in a clean
checkout:
cargo xtask e2ebenchthroughput scenarios on a loaded host, unrelated to this change. The harness reconciles expected xfails separately from passes, so the two counts do not sum to the total.prek run --all-fileshawkeye check --config licenserc.tomlcargo xtask tpn --checkprek run --all-filesalso passes on the edited files.Checklist
tests/e2e-cucumber/expectations.tomlfor the fixed ticket ID and removed/narrowed any now-stale xfail rows. — Not applicable: documentation-only, no code behaviour changes, so no xfail row can become stale. Confirmedexpectations.tomlis untouched by this PR.Notes for reviewers
-p e2e-cucumber --test e2e) addresses the sametest = falseroot cause as the e2e gap and comes from the CI workflow. It isadjacent to the originally reported problem rather than part of it — happy to
drop it if you would rather keep this change narrower.
cargo clippy --workspace --all-targetsline lacks--lockedwhile CI uses it; the newly added clippy line matches CI byte-for-byte. Left
the pre-existing line alone to keep this change scoped, but happy to align it.