Skip to content

docs(agents): name the commands AGENTS.md's rules require - #372

Open
siloteemu wants to merge 1 commit into
mainfrom
docs-agents-md-runnable-commands
Open

docs(agents): name the commands AGENTS.md's rules require#372
siloteemu wants to merge 1 commit into
mainfrom
docs-agents-md-runnable-commands

Conversation

@siloteemu

@siloteemu siloteemu commented Sep 9, 2026

Copy link
Copy Markdown

AGENTS.md states rules but, in several places, never names the command that
satisfies them. Someone following it exactly runs none of the behaviour tests it
mandates and fails CI checks with no way to learn from the file what to run.

What was wrong

  • The prescribed gate could not run the tests the same file mandates.
    AGENTS.md requires user-observable behaviour changes to be covered by a
    Gherkin scenario under tests/e2e-cucumber/features/. Its quality gate
    prescribed cargo test --workspace --all-targets — but the e2e target sets
    test = false, so cargo test and cargo nextest skip every scenario
    silently and still report success. 17 feature files were reachable only via
    cargo xtask e2e, which the file never mentioned.
  • The same test = false excludes the harness from the prescribed clippy
    run
    , so the e2e step and harness code went unlinted by the gate as written.
    CI covers it with a second, package-scoped invocation; the file did not.
  • Two checks CI enforces hard were named in no contributor-facing document:
    the licence-header check and the third-party-notices staleness check. Their
    pre-commit hooks also fail differently when the underlying tool is absent
    locally — the notices hook passes --if-available and no-ops, so a green
    local run did not mean what CI means, while the licence-header hook is a
    plain language: system hook and aborts the run outright.

What changed

  • The quality gate now lists prek run --all-files, the package-scoped clippy
    invocation for the e2e harness, and cargo xtask e2e, alongside the three
    commands it already had (kept verbatim).
  • The scenario rule now names cargo xtask e2e at the point where the decision
    to write a scenario is made, including the filter form.
  • The licence-header and third-party-notices checks are named with their direct
    CI-equivalent commands and install hints, and the file states how each hook
    behaves when its tool is missing — the notices hook no-ops, the licence-header
    hook aborts — so the difference between a locally green run and a CI-green run
    is explicit.
  • AGENTS.md and CONTRIBUTING.md now cross-reference each other; previously
    neither mentioned the other.

Commands were sourced from the tool configuration and the CI workflow rather
than copied between documents, since for several tools the content did not
exist in either document.

Test plan

Documentation-only change; no behaviour is added, so no regression test applies.
Every command added to the file was verified by executing it in a clean
checkout:

Command Result
cargo xtask e2e ran 78 scenarios, 74 passed, 2 unexpected failures — pre-existing timing-sensitive bench throughput scenarios on a loaded host, unrelated to this change. The harness reconciles expected xfails separately from passes, so the two counts do not sum to the total.
prek run --all-files exit 0, 14 hooks passed
hawkeye check --config licenserc.toml exit 0, 230 files
cargo xtask tpn --check exit 0

prek run --all-files also passes on the edited files.

Checklist

  • If this PR fixes a bug, searched tests/e2e-cucumber/expectations.toml for the fixed ticket ID and removed/narrowed any now-stale xfail rows. — Not applicable: documentation-only, no code behaviour changes, so no xfail row can become stale. Confirmed expectations.toml is untouched by this PR.

Notes for reviewers

  • The second clippy line (-p e2e-cucumber --test e2e) addresses the same
    test = false root cause as the e2e gap and comes from the CI workflow. It is
    adjacent to the originally reported problem rather than part of it — happy to
    drop it if you would rather keep this change narrower.
  • The pre-existing cargo clippy --workspace --all-targets line lacks --locked
    while CI uses it; the newly added clippy line matches CI byte-for-byte. Left
    the pre-existing line alone to keep this change scoped, but happy to align it.

AGENTS.md required behavior changes to be covered by Gherkin scenarios
under tests/e2e-cucumber/, but the quality gate it prescribed could not
run them: the e2e target sets `test = false`, so
`cargo test --workspace --all-targets` skips every scenario silently and
still reports success. The same setting excludes the harness from
`cargo clippy --workspace --all-targets`. Both gaps are now named, with
the commands that actually cover them (`cargo xtask e2e` and a second,
package-scoped clippy invocation), matching what CI runs.

The scenario rule now names `cargo xtask e2e` where the decision to write
a scenario is made, with the caveat that `-n` selection replaces the
harness's own filter rather than composing with it: a filtered run drops
OS applicability, xfail expectations and lifecycle selection, so it is a
quick loop and not the gate. docs/testing.md already gives that warning.

Two more checks CI enforces hard - license headers and third-party
notices - appeared in no contributor-facing document. Their pre-commit
hooks fail differently when the tool is absent locally, and the file now
says which is which: the notices hook passes `--if-available` and no-ops,
so a green local run can silently omit it, while the license-header hook
is a plain `language: system` hook and aborts the run outright. The gate
lists `prek run --all-files` plus the direct CI forms (`hawkeye check`
and `cargo xtask tpn --check`), so a locally green run means what CI
means.

CONTRIBUTING.md gains a pointer to AGENTS.md so contributors arriving
from the setup instructions find the agent rules, and AGENTS.md points
back for setup and commit conventions rather than restating them.

Signed-off-by: Teemu Kuusisto <teemu.kuusisto@amd.com>
@siloteemu
siloteemu marked this pull request as ready for review September 9, 2026 08:46
@siloteemu
siloteemu requested a review from a team as a code owner September 9, 2026 08:46

@jussielo-amd jussielo-amd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed. Docs-only change (AGENTS.md, CONTRIBUTING.md), no runtime/CLI-facing impact.

Correctness: All 11 new factual claims (e2e test = false quirk, exact CI clippy invocation, cargo xtask e2e, license-header hook aborting vs. third-party-notices hook silently no-oping, etc.) were independently verified against the repo — all accurate. Fixes the four gaps it describes; a few other "rule with no named command" spots remain elsewhere in AGENTS.md (§3/§13, §6, §8, §12) — same defect class, out of scope here, good follow-up candidate.

Note: you already flagged it yourself, but the new e2e-cucumber clippy line uses --locked (matching CI) while the adjacent pre-existing workspace clippy line still doesn't, even though CI's version has it too — worth aligning whenever convenient.

UX: minor flow nits only — the new §3 paragraph sits between the topic sentence and the bullets it explains (slightly breaks the antecedent), and the AGENTS.md cross-reference in CONTRIBUTING.md lands mid-setup-section without a transition. Neither is misleading, just placement polish.

No blocking issues, CI green. Approving.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants