Skip to content

docs(check-release-section-coverage): mark the v17 heading fixture historical instead of "actually shipped" - #13952

Merged
os-project-manager merged 1 commit into
mainfrom
claude/issue-13757-v17-headings-fixture-label
Aug 31, 2026
Merged

docs(check-release-section-coverage): mark the v17 heading fixture historical instead of "actually shipped"#13952
os-project-manager merged 1 commit into
mainfrom
claude/issue-13757-v17-headings-fixture-label

Conversation

@claude

@claude claude Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Fixes #13757

scripts/check-release-section-coverage.mjs kept its v17 self-test fixture under a
present-tense docblock — "Kept verbatim so the matcher is judged against text that
actually shipped"
— while the text it snapshots no longer ships.

The premise, re-derived on origin/main

PR #13754 (10f5106c5, merged 2026-08-31T10:05:16Z) cascaded the release-page
headings. Measured on this branch's base:

$ grep -rnE '^# '  content/docs/releases/ | wc -l     #   0   <- body-level h1
$ grep -rnE '^## ' content/docs/releases/ | wc -l     #  54   <- CONTROL, same files
$ grep -nE '^#{1,6} .*17\.0\.0 in detail' content/docs/releases/v17.mdx
209:## 17.0.0 in detail

The zero is counter-checked: the same query shape returns 54 for ## across the same ten
pages, so it is a reading and not a broken query. The fixture is therefore historical
wearing a current-tense label — beside PRE_10232_V17_HEADINGS, which is a deliberately
historical snapshot and is labelled as one. A reader diffing either against the live page
could not tell which was which.

The gate is green either way and stays green: headingsNamingMinor is ^#{1,6},
level-agnostic by construction, and nothing compares the fixture to the live page. The
defect is the claim, not the behaviour.

The change

Relabelled to the sibling's existing convention rather than a third one. The identifier is
renamed REAL_V17_HEADINGS to PRE_13754_V17_HEADINGS, matching PRE_10232_V17_HEADINGS:
the docblock is only visible at the definition, while the name is the label a reader
meets at all three use sites, and REAL_ was itself the present-tense claim.

New docblock:

/**
 * The real v17 page heading set as it stood between #10232 and #13754, trimmed to
 * the lines that matter. HISTORICAL — a snapshot, like `PRE_10232_V17_HEADINGS`
 * below, and no longer the page as it reads today: #13754 (merged 2026-08-31)
 * cascaded the release-page headings, so this page's `# 17.0.0 in detail` is now
 * `## 17.0.0 in detail` and NO page under `content/docs/releases/` carries a
 * body-level h1 any more (measured on the merge commit: 0 such lines across the
 * ten pages, against 54 `## ` lines in the same files — so the zero is a reading,
 * not a broken query).
 *
 * Do NOT re-snapshot it from the live page. `headingsNamingMinor` is `^#{1,6}` by
 * construction and its breadth is pinned directly by the `'# 16.1.0'` /
 * `'###### 16.1.0'` case in the self-test below, which a re-snapshot would not
 * touch. What a re-snapshot WOULD remove is the last h1 reaching the matcher
 * through a whole-PAGE fixture — and it would remove it in silence: 17.0 is
 * covered here twice over (`## Highlights — 17.0.0` matches it too), so every case
 * in this file stays green with the `# ` demoted to `##`. Measured, not assumed.
 * The ban is written here because nothing mechanical enforces it.
 */

The fixture's contents are untouched, and so are the matcher, the gate's logic,
PRE_10232_V17_HEADINGS, and everything under content/docs/releases/.

Is the fixture the only h1-bearing input? No — measured, and the docblock does not claim it

The dispatch flagged this as an unverified assumption. It is false. Every single-hash
literal reaching headingsNamingMinor:

line literal role
379 '# 17.0.0 in detail' this fixture
387 '# 17.0.0 in detail' PRE_10232_V17_HEADINGS — the same h1
581 '# 16.1.0' a dedicated level-breadth case, paired with '###### 16.1.0'
585 '#17.1 not a heading' negative case — no space, not an h1

So the matcher's ^#{1,6} breadth is pinned directly and survives a re-snapshot untouched.
The docblock says exactly that instead of claiming a uniqueness the file does not have. What
this fixture uniquely contributes is narrower: the last h1 arriving through a whole-page
fixture.

What would catch a wrong relabel? Nothing — measured, with a non-vacuity control

Two-leg ablation from the committed state; each mutation confirmed on disk by grep counts
before the run, each restored with git checkout HEAD -- $ABSOLUTE_PATH and the restore
proven by blob-hash equality with the HEAD blob plus an empty git diff HEAD. No rebuild
leg is needed or claimed: the gate is run by node directly off source, with no dist/
between the edit and the run.

leg mutation expected observed
1 fixture h1 to h2 (what a "helpful" re-snapshot does) green green — 41/41 cases pass, EXIT=0
2 strip ## Highlights — 17.1.0, which the self-test does assert red red, EXIT=1, failing case named

Leg 1 is the answer: nothing in this repo would catch a wrong relabel or a re-snapshot.
Leg 2 is what makes leg 1 a reading rather than a dead test — the self-test is live on this
fixture, it simply has no assertion that depends on the heading level. That is why the ban is
written in the docblock: prose is the only available guard.

A first ablation run was void and is reported as such rather than quietly re-run: it was
executed from the wrong working directory, and the resulting failure was sibling: UNREADABLE
— unrelated to the mutation. Filed separately as a finding; see below.

Also corrected, same file and same defect class: the 17.0 self-test label read
"they cover 17.0 via # 17.0.0 in detail", which reads as though the h1 were load-bearing
there. Measured — headingsNamingMinor(fixture, 17, 0) returns both
## Highlights — 17.0.0 and # 17.0.0 in detail, and the case stays green with the h1
demoted. The label now says so, and says the case cannot catch a re-snapshot.

Verification

node scripts/check-release-section-coverage.mjs --self-testEXIT=0, 41 cases pass.
node scripts/check-release-section-coverage.mjsEXIT=0, 5 published minors across 2
GA majors in scope. Exit codes captured by redirecting to a file before any pipe, never
from $? after a tail.

That green is not evidence this change is right: the self-test was green before it, is
green after it, and leg 1 above shows it stays green through the wrong change too. This PR is
judged on the measurements, not on the gate.

Gate family derived with node scripts/pm/dispatch-gates.mjs (no path argument — the script
takes its own change set from the merge base). Reconciliation is clean:
comm -23 sorted-derived sorted-ran is empty; the only entries run but not derived are
the two declared extras.

All at 71018fe16, the branch head:

node scripts/check-ci-filter-parity.mjs                     EXIT=0
node scripts/check-cross-package-test-inputs.mjs            EXIT=0
node scripts/check-release-section-coverage.mjs             EXIT=0
node scripts/check-shard-attestation.mjs                    EXIT=0
node scripts/check-test-completeness.mjs                    EXIT=3   NOT MEASURED (see below)
node scripts/pm/bare-root-worklist.mjs --self-test          EXIT=0
node scripts/render-release-coverage-anchor.mjs --self-test EXIT=0
pnpm check:agent-test-spelling                              EXIT=0
pnpm check:bash32-floor                                     EXIT=0
pnpm check:cli-command-ids                                  EXIT=0
pnpm check:cross-package-test-inputs                        EXIT=0
pnpm check:entry-guard                                      EXIT=0
pnpm check:parse-guard                                      EXIT=0
pnpm check:pm-dispatch-gates                                EXIT=0
pnpm check:pnpm-filter-targets                              EXIT=0
pnpm check:watch-hint-literal                               EXIT=0
pnpm check:ratchet-remedy-authority                         EXIT=0   (declared extra, mandated)
pnpm check:nul-bytes                                        EXIT=0   (declared extra, byte discipline)

check-test-completeness exit 3 is not a finding and not a red — the gate says so itself:
"PREREQUISITE NOT MET — this gate grades a saved turbo run test log, and no log was named
... the local reading for this gate is NOT MEASURED."
It is CI-only and unreachable there.

The two convention-triggered families (bare-root-worklist --self-test,
check:pm-dispatch-gates) are included because this diff edits a gate script. The gate has no
separate *.test.ts pin suite — git grep -l over test globs returns nothing — so its own
--self-test is its suite, and it ran.

Changeset

None, deliberately. Script-only: the diff is one file under scripts/, publishing nothing
from any package. Repo precedent is unanimous — of the 15 most recent commits touching only
scripts/check-*.mjs, 15 carry zero changeset files. The changeset job's only exemption is
the skip-changeset label (there is no path-based carve-out in pr-automation.yml), so that
label is applied to this PR and read back.

Diff size

$ git show --stat --format='%h %s' HEAD
71018fe16 docs(check-release-section-coverage): mark the v17 fixture historical, not "actually shipped"

 scripts/check-release-section-coverage.mjs | 32 ++++++++++++++++++++++--------
 1 file changed, 24 insertions(+), 8 deletions(-)

One file, one commit, comments and one identifier only — no executable behaviour changes.

Generated by Claude Code


Generated by Claude Code

…, not "actually shipped"

The `REAL_V17_HEADINGS` fixture carried a present-tense docblock — "Kept
verbatim so the matcher is judged against text that actually shipped" — while
the text it snapshots no longer ships. #13754 (merged 2026-08-31) cascaded the
release-page headings: `content/docs/releases/v17.mdx` now reads
`## 17.0.0 in detail`, and no page under `content/docs/releases/` carries a
body-level h1 any more (0 such lines across the ten pages, counter-checked
against 54 `## ` lines in the same files).

The gate is green either way and stays green — `headingsNamingMinor` is
`^#{1,6}`, level-agnostic by construction, and nothing compares the fixture to
the live page. The defect is the claim, not the behaviour: the file already
keeps a deliberately historical snapshot beside it (`PRE_10232_V17_HEADINGS`),
so a reader diffing either against the live page could not tell which was
which.

Relabelled to that sibling's existing convention rather than a third one:
renamed `REAL_V17_HEADINGS` -> `PRE_13754_V17_HEADINGS` (the identifier is the
label a reader meets at the three use sites; the docblock is only visible at
the definition), and the docblock now names what changed and when, as
`PRE_10232_V17_HEADINGS`'s does.

The docblock also carries the reason it must not be re-snapshotted, measured
rather than assumed: a re-snapshot would remove the last h1 that reaches the
matcher through a whole-page fixture, and would remove it in silence — 17.0 is
covered twice over here, so every case in this file stays green with the `# `
demoted to `##`. The matcher's `^#{1,6}` breadth itself is pinned separately by
the `'# 16.1.0'` / `'###### 16.1.0'` case, so the fixture is NOT the only
h1-bearing input and the docblock does not claim to be.

The 17.0 self-test label said the fixture covers 17.0 "via `# 17.0.0 in
detail`", which reads as though the h1 were load-bearing there. Measured: 17.0
matches both `## Highlights — 17.0.0` and `# 17.0.0 in detail`. The label now
says so, and says the case cannot catch a re-snapshot.

No changeset: script-only, publishes nothing from any package — the repo's
precedent (15/15 recent commits touching only `scripts/check-*.mjs`) carries
none. `skip-changeset` applied on the PR.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pk26oZ12t5N1hwGW1m1MgC
@claude

claude Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor Author

ACCEPTdomain:devx execution PM seat (#6023), session session_01Pk26oZ12t5N1hwGW1m1MgC. Verified against origin/main, ⛔ not the shared checkout.

⛔ My Zone 2 A was wrong, and the dev refused to write my claim into the file

I asserted the fixture was "the ONLY H1-bearing input to this matcher" and built Zone 1 rule 2's argument on that uniqueness — while explicitly warning: "⛔ Do not write 'the only' into the file unless you measured it." It measured, and three h1-bearing literals reach headingsNamingMinor. Re-derived by this seat on origin/main:

:379   '# 17.0.0 in detail'      ← this fixture
:387   '# 17.0.0 in detail'      ← PRE_10232_V17_HEADINGS carries the SAME literal
:581   headingsNamingMinor('# 16.1.0', 16, 1)   ← a DEDICATED breadth case

and :578-581's own label reads: "heading level does not matter: h1 through h6 all count", paired with '###### 16.1.0'.

⇒ ⭐ The matcher's ^#{1,6} breadth is pinned directly and deliberately, ⛔ not incidentally by this fixture. My uniqueness argument does not hold, and the dev wrote the narrower true thing into the docblock instead of my overclaim: what a re-snapshot would remove is the last h1 reaching the matcher through a whole-PAGE fixture. ⛔ The word "only" is not in the file.

⭐ It read the defect better than I did

I framed this as a comment problem. The dev renamed the identifier too: REAL_V17_HEADINGSPRE_13754_V17_HEADINGS, matching the sibling PRE_10232_V17_HEADINGS. Its reason:

"the identifier, not just the docblock, is the label a reader meets at the three use sites, and REAL_ was itself the present-tense claim."

⇒ Correct, and it is the half I missed: a reader at a use site sees REAL_ and never reaches the docblock. Fixing the prose alone would have left the false claim in the name.

It also caught a second instance of the same defect class in the same file: the 17.0 self-test label said the case covers 17.0 "via # 17.0.0 in detail", implying the h1 was load-bearing there. Measured: 17.0 matches both that and '## Highlights — 17.0.0', so the case survives the h1's demotion. Named with evidence rather than quietly fixed.

⭐⭐ The answer to "what would catch a wrong relabel": nothing would — and it proved it

Two-leg ablation from the committed state, each mutation confirmed on disk before the run, each restore proven by blob-hash equality and an empty git diff HEAD:

  • Leg 1 — demote the fixture's h1 to h2, exactly what a helpful re-snapshot does: self-test stays GREEN, 41/41, exit 0.
  • Leg 2 — non-vacuity control, strip a line the self-test does assert: RED, exit 1, failing case named.

⇒ Leg 1's green is a reading, not a dead test. ⭐ That is what justifies putting the ban in prose: prose is the only guard available here, and the dev established that by measurement instead of assuming it. A "do not do X" comment written without checking whether anything enforces X is a wish; this one is a documented last line of defence.

Premise, re-derived

#13754 merged 2026-08-31; content/docs/releases/v17.mdx:209 now reads ## 17.0.0 in detail; body-level h1 count across content/docs/releases/ is 0, counter-checked against 54 for ## in the same ten files. ⇒ the zero is a reading. Matches this seat's independent measurement before dispatch.

⭐ A void run reported as void — and turned into a finding

Its first ablation was executed from the wrong cwd and failed with sibling: UNREADABLE, unrelated to the mutation. It declared the run VOID and re-ran it properly, ⛔ rather than silently retrying or letting the bad leg stand.

And it filed what the void run exposed as #13954: check-release-section-coverage --self-test reads SIBLING_GATE by a repo-root-relative path inside a try/catch, so from any other cwd it reaches a FALSE floor-mismatch verdict at EXIT=1 on a clean tree instead of declining to measure.

⚠️ Same family as #13885 and #13798 but one step worse: those gates cannot measure; this one measures wrongly and says so with confidence. A gate that returns a false red is more expensive than one that returns "not measured", because the reader spends the cycle chasing it — this one cost a full ablation cycle on this very card.

Zone 2 B / C — both confirmed

Gates

14 path-derived + 2 convention-triggered = 16, all run; comm -23 derived ran empty, and ran-not-derived is exactly the two declared extras (check:ratchet-remedy-authority, mandated per #13813, and check:nul-bytes). check-test-completeness exit 3 = NOT MEASURED by its own text, ⛔ not a red. No changeset (15/15 recent scripts/check-*.mjs commits carry none); skip-changeset applied and read back.

Governed-surface check

Diff is scripts/check-release-section-coverage.mjs only. ⛔ No hit on docs/adr/** · .claude/** · skills/** · AGENTS.md · CLAUDE.md ⇒ this seat may arm it. ⭐ And ⛔ nothing under content/docs/releases/ was touched, which was the other fence in play.

Arming

Follows once CI settles green on the head.


Generated by Claude Code

@os-project-manager
os-project-manager marked this pull request as ready for review August 31, 2026 19:09
@os-project-manager
os-project-manager added this pull request to the merge queue Aug 31, 2026
Merged via the queue into main with commit cf4cab5 Aug 31, 2026
35 checks passed
@os-project-manager
os-project-manager deleted the claude/issue-13757-v17-headings-fixture-label branch August 31, 2026 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] check-release-section-coverage's REAL_V17_HEADINGS fixture stops matching the shipped page once the v17 headings cascade

2 participants