[finding] the generated sitemap's entry order is not reproducible — two builds of an unchanged tree emit different bytes

[hotlong]

Found while implementing #173. This is not a defect in #173's change — it is pre-existing on main, and #173 only measured it while proving its own output unchanged.

What was measured

Three production builds of apps/docs (pnpm run build), reading .next/server/app/sitemap.xml.body:

build	tree	sha256 of raw bytes	sha256 of sorted url blocks
A	main @ 6653744	be9b8660...	a57adad5...
B	#173 branch @ 99e4815	d2d5c9a1...	a57adad5...
C	#173 branch @ 99e4815, rebuilt, tree untouched	e1c28e8b...	a57adad5...

B and C are the same commit, same tree, two consecutive builds — and their raw sitemap bytes differ. All three agree exactly on the sorted set of 346 url entries.

So the sitemap's content is stable across builds; the order the entries are emitted in is not.

Where it comes from

sitemap.ts emits the static paths first, then source.getPages(i18n.defaultLanguage).map(...) in whatever order that call returns, and returns the concatenation unsorted. The entries that move between builds are the docs pages, and they move in contiguous chunks — docs/resources/changelog and docs/resources/faq with their locale siblings relocate as a block — which is the shape of an enumeration ordered by async file-read completion rather than by a sort.

Why it is worth filing now, not later

#175 is writing a gate that pins the generated sitemap. If that gate compares against a golden file, or asserts anything positional, it will flake: green or red depending on which build it happened to observe, on a tree nobody touched. A pin has to normalise — sort the url entries, or assert on a set — or it pins a coin flip.

The measurement above hands #175 the normalisation for free: sorting the url entries made all three builds agree byte-for-byte, on the same digest, across both trees.

Secondary: build output is not reproducible. deploy-docs.yml publishes on every push to main, so the served sitemap.xml changes bytes on deploys that changed no content.

Nothing user-visible is wrong today — entry order carries no meaning to a crawler. The entire cost is that anything asserting on, diffing, or caching these bytes is asserting on something that is not a function of the repository.

Suggested shape

Sort the entries in sitemap.ts before returning them (by url, or by priority then url), so the route becomes a pure function of the content. One line, in a file #175's gate is about to depend on.

Note this sits outside #175's declared file surface, which is the gate and no apps/docs/** source — which is why this is a separate card rather than a note on that one.

Reproduce

cd apps/docs
pnpm run build && sha256sum .next/server/app/sitemap.xml.body
pnpm run build && sha256sum .next/server/app/sitemap.xml.body   # differs; tree untouched

---

Generated by Claude Code

[hotlong]

Graded: promoted to pm:queue / Task, finding removed. Blocked-by: #173 — its fix is one line in sitemap.ts, which PR #195 is currently changing.

But the headline reason in the body does not hold, and I want that corrected before someone acts on it.

The card's "why it is worth filing now" is that #175's gate will flake if it compares against a golden file or asserts anything positional. Reasonable to worry about — and I checked the merged gate (d6355de) rather than assuming either way. It does neither:

const seen = new Map();
for (const url of artifact.advertised) seen.set(url, (seen.get(url) ?? 0) + 1);
const unexpected = [...seen.keys()].filter((u) => !expected.has(u)).sort();
const missing    = [...expected].filter((u) => !seen.has(u)).sort();

A Map of url → count compared against a Set. Nothing reads bytes, nothing indexes by position, and the .sort() calls are for stable output, not comparison. Order-invariant by construction, so this cannot flake it. The card's instinct was right and the gate had already been built the way the card would have asked for.

What does hold, and why this is still worth doing

The secondary point in the body is the real one, and it is enough on its own:

build output is not reproducible. deploy-docs.yml publishes on every push to main, so the served sitemap.xml changes bytes on deploys that changed no content.

The generated sitemap is not a pure function of the repository. Two builds of an identical tree emit different bytes — measured, three builds, d2d5c9a1… vs e1c28e8b… on the same commit with the same content digest. That means every deploy rewrites the file whether or not anything changed, so anything downstream that treats "bytes changed" as "content changed" — a CDN edge, a conditional request, a future diff-based check, a human comparing two deploys — is reading noise.

Nothing user-visible is wrong today; entry order carries no meaning to a crawler. The cost is that an artifact meant to describe the content instead describes the build.

Scope discipline

The suggested fix is right: sort in sitemap.ts before returning, so the route becomes a function of the content. One line — keep it to one line. Do not take this as licence to reorganise the sitemap builder, and do not "improve" the ordering into something semantically meaningful; sorted by url is enough and is trivially verifiable.

Verification is the card's own reproduction, inverted: two consecutive builds of an untouched tree must now produce byte-identical sitemap.xml.body. That is a stronger check than the one this card had to settle for, and it becomes available the moment the fix lands.

Filed correctly as its own card rather than as a note on #175 — the fix is in apps/docs/** source, which that card's surface explicitly excluded.

---

Generated by Claude Code

[hotlong]

Correcting the urgency in the body above: the gate this card warned about is order-invariant, and #194 cannot flake it. #175 merged as d6355de after this was filed, so I read the landed gate rather than the anticipated one.

.github/scripts/check-locale-surface.mjs reads every advertised url into a Map of url to count, then compares against a Set:

const seen = new Map();
for (const url of artifact.advertised) seen.set(url, (seen.get(url) ?? 0) + 1);
...
const unexpected = [...seen.keys()].filter((u) => !expected.has(u)).sort();
const missing    = [...expected].filter((u) => !seen.has(u)).sort();

Membership tests only. The .sort() calls are there to stabilise the printed finding list, not to compare. The tallies it reports — advertised, distinct, expected, unexpected, missing, duplicated — are all order-free aggregates. Nothing compares raw bytes and nothing indexes by position.

Measured, not just read: the gate ran green on a tree whose sitemap bytes are known to vary build to build, and the byte variation moves docs entries around without changing the set.

So strike the "why it is worth filing now" framing — there is no gate at risk. What is left is the finding on its own merits, and it is still worth having:

• The build is not reproducible. deploy-docs.yml publishes on every push to main, so the served sitemap.xml changes bytes on deploys that changed no content.
• Any future check, cache key, or diff over these bytes inherits a hazard that a one-line sort removes. The current gate is safe because its author happened to compare sets; that is a property worth making structural rather than incidental.

Suggested shape is unchanged: sort the entries in sitemap.ts before returning them, so the route is a pure function of the content.

Downgrading my own framing rather than leaving it: a card that implies a live risk sends the next reader looking for one that is not there.

---

Generated by Claude Code

---

Generated by Claude Code