fix(docs): emit absolute URLs from llms.txt and llms-full.txt - #198
Merged
Conversation
Both llms endpoints emitted site-relative links. That file is fetched and its text lifted into a context window where the origin is no longer attached to it, so a relative path resolves only if whatever moved it there also carried the base URL — which is exactly what a consumer of an llms.txt will not do. The marketing site's own /llms.txt has always emitted absolute URLs; this is the docs site catching up to it. llms.txt rewrites page-tree node URLs before handing them to fumadocs' indexNode, so titles, descriptions, indentation and nesting come out untouched and no pattern runs over an authored description. Node identity is carried by $ref, not url, so the spread leaves title lookup working. The three prose path examples go through localeUrl. llms-full.txt resolves each page's body links against that page's own absolute URL, which also fixes ./sibling and #fragment links that needed the page identity the file never carried. Fenced code blocks are skipped. No new base-URL literal: both routes import SITE_URL and localeUrl from lib/seo.ts, which is unchanged. robots.ts's own BASE collapses into the same constant, taking the count of hostname literals in apps/docs from 4 to 3. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5zjYc2BoFV2NjKBBapC7C
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #178
Both
llmsendpoints emitted site-relative links. That file is fetched and its text is then lifted into a context window where the origin is no longer attached to it, so a relative path resolves only if whatever moved it there also carried the base URL — which is exactly what a consumer of anllms.txtwill not do. The marketing site's own/llms.txthas always emitted absolute URLs; this is the docs site catching up to it.What changed
llms.txtrewrites page-tree node URLs before handing them to fumadocs'indexNode, rather than rewriting the rendered bullet. Titles, descriptions, indentation and nesting therefore come out of the generator untouched and no pattern ever runs over an authored description. Node identity is carried by$ref, noturl—getNodePageandgetNodeMetaboth look up by$ref— so the spread leaves title and description lookup working.The three prose path examples in the header and the "Other Languages" footer go through
localeUrl, which is what that helper is for: they are logical paths, andlocaleUrlis the thing that knows the default locale carries no prefix.llms-full.txtresolves each page's body links against that page's own absolute URL. Site-root links (/docs/x) were the card's subject; resolving per page also fixes the page-relative (./sibling) and bare-fragment (#anchor) links, which needed the page identity that this file has never carried and were therefore less resolvable than the ones the card named. Fenced code blocks are skipped, so a future page that documents markdown link syntax is not silently edited.Targets carrying a scheme cannot match the pattern at all, so no
https:ormailto:URL is re-parsed and re-serialized; nor is a protocol-relative//host/x, which names a different origin.robots.ts— the adjacent question the card names. It stayed a one-line change, so it is in:BASEis deleted andSITE_URLimported in its place. That takes hostname literals inapps/docsfrom 4 to 3 and adds none.apps/docs/lib/seo.tsis unchanged — its existing exports were already sufficient, so there is no new helper either.The two literals that remain are
app/layout.tsx'smetadataBaseandmiddleware.ts's canonical-domain redirect. Both are outside this card's declared file surface, andmiddleware.tsruns in the edge runtime where importinglib/seowould pull the fumadocs source loader in with it — so collapsing them is a real design question, not the same one-line move. Left alone, not filed.Verification
Production builds (
turbo run build --filter=@objectos/docs), served withnext start, both bodies fetched over HTTP. Head build and the counts below are from9073d14.llms.txtllms-full.txtrobots.txtEvery one of the 79
llms.txttargets and all 549 rewrittenllms-full.txttargets beginhttps://docs.objectos.aiand parse as absolute URLs. The 97 targets left untouched are the ones that already carried a scheme (https:,mailto:), verified byte-identical.Composition is unchanged — the invariant #175's gate asserts.
llms.txt: the served body is byte-for-byte identical tomainonce the stringhttps://docs.objectos.aiis deleted from the new side. Same 79 links in the same order, same 79 slugs, same 10 headings, same 116 lines. Nothing but the spelling of a URL moved.llms-full.txtneeds a per-link check rather than a string deletion, because./siblingand#anchorexpand to more than a prefix. Pairing the two served bodies page by page:new URL(beforeTarget, thatPagesUrl).href(549)Locale surface unchanged: 0 non-default-locale link targets in either body, before and after.
Two pre-existing defects surfaced by the comparison, filed rather than fixed here
llms-full.txtpage order is not reproducible across builds. Two builds of the same commit emit the pages in different orders (the multiset is identical). Same root cause as [finding] the generated sitemap's entry order is not reproducible — two builds of an unchanged tree emit different bytes #194 on a second surface. This is why the check above pairs pages by title instead of by position, and [finding] nothing pins the generated sitemap — the seven-locales-for-every-page defect was green on every gate #175 will need the same normalisation.#196remains open.getText('processed')emits HTML numeric character references into this plain-text file, and in 2 places the escaped character is a link's closing parenthesis, so the link is malformed. Present identically before and after; those 2 of 646 targets are the only ones this PR leaves site-relative, because they are not well-formed links for it to rewrite.#197remains open.Gates
Run on
9073d14:pnpm turbo run type-check --filter=@objectos/docs—Tasks: 1 successful, cache miss (really executed)pnpm turbo run build --filter=@objectos/docs—Compiled successfully,Tasks: 1 successfulpnpm turbo run test—Tasks: 1 successfulnode .github/scripts/check-node-floor.mjs --self-testthen the real scan — both exit 0node .github/scripts/check-translation-ownership.mjsagainst the real changed-file list — exit 0, "touches 0 translation artifact(s)"The 310
Failed to load dynamic fontlines in the build log are the OG-image route reaching for Google Fonts through a proxy with a self-signed certificate. Identical count on the baseline build ofmain; environment noise, not this change.No changeset: this repository has no changeset flow.
Generated by Claude Code