Make prisma.io docs agent-ready: llms.txt split, agent directives, skill.md, MCP discovery#8087
Merged
Conversation
…discovery) Fixes failing checks from the Mintlify agent-score audit of www.prisma.io/docs: - Split the 116k-char llms.txt into a small root index plus per-area section indexes (each under 50k chars), with a coverage guarantee for unmatched pages - Add llms.txt directives to both HTML (visually-hidden element) and markdown (blockquote) versions of every docs page - Improve markdown/HTML parity: include page descriptions in markdown, mark HTML-only page chrome with data-markdown-ignore - Slim llms-full.txt from ~7 MB to ~4.5 MB by excluding legacy /orm/v6 and deprecated product pages; absolutize in-body links so they resolve - Serve an agentskills.io-format skill at /skill.md, /docs/skill.md and /.well-known/agent-skills/* - Serve MCP discovery documents at /.well-known/mcp, /.well-known/mcp.json and server cards, pointing at the existing https://mcp.prisma.io/mcp Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (1)
WalkthroughThe change adds AI-readable documentation links and markdown processing, reorganizes LLM indexes and feeds, introduces Prisma agent-skill and MCP discovery endpoints, and adds automated agent-readiness validation. ChangesAI documentation and agent discovery
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Agent
participant WellKnownRoute
participant Builder
Agent->>WellKnownRoute: Request skill or MCP metadata
WellKnownRoute->>Builder: Generate payload
Builder-->>WellKnownRoute: Return JSON or Markdown
WellKnownRoute-->>Agent: Send typed response
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
|
The latest updates on your projects. Learn more about Argos notifications ↗︎
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/docs/src/lib/get-llm-text.ts`:
- Around line 201-210: Update absolutizeInBodyLinks to preserve inline backtick
code spans as well as fenced blocks before rewriting links, ensuring markdown
examples such as [label](/path) inside inline code remain unchanged while actual
links continue through resolveInBodyHref.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: bbdd4c54-76b7-4fd1-b989-2e2283a2db85
📒 Files selected for processing (18)
apps/docs/src/app/(docs)/(default)/[[...slug]]/page.tsxapps/docs/src/app/.well-known/mcp.json/route.tsapps/docs/src/app/.well-known/mcp/route.tsapps/docs/src/app/llms-full.txt/route.tsapps/docs/src/app/llms.txt/route.tsapps/docs/src/app/skill.md/route.tsapps/docs/src/lib/get-llm-text.tsapps/docs/src/lib/llm-markdown.tsapps/docs/src/lib/llms.tsapps/docs/src/lib/mcp-discovery.tsapps/site/src/app/.well-known/agent-skills/index.json/route.tsapps/site/src/app/.well-known/agent-skills/prisma/SKILL.md/route.tsapps/site/src/app/.well-known/mcp.json/route.tsapps/site/src/app/.well-known/mcp/route.tsapps/site/src/app/.well-known/mcp/server-card.json/route.tsapps/site/src/app/.well-known/mcp/server-cards.json/route.tsapps/site/src/app/skill.md/route.tsapps/site/src/lib/agent-skills.ts
…-ready skill Make the agent-score setup self-maintaining: - Extract llms.txt / section-index generation into pure builders in lib/llms.ts (byte-identical output) so routes and checks share one source of truth; llms-full page filtering exported as getLLMsFullPages - Add scripts/lint-agent-ready.ts (pnpm --filter docs lint:agent-ready), wired into turbo and the Links CI workflow: enforces size budgets, full page coverage, catch-all creep, markdown directives, common-query link resolution, llms-full exclusions, and skill/MCP content sanity - Add .claude/skills/docs-agent-ready with invariants, endpoint file map, and playbooks for adding new docs areas Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/docs/scripts/lint-agent-ready.ts`:
- Around line 232-240: Update the MCP discovery validation around
mcpDiscoveryDocument to validate both docs and site discovery payloads, not just
the docs builder. For every server, verify the canonical server URL, transport,
and authentication fields against the expected values, and fail with actionable
field-specific messages when either payload is incomplete or incorrect; retain
the existing pass behavior only when both roots are fully valid.
- Around line 136-168: Update the Check 5 validation loop to cover every indexed
page in the relevant sections rather than sampling only the first page. For each
page, validate that the markdown directive is the first required content and
that the rendered HTML has the directive as its first child, using the existing
page text/rendering utilities and reporting failures per page; retain the
description check for each page.
- Around line 28-30: Update the baseUrl declaration in the lint-agent-ready
script to always use the fixed production URL https://www.prisma.io, removing
the NEXT_PUBLIC_PRISMA_URL environment-variable fallback so byte-budget
measurements remain stable across environments.
In `@apps/docs/src/lib/agent-skill.ts`:
- Around line 52-56: Update the setup instructions around the `npx create-db`
command to require explicit user consent immediately before provisioning the
hosted Prisma Postgres database. Make clear that the agent must pause and obtain
confirmation before executing this external resource-creation step, while
preserving the subsequent connection-string configuration guidance.
- Around line 35-43: Update the TypeScript project setup instructions before
`prisma init` to include `npx tsc --init` and configure `tsconfig.json` with
`module: "ESNext"`, `moduleResolution: "bundler"`, and `target: "ES2023"`, plus
set `"type": "module"` in `package.json`.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 16eb7c99-7e57-49e5-8f04-50c19851a619
📒 Files selected for processing (12)
.claude/skills/README.md.claude/skills/docs-agent-ready/SKILL.md.github/workflows/links.ymlapps/docs/package.jsonapps/docs/scripts/lint-agent-ready.tsapps/docs/src/app/llms-full.txt/route.tsapps/docs/src/app/llms.txt/route.tsapps/docs/src/app/llms/[...slug]/route.tsapps/docs/src/app/skill.md/route.tsapps/docs/src/lib/agent-skill.tsapps/docs/src/lib/llms.tsturbo.json
- Protect inline code spans (not just fenced blocks) when absolutizing in-body markdown links - Pin the guard's byte-budget measurements to the production base URL - Guard now asserts the directive blockquote positionally on all 621 pages, checks descriptions across all pages, and adds a source-level guard for the hidden HTML directive - Field-level MCP discovery validation for both the docs and site payloads - Skill workflows now mirror the quickstart's ESM/TypeScript setup steps verbatim, and require explicit user approval before provisioning a hosted database with npx create-db Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/docs/src/lib/llm-markdown.ts`:
- Around line 409-419: Update the placeholder generation and restore logic in
the inline-code protector around the restore method so restoration cannot
replace sentinel text reintroduced by an earlier span. Coordinate token
generation across both inline and fenced-code protectors in get-llm-text.ts,
using collision-free unique placeholders rather than sequential per-protector
indices. Add regression coverage for sentinel-looking text inside inline and
fenced code spans, preserving literal content during restoration.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 8c2f4193-5345-4f4b-97ef-33057c3a7070
📒 Files selected for processing (5)
apps/docs/scripts/lint-agent-ready.tsapps/docs/src/lib/agent-skill.tsapps/docs/src/lib/get-llm-text.tsapps/docs/src/lib/llm-markdown.tsapps/site/src/lib/agent-skills.ts
🚧 Files skipped from review as they are similar to previous changes (3)
- apps/docs/src/lib/agent-skill.ts
- apps/site/src/lib/agent-skills.ts
- apps/docs/src/lib/get-llm-text.ts
Prisma Next content (/next, /orm/next, /guides/next, /cli/next — 68 pages) was split across the orm/guides/cli indexes and the root catch-all; Compute was folded into the platform section. Both now get their own section index (llms/next.txt, llms/compute.txt) with excludePrefixes keeping the sibling sections version-clean, plus a Prisma Next common-query entry. Root catch-all drops from 12 to 3 pages; all sections remain under the 50k budget. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
apps/docs/src/lib/llms.ts (1)
355-448: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winAdd regression coverage for section partitioning.
Test that
/orm/next,/guides/next,/cli/next,/orm/v6, and unmatched pages appear only in their intended feed. The existing coverage check can pass even if a page is duplicated across a parent and Prisma Next section.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/docs/src/lib/llms.ts` around lines 355 - 448, Add regression tests for buildLLMsIndexContent, buildLLMsSectionContent, and getLLMsFullPages covering /orm/next, /guides/next, /cli/next, /orm/v6, and an unmatched page. Assert each page appears only in its intended feed, including that Prisma Next pages are not duplicated in parent sections, /orm/v6 is excluded from the full feed, and unmatched pages appear only in the index’s other-pages section.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@apps/docs/src/lib/llms.ts`:
- Around line 355-448: Add regression tests for buildLLMsIndexContent,
buildLLMsSectionContent, and getLLMsFullPages covering /orm/next, /guides/next,
/cli/next, /orm/v6, and an unmatched page. Assert each page appears only in its
intended feed, including that Prisma Next pages are not duplicated in parent
sections, /orm/v6 is excluded from the full feed, and unmatched pages appear
only in the index’s other-pages section.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: c92d7302-8726-4f75-a83d-c889f05946c6
📒 Files selected for processing (1)
apps/docs/src/lib/llms.ts
Sequential per-token replace could corrupt output when protected code literally contained sentinel-looking text. Both protectors now use PUA-character delimiters lengthened deterministically until absent from the input, with a single-pass regex restore, and the guard gains a protector round-trip check with adversarial inputs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@coderabbitai resolve |
Contributor
Author
|
@coderabbitai review |
Contributor
✅ Action performedReview finished.
|
Contributor
✅ Action performedComments resolved and changes approved. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Mintlify's agent-score audit of www.prisma.io/docs scored 85/100 (B) with failing checks in Content Discoverability, Markdown Parity, Full Content Discoverability, Agent Skills, and MCP Server discovery. This PR addresses all of them so AI agents (Claude, ChatGPT, Cursor, etc.) can reliably discover, crawl, and trust our docs.
Key changes
1.
llms.txtwas too big to be usable (116k chars → ~7.5k)Agents truncate
llms.txtabove 100k chars, so ours was effectively cut off mid-file. The root index no longer dumps all 631 pages; it now links to per-area section indexes (/docs/llms/orm.txt,/docs/llms/postgres.txt, … 10 sections total, each under 50k chars). A coverage helper guarantees every page is reachable from the root index — pages that match no section are listed directly under "Other pages."2. Agents visiting a page had no pointer to the index
Every docs page now carries an
llms.txtdirective in both formats:/docs/llms.txtand the page's.mdURL.mdpage3. Markdown pages were missing content the HTML had
The page description (rendered in HTML) is now included in the markdown output, and HTML-only chrome (copy buttons, "Edit on GitHub" footer) is marked
data-markdown-ignoreso parity checkers don't count it against us.4.
llms-full.txtwas 7 MB with broken links (→ ~4.5 MB, links fixed)The full-content feed now excludes legacy
/orm/v6pages (191 pages, still available per-page via.md) and deprecated Accelerate/Optimize pages (already excluded from the index, but leaking into the full feed). In-body root-relative links are now resolved to absolute URLs — before, ~2,500 links would lose the/docsprefix when resolved against the feed URL. After the fix, 4,591 of 4,597 internal links resolve (the 6 remaining point at a pre-existing broken link target in Accelerate content).5. No agent skill definition existed (score: 0/100)
An agentskills.io-format skill is now served at
/skill.md,/docs/skill.md, and/.well-known/agent-skills/*. Content is grounded strictly in existing docs (quickstart + MCP server pages): the core CLI workflow, Prisma Postgres setup, MCP config, and where agents should read docs.6. Our MCP server wasn't discoverable (score: 0/100)
Prisma already runs a real MCP server at
mcp.prisma.io/mcp— checkers just couldn't find it. Standard discovery documents are now served at/.well-known/mcp,/.well-known/mcp.json, and/.well-known/mcp/server-card(s).json, listing the 11 documented tools.Impact
llms.txtsizellms-full.txt/skill.md+ well-known endpoints/.well-known/mcp*→ mcp.prisma.ioVerification
pnpm --filter docs types:checkandpnpm --filter site types:checkpass🤖 Generated with Claude Code
Summary by CodeRabbit
llms.txtcross-links in the docs renderer for better offline/AI access.llms.txt/llms-full.txtgeneration with clearer legacy ORM v6 exclusions and expanded, consistent section coverage.Keeping the score from regressing (added in follow-up commit)
The setup is now self-maintaining:
llms.txtroot index and section files are generated by pure functions inlib/llms.ts; the route handlers are thin wrappers. New content is picked up automatically via section prefixes, and pages matching no section surface in the root "Other pages" list, so nothing can silently fall out of the index.pnpm --filter docs lint:agent-ready(new step in the Links workflow) fails the build if any invariant regresses: size budgets (root < 50k, each section < 50k with early warnings), full page coverage, catch-all creep (> 25 uncategorized pages → time to add a section), the markdown llms.txt directive, stale common-query links, v6/deprecated pages leaking intollms-full.txt, or missing skill.md / MCP-discovery fields. It also prints a size-budget table with per-file headroom for reviewers..claude/skills/docs-agent-ready/SKILL.mddocuments the invariants, the endpoint→file map for both apps, and playbooks (adding a docs area, splitting an over-budget section, page-chrome rules, keeping skill.md in sync with the quickstart/MCP docs).Current guard output: 0 failures, 0 warnings — largest file (
llms/orm.txt) at 38k of its 50k budget.