Skip to content

perf(mdx): Optimize frontmatter scanning and local bundle cache #17706

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
a-hariti wants to merge 4 commits into from
Closed

perf(mdx): Optimize frontmatter scanning and local bundle cache #17706

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4eb4aa9
perf(mdx): Reuse optimized frontmatter scanner
a-hariti May 10, 2026
4126202
perf(mdx): Enable local production bundle cache
a-hariti May 10, 2026
b3da3fe
ref(mdx): Remove duplicated frontmatter scanner
a-hariti May 10, 2026
2d3ced2
fix(mdx): Write bundle cache atomically
a-hariti May 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions app/[[...path]]/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,14 @@ import {
getPreviousNode,
nodeForPath,
} from 'sentry-docs/docTree';
import {getMDXComponent} from 'sentry-docs/getMDXComponent';
import {isDeveloperDocs} from 'sentry-docs/isDeveloperDocs';
import {
getDevDocsFrontMatter,
getDocsFrontMatter,
getFileBySlugWithCache,
getVersionsFromDoc,
} from 'sentry-docs/mdx';
} from 'sentry-docs/frontmatter';
import {getMDXComponent} from 'sentry-docs/getMDXComponent';
import {isDeveloperDocs} from 'sentry-docs/isDeveloperDocs';
import {getFileBySlugWithCache} from 'sentry-docs/mdx';
import {mdxComponents} from 'sentry-docs/mdxComponents';
import {PageType} from 'sentry-docs/metrics';
import {setServerContext} from 'sentry-docs/serverContext';
Expand Down
2 changes: 1 addition & 1 deletion scripts/algolia.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ import fs from 'fs';
import {join} from 'path';
import {isDeveloperDocs} from 'sentry-docs/isDeveloperDocs';

import {getDevDocsFrontMatter, getDocsFrontMatter} from '../src/mdx';
import {getDevDocsFrontMatter, getDocsFrontMatter} from '../src/frontmatter';
import {FrontMatter} from '../src/types';

// This is the path to the static files generated by Next.js for the app directory
Expand Down
30 changes: 25 additions & 5 deletions src/frontmatter.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,13 +54,35 @@ export function getDocsFrontMatter(): Promise<FrontMatter[]> {
return getDocsFrontMatterCache;
}

/**
* Collect all available versions for a given document path.
*/
Comment on lines 54 to +59
Copy link
Copy Markdown
Contributor

@sentry sentry Bot May 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The new getDocsFrontMatter() and getDevDocsFrontMatter() functions lack the Vercel runtime guard, causing expensive filesystem scans in production serverless environments, which was previously prevented.
Severity: HIGH

Suggested Fix

Reintroduce the Vercel runtime guard in getDocsFrontMatter() and getDevDocsFrontMatter() in src/frontmatter.ts. The check should prevent execution in a Vercel production environment (e.g., if (process.env.VERCEL && !process.env.CI && process.env.NODE_ENV !== 'development')) by rejecting the promise or throwing an error, similar to the logic in the old src/mdx.ts file.

Prompt for AI Agent 
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: src/frontmatter.ts#L54-L59

Potential issue: The refactored functions `getDocsFrontMatter()` and
`getDevDocsFrontMatter()` in `src/frontmatter.ts` are missing a runtime guard that was
present in the previous implementation. This guard prevented expensive filesystem
scanning operations on Vercel production deployments. Without it, runtime calls to these
functions, such as from the `Page` component in `app/[[...path]]/page.tsx`, will trigger
a recursive scan of the entire docs folder. This can lead to `EMFILE` errors, severe
performance degradation, or failures on Vercel's read-only filesystem, issues the
original guard was designed to prevent.

Also affects:

  • src/frontmatter.ts:62~67

Did we get this right? 👍 / 👎 to inform future reviews.

export const getVersionsFromDoc = (frontMatter: FrontMatter[], docPath: string) => {
const versions = frontMatter
.filter(({slug}) => {
return (
slug.includes(VERSION_INDICATOR) &&
slug.split(VERSION_INDICATOR)[0] === docPath.split(VERSION_INDICATOR)[0]
);
})
.map(({slug}) => {
const segments = slug.split(VERSION_INDICATOR);
return segments[segments.length - 1];
});

return [...new Set(versions)];
};

async function getDocsFrontMatterUncached(): Promise<FrontMatter[]> {
const docsPath = path.join(root, 'docs');
const files = await getAllFilesRecursively(docsPath);
const allFrontMatter: FrontMatter[] = [];

// Create a Set of existing file paths for fast lookups
const existingFilesSet = new Set(files);
const existingFilesLowercaseSet = new Set(files.map(file => file.toLowerCase()));
const hasExistingFile = (file: string) =>
existingFilesSet.has(file) || existingFilesLowercaseSet.has(file.toLowerCase());

// First, collect all non-common files
await Promise.all(
Expand Down Expand Up @@ -191,10 +213,8 @@ async function getDocsFrontMatterUncached(): Promise<FrontMatter[]> {
.replace(/\/common\//, '/');
// Check if the file exists using the pre-computed Set
const noFrontMatter =
!existingFilesSet.has(path.join(docsPath, slug)) &&
!existingFilesSet.has(
path.join(docsPath, slug.replace('/index.mdx', '.mdx'))
);
!hasExistingFile(path.join(docsPath, slug)) &&
!hasExistingFile(path.join(docsPath, slug.replace('/index.mdx', '.mdx')));
if (noFrontMatter) {
let frontmatter = commonFile.frontmatter;
if (subpath === 'index.mdx') {
Expand Down Expand Up @@ -278,7 +298,7 @@ async function getDocsFrontMatterUncached(): Promise<FrontMatter[]> {
subpath
);
// Check if file exists using pre-computed Set
if (existingFilesSet.has(path.join(docsPath, slug))) {
if (hasExistingFile(path.join(docsPath, slug))) {
return;
}

Expand Down
3 changes: 2 additions & 1 deletion src/mdx.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import {describe, expect, test} from 'vitest';

import {addVersionToFilePath, getVersionedIndexPath, getVersionsFromDoc} from './mdx';
import {getVersionsFromDoc} from './frontmatter';
import {addVersionToFilePath, getVersionedIndexPath} from './mdx';
import {FrontMatter} from './types';

const mockFm: FrontMatter[] = [
Expand Down
Loading
Loading