Skip to content

docs site: any single-segment path containing a dot renders the homepage with 200 (soft-404 class) #12233

Description

@os-zhuang

One-liner

/foo.txt, /ads.txt, /sitemap_index.xml, /security.txt — every single-segment path with a dot in it returns 200 with the full homepage. Paths without a dot 404 correctly. So the site publishes an unbounded set of duplicate homepages at exactly the URLs crawlers probe by default.

Measured (local dev server, same routing in production)

/foo.txt               200  title=<title>ObjectStack — Apps small enough for AI to hold whole.</title>
/ads.txt               200  title=<title>ObjectStack — Apps small enough for AI to hold whole.</title>
/sitemap_index.xml     200  title=<title>ObjectStack — Apps small enough for AI to hold whole.</title>
/security.txt          200  title=<title>ObjectStack — Apps small enough for AI to hold whole.</title>
/this-page-does-not-exist   404   ← no dot, correct

Root cause

apps/docs/proxy.ts matcher excludes dotted paths from locale rewriting:

matcher: ['/((?!api|_next/static|_next/image|favicon.ico|.*\\..*).*)'],

so /robots.txt skips the proxy and falls straight through to the single-segment dynamic route app/[lang]/page.tsx, which matches with lang = "robots.txt" and renders HomePage() — it never checks whether lang is a real locale. A dotless path is rewritten to /en/<path>, becomes two segments, matches nothing, and 404s properly.

Expected

[lang] accepts only the locales in lib/i18n.ts. Either export const dynamicParams = false alongside the existing generateStaticParams, or an explicit if (!i18n.languages.includes(lang)) notFound() in app/[lang]/layout.tsx — whichever the Next 16 app router honours for both the static and the dev path. Verify both, do not assume.

Why it matters

Soft-404s at this scale burn crawl budget and split ranking signal across junk URLs; and the two paths that matter most (/robots.txt, /sitemap.xml) are in the affected set, so this card and the robots/sitemap card must both land for either to be worth anything.

Acceptance

  • /foo.txt, /ads.txt, /security.txt, /anything.html all return 404
  • /robots.txt and /sitemap.xml still resolve to their real routes (regression check against the sibling card)
  • /docs, /docs/<any real page>, /blog unchanged
  • a test or gate pins the behaviour so the catch-all cannot silently return

Source

Found in an SEO review of the docs site (apps/docs) run on 2026-08-25, measured against the local dev server and against production. The canonical origin is https://objectstack.ai — maintainer ruling recorded in #10659:

这个仓的文档站规范 URL 是 https://objectstack.ai

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions