vfs: integrate with CJS and ESM module loaders by mcollina · Pull Request #63653 · nodejs/node

mcollina · 2026-05-30T05:41:48Z

Makes require() and import resolve files served by node:vfs. Before this PR, mounted VFS files were only visible through fs.*; the loaders went straight to the real filesystem.

Design

vfs.mount() takes no arguments and returns the reserved absolute mount point of the instance: ${os.devNull}/vfs/<layerId> (for example /dev/null/vfs/0). os.devNull is a character device on POSIX and a device-namespace path on Windows; neither can have child filesystem entries, so no real path can ever exist under this root.

Everything about ownership is decidable from the path alone:

Dispatch: one path normalization, one prefix comparison against the namespace root, one map lookup on the layer id. O(1) in the number of mounted layers — the benchmark (benchmark/vfs/bench-fs-dispatch.js) reports flat per-call latency across 1..10 mounted layers.
Real-fs paths short-circuit with a single prefix check.
Unmount purges loader caches by prefix-scanning the mount point — no per-VFS ownership tracking is needed, and no other-VFS or real-fs entries are ever touched.
Symlink correctness: realpathSync of a VFS entry always resolves to another path under the same mount point, so cache entries hidden behind symlinks are captured by the prefix scan.
Dynamic-import identity: resolved URLs are plain file: URLs under the mount point; import(import.meta.resolve(x)) re-hits the same module job.

Two instances mounting simultaneously never collide (each gets its own layer-<id> segment), so there is no overlap validation and no ordering hazard between mounts.

Loader integration

Toggleable wrappers in the loaders. Null fast-path when no VFS is mounted; otherwise the VFS answers stat / readFile / realpath / legacyMainResolve / getFormatOfExtensionlessFile and the four package.json C++-binding calls.

Module identity follows the path: __filename, module.filename, and import.meta.url are the plain absolute path (or file: URL) of the module under the mount point — no synthetic decorations.

Review guide

Refs

The reserved-namespace design follows the "no interference with valid paths in the file system" requirement from the SEA VFS requirements doc.

Out of scope

SEA + VFS, overlay/stacking of multiple VFS layers under one prefix, migrating the C++ package_configs_ cache, broader permission-model integration.

nodejs-github-bot · 2026-05-30T05:41:54Z

Review requested:

@nodejs/loaders

codecov · 2026-05-30T06:50:02Z

Codecov Report

❌ Patch coverage is 92.03445% with 74 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.23%. Comparing base (0e2126d) to head (0d5fb61).
⚠️ Report is 112 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/vfs/setup.js	87.52%	57 Missing ⚠️
src/node_modules.cc	77.96%	4 Missing and 9 partials ⚠️
lib/internal/vfs/file_system.js	97.18%	2 Missing ⚠️
lib/internal/vfs/router.js	96.61%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #63653      +/-   ##
==========================================
- Coverage   92.04%   90.23%   -1.81%     
==========================================
  Files         381      741     +360     
  Lines      169208   242261   +73053     
  Branches    25926    45640   +19714     
==========================================
+ Hits       155746   218611   +62865     
- Misses      13174    15197    +2023     
- Partials      288     8453    +8165

Files with missing lines	Coverage Δ
lib/fs.js	`98.36% <100.00%> (+4.13%)`	⬆️
lib/internal/modules/cjs/loader.js	`98.16% <100.00%> (+18.59%)`	⬆️
lib/internal/modules/esm/get_format.js	`95.20% <100.00%> (+20.89%)`	⬆️
lib/internal/modules/esm/load.js	`91.47% <100.00%> (+8.22%)`	⬆️
lib/internal/modules/esm/resolve.js	`99.03% <100.00%> (+12.10%)`	⬆️
lib/internal/modules/helpers.js	`98.80% <100.00%> (+8.01%)`	⬆️
lib/internal/modules/package_json_reader.js	`99.47% <100.00%> (+11.87%)`	⬆️
src/node_modules.h	`100.00% <ø> (ø)`
lib/internal/vfs/file_system.js	`99.54% <97.18%> (+0.33%)`	⬆️
lib/internal/vfs/router.js	`97.64% <96.61%> (+12.86%)`	⬆️
... and 2 more

... and 503 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

mcollina · 2026-05-30T10:27:16Z

@joyeecheung take a look, should be easier to review.

@jsumners-nr

Restore the "DO NOT depend on the patchability" warnings in esm/load.js and esm/resolve.js that were dropped along with the fs imports. The warning still applies; it now also points at node:vfs as one of the formal hook mechanisms callers should reach for instead. Addresses review feedback from @jsumners-nr in nodejs#63653

nodejs-github-bot · 2026-06-03T08:04:09Z

CI: https://ci.nodejs.org/job/node-test-pull-request/73901/

nodejs-github-bot · 2026-06-03T11:52:41Z

CI: https://ci.nodejs.org/job/node-test-pull-request/73902/

joyeecheung

A design question recently occurred to me: have we explored the versioning of the mounting?

mcollina · 2026-06-03T12:40:18Z

A design question recently occurred to me: have we explored the versioning of the mounting?

what do you mean? You mean multiple vfs layers on top of each other?

joyeecheung · 2026-06-03T13:02:48Z

what do you mean? You mean multiple vfs layers on top of each other?

For the stacks to have some kind of version number/ID to identify the current status?

BTW I just noticed that there's no mention of unmount() and mount() in the VFS docs..

mcollina · 2026-06-03T20:10:47Z

BTW I just noticed that there's no mention of unmount() and mount() in the VFS docs..

I did purge them when doing the splitting; I forgot to bring them back. I'll add them to this PR.

mcollina · 2026-06-03T20:11:45Z

For the stacks to have some kind of version number/ID to identify the current status?

No but we totally should.

Each VirtualFileSystem now exposes a per-process monotonically increasing `layerId`, assigned at construction. The id is stable across mount/unmount cycles for the lifetime of the instance and surfaces in: - debug() output for register / deregister so the layer stack is visible when NODE_DEBUG=vfs is enabled; - the overlap ERR_INVALID_STATE message, which now names the layer ids of the conflicting mounts. The id is the building block for tagging cache entries with the owning VFS, which a follow-up will use to replace the global loader-cache flush in deregisterVFS with a scoped purge. Refs: nodejs#63653 Signed-off-by: Matteo Collina <hello@matteocollina.com>

Replace the global loader-cache flush in deregisterVFS with a scope-purge that only drops entries owned by the unmounting VFS. Per-layer ownership is determined two ways: - For CJS-style filename-keyed caches (Module._cache, Module._pathCache, the CJS stat cache, the helpers.js realpath cache, and the package.json caches) entries are filtered with `vfs.shouldHandle(filename)`. __filename stays a clean absolute path so user code that does `path.dirname(__filename)` or similar is unaffected. - For the ESM cascaded loader's loadCache, entries are tagged at resolve time: when finalizeResolution() detects the resolved path is VFS-owned (via the new loaderGetLayerForPath hook), it appends `?vfs-layer=<id>` to the URL. The tag surfaces in `import.meta.url`, matching the cache-busting pattern used by HMR tooling. On deregister, entries whose URL carries the tag for the unmounting layer are deleted. Multi-mount setups no longer pay the cross-VFS cache-warmup penalty when a single VFS unmounts, and ESM modules loaded from a VFS become reachable for purge instead of leaking forever in the cascaded loader. New helpers exposed for VFS: - cjs/loader.js: clearStatCacheForVFS - helpers.js: purgeRealpathCacheForVFS, loaderGetLayerForPath - package_json_reader.js: purgePackageJSONCacheForVFS Adds test-vfs-scoped-cache-purge covering both the multi-mount isolation and the import.meta.url tag visibility. Refs: nodejs#63653 Signed-off-by: Matteo Collina <hello@matteocollina.com>

mcollina · 2026-06-05T06:47:09Z

@joyeecheung PTAL

`mount()` no longer accepts an argument. The user-supplied prefix served no purpose beyond a cosmetic label - actual layer identity comes from the per-instance `layerId`, so distinct instances never shared a mount point even when they passed the same prefix. Removing the argument eliminates the input validation, the escape check for `..` segments, and the API-shape doubt about what a "logical prefix" means. The `layer-` prefix on the layer segment of the mount path is likewise gone: mount points are now `${os.devNull}/vfs/<id>` (for example `/dev/null/vfs/0`) instead of `${os.devNull}/vfs/layer-0/<prefix>`. The parser in `getLayerIdFromPath` simplifies to reading the digit segment immediately after the VFS root. Docs, tests and benchmark adapted. The dispatch benchmark still reports flat per-call latency across 1..10 mounted layers. Signed-off-by: Matteo Collina <hello@matteocollina.com>

mcollina · 2026-07-08T08:02:10Z

@joyeecheung ping.

nodejs-github-bot · 2026-07-08T19:59:03Z

CI: https://ci.nodejs.org/job/node-test-pull-request/74678/

joyeecheung

High level direction looks much better and more aligned with https://github.com/nodejs/single-executable/blob/main/docs/virtual-file-system-requirements.md#no-interference-with-valid-paths-in-the-file-system now, thanks, though I think the implementation still leaves quite a bit hard-codes and repeitions that should be cleaned up before landing to avoid taxing future changes to module loading with extra maintenance complexity & divergence..

joyeecheung · 2026-07-12T16:59:49Z

+ * @returns {object|undefined}
+ */
+function loaderReadPackageJSON(jsonPath, isESM, base, specifier) {
+  if (readPackageJSONOverride !== null) {


Helpers like this look like a lot of maintainance burden - every time someone adds a new binding, they have to update all these, and if they end up updating some internal parameters as part of a semver major change, it can cause another wave of churns for backports. Can you just use a helper to forward them all? Something like

function wrapLoaderMethod(originalFn) { return function(...args) { const override = overrideMap.get(originalFn); if (override) return ReflectApply(override, this, args); return ReflectApply(originalFn, this, args); } }

(if a benchmark shows it actually matters for perf we could even switch on originalFn.length and build arity-based dispatches like processTicksAndRejections but I suspect the overhead is negligible for this path in the bigger picture). This would also shrink the diff here quite a bit.

joyeecheung · 2026-07-12T17:04:09Z

@@ -174,11 +219,26 @@ class VirtualFileSystem {
   * @returns {boolean}
   */
  shouldHandle(inputPath) {


I think this is dead code now?

joyeecheung · 2026-07-12T17:07:40Z

+        const parsed = JSONParse(content);
+        return { vfs, pjsonPath, parsed, sentinel: pjsonPath };
+      } catch {
+        // SyntaxError or other errors, continue walking


This diverges from GetPackageJSON which would throw ERR_INVALID_PACKAGE_CONFIG on the first syntactically malformed package.json instead of keep walking, I think we should at least align with the actual behavior here or otherwise this can introduce very subtle bugs for users that create a tree into VFS and one of the package.json end up corrupted somehow.

What would be even better is to simply split the C++ GetPackageJSON into read + parse, and let the VFS side reuse the parsing logic, that also eliminates the duplication of serializePackageJSON etc and would make use of partial parsing with simdjson (we could technically just read the pacakge.json as buffer and avoid the UTF8 encoding cost too) instead of a wasteful full parse with JSON.parse.

joyeecheung · 2026-07-12T17:14:07Z

+        // mount. Treat that as JS (the caller will surface the real
+        // error when it later tries to load source). Propagate every
+        // other code (EACCES, ELOOP, etc).
+        if (e?.code === 'ENOENT') return 0; // EXTENSIONLESS_FORMAT_JAVASCRIPT


The original version normalizes all errors for EXTENSIONLESS_FORMAT_JAVASCRIPT, not just ENOENT.

joyeecheung · 2026-07-12T18:14:44Z

+      if (content && content.length >= 4 &&
+          content[0] === 0x00 && content[1] === 0x61 &&
+          content[2] === 0x73 && content[3] === 0x6d) {
+        return 1; // EXTENSIONLESS_FORMAT_WASM


This shouldn't hard-code and should return internalBinding('constants').EXTENSIONLESS_FORMAT_WASM etc.

joyeecheung · 2026-07-12T18:42:04Z

+      // 7-9: try pkgPath + ./index.ext
+      const mainExts = ['', '.js', '.json', '.node',
+                        '/index.js', '/index.json', '/index.node'];
+      const indexExts = ['./index.js', './index.json', './index.node'];


Do we need to duplicate all these? It can go out of sync easily. legacyMainResolveExtensions is already (inappropriately) located in esm/resolve.js, I think we can simply share the arrays and do something like

legacyMainResolve(pkgPath, main, base) { if (findVFS(pkgPath) === null) return undefined; for (let i = 0; i < legacyMainResolveExtensions.length; i++) { const shouldResolveByMain = i <= kMaxResolvedByMainIndex; // kResolvedByMainIndexNode if (shouldResolveByMain && !main) continue; const prefix = shouldResolveByMain ? main : ''; const candidate = join(pkgPath, prefix + legacyMainResolveExtensions[i]); if (findVFSForStat(candidate)?.result === 0) return i; } const initial = main ? join(pkgPath, main) : join(pkgPath, 'index.js'); throw new ERR_MODULE_NOT_FOUND(initial, base, undefined); }

joyeecheung · 2026-07-12T18:55:34Z

+        StringPrototypeEndsWith(currentDir, '\\node_modules')) {
+      break;
+    }
+    const pjsonPath = join(currentDir, 'package.json');


Since dirname/join preserves the prefix if it's absolute, I think can just do something like if (!vfs.shouldHandleNormalized(currentDir)) break; to save on the repeated normalization in stat/read etc.

joyeecheung · 2026-07-12T19:03:33Z

+      } else {
+        filePath = resolved;
+      }
+      const vfs = findVFS(filePath);


Nit: findVFS discards the normalized filePath which is a bit wasteful, if it returns that back we can reuse that in e.g. findVFSPackageJSON.

Co-authored-by: Joyee Cheung <joyeec9h3@gmail.com>

`VirtualFileSystem#shouldHandle` had no remaining callers: dispatch goes through `shouldHandleNormalized` and every code path outside dispatch was rewritten to `findVFS` after the reserved-namespace refactor. Two router.js exports were also unreferenced: `getVfsRoot` (used only internally by `getNormalizedVfsRoot` and `getLayerRoot`) and the re-export of `path.isAbsolute` as `isAbsolutePath`. The corresponding test-vfs-router assertions and the `isAbsolute` local binding on the `require('path')` destructure go with them. Signed-off-by: Matteo Collina <hello@matteocollina.com>

Signed-off-by: Matteo Collina <hello@matteocollina.com>

This reverts commit a95717a.

Signed-off-by: Matteo Collina <hello@matteocollina.com>

mcollina · 2026-07-13T16:59:36Z

@joyeecheung done

nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels May 30, 2026

mcollina requested review from ShogunPanda, jasnell and joyeecheung May 30, 2026 10:26

ShogunPanda approved these changes May 30, 2026

View reviewed changes

jsumners-nr reviewed Jun 1, 2026

View reviewed changes

Comment thread lib/internal/modules/esm/load.js

Comment thread lib/internal/modules/esm/resolve.js

mcollina force-pushed the vfs-module-loader-integration branch from 6321e08 to 51b033a Compare June 1, 2026 15:54

ShogunPanda approved these changes Jun 2, 2026

View reviewed changes

mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Jun 3, 2026

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jun 3, 2026

joyeecheung reviewed Jun 3, 2026

View reviewed changes

Comment thread lib/internal/vfs/setup.js Outdated

mcollina force-pushed the vfs-module-loader-integration branch from 51b033a to 294a19c Compare June 4, 2026 07:23

coderabbitai Bot mentioned this pull request Jun 12, 2026

build: migrate from Workers to Docker jsdelivr/globalping-dash#204

Merged

arcanis mentioned this pull request Jun 15, 2026

module: add --experimental-strip-private-modules #63869

Open

ShogunPanda approved these changes Jun 15, 2026

View reviewed changes

mcollina force-pushed the vfs-module-loader-integration branch from 75bb5c7 to 99a5a5c Compare June 15, 2026 09:00

mcollina added 2 commits July 7, 2026 13:21

doc: remove em dashes from VFS docs

de17e81

ShogunPanda approved these changes Jul 8, 2026

View reviewed changes

mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Jul 8, 2026

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 8, 2026

This comment was marked as outdated.

Sign in to view

trivikr requested a review from joyeecheung July 9, 2026 16:55

joyeecheung reviewed Jul 12, 2026

View reviewed changes

mcollina and others added 18 commits July 13, 2026 02:15

Update lib/internal/vfs/setup.js

023516e

Co-authored-by: Joyee Cheung <joyeec9h3@gmail.com>

Update lib/internal/vfs/setup.js

e378435

Co-authored-by: Joyee Cheung <joyeec9h3@gmail.com>

vfs: use internal EXTENSIONLESS_FORMAT_* constants

2203b59

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: raise ERR_INVALID_PACKAGE_CONFIG for CJS too

f2f7b04

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: default all read errors in getFormatOfExtensionlessFile to JS

2462d17

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: return normalized path from findVFS

902254d

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: inline isUnderMountPoint and use shouldHandleNormalized

2716996

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs,esm: share legacyMainResolveExtensions arrays

6fafe07

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: add cleanForVfsPrefix helper for cache purges

6e3f1b9

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: use indexed for loops in cleanForVfsPrefix

a95717a

Signed-off-by: Matteo Collina <hello@matteocollina.com>

Revert "vfs: use indexed for loops in cleanForVfsPrefix"

ffd68a2

This reverts commit a95717a.

vfs: fold loader wrappers into wrapLoaderMethod factory

7b6e3de

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: fix lint on wrapLoaderMethod curly braces

2e463ca

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: throw ERR_INVALID_PACKAGE_CONFIG on malformed ancestor pjson

5b71a94

Signed-off-by: Matteo Collina <hello@matteocollina.com>

src: split GetPackageJSON and expose parsePackageJSON binding

30d5115

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: use native parsePackageJSON binding, drop serializePackageJSON

337c130

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: read pjson as Buffer, skip UTF-8 decode

0d5fb61

Signed-off-by: Matteo Collina <hello@matteocollina.com>

mcollina force-pushed the vfs-module-loader-integration branch from d1d72c3 to 0d5fb61 Compare July 13, 2026 13:59

#	File	Role
1	`lib/internal/vfs/router.js`	Reserved-namespace helpers: `getVfsRoot`, `getLayerRoot`, `getLayerIdFromPath`.
2	`lib/internal/vfs/file_system.js`	`mount()` returns the layer's reserved mount point.
3	`lib/internal/vfs/setup.js`	Heart of the PR. `findVFS` (O(1) lookup), fs handler, loader overrides with parity to `src/node_modules.cc` / `src/node_file.cc`, prefix-scan cache purge.
4	`lib/internal/modules/helpers.js`	Hook surface: `loader*` wrappers, `setLoaderFsOverrides` / `setLoaderPackageOverrides`, `purgeRealpathCacheForPrefix`.
5	`lib/internal/modules/cjs/loader.js`	`stat()` + TS read routed through wrappers; `purgeModuleCachesForPrefix` for unmount.
6	`lib/internal/modules/esm/resolve.js`	`legacyMainResolve` + `internalModuleStat` + `toRealPath` routed. No URL decoration.
7	`lib/internal/modules/esm/load.js`	`getSourceSync` reads via the wrapper.
8	`lib/internal/modules/esm/get_format.js`	Extensionless format detection routed.
9	`lib/internal/modules/package_json_reader.js`	4 C++ binding calls routed; `purgePackageJSONCacheForPrefix`.
10	`lib/fs.js`	`statSync` / `lstatSync` honour `throwIfNoEntry:false` on ENOENT from the VFS handler.
11	`doc/api/vfs.md`	Mount semantics + Module loader integration section.

Uh oh!

Uh oh!

Conversation

mcollina commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Design

Loader integration

Review guide

Refs

Out of scope

Uh oh!

nodejs-github-bot commented May 30, 2026

Uh oh!

codecov Bot commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mcollina commented May 30, 2026

Uh oh!

Uh oh!

Uh oh!

nodejs-github-bot commented Jun 3, 2026

Uh oh!

nodejs-github-bot commented Jun 3, 2026

Uh oh!

joyeecheung left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mcollina commented Jun 3, 2026

Uh oh!

joyeecheung commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mcollina commented Jun 3, 2026

Uh oh!

mcollina commented Jun 3, 2026

Uh oh!

mcollina commented Jun 5, 2026

Uh oh!

mcollina commented Jul 8, 2026

Uh oh!

This comment was marked as outdated.

nodejs-github-bot commented Jul 8, 2026

Uh oh!

joyeecheung left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mcollina commented Jul 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

mcollina commented May 30, 2026 •

edited

Loading

codecov Bot commented May 30, 2026 •

edited

Loading

joyeecheung commented Jun 3, 2026 •

edited

Loading

joyeecheung left a comment •

edited

Loading