feat(cli): auto-detect embedded private packages [RED-862]

packages/cli/src/ai-context/references/configure-playwright-checks.md

@@ -14,7 +14,8 @@
 - Use `installCommand` only when the default package-manager install command is not enough.
 - Checkly caches installed dependencies between runs, keyed off the lock file, `package.json` and `.npmrc` contents. To force a reinstall declaratively, set `caching.dependencyCache.version` (a string or a safe integer) at the top level of `checkly.config.ts` (not per check — one code bundle serves all Playwright Check Suites) and change its value whenever the cache should be invalidated; scheduled checks pick up the change on the next `checkly deploy`. Unset or empty-string values leave the cache key unchanged, so a dynamic value such as `version: process.env.DEPENDENCY_CACHE_VERSION` is safe when the variable is not always set. For a one-off reinstall during an ad-hoc run, use the `--refresh-cache` flag available on the run/test commands (`checkly test`, `checkly pw-test`, `checkly trigger`, `checkly checks run`) instead; the config value is the persistent knob that also applies to deployed, scheduled checks.
 - In Checkly CLI v8.0.0 and later, `include` patterns resolve relative to the Playwright config directory, not the project root. If `playwrightConfigPath` points to a subdirectory, adjust `include` globs. Example: `playwrightConfigPath: "./e2e/playwright.config.ts"` with a root fixture at `fixtures/data.json` needs `include: ["../fixtures/data.json"]`.
-- If dependencies come from a private registry that Checkly's infrastructure cannot reach (for example an intranet-only Nexus mirror), list them in `checks.embeddedPackages` in `checkly.config.ts`. Each entry is a package name (embeds every version found in the lockfile) or an exact `name@version` pin; names may contain `*` wildcards (`@acme/*`, `acme-*`, `@acme/*-utils`) where each `*` matches any run of characters except `/` (never crossing the scope separator); as long as a spec matches at least one registry package, matches that cannot be embedded are skipped (workspace members silently, git/file/URL dependencies and integrity-less entries with a warning since the runner must fetch those itself), while a spec whose only matches cannot be embedded — or that matches nothing at all — is an error; a pattern embeds every lockfile version of every package it matches, so scope it to the packages the runner genuinely cannot fetch. List every unreachable package by name, including private packages that only appear as transitive dependencies of other private packages — dependencies of listed packages are not embedded automatically. The CLI resolves entries against the workspace-root lockfile (`pnpm-lock.yaml` or `package-lock.json`), reuses tarballs from local caches (its own, then npm's) or downloads them from the registry configured in `.npmrc`, verifies each against the lockfile's integrity hash, and ships them inside the code bundle at `.checkly/embedded-packages/*.tgz`, where the runner serves them through a local registry during install. Downloads are cached under the workspace root's `node_modules/.cache/checkly` (in a monorepo that is the repo root, not the member package; override with `CHECKLY_CACHE_DIR`; a per-user cache dir is the fallback when the project location isn't writable), so nothing lands in the project outside `node_modules`. CI setups that cache `node_modules` — or platforms that preserve `node_modules/.cache` — persist the tarballs automatically; otherwise persist `CHECKLY_CACHE_DIR` in CI to avoid re-downloading (note `npm ci` deletes `node_modules` wholesale, unlike incremental pnpm installs). The machine running `checkly deploy`/`test` needs registry access on a cold cache. Applies to Playwright Check Suites only, not browser or multistep checks.
+- Dependencies that Checkly's infrastructure cannot fetch from the public npm registry (for example packages from an intranet-only Nexus mirror) are **detected and embedded into the code bundle automatically** during `deploy`/`test`/`pw-test` (`checks.detectEmbeddedPackages`, default `true`; per-run override `--no-detect-embedded-packages`). Private package names never leave the machine by default: detection needs no network when the effective registry is the public one, packages from a scoped registry (`@scope:registry` in `.npmrc`) are embedded without any lookup, and remaining undecided packages are resolved by asking the private registry which packages it hosts (Sonatype Nexus REST API, using the `.npmrc` credentials; the credentials must be able to browse every npm hosted repository on the instance) — each package is checked against the instance its lockfile-recorded `resolved` URL points at; packages whose lockfile records no source URL (`pnpm-lock.yaml` records none) are classified by the configured registry's inventory, where hosted means embed and absent means public; a recorded source that isn't a Nexus content URL is checked against the configured registry only when it shares that registry's host, and only to confirm the package is hosted there (a package the instance doesn't host stays undecided rather than being assumed public). Results are cached keyed by the lockfile, so repeat runs are free. Whatever detection leaves undecided — because the registry API is unavailable (no REST access, or a non-Nexus registry), because a same-origin recorded source is not hosted on the instance even though the API works, or because the registry configuration itself cannot be resolved (e.g. an unset environment variable referenced in `.npmrc`, which is also warned about separately) — is skipped with a warning; set `checks.detectEmbeddedPackagesFallback: "public-registry"` to instead allow integrity lookups against public npm for those undecided packages (accurate for any registry product, but it transmits the undecided package names, potentially private ones, to the public registry — the exact versions listed in `checks.embeddedPackages` are exempt, though *other* lockfile versions of a pinned name still count as undecided and are transmitted; verdicts obtained this way are cached as immutable proofs and continue to apply after the option is set back to `"skip"` — clear the CLI cache to discard them), or list the packages explicitly. Detection assumes proxy repositories front public npm; packages proxied from *another private* registry are not detected and must be listed explicitly. Detection state lives in the CLI cache — delete `node_modules/.cache/checkly`, plus the per-user cache directory used when the project location isn't writable (`~/Library/Caches/checkly` on macOS, `~/.cache/checkly` on Linux, `%LOCALAPPDATA%\checkly\Cache` on Windows), or point `CHECKLY_CACHE_DIR` elsewhere, to reset it.
+- To embed packages explicitly — pinning a version, forcing a public package in, or working with detection off — list them in `checks.embeddedPackages` in `checkly.config.ts`. An explicit entry takes over its package name: detection never adds other versions of an explicitly listed name. Each entry is a package name (embeds every version found in the lockfile) or an exact `name@version` pin; names may contain `*` wildcards (`@acme/*`, `acme-*`, `@acme/*-utils`) where each `*` matches any run of characters except `/` (never crossing the scope separator); as long as a spec matches at least one registry package, matches that cannot be embedded are skipped (workspace members silently, git/file/URL dependencies and integrity-less entries with a warning since the runner must fetch those itself), while a spec whose only matches cannot be embedded — or that matches nothing at all — is an error; a pattern embeds every lockfile version of every package it matches, so scope it to the packages the runner genuinely cannot fetch. With detection disabled, list every unreachable package by name, including private packages that only appear as transitive dependencies of other private packages — dependencies of listed packages are not embedded implicitly. The CLI resolves entries against the workspace-root lockfile (`pnpm-lock.yaml` or `package-lock.json`), reuses tarballs from local caches (its own, then npm's) or downloads them from the registry configured in `.npmrc`, verifies each against the lockfile's integrity hash, and ships them inside the code bundle at `.checkly/embedded-packages/*.tgz`, where the runner serves them through a local registry during install. Downloads are cached under the workspace root's `node_modules/.cache/checkly` (in a monorepo that is the repo root, not the member package; override with `CHECKLY_CACHE_DIR`; a per-user cache dir is the fallback when the project location isn't writable), so nothing lands in the project outside `node_modules`. CI setups that cache `node_modules` — or platforms that preserve `node_modules/.cache` — persist the tarballs automatically; otherwise persist `CHECKLY_CACHE_DIR` in CI to avoid re-downloading (note `npm ci` deletes `node_modules` wholesale, unlike incremental pnpm installs). The machine running `checkly deploy`/`test` needs registry access on a cold cache. Applies to Playwright Check Suites only, not browser or multistep checks.
 
 ## Install troubleshooting
 

packages/cli/src/commands/debug/parse-project.ts

@@ -150,6 +150,8 @@ export default class ParseProjectCommand extends Command {
         playwrightConfigPath: checklyConfig.checks?.playwrightConfigPath,
         include: includeFlag.length ? includeFlag : checklyConfig.checks?.include,
         embeddedPackages: checklyConfig.checks?.embeddedPackages,
+        detectEmbeddedPackages: checklyConfig.checks?.detectEmbeddedPackages,
+        detectEmbeddedPackagesFallback: checklyConfig.checks?.detectEmbeddedPackagesFallback,
         playwrightChecks: checklyConfig.checks?.playwrightChecks,
         loadPlaywrightChecksOnly: emulatePwTest,
         warnOnWebServerConfig: emulatePwTest && !(includeFlag.length > 0),

packages/cli/src/commands/deploy.ts

@@ -97,6 +97,11 @@ export default class Deploy extends AuthCommand {
       allowNo: true,
       env: 'CHECKLY_VERIFY_RUNTIME_DEPENDENCIES',
     }),
+    'detect-embedded-packages': Flags.boolean({
+      description: '[default: true] Automatically embed dependencies that Checkly cannot fetch from the public npm registry (see checks.detectEmbeddedPackages).',
+      allowNo: true,
+      env: 'CHECKLY_DETECT_EMBEDDED_PACKAGES',
+    }),
     'debug-bundle': Flags.boolean({
       description: 'Output the project bundle to a file without deploying any resources.',
       default: false,
@@ -179,6 +184,8 @@ export default class Deploy extends AuthCommand {
       playwrightConfigPath: checklyConfig.checks?.playwrightConfigPath,
       include: checklyConfig.checks?.include,
       embeddedPackages: checklyConfig.checks?.embeddedPackages,
+      detectEmbeddedPackages: flags['detect-embedded-packages'] ?? checklyConfig.checks?.detectEmbeddedPackages,
+      detectEmbeddedPackagesFallback: checklyConfig.checks?.detectEmbeddedPackagesFallback,
       playwrightChecks: checklyConfig.checks?.playwrightChecks,
     })
     const repoInfo = getGitInformation(project.repoUrl)

packages/cli/src/commands/pw-test.ts

@@ -112,6 +112,11 @@ export default class PwTestCommand extends AuthCommand {
       multiple: true,
       default: [],
     }),
+    'detect-embedded-packages': Flags.boolean({
+      description: '[default: true] Automatically embed dependencies that Checkly cannot fetch from the public npm registry (see checks.detectEmbeddedPackages).',
+      allowNo: true,
+      env: 'CHECKLY_DETECT_EMBEDDED_PACKAGES',
+    }),
     'install-command': Flags.string({
       description: 'Command to install dependencies before running tests.',
     }),
@@ -215,6 +220,8 @@ export default class PwTestCommand extends AuthCommand {
       playwrightConfigPath,
       include: includeFlag.length ? includeFlag : checklyConfig.checks?.include,
       embeddedPackages: checklyConfig.checks?.embeddedPackages,
+      detectEmbeddedPackages: flags['detect-embedded-packages'] ?? checklyConfig.checks?.detectEmbeddedPackages,
+      detectEmbeddedPackagesFallback: checklyConfig.checks?.detectEmbeddedPackagesFallback,
       playwrightChecks: [playwrightCheck],
       loadPlaywrightChecksOnly: true,
       warnOnWebServerConfig: !(includeFlag.length > 0),

packages/cli/src/commands/test.ts

@@ -116,6 +116,11 @@ export default class Test extends AuthCommand {
       allowNo: true,
       env: 'CHECKLY_VERIFY_RUNTIME_DEPENDENCIES',
     }),
+    'detect-embedded-packages': Flags.boolean({
+      description: '[default: true] Automatically embed dependencies that Checkly cannot fetch from the public npm registry (see checks.detectEmbeddedPackages).',
+      allowNo: true,
+      env: 'CHECKLY_DETECT_EMBEDDED_PACKAGES',
+    }),
     'refresh-cache': Flags.boolean({
       description: 'Force a fresh install of dependencies and update the cached version.',
       default: false,
@@ -207,6 +212,8 @@ export default class Test extends AuthCommand {
       playwrightConfigPath: checklyConfig.checks?.playwrightConfigPath,
       include: checklyConfig.checks?.include,
       embeddedPackages: checklyConfig.checks?.embeddedPackages,
+      detectEmbeddedPackages: flags['detect-embedded-packages'] ?? checklyConfig.checks?.detectEmbeddedPackages,
+      detectEmbeddedPackagesFallback: checklyConfig.checks?.detectEmbeddedPackagesFallback,
       playwrightChecks: checklyConfig.checks?.playwrightChecks,
       checkFilter: check => {
         if (check instanceof HeartbeatMonitor) {

packages/cli/src/commands/validate.ts

@@ -63,6 +63,8 @@ export default class Validate extends AuthCommand {
       playwrightConfigPath: checklyConfig.checks?.playwrightConfigPath,
       include: checklyConfig.checks?.include,
       embeddedPackages: checklyConfig.checks?.embeddedPackages,
+      detectEmbeddedPackages: checklyConfig.checks?.detectEmbeddedPackages,
+      detectEmbeddedPackagesFallback: checklyConfig.checks?.detectEmbeddedPackagesFallback,
       playwrightChecks: checklyConfig.checks?.playwrightChecks,
     })
 

packages/cli/src/constructs/__tests__/fixtures/playwright-check/test-cases/test-embedded-packages-detect/.npmrc

@@ -0,0 +1,2 @@
+registry=https://registry.npmjs.org/
+@acme:registry=https://nexus.local/repository/npm-private/

packages/cli/src/constructs/__tests__/fixtures/playwright-check/test-cases/test-embedded-packages-detect/checkly.config.ts

@@ -0,0 +1,19 @@
+import { defineConfig } from 'checkly'
+
+const config = defineConfig({
+  projectName: 'Check Fixture',
+  logicalId: 'check-fixture',
+  checks: {
+    checkMatch: '**/*.check.ts',
+    ignoreDirectoriesMatch: [],
+    playwrightConfigPath: './playwright.config.ts',
+    playwrightChecks: [
+      {
+        logicalId: 'playwright-check-suite',
+        name: 'Playwright Check Suite',
+      }
+    ],
+  },
+})
+
+export default config

packages/cli/src/constructs/__tests__/fixtures/playwright-check/test-cases/test-embedded-packages-detect/checkly.detect-off.config.ts

@@ -0,0 +1,20 @@
+import { defineConfig } from 'checkly'
+
+const config = defineConfig({
+  projectName: 'Check Fixture',
+  logicalId: 'check-fixture',
+  checks: {
+    checkMatch: '**/*.check.ts',
+    ignoreDirectoriesMatch: [],
+    playwrightConfigPath: './playwright.config.ts',
+    detectEmbeddedPackages: false,
+    playwrightChecks: [
+      {
+        logicalId: 'playwright-check-suite',
+        name: 'Playwright Check Suite',
+      }
+    ],
+  },
+})
+
+export default config

packages/cli/src/constructs/__tests__/fixtures/playwright-check/test-cases/test-embedded-packages-detect/package.json

@@ -0,0 +1,7 @@
+{
+  "name": "playwright-bundle-test",
+  "version": "1.0.0",
+  "dependencies": {
+    "@playwright/test": "^1.55.1"
+  }
+}

packages/cli/src/constructs/__tests__/fixtures/playwright-check/test-cases/test-embedded-packages-detect/playwright.config.ts

@@ -0,0 +1,6 @@
+import { defineConfig } from '@playwright/test'
+
+export default defineConfig({
+  testDir: './tests',
+  timeout: 30000,
+})

packages/cli/src/constructs/__tests__/fixtures/playwright-check/test-cases/test-embedded-packages-detect/tests/example.spec.ts

@@ -0,0 +1,6 @@
+import { test, expect } from '@playwright/test'
+
+test('basic test', async ({ page }) => {
+  await page.goto('https://playwright.dev/')
+  expect(await page.title()).toContain('Playwright')
+})