diff --git a/README.md b/README.md index b93a305..886a307 100644 --- a/README.md +++ b/README.md @@ -37,33 +37,116 @@ _See code: [src/commands/telemetry/index.js](https://github.com/adobe/aio-cli-pl ## Configuration -The following values need to be set when this plugin is hosted by different CLIs -- `aioTelemetry`: defined object in root cli package.json with values: - - `postUrl` : Where to post telemetry data - - `postHeaders`: Any specific headers that need to be posted with telemetry data (ex. x-api-key) - - `productPrivacyPolicyLink`: A link to display to users when prompting to optIn -- `productName`: How to refer to the cli when user is prompted to enable telemetry - - this value is read from `displayName` or `name` of the cli's package.json -- `productBin`: Output in help text - - ex. To turn telemetry on run `${productBin} telemetry on` - - this value is read from 'bin' of the cli's package.json, if the package exports more than 1 bin the first is used -## POST data +When this plugin is hosted by different CLIs: -Here is an example of the event data as posted: -``` -{ "id": 656915165813, - "timestamp": 1673404918437, - "_adobeio": { - "eventType": "telemetry-prompt", - "eventData": "accepted", - "cliVersion": "@adobe/aio-cli@9.1.1", - "clientId": 264421030538, - "commandDuration": 5661, - "commandFlags": "", - "commandSuccess": true, - "nodeVersion": "v14.20.0", - "osNameVersion": "macOS" +- `aioTelemetry` (optional object in the root `package.json` of the **host CLI** — the same `pjson` oclif passes into the init hook): + - `postUrl` (optional string): HTTPS URL of the telemetry proxy that receives POSTed metric batches. Use this when your CLI should send telemetry to a different App Builder action or gateway than the plugin default. + - `fetchHeaders`: Optional extra headers merged into telemetry requests (`Content-Type` is always set) + - `productPrivacyPolicyLink`: A link shown in the one-time telemetry notice (what is collected and how to opt out) +- `productName`: How to refer to the CLI in the telemetry notice (from `displayName` or `name` in `package.json`) +- `productBin`: Shown in help text (from `bin` in `package.json`; if several bins exist, the first is used). Example: run `${productBin} telemetry on` + +### Overriding the telemetry POST URL + +Resolution order (first match wins): + +1. **`aioTelemetry.postUrl`** in the host CLI `package.json` +2. **`AIO_TELEMETRY_POST_URL`** environment variable (non-empty string) +3. **Built-in default** in the plugin (`DEFAULT_TELEMETRY_POST_URL` in [`src/telemetry-lib.js`](https://github.com/adobe/aio-cli-plugin-telemetry/blob/master/src/telemetry-lib.js)) + +Host `package.json` example: + +```json +{ + "name": "my-cli", + "bin": { "mycli": "./bin/run.js" }, + "aioTelemetry": { + "postUrl": "https://-.adobeio-static.net/api/v1/web//" } } ``` + +Environment override (no `package.json` change; useful for CI, staging, or local proxy debugging): + +```sh +export AIO_TELEMETRY_POST_URL='https://-.adobeio-static.net/api/v1/web//' +mycli app deploy +``` + +The resolved URL is passed to the flush worker on each telemetry send; it applies for the rest of that CLI process after `init` runs. + +## Opting out via environment variable + +Telemetry is suppressed when `AIO_TELEMETRY_DISABLED` is set to one of **`true`**, **`1`**, or **`yes`** (exact match; case-sensitive). Other values such as `0`, `false`, `no`, or an empty string do **not** disable telemetry via this variable. + +This does not change the persisted opt-out state. Useful for CI pipelines and scripted environments. + +```sh +AIO_TELEMETRY_DISABLED=true aio app deploy +AIO_TELEMETRY_DISABLED=1 aio app deploy +AIO_TELEMETRY_DISABLED=yes aio app deploy +``` + +## Flush architecture + +Telemetry is **best-effort**: events are not persisted when the proxy is down or the network fails. + +A flush happens on a **terminal event** — `postrun` (command succeeded), `command-error` (command threw), or `command-not-found`. oclif runs `postrun` only on success, so `command-error`/`command-not-found` must flush on their own; otherwise error telemetry (the most valuable signal) would be buffered and silently dropped on exit. On a terminal event, any in-memory metrics from earlier hooks in the same command are merged with the terminal metric and the combined batch is handed off to a **fire-and-forget detached subprocess** (`src/flush-worker.js`). The parent CLI spawns the worker and immediately `unref()`s it, so the CLI can exit without waiting for the HTTP POST. If the POST fails (network error or non-2xx response), the batch is dropped; telemetry must not block or slow normal CLI use. + +Non-terminal events (for example `telemetry-notice`) are held in an **in-memory buffer** until the next terminal event flushes them. If the process exits before any terminal event (crash, `SIGKILL`), buffered events are lost. The buffer is cleared when telemetry is disabled or when `init` runs again (new command session). + +## Agent detection + +The plugin detects whether the CLI is being invoked by an AI agent or a human by inspecting environment variables at the time of the event. The detected context is included in every event as `invocation_context` (`"agent"` or `"human"`) and `agent_name`. + +Supported agents detected automatically: + +| Environment variable | Detected agent name | +|---|---| +| `AGENT` | value of the variable (or `"generic"` if `1`/`true`) | +| `AI_AGENT` | value of the variable (or `"generic"` if `1`/`true`) | +| `AIO_AGENT` | `aio-opt-in` | +| `AIO_INVOCATION_CONTEXT=agent` | `aio-opt-in` | +| `CURSOR_AGENT` | `cursor` | +| `CLAUDECODE` / `CLAUDE_CODE` | `claude` | +| `GEMINI_CLI` | `gemini` | +| `CODEX_SANDBOX` | `codex` | +| `AUGMENT_AGENT` | `augment` | +| `CLINE_ACTIVE` | `cline` | +| `OPENCODE_CLIENT` | `opencode` | +| `REPL_ID` | `replit` | +| `PATH` containing `github.copilot-chat` | `github-copilot` | + +To opt into agent tracking without setting a tool-specific variable, set `AIO_INVOCATION_CONTEXT=agent`. + +## POST data + +The `eventData` attribute is always a string. Objects and arrays are stored as a JSON text (e.g. `"{}"`, `"{\"message\":\"…\"}"`). String payloads (such as the telemetry notice outcome `shown`) are stored as that plain text without an extra layer of JSON quoting. Numbers and booleans use their usual string forms (`"0"`, `"false"`). + +Example shape of the metric payload: + +```json +[{ + "metrics": [{ + "name": "aio.cli.telemetry", + "type": "gauge", + "value": 1, + "timestamp": 1673404918437, + "attributes": { + "eventName": "postrun", + "eventData": "{}", + "cliVersion": "@adobe/aio-cli@11.0.2", + "clientId": 264421030538, + "command": "app:deploy", + "commandDuration": 5661, + "commandFlags": "", + "commandSuccess": true, + "nodeVersion": "v22.21.1", + "osNameVersion": "macOS Sequoia 15.4", + "invocation_context": "human", + "agent_name": "unknown" + } + }] +}] +``` diff --git a/package.json b/package.json index 730529a..c0fd77b 100644 --- a/package.json +++ b/package.json @@ -9,10 +9,7 @@ "@adobe/aio-lib-core-networking": "^5.0.4", "@oclif/core": "^4", "ci-info": "^4.0.0", - "debug": "^4.1.1", - "inquirer": "^8.2.1", - "os-name": "^4.0.1", - "splunk-logging": "^0.11.1" + "debug": "^4.1.1" }, "devDependencies": { "@adobe/eslint-config-aio-lib-config": "5.0.0", diff --git a/src/flush-worker.js b/src/flush-worker.js new file mode 100644 index 0000000..5014eaa --- /dev/null +++ b/src/flush-worker.js @@ -0,0 +1,87 @@ +/* +Copyright 2026 Adobe. All rights reserved. +This file is licensed to you under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. You may obtain a copy +of the License at http://www.apache.org/licenses/LICENSE-2.0 +Unless required by applicable law or agreed to in writing, software distributed under +the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS +OF ANY KIND, either express or implied. See the License for the specific language +governing permissions and limitations under the License. +*/ + +/** + * Telemetry flush worker — spawned as a detached subprocess so the parent CLI can + * exit immediately without waiting on the HTTP POST. + * + * Accepts a single CLI argument: a JSON-encoded object with shape + * { body: string, postUrl: string, headers?: object } where `body` is a serialised + * New Relic metric batch array (same shape as the parent builds for fetch), `postUrl` + * is the App Builder proxy, and `headers` (when present) are optional overrides merged + * after the worker defaults (never pass secrets such as api-key). + * + * Failed deliveries are dropped; telemetry is best-effort and must not affect the CLI. + */ + +'use strict' + +const debug = require('debug')('aio-telemetry:flush-worker') +const { createFetch } = require('@adobe/aio-lib-core-networking') + +const fetch = createFetch() + +const DEFAULT_HEADERS = { + 'Content-Type': 'application/json' +} + +/** + * POSTs the metric batch from argv. Swallows all errors. + * @returns {Promise} + */ +async function main () { + let batches + let postUrl + let requestHeaders = { ...DEFAULT_HEADERS } + try { + const parsed = JSON.parse(process.argv[2]) + const { body, postUrl: url, headers } = parsed + if (!url || typeof url !== 'string') { + return + } + postUrl = url + if (headers && typeof headers === 'object') { + const safe = Object.fromEntries( + Object.entries(headers).filter(([key]) => key.toLowerCase() !== 'api-key') + ) + requestHeaders = { ...DEFAULT_HEADERS, ...safe } + } + const parsedBody = JSON.parse(body) + if (!Array.isArray(parsedBody)) { + return + } + batches = parsedBody + } catch { + return + } + + try { + debug('POST %s requestHeaders=%o', postUrl, requestHeaders) + const res = await fetch(postUrl, { + method: 'POST', + headers: requestHeaders, + body: JSON.stringify({ batches }) + }) + if (!res?.ok) { + const status = res?.status ?? 'unknown' + debug('telemetry flush non-ok: HTTP %s', status) + } + } catch (err) { + debug('telemetry flush failed: %O', err) + } +} + +/* istanbul ignore next */ +if (require.main === module) { + main() +} + +module.exports = { main } diff --git a/src/hooks/init.js b/src/hooks/init.js index f1d0910..1631fc9 100644 --- a/src/hooks/init.js +++ b/src/hooks/init.js @@ -37,17 +37,16 @@ module.exports = async function (opts) { opts.argv.filter(arg => arg.indexOf('-') === 0).join(','), global.prerunTimer) - // init event does not post telemetry, it stores some info that will be used later - // this will prompt to optIn/Out if telemetry.optIn is undefined + // Telemetry is opt-out (on by default). On the first run we show a one-time, + // non-blocking notice instead of an opt-in prompt. isNull() is true only until the + // notice records the default, so this shows once. if ((opts.argv.indexOf('--no-telemetry') < 0) && !inCI && telemetryLib.isNull()) { - // let's ask! - // unfortunately the `oclif-dev readme` run by prepack fires this event, which hangs CI - // Also we don't prompt for telemetry if the first command is a telemetry command because they - // are probably setting it on or off already + // skip in CI (handled above) and when oclif-dev readme runs (it fires this event); + // also skip for telemetry commands, where the user is already setting state. if (['readme', 'telemetry'].indexOf(opts.id) < 0) { - return telemetryLib.prompt(productName, binName, opts?.config?.pjson?.aioTelemetry?.productPrivacyPolicyLink) + telemetryLib.notice(productName, opts?.config?.pjson?.aioTelemetry?.productPrivacyPolicyLink) } } } diff --git a/src/telemetry-lib.js b/src/telemetry-lib.js index f40bfac..662490d 100644 --- a/src/telemetry-lib.js +++ b/src/telemetry-lib.js @@ -9,30 +9,105 @@ OF ANY KIND, either express or implied. See the License for the specific languag governing permissions and limitations under the License. */ -const { createFetch } = require('@adobe/aio-lib-core-networking') +const { spawn } = require('child_process') +const path = require('path') +const os = require('os') const config = require('@adobe/aio-lib-core-config') -const fetch = createFetch() -const osName = require('os-name') -const inquirer = require('inquirer') const debug = require('debug')('aio-telemetry:telemetry-lib') +/** Adobe I/O App Builder web action that forwards CLI metrics to New Relic (ingest key stays server-side). */ +const DEFAULT_TELEMETRY_POST_URL = 'https://53444-aioclitelemetryproxy.adobeio-static.net/api/v1/web/dx-excshell-1/telemetry' + +/** @returns {boolean} Whether `AIO_TELEMETRY_DISABLED` opts out (only the literal string `"true"`). */ +function isEnvTelemetryDisabled () { + return ['true', '1', 'yes'].includes(process.env.AIO_TELEMETRY_DISABLED) +} + let isDisabledForCommand = false -const osNameVersion = osName() +/** Metrics for non-terminal events in the current command; merged into the POST on a terminal event. */ +const pendingCommandMetrics = [] + +/** Events with no `postrun` after them; each flushes immediately so error telemetry isn't lost. */ +const TERMINAL_EVENTS = ['postrun', 'command-error', 'command-not-found'] + +/** Events that mean the command did not succeed (so `commandSuccess` is false). */ +const FAILED_COMMAND_EVENTS = ['command-error', 'command-not-found'] + +/** Set by `command-not-found` so we can drop the host's immediate `command-error` rethrow of the same typo. */ +let commandNotFoundFired = false + +/** + * Detects GitHub Copilot Chat on PATH via the extension id in globalStorage paths (any OS path separator). + * + * @param {string|null|undefined} [pathValue] - PATH environment variable value. + * @returns {string|null} Agent name when detected, otherwise null. + */ +function detectCopilotAgent (pathValue) { + if (!pathValue) return null + // Extension id appears in globalStorage paths on all platforms; do not tie to '/' (Windows uses '\'). + if (pathValue.includes('github.copilot-chat')) { + return 'github-copilot' + } + return null +} + +// TODO: detect VSCODE run as an agent +/** + * Environment variables checked for agent detection (proposed standard first, then tool-specific). + * Used for metrics only. See aio-cli README "Agent detection" for full list. + */ +const AGENT_ENV_VARS = [ + { env: 'AGENT', name: (v) => (v && v !== '1' && v !== 'true' ? String(v).toLowerCase() : 'generic') }, + { env: 'AI_AGENT', name: (v) => (v && v !== '1' && v !== 'true' ? String(v).toLowerCase() : 'generic') }, + { env: 'AIO_AGENT', name: () => 'aio-opt-in' }, + { env: 'AIO_INVOCATION_CONTEXT', name: (v) => (v === 'agent' ? 'aio-opt-in' : null) }, + { env: 'CURSOR_AGENT', name: () => 'cursor' }, + { env: 'CLAUDECODE', name: () => 'claude' }, + { env: 'CLAUDE_CODE', name: () => 'claude' }, + { env: 'GEMINI_CLI', name: () => 'gemini' }, + { env: 'CODEX_SANDBOX', name: () => 'codex' }, + { env: 'AUGMENT_AGENT', name: () => 'augment' }, + { env: 'CLINE_ACTIVE', name: () => 'cline' }, + { env: 'OPENCODE_CLIENT', name: () => 'opencode' }, + { env: 'PATH', name: detectCopilotAgent }, + { env: 'REPL_ID', name: () => 'replit' } +] + +/** + * Detects whether the CLI is being invoked by an AI agent (vs a human) using env vars. + * Used for metrics only. + * + * @param {object} [env] - Environment object to read (defaults to process.env when omitted). + * @returns {{ isAgent: boolean, agentName: string|null }} Invocation context metadata. + */ +function getInvocationContext (env) { + const envToUse = env !== undefined ? env : process.env + for (const { env: key, name } of AGENT_ENV_VARS) { + const value = envToUse[key] + if (value !== undefined && value !== '') { + const agentName = name(value) + if (agentName) { + return { isAgent: true, agentName } + } + } + } + return { isAgent: false, agentName: null } +} + +const osNameVersion = `${os.type()} ${os.release()}` // this is set by the init hook, ex. @adobe/aio-cli@8.2.0 let rootCliVersion = '?' let prerunEvent = { flags: [] } -// postUrl and fetchHeaders are set by the init hook if these values are set in the root cli package.json -let postUrl = 'https://dcs.adobedc.net/collection/ffb5bdcefe744485c5c968662012f91293eee10f5dac4ca009beb14d7c028424?asynchronous=true' -let fetchHeaders = { - 'Content-Type': 'application/json', - 'x-adobe-flow-id': '18dce8db-f523-4ff1-8806-0719de3fd367', - 'x-api-key': 'adobe_io', - 'sandbox-name': 'developer-lifecycle-dev1' -} + +/** @type {Record} Host-only headers from `aioTelemetry.fetchHeaders` (passed to flush worker as overrides). */ +let extraFetchHeaders = {} +/** @type {string} Resolved proxy URL (defaults at module load; init may override). */ +let postUrl = DEFAULT_TELEMETRY_POST_URL let configKey = 'aio-cli-telemetry' + const defaultPrivacyPolicyLink = 'https://developer.adobe.com/app-builder/docs/guides/telemetry/' /** @@ -53,48 +128,142 @@ const getOnMessage = (productName, binName) => { const getOffMessage = (binName) => { return `\nTelemetry is off.\nIf you would like to turn telemetry on, simply run \`${binName} telemetry on\`` } +const getNoticeMessage = (productName, privacyPolicyLink) => { + return `${productName} collects anonymous usage data to help us improve our products.\n` + + `Telemetry is on by default; read what we collect and how it is used here: ${privacyPolicyLink || defaultPrivacyPolicyLink}` +} + +/** + * Builds the value stored in the metric `eventData` attribute. For `postrun`, an empty object is + * replaced with `{ durationMs }` from `global.prerunTimer` so older CLIs / hooks that omit the + * second argument still send timing. + * + * @param {string} eventType - telemetry event name (e.g. postrun, command-error) + * @param {object|string|number|undefined} raw - argument passed to trackEvent + * @returns {object|string|number} payload serialized into the metric attribute + */ +function resolveEventData (eventType, raw) { + if (eventType !== 'postrun') { + return raw === undefined ? {} : raw + } + const empty = raw === undefined || raw === null || + (typeof raw === 'object' && !Array.isArray(raw) && Object.keys(raw).length === 0) + if (!empty) { + return raw + } + if (typeof global.prerunTimer === 'number') { + return { durationMs: Date.now() - global.prerunTimer } + } + return {} +} + +/** + * Serializes `eventData` for the metric `eventData` attribute (a string on the wire). + * Objects and arrays use `JSON.stringify`; string primitives are not double-encoded + * (so e.g. telemetry-prompt `"accepted"` stays `accepted`, not `"\"accepted\""`). + * + * @param {object|string|number|boolean|undefined|null} eventData - resolved payload from {@link resolveEventData} + * @returns {string} Serialized value for the metric `eventData` attribute (JSON for objects/arrays; plain text for strings). + */ +function formatEventDataAttribute (eventData) { + if (eventData === undefined) { + return '{}' + } + if (typeof eventData === 'string') { + return eventData + } + if (['number', 'boolean', 'bigint'].includes(typeof eventData)) { + return String(eventData) + } + if (typeof eventData === 'object') { + return JSON.stringify(eventData) + } + return String(eventData) +} /** - * @description tracks the event + * Records a telemetry event. Non-terminal metrics are held in memory and sent in a single + * batched POST on the next terminal event (`postrun`, `command-error`, `command-not-found`). + * When enabled, the flush worker is detached so the CLI never waits on the network; failed + * deliveries are dropped (no disk queue). + * * @param {string} eventType prerun, postrun, command-error, command-not-found, telemetry - * @param {string} eventData additional data, like the error message, or custom telemetry payload + * @param {object|string|number|undefined} [rawEventData] Optional hook payload (e.g. `{ message }` on errors). + * Command/flags/duration are also sent as separate metric attributes from prerun/postrun. * @returns {undefined} */ -async function trackEvent (eventType, eventData = '') { - // prerunEvent will be null when telemetry-prompt event fires, this happens before - // any command is actually run, so we want to ignore the command+flags in this case +async function trackEvent (eventType, rawEventData = {}) { + // prerunEvent is minimal before prerun; telemetry-prompt and similar fire before a command runs. + + const eventData = resolveEventData(eventType, rawEventData) - if (isDisabledForCommand || config.get(`${configKey}.optOut`, 'global') === true) { + const optedOut = isDisabledForCommand || isEnvTelemetryDisabled() || config.get(`${configKey}.optOut`, 'global') === true + const willSend = !optedOut + debug(`trackEvent ${eventType} eventData=${JSON.stringify(eventData)} postUrl=${postUrl} willSend=${willSend}`) + + if (optedOut) { + pendingCommandMetrics.length = 0 debug('Telemetry is off. Not logging telemetry event', eventType) } else { const clientId = getClientId() const timestamp = Date.now() - const fetchConfig = { - method: 'POST', - headers: fetchHeaders, - body: JSON.stringify({ - id: Math.floor(timestamp * Math.random()), - timestamp, - _adobeio: { - eventType, - eventData, - cliVersion: rootCliVersion, - clientId, - command: prerunEvent.command, - commandDuration: timestamp - prerunEvent.start, - commandFlags: prerunEvent.flags.toString(), - commandSuccess: eventType !== 'command-error', - nodeVersion: process.version, - osNameVersion - } - }) + const invocationContext = getInvocationContext() + const metric = { + name: 'aio.cli.telemetry', + type: 'gauge', + value: 1, + timestamp, + attributes: { + // NB: the wire attribute is `eventName`, not `eventType` — `eventType` is a New Relic + // reserved word and is dropped on Metric API ingest, so it is unqueryable in NRQL. + eventName: eventType, + eventData: formatEventDataAttribute(eventData), + cliVersion: rootCliVersion, + clientId, + command: prerunEvent.command, + commandDuration: timestamp - prerunEvent.start, + commandFlags: prerunEvent.flags.toString(), + commandSuccess: !FAILED_COMMAND_EVENTS.includes(eventType), + nodeVersion: process.version, + osNameVersion, + invocation_context: /* istanbul ignore next */ invocationContext.isAgent ? 'agent' : 'human', + agent_name: /* istanbul ignore next */ invocationContext.agentName || 'unknown' + } + } + + // Non-terminal events buffer; terminal events flush (oclif runs no postrun after error/not-found). + if (!TERMINAL_EVENTS.includes(eventType)) { + pendingCommandMetrics.push(metric) + return } + + // A typo fires command-not-found, then the host rethrows it as command-error; drop that rethrow. + if (eventType === 'command-not-found') { + commandNotFoundFired = true + } else if (eventType === 'command-error' && commandNotFoundFired) { + commandNotFoundFired = false + debug('dropping command-error that rethrows a command-not-found (same typo)') + return + } + + const mergedMetrics = [...pendingCommandMetrics, metric] + pendingCommandMetrics.length = 0 + const mergedBody = JSON.stringify([{ metrics: mergedMetrics }]) + + const flushPayload = JSON.stringify({ + body: mergedBody, + postUrl, + ...(Object.keys(extraFetchHeaders).length > 0 && { headers: { ...extraFetchHeaders } }) + }) try { - debug('posting telemetry event', fetchConfig) - const response = await fetch(postUrl, fetchConfig) - debug('response.ok = ', response.ok) - } catch (exc) { - debug('error reaching telemetry server : ', exc) + // Omit `env`: child inherits `process.env` (proxy / TLS vars for fetch in the worker). + const child = spawn(process.execPath, [path.join(__dirname, 'flush-worker.js'), flushPayload], { + detached: true, + stdio: 'ignore' + }) + child.unref() + } catch (err) { + debug('Failed to launch telemetry flush worker: %O', err) } } } @@ -106,18 +275,25 @@ async function trackEvent (eventType, eventData = '') { */ function trackPrerun (command, flags, start) { prerunEvent = { command, flags, start } + // A real command is now running (e.g. an accepted "did you mean" suggestion), so its errors count. + commandNotFoundFired = false } module.exports = { + getInvocationContext, init: (versionString, binName, remoteConf = {}) => { + pendingCommandMetrics.length = 0 + commandNotFoundFired = false global.commandHookStartTime = Date.now() rootCliVersion = versionString - if (remoteConf.fetchHeaders) { - fetchHeaders = remoteConf.fetchHeaders - } - if (remoteConf.postUrl) { - postUrl = remoteConf.postUrl - } + postUrl = remoteConf.postUrl || process.env.AIO_TELEMETRY_POST_URL || DEFAULT_TELEMETRY_POST_URL + const rawExtra = remoteConf.fetchHeaders && typeof remoteConf.fetchHeaders === 'object' + ? remoteConf.fetchHeaders + : {} + const BLOCKED_HEADERS = new Set(['api-key', 'authorization', 'x-api-key', 'x-ingest-key']) + extraFetchHeaders = Object.fromEntries( + Object.entries(rawExtra).filter(([key]) => !BLOCKED_HEADERS.has(key.toLowerCase())) + ) configKey = binName + '-cli-telemetry' }, getClientId, @@ -128,46 +304,30 @@ module.exports = { config.set(`${configKey}.optOut`, true) }, isEnabled: () => { - return !isDisabledForCommand && config.get(`${configKey}.optOut`, 'global') === false + return !isDisabledForCommand && !isEnvTelemetryDisabled() && config.get(`${configKey}.optOut`, 'global') !== true }, disableForCommand: () => { isDisabledForCommand = true }, isNull: () => { - return config.get(`${configKey}.optOut`, 'global') === undefined + return !isEnvTelemetryDisabled() && config.get(`${configKey}.optOut`, 'global') === undefined }, trackEvent, trackPrerun, // secret api for testing + DEFAULT_TELEMETRY_POST_URL, + resolveEventData, + formatEventDataAttribute, reset: () => { + pendingCommandMetrics.length = 0 config.delete(configKey) }, getOnMessage, getOffMessage, - prompt: async (productName, binName, privacyPolicyLink) => { - console.log(` - How you use ${productName} provides us with important data that we can use - to make our products better. Please read our guide for more information on - the data we anonymously collect, and how we use it. - ${privacyPolicyLink || defaultPrivacyPolicyLink} - `) - - const response = await inquirer.prompt([{ - name: 'accept', - type: 'confirm', - message: `Would you like to allow ${productName} to collect anonymous usage data?` - }]) - if (response.accept) { - config.set(`${configKey}.optOut`, false) - console.log(getOnMessage(productName, binName)) - trackEvent('telemetry-prompt', 'accepted') - } else { - // we will set optOut to true after tracking this one event - // todo: check if tty error - config.set(`${configKey}.optOut`, false) - console.log(getOffMessage(binName)) - trackEvent('telemetry-prompt', 'declined') - config.set(`${configKey}.optOut`, true) - } + getNoticeMessage, + notice: (productName, privacyPolicyLink) => { + console.log(getNoticeMessage(productName, privacyPolicyLink)) + config.set(`${configKey}.optOut`, false) + trackEvent('telemetry-notice', 'shown') } } diff --git a/test/flush-worker.test.js b/test/flush-worker.test.js new file mode 100644 index 0000000..5dd9c18 --- /dev/null +++ b/test/flush-worker.test.js @@ -0,0 +1,134 @@ +/* + * Copyright 2026 Adobe Inc. All rights reserved. + * This file is licensed to you under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. You may obtain a copy + * of the License at http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under + * the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS + * OF ANY KIND, either express or implied. See the License for the specific language + * governing permissions and limitations under the License. + */ + +const { createFetch } = require('@adobe/aio-lib-core-networking') + +const fetch = createFetch() +const { main } = require('../src/flush-worker') + +const PROXY = 'https://telemetry-proxy.example/api/v1/web/dx-excshell-1/telemetry' +const METRIC = { name: 'aio.cli.telemetry', type: 'gauge', value: 1, timestamp: 1000, attributes: { eventName: 'postrun' } } +const BODY = JSON.stringify([{ metrics: [METRIC] }]) +const flushArg = (body = BODY, extraHeaders) => { + const base = { body, postUrl: PROXY } + if (extraHeaders !== undefined) { + base.headers = extraHeaders + } + return JSON.stringify(base) +} + +describe('flush-worker main()', () => { + let origArgv + + beforeEach(() => { + origArgv = process.argv + fetch.mockReset() + }) + + afterEach(() => { + process.argv = origArgv + }) + + test('POSTs batches from payload on success', async () => { + fetch.mockResolvedValue({ ok: true }) + + process.argv = ['node', 'flush-worker.js', flushArg()] + await main() + + expect(fetch).toHaveBeenCalledTimes(1) + const [url, opts] = fetch.mock.calls[0] + expect(url).toBe(PROXY) + expect(opts.method).toBe('POST') + expect(opts.headers['Content-Type']).toBe('application/json') + expect(opts.headers['Api-Key']).toBeUndefined() + + const posted = JSON.parse(opts.body) + expect(posted.batches).toHaveLength(1) + expect(posted.batches[0].metrics).toHaveLength(1) + expect(posted.batches[0].metrics[0]).toEqual(METRIC) + }) + + test('does not throw when fetch rejects', async () => { + fetch.mockRejectedValue(new Error('network error')) + + process.argv = ['node', 'flush-worker.js', flushArg()] + await expect(main()).resolves.toBeUndefined() + expect(fetch).toHaveBeenCalledTimes(1) + }) + + test('does not throw when fetch resolves with non-ok response', async () => { + fetch.mockResolvedValue({ ok: false, status: 503 }) + + process.argv = ['node', 'flush-worker.js', flushArg()] + await expect(main()).resolves.toBeUndefined() + expect(fetch).toHaveBeenCalledTimes(1) + }) + + test('does not throw when fetch resolves with ok false and no status', async () => { + fetch.mockResolvedValue({ ok: false }) + + process.argv = ['node', 'flush-worker.js', flushArg()] + await expect(main()).resolves.toBeUndefined() + }) + + test('does not throw when fetch resolves with undefined response', async () => { + fetch.mockResolvedValue(undefined) + + process.argv = ['node', 'flush-worker.js', flushArg()] + await expect(main()).resolves.toBeUndefined() + }) + + test('merges optional payload headers as overrides after defaults', async () => { + fetch.mockResolvedValue({ ok: true }) + process.argv = ['node', 'flush-worker.js', flushArg(BODY, { 'X-Custom-Proxy': 'unit-test' })] + await main() + expect(fetch.mock.calls[0][1].headers).toEqual( + expect.objectContaining({ + 'Content-Type': 'application/json', + 'X-Custom-Proxy': 'unit-test' + }) + ) + }) + + test('returns silently when argv[2] is missing', async () => { + process.argv = ['node', 'flush-worker.js'] + await main() + expect(fetch).not.toHaveBeenCalled() + }) + + test('returns silently when argv[2] is malformed JSON', async () => { + process.argv = ['node', 'flush-worker.js', 'not-json{{{'] + await main() + expect(fetch).not.toHaveBeenCalled() + }) + + test('returns silently when postUrl is missing', async () => { + process.argv = ['node', 'flush-worker.js', JSON.stringify({ body: BODY })] + await main() + expect(fetch).not.toHaveBeenCalled() + }) + + test('returns silently when body is not a JSON array', async () => { + process.argv = ['node', 'flush-worker.js', JSON.stringify({ body: '{"foo":1}', postUrl: PROXY })] + await main() + expect(fetch).not.toHaveBeenCalled() + }) + + test('uses default headers when headers omitted from payload', async () => { + fetch.mockResolvedValue({ ok: true }) + process.argv = ['node', 'flush-worker.js', JSON.stringify({ body: BODY, postUrl: PROXY })] + await main() + expect(fetch.mock.calls[0][1].headers).toEqual( + expect.objectContaining({ 'Content-Type': 'application/json' }) + ) + }) +}) diff --git a/test/hooks.test.js b/test/hooks.test.js index ff23cd6..2009a1b 100644 --- a/test/hooks.test.js +++ b/test/hooks.test.js @@ -11,149 +11,140 @@ */ const { createFetch } = require('@adobe/aio-lib-core-networking') -const inquirer = require('inquirer') const config = require('@adobe/aio-lib-core-config') -jest.mock('inquirer') jest.mock('@adobe/aio-lib-core-config') +jest.mock('child_process', () => ({ + spawn: jest.fn(() => ({ unref: jest.fn() })) +})) const fetch = createFetch() +const { spawn } = require('child_process') +const telemetryLib = require('../src/telemetry-lib') const mockPackageJson = { bin: { aio: '' }, name: 'name', - aioTelemetry: { - fetchHeaders: { 'Content-Type': 'application/json' }, - postUrl: 'https://httpstat.us/200' - } + aioTelemetry: {} } describe('hook interfaces', () => { + let noticeSpy beforeEach(() => { fetch.mockReset() + spawn.mockClear() + config.get.mockReset() + config.set.mockClear() + noticeSpy = jest.spyOn(telemetryLib, 'notice') + }) + afterEach(() => { + noticeSpy.mockRestore() }) - test('command-error', async () => { + // oclif does not run `postrun` after a command throws, so the error hook itself must flush. + test('command-error flushes immediately (no postrun follows an error)', async () => { + config.get.mockImplementation((key) => (String(key).includes('optOut') ? false : 'clientid')) + telemetryLib.init('name@0.0.1', 'aio', mockPackageJson.aioTelemetry) const hook = require('../src/hooks/command-error') expect(typeof hook).toBe('function') await hook({ message: 'msg' }) - expect(fetch).toHaveBeenCalledTimes(1) - expect(fetch).toHaveBeenCalledWith(expect.any(String), - expect.objectContaining({ body: expect.stringContaining('"_adobeio":{"eventType":"command-error"') })) + expect(spawn).toHaveBeenCalledTimes(1) + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + const body = JSON.parse(flushPayload.body) + expect(body[0].metrics.map((m) => m.attributes.eventName)).toEqual(['command-error']) + expect(body[0].metrics[0].attributes.commandSuccess).toBe(false) }) - test('command-not-found', async () => { + // command-not-found is terminal too: no command runs, so no postrun. + test('command-not-found flushes immediately', async () => { + config.get.mockImplementation((key) => (String(key).includes('optOut') ? false : 'clientid')) + telemetryLib.init('name@0.0.1', 'aio', mockPackageJson.aioTelemetry) const hook = require('../src/hooks/command-not-found') expect(typeof hook).toBe('function') await hook({ id: 'id' }) - expect(fetch).toHaveBeenCalledTimes(1) - expect(fetch).toHaveBeenCalledWith(expect.any(String), - expect.objectContaining({ body: expect.stringContaining('"_adobeio":{"eventType":"command-not-found"') })) + expect(spawn).toHaveBeenCalledTimes(1) + const flushPayloadNf = JSON.parse(spawn.mock.calls[0][1][1]) + const bodyNf = JSON.parse(flushPayloadNf.body) + expect(bodyNf[0].metrics.map((m) => m.attributes.eventName)).toEqual(['command-not-found']) + expect(bodyNf[0].metrics[0].attributes.commandSuccess).toBe(false) }) - /** - * Should prompt when config.get(optOut) returns undefined - * post results - */ - test('init prompt accept:true', async () => { + test('init shows one-time notice on first run', async () => { const preEnv = process.env process.env = { ...preEnv, CI: undefined, GITHUB_ACTIONS: undefined } const hook = require('../src/hooks/init') expect(typeof hook).toBe('function') - inquirer.prompt = jest.fn().mockResolvedValue({ accept: true }) config.get = jest.fn().mockReturnValue(undefined) await hook({ config: { name: 'name', version: '0.0.1', pjson: mockPackageJson }, argv: [] }) - expect(inquirer.prompt).toHaveBeenCalled() - expect(fetch).toHaveBeenCalledWith(expect.any(String), - expect.objectContaining({ body: expect.stringContaining('"_adobeio":{"eventType":"telemetry-prompt","eventData":"accepted"') })) - expect(fetch).toHaveBeenCalledTimes(1) + expect(noticeSpy).toHaveBeenCalled() + expect(config.set).toHaveBeenCalledWith('aio-cli-telemetry.optOut', false) + expect(spawn).not.toHaveBeenCalled() + await telemetryLib.trackEvent('postrun') + expect(spawn).toHaveBeenCalledTimes(1) + const flushPayloadAcc = JSON.parse(spawn.mock.calls[0][1][1]) + const bodyAcc = JSON.parse(flushPayloadAcc.body) + expect(bodyAcc[0].metrics.map((m) => m.attributes.eventName)).toEqual(['telemetry-notice', 'postrun']) + expect(bodyAcc[0].metrics[0].attributes.eventData).toBe('shown') process.env = preEnv }) - test('init prompt - full coverage when run by gh actions', async () => { + test('init - no notice for telemetry commands', async () => { const preEnv = process.env process.env = { ...preEnv, CI: undefined, GITHUB_ACTIONS: undefined } const hook = require('../src/hooks/init') expect(typeof hook).toBe('function') - inquirer.prompt = jest.fn().mockResolvedValue({ accept: true }) config.get = jest.fn().mockReturnValue(undefined) await hook({ id: 'telemetry', config: { name: 'name', version: '0.0.1' }, argv: [] }) - expect(inquirer.prompt).not.toHaveBeenCalled() - expect(fetch).not.toHaveBeenCalled() + expect(noticeSpy).not.toHaveBeenCalled() + expect(spawn).not.toHaveBeenCalled() process.env = preEnv }) - test('init prompt - dont ask for telemetry for telemetry commands', async () => { - const hook = require('../src/hooks/init') - expect(typeof hook).toBe('function') - inquirer.prompt = jest.fn().mockResolvedValue({ accept: true }) - config.get = jest.fn().mockReturnValue(undefined) - await hook({ id: 'telemetry', config: { name: 'name', version: '0.0.1' }, argv: [] }) - expect(inquirer.prompt).not.toHaveBeenCalled() - expect(fetch).not.toHaveBeenCalled() - }) - - test('init prompt - dont run when oclif is generating readme', async () => { - const hook = require('../src/hooks/init') - expect(typeof hook).toBe('function') - inquirer.prompt = jest.fn().mockResolvedValue({ accept: true }) - config.get = jest.fn().mockReturnValue(undefined) - await hook({ id: 'readme', config: { name: 'name', version: '0.0.1' }, argv: [] }) - expect(inquirer.prompt).not.toHaveBeenCalled() - expect(fetch).not.toHaveBeenCalled() - }) - - test('init prompt - dont run when oclif is generating readme and CI is off', async () => { + test('init - no notice when oclif is generating readme', async () => { const preEnv = process.env - process.env = { ...preEnv, CI: undefined } + process.env = { ...preEnv, CI: undefined, GITHUB_ACTIONS: undefined } const hook = require('../src/hooks/init') expect(typeof hook).toBe('function') - inquirer.prompt = jest.fn().mockResolvedValue({ accept: true }) config.get = jest.fn().mockReturnValue(undefined) await hook({ id: 'readme', config: { name: 'name', version: '0.0.1' }, argv: [] }) - expect(inquirer.prompt).not.toHaveBeenCalled() - expect(fetch).not.toHaveBeenCalled() + expect(noticeSpy).not.toHaveBeenCalled() + expect(spawn).not.toHaveBeenCalled() process.env = preEnv }) - test('no prompt when process.env.CI', async () => { + test('init - no notice when process.env.CI', async () => { const preEnv = process.env process.env = { ...preEnv, CI: 'true' } let hook jest.isolateModules(() => { hook = require('../src/hooks/init') }) - expect(typeof hook).toBe('function') - inquirer.prompt = jest.fn().mockResolvedValue({ accept: false }) config.get = jest.fn().mockReturnValue(undefined) - expect(inquirer.prompt).not.toHaveBeenCalled() await hook({ config: { name: 'name', version: '0.0.1' }, argv: ['--verbose'] }) - expect(fetch).not.toHaveBeenCalled() - expect(inquirer.prompt).not.toHaveBeenCalled() + expect(noticeSpy).not.toHaveBeenCalled() + expect(spawn).not.toHaveBeenCalled() process.env = preEnv }) /** - * Should prompt when config.get(optOut) returns undefined - * should still post after prompt even though it is declined, this is the last post + * When the user has already chosen a state (optOut defined), isNull() is false, + * so the notice is not shown again. */ - test('init prompt accept:false', async () => { + test('init - no notice when telemetry state already set', async () => { const preEnv = process.env process.env = { ...preEnv, CI: undefined, GITHUB_ACTIONS: undefined } const hook = require('../src/hooks/init') expect(typeof hook).toBe('function') - inquirer.prompt = jest.fn().mockResolvedValue({ accept: false }) - config.get = jest.fn().mockReturnValue(undefined) - await hook({ config: { name: 'name', version: '0.0.1' }, argv: ['--verbose'] }) - expect(inquirer.prompt).toHaveBeenCalled() - expect(fetch).toHaveBeenCalledTimes(1) - expect(fetch).toHaveBeenCalledWith(expect.any(String), - expect.objectContaining({ body: expect.stringContaining('"_adobeio":{"eventType":"telemetry-prompt","eventData":"declined"') })) + config.get = jest.fn().mockReturnValue(false) // optOut already set -> isNull() false + await hook({ config: { name: 'name', version: '0.0.1', pjson: mockPackageJson }, argv: [] }) + expect(noticeSpy).not.toHaveBeenCalled() + expect(spawn).not.toHaveBeenCalled() process.env = preEnv }) test('telemetry', async () => { + telemetryLib.init('name@0.0.1', 'aio', mockPackageJson.aioTelemetry) const hook = require('../src/hooks/telemetry') expect(typeof hook).toBe('function') config.get = jest @@ -161,42 +152,47 @@ describe('hook interfaces', () => { .mockReturnValueOnce('clientid') .mockReturnValueOnce(false) - await hook({ message: 'msg' }) - expect(fetch).toHaveBeenCalledTimes(1) - expect(fetch).toHaveBeenCalledWith(expect.any(String), - expect.objectContaining({ body: expect.stringContaining('"_adobeio":{"eventType":"telemetry-custom-event"') })) + await hook({ data: { feature: 'x' } }) + expect(spawn).not.toHaveBeenCalled() + await telemetryLib.trackEvent('postrun') + expect(spawn).toHaveBeenCalledTimes(1) + const flushPayloadCe = JSON.parse(spawn.mock.calls[0][1][1]) + const bodyCe = JSON.parse(flushPayloadCe.body) + expect(bodyCe[0].metrics.map((m) => m.attributes.eventName)).toEqual(['telemetry-custom-event', 'postrun']) }) test('postrun', async () => { + config.get.mockImplementation((key) => (String(key).includes('optOut') ? false : 'clientid')) + telemetryLib.init('name@0.0.1', 'aio', mockPackageJson.aioTelemetry) const hook = require('../src/hooks/postrun') expect(typeof hook).toBe('function') await hook({ Command: { id: 'id' }, argv: ['--hello'] }) - expect(fetch).toHaveBeenCalledTimes(1) - expect(fetch).toHaveBeenCalledWith(expect.any(String), - expect.objectContaining({ body: expect.stringContaining('"_adobeio":{"eventType":"postrun"') })) + expect(spawn).toHaveBeenCalledTimes(1) + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + expect(flushPayload.headers).toBeUndefined() + expect(flushPayload.body).toContain('"eventName":"postrun"') }) /** * Should NOT prompt even though config.get(optOut) returned undefined * --no-telemetry flag wins */ - test('init --no-telemetry no prompt', async () => { + test('init --no-telemetry no notice', async () => { const hook = require('../src/hooks/init') expect(typeof hook).toBe('function') - inquirer.prompt = jest.fn() config.get = jest.fn().mockReturnValue(undefined) await hook({ config: { name: 'name', version: '0.0.1' }, argv: ['--no-telemetry'] }) - expect(inquirer.prompt).not.toHaveBeenCalled() - expect(fetch).not.toHaveBeenCalled() + expect(noticeSpy).not.toHaveBeenCalled() + expect(spawn).not.toHaveBeenCalled() }) test('prerun', async () => { const hook = require('../src/hooks/prerun') expect(typeof hook).toBe('function') await hook({ Command: { id: 'id' }, argv: ['--hello'] }) - expect(fetch).not.toHaveBeenCalled() + expect(spawn).not.toHaveBeenCalled() await hook({ Command: { id: 'id' }, argv: ['--hello', '--no-telemetry'] }) - expect(fetch).not.toHaveBeenCalled() + expect(spawn).not.toHaveBeenCalled() }) test('prerun disables telemetry for postrun', async () => { @@ -205,6 +201,6 @@ describe('hook interfaces', () => { config.get.mockResolvedValue('clientidxyz') await preHook({ Command: { id: 'id' }, argv: ['--hello', '--no-telemetry'] }) await postHook({ Command: { id: 'id' }, argv: ['--hello', '--no-telemetry'] }) - expect(fetch).not.toHaveBeenCalled() + expect(spawn).not.toHaveBeenCalled() }) }) diff --git a/test/index.test.js b/test/index.test.js index b0306e9..7eafdae 100644 --- a/test/index.test.js +++ b/test/index.test.js @@ -15,7 +15,6 @@ const TheCommand = require('../src/commands/telemetry') const { stdout } = require('stdout-stderr') const config = require('@adobe/aio-lib-core-config') -jest.mock('inquirer') jest.mock('@adobe/aio-lib-core-config') const fetch = createFetch() diff --git a/test/telemetry-lib.test.js b/test/telemetry-lib.test.js index 45f4255..df40047 100644 --- a/test/telemetry-lib.test.js +++ b/test/telemetry-lib.test.js @@ -15,13 +15,18 @@ const telemetryLib = require('../src/telemetry-lib') const config = require('@adobe/aio-lib-core-config') jest.mock('@adobe/aio-lib-core-config') +jest.mock('child_process', () => ({ + spawn: jest.fn(() => ({ unref: jest.fn() })) +})) const fetch = createFetch() +const { spawn } = require('child_process') describe('telemetry-lib', () => { beforeEach(() => { jest.resetModules() fetch.mockReset() + spawn.mockClear() }) test('exports messages', async () => { @@ -30,25 +35,503 @@ describe('telemetry-lib', () => { expect(telemetryLib.getOnMessage).toBeDefined() expect(telemetryLib.getOnMessage).toBeInstanceOf(Function) + + expect(telemetryLib.getNoticeMessage).toBeInstanceOf(Function) + expect(telemetryLib.notice).toBeInstanceOf(Function) + }) + + test('getNoticeMessage uses the default privacy link, or a provided one', async () => { + const withDefault = telemetryLib.getNoticeMessage('Adobe Developer CLI') + expect(withDefault).toMatch('on by default') + expect(withDefault).toMatch('developer.adobe.com/app-builder/docs/guides/telemetry') + + const withLink = telemetryLib.getNoticeMessage('Adobe Developer CLI', 'https://example.com/privacy') + expect(withLink).toMatch('https://example.com/privacy') }) test('exports init function', async () => { expect(telemetryLib.init).toBeDefined() expect(telemetryLib.init).toBeInstanceOf(Function) - telemetryLib.init('a@4', 'binTest') + telemetryLib.init('a@4', 'binTest', {}) telemetryLib.enable() expect(config.set).toHaveBeenCalledWith('binTest-cli-telemetry.optOut', false) telemetryLib.disable() expect(config.set).toHaveBeenCalledWith('binTest-cli-telemetry.optOut', true) }) - test('uses client id from config', async () => { + test('trackEvent does not throw when spawn fails while launching flush worker', async () => { + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binSpawnFail', {}) + spawn.mockImplementationOnce(() => { + throw new Error('spawn EPERM') + }) + await expect(telemetryLib.trackEvent('postrun')).resolves.toBeUndefined() + }) + + test('buffers non-postrun until postrun merges one batch', async () => { + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binBuf', {}) + await telemetryLib.trackEvent('telemetry-custom-event') + expect(spawn).not.toHaveBeenCalled() + await telemetryLib.trackEvent('postrun') + expect(spawn).toHaveBeenCalledTimes(1) + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + const body = JSON.parse(flushPayload.body) + expect(body[0].metrics).toHaveLength(2) + expect(body[0].metrics[0].attributes.eventName).toBe('telemetry-custom-event') + expect(body[0].metrics[1].attributes.eventName).toBe('postrun') + }) + + test('terminal command-error flushes buffered events without a postrun', async () => { + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binErr', {}) + await telemetryLib.trackEvent('telemetry-custom-event') + expect(spawn).not.toHaveBeenCalled() + await telemetryLib.trackEvent('command-error', { message: 'boom' }) + expect(spawn).toHaveBeenCalledTimes(1) + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + const body = JSON.parse(flushPayload.body) + expect(body[0].metrics.map((m) => m.attributes.eventName)).toEqual(['telemetry-custom-event', 'command-error']) + }) + + test('a typo (command-not-found then host command-error) is recorded as a single command-not-found', async () => { + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binOnce', {}) + // a typo fires command-not-found (from oclif) and then command-error (rethrown by the host) + await telemetryLib.trackEvent('command-not-found', 'bogus') + await telemetryLib.trackEvent('command-error', { message: 'Run aio help' }) + expect(spawn).toHaveBeenCalledTimes(1) + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + const body = JSON.parse(flushPayload.body) + expect(body[0].metrics.map((m) => m.attributes.eventName)).toEqual(['command-not-found']) + }) + + test('only the first command-error after a not-found is suppressed; a later one still flushes', async () => { + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binReset', {}) + await telemetryLib.trackEvent('command-not-found', 'bogus') // flush #1 + await telemetryLib.trackEvent('command-error', { message: 'Run aio help' }) // dropped: the typo rethrow + await telemetryLib.trackEvent('command-error', { message: 'unrelated later error' }) // flush #2 + expect(spawn).toHaveBeenCalledTimes(2) + expect(JSON.parse(JSON.parse(spawn.mock.calls[0][1][1]).body)[0].metrics.map((m) => m.attributes.eventName)).toEqual(['command-not-found']) + expect(JSON.parse(JSON.parse(spawn.mock.calls[1][1][1]).body)[0].metrics.map((m) => m.attributes.eventName)).toEqual(['command-error']) + }) + + test('"Did you mean ...? Yes": not-found then the suggested command runs and its postrun flushes', async () => { + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binYes', {}) + await telemetryLib.trackEvent('command-not-found', 'telemetryd') // flush #1 + telemetryLib.trackPrerun('telemetry', [], Date.now()) // suggestion accepted -> command runs + await telemetryLib.trackEvent('postrun') // flush #2 + expect(spawn).toHaveBeenCalledTimes(2) + expect(JSON.parse(JSON.parse(spawn.mock.calls[1][1][1]).body)[0].metrics.map((m) => m.attributes.eventName)).toEqual(['postrun']) + }) + + test('"Did you mean ...? Yes" then the suggested command errors: that command-error is NOT masked', async () => { + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binYesErr', {}) + await telemetryLib.trackEvent('command-not-found', 'telemetryd') // flush #1 + telemetryLib.trackPrerun('telemetry', [], Date.now()) // suggestion accepted -> command runs (clears the flag) + await telemetryLib.trackEvent('command-error', { message: 'the real command failed' }) // flush #2 + expect(spawn).toHaveBeenCalledTimes(2) + expect(JSON.parse(JSON.parse(spawn.mock.calls[1][1][1]).body)[0].metrics.map((m) => m.attributes.eventName)).toEqual(['command-error']) + }) + + test('nested commands in one process each flush (no masking): two postruns => two flushes', async () => { + // A command that calls config.runCommand re-fires prerun/postrun for the child; init runs once. + // The child postrun must NOT suppress the parent terminal event. + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binNested', {}) + telemetryLib.trackPrerun('app:clean', [], Date.now()) + await telemetryLib.trackEvent('postrun') // child (e.g. app:clean) succeeds + telemetryLib.trackPrerun('app:undeploy', [], Date.now()) + await telemetryLib.trackEvent('command-error', { message: 'parent failed after child succeeded' }) + expect(spawn).toHaveBeenCalledTimes(2) + expect(JSON.parse(JSON.parse(spawn.mock.calls[0][1][1]).body)[0].metrics.map((m) => m.attributes.eventName)).toEqual(['postrun']) + expect(JSON.parse(JSON.parse(spawn.mock.calls[1][1][1]).body)[0].metrics.map((m) => m.attributes.eventName)).toEqual(['command-error']) + }) + + test('postrun adds durationMs to eventData when the hook omits payload and prerunTimer is set', async () => { + global.prerunTimer = Date.now() - 40 + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binDur', {}) + await telemetryLib.trackEvent('postrun') + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + const body = JSON.parse(flushPayload.body) + const ed = JSON.parse(body[0].metrics[0].attributes.eventData) + expect(ed.durationMs).toBeGreaterThanOrEqual(35) + expect(ed.durationMs).toBeLessThan(60000) + }) + + test('postrun keeps non-empty eventData as-is', async () => { + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binKeep', {}) + await telemetryLib.trackEvent('postrun', { source: 'test' }) + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + const body = JSON.parse(flushPayload.body) + expect(JSON.parse(body[0].metrics[0].attributes.eventData)).toEqual({ source: 'test' }) + }) + + test('postrun eventData is {} when payload empty and prerunTimer is not a number', async () => { + const prev = global.prerunTimer + delete global.prerunTimer + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binNoTimer', {}) + await telemetryLib.trackEvent('postrun') + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + const body = JSON.parse(flushPayload.body) + expect(body[0].metrics[0].attributes.eventData).toBe('{}') + global.prerunTimer = prev + }) + + test('trackEvent includes invocation_context and agent_name in payload', async () => { + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binTest', {}) + await telemetryLib.trackEvent('postrun') + expect(spawn).toHaveBeenCalled() + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + const body = JSON.parse(flushPayload.body) + const attributes = body[0].metrics[0].attributes + expect(attributes).toHaveProperty('invocation_context') + expect(attributes).toHaveProperty('agent_name') + expect(['agent', 'human']).toContain(attributes.invocation_context) + }) + + test('init uses built-in default postUrl when host omits aioTelemetry.postUrl and env', async () => { + const orig = process.env.AIO_TELEMETRY_POST_URL + delete process.env.AIO_TELEMETRY_POST_URL + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binDefaultUrl', {}) + await telemetryLib.trackEvent('postrun') + expect(spawn).toHaveBeenCalled() + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + expect(flushPayload.postUrl).toBe(telemetryLib.DEFAULT_TELEMETRY_POST_URL) + if (orig !== undefined) process.env.AIO_TELEMETRY_POST_URL = orig + }) + + test('init uses AIO_TELEMETRY_POST_URL when remoteConf.postUrl is omitted', async () => { + const orig = process.env.AIO_TELEMETRY_POST_URL + process.env.AIO_TELEMETRY_POST_URL = 'https://env.example/ingest' + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binEnv', {}) + await telemetryLib.trackEvent('postrun') + expect(spawn).toHaveBeenCalled() + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + expect(flushPayload.postUrl).toBe('https://env.example/ingest') + if (orig !== undefined) process.env.AIO_TELEMETRY_POST_URL = orig + else delete process.env.AIO_TELEMETRY_POST_URL + }) + + test('init with two args defaults remoteConf and uses AIO_TELEMETRY_POST_URL', async () => { + const orig = process.env.AIO_TELEMETRY_POST_URL + process.env.AIO_TELEMETRY_POST_URL = 'https://env-default.example/ingest' + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binTwoArg') + await telemetryLib.trackEvent('postrun') + expect(spawn).toHaveBeenCalled() + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + expect(flushPayload.postUrl).toBe('https://env-default.example/ingest') + if (orig !== undefined) process.env.AIO_TELEMETRY_POST_URL = orig + else delete process.env.AIO_TELEMETRY_POST_URL + }) + + test('remoteConf.postUrl takes precedence over AIO_TELEMETRY_POST_URL', async () => { + const orig = process.env.AIO_TELEMETRY_POST_URL + process.env.AIO_TELEMETRY_POST_URL = 'https://env.example/ingest' config.get.mockReturnValue('clientidxyz') - telemetryLib.init('a@4', 'binTest2') - await telemetryLib.trackEvent('test-event') - expect(config.get).toHaveBeenCalledWith('binTest2-cli-telemetry.clientId') - expect(config.get).toHaveBeenCalledWith('binTest2-cli-telemetry.optOut', 'global') - expect(fetch).toHaveBeenCalledWith(expect.any(String), - expect.objectContaining({ body: expect.stringContaining('"clientId":"clientidxyz"') })) + telemetryLib.init('a@4', 'binPrec', { postUrl: 'https://cli-config.example/proxy' }) + await telemetryLib.trackEvent('postrun') + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + expect(flushPayload.postUrl).toBe('https://cli-config.example/proxy') + if (orig !== undefined) process.env.AIO_TELEMETRY_POST_URL = orig + else delete process.env.AIO_TELEMETRY_POST_URL + }) + + test.each(['true', '1', 'yes'])( + 'trackEvent does not post when AIO_TELEMETRY_DISABLED is %s', + async (value) => { + const orig = process.env.AIO_TELEMETRY_DISABLED + process.env.AIO_TELEMETRY_DISABLED = value + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binTest', {}) + await telemetryLib.trackEvent('postrun') + expect(spawn).not.toHaveBeenCalled() + if (orig !== undefined) process.env.AIO_TELEMETRY_DISABLED = orig + else delete process.env.AIO_TELEMETRY_DISABLED + } + ) + + test.each(['0', 'false', 'no', ''])( + 'trackEvent posts when AIO_TELEMETRY_DISABLED is %j', + async (value) => { + const orig = process.env.AIO_TELEMETRY_DISABLED + if (value === '') { + delete process.env.AIO_TELEMETRY_DISABLED + } else { + process.env.AIO_TELEMETRY_DISABLED = value + } + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binEnvNotDisabled', {}) + await telemetryLib.trackEvent('postrun') + expect(spawn).toHaveBeenCalled() + if (orig !== undefined) process.env.AIO_TELEMETRY_DISABLED = orig + else delete process.env.AIO_TELEMETRY_DISABLED + } + ) + + test.each(['true', '1', 'yes'])( + 'trackEvent does not buffer when AIO_TELEMETRY_DISABLED is %s for non-postrun', + async (value) => { + const orig = process.env.AIO_TELEMETRY_DISABLED + process.env.AIO_TELEMETRY_DISABLED = value + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binNoQueue', {}) + await telemetryLib.trackEvent('command-error', { message: 'x' }) + expect(spawn).not.toHaveBeenCalled() + if (orig !== undefined) process.env.AIO_TELEMETRY_DISABLED = orig + else delete process.env.AIO_TELEMETRY_DISABLED + } + ) + + test('string eventData is stored without extra JSON quotes in flush payload', async () => { + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binStrEd', {}) + await telemetryLib.trackEvent('telemetry-prompt', 'accepted') + expect(spawn).not.toHaveBeenCalled() + await telemetryLib.trackEvent('postrun') + expect(spawn).toHaveBeenCalledTimes(1) + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + const body = JSON.parse(flushPayload.body) + const metric = body[0].metrics[0] + expect(metric.attributes.eventData).toBe('accepted') + expect(metric.attributes.eventData).not.toMatch(/^"/) + }) + + test('postrun flush payload omits headers when host has no aioTelemetry.fetchHeaders', async () => { + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binNoFlushHdr', {}) + await telemetryLib.trackEvent('postrun') + expect(spawn).toHaveBeenCalled() + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + expect(flushPayload.headers).toBeUndefined() + }) + + test('postrun flush payload passes only aioTelemetry.fetchHeaders as worker overrides', async () => { + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binFlushHdr', { + fetchHeaders: { 'x-correlation-id': 'abc', 'api-key': 'nope' } + }) + await telemetryLib.trackEvent('postrun') + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + expect(flushPayload.headers).toEqual({ 'x-correlation-id': 'abc' }) + }) + + test('trackEvent sends agent context when CURSOR_AGENT env is set', async () => { + const orig = process.env.CURSOR_AGENT + process.env.CURSOR_AGENT = '1' + config.get.mockReturnValue('clientidxyz') + telemetryLib.init('a@4', 'binTest', {}) + await telemetryLib.trackEvent('postrun') + expect(spawn).toHaveBeenCalled() + const flushPayload = JSON.parse(spawn.mock.calls[0][1][1]) + const body = JSON.parse(flushPayload.body) + const attributes = body[0].metrics[0].attributes + expect(attributes.invocation_context).toBe('agent') + expect(attributes.agent_name).toBe('cursor') + if (orig !== undefined) process.env.CURSOR_AGENT = orig + else delete process.env.CURSOR_AGENT + }) +}) + +describe('resolveEventData', () => { + test('non-postrun with undefined raw yields {}', () => { + expect(telemetryLib.resolveEventData('command-error', undefined)).toEqual({}) + }) +}) + +describe('formatEventDataAttribute', () => { + test('string is returned as-is', () => { + expect(telemetryLib.formatEventDataAttribute('accepted')).toBe('accepted') + expect(telemetryLib.formatEventDataAttribute('')).toBe('') }) + + test('object and array use JSON.stringify', () => { + expect(telemetryLib.formatEventDataAttribute({ a: 1 })).toBe('{"a":1}') + expect(telemetryLib.formatEventDataAttribute([1, 2])).toBe('[1,2]') + expect(telemetryLib.formatEventDataAttribute({})).toBe('{}') + }) + + test('number and boolean use String()', () => { + expect(telemetryLib.formatEventDataAttribute(0)).toBe('0') + expect(telemetryLib.formatEventDataAttribute(false)).toBe('false') + }) + + test('bigint uses String()', () => { + expect(telemetryLib.formatEventDataAttribute(42n)).toBe('42') + }) + + test('symbol falls through to String()', () => { + expect(telemetryLib.formatEventDataAttribute(Symbol('s'))).toBe('Symbol(s)') + }) + + test('undefined yields {}', () => { + expect(telemetryLib.formatEventDataAttribute(undefined)).toBe('{}') + }) + + test('null yields JSON null token', () => { + expect(telemetryLib.formatEventDataAttribute(null)).toBe('null') + }) +}) + +describe('getInvocationContext', () => { + test('returns human when no agent env vars are set', () => { + const result = telemetryLib.getInvocationContext({}) + expect(result).toEqual({ isAgent: false, agentName: null }) + }) + + test('returns agent cursor when CURSOR_AGENT is set', () => { + const result = telemetryLib.getInvocationContext({ CURSOR_AGENT: '1' }) + expect(result).toEqual({ isAgent: true, agentName: 'cursor' }) + }) + + test('returns agent with name when AGENT is set to a value', () => { + const result = telemetryLib.getInvocationContext({ AGENT: 'goose' }) + expect(result).toEqual({ isAgent: true, agentName: 'goose' }) + }) + + test('returns agent generic when AGENT=1', () => { + const result = telemetryLib.getInvocationContext({ AGENT: '1' }) + expect(result).toEqual({ isAgent: true, agentName: 'generic' }) + }) + + test('returns aio-opt-in when AIO_AGENT is set', () => { + const result = telemetryLib.getInvocationContext({ AIO_AGENT: '1' }) + expect(result).toEqual({ isAgent: true, agentName: 'aio-opt-in' }) + }) + + test('returns aio-opt-in when AIO_INVOCATION_CONTEXT=agent', () => { + const result = telemetryLib.getInvocationContext({ AIO_INVOCATION_CONTEXT: 'agent' }) + expect(result).toEqual({ isAgent: true, agentName: 'aio-opt-in' }) + }) + + test('returns human when AIO_INVOCATION_CONTEXT is not agent', () => { + const result = telemetryLib.getInvocationContext({ AIO_INVOCATION_CONTEXT: 'human' }) + expect(result).toEqual({ isAgent: false, agentName: null }) + }) + + test('returns github-copilot when Copilot Chat PATH markers are present', () => { + const result = telemetryLib.getInvocationContext({ + PATH: '/usr/local/bin:/Users/test/Library/Application Support/Code/User/globalStorage/github.copilot-chat/debugCommand:/Users/test/Library/Application Support/Code/User/globalStorage/github.copilot-chat/copilotCli' + }) + expect(result).toEqual({ isAgent: true, agentName: 'github-copilot' }) + }) + + test('returns github-copilot when PATH uses Windows separators', () => { + const result = telemetryLib.getInvocationContext({ + PATH: 'C:\\Program Files\\Git\\cmd;C:\\Users\\test\\AppData\\Roaming\\Code\\User\\globalStorage\\github.copilot-chat\\debugCommand' + }) + expect(result).toEqual({ isAgent: true, agentName: 'github-copilot' }) + }) + + test('returns human when PATH does not contain Copilot Chat markers', () => { + const result = telemetryLib.getInvocationContext({ PATH: '/usr/local/bin:/usr/bin:/bin' }) + expect(result).toEqual({ isAgent: false, agentName: null }) + }) + + test('returns human when PATH is null or undefined', () => { + expect(telemetryLib.getInvocationContext({ PATH: null })).toEqual({ isAgent: false, agentName: null }) + expect(telemetryLib.getInvocationContext({ PATH: undefined })).toEqual({ isAgent: false, agentName: null }) + }) + + test('AGENT takes precedence over tool-specific when both set', () => { + const result = telemetryLib.getInvocationContext({ AGENT: 'goose', CURSOR_AGENT: '1' }) + expect(result).toEqual({ isAgent: true, agentName: 'goose' }) + }) + + test('ignores empty string env values', () => { + const result = telemetryLib.getInvocationContext({ CURSOR_AGENT: '' }) + expect(result).toEqual({ isAgent: false, agentName: null }) + }) + + test('returns agent generic when AGENT=true', () => { + const result = telemetryLib.getInvocationContext({ AGENT: 'true' }) + expect(result).toEqual({ isAgent: true, agentName: 'generic' }) + }) + + test('returns aio-opt-in when AI_AGENT is set', () => { + const result = telemetryLib.getInvocationContext({ AI_AGENT: 'my-agent' }) + expect(result).toEqual({ isAgent: true, agentName: 'my-agent' }) + }) + + test('returns generic when AI_AGENT=1', () => { + const result = telemetryLib.getInvocationContext({ AI_AGENT: '1' }) + expect(result).toEqual({ isAgent: true, agentName: 'generic' }) + }) + + test('returns claude when CLAUDECODE is set', () => { + expect(telemetryLib.getInvocationContext({ CLAUDECODE: '1' })).toEqual({ isAgent: true, agentName: 'claude' }) + }) + + test('returns claude when CLAUDE_CODE is set', () => { + expect(telemetryLib.getInvocationContext({ CLAUDE_CODE: '1' })).toEqual({ isAgent: true, agentName: 'claude' }) + }) + + test('returns gemini when GEMINI_CLI is set', () => { + expect(telemetryLib.getInvocationContext({ GEMINI_CLI: '1' })).toEqual({ isAgent: true, agentName: 'gemini' }) + }) + + test('returns codex when CODEX_SANDBOX is set', () => { + expect(telemetryLib.getInvocationContext({ CODEX_SANDBOX: '1' })).toEqual({ isAgent: true, agentName: 'codex' }) + }) + + test('returns augment when AUGMENT_AGENT is set', () => { + expect(telemetryLib.getInvocationContext({ AUGMENT_AGENT: '1' })).toEqual({ isAgent: true, agentName: 'augment' }) + }) + + test('returns cline when CLINE_ACTIVE is set', () => { + expect(telemetryLib.getInvocationContext({ CLINE_ACTIVE: '1' })).toEqual({ isAgent: true, agentName: 'cline' }) + }) + + test('returns opencode when OPENCODE_CLIENT is set', () => { + expect(telemetryLib.getInvocationContext({ OPENCODE_CLIENT: '1' })).toEqual({ isAgent: true, agentName: 'opencode' }) + }) + + test('returns replit when REPL_ID is set', () => { + expect(telemetryLib.getInvocationContext({ REPL_ID: 'abc123' })).toEqual({ isAgent: true, agentName: 'replit' }) + }) +}) + +describe('AIO_TELEMETRY_DISABLED', () => { + let orig + + beforeEach(() => { + orig = process.env.AIO_TELEMETRY_DISABLED + }) + + afterEach(() => { + if (orig !== undefined) process.env.AIO_TELEMETRY_DISABLED = orig + else delete process.env.AIO_TELEMETRY_DISABLED + }) + + test.each(['true', '1', 'yes'])( + 'isEnabled returns false when AIO_TELEMETRY_DISABLED is %s', + (value) => { + process.env.AIO_TELEMETRY_DISABLED = value + telemetryLib.init('a@4', 'binTest', {}) + const config = require('@adobe/aio-lib-core-config') + config.get.mockReturnValue(false) + expect(telemetryLib.isEnabled()).toBe(false) + } + ) + + test.each(['true', '1', 'yes'])( + 'isNull returns false when AIO_TELEMETRY_DISABLED is %s', + (value) => { + process.env.AIO_TELEMETRY_DISABLED = value + telemetryLib.init('a@4', 'binTest', {}) + const config = require('@adobe/aio-lib-core-config') + config.get.mockReturnValue(undefined) + expect(telemetryLib.isNull()).toBe(false) + } + ) })