Telemetry proxy - agent detection

README.md

@@ -37,33 +37,116 @@ _See code: [src/commands/telemetry/index.js](https://github.com/adobe/aio-cli-pl
 <!-- commandsstop -->
 
 ## Configuration
-The following values need to be set when this plugin is hosted by different CLIs
-- `aioTelemetry`: defined object in root cli package.json with values:
-  - `postUrl` : Where to post telemetry data
-  - `postHeaders`: Any specific headers that need to be posted with telemetry data (ex. x-api-key)
-  - `productPrivacyPolicyLink`: A link to display to users when prompting to optIn
-- `productName`: How to refer to the cli when user is prompted to enable telemetry
-  - this value is read from `displayName` or `name` of the cli's package.json
-- `productBin`: Output in help text
-  - ex. To turn telemetry on run `${productBin} telemetry on`
-  - this value is read from 'bin' of the cli's package.json, if the package exports more than 1 bin the first is used
 
-## POST data
+When this plugin is hosted by different CLIs:
 
-Here is an example of the event data as posted:
-```
-{ "id": 656915165813,
-  "timestamp": 1673404918437,
-  "_adobeio": {
-    "eventType": "telemetry-prompt",
-    "eventData": "accepted",
-    "cliVersion": "@adobe/aio-cli@9.1.1",
-    "clientId": 264421030538,
-    "commandDuration": 5661,
-    "commandFlags": "",
-    "commandSuccess": true,
-    "nodeVersion": "v14.20.0",
-    "osNameVersion": "macOS"
+- `aioTelemetry` (optional object in the root `package.json` of the **host CLI** — the same `pjson` oclif passes into the init hook):
+  - `postUrl` (optional string): HTTPS URL of the telemetry proxy that receives POSTed metric batches. Use this when your CLI should send telemetry to a different App Builder action or gateway than the plugin default.
+  - `fetchHeaders`: Optional extra headers merged into telemetry requests (`Content-Type` is always set)
+  - `productPrivacyPolicyLink`: A link shown in the one-time telemetry notice (what is collected and how to opt out)
+- `productName`: How to refer to the CLI in the telemetry notice (from `displayName` or `name` in `package.json`)
+- `productBin`: Shown in help text (from `bin` in `package.json`; if several bins exist, the first is used). Example: run `${productBin} telemetry on`
+
+### Overriding the telemetry POST URL
+
+Resolution order (first match wins):
+
+1. **`aioTelemetry.postUrl`** in the host CLI `package.json`
+2. **`AIO_TELEMETRY_POST_URL`** environment variable (non-empty string)
+3. **Built-in default** in the plugin (`DEFAULT_TELEMETRY_POST_URL` in [`src/telemetry-lib.js`](https://github.com/adobe/aio-cli-plugin-telemetry/blob/master/src/telemetry-lib.js))
+
+Host `package.json` example:
+
+```json
+{
+  "name": "my-cli",
+  "bin": { "mycli": "./bin/run.js" },
+  "aioTelemetry": {
+    "postUrl": "https://<namespace>-<project>.adobeio-static.net/api/v1/web/<package>/<action>"
   }
 }
 ```
+
+Environment override (no `package.json` change; useful for CI, staging, or local proxy debugging):
+
+```sh
+export AIO_TELEMETRY_POST_URL='https://<namespace>-<project>.adobeio-static.net/api/v1/web/<package>/<action>'
+mycli app deploy
+```
+
+The resolved URL is passed to the flush worker on each telemetry send; it applies for the rest of that CLI process after `init` runs.
+
+## Opting out via environment variable
+
+Telemetry is suppressed when `AIO_TELEMETRY_DISABLED` is set to one of **`true`**, **`1`**, or **`yes`** (exact match; case-sensitive). Other values such as `0`, `false`, `no`, or an empty string do **not** disable telemetry via this variable.
+
+This does not change the persisted opt-out state. Useful for CI pipelines and scripted environments.
+
+```sh
+AIO_TELEMETRY_DISABLED=true aio app deploy
+AIO_TELEMETRY_DISABLED=1 aio app deploy
+AIO_TELEMETRY_DISABLED=yes aio app deploy
+```
+
+## Flush architecture
+
+Telemetry is **best-effort**: events are not persisted when the proxy is down or the network fails.
+
+A flush happens on a **terminal event** — `postrun` (command succeeded), `command-error` (command threw), or `command-not-found`. oclif runs `postrun` only on success, so `command-error`/`command-not-found` must flush on their own; otherwise error telemetry (the most valuable signal) would be buffered and silently dropped on exit. On a terminal event, any in-memory metrics from earlier hooks in the same command are merged with the terminal metric and the combined batch is handed off to a **fire-and-forget detached subprocess** (`src/flush-worker.js`). The parent CLI spawns the worker and immediately `unref()`s it, so the CLI can exit without waiting for the HTTP POST. If the POST fails (network error or non-2xx response), the batch is dropped; telemetry must not block or slow normal CLI use.
+
+Non-terminal events (for example `telemetry-notice`) are held in an **in-memory buffer** until the next terminal event flushes them. If the process exits before any terminal event (crash, `SIGKILL`), buffered events are lost. The buffer is cleared when telemetry is disabled or when `init` runs again (new command session).
+
+## Agent detection
+
+The plugin detects whether the CLI is being invoked by an AI agent or a human by inspecting environment variables at the time of the event. The detected context is included in every event as `invocation_context` (`"agent"` or `"human"`) and `agent_name`.
+
+Supported agents detected automatically:
+
+| Environment variable | Detected agent name |
+|---|---|
+| `AGENT` | value of the variable (or `"generic"` if `1`/`true`) |
+| `AI_AGENT` | value of the variable (or `"generic"` if `1`/`true`) |
+| `AIO_AGENT` | `aio-opt-in` |
+| `AIO_INVOCATION_CONTEXT=agent` | `aio-opt-in` |
+| `CURSOR_AGENT` | `cursor` |
+| `CLAUDECODE` / `CLAUDE_CODE` | `claude` |
+| `GEMINI_CLI` | `gemini` |
+| `CODEX_SANDBOX` | `codex` |
+| `AUGMENT_AGENT` | `augment` |
+| `CLINE_ACTIVE` | `cline` |
+| `OPENCODE_CLIENT` | `opencode` |
+| `REPL_ID` | `replit` |
+| `PATH` containing `github.copilot-chat` | `github-copilot` |
+
+To opt into agent tracking without setting a tool-specific variable, set `AIO_INVOCATION_CONTEXT=agent`.
+
+## POST data
+
+The `eventData` attribute is always a string. Objects and arrays are stored as a JSON text (e.g. `"{}"`, `"{\"message\":\"…\"}"`). String payloads (such as the telemetry notice outcome `shown`) are stored as that plain text without an extra layer of JSON quoting. Numbers and booleans use their usual string forms (`"0"`, `"false"`).
+
+Example shape of the metric payload:
+
+```json
+[{
+  "metrics": [{
+    "name": "aio.cli.telemetry",
+    "type": "gauge",
+    "value": 1,
+    "timestamp": 1673404918437,
+    "attributes": {
+      "eventName": "postrun",
+      "eventData": "{}",
+      "cliVersion": "@adobe/aio-cli@11.0.2",
+      "clientId": 264421030538,
+      "command": "app:deploy",
+      "commandDuration": 5661,
+      "commandFlags": "",
+      "commandSuccess": true,
+      "nodeVersion": "v22.21.1",
+      "osNameVersion": "macOS Sequoia 15.4",
+      "invocation_context": "human",
+      "agent_name": "unknown"
+    }
+  }]
+}]
+```

package.json

@@ -9,10 +9,7 @@
     "@adobe/aio-lib-core-networking": "^5.0.4",
     "@oclif/core": "^4",
     "ci-info": "^4.0.0",
-    "debug": "^4.1.1",
-    "inquirer": "^8.2.1",
-    "os-name": "^4.0.1",
-    "splunk-logging": "^0.11.1"
+    "debug": "^4.1.1"
   },
   "devDependencies": {
     "@adobe/eslint-config-aio-lib-config": "5.0.0",

src/flush-worker.js

@@ -0,0 +1,87 @@
+/*
+Copyright 2026 Adobe. All rights reserved.
+This file is licensed to you under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License. You may obtain a copy
+of the License at http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing, software distributed under
+the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS
+OF ANY KIND, either express or implied. See the License for the specific language
+governing permissions and limitations under the License.
+*/
+
+/**
+ * Telemetry flush worker — spawned as a detached subprocess so the parent CLI can
+ * exit immediately without waiting on the HTTP POST.
+ *
+ * Accepts a single CLI argument: a JSON-encoded object with shape
+ * { body: string, postUrl: string, headers?: object } where `body` is a serialised
+ * New Relic metric batch array (same shape as the parent builds for fetch), `postUrl`
+ * is the App Builder proxy, and `headers` (when present) are optional overrides merged
+ * after the worker defaults (never pass secrets such as api-key).
+ *
+ * Failed deliveries are dropped; telemetry is best-effort and must not affect the CLI.
+ */
+
+'use strict'
+
+const debug = require('debug')('aio-telemetry:flush-worker')
+const { createFetch } = require('@adobe/aio-lib-core-networking')
+
+const fetch = createFetch()
+
+const DEFAULT_HEADERS = {
+  'Content-Type': 'application/json'
+}
+
+/**
+ * POSTs the metric batch from argv. Swallows all errors.
+ * @returns {Promise<void>}
+ */
+async function main () {
+  let batches
+  let postUrl
+  let requestHeaders = { ...DEFAULT_HEADERS }
+  try {
+    const parsed = JSON.parse(process.argv[2])
+    const { body, postUrl: url, headers } = parsed
+    if (!url || typeof url !== 'string') {
+      return
+    }
+    postUrl = url
+    if (headers && typeof headers === 'object') {
+      const safe = Object.fromEntries(
+        Object.entries(headers).filter(([key]) => key.toLowerCase() !== 'api-key')
+      )
+      requestHeaders = { ...DEFAULT_HEADERS, ...safe }
+    }
+    const parsedBody = JSON.parse(body)
+    if (!Array.isArray(parsedBody)) {
+      return
+    }
+    batches = parsedBody
+  } catch {
+    return
+  }
+
+  try {
+    debug('POST %s requestHeaders=%o', postUrl, requestHeaders)
+    const res = await fetch(postUrl, {
+      method: 'POST',
+      headers: requestHeaders,
+      body: JSON.stringify({ batches })
+    })
+    if (!res?.ok) {
+      const status = res?.status ?? 'unknown'
+      debug('telemetry flush non-ok: HTTP %s', status)
+    }
+  } catch (err) {
+    debug('telemetry flush failed: %O', err)
+  }
+}
+
+/* istanbul ignore next */
+if (require.main === module) {
+  main()
+}
+
+module.exports = { main }

src/hooks/init.js

@@ -37,17 +37,16 @@ module.exports = async function (opts) {
     opts.argv.filter(arg => arg.indexOf('-') === 0).join(','),
     global.prerunTimer)
 
-  // init event does not post telemetry, it stores some info that will be used later
-  // this will prompt to optIn/Out if telemetry.optIn is undefined
+  // Telemetry is opt-out (on by default). On the first run we show a one-time,
+  // non-blocking notice instead of an opt-in prompt. isNull() is true only until the
+  // notice records the default, so this shows once.
   if ((opts.argv.indexOf('--no-telemetry') < 0) &&
     !inCI &&
     telemetryLib.isNull()) {
-    // let's ask!
-    // unfortunately the `oclif-dev readme` run by prepack fires this event, which hangs CI
-    // Also we don't prompt for telemetry if the first command is a telemetry command because they
-    // are probably setting it on or off already
+    // skip in CI (handled above) and when oclif-dev readme runs (it fires this event);
+    // also skip for telemetry commands, where the user is already setting state.
     if (['readme', 'telemetry'].indexOf(opts.id) < 0) {
-      return telemetryLib.prompt(productName, binName, opts?.config?.pjson?.aioTelemetry?.productPrivacyPolicyLink)
+      telemetryLib.notice(productName, opts?.config?.pjson?.aioTelemetry?.productPrivacyPolicyLink)
     }
   }
 }