-
Notifications
You must be signed in to change notification settings - Fork 27
feat(cli): overhaul cli #617
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
7c174b4
feat(cli): typed surface + manifest snapshot + command migration
sarahxsanders 4b8a589
fix(cli): harden command surface + make skill/audit runs actually exe…
sarahxsanders 6d95cb4
fix(cli): default the audit family to events instead of the comprehen…
sarahxsanders 684167a
fix(audit): match the events-audit slide guard to the audit-events sk…
sarahxsanders 7e50262
feat(cli): resolve skill subcommands at runtime; retire audit-3000
sarahxsanders 62926f1
Merge branch 'main' into posthog-code/cli-overhaul-core
sarahxsanders e7a00af
Merge branch 'main' into posthog-code/cli-overhaul-core
sarahxsanders b050e24
chore: regenerate pnpm-lock.yaml after dropping ajv
sarahxsanders 38ea54f
docs(cli): document old→new command mapping (README + AGENTS.md)
sarahxsanders 054f08e
Merge origin/main into posthog-code/cli-overhaul-core
sarahxsanders 90c4dbf
docs: make AGENTS.md the single source of truth; CLAUDE.md points to it
sarahxsanders a358292
refactor(cli): read cli.default (was cli.recommended) from the manifest
sarahxsanders 22f3955
fix(cli): bare `wizard audit` opens the picker (events-only for now)
sarahxsanders a1b189c
docs(cli): list audit subcommands and clarify commands vs. skills
sarahxsanders f944e1e
fix(cli): bare `wizard audit` runs the default audit, not a one-item …
sarahxsanders 60ee51c
docs(cli): `wizard audit` runs events by default (not a picker)
sarahxsanders b9f3a91
fix(cli): reject unknown commands with a concise error
sarahxsanders 7cc1d55
analytics
edwinyjlim 23fb7c4
Merge branch 'posthog-code/cli-overhaul-core' of github.com:PostHog/w…
edwinyjlim 871fd49
fix(cli): surface errors from async command handlers
sarahxsanders d8dc035
Merge remote-tracking branch 'origin/posthog-code/cli-overhaul-core' …
sarahxsanders File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| # Generated at build time by scripts/generate-cli-manifest.cjs | ||
| src/lib/programs/cli-manifest.generated.ts |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,156 @@ | ||
| # AGENTS.md — PostHog Wizard | ||
|
|
||
| Instructions for all agents (and humans) working in this repo. This is the | ||
| single source of truth; [`CLAUDE.md`](CLAUDE.md) just points here. User-facing | ||
| docs: https://posthog.com/docs/ai-engineering/ai-wizard | ||
|
|
||
| The PostHog wizard (`npx @posthog/wizard`) is a CLI that adds PostHog to a user's project using an AI agent. It authenticates the user, detects their framework, runs an agent that integrates the SDK and instruments events, and walks the user through their first dashboard. All from the terminal. | ||
|
|
||
| ## Design discipline | ||
|
|
||
| This codebase follows a specific design discipline: **product knowledge never enters infrastructure code.** The runner pipeline, the TUI store, the detection loop, and the prompt assembler are machinery. They don't know what PostHog is. They don't know what a framework is. They execute a pipeline driven by typed configuration surfaces. | ||
|
|
||
| Each domain has a dedicated boundary: | ||
|
|
||
| - **Frameworks** → `FrameworkConfig` in `src/frameworks/<name>/` | ||
| - **Integration knowledge** → markdown skills in the | ||
| [context-mill](https://github.com/PostHog/context-mill) repo | ||
| - **Security policy** → YARA-X rules in the [warlock](https://github.com/PostHog/warlock) sibling repo. The wizard wires the scanner via PostToolUse/PreToolUse hooks (`src/lib/yara-hooks.ts`); the rule content itself lives in warlock. | ||
| - **Programs** → step arrays in `src/lib/programs/` | ||
| - **TUI** → screen components and primitives in `src/ui/tui/` | ||
|
|
||
| Adding a new concern means finding the narrowest existing surface, not adding logic to the runner. The wizard is small (~20K lines) because boundaries prevent damage from propagating between concerns. | ||
|
|
||
| ## Before making structural changes | ||
|
|
||
| Read `.claude/skills/wizard-development/SKILL.md` first. It covers the design discipline, a decision framework for new extensions, and warning signs that a change is drifting off-pattern. Two reference files extend it: | ||
|
|
||
| - `references/ARCHITECTURE.md` — pipeline anatomy, data flow, security | ||
| boundaries, screen resolution | ||
| - `references/ANTI-PATTERNS.md` — concrete failure modes with alternatives | ||
|
|
||
| ## Skills available | ||
|
|
||
| Four skills live under `.claude/skills/`. Read `wizard-development` first for any structural change; then load the relevant procedural skill: | ||
|
|
||
| | Skill | When to use | | ||
| |---|---| | ||
| | `wizard-development` | Before any structural change. Design principles + decision framework. | | ||
| | `adding-framework-support` | Adding a new framework integration (e.g. Ruby on Rails, Go, Angular). | | ||
| | `adding-skill-program` | Adding a new skill-based program (e.g. a new product feature setup). | | ||
| | `ink-tui` | Building or modifying TUI screens, layouts, and primitives. | | ||
|
|
||
| ## CLI command surface | ||
|
|
||
| The CLI was overhauled to a smaller, extensible command surface. **Use the new | ||
| command names.** Old names mostly no longer exist — only some are kept as aliases. | ||
|
|
||
| | Old command | New command | Status | | ||
| |---|---|---| | ||
| | `wizard integrate` | `wizard` (default flow) | command removed | | ||
| | `wizard events-audit` | `wizard audit events` | moved into `audit` family | | ||
| | `wizard audit` (single) | `wizard audit <subcommand>` | now a family — see [Audit subcommands](#audit-subcommands) | | ||
| | `wizard audit-3000` | *removed* | retired | | ||
| | `wizard revenue` | `wizard revenue-analytics` | renamed (old `revenue` removed) | | ||
| | `wizard upload-sourcemaps` | `wizard upload-source-maps` | renamed; `upload-sourcemaps` kept as alias | | ||
|
|
||
| ### Audit subcommands | ||
|
|
||
| `audit` is the only family with skill-backed subcommands today: | ||
|
|
||
| | Subcommand | What it audits | | ||
| |---|---| | ||
| | `wizard audit events` | event capture quality + cost (**default** leaf) | | ||
| | `wizard audit all` | comprehensive audit across every area | | ||
| | `wizard audit autocapture` | autocapture setup + cost | | ||
| | `wizard audit feature-flags` | feature flag usage + cost | | ||
| | `wizard audit identify` | `$identify` implementation | | ||
| | `wizard audit session-replay` | session replay setup | | ||
| | `wizard audit web-analytics` | web analytics setup (**wizard-native**, not a skill) | | ||
|
|
||
| ### Commands vs. skills (the `audit [skill]` gotcha) | ||
|
|
||
| A skill and a command are the **same machinery** — a context-mill skill becomes a | ||
| command when its `cli:` block sets `role: command`. So `wizard audit events` | ||
| *is* the `audit-events` skill, just promoted. `wizard skill <skill-name>` | ||
| ([`skill.ts`](src/commands/skill.ts)) runs a skill that **wasn't** promoted. | ||
|
|
||
| Two surfaces, one mechanism. So `wizard audit <subcommand>` is choosing an audit | ||
| area — it is **not** asking for a skill name, despite `wizard audit --help` | ||
| labelling the positional `[skill]` (a wizard-internal name we left as-is). Don't | ||
| confuse it with the top-level `wizard skill` command. | ||
|
|
||
| ### Where the surface is defined (source of truth) | ||
|
|
||
| - **Registration:** [`bin.ts`](bin.ts) — the `.use()` chain wires each command. | ||
| - **Command shape:** [`src/commands/command.ts`](src/commands/command.ts) — the | ||
| `Command` interface every command implements. | ||
| - **Flat native commands** (e.g. `revenue-analytics`, `upload-source-maps`) are | ||
| built with `nativeCommandFactory` | ||
| ([`src/commands/factories/native-command-factory.ts`](src/commands/factories/native-command-factory.ts)). | ||
| - **Family commands** (e.g. `audit`) resolve subcommands at runtime against the | ||
| `cliEntries` in `skill-menu.json`. Logic lives in | ||
| [`src/lib/programs/dispatch-family.ts`](src/lib/programs/dispatch-family.ts). | ||
| Adding a skill-backed subcommand is a **context-mill** release, not a wizard | ||
| change. | ||
|
|
||
| ### Commands vs. programs (don't confuse these) | ||
|
|
||
| - A **command** is the word a user types (`audit`, `revenue-analytics`). | ||
| - A **program** is the internal business logic (`posthog-integration`, | ||
| `revenue-analytics-setup`) that a command invokes, and that other programs | ||
| depend on via `requires: [...]`. | ||
| - `posthog-integration` is a **program id, not a command**. It powers the default | ||
| flow and is a dependency of most other programs. Do not treat it as a CLI | ||
| command or reference it in CI as one. | ||
|
|
||
| ### Adding a command alias (keep an old name working) | ||
|
|
||
| Give the `Command.name` an array of `[newName, ...legacyNames]`. yargs treats the | ||
| extra entries as aliases. See | ||
| [`src/commands/upload-sourcemaps.ts`](src/commands/upload-sourcemaps.ts). Reserve | ||
| aliases for names that external callers (users' scripts) may still use — when the | ||
| only caller is one we control, update the caller instead. | ||
|
|
||
| ## Commands | ||
|
|
||
| ```bash | ||
| pnpm install # Install dependencies | ||
| pnpm try --install-dir=<path> # Run the wizard locally against a test project | ||
| pnpm build # Compile TypeScript | ||
| pnpm test # Unit tests (builds first) | ||
| pnpm test:watch # Unit tests in watch mode | ||
| pnpm test:e2e # End-to-end tests | ||
| pnpm lint # Prettier + ESLint checks | ||
| pnpm fix # Auto-fix lint issues | ||
| pnpm dev # Build, link globally, watch for changes | ||
| ``` | ||
|
|
||
| After any change, verify with: | ||
|
|
||
| ```bash | ||
| pnpm build && pnpm test && pnpm fix | ||
| ``` | ||
|
|
||
| ## Repository conventions | ||
|
|
||
| - TypeScript everywhere. Use `type` (not `interface`) for framework context | ||
| types so they satisfy `Record<string, unknown>`. | ||
| - All UI calls go through `getUI()` (returns `WizardUI` interface). Never import | ||
| the store directly from business logic. | ||
| - Session mutations go through explicit store setters that call `emitChange()`. | ||
| Never mutate `session` directly — nanostore holds a shallow copy. | ||
| - The router resolves the active screen from session state. No imperative | ||
| navigation (`goTo`, `navigate`, `push`) anywhere. | ||
| - Never write secrets to source code or hardcode API keys. Use the | ||
| `wizard-tools` MCP server (`check_env_keys` / `set_env_values`) for `.env` file operations. | ||
| - Feedback / issues: wizard@posthog.com or | ||
| [GitHub Issues](https://github.com/posthog/wizard/issues). | ||
|
|
||
| ## Companion projects | ||
|
|
||
| - **[context-mill](https://github.com/PostHog/context-mill)** — builds and | ||
| publishes the markdown skills the wizard agent uses for framework-specific integration knowledge. Skills are decoupled from the wizard release cycle so docs and integration patterns can update independently. | ||
| - **[wizard-workbench](https://github.com/PostHog/wizard-workbench)** — the | ||
| development and testing environment. Houses framework test apps (Next.js, React Router, Django, Flask, Laravel, SvelteKit, Swift, TanStack, FastAPI) with no PostHog installed, plus an `mprocs`-driven local dev stack that runs context-mill + MCP + the wizard together with hot reload. Use this to develop and test wizard changes against real projects. | ||
| - **[warlock](https://github.com/PostHog/warlock)** — the security scanner engine for PostHog's agentic flows. Bundles YARA-X rules for prompt injection, exfiltration, destructive operations, supply chain attacks, hardcoded secrets, and PII. Engine-only: it returns matches with category/severity/action metadata; the wizard decides how to respond. New security rules belong in warlock, not in the wizard. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,84 +1,6 @@ | ||
| # PostHog Wizard | ||
|
|
||
| The PostHog wizard (`npx @posthog/wizard`) is a CLI that adds PostHog to a user's project using an AI agent. It authenticates the user, detects their framework, runs an agent that integrates the SDK and instruments events, and walks the user through their first dashboard. All from the terminal. | ||
|
|
||
| User-facing docs: https://posthog.com/docs/ai-engineering/ai-wizard | ||
|
|
||
| ## Design discipline | ||
|
|
||
| This codebase follows a specific design discipline: **product knowledge never enters infrastructure code.** The runner pipeline, the TUI store, the detection loop, and the prompt assembler are machinery. They don't know what PostHog is. They don't know what a framework is. They execute a pipeline driven by typed configuration surfaces. | ||
|
|
||
| Each domain has a dedicated boundary: | ||
|
|
||
| - **Frameworks** → `FrameworkConfig` in `src/frameworks/<name>/` | ||
| - **Integration knowledge** → markdown skills in the | ||
| [context-mill](https://github.com/PostHog/context-mill) repo | ||
| - **Security policy** → YARA-X rules in the [warlock](https://github.com/PostHog/warlock) sibling repo. The wizard wires the scanner via PostToolUse/PreToolUse hooks (`src/lib/yara-hooks.ts`); the rule content itself lives in warlock. | ||
| - **Programs** → step arrays in `src/lib/programs/` | ||
| - **TUI** → screen components and primitives in `src/ui/tui/` | ||
|
|
||
| Adding a new concern means finding the narrowest existing surface, not adding logic to the runner. The wizard is small (~20K lines) because boundaries prevent damage from propagating between concerns. | ||
|
|
||
| ## Before making structural changes | ||
|
|
||
| Read `.claude/skills/wizard-development/SKILL.md` first. It covers the design discipline, a decision framework for new extensions, and warning signs that a change is drifting off-pattern. Two reference files extend it: | ||
|
|
||
| - `references/ARCHITECTURE.md` — pipeline anatomy, data flow, security | ||
| boundaries, screen resolution | ||
| - `references/ANTI-PATTERNS.md` — concrete failure modes with alternatives | ||
|
|
||
| ## Skills available | ||
|
|
||
| Four skills live under `.claude/skills/`. Read `wizard-development` first for any structural change; then load the relevant procedural skill: | ||
|
|
||
| | Skill | When to use | | ||
| |---|---| | ||
| | `wizard-development` | Before any structural change. Design principles + decision framework. | | ||
| | `adding-framework-support` | Adding a new framework integration (e.g. Ruby on Rails, Go, Angular). | | ||
| | `adding-skill-program` | Adding a new skill-based program (e.g. a new product feature setup). | | ||
| | `ink-tui` | Building or modifying TUI screens, layouts, and primitives. | | ||
|
|
||
| ## Commands | ||
|
|
||
| ```bash | ||
| pnpm install # Install dependencies | ||
| pnpm try --install-dir=<path> # Run the wizard locally against a test project | ||
| pnpm build # Compile TypeScript | ||
| pnpm test # Unit tests (builds first) | ||
| pnpm test:watch # Unit tests in watch mode | ||
| pnpm test:e2e # End-to-end tests | ||
| pnpm lint # Prettier + ESLint checks | ||
| pnpm fix # Auto-fix lint issues | ||
| pnpm dev # Build, link globally, watch for changes | ||
| ``` | ||
|
|
||
| After any change, verify with: | ||
|
|
||
| ```bash | ||
| pnpm build && pnpm test && pnpm fix | ||
| ``` | ||
|
|
||
| ## Companion projects | ||
|
|
||
| - **[context-mill](https://github.com/PostHog/context-mill)** — builds and | ||
| publishes the markdown skills the wizard agent uses for framework-specific integration knowledge. Skills are decoupled from the wizard release cycle so docs and integration patterns can update independently. | ||
| - **[wizard-workbench](https://github.com/PostHog/wizard-workbench)** — the | ||
| development and testing environment. Houses framework test apps (Next.js, React Router, Django, Flask, Laravel, SvelteKit, Swift, TanStack, FastAPI) with no PostHog installed, plus an `mprocs`-driven local dev stack that runs context-mill + MCP + the wizard together with hot reload. Use this to develop and test wizard changes against real projects. | ||
| - **[warlock](https://github.com/PostHog/warlock)** — the security scanner engine for PostHog's agentic flows. Bundles YARA-X rules for prompt injection, exfiltration, destructive operations, supply chain attacks, hardcoded secrets, and PII. Engine-only: it returns matches with category/severity/action metadata; the wizard decides how to respond. New security rules belong in warlock, not in the wizard. | ||
|
|
||
| ## Repository conventions | ||
|
|
||
| - TypeScript everywhere. Use `type` (not `interface`) for framework context | ||
| types so they satisfy `Record<string, unknown>`. | ||
| - All UI calls go through `getUI()` (returns `WizardUI` interface). Never import | ||
| the store directly from business logic. | ||
| - Session mutations go through explicit store setters that call `emitChange()`. | ||
| Never mutate `session` directly — nanostore holds a shallow copy. | ||
| - The router resolves the active screen from session state. No imperative | ||
| navigation (`goTo`, `navigate`, `push`) anywhere. | ||
| - Never write secrets to source code or hardcode API keys. Use the | ||
| `wizard-tools` MCP server (`check_env_keys` / `set_env_values`) for `.env` file operations. | ||
| - Feedback / issues: wizard@posthog.com or | ||
| [GitHub Issues](https://github.com/posthog/wizard/issues). | ||
| # CLAUDE.md | ||
|
|
||
| Repo guidance for all agents lives in [AGENTS.md](AGENTS.md) — the single source | ||
| of truth. It's imported below so Claude Code picks it up automatically. | ||
|
|
||
| @AGENTS.md |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -32,12 +32,9 @@ if (process.env.NODE_ENV === 'test') { | |
| import { Wizard } from './src/wizard'; | ||
| import { basicIntegrationCommand } from './src/commands/basic-integration'; | ||
| import { mcpCommand } from './src/commands/mcp'; | ||
| import { integrateCommand } from './src/commands/integrate'; | ||
| import { auditCommand } from './src/commands/audit'; | ||
| import { audit3000Command } from './src/commands/audit-3000'; | ||
| import { doctorCommand } from './src/commands/doctor'; | ||
| import { migrateCommand } from './src/commands/migrate'; | ||
| import { eventsAuditCommand } from './src/commands/events-audit'; | ||
| import { revenueCommand } from './src/commands/revenue'; | ||
| import { slackCommand } from './src/commands/slack'; | ||
| import { uploadSourcemapsCommand } from './src/commands/upload-sourcemaps'; | ||
|
|
@@ -61,12 +58,9 @@ function resolveInstallDir(): string { | |
|
|
||
| Wizard.use(basicIntegrationCommand) | ||
| .use(mcpCommand) | ||
| .use(integrateCommand) | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What did this do btw I don't recall |
||
| .use(auditCommand) | ||
| .use(audit3000Command) | ||
| .use(doctorCommand) | ||
| .use(migrateCommand) | ||
| .use(eventsAuditCommand) | ||
| .use(revenueCommand) | ||
| .use(slackCommand) | ||
| .use(uploadSourcemapsCommand) | ||
|
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
changes? or should we position this as just the way it works now?