feat(cli): cli overhaul#178
Open
sarahxsanders wants to merge 23 commits into
Open
Conversation
JSDoc typedef in scripts/lib/skill-generator.js describing the optional `cli:` block that skill `config.yaml` files will adopt. Tells the wizard whether and how each skill appears in the wizard CLI (public, catalog, internal). Phase 0: schema only — the build does not parse or emit anything from this block yet. Phase 1 adds parsing and emits dist/skills/cli-manifest.json. Pairs with PostHog/wizard feat/cli-overhaul branch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
Adds a parser for the optional `cli:` block in skill `config.yaml` files, propagates the resolved block onto every expanded variant, and emits `dist/skills/cli-manifest.json` alongside `manifest.json` and `skill-menu.json`. Skills with no `cli:` block default to catalog and are not written into the new manifest. Group-level blocks set defaults for every variant; variant-level blocks override field by field. For `surface: public`, the leaf defaults to the variant's short id (so `migrate-statsig` becomes `wizard migrate statsig`); the magic `id: all` variant requires an explicit leaf at the group level. Tags the audit family, `migrate`, and `revenue-analytics` as `surface: public` so the new manifest carries real content. The multi-variant + auto-detected surfaces (`source-maps`, `error-tracking`, `logs`, `llm-analytics`) need a richer manifest shape to express auto-detection and will be tagged alongside their wizard-side renames. Pairs with PostHog/wizard feat/cli-overhaul branch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Renames the cli block fields to match the wizard's existing
`ProgramConfig.command` / `parentCommand` convention so contributors
learn one vocabulary across both repos. Field semantics are unchanged:
- `command` — the user-typed word that registers a skill
(e.g. `events` in `wizard audit events`)
- `parentCommand` — the command this skill nests under
(e.g. `audit`)
Updates the parser, the manifest emitter, the 8 tagged skill configs,
and the test suite. Also adds a YAML-to-command mapping table to the
JSDoc so contributors can read the schema as "write this to register
that command" rather than as a tree of abstract positions.
Pairs with PostHog/wizard feat/cli-overhaul.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Deletes transformation-config/skills/audit-3000/. The skill was a hackathon experiment whose content has since been split into the individual audit-* skills (audit-events, audit-feature-flags, audit-identify, audit-session-replay, audit-autocapture, plus the parent audit) — those are now what backs the new `wizard audit <leaf>` family on the wizard side. The next context-mill release will stop publishing audit-3000.zip. Build output verified: 127 skills generated (was 128), and the audit-3000 id no longer appears in manifest.json, skill-menu.json, or cli-manifest.json. Coordinated with PostHog/wizard feat/cli-overhaul which already removed the `wizard audit-3000` command in the surface migration. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 8 of the wizard CLI overhaul — companion to the new
CONTRIBUTING in PostHog/wizard.
Covers:
- The cli: block schema (surface, command, parentCommand) with the
YAML→registered-command mapping table from the JSDoc, lifted into
contributor-facing docs.
- Promotion criterion for surface: public (five checks before
promoting a skill to the public CLI; lean toward `catalog` when
in doubt, since promoting catalog→public is cheap but demoting
public→catalog breaks user scripts).
- "Adding a new skill" and "Adding a new public command" walkthroughs.
- A which-repo-does-this-change-go-in matrix.
No build behavior changes — pure docs. `npm test` 83/83.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ention
Phase 9 (context-mill side). Two concerns the plan groups together:
1. Tighter parseCliBlock validation. Every `command` and `parentCommand`
value must now satisfy:
- kebab-case, /^[a-z][a-z0-9-]*$/
- 2–20 characters
- not a yargs reserved word (help, version, completion)
- not a collision with a wizard internal flag (playground,
benchmark, yara-report, local-mcp, ci, skill)
Failures throw at build time, so a contributor can't sneak in
`cli.command: helpme` or `cli.parentCommand: snake_case`.
2. JSON Schema co-published from the build. New
`transformation-config/cli-manifest.schema.json` is copied to
`dist/skills/cli-manifest.schema.json` alongside the manifest on
every release. The wizard's prebuild fetches and validates against
it via ajv so schema drift between the two repos gets caught at
build time, not runtime.
Test coverage: 7 new cases in cli-block.test.js exercise each rejection
(kebab-case, length, reserved words, internal-flag collisions) on both
command and parentCommand. 90/90 tests passing.
Pairs with PostHog/wizard feat/cli-overhaul.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the wizard's CLAUDE.md pattern: a brief overview of where things live, a pointer to CONTRIBUTING.md for the cli: block schema and surface conventions, and the naming rules enforced by parseCliBlock at build time. Agents working on context-mill skills now see the cli: block schema and the public-surface promotion criterion as session-load context, not just buried in CONTRIBUTING.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an optional `default: true` field on the cli block. When a family
parent is invoked with no subcommand, the wizard runs the leaf marked
default — same as if the user had typed `wizard <family> <leaf>`.
Adding a non-default leaf to a family later doesn't change what bare
`wizard <family>` does, so existing usage stays stable across releases.
Two skill configs use the new field:
- audit/all → default: true. `wizard audit` runs the comprehensive
audit directly. Users who want a specific area still type the
explicit leaf (`wizard audit events`, etc.).
- revenue-analytics → restructured from a flat `command: revenue` to
`parentCommand: revenue, command: stripe, default: true`.
`wizard revenue` runs Stripe today; when chargebee lands, it'll be
added as a non-default sibling and `wizard revenue` keeps running
Stripe for existing users.
Schema + parser + manifest emit updated. The JSON Schema gains an
optional `default` boolean. `parseCliBlock` validates the type and
includes it in the resolved cli block.
Pairs with PostHog/wizard feat/cli-overhaul.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a context-mill PR changes a cli: block, the wizard repo ships a committed docs/cli.md that needs regenerating once the wizard upgrades to a release containing the change. Contributors editing cli: blocks here can't trigger that regen themselves (it's a wizard-side step) — so the right thing is to flag it in the PR description or open a tracking issue on the wizard side. Adds step 5 to the "When you're about to change a cli: block" checklist in CLAUDE.md and a "Heads up" subsection to CONTRIBUTING.md under "Adding a new public command". Cross-links to the wizard's CONTRIBUTING for the canonical regen instructions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Promoting single-option commands to family form was forced abstraction.
`wizard migrate statsig` and `wizard revenue stripe` aren't real choices
— they're one option dressed up as two-word commands. Reverting:
- revenue-analytics: cli: { surface: public, command: revenue }
(was parentCommand: revenue, command: stripe, default: true)
- migrate: cli: { surface: public, command: migrate }
(was parentCommand: migrate; command derived from variant id)
Audit stays a family because it's genuinely multi-option. `default: true`
stays on audit/all — it pre-highlights the comprehensive audit in the
picker but no longer skips the picker entirely.
Rule going forward: family form only when there are multiple distinct
options to pick from. When a second migration vendor (or a second
revenue provider) lands, restructure to a family at that moment and
document the breaking UX change in release notes.
Pairs with PostHog/wizard feat/cli-overhaul.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The schema description for the `default` field was outright wrong
after the wizard-side picker behavior changed. The agent-facing docs
were missing the flat-vs-family rule entirely.
Updated:
- transformation-config/cli-manifest.schema.json — `default` field
description rewritten. Was "this leaf runs when invoked with no
subcommand" (auto-run); now "this leaf is pre-highlighted in the
family picker" (picker always opens; default sorts it first).
- scripts/lib/skill-generator.js — JSDoc typedef for the cli: block
adds the `default` property and a new mapping-table example for
the default-leaf pattern. Also adds a "Flat vs. family" section.
- CONTRIBUTING.md — new section "Flat vs. family — the convention"
explaining the rule (flat when one option, family when many).
Mapping table gets a new example showing `default: true`. New
"What `default: true` does (and doesn't do)" section clarifies
the picker-pre-highlight semantics.
- CLAUDE.md — adds the `default` field to the parser checklist
and a brief reference to the flat-vs-family rule.
No build changes — pure docs. 90/90 tests passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Renames two commands to follow PostHog's product naming verbatim: - audit-feature-flags: command \`flags\` → \`feature-flags\` - revenue-analytics: command \`revenue\` → \`revenue-analytics\` \`flags\` was shorthand for the product "Feature Flags" — every other PostHog surface (docs, dashboard, GitHub) calls it feature-flags; the CLI shouldn't invent its own abbreviation. Same logic for \`revenue\` (the product is "Revenue Analytics"). The naming taste rule is added explicitly to CONTRIBUTING.md and the parser JSDoc with a good/bad table: | Use | Don't use | | wizard audit feature-flags | wizard audit flags | | wizard audit session-replay| wizard audit replay | | wizard revenue-analytics | wizard revenue | Mechanics (kebab-case, length, reserved words) stay where they were — parseCliBlock enforces them. This rule sits on top as taste. Pairs with PostHog/wizard feat/cli-overhaul. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Validate the auto-filled command name (variant id fallback) so a reserved word or non-kebab id can't slip past the checks an explicit command gets. Enforce at most one default: leaf per family in generateCliManifest, and reject default: on a flat command with no parentCommand. Add tests for all three and for default: flowing through into the manifest entry. Generated-By: PostHog Code Task-Id: 75f1264c-4512-4b8c-abb4-094f60f53de7
surface -> role, with values public -> command and catalog -> skill (internal unchanged); default -> recommended. Plainer vocabulary that reads as "what is this skill" rather than an abstract visibility level, and `recommended` no longer collides with the programming sense of "default". command/parentCommand keep the wizard's existing names. Touches the parser, manifest builder + validation, JSON schema, all 8 config.yaml cli blocks, tests, and the CONTRIBUTING/CLAUDE docs. The wizard's consumer side (PR #605) must move in lockstep, the manifest field names are a cross-repo contract. Generated-By: PostHog Code Task-Id: 75f1264c-4512-4b8c-abb4-094f60f53de7
# Conflicts: # transformation-config/skills/audit-3000/config.yaml
context-mill now checks its own emitted cli-manifest.json against the
cli-manifest.schema.json it publishes, using the same validator (ajv)
the wizard's prebuild uses. A manifest that violates the schema fails
`npm run build` here, in this repo, instead of surfacing later in the
wizard. Closes the gap where the emitter and the schema could silently
drift apart.
Also tightens the schema to match the build's semantics:
- if/then rule: a `role: command` entry must carry `command`
(drift-free, structural).
- documents that semantic naming rules (reserved-word / internal-flag
collisions) are enforced by the build, not the schema, so consumers
don't over-trust schema-alone validation.
Adds ajv as a dev dependency (build/release-time only; never ships to
skill consumers) and 7 tests covering valid/invalid manifests.
Generated-By: PostHog Code
Task-Id: 95407a79-1f13-4f1d-a37b-43fe4a62b857
# Conflicts: # context/cli-manifest.schema.json # context/skills/audit-3000/BUILD_NOTES.md # context/skills/audit-3000/config.yaml # context/skills/audit-3000/description.md # context/skills/audit-3000/references/1-version.md # context/skills/audit-3000/references/10-report.md # context/skills/audit-3000/references/2-init.md # context/skills/audit-3000/references/3-identification.md # context/skills/audit-3000/references/4-event-capture.md # context/skills/audit-3000/references/5-event-quality.md # context/skills/audit-3000/references/6-feature-flags.md # context/skills/audit-3000/references/6b-session-replay.md # context/skills/audit-3000/references/7-customer-enrichment.md # context/skills/audit-3000/references/8-use-case-match.md # context/skills/audit-3000/references/9-use-case-expansion.md # context/skills/audit-3000/references/use-case-match-example.md
…ifest The CLI manifest reuses uri-schema.yaml's manifest_version — there's no CLI-manifest-only version knob. Comment so nobody bumps it expecting it to affect only the CLI side. Generated-By: PostHog Code Task-Id: 95407a79-1f13-4f1d-a37b-43fe4a62b857
Moves the family's `recommended` default from `audit all` to `audit-events`, so bare `wizard audit` defaults to the events audit. `audit all` stays a registered command (`wizard audit all`); explicit leaves like `wizard audit feature-flags` still run their own audit. Only the default moves here. Dropping `audit all` entirely and the wizard-side cleanup come later in the multi-select PR, coordinated across both repos. Audit family: all, autocapture, events (recommended), feature-flags, identify, session-replay. Generated-By: PostHog Code Task-Id: 95407a79-1f13-4f1d-a37b-43fe4a62b857
gewenyu99
reviewed
Jun 10, 2026
gewenyu99
left a comment
gewenyu99
left a comment
Collaborator
There was a problem hiding this comment.
an initial fly by, will return to this after looking at wizard PR, too.
Minorly concerned about tight coupling wizard builds to context mill for command registering.
My initial thoughts is that commands are not context and are too wizard specific. What part of the command is strictly context/information? And what parts are just wizard specific?
Collaborator
There was a problem hiding this comment.
Move this to agents.md so all agents will read i think
Validation of the cli manifest now lives on the consumer side (the wizard validates the manifest it fetches against the published schema). context-mill still publishes cli-manifest.schema.json alongside the manifest, but no longer validates its own output against it. Removes the ajv self-check added earlier: - validateCliManifest() + its call in writeManifestAndMenu - the ajv import and dev-dependency - the validateCliManifest test block Keeps the schema file (the contract the wizard validates against) and the schema co-publish to dist/skills/. Config-level validation (parseCliBlock naming rules, one-recommended-per-family) stays — that's input checking, not self-validation. Generated-By: PostHog Code Task-Id: 95407a79-1f13-4f1d-a37b-43fe4a62b857
The JSON Schema was belt-and-suspenders. parseCliBlock already enforces the naming rules at build time, and the wizard side no longer validates against the schema at runtime — so context-mill was publishing a contract nobody consumed. Removing the schema file, the build-time copy step, and the project CLAUDE.md reference. Also documents how to migrate a flat command into a family (e.g. `wizard investigate` -> `wizard investigate events`). This is a breaking UX change for users, so the docs spell out the YAML diff, the release- notes obligation, and the `recommended:` escape hatch for preserving the old default-action ergonomics. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The wizard's runtime resolver needs a flat list of CLI entries to look up parentCommand+command -> skillId. Lifting that array into skill-menu.json lets the wizard reuse the existing fetchSkillMenu plumbing instead of fetching a second URL and maintaining a parallel cache. cli-manifest.json still emits unchanged for one release while the older still-baked wizard is in the wild; planned removal once that release ages out (tracked separately). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5 tasks
…-menu.json The wizard now reads cliEntries from skill-menu.json at runtime — no published consumer reads cli-manifest.json. The standalone file was dead-on-arrival, so removing it now keeps the contract honest (one file, one consumer). generateCliManifest was producing a wrapped object solely for the file write. Renamed to generateCliEntries, returns the entries array directly. Caller embeds it under skill-menu.json's cliEntries. CLAUDE.md and CONTRIBUTING.md updated to point contributors at the new location. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
sarahxsanders
force-pushed
the
feat/cli-overhaul
branch
from
00b6fc3 to
549cfc5
Compare
Collaborator
gewenyu99
approved these changes
Jun 16, 2026
| */ | ||
|
|
||
| /** | ||
| * Optional `cli:` block in a skill's `config.yaml`. |
Collaborator
There was a problem hiding this comment.
I would honestly throw this in a markdown. This comment block is just a little tooo chonky
| * role: command Pre-highlighted in the | ||
| * parentCommand: audit family picker so | ||
| * command: all `wizard audit` → Enter | ||
| * recommended: true runs this leaf. |
Collaborator
There was a problem hiding this comment.
Should this be called default
|
|
||
| const CLI_ROLES = ['command', 'skill', 'internal']; | ||
|
|
||
| // Naming convention enforcement — see context-mill/CONTRIBUTING.md and the |
Collaborator
There was a problem hiding this comment.
Maybe move this to its own file. Great that we're enforcing, not super relevant to the flow of this which makes it just a lil harder to read
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CLI OVERHAULLLLLLLLLLLLLL 🤠
two things happening here:
cli:block to skillconfig.yamlto tell the wizard how askill's command should appear
audit-3000in favor of a properauditskill family:audit events,audit feature-flagsetc.cli:blockeach skill has the option to declare:
the build parses + validates it (kebab-case, length limits, reserved-word
collisions, etc.) and emits the entries into skill-menu.json under
cliEntries. the wizard fetches skill-menu.json at runtime and uses
that array to register its command surface – no baked snapshot, no
build-time coupling between the two repos.
cli-manifest.json still ships as a standalone file for one release as
back-compat with the still-baked older wizard. it gets deleted in a
follow-up once that release ages out.
full schema and semantics in CONTRIBUTING.md.
how the commands come together
command:
wizard revenue-analyticscommand:
wizard audit eventscommand:
wizard auditthen Enter runs this one. the picker still opens(so people can see the other audit options + opt in), recommended just
sorts this to the top so the common case is one keystroke. only one leaf
per family gets to be the recommended one, and the build will yell at
you if two try to claim it.
command:
wizard skill <skill-id>call with
--skill=<id>for devflat vs family
a command is flat when there's only one option today, and a family when
the user has to pick between a few.
the rule of thumb: don't build the family shape before you need it. no
wizard migrate <vendor>while there's only one vendor – that's justabstraction for its own sake. when a second option shows up, that's the
moment to restructure into a family. the migration steps for that
restructure (and the breaking-change call-out) are in
CONTRIBUTING.md.sister PR
PostHog/wizard#617
something to note: when the wizard upgrades to a release with these
changes, we need to run
pnpm docs:clito refresh the committedcli.md(I can do this).release order (IMPORTANT):
CANT SHIP WIZARD PR FIRST — the wizard fetches
skill-menu.jsonatruntime, so the new shape (with
cliEntries) has to exist in apublished release before the wizard release goes out.
testing
npm testall passingnpm run buildgreen;dist/skills/skill-menu.jsonhascliEntrieswith the expected entries and recommended flags;
dist/skills/cli-manifest.jsonstill emits the same shape for back-compatibility