feat: add lazy CLI extensions

[andreatnvidia]

Summary

Adds a minimal entry-point seam for optional packages to provide top-level data-designer command groups without importing their implementation until selected.

Related issues

Closes #853

Depends on #856
Related to #850

Changes

• Discover optional groups from data_designer.cli entry-point metadata.
• Keep root help metadata-only and load only the selected extension factory.
• Validate Data Designer version compatibility and the returned Click command only after command selection.
• Reject built-in collisions and duplicate names deterministically without loading either target.
• Ignore malformed registrations with targeted warnings so they do not disable built-in commands.
• Register the optional Slurm package as the first extension.
• Extend installed-wheel smoke tests with base-only, extra-installed, lazy-import, and absolute/relative startup-budget checks.
• Document the entry-point and callable contract.

Impact

A base-only install has no slurm command. Installing data-designer[slurm] adds data-designer slurm, while root help and unrelated commands do not import data_designer.slurm or compatibility-checking modules.

F2 intentionally establishes the extension seam and an empty Slurm group. Leaf subcommands are owned by follow-up API work. A generic nested-command test proves the seam supports deeper command trees and preserves leaf exit codes.

This PR is stacked on #856 and should be retargeted to main after F1 merges.

Testing

• .venv/bin/pytest packages/data-designer/tests packages/data-designer-slurm/tests -p no:cacheprovider (1146 passed, 1 skipped)
• make test-slurm-wheel-install (0.574 s base median, 0.568 s extension median; 1 s base and 100 ms overhead budgets)
• .venv/bin/ruff check --fix .
• .venv/bin/ruff format .
• Pre-commit hooks on changed files

[greptile-apps]

Greptile Summary

Adds metadata-only discovery and lazy loading of optional top-level CLI command groups.

• Introduces entry-point discovery, collision handling, compatibility checks, and deferred extension factories.
• Registers the optional Slurm package as the first CLI extension.
• Adds unit and installed-wheel coverage for dispatch, isolation, metadata failures, and startup budgets.
• Documents the extension contract and lazy-loading behavior.

Confidence Score: 5/5

The PR appears safe to merge; no concrete blocking or independently actionable issue was identified.

The extension remains metadata-only during root help, resolves only after selection, preserves nested command dispatch and exit status, and handles malformed or conflicting registrations according to the documented contract.

Important Files Changed

Filename	Overview
packages/data-designer/src/data_designer/cli/lazy_group.py	Implements metadata-only extension discovery, deferred compatibility validation and loading, and deterministic collision handling; no actionable defect was established.
packages/data-designer/src/data_designer/cli/main.py	Enables extension discovery only on the root CLI group while preserving existing lazy built-ins.
packages/data-designer-slurm/src/data_designer/slurm/cli.py	Adds the minimal Slurm Typer group factory required by the new entry-point contract.
packages/data-designer-slurm/pyproject.toml	Registers the Slurm command factory under the new data_designer.cli entry-point group.
packages/data-designer/tests/cli/test_lazy_group.py	Provides broad coverage of lazy loading, nested dispatch, compatibility failures, malformed registrations, and collisions.
scripts/test_slurm_package_install.py	Extends installed-wheel smoke tests to verify extension visibility, import isolation, dispatch, and CLI startup budgets.

Sequence Diagram

sequenceDiagram
    participant User
    participant Root as Data Designer CLI
    participant Metadata as Entry-point metadata
    participant Factory as Extension factory
    participant Command as Extension command
    User->>Root: data-designer --help
    Root->>Metadata: Discover names and summaries
    Metadata-->>Root: Extension metadata
    Root-->>User: Root help without imports
    User->>Root: data-designer slurm ...
    Root->>Metadata: Resolve selected provider
    Root->>Root: Validate compatibility
    Root->>Factory: Load and call selected factory
    Factory-->>Root: click.Command
    Root->>Command: Dispatch remaining arguments
    Command-->>User: Output and exit status

Loading

_{Reviews (1): Last reviewed commit: "fix: isolate CLI extension failures" | Re-trigger Greptile}