Align MCPRegistry CRD with registry server v2 config format

[rdimitrov]

Summary

The registry server (toolhive-registry-server) moved to a v2 config format that separates data sources from registry views and adds claims-based authorization. The operator's MCPRegistry CRD was still generating v1 config (flat registries[] with inline source configs), so operators deploying via the CRD got none of the v2 authorization features.

This updates the MCPRegistry CRD and config generation to produce v2-compatible config with full parity:

• Split spec.registries into spec.sources (data source definitions) and spec.registries (lightweight views referencing sources by name)
• Add claims support on both sources and registries via apiextensionsv1.JSON
• Add ManagedSource, KubernetesSource (with namespace selectors), and URLSource as new source types
• Add authz (role-based access control) and publicPaths to auth config
• Add maxMetaSize and dynamicAuth (AWS RDS IAM) to database config
• Add telemetryConfig with tracing (sampling) and metrics sub-configs
• Remove PVC source type (not a supported use case)

Closes #4572

Type of change

• New feature

Test plan

• Unit tests (task test)
• Linting (task lint-fix)
• Manual testing (describe below)

Deployed updated CRDs to a Kind cluster, applied a v2 MCPRegistry resource with sources + registries + claims, ran the operator locally, and verified the generated ConfigMap produces correct v2 YAML with sources[] and registries[].

Changes

File	Change
cmd/thv-operator/api/v1alpha1/mcpregistry_types.go	New MCPRegistrySourceConfig, MCPRegistryViewConfig, ManagedSource, KubernetesSource, URLSource, MCPRegistryAuthzConfig, MCPRegistryRolesConfig, MCPRegistryTelemetryConfig, MCPRegistryDynamicAuthConfig; claims via apiextensionsv1.JSON; PublicPaths on auth; MaxMetaSize/DynamicAuth on database; removed PVCSource
cmd/thv-operator/api/v1alpha1/zz_generated.deepcopy.go	Regenerated
cmd/thv-operator/pkg/registryapi/config/config.go	v2 SourceConfig/RegistryConfig structs, claims deserialization, authz mapping, telemetry/database config mapping, URL source support, removed PVC config building
cmd/thv-operator/pkg/registryapi/config/config_test.go	All tests migrated to v2 types, removed PVC tests
cmd/thv-operator/pkg/registryapi/deployment.go	Iterate Spec.Sources for git auth mounts
cmd/thv-operator/pkg/registryapi/podtemplatespec.go	WithRegistrySourceMounts takes []MCPRegistrySourceConfig, removed PVC volume mounting
cmd/thv-operator/pkg/registryapi/{*_test.go}	All tests migrated to v2 types
cmd/thv-operator/test-integration/mcp-registry/*.go	Test helpers and integration tests migrated; builder syncs source names; removed PVC test file
deploy/charts/operator-crds/**	Regenerated CRD manifests
docs/operator/crd-api.md	Regenerated CRD API reference docs
examples/operator/mcp-registries/*.yaml	Example manifests updated to v2 format; removed PVC and multi-source PVC examples

Does this introduce a user-facing change?

Yes. This is a breaking CRD change — spec.registries changes meaning from source definitions to registry views. Since this is v1alpha1 (explicitly unstable), an in-place update is acceptable. Existing MCPRegistry resources must be updated to the new format.

Migration example:

Before:

spec:
  registries:
    - name: production
      format: toolhive
      configMapRef: { name: prod-registry, key: registry.json }

After:

spec:
  sources:
    - name: production
      format: toolhive
      configMapRef: { name: prod-registry, key: registry.json }
  registries:
    - name: default
      sources: ["production"]

Special notes for reviewers

• Claims typing: Uses apiextensionsv1.JSON for claims fields on sources and registries, and for role entries in authz config. The config generator deserializes and validates that values are string or []string before emitting map[string]any in the YAML output.
• No auto-injection: Users must explicitly define all sources including Kubernetes discovery sources. This avoids implicit behavior that would be surprising with claims-based authorization.
• PVC source removed: The PVC source type was removed as it is not a supported use case.
• PR size: The diff is large but most of it is generated code (CRD manifests, deepcopy), test migrations, and example YAML updates. The core logic changes are in mcpregistry_types.go and config.go.

Generated with Claude Code

[github-actions]

Large PR Detected

This PR exceeds 1000 lines of changes and requires justification before it can be reviewed.

How to unblock this PR:

Add a section to your PR description with the following format:

## Large PR Justification

[Explain why this PR must be large, such as:]
- Generated code that cannot be split
- Large refactoring that must be atomic
- Multiple related changes that would break if separated
- Migration or data transformation

Alternative:

Consider splitting this PR into smaller, focused changes (< 1000 lines each) for easier review and reduced risk.

See our Contributing Guidelines for more details.

---

This review will be automatically dismissed once you add the justification section.

[codecov]

Codecov Report

❌ Patch coverage is 35.24229% with 147 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.70%. Comparing base (d68ea47) to head (2741ab3).
⚠️ Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
cmd/thv-operator/pkg/registryapi/config/config.go	31.94%	127 Missing and 20 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4653      +/-   ##
==========================================
- Coverage   68.80%   68.70%   -0.11%     
==========================================
  Files         506      507       +1     
  Lines       52597    52837     +240     
==========================================
+ Hits        36189    36301     +112     
- Misses      13613    13720     +107     
- Partials     2795     2816      +21     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ChrisJBurns]

Multi-Agent Code Review

Reviewed by three specialized agents: Code Reviewer (Go style/conventions), Kubernetes Expert (CRD/operator patterns), and Security Advisor (auth, claims, threat modeling). See inline comments.

Legend: 🔴 Must Fix | 🟡 Strongly Recommended | 🟢 Nice to Have

Confirmed Positive

• Sources/registries split is well-designed (K8s Expert)
• apiextensionsv1.JSON is the correct type choice for claims (K8s Expert)
• DeepCopy generation is correct for all apiextensionsv1.JSON fields (K8s Expert)
• deserializeClaims is safe and well-implemented (Security Advisor)
• PVC removal is clean across all files (Code Reviewer)
• Test coverage is thorough (all agents)

Review performed by Claude Code using code-reviewer, kubernetes-expert, and security-advisor agents.

[rdimitrov]

Local Validation Results

Tested end-to-end on a Kind cluster with thv-registry-api:v1.0.0 (the new release that supports v2 config format).

Setup

• Kind cluster with CNPG PostgreSQL
• Operator built from this branch with task operator-deploy-local
• Registry API image: ghcr.io/stacklok/thv-registry-api:v1.0.0

MCPRegistry Applied

sources:
  - name: toolhive-catalog
    format: toolhive
    configMapRef: { name: toolhive-registry-data, key: registry.json }
    syncPolicy: { interval: "1h" }
    filter:
      tags:
        include: ["production"]
  - name: k8s-discovery
    kubernetes:
      namespaces: [test-registry]
registries:
  - name: default
    sources: [toolhive-catalog, k8s-discovery]

Config Generation

Generated v2 config.yaml with correct structure:

• sources[] with file path for ConfigMap, kubernetes with namespace selector
• registries[] as view referencing both sources
• Tag filter correctly included
• Database and auth config passed through

Registry Server

"Loaded configuration","source_count":2,"registry_count":1,"auth_mode":"anonymous"

• DB migrations ran successfully
• ConfigMap source synced 2 servers with tag filtering (both matched production tag)
• Kubernetes source initialized (RBAC errors for cluster-scope listing are pre-existing)
• /health → {"status":"healthy"}
• /v1/sources → both sources with sync=complete
• /v1/registries → default view with both source references
• MCPRegistry status: Phase=Running, Condition Ready=True

CEL Validation Rules

All admission-time validations working:

Test	Expected	Result
No source type on a source	Rejected	exactly one source type must be specified
Two source types on one source	Rejected	exactly one source type must be specified
Two managed sources	Rejected	at most one managed source is allowed
Authz with anonymous auth	Rejected	authz configuration has no effect when auth mode is anonymous
pvcRef field (removed)	Rejected	unknown field "spec.sources[0].pvcRef"
Valid v2 MCPRegistry	Accepted	Reconciled to Ready

Generated with Claude Code

chunky changes