Scope schema discovery to target host's cluster in multi-cluster CHI

[lukas-pfannschmidt-tr]

Summary

Fixes #1964.

HostCreateTables used api.ClickHouseInstallation{} as the scope for Names(NameFQDNs, ...) when building the endpoint list passed to QueryUnzip2Columns / QueryAny. In a CHI that defines multiple clusters, this walked hosts from every cluster and allowed QueryAny to pick a source node outside the target cluster.

The SQL does filter by cluster name via clusterAllReplicas('<target>', system.tables), but sqlCreateTableReplicated joins against the executing node's local system.databases:

FROM clusterAllReplicas('<cluster>', system.tables) tables
LOCAL JOIN system.databases databases on (databases.name = tables.database)
WHERE database NOT IN (...) AND databases.engine IN ('Ordinary','Atomic','Memory','Lazy')

So the returned set of CREATE statements depends on which cluster's node answered first — resulting in missing or incorrect schemas on newly added replicas in multi-cluster CHIs.

Timeline (for context)

• The CHI-wide scoping in schema discovery dates back to CreatePodFQDNsOfCHI(host.GetCHI()) pre-2021 and was preserved through the 2021 unification (6b946799d) and the 2024 schemer refactor (b64a6241d).
• The LOCAL JOIN system.databases form landed in d49187d0b (first in release-0.23.6), which made the executing-node dependency stricter.
• The 0.26 "rework k8s DNS" commit 6d625de69 added a trailing dot to patternNamespaceDomain (%s.svc.cluster.local.). That fixed slow/failing DNS resolution under ndots:5, but it also removed an accidental failure-mode that had been masking the scoping bug: before 0.26, cross-cluster endpoints in the CHI-wide list could fail DNS quickly and QueryAny would fall through to a same-cluster endpoint. After 0.26, every CHI endpoint resolves reliably, so QueryAny returns from whichever is first in the slice — which, with CHI-wide scoping, can be a node from a different cluster.

The 0.26 DNS change didn't cause this bug — it exposed it. The root cause is the schema-discovery endpoint scope.

Change

Use api.Cluster{} scope in getReplicatedObjectsSQLs and getDistributedObjectsSQLs, so schema-discovery endpoints are restricted to the target host's own cluster. This matches the scoping already used by shouldCreateReplicatedObjects / shouldCreateDistributedObjects for the related gating logic.

Six call sites updated:

• pkg/model/chi/schemer/replicated.go (databases, tables, functions)
• pkg/model/chi/schemer/distributed.go (databases, tables, functions)

Test plan

• go build ./...
• Manual verification on a multi-cluster CHI: scale one cluster and confirm new replicas receive the correct schemas sourced from the same cluster.

Single-cluster CHIs are unaffected: cluster-scoped FQDNs equal CHI-scoped FQDNs in that case.

[sunsingerus]

LGTM after a 3-angle review (correctness / security / code quality).

What this fixes: schema discovery in multi-cluster CHIs was using CHI-wide endpoint scope (api.ClickHouseInstallation{}) but the surrounding gating (shouldCreateReplicatedObjects / shouldCreateDistributedObjects at replicated.go:28-29 and distributed.go:28) already used cluster scope. With LOCAL JOIN system.databases semantics introduced in d49187d and trailing-dot FQDNs in 6d625de, an off-cluster host could respond to QueryAny and surface a different cluster's database catalog — leaking schemas across cluster boundaries inside the same CHI.

Why it's safe to merge as-is:

• Six mechanical replacements in two files. Type-switch in createFQDNs (pkg/model/chi/namer/name.go:279) already handles the api.Cluster scope path.
• Single-cluster CHIs unaffected — cluster-scoped FQDNs equal CHI-scoped FQDNs in that case.
• Aligns discovery scope with the gating that was already cluster-scoped — closes an inconsistency, doesn't introduce one.

Optional follow-up (non-blocking): unit test asserting the scope argument passed to s.Names(interfaces.NameFQDNs, ...) is api.Cluster{} for both getReplicatedObjectsSQLs and getDistributedObjectsSQLs. The schemer package currently has no tests, so this would establish the pattern.