adapter: don't leak arrangement compression into the realized config

doc/user/content/reference/system-catalog/mz_internal.md

@@ -181,7 +181,7 @@ shape is in [`mz_clusters`](../mz_catalog/#mz_clusters).
 | `status`       | [`text`]         | The lifecycle status of the reconfiguration: `in-progress` while the controller converges on the target, then a terminal `finalized`, `timed-out`, `cancelled`, or `resource-exhausted`. The record is retained after it settles, so the latest outcome stays inspectable until a later reconfiguration overwrites it. |
 | `deadline`     | [`mz_timestamp`] | The deadline by which the reconfiguration must complete. After it passes, the `on_timeout` action applies. |
 | `on_timeout`   | [`text`]         | The action applied if `deadline` passes before the target hydrates: `commit` (cut over to the not-yet-hydrated target) or `rollback` (revert to the pre-reconfiguration shape). |
-| `target`       | [`jsonb`]        | The config shape the cluster is reconfiguring to, as JSON: `size`, `replication_factor`, `availability_zones`, and `logging`. The realized (current) shape is in `mz_clusters`. |
+| `target`       | [`jsonb`]        | The config shape the cluster is reconfiguring to, as JSON: `size`, `replication_factor`, `availability_zones`, `logging`, and `arrangement_compression`. The realized (current) shape is in `mz_clusters`. |
 | `changes`      | [`jsonb`]        | The dimensions in which `target` differs from the cluster's realized configuration, as a JSON object holding the target value per changed dimension. Empty (`{}`) once a record settles with its target applied. A rolled-back record keeps the abandoned diff. |
 
 ## `mz_cluster_auto_scaling_strategies`

doc/user/content/self-managed-deployments/troubleshooting.md

@@ -113,13 +113,19 @@ To increase the cluster's size, you can follow the following steps:
    show clusters;
    ```
 
+   Resizing a cluster is a graceful reconfiguration: Materialize brings up a
+   replica at the new size, waits for it to hydrate, and only then retires the
+   old one. Until that finishes, `SHOW CLUSTERS` reports the old size, and
+   briefly both. Re-run the statement until it settles. The replacement replica
+   also gets a fresh name, so the resized cluster reports `r2` rather than `r1`.
+
    The output should include the `mz_catalog_server` cluster with a size of `50cc`:
 
    ```none
           name        | replicas  | comment
     -------------------+-----------+---------
     mz_analytics      |           |
-    mz_catalog_server | r1 (50cc) |
+    mz_catalog_server | r2 (50cc) |
     mz_probe          |           |
     mz_support        |           |
     mz_system         |           |

doc/user/content/sql/alter-cluster.md

@@ -177,10 +177,6 @@ by default), Materialize rolls back the resize and the cluster keeps its current
 size. To customize the timeout behavior, use the `WAIT UNTIL READY` or `WAIT FOR` options.
 The resize still proceeds in the background.
 
-{{< private-preview >}}
-Customizing the resize timeout with `WAIT UNTIL READY` or `WAIT FOR`
-{{< /private-preview >}}
-
 - `WAIT UNTIL READY (TIMEOUT = ..., ON TIMEOUT = ...)` sets the timeout for the
   resize. On timeout, `ON TIMEOUT` selects whether to `COMMIT` (retire the old
   replicas and proceed with the not-yet-hydrated new ones, which can cause
@@ -225,7 +221,6 @@ current size. Materialize drops the pending replicas and keeps the current
 configuration.
 
 #### Downtime considerations for v26.34 or before
-{{< private-preview />}}
 
 You can use the `WAIT UNTIL READY` option to perform a zero-downtime resizing,
 which incurs **no downtime**. Instead of restarting the cluster, this approach

doc/user/data/metrics.yml

@@ -453,28 +453,6 @@ metrics:
   help: Total number of started transactions.
   source: src/catalog/src/durable/metrics.rs
   visibility: internal
-- name: mz_check_scheduling_policies_seconds_bucket
-  help: The time each policy in `check_scheduling_policies` takes.
-  labels:
-  - le
-  - policy
-  - thread
-  source: src/adapter/src/metrics.rs
-  visibility: internal
-- name: mz_check_scheduling_policies_seconds_count
-  help: The time each policy in `check_scheduling_policies` takes.
-  labels:
-  - policy
-  - thread
-  source: src/adapter/src/metrics.rs
-  visibility: internal
-- name: mz_check_scheduling_policies_seconds_sum
-  help: The time each policy in `check_scheduling_policies` takes.
-  labels:
-  - policy
-  - thread
-  source: src/adapter/src/metrics.rs
-  visibility: internal
 - name: mz_cluster_handle_command_duration_seconds_bucket
   help: Time spent in handling commands.
   labels:
@@ -971,25 +949,6 @@ metrics:
   help: The time it takes to advance the catalog shard upper for a txns-shard write (group commits and table register/forget).
   source: src/adapter/src/metrics.rs
   visibility: internal
-- name: mz_handle_scheduling_decisions_seconds_bucket
-  help: The time `handle_scheduling_decisions` takes.
-  labels:
-  - altered_a_cluster
-  - le
-  source: src/adapter/src/metrics.rs
-  visibility: internal
-- name: mz_handle_scheduling_decisions_seconds_count
-  help: The time `handle_scheduling_decisions` takes.
-  labels:
-  - altered_a_cluster
-  source: src/adapter/src/metrics.rs
-  visibility: internal
-- name: mz_handle_scheduling_decisions_seconds_sum
-  help: The time `handle_scheduling_decisions` takes.
-  labels:
-  - altered_a_cluster
-  source: src/adapter/src/metrics.rs
-  visibility: internal
 - name: mz_index_peek_cursor_setup_seconds_bucket
   help: Time setting up cursor and literal constraints.
   labels:

misc/python/materialize/checks/all_checks/builtin_cluster_replication_factor.py

@@ -21,6 +21,10 @@ class BuiltinClusterReplicationFactor(Check):
     the break-glass path a support engineer scales up by hand. A catalog open that
     does not read the cluster's replication factor tears that replica down, leaving
     the cluster reporting a factor it is not honoring.
+
+    The replica arrives asynchronously: the cluster controller materializes it a
+    tick after the ALTER commits. Every assertion below is a retrying testdrive
+    query for that reason.
     """
 
     def _can_run(self, e: Executor) -> bool:

misc/python/materialize/checks/all_checks/cluster.py

@@ -152,9 +152,6 @@ def _can_run(self, e: Executor) -> bool:
 
     def initialize(self) -> Testdrive:
         return Testdrive(dedent("""
-            $ postgres-execute connection=postgres://mz_system@${testdrive.materialize-internal-sql-addr}
-            ALTER SYSTEM SET enable_zero_downtime_cluster_reconfiguration = true
-
             $ postgres-execute connection=postgres://postgres:postgres@postgres
             CREATE USER graceful_reconfig WITH SUPERUSER PASSWORD 'postgres';
             ALTER USER graceful_reconfig WITH replication;

misc/python/materialize/mzcompose/__init__.py

@@ -97,14 +97,9 @@ def get_minimal_system_parameters(
         "enable_refresh_every_mvs": "true",
         "enable_replacement_materialized_views": "true",
         "enable_cluster_schedule_refresh": "true",
-        # The cluster controller and background ALTER CLUSTER dyncfgs default on
-        # in current versions. Pin them explicitly so runs against older versions
-        # (which predate the flags or defaulted them off) exercise the legacy
-        # paths while current versions exercise the controller owning the
-        # managed-cluster replica set.
-        "enable_cluster_controller": (
-            "true" if version >= MzVersion.parse_mz("v26.29.0-dev") else "false"
-        ),
+        # Pinned explicitly so runs against older versions (which predate the
+        # flag or defaulted it off) behave like current ones, where it defaults
+        # on.
         "enable_background_alter_cluster": (
             "true" if version >= MzVersion.parse_mz("v26.29.0-dev") else "false"
         ),
@@ -127,6 +122,23 @@ def get_minimal_system_parameters(
     if version < MzVersion.parse_mz("v0.163.0-dev"):
         config["enable_compute_active_dataflow_cancelation"] = "true"
 
+    # The cluster controller's break-glass gate. Removed in v26.38, where the
+    # controller runs unconditionally. Older binaries still read it, and
+    # defaulted it off before v26.29, so pin it on for them to keep mixed-version
+    # runs exercising the same path as current versions.
+    if version < MzVersion.parse_mz("v26.38.0-dev"):
+        config["enable_cluster_controller"] = (
+            "true" if version >= MzVersion.parse_mz("v26.29.0-dev") else "false"
+        )
+
+    # The `WITH (WAIT ...)` graceful-reconfiguration surface. Always accepted
+    # from v26.38 on. Older binaries still gate it behind this feature flag, so
+    # pin it on for them: the tests that use the surface no longer enable it
+    # themselves, and in a mixed-version run some of their phases execute
+    # against the old binary.
+    if version < MzVersion.parse_mz("v26.38.0-dev"):
+        config["enable_zero_downtime_cluster_reconfiguration"] = "true"
+
     return config
 
 

misc/python/materialize/parallel_workload/action.py

@@ -3270,7 +3270,6 @@ def __init__(
             "oidc_group_role_sync_strict",
             "console_oidc_client_id",
             "console_oidc_scopes",
-            "enable_cluster_controller",
             "cluster_controller_tick_interval",
             "enable_background_alter_cluster",
             "default_cluster_reconfiguration_timeout",

misc/python/materialize/parallel_workload/parallel_workload.py

@@ -168,10 +168,6 @@ def run(
         system_exe.execute("ALTER SYSTEM SET max_sql_server_connections = 1000000")
         system_exe.execute("ALTER SYSTEM SET max_kafka_connections = 1000000")
         system_exe.execute("ALTER SYSTEM SET idle_in_transaction_session_timeout = 0")
-        # Gates the WITH (WAIT ...) clause used by ReconfigureClusterAction.
-        system_exe.execute(
-            "ALTER SYSTEM SET enable_zero_downtime_cluster_reconfiguration = true"
-        )
         # Most queries should not fail because of privileges
         for object_type in [
             "TABLES",

src/adapter-types/src/dyncfgs.rs

@@ -353,23 +353,7 @@ pub const ENABLE_SCOPED_SYSTEM_PARAMETERS: Config<bool> = Config::new(
     "Whether per-cluster and per-replica scoped system parameters are evaluated and applied.",
 );
 
-/// Top-level gate for the cluster controller. When on, the controller owns the
-/// managed-cluster replica set and the legacy paths (the graceful 3-stage
-/// machine and `cluster_scheduling.rs`) are bypassed. The replica set cannot
-/// have two writers, so this is a clean switch, not a per-strategy toggle.
-///
-/// Defaults on. This is the break-glass switch to fall back to the legacy
-/// paths if the controller misbehaves.
-pub const ENABLE_CLUSTER_CONTROLLER: Config<bool> = Config::new(
-    "enable_cluster_controller",
-    true,
-    "Whether the cluster controller owns the managed-cluster replica set. When false, the legacy scheduling and graceful-reconfiguration paths run instead.",
-);
-
 /// Cadence of the cluster controller's reconcile tick.
-///
-/// Replaces `cluster_check_scheduling_policies_interval` once the controller is
-/// the sole owner; while the controller is dark both intervals exist.
 pub const CLUSTER_CONTROLLER_TICK_INTERVAL: Config<Duration> = Config::new(
     "cluster_controller_tick_interval",
     Duration::from_secs(5),
@@ -380,9 +364,6 @@ pub const CLUSTER_CONTROLLER_TICK_INTERVAL: Config<Duration> = Config::new(
 /// controller converging in the background, or blocks the session on a
 /// wait-shim until the reconfiguration completes or its deadline passes.
 ///
-/// Only consulted while [`ENABLE_CLUSTER_CONTROLLER`] is on, when the
-/// controller owns the reconfiguration.
-///
 /// Defaults on. This is the break-glass switch back to the blocking wait-shim
 /// if returning immediately causes trouble.
 pub const ENABLE_BACKGROUND_ALTER_CLUSTER: Config<bool> = Config::new(
@@ -404,9 +385,9 @@ pub const DEFAULT_CLUSTER_RECONFIGURATION_TIMEOUT: Config<Duration> = Config::ne
 /// runs a burst replica; graceful reconfiguration and `ON REFRESH` scheduling
 /// are unaffected.
 ///
-/// Only consulted while [`ENABLE_CLUSTER_CONTROLLER`] is on. A cluster can only
-/// carry an `AUTO SCALING STRATEGY` while its SQL acceptance feature flag is
-/// on, so this is the second of the two gates burst sits behind.
+/// A cluster can only carry an `AUTO SCALING STRATEGY` while its SQL acceptance
+/// feature flag is on, so this is the second of the two gates burst sits
+/// behind.
 pub const ENABLE_HYDRATION_BURST: Config<bool> = Config::new(
     "enable_hydration_burst",
     true,
@@ -426,7 +407,6 @@ pub const DEFAULT_HYDRATION_BURST_LINGER: Config<Duration> = Config::new(
 pub fn all_dyncfgs(configs: ConfigSet) -> ConfigSet {
     configs
         .add(&ALLOW_USER_SESSIONS)
-        .add(&ENABLE_CLUSTER_CONTROLLER)
         .add(&CLUSTER_CONTROLLER_TICK_INTERVAL)
         .add(&ENABLE_BACKGROUND_ALTER_CLUSTER)
         .add(&DEFAULT_CLUSTER_RECONFIGURATION_TIMEOUT)

src/adapter/src/catalog/open.rs

@@ -1155,9 +1155,14 @@ fn add_new_remove_old_builtin_roles_migration(
 /// recorded durably and runs before the cluster controller is spawned, and the
 /// controller does not run at all while a deployment is read-only.
 ///
-/// The controller derives its target from the same cluster config, so it converges
-/// on the same replica set rather than competing for it. It excludes system
-/// clusters today, but nothing here depends on that staying true.
+/// The cluster controller owns these replica sets at runtime and derives its
+/// target from the same cluster config, so the two converge on the same set
+/// rather than competing for it. The controller matches replicas by shape and
+/// count, never by name, so the `r1..rN` this creates satisfy it. This converges
+/// by name, so a boot after the controller reshaped a cluster renames or
+/// re-creates replicas it had materialized under generator names. That is
+/// harmless: every replica is a cold process at boot anyway, so the cost is
+/// replica-id and audit-log noise.
 fn reconcile_builtin_cluster_replicas(
     txn: &mut Transaction<'_>,
     builtin_cluster_config_map: &BuiltinBootstrapClusterConfigMap,
@@ -1231,9 +1236,9 @@ fn reconcile_builtin_cluster_replicas(
         }
 
         // Reading the cluster's factor is what makes this compose with the other
-        // writers of a replica set. The refresh scheduler parks a scheduled cluster
-        // by writing its factor to 0, so converging on the factor honors that
-        // instead of resurrecting a replica the scheduler just dropped.
+        // writers of a replica set. The controller's on-refresh strategy parks a
+        // scheduled cluster by writing its factor to 0, so converging on the
+        // factor honors that instead of resurrecting a replica it just dropped.
         let mut surplus = replicas_by_cluster.remove(&cluster.id).unwrap_or_default();
         for index in 0..managed.replication_factor {
             let replica_name = managed_cluster_replica_name(index);
@@ -1425,8 +1430,14 @@ fn remove_invalid_config_param_role_defaults_migration(
     Ok(())
 }
 
-/// Cluster Replicas may be created ephemerally during an alter statement, these replicas
-/// are marked as pending and should be cleaned up on catalog open.
+/// Drops replicas left durably marked `pending`.
+///
+/// No runtime path creates one anymore. An upgrade can still come from a version
+/// whose staged reconfiguration machine crashed between the pending-create commit
+/// and the finalize, and those replicas are excluded from the cluster
+/// controller's ownership test, so this catalog-open sweep is their only
+/// remaining cleaner. It goes away together with the durable `pending` field,
+/// once no supported upgrade source can still write one.
 fn remove_pending_cluster_replicas_migration(
     tx: &mut Transaction,
     boot_ts: mz_repr::Timestamp,

src/adapter/src/catalog/open/builtin_schema_migration.rs

@@ -355,6 +355,16 @@ static MIGRATIONS: LazyLock<Vec<MigrationStep>> = LazyLock::new(|| {
             MZ_CATALOG_SCHEMA,
             "mz_aws_privatelink_connections",
         ),
+        // The mz_cluster_reconfigurations MV definition changed (the `changes`
+        // diff now includes the `arrangement_compression` dimension). See the
+        // NOTE above: this version must stay at the workspace's current dev
+        // version until the change ships.
+        MigrationStep::replacement(
+            "26.38.0-dev.0",
+            CatalogItemType::MaterializedView,
+            MZ_INTERNAL_SCHEMA,
+            "mz_cluster_reconfigurations",
+        ),
     ]
 });