SystemVm placement fails when all clusters are dedicated to sub-domains

[mbertolina]

Is there a way to instantiate SystemVMs having all clusters dedicated to subdomains (different that ROOT)?

When all clusters are dedicated to a subdomain different than ROOT I see that Cloudstack is not able to find any suitable cluster/host for systemvm placement, reporting the following error message: "No clusters found having a host with enough capacity, returning."

I tried to use system offering for console proxy VM and did not work.

2026-04-29 12:42:10,362 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Enabled non-edge zones available for scan: 1
2026-04-29 12:42:10,370 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Zone Zone {"id": "1", "name": "wtc1a2f_zone", "uuid": "0d32c3fb-82c5-4736-aef6-3202718278ad"} is ready to launch console proxy
2026-04-29 12:42:10,372 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Expand console proxy standby capacity for zone wtc1a2f_zone
2026-04-29 12:42:10,373 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Allocating console proxy standby capacity for zone [Zone {"id": "1", "name": "wtc1a2f_zone", "uuid": "0d32c3fb-82c5-4736-aef6-3202718278ad"}].
2026-04-29 12:42:10,375 INFO  [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) No stopped console proxy is available, need to allocate a new console proxy
2026-04-29 12:42:10,376 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Assign console proxy from a newly started instance for request from data center : 1
2026-04-29 12:42:10,415 DEBUG [o.a.c.e.o.NetworkOrchestrator] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Found existing network configuration for offering NetworkOffering {"id":1,"name":"System-Public-Network","trafficType":"Public","uuid":"69655e3e-e4b1-40ad-b148-7daa4f747aa8"}: Network {"id": 200, "name": "null", "uuid": "bcca2494-e138-454a-9529-ab68d08ca219", "networkofferingid": 1}
2026-04-29 12:42:10,415 DEBUG [o.a.c.e.o.NetworkOrchestrator] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Releasing lock for Account [{"accountName":"system","id":1,"uuid":"3496060f-344d-11f1-82a0-52540033903c"}]
2026-04-29 12:42:10,433 DEBUG [o.a.c.e.o.NetworkOrchestrator] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Found existing network configuration for offering NetworkOffering {"id":3,"name":"System-Control-Network","trafficType":"Control","uuid":"012722f7-1104-4f2a-a93c-e6c32250ad43"}: Network {"id": 202, "name": "null", "uuid": "4adb766c-ebfb-416e-a072-9c844695b3eb", "networkofferingid": 3}
2026-04-29 12:42:10,434 DEBUG [o.a.c.e.o.NetworkOrchestrator] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Releasing lock for Account [{"accountName":"system","id":1,"uuid":"3496060f-344d-11f1-82a0-52540033903c"}]
2026-04-29 12:42:10,450 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Enabled non-edge zones available for scan: 1
2026-04-29 12:42:10,451 DEBUG [o.a.c.e.o.NetworkOrchestrator] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Found existing network configuration for offering NetworkOffering {"id":2,"name":"System-Management-Network","trafficType":"Management","uuid":"511cdbac-9cde-4ff0-a400-34170d22c595"}: Network {"id": 201, "name": "null", "uuid": "ad21dd45-ffe7-484b-b74e-d82521c374e2", "networkofferingid": 2}
2026-04-29 12:42:10,452 DEBUG [o.a.c.e.o.NetworkOrchestrator] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Releasing lock for Account [{"accountName":"system","id":1,"uuid":"3496060f-344d-11f1-82a0-52540033903c"}]
2026-04-29 12:42:10,455 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Zone [1] is ready to launch secondary storage VM.
2026-04-29 12:42:10,460 INFO  [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) No running secondary storage vms found in datacenter id=1, starting one
2026-04-29 12:42:10,461 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Allocate secondary storage VM standby capacity for zone [1].
2026-04-29 12:42:10,463 INFO  [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) No stopped secondary storage VM is available, need to allocate a new secondary storage VM.
2026-04-29 12:42:10,464 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Service offering for console proxy VM is not set or not a system service offering. Creating a default service offering.
2026-04-29 12:42:10,465 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Assign secondary storage VM from a newly started instance for request from data center [1].
2026-04-29 12:42:10,488 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Found existing network configuration for offering NetworkOffering {"id":1,"name":"System-Public-Network","trafficType":"Public","uuid":"69655e3e-e4b1-40ad-b148-7daa4f747aa8"}: Network {"id": 200, "name": "null", "uuid": "bcca2494-e138-454a-9529-ab68d08ca219", "networkofferingid": 1}
2026-04-29 12:42:10,488 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Releasing lock for Account [{"accountName":"system","id":1,"uuid":"3496060f-344d-11f1-82a0-52540033903c"}]
2026-04-29 12:42:10,492 INFO  [c.c.v.ClusteredVirtualMachineManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) allocating virtual machine from template: Template {"format":"QCOW2","id":3,"name":"SystemVM Template (KVM)","uniqueName":"routing-3","uuid":"dcacad71-d6fd-469a-a047-4399ecf09408"} with hostname: v-5-VM and 3 networks
2026-04-29 12:42:10,496 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Allocating entries for VM: VM instance {"id":5,"instanceName":"v-5-VM","state":"Stopped","type":"ConsoleProxy","uuid":"c3a2090b-216b-4521-8abd-92fc2fa04ae6"}
2026-04-29 12:42:10,499 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Allocating nics for VM instance {"id":5,"instanceName":"v-5-VM","state":"Stopped","type":"ConsoleProxy","uuid":"c3a2090b-216b-4521-8abd-92fc2fa04ae6"}
2026-04-29 12:42:10,503 DEBUG [o.a.c.e.o.NetworkOrchestrator] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Allocating nic for vm VM instance {"id":5,"instanceName":"v-5-VM","state":"Stopped","type":"ConsoleProxy","uuid":"c3a2090b-216b-4521-8abd-92fc2fa04ae6"} in network Network {"id": 200, "name": "null", "uuid": "bcca2494-e138-454a-9529-ab68d08ca219", "networkofferingid": 1} with requested profile NicProfile {"broadcastUri":null,"deviceId":2,"iPv4Address":null,"id":0,"reservationId":null,"uuid":null,"vmId":0}
2026-04-29 12:42:10,505 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Found existing network configuration for offering NetworkOffering {"id":3,"name":"System-Control-Network","trafficType":"Control","uuid":"012722f7-1104-4f2a-a93c-e6c32250ad43"}: Network {"id": 202, "name": "null", "uuid": "4adb766c-ebfb-416e-a072-9c844695b3eb", "networkofferingid": 3}
2026-04-29 12:42:10,505 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Releasing lock for Account [{"accountName":"system","id":1,"uuid":"3496060f-344d-11f1-82a0-52540033903c"}]
2026-04-29 12:42:10,511 DEBUG [c.c.n.IpAddressManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Public IP address [IPAddress {"address":"155.165.132.236","dataCenterId":1,"id":1,"uuid":"1ef86e83-2a73-4580-aeba-bba1d8d95415"}] is not in quarantine; therefore, it is allowed to be allocated.
2026-04-29 12:42:10,512 DEBUG [c.c.n.IpAddressManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) trying ip address 155.165.132.236
2026-04-29 12:42:10,523 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Found existing network configuration for offering NetworkOffering {"id":2,"name":"System-Management-Network","trafficType":"Management","uuid":"511cdbac-9cde-4ff0-a400-34170d22c595"}: Network {"id": 201, "name": "null", "uuid": "ad21dd45-ffe7-484b-b74e-d82521c374e2", "networkofferingid": 2}
2026-04-29 12:42:10,523 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Releasing lock for Account [{"accountName":"system","id":1,"uuid":"3496060f-344d-11f1-82a0-52540033903c"}]
2026-04-29 12:42:10,527 DEBUG [c.c.n.IpAddressManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) locked row for ip address 155.165.132.236 (id: 1ef86e83-2a73-4580-aeba-bba1d8d95415)
2026-04-29 12:42:10,529 INFO  [c.c.n.IpAddressManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) successfully allocated ip address 155.165.132.236
2026-04-29 12:42:10,549 DEBUG [c.c.n.IpAddressManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) IP address [IPAddress {"address":"155.165.132.236","dataCenterId":1,"id":1,"uuid":"1ef86e83-2a73-4580-aeba-bba1d8d95415"}] is direct; therefore, the resource count should not be updated.
2026-04-29 12:42:10,553 INFO  [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) allocating virtual machine from template: Template {"format":"QCOW2","id":3,"name":"SystemVM Template (KVM)","uniqueName":"routing-3","uuid":"dcacad71-d6fd-469a-a047-4399ecf09408"} with hostname: s-6-VM and 3 networks
2026-04-29 12:42:10,556 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Allocating entries for VM: VM instance {"id":6,"instanceName":"s-6-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"db2c3a0b-5f36-4f74-8d37-91f0f1a1ac48"}
2026-04-29 12:42:10,557 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Allocating nics for VM instance {"id":6,"instanceName":"s-6-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"db2c3a0b-5f36-4f74-8d37-91f0f1a1ac48"}
2026-04-29 12:42:10,558 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Allocating nic for vm VM instance {"id":6,"instanceName":"s-6-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"db2c3a0b-5f36-4f74-8d37-91f0f1a1ac48"} in network Network {"id": 200, "name": "null", "uuid": "bcca2494-e138-454a-9529-ab68d08ca219", "networkofferingid": 1} with requested profile NicProfile {"broadcastUri":null,"deviceId":2,"iPv4Address":null,"id":0,"reservationId":null,"uuid":null,"vmId":0}
2026-04-29 12:42:10,559 INFO  [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-dbf92489]) (logid:bbe36338) Found 1 inactive management server node based on timestamp
2026-04-29 12:42:10,559 INFO  [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-dbf92489]) (logid:bbe36338) management server node msid: 90520734109756, name: wtc1a2facs01v, service ip: 108.145.116.98, version: 4.22.0.0
2026-04-29 12:42:10,559 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-dbf92489]) (logid:bbe36338) Peer scan is finished. profiler: Done. Duration: 2ms , profilerQueryActiveList: Done. Duration: 0ms, , profilerSyncClusterInfo: Done. Duration: 0ms, profilerInvalidatedNodeList: Done. Duration: 0ms, profilerRemovedList: Done. Duration: 0ms,, profilerNewList: Done. Duration: 0ms, profilerInactiveList: Done. Duration: 0ms
2026-04-29 12:42:10,562 DEBUG [c.c.n.IpAddressManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Public IP address [IPAddress {"address":"155.165.132.237","dataCenterId":1,"id":2,"uuid":"3fe12d76-820c-4f51-8c02-bdee16c1c99c"}] is not in quarantine; therefore, it is allowed to be allocated.
2026-04-29 12:42:10,562 DEBUG [c.c.n.IpAddressManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) trying ip address 155.165.132.237
2026-04-29 12:42:10,570 DEBUG [c.c.n.IpAddressManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) locked row for ip address 155.165.132.237 (id: 3fe12d76-820c-4f51-8c02-bdee16c1c99c)
2026-04-29 12:42:10,571 INFO  [c.c.n.IpAddressManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) successfully allocated ip address 155.165.132.237
2026-04-29 12:42:10,582 DEBUG [c.c.n.IpAddressManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) IP address [IPAddress {"address":"155.165.132.237","dataCenterId":1,"id":2,"uuid":"3fe12d76-820c-4f51-8c02-bdee16c1c99c"}] is direct; therefore, the resource count should not be updated.
2026-04-29 12:42:10,608 DEBUG [o.a.c.e.o.NetworkOrchestrator] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Allocating nic for vm VM instance {"id":5,"instanceName":"v-5-VM","state":"Stopped","type":"ConsoleProxy","uuid":"c3a2090b-216b-4521-8abd-92fc2fa04ae6"} in network Network {"id": 202, "name": "null", "uuid": "4adb766c-ebfb-416e-a072-9c844695b3eb", "networkofferingid": 3} with requested profile null
2026-04-29 12:42:10,620 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Allocating nic for vm VM instance {"id":6,"instanceName":"s-6-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"db2c3a0b-5f36-4f74-8d37-91f0f1a1ac48"} in network Network {"id": 202, "name": "null", "uuid": "4adb766c-ebfb-416e-a072-9c844695b3eb", "networkofferingid": 3} with requested profile null
2026-04-29 12:42:10,621 DEBUG [o.a.c.e.o.NetworkOrchestrator] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Allocating nic for vm VM instance {"id":5,"instanceName":"v-5-VM","state":"Stopped","type":"ConsoleProxy","uuid":"c3a2090b-216b-4521-8abd-92fc2fa04ae6"} in network Network {"id": 201, "name": "null", "uuid": "ad21dd45-ffe7-484b-b74e-d82521c374e2", "networkofferingid": 2} with requested profile null
2026-04-29 12:42:10,630 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Allocating nic for vm VM instance {"id":6,"instanceName":"s-6-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"db2c3a0b-5f36-4f74-8d37-91f0f1a1ac48"} in network Network {"id": 201, "name": "null", "uuid": "ad21dd45-ffe7-484b-b74e-d82521c374e2", "networkofferingid": 2} with requested profile null
2026-04-29 12:42:10,641 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Allocating disks for VM instance {"id":5,"instanceName":"v-5-VM","state":"Stopped","type":"ConsoleProxy","uuid":"c3a2090b-216b-4521-8abd-92fc2fa04ae6"}
2026-04-29 12:42:10,643 INFO  [o.a.c.e.o.VolumeOrchestrator] (consoleproxy-1:[ctx-813245b3, ctx-19908da0]) (logid:13b41cc6) Adding disk object [ROOT-5] to VM [VM instance {"id":5,"instanceName":"v-5-VM","state":"Stopped","type":"ConsoleProxy","uuid":"c3a2090b-216b-4521-8abd-92fc2fa04ae6"}]
2026-04-29 12:42:10,645 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Allocating disks for VM instance {"id":6,"instanceName":"s-6-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"db2c3a0b-5f36-4f74-8d37-91f0f1a1ac48"}
2026-04-29 12:42:10,646 INFO  [o.a.c.e.o.VolumeOrchestrator] (secstorage-1:[ctx-dc656192, ctx-679ab9e7]) (logid:bb00d5ed) Adding disk object [ROOT-6] to VM [VM instance {"id":6,"instanceName":"s-6-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"db2c3a0b-5f36-4f74-8d37-91f0f1a1ac48"}]
2026-04-29 12:42:10,685 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Allocation completed for VM: VM instance {"id":6,"instanceName":"s-6-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"db2c3a0b-5f36-4f74-8d37-91f0f1a1ac48"}
2026-04-29 12:42:10,689 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Allocation completed for VM: VM instance {"id":5,"instanceName":"v-5-VM","state":"Stopped","type":"ConsoleProxy","uuid":"c3a2090b-216b-4521-8abd-92fc2fa04ae6"}
2026-04-29 12:42:10,690 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Trying to deploy VM [error decoding VM instance {"id":6,"instanceName":"s-6-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"db2c3a0b-5f36-4f74-8d37-91f0f1a1ac48"}] and details: Plan [{"_dcId":1,"_recreateDisks":false,"preferredHostIds":[],"migrationPlan":false,"hostPriorities":{}}]; avoid list [{}] and planner: [null].
2026-04-29 12:42:10,692 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Trying to allocate a host and storage pools from datacenter [Zone {"id": "1", "name": "wtc1a2f_zone", "uuid": "0d32c3fb-82c5-4736-aef6-3202718278ad"}], pod [null], cluster [null], to deploy VM [VM instance {"id":6,"instanceName":"s-6-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"db2c3a0b-5f36-4f74-8d37-91f0f1a1ac48"}] with requested CPU [500] and requested RAM [(512.00 MB) 536870912].
2026-04-29 12:42:10,694 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Trying to deploy VM [error decoding Console VM instance {"id":5,"instanceName":"v-5-VM","state":"Stopped","type":"ConsoleProxy","uuid":"c3a2090b-216b-4521-8abd-92fc2fa04ae6"}] and details: Plan [{"_dcId":1,"_recreateDisks":false,"preferredHostIds":[],"migrationPlan":false,"hostPriorities":{}}]; avoid list [{}] and planner: [null].
2026-04-29 12:42:10,693 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) ROOT volume [Volume {"id":5,"instanceId":6,"name":"ROOT-6","uuid":"7216c413-d93c-4123-9356-a764dadb7a19","volumeType":"ROOT"}] is not ready to deploy VM [VM instance {"id":6,"instanceName":"s-6-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"db2c3a0b-5f36-4f74-8d37-91f0f1a1ac48"}].
2026-04-29 12:42:10,696 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Adding pods [] to the avoid set because these pods are in the Disabled state.
2026-04-29 12:42:10,697 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Trying to allocate a host and storage pools from datacenter [Zone {"id": "1", "name": "wtc1a2f_zone", "uuid": "0d32c3fb-82c5-4736-aef6-3202718278ad"}], pod [null], cluster [null], to deploy VM [Console VM instance {"id":5,"instanceName":"v-5-VM","state":"Stopped","type":"ConsoleProxy","uuid":"c3a2090b-216b-4521-8abd-92fc2fa04ae6"}] with requested CPU [500] and requested RAM [(1.00 GB) 1073741824].
2026-04-29 12:42:10,697 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Adding clusters [] of pod [1] to the void set because these clusters are in the Disabled state.
2026-04-29 12:42:10,698 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Adding hosts [] of datacenter [0d32c3fb-82c5-4736-aef6-3202718278ad] to the avoid set, because these hosts are in the Disabled state.
2026-04-29 12:42:10,698 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) ROOT volume [Volume {"id":6,"instanceId":5,"name":"ROOT-5","uuid":"8d9c12d0-01a7-4e94-a278-0d397e5891ee","volumeType":"ROOT"}] is not ready to deploy VM [Console VM instance {"id":5,"instanceName":"v-5-VM","state":"Stopped","type":"ConsoleProxy","uuid":"c3a2090b-216b-4521-8abd-92fc2fa04ae6"}].
2026-04-29 12:42:10,700 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) DeploymentPlan [DataCenterDeployment] has not specified host. Trying to find another destination to deploy VM [VM instance {"id":6,"instanceName":"s-6-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"db2c3a0b-5f36-4f74-8d37-91f0f1a1ac48"}], avoiding pods [], clusters [] and hosts [null].
2026-04-29 12:42:10,700 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Deploy avoids pods: [], clusters: [], hosts: null.
2026-04-29 12:42:10,700 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Deploy hosts with priorities {}, hosts have NORMAL priority by default
2026-04-29 12:42:10,702 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Adding pods [] to the avoid set because these pods are in the Disabled state.
2026-04-29 12:42:10,703 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Searching all possible resources under this Zone: Zone {"id": "1", "name": "wtc1a2f_zone", "uuid": "0d32c3fb-82c5-4736-aef6-3202718278ad"}
2026-04-29 12:42:10,704 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Listing clusters in order of aggregate capacity, that have (at least one host with) enough CPU and RAM capacity under this Zone: 1
2026-04-29 12:42:10,704 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Adding clusters [] of pod [1] to the void set because these clusters are in the Disabled state.
2026-04-29 12:42:10,706 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Adding hosts [] of datacenter [0d32c3fb-82c5-4736-aef6-3202718278ad] to the avoid set, because these hosts are in the Disabled state.
2026-04-29 12:42:10,707 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) CapacityType: CPU is used for Cluster ordering
2026-04-29 12:42:10,708 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) DeploymentPlan [DataCenterDeployment] has not specified host. Trying to find another destination to deploy VM [Console VM instance {"id":5,"instanceName":"v-5-VM","state":"Stopped","type":"ConsoleProxy","uuid":"c3a2090b-216b-4521-8abd-92fc2fa04ae6"}], avoiding pods [], clusters [] and hosts [null].
2026-04-29 12:42:10,709 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Deploy avoids pods: [], clusters: [], hosts: null.
2026-04-29 12:42:10,709 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Deploy hosts with priorities {}, hosts have NORMAL priority by default
2026-04-29 12:42:10,709 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) No clusters found having a host with enough capacity, returning.
2026-04-29 12:42:10,711 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) received secondary storage vm alert
2026-04-29 12:42:10,711 DEBUG [c.c.d.FirstFitPlanner] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Searching all possible resources under this Zone: Zone {"id": "1", "name": "wtc1a2f_zone", "uuid": "0d32c3fb-82c5-4736-aef6-3202718278ad"}
2026-04-29 12:42:10,712 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Secondary Storage Vm creation failure in zone [wtc1a2f_zone].
2026-04-29 12:42:10,713 DEBUG [c.c.d.FirstFitPlanner] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Listing clusters in order of aggregate capacity, that have (at least one host with) enough CPU and RAM capacity under this Zone: 1
2026-04-29 12:42:10,715 WARN  [c.c.a.AlertManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) alertType=[19] dataCenterId=[1] podId=[null] clusterId=[null] message=[Secondary Storage Vm creation failure in zone [wtc1a2f_zone]. Error details: Unable to allocate capacity on zone [1] due to [null].].
2026-04-29 12:42:10,717 DEBUG [c.c.d.FirstFitPlanner] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) CapacityType: CPU is used for Cluster ordering
2026-04-29 12:42:10,721 DEBUG [c.c.d.FirstFitPlanner] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) No clusters found having a host with enough capacity, returning.
2026-04-29 12:42:10,722 WARN  [c.c.a.AlertManagerImpl] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) No recipients set in global setting 'alert.email.addresses', skipping sending alert with subject [Secondary Storage Vm creation failure in zone [wtc1a2f_zone]. Error details: Unable to allocate capacity on zone [1] due to [null].] and content [Secondary Storage Vm creation failure (zone wtc1a2f_zone)].
2026-04-29 12:42:10,722 WARN  [c.c.v.SystemVmLoadScanner$1] (secstorage-1:[ctx-dc656192]) (logid:bb00d5ed) Unexpected exception Failed to allocate secondary storage VM [%s] in zone [%s] with available templates com.cloud.utils.exception.CloudRuntimeException: Failed to allocate secondary storage VM [%s] in zone [%s] with available templates
        at org.apache.cloudstack.secondarystorage.SecondaryStorageManagerImpl.createSecStorageVmInstance(SecondaryStorageManagerImpl.java:707)
        at org.apache.cloudstack.secondarystorage.SecondaryStorageManagerImpl.startNew(SecondaryStorageManagerImpl.java:528)
        at org.apache.cloudstack.secondarystorage.SecondaryStorageManagerImpl.allocCapacity(SecondaryStorageManagerImpl.java:779)
        at org.apache.cloudstack.secondarystorage.SecondaryStorageManagerImpl.expandPool(SecondaryStorageManagerImpl.java:1461)
        at org.apache.cloudstack.secondarystorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:146)
        at org.apache.cloudstack.secondarystorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:55)
        at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:115)
        at com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:92)
        at com.cloud.vm.SystemVmLoadScanner$1.runInContext(SystemVmLoadScanner.java:83)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {"id":6,"instanceName":"s-6-VM","state":"Stopped","type":"SecondaryStorageVm","uuid":"db2c3a0b-5f36-4f74-8d37-91f0f1a1ac48"}Scope=interface com.cloud.dc.DataCenter; id=1
        at com.cloud.vm.VirtualMachineManagerImpl.checkDeploymentPlan(VirtualMachineManagerImpl.java:6576)
        at org.apache.cloudstack.secondarystorage.SecondaryStorageManagerImpl.createSecStorageVmInstance(SecondaryStorageManagerImpl.java:700)
        ... 19 more
2026-04-29 12:42:10,724 WARN  [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Unable to allocate console proxy standby capacity for zone [Zone {"id": "1", "name": "wtc1a2f_zone", "uuid": "0d32c3fb-82c5-4736-aef6-3202718278ad"}] due to [Failed to allocate proxy [%s] in zone [%s] with available templates]. com.cloud.utils.exception.CloudRuntimeException: Failed to allocate proxy [%s] in zone [%s] with available templates
        at com.cloud.consoleproxy.ConsoleProxyManagerImpl.createProxyInstance(ConsoleProxyManagerImpl.java:739)
        at com.cloud.consoleproxy.ConsoleProxyManagerImpl.startNew(ConsoleProxyManagerImpl.java:583)
        at com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:846)
        at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1529)
        at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:170)
        at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:122)
        at com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:92)
        at com.cloud.vm.SystemVmLoadScanner$1.runInContext(SystemVmLoadScanner.java:83)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for Console VM instance {"id":5,"instanceName":"v-5-VM","state":"Stopped","type":"ConsoleProxy","uuid":"c3a2090b-216b-4521-8abd-92fc2fa04ae6"}Scope=interface com.cloud.dc.DataCenter; id=1
        at com.cloud.vm.VirtualMachineManagerImpl.checkDeploymentPlan(VirtualMachineManagerImpl.java:6576)
        at com.cloud.consoleproxy.ConsoleProxyManagerImpl.createProxyInstance(ConsoleProxyManagerImpl.java:732)
        ... 18 more
2026-04-29 12:42:10,724 DEBUG [c.c.a.ConsoleProxyAlertAdapter] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) received console proxy alert
2026-04-29 12:42:10,726 DEBUG [c.c.a.ConsoleProxyAlertAdapter] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Console proxy creation failure. Zone [wtc1a2f_zone].
2026-04-29 12:42:10,728 WARN  [c.c.a.AlertManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) alertType=[10] dataCenterId=[1] podId=[null] clusterId=[null] message=[Console proxy creation failure. Zone [wtc1a2f_zone]. Error details: Failed to allocate proxy [%s] in zone [%s] with available templates].
2026-04-29 12:42:10,736 WARN  [c.c.a.AlertManagerImpl] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) No recipients set in global setting 'alert.email.addresses', skipping sending alert with subject [Console proxy creation failure. Zone [wtc1a2f_zone]. Error details: Failed to allocate proxy [%s] in zone [%s] with available templates] and content [Console proxy creation failure (zone wtc1a2f_zone)].
2026-04-29 12:42:10,737 WARN  [c.c.v.SystemVmLoadScanner$1] (consoleproxy-1:[ctx-813245b3]) (logid:13b41cc6) Unexpected exception Failed to allocate proxy [%s] in zone [%s] with available templates com.cloud.utils.exception.CloudRuntimeException: Failed to allocate proxy [%s] in zone [%s] with available templates
        at com.cloud.consoleproxy.ConsoleProxyManagerImpl.createProxyInstance(ConsoleProxyManagerImpl.java:739)
        at com.cloud.consoleproxy.ConsoleProxyManagerImpl.startNew(ConsoleProxyManagerImpl.java:583)
        at com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:846)
        at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1529)
        at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:170)
        at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:122)
        at com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:92)
        at com.cloud.vm.SystemVmLoadScanner$1.runInContext(SystemVmLoadScanner.java:83)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for Console VM instance {"id":5,"instanceName":"v-5-VM","state":"Stopped","type":"ConsoleProxy","uuid":"c3a2090b-216b-4521-8abd-92fc2fa04ae6"}Scope=interface com.cloud.dc.DataCenter; id=1
        at com.cloud.vm.VirtualMachineManagerImpl.checkDeploymentPlan(VirtualMachineManagerImpl.java:6576)
        at com.cloud.consoleproxy.ConsoleProxyManagerImpl.createProxyInstance(ConsoleProxyManagerImpl.java:732)
        ... 18 more
2026-04-29 12:42:11,857 DEBUG [c.c.s.S.VmStatsCollector] (StatsCollector-3:[ctx-7144950d]) (logid:45a2e600) VmStatsCollector is running to process VMs across 6 UP hosts