JustPaste.it - Share Text & Images the Easy Way

root@host02 ~ $ tail -f /var/log/cloudstack/management/management-server.log
2023-02-15 10:05:15,512 DEBUG [c.c.a.ApiServer] (qtp262366552-20:ctx-fc81f719 ctx-a4432026) (logid:75348244) CIDRs from which account 'Acct[88b47694-3827-11ed-85ff-3cecefa93384-admin] -- Account {"id": 2, "name": "admin", "uuid": "88b47694-3827-11ed-85ff-3cecefa93384"}' is allowed to perform API calls: 0.0.0.0/0,::/0
2023-02-15 10:05:15,514 WARN [c.c.a.d.ParamGenericValidationWorker] (qtp262366552-20:ctx-fc81f719 ctx-a4432026) (logid:75348244) Received unknown parameters for command listSystemVms. Unknown parameters : listall
2023-02-15 10:05:15,552 DEBUG [c.c.a.ApiServlet] (qtp262366552-20:ctx-fc81f719 ctx-a4432026) (logid:75348244) ===END=== 160.242.108.35 -- GET listall=true&page=1&pagesize=20&command=listSystemVms&response=json
2023-02-15 10:05:16,500 DEBUG [o.a.c.h.HAManagerImpl] (BackgroundTaskPollManager-2:ctx-5c93cdf3) (logid:0dc2f777) HA health check task is running...
2023-02-15 10:05:17,693 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-60314da0) (logid:48d8c3cf) Resetting hosts suitable for reconnect
2023-02-15 10:05:17,694 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-60314da0) (logid:48d8c3cf) Completed resetting hosts suitable for reconnect
2023-02-15 10:05:17,694 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-60314da0) (logid:48d8c3cf) Acquiring hosts for clusters already owned by this management server
2023-02-15 10:05:17,694 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-60314da0) (logid:48d8c3cf) Completed acquiring hosts for clusters already owned by this management server
2023-02-15 10:05:17,694 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-60314da0) (logid:48d8c3cf) Acquiring hosts for clusters not owned by any management server
2023-02-15 10:05:17,694 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-60314da0) (logid:48d8c3cf) Completed acquiring hosts for clusters not owned by any management server
2023-02-15 10:05:20,154 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-12:null) (logid:) SeqA 42-315: Processing Seq 42-315: { Cmd , MgmtId: -1, via: 42, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":"96","_loadInfo":"{
"connections": []
}","wait":"0","bypassHostMaintenance":"false"}}] }
2023-02-15 10:05:20,157 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-12:null) (logid:) SeqA 42-315: Sending Seq 42-315: { Ans: , MgmtId: 66988330791812, via: 42, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
2023-02-15 10:05:20,507 DEBUG [o.a.c.h.HAManagerImpl] (BackgroundTaskPollManager-4:ctx-94c82f2f) (logid:8709742a) HA health check task is running...
2023-02-15 10:05:24,514 DEBUG [o.a.c.h.HAManagerImpl] (BackgroundTaskPollManager-5:ctx-381d7534) (logid:62d27d32) HA health check task is running...
2023-02-15 10:05:28,521 DEBUG [o.a.c.h.HAManagerImpl] (BackgroundTaskPollManager-2:ctx-504c80b6) (logid:95b864c6) HA health check task is running...
2023-02-15 10:05:30,160 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-13:null) (logid:) SeqA 42-316: Processing Seq 42-316: { Cmd , MgmtId: -1, via: 42, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":"96","_loadInfo":"{
"connections": []
}","wait":"0","bypassHostMaintenance":"false"}}] }
2023-02-15 10:05:30,163 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-13:null) (logid:) SeqA 42-316: Sending Seq 42-316: { Ans: , MgmtId: 66988330791812, via: 42, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
2023-02-15 10:05:32,528 DEBUG [o.a.c.h.HAManagerImpl] (BackgroundTaskPollManager-4:ctx-41221623) (logid:476a8e7d) HA health check task is running...
2023-02-15 10:05:36,534 DEBUG [o.a.c.h.HAManagerImpl] (BackgroundTaskPollManager-5:ctx-b48d6a36) (logid:bfde8a65) HA health check task is running...
2023-02-15 10:05:37,181 DEBUG [c.c.a.ApiServlet] (qtp262366552-311:ctx-b6e25249) (logid:1b57c596) ===START=== 160.242.108.35 -- GET listall=true&page=1&pagesize=20&command=listSystemVms&response=json
2023-02-15 10:05:37,185 DEBUG [c.c.a.ApiServer] (qtp262366552-311:ctx-b6e25249 ctx-d108e762) (logid:1b57c596) CIDRs from which account 'Acct[88b47694-3827-11ed-85ff-3cecefa93384-admin] -- Account {"id": 2, "name": "admin", "uuid": "88b47694-3827-11ed-85ff-3cecefa93384"}' is allowed to perform API calls: 0.0.0.0/0,::/0
2023-02-15 10:05:37,187 WARN [c.c.a.d.ParamGenericValidationWorker] (qtp262366552-311:ctx-b6e25249 ctx-d108e762) (logid:1b57c596) Received unknown parameters for command listSystemVms. Unknown parameters : listall
2023-02-15 10:05:37,224 DEBUG [c.c.a.ApiServlet] (qtp262366552-311:ctx-b6e25249 ctx-d108e762) (logid:1b57c596) ===END=== 160.242.108.35 -- GET listall=true&page=1&pagesize=20&command=listSystemVms&response=json
2023-02-15 10:05:37,255 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-1df0d432) (logid:37dc81e4) Zone 16 is ready to launch console proxy
2023-02-15 10:05:37,281 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-c95c714e) (logid:18f40720) Zone [16] is ready to launch secondary storage VM.
2023-02-15 10:05:37,430 DEBUG [o.a.c.c.p.RootCACustomTrustManager] (pool-12-thread-1:null) (logid:) A client/agent attempting connection from address=defaultgatewaypublicIP has presented these certificate(s):
Certificate [1] :
Serial: e93da13b2680bffd
Not Before:Tue Feb 14 21:56:21 SAST 2023
Not After:Thu Feb 15 09:56:21 SAST 2024
Signature Algorithm:SHA256withRSA
Version:3
Subject DN:CN=s-95-VM
Issuer DN:CN=ca.cloudstack.apache.org
Alternative Names:[[7, 197.189.211.5], [7, 192.168.50.87], [7, 192.168.50.64], [7, 169.254.172.153], [2, s-95-VM]]
Certificate [2] :
Serial: 7f5d24fd310b037a
Not Before:Mon Sep 19 04:30:01 SAST 2022
Not After:Wed Sep 11 16:30:01 SAST 2052
Signature Algorithm:SHA256withRSA
Version:3
Subject DN:CN=ca.cloudstack.apache.org
Issuer DN:CN=ca.cloudstack.apache.org
Alternative Names:null
2023-02-15 10:05:37,431 ERROR [o.a.c.c.p.RootCACustomTrustManager] (pool-12-thread-1:null) (logid:) Certificate ownership verification failed for client: defaultgatewaypublicIP
2023-02-15 10:05:37,431 DEBUG [o.a.c.c.p.RootCACustomTrustManager] (pool-12-thread-1:null) (logid:) Client/agent connection from ip=defaultgatewaypublicIP accepted without certificate validation.
2023-02-15 10:05:37,612 DEBUG [c.c.a.t.Request] (AgentManager-Handler-14:null) (logid:) Seq 0-0: Scheduling the first command { Cmd , MgmtId: -1, via: 0, Ver: v1, Flags: 101, [{"com.cloud.agent.api.StartupSecondaryStorageCommand":{"type":"SecondaryStorage","dataCenter":"16","pod":"15","guid":"s-95-VM-PremiumSecondaryStorageResource","name":"s-95-VM","id":"0","version":"4.17.2.0","iqn":"NoIqn","publicIpAddress":"197.189.211.5","publicNetmask":"255.255.255.248","publicMacAddress":"1e:00:21:00:00:21","privateIpAddress":"192.168.50.64","privateMacAddress":"1e:00:56:00:00:0f","privateNetmask":"255.255.255.0","storageIpAddress":"192.168.50.64","storageNetmask":"255.255.255.0","storageMacAddress":"1e:00:56:00:00:0f","resourceName":"PremiumSecondaryStorageResource","wait":"0","bypassHostMaintenance":"false"}}] }
2023-02-15 10:05:37,613 DEBUG [c.c.a.t.Request] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Seq 0-0: Processing the first command { Cmd , MgmtId: -1, via: 0, Ver: v1, Flags: 101, [{"com.cloud.agent.api.StartupSecondaryStorageCommand":{"type":"SecondaryStorage","dataCenter":"16","pod":"15","guid":"s-95-VM-PremiumSecondaryStorageResource","name":"s-95-VM","id":"0","version":"4.17.2.0","iqn":"NoIqn","publicIpAddress":"197.189.211.5","publicNetmask":"255.255.255.248","publicMacAddress":"1e:00:21:00:00:21","privateIpAddress":"192.168.50.64","privateMacAddress":"1e:00:56:00:00:0f","privateNetmask":"255.255.255.0","storageIpAddress":"192.168.50.64","storageNetmask":"255.255.255.0","storageMacAddress":"1e:00:56:00:00:0f","resourceName":"PremiumSecondaryStorageResource","wait":"0","bypassHostMaintenance":"false"}}] }
2023-02-15 10:05:37,616 DEBUG [c.c.r.ResourceManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Dispatching resource state event CREATE_HOST_VO_FOR_CONNECTED to CiscoVnmcElement
2023-02-15 10:05:37,616 DEBUG [c.c.r.ResourceManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Dispatching resource state event CREATE_HOST_VO_FOR_CONNECTED to BareMetalDiscoverer
2023-02-15 10:05:37,616 DEBUG [c.c.r.ResourceManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Dispatching resource state event CREATE_HOST_VO_FOR_CONNECTED to NetscalerElement
2023-02-15 10:05:37,616 DEBUG [c.c.r.ResourceManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Dispatching resource state event CREATE_HOST_VO_FOR_CONNECTED to JuniperSRXExternalFirewallElement
2023-02-15 10:05:37,616 DEBUG [c.c.r.ResourceManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Dispatching resource state event CREATE_HOST_VO_FOR_CONNECTED to HypervServerDiscoverer
2023-02-15 10:05:37,616 DEBUG [c.c.r.ResourceManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Dispatching resource state event CREATE_HOST_VO_FOR_CONNECTED to BaremetalPxeManagerImpl
2023-02-15 10:05:37,616 DEBUG [c.c.r.ResourceManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Dispatching resource state event CREATE_HOST_VO_FOR_CONNECTED to XcpServerDiscoverer
2023-02-15 10:05:37,616 DEBUG [c.c.r.ResourceManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Dispatching resource state event CREATE_HOST_VO_FOR_CONNECTED to NiciraNvp
2023-02-15 10:05:37,616 DEBUG [c.c.r.ResourceManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Dispatching resource state event CREATE_HOST_VO_FOR_CONNECTED to BrocadeVcsElement
2023-02-15 10:05:37,616 DEBUG [c.c.r.ResourceManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Dispatching resource state event CREATE_HOST_VO_FOR_CONNECTED to Ovm3Discoverer
2023-02-15 10:05:37,616 DEBUG [c.c.h.o.r.Ovm3Discoverer] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) createHostVOForConnectedAgent: Host {"id": "43", "name": "s-95-VM", "uuid": "629aeb02-3075-4bde-a696-14b87440b016", "type"="SecondaryStorageVM"}
2023-02-15 10:05:37,616 DEBUG [c.c.r.ResourceManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Dispatching resource state event CREATE_HOST_VO_FOR_CONNECTED to LxcServerDiscoverer
2023-02-15 10:05:37,616 DEBUG [c.c.r.ResourceManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Dispatching resource state event CREATE_HOST_VO_FOR_CONNECTED to NetworkUsageManagerImpl
2023-02-15 10:05:37,616 DEBUG [c.c.r.ResourceManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Dispatching resource state event CREATE_HOST_VO_FOR_CONNECTED to PremiumSecondaryStorageManagerImpl
2023-02-15 10:05:37,627 DEBUG [c.c.r.ResourceState] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Resource state update: [id = 43; name = s-95-VM; old state = Enabled; event = InternalCreated; new state = Enabled]
2023-02-15 10:05:37,627 DEBUG [c.c.a.m.AgentManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Transition:[Resource state = Enabled, Agent event = AgentConnected, Host id = 43, name = s-95-VM]
2023-02-15 10:05:37,645 DEBUG [c.c.a.m.AgentManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Agent's management server host list is not up to date, sending list update:[managementpublicIP]
2023-02-15 10:05:37,645 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) create ClusteredAgentAttache for 43
2023-02-15 10:05:37,646 DEBUG [c.c.a.m.AgentManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Sending Connect to listener: XcpServerDiscoverer
2023-02-15 10:05:37,646 DEBUG [c.c.a.m.AgentManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Sending Connect to listener: HypervServerDiscoverer
2023-02-15 10:05:37,646 DEBUG [c.c.a.m.AgentManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Sending Connect to listener: ClusteredVirtualMachineManagerImpl
2023-02-15 10:05:37,646 DEBUG [c.c.a.m.AgentManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Sending Connect to listener: VmwareManagerImpl
2023-02-15 10:05:37,646 DEBUG [c.c.a.m.AgentManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Sending Connect to listener: StoragePoolMonitor
2023-02-15 10:05:37,646 DEBUG [c.c.a.m.AgentManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Sending Connect to listener: SecondaryStorageListener
2023-02-15 10:05:37,646 INFO [c.c.s.s.SecondaryStorageListener] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Received a host startup notification com.cloud.agent.api.StartupSecondaryStorageCommand
2023-02-15 10:05:37,655 DEBUG [c.c.a.t.Request] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Seq 43-4050424914866339841: Sending { Cmd , MgmtId: 66988330791812, via: 43(s-95-VM), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.SecStorageSetupCommand":{"store":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://192.168.50.2/mnt/datasecond","_role":"Image"}},"secUrl":"nfs://192.168.50.2/mnt/datasecond","certs":{},"postUploadKey":"fneB0iq9ETvOUHKxqHQBai1GeeKaBjvUxSQXHuZlv2mTW13wO2OOWH0OgrAIRH2eMMvkFORM9jwez1LPGnhXAg","wait":"0","bypassHostMaintenance":"false"}}] }
2023-02-15 10:05:37,731 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-10:null) (logid:) Ping from 43(s-95-VM)
2023-02-15 10:05:37,731 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-10:null) (logid:) PingMap for agent: 43 will not be updated because agent is no longer in the PingMap
2023-02-15 10:05:40,155 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-2:null) (logid:) SeqA 42-317: Processing Seq 42-317: { Cmd , MgmtId: -1, via: 42, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":"96","_loadInfo":"{
"connections": []
}","wait":"0","bypassHostMaintenance":"false"}}] }
2023-02-15 10:05:40,159 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-2:null) (logid:) SeqA 42-317: Sending Seq 42-317: { Ans: , MgmtId: 66988330791812, via: 42, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
2023-02-15 10:05:40,541 DEBUG [o.a.c.h.HAManagerImpl] (BackgroundTaskPollManager-2:ctx-d269f76e) (logid:84372817) HA health check task is running...
2023-02-15 10:05:40,947 DEBUG [c.c.a.t.Request] (AgentManager-Handler-15:null) (logid:) Seq 43-4050424914866339841: Processing: { Ans: , MgmtId: 66988330791812, via: 43, Ver: v1, Flags: 110, [{"com.cloud.agent.api.SecStorageSetupAnswer":{"_dir":"a863afaf-19a6-3827-8c49-0c5e6346b6c8","result":"true","details":"success","wait":"0","bypassHostMaintenance":"false"}}] }
2023-02-15 10:05:40,947 DEBUG [c.c.a.t.Request] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Seq 43-4050424914866339841: Received: { Ans: , MgmtId: 66988330791812, via: 43(s-95-VM), Ver: v1, Flags: 110, { SecStorageSetupAnswer } }
2023-02-15 10:05:40,947 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-15:null) (logid:) Seq 43-4050424914866339841: No more commands found
2023-02-15 10:05:40,947 DEBUG [c.c.a.m.AgentManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Details from executing class com.cloud.agent.api.SecStorageSetupCommand: success
2023-02-15 10:05:40,949 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Successfully programmed secondary storage [Local Second] in secondary storage VM [s-95-VM].
2023-02-15 10:05:40,954 DEBUG [c.c.a.t.Request] (AgentConnectTaskPool-5:ctx-8d5eb6c7) (logid:db512f8a) Seq 43-4050424914866339842: Sending { Cmd , MgmtId: 66988330791812, via: 43(s-95-VM), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.SecStorageSetupCommand":{"store":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://102.165.20.210/data/secondary","_role":"Image"}},"secUrl":"nfs://102.165.20.210/data/secondary","certs":{},"postUploadKey":"fneB0iq9ETvOUHKxqHQBai1GeeKaBjvUxSQXHuZlv2mTW13wO2OOWH0OgrAIRH2eMMvkFORM9jwez1LPGnhXAg","wait":"0","bypassHostMaintenance":"false"}}] }
2023-02-15 10:05:42,572 DEBUG [c.c.s.StatsCollector] (StatsCollector-1:ctx-bc5a57f6) (logid:dfa9557a) DbCollector is running...
2023-02-15 10:05:42,625 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-faa3f25d) (logid:f5e43ffa) Found 0 routers to update status.
2023-02-15 10:05:42,626 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-faa3f25d) (logid:f5e43ffa) Found 0 VPC's to update Redundant State.
2023-02-15 10:05:42,627 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-faa3f25d) (logid:f5e43ffa) Found 0 networks to update RvR status.
2023-02-15 10:05:43,556 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-1:null) (logid:) Ping from 42(v-96-VM)
2023-02-15 10:05:43,571 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) ManagementServerCollector is running...
2023-02-15 10:05:43,572 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing: /bin/bash -c systemctl status cloudstack-usage | grep " Loaded:"
2023-02-15 10:05:43,573 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing while with timeout : 3600000
2023-02-15 10:05:43,585 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-5:null) (logid:) Ping from Routing host 44(KVMhostname)
2023-02-15 10:05:43,585 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-5:null) (logid:) Process host VM state report from ping process. host: 44
2023-02-15 10:05:43,585 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-5:null) (logid:) Process VM state report. host: 44, number of records in report: 0
2023-02-15 10:05:43,586 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-5:null) (logid:) Done with process of VM state report. host: 44
2023-02-15 10:05:43,591 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Execution is successful.
2023-02-15 10:05:43,592 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) usage install: Loaded: loaded (/usr/lib/systemd/system/cloudstack-usage.service; enabled; vendor preset: disabled)
2023-02-15 10:05:43,592 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) Sessions active: 1
2023-02-15 10:05:43,592 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing: /bin/bash -c cat /etc/os-release | grep PRETTY_NAME | cut -f2 -d '=' | tr -d '"'
2023-02-15 10:05:43,592 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing while with timeout : 3600000
2023-02-15 10:05:43,596 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Execution is successful.
2023-02-15 10:05:43,596 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing: /bin/bash -c uname -r
2023-02-15 10:05:43,597 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing while with timeout : 3600000
2023-02-15 10:05:43,599 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Execution is successful.
2023-02-15 10:05:43,599 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing: /bin/bash -c cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z '
2023-02-15 10:05:43,600 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing while with timeout : 3600000
2023-02-15 10:05:43,602 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Execution is successful.
2023-02-15 10:05:43,603 INFO [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) system memory from /proc: 16543047680
2023-02-15 10:05:43,603 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing: /bin/bash -c cat /proc/meminfo | grep MemFree | cut -f 2 -d ':' | tr -d 'a-zA-z '
2023-02-15 10:05:43,603 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-4:null) (logid:) Ping from Routing host 36(hostname)
2023-02-15 10:05:43,603 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing while with timeout : 3600000
2023-02-15 10:05:43,604 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) Process host VM state report from ping process. host: 36
2023-02-15 10:05:43,605 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Execution is successful.
2023-02-15 10:05:43,606 INFO [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) free memory from /proc: 10989355008
2023-02-15 10:05:43,606 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing: /bin/bash -c ps -o rss= 2549994
2023-02-15 10:05:43,607 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing while with timeout : 3600000
2023-02-15 10:05:43,620 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) Process VM state report. host: 36, number of records in report: 16
2023-02-15 10:05:43,620 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) VM state report. host: 36, vm id: 96, power state: PowerOn
2023-02-15 10:05:43,621 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) VM state report. host: 36, vm id: 128, power state: PowerOn
2023-02-15 10:05:43,621 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Execution is successful.
2023-02-15 10:05:43,622 INFO [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) used memory from /proc: 1215524
2023-02-15 10:05:43,622 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing: /bin/bash -c uptime -s
2023-02-15 10:05:43,622 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing while with timeout : 3600000
2023-02-15 10:05:43,624 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) VM state report. host: 36, vm id: 129, power state: PowerOn
2023-02-15 10:05:43,626 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) VM state report. host: 36, vm id: 130, power state: PowerOn
2023-02-15 10:05:43,626 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Execution is successful.
2023-02-15 10:05:43,629 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing: /bin/bash -c ps -o vsz= 2549994
2023-02-15 10:05:43,629 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) VM state report. host: 36, vm id: 131, power state: PowerOn
2023-02-15 10:05:43,629 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing while with timeout : 3600000
2023-02-15 10:05:43,631 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) VM state report. host: 36, vm id: 132, power state: PowerOn
2023-02-15 10:05:43,632 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) VM state report. host: 36, vm id: 133, power state: PowerOn
2023-02-15 10:05:43,633 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) VM state report. host: 36, vm id: 134, power state: PowerOn
2023-02-15 10:05:43,634 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) VM state report. host: 36, vm id: 135, power state: PowerOn
2023-02-15 10:05:43,635 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) VM state report. host: 36, vm id: 136, power state: PowerOn
2023-02-15 10:05:43,637 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) VM state report. host: 36, vm id: 117, power state: PowerOn
2023-02-15 10:05:43,637 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Execution is successful.
2023-02-15 10:05:43,638 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing: /bin/bash -c cat /proc/cpuinfo | grep "cpu MHz" | grep "cpu MHz" | cut -f 2 -d : | tr -d ' '| tr '\n' " "
2023-02-15 10:05:43,638 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing while with timeout : 3600000
2023-02-15 10:05:43,639 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) VM state report. host: 36, vm id: 118, power state: PowerOn
2023-02-15 10:05:43,640 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) VM state report. host: 36, vm id: 119, power state: PowerOn
2023-02-15 10:05:43,642 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) VM state report. host: 36, vm id: 120, power state: PowerOn
2023-02-15 10:05:43,643 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) VM state report. host: 36, vm id: 126, power state: PowerOn
2023-02-15 10:05:43,644 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) VM state report. host: 36, vm id: 95, power state: PowerOn
2023-02-15 10:05:43,650 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-4:null) (logid:) Done with process of VM state report. host: 36
2023-02-15 10:05:43,654 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Execution is successful.
2023-02-15 10:05:43,655 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing: /bin/bash -c cat /proc/loadavg
2023-02-15 10:05:43,655 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing while with timeout : 3600000
2023-02-15 10:05:43,657 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Execution is successful.
2023-02-15 10:05:43,657 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing: /bin/bash -c cat /proc/stat | grep "cpu " | tr -d "cpu"
2023-02-15 10:05:43,658 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing while with timeout : 3600000
2023-02-15 10:05:43,660 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Execution is successful.
2023-02-15 10:05:43,661 DEBUG [c.c.u.LogUtils] (StatsCollector-4:ctx-14263f59) (logid:be715028) file for FILE : /var/log/cloudstack/management/management-server.log
2023-02-15 10:05:43,661 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing: /bin/bash -c du -sh /var/log/cloudstack/management/management-server.log | cut -f '1'
2023-02-15 10:05:43,661 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing while with timeout : 3600000
2023-02-15 10:05:43,665 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Execution is successful.
2023-02-15 10:05:43,665 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing: /bin/bash -c df -h /var/log/cloudstack/management/management-server.log | grep -v Filesystem | awk '{print "on disk " $1 " mounted on " $6 " (" $5 " full)"}'
2023-02-15 10:05:43,666 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Executing while with timeout : 3600000
2023-02-15 10:05:43,680 DEBUG [c.c.u.s.Script] (StatsCollector-4:ctx-14263f59) (logid:be715028) Execution is successful.
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail buffersdirect.capacity, 153819
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail buffersdirect.count, 11
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail buffersdirect.used, 153819
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail buffersmapped.capacity, 0
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail buffersmapped.count, 0
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail buffersmapped.used, 0
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail gcPS-MarkSweep.count, 4
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail gcPS-MarkSweep.time, 304
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail gcPS-Scavenge.count, 27
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail gcPS-Scavenge.time, 217
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail jvmname, 2549994@managementhostname
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail jvmuptime, 500580
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail jvmvendor, Red Hat, Inc. OpenJDK 64-Bit Server VM 11.0.18+10-LTS (11)
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memoryheap.committed, 672661504
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memoryheap.init, 260046848
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memoryheap.usage, 0.31001446856734716
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorynon-heap.committed, 183025664
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorynon-heap.init, 7667712
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorynon-heap.max, -1
2023-02-15 10:05:43,682 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorynon-heap.usage, -1.75878976E8
2023-02-15 10:05:43,683 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorynon-heap.used, 175878976
2023-02-15 10:05:43,683 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorypools.CodeHeap-'non-nmethods'.usage, 0.312719298245614
2023-02-15 10:05:43,683 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorypools.CodeHeap-'non-profiled-nmethods'.usage, 0.1325646494268195
2023-02-15 10:05:43,683 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorypools.CodeHeap-'profiled-nmethods'.usage, 0.2350847302296131
2023-02-15 10:05:43,683 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorypools.Compressed-Class-Space.usage, 0.010788261890411377
2023-02-15 10:05:43,683 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorypools.Metaspace.usage, 0.9733642080217539
2023-02-15 10:05:43,683 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorypools.PS-Eden-Space.usage, 0.6741430044728227
2023-02-15 10:05:43,683 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorypools.PS-Old-Gen.usage, 0.09124143496401273
2023-02-15 10:05:43,683 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorypools.PS-Survivor-Space.usage, 0.9263702392578125
2023-02-15 10:05:43,683 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorytotal.committed, 855687168
2023-02-15 10:05:43,683 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorytotal.init, 267714560
2023-02-15 10:05:43,683 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorytotal.max, 1908932607
2023-02-15 10:05:43,683 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail memorytotal.used, 767668792
2023-02-15 10:05:43,684 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail threadsdeadlocks, []
2023-02-15 10:05:43,684 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail threadsnew.count, 0
2023-02-15 10:05:43,684 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-14263f59) (logid:be715028) not storing detail threadstimed_waiting.count, 70
2023-02-15 10:05:43,686 DEBUG [c.c.c.ClusterManagerImpl] (StatsCollector-4:ctx-14263f59) (logid:be715028) Forwarding {"managementServerHostId":2,"managementServerHostUuid":"14ce1e48-023f-479d-b86b-30ffc769d14f","collectionTime":"Feb 15, 2023, 10:05:43 AM","sessions":1,"cpuUtilization":0.0,"totalJvmMemoryBytes":672661504,"freeJvmMemoryBytes":87934752,"maxJvmMemoryBytes":1908932607,"processJvmMemoryBytes":0,"jvmUptime":500490,"jvmStartTime":1676447843122,"availableProcessors":8,"loadAverage":0.23,"totalInit":267714560,"totalUsed":760564184,"totalCommitted":855687168,"pid":2549994,"jvmName":"2549994@managementhostname","jvmVendor":"Red Hat, Inc.","jvmVersion":"11.0.18+10-LTS","osDistribution":"AlmaLinux 8.7 (Stone Smilodon)","agentCount":4,"heapMemoryUsed":591796728,"heapMemoryTotal":1908932608,"threadsBlockedCount":0,"threadsDaemonCount":13,"threadsRunnableCount":17,"threadsTerminatedCount":0,"threadsTotalCount":192,"threadsWaitingCount":105,"systemMemoryTotal":16543047680,"systemMemoryFree":10989355008,"systemMemoryUsed":1215524,"systemMemoryVirtualSize":8401260544,"logInfo":"/var/log/cloudstack/management/management-server.log using: 25M\non disk /dev/md1 mounted on / (82% full)","systemTotalCpuCycles":29012.850000000002,"systemLoadAverages":[0.23,0.42,0.33],"systemCyclesUsage":[11402096,18365908,6039988732],"dbLocal":true,"usageLocal":true,"systemBootTime":"Nov 18, 2022, 7:14:33 PM","kernelVersion":"4.18.0-372.32.1.el8_6.x86_64"} to 66988330791812
2023-02-15 10:05:43,686 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Worker-4:ctx-784129e9) (logid:f06a5bed) Cluster PDU 66988330791812 -> 66988330791812. agent: 0, pdu seq: 17, pdu ack seq: 0, json: {"managementServerHostId":2,"managementServerHostUuid":"14ce1e48-023f-479d-b86b-30ffc769d14f","collectionTime":"Feb 15, 2023, 10:05:43 AM","sessions":1,"cpuUtilization":0.0,"totalJvmMemoryBytes":672661504,"freeJvmMemoryBytes":87934752,"maxJvmMemoryBytes":1908932607,"processJvmMemoryBytes":0,"jvmUptime":500490,"jvmStartTime":1676447843122,"availableProcessors":8,"loadAverage":0.23,"totalInit":267714560,"totalUsed":760564184,"totalCommitted":855687168,"pid":2549994,"jvmName":"2549994@managementhostname","jvmVendor":"Red Hat, Inc.","jvmVersion":"11.0.18+10-LTS","osDistribution":"AlmaLinux 8.7 (Stone Smilodon)","agentCount":4,"heapMemoryUsed":591796728,"heapMemoryTotal":1908932608,"threadsBlockedCount":0,"threadsDaemonCount":13,"threadsRunnableCount":17,"threadsTerminatedCount":0,"threadsTotalCount":192,"threadsWaitingCount":105,"systemMemoryTotal":16543047680,"systemMemoryFree":10989355008,"systemMemoryUsed":1215524,"systemMemoryVirtualSize":8401260544,"logInfo":"/var/log/cloudstack/management/management-server.log using: 25M\non disk /dev/md1 mounted on / (82% full)","systemTotalCpuCycles":29012.850000000002,"systemLoadAverages":[0.23,0.42,0.33],"systemCyclesUsage":[11402096,18365908,6039988732],"dbLocal":true,"usageLocal":true,"systemBootTime":"Nov 18, 2022, 7:14:33 PM","kernelVersion":"4.18.0-372.32.1.el8_6.x86_64"}
2023-02-15 10:05:43,689 DEBUG [c.c.c.ClusterServiceServletImpl] (Cluster-Worker-4:ctx-784129e9) (logid:f06a5bed) POST http://managementpublicIP:9090/clusterservice response :true, responding time: 2 ms
2023-02-15 10:05:43,689 DEBUG [c.c.s.StatsCollector] (Cluster-Worker-10:ctx-ca6ddbef) (logid:9fa8e606) StatusUpdate from 66988330791812, json: {"managementServerHostId":2,"managementServerHostUuid":"14ce1e48-023f-479d-b86b-30ffc769d14f","collectionTime":"Feb 15, 2023, 10:05:43 AM","sessions":1,"cpuUtilization":0.0,"totalJvmMemoryBytes":672661504,"freeJvmMemoryBytes":87934752,"maxJvmMemoryBytes":1908932607,"processJvmMemoryBytes":0,"jvmUptime":500490,"jvmStartTime":1676447843122,"availableProcessors":8,"loadAverage":0.23,"totalInit":267714560,"totalUsed":760564184,"totalCommitted":855687168,"pid":2549994,"jvmName":"2549994@managementhostname","jvmVendor":"Red Hat, Inc.","jvmVersion":"11.0.18+10-LTS","osDistribution":"AlmaLinux 8.7 (Stone Smilodon)","agentCount":4,"heapMemoryUsed":591796728,"heapMemoryTotal":1908932608,"threadsBlockedCount":0,"threadsDaemonCount":13,"threadsRunnableCount":17,"threadsTerminatedCount":0,"threadsTotalCount":192,"threadsWaitingCount":105,"systemMemoryTotal":16543047680,"systemMemoryFree":10989355008,"systemMemoryUsed":1215524,"systemMemoryVirtualSize":8401260544,"logInfo":"/var/log/cloudstack/management/management-server.log using: 25M\non disk /dev/md1 mounted on / (82% full)","systemTotalCpuCycles":29012.850000000002,"systemLoadAverages":[0.23,0.42,0.33],"systemCyclesUsage":[11402096,18365908,6039988732],"dbLocal":true,"usageLocal":true,"systemBootTime":"Nov 18, 2022, 7:14:33 PM","kernelVersion":"4.18.0-372.32.1.el8_6.x86_64"}
2023-02-15 10:05:43,689 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Worker-4:ctx-784129e9) (logid:f06a5bed) Cluster PDU 66988330791812 -> 66988330791812 completed. time: 2ms. agent: 0, pdu seq: 17, pdu ack seq: 0, json: {"managementServerHostId":2,"managementServerHostUuid":"14ce1e48-023f-479d-b86b-30ffc769d14f","collectionTime":"Feb 15, 2023, 10:05:43 AM","sessions":1,"cpuUtilization":0.0,"totalJvmMemoryBytes":672661504,"freeJvmMemoryBytes":87934752,"maxJvmMemoryBytes":1908932607,"processJvmMemoryBytes":0,"jvmUptime":500490,"jvmStartTime":1676447843122,"availableProcessors":8,"loadAverage":0.23,"totalInit":267714560,"totalUsed":760564184,"totalCommitted":855687168,"pid":2549994,"jvmName":"2549994@managementhostname","jvmVendor":"Red Hat, Inc.","jvmVersion":"11.0.18+10-LTS","osDistribution":"AlmaLinux 8.7 (Stone Smilodon)","agentCount":4,"heapMemoryUsed":591796728,"heapMemoryTotal":1908932608,"threadsBlockedCount":0,"threadsDaemonCount":13,"threadsRunnableCount":17,"threadsTerminatedCount":0,"threadsTotalCount":192,"threadsWaitingCount":105,"systemMemoryTotal":16543047680,"systemMemoryFree":10989355008,"systemMemoryUsed":1215524,"systemMemoryVirtualSize":8401260544,"logInfo":"/var/log/cloudstack/management/management-server.log using: 25M\non disk /dev/md1 mounted on / (82% full)","systemTotalCpuCycles":29012.850000000002,"systemLoadAverages":[0.23,0.42,0.33],"systemCyclesUsage":[11402096,18365908,6039988732],"dbLocal":true,"usageLocal":true,"systemBootTime":"Nov 18, 2022, 7:14:33 PM","kernelVersion":"4.18.0-372.32.1.el8_6.x86_64"}
2023-02-15 10:05:44,548 DEBUG [o.a.c.h.HAManagerImpl] (BackgroundTaskPollManager-4:ctx-a563409b) (logid:fe739fc3) HA health check task is running...
2023-02-15 10:05:47,701 DEBUG [c.c.v.VirtualMachineManagerImpl] (Vm-Operations-Cleanup-1:ctx-7bd8a065) (logid:796d2720) VM Operation Thread Running
2023-02-15 10:05:48,555 DEBUG [o.a.c.h.HAManagerImpl] (BackgroundTaskPollManager-5:ctx-6b86cb99) (logid:a143684f) HA health check task is running...
2023-02-15 10:05:50,156 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-3:null) (logid:) SeqA 42-319: Processing Seq 42-319: { Cmd , MgmtId: -1, via: 42, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":"96","_loadInfo":"{
"connections": []
}","wait":"0","bypassHostMaintenance":"false"}}] }
2023-02-15 10:05:50,159 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-3:null) (logid:) SeqA 42-319: Sending Seq 42-319: { Ans: , MgmtId: 66988330791812, via: 42, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
2023-02-15 10:05:51,582 DEBUG [c.c.a.ApiServlet] (qtp262366552-20:ctx-666cdab0) (logid:a268718e) ===START=== 160.242.108.35 -- GET listall=true&page=1&pagesize=20&command=listSystemVms&response=json
2023-02-15 10:05:51,585 DEBUG [c.c.a.ApiServer] (qtp262366552-20:ctx-666cdab0 ctx-3132add4) (logid:a268718e) CIDRs from which account 'Acct[88b47694-3827-11ed-85ff-3cecefa93384-admin] -- Account {"id": 2, "name": "admin", "uuid": "88b47694-3827-11ed-85ff-3cecefa93384"}' is allowed to perform API calls: 0.0.0.0/0,::/0
2023-02-15 10:05:51,587 WARN [c.c.a.d.ParamGenericValidationWorker] (qtp262366552-20:ctx-666cdab0 ctx-3132add4) (logid:a268718e) Received unknown parameters for command listSystemVms. Unknown parameters : listall
2023-02-15 10:05:51,620 DEBUG [c.c.a.ApiServlet] (qtp262366552-20:ctx-666cdab0 ctx-3132add4) (logid:a268718e) ===END=== 160.242.108.35 -- GET listall=true&page=1&pagesize=20&command=listSystemVms&response=json
2023-02-15 10:05:52,563 DEBUG [o.a.c.h.HAManagerImpl] (BackgroundTaskPollManager-2:ctx-f813ae3b) (logid:c8c7e9eb) HA health check task is running...
2023-02-15 10:05:56,570 DEBUG [o.a.c.h.HAManagerImpl] (BackgroundTaskPollManager-4:ctx-6911b2a6) (logid:2b537544) HA health check task is running...
2023-02-15 10:05:57,584 DEBUG [c.c.s.StatsCollector] (StatsCollector-3:ctx-45488043) (logid:6e6b1ce6) AutoScaling Monitor is running...
2023-02-15 10:05:57,987 DEBUG [c.c.s.StatsCollector] (StatsCollector-5:ctx-6279652d) (logid:0e6083c7) HostStatsCollector is running...
2023-02-15 10:05:57,995 DEBUG [c.c.a.t.Request] (StatsCollector-5:ctx-6279652d) (logid:0e6083c7) Seq 36-3755157664296861724: Received: { Ans: , MgmtId: 66988330791812, via: 36(hostname), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
2023-02-15 10:05:58,040 DEBUG [c.c.a.t.Request] (StatsCollector-5:ctx-6279652d) (logid:0e6083c7) Seq 44-5623870034678906897: Received: { Ans: , MgmtId: 66988330791812, via: 44(KVMhostname), Ver: v1, Flags: 10, { GetHostStatsAnswer } }
2023-02-15 10:06:00,155 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-9:null) (logid:) SeqA 42-320: Processing Seq 42-320: { Cmd , MgmtId: -1, via: 42, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":"96","_loadInfo":"{
"connections": []
}","wait":"0","bypassHostMaintenance":"false"}}] }
2023-02-15 10:06:00,160 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-9:null) (logid:) SeqA 42-320: Sending Seq 42-320: { Ans: , MgmtId: 66988330791812, via: 42, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
2023-02-15 10:06:00,577 DEBUG [o.a.c.h.HAManagerImpl] (BackgroundTaskPollManager-5:ctx-8836b063) (logid:8fed1abe) HA health check task is running...
2023-02-15 10:06:00,986 DEBUG [c.c.a.t.Request] (StatsCollector-6:ctx-1c90b6bf) (logid:83932347) Seq 36-3755157664296861725: Received: { Ans: , MgmtId: 66988330791812, via: 36(hostname), Ver: v1, Flags: 10, { GetVmStatsAnswer } }
^Z