2018-03-28 23:22:29.852 25497 ERROR octavia.amphorae.drivers.haproxy.rest_api_driver [-] Connection retries (currently set to 1500) exhausted. The amphora is unavailable. Reason: HTTPSConnectionPool(host='172.16.0.9', port=9443): Max retries exceeded with url: /0.5/plug/vip/10.10.200.47 (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f42b0f17f90>, 'Connection to 172.16.0.9 timed out. (connect timeout=10.0)'))
2018-03-28 23:22:29.880 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'octavia.controller.worker.tasks.amphora_driver_tasks.AmphoraePostVIPPlug' (7d1a20fa-30e7-40d2-9f98-2b8d7f247cd0) transitioned into state 'FAILURE' from state 'RUNNING'
33 predecessors (most recent first):
Atom 'octavia.controller.worker.tasks.network_tasks.GetAmphoraeNetworkConfigs' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer': <octavia.common.data_models.LoadBalancer object at 0x7f42b0f6da10>}, 'provides': {u'bc084b0c-eb2a-4f83-95c9-2a751ee26f79': <octavia.network.data_models.AmphoraNetworkConfig object at 0x7f42b0eebf10>}}
|__Atom 'reload-lb-after-plug-vip' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer_id': u'91b1165e-c44d-484a-a8bd-a6c8241d9cb4'}, 'provides': <octavia.common.data_models.LoadBalancer object at 0x7f42b0f6da10>}
|__Atom 'octavia.controller.worker.tasks.database_tasks.UpdateAmphoraVIPData' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'amps_data': [<octavia.common.data_models.Amphora object at 0x7f42b0f46950>]}, 'provides': None}
|__Atom 'octavia.controller.worker.tasks.network_tasks.PlugVIP' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer': <octavia.common.data_models.LoadBalancer object at 0x7f42b0f6d3d0>}, 'provides': [<octavia.common.data_models.Amphora object at 0x7f42b0f46950>]}
|__Atom 'octavia.controller.worker.tasks.database_tasks.UpdateVIPAfterAllocation' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'vip': <octavia.common.data_models.Vip object at 0x7f42b0f6de10>, 'loadbalancer_id': u'91b1165e-c44d-484a-a8bd-a6c8241d9cb4'}, 'provides': <octavia.common.data_models.LoadBalancer object at 0x7f42b0f6d3d0>}
|__Atom 'octavia.controller.worker.tasks.network_tasks.AllocateVIP' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer': <octavia.common.data_models.LoadBalancer object at 0x7f42b0fc2290>}, 'provides': <octavia.common.data_models.Vip object at 0x7f42b0f6de10>}
|__Flow 'octavia-new-loadbalancer-net-subflow'
|__Atom 'octavia-post-loadbalancer-amp_association-subflow-octavia-post-loadbalancer-amp_association-subflow-reload-lb-after-amp-assoc' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer_id': u'91b1165e-c44d-484a-a8bd-a6c8241d9cb4'}, 'provides': <octavia.common.data_models.LoadBalancer object at 0x7f42b0fc2290>}
|__Flow 'octavia-post-loadbalancer-amp_association-subflow-octavia-post-loadbalancer-amp_association-subflow'
|__Atom 'STANDALONE-octavia-post-map-amp-to-lb-subflow-octavia-mark-amp-standalone-indb' {'intention': 'IGNORE', 'state': 'IGNORE'}
| |__Atom 'STANDALONE-octavia-post-map-amp-to-lb-subflow-octavia-reload-amphora' {'intention': 'IGNORE', 'state': 'IGNORE', 'requires': {'amphora_id': None}}
| |__Flow 'STANDALONE-octavia-post-map-amp-to-lb-subflow'
| |__Atom 'STANDALONE-octavia-get-amphora-for-lb-subflow-octavia-mapload-balancer-to-amphora' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer_id': u'91b1165e-c44d-484a-a8bd-a6c8241d9cb4'}, 'provides': None}
| |__Flow 'STANDALONE-octavia-get-amphora-for-lb-subflow'
| |__Atom 'octavia.controller.worker.tasks.lifecycle_tasks.LoadBalancerIDToErrorOnRevertTask' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer_id': u'91b1165e-c44d-484a-a8bd-a6c8241d9cb4'}, 'provides': None}
| |__Flow 'octavia-create-loadbalancer-flow'
|__Atom 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-mark-amp-standalone-indb' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'amphora': <octavia.common.data_models.Amphora object at 0x7f42b0f6dcd0>}, 'provides': None}
|__Atom 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-reload-amphora' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'amphora_id': u'bc084b0c-eb2a-4f83-95c9-2a751ee26f79'}, 'provides': <octavia.common.data_models.Amphora object at 0x7f42b0f6dcd0>}
|__Atom 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-mark-amphora-allocated-indb' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'amphora': <octavia.common.data_models.Amphora object at 0x7f42b122bf10>, 'loadbalancer_id': u'91b1165e-c44d-484a-a8bd-a6c8241d9cb4'}, 'provides': None}
|__Atom 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-amphora-finalize' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'amphora': <octavia.common.data_models.Amphora object at 0x7f42b122bf10>}, 'provides': None}
|__Atom 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-amphora-info' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'amphora_id': u'bc084b0c-eb2a-4f83-95c9-2a751ee26f79', 'compute_obj': <octavia.common.data_models.Amphora object at 0x7f42b2190c90>}, 'provides': <octavia.common.data_models.Amphora object at 0x7f42b122bf10>}
|__Atom 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-compute-wait' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'compute_id': u'1d273ae2-3680-4328-b0a4-e60f1a69200e'}, 'provides': <octavia.common.data_models.Amphora object at 0x7f42b2190c90>}
|__Atom 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-mark-amphora-booting-indb' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'amphora_id': u'bc084b0c-eb2a-4f83-95c9-2a751ee26f79', 'compute_id': u'1d273ae2-3680-4328-b0a4-e60f1a69200e'}, 'provides': None}
|__Atom 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-amphora-computeid' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'amphora_id': u'bc084b0c-eb2a-4f83-95c9-2a751ee26f79', 'compute_id': u'1d273ae2-3680-4328-b0a4-e60f1a69200e'}, 'provides': None}
|__Atom 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-cert-compute-create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'server_pem': '-----BEGIN CERTIFICATE-----\nMIIDiDCCAnCgAwIBAgIQMncd+jCkQx6NIwMy1zJm3jANBgkqhkiG9w0BAQsFADBc\nMQswCQYDVQQGEwJVUzEPMA0GA1UECAwGRGVuaWFsMRQwEgYDVQQHDAtTcHJpbmdm\naWVsZDEMMAoGA1UECgwDRGlzMRgwFgYDVQQDDA93d3cuZXhhbXBsZS5jb20wHhcN\nMTgwMzI4MTc1NjQyWhcNMjAwMzI3MTc1NjQyWjAvMS0wKwYDVQQDDCRiYzA4NGIw\nYy1lYjJhLTRmODMtOTVjOS0yYTc1MWVlMjZmNzkwggEiMA0GCSqGSIb3DQEBAQUA\nA4IBDwAwggEKAoIBAQDVZYK37pNu3Vr6NMr0jUPgFRYEjRfCFaSgRqXskfM46juj\nVqB4inpdTr7kqqRmNaOfBURNfklhGqzIEFSG66sszdMN8XgZNfKhHIdlTOKf/CTl\nvuxI3nJgDOHG0W4DHacH8M7K7DF3Qm0Hk8k5exlQq6NVaHHdt/UXXO8MhTGaAWk6\nfuI0/kFquQoAkngSr4a0PttLca661uYRVfsGMwLW4P4tJCOGOmTr8gmvI4t2g0O7\ns/MiRtuh5lrYjJdCGf2is0eFy8WSIIjSisCsP79eTYRkM++h5Juk1YSIozmWEOS4\nbC0jF0tQtFKAJnBWaZfwBDZ4tka6PUovrI0Eoh0XAgMBAAGjczBxMAwGA1UdEwEB\n/wQCMAAwLwYDVR0RBCgwJoIkYmMwODRiMGMtZWIyYS00ZjgzLTk1YzktMmE3NTFl\nZTI2Zjc5MA4GA1UdDwEB/wQEAwIDuDAgBgNVHSUBAf8EFjAUBggrBgEFBQcDAQYI\nKwYBBQUHAwIwDQYJKoZIhvcNAQELBQADggEBACfxCeL6DvTfSLzk7+hJyvk28V2f\nmPq9VefRqvyHW8TXqOvv0MYQC7I2MbAaCebZ/CsxeE4DcvY1+8Mi1VUBSqIMT/KP\nH7K0S+RSF5vlucZeFv3iOnAvRnJzkFp1R+W1NcRkenuKc6uxXg67oaRV48SjrBJZ\n/PFoS+e4ouBq8JJQZAwOrR81t3MLQYRUWM2L8N+jydd/gojSQtR5SK6LcHKvdNWh\nPstg8VjHSLrcKxI0mYtMu4ug4OsmVFaUdojm08HG+6kpsCHBz08OveokboEMbClk\n+F4MN38ExFJAgxM4a/QDD1hgHPsRFrGsQ6KCKxYJtaijmhiKXeWuBC3RZIk=\n-----END CERTIFICATE-----\n-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEA1WWCt+6Tbt1a+jTK9I1D4BUWBI0XwhWkoEal7JHzOOo7o1ag\neIp6XU6+5KqkZjWjnwVETX5JYRqsyBBUhuurLM3TDfF4GTXyoRyHZUzin/wk5b7s\nSN5yYAzhxtFuAx2nB/DOyuwxd0JtB5PJOXsZUKujVWhx3bf1F1zvDIUxmgFpOn7i\nNP5BarkKAJJ4Eq+GtD7bS3GuutbmEVX7BjMC1uD+LSQjhjpk6/IJryOLdoNDu7Pz\nIkbboeZa2IyXQhn9orNHhcvFkiCI0orArD+/Xk2EZDPvoeSbpNWEiKM5lhDkuGwt\nIxdLULRSgCZwVmmX8AQ2eLZGuj1KL6yNBKIdFwIDAQABAoIBAQC13ukUt9mhoDh8\njGf5GzihD3ePc2yvNmDzVKLvR8iiVkdzw03KzkLRqV5aZvDsR4Tm78cxwJLrlp9x\ndzGDN/HQoq7g8E6qNedO1pCyjl+LeIs934Vj6WpTYEdAepenQitsQhkMq1bRTKS9\n03+pZLfDoDT5gqWhELprKgJ2ikIm/MPyoaqd04GPsZ4h/1HAvAcv7V71FRSTC5+v\nMbk/PuFZWQvhgr8t5NpwDZhVh3JaFbgp0nf3G/qcIn4fWafWynsRHe3Jx2Ko3xPI\nG+Mr1qJ6orILMUr+OSQORHSSabsDt9DRhB9DMhOPywJauCtu3lbOtkWY1r/7W1T6\nC3+8WL7ZAoGBAPYPb4wYynBU967QUuRc7aYZ031tEhat/MnL+VW3u6MhUyhuMvs5\nTcT3+ft2ECMtD3iJC8CLuqaXatfXzycynuyB/u1pySZ4j7KgBXyX26H486ka9KSR\nJYb0RwPZWB6XolZ87fa9rIm3XsX0RdcWZ2Y7kMFeUWe/WnhmeOfYBIKFAoGBAN4E\nSqShAeciYac3y4LAZWs7/xGJTqkFhbuyeFAk+R/OzmKR1b3XYYZcPSIazwjIR+0G\nUxGFgwcz8BrihyXRdUY2nNg+OUelEsZVbbAsYYTsElTFf7S9d6Fq/AVx8c/MWwTl\noTwF4sVd9aSSwlDrvbr5cDvk1iztenqtvbRKaynrAoGAHKDVIJ6txv4LxzoTuuAZ\nAk9U7O3owdzCYZQq0U9MNIGnFDgmFFapdrZHTnuBCyat+x8uz9sx/zn0w2Inhu77\n0JabW3IIZkJ2rZiyqrlcY278onsxZYhBrn7zWD5MY1rPF3L3mNKepXNBQpTWH11K\n8HkTsXvQytpXo1SA0rCubaUCgYAs0Lb0hl+o9WK662NUMFwIimLRemL7cVbubXcl\nzmDPT+KPORrpKEnSYlogwUKbZjpGmSk5dzbcxFoqQyG+u4otsyZz7BOJ58Njb6kl\nj7/bof+NRxShOkGkvsDCpJ6IMJsK4QNr51BAQNgt8UXyKbPxwxoprOaiyQkZLGsH\nsqHxIwKBgQC/sCnW51Y7A5QQuhUIYWHpTsYagXuE6fSdS2/AISGXAesAXT4kAtDE\nbumV74GVQpfn6+bqc0NiZwIMrwYnt0Is5fwvRpt896XkIfyPdpNA/aXpwoZTaObN\nwFuH2a5QTCejYILhD3JHqbNXswfTUbwsGl1MI054zJfrqozO3qAcaQ==\n-----END RSA PRIVATE KEY-----\n', 'amphora_id': u'bc084b0c-eb2a-4f83-95c9-2a751ee26f79'}, 'provides': u'1d273ae2-3680-4328-b0a4-e60f1a69200e'}
|__Atom 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-cert-expiration' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'server_pem': '-----BEGIN CERTIFICATE-----\nMIIDiDCCAnCgAwIBAgIQMncd+jCkQx6NIwMy1zJm3jANBgkqhkiG9w0BAQsFADBc\nMQswCQYDVQQGEwJVUzEPMA0GA1UECAwGRGVuaWFsMRQwEgYDVQQHDAtTcHJpbmdm\naWVsZDEMMAoGA1UECgwDRGlzMRgwFgYDVQQDDA93d3cuZXhhbXBsZS5jb20wHhcN\nMTgwMzI4MTc1NjQyWhcNMjAwMzI3MTc1NjQyWjAvMS0wKwYDVQQDDCRiYzA4NGIw\nYy1lYjJhLTRmODMtOTVjOS0yYTc1MWVlMjZmNzkwggEiMA0GCSqGSIb3DQEBAQUA\nA4IBDwAwggEKAoIBAQDVZYK37pNu3Vr6NMr0jUPgFRYEjRfCFaSgRqXskfM46juj\nVqB4inpdTr7kqqRmNaOfBURNfklhGqzIEFSG66sszdMN8XgZNfKhHIdlTOKf/CTl\nvuxI3nJgDOHG0W4DHacH8M7K7DF3Qm0Hk8k5exlQq6NVaHHdt/UXXO8MhTGaAWk6\nfuI0/kFquQoAkngSr4a0PttLca661uYRVfsGMwLW4P4tJCOGOmTr8gmvI4t2g0O7\ns/MiRtuh5lrYjJdCGf2is0eFy8WSIIjSisCsP79eTYRkM++h5Juk1YSIozmWEOS4\nbC0jF0tQtFKAJnBWaZfwBDZ4tka6PUovrI0Eoh0XAgMBAAGjczBxMAwGA1UdEwEB\n/wQCMAAwLwYDVR0RBCgwJoIkYmMwODRiMGMtZWIyYS00ZjgzLTk1YzktMmE3NTFl\nZTI2Zjc5MA4GA1UdDwEB/wQEAwIDuDAgBgNVHSUBAf8EFjAUBggrBgEFBQcDAQYI\nKwYBBQUHAwIwDQYJKoZIhvcNAQELBQADggEBACfxCeL6DvTfSLzk7+hJyvk28V2f\nmPq9VefRqvyHW8TXqOvv0MYQC7I2MbAaCebZ/CsxeE4DcvY1+8Mi1VUBSqIMT/KP\nH7K0S+RSF5vlucZeFv3iOnAvRnJzkFp1R+W1NcRkenuKc6uxXg67oaRV48SjrBJZ\n/PFoS+e4ouBq8JJQZAwOrR81t3MLQYRUWM2L8N+jydd/gojSQtR5SK6LcHKvdNWh\nPstg8VjHSLrcKxI0mYtMu4ug4OsmVFaUdojm08HG+6kpsCHBz08OveokboEMbClk\n+F4MN38ExFJAgxM4a/QDD1hgHPsRFrGsQ6KCKxYJtaijmhiKXeWuBC3RZIk=\n-----END CERTIFICATE-----\n-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEA1WWCt+6Tbt1a+jTK9I1D4BUWBI0XwhWkoEal7JHzOOo7o1ag\neIp6XU6+5KqkZjWjnwVETX5JYRqsyBBUhuurLM3TDfF4GTXyoRyHZUzin/wk5b7s\nSN5yYAzhxtFuAx2nB/DOyuwxd0JtB5PJOXsZUKujVWhx3bf1F1zvDIUxmgFpOn7i\nNP5BarkKAJJ4Eq+GtD7bS3GuutbmEVX7BjMC1uD+LSQjhjpk6/IJryOLdoNDu7Pz\nIkbboeZa2IyXQhn9orNHhcvFkiCI0orArD+/Xk2EZDPvoeSbpNWEiKM5lhDkuGwt\nIxdLULRSgCZwVmmX8AQ2eLZGuj1KL6yNBKIdFwIDAQABAoIBAQC13ukUt9mhoDh8\njGf5GzihD3ePc2yvNmDzVKLvR8iiVkdzw03KzkLRqV5aZvDsR4Tm78cxwJLrlp9x\ndzGDN/HQoq7g8E6qNedO1pCyjl+LeIs934Vj6WpTYEdAepenQitsQhkMq1bRTKS9\n03+pZLfDoDT5gqWhELprKgJ2ikIm/MPyoaqd04GPsZ4h/1HAvAcv7V71FRSTC5+v\nMbk/PuFZWQvhgr8t5NpwDZhVh3JaFbgp0nf3G/qcIn4fWafWynsRHe3Jx2Ko3xPI\nG+Mr1qJ6orILMUr+OSQORHSSabsDt9DRhB9DMhOPywJauCtu3lbOtkWY1r/7W1T6\nC3+8WL7ZAoGBAPYPb4wYynBU967QUuRc7aYZ031tEhat/MnL+VW3u6MhUyhuMvs5\nTcT3+ft2ECMtD3iJC8CLuqaXatfXzycynuyB/u1pySZ4j7KgBXyX26H486ka9KSR\nJYb0RwPZWB6XolZ87fa9rIm3XsX0RdcWZ2Y7kMFeUWe/WnhmeOfYBIKFAoGBAN4E\nSqShAeciYac3y4LAZWs7/xGJTqkFhbuyeFAk+R/OzmKR1b3XYYZcPSIazwjIR+0G\nUxGFgwcz8BrihyXRdUY2nNg+OUelEsZVbbAsYYTsElTFf7S9d6Fq/AVx8c/MWwTl\noTwF4sVd9aSSwlDrvbr5cDvk1iztenqtvbRKaynrAoGAHKDVIJ6txv4LxzoTuuAZ\nAk9U7O3owdzCYZQq0U9MNIGnFDgmFFapdrZHTnuBCyat+x8uz9sx/zn0w2Inhu77\n0JabW3IIZkJ2rZiyqrlcY278onsxZYhBrn7zWD5MY1rPF3L3mNKepXNBQpTWH11K\n8HkTsXvQytpXo1SA0rCubaUCgYAs0Lb0hl+o9WK662NUMFwIimLRemL7cVbubXcl\nzmDPT+KPORrpKEnSYlogwUKbZjpGmSk5dzbcxFoqQyG+u4otsyZz7BOJ58Njb6kl\nj7/bof+NRxShOkGkvsDCpJ6IMJsK4QNr51BAQNgt8UXyKbPxwxoprOaiyQkZLGsH\nsqHxIwKBgQC/sCnW51Y7A5QQuhUIYWHpTsYagXuE6fSdS2/AISGXAesAXT4kAtDE\nbumV74GVQpfn6+bqc0NiZwIMrwYnt0Is5fwvRpt896XkIfyPdpNA/aXpwoZTaObN\nwFuH2a5QTCejYILhD3JHqbNXswfTUbwsGl1MI054zJfrqozO3qAcaQ==\n-----END RSA PRIVATE KEY-----\n', 'amphora_id': u'bc084b0c-eb2a-4f83-95c9-2a751ee26f79'}, 'provides': None}
|__Atom 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-generate-serverpem' {'intention': 'EXECUTE', 'state': 'SUCCESS'}
|__Atom 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-create-amphora-indb' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {}, 'provides': u'bc084b0c-eb2a-4f83-95c9-2a751ee26f79'}
|__Flow 'STANDALONE-octavia-create-amp-for-lb-subflow'
|__Atom 'STANDALONE-octavia-get-amphora-for-lb-subflow-octavia-mapload-balancer-to-amphora' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer_id': u'91b1165e-c44d-484a-a8bd-a6c8241d9cb4'}, 'provides': None}
|__Flow 'STANDALONE-octavia-get-amphora-for-lb-subflow'
|__Atom 'octavia.controller.worker.tasks.lifecycle_tasks.LoadBalancerIDToErrorOnRevertTask' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer_id': u'91b1165e-c44d-484a-a8bd-a6c8241d9cb4'}, 'provides': None}
|__Flow 'octavia-create-loadbalancer-flow'
2018-03-28 23:22:29.880 25497 ERROR octavia.controller.worker.controller_worker Traceback (most recent call last):
2018-03-28 23:22:29.880 25497 ERROR octavia.controller.worker.controller_worker File "/usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2018-03-28 23:22:29.880 25497 ERROR octavia.controller.worker.controller_worker result = task.execute(**arguments)
2018-03-28 23:22:29.880 25497 ERROR octavia.controller.worker.controller_worker File "/usr/local/lib/python2.7/dist-packages/octavia/controller/worker/tasks/amphora_driver_tasks.py", line 241, in execute
2018-03-28 23:22:29.880 25497 ERROR octavia.controller.worker.controller_worker amphorae_network_config)
2018-03-28 23:22:29.880 25497 ERROR octavia.controller.worker.controller_worker File "/usr/local/lib/python2.7/dist-packages/octavia/controller/worker/tasks/amphora_driver_tasks.py", line 220, in execute
2018-03-28 23:22:29.880 25497 ERROR octavia.controller.worker.controller_worker amphora, loadbalancer, amphorae_network_config)
2018-03-28 23:22:29.880 25497 ERROR octavia.controller.worker.controller_worker File "/usr/local/lib/python2.7/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 136, in post_vip_plug
2018-03-28 23:22:29.880 25497 ERROR octavia.controller.worker.controller_worker net_info)
2018-03-28 23:22:29.880 25497 ERROR octavia.controller.worker.controller_worker File "/usr/local/lib/python2.7/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 375, in plug_vip
2018-03-28 23:22:29.880 25497 ERROR octavia.controller.worker.controller_worker json=net_info)
2018-03-28 23:22:29.880 25497 ERROR octavia.controller.worker.controller_worker File "/usr/local/lib/python2.7/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 299, in request
2018-03-28 23:22:29.880 25497 ERROR octavia.controller.worker.controller_worker raise driver_except.TimeOutException()
2018-03-28 23:22:29.880 25497 ERROR octavia.controller.worker.controller_worker TimeOutException: contacting the amphora timed out
2018-03-28 23:22:29.880 25497 ERROR octavia.controller.worker.controller_worker
2018-03-28 23:22:29.896 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'octavia.controller.worker.tasks.amphora_driver_tasks.AmphoraePostVIPPlug' (7d1a20fa-30e7-40d2-9f98-2b8d7f247cd0) transitioned into state 'REVERTING' from state 'FAILURE' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:29.899 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'octavia.controller.worker.tasks.amphora_driver_tasks.AmphoraePostVIPPlug' (7d1a20fa-30e7-40d2-9f98-2b8d7f247cd0) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:29.903 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'octavia.controller.worker.tasks.network_tasks.GetAmphoraeNetworkConfigs' (e93473af-1021-4973-b1ae-2392eda41692) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:29.905 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'octavia.controller.worker.tasks.network_tasks.GetAmphoraeNetworkConfigs' (e93473af-1021-4973-b1ae-2392eda41692) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:29.910 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'reload-lb-after-plug-vip' (e6eed8e6-895a-4d56-be21-bda4d7c0564b) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:29.912 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'reload-lb-after-plug-vip' (e6eed8e6-895a-4d56-be21-bda4d7c0564b) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:29.915 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'octavia.controller.worker.tasks.database_tasks.UpdateAmphoraVIPData' (b4088122-cf34-428c-94f9-e684ede0ef16) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:29.917 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'octavia.controller.worker.tasks.database_tasks.UpdateAmphoraVIPData' (b4088122-cf34-428c-94f9-e684ede0ef16) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:29.921 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'octavia.controller.worker.tasks.network_tasks.PlugVIP' (1fa56b67-f836-4ac5-b819-2f9ec0d6a5fe) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:29.921 25497 WARNING octavia.controller.worker.tasks.network_tasks [-] Unable to plug VIP for loadbalancer id 91b1165e-c44d-484a-a8bd-a6c8241d9cb4
2018-03-28 23:22:31.209 25497 DEBUG neutronclient.v2_0.client [-] GET call to neutron for http://testcont:9696/v2.0/subnets/f2abcdfd-73d9-454d-91ba-f0f111878775.json used request id req-139546da-91ce-4143-8042-2d75e7396266 _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:31.275 25497 DEBUG neutronclient.v2_0.client [-] GET call to neutron for http://testcont:9696/v2.0/ports.json?device_id=1d273ae2-3680-4328-b0a4-e60f1a69200e used request id req-181c311e-075a-4cf1-912e-67d9e8bc09c3 _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:31.276 25497 DEBUG octavia.network.drivers.neutron.allowed_address_pairs [-] Cannot get amphora 1d273ae2-3680-4328-b0a4-e60f1a69200e interface, skipped unplug_vip /usr/local/lib/python2.7/dist-packages/octavia/network/drivers/neutron/allowed_address_pairs.py:374
2018-03-28 23:22:31.278 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'octavia.controller.worker.tasks.network_tasks.PlugVIP' (1fa56b67-f836-4ac5-b819-2f9ec0d6a5fe) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:31.282 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'octavia.controller.worker.tasks.database_tasks.UpdateVIPAfterAllocation' (1d8e607c-ddd1-4c7e-898b-0cf03a32e32e) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:31.285 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'octavia.controller.worker.tasks.database_tasks.UpdateVIPAfterAllocation' (1d8e607c-ddd1-4c7e-898b-0cf03a32e32e) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:31.289 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'octavia.controller.worker.tasks.network_tasks.AllocateVIP' (3df7c97c-ffd4-4e87-b259-d2c86a182302) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:31.290 25497 WARNING octavia.controller.worker.tasks.network_tasks [-] Deallocating vip 10.10.200.47
2018-03-28 23:22:31.309 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"message": "The resource could not be found.<br /><br />\n\n\n", "code": "404 Not Found", "title": "Not Found"} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:31.310 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/ports/None.json used request id req-b6a9f7d8-6253-4b10-af11-44744978cfa7 _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:31.310 25497 DEBUG octavia.network.drivers.neutron.allowed_address_pairs [-] VIP instance port None already deleted. Skipping. deallocate_vip /usr/local/lib/python2.7/dist-packages/octavia/network/drivers/neutron/allowed_address_pairs.py:270
2018-03-28 23:22:31.437 25497 DEBUG neutronclient.v2_0.client [-] GET call to neutron for http://testcont:9696/v2.0/ports/54f5577d-f590-4c99-b17a-f060dad1c0e5.json used request id req-0b45cf05-8ea7-43f9-b703-e84d4896648a _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:31.576 25497 DEBUG neutronclient.v2_0.client [-] GET call to neutron for http://testcont:9696/v2.0/security-groups.json?name=lb-91b1165e-c44d-484a-a8bd-a6c8241d9cb4 used request id req-56ef8cd4-3094-418c-aaca-ae2172faa310 _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:31.577 25497 INFO octavia.network.drivers.neutron.allowed_address_pairs [-] Removing security group 4d3082c2-4d79-46df-b284-96ca3a54e56f from port 54f5577d-f590-4c99-b17a-f060dad1c0e5
2018-03-28 23:22:31.714 25497 DEBUG neutronclient.v2_0.client [-] GET call to neutron for http://testcont:9696/v2.0/ports/54f5577d-f590-4c99-b17a-f060dad1c0e5.json used request id req-6d73da9d-f77d-4768-8abd-ecd5232d858d _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:32.556 25497 DEBUG neutronclient.v2_0.client [-] PUT call to neutron for http://testcont:9696/v2.0/ports/54f5577d-f590-4c99-b17a-f060dad1c0e5.json used request id req-5db28785-1662-46f6-b9d8-41e4603bbf92 _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:32.630 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"message": "Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.", "type": "SecurityGroupInUse", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:32.630 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/security-groups/4d3082c2-4d79-46df-b284-96ca3a54e56f.json used request id req-55345f2e-dffa-4b1d-8132-96c18c91b6b1 _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:32.630 25497 WARNING octavia.network.drivers.neutron.allowed_address_pairs [-] Attempt 1 to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f failed.
2018-03-28 23:22:33.710 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"message": "Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.", "type": "SecurityGroupInUse", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:33.711 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/security-groups/4d3082c2-4d79-46df-b284-96ca3a54e56f.json used request id req-4e4a1fce-dfbd-4793-9026-24f866b065ac _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:33.711 25497 WARNING octavia.network.drivers.neutron.allowed_address_pairs [-] Attempt 2 to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f failed.
2018-03-28 23:22:34.790 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"message": "Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.", "type": "SecurityGroupInUse", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:34.790 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/security-groups/4d3082c2-4d79-46df-b284-96ca3a54e56f.json used request id req-5d69ea5b-2475-42c4-babd-f84ee4c8e400 _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:34.791 25497 WARNING octavia.network.drivers.neutron.allowed_address_pairs [-] Attempt 3 to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f failed.
2018-03-28 23:22:35.875 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"message": "Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.", "type": "SecurityGroupInUse", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:35.876 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/security-groups/4d3082c2-4d79-46df-b284-96ca3a54e56f.json used request id req-037e0d91-c12d-44c6-9491-aed784a45919 _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:35.877 25497 WARNING octavia.network.drivers.neutron.allowed_address_pairs [-] Attempt 4 to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f failed.
2018-03-28 23:22:36.955 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"message": "Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.", "type": "SecurityGroupInUse", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:36.957 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/security-groups/4d3082c2-4d79-46df-b284-96ca3a54e56f.json used request id req-858790ee-18b4-4099-a35f-b0b087e9d7fe _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:36.957 25497 WARNING octavia.network.drivers.neutron.allowed_address_pairs [-] Attempt 5 to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f failed.
2018-03-28 23:22:38.057 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"message": "Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.", "type": "SecurityGroupInUse", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:38.058 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/security-groups/4d3082c2-4d79-46df-b284-96ca3a54e56f.json used request id req-4fbaaec4-b585-4492-847b-af9b86994092 _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:38.059 25497 WARNING octavia.network.drivers.neutron.allowed_address_pairs [-] Attempt 6 to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f failed.
2018-03-28 23:22:39.141 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"message": "Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.", "type": "SecurityGroupInUse", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:39.142 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/security-groups/4d3082c2-4d79-46df-b284-96ca3a54e56f.json used request id req-3c9351df-7e79-47d9-9896-3f57802697e5 _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:39.143 25497 WARNING octavia.network.drivers.neutron.allowed_address_pairs [-] Attempt 7 to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f failed.
2018-03-28 23:22:40.242 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"message": "Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.", "type": "SecurityGroupInUse", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:40.243 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/security-groups/4d3082c2-4d79-46df-b284-96ca3a54e56f.json used request id req-4abddef3-d965-4664-9861-bd986d70ac12 _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:40.244 25497 WARNING octavia.network.drivers.neutron.allowed_address_pairs [-] Attempt 8 to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f failed.
2018-03-28 23:22:41.336 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"message": "Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.", "type": "SecurityGroupInUse", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:41.337 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/security-groups/4d3082c2-4d79-46df-b284-96ca3a54e56f.json used request id req-2f717a54-0a19-4250-b00f-83e45c5f941a _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:41.338 25497 WARNING octavia.network.drivers.neutron.allowed_address_pairs [-] Attempt 9 to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f failed.
2018-03-28 23:22:42.426 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"message": "Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.", "type": "SecurityGroupInUse", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:42.426 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/security-groups/4d3082c2-4d79-46df-b284-96ca3a54e56f.json used request id req-02cd77fb-5b9a-47a4-b071-71aeaafeed3b _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:42.427 25497 WARNING octavia.network.drivers.neutron.allowed_address_pairs [-] Attempt 10 to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f failed.
2018-03-28 23:22:43.511 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"message": "Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.", "type": "SecurityGroupInUse", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:43.512 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/security-groups/4d3082c2-4d79-46df-b284-96ca3a54e56f.json used request id req-29ea28b6-32f9-4a88-89c7-d4ad8cf011bf _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:43.512 25497 WARNING octavia.network.drivers.neutron.allowed_address_pairs [-] Attempt 11 to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f failed.
2018-03-28 23:22:44.599 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"message": "Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.", "type": "SecurityGroupInUse", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:44.599 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/security-groups/4d3082c2-4d79-46df-b284-96ca3a54e56f.json used request id req-35072452-3ef7-4363-aba4-c8ed6fd47d10 _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:44.600 25497 WARNING octavia.network.drivers.neutron.allowed_address_pairs [-] Attempt 12 to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f failed.
2018-03-28 23:22:45.682 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"message": "Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.", "type": "SecurityGroupInUse", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:45.683 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/security-groups/4d3082c2-4d79-46df-b284-96ca3a54e56f.json used request id req-682f3072-f727-4927-9e96-b6b201971940 _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:45.684 25497 WARNING octavia.network.drivers.neutron.allowed_address_pairs [-] Attempt 13 to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f failed.
2018-03-28 23:22:46.787 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"message": "Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.", "type": "SecurityGroupInUse", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:46.788 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/security-groups/4d3082c2-4d79-46df-b284-96ca3a54e56f.json used request id req-74a5e47a-5b03-4e11-b659-ebc374f6a954 _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:46.788 25497 WARNING octavia.network.drivers.neutron.allowed_address_pairs [-] Attempt 14 to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f failed.
2018-03-28 23:22:47.871 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"message": "Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.", "type": "SecurityGroupInUse", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:47.872 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/security-groups/4d3082c2-4d79-46df-b284-96ca3a54e56f.json used request id req-05ec4c06-8138-43bf-9c47-3e637af7b705 _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:47.873 25497 WARNING octavia.network.drivers.neutron.allowed_address_pairs [-] Attempt 15 to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f failed.
2018-03-28 23:22:48.952 25497 DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"message": "Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.", "type": "SecurityGroupInUse", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:266
2018-03-28 23:22:48.953 25497 DEBUG neutronclient.v2_0.client [-] DELETE call to neutron for http://testcont:9696/v2.0/security-groups/4d3082c2-4d79-46df-b284-96ca3a54e56f.json used request id req-2ed2f458-f990-4379-a3b6-bd4ac8264866 _append_request_id /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128
2018-03-28 23:22:48.953 25497 WARNING octavia.network.drivers.neutron.allowed_address_pairs [-] Attempt 16 to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f failed.
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs [-] All attempts to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f have failed.
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs Traceback (most recent call last):
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/local/lib/python2.7/dist-packages/octavia/network/drivers/neutron/allowed_address_pairs.py", line 220, in _delete_vip_security_group
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs self.neutron_client.delete_security_group(sec_grp)
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 945, in delete_security_group
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs return self.delete(self.security_group_path % (security_group))
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 357, in delete
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs headers=headers, params=params)
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 338, in retry_request
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs headers=headers, params=params)
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 301, in do_request
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs self._handle_fault_response(status_code, replybody, resp)
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 276, in _handle_fault_response
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs exception_handler_v20(status_code, error_body)
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 92, in exception_handler_v20
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs request_ids=request_ids)
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs Conflict: Security Group 4d3082c2-4d79-46df-b284-96ca3a54e56f in use.
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs Neutron server returns request_ids: ['req-2ed2f458-f990-4379-a3b6-bd4ac8264866']
2018-03-28 23:22:49.956 25497 ERROR octavia.network.drivers.neutron.allowed_address_pairs
2018-03-28 23:22:49.960 25497 ERROR octavia.controller.worker.tasks.network_tasks [-] Failed to deallocate VIP. Resources may still be in use from vip: 10.10.200.47 due to error: All attempts to remove security group 4d3082c2-4d79-46df-b284-96ca3a54e56f have failed.
2018-03-28 23:22:49.965 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'octavia.controller.worker.tasks.network_tasks.AllocateVIP' (3df7c97c-ffd4-4e87-b259-d2c86a182302) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:49.971 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'octavia-post-loadbalancer-amp_association-subflow-octavia-post-loadbalancer-amp_association-subflow-reload-lb-after-amp-assoc' (0051d205-ac68-4bab-92fa-4098c93d3c82) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:49.973 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'octavia-post-loadbalancer-amp_association-subflow-octavia-post-loadbalancer-amp_association-subflow-reload-lb-after-amp-assoc' (0051d205-ac68-4bab-92fa-4098c93d3c82) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:49.979 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-mark-amp-standalone-indb' (f196be36-10b7-46df-a965-5b3b5f5d9b3e) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:49.980 25497 WARNING octavia.controller.worker.tasks.database_tasks [-] Reverting amphora role in DB for amp id bc084b0c-eb2a-4f83-95c9-2a751ee26f79
2018-03-28 23:22:50.006 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-mark-amp-standalone-indb' (f196be36-10b7-46df-a965-5b3b5f5d9b3e) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:50.010 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-reload-amphora' (af41d128-bedc-4d85-98d7-21adf129c201) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:50.012 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-reload-amphora' (af41d128-bedc-4d85-98d7-21adf129c201) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:50.015 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-mark-amphora-allocated-indb' (d3bd2388-f7f0-47cd-8f51-aa9dbcd7472a) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:50.016 25497 WARNING octavia.controller.worker.tasks.database_tasks [-] Reverting mark amphora ready in DB for amp id bc084b0c-eb2a-4f83-95c9-2a751ee26f79 and compute id 1d273ae2-3680-4328-b0a4-e60f1a69200e
2018-03-28 23:22:50.026 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-mark-amphora-allocated-indb' (d3bd2388-f7f0-47cd-8f51-aa9dbcd7472a) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:50.030 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-amphora-finalize' (ebd1a23a-2112-4044-bfb1-27846fffe9b3) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:50.031 25497 WARNING octavia.controller.worker.tasks.amphora_driver_tasks [-] Reverting amphora finalize.
2018-03-28 23:22:50.040 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-amphora-finalize' (ebd1a23a-2112-4044-bfb1-27846fffe9b3) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:50.044 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-amphora-info' (ed1dd8eb-f3aa-429d-9acf-d2a529c93139) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:50.046 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-amphora-info' (ed1dd8eb-f3aa-429d-9acf-d2a529c93139) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:50.050 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-compute-wait' (f65abd54-f22f-446d-84f1-0c2c7437f496) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:50.054 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-compute-wait' (f65abd54-f22f-446d-84f1-0c2c7437f496) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:50.059 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-mark-amphora-booting-indb' (a6804c65-1b30-4bab-9957-c624a63c2fd1) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:50.061 25497 WARNING octavia.controller.worker.tasks.database_tasks [-] Reverting mark amphora booting in DB for amp id bc084b0c-eb2a-4f83-95c9-2a751ee26f79 and compute id 1d273ae2-3680-4328-b0a4-e60f1a69200e
2018-03-28 23:22:50.071 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-mark-amphora-booting-indb' (a6804c65-1b30-4bab-9957-c624a63c2fd1) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:50.076 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-amphora-computeid' (03d2f57f-2604-4c4e-acfe-00c934e01e35) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:50.079 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-amphora-computeid' (03d2f57f-2604-4c4e-acfe-00c934e01e35) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:50.081 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-cert-compute-create' (b390350f-ffc4-406d-82fa-8c07b3915042) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:50.082 25497 WARNING octavia.controller.worker.tasks.compute_tasks [-] Reverting compute create for amphora with idbc084b0c-eb2a-4f83-95c9-2a751ee26f79 and compute id: 1d273ae2-3680-4328-b0a4-e60f1a69200e
2018-03-28 23:22:50.400 25497 DEBUG novaclient.v2.client [-] REQ: curl -g -i -X DELETE http://testcont:8774/v2.1/servers/1d273ae2-3680-4328-b0a4-e60f1a69200e -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.1" -H "X-Auth-Token: {SHA1}b323a1e636f30e1c1138035588855d93a92998b8" _http_log_request /usr/lib/python2.7/dist-packages/keystoneauth1/session.py:347
2018-03-28 23:22:50.883 25497 DEBUG novaclient.v2.client [-] RESP: [404] Openstack-Api-Version: compute 2.1 X-Openstack-Nova-Api-Version: 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version Content-Type: application/json; charset=UTF-8 Content-Length: 111 X-Compute-Request-Id: req-b6330902-64ed-4756-be23-f16594b148d0 Date: Wed, 28 Mar 2018 20:22:50 GMT Connection: keep-alive
RESP BODY: {"itemNotFound": {"message": "Instance 1d273ae2-3680-4328-b0a4-e60f1a69200e could not be found.", "code": 404}}
_http_log_response /usr/lib/python2.7/dist-packages/keystoneauth1/session.py:395
2018-03-28 23:22:50.883 25497 DEBUG novaclient.v2.client [-] DELETE call to compute for http://testcont:8774/v2.1/servers/1d273ae2-3680-4328-b0a4-e60f1a69200e used request id req-b6330902-64ed-4756-be23-f16594b148d0 request /usr/lib/python2.7/dist-packages/keystoneauth1/session.py:640
2018-03-28 23:22:50.884 25497 WARNING octavia.compute.drivers.nova_driver [-] Nova instance with id: 1d273ae2-3680-4328-b0a4-e60f1a69200e not found. Assuming already deleted.
2018-03-28 23:22:50.887 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-cert-compute-create' (b390350f-ffc4-406d-82fa-8c07b3915042) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:50.890 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-cert-expiration' (79ce3e69-b045-4404-9833-af9d0b29ba3a) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:50.893 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-cert-expiration' (79ce3e69-b045-4404-9833-af9d0b29ba3a) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:50.896 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-generate-serverpem' (d22061fb-17ca-4a26-8b67-8a2caee22b00) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:50.899 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-generate-serverpem' (d22061fb-17ca-4a26-8b67-8a2caee22b00) transitioned into state 'REVERTED' from state 'REVERTING'
2018-03-28 23:22:50.902 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-create-amphora-indb' (e49049dd-cbb2-4e89-9a97-e46d9a0cc33c) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:50.903 25497 WARNING octavia.controller.worker.tasks.database_tasks [-] Reverting create amphora in DB for amp id bc084b0c-eb2a-4f83-95c9-2a751ee26f79
2018-03-28 23:22:50.911 25497 ERROR octavia.controller.worker.tasks.database_tasks [-] Failed to delete amphora bc084b0c-eb2a-4f83-95c9-2a751ee26f79 in the database due to: No row was found for one()
2018-03-28 23:22:50.913 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-create-amphora-indb' (e49049dd-cbb2-4e89-9a97-e46d9a0cc33c) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:50.916 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-get-amphora-for-lb-subflow-octavia-mapload-balancer-to-amphora' (6f5b2ae9-8f4b-460f-89cd-c361b839211f) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:50.917 25497 WARNING octavia.controller.worker.tasks.database_tasks [-] Reverting Amphora allocation for the load balancer 91b1165e-c44d-484a-a8bd-a6c8241d9cb4 in the database.
2018-03-28 23:22:50.928 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-get-amphora-for-lb-subflow-octavia-mapload-balancer-to-amphora' (6f5b2ae9-8f4b-460f-89cd-c361b839211f) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:50.930 25497 DEBUG octavia.controller.worker.controller_worker [-] Task 'octavia.controller.worker.tasks.lifecycle_tasks.LoadBalancerIDToErrorOnRevertTask' (64857afb-d684-43cb-9ace-ed3a9a35237e) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194
2018-03-28 23:22:50.940 25497 WARNING octavia.controller.worker.controller_worker [-] Task 'octavia.controller.worker.tasks.lifecycle_tasks.LoadBalancerIDToErrorOnRevertTask' (64857afb-d684-43cb-9ace-ed3a9a35237e) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
2018-03-28 23:22:50.949 25497 WARNING octavia.controller.worker.controller_worker [-] Flow 'octavia-create-loadbalancer-flow' (ec02a034-7e46-44f9-ada2-65a321f1077c) transitioned into state 'REVERTED' from state 'RUNNING'
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server [-] Exception during message handling
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 222, in dispatch
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 192, in _do_dispatch
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/octavia/controller/queue/endpoint.py", line 45, in create_load_balancer
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server self.worker.create_load_balancer(load_balancer_id)
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/octavia/controller/worker/controller_worker.py", line 280, in create_load_balancer
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server create_lb_tf.run()
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 247, in run
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout):
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures)
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/taskflow/types/failure.py", line 336, in reraise_if_any
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server failures[0].reraise()
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/taskflow/types/failure.py", line 343, in reraise
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info)
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server result = task.execute(**arguments)
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/octavia/controller/worker/tasks/amphora_driver_tasks.py", line 241, in execute
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server amphorae_network_config)
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/octavia/controller/worker/tasks/amphora_driver_tasks.py", line 220, in execute
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server amphora, loadbalancer, amphorae_network_config)
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 136, in post_vip_plug
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server net_info)
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 375, in plug_vip
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server json=net_info)
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 299, in request
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server raise driver_except.TimeOutException()
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server TimeOutException: contacting the amphora timed out
2018-03-28 23:22:50.950 25497 ERROR oslo_messaging.rpc.server