我们正在尝试创建 Cinder 卷。Cinder API 接受请求,但卷卡在creating
状态。无论使用哪个后端(即也使用lvm
),都会发生这种情况。似乎 Cinder API 和其余进程之间发生了一些超时。如何解决这个问题?非常感谢任何线索。
以下是 cinder-api 日志的摘录:
req-00001bfc-808d-4525-904d-207e40ac1e2d 9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - - default default] POST http://10.10.1.83:8776/v3/949326f6d40c4d49a93fffac8b958b49/volumes 2022-08-11 17:17:01.415 21 INFO cinder.api.v3.volumes [None req-00001bfc-808d-4525-904d-207e40ac1e2d 9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - - default default] Create volume of 4 GB 2022-08-11 17:17:01.435 21 INFO cinder.volume.api [None req-00001bfc-808d-4525-904d-207e40ac1e2d 9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - - default default] Availability Zones retrieved successfully. 2022-08-11 17:17:01.619 21 INFO cinder.volume.api [None req-00001bfc-808d-4525-904d-207e40ac1e2d 9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - - default default] Create volume request issued successfully. 2022-08-11 17:17:01.620 21 INFO cinder.api.openstack.wsgi [None req-00001bfc-808d-4525-904d-207e40ac1e2d 9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - - default default] http://10.10.1.83:8776/v3/949326f6d40c4d49a93fffac8b958b49/volumes returned with HTTP 202 2022-08-11 17:17:11.119 23 INFO cinder.api.openstack.wsgi [None req-a50d2c2f-784b-481e-a34a-48f34225eed2 - - - - - -] GET http://10.10.1.80:8776/ 2022-08-11 17:17:11.120 23 INFO cinder.api.openstack.wsgi [None req-a50d2c2f-784b-481e-a34a-48f34225eed2 - - - - - -] http://10.10.1.80:8776/ returned with HTTP 300 2022-08-11 17:17:26.317 19 INFO cinder.api.openstack.wsgi [None req-5ffc061d-f253-45fa-ad32-5d3bb3f2274a 9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - - default default] GET http://10.10.1.83:8776/v3/949326f6d40c4d49a93fffac8b958b49/volumes/detail?all_tenants=1&name=test1 2022-08-11 17:17:26.341 19 INFO cinder.volume.api [None req-5ffc061d-f253-45fa-ad32-5d3bb3f2274a 9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - - default default] Get all volumes completed successfully. 2022-08-11 17:17:26.354 19 INFO cinder.api.openstack.wsgi [None req-5ffc061d-f253-45fa-ad32-5d3bb3f2274a 9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - - default default] http://10.10.1.83:8776/v3/949326f6d40c4d49a93fffac8b958b49/volumes/detail?all_tenants=1&name=test1 returned with HTTP 200 2022-08-11 17:17:28.081 21 INFO cinder.api.openstack.wsgi [None req-a2a433a3-50e6-4f2f-923b-21b0251338ca 9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - - default default] GET http://10.10.1.83:8776/v3/949326f6d40c4d49a93fffac8b958b49/scheduler-stats/get_pools?detail=true 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault [None req-8ffc0660-49a5-4533-a60d-b4494d039203 9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - - default default] Caught error: <class 'oslo_messaging.exceptions.MessagingTimeout'> Timed out waiting for a reply to message ID 56b3bf40bf824e1085283682367d1793: oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply to message ID 56b3bf40bf824e1085283682367d1793 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault Traceback (most recent call last): 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 441, in get 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault return self._queues[msg_id].get(block=True, timeout=timeout) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/usr/lib/python3.10/queue.py", line 179, in get 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault raise Empty 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault _queue.Empty 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault During handling of the above exception, another exception occurred: 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault Traceback (most recent call last): 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/cinder/api/middleware/fault.py", line 84, in __call__ 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault return req.get_response(self.application) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/webob/request.py", line 1313, in send 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault status, headers, app_iter = self.call_application( 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/webob/request.py", line 1278, in call_application 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault app_iter = application(self.environ, start_response) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line 143, in __call__ 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault return resp(environ, start_response) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line 129, in __call__ 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault resp = self.call_func(req, *args, **kw) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line 193, in call_func 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault return self.func(req, *args, **kwargs) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/osprofiler/web.py", line 111, in __call__ 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault return request.get_response(self.application) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/webob/request.py", line 1313, in send 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault status, headers, app_iter = self.call_application( 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/webob/request.py", line 1278, in call_application 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault app_iter = application(self.environ, start_response) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line 129, in __call__ 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault resp = self.call_func(req, *args, **kw) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line 193, in call_func 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault return self.func(req, *args, **kwargs) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/keystonemiddleware/auth_token/__init__.py", line 341, in __call__ 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault response = req.get_response(self._app) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/webob/request.py", line 1313, in send 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault status, headers, app_iter = self.call_application( 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/webob/request.py", line 1278, in call_application 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault app_iter = application(self.environ, start_response) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line 143, in __call__ 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault return resp(environ, start_response) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line 143, in __call__ 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault return resp(environ, start_response) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/routes/middleware.py", line 153, in __call__ 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault response = self.app(environ, start_response) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line 143, in __call__ 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault return resp(environ, start_response) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line 129, in __call__ 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault resp = self.call_func(req, *args, **kw) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line 193, in call_func 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault return self.func(req, *args, **kwargs) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/cinder/api/openstack/wsgi.py", line 839, in __call__ 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault return self._process_stack(request, action, action_args, 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/cinder/api/openstack/wsgi.py", line 900, in _process_stack 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault action_result = self.dispatch(meth, request, action_args) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/cinder/api/openstack/wsgi.py", line 995, in dispatch 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault return method(req=request, **action_args) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/cinder/api/contrib/scheduler_stats.py", line 60, in get_pools 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault pools = self.scheduler_api.get_pools(context, filters=filters) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/cinder/scheduler/rpcapi.py", line 199, in get_pools 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault return cctxt.call(ctxt, 'get_pools', filters=filters) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 189, in call 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault result = self.transport._send( 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault return self._driver.send(target, ctxt, message, 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault return self._send(target, ctxt, message, wait_for_reply, timeout, 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 678, in _send 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault result = self._waiter.wait(msg_id, timeout, 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 567, in wait 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault message = self.waiters.get(msg_id, timeout=timeout) 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 443, in get 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault raise oslo_messaging.MessagingTimeout( 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply to message ID 56b3bf40bf824e1085283682367d1793 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault 2022-08-11 17:17:28.362 22 INFO cinder.api.middleware.fault [None req-8ffc0660-49a5-4533-a60d-b4494d039203 9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - - default default] http://10.10.1.83:8776/v3/949326f6d40c4d49a93fffac8b958b49/scheduler-stats/get_pools?detail=true returned with HTTP HTTPStatus.INTERNAL_SERVER_ERROR
答案1
您是否已配置服务令牌? “对于 2023-05-10 之后的所有 OpenStack 版本,都需要将 Nova 配置为向 Cinder 发送服务令牌,并且 Cinder 可以接收该令牌”
防止长时间运行的作业失败:如果操作链需要很长时间,用户的令牌可能会在操作完成之前过期,从而导致用户原始请求失败。处理此问题的一种方法是在 Keystone 中设置较长的令牌生命周期,这可能就是您当前正在做的事情。但对于安全策略更倾向于较短用户令牌生命周期的安装,这可能会有问题。从 Queens 版本开始,可以使用替代解决方案。您可以配置某些服务(特别是 Nova 和 Cinder)以将“服务令牌”与用户令牌一起发送。正确配置后,身份服务将在过期的用户令牌伴随有效服务令牌时对其进行验证。因此,如果用户的令牌在各种 OpenStack 服务之间的长时间运行的操作链中的某个地方过期,操作可以继续。
有关更多详细信息和说明,请参阅Cinder 文档