openstack yoga 卷创建失败

openstack yoga 卷创建失败

我们正在尝试创建 Cinder 卷。Cinder API 接受请求,但卷卡在creating状态。无论使用哪个后端(即也使用lvm),都会发生这种情况。似乎 Cinder API 和其余进程之间发生了一些超时。如何解决这个问题?非常感谢任何线索。

以下是 cinder-api 日志的摘录:

req-00001bfc-808d-4525-904d-207e40ac1e2d
9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - -
default default] POST
http://10.10.1.83:8776/v3/949326f6d40c4d49a93fffac8b958b49/volumes
2022-08-11 17:17:01.415 21 INFO cinder.api.v3.volumes [None
req-00001bfc-808d-4525-904d-207e40ac1e2d
9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - -
default default] Create volume of 4 GB 2022-08-11 17:17:01.435 21 INFO
cinder.volume.api [None req-00001bfc-808d-4525-904d-207e40ac1e2d
9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - -
default default] Availability Zones retrieved successfully. 2022-08-11
17:17:01.619 21 INFO cinder.volume.api [None
req-00001bfc-808d-4525-904d-207e40ac1e2d
9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - -
default default] Create volume request issued successfully. 2022-08-11
17:17:01.620 21 INFO cinder.api.openstack.wsgi [None
req-00001bfc-808d-4525-904d-207e40ac1e2d
9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - -
default default]
http://10.10.1.83:8776/v3/949326f6d40c4d49a93fffac8b958b49/volumes
returned with HTTP 202 2022-08-11 17:17:11.119 23 INFO
cinder.api.openstack.wsgi [None
req-a50d2c2f-784b-481e-a34a-48f34225eed2 - - - - - -] GET
http://10.10.1.80:8776/ 2022-08-11 17:17:11.120 23 INFO
cinder.api.openstack.wsgi [None
req-a50d2c2f-784b-481e-a34a-48f34225eed2 - - - - - -]
http://10.10.1.80:8776/ returned with HTTP 300 2022-08-11 17:17:26.317
19 INFO cinder.api.openstack.wsgi [None
req-5ffc061d-f253-45fa-ad32-5d3bb3f2274a
9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - -
default default] GET
http://10.10.1.83:8776/v3/949326f6d40c4d49a93fffac8b958b49/volumes/detail?all_tenants=1&name=test1
2022-08-11 17:17:26.341 19 INFO cinder.volume.api [None
req-5ffc061d-f253-45fa-ad32-5d3bb3f2274a
9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - -
default default] Get all volumes completed successfully. 2022-08-11
17:17:26.354 19 INFO cinder.api.openstack.wsgi [None
req-5ffc061d-f253-45fa-ad32-5d3bb3f2274a
9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - -
default default]
http://10.10.1.83:8776/v3/949326f6d40c4d49a93fffac8b958b49/volumes/detail?all_tenants=1&name=test1
returned with HTTP 200 2022-08-11 17:17:28.081 21 INFO
cinder.api.openstack.wsgi [None
req-a2a433a3-50e6-4f2f-923b-21b0251338ca
9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - -
default default] GET
http://10.10.1.83:8776/v3/949326f6d40c4d49a93fffac8b958b49/scheduler-stats/get_pools?detail=true
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault [None
req-8ffc0660-49a5-4533-a60d-b4494d039203
9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - -
default default] Caught error: <class
'oslo_messaging.exceptions.MessagingTimeout'> Timed out waiting for a
reply to message ID 56b3bf40bf824e1085283682367d1793:
oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a
reply to message ID 56b3bf40bf824e1085283682367d1793 2022-08-11
17:17:28.359 22 ERROR cinder.api.middleware.fault Traceback (most
recent call last): 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 441, in get 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  return
self._queues[msg_id].get(block=True, timeout=timeout) 2022-08-11
17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/usr/lib/python3.10/queue.py", line 179, in get 2022-08-11
17:17:28.359 22 ERROR cinder.api.middleware.fault  raise Empty
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault
_queue.Empty 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault During handling of the above exception,
another exception occurred: 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault Traceback (most recent call last):
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/cinder/api/middleware/fault.py",
line 84, in __call__ 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  return req.get_response(self.application)
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/webob/request.py",
line 1313, in send 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  status, headers, app_iter =
self.call_application( 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/webob/request.py",
line 1278, in call_application 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  app_iter = application(self.environ,
start_response) 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line
143, in __call__ 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  return resp(environ, start_response)
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line
129, in __call__ 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  resp = self.call_func(req, *args, **kw)
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line
193, in call_func 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  return self.func(req, *args, **kwargs)
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/osprofiler/web.py",
line 111, in __call__ 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  return
request.get_response(self.application) 2022-08-11 17:17:28.359 22
ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/webob/request.py",
line 1313, in send 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  status, headers, app_iter =
self.call_application( 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/webob/request.py",
line 1278, in call_application 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  app_iter = application(self.environ,
start_response) 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line
129, in __call__ 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  resp = self.call_func(req, *args, **kw)
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line
193, in call_func 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  return self.func(req, *args, **kwargs)
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/keystonemiddleware/auth_token/__init__.py",
line 341, in __call__ 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  response = req.get_response(self._app)
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/webob/request.py",
line 1313, in send 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  status, headers, app_iter =
self.call_application( 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/webob/request.py",
line 1278, in call_application 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  app_iter = application(self.environ,
start_response) 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line
143, in __call__ 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  return resp(environ, start_response)
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line
143, in __call__ 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  return resp(environ, start_response)
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/routes/middleware.py",
line 153, in __call__ 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  response = self.app(environ,
start_response) 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line
143, in __call__ 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  return resp(environ, start_response)
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line
129, in __call__ 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  resp = self.call_func(req, *args, **kw)
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/webob/dec.py", line
193, in call_func 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  return self.func(req, *args, **kwargs)
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/cinder/api/openstack/wsgi.py",
line 839, in __call__ 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  return self._process_stack(request,
action, action_args, 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/cinder/api/openstack/wsgi.py",
line 900, in _process_stack 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  action_result = self.dispatch(meth,
request, action_args) 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/cinder/api/openstack/wsgi.py",
line 995, in dispatch 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  return method(req=request, **action_args)
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/cinder/api/contrib/scheduler_stats.py",
line 60, in get_pools 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  pools =
self.scheduler_api.get_pools(context, filters=filters) 2022-08-11
17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/cinder/scheduler/rpcapi.py",
line 199, in get_pools 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  return cctxt.call(ctxt, 'get_pools',
filters=filters) 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py",
line 189, in call 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  result = self.transport._send( 2022-08-11
17:17:28.359 22 ERROR cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/transport.py",
line 123, in _send 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  return self._driver.send(target, ctxt,
message, 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault 
File
"/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 689, in send 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  return self._send(target, ctxt, message,
wait_for_reply, timeout, 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 678, in _send 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  result = self._waiter.wait(msg_id,
timeout, 2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault 
File
"/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 567, in wait 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  message = self.waiters.get(msg_id,
timeout=timeout) 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault   File
"/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 443, in get 2022-08-11 17:17:28.359 22 ERROR
cinder.api.middleware.fault  raise oslo_messaging.MessagingTimeout(
2022-08-11 17:17:28.359 22 ERROR cinder.api.middleware.fault
oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a
reply to message ID 56b3bf40bf824e1085283682367d1793 2022-08-11
17:17:28.359 22 ERROR cinder.api.middleware.fault 2022-08-11
17:17:28.362 22 INFO cinder.api.middleware.fault [None
req-8ffc0660-49a5-4533-a60d-b4494d039203
9c8f8ded6e5947278acc0e5f10d0c7bf 949326f6d40c4d49a93fffac8b958b49 - -
default default]
http://10.10.1.83:8776/v3/949326f6d40c4d49a93fffac8b958b49/scheduler-stats/get_pools?detail=true
returned with HTTP HTTPStatus.INTERNAL_SERVER_ERROR

答案1

您是否已配置服务令牌? “对于 2023-05-10 之后的所有 OpenStack 版本,都需要将 Nova 配置为向 Cinder 发送服务令牌,并且 Cinder 可以接收该令牌”

防止长时间运行的作业失败:如果操作链需要很长时间,用户的令牌可能会在操作完成之前过期,从而导致用户原始请求失败。处理此问题的一种方法是在 Keystone 中设置较长的令牌生命周期,这可能就是您当前正在做的事情。但对于安全策略更倾向于较短用户令牌生命周期的安装,这可能会有问题。从 Queens 版本开始,可以使用替代解决方案。您可以配置某些服务(特别是 Nova 和 Cinder)以将“服务令牌”与用户令牌一起发送。正确配置后,身份服务将在过期的用户令牌伴随有效服务令牌时对其进行验证。因此,如果用户的令牌在各种 OpenStack 服务之间的长时间运行的操作链中的某个地方过期,操作可以继续。

有关更多详细信息和说明,请参阅Cinder 文档

相关内容