Hi,
We have recently updated our hyperv nova compute driver from Mitaka to Newton. So far, the update went with no major issues until recently; we updated our centos7 image through glance and for some reason, we are no longer able to spin up VMs.
The error message we are seeing from nova on our hyperv node is the following:
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [req-de5bd8d6-13e2-46fd-91b3-c6369bd326b1 1dd5c11bf9ef453381328c70ddd20b4c 52977a62b18e44f0955a2c7d0ef1ea6e - - -] [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] Instance failed to spawn
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] Traceback (most recent call last):
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\compute\manager.py", line 2078, in _build_resources
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] yield resources
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\compute\manager.py", line 1920, in _build_and_run_instance
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] block_device_info=block_device_info)
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\nova\driver.py", line 70, in wrapper
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] six.reraise(raised_exception, exc, exc_info[2])
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\nova\driver.py", line 54, in wrapper
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] return function(*args, **kwargs)
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\nova\driver.py", line 162, in spawn
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] admin_password, network_info, block_device_info)
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\nova\vmops.py", line 82, in wrapper
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] return function(self, *args, **kwds)
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\nova\vmops.py", line 299, in spawn
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] self._create_root_device(context, instance, root_device, vm_gen)
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\nova\vmops.py", line 145, in _create_root_device
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] path = self._create_root_vhd(context, instance)
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\nova\vmops.py", line 164, in _create_root_vhd
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] base_vhd_info = self._vhdutils.get_vhd_info(base_vhd_path)
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\os_win\utils\storage\virtdisk\vhdutils.py", line 203, in get_vhd_info
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] open_access_mask=open_access_mask)
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\os_win\utils\storage\virtdisk\vhdutils.py", line 96, in _open
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] ctypes.byref(handle))
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\os_win\utils\storage\virtdisk\vhdutils.py", line 76, in _run_and_check_output
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] return self._win32_utils.run_and_check_output(*args, **kwargs)
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\os_win\utils\win32utils.py", line 48, in run_and_check_output
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] self._run_and_check_output, *args, **kwargs)
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\os_win\_utils.py", line 204, in avoid_blocking_call
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] return tpool.execute(f, *args, **kwargs)
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\eventlet\tpool.py", line 125, in execute
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] six.reraise(c, e, tb)
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\eventlet\tpool.py", line 83, in tworker
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] rv = meth(*args, **kwargs)
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] File "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\os_win\utils\win32utils.py", line 94, in _run_and_check_output
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] func_name=func_name)
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681] NovaException: Executing Win32 API function OpenVirtualDisk failed. Error code: 5. Error message: Access is denied.
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681]
2017-01-25 16:26:30.236 4016 ERROR nova.compute.manager [instance: b1ab1e16-bc01-409c-9b7f-c9935f29e681]
Looks like to be a problem with permissions being configured on the VHD file.. Any ideas what might be wrong ?
Thanks! -Luc
