Nova reports false free disk space values on Hyper-V

asked 2016-02-13 15:54:50 +0300

tomcsanyid gravatar image

Hi All,

We recently faced a weird issue: ever since we upgraded to Liberty we were not able to spin up any new VMs because of a strange "No valid host was found" issue. We debugged the issue and we came to the following conclusion: Ever since we have used OpenStack we had absolutely false numbers on graphs in Horizon, showing for example 55 TB used out of 130 GB. We didn't pay attention to this because it was a known bug in Kilo. The bug is causing Nova to compare all the storage used by VMs to the local storage of the hosting hypervisor, meaning that even if the machine is stored on an iSCSI LUN, or on an SMB share the amount of space defined in its flavor will be deducted from the local disk space of the hypervisor. This caused no error in Kilo, because cinder-scheduler ignored these values. However in Liberty the bug was fixed and therefore the developers of OpenStack decided to add the DiskFilter to the enabled DefaultFilters of Cinder scheduler immediately causing the behavior I described above. We solved this problem via creating a workaround: in cinder.conf it is possible to define a new set of filters, so we simply omitted the DiskFilter from that list and everything started to work as it did before.

We would like to however fix this in a better way, and that would require CloudBase to pull the fix from the official OpenStack repository and apply it to the HyperV driver as well (it was applied to the KVM-libvirt driver as far as I remember).

I'd have reported this on Github as an issue of the nova fork you are maintaining but I think issues are disabled for that repository.

Thanks, Domi

edit retag flag offensive close merge delete


Can you please add a link to the git commit?

alexpilotti gravatar imagealexpilotti ( 2016-02-14 14:42:50 +0300 )edit

Hm that patch never merged, but the bug it refers to has a separate fix:

alexpilotti gravatar imagealexpilotti ( 2016-02-14 18:21:52 +0300 )edit

And here is the actual patch:

alexpilotti gravatar imagealexpilotti ( 2016-02-14 18:31:51 +0300 )edit

This fix is specific to the LVM cinder driver. What cinder driver(s) are you using?

alexpilotti gravatar imagealexpilotti ( 2016-02-14 18:34:45 +0300 )edit

We are using SMB and iSCSI, experienced the issue with both backends in Kilo. In Liberty I think we only tried with SMB and the bug was there. Thanks

tomcsanyid gravatar imagetomcsanyid ( 2016-02-14 19:10:31 +0300 )edit