openstack launch instance hangs on 'Getting metadata...'

asked 2014-12-19 13:52:45 +0300

spiffly
1 ●3 ●4 ●4

updated 2015-01-20 14:02:37 +0300

Hi,

I've prepped a windows install image locally on KVM with the last step being install of cloudbase-init. After importing this image into glance and then launching the instance, cloudbase-init.exe shell hangs on the following message:

2014-12-19 09:36:41.155 1516 DEBUG cloudbaseinit.metadata.services.httpservice [-] Getting metadata from: http://169.254.169.254/openstack/latest/meta_data.json _get_data c:\program files (x86)\Cloudbase Solutions\Cloudbase-Init\Python27\lib\site-packages\cloudbaseinit\metadata\services\httpservice.py:72

(please pardon any minor typos above - typed by hand)

It seems fetching metadata is somehow failing. I can close this window to continue, but not surprisingly, hostname renaming, password etc are not setup.

It's probably worth noting that I have no problems with the metadata service in my openstack environment when launching linux instances: they pickup the SSH key injection and hostname just fine, so I'm assuming it's a configuration problem on the windows instance side.

I'm assuming that this may be a setting in the cloudbase-init.conf file that I need to get right, but I can't find any documentation on how.

The github readme explains what plugins are available, but again, I don't see any documentation anywhere explaining how to use them.

any help is appreciated!

edit: Here are the logs:

http://paste.openstack.org/show/159132/

Note that the 'Getting metadata' line is the last entry in the logs since it hung there indefinitely, so I closed the shell window

edit2: I've also configured DHCP in Neutron to tell clients to use an MTU of 1438. Basically following instructions found here:

https://ask.openstack.org/en/question...

This solved the terminal freeze problems I was seeing on Linux guests, but it looks like the Windows 7 guest ignores the DHCP directive for MTU.

edit retag flag offensive close merge delete

add a comment

4 answers

Sort by » oldest newest most voted

answered 2014-12-22 02:43:24 +0300

alexpilotti
726 ●1 ●3 ●6

Hi, a possible networking issue might be related to the MTU, especially when used in conjunction with GRE tunnels. Cloudbase-Init picks the MTU value from the DHCP server, if present. Can you please post your full Cloudbase-Init logs?

What MTU value are you using? If you are using the default (1500), please try to lower it to 1492 first and 1450 if it still does not work.

edit flag offensive delete link

Comments

Thanks for your reply. I am using GRE tunnels and had previously hit the problem you mention. My MTU is currently set to 1438. Linux instances are properly picking up the size from DHCP, but on windows instances, netsh shows the NIC still has an MTU of 1500.

spiffly ( 2015-01-20 12:42:38 +0300 )edit

I manually changed the MTU on the guest to 1438 and now the browser is hitting websites just fine. Any reason why the guest isn't properly adjusting MTU per the DHCP directive? I could set the MTU in image before running sysprep and uploading to glance, but I don't know if sysprep will remove it.

spiffly ( 2015-01-20 14:11:33 +0300 )edit

add a comment

answered 2015-01-25 14:21:02 +0300

schegi
1 ●1 ●1

updated 2015-01-25 14:21:39 +0300

Ok, I builded a windows server 2012 R2 image following the official openstack documentation http://docs.openstack.org/image-guide/content/windows-image.html and before 7.) running cloudbasinit.msi, I set the mtu of interface Ethernet to 1458 by 'interface ipv4 set subinterface "Ethernet" mtu=1458 store=persistent'. With the so build image everything works fine. First boot succeeds, Webaccess from an instanced created from this image works fine.

edit flag offensive delete link

add a comment

answered 2015-01-21 20:05:38 +0300

schegi
1 ●1 ●1

updated 2015-01-24 14:09:22 +0300

I had a similar problem with a Windows server 2012 R2 vm started on openstack.The maschine stuck in "Getting Ready" during the first boot with the log showing

DEBUG cloudbaseinit.metadata.services.httpservice [-] Getting data from: http://169.254.169.254/openstack/late...data.json (http://169.254.169.254/openstack/late...) _getdata C:\Program Files (x86)\Cloudbase Solutions\Cloudbase-Init\Python27\lib\site-packages\cloudbaseinit\metadata\services\httpservice.py:68

The only workaround i found was to not assign a key-pair. The i did this the machine booted just fine the first time.

edit flag offensive delete link

Comments

Also played around with the mtu. when setting it to 1458 in the windows server with netsh interface ipv4 set subinterface "Ethernet" mtu=1458 store=persistent and reboot. The bootprocess went through and the key is submittet to the Server. I will try now to build an image with preset mtu and report back if it succeeds.

schegi ( 2015-01-24 14:02:40 +0300 )edit

add a comment

answered 2017-04-10 15:36:57 +0300

jacolex
1 ●2

updated 2017-04-10 15:38:36 +0300

Hello I have similar issue with Windows Server 2016. Openstack Liberty. When ssh public key is added when creating instance, the Cloudbase-Init hangs on:

2017-04-10 14:12:42.904 2688 DEBUG cloudbaseinit.utils.network [-] Testing url: http://169.254.169.254/ checkurl c:\program files\cloudbase solutions\cloudbase-init\python\lib\site-packages\cloudbaseinit\utils\network.py:35 2017-04-10 14:13:03.920 2688 DEBUG cloudbaseinit.utils.network [-] Testing url: http://169.254.169.254/ checkurl c:\program files\cloudbase solutions\cloudbase-init\python\lib\site-packages\cloudbaseinit\utils\network.py:35 2017-04-10 14:13:24.951 2688 DEBUG cloudbaseinit.utils.network [-] Testing url: http://169.254.169.254/ checkurl c:\program files\cloudbase solutions\cloudbase-init\python\lib\site-packages\cloudbaseinit\utils\network.py:35 2017-04-10 14:13:42.623 2688 DEBUG cloudbaseinit.utils.network [-] Setting gateway for host: 169.254.169.254 checkmetadataiproute c:\program files\cloudbase solutions\cloudbase-init\python\lib\site-packages\cloudbaseinit\utils\network.py:60 2017-04-10 14:13:42.732 2688 ERROR cloudbaseinit.utils.network [-] Unable to add route: b'The route addition failed: The object already exists.\r\n\r\n' 2017-04-10 14:13:42.732 2688 ERROR cloudbaseinit.utils.network Traceback (most recent call last): 2017-04-10 14:13:42.732 2688 ERROR cloudbaseinit.utils.network File "c:\program files\cloudbase solutions\cloudbase-init\python\lib\site-packages\cloudbaseinit\utils\network.py", line 65, in checkmetadataiproute 2017-04-10 14:13:42.732 2688 ERROR cloudbaseinit.utils.network 10) 2017-04-10 14:13:42.732 2688 ERROR cloudbaseinit.utils.network File "c:\program files\cloudbase solutions\cloudbase-init\python\lib\site-packages\cloudbaseinit\osutils\windows.py", line 1146, in addstaticroute 2017-04-10 14:13:42.732 2688 ERROR cloudbaseinit.utils.network 'Unable to add route: %s' % err) 2017-04-10 14:13:42.732 2688 ERROR cloudbaseinit.utils.network cloudbaseinit.exception.CloudbaseInitException: Unable to add route: b'The route addition failed: The object already exists.\r\n\r\n' 2017-04-10 14:13:42.732 2688 ERROR cloudbaseinit.utils.network 2017-04-10 14:13:42.764 2688 DEBUG cloudbaseinit.metadata.services.base [-] Getting metadata from: http://169.254.169.254/openstack/late...data.json httprequest c:\program files\cloudbase solutions\cloudbase-init\python\lib\site-packages\cloudbaseinit\metadata\services\base.py:264

Additionally I have error : "Unable to add route". Without ssh key it gives me the same error, but cloudbase scripts are not hanging after "getting metadata" and it continues running.

Second strange thing is setting MTU. I have no information such like Setting MTU for interface "FA:16:3E:B1:9D:32" with value "xx". But I have also Windows 2012 R2 image, there is no such information about setting MTU. but it's working ok.

I have also another Openstack deployment (Kilo) and there everything is running correctly (the same Windows 2016 image).

I don't know, where is the problem - MTU or maybe something else?

edit flag offensive delete link

Comments

The critical setting is to disable Internet Explorer Enhanced Security during creating the new image. After that everything works, after spending one week of resolving the issue.

jacolex ( 2017-04-10 17:07:55 +0300 )edit

Thanks for sharing this, much appreciated.

avladu ( 2017-04-10 20:08:02 +0300 )edit

add a comment

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

openstack launch instance hangs on 'Getting metadata...'

4 answers

Comments

Comments

Comments

Your Answer

Question Tools

Stats

Related questions

openstack launch instance hangs on 'Getting metadata...' edit

4 answers

Comments

Comments

Comments

Your Answer

Question Tools

Stats

Related questions

openstack launch instance hangs on 'Getting metadata...'