New Question

awestin1's profile - activity

2018-01-30 04:24:31 +0200 edited answer Gnocchi - Keystone Unable to Validate Token

It turns out that there were actually a few issues. 1) Apparently, Gnocchi doesn't like you having admin creds loaded f

2018-01-30 04:24:31 +0200 received badge  Associate Editor (source)
2018-01-30 04:21:52 +0200 marked best answer Gnocchi - Keystone Unable to Validate Token

I am trying to set up Ceilometer with Gnocchi. Gnocchi has been installed and configured. However, after starting the Gnocchi api and metricd, I keep getting an error when attempting to test Gnicchi. When I run gnocchi metric list, I get the following error:

[~]# gnocchi metric list
{"message": "The server is currently unavailable. Please try again at a later time.<br /><br />\n\n\n", "code": "503 Service Unavailable", "title": "Service Unavailable"} (HTTP 503)

At the same time, I get the following error from /etc/var/log/gnocchi/gnocchi.log:

2017-10-27 00:32:27.241 3231 WARNING keystonemiddleware.auth_token [-] Identity response: {"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}
2017-10-27 00:32:27.242 3231 CRITICAL keystonemiddleware.auth_token [-] Unable to validate token: Identity server rejected authorization necessary to fetch token data

I also get the following from stderr on gnocchi-api:

[~]# gnocchi-api -p 8041 -b 0.0.0.0
Option "verbose" from group "DEFAULT" is deprecated for removal.  Its value may be silently ignored in the future.
********************************************************************************
STARTING test server gnocchi.rest.app.build_wsgi_app
Available at http://gcp-openstack:8041/
DANGER! For testing only, do not use in production
********************************************************************************
10.9.47.55 - - [26/Oct/2017 23:32:21] "GET /v1/metric HTTP/1.1" 503 170

The [keystone_authtoken] section in my /etc/gnocchi/gnocchi.conf is as follows:

[keystone_authtoken]
auth_type = password
auth_url = http://controller:5000/v3
project_domain_name = Default
user_domain_name = Default
project_name = service
username = gnocchi
password = P@ssw0rd
interface = internalURL
region_name = RegionOne

Does anyone have any suggestions for troubleshooting this issue? I have already sourced admin-openrc before running the Gnocchi client. So, I am not sure why I am receiving this message. My admin-openrc is as follows:

[~]# cat admin-openrc 
export OS_PROJECT_DOMAIN_NAME=Default
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=admin_P@ssw0rd
export OS_AUTH_URL=http://controller:35357/v3
export OS_IDENTITY_API_VERSION=3
export OS_IMAGE_API_VERSION=2
export OS_AUTH_TYPE=password

Are there any other environment variables I am supposed to be setting?

Edit: I am running Ocata RDO on RHEL 7.

2018-01-30 04:20:45 +0200 answered a question Gnocchi - Keystone Unable to Validate Token

It turns out that there were actually a few issues. 1) Apparently, Gnocchi doesn't like you having admin creds loaded f

2018-01-30 03:53:28 +0200 received badge  Famous Question (source)
2018-01-30 03:53:28 +0200 received badge  Notable Question (source)
2017-11-21 12:46:58 +0200 received badge  Popular Question (source)
2017-10-27 13:53:55 +0200 marked best answer HyperV-Agent Error: PortBindingFailed

I am attempting to configure OpenStack to launch instances on a specific VLAN on one of the interfaces on my compute nodes. The VMswitch name is Back-end zone. I want to launch VMs on VLAN 150 on this VMswitch.

I have installed the latest Cloudbase Nova driver for Ocata and am using the default Hyper-V neutron agent.

Do I need to do anything with the segment_id directive for the subnet?

I have tried setting the router:external directive for the network to 'internal' but this doesn't seem to help.

All relevant information I could think of has been included below. Please let me know if there is any additional info you would like to see. Any help would be greatly appreciated.

Here is my network agent list on the controller:

# openstack network agent list
+--------------------------------------+----------------+---------------+-------------------+-------+-------+------------------------+
| ID                                   | Agent Type     | Host          | Availability Zone | Alive | State | Binary                 |
+--------------------------------------+----------------+---------------+-------------------+-------+-------+------------------------+
| 1f512938-132e-47d3-b1a5-eb0be5aed61c | DHCP agent     | gcp-openstack | nova              | True  | UP    | neutron-dhcp-agent     |
| 46d88273-4245-4206-b636-6944addb1c1f | Metadata agent | gcp-openstack | None              | True  | UP    | neutron-metadata-agent |
| 4d1ca29e-ece8-4380-bde7-17d167d6a103 | L3 agent       | gcp-openstack | nova              | True  | UP    | neutron-l3-agent       |
| da6107b9-c6ac-41cd-8063-28ce0d702c24 | HyperV agent   | T-Cloud-1     | None              | True  | UP    | neutron-hyperv-agent   |
+--------------------------------------+----------------+---------------+-------------------+-------+-------+------------------------+

Here are my VMswitches:

PS C:\Windows\system32> get-vmswitch

Name          SwitchType NetAdapterInterfaceDescription
----          ---------- ------------------------------
Back-end zone External   Broadcom BCM5709C NetXtreme II GigE (NDIS VBD Client) #40
Proxy zone    External   Broadcom NetXtreme Gigabit Ethernet
Web zone      External   Broadcom BCM5709C NetXtreme II GigE (NDIS VBD Client) #39

I receive the following error message in Horizon when launching an instance:

2017-05-16 16:53:33.328 2112 ERROR nova.compute.manager [req-df281722-7a6a-45d8-a5e1-96cf329a669d 0797a3b035234af49fa8fc8a80680060 f0d140af365f4c65b8144f9d5a784bd8 - - -] Instance failed network setup after 1 attempt(s)
2017-05-16 16:53:33.328 2112 ERROR nova.compute.manager Traceback (most recent call last):
[...]
2017-05-16 16:53:33.328 2112 ERROR nova.compute.manager PortBindingFailed: Binding failed for port 5c6b4f72-635e-44ed-8681-01fde769998b, please check neutron logs for more information.
2017-05-16 16:53:33.375 2112 ERROR nova.compute.manager [req-df281722-7a6a-45d8-a5e1-96cf329a669d 0797a3b035234af49fa8fc8a80680060 f0d140af365f4c65b8144f9d5a784bd8 - - -] [instance: 109a7120-b447-4313-8050-20a7099118d4] Instance failed to spawn
2017-05-16 16:53:33.375 2112 ERROR nova.compute.manager [instance: 109a7120-b447-4313-8050-20a7099118d4] Traceback (most recent call last):
[...]
2017-05-16 16:53:33.375 2112 ERROR nova.compute.manager [instance: 109a7120-b447-4313-8050-20a7099118d4] PortBindingFailed: Binding failed for port 5c6b4f72-635e-44ed-8681-01fde769998b, please check neutron logs for more information.

I don't see any new logs in the neutron-hyperv-agent.log file on the compute node.

I see the following errors in /var/log/neutron/server.log on the controller:

2017-05-16 16:53:29.701 7634 WARNING neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api [req-0825988b-430f-41e6-83f3-422bb61927ff - - - - -] Unable to schedule network c8321bc0-f38c-4943-988b-f298313ae1be: no agents available; will retry on subsequent port and subnet creation events.
2017-05-16 16:53:29.890 7634 INFO neutron.wsgi [req-0825988b-430f-41e6-83f3-422bb61927ff - - - - -] 10.9.47.71 - - [16/May/2017 16:53:29] "POST /v2.0/ports.json HTTP/1.1" 201 1087 1.430238
2017-05-16 16:53:30.174 7634 INFO neutron.wsgi [req-b6a0a893-439c-4bf1-a548-7f924ed19755 - - - - -] 10.9.47.71 - - [16/May/2017 16:53:30] "GET /v2.0/extensions.json HTTP/1.1" 200 7503 0.278365
2017-05-16 16:53:31.892 7634 ERROR neutron.plugins.ml2.managers [req-69aaa9cb-d10b-49fd-8812-5a973b188d11 - - - - -] Failed to bind port 5c6b4f72-635e-44ed-8681-01fde769998b on host T-Cloud-1 for vnic_type normal using segments ...
(more)
2017-10-27 07:51:36 +0200 edited question Gnocchi - Keystone Unable to Validate Token

Gnocchi - Keystone Unable to Validate Token I am trying to set up Ceilometer with Gnocchi. Gnocchi has been installed an

2017-10-27 07:50:36 +0200 edited question Gnocchi - Keystone Unable to Validate Token

Gnocchi - Keystone Unable to Validate Token I am trying to set up Ceilometer with Gnocchi. Gnocchi has been installed an

2017-10-27 07:48:19 +0200 asked a question Gnocchi - Keystone Unable to Validate Token

Gnocchi - Keystone Unable to Validate Token I am trying to set up Ceilometer with Gnocchi. Gnocchi has been installed an

2017-10-27 07:01:47 +0200 edited answer No hosts availble error when launching Hyper-V guest

What port is your placement endpoint listen on? openstack endpoint list I know there is a typo in the OpenStack RD

2017-10-27 06:54:26 +0200 answered a question No hosts availble error when launching Hyper-V guest

What port is your placement endpoint listen on? openstack endpoint list I know there is a typo in the OpenStack RD

2017-10-24 10:36:39 +0200 received badge  Famous Question (source)
2017-10-24 10:36:39 +0200 received badge  Notable Question (source)
2017-10-24 10:36:39 +0200 received badge  Popular Question (source)
2017-08-30 22:21:53 +0200 received badge  Supporter (source)
2017-08-30 22:21:51 +0200 marked best answer No ports have port_id starting with Network Adapter

I have installed OpenStack Hyper-V Compute Ocata msi installer from Cloudbase and have successfully attached the compute node to the OpenStack controller and I am using the hypervneutronagent on the target compute node.

When I attempt to launch an instance OpenStack begins building the VM on my compute node but then the build fails with the following error in Horizon:

Error: Failed to perform requested operation on instance "testinstance-vfjdiow84392", the instance has an error status: Please try again later [Error: Binding failed for port b28593fa-646d-409d-98ca-ebde69cacb04, please check neutron logs for more information.].

Looking through the neutron logs on the controller, I am seeing the following error:

No ports have port_id starting with Network Adapter

I'm not sure I understand what this means. Any ideas?

UPDATE:

I'm also seeing the following on the hypervneutronagent log when the agent starts:

2017-05-22 16:18:13.388 2160 INFO hyperv.neutron.l2_agent [req-1c98cd35-de4a-4c89-976e-230668df5e4c - - - - -] Agent initialized successfully, now running... 
2017-05-22 16:19:57.907 2160 INFO hyperv.neutron.hyperv_neutron_agent [-] Hyper-V VM vNIC added: Network Adapter
2017-05-22 16:19:59.688 2160 INFO hyperv.neutron.hyperv_neutron_agent [req-1c98cd35-de4a-4c89-976e-230668df5e4c - - - - -] Adding port Network Adapter

I didn't notice this before as this only logs during agent start and I thought Network Adapter was just a generic term, not a specific name that the agent was looking for. I never gave the config a string "Network Adapter" as far as I know. So, I'm not sure where the agent is getting "Network Adapter" from...

When stopping the hypervneutronagent, I see the following:

2017-05-22 15:11:36.931 704 INFO hyperv.neutron.hyperv_neutron_agent [-] Hyper-V VM vNIC removed: Network Adapter
2017-05-22 15:11:36.931 704 INFO hyperv.neutron.hyperv_neutron_agent [-] Hyper-V VM vNIC removed: None
2017-05-22 15:11:36.931 704 INFO hyperv.neutron.hyperv_neutron_agent [-] Hyper-V VM vNIC removed: None
2017-05-22 15:11:36.931 704 INFO hyperv.neutron.hyperv_neutron_agent [-] Hyper-V VM vNIC removed: None
2017-05-22 15:11:36.931 704 INFO hyperv.neutron.hyperv_neutron_agent [-] Hyper-V VM vNIC removed: None
2017-05-22 15:11:37.799 704 INFO neutron.agent.securitygroups_rpc [-] Remove device filter for [u'Network Adapter']
2017-05-22 15:11:37.799 704 INFO neutron.agent.securitygroups_rpc [-] Remove device filter for [None]

Anyone know where this "Network Adapter" string is comping from? I double-checked the nova.conf and neutronhypervagent.conf on the compute node and I'm seeing no string "Network Adapter." None of my Hyper-V VMswitches are named "Network Adapter" and none of my physical interfaces are names "Network Adapter"...

2017-08-30 22:21:51 +0200 received badge  Scholar (source)
2017-08-29 02:25:36 +0200 edited answer HyperV-Agent Error: PortBindingFailed

As Claudiu mentioned in the comments "Neutron Ocata doesn't fail to start if a configured mechanism_driver is not instal

2017-08-29 02:25:27 +0200 edited answer HyperV-Agent Error: PortBindingFailed

As Claudiu mentioned in the comments "Neutron Ocata doesn't fail to start if a configured mechanism_driver is not instal

2017-08-29 02:24:28 +0200 answered a question HyperV-Agent Error: PortBindingFailed

As Claudiu mentioned in the comments "Neutron Ocata doesn't fail to start if a configured mechanism_driver is not instal

2017-08-29 02:10:04 +0200 commented answer HyperV-Agent Error: PortBindingFailed

@Claudiu: That did it! Had to "pip install networking-hyperv==4.0.0"

2017-08-29 01:16:10 +0200 commented answer HyperV-Agent Error: PortBindingFailed

@Claudiu: Thanks! I'm not seeing networking-hyperv in my list of installed Python packages! I'll go ahead and install an

2017-08-29 00:13:16 +0200 commented answer HyperV-Agent Error: PortBindingFailed

Updated the post with fresh logs that might be more helpful.

2017-08-29 00:12:41 +0200 commented answer HyperV-Agent Error: PortBindingFailed

False alarm. Disabled all mech_agent types except hyperv in my /etc/neutron/plugin.ini. This stopped some of the DEBUG l

2017-08-29 00:10:12 +0200 edited question HyperV-Agent Error: PortBindingFailed

HyperV-Agent Error: PortBindingFailed I am attempting to configure OpenStack to launch instances on a specific VLAN on o

2017-08-28 23:41:39 +0200 commented answer HyperV-Agent Error: PortBindingFailed

Looks as though, for some reason, the neutron server is only trying to use Linux Bridge and OpenVSwitch agents on the ta

2017-08-28 23:00:05 +0200 edited question HyperV-Agent Error: PortBindingFailed

HyperV-Agent Error: PortBindingFailed I am attempting to configure OpenStack to launch instances on a specific VLAN on o

2017-08-28 22:56:39 +0200 edited question HyperV-Agent Error: PortBindingFailed

HyperV-Agent Error: PortBindingFailed I am attempting to configure OpenStack to launch instances on a specific VLAN on o

2017-08-28 22:52:24 +0200 commented answer HyperV-Agent Error: PortBindingFailed

I am seeing the following warning in the neutron-hyperv-agent log on the node when starting the neutron agent svc 2017-0

2017-08-28 22:25:47 +0200 commented answer HyperV-Agent Error: PortBindingFailed

Also, yes. I have confirmed that the Hyper-V ML2 mechanism driver is enabled in my config (/etc/neutron/plugin.ini): [m

2017-08-28 22:25:15 +0200 commented answer HyperV-Agent Error: PortBindingFailed

Also, yes. I have confirmed that the Hyper-V ML2 mechanism driver is enabled in my config (/etc/neutron/plugin.ini): [m

2017-08-28 22:24:43 +0200 commented answer HyperV-Agent Error: PortBindingFailed

Also, yes. I have confirmed that the Hyper-V ML2 mechanism driver is enabled in my config (/etc/neutron/plugin.ini): [m

2017-08-28 22:24:34 +0200 commented answer HyperV-Agent Error: PortBindingFailed

Also, yes. I have confirmed that the Hyper-V ML2 mechanism driver is enabled in my config (/etc/neutron/plugin.ini): [ml

2017-08-28 22:18:23 +0200 commented answer HyperV-Agent Error: PortBindingFailed

Turned off DHCP for the subnet. This does not appear to have helped: openstack subnet set --no-dhcp subnetnamehere

2017-08-28 20:59:00 +0200 commented answer HyperV-Agent Error: PortBindingFailed

Thanks for the input! I ran "neutron dhcp-agent-list-hosting-net [networkidhere]" and confirmed "adminstateup" is True a

2017-08-28 20:58:25 +0200 commented answer HyperV-Agent Error: PortBindingFailed

I am pretty sure the instance being created will not have network access to the DHCP agent. Will this keep the instance

2017-08-28 20:56:10 +0200 commented answer HyperV-Agent Error: PortBindingFailed

Thanks for the input! I ran "neutron dhcp-agent-list-hosting-net [networkidhere]" and confirmed "adminstateup" is True a

2017-08-28 20:54:52 +0200 commented answer HyperV-Agent Error: PortBindingFailed

Thanks for the input! I ran neutron dhcp-agent-list-hosting-net [network_id_here] and confirmed admin_state_up is True a

2017-07-29 19:23:40 +0200 received badge  Famous Question (source)
2017-07-29 19:23:40 +0200 received badge  Notable Question (source)
2017-07-21 00:59:34 +0200 asked a question Error - Interface provider for physical network provider does not exist. Agent terminated!

I keep seeing the same logs over and over again in my /var/log/neutron/linuxbridge-agent.log. Does anyone know why or have any suggestions where I should look to dig into this further? I am not using OVS on my nodes. I'm just using the neutron hyper-v agent. Do I have something enabled somewhere in my config that should be disabled?

2017-07-20 17:16:52.349 61252 INFO neutron.common.config [-] Logging enabled!
2017-07-20 17:16:52.349 61252 INFO neutron.common.config [-] /usr/bin/neutron-linuxbridge-agent version 10.0.0
2017-07-20 17:16:52.349 61252 INFO neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] Interface mappings: {'provider': 'provider'}
2017-07-20 17:16:52.350 61252 INFO neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] Bridge mappings: {}
2017-07-20 17:16:52.355 61252 ERROR neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] Interface provider for physical network provider does not exist. Agent terminated!

2017-07-20 17:16:55.570 61276 INFO neutron.common.config [-] Logging enabled!
2017-07-20 17:16:55.570 61276 INFO neutron.common.config [-] /usr/bin/neutron-linuxbridge-agent version 10.0.0
2017-07-20 17:16:55.571 61276 INFO neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] Interface mappings: {'provider': 'provider'}
2017-07-20 17:16:55.571 61276 INFO neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] Bridge mappings: {}
2017-07-20 17:16:55.577 61276 ERROR neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] Interface provider for physical network provider does not exist. Agent terminated!
2017-06-09 11:40:17 +0200 marked best answer Can't Start Neutron OVS Agent/Service After Install (OVS 2.6.1)

I followed the instructions from the Cloudbase blog, for installing Open vSwitch and configuring the neutron agent but the neutron-ovs-agent Windows service will not start. Each time I try to start the service, it crashes.

I have installed OpenStack Hyper-V Compute Ocata msi installer from Cloudbase and have successfully attached the compute node to the OpenStack controller.

I have installed the Cloudbase Open vSwitch for Windows 2.6.1 msi installer and, per the blog post linked above, created the virtual switch and bridges using ovs-vsctl. I manually created the neutron-ovs-agent Windows service per the instructions but the service crashes each time and will not start.

I find the following error in the Windows Event Logs:

The description for Event ID 0 from source neutron-hyperv-agent cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event: 

neutron-hyperv-agent
Child process ended

The instructions indicate that running the service is supposed to create additional bridges. None of these are created. When checking my list of bridges, I only see the first one that I had created.

The config file is set to log to C:\OpenStack\Log\neutron-ovs-agent.log. When I check the log directory, the neutron-ovs-agent.log file does not exist. This leads me to believe that the neutron agent crashes before it even begins logging...

Just out of curiosity, I tried running the OpenStackServiceNeutron.exe command in my cmd prompt (rather than from the Windows service) to see what happens. The exits with the following error:

C:\Windows\system32>"C:\Program Files\Cloudbase Solutions\OpenStack\Nova\bin\Ope
nStackServiceNeutron.exe" neutron-hyperv-agent "C:\Program Files\Cloudbase Solut
ions\OpenStack\Nova\Python27\Scripts\neutron-openvswitch-agent.exe" --config-fil
e "C:\Program Files\Cloudbase Solutions\OpenStack\Nova\etc\neutron_ovs_agent.con
f"
Service failed to run w/err 0x00000427

I have been researching for a couple of days now with no solution.

I noticed that the Cloudbase blog post is for Open vSwitch 2.5 but I'm not seeing and subsequent blog posts or instructions for configuring 2.6. So, I'm not sure if setup is supposed to be different for 2.6.

I noticed the following question here but none of the suggestions here seem to work for me . The poster mentions uninstalling the old versions of nova and Open vSwitch but I am already using the latest version.

Aserdean suggests adding the following to the config file but this doesn't change anything for me. The service still crashes.

[ovs]
...
of_interface = ovs-ofctl 
ovsdb_interface = vsctl

Any thoughts would be greatly appreciated!

Here is my neutronovsagent.conf file but I doubt my conf file is the issue considering the agent doesn't even start logging before it crashes. Given the Windows Event Log (provided above ... (more)

2017-06-09 11:40:07 +0200 received badge  Famous Question (source)
2017-06-05 23:06:25 +0200 commented question PortBindingFailed: Binding failed for port