New Question

Revision history [back]

click to hide/show revision 1
initial version

VXLAN port not open in Hyper-V

Hi,

I'm trying to get instances in a Hyper-V compute node to communicate with instances on other hypervisors using Open vSwitch and neutron-ova-agent over VXLAN. My other hypervisors run CentOS/KVM. Instances on the Linux machines can communicate just fine, but I don't get the ones running in Hyper-V to communicate.

The setup runs two controller nodes (10.60.11.21 and 10.60.11.22 with a shared VIP 10.60.11.201) with the L3-agent running the router on .21. In addition I have two separate hypervisors installed, one CentOS/KVM (10.60.11.23) and one Windows Server 2016 Datacenter/Hyper-V (10.60.11.25).

The Hyper-V host runs 3 VMs, one CirrOS (192.168.0.7) and two Windows Server (one with 192.168.0.12 and one trying to use DHCP).

I have created a Hyper-V VMswitch named "br-ex" and enabled the Open vSwitch extension. The VMswitch is connected to an interface named "Tenant" (without IP). I have also manually created a Open vSwitch bridge named "br-ex" and connected the "Tenant" port above to it. The hypervisor also has a Management interface with IP (i.e. 10.60.11.25) with connectivity to the other Openstack nodes.

The VMs on Hyper-V can ping themselves and other VMs on the same Hyper-V host. They can not access the router, DHCP-server on the controller node or VMs on other hypervisors.

I think the problem has something to do with that Open vSwitch does not open a listening socket for VXLAN (udp/4789), i.e. according to netstat -na | find "4789".

If I try to connect to an instance from the outside (10.20.150.115) I see the incoming TCP SYN on the Hyper-V host.

Frame 1195965: 128 bytes on wire (1024 bits), 128 bytes captured (1024 bits) on interface 0
Ethernet II, Src: HewlettP_70:57:80 (ac:16:2d:70:57:80), Dst: HewlettP_70:a9:d0 (ac:16:2d:70:a9:d0)
Internet Protocol Version 4, Src: 10.60.11.21, Dst: 10.60.11.25
User Datagram Protocol, Src Port: 44982, Dst Port: 4789
Virtual eXtensible Local Area Network
Ethernet II, Src: fa:16:3e:e2:06:b1 (fa:16:3e:e2:06:b1), Dst: fa:16:3e:9e:f7:c6 (fa:16:3e:9e:f7:c6)
Internet Protocol Version 4, Src: 10.20.150.115, Dst: 192.168.0.12
Transmission Control Protocol, Src Port: 53839, Dst Port: 22, Seq: 0, Len: 0

I.e. the NAT/routing into Hyper-V works. But the VXLAN port is not open, so Hyper-V-host responds with ICMP Port Unreachable.

Frame 1195966: 156 bytes on wire (1248 bits), 156 bytes captured (1248 bits) on interface 0
Ethernet II, Src: HewlettP_70:a9:d0 (ac:16:2d:70:a9:d0), Dst: HewlettP_70:57:80 (ac:16:2d:70:57:80)
Internet Protocol Version 4, Src: 10.60.11.25, Dst: 10.60.11.21
Internet Control Message Protocol
    Type: 3 (Destination unreachable)
    Code: 3 (Port unreachable)
    Checksum: 0xfe7a [correct]
    [Checksum Status: Good]
    Unused: 00000000
    Internet Protocol Version 4, Src: 10.60.11.21, Dst: 10.60.11.25
    User Datagram Protocol, Src Port: 44982, Dst Port: 4789
    Virtual eXtensible Local Area Network
    Ethernet II, Src: fa:16:3e:e2:06:b1 (fa:16:3e:e2:06:b1), Dst: fa:16:3e:9e:f7:c6 (fa:16:3e:9e:f7:c6)
    Internet Protocol Version 4, Src: 10.20.150.115, Dst: 192.168.0.12
    Transmission Control Protocol, Src Port: 53839, Dst Port: 22, Seq: 1062324327

Firewalls in both the hypervisor and VMs are turned off.

I use Newton release with the Windows components https://cloudbase.it/downloads/HyperVNovaComputeNewton1401.zip and https://cloudbase.it/downloads/openvswitch-hyperv-2.6.1-certified.msi.

My neutron-ovs-agent.conf

[DEFAULT]
verbose=true
debug = False

control_exchange=neutron
policy_file=C:\Program Files\Cloudbase Solutions\OpenStack\Nova\etc\policy.json
logdir=C:\OpenStack\Log\
logfile=neutron-ovs-agent.log
transport_url = rabbit://openstack:password@10.60.11.21:5672,openstack:password@10.60.11.22:5672

[agent]
tunnel_types = vxlan
enable_metrics_collection=false
vxlan_udp_port = 4789

[SECURITYGROUP]
enable_security_group=false

[ovs]
of_interface = ovs-ofctl 
ovsdb_interface = vsctl
local_ip = 10.60.11.25
tunnel_bridge = br-tun
integration_bridge = br-int
tenant_network_type = vxlan
enable_tunneling = True
vni_ranges = 1:1000
vxlan_group = 239.1.1.1
bridge_mappings = physnet1:br-ex

My nova.conf

[DEFAULT]
auth_strategy=keystone
image_service=nova.image.glance.GlanceImageService
compute_driver=compute_hyperv.driver.HyperVDriver
network_api_class=nova.network.neutronv2.api.API
volume_api_class=nova.volume.cinder.API
instances_path=C:\OpenStack\Instances
use_cow_images=true
flat_injected=true
mkisofs_cmd=C:\Program Files\Cloudbase Solutions\OpenStack\Nova\bin\mkisofs.exe
verbose=true
allow_resize_to_same_host=true
running_deleted_instance_action=reap
running_deleted_instance_poll_interval=120
resize_confirm_window=5
resume_guests_state_on_host_boot=true
rpc_response_timeout=1800
use_neutron=true
vif_plugging_is_fatal=false
vif_plugging_timeout=60
rpc_backend=rabbit
logdir=C:\OpenStack\Log\
logfile=nova-compute.log
force_config_drive=True
debug = False
transport_url = rabbit://openstack:password@10.60.11.21:5672,openstack:password@10.60.11.22:5672

[oslo_policy]
policy_file=C:\Program Files\Cloudbase Solutions\OpenStack\Nova\etc\policy.json

[oslo_concurrency]
lock_path=C:\OpenStack\Log\

[glance]
debug = False
api_servers=http://10.60.11.21:9292,http://10.60.11.22:9292

[hyperv]
vswitch_name=br-ex
limit_cpu_features=true
config_drive_inject_password=false
qemu_img_cmd=C:\Program Files\Cloudbase Solutions\OpenStack\Nova\bin\qemu-img.exe
config_drive_cdrom=true
dynamic_memory_ratio=1
enable_instance_metrics_collection=false

[rdp]
enabled=true
html5_proxy_base_url=http://10.60.11.25:8000/

[neutron]
url = http://10.60.11.201:9696
auth_strategy=keystone
project_domain_name=Default
project_name=service
user_domain_name=Default
username = neutron
password = password
auth_url = http://10.60.11.201:35357/v3
auth_type=v3password

Output from ovs-vsctl show

4ac94d05-c741-4aab-8aae-db750d325647
    Bridge br-tun
        fail_mode: secure
        Port "vxlan-0a3c0b15"
            Interface "vxlan-0a3c0b15"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="10.60.11.25", out_key=flow, remote_ip="10.60.11.21"}
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
        Port "vxlan-0a3c0b17"
            Interface "vxlan-0a3c0b17"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="10.60.11.25", out_key=flow, remote_ip="10.60.11.23"}
        Port "vxlan-0a3c0b16"
            Interface "vxlan-0a3c0b16"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="10.60.11.25", out_key=flow, remote_ip="10.60.11.22"}
    Bridge br-int
        fail_mode: secure
        Port "84a9315b-4b33-4185-9a98-55b10842569e"
            tag: 1
            Interface "84a9315b-4b33-4185-9a98-55b10842569e"
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port br-int
            Interface br-int
                type: internal
        Port "9f422822-edca-4d41-8b2b-a39b6f2c9383"
            tag: 1
            Interface "9f422822-edca-4d41-8b2b-a39b6f2c9383"
        Port "34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b"
            tag: 1
            Interface "34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b"
        Port int-br-ex
            Interface int-br-ex
                type: patch
                options: {peer=phy-br-ex}
    Bridge br-ex
        fail_mode: secure
        Port br-ex
            Interface br-ex
                type: internal
        Port Tenant
            Interface Tenant
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
                options: {peer=int-br-ex}

Output from ovs-dpctl show

system@ovs-system:
    lookups: hit:53116 missed:53200 lost:0
    flows: 2
    port 1: br-ex (internal)
    port 2: Tenant
    port 3: br-int (internal)
    port 4: br-tun (internal)
    port 5: vxlan_sys_4789 (vxlan)
    port 6: 9f422822-edca-4d41-8b2b-a39b6f2c9383
    port 8: 84a9315b-4b33-4185-9a98-55b10842569e
    port 10: 34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b

Output from ovs-ofctl dump-ports-desc br-int

OFPST_PORT_DESC reply (xid=0x2):
 1(int-br-ex): addr:4a:f3:6a:3d:e9:9b
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(patch-tun): addr:ca:b5:cf:6b:51:d5
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 3(9f422822-edca-4): addr:fa:16:3e:c7:90:50
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 12(84a9315b-4b33-4): addr:fa:16:3e:9e:f7:c6
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 16(34cecd4b-c5cc-4): addr:fa:16:3e:62:81:7a
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 LOCAL(br-int): addr:00:15:5d:0c:19:14
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max

Output from ovs-ofctl dump-ports-desc br-tun

OFPST_PORT_DESC reply (xid=0x2):
 1(patch-int): addr:32:52:d2:71:d9:d0
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(vxlan-0a3c0b15): addr:c2:27:30:e3:a4:d0
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 3(vxlan-0a3c0b16): addr:e6:a2:aa:1d:bd:85
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 4(vxlan-0a3c0b17): addr:e6:9e:c7:01:aa:fa
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 LOCAL(br-tun): addr:00:15:5d:0c:19:15
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max

Output from ovs-ofctl dump-ports-desc br-ex

OFPST_PORT_DESC reply (xid=0x2):
 1(Tenant): addr:ac:16:2d:70:a9:d1
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(phy-br-ex): addr:62:04:7b:dc:39:59
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 LOCAL(br-ex): addr:00:15:5d:0c:19:13
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max

Output from ovs-ofctl dump-flows br-int

NXST_FLOW reply (xid=0x4):
 cookie=0x953fea9b637ac6b5, duration=20369.454s, table=0, n_packets=0, n_bytes=0, idle_age=20369, priority=10,arp,in_port=16 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20368.955s, table=0, n_packets=17, n_bytes=714, idle_age=3982, priority=10,arp,in_port=3 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20368.454s, table=0, n_packets=571, n_bytes=23982, idle_age=456, priority=10,arp,in_port=12 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20369.610s, table=0, n_packets=0, n_bytes=0, idle_age=20369, priority=10,icmp6,in_port=16,icmp_type=136 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20369.079s, table=0, n_packets=0, n_bytes=0, idle_age=20369, priority=10,icmp6,in_port=3,icmp_type=136 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20368.578s, table=0, n_packets=0, n_bytes=0, idle_age=20368, priority=10,icmp6,in_port=12,icmp_type=136 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20369.736s, table=0, n_packets=600, n_bytes=179319, idle_age=58, priority=9,in_port=16 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20369.204s, table=0, n_packets=856, n_bytes=1256064, idle_age=3969, priority=9,in_port=3 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20368.704s, table=0, n_packets=920, n_bytes=1266011, idle_age=105, priority=9,in_port=12 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20375.434s, table=0, n_packets=0, n_bytes=0, idle_age=20375, priority=0 actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20375.377s, table=23, n_packets=0, n_bytes=0, idle_age=20375, priority=0 actions=drop
 cookie=0x953fea9b637ac6b5, duration=20369.533s, table=24, n_packets=0, n_bytes=0, idle_age=20369, priority=2,arp,in_port=16,arp_spa=192.168.0.13 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20369.017s, table=24, n_packets=17, n_bytes=714, idle_age=3982, priority=2,arp,in_port=3,arp_spa=192.168.0.7 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20368.517s, table=24, n_packets=571, n_bytes=23982, idle_age=456, priority=2,arp,in_port=12,arp_spa=192.168.0.12 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20369.673s, table=24, n_packets=0, n_bytes=0, idle_age=20369, priority=2,icmp6,in_port=16,icmp_type=136,nd_target=fe80::f816:3eff:fe62:817a actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20369.141s, table=24, n_packets=0, n_bytes=0, idle_age=20369, priority=2,icmp6,in_port=3,icmp_type=136,nd_target=fe80::f816:3eff:fec7:9050 actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20368.641s, table=24, n_packets=0, n_bytes=0, idle_age=20368, priority=2,icmp6,in_port=12,icmp_type=136,nd_target=fe80::f816:3eff:fe9e:f7c6 actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20375.314s, table=24, n_packets=0, n_bytes=0, idle_age=20375, priority=0 actions=drop
 cookie=0x953fea9b637ac6b5, duration=20369.861s, table=25, n_packets=600, n_bytes=179319, idle_age=58, priority=2,in_port=16,dl_src=fa:16:3e:62:81:7a actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20369.329s, table=25, n_packets=873, n_bytes=1256778, idle_age=3969, priority=2,in_port=3,dl_src=fa:16:3e:c7:90:50 actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20368.830s, table=25, n_packets=1491, n_bytes=1289993, idle_age=105, priority=2,in_port=12,dl_src=fa:16:3e:9e:f7:c6 actions=NORMAL

Output from ovs-ofctl dumpflows br-tun

NXST_FLOW reply (xid=0x4):
 cookie=0xaf522f8bd636f703, duration=20385.783s, table=0, n_packets=1231, n_bytes=213080, idle_age=70, priority=1,in_port=1 actions=resubmit(,2)
 cookie=0xaf522f8bd636f703, duration=20384.081s, table=0, n_packets=0, n_bytes=0, idle_age=20384, priority=1,in_port=2 actions=resubmit(,4)
 cookie=0xaf522f8bd636f703, duration=20383.862s, table=0, n_packets=0, n_bytes=0, idle_age=20383, priority=1,in_port=3 actions=resubmit(,4)
 cookie=0xaf522f8bd636f703, duration=20383.643s, table=0, n_packets=0, n_bytes=0, idle_age=20383, priority=1,in_port=4 actions=resubmit(,4)
 cookie=0xaf522f8bd636f703, duration=20385.782s, table=0, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=drop
 cookie=0xaf522f8bd636f703, duration=20385.779s, table=2, n_packets=0, n_bytes=0, idle_age=20385, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20)
 cookie=0xaf522f8bd636f703, duration=20385.777s, table=2, n_packets=1231, n_bytes=213080, idle_age=70, priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,22)
 cookie=0xaf522f8bd636f703, duration=20385.777s, table=3, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=drop
 cookie=0xaf522f8bd636f703, duration=20381.923s, table=4, n_packets=0, n_bytes=0, idle_age=20381, priority=1,tun_id=0x1d actions=mod_vlan_vid:1,resubmit(,10)
 cookie=0xaf522f8bd636f703, duration=20385.776s, table=4, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=drop
 cookie=0xaf522f8bd636f703, duration=20385.775s, table=6, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=drop
 cookie=0xaf522f8bd636f703, duration=20385.774s, table=10, n_packets=0, n_bytes=0, idle_age=20385, priority=1 actions=learn(table=20,hard_timeout=300,priority=1,cookie=0xaf522f8bd636f703,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1
 cookie=0xaf522f8bd636f703, duration=20385.773s, table=20, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=resubmit(,22)
 cookie=0xaf522f8bd636f703, duration=20381.987s, table=22, n_packets=1231, n_bytes=213080, idle_age=70, dl_vlan=1 actions=strip_vlan,set_tunnel:0x1d,output:2,output:4,output:3
 cookie=0xaf522f8bd636f703, duration=20385.706s, table=22, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=drop

Output from ovs-ofctl dump-flows br-ex

NXST_FLOW reply (xid=0x4):

The ovs-vswitchd.log

    2017-02-24T12:57:07.407Z|00001|vlog|INFO|opened log file c:\Program Files\Cloudbase Solutions\Open vSwitch\logs\ovs-vswitchd.log
    2017-02-24T12:57:07.422Z|00002|ovs_numa|INFO|Discovered 0 NUMA nodes and 0 CPU cores
    2017-02-24T12:57:07.423Z|00003|reconnect|INFO|unix:C:\ProgramData\openvswitch\db.sock: connecting...
    2017-02-24T12:57:07.424Z|00004|reconnect|INFO|unix:C:\ProgramData\openvswitch\db.sock: connected
    2017-02-24T12:57:07.454Z|00005|ofproto_dpif|INFO|system@ovs-system: Datapath supports recirculation
    2017-02-24T12:57:07.455Z|00006|ofproto_dpif|INFO|system@ovs-system: MPLS label stack length probed as 3
    2017-02-24T12:57:07.456Z|00007|ofproto_dpif|INFO|system@ovs-system: datapath does not support masked set action feature.
    2017-02-24T12:57:07.456Z|00008|ofproto_dpif|INFO|system@ovs-system: Datapath supports truncate action
    2017-02-24T12:57:07.456Z|00009|ofproto_dpif|INFO|system@ovs-system: Datapath does not support unique flow ids
    2017-02-24T12:57:07.456Z|00010|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_state
    2017-02-24T12:57:07.456Z|00011|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_zone
    2017-02-24T12:57:07.457Z|00012|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_mark
    2017-02-24T12:57:07.457Z|00013|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_label
    2017-02-24T12:57:07.457Z|00014|ofproto_dpif|INFO|system@ovs-system: Datapath does not support ct_state_nat
    2017-02-24T12:57:07.461Z|00015|ofproto_dpif|WARN|system@ovs-system: variable-length userdata feature probe failed (Unknown error)
    2017-02-24T12:57:07.474Z|00016|bridge|INFO|bridge br-int: added interface 34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b on port 16
    2017-02-24T12:57:07.475Z|00017|bridge|INFO|bridge br-int: added interface br-int on port 65534
    2017-02-24T12:57:07.475Z|00018|bridge|INFO|bridge br-int: added interface int-br-ex on port 1
    2017-02-24T12:57:07.475Z|00019|bridge|INFO|bridge br-int: added interface 84a9315b-4b33-4185-9a98-55b10842569e on port 12
    2017-02-24T12:57:07.475Z|00020|bridge|INFO|bridge br-int: added interface 9f422822-edca-4d41-8b2b-a39b6f2c9383 on port 3
    2017-02-24T12:57:07.476Z|00021|bridge|INFO|bridge br-int: added interface patch-tun on port 2
    2017-02-24T12:57:07.476Z|00022|bridge|INFO|bridge br-ex: added interface br-ex on port 65534
    2017-02-24T12:57:07.476Z|00023|bridge|INFO|bridge br-ex: added interface phy-br-ex on port 2
    2017-02-24T12:57:07.476Z|00024|bridge|INFO|bridge br-ex: added interface Tenant on port 1
    2017-02-24T12:57:07.478Z|00025|bridge|INFO|bridge br-tun: added interface vxlan-0a3c0b15 on port 2
    2017-02-24T12:57:07.478Z|00026|bridge|INFO|bridge br-tun: added interface patch-int on port 1
    2017-02-24T12:57:07.478Z|00027|bridge|INFO|bridge br-tun: added interface vxlan-0a3c0b16 on port 3
    2017-02-24T12:57:07.478Z|00028|bridge|INFO|bridge br-tun: added interface vxlan-0a3c0b17 on port 4
    2017-02-24T12:57:07.479Z|00029|bridge|INFO|bridge br-tun: added interface br-tun on port 65534
    2017-02-24T12:57:07.479Z|00030|bridge|INFO|bridge br-int: using datapath ID 000056ba7b1ba641
    2017-02-24T12:57:07.480Z|00031|connmgr|INFO|br-int: added service controller "punix:c:\ProgramData\openvswitch\/br-int.mgmt"
    2017-02-24T12:57:07.493Z|00032|bridge|INFO|bridge br-ex: using datapath ID 0000ac162d70a9d1
    2017-02-24T12:57:07.493Z|00033|connmgr|INFO|br-ex: added service controller "punix:c:\ProgramData\openvswitch\/br-ex.mgmt"
    2017-02-24T12:57:07.505Z|00034|bridge|INFO|bridge br-tun: using datapath ID 0000ba4702014949
    2017-02-24T12:57:07.505Z|00035|connmgr|INFO|br-tun: added service controller "punix:c:\ProgramData\openvswitch\/br-tun.mgmt"
    2017-02-24T12:57:07.524Z|00036|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.6.1.0
    2017-02-24T12:57:17.216Z|00037|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:17.277Z|00038|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:17.340Z|00039|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:17.430Z|00040|memory|INFO|10192 kB peak resident set size after 10.0 seconds
    2017-02-24T12:57:17.431Z|00041|memory|INFO|handlers:17 ports:14 revalidators:7 rules:15 udpif keys:2
    2017-02-24T12:57:17.902Z|00042|connmgr|INFO|br-ex<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:18.277Z|00043|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:18.340Z|00044|connmgr|INFO|br-ex<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:19.223Z|00045|connmgr|INFO|br-tun<->unix: 9 flow_mods in the last 0 s (9 adds)
    2017-02-24T12:57:19.293Z|00046|connmgr|INFO|br-tun<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:21.823Z|00047|connmgr|INFO|br-tun<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:22.073Z|00048|connmgr|INFO|br-tun<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:22.324Z|00049|connmgr|INFO|br-tun<->unix: 1 flow_mods in the last 0 s (1 adds)

and the neutron-ovs-agent.log

2017-02-24 13:57:16.726 8524 INFO neutron.common.config [-] Logging enabled!
2017-02-24 13:57:16.726 8524 INFO neutron.common.config [-] c:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python\Scripts\neutron-openvswitch-agent version 9.0.1.dev14
2017-02-24 13:57:17.492 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-143b7ba8-7253-4d83-8cc6-d611cff6d65d - - - - -] Mapping physical network physnet1 to bridge br-ex
2017-02-24 13:57:19.164 8524 INFO neutron.agent.agent_extensions_manager [req-143b7ba8-7253-4d83-8cc6-d611cff6d65d - - - - -] Loaded agent extensions: []
2017-02-24 13:57:20.945 8524 INFO oslo.messaging._drivers.impl_rabbit [-] [ccd7f232-990e-4bdb-a9ca-6cd292881c76] Reconnected to AMQP server on 10.60.11.22:5672 via [amqp] client with port 51875.
2017-02-24 13:57:20.961 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-143b7ba8-7253-4d83-8cc6-d611cff6d65d - - - - -] Agent has just been revived. Doing a full sync.
2017-02-24 13:57:21.476 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Agent initialized successfully, now running... 
2017-02-24 13:57:21.476 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] rpc_loop doing a full sync.
2017-02-24 13:57:22.336 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Agent out of sync with plugin!
2017-02-24 13:57:23.758 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Port 9f422822-edca-4d41-8b2b-a39b6f2c9383 updated. Details: {'port_security_enabled': True, 'device': '9f422822-edca-4d41-8b2b-a39b6f2c9383', 'network_qos_policy_id': None, 'admin_state_up': True, 'port_id': '9f422822-edca-4d41-8b2b-a39b6f2c9383', 'mac_address': 'fa:16:3e:c7:90:50', 'network_type': 'vxlan', 'security_groups': ['2e4bd637-ecaa-4d27-9291-7900fabe1ac1'], 'qos_policy_id': None, 'network_id': 'c3e0d3b2-86de-4c14-b1d0-e71398e6c0b8', 'fixed_ips': [{'ip_address': '192.168.0.7', 'subnet_id': '6bb7c5dc-bc6c-4751-920a-664db1ca4165'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'physical_network': None, 'segmentation_id': 29, 'profile': {}}
2017-02-24 13:57:23.758 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Assigning 1 as local vlan for net-id=c3e0d3b2-86de-4c14-b1d0-e71398e6c0b8
2017-02-24 13:57:24.039 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Port 34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b updated. Details: {'port_security_enabled': True, 'device': '34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b', 'network_qos_policy_id': None, 'admin_state_up': True, 'port_id': '34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b', 'mac_address': 'fa:16:3e:62:81:7a', 'network_type': 'vxlan', 'security_groups': ['2e4bd637-ecaa-4d27-9291-7900fabe1ac1'], 'qos_policy_id': None, 'network_id': 'c3e0d3b2-86de-4c14-b1d0-e71398e6c0b8', 'fixed_ips': [{'ip_address': '192.168.0.13', 'subnet_id': '6bb7c5dc-bc6c-4751-920a-664db1ca4165'}], 'device_owner': 'compute:None', 'allowed_address_pairs': [], 'physical_network': None, 'segmentation_id': 29, 'profile': {}}
2017-02-24 13:57:24.226 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Port 84a9315b-4b33-4185-9a98-55b10842569e updated. Details: {'port_security_enabled': True, 'device': '84a9315b-4b33-4185-9a98-55b10842569e', 'network_qos_policy_id': None, 'admin_state_up': True, 'port_id': '84a9315b-4b33-4185-9a98-55b10842569e', 'mac_address': 'fa:16:3e:9e:f7:c6', 'network_type': 'vxlan', 'security_groups': ['2e4bd637-ecaa-4d27-9291-7900fabe1ac1'], 'qos_policy_id': None, 'network_id': 'c3e0d3b2-86de-4c14-b1d0-e71398e6c0b8', 'fixed_ips': [{'ip_address': '192.168.0.12', 'subnet_id': '6bb7c5dc-bc6c-4751-920a-664db1ca4165'}], 'device_owner': 'compute:None', 'allowed_address_pairs': [], 'physical_network': None, 'segmentation_id': 29, 'profile': {}}
2017-02-24 13:57:24.445 8524 INFO neutron.agent.securitygroups_rpc [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Skipping method prepare_devices_filter as firewall is disabled or configured as NoopFirewallDriver.
2017-02-24 13:57:26.273 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Configuration for devices up ['9f422822-edca-4d41-8b2b-a39b6f2c9383', '34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b', '84a9315b-4b33-4185-9a98-55b10842569e'] and devices down [] completed.
2017-02-24 13:57:26.289 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Cleaning stale br-int flows
2017-02-24 13:57:26.367 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Cleaning stale br-ex flows
2017-02-24 13:57:26.429 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Cleaning stale br-tun flows

Any ideas are appreciated since Openstack and Open vSwitch are pretty new to me.

VXLAN port not open in Hyper-V

Hi,

I'm trying to get instances in a Hyper-V compute node to communicate with instances on other hypervisors using Open vSwitch and neutron-ova-agent over VXLAN. My other hypervisors run CentOS/KVM. Instances on the Linux machines can communicate just fine, but I don't get the ones running in Hyper-V to communicate.

The setup runs two controller nodes (10.60.11.21 and 10.60.11.22 with a shared VIP 10.60.11.201) with the L3-agent running the router on .21. In addition I have two separate hypervisors installed, one CentOS/KVM (10.60.11.23) and one Windows Server 2016 Datacenter/Hyper-V (10.60.11.25).

The Hyper-V host runs 3 VMs, one CirrOS (192.168.0.7) and two Windows Server (one with 192.168.0.12 and one trying to use DHCP).

I have created a Hyper-V VMswitch named "br-ex" and enabled the Open vSwitch extension. The VMswitch is connected to an interface named "Tenant" (without IP). I have also manually created a Open vSwitch bridge named "br-ex" and connected the "Tenant" port above to it. The hypervisor also has a Management interface with IP (i.e. 10.60.11.25) with connectivity to the other Openstack nodes.

The VMs on Hyper-V can ping themselves and other VMs on the same Hyper-V host. They can not access the router, DHCP-server on the controller node or VMs on other hypervisors.

I think the problem has something to do with that Open vSwitch does not open a listening socket for VXLAN (udp/4789), i.e. according to netstat -na | find "4789".. I also can see outgoing traffic from br-tun to each vxlan port according to ovs-ofctl dump-flows br-tun but I don't see this traffic in wireshark or at incoming br-tun on the other vxlan vteps, my guess is that this is outgoing ARP requests that for some reason are not sent out on the physical interface.

If I try to connect to an instance from the outside (10.20.150.115) I see the incoming TCP SYN on the Hyper-V host.

Frame 1195965: 128 bytes on wire (1024 bits), 128 bytes captured (1024 bits) on interface 0
Ethernet II, Src: HewlettP_70:57:80 (ac:16:2d:70:57:80), Dst: HewlettP_70:a9:d0 (ac:16:2d:70:a9:d0)
Internet Protocol Version 4, Src: 10.60.11.21, Dst: 10.60.11.25
User Datagram Protocol, Src Port: 44982, Dst Port: 4789
Virtual eXtensible Local Area Network
Ethernet II, Src: fa:16:3e:e2:06:b1 (fa:16:3e:e2:06:b1), Dst: fa:16:3e:9e:f7:c6 (fa:16:3e:9e:f7:c6)
Internet Protocol Version 4, Src: 10.20.150.115, Dst: 192.168.0.12
Transmission Control Protocol, Src Port: 53839, Dst Port: 22, Seq: 0, Len: 0

I.e. the NAT/routing into Hyper-V works. But the VXLAN port is not open, so Hyper-V-host responds with ICMP Port Unreachable.

Frame 1195966: 156 bytes on wire (1248 bits), 156 bytes captured (1248 bits) on interface 0
Ethernet II, Src: HewlettP_70:a9:d0 (ac:16:2d:70:a9:d0), Dst: HewlettP_70:57:80 (ac:16:2d:70:57:80)
Internet Protocol Version 4, Src: 10.60.11.25, Dst: 10.60.11.21
Internet Control Message Protocol
    Type: 3 (Destination unreachable)
    Code: 3 (Port unreachable)
    Checksum: 0xfe7a [correct]
    [Checksum Status: Good]
    Unused: 00000000
    Internet Protocol Version 4, Src: 10.60.11.21, Dst: 10.60.11.25
    User Datagram Protocol, Src Port: 44982, Dst Port: 4789
    Virtual eXtensible Local Area Network
    Ethernet II, Src: fa:16:3e:e2:06:b1 (fa:16:3e:e2:06:b1), Dst: fa:16:3e:9e:f7:c6 (fa:16:3e:9e:f7:c6)
    Internet Protocol Version 4, Src: 10.20.150.115, Dst: 192.168.0.12
    Transmission Control Protocol, Src Port: 53839, Dst Port: 22, Seq: 1062324327

Firewalls in both the hypervisor and VMs are turned off.

I use Newton release with the Windows components https://cloudbase.it/downloads/HyperVNovaComputeNewton1401.zip and https://cloudbase.it/downloads/openvswitch-hyperv-2.6.1-certified.msi.

My neutron-ovs-agent.conf

[DEFAULT]
verbose=true
debug = False

control_exchange=neutron
policy_file=C:\Program Files\Cloudbase Solutions\OpenStack\Nova\etc\policy.json
logdir=C:\OpenStack\Log\
logfile=neutron-ovs-agent.log
transport_url = rabbit://openstack:password@10.60.11.21:5672,openstack:password@10.60.11.22:5672

[agent]
tunnel_types = vxlan
enable_metrics_collection=false
vxlan_udp_port = 4789

[SECURITYGROUP]
enable_security_group=false

[ovs]
of_interface = ovs-ofctl 
ovsdb_interface = vsctl
local_ip = 10.60.11.25
tunnel_bridge = br-tun
integration_bridge = br-int
tenant_network_type = vxlan
enable_tunneling = True
vni_ranges = 1:1000
vxlan_group = 239.1.1.1
bridge_mappings = physnet1:br-ex

My nova.conf

[DEFAULT]
auth_strategy=keystone
image_service=nova.image.glance.GlanceImageService
compute_driver=compute_hyperv.driver.HyperVDriver
network_api_class=nova.network.neutronv2.api.API
volume_api_class=nova.volume.cinder.API
instances_path=C:\OpenStack\Instances
use_cow_images=true
flat_injected=true
mkisofs_cmd=C:\Program Files\Cloudbase Solutions\OpenStack\Nova\bin\mkisofs.exe
verbose=true
allow_resize_to_same_host=true
running_deleted_instance_action=reap
running_deleted_instance_poll_interval=120
resize_confirm_window=5
resume_guests_state_on_host_boot=true
rpc_response_timeout=1800
use_neutron=true
vif_plugging_is_fatal=false
vif_plugging_timeout=60
rpc_backend=rabbit
logdir=C:\OpenStack\Log\
logfile=nova-compute.log
force_config_drive=True
debug = False
transport_url = rabbit://openstack:password@10.60.11.21:5672,openstack:password@10.60.11.22:5672

[oslo_policy]
policy_file=C:\Program Files\Cloudbase Solutions\OpenStack\Nova\etc\policy.json

[oslo_concurrency]
lock_path=C:\OpenStack\Log\

[glance]
debug = False
api_servers=http://10.60.11.21:9292,http://10.60.11.22:9292

[hyperv]
vswitch_name=br-ex
limit_cpu_features=true
config_drive_inject_password=false
qemu_img_cmd=C:\Program Files\Cloudbase Solutions\OpenStack\Nova\bin\qemu-img.exe
config_drive_cdrom=true
dynamic_memory_ratio=1
enable_instance_metrics_collection=false

[rdp]
enabled=true
html5_proxy_base_url=http://10.60.11.25:8000/

[neutron]
url = http://10.60.11.201:9696
auth_strategy=keystone
project_domain_name=Default
project_name=service
user_domain_name=Default
username = neutron
password = password
auth_url = http://10.60.11.201:35357/v3
auth_type=v3password

Output from ovs-vsctl show

4ac94d05-c741-4aab-8aae-db750d325647
    Bridge br-tun
        fail_mode: secure
        Port "vxlan-0a3c0b15"
            Interface "vxlan-0a3c0b15"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="10.60.11.25", out_key=flow, remote_ip="10.60.11.21"}
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
        Port "vxlan-0a3c0b17"
            Interface "vxlan-0a3c0b17"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="10.60.11.25", out_key=flow, remote_ip="10.60.11.23"}
        Port "vxlan-0a3c0b16"
            Interface "vxlan-0a3c0b16"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="10.60.11.25", out_key=flow, remote_ip="10.60.11.22"}
    Bridge br-int
        fail_mode: secure
        Port "84a9315b-4b33-4185-9a98-55b10842569e"
            tag: 1
            Interface "84a9315b-4b33-4185-9a98-55b10842569e"
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port br-int
            Interface br-int
                type: internal
        Port "9f422822-edca-4d41-8b2b-a39b6f2c9383"
            tag: 1
            Interface "9f422822-edca-4d41-8b2b-a39b6f2c9383"
        Port "34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b"
            tag: 1
            Interface "34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b"
        Port int-br-ex
            Interface int-br-ex
                type: patch
                options: {peer=phy-br-ex}
    Bridge br-ex
        fail_mode: secure
        Port br-ex
            Interface br-ex
                type: internal
        Port Tenant
            Interface Tenant
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
                options: {peer=int-br-ex}

Output from ovs-dpctl show

system@ovs-system:
    lookups: hit:53116 missed:53200 lost:0
    flows: 2
    port 1: br-ex (internal)
    port 2: Tenant
    port 3: br-int (internal)
    port 4: br-tun (internal)
    port 5: vxlan_sys_4789 (vxlan)
    port 6: 9f422822-edca-4d41-8b2b-a39b6f2c9383
    port 8: 84a9315b-4b33-4185-9a98-55b10842569e
    port 10: 34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b

Output from ovs-ofctl dump-ports-desc br-int

OFPST_PORT_DESC reply (xid=0x2):
 1(int-br-ex): addr:4a:f3:6a:3d:e9:9b
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(patch-tun): addr:ca:b5:cf:6b:51:d5
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 3(9f422822-edca-4): addr:fa:16:3e:c7:90:50
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 12(84a9315b-4b33-4): addr:fa:16:3e:9e:f7:c6
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 16(34cecd4b-c5cc-4): addr:fa:16:3e:62:81:7a
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 LOCAL(br-int): addr:00:15:5d:0c:19:14
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max

Output from ovs-ofctl dump-ports-desc br-tun

OFPST_PORT_DESC reply (xid=0x2):
 1(patch-int): addr:32:52:d2:71:d9:d0
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(vxlan-0a3c0b15): addr:c2:27:30:e3:a4:d0
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 3(vxlan-0a3c0b16): addr:e6:a2:aa:1d:bd:85
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 4(vxlan-0a3c0b17): addr:e6:9e:c7:01:aa:fa
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 LOCAL(br-tun): addr:00:15:5d:0c:19:15
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max

Output from ovs-ofctl dump-ports-desc br-ex

OFPST_PORT_DESC reply (xid=0x2):
 1(Tenant): addr:ac:16:2d:70:a9:d1
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(phy-br-ex): addr:62:04:7b:dc:39:59
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 LOCAL(br-ex): addr:00:15:5d:0c:19:13
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max

Output from ovs-ofctl dump-flows br-int

NXST_FLOW reply (xid=0x4):
 cookie=0x953fea9b637ac6b5, duration=20369.454s, table=0, n_packets=0, n_bytes=0, idle_age=20369, priority=10,arp,in_port=16 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20368.955s, table=0, n_packets=17, n_bytes=714, idle_age=3982, priority=10,arp,in_port=3 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20368.454s, table=0, n_packets=571, n_bytes=23982, idle_age=456, priority=10,arp,in_port=12 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20369.610s, table=0, n_packets=0, n_bytes=0, idle_age=20369, priority=10,icmp6,in_port=16,icmp_type=136 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20369.079s, table=0, n_packets=0, n_bytes=0, idle_age=20369, priority=10,icmp6,in_port=3,icmp_type=136 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20368.578s, table=0, n_packets=0, n_bytes=0, idle_age=20368, priority=10,icmp6,in_port=12,icmp_type=136 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20369.736s, table=0, n_packets=600, n_bytes=179319, idle_age=58, priority=9,in_port=16 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20369.204s, table=0, n_packets=856, n_bytes=1256064, idle_age=3969, priority=9,in_port=3 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20368.704s, table=0, n_packets=920, n_bytes=1266011, idle_age=105, priority=9,in_port=12 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20375.434s, table=0, n_packets=0, n_bytes=0, idle_age=20375, priority=0 actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20375.377s, table=23, n_packets=0, n_bytes=0, idle_age=20375, priority=0 actions=drop
 cookie=0x953fea9b637ac6b5, duration=20369.533s, table=24, n_packets=0, n_bytes=0, idle_age=20369, priority=2,arp,in_port=16,arp_spa=192.168.0.13 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20369.017s, table=24, n_packets=17, n_bytes=714, idle_age=3982, priority=2,arp,in_port=3,arp_spa=192.168.0.7 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20368.517s, table=24, n_packets=571, n_bytes=23982, idle_age=456, priority=2,arp,in_port=12,arp_spa=192.168.0.12 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20369.673s, table=24, n_packets=0, n_bytes=0, idle_age=20369, priority=2,icmp6,in_port=16,icmp_type=136,nd_target=fe80::f816:3eff:fe62:817a actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20369.141s, table=24, n_packets=0, n_bytes=0, idle_age=20369, priority=2,icmp6,in_port=3,icmp_type=136,nd_target=fe80::f816:3eff:fec7:9050 actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20368.641s, table=24, n_packets=0, n_bytes=0, idle_age=20368, priority=2,icmp6,in_port=12,icmp_type=136,nd_target=fe80::f816:3eff:fe9e:f7c6 actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20375.314s, table=24, n_packets=0, n_bytes=0, idle_age=20375, priority=0 actions=drop
 cookie=0x953fea9b637ac6b5, duration=20369.861s, table=25, n_packets=600, n_bytes=179319, idle_age=58, priority=2,in_port=16,dl_src=fa:16:3e:62:81:7a actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20369.329s, table=25, n_packets=873, n_bytes=1256778, idle_age=3969, priority=2,in_port=3,dl_src=fa:16:3e:c7:90:50 actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20368.830s, table=25, n_packets=1491, n_bytes=1289993, idle_age=105, priority=2,in_port=12,dl_src=fa:16:3e:9e:f7:c6 actions=NORMAL

Output from ovs-ofctl dumpflows br-tun

NXST_FLOW reply (xid=0x4):
 cookie=0xaf522f8bd636f703, duration=20385.783s, table=0, n_packets=1231, n_bytes=213080, idle_age=70, priority=1,in_port=1 actions=resubmit(,2)
 cookie=0xaf522f8bd636f703, duration=20384.081s, table=0, n_packets=0, n_bytes=0, idle_age=20384, priority=1,in_port=2 actions=resubmit(,4)
 cookie=0xaf522f8bd636f703, duration=20383.862s, table=0, n_packets=0, n_bytes=0, idle_age=20383, priority=1,in_port=3 actions=resubmit(,4)
 cookie=0xaf522f8bd636f703, duration=20383.643s, table=0, n_packets=0, n_bytes=0, idle_age=20383, priority=1,in_port=4 actions=resubmit(,4)
 cookie=0xaf522f8bd636f703, duration=20385.782s, table=0, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=drop
 cookie=0xaf522f8bd636f703, duration=20385.779s, table=2, n_packets=0, n_bytes=0, idle_age=20385, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20)
 cookie=0xaf522f8bd636f703, duration=20385.777s, table=2, n_packets=1231, n_bytes=213080, idle_age=70, priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,22)
 cookie=0xaf522f8bd636f703, duration=20385.777s, table=3, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=drop
 cookie=0xaf522f8bd636f703, duration=20381.923s, table=4, n_packets=0, n_bytes=0, idle_age=20381, priority=1,tun_id=0x1d actions=mod_vlan_vid:1,resubmit(,10)
 cookie=0xaf522f8bd636f703, duration=20385.776s, table=4, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=drop
 cookie=0xaf522f8bd636f703, duration=20385.775s, table=6, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=drop
 cookie=0xaf522f8bd636f703, duration=20385.774s, table=10, n_packets=0, n_bytes=0, idle_age=20385, priority=1 actions=learn(table=20,hard_timeout=300,priority=1,cookie=0xaf522f8bd636f703,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1
 cookie=0xaf522f8bd636f703, duration=20385.773s, table=20, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=resubmit(,22)
 cookie=0xaf522f8bd636f703, duration=20381.987s, table=22, n_packets=1231, n_bytes=213080, idle_age=70, dl_vlan=1 actions=strip_vlan,set_tunnel:0x1d,output:2,output:4,output:3
 cookie=0xaf522f8bd636f703, duration=20385.706s, table=22, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=drop

Output from ovs-ofctl dump-flows br-ex

NXST_FLOW reply (xid=0x4):

The ovs-vswitchd.log

    2017-02-24T12:57:07.407Z|00001|vlog|INFO|opened log file c:\Program Files\Cloudbase Solutions\Open vSwitch\logs\ovs-vswitchd.log
    2017-02-24T12:57:07.422Z|00002|ovs_numa|INFO|Discovered 0 NUMA nodes and 0 CPU cores
    2017-02-24T12:57:07.423Z|00003|reconnect|INFO|unix:C:\ProgramData\openvswitch\db.sock: connecting...
    2017-02-24T12:57:07.424Z|00004|reconnect|INFO|unix:C:\ProgramData\openvswitch\db.sock: connected
    2017-02-24T12:57:07.454Z|00005|ofproto_dpif|INFO|system@ovs-system: Datapath supports recirculation
    2017-02-24T12:57:07.455Z|00006|ofproto_dpif|INFO|system@ovs-system: MPLS label stack length probed as 3
    2017-02-24T12:57:07.456Z|00007|ofproto_dpif|INFO|system@ovs-system: datapath does not support masked set action feature.
    2017-02-24T12:57:07.456Z|00008|ofproto_dpif|INFO|system@ovs-system: Datapath supports truncate action
    2017-02-24T12:57:07.456Z|00009|ofproto_dpif|INFO|system@ovs-system: Datapath does not support unique flow ids
    2017-02-24T12:57:07.456Z|00010|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_state
    2017-02-24T12:57:07.456Z|00011|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_zone
    2017-02-24T12:57:07.457Z|00012|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_mark
    2017-02-24T12:57:07.457Z|00013|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_label
    2017-02-24T12:57:07.457Z|00014|ofproto_dpif|INFO|system@ovs-system: Datapath does not support ct_state_nat
    2017-02-24T12:57:07.461Z|00015|ofproto_dpif|WARN|system@ovs-system: variable-length userdata feature probe failed (Unknown error)
    2017-02-24T12:57:07.474Z|00016|bridge|INFO|bridge br-int: added interface 34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b on port 16
    2017-02-24T12:57:07.475Z|00017|bridge|INFO|bridge br-int: added interface br-int on port 65534
    2017-02-24T12:57:07.475Z|00018|bridge|INFO|bridge br-int: added interface int-br-ex on port 1
    2017-02-24T12:57:07.475Z|00019|bridge|INFO|bridge br-int: added interface 84a9315b-4b33-4185-9a98-55b10842569e on port 12
    2017-02-24T12:57:07.475Z|00020|bridge|INFO|bridge br-int: added interface 9f422822-edca-4d41-8b2b-a39b6f2c9383 on port 3
    2017-02-24T12:57:07.476Z|00021|bridge|INFO|bridge br-int: added interface patch-tun on port 2
    2017-02-24T12:57:07.476Z|00022|bridge|INFO|bridge br-ex: added interface br-ex on port 65534
    2017-02-24T12:57:07.476Z|00023|bridge|INFO|bridge br-ex: added interface phy-br-ex on port 2
    2017-02-24T12:57:07.476Z|00024|bridge|INFO|bridge br-ex: added interface Tenant on port 1
    2017-02-24T12:57:07.478Z|00025|bridge|INFO|bridge br-tun: added interface vxlan-0a3c0b15 on port 2
    2017-02-24T12:57:07.478Z|00026|bridge|INFO|bridge br-tun: added interface patch-int on port 1
    2017-02-24T12:57:07.478Z|00027|bridge|INFO|bridge br-tun: added interface vxlan-0a3c0b16 on port 3
    2017-02-24T12:57:07.478Z|00028|bridge|INFO|bridge br-tun: added interface vxlan-0a3c0b17 on port 4
    2017-02-24T12:57:07.479Z|00029|bridge|INFO|bridge br-tun: added interface br-tun on port 65534
    2017-02-24T12:57:07.479Z|00030|bridge|INFO|bridge br-int: using datapath ID 000056ba7b1ba641
    2017-02-24T12:57:07.480Z|00031|connmgr|INFO|br-int: added service controller "punix:c:\ProgramData\openvswitch\/br-int.mgmt"
    2017-02-24T12:57:07.493Z|00032|bridge|INFO|bridge br-ex: using datapath ID 0000ac162d70a9d1
    2017-02-24T12:57:07.493Z|00033|connmgr|INFO|br-ex: added service controller "punix:c:\ProgramData\openvswitch\/br-ex.mgmt"
    2017-02-24T12:57:07.505Z|00034|bridge|INFO|bridge br-tun: using datapath ID 0000ba4702014949
    2017-02-24T12:57:07.505Z|00035|connmgr|INFO|br-tun: added service controller "punix:c:\ProgramData\openvswitch\/br-tun.mgmt"
    2017-02-24T12:57:07.524Z|00036|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.6.1.0
    2017-02-24T12:57:17.216Z|00037|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:17.277Z|00038|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:17.340Z|00039|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:17.430Z|00040|memory|INFO|10192 kB peak resident set size after 10.0 seconds
    2017-02-24T12:57:17.431Z|00041|memory|INFO|handlers:17 ports:14 revalidators:7 rules:15 udpif keys:2
    2017-02-24T12:57:17.902Z|00042|connmgr|INFO|br-ex<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:18.277Z|00043|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:18.340Z|00044|connmgr|INFO|br-ex<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:19.223Z|00045|connmgr|INFO|br-tun<->unix: 9 flow_mods in the last 0 s (9 adds)
    2017-02-24T12:57:19.293Z|00046|connmgr|INFO|br-tun<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:21.823Z|00047|connmgr|INFO|br-tun<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:22.073Z|00048|connmgr|INFO|br-tun<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:22.324Z|00049|connmgr|INFO|br-tun<->unix: 1 flow_mods in the last 0 s (1 adds)

and the neutron-ovs-agent.log

2017-02-24 13:57:16.726 8524 INFO neutron.common.config [-] Logging enabled!
2017-02-24 13:57:16.726 8524 INFO neutron.common.config [-] c:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python\Scripts\neutron-openvswitch-agent version 9.0.1.dev14
2017-02-24 13:57:17.492 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-143b7ba8-7253-4d83-8cc6-d611cff6d65d - - - - -] Mapping physical network physnet1 to bridge br-ex
2017-02-24 13:57:19.164 8524 INFO neutron.agent.agent_extensions_manager [req-143b7ba8-7253-4d83-8cc6-d611cff6d65d - - - - -] Loaded agent extensions: []
2017-02-24 13:57:20.945 8524 INFO oslo.messaging._drivers.impl_rabbit [-] [ccd7f232-990e-4bdb-a9ca-6cd292881c76] Reconnected to AMQP server on 10.60.11.22:5672 via [amqp] client with port 51875.
2017-02-24 13:57:20.961 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-143b7ba8-7253-4d83-8cc6-d611cff6d65d - - - - -] Agent has just been revived. Doing a full sync.
2017-02-24 13:57:21.476 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Agent initialized successfully, now running... 
2017-02-24 13:57:21.476 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] rpc_loop doing a full sync.
2017-02-24 13:57:22.336 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Agent out of sync with plugin!
2017-02-24 13:57:23.758 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Port 9f422822-edca-4d41-8b2b-a39b6f2c9383 updated. Details: {'port_security_enabled': True, 'device': '9f422822-edca-4d41-8b2b-a39b6f2c9383', 'network_qos_policy_id': None, 'admin_state_up': True, 'port_id': '9f422822-edca-4d41-8b2b-a39b6f2c9383', 'mac_address': 'fa:16:3e:c7:90:50', 'network_type': 'vxlan', 'security_groups': ['2e4bd637-ecaa-4d27-9291-7900fabe1ac1'], 'qos_policy_id': None, 'network_id': 'c3e0d3b2-86de-4c14-b1d0-e71398e6c0b8', 'fixed_ips': [{'ip_address': '192.168.0.7', 'subnet_id': '6bb7c5dc-bc6c-4751-920a-664db1ca4165'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'physical_network': None, 'segmentation_id': 29, 'profile': {}}
2017-02-24 13:57:23.758 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Assigning 1 as local vlan for net-id=c3e0d3b2-86de-4c14-b1d0-e71398e6c0b8
2017-02-24 13:57:24.039 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Port 34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b updated. Details: {'port_security_enabled': True, 'device': '34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b', 'network_qos_policy_id': None, 'admin_state_up': True, 'port_id': '34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b', 'mac_address': 'fa:16:3e:62:81:7a', 'network_type': 'vxlan', 'security_groups': ['2e4bd637-ecaa-4d27-9291-7900fabe1ac1'], 'qos_policy_id': None, 'network_id': 'c3e0d3b2-86de-4c14-b1d0-e71398e6c0b8', 'fixed_ips': [{'ip_address': '192.168.0.13', 'subnet_id': '6bb7c5dc-bc6c-4751-920a-664db1ca4165'}], 'device_owner': 'compute:None', 'allowed_address_pairs': [], 'physical_network': None, 'segmentation_id': 29, 'profile': {}}
2017-02-24 13:57:24.226 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Port 84a9315b-4b33-4185-9a98-55b10842569e updated. Details: {'port_security_enabled': True, 'device': '84a9315b-4b33-4185-9a98-55b10842569e', 'network_qos_policy_id': None, 'admin_state_up': True, 'port_id': '84a9315b-4b33-4185-9a98-55b10842569e', 'mac_address': 'fa:16:3e:9e:f7:c6', 'network_type': 'vxlan', 'security_groups': ['2e4bd637-ecaa-4d27-9291-7900fabe1ac1'], 'qos_policy_id': None, 'network_id': 'c3e0d3b2-86de-4c14-b1d0-e71398e6c0b8', 'fixed_ips': [{'ip_address': '192.168.0.12', 'subnet_id': '6bb7c5dc-bc6c-4751-920a-664db1ca4165'}], 'device_owner': 'compute:None', 'allowed_address_pairs': [], 'physical_network': None, 'segmentation_id': 29, 'profile': {}}
2017-02-24 13:57:24.445 8524 INFO neutron.agent.securitygroups_rpc [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Skipping method prepare_devices_filter as firewall is disabled or configured as NoopFirewallDriver.
2017-02-24 13:57:26.273 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Configuration for devices up ['9f422822-edca-4d41-8b2b-a39b6f2c9383', '34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b', '84a9315b-4b33-4185-9a98-55b10842569e'] and devices down [] completed.
2017-02-24 13:57:26.289 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Cleaning stale br-int flows
2017-02-24 13:57:26.367 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Cleaning stale br-ex flows
2017-02-24 13:57:26.429 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Cleaning stale br-tun flows

Any ideas are appreciated since Openstack and Open vSwitch are pretty new to me.

VXLAN port not open in Hyper-V

Hi,

I'm trying to get instances in a Windows Server 2016 Datacenter based Hyper-V compute node to communicate with instances on other hypervisors using Open vSwitch and neutron-ova-agent neutron-ovs-agent over VXLAN. My other hypervisors run CentOS/KVM. Instances on the Linux machines can communicate just fine, but I don't get the ones running in Hyper-V to communicate.communicate with the world outside the hypervisor.

The setup runs two controller nodes (10.60.11.21 and 10.60.11.22 with a shared VIP 10.60.11.201) with the L3-agent running the router on .21. In addition I have two separate hypervisors installed, one CentOS/KVM KVM (10.60.11.23) and one Windows Server 2016 Datacenter/Hyper-V Hyper-V (10.60.11.25).

The Hyper-V host runs 3 VMs, one CirrOS (192.168.0.7) and two Windows Server (one with 192.168.0.12 and one trying to use DHCP).

I have created a Hyper-V VMswitch named "br-ex" "external" and enabled the Open vSwitch extension. The VMswitch is connected to an a physical interface named "Tenant" (without IP). I have also manually created a Open vSwitch bridge named "br-ex" and connected the "Tenant" port above to it. The hypervisor also has a Management interface with IP (i.e. 10.60.11.25) with connectivity to the other Openstack nodes.nodes, this is the interface that should be used for the VXLAN traffic.

The VMs on Hyper-V can ping themselves and other VMs on the same Hyper-V host. They can not access the router, DHCP-server on the controller node or VMs on other hypervisors.

I think the problem has something to do with that Open vSwitch does not open a listening socket for VXLAN (udp/4789), i.e. according to netstat -na | find "4789". I also can see outgoing flood traffic from br-tun to each vxlan VXLAN port according to ovs-ofctl dump-flows br-tun but I don't see this traffic in wireshark or at the incoming br-tun VXLAN port on the other vxlan vteps, hypervisors, my guess is that this is outgoing ARP requests that for some reason are not sent out on the physical interface. interface.

If I try to connect to an instance on the Hyper-V host from the outside (10.20.150.115) I see the incoming TCP SYN on the Hyper-V host. host (but not on the VM).

Frame 1195965: 128 bytes on wire (1024 bits), 128 bytes captured (1024 bits) on interface 0
Ethernet II, Src: HewlettP_70:57:80 (ac:16:2d:70:57:80), Dst: HewlettP_70:a9:d0 (ac:16:2d:70:a9:d0)
Internet Protocol Version 4, Src: 10.60.11.21, Dst: 10.60.11.25
User Datagram Protocol, Src Port: 44982, Dst Port: 4789
Virtual eXtensible Local Area Network
Ethernet II, Src: fa:16:3e:e2:06:b1 (fa:16:3e:e2:06:b1), Dst: fa:16:3e:9e:f7:c6 (fa:16:3e:9e:f7:c6)
Internet Protocol Version 4, Src: 10.20.150.115, Dst: 192.168.0.12
Transmission Control Protocol, Src Port: 53839, Dst Port: 22, Seq: 0, Len: 0

I.e. the NAT/routing into Hyper-V works. But since the VXLAN port is not open, so Hyper-V-host responds with ICMP Port Unreachable.

Frame 1195966: 156 bytes on wire (1248 bits), 156 bytes captured (1248 bits) on interface 0
Ethernet II, Src: HewlettP_70:a9:d0 (ac:16:2d:70:a9:d0), Dst: HewlettP_70:57:80 (ac:16:2d:70:57:80)
Internet Protocol Version 4, Src: 10.60.11.25, Dst: 10.60.11.21
Internet Control Message Protocol
    Type: 3 (Destination unreachable)
    Code: 3 (Port unreachable)
    Checksum: 0xfe7a [correct]
    [Checksum Status: Good]
    Unused: 00000000
    Internet Protocol Version 4, Src: 10.60.11.21, Dst: 10.60.11.25
    User Datagram Protocol, Src Port: 44982, Dst Port: 4789
    Virtual eXtensible Local Area Network
    Ethernet II, Src: fa:16:3e:e2:06:b1 (fa:16:3e:e2:06:b1), Dst: fa:16:3e:9e:f7:c6 (fa:16:3e:9e:f7:c6)
    Internet Protocol Version 4, Src: 10.20.150.115, Dst: 192.168.0.12
    Transmission Control Protocol, Src Port: 53839, Dst Port: 22, Seq: 1062324327

Firewalls in both the hypervisor and VMs are turned off.off. I have tried reinstalling Open vSwitch and rebooted the Hyper-V host without change.

I use Newton release with the Windows components https://cloudbase.it/downloads/HyperVNovaComputeNewton1401.zip and https://cloudbase.it/downloads/openvswitch-hyperv-2.6.1-certified.msi.

Have anyone got VXLAN to work on Windows Server 2016 using these components?

My neutron-ovs-agent.conf

[DEFAULT]
verbose=true
debug = False

control_exchange=neutron
policy_file=C:\Program Files\Cloudbase Solutions\OpenStack\Nova\etc\policy.json
logdir=C:\OpenStack\Log\
logfile=neutron-ovs-agent.log
transport_url = rabbit://openstack:password@10.60.11.21:5672,openstack:password@10.60.11.22:5672

[agent]
tunnel_types = vxlan
enable_metrics_collection=false
vxlan_udp_port = 4789

[SECURITYGROUP]
enable_security_group=false

[ovs]
of_interface = ovs-ofctl 
ovsdb_interface = vsctl
local_ip = 10.60.11.25
tunnel_bridge = br-tun
integration_bridge = br-int
tenant_network_type = vxlan
enable_tunneling = True
vni_ranges = 1:1000
vxlan_group = 239.1.1.1
bridge_mappings = physnet1:br-ex

My nova.conf

[DEFAULT]
auth_strategy=keystone
image_service=nova.image.glance.GlanceImageService
compute_driver=compute_hyperv.driver.HyperVDriver
network_api_class=nova.network.neutronv2.api.API
volume_api_class=nova.volume.cinder.API
instances_path=C:\OpenStack\Instances
use_cow_images=true
flat_injected=true
mkisofs_cmd=C:\Program Files\Cloudbase Solutions\OpenStack\Nova\bin\mkisofs.exe
verbose=true
allow_resize_to_same_host=true
running_deleted_instance_action=reap
running_deleted_instance_poll_interval=120
resize_confirm_window=5
resume_guests_state_on_host_boot=true
rpc_response_timeout=1800
use_neutron=true
vif_plugging_is_fatal=false
vif_plugging_timeout=60
rpc_backend=rabbit
logdir=C:\OpenStack\Log\
logfile=nova-compute.log
force_config_drive=True
debug = False
transport_url = rabbit://openstack:password@10.60.11.21:5672,openstack:password@10.60.11.22:5672

[oslo_policy]
policy_file=C:\Program Files\Cloudbase Solutions\OpenStack\Nova\etc\policy.json

[oslo_concurrency]
lock_path=C:\OpenStack\Log\

[glance]
debug = False
api_servers=http://10.60.11.21:9292,http://10.60.11.22:9292

[hyperv]
vswitch_name=br-ex
vswitch_name=external
limit_cpu_features=true
config_drive_inject_password=false
qemu_img_cmd=C:\Program Files\Cloudbase Solutions\OpenStack\Nova\bin\qemu-img.exe
config_drive_cdrom=true
dynamic_memory_ratio=1
enable_instance_metrics_collection=false

[rdp]
enabled=true
html5_proxy_base_url=http://10.60.11.25:8000/

[neutron]
url = http://10.60.11.201:9696
auth_strategy=keystone
project_domain_name=Default
project_name=service
user_domain_name=Default
username = neutron
password = password
auth_url = http://10.60.11.201:35357/v3
auth_type=v3password

Output from ovs-vsctl show

4ac94d05-c741-4aab-8aae-db750d325647
    Bridge br-tun
        fail_mode: secure
        Port "vxlan-0a3c0b15"
            Interface "vxlan-0a3c0b15"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="10.60.11.25", out_key=flow, remote_ip="10.60.11.21"}
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
        Port "vxlan-0a3c0b17"
            Interface "vxlan-0a3c0b17"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="10.60.11.25", out_key=flow, remote_ip="10.60.11.23"}
        Port "vxlan-0a3c0b16"
            Interface "vxlan-0a3c0b16"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="10.60.11.25", out_key=flow, remote_ip="10.60.11.22"}
    Bridge br-int
        fail_mode: secure
        Port "84a9315b-4b33-4185-9a98-55b10842569e"
            tag: 1
            Interface "84a9315b-4b33-4185-9a98-55b10842569e"
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port br-int
            Interface br-int
                type: internal
        Port "9f422822-edca-4d41-8b2b-a39b6f2c9383"
            tag: 1
            Interface "9f422822-edca-4d41-8b2b-a39b6f2c9383"
        Port "34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b"
            tag: 1
            Interface "34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b"
        Port int-br-ex
            Interface int-br-ex
                type: patch
                options: {peer=phy-br-ex}
    Bridge br-ex
        fail_mode: secure
        Port br-ex
            Interface br-ex
                type: internal
        Port Tenant
            Interface Tenant
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
                options: {peer=int-br-ex}

Output from ovs-dpctl show

system@ovs-system:
    lookups: hit:53116 missed:53200 lost:0
    flows: 2
    port 1: br-ex (internal)
    port 2: Tenant
    port 3: br-int (internal)
    port 4: br-tun (internal)
    port 5: vxlan_sys_4789 (vxlan)
    port 6: 9f422822-edca-4d41-8b2b-a39b6f2c9383
    port 8: 84a9315b-4b33-4185-9a98-55b10842569e
    port 10: 34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b

Output from ovs-ofctl dump-ports-desc br-int

OFPST_PORT_DESC reply (xid=0x2):
 1(int-br-ex): addr:4a:f3:6a:3d:e9:9b
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(patch-tun): addr:ca:b5:cf:6b:51:d5
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 3(9f422822-edca-4): addr:fa:16:3e:c7:90:50
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 12(84a9315b-4b33-4): addr:fa:16:3e:9e:f7:c6
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 16(34cecd4b-c5cc-4): addr:fa:16:3e:62:81:7a
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 LOCAL(br-int): addr:00:15:5d:0c:19:14
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max

Output from ovs-ofctl dump-ports-desc br-tun

OFPST_PORT_DESC reply (xid=0x2):
 1(patch-int): addr:32:52:d2:71:d9:d0
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(vxlan-0a3c0b15): addr:c2:27:30:e3:a4:d0
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 3(vxlan-0a3c0b16): addr:e6:a2:aa:1d:bd:85
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 4(vxlan-0a3c0b17): addr:e6:9e:c7:01:aa:fa
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 LOCAL(br-tun): addr:00:15:5d:0c:19:15
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max

Output from ovs-ofctl dump-ports-desc br-ex

OFPST_PORT_DESC reply (xid=0x2):
 1(Tenant): addr:ac:16:2d:70:a9:d1
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(phy-br-ex): addr:62:04:7b:dc:39:59
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 LOCAL(br-ex): addr:00:15:5d:0c:19:13
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max

Output from ovs-ofctl dump-flows br-int

NXST_FLOW reply (xid=0x4):
 cookie=0x953fea9b637ac6b5, duration=20369.454s, table=0, n_packets=0, n_bytes=0, idle_age=20369, priority=10,arp,in_port=16 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20368.955s, table=0, n_packets=17, n_bytes=714, idle_age=3982, priority=10,arp,in_port=3 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20368.454s, table=0, n_packets=571, n_bytes=23982, idle_age=456, priority=10,arp,in_port=12 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20369.610s, table=0, n_packets=0, n_bytes=0, idle_age=20369, priority=10,icmp6,in_port=16,icmp_type=136 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20369.079s, table=0, n_packets=0, n_bytes=0, idle_age=20369, priority=10,icmp6,in_port=3,icmp_type=136 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20368.578s, table=0, n_packets=0, n_bytes=0, idle_age=20368, priority=10,icmp6,in_port=12,icmp_type=136 actions=resubmit(,24)
 cookie=0x953fea9b637ac6b5, duration=20369.736s, table=0, n_packets=600, n_bytes=179319, idle_age=58, priority=9,in_port=16 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20369.204s, table=0, n_packets=856, n_bytes=1256064, idle_age=3969, priority=9,in_port=3 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20368.704s, table=0, n_packets=920, n_bytes=1266011, idle_age=105, priority=9,in_port=12 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20375.434s, table=0, n_packets=0, n_bytes=0, idle_age=20375, priority=0 actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20375.377s, table=23, n_packets=0, n_bytes=0, idle_age=20375, priority=0 actions=drop
 cookie=0x953fea9b637ac6b5, duration=20369.533s, table=24, n_packets=0, n_bytes=0, idle_age=20369, priority=2,arp,in_port=16,arp_spa=192.168.0.13 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20369.017s, table=24, n_packets=17, n_bytes=714, idle_age=3982, priority=2,arp,in_port=3,arp_spa=192.168.0.7 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20368.517s, table=24, n_packets=571, n_bytes=23982, idle_age=456, priority=2,arp,in_port=12,arp_spa=192.168.0.12 actions=resubmit(,25)
 cookie=0x953fea9b637ac6b5, duration=20369.673s, table=24, n_packets=0, n_bytes=0, idle_age=20369, priority=2,icmp6,in_port=16,icmp_type=136,nd_target=fe80::f816:3eff:fe62:817a actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20369.141s, table=24, n_packets=0, n_bytes=0, idle_age=20369, priority=2,icmp6,in_port=3,icmp_type=136,nd_target=fe80::f816:3eff:fec7:9050 actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20368.641s, table=24, n_packets=0, n_bytes=0, idle_age=20368, priority=2,icmp6,in_port=12,icmp_type=136,nd_target=fe80::f816:3eff:fe9e:f7c6 actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20375.314s, table=24, n_packets=0, n_bytes=0, idle_age=20375, priority=0 actions=drop
 cookie=0x953fea9b637ac6b5, duration=20369.861s, table=25, n_packets=600, n_bytes=179319, idle_age=58, priority=2,in_port=16,dl_src=fa:16:3e:62:81:7a actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20369.329s, table=25, n_packets=873, n_bytes=1256778, idle_age=3969, priority=2,in_port=3,dl_src=fa:16:3e:c7:90:50 actions=NORMAL
 cookie=0x953fea9b637ac6b5, duration=20368.830s, table=25, n_packets=1491, n_bytes=1289993, idle_age=105, priority=2,in_port=12,dl_src=fa:16:3e:9e:f7:c6 actions=NORMAL

Output from ovs-ofctl dumpflows br-tun

NXST_FLOW reply (xid=0x4):
 cookie=0xaf522f8bd636f703, duration=20385.783s, table=0, n_packets=1231, n_bytes=213080, idle_age=70, priority=1,in_port=1 actions=resubmit(,2)
 cookie=0xaf522f8bd636f703, duration=20384.081s, table=0, n_packets=0, n_bytes=0, idle_age=20384, priority=1,in_port=2 actions=resubmit(,4)
 cookie=0xaf522f8bd636f703, duration=20383.862s, table=0, n_packets=0, n_bytes=0, idle_age=20383, priority=1,in_port=3 actions=resubmit(,4)
 cookie=0xaf522f8bd636f703, duration=20383.643s, table=0, n_packets=0, n_bytes=0, idle_age=20383, priority=1,in_port=4 actions=resubmit(,4)
 cookie=0xaf522f8bd636f703, duration=20385.782s, table=0, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=drop
 cookie=0xaf522f8bd636f703, duration=20385.779s, table=2, n_packets=0, n_bytes=0, idle_age=20385, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20)
 cookie=0xaf522f8bd636f703, duration=20385.777s, table=2, n_packets=1231, n_bytes=213080, idle_age=70, priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,22)
 cookie=0xaf522f8bd636f703, duration=20385.777s, table=3, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=drop
 cookie=0xaf522f8bd636f703, duration=20381.923s, table=4, n_packets=0, n_bytes=0, idle_age=20381, priority=1,tun_id=0x1d actions=mod_vlan_vid:1,resubmit(,10)
 cookie=0xaf522f8bd636f703, duration=20385.776s, table=4, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=drop
 cookie=0xaf522f8bd636f703, duration=20385.775s, table=6, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=drop
 cookie=0xaf522f8bd636f703, duration=20385.774s, table=10, n_packets=0, n_bytes=0, idle_age=20385, priority=1 actions=learn(table=20,hard_timeout=300,priority=1,cookie=0xaf522f8bd636f703,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1
 cookie=0xaf522f8bd636f703, duration=20385.773s, table=20, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=resubmit(,22)
 cookie=0xaf522f8bd636f703, duration=20381.987s, table=22, n_packets=1231, n_bytes=213080, idle_age=70, dl_vlan=1 actions=strip_vlan,set_tunnel:0x1d,output:2,output:4,output:3
 cookie=0xaf522f8bd636f703, duration=20385.706s, table=22, n_packets=0, n_bytes=0, idle_age=20385, priority=0 actions=drop

Output from ovs-ofctl dump-flows br-ex

NXST_FLOW reply (xid=0x4):

The ovs-vswitchd.log

    2017-02-24T12:57:07.407Z|00001|vlog|INFO|opened log file c:\Program Files\Cloudbase Solutions\Open vSwitch\logs\ovs-vswitchd.log
    2017-02-24T12:57:07.422Z|00002|ovs_numa|INFO|Discovered 0 NUMA nodes and 0 CPU cores
    2017-02-24T12:57:07.423Z|00003|reconnect|INFO|unix:C:\ProgramData\openvswitch\db.sock: connecting...
    2017-02-24T12:57:07.424Z|00004|reconnect|INFO|unix:C:\ProgramData\openvswitch\db.sock: connected
    2017-02-24T12:57:07.454Z|00005|ofproto_dpif|INFO|system@ovs-system: Datapath supports recirculation
    2017-02-24T12:57:07.455Z|00006|ofproto_dpif|INFO|system@ovs-system: MPLS label stack length probed as 3
    2017-02-24T12:57:07.456Z|00007|ofproto_dpif|INFO|system@ovs-system: datapath does not support masked set action feature.
    2017-02-24T12:57:07.456Z|00008|ofproto_dpif|INFO|system@ovs-system: Datapath supports truncate action
    2017-02-24T12:57:07.456Z|00009|ofproto_dpif|INFO|system@ovs-system: Datapath does not support unique flow ids
    2017-02-24T12:57:07.456Z|00010|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_state
    2017-02-24T12:57:07.456Z|00011|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_zone
    2017-02-24T12:57:07.457Z|00012|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_mark
    2017-02-24T12:57:07.457Z|00013|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_label
    2017-02-24T12:57:07.457Z|00014|ofproto_dpif|INFO|system@ovs-system: Datapath does not support ct_state_nat
    2017-02-24T12:57:07.461Z|00015|ofproto_dpif|WARN|system@ovs-system: variable-length userdata feature probe failed (Unknown error)
    2017-02-24T12:57:07.474Z|00016|bridge|INFO|bridge br-int: added interface 34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b on port 16
    2017-02-24T12:57:07.475Z|00017|bridge|INFO|bridge br-int: added interface br-int on port 65534
    2017-02-24T12:57:07.475Z|00018|bridge|INFO|bridge br-int: added interface int-br-ex on port 1
    2017-02-24T12:57:07.475Z|00019|bridge|INFO|bridge br-int: added interface 84a9315b-4b33-4185-9a98-55b10842569e on port 12
    2017-02-24T12:57:07.475Z|00020|bridge|INFO|bridge br-int: added interface 9f422822-edca-4d41-8b2b-a39b6f2c9383 on port 3
    2017-02-24T12:57:07.476Z|00021|bridge|INFO|bridge br-int: added interface patch-tun on port 2
    2017-02-24T12:57:07.476Z|00022|bridge|INFO|bridge br-ex: added interface br-ex on port 65534
    2017-02-24T12:57:07.476Z|00023|bridge|INFO|bridge br-ex: added interface phy-br-ex on port 2
    2017-02-24T12:57:07.476Z|00024|bridge|INFO|bridge br-ex: added interface Tenant on port 1
    2017-02-24T12:57:07.478Z|00025|bridge|INFO|bridge br-tun: added interface vxlan-0a3c0b15 on port 2
    2017-02-24T12:57:07.478Z|00026|bridge|INFO|bridge br-tun: added interface patch-int on port 1
    2017-02-24T12:57:07.478Z|00027|bridge|INFO|bridge br-tun: added interface vxlan-0a3c0b16 on port 3
    2017-02-24T12:57:07.478Z|00028|bridge|INFO|bridge br-tun: added interface vxlan-0a3c0b17 on port 4
    2017-02-24T12:57:07.479Z|00029|bridge|INFO|bridge br-tun: added interface br-tun on port 65534
    2017-02-24T12:57:07.479Z|00030|bridge|INFO|bridge br-int: using datapath ID 000056ba7b1ba641
    2017-02-24T12:57:07.480Z|00031|connmgr|INFO|br-int: added service controller "punix:c:\ProgramData\openvswitch\/br-int.mgmt"
    2017-02-24T12:57:07.493Z|00032|bridge|INFO|bridge br-ex: using datapath ID 0000ac162d70a9d1
    2017-02-24T12:57:07.493Z|00033|connmgr|INFO|br-ex: added service controller "punix:c:\ProgramData\openvswitch\/br-ex.mgmt"
    2017-02-24T12:57:07.505Z|00034|bridge|INFO|bridge br-tun: using datapath ID 0000ba4702014949
    2017-02-24T12:57:07.505Z|00035|connmgr|INFO|br-tun: added service controller "punix:c:\ProgramData\openvswitch\/br-tun.mgmt"
    2017-02-24T12:57:07.524Z|00036|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.6.1.0
    2017-02-24T12:57:17.216Z|00037|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:17.277Z|00038|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:17.340Z|00039|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:17.430Z|00040|memory|INFO|10192 kB peak resident set size after 10.0 seconds
    2017-02-24T12:57:17.431Z|00041|memory|INFO|handlers:17 ports:14 revalidators:7 rules:15 udpif keys:2
    2017-02-24T12:57:17.902Z|00042|connmgr|INFO|br-ex<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:18.277Z|00043|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:18.340Z|00044|connmgr|INFO|br-ex<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:19.223Z|00045|connmgr|INFO|br-tun<->unix: 9 flow_mods in the last 0 s (9 adds)
    2017-02-24T12:57:19.293Z|00046|connmgr|INFO|br-tun<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:21.823Z|00047|connmgr|INFO|br-tun<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:22.073Z|00048|connmgr|INFO|br-tun<->unix: 1 flow_mods in the last 0 s (1 adds)
    2017-02-24T12:57:22.324Z|00049|connmgr|INFO|br-tun<->unix: 1 flow_mods in the last 0 s (1 adds)

and the neutron-ovs-agent.log

2017-02-24 13:57:16.726 8524 INFO neutron.common.config [-] Logging enabled!
2017-02-24 13:57:16.726 8524 INFO neutron.common.config [-] c:\Program Files\Cloudbase Solutions\OpenStack\Nova\Python\Scripts\neutron-openvswitch-agent version 9.0.1.dev14
2017-02-24 13:57:17.492 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-143b7ba8-7253-4d83-8cc6-d611cff6d65d - - - - -] Mapping physical network physnet1 to bridge br-ex
2017-02-24 13:57:19.164 8524 INFO neutron.agent.agent_extensions_manager [req-143b7ba8-7253-4d83-8cc6-d611cff6d65d - - - - -] Loaded agent extensions: []
2017-02-24 13:57:20.945 8524 INFO oslo.messaging._drivers.impl_rabbit [-] [ccd7f232-990e-4bdb-a9ca-6cd292881c76] Reconnected to AMQP server on 10.60.11.22:5672 via [amqp] client with port 51875.
2017-02-24 13:57:20.961 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-143b7ba8-7253-4d83-8cc6-d611cff6d65d - - - - -] Agent has just been revived. Doing a full sync.
2017-02-24 13:57:21.476 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Agent initialized successfully, now running... 
2017-02-24 13:57:21.476 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] rpc_loop doing a full sync.
2017-02-24 13:57:22.336 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Agent out of sync with plugin!
2017-02-24 13:57:23.758 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Port 9f422822-edca-4d41-8b2b-a39b6f2c9383 updated. Details: {'port_security_enabled': True, 'device': '9f422822-edca-4d41-8b2b-a39b6f2c9383', 'network_qos_policy_id': None, 'admin_state_up': True, 'port_id': '9f422822-edca-4d41-8b2b-a39b6f2c9383', 'mac_address': 'fa:16:3e:c7:90:50', 'network_type': 'vxlan', 'security_groups': ['2e4bd637-ecaa-4d27-9291-7900fabe1ac1'], 'qos_policy_id': None, 'network_id': 'c3e0d3b2-86de-4c14-b1d0-e71398e6c0b8', 'fixed_ips': [{'ip_address': '192.168.0.7', 'subnet_id': '6bb7c5dc-bc6c-4751-920a-664db1ca4165'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'physical_network': None, 'segmentation_id': 29, 'profile': {}}
2017-02-24 13:57:23.758 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Assigning 1 as local vlan for net-id=c3e0d3b2-86de-4c14-b1d0-e71398e6c0b8
2017-02-24 13:57:24.039 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Port 34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b updated. Details: {'port_security_enabled': True, 'device': '34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b', 'network_qos_policy_id': None, 'admin_state_up': True, 'port_id': '34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b', 'mac_address': 'fa:16:3e:62:81:7a', 'network_type': 'vxlan', 'security_groups': ['2e4bd637-ecaa-4d27-9291-7900fabe1ac1'], 'qos_policy_id': None, 'network_id': 'c3e0d3b2-86de-4c14-b1d0-e71398e6c0b8', 'fixed_ips': [{'ip_address': '192.168.0.13', 'subnet_id': '6bb7c5dc-bc6c-4751-920a-664db1ca4165'}], 'device_owner': 'compute:None', 'allowed_address_pairs': [], 'physical_network': None, 'segmentation_id': 29, 'profile': {}}
2017-02-24 13:57:24.226 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Port 84a9315b-4b33-4185-9a98-55b10842569e updated. Details: {'port_security_enabled': True, 'device': '84a9315b-4b33-4185-9a98-55b10842569e', 'network_qos_policy_id': None, 'admin_state_up': True, 'port_id': '84a9315b-4b33-4185-9a98-55b10842569e', 'mac_address': 'fa:16:3e:9e:f7:c6', 'network_type': 'vxlan', 'security_groups': ['2e4bd637-ecaa-4d27-9291-7900fabe1ac1'], 'qos_policy_id': None, 'network_id': 'c3e0d3b2-86de-4c14-b1d0-e71398e6c0b8', 'fixed_ips': [{'ip_address': '192.168.0.12', 'subnet_id': '6bb7c5dc-bc6c-4751-920a-664db1ca4165'}], 'device_owner': 'compute:None', 'allowed_address_pairs': [], 'physical_network': None, 'segmentation_id': 29, 'profile': {}}
2017-02-24 13:57:24.445 8524 INFO neutron.agent.securitygroups_rpc [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Skipping method prepare_devices_filter as firewall is disabled or configured as NoopFirewallDriver.
2017-02-24 13:57:26.273 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Configuration for devices up ['9f422822-edca-4d41-8b2b-a39b6f2c9383', '34cecd4b-c5cc-405a-af7e-c0a3d80f3e0b', '84a9315b-4b33-4185-9a98-55b10842569e'] and devices down [] completed.
2017-02-24 13:57:26.289 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Cleaning stale br-int flows
2017-02-24 13:57:26.367 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Cleaning stale br-ex flows
2017-02-24 13:57:26.429 8524 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-0c92e8a7-46c1-43b9-b30f-cc62986f9a8b - - - - -] Cleaning stale br-tun flows

Any ideas are appreciated since Openstack and Open vSwitch are pretty new to me.