Cinder local loopback

asked 2017-03-31 20:32:04 +0300

jrack
18 ●7 ●10 ●12

I am using cinder AZs to expose local disk for volume services running on a Hyper-V 2016 instance. Using the mitaka cinder driver it can do the local detect (for S2D efficiency) which works most of the time. But after a while I will start getting what I would imagine is a token expiration that breaks attaching volumes to instances.

2017-03-31 13:07:02.532 3436 ERROR cinder.volume.drivers.remotefs [req-9695f713-81a8-4b0e-b04f-1b9d26f9bf2e - - - - -] Exception during mounting Unable to mount SMBFS share: \localhost\Cinder$ WMI exception: <x_wmi: a="" specified="" logon="" session="" does="" not="" exist.="" it="" may="" already="" have="" been="" terminated.="" ><="" p="">

I am betting I just need to do a more precise configuration in my New-SmbShare with perms or something, but wasn't sure. And HV server is a bit of a black box so harder to track down might might be going on. As a note volume creation and nova ops all work, but that attachment is what is blowing up.

edit retag flag offensive close merge delete

add a comment

answered 2017-04-04 10:52:46 +0300

lpetrut
206 ●3 ●3

I think that happens when it tries to cleanup a failed mount. Could you please include the whole trace? Thank you.

I guess you're using CSVs, right? We're usually placing scale-out shares on top of them, simplifying share address resolution. I'd suggest using share locations such as //<sofs-name>/<volumes_share>. Those will be resolved to their local mount points.

Lucian

edit flag offensive delete link

Comments

Nope just a local SMB share configured as //localhost/Cinder$ no CSV or anything complex. Just trying to use the local storage in the box. I will flip on debug as the local cinder-volume-smb just contains the above over and over.

jrack ( 2017-04-04 15:11:22 +0300 )edit

Flipped on verbose in cinder-volume-smb.conf and nothing really enlightening. http://paste.openstack.org/show/605417/

jrack ( 2017-04-04 17:53:48 +0300 )edit

And added a debug=true statement to the conf and oddly I did bsod the box, but after booting still getting the context error. http://paste.openstack.org/show/605425/

jrack ( 2017-04-04 18:12:28 +0300 )edit

I mean honestly I don't even want to use the SMB share which I thought the mitaka driver handled internally to make the S2D lookup try local first.

jrack ( 2017-04-04 18:25:59 +0300 )edit

As a workaround for now, just set the actual hostname instead of 'localhost'. This should let the driver pick it up as a local share (which is simply resolving addresses). I'm going to send a fix ASAP, sorry for the inconvenience. Let me know how it went.

lpetrut ( 2017-04-05 13:28:25 +0300 )edit

I had the same thought yesterday, but it didn't resolve the core issue with the attach. I am not getting the WMI error so that is good. Likely in my TP5 test I was using hostname, but used localhost in the deploy script thinking it wouldn't matter. http://paste.openstack.org/show/605513/

jrack ( 2017-04-05 17:14:29 +0300 )edit

So I figured out one err... using gen1s and you can't attach when powered on. so I need to handle that better as the smbfs driver seems to just do a terminate with no error. After powering down though I get the same wmi session err even w \\hostname\share$

jrack ( 2017-04-05 22:20:42 +0300 )edit

So the issue is on the Nova side. Could you please check the logs and paste them here? What Nova MSI are you using? You can in fact attach disks to powered on Gen1 VMS. It's just that you can't attach/detach IDE disks in this case (we're using the SCSI bus in this case, except for BFV).

lpetrut ( 2017-04-06 08:31:29 +0300 )edit

Not sure on that, but it's the mitaka release? I had tested using hv manager to try and attach the vmdk and that's where I noticed it would fail if the g1 vm's were on. I know I need to set the metadata for g2. I'd there any more granular debugging in cinder?

jrack ( 2017-04-06 14:49:33 +0300 )edit

On pto till next week, but we'll try to grab versions and any Nova logs that corelate to the attach events when I am back in the office.

jrack ( 2017-04-06 14:51:17 +0300 )edit

So, at this point, everything works fine on the Cinder side, there's nothing else to debug there. When Nova fails to attach a disk, it will request Cinder to terminate the volume connection, which is expected.

lpetrut ( 2017-04-06 15:11:59 +0300 )edit

Ahh I follow now. Honestly didn't even think of checking Nova since it was all volume work, but makes more sense now. We'll try to remote in and take a look.

jrack ( 2017-04-06 15:13:56 +0300 )edit

You can issue the following powershell command to fetch the exact Nova product version: http://paste.openstack.org/raw/605618/ The idea is that from time to time we release updated MSIs for stable releases.

lpetrut ( 2017-04-06 15:14:53 +0300 )edit

Man feel like a dunce now as there is some solid info in there. http://paste.openstack.org/show/sQzF5Fox30oXmi3VMaOe/ including the session expire error. I don't have an A record for this lab and it is not a domain member, but maybe just create a hosts entry if the hostname isn't cutting it?

jrack ( 2017-04-06 15:28:25 +0300 )edit

And compute version 13.0.0.0

jrack ( 2017-04-06 15:28:55 +0300 )edit

@lpetrut any additional thoughts on this one?

jrack ( 2017-04-10 19:50:03 +0300 )edit

see more comments

Cinder local loopback

1 answer

Comments

Your Answer

Question Tools

Stats

Cinder local loopback edit

1 answer

Comments

Your Answer

Question Tools

Stats

Cinder local loopback