CSR1000V, VMWARE Workstation Pro, VxVLAN, Jumbo Frames?

[Christopher Smith]

 

Everyone, keen to see if anyone's has this drama too??

- Have four (4) CSR1000v serially connected running on VMWARE Workstation Pro 12.5, as ...

[R1] <--LAN1 --> [R2] <--LAN2 --> [R3] <--LAN3 --> [R4]

- [R1:Gi2] has a vrf bound to subinterface dot1q 100, IP address 10.1.1.253/23

- [R1:Gi2] connects to [R2:Gi2] along subinterface dot1 255 (subnet 10.3.253.0/24), mtu between them updated to 9216

- [R2:Gi1] connects to [R3:Gi1] natively (subnet 10.255.253.0/24), mtu between them updated to 9216,

- [R3:Gi2] connects to [R4:Gi2] along subinterface dot1q 255 (subnet 10.2.253.0/24), mtu between them updated to 9216

- [R4:Gi2] has a vrf bound to subninterface dot1q 100, IP address 10.1.1.254/23

- OSPF (area 0 everywhere except vrf interfaces) for IGP

- VxVLAN:

> "Cisco CSR 1000V VxLAN Support "

@ http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/vxlan/m_cs...

> Followed most of the docs config

> Source Loopbacks on [R2] and [R3]

> A Service Instance 20 legs [R2:Gi2] dot1q 100 to template

> And, mirrored on the other side [G3:Gi2]

Standard pings work :)

(On R1, "ping vrf blah 10.1.1.254")

LARGE pings panics VM :(

(On R1, "ping vrf blah 10.1.1.254 size 1600")

In the VM's VMware.log, there's this panic ...

2017-04-28T07:37:56.517+10:00| vcpu-0| E105: PANIC: VERIFY bora\devices\vmxnet3\vmxnet3_hosted.c:719

2017-04-28T07:37:56.517+10:00| vcpu-0| W115: Win32 object usage: GDI 10, USER 16

My theory is that VMWARE Workstation PRO vmxnet3 NICS per guest ALSO needs an MTU kick / tweak (can do this quite easily with ESXi's via virtual switches), the closet I could find was updating their *.vmx files to accommodate (for example) ethernet2.features = "15" - this apparently enables TSO as well as Jumbo Frames. But still no joy -

Anyone?

Looking forward to 'flames'

Regards,

Christopher

[Philip D'Ath]

This sounds like a VMWare Workstation bug to me.  Nothing should cause a "panic" - except a bug.

[Christopher Smith]

My own RTFM :(

1) At http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/configuration/b_CSR1000v_Configuration_Guide/b_CSR1000v_Configuration_Guide_chapter_00.html#con_1213069

... the PCI passthrough (enic) section, there's blah around ... 

• Jumbo packet support: In this release, jumbo packet (MTU > 1518) is not supported

Not directly related, but an eyebrow raiser.  Then ...

2) At http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/configuration/b_CSR1000v_Configuration_Guide/b_CSR1000v_Configuration_Guide_chapter_01111.html#con_1027695

... there's blah around ...

• Verify that the router is configured for the correct MTU setting.  By default, the maximum MTU setting on the router is set to 1500. To support jumbo frames, you need to edit the default VMware vSwitch settings. For more information, see the VMware vSwitch documentation.

Reading between the lines, tells me the CSR1000v are tested against proper ESXi NOT the likes of Workstation Pro, that if you get it to work in those environments, great - but not supported.  More RTFM ....

http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/configuration/b_CSR1000v_Configuration_Guide/b_CSR1000v_Configuration_Guide_chapter_00.html#con_1081607

• Installation of the Cisco CSR 1000v is supported on selected Type 1 (native, bare metal) hypervisors. Installation is not supported on Type 2 (hosted) hypervisors, such as VMware Fusion, VMware Player, or Virtual Box.

Long story longer, went and begged for some "loaner compute" from the Server Team, spun up the same lab off a ESXi 5.5 Hypervisor, adjusted the relevant vSwitches for 9000 MTUs, booted up these same CSRs after a motion in, and it worked the first time!

Hopefully this have given everyone a fruitful shaking-of-their-head-in-disbelief!

Regards,

Christopher