CSR1000v cannot communicate via dot1q subinterface

jangossens · ‎12-21-2015

Hello community,

I am struggling with problem with some CSR1000v for about 2 weeks now and since I think that I nearly tried and read everything I could on the internet, I will try to find some help here now.

The situation is the following: To prepare myself for the CCIE R&S I am trying to build the CCIE v5 INE lab on my desktop computer using CSR1000v virtual routers. I just downloaded the CSR1000v software (.iso), created some VMs in Virtualbox and imported these VMs into GNS3 to work with them. Everything worked so far and I could start and ping directly connected CSR1000vs succesfully.

However as soon as I use subinterfaces (like they do in the INE lab) the basic connectivity does not work anymore. I connect two routers directly via GigabitEthernet1 and I configure them for example like this:

interface GigabitEthernet1

negotiation auto

no ip address

no shutdown

interface GigabitEthernet1.100

encapsulation dot1q 100

ip address 10.0.0.1 255.255.255.0

no shut

So absolute basic connectivity - but I can not ping the routers like this. If I put an IP address to Gig1 or use the "encapsulation dot1q 100 native" command on the subinterfaces it is working. I googled this problem and saw that lots of other people seem to have the same or similar problems when it comes to dot1q tagging with the CSR1000v series. Some claim that it worked for them after lots of troubleshooting for me it is not.

I thought that maybe some kind of software or virtualization bug may cause my problems, so I started to use different software versions whereever possible.

OS: Windows 8 (fixed)

GNS3: 1.3.10 first, then 1.3.13, currently the 1.4 beta version because if integration of VMware.

Virtualbox: 5.0.11 first, then update to 5.0.12 and currently 5.0.10 (because someone who brought tagged subinterfaces to work said that he used this version). I tried every network interface card possible in Virtualbox as well, some people say that the paravirtualized one worked for them.

CSR1000V: 3.17.00.S, 03.16.01a, 03.15.02, 03.11.02

After using all these software versions in every combination I tried to virtualize the CSR1000V in VMware Workstation Pro but I got the same issues. At this point now I don't think that my problem has anything to do with some kind of virtualization bug because I really changed everything I could and the same problem just occurs again and again...

With wireshark and the "debug XXX" commands in the IOS-XE I tried to troubleshoot the problems and I think I figured out what the actual problem is.

When I connect the CSR1000v into GNS3 to some 7200 dynamips emulated router via a tagged subinterface and I do "debug arp" on both sides while trying to ping, the 7200er router says the following:

*Dec 21 02:50:34.563: IP ARP rep filtered src 17.0.0.6 001e.7ae9.61bf, dst 17.0.0.6 ffff.ffff.ffff wrong cable, interface GigabitEthernet1/0

Which means (I googled this) that the ARP reply is being received on a wrong interface. Why is it the wrong interface? Because the ARP Request is going out of router 7200 with a VLAN tag but the reply is being received untagged (so basically it is going out of interface Gi1.100 but is being received on Gi1 because of the missing tag). I started a capture with wireshark then to have a more detailed look and it seems that my suspicion is correct and the CSR1000v is sending untagged frames out of its subinterface, even though it is configured with "encapsulation dot1q 100" for example.

Now the question is how is this possible? If I would not see lots of people (even the INE people with their lab videos itself) saying that the CSR1000v is working for them, I would say that this is some major bug in the CSR1000 software itself. However as stated above I used lots of different CSR1000v IOS-XEs, I cannot believe that this should really be a simple bug.

Does anyone have an idea why this problem keeps occuring or what I could do to get the subinterfaces work? I spent so many hours the last 2 weeks of troubleshooting and googling and using other software, I do not have an idea of what to do anymore.

PS: Some guy in the GNS3 board was using BDI interfaces instead of subinterfaces, I tried this as well with no success.

Regards

Jan

jangossens · ‎12-22-2015

SOLVED: In another forum I got the answer: GNS3 is at fault. If you use a native VM in VMware Workstation and connect the CSR1000Vs via vmnet5 it works.

nicolas.vallet · ‎08-20-2016

Hello everyone,

I know this post was 8 months ago but the issue was still here.
I didn't want to use WMware Workstation (well i tried but i had to create a crazy amounts of networks interfaces and at the end it wasn't working for me).
I also didn't want to invest in a server with ESXi.
I just wanted to use my Desktop with VirtualBox and GNS3.

After several days searching and testing i finally made the CSRv ping each other using sub-interfaces.
They are connected either with direct back-to-back connection or through a switch.

I don't know if it works with other versions of softwares (Windows, CSRv, GNS3, VirtualBox...) but for sure the setup i found works perfectly:

So here is my setup:

My PC: Windows 10, 24Gig DDR3 RAM, Intel i7-920 2.66GHz (4 cores), 4 Hybrid disks Seagate SSHD SATA3 in RAID10.
VirtualBox version: 5.0.26 r 108824
GNS3 version: 1.5.1
CSRv image: csr1000v-universalk9.03.14.01.S.155-1.S1-std.iso

VirtualBox Setup:
I have right now 7 VMs (one for each CSR). It's the max i can run with my 24Gig Memory with this setup.

Each VM is configured like this:
Operating System: Other Linux (64 bit)
Base Memory: 3072 MB (when i put 2048 MB the CSR loads but soon after i see memory error syslog messages and my interfaces stays down, i didn't try yet a value in between those 2).
Boot Order: i just let Optical first then Hard Drive second (i removed Floppy disk and Network).
Acceleration: KVM (and i've let the 2 boxes checked: Enable VT-x...and Enable Nested Paging)

Network:
For each adapters you want to activate:
Enable Network Adapter (checked)
Attached to: Not attached
Advanced:
Adapter Type: Paravirtualized Network (virtio-net)
There should be a mac address already generated in the field
Don't check the box "Cable Connected".

Serial Ports: none are activated.

GNS3 Setup:
Pay attention to this: GNS3 can modify the memory allocated to the VMs and also manipulate the NICs.

Each GNS3 VMs is linked and controls one of the VirtualBox VM previously created.

If you have both windows opened: GNS3 and VirtualBox Manager (VBM), you can see live the changes you make in GNS3 modifying the VB VMs.
But if you make any modifications directly in VBM it will NOT be updated in GNS3 until you restart it.
GNS3 has to be launch "above" VB i mean "after" any VBM modifications.

When i edit a GNS3 VM i have 2 tabs:
Tab General Settings:
- Category: Routers
- RAM: 3072MB
- Enable remote console (checked)
- Start VM in headless mode (checked)

Tab Network:
Type: Paravirtualized Network (virtio-net)
Allow GNS3 to use any configured VB adapter (checked) -> I noticed if i don't check this box GNS3 can not start the VM.

Finally the pings works also fine through a GNS3 Ethernet switch. I've let the default config which is all ports in access vlan 1.

i hope this post will help people. I explained here the most important options because i could not list the full config.

Regards,
Nicolas.

amurphy636 · ‎12-11-2016

Nicolas....thank you this was very helpful, I think it comes down to IOS Code....I've tried 3.12 and 3.17 w/ no luck but 3.14 worked perfectly. Thanks

ryan-hitch · ‎06-07-2018

I was experiencing this problem with CSR 1000V IOS-XE 16.3.5. Switching the Virtual NIC driver from virtio-net-pci to vmxnet3 corrected the problem.