Solved: Re: Packet Loss between VLANS

metuckness · ‎03-03-2014

Hi all,

I have a C2970 Switch and I am having some real bad packet loss. I finally moved some network devices from VLAN1 to VLAN20 and now I am getting very bad packet loss from items outside of VLAN20.

I configured two ports to VLAN20 and the rest are on VLAN1. I am getting very bad packet loss between devices on the two different VLANS, but none between devices on the same VLAN20.

So, devices on port 22, 21, 20, et cetera (VLAN1) pinging devices on port 3 (VLAN20) are getting this:

Reply from 192.168.20.5: bytes=32 time=37ms TTL=63

Reply from 192.168.20.5: bytes=32 time=35ms TTL=63

Reply from 192.168.20.5: bytes=32 time=26ms TTL=63

Reply from 192.168.20.5: bytes=32 time=19ms TTL=63

Reply from 192.168.20.5: bytes=32 time=4ms TTL=63

Reply from 192.168.20.5: bytes=32 time=2ms TTL=63

Reply from 192.168.20.5: bytes=32 time=5ms TTL=63

Reply from 192.168.20.5: bytes=32 time=2ms TTL=63

Request timed out.

Reply from 192.168.20.5: bytes=32 time=3ms TTL=63

Reply from 192.168.20.5: bytes=32 time=2ms TTL=63

Reply from 192.168.20.5: bytes=32 time=3ms TTL=63

Reply from 192.168.20.5: bytes=32 time=2ms TTL=63

Reply from 192.168.20.5: bytes=32 time=3ms TTL=63

Reply from 192.168.20.5: bytes=32 time=2ms TTL=63

Reply from 192.168.20.5: bytes=32 time=10ms TTL=63

Reply from 192.168.20.5: bytes=32 time=15ms TTL=63

Reply from 192.168.20.5: bytes=32 time=26ms TTL=63

Reply from 192.168.20.5: bytes=32 time=18ms TTL=63

Reply from 192.168.20.5: bytes=32 time=3ms TTL=63

Request timed out.

Reply from 192.168.20.5: bytes=32 time=23ms TTL=63

Reply from 192.168.20.5: bytes=32 time=26ms TTL=63

Reply from 192.168.20.5: bytes=32 time=16ms TTL=63

Reply from 192.168.20.5: bytes=32 time=5ms TTL=63

Reply from 192.168.20.5: bytes=32 time=2ms TTL=63

Reply from 192.168.20.5: bytes=32 time=1ms TTL=63

Reply from 192.168.20.5: bytes=32 time=34ms TTL=63

Reply from 192.168.20.5: bytes=32 time=31ms TTL=63

Reply from 192.168.20.5: bytes=32 time=2ms TTL=63

Reply from 192.168.20.5: bytes=32 time=8ms TTL=63

Reply from 192.168.20.5: bytes=32 time=2ms TTL=63

Request timed out.

Reply from 192.168.20.5: bytes=32 time=3ms TTL=63

Reply from 192.168.20.5: bytes=32 time=30ms TTL=63

Reply from 192.168.20.5: bytes=32 time=15ms TTL=63

Reply from 192.168.20.5: bytes=32 time=5ms TTL=63

Reply from 192.168.20.5: bytes=32 time=2ms TTL=63

Reply from 192.168.20.5: bytes=32 time=4ms TTL=63

Reply from 192.168.20.5: bytes=32 time=6ms TTL=63

Reply from 192.168.20.5: bytes=32 time=37ms TTL=63

Reply from 192.168.20.5: bytes=32 time=27ms TTL=63

Devices on port 4 (VLAN20) pinging devices on port 3 (VLAN20) are getting no packet loss.

Here is my switch config, I am hoping someone can find a config issue that would be the cause of this. I have changed cables et cetera to confirm it wasn't a hardware issue (at least on the cable; devices ports).

C2970#sh run

Building configuration...

Current configuration : 5566 bytes

!

! Last configuration change at 09:12:38 MST Mon Mar 3 2014

! NVRAM config last updated at 09:12:49 MST Mon Mar 3 2014

!

version 12.2

no service pad

service timestamps debug uptime

service timestamps log uptime

service password-encryption

!

hostname C2970

!

boot-start-marker

boot-end-marker

!

enable secret REDACTED

!

username REDACTED

aaa new-model

!

aaa session-id common

clock timezone MST -7

system mtu routing 1500

ip subnet-zero

!

ip domain-lookup source-interface GigabitEthernet0/22

ip domain-name internal.int

ip name-server 192.168.1.2

!

password encryption aes

!

crypto pki trustpoint TP-self-signed-1551050880

enrollment selfsigned

subject-name cn=IOS-Self-Signed-Certificate-1551050880

revocation-check none

rsakeypair TP-self-signed-1551050880

!

crypto pki certificate chain TP-self-signed-1551050880

certificate self-signed 01

3082024C 308201B5 A0030201 02020101 300D0609 2A864886 F70D0101 04050030

31312F30 2D060355 04031326 494F532D 53656C66 2D536967 6E65642D 43657274

69666963 6174652D 31353531 30353038 3830301E 170D3933 30333031 30303031

32305A17 0D323030 31303130 30303030 305A3031 312F302D 06035504 03132649

4F532D53 656C662D 5369676E 65642D43 65727469 66696361 74652D31 35353130

35303838 3030819F 300D0609 2A864886 F70D0101 01050003 818D0030 81890281

8100DFD8 D9E0510E 6B01916F 1FB2E137 DD4589A9 B0811694 C0547F4D 4A7F7F59

37D96387 C2A59577 4F6DA108 5B1BC0EA F52EB656 4EB0FE60 1941308D 4F9B3AD9

257190AA E7574BD7 BF45EEEF EF8FEF00 A4028694 2AE22AB7 B76E9AA3 D16278E8

2A757A06 D58B1E4E 7A441369 304BE9CE E1513096 4C258796 3863757E 76C042E8

39ED0203 010001A3 74307230 0F060355 1D130101 FF040530 030101FF 301F0603

551D1104 18301682 14433239 37302E6D 616C6164 6F6D696E 692E696E 74301F06

03551D23 00000000 80140D57 3D3AAFCA B4A0BB44 8E48207F 7FF9099A C0F1301D

0603551D 0E041604 140D573D 3AAFCAB4 A0BB448E 48207F7F F9099AC0 F1300D06

092A8648 86F70D01 01040500 03818100 53AB91E3 DD31CE2D 78523C46 D92E41D1

C9C1F831 21CBD887 7F429BF5 E092B794 718E93B0 304EE2DD 110DD53B 278E6081

FCE47A4C CE959AD2 02030C82 0182D800 2FC3D5C7 4A1F9248 FDF1BBA7 268BB8BA

B20B95DE C71E784A 3646668A 1006BE67 4F145B9D 408C482D 3638619E CB9CE11A

537F53F9 DE36C1A4 FBCEFAA1 3EA746DB

quit

!

spanning-tree mode pvst

spanning-tree portfast bpduguard default

no spanning-tree optimize bpdu transmission

spanning-tree extend system-id

!

vlan internal allocation policy ascending

!

ip ssh time-out 60

ip ssh authentication-retries 5

ip ssh version 2

!

interface GigabitEthernet0/1

switchport access vlan 10

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/2

switchport access vlan 10

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/3

switchport access vlan 20

switchport mode access

!

interface GigabitEthernet0/4

switchport access vlan 20

switchport mode access

!

interface GigabitEthernet0/5

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/6

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/7

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/8

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/9

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/10

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/11

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/12

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/13

switchport mode access

!

interface GigabitEthernet0/14

switchport mode access

!

interface GigabitEthernet0/15

switchport mode access

!

interface GigabitEthernet0/16

switchport mode access

!

interface GigabitEthernet0/17

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/18

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/19

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/20

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/21

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/22

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/23

switchport mode access

spanning-tree portfast

!

interface GigabitEthernet0/24

description CONNECTION TO 2811 ROUTER - TRUNK

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 1,10,20

switchport mode trunk

spanning-tree portfast

!

interface Vlan1

ip address 192.168.1.3 255.255.255.0

ip helper-address 192.168.1.2

no ip route-cache

!

interface Vlan10

ip address 192.168.10.3 255.255.255.0

ip helper-address 192.168.1.2

no ip route-cache

!

interface Vlan20

ip address 192.168.20.1 255.255.255.0

ip helper-address 192.168.1.2

no ip route-cache

!

ip default-gateway 10.10.1.2

no ip http server

ip http authentication local

ip http secure-server

access-list 1 permit any

!

control-plane

!

line con 0

exec-timeout 0 0

password REDACTED

transport preferred ssh

line vty 0 4

exec-timeout 0 0

password REDACTED

transport input ssh

line vty 5 15

exec-timeout 0 0

password REDACTED

transport input ssh

!

ntp clock-period 36029640

ntp source GigabitEthernet0/22

ntp server 192.168.1.1

ntp peer 192.168.1.2

ntp server 96.226.242.9

end

Jeff Van Houten · ‎03-04-2014

No there is a router connected to port 24. Do you have sub interfaces configured on that router? If so, what are the ip addresses? You cannot have the same ip addresses configured on the switch svis as you do on the router sub-interfaces.

Sent from Cisco Technical Support iPad App

View solution in original post

Jeff Van Houten · ‎03-03-2014

Do you have sub-interfaces defined on the router port? If so, do you have the same addresses as on those svi interfaces?

Sent from Cisco Technical Support iPad App

metuckness · ‎03-04-2014

Yes:

!

interface Vlan1

ip address 192.168.1.3 255.255.255.0

ip helper-address 192.168.1.2

no ip route-cache

!

interface Vlan10

ip address 192.168.10.3 255.255.255.0

ip helper-address 192.168.1.2

no ip route-cache

!

interface Vlan20

ip address 192.168.20.1 255.255.255.0

ip helper-address 192.168.1.2

no ip route-cache

Jeff Van Houten · ‎03-04-2014

No there is a router connected to port 24. Do you have sub interfaces configured on that router? If so, what are the ip addresses? You cannot have the same ip addresses configured on the switch svis as you do on the router sub-interfaces.

Sent from Cisco Technical Support iPad App

glen.grant · ‎03-04-2014

If the router is doing the routing for those vlans you don't need all those SVI's configured on the 2970. You only need a single SVI defined to manage the switch along with the correct default gateway address.

metuckness · ‎03-04-2014

The router has them as well, but I always give routers a .1 IP and switches a .3 address:

interface FastEthernet0/0

description CONNECTION TO INSIDE INT. OF ASA

ip address 10.10.1.2 255.255.255.252

ip virtual-reassembly in

duplex auto

speed auto

!

interface FastEthernet0/1

no ip address

ip virtual-reassembly in

duplex auto

speed auto

!

interface FastEthernet0/1.1

description VLAN 10

encapsulation dot1Q 10

ip address 192.168.10.1 255.255.255.0

ip helper-address 192.168.1.2

ip virtual-reassembly in

!

interface FastEthernet0/1.2

description VLAN 20

encapsulation dot1Q 20

ip address 192.168.20.1 255.255.255.0

ip helper-address 192.168.1.2

ip virtual-reassembly in

!

interface FastEthernet0/1.3

description Trunk Interface VLAN 1

encapsulation dot1Q 1 native

ip address 192.168.1.1 255.255.255.0

ip helper-address 192.168.1.2

ip virtual-reassembly in

!

ip default-gateway 10.10.1.1

metuckness · ‎03-04-2014

I am beginning to think it is a VLAN issues on the wireless AP that is attached to the ports. I have a WRT300N running DD-WRT that I use as an Access Point for the wireless devices that I wun on the VLAN20 and it was the AP and the devices behind the AP that were having issues.

I did some testing last night and once I moved the AP back to VLAN1 it worked, no packet loss. So, I think I have to try and figure out how to configure the VLANS on the DD-WRT router. Which is going to be a nightmare because they are nothing like Ciscos AND I just read that they don't go up to VLANS higher than 15. So I am going to have to put the wireless network on VLAN10 or make a new one.

http://www.dd-wrt.com/wiki/index.php/Switched_Ports

Thank you all for checking, but I think that is my problem, it's not tagging the packets that it is passing properly, hence the constant and severe packet loss.

I have asked on the DD-WRT forums if they think that is what it could be and how I might configure the VLANS for VLAN10.

Jeff Van Houten · ‎03-04-2014

Thank you for pulling what I like to refer to as a "Paul Harvey".

Sent from Cisco Technical Support iPad App

metuckness · ‎03-04-2014

LOL, now I have to figure out what a Paul Harvey is!

Really, i didn't think that changing the VLAn the AP was on would be impacted by VLANS, honestly! I was up until 3Am trying to figure this out gimme a little break!

I do appreciate the help. I honestly thought it was something wrong with the switch. At first I thought it was a bad port, then cable, then config, then I started looking at the AP...

Jeff Van Houten · ‎03-04-2014

I guess I'm giving away my age but In the U.S. For decades there was a syndicated radio host named Paul Harvey. He was famous for telling stories that lead up to a dramatic conclusion that was generally not discernible until the last sentence or two. He would always conclude the lesson of the day with, "And now you know the rest of the story".

Sent from Cisco Technical Support iPad App

metuckness · ‎03-04-2014

Haha. yeah I Googled him. I had never listened to him, but I did hear his famous speech about social and moral decay and the Devil; which I thought was pretty accurate.

I really am sorry, but if you do read that link and are able to tell me how to create and assign VLAN10 to a port using that DD-WRT software that would be fantastic! . I know it's not Cisco related, but I just learned how to setup Cisco devices and then I find something totally different! And their GUI doesn't work, so it all has to be done command line, which is fine, except I don't normally use CLI on DD-WRT devices; their GUI always worked well enough.

http://www.dd-wrt.com/wiki/index.php/Switched_Ports

The router has Gig ports, so it would follow the Gig examples they show.

But anyway, The only reason I decided that was the issues was because I put a laptop on port 4 which was also in the VLAN20 and it never had packet loss, but the AP and devices behind it did. Even though sometimes the AP wouldn't respond, but I could still get to devices behind it.

Thanks all!

Mitch

Jeff Van Houten · ‎03-04-2014

Vlan tagging occurs at the switch as traffic is received through the port. If you have an access vlan assigned at the port theres no reason to try to get the ap to tag the packets. I would suggest you are seeing a problem with the ap or the wireless devices behind the ap.

Sent from Cisco Technical Support iPad App

metuckness · ‎03-04-2014

I thought that at first as well, but they work perfect when they are on VLAN1. No packet loss, no drops, nothing. Absolutely flawless like they are supposed to. but i move it to another VLAN and then they start dropping packets.

And it is weird. The AP will stop responding to pings, but the AP devices will continue. Then sometimes all stop, and visa-versa.

But put them on VLAN1 ports and they work fine.

Jeff Van Houten · ‎03-04-2014

I don't have any experience with dd-wrt hacked aps, but I have seen something similar. Years ago I ran across some dell laptops that wouldn't communicate with a Cisco switch on anything other than vlan 1. Updated drivers alleviated most of the problem there. Right now we have issues with the administrative interface on some video DVR devices. They work fine on the cheapest dlink switch around, but absolutely will not work on a Cisco switch port. I suspect drivers there as well.

You might try to turn off dtp with "switchport nonegotiate" pointing to the ap and see if that gets you somewhere.

Sent from Cisco Technical Support iPad App

metuckness · ‎03-04-2014

I will try that tonight and see if it helps. I am also going to mess around with the tagging on the AP and see if I can figure it out, it's the only logical thing that can be the cause of the packet loss that I can think of considering how it has acted.

DD-WRT is great for the routers they have versions for. It adds tons of features the default firmware doesn't do, including VPN's, NAS support, iptables, SSH et cetera.

It's just figuring out how they work

It's so good and populare, Buffalo has came out with a few models that ship with DD-WRT already on them. They just annouced it.

http://www.buffalotech.com/about-buffalo/news-and-press/press-releases/buffalo-introduces-open-source-dd-wrt-wireless-networking-solutions

http://smile.amazon.com/gp/product/B00I2N6O0W/ref=oh_details_o01_s00_i00?ie=UTF8&psc=1