3750G interVLAN routing slow

jasonrothwell · ‎05-07-2018

Hello,

I have a Catalyst 3750G 48 Port switch with several routed VLANs. No QOS, no ACLs being applied. Traffic in the same VLAN is as fast as expected, but traffic being routed between VLANs is noticeably slower. For example, VLAN 10 has the edge router (internet gateway) and if I plug a computer into VLAN 24 and run an internet speed test, I'm getting around 14Mbps; if I plug the computer directly into a switchport on VLAN 10 and run the internet speed test, I get around 90Mbps.

Any recommendations?

Thanks

CISCO3750#show run
Building configuration...

Current configuration : 11456 bytes
!
version 12.2
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
service sequence-numbers
!
hostname CISCO3750
!
boot-start-marker
boot-end-marker
!
enable secret
enable password
!
username password 0 priv 15
no aaa new-model
clock timezone EST -5 0
clock summer-time EDT recurring
switch 1 provision ws-c3750g-48ps
system mtu routing 1500
ip routing
ip domain-name
!
!
!
mls qos map cos-dscp 0 8 16 24 32 46 48 56
mls qos srr-queue input bandwidth 70 30
mls qos srr-queue input threshold 1 80 90
mls qos srr-queue input priority-queue 2 bandwidth 30
mls qos srr-queue input cos-map queue 1 threshold 2 3
mls qos srr-queue input cos-map queue 1 threshold 3 6 7
mls qos srr-queue input cos-map queue 2 threshold 1 4
mls qos srr-queue input dscp-map queue 1 threshold 2 24
mls qos srr-queue input dscp-map queue 1 threshold 3 48 49 50 51 52 53 54 55
mls qos srr-queue input dscp-map queue 1 threshold 3 56 57 58 59 60 61 62 63
mls qos srr-queue input dscp-map queue 2 threshold 3 32 33 40 41 42 43 44 45
mls qos srr-queue input dscp-map queue 2 threshold 3 46 47
mls qos srr-queue output cos-map queue 1 threshold 3 4 5
mls qos srr-queue output cos-map queue 2 threshold 1 2
mls qos srr-queue output cos-map queue 2 threshold 2 3
mls qos srr-queue output cos-map queue 2 threshold 3 6 7
mls qos srr-queue output cos-map queue 3 threshold 3 0
mls qos srr-queue output cos-map queue 4 threshold 3 1
mls qos srr-queue output dscp-map queue 1 threshold 3 32 33 40 41 42 43 44 45
mls qos srr-queue output dscp-map queue 1 threshold 3 46 47
mls qos srr-queue output dscp-map queue 2 threshold 1 16 17 18 19 20 21 22 23
mls qos srr-queue output dscp-map queue 2 threshold 1 26 27 28 29 30 31 34 35
mls qos srr-queue output dscp-map queue 2 threshold 1 36 37 38 39
mls qos srr-queue output dscp-map queue 2 threshold 2 24
mls qos srr-queue output dscp-map queue 2 threshold 3 48 49 50 51 52 53 54 55
mls qos srr-queue output dscp-map queue 2 threshold 3 56 57 58 59 60 61 62 63
mls qos srr-queue output dscp-map queue 3 threshold 3 0 1 2 3 4 5 6 7
mls qos srr-queue output dscp-map queue 4 threshold 1 8 9 11 13 15
mls qos srr-queue output dscp-map queue 4 threshold 2 10 12 14
mls qos queue-set output 1 threshold 1 100 100 50 200
mls qos queue-set output 1 threshold 2 125 125 100 400
mls qos queue-set output 1 threshold 3 100 100 100 400
mls qos queue-set output 1 threshold 4 60 150 50 200
mls qos queue-set output 1 buffers 15 25 40 20
!
crypto pki trustpoint TP-self-signed-
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-
revocation-check none
rsakeypair TP-self-signed-
!
!
crypto pki certificate chain TP-self-signed-
certificate self-signed 01
quit
auto qos srnd4
!
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
no spanning-tree vlan 1,10,20,22,24,26,28,30,40
!
vlan internal allocation policy ascending
!
!
!
!
!
!
interface GigabitEthernet1/0/1
description EDGE ROUTER
switchport access vlan 10
switchport mode access
!
interface GigabitEthernet1/0/2
description IDRAC
switchport access vlan 10
switchport mode access
!
interface GigabitEthernet1/0/3
description IDRAC
switchport access vlan 20
switchport mode access
!
interface GigabitEthernet1/0/4
description IDRAC
switchport access vlan 20
switchport mode access
!
interface GigabitEthernet1/0/5
description ILO
switchport access vlan 20
switchport mode access
!
interface GigabitEthernet1/0/6
description SOPHOS SERVER - VM
switchport access vlan 22
switchport mode access
!
interface GigabitEthernet1/0/7
description WORKSTATION
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/8
description WORKSTATION
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/9
description WORKSTATION
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/10
description WORKSTATION
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/11
description WORKSTATION
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/12
description WORKSTATION
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/13
description WORKSTATION
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/14
description WORKSTATION
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/15
description WORKSTATION
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/16
description WORKSTATION
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/17
description WORKSTATION
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/18
description WORKSTATION
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/19
description SERVER
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/20
description SERVER
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/21
description SERVER
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/22
description SERVER
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/23
description CISCO VPN EXTERNAL
switchport access vlan 26
switchport mode access
!
interface GigabitEthernet1/0/24
description CISCO VPN INTERNAL
switchport access vlan 26
switchport mode access
!
interface GigabitEthernet1/0/25
description WEB SERVER - VM
switchport access vlan 28
switchport mode access
!
interface GigabitEthernet1/0/26
description EXT DNS SVR - VM
switchport access vlan 28
switchport mode access
!
interface GigabitEthernet1/0/27
description CISCO IP PHONE
switchport access vlan 30
switchport mode access
!
interface GigabitEthernet1/0/28
description CISCO IP PHONE
switchport access vlan 30
switchport mode access
!
interface GigabitEthernet1/0/29
description CISCO IP PHONE
switchport access vlan 30
switchport mode access
!
interface GigabitEthernet1/0/30
description CISCO IP PHONE
switchport access vlan 30
switchport mode access
!
interface GigabitEthernet1/0/31
description CISCO IP PHONE
switchport access vlan 30
switchport mode access
!
interface GigabitEthernet1/0/32
description CISCO IP PHONE
switchport access vlan 30
switchport mode access
!
interface GigabitEthernet1/0/33
description CISCO IP PHONE
switchport access vlan 30
switchport mode access
!
interface GigabitEthernet1/0/34
description CISCO IP PHONE
switchport access vlan 30
switchport mode access
!
interface GigabitEthernet1/0/35
description CISCO IP PHONE
switchport access vlan 30
switchport mode access
!
interface GigabitEthernet1/0/36
description VOICE SERVER
switchport access vlan 30
switchport mode access
!
interface GigabitEthernet1/0/37
description VOICE SERVER
switchport access vlan 30
switchport mode access
!
interface GigabitEthernet1/0/38
description IP CAMERA
switchport access vlan 40
switchport mode access
!
interface GigabitEthernet1/0/39
description IP CAMERA
switchport access vlan 40
switchport mode access
!
interface GigabitEthernet1/0/40
description IP CAMERA
switchport access vlan 40
switchport mode access
!
interface GigabitEthernet1/0/41
description IP CAMERA
switchport access vlan 40
switchport mode access
!
interface GigabitEthernet1/0/42
description IP CAMERA
switchport access vlan 40
switchport mode access
!
interface GigabitEthernet1/0/43
description IP CAMERA
switchport access vlan 40
switchport mode access
!
interface GigabitEthernet1/0/44
description IP CAMERA
switchport access vlan 40
switchport mode access
!
interface GigabitEthernet1/0/45
description IP CAMERA
switchport access vlan 40
switchport mode access
!
interface GigabitEthernet1/0/46
description VIDEO SERVER
switchport access vlan 40
switchport mode access
!
interface GigabitEthernet1/0/47
description VIDEO SERVER
switchport access vlan 40
switchport mode access
!
interface GigabitEthernet1/0/48
description CISCO AP
switchport trunk encapsulation dot1q
switchport trunk native vlan 24
switchport mode trunk
!
interface GigabitEthernet1/0/49
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/50
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/51
switchport access vlan 24
switchport mode access
!
interface GigabitEthernet1/0/52
switchport access vlan 24
switchport mode access
!
interface Vlan1
no ip address
!
interface Vlan10
ip address 10.10.10.1 255.255.255.0
!
interface Vlan20
ip address 10.10.20.1 255.255.255.0
ip helper-address 10.10.24.124
!
interface Vlan22
ip address 10.10.22.1 255.255.255.0
ip helper-address 10.10.24.124
!
interface Vlan24
ip address 10.10.24.1 255.255.255.0
!
interface Vlan26
ip address 10.10.26.1 255.255.255.0
ip helper-address 10.10.24.124
!
interface Vlan28
ip address 10.10.28.1 255.255.255.0
ip helper-address 10.10.24.124
!
interface Vlan30
ip address 10.10.30.1 255.255.255.0
ip helper-address 10.10.24.124
!
interface Vlan40
ip address 10.10.40.1 255.255.255.0
ip helper-address 10.10.24.124
!
ip default-gateway 10.10.10.10
ip http server
ip http secure-server
!
ip route 0.0.0.0 0.0.0.0 10.10.10.10
!
logging esm config
!
!
!
line con 0
password
login
line vty 0 4
password
login
length 0
line vty 5 15
password
login
!
end

jasonrothwell · ‎05-07-2018

jasonrothwell · ‎05-11-2018

Bump

Georg Pauwen · ‎05-11-2018

Hello,

do you have CEF enabled ?

3750(config)#ip cef distributed

jasonrothwell · ‎05-11-2018

@Georg Pauwen wrote:

Hello,

do you have CEF enabled ?

3750(config)#ip cef distributed

Yes, it is.

Georg Pauwen · ‎05-11-2018

Hello,

try:

system mtu routing 1504

Also, remove:

ip default-gateway 10.10.10.10

jasonrothwell · ‎05-12-2018

Thanks. I made those changes yesterday and I’m still seeing the same issue.

Jason

Georg Pauwen · ‎05-12-2018

Hello,

post the output of 'sh interface vlan 10'...

jasonrothwell · ‎05-12-2018

CISCO3750#sh run int vlan 10
Building configuration...

Current configuration : 61 bytes
!
interface Vlan10
ip address 10.10.10.1 255.255.255.0
end

Georg Pauwen · ‎05-12-2018

Hello,

I meant the output of:

CISCO3750#sh interfaces vlan 10

jasonrothwell · ‎05-12-2018

oh, sorry...

CISCO3750#sh interfaces vlan 10
Vlan10 is up, line protocol is up
Hardware is EtherSVI, address is 0024.5133.b941 (bia 0024.5133.b941)
Internet address is 10.10.10.1/24
MTU 1546 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not supported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:06, output 00:00:06, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 16000 bits/sec, 11 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
18945536 packets input, 26268458516 bytes, 0 no buffer
Received 0 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
8978 packets output, 616823 bytes, 0 underruns
0 output errors, 2 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out

Georg Pauwen · ‎05-12-2018

Hello,

no errors on the interface.

Can you post the config of the router ?

jasonrothwell · ‎05-12-2018

It’s a Fortigate 60c. Do you still want the config? If I plug a laptop in VLAN 10, there is no issue. If I plug into any other VLAN, that’s when I see the dramatic speed decrease.

Thanks.

Georg Pauwen · ‎05-12-2018

Hello,

actually, looking at the output you previously posted, the below looks odd:

CISCO3750#sh interfaces vlan 10
Vlan10 is up, line protocol is up
Hardware is EtherSVI, address is 0024.5133.b941 (bia 0024.5133.b941)
Internet address is 10.10.10.1/24
MTU 1546 bytes, BW 1000000 Kbit/sec, DLY 10 usec,

Either way, if MTU 1546 is working, set the MTU on the other VLAN interfaces to that same value, e.g.:

CISCO3750#conf t

CISCO3750(config)#interface vlan 20

CISCO3750(config-if)#mtu 1546

jasonrothwell · ‎05-12-2018

i just ran the command on VLAN24 that was having issues and the MTU is already the same.

CISCO3750#sh interfaces vlan 24
Vlan24 is up, line protocol is up
Hardware is EtherSVI, address is 0024.5133.b944 (bia 0024.5133.b944)
Internet address is 10.10.24.1/24
MTU 1546 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not supported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 4000 bits/sec, 6 packets/sec
5 minute output rate 37000 bits/sec, 33 packets/sec
507061 packets input, 55197343 bytes, 0 no buffer
Received 0 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
18824364 packets output, 26407216069 bytes, 0 underruns
0 output errors, 2 interface resets
30 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out