09-30-2015 05:34 AM
Hi
i have a scenario include Bladecenter -->nexus 4001--->nexus N5K96UP--->MDS 9148--Storage
I am seeing a very high "rx Pause" on the Nexus 4001 side portchannel connected to nexus 5000. Here is the output:
SW-4001i(config)# sh int port-channel 10
port-channel10 is up
Hardware: Port-Channel, address: c08c.609c.269a (bia c08c.609c.269a)
MTU 1500 bytes, BW 40000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is trunk
full-duplex, 10 Gb/s
Input flow-control is off, output flow-control is off
Switchport monitor is off
Members in this channel: Eth1/15, Eth1/16, Eth1/17, Eth1/18
Last clearing of "show interface" counters never
1 minute input rate 35640 bits/sec, 0 packets/sec
1 minute output rate 188496 bits/sec, 0 packets/sec
Rx
6353997 input packets 6333875 unicast packets 11048 multicast packets
310 broadcast packets 4237344 jumbo packets 0 storm suppression packets
9153613971 bytes
41648 Rx pause
0 input discard
Tx
15469076 output packets 3773 multicast packets
686 broadcast packets 14714882 jumbo packets
31221537507 bytes
0 input error 0 short frame 0 watchdog
0 no buffer 0 runt 0 CRC 0 ecc
0 overrun 0 underrun 0 ignored 0 bad etype drop
0 bad proto drop 0 if down drop 0 input with dribble
0 output error 0 collision 0 deferred
0 late collision 0 lost carrier 0 no carrier
0 babble
928 Tx pause
0 interface resets
_______________________________
SW-4001i(config)# sh interface port-channel 10 counters detailed
port-channel10
Rx Packets: 6356736
Rx Unicast Packets: 6336244
Rx Multicast Packets: 11421
Rx Broadcast Packets: 317
Rx Jumbo Packets: 4237563
Rx Bytes: 9154435142
Rx Packets from 0 to 64 bytes: 7007
Rx Packets from 65 to 127 bytes: 2086004
Rx Packets from 128 to 255 bytes: 9270
Rx Packets from 256 to 511 bytes: 6962
Rx Packets from 512 to 1023 bytes: 5981
Rx Packets from 1024 to 1518 bytes: 364
Rx Packets from 1519 to 1548 bytes: 42
Rx Trunk Packets: 6341124
Tx Packets: 15472247
Tx Unicast Packets: 15458153
Tx Multicast Packets: 3878
Tx Broadcast Packets: 694
Tx Jumbo Packets: 14715681
Tx Bytes: 31223603330
Tx Packets from 0 to 64 bytes: 90144
Tx Packets from 65 to 127 bytes: 745823
Tx Packets from 128 to 255 bytes: 785839
Tx Packets from 256 to 511 bytes: 78299
Tx Packets from 512 to 1023 bytes: 103157
Tx Packets from 1024 to 1518 bytes: 773482
Tx Packets from 1519 to 1548 bytes: 788476
Tx Trunk Packets: 15459562
Rx Pause: 41648
Tx Pause: 928
SQETest 928
SW-4001i(config)# sh interface flowcontrol
----------------------------------------------------------------------------------------------------
Port Send FlowControl Receive FlowControl RxPause TxPause
admin oper admin oper
----------------------------------------------------------------------------------------------------
Eth1/1 off off off off 0 278
Eth1/2 off off off off 0 278
Eth1/3 off off off off 0 278
Eth1/4 off off off off 0 2582
Eth1/5 off off off off 0 278
Eth1/6 off off off off 0 278
Eth1/7 off off off off 0 278
Eth1/8 off off off off 0 278
Eth1/9 off off off off 0 278
Eth1/10 off off off off 0 278
Eth1/11 off off off off 0 278
Eth1/12 off off off off 0 278
Eth1/13 off off off off 0 278
Eth1/14 off off off off 0 278
Eth1/15 off off off off 0 274
Eth1/16 off off off off 0 272
Eth1/17 off off off off 0 278
Eth1/18 off off off off 41648 104
Eth1/19 off off off off 0 0
Eth1/20 off off off off 0 0
Po10 off off off off 41648 928
also you see sh run :
SW-4001i(config)# sh run
version 4.1(2)E1(1n)
feature telnet
feature lacp
feature fip-snooping
username admin password 5 ! role network-admin
username USERID password 5 $1$JY2o30Ad$tUi8chLhL0SAxnFaC6guH/ role network-admin
ip domain-lookup
ip host SW-4001i 192.168.200.1
hostname SW-4001i
policy-map type queuing policy-fcoe-bandwidth
class type queuing 1p7q4t-out-q-default
bandwidth percent 50
class type queuing 1p7q4t-out-pq1
bandwidth percent 7
class type queuing 1p7q4t-out-q2
bandwidth percent 7
class type queuing 1p7q4t-out-q3
bandwidth percent 7
class type queuing 1p7q4t-out-q4
bandwidth percent 7
class type queuing 1p7q4t-out-q5
bandwidth percent 7
class type queuing 1p7q4t-out-q6
bandwidth percent 7
class type queuing 1p7q4t-out-q7
bandwidth percent 7
class-map type network-qos class-fcoe
match cos 3
class-map type network-qos class-non-fcoe
match cos 0-2,4-7
policy-map type network-qos policy-fcoe
class type network-qos class-fcoe
pause no-drop
mtu 2500
class type network-qos class-non-fcoe
system qos
service-policy type network-qos policy-fcoe
service-policy type queuing output policy-fcoe-bandwidth
snmp-server user USERID network-admin auth md5 0xd12480fdd8d0bf8074815b83070e4f7a priv 0xd12480fdd8d0bf8074815b83070e4f7a localizedk
ey
vrf context management
ip route 0.0.0.0/0 10.10.10.2
vlan 1
vlan 10
fip-snooping enable
vlan 485
port-channel load-balance ethernet source-dest-port
ele-fwd pause rate threshold is 1000 pps
interface port-channel10
switchport mode trunk
switchport trunk allowed vlan 1,10,485
priority-flow-control mode on
fip-snooping port-mode fcf
speed 10000
service-policy type queuing output policy-fcoe-bandwidth
interface Ethernet1/1
switchport mode trunk
priority-flow-control mode on
spanning-tree port type edge
speed 10000
interface Ethernet1/2
description balde 2
switchport mode trunk
switchport trunk native vlan 485
priority-flow-control mode on
spanning-tree port type edge
speed 10000
service-policy type queuing output policy-fcoe-bandwidth
interface Ethernet1/3
switchport mode trunk
priority-flow-control mode on
spanning-tree port type edge
speed 10000
interface Ethernet1/4
switchport mode trunk
priority-flow-control mode on
spanning-tree port type edge
speed 10000
interface Ethernet1/5
switchport mode trunk
priority-flow-control mode on
spanning-tree port type edge
speed 10000
interface Ethernet1/6
switchport mode trunk
priority-flow-control mode on
spanning-tree port type edge
speed 10000
interface Ethernet1/7
switchport mode trunk
priority-flow-control mode on
spanning-tree port type edge
speed 10000
interface Ethernet1/8
switchport mode trunk
priority-flow-control mode on
spanning-tree port type edge
speed 10000
interface Ethernet1/9
switchport mode trunk
priority-flow-control mode on
spanning-tree port type edge
speed 10000
interface Ethernet1/10
switchport mode trunk
priority-flow-control mode on
spanning-tree port type edge
speed 10000
interface Ethernet1/11
switchport mode trunk
priority-flow-control mode on
spanning-tree port type edge
speed 10000
interface Ethernet1/12
switchport mode trunk
priority-flow-control mode on
spanning-tree port type edge
speed 10000
interface Ethernet1/13
switchport mode trunk
priority-flow-control mode on
spanning-tree port type edge
speed 10000
interface Ethernet1/14
switchport mode trunk
priority-flow-control mode on
spanning-tree port type edge
speed 10000
interface Ethernet1/15
switchport mode trunk
switchport trunk allowed vlan 1,10,485
priority-flow-control mode on
speed 10000
service-policy type queuing output policy-fcoe-bandwidth
channel-group 10 mode active
interface Ethernet1/16
switchport mode trunk
switchport trunk allowed vlan 1,10,485
priority-flow-control mode on
speed 10000
service-policy type queuing output policy-fcoe-bandwidth
channel-group 10 mode active
interface Ethernet1/17
switchport mode trunk
switchport trunk allowed vlan 1,10,485
priority-flow-control mode on
speed 10000
service-policy type queuing output policy-fcoe-bandwidth
channel-group 10 mode active
interface Ethernet1/18
switchport mode trunk
switchport trunk allowed vlan 1,10,485
priority-flow-control mode on
speed 10000
service-policy type queuing output policy-fcoe-bandwidth
channel-group 10 mode active
interface Ethernet1/19
speed 10000
interface Ethernet1/20
speed 10000
interface mgmt0
ip address 192.168.200.1/24
interface mgmt1
boot kickstart bootflash:/n4000-bk9-kickstart.4.1.2.E1.1n.bin
boot system bootflash:/n4000-bk9.4.1.2.E1.1n.bin
-----------------------------------------------------------------------------------------
N5K96UP-1# sh int port-channel 10
port-channel10 is up
Hardware: Port-Channel, address: 002a.6ae6.7aa8 (bia 002a.6ae6.7aa8)
MTU 1500 bytes, BW 40000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is trunk
full-duplex, 10 Gb/s
Input flow-control is off, output flow-control is off
Switchport monitor is off
EtherType is 0x8100
Members in this channel: Eth1/33, Eth1/34, Eth1/35, Eth1/36
Last clearing of "show interface" counters never
6 interface resets
30 seconds input rate 108904 bits/sec, 13 packets/sec
30 seconds output rate 26016 bits/sec, 11 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 238.02 Kbps, 14 pps; output rate 39.28 Kbps, 11 pps
RX
70222975 unicast packets 30621 multicast packets 4935 broadcast packets
70258531 input packets 137731035809 bytes
64715012 jumbo packets 0 storm suppression bytes
0 runts 0 giants 0 CRC 0 no buffer
0 input error 0 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
TX
30125787 unicast packets 144949 multicast packets 12932 broadcast packets
30283668 output packets 44180775880 bytes
21111539 jumbo packets
0 output error 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause
6 interface resets
------------------------------------------------------------------------------------------
version 7.0(6)N1(1)
feature fcoe
hostname N5K96UP-1
feature telnet
feature udld
feature interface-vlan
feature lacp
feature lldp
feature vtp
clock protocol none
feature fabric access
username admin password 5 $1$LJ5WKtAc$voXI4bFYbC.alKLDI0Xp71 role network-admin
no password strength-check
ip domain-lookup
system qos
service-policy type queuing input fcoe-default-in-policy
service-policy type queuing output fcoe-default-out-policy
service-policy type qos input fcoe-default-in-policy
service-policy type network-qos fcoe-default-nq-policy
slot 1
port 43-48 type fc
snmp-server user admin network-admin auth md5 0x35d624191b34e07e6c4da9079363ea9c priv 0x35d624191b34e07e6c4da9079363ea9c localizedke
y
snmp-server host 172.20.16.171 traps version 2c public udp-port 1163
rmon event 1 log trap public description FATAL(1) owner PMON@FATAL
rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO
ip route 0.0.0.0/0 172.20.16.254
vrf context management
ip route 0.0.0.0/0 172.20.16.254
port-channel load-balance ethernet source-dest-port
fcdomain fcid database
vsan 1 wwn 10:00:40:f2:e9:87:9f:d9 fcid 0x00ca0000 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:aa:81 fcid 0x00ca0020 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:9e:89 fcid 0x00ca0040 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:a7:91 fcid 0x00ca0080 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:9f:dd fcid 0x00ca00a0 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:9e:8d fcid 0x00ca00c0 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:a7:95 fcid 0x00ca0100 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:9c:d1 fcid 0x00ca0120 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:9c:d9 fcid 0x00ca0140 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:9d:31 fcid 0x00ca0160 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:a7:81 fcid 0x00ca0180 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:a7:f1 fcid 0x00ca01a0 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:85:b9 fcid 0x00ca01c0 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:a1:51 fcid 0x00ca0200 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:a9:b1 fcid 0x00ca0220 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:a9:99 fcid 0x00ca0081 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:a8:91 fcid 0x00ca0240 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:85:bd fcid 0x00ca0260 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:9d:35 fcid 0x00ca0280 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:a9:b9 fcid 0x00ca02c0 dynamic
vsan 1 wwn 10:00:40:f2:e9:87:a9:bd fcid 0x00ca0300 dynamic
interface Vlan1
no shutdown
interface Vlan485
no shutdown
ip address 172.20.16.181/24
interface san-port-channel 1
interface port-channel10
switchport mode trunk
priority-flow-control mode on
switchport trunk allowed vlan 1,10,485
interface vfc105
bind mac-address 40F2.E987.9D31
no shutdown
interface vfc106
bind mac-address 40F2.E987.A7F1
no shutdown
interface vfc107
bind mac-address 40F2.E987.9CD1
no shutdown
interface vfc108
bind mac-address 40F2.E987.A9B1
no shutdown
interface vfc109
bind mac-address 40F2.E987.A151
no shutdown
interface vfc110
bind mac-address 40F2.E987.A891
no shutdown
interface vfc111
bind mac-address 40F2.E987.A9B9
no shutdown
interface vfc112
bind mac-address 40F2.E987.A999
no shutdown
interface vfc114
bind mac-address 40F2.E987.9CD9
no shutdown
interface fc1/43
interface fc1/44
interface fc1/45
channel-group 1 force
no shutdown
interface fc1/46
channel-group 1 force
no shutdown
interface fc1/47
channel-group 1 force
no shutdown
interface fc1/48
channel-group 1 force
no shutdown
interface Ethernet1/1
switchport access vlan 485
interface Ethernet1/2
switchport access vlan 485
interface Ethernet1/3
switchport access vlan 485
interface Ethernet1/4
interface Ethernet1/5
switchport access vlan 485
speed 1000
interface Ethernet1/6
switchport mode trunk
interface Ethernet1/7
interface Ethernet1/8
interface Ethernet1/9
interface Ethernet1/33
priority-flow-control mode on
switchport mode trunk
switchport trunk allowed vlan 1,10,485
channel-group 10 mode active
interface Ethernet1/34
priority-flow-control mode on
switchport mode trunk
switchport trunk allowed vlan 1,10,485
channel-group 10 mode active
interface Ethernet1/35
priority-flow-control mode on
switchport mode trunk
switchport trunk allowed vlan 1,10,485
channel-group 10 mode active
interface Ethernet1/36
priority-flow-control mode on
switchport mode trunk
switchport trunk allowed vlan 1,10,485
channel-group 10 mode active
interface Ethernet1/37
interface Ethernet1/42
interface mgmt0
vrf member management
ip address 172.20.16.181/24
line console
line vty
boot kickstart bootflash:/n5000-uk9-kickstart.7.0.6.N1.1.bin
boot system bootflash:/n5000-uk9.7.0.6.N1.1.bin
----------------------------------------------------------------------------------------
also i install 2 VM in the storage and communication between those host are drop about speed 100 M instead 4g .
please help me about this issue.
mant thanks
Solved! Go to Solution.
10-01-2015 09:43 AM
Hi,
I am not very clear on the problem description but will try my best to answer it.
Rx pauses received indicate that the end host connected to po10 is receiving traffic beyond its capacity to handle thereby causing the NIC to send pause frames to the upstream device. This will definitely slow down the data transfer rate. You may want to see if the end host performance specs are not being exceeded in any way.
-Raj
10-01-2015 09:43 AM
Hi,
I am not very clear on the problem description but will try my best to answer it.
Rx pauses received indicate that the end host connected to po10 is receiving traffic beyond its capacity to handle thereby causing the NIC to send pause frames to the upstream device. This will definitely slow down the data transfer rate. You may want to see if the end host performance specs are not being exceeded in any way.
-Raj
10-02-2015 02:01 AM
Hi Raj
thanks for your support
i have a storwize connect to MDS by 4 I/O in the downstream .when I do config Qos for a Pww of a blade to Pww of storwize this low speed (10)M gets over 600 M and stop counter of rx pause in the 4001. I have 14 Blade and I need a solution for automatically for 14 blade .also in port channel between MDS and nexus 5K I did config dedicate mode .in the scenario its better dedicate mode or share mode .and also how do I see congestion on the mds or interfaces ?
many thanks
Mohammad
10-02-2015 06:33 AM
Hi Mohammad,
My expertise on the MDS side is limited.I did discuss your issue with one of the TAC engineers in the MDS team and the recommendation is to have a TAC case open to troubleshooting this further.
Thanks,
Raj
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide