Nexus 2K 10Gb connectivity issue with Netapp SAN

abbasali5 · ‎07-22-2015

I have a customer who have Nexus 2K 10Gb connected to NetAPP SAN, but they are only getting 10Mb. The statistics also shows a lots of CRC error. Not sure, if there is any need to turn on some additional features on the port or could be a bug on the Nexus OS.

Any advise will be appreciated.

Regards,

-----------------------------------------------------------------------------------------------

Nexus 2K port configuration below:

interface Ethernet103/1/25

description < DR SAN REPLICATION TEMP - NIC N1-e0b >

switchport

switchport access vlan 220

spanning-tree port type edge

no shutdown

netappcluster::> net port show -port e0b -fields node,port,mtu,duplex-admin,duplex-oper,speed-admin,flowcontrol-admin,ipspace

(network port show)

node port mtu duplex-admin duplex-oper speed-admin flowcontrol-admin ipspace

---------------- ---- ---- ------------ ----------- ----------- ----------------- -------

netappcluster-01 e0b 1500 full full 10000 none Default

netappcluster-02 e0b 1500 full full 10000 none Default

2 entries were displayed.

Mark Malone · ‎07-22-2015

Hi crcs are a layer 1 issue are they been seen on Nexus port can you post the output please of the show interface e103/1/25 ?

How may interfaces are connected to the SAN from the Nexus and whats the uplink speed to the 5ks 20 or 40?

Are there many other devices connected to the 2k and do they receive the speed they should

Walter Dey · ‎07-22-2015

We need more information:

- what kind of IP storage protocol ? iSCSI or NFS ?

- which OS ? and of ESXi, which virtual switch ? vswitch, VMware DVS, Cisco N1k ?

- detailed configuration of the virtual switch ? load balancing setup ?

and

it is not best practise, to attach a Netapp filer to a N2k, due to oversucription; storage folks would like to have a 10G non oversubsribed connectivity ! connect the Netapp directly to a 10G ports on a N5k,... switch.

abbasali5 · ‎07-22-2015

Thanks Mark and Walter,

Please see below the output from show interfaces ethernet 103/1/25-28 in question.

Also included the hardware and software version at the end.

MDF-DC1# show int ethernet 103/1/25

Ethernet103/1/25 is up

admin state is up

Hardware: 1000/10000 Ethernet, address: 5ca4.8a8c.2c1a (bia 5ca4.8a8c.2c1a)

Description: < DR SAN REPLICATION TEMP - NIC N1-e0b >

MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, medium is broadcast

Port mode is access

full-duplex, 10 Gb/s, media type is 10G

Beacon is turned off

Auto-Negotiation is turned on

Input flow-control is off, output flow-control is off

Auto-mdix is turned off

Rate mode is dedicated

Switchport monitor is off

EtherType is 0x8100

Last link flapped 1week(s) 0day(s)

Last clearing of "show interface" counters 6w5d

2 interface resets

Load-Interval #1: 30 seconds

30 seconds input rate 224 bits/sec, 0 packets/sec

30 seconds output rate 552 bits/sec, 0 packets/sec

input rate 224 bps, 0 pps; output rate 552 bps, 0 pps

Load-Interval #2: 5 minute (300 seconds)

300 seconds input rate 3968 bits/sec, 1 packets/sec

300 seconds output rate 4520 bits/sec, 1 packets/sec

input rate 3.97 Kbps, 1 pps; output rate 4.52 Kbps, 1 pps

RX

4885014 unicast packets 65990 multicast packets 2685 broadcast packets

4953689 input packets 1952839989 bytes

0 jumbo packets 0 storm suppression packets

0 runts 0 giants 0 CRC/FCS 0 no buffer

0 input error 0 short frame 0 overrun 0 underrun 0 ignored

0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop

0 input with dribble 0 input discard

0 Rx pause

TX

4956095 unicast packets 992224 multicast packets 196687 broadcast packets

6215724 output packets 2149545682 bytes

8524 jumbo packets

0 output error 0 collision 0 deferred 0 late collision

0 lost carrier 0 no carrier 0 babble 0 output discard

0 Tx pause

MDF-DC1# show int ethernet 103/1/26

Ethernet103/1/26 is up

admin state is up

Hardware: 1000/10000 Ethernet, address: 5ca4.8a8c.2c1b (bia 5ca4.8a8c.2c1b)

Description: < DR SAN REPLICATION TEMP - NIC N2-e0b >

MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, medium is broadcast

Port mode is access

full-duplex, 10 Gb/s, media type is 10G

Beacon is turned off

Auto-Negotiation is turned on

Input flow-control is off, output flow-control is off

Auto-mdix is turned off

Rate mode is dedicated

Switchport monitor is off

EtherType is 0x8100

Last link flapped 1week(s) 0day(s)

Last clearing of "show interface" counters 6w5d

3 interface resets

Load-Interval #1: 30 seconds

30 seconds input rate 1960 bits/sec, 1 packets/sec

30 seconds output rate 2448 bits/sec, 2 packets/sec

input rate 1.96 Kbps, 1 pps; output rate 2.45 Kbps, 2 pps

Load-Interval #2: 5 minute (300 seconds)

300 seconds input rate 29488 bits/sec, 42 packets/sec

300 seconds output rate 1605320 bits/sec, 136 packets/sec

input rate 29.49 Kbps, 42 pps; output rate 1.61 Mbps, 136 pps

RX

75478175 unicast packets 65991 multicast packets 2757 broadcast packets

75546923 input packets 7471402824 bytes

0 jumbo packets 0 storm suppression packets

0 runts 0 giants 0 CRC/FCS 0 no buffer

0 input error 0 short frame 0 overrun 0 underrun 0 ignored

0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop

0 input with dribble 0 input discard

0 Rx pause

TX

229664577 unicast packets 992257 multicast packets 196620 broadcast packet

s

235653630 output packets 336314440639 bytes

147281 jumbo packets

0 output error 0 collision 0 deferred 0 late collision

0 lost carrier 0 no carrier 0 babble 0 output discard

0 Tx pause

MDF-DC1# show int ethernet 103/1/27

Ethernet103/1/27 is up

admin state is up

Hardware: 1000/10000 Ethernet, address: 5ca4.8a8c.2c1c (bia 5ca4.8a8c.2c1c)

Description: < NetApp Prod SAN FCoE - N1-e0b >

MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, medium is broadcast

Port mode is access

full-duplex, 10 Gb/s, media type is 10G

Beacon is turned off

Auto-Negotiation is turned on

Input flow-control is off, output flow-control is off

Auto-mdix is turned off

Rate mode is dedicated

Switchport monitor is off

EtherType is 0x8100

Last link flapped 1week(s) 0day(s)

Last clearing of "show interface" counters 6w5d

2 interface resets

Load-Interval #1: 30 seconds

30 seconds input rate 1248 bits/sec, 1 packets/sec

30 seconds output rate 1904 bits/sec, 1 packets/sec

input rate 1.25 Kbps, 1 pps; output rate 1.90 Kbps, 1 pps

Load-Interval #2: 5 minute (300 seconds)

300 seconds input rate 1509536 bits/sec, 126 packets/sec

300 seconds output rate 27192 bits/sec, 38 packets/sec

input rate 1.51 Mbps, 126 pps; output rate 27.19 Kbps, 38 pps

RX

201528825 unicast packets 65993 multicast packets 4023 broadcast packets

201598841 input packets 292255465031 bytes

0 jumbo packets 0 storm suppression packets

0 runts 0 giants 0 CRC/FCS 0 no buffer

0 input error 0 short frame 0 overrun 0 underrun 0 ignored

0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop

0 input with dribble 0 input discard

0 Rx pause

TX

62933326 unicast packets 1022485 multicast packets 195360 broadcast packet

s

64239330 output packets 6684406470 bytes

8984 jumbo packets

0 output error 0 collision 0 deferred 0 late collision

0 lost carrier 0 no carrier 0 babble 0 output discard

30222 Tx pause

MDF-DC1# show int ethernet 103/1/28

Ethernet103/1/28 is up

admin state is up

Hardware: 1000/10000 Ethernet, address: 5ca4.8a8c.2c1d (bia 5ca4.8a8c.2c1d)

Description: < NetApp Prod SAN FCoE - N2-e0b >

MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, medium is broadcast

Port mode is access

full-duplex, 10 Gb/s, media type is 10G

Beacon is turned off

Auto-Negotiation is turned on

Input flow-control is off, output flow-control is off

Auto-mdix is turned off

Rate mode is dedicated

Switchport monitor is off

EtherType is 0x8100

Last link flapped 1week(s) 0day(s)

Last clearing of "show interface" counters 6w5d

2 interface resets

Load-Interval #1: 30 seconds

30 seconds input rate 1208 bits/sec, 1 packets/sec

30 seconds output rate 1312 bits/sec, 1 packets/sec

input rate 1.21 Kbps, 1 pps; output rate 1.31 Kbps, 1 pps

Load-Interval #2: 5 minute (300 seconds)

300 seconds input rate 119536 bits/sec, 12 packets/sec

300 seconds output rate 6768 bits/sec, 5 packets/sec

input rate 119.54 Kbps, 12 pps; output rate 6.77 Kbps, 5 pps

RX

40794461 unicast packets 65994 multicast packets 4030 broadcast packets

40864485 input packets 50256267733 bytes

0 jumbo packets 0 storm suppression packets

0 runts 0 giants 0 CRC/FCS 0 no buffer

0 input error 0 short frame 0 overrun 0 underrun 0 ignored

0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop

0 input with dribble 0 input discard

0 Rx pause

TX

17091067 unicast packets 1655188 multicast packets 195348 broadcast packet

s

19009825 output packets 3050691908 bytes

8962 jumbo packets

0 output error 0 collision 0 deferred 0 late collision

0 lost carrier 0 no carrier 0 babble 0 output discard

662912 Tx pause

-------------------------------------------------------

NX-OS Version of Nexus 7004 (parent to Nexus 2Ks):

MDF-DC1# show ver

Cisco Nexus Operating System (NX-OS) Software

TAC support: http://www.cisco.com/tac

Documents: http://www.cisco.com/en/US/products/ps9372/tsd_products_support_series_home.html

The copyrights to certain works contained in this software are

owned by other third parties and used and distributed under

license. Certain components of this software are licensed under

the GNU General Public License (GPL) version 2.0 or the GNU

Lesser General Public License (LGPL) Version 2.1. A copy of each

such license is available at

http://www.opensource.org/licenses/gpl-2.0.php and

http://www.opensource.org/licenses/lgpl-2.1.php

Software

BIOS: version 2.12.0

kickstart: version 6.2(8a)

system: version 6.2(8a)

BIOS compile time: 05/29/2013

kickstart image file is: bootflash:///n7000-s2-kickstart-npe.6.2.8a.bin

kickstart compile time: 5/15/2014 20:00:00 [06/20/2014 21:56:57]

system image file is: bootflash:///n7000-s2-dk9-npe.6.2.8a.bin

system compile time: 5/15/2014 20:00:00 [06/20/2014 23:25:18]

---------------------------------------------------------------------------------------------

Hardware

cisco Nexus7000 C7004 (4 Slot) Chassis ("Supervisor Module-2")

Intel(R) Xeon(R) CPU with 12224948 kB of memory.

Processor Board ID JAF1716ARAC

Device name: MDF-DC1

bootflash: 2007040 kB

slot0: 0 kB (expansion flash)

Kernel uptime is 80 day(s), 22 hour(s), 10 minute(s), 44 second(s)

Last reset

Reason: Unknown

System version: 6.2(8a)

Service:

plugin

Core Plugin, Ethernet Plugin

MDF-DC1#