cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
13628
Views
150
Helpful
23
Replies

Ask the Expert: Cisco Nexus 3000 Series Switches

ciscomoderator
Community Manager
Community Manager
 

This session provides an opportunity to learn and ask questions about Cisco Nexus 3000 Series Switches hardware architecture and their role in Data Center / Enterprise switching design, configuration and troubleshooting.

Cisco Nexus 3000 Series Switches extend Cisco Unified Fabric to the data center. Highly programmable, these low-latency switches offer simplified management, enhanced network visibility, advanced monitoring features, and wire-speed Layer 2 and 3 switching for data center top-of-rack (ToR) deployments. These highly programmable, cost-effective, deliver flexible port densities and low power consumption for data centers.

 

Ask questions from Monday, September 21 to Friday, October 2nd, 2015

Featured Experts

Vikash Kumar is a Customer support engineer in High-Touch Technical Services (HTTS) team supporting Data center Products. His areas of expertise include Cisco Nexus 9000, Nexus 7000/7700 , Nexus 5000, Nexus 3000, Nexus 2000, UCS, and MDS SAN Switches. Kumar has over 6 years of industry experience working with large enterprise and Service Provider networks. He has delivered several internal and external trainings on Datacenter technologies and created external documents on Datacenter products and technologies. He holds Bachelors in Information Technologies and PGDM in Cyber security from IMT Ghaziabad, India and these Cisco  certifications: CCNA, CCNP (Routing and Swtiching), CCNP (security) and he is triple CCIE (#27857) (in R&S, Security and Datacenter). He is pursuing for CCDE

 

 

 

Find other  https://supportforums.cisco.com/expert-corner/events.

** Ratings Encourage Participation! **
Please be sure to rate the Answers to Questions

 


 

23 Replies 23

Sarah Staker
Level 1
Level 1

Hello Vikash,

We are receiving following UDLD empty echo error message on one of the Nexus 3000 switch connected to core Nexus 7000.
%ETHPORT-5-IF_DOWN_ERROR_DISABLED: Interface Ethernet1/48 is down (Error disabled. Reason:UDLD empty echo)

 

Can you kindly explain what it means and how to stop receiving it?

 

Thank you

Sarah

Hi Sarah,

When this error is detected, the cause could at either side of the peer port, let say we are having topology as following:

Switch-A(N3k)—Ethx/y————————————————————Ethw/z—Switch-B(N7k)

Empty-Echo condition:
- Echo Packet from A to B has “My Switch-ID A, My Port-ID e x/y”, But when N7k sends the echo-reply back,  it is expected to have “My Switch-ID of B, My Port-ID e w/z” AND “Your Switch-ID A, Your Port-ID e x/y”
-     If for some reason packets from N3k did not received by N7k, it will not have the switch A(N3k) info. So when B (N7k) sends the echo-reply back, the echo-reply packet will have only “My Switch-ID of B , My Port-ID e w/z”.
    After receiving such “Empty-echo” packets continuously, switch A will declare a “Empty-echo” error is detected.  

What is wrong here?: The cause could be:

 - Packet did not sent out from e w/z of switch-B,
-  Packet arrived at e x/y of switch-A did not get to UDLD (CPU).

Troubleshooting Steps:

First look at Switch-B(N7k):

-- "show interface e w/z counters” will indicate if there is any packet sent at port (do a clear counter first will make it easier to ser any increase on packet count).
-- "show system internal pktmgr interface e w/z” will show if there is any packet passing thru the pktmgr to UDLD from port e w/z

Note:- To further debug the packet flow between Sup (pktmgr) and Module, elam on N7k  can be used to capture the UDLD packet at Sup and module level

-- "show udld internal event-history msgs  | grep –A 3 –B 3 L2_RX_DATA” will show if there is any packet arrived at UDLD from pktmgr. If above steps show no packets coming at port e w/z, then the issue may be at N3k switch-A side

Next look at Switch-A: (N3k):

Repeat the same steps as with switch-B, with following commands:
-- "show interface e x/y counters”
-- "show system internal pktmgr interface e x/y”
-- "debug logfile udld-log” to log into a file with “debug udld trace”, from the log file check if there is any msg as follow to show the packet is sent to the port:
             
udld_send_update(3610): Dst Port: Ethernet x/y [0xaabbcddee]

-- These steps will pin point where the UDLD packet is lost, i.e., did it get to the pktmgr? If it got to the pktmgr, did it get to the module? Did the module sent it out?

Should you be having any further query in this regards, please feel free to revert back.

Regards,
Vikash

 

we are planning to upgrade our nexus N3K-C3064PQ-10GX switches to 5.0(3)U5(1a), however in the compatibility check it is showing upgrade is disruptive?There is any possibility go with non disruptive?

Hi Vedprakash,

non-disruptive upgrade (ISSU) is not supported on 5.0(3)U5(1a).

regards,
Vikash

Hi Vikash,

Cisco N3548 provides normal port latency is 250ns

Can you explain how much latency of UCS Mini FI6324 is? I mean the ASIC chipset on FI6324.

If UCS mini is using the same ASIC with N3548, that will be great for HFT or Algo trading

Thanks

James

Hi James,

UCS Mini has Malibu ASIC which is based on the Monticello-CR (MTC-CR) ASIC same as on Nexus 3548/3524. And since it is the same Monticello-CR chip with just less ports hence latency is also same (~250ns) like N3500.

 

regards,
Vikash kumar

Hi Vedprakash,

non-disruptive upgrade (ISSU) is not supported on 5.0(3)U5(1a).

regards,
Vikash

Sandeep Maurya
Level 1
Level 1

Hi Vikash,

What are the advantages of Nexus3000 over N5500/5600 and in which scenario/Layer should a customer prefer putting N3K in his DataCenter.  Also N3k works in which mode i.e Cut-through or Store and Forward ?

 

Thanks & Regards,

Sandeep Maurya.

Hi Sandeep,

Both N3k and N5k series switch are designed for low latency, top-of-rack(TOR) / Access layer deployment. And both  switch equipped with Cut-through switching mechanism. However following are few key difference in both generations of the switch, on which base you can choose the best switch for your design requirement.


Nexus 3K has comprehensive Layer-2 and Layer-3 functionality along with rich programmability feature-set for traditional L2/L3 deployments , lower per port power consumption with improved latency of 480ns and increased buffering and table sizes(Multicast,Unicast).


Where N5k have feature/functionality, which are not available in the Nexus 3000 series switches, such as:

The Nexus 5000 offers the following unique features: FEX support, Unified Port, Unified Fabric - FCoE, FabricPath, Adapter FEX, VM FEX. Further, the Nexus 5000 offers a flexible port configuration from 32 to 96 ports in 16 port increments.

 

Positioning:

Nexus 3000/3500 for competitive data center switching opportunities that require a smaller form factor 10GE purpose-built, line-rate switch, but do not need advanced features such as FEX, FCoE and FabricPath, Dynamic Fabric Automation (DFA).


Quick Comparison:
Apart from the cost factor, following are some hardware and software features and capacity difference:

 

 N3548N3048N5548/N5596
L2 Switching Capacity960Gbs176Gbps1.92Tbps
L3 Switching Capacity960Gbs line rate48x1GE + 4x10GE line rate16x10GE line rate
L2 Throughput720 Mpps131 Mpps1428 Mpps
L3 Throughput720 Mpps131 Mpps240 Mpps
Power (Typical/Max)152W/265W143W/267W 390W/600W (N5548)
660W/882W (N5596)
Route Table24KUp to 16K8K
Multicast Routes8KUp to 8KUp to 4K
MAC Unicast Table64K128K25K
MAC Multicast Table8K 8K 4K
SVI4K1K4K
Buffer18MB Shared9MB Shared640KB/port
Logical VLAN ports9000900048K (H+)
Ether Channel24 ports16 ports16 ports
SPAN Session444
ACL Table4K2K4K
Switching ModeCut-throughCut-ThroughCut-Through
100Mb supportNoYesNo
    

 

Hope this is helpful and answers your query, let me know if you need any further information in this regards.

Thanks !

Regards,
Vikash Kumar

keshav.kumar
Level 1
Level 1

Hi Vikash,

I hope you will help me for this issue.

I am trying to upgrade a nexus switch but it is throwing this error.

#install all system bootflash:n3500-uk9.6.0.2.A3.0.674.bin kickstart bootflash:n3500-uk9-kickstart.6.0.2.A3.0.674.bin
Installer is forced disruptive
Pre-upgrade check failed. Return code 0x40930062 (free space in the filesystem is below threshold).

Hi Keshav,

i am suspecting that your switch /var/tmp directory is not having enough space, which is causing upgrade failed.

please use following command and check the status of "/var" directory.

example:-

HTTS-DCN-N3k# show system internal flash
Mount-on                  1K-blocks      Used   Available   Use%  Filesystem

/                             89243     67554       21689     76   /dev/root
/proc                             0         0           0      0   proc
/post                          2048         4        2044      1   none
/var                          89243     67554       21689     76   none <<<<<
/sys                              0         0           0      0   none
/debugfs                          0         0           0      0   nodev
/isan                       1536000    593280      942720     39   none
/nxos/tmp                     40960        36       40924      1   none
/var/tmp                     409600     399100      10500      93   none
/var/sysmgr                  921600     60772      860828      7   none
/var/sysmgr/ftp              307200        72      307128      1   none
/var/sysmgr/ftp/cores        102400         0      102400      0   none
/dev/shm                     409600    257412      152188     63   none
/volatile                    102400         0      102400      0   none
/debug                        20480         0       20480      0   none
/dev/mqueue                       0         0           0      0   none

As you can see from the logs above  "/var/tmp" folder is almost full. To troubleshoot and fix the issue further we need to check the files under "/var/tmp" directory and need to delete the files, larger in size, as following:


HTTS-DCN-N3k# show system internal dir /var/tmp
                                                                ./         800
                                                               ../        1024
                                                      etherfilter          784
                                           util_cli_history_admin            0
                                                   bootloader_ver            0
                                                     libvirtd.log          116
                                                 m2rib_ascii.3480          374
                                            _clis_parse_tree_.log         2706
                                                 sdwrap.378.378.5      6291456
                                                 sdwrap.378.378.4       524288
                                                 sdwrap.378.378.3      1048576
                                                 sdwrap.378.378.2      1048576
                                                 sdwrap.378.378.1      1048576
                                                 sdwrap.378.115.0        65536
                                                   sdwrap.378.0.0       524288
                                                  pim_restart.log            0
                                                        stp.log.1         4948
                                                           ntpd_1            0
                                                      cfs_mac.log          166
                                                     ntp_client_1            0
                                                 igmp_restart.log            0
                                                         fwm1.out            0
                                                     ufdm_log.txt          109
                                                          vdc.log        269399
                                                  security_stderr           77
                                                  security_stdout            0
                                               security_debug.log          647
                                          
                                                              vsh/          40

HTTS-DCN-N3k# filesys delete /var/tmp/vdc.log
Please Wait.File is being deleted.
Successfully deleted the file.

If you fails to delete unnecessary files from /var/tmp directory, please reache out to cisco TAC so that CSE can load debug plugin file on the switch and can help you deleting unnecessary files from linux kernel shell.

For further information please refer following software defect:

CSCuh69073    Pre-upgrade check failed. Return code 0x40930062 (free space in the file
CSCuo00926    SNMP get operations causing /var/tmp/vdc.log to grow too large

 

should you be having any further query in this regards, please feel free to revert back.

 

thanks !

regards,
Vikash kumar
 

susim
Level 3
Level 3

Hi vikash 

How would you compare n3k and n9k . 

Is it  wrong  choice  for DMZ where only web traffic passing mainly ?

Thanks

 

Hi SBT team,

Both N3k and N9k use same Trident 2 ASIC and best for low latency data-center environment.  Where N3k is designed for top-of-rack(TOR) implementation and best match for 10G / 40G line rate switching requirement.  At the same time N9k is designed for leaf or spine (distribution and core) layer implementation and offers more port density, redundancy, high availability,  more throughput and less power consumption, alone with rich features like VXLAN, ACI , FEX, single binary image, patch upgrade etc.

 

If your requirement is to provide high speed with ultra low latency to the traffic in DMZ, Nexus 3100 or 3500 cut-through switches is the good choice. which offers 960 Gbs L2/L3 line-rate speed with less power consumption.

regards,
Vikash kumar

 

kp-tkr2014
Level 1
Level 1

Hi ,

Can  we build  double sided vpc topology using n3k ?

Thanks

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: