Ask the Expert: Nexus 5000 Architecture, Configuration and Troubleshooting

[Monica Lluis]

This session will provide an opportunity to learn and ask questions about Cisco Nexus 5000 Architecture, configuration and how to do basic troubleshooting for issues related specifically to this platform. To participate in this event, ask your questions below by clicking on the "reply" button. 

Ask questions from Monday April 18 to Friday April 29, 2016

Featured Experts

Ivan Shirshin is a customer support engineer in High-Touch Technical Services (HTTS). He is an expert on Routing, LAN Switching and Data Center products. His areas of expertise include Cisco Catalyst 2x00, 3x00, 4x00, 6500, Cisco Nexus 7000, ISRs, as well as Cisco routers ASR1000, 7600, 10000 and XR platforms. He has over 8 years of industry experience working with large Enterprise and Service Provider networks. Ivan is double CCIE (# 43481) in R&S and DC specializations and also holds CCDP and XR specialist certifications. 

Naveen Venkateshaiah is a customer support engineer in High-Touch Technical Services (HTTS). He is an expert on Routing, LAN Switching and Data Center products. His areas of expertise include Cisco Catalyst 3000, 4000, 6500, and Cisco Nexus 7000,Nexus 5000, Nexus 3000, Nexus 2000, UCS, and MDS SAN Switches. He has over 8 years of industry experience working with large enterprise and Service Provider networks. Venkateshaiah holds a CCNA, CCNP, and CCDP-ARCH, AWLANFE, LCSAWLAN Certification. He is currently working to obtain a CCIE in Data Center.

Find other  https://supportforums.cisco.com/expert-corner/events.

** Ratings Encourage Participation! **
Please be sure to rate the Answers to Questions

[Ivan Shirshin]

Hi,

Is there a reason you are using single link? It is strongly recommended to connect ASA to vPC domain using vPC link. You may encounters issue in case of link failures otherwise.

The ingress vlan in your case would be vlan 100 but you also need egress vlan, e.g. vlan 200. Since it is transparent mode, they have to be in the same subnet 10.10.100.0/24.

As for routed interfaces, for such setup you need SVI for egress vlan on Nexus (default gateway) and BVI interface bridging ingress and egress vlans on ASA.

Static routes are not needed in this case, since it is transparent mode and servers are in the same subnet with default gateway.

Kind Regards,

Ivan

[bkoch1]

We're currently running into an odd L2 issue. We have a 6509 VSS pair connected to a nexus 5548up pair (connected using vpc). The vpc links are lacp, the links between the core and nexus pair are not (plain etherchannel). Connected to both 5K's are 4 2200TP fabric extenders. All routing is done at the core.

I have an ESX cluster connected to a fabric extender utilizing 4 trunk ports. 2 ports have vlan 64,118 and 2 ports have vlan 117, 118. a host in the ESX cluster has an ip of 10.64.255.191 (vlan 64 IP scheme).

Here is my issue: If I'm not on vlan 64, i can ping the 10.64.x.x IP. If I have another PC inside vlan 64, but not inside the nexus structure, I can't ping the ESX host inside. The ESX host inside can ping out through the nexus and the core to 10.64.x.x though. it's as though arp requests going INTO the nexus tree is getting lost.

[Ivan Shirshin]

Hi,

Please see reply to your another post above.

Kind Regards,

Ivan

[bkoch1]

Resubmitting cause first one didn't get posted.

Network layout: 6509 VSS pair connects to N5548UP pair (4x10G links) which connects to a 2248TP fabric extender (2x10G links).

I have an ESX cluster connected to the N2K using 4 trunk ports, which have vlans 64, 117, 118. A host in the ESX cluster has IP 10.64.255.191 (vlan 64 scheme).

Problem: 10.64.255.191 host inside the nexus structure can communicate(ping) out through nexus-->core to another host on 10.64.x.x. Anything else connected through the core from vlan 64 can't ping back through core-->nexus structure to 10.64.255.191.

It seems arp requests are being trashed somewhere.

nexus 5K configs are identical.

[Ivan Shirshin]

Hi,

The problem indeed seems to be caused by a packet loss in one direction from your description.

I do not see anything wrong related to the design, and I recommend to narrow down the point of failure to proceed with the issue analysis.

As the first step, I would trace the packets that are getting lost and try to find where they are getting lost, and in which direction. You can do it using several methods: ACL on the interfaces iin the traffic path for this specific traffic, SPAN and/or ELAM on the switches that support it (6500 support ELAM). You know the source, destination, type of traffic, so you specify all required filters for this flow specifically.

When the point of failure is identified, you can research in more details the reason packets are getting dropped there (due to software issue, or configuration, or vlan connectivity, etc.).

Also, with this symptoms and complex investigation needed I recommend to open a TAC case to have dedicated Cisco engineer assist you with troubleshooting.

Kind Regards,

Ivan

[bkoch1]

Finally found it, there was vtp pruning on the PO interface going from core to Nexus pair that was stopping the arp broadcasts on 2 vlans.

Thanks!

[Ivan Shirshin]

Great, I am glad the issue is resolved! You are welcome!

Kind Regards,

Ivan

[Wilson Gabriel Veliz Plua]

Good Day

Do you have any example of configuration a Nexus 7K/5K in HCS enviroment ?

Regards

Wilson Veliz

[Naveen Venkateshaiah]

Hi,


Below link provides you the details of typical Cisco HCS Large PoD deployment, Cisco HCS Small Pod deployment.

http://www.cisco.com/c/en/us/td/docs/voice_ip_comm/hcs/9_2_1_SU1/Solution_Install/CHCS_BK_I13CF47E_00_installation-guide-for-chcs-9_2_1SU1/CHCS_BK_I13CF47E_00_installation-guide-for-chcs-9_2_1SU1_chapter_01100.html#CHCS_TK_N9DD0BB9_00

For more information HCS ,Please go through below link.

http://www.cisco.com/c/en/us/solutions/hosted-collaboration-solution/hcs-customers.html

Addition to this we have detailed Presentation on Hosted collaboration services.

http://d2zmdbbm9feqrf.cloudfront.net/2014/usa/pdf/BRKCOL-1350.pdf
http://d2zmdbbm9feqrf.cloudfront.net/2015/lat/pdf/BRKCOL-1351.pdf

Regards,

Naveen Venkateshaiah

[Wilson Gabriel Veliz Plua]

Tks Naveen.

Ivan as expert can yo7u help me with some configuration examples of ASR as a CUBE SP or a forum where I can learn more about it.

Regards

Wilson Veliz P.

[Ivan Shirshin · Accepted solution]

Hi,

You can find CUBE (SP edition) configuration examples in the following guides:

http://www.cisco.com/c/en/us/support/docs/voice-unified-communications/unified-border-element/116415-configure-product-00.html

http://www.cisco.com/c/en/us/td/docs/routers/asr1000/profiles/SBC_Config_Examplebook.html

Kind Regards,

Ivan

[jjreyes1986]

good day

My question is the following, when we talk about use VPC peer Gateway, letting pass Layer 3 messages to build a HRSP between both Nexus and the messages pass through the VPC, if peer Gateway is enabled but is already working as VPC peer link and are connected switch access with LACP toward the Nexus, exists possibility to have some issues? Besides HSRP usage, VRRP protocol would let pass peer Gateway messages from routing protocols (OSPF, EIGRP) ?

regards

[Naveen Venkateshaiah]

Hi,

Packets reaching a vPC device for the non-local router MAC address are sent across the peer-link and could be dropped by the built in vPC loop avoidance mechanism if the final destination is behind another vPC.

Basically The vPC peer-gateway functionality allows a vPC switch to act as the active gateway for packets that are addressed to the router MAC address of the vPC peer.

This feature enables local forwarding of such packets without the need to cross the vPC peer-link. In this scenario, the feature optimizes use of the peer-link and avoids potential traffic loss.

Packets arriving at the peer-gateway vPC device will have their TTL decremented, so packets carrying TTL = 1 may be dropped in transit due to TTL expire. This needs to be taken into account when the peer-gateway feature is enabled and particular network protocols sourcing packets with TTL = 1 operate on a vPC VLAN.

below is the detail example based explanation on Peer gateway.

https://supportforums.cisco.com/document/98811/peer-gateway-feature-nexus-7000

With respect to having HSRP or VRRP it shouldn’t affect any routing protocol to either eigrp/ospf to get pass through the peer gateway.

Below is the useful link  to configuring peer gateway with example.

http://www.cisco.com/c/en/us/products/collateral/switches/nexus-5000-series-switches/guide_c07-673997.html#wp9000332

Regards,

Naveen

[ssalin_nico]

Good evening Team;

What security challenges are associated with switching and routing in the Nexus 5000 architecture?

Kind regards.

Nicholas.

[Ivan Shirshin]

Hello Nicholas,

Are you interested in something specific? There are certain things to be taken into account when you deploy security in N5k network, e.g.:

1. Features implementation can not exist in older release, such as Login Block Per User was added to Dynamic ARP Inspection only starting with .3(0)N1(1), or 5.1(3)N1(1) added information to configure the Cisco TrustSec feature.

2. There are limitations and guidelines for specific security areas, such vPC security or ACLs which you can see in corresponding guides:

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5000/sw/security/513_n1_1/b_Cisco_n5k_security_config_gd_513_n1_1/b_Cisco_n5k_security_config_gd_513_n1_1_chapter_01001.html#concept_22FB5A78C7A94A028979AFF30B34FDF8

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5000/sw/security/521_n1_1/b_5k_Security_Config_521N11/b_5k_Security_Config_521N11_chapter_01000.html#concept_C4D43EE804DE433790286B93867E5EC1

3. Platform specific limitations, such as the fact Nexus 5000 switches do not support all numeric usernames, whether created with TACACS+ or RADIUS, or created locally, and do not create local users with all numeric names. If an all numeric username exists on an AAA server and is entered during login, the Nexus 5000 Series switch will log in the user.

The recommended approach to this is to verify the guidelines and limitation of security features you need in the corresponding guides at cisco.com.

Kind Regards,

Ivan