cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
396
Views
1
Helpful
10
Replies

Trunk between Catalyst 9600 and other Cisco switches - Incomplete ARP

BrendonFranca
Spotlight
Spotlight

Hello everyone, I hope you are all well.
I came to ask for help regarding a very intriguing case.
I'm in a project that involves high convergence connections between the Cisco C9600 (with two supervisors, 2 SFP modules of 100 and 50GB and 1 module with 25GB SFPs), two nexus 9300 and two nexus and 2 slaves from the 5000 series.
When trying to create a trunk between the 9600 chassis and the Nexus 5000, the ARP appears as incomplete.
I've already validated everything, network settings, MTU, configurations of the SVI's involved and everything is fine.
The equipment can be seen via CDP but does not exchange TCP (as we say here in Brazil) "nem com reza braba"

I will leave some information about the environment

 

Chassi 9600: 
IOS Version: 17.09.04a

sh int vlan 520 (VLAN Used for the link)
Vlan520 is up, line protocol is up , Autostate Enabled
Hardware is Ethernet SVI, address is cc36.cfae.8a05 (bia cc36.cfae.8a05)
Internet address is 192.168.110.12/24
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not supported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:24, output 00:00:24, output hang never
Last clearing of "show interface" counters 4d01h
Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
13914 packets input, 1472594 bytes, 0 no buffer
Received 0 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
12939 packets output, 1584806 bytes, 0 underruns
Output 0 broadcasts (0 IP multicasts)
0 output errors, 0 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out
--------------

sh int vlan 1 (another vlan used for the link)
Vlan1 is up, line protocol is up , Autostate Enabled
Hardware is Ethernet SVI, address is cc36.cfae.8a05 (bia cc36.cfae.8a05)
Internet address is 192.168.10.218/24
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not supported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 4d01h, output 00:00:09, output hang never
Last clearing of "show interface" counters 3d23h
Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
0 packets input, 0 bytes, 0 no buffer
Received 0 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 packets output, 0 bytes, 0 underruns
Output 0 broadcasts (0 IP multicasts)
0 output errors, 3 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out

-----------

sh cdp nei detail (CDP with N5k)

Device ID: NEXUS-5548-01.*****.local(SSI1745092K)
Entry address(es):
IP address: 192.168.10.1
Platform: N5K-C5548UP, Capabilities: Router Switch IGMP CVTA phone port
Interface: FiftyGigE5/0/5, Port ID (outgoing port): Ethernet1/20
Holdtime : 176 sec

Version :
Cisco Nexus Operating System (NX-OS) Software, Version 7.1(4)N1(1)

advertisement version: 2
Peer Source MAC: 002a.6aa8.76fb
Native VLAN: 1
Duplex: full

---------------

SW-CORE-CAMPUS-9600#sh int fi5/0/5
FiftyGigE5/0/5 is up, line protocol is up (connected)
Hardware is Fifty Gigabit Ethernet, address is 24d5.e4ae.d204 (bia 24d5.e4ae.d204)
MTU 1500 bytes, BW 10000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 10Gb/s, link type is auto, media type is SFP-10GBase-SR
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:03, output hang never
Last clearing of "show interface" counters 3d23h
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 1740000 bits/sec, 256 packets/sec
5 minute output rate 1000 bits/sec, 1 packets/sec
606721 packets input, 492376337 bytes, 0 no buffer
Received 0 broadcasts (0 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
1539 packets output, 151932 bytes, 0 underruns
Output 0 broadcasts (0 multicasts)
0 output errors, 0 collisions, 1 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

-------------

SW-CORE-CAMPUS-9600#sh int status | inc connected
Hu1/0/1 connected trunk full 100G QSFP 40/100G SRBD
Hu1/0/3 connected trunk full 40G QSFP 40GE CSR-S
Hu2/0/1 connected trunk full 100G QSFP 40/100G SRBD
Hu2/0/3 connected trunk full 40G QSFP 40GE CSR-S
Fif5/0/5 connected trunk full 10G SFP-10GBase-SR
SW-CORE-CAMPUS-9600#
SW-CORE-CAMPUS-9600#sh ip int br
Interface IP-Address OK? Method Status Protocol
Vlan1 192.168.10.218 YES manual up up
Vlan520 192.168.110.12 YES NVRAM up up

--------

SW-CORE-CAMPUS-9600#sh arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.10.1 0 Incomplete ARPA
Internet 192.168.10.218 - cc36.cfae.8a05 ARPA Vlan1

 

Logs N5k

IOS Version: 7.1.4.N1.1

 

NEXUS-5548-01# sh int vlan 520
Vlan520 is up, line protocol is up
  Hardware is EtherSVI, address is  002a.6aa8.763c
  Description: GERENCIA_NOVO_CORE_CISCO
  Internet Address is 192.168.110.10/24
  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec
------------------
NEXUS-5548-01# sh int vlan 1
Vlan1 is up, line protocol is up
  Hardware is EtherSVI, address is  002a.6aa8.763c
  Internet Address is 192.168.10.1/24
  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec
---------------
NEXUS-5548-01# sh interface eth1/20
Ethernet1/20 is up
 Dedicated Interface
 
  Hardware: 1000/10000 Ethernet, address: 002a.6aa8.76fb (bia 002a.6aa8.76fb)
  Description: *** TESTE2 GBIC UPLINK DTC CONTEINER ***
  MTU 1500 bytes, BW 10000000 Kbit,, BW 10000000 Kbit, DLY 10 usec
  reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, medium is broadcast
  Port mode is trunk
  full-duplex, 10 Gb/s, media type is 10G
  Beacon is turned off
  Input flow-control is off, output flow-control is off
  Rate mode is dedicated
  Switchport monitor is off
  EtherType is 0x8100
  Last link flapped 00:57:41
  Last clearing of "show interface" counters 02:37:26
  2 interface resets
  30 seconds input rate 88 bits/sec, 0 packets/sec
  30 seconds output rate 890952 bits/sec, 187 packets/sec
  Load-Interval #2: 5 minute (300 seconds)
    input rate 344 bps, 0 pps; output rate 2.25 Mbps, 329 pps
  RX
    1852 unicast packets  290 multicast packets  324 broadcast packets
    2466 input packets  242395 bytes
    0 jumbo packets  0 storm suppression bytes
    0 runts  0 giants  0 CRC  0 no buffer
    0 input error  0 short frame  0 overrun   0 underrun  0 ignored
    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop
    0 input with dribble  0 input discard
    0 Rx pause
  TX
    1330516 unicast packets  27047 multicast packets  120542 broadcast packets
    1478105 output packets  1221482079 bytes
    579255 jumbo packets
    0 output error  0 collision  0 deferred  0 late collision
    0 lost carrier  0 no carrier  0 babble 0 output discard
    0 Tx pause
-------------------
NEXUS-5548-01# sh ip arp | inc 192.168.10.218
192.168.10.218  00:00:35  cc36.cfae.8a05  Vlan1
 
NEXUS-5548-01# ping 192.168.10.218 source 192.168.10.1
PING 192.168.10.218 (192.168.10.218) from 192.168.10.1: 56 data bytes
Request 0 timed out
Request 1 timed out
 
 
 

As you can see, the N5k "sees" the 9600 through ARP but not the other way around.
When I use the interface in access mode, the link is established without problems. The fault is only in TRUNK.

Has anyone experienced this or seen something similar so I can have troubleshooting information? Because I've already exhausted my possibilities hahaha

10 Replies 10

Show interface status 

Show spanning tree

Show etherchannel (or port channel) summary 

MHM

Hello MHM, thanks for the response.

I can't bring you that information right now but look,
The Show Status interface I have:
SW-CORE-CAMPUS-9600#sh int status | inc connected
Hu1/0/1 connected trunk full 100G QSFP 40/100G SRBD
Hu1/0/3 connected trunk full 40G QSFP 40GE CSR-S
Hu2/0/1 connected trunk full 100G QSFP 40/100G SRBD
Hu2/0/3 connected trunk full 40G QSFP 40GE CSR-S
Fif5/0/5 connected trunk full 10G SFP-10GBase-SR

The spanning-tree show has chassis 9600 (which is currently L2) with vlans 1-3,520 as Root FWD (Cost 2000) for the N5k (which is currently the L3 of the network).

Regarding ehterchannel, I still don't have anything configured for this environment.

 

Hello,

I assume the trunk on both sides allows Vlans 1(native) and 520 ? 

Hello, Geaorg! 

Thanks for reply!

Yes, I tested with native VLAN 1 allowing 1-3,520 and also tested with native 520 on both sides.

this is your topolgy if I am correct ?
add to this topolgy link between SW

Screenshot (376).png

Hello MHM, 

This's my topology:

BrendonFranca_1-1714669819106.png

 

 

C9600 and N5K ARP is OK

C9600 and both N9K arp incomplete?

Are N9K form vPC pair?

MHM

Incomplete ARP is between 9600 and N5K.
On the N5K side it learned the MAC address of the 9600 but the 9600 did not learn the MAC of the N5K, making it incomplete.

The other trunks work, but I had to enable the "VLAN tag native enable" command on the N9K for it to work. Since it's a non-production environment, I had no problem doing this, but the N5K is a production device so I couldn't do the same.

N9k do not have vPC, only the physical link.

in C9600
can I see 
show interface trunk 
show vlan brief 

MHM

Hello,

there must be something very basic missing. Post the full running configs of both neighboring switches (with the trunks configured)...

Review Cisco Networking for a $25 gift card