Hey guys, I'm running a pair of Cisco Nexus 9K's in HSRP mode as the core routers. I created a a 10.29.0.0/19 subnet to accomodate our ever-increasing list of BOYD devices. I noticed that the latency is extremely high when pinging clients in this subnet:
64 bytes from 10.29.28.184: icmp_seq=6627 ttl=63 time=424.914 ms 64 bytes from 10.29.28.184: icmp_seq=6628 ttl=63 time=289.047 ms 64 bytes from 10.29.28.184: icmp_seq=6629 ttl=63 time=219.505 ms 64 bytes from 10.29.28.184: icmp_seq=6630 ttl=63 time=396.309 ms 64 bytes from 10.29.28.184: icmp_seq=6631 ttl=63 time=251.964 ms 64 bytes from 10.29.28.184: icmp_seq=6632 ttl=63 time=379.557 ms 64 bytes from 10.29.28.184: icmp_seq=6633 ttl=63 time=549.037 ms 64 bytes from 10.29.28.184: icmp_seq=6634 ttl=63 time=436.171 ms 64 bytes from 10.29.28.184: icmp_seq=6635 ttl=63 time=635.576 ms 64 bytes from 10.29.28.184: icmp_seq=6636 ttl=63 time=470.314 ms 64 bytes from 10.29.28.184: icmp_seq=6637 ttl=63 time=390.080 ms 64 bytes from 10.29.28.184: icmp_seq=6638 ttl=63 time=304.494 ms 64 bytes from 10.29.28.184: icmp_seq=6639 ttl=63 time=478.791 ms 64 bytes from 10.29.28.184: icmp_seq=6640 ttl=63 time=556.774 ms 64 bytes from 10.29.28.184: icmp_seq=6641 ttl=63 time=469.451 ms 64 bytes from 10.29.28.184: icmp_seq=6642 ttl=63 time=1564.194 ms 64 bytes from 10.29.28.184: icmp_seq=6643 ttl=63 time=559.249 ms 64 bytes from 10.29.28.184: icmp_seq=6644 ttl=63 time=423.030 ms
I took a PCAP and discovered there are tons of arp requests from devices asking who the default gateway is (ex: Who has 10.29.0.1? Tell 10.29.28.53). It looks like the mac address of the default gateway is not being resolved. Could this be the culprit to our latency issue? I am able to ping 10.29.0.1 but this address only shows as active in the arp table on the standby router for some reason.
Address Age MAC Address Interface Flags 10.29.0.1 00:07:48 0000.0c9f.f320 Vlan800
Here is our VLAN configuration. The standby router looks identical for the most part with the exception of the interface IP and HSRP priority (10.29.31.252, priority 125). Any help would be much appreciated. Thanks in advance!
IP Interface Status for VRF "default"(1) Vlan800, Interface status: protocol-up/link-up/admin-up, iod: 139, IP address: 10.29.31.251, IP subnet: 10.29.0.0/19 route-preference: 0, tag: 0 IP broadcast address: 255.255.255.255 IP multicast groups locally joined: 220.127.116.11 IP MTU: 1500 bytes (using link MTU) IP primary address route-preference: 0, tag: 0 IP proxy ARP : disabled IP Local Proxy ARP : disabled IP multicast routing: disabled IP icmp redirects: disabled IP directed-broadcast: disabled IP Forwarding: disabled IP icmp unreachables (except port): disabled IP icmp port-unreachable: enabled IP unicast reverse path forwarding: none IP load sharing: none IP interface statistics last reset: never IP interface software stats: (sent/received/forwarded/originated/consumed) Unicast packets : 62804/155833/646/62238/359075 Unicast bytes : 42174958/68353903/187096/41994582/159289835 Multicast packets : 1/2149160/0/1/753830 Multicast bytes : 82/369070078/0/82/30153200 Broadcast packets : 0/0/0/0/0 Broadcast bytes : 0/0/0/0/0 Labeled packets : 0/0/0/0/0 Labeled bytes : 0/0/0/0/0 WCCP Redirect outbound: disabled WCCP Redirect inbound: disabled WCCP Redirect exclude: disabled
Vlan800 - Group 800 (HSRP-V2) (IPv4) Local state is Active, priority 150 (Cfged 150), may preempt Forwarding threshold(for vPC), lower: 0 upper: 150 Hellotime 3 sec, holdtime 10 sec Next hello sent in 1.556000 sec(s) Virtual IP address is 10.29.0.1 (Cfged) Active router is local Standby router is 10.29.31.252 , priority 125 expires in 7.817000 sec(s) Authentication text "cisco" Virtual mac address is 0000.0c9f.f320 (Default MAC) 8 state changes, last state change 1w0d IP redundancy name is hsrp-Vlan800-800 (default)
interface Vlan800 description no shutdown no ip redirects ip address 10.29.31.251/19 ip ospf passive-interface ip router ospf 100 area 0.0.0.0 hsrp version 2 hsrp 800 preempt priority 150 ip 10.29.0.1 ip dhcp relay address 172.18.0.19 ip dhcp relay address 172.18.0.127
It's interesting that the primary HSRP interface does not show any broadcasts in the counters. It appears it is not seeing the ARP requests or any broadcasts are not reaching it or are blocked. You may have a topology or spanning tree type issue. Take a close look at the root bridges for the VLAN, what interfaces are blocking and that the VLAN is properly allowed on the necessary interfaces. A good start anyway.
Game on! As a part of Cisco Live US auxiliary programs, we invite you to learn new technologies and obtain hands-on experience in a fun way by playing Capture the Flag (CTF).
Your mission: solve interesting challenges based on use-cases, technologies and ...
Hi,I'm trying to setup a cellular connection on my Cisco 1111 router.The interface is UP and it gets a private IPv4 and a public IPv6 address.If I try to send something on IPv4 through the cellular interface it works fine.But there seems to be an issue wi...
For an offline or printed copy of this document, simply choose ⋮ Options > Printer Friendly Page. You may then Print, Print to PDF or copy and paste to any other document format you like.
Is your WAN ready for a multicloud transformation?
Network Insider Live Webinar
Tuesday, July 21, 2020 10:00 am Pacific Time (San Francisco, GMT-08:00)
This webinar will show how convergence between SD-WAN and Security is emerging as important new SASE a...