Solved: Re: Nexus 9K doesn't age out dynamic mac entries - Page 2

EdvinasKairys11982 · ‎07-16-2021

Hello,

We got couple of N9K's - N9K-C9372PX-E 7.0(3)I7(8)

For some reason they doesn't age out old MAC entries. Where're no custom configurations, the age time is left a default:

Switch# show mac address-table aging-time
Aging Time
----------
1800

Dynamic Local Address Count: 48137

Because of that - our mac address table has grown to 48K, which i think is quite near limits.

ARP entries ages out successfully - after about 25mins, but the MAC address is hanging here for more than 3 hours already.

Switch# show mac address-table address fa16.3e8b.3b4d
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False, C - ControlPlane MAC, ~ - vsan
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
+ 2232 fa16.3e8b.3b4d dynamic 0 F F Po3707

If i delete it manually - ir never reappears, what proves that it's really inactive.

Maybe you have some advices ?

Tried to search for a bug - but not successful. There's no official bug reported.

MHM Cisco World · ‎07-18-2021

show mac address notification mac-move

can you share the output of above command

EdvinasKairys11982 · ‎07-18-2021

SwitchA# show mac address-table notification mac-move 
AM MAC Registration/Deletion Notifications: 594227
  Number of MAC Addresses added: 279849 
  Number of MAC Addresses moved: 9836 
  Number of MAC Addresses removed: 208707

Yes, it's a lot. But the mac which I'd provided here before hasn't had any move - it's still on the same interface. Are you thinking that these numbers could indicate that the dynamic age out was disabled on the switch due to these numbers ? Strange.. For example on some other identical 9Ks (just different version) i see also big numbers:

RandomSw# show mac address-table notification mac-move
MAC Move Notify Triggers: 11238023
  Number of MAC Addresses added: 6302773 
  Number of MAC Addresses moved: 16692 
  Number of MAC Addresses removed: 6226143

But here mac address aging works fine.

The mac move is big because the VM's is moving between cloud hypervisors i think.

MHM Cisco World · ‎07-18-2021

OK, the value is nearly the same compare to total MAC-Move but,
let monitor these value,
we must see the add and remove increase in same or nearly same percentage but if the ADD is more than remove increase then we will deep search the cause of this issue.

EdvinasKairys11982 · ‎07-19-2021

Thanks,

i see in logging few moves has occured, but overal +~200 per day macs are added and non removed. Even they're inactive..

SwitchA# show logging | include MAC
2021 Jul 18 20:12:33 JAYNET03A %L2FM-4-L2FM_MAC_MOVE2: Mac fa16.3eaa.3e0c in vlan 2232 has moved between Po3707 to Po4007
2021 Jul 19 05:16:45 JAYNET03A %L2FM-4-L2FM_MAC_MOVE2: Mac fa16.3e77.4bb7 in vlan 2232 has moved between Po3724 to Po3813
2021 Jul 19 08:07:10 JAYNET03A %L2FM-4-L2FM_MAC_MOVE2: Mac fa16.3e86.82cd in vlan 2213 has moved between Po3815 to Po3916

SwitchA# show  mac address-table notification mac-move 
AM MAC Registration/Deletion Notifications: 594295
  Number of MAC Addresses added: 279984 
  Number of MAC Addresses moved: 9847 
  Number of MAC Addresses removed: 208707 

SwitchA#show mac address-table count 
Legend:
DLAC - Dynamic Local Address Count
DRAC - Dynamic Remote Address Count
SLAC - Static Local Address (User Defined) Count
SRAC - Static Remote Address (User Defined) Count
SAC - Secure Address Count

MAC Entries for all VLANS:
Dynamic Local Address Count:                                   48620
Dynamic Remote Address Count:                                      0
Static Remote Address (User-defined) Count:                        0
Static Local Address (User-defined) Count:                         0
Secure Address Count:                                              0
Total MAC Addresses in Use (DLAC + DRAC + SLAC + SRAC + SAC):  48620

i'm afraid we will hit some Nexus limit - where it will go nuts

Also, lowered mac age timer from 1800 to 1700, no difference - same situation.

Edit seems entering that mac age 1700 command actually helped. I noticed that on one of the paired VPC Nexuses there was entered mac age limit to every vlan, where on the other one - no. So i deleted all exceptions and it started to work. I think the VPC saw some differences and because of that hasn't aged any entries.

Will watch that, but i think it's fixes. Thanks @MHM Cisco World