MAC Address not captured on switch ports

[esamaniego]

In my environment we have 3750x switches running ios 15.0 (1) SE2.  We have port security mac address sticky configured on all our switch ports.  I noticed that we have several interfaces (on different switches) that are up but have not captured the MAC address from the workstation.  Here is one example:

interface GigabitEthernet2/0/11

switchport mode access

switchport port-security

switchport port-security mac-address sticky

spanning-tree portfast

end

SWIITCH(config)#do show mac address-t int g2/0/11
          Mac Address Table
-------------------------------------------

Vlan    Mac Address       Type        Ports
----    -----------       --------    -----

SWITCH(config)#do show int g2/0/11

GigabitEthernet2/0/11 is up, line protocol is up (connected)

  Hardware is Gigabit Ethernet, address is 70ca.9bca.760b (bia 70ca.9bca.760b)

    MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,

     reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX

  input flow-control is off, output flow-control is unsupported

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input never, output 00:00:00, output hang never

  Last clearing of "show interface" counters never

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 5454502

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 0 bits/sec, 0 packets/sec

  5 minute output rate 63000 bits/sec, 51 packets/sec

     0 packets input, 0 bytes, 0 no buffer

     Received 0 broadcasts (0 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 0 multicast, 0 pause input

     0 input packets with dribble condition detected

     56272056 packets output, 9223565276 bytes, 0 underruns

     0 output errors, 0 collisions, 2 interface resets

     0 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 pause output

     0 output buffer failures, 0 output buffers swapped out

[MichaelJAssels]

I think I've found a definitive but disappointing answer.  It's a known bug:  CSCtx96215.

http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCtx96215&from=summary

Slave switches missing port-security sticky mac-address entries

Symptom:

A port-security sticky mac-address is missing from a slave's CAM table

Conditions:

The problem is seen only when port-security with port-security

Workaround:

None at this time

Its status is "Severe".  It appears to affect every version of IOS since 12.2(40)SE.

So, while our problem isn't solved, at least it's understood.

Michael

[MichaelJAssels]

Since there seems to be renewed interest here, I'll add my two cents worth:  Cisco engineers managed to identify the problem we were experiencing as a new bug -- CSCtz07523 -- first found in 15.0(1)SE, that is fixed in a pending release.  The bug details are copied below, although in our experience, the "workaround" doesn't work at all.  We've been working with a specially patched engineering image for a few months now, and are anxiously awaiting the pending release.

Michael

CSCtz07523            Bug Details

Slave switches miss port-sec sticky mac entries

Symptom: Slave switches miss port-security sticky mac entriesConditions: 15.0SE2, 3750X stack with port-security enabledWorkaround: removing "switchport port-security" and re-applying may workaround the issue

[vladakoci]

Just to add my recent experience.

This recently happened on one of my many 3750X stacks. I am currently running IOS 12.2.-58.SE1 version, going to upgrade to the most recent 15.0-1.SE2, but not sure if this is going to resolve.

Only some of the ports were affected. As soon as I removed portswitch security and applied it again, it worked. I did not have much time to capture anything, had to fix quickly, but am absolutely sure it has nothing to do with physical layer.

Heard that this could be related to the way how the stack is booted, means the boot sequence of the members in the stack. I have a lot of such stacks and this is the first time I've seen something like this.

My port settings are like these, so I do not use mac address sticky

!

interface GigabitEthernet1/0/23

switchport access vlan xx

switchport mode access

switchport voice vlan yy

switchport port-security maximum 25

switchport port-security

switchport port-security aging time 2

switchport port-security violation restrict

switchport port-security aging type inactivity

ip device tracking maximum 10

ip arp inspection trust

ip arp inspection limit rate 256 burst interval 15

no logging event link-status

srr-queue bandwidth share 10 80 5 5

srr-queue bandwidth shape 0 0 0 0

priority-queue out

snmp trap mac-notification change added

snmp trap mac-notification change removed

spanning-tree portfast

spanning-tree bpduguard enable

service-policy input Policy_Voice

ip dhcp snooping limit rate 10

!

Vlad

[vladakoci]

The same happened on one of our other new 3750X stacks recently. This time the IOS version is 12.2(55)SE3, basically because we plan to add older 3750 switches into this stack in the near future.

I've opened a Cisco case for this particular one.

Will see their answer.

Vlad

[vladakoci]

I am able to reproduce my issue in my lab. Had two calls with Cisco, showed them it. Their expert confirms this is a bug in IOS,  and is a stack synchronization issue. They are going to fix in next IOS release.

The way how you can verify that you have THE same kind of issue as is mine is that you enter this command

remote command all show port-security int g2/0/1

and if you see that port security is enabled on the primary stack member  1 but is disabled on the stack member 2 it is a stack synchronization issue.

[Jan Hrnko]

Vlad,

thank you for sharing this kind of information! I believe it is very useful!

This deserves points and a big THANKS.

Best regards,

Jan

[vladakoci]

Cisco inform they are able  to recreate the issue in their lab and have filed a bug  - CSCub20606, which is Cisco internal bug ID only.

Vlad