03-01-2012 08:30 AM - edited 03-07-2019 05:17 AM
In my environment we have 3750x switches running ios 15.0 (1) SE2. We have port security mac address sticky configured on all our switch ports. I noticed that we have several interfaces (on different switches) that are up but have not captured the MAC address from the workstation. Here is one example:
interface GigabitEthernet2/0/11
switchport mode access
switchport port-security
switchport port-security mac-address sticky
spanning-tree portfast
end
SWIITCH(config)#do show mac address-t int g2/0/11
Mac Address Table
-------------------------------------------
Vlan Mac Address Type Ports
---- ----------- -------- -----
SWITCH(config)#do show int g2/0/11
GigabitEthernet2/0/11 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 70ca.9bca.760b (bia 70ca.9bca.760b)
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 5454502
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 63000 bits/sec, 51 packets/sec
0 packets input, 0 bytes, 0 no buffer
Received 0 broadcasts (0 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
56272056 packets output, 9223565276 bytes, 0 underruns
0 output errors, 0 collisions, 2 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out
Solved! Go to Solution.
03-20-2012 08:23 AM
I've recently experienced the same issue on a 3750 stack. The switch was initially installed however the first 12 ports would not capture the MAC address of directly connected hosts. Rebooting the switch changed nothing.
Removing the "switchport port-security" command on the individual interfaces and then re-applying it, solved the problem for me.
03-01-2012 09:08 AM
Hi Erik,
please can you also post output from this command?
show port-security interface g2/0/11
Best regards,
Jan
03-01-2012 09:22 AM
Jan, here is the output:
SWITCH#show port-security int g2/0/11
Port Security : Enabled
Port Status : Secure-up
Violation Mode : Shutdown
Aging Time : 0 mins
Aging Type : Absolute
SecureStatic Address Aging : Disabled
Maximum MAC Addresses : 1
Total MAC Addresses : 0
Configured MAC Addresses : 0
Sticky MAC Addresses : 0
Last Source Address:Vlan : 0021.9b38.1ea9:1
Security Violation Count : 0
03-01-2012 09:23 AM
When a bridge receives a BPDU with the TC bit set from a neighbor, these occur:
It clears the MAC addresses learned on all its ports, except the one that receives the topology change.
It starts the TC While timer and sends BPDUs with TC set on all its designated ports and root port (RSTP no longer uses the specific TCN BPDU, unless a legacy bridge needs to be notified).
So, do a sh spann vlan xx detail and see if there is a lots of topology changes
http://www.cisco.com/en/US/tech/tk389/tk621/technologies_white_paper09186a0080094cfa.shtml
I also have seen some IP camera decoders that in order to see the MAC in the MAC table, you have to ping the device first.
HTH
03-01-2012 10:46 AM
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 63000 bits/sec, 51 packets/sec
0 packets input, 0 bytes, 0 no buffer
Received 0 broadcasts (0 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
56272056 packets output, 9223565276 bytes, 0 underruns
From that output I can see that there are no packets coming in to the interface, and many packets out of it. A switch will not learn a MAC address if it doesn´t receive any packet from the host.
I´m curious on why it is not receiving any packets, I´ve seen this in a load-balancing configuration on the NICs(on servers for example, one NIC receives packets and the other NIC sends) but since this is a workstation I really doubt this is the cause.
So, the first question to ask is why the host is not sending any packets through this interface?
You may check NIC config, if you are using any sort of redundancy let us know.
Carlos
03-01-2012 11:09 AM
Carlos, I noticed that also, however since there are several ports behaving this way (on different switches) I check the other ports and saw that input packets are detected. All of these ports have workstations (Windows 7) connected to them. I checked a few of the workstations and there are no network issues (just the MAC capture issue).
03-05-2012 09:11 PM
I see, then it would be interesting to see the output of a debug arp command if it´s supported by your switch, as always recommended, if you are going to issue that command be careful of not doing it when the switch already has a lot of work to do because it overloads the processor, and the debug arp comes with an option of debugging just one interface so I would start there.
03-20-2012 03:33 AM
Erik, we see exactly the same problem on stacks of 3750X switches running IOS 15.0 (1) SE2 (ipbase image). It appears to affect only ports on stack members 2 or higher; we've configured stack member 1 as master in all cases, but I don't know if this is relevant. In all cases the affected ports are "secured" to a single computer, and are configured with "spanning-tree portfast". Here is a typical case:
#sh ru int g 2/0/1
interface GigabitEthernet2/0/1
switchport access vlan 502
switchport mode access
switchport port-security
switchport port-security violation restrict
switchport port-security mac-address sticky
speed auto 100
spanning-tree portfast
#sh port-security int g 2/0/1
Port Security : Enabled
Port Status : Secure-up
Violation Mode : Restrict
Aging Time : 0 mins
Aging Type : Absolute
SecureStatic Address Aging : Disabled
Maximum MAC Addresses : 1
Total MAC Addresses : 0
Configured MAC Addresses : 0
Sticky MAC Addresses : 0
Last Source Address:Vlan : 0021.86fa.95c4:502
Security Violation Count : 0
#sh mac address-table int g 2/0/1
Mac Address Table
-------------------------------------------
Vlan Mac Address Type Ports
---- ----------- -------- -----
These commands were run while the attached computer was happily answering "ping". There doesn't appear to be anything at all wrong with the computer's network connectivity.
Something that may or may not be relevant: With "debug port-security", logging shows many lines like this:
Mar 20 10:05:22.794: PSECURE: psecure_delete_address_not_ok: no port security subblock for Po48
Mar 20 10:05:23.817: PSECURE: psecure_delete_address_not_ok: no port security subblock for Po48
Mar 20 10:05:24.824: PSECURE: psecure_delete_address_not_ok: no port security subblock for Po48
Mar 20 10:05:24.824: PSECURE: psecure_delete_address_not_ok: no port security subblock for Po48
Mar 20 10:05:25.830: PSECURE: psecure_delete_address_not_ok: no port security subblock for Po48
Po48 is the port-channel uplink, configured as a dot1q trunk, to a distribution switch. I haven't been able to find any reference to this debug message anywhere.
In short, we're seeing the same problem. The only data points I have to add are (a) that it happens only on stack members greater than 1, and (b) that the affected ports are all configured with "spanning-tree portfast", and bpduguard is on, so I don't think BPDUs should be an issue.
I'm stumped.
Michael Assels
03-20-2012 06:03 AM
Hi Erik,
Hope ip routing is disabled on 3750 switch, use below method to find out interface using mac address database,
Ex- you are trying to find 172.16.1.20 host interface,
First ping from your core switch (L3) to 172.16.1.20 where host gateway (Ex.172.16.1.1) was configured,
then issue sh ip arp 172.16.1.20 on core,
here is your host mac address, then copy mac address and telnet to your host connected switch,
and issue sh mac address add xxxx.xxxx.xxxx
whoh! you have found interface of connected host,
HTH
03-20-2012 06:11 AM
Hi all,
The issue is that the host connected only receives traffic and does not sent - maybe does not send on this interface.
GigabitEthernet2/0/11 is up, line protocol is up (connected)
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 5454502
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 63000 bits/sec, 51 packets/sec
Having in mind that the FDB is build by looking in the frame received on the port , and source mac address , and looking at the show interface output.
Regards
Dan
03-20-2012 06:39 AM
Dan-Ciprian Cicioiu wrote:
The issue is that the host connected only receives traffic and does not sent - maybe does not send on this interface.
Dan,
I think the lack of traffic sent by the host on Erik's Gi2/0/11 interface is a red herring. I see dozens of ports with the same problem -- i.e., MAC addresses not captured on switch ports -- but all of them send and receive traffic normally. Again, I emphasize that this only happens on stacked 3750X switches, and never on the stack master (or at least, never on stack member 1, which happens to be the master). I should also add that it happens on very roughly half the access ports that have link and line protocol up, and there doesn't appear to be any obvious pattern to the distribution of ports with the problem.
I can't speak for Erik, who originally raised the issue, but it certainly appears that he and I are seeing the same thing.
Regards,
Michael
03-23-2012 07:31 AM
Thanks Kyle, removing and reapplying the port-security commands looks like it fixed the issue.
03-23-2012 11:17 AM
Hi Erik,
Are you sure the problem is fixed? I was able to "bring back" a port's sticky MAC address by removing and reapplying the port-security commands, but it disappears again as soon as there's a significant change to the port's state. In particular, if you unplug that attached host and then plug it in again, the MAC address disappears again. I get the same result by changing the VLAN and then changing it back, or by shutting down the port and then bringing it back up.
Michael
03-20-2012 08:23 AM
I've recently experienced the same issue on a 3750 stack. The switch was initially installed however the first 12 ports would not capture the MAC address of directly connected hosts. Rebooting the switch changed nothing.
Removing the "switchport port-security" command on the individual interfaces and then re-applying it, solved the problem for me.
03-20-2012 09:34 AM
Kyle McKay wrote:
[...]
Removing the "switchport port-security" command on the individual interfaces and then re-applying it, solved the problem for me.
Hmm. That looked really promising, but unfortunately, it didn't work for me.
In case in stimulates anyone's memory, I have more snippets of logging output from "debug port-security":
[...]
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/15
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/16
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning 0x7E3FE6C<001a.a015.6011:540> on Gi2/0/17
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/17
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/18
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning 0x7DEEE84<001a.a016.5892:540> on Gi2/0/19
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/19
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning 0x7DD808C<001a.a015.498a:540> on Gi2/0/20
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/20
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/22
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning 0x7B16A94<0000.a703.674b:601> on Gi2/0/23
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/23
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/33
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning 0x7800AD4<0021.86fa.93b1:540> on Gi2/0/34
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/34
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning 0x7F1D30C<0021.86fa.91f0:540> on Gi2/0/35
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/35
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/36
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/37
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning 0x7C4B634<0021.86fa.933a:540> on Gi2/0/38
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/38
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning 0x7C9AA84<0021.86fa.932d:540> on Gi2/0/39
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/39
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/40
Mar 20 15:08:47: PSECURE: psecure_get_next_mac_from_hat: returning NULL on Gi2/0/41
[... etc., up to Gi6/0/48 ...]
The cases where only "NULL" is returned correspond exactly to the ports that are up and working but apparently without recorded MAC addresses. The "missing" ports (e.g., Gi2/0/21, and Gi2/0/24-32) are not configured for port-security. Again, searching for "psecure_get_next_mac_from_hat" yields no information on either Cisco's site or elsewhere.
Michael
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide