Re: Issue with downlink ports in Cisco Catalyst 9500-48Y4C switch - Page 2

jshayne · ‎02-16-2023

I connected a computer with a 25G interface in Cisco Catalyst 9500-48Y4C switch. I saw ports on both PC and the switch are UP (green light). However, the switch could not learn the computer's MAC address. Also, the computer did not know the MAC address of the connected switch port. It seems to be a layer-2 problem. When I tried the ping from the computer to the switch port, as the MAC address of the switch port is not listed on the computer, the broadcast messages were sent out. But what surprised me is that switch still received the broadcast messages from the computer. What is the problem here?

-------------On the switch---------------------

show interface TwentyFiveGigE1/0/X
TwentyFiveGigE1/0/X is up, line protocol is up (connected)
Hardware is Twenty Five Gigabit Ethernet, address is 8024.8f2f.451f (bia 8024.8f2f.451f)
Internet address is 123.123.123.123/24
MTU 1500 bytes, BW 10000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 10Gb/s, link type is auto, media type is SFP-10GBase-CU3M
input flow-control is on, output flow-control is off
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:00, output hang never
Last clearing of "show interface" counters 1w2d
Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
3052 packets input, 201865 bytes, 0 no buffer
Received 3052 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 235 multicast, 0 pause input
0 input packets with dribble condition detected
500755 packets output, 45412032 bytes, 0 underruns
Output 2 broadcasts (0 IP multicasts)
0 output errors, 0 collisions, 6 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

When I ran "ping 123.123.123.123" from the computer, the switch port updated broadcast messages:

show interface TwentyFiveGigE1/0/X
TwentyFiveGigE1/0/X is up, line protocol is up (connected)
Hardware is Twenty Five Gigabit Ethernet, address is 8024.8f2f.451f (bia 8024.8f2f.451f)
Internet address is 123.123.123.123/24
MTU 1500 bytes, BW 10000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 10Gb/s, link type is auto, media type is SFP-10GBase-CU3M
input flow-control is on, output flow-control is off
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:02, output hang never
Last clearing of "show interface" counters 1w2d
Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
3058 packets input, 202249 bytes, 0 no buffer
Received 3058 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 235 multicast, 0 pause input
0 input packets with dribble condition detected
500773 packets output, 45415012 bytes, 0 underruns
Output 2 broadcasts (0 IP multicasts)
0 output errors, 0 collisions, 6 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

Switch did not learn MAC address at all:

show mac address-table interface TwentyFiveGigE1/0/X
Mac Address Table
-------------------------------------------

Vlan Mac Address Type Ports
---- ----------- -------- -----

Other information on the switch:

Cisco IOS XE Software, Version 17.03.04

show vlan

VLAN Name Status Ports
---- -------------------------------- --------- -------------------------------
1 default active Twe1/0/1, Twe1/0/2, Twe1/0/3
Twe1/0/4, Twe1/0/5, Twe1/0/6
Twe1/0/7, Twe1/0/8, Twe1/0/9
Twe1/0/10, Twe1/0/11, Twe1/0/12
Twe1/0/13, Twe1/0/14, Twe1/0/15
Twe1/0/16, Twe1/0/17, Twe1/0/18
Twe1/0/19, Twe1/0/20, Twe1/0/21
Twe1/0/22, Twe1/0/23, Twe1/0/24
Twe1/0/25, Twe1/0/26, Twe1/0/27
Twe1/0/28, Twe1/0/29, Twe1/0/30
Twe1/0/31, Twe1/0/32, Twe1/0/33
Twe1/0/34, Twe1/0/35, Twe1/0/36
Twe1/0/37, Twe1/0/38, Twe1/0/39
Twe1/0/40, Twe1/0/41, Twe1/0/42
Twe1/0/43, Twe1/0/44, Twe1/0/45
Twe1/0/46, Twe1/0/47, Hu1/0/50
Hu1/0/51, Hu1/0/52
2 VLAN0002 active
1002 fddi-default act/unsup
1003 token-ring-default act/unsup
1004 fddinet-default act/unsup
1005 trnet-default act/unsup

--------------On the computer--------------------------

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 123.123.123.1 netmask 255.255.255.0 broadcast 123.123.123.255
ether 90:e2:ba:e2:76:cc txqueuelen 1000 (Ethernet)
RX packets 1894 bytes 266637 (266.6 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 6387 bytes 300306 (300.3 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

user@PC:~$ arp
Address HWtype HWaddress Flags Mask Iface
123.123.123.123 (incomplete) enp2s0f0

jshayne · ‎02-27-2023

@MHM Cisco World : I already tried switch port mode before (switchport command). The PCs could not reach each other. Afterwards, I decided to change to L3 mode for the switch port. I tried to ping directly from the PC to the switch port. But this is not successful as well. I really wonder what is the reason behind it. Let me know if you need any output from the switch.

jshayne · ‎03-01-2023

Any one could please help me with that?

MHM Cisco World · ‎03-01-2023

I will share some point to check today

Leo Laohoo · ‎03-02-2023

Can you try upgrading the 9500 firmware to 17.6.4 or 17.6.5 and try again?

jshayne · ‎03-02-2023

@Leo Laohoo Thanks. I will try to fix the issue with the current firmware. Maybe I will try to upgrade the firmware later.

Do you know how to upgrade it? And will my Cisco license still remain? I don't think I will be able to add the license again (for some reasons) after the upgrade if it is the case.

balaji.bandi · ‎03-02-2023

yes the License not going to change.

read the release notes it has upgrade steps.

https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst9500/software/release/17-6/release_notes/ol-17-6-9500.html

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

Leo Laohoo · ‎03-02-2023

@jshayne wrote:
Do you know how to upgrade it?

Please refer to the attachment.

MHM Cisco World · ‎03-05-2023

friend can you update us are this issue solve after upgrade ??
can you share
show platform software fed switch active ifm mappings <<- share this please when you have time

if the mac not appear
connect host to different port from different ASIC (two or more port is collect in one rectangle, you can see in from view of SW, this meaning that these port is control by one ASIC) it can ASIC is damaged.

then do
show platform software fed switch active ifm mappings

sorry for late reply

jshayne · ‎03-06-2023

@MHM Cisco World I will upgrade the firmware soon and then give the update here.

Here is the output of the command you suggested with the current firmware:

show platform software fed active ifm mappings
Interface                 IF_ID    Inst Asic Core Port SubPort Mac  Cntx LPN  GPN  Type Active
TwentyFiveGigE1/0/1       0x8        1   0   1    20     0      16   4    1    101  NIF  Y
TwentyFiveGigE1/0/2       0x9        1   0   1    21     0      17   5    2    102  NIF  Y
TwentyFiveGigE1/0/3       0xa        1   0   1    22     0      18   6    3    103  NIF  Y
TwentyFiveGigE1/0/4       0xb        1   0   1    23     0      19   7    4    104  NIF  Y
TwentyFiveGigE1/0/5       0xc        1   0   1    24     0      20   8    5    105  NIF  Y
TwentyFiveGigE1/0/6       0xd        1   0   1    25     0      21   9    6    106  NIF  Y
TwentyFiveGigE1/0/7       0xe        1   0   1    26     0      22   10   7    107  NIF  Y
TwentyFiveGigE1/0/8       0xf        1   0   1    27     0      23   11   8    108  NIF  Y
TwentyFiveGigE1/0/9       0x10       1   0   1    28     0      24   12   9    109  NIF  Y
TwentyFiveGigE1/0/10      0x11       1   0   1    29     0      25   13   10   110  NIF  Y
TwentyFiveGigE1/0/11      0x12       1   0   1    30     0      26   14   11   111  NIF  Y
TwentyFiveGigE1/0/12      0x13       1   0   1    31     0      27   15   12   112  NIF  Y
TwentyFiveGigE1/0/13      0x14       1   0   1    4      0      0    4    13   113  NIF  Y
TwentyFiveGigE1/0/14      0x15       1   0   1    5      0      1    5    14   114  NIF  Y
TwentyFiveGigE1/0/15      0x16       1   0   1    6      0      2    6    15   115  NIF  Y
TwentyFiveGigE1/0/16      0x17       1   0   1    7      0      3    7    16   116  NIF  Y
TwentyFiveGigE1/0/17      0x18       1   0   1    8      0      4    8    17   117  NIF  Y
TwentyFiveGigE1/0/18      0x19       1   0   1    9      0      5    9    18   118  NIF  Y
TwentyFiveGigE1/0/19      0x1a       1   0   1    10     0      6    10   19   119  NIF  Y
TwentyFiveGigE1/0/20      0x1b       1   0   1    11     0      7    11   20   120  NIF  Y
TwentyFiveGigE1/0/21      0x1c       1   0   1    12     0      8    12   21   121  NIF  Y
TwentyFiveGigE1/0/22      0x1d       1   0   1    13     0      9    13   22   122  NIF  Y
TwentyFiveGigE1/0/23      0x1e       1   0   1    14     0      10   14   23   123  NIF  Y
TwentyFiveGigE1/0/24      0x1f       1   0   1    15     0      11   15   24   124  NIF  Y
TwentyFiveGigE1/0/25      0x20       0   0   0    20     0      31   4    25   125  NIF  Y
TwentyFiveGigE1/0/26      0x21       0   0   0    21     0      30   5    26   126  NIF  Y
TwentyFiveGigE1/0/27      0x22       0   0   0    22     0      29   6    27   127  NIF  Y
TwentyFiveGigE1/0/28      0x23       0   0   0    23     0      28   7    28   128  NIF  Y
TwentyFiveGigE1/0/29      0x24       0   0   0    24     0      27   8    29   129  NIF  Y
TwentyFiveGigE1/0/30      0x25       0   0   0    25     0      26   9    30   130  NIF  Y
TwentyFiveGigE1/0/31      0x26       0   0   0    26     0      25   10   31   131  NIF  Y
TwentyFiveGigE1/0/32      0x27       0   0   0    27     0      24   11   32   132  NIF  Y
TwentyFiveGigE1/0/33      0x28       0   0   0    28     0      23   12   33   133  NIF  Y
TwentyFiveGigE1/0/34      0x29       0   0   0    29     0      22   13   34   134  NIF  Y
TwentyFiveGigE1/0/35      0x2a       0   0   0    30     0      21   14   35   135  NIF  Y
TwentyFiveGigE1/0/36      0x2b       0   0   0    31     0      20   15   36   136  NIF  Y
TwentyFiveGigE1/0/37      0x2c       0   0   0    4      0      15   4    37   137  NIF  Y
TwentyFiveGigE1/0/38      0x2d       0   0   0    5      0      14   5    38   138  NIF  Y
TwentyFiveGigE1/0/39      0x2e       0   0   0    6      0      13   6    39   139  NIF  Y
TwentyFiveGigE1/0/40      0x2f       0   0   0    7      0      12   7    40   140  NIF  Y
TwentyFiveGigE1/0/41      0x30       0   0   0    8      0      11   8    41   141  NIF  Y
TwentyFiveGigE1/0/42      0x31       0   0   0    9      0      10   9    42   142  NIF  Y
TwentyFiveGigE1/0/43      0x32       0   0   0    10     0      9    10   43   143  NIF  Y
TwentyFiveGigE1/0/44      0x33       0   0   0    11     0      8    11   44   144  NIF  Y
TwentyFiveGigE1/0/45      0x34       0   0   0    12     0      7    12   45   145  NIF  Y
TwentyFiveGigE1/0/46      0x35       0   0   0    13     0      6    13   46   146  NIF  Y
TwentyFiveGigE1/0/47      0x36       0   0   0    14     0      5    14   47   147  NIF  Y
TwentyFiveGigE1/0/48      0x37       0   0   0    15     0      4    15   48   148  NIF  Y
HundredGigE1/0/49         0x3e       1   0   1    0      0      12   2    49   149  NIF  Y
HundredGigE1/0/50         0x39       1   0   1    16     0      28   2    50   150  NIF  Y
HundredGigE1/0/51         0x3a       0   0   0    16     0      16   2    51   151  NIF  Y
HundredGigE1/0/52         0x3b       0   0   0    0      0      0    2    52   152  NIF  Y

MHM Cisco World · ‎03-06-2023

But I see mac with each interface, can you point which interface (or interfaces) we talk about

jshayne · ‎03-06-2023

Two interfaces I used to connect to the PCs are TwentyFiveGigE1/0/1 and TwentyFiveGigE1/0/4.

But their MAC tables are empty:

show mac address-table int TwentyFiveGigE1/0/1
          Mac Address Table
-------------------------------------------

Vlan    Mac Address       Type        Ports
----    -----------       --------    -----

show mac address-table int TwentyFiveGigE1/0/4
          Mac Address Table
-------------------------------------------

Vlan    Mac Address       Type        Ports
----    -----------       --------    -----

MHM Cisco World · ‎03-06-2023

show platform software fed active ifm if-id 0x8
show platform software fed active ifm if-id 0xb

from the the above get the interface pointer

Interface Block Pointer : 0x1111111111

then use value above in command below and share result here.

show platform hardware fed active fwd-asic abstraction print-resource-handle 0x11111111111 1

thanks a lot

MHM Cisco World · ‎03-06-2023

Starting in Cisco IOS XE Gibraltar 16.11.1, the C9500-32C, C9500-32QC, C9500-48Y4C, and C9500-24Y4C models of the Cisco Catalyst 9500 Series Switches bootup with interfaces in the default Layer 2 state. In all earlier releases, the default is Layer 3. (For all other models of the Cisco Catalyst 9500 Series Switches, the default interface continues to be Layer 2)

if you run 9500 with earlier than 16.11.x then the default is L3 not L2
try add
switchport to these two port and check again
if this not success please share the output I ask earlier

jshayne · ‎03-06-2023

The current IOS XE is 17.3.4 and therefore should be fine. I added switchport to the switch interfaces. The problem still remains.

I tried the command "

show platform hardware fed active fwd-asic abstraction print-resource-handle [Interface Resource Pointer] 1

But I got the error "[Interface Resource Pointer] not valid handle".

"

show platform software fed active ifm if-id 0x8
Interface IF_ID         : 0x0000000000000008
Interface Name          : TwentyFiveGigE1/0/1
Interface Block Pointer : 0x7f35643feab8
Interface Block State   : READY
Interface State         : Enabled
Interface Status        : ADD, UPD
Interface Ref-Cnt       : 3
Interface Type          : ETHER
        Port Type         : SWITCH PORT
        Port Location     : LOCAL
        Slot              : 1
        Unit              : 0
        Slot Unit         : 1
        SNMP IF Index     : 3
        GPN               : 101
        EC Channel        : 0
        EC Index          : 0
        Port Handle       : 0xae000065
        LISP v4 Mobility  : false
        LISP v6 Mobility  : false
        QoS Trust Type    : 3

        Port Information
        Handle ............ [0xae000065]
        Type .............. [Layer2]
        Identifier ........ [0x8]
        Slot .............. [1]
        Unit .............. [1]
        Port Physical Subblock
                Affinity .......... [local]
                Asic Instance ..... [1 (A:0,C:1)]
                AsicPort .......... [20]
                AsicSubPort ....... [0]
                MacNum ............ [16]
                ContextId ......... [4]
                LPN ............... [1]
                GPN ............... [101]
                Speed ............. [25GB]
                type .............. [NIF]
                PORT_LE ........... [0x7f35644133d8]
                L3IF_LE ........... [0x0]
                DI ................ [0x7f3564405b78]
                SubIf count ....... [0]
        Port QoS Subblock
                Trust Type .................... [0x2]
                Default Value ................. [0]
                Ingress Table Map ............. [0x0]
                Egress Table Map .............. [0x0]
                Queue Map ..................... [0x0]
        Port Policy Subblock
        List of Ingress Policies attached to an interface
        List of Egress Policies attached to an interface
        Openflow Subblock
                Enabled ............... [No]
                Mirror Count .......... [0]
Ref Count : 3 (feature Ref Counts + 1)
IFM Feature Ref Counts
        FID : 59 (AAL_FEATURE_NETFLOW_VACL), Ref Count : 1
        FID : 41 (AAL_FEATURE_QOS), Ref Count : 1
IFM Feature Sub block information
        FID : 59 (AAL_FEATURE_NETFLOW_VACL), Private Data Ptr: 0x7f3564901058
        FID : 41 (AAL_FEATURE_QOS), Private Data Ptr: 0x7f3564412cd8

show platform software fed active ifm if-id 0xb
Interface IF_ID         : 0x000000000000000b
Interface Name          : TwentyFiveGigE1/0/4
Interface Block Pointer : 0x7f356443be98
Interface Block State   : READY
Interface State         : Enabled
Interface Status        : ADD, UPD
Interface Ref-Cnt       : 3
Interface Type          : ETHER
        Port Type         : SWITCH PORT
        Port Location     : LOCAL
        Slot              : 1
        Unit              : 0
        Slot Unit         : 4
        SNMP IF Index     : 6
        GPN               : 104
        EC Channel        : 0
        EC Index          : 0
        Port Handle       : 0xb7000067
        LISP v4 Mobility  : false
        LISP v6 Mobility  : false
        QoS Trust Type    : 3

        Port Information
        Handle ............ [0xb7000067]
        Type .............. [Layer2]
        Identifier ........ [0xb]
        Slot .............. [1]
        Unit .............. [4]
        Port Physical Subblock
                Affinity .......... [local]
                Asic Instance ..... [1 (A:0,C:1)]
                AsicPort .......... [23]
                AsicSubPort ....... [0]
                MacNum ............ [19]
                ContextId ......... [7]
                LPN ............... [4]
                GPN ............... [104]
                Speed ............. [25GB]
                type .............. [NIF]
                PORT_LE ........... [0x7f356444f998]
                L3IF_LE ........... [0x0]
                DI ................ [0x7f3564442e38]
                SubIf count ....... [0]
        Port QoS Subblock
                Trust Type .................... [0x2]
                Default Value ................. [0]
                Ingress Table Map ............. [0x0]
                Egress Table Map .............. [0x0]
                Queue Map ..................... [0x0]
        Port Policy Subblock
        List of Ingress Policies attached to an interface
        List of Egress Policies attached to an interface
        Openflow Subblock
                Enabled ............... [No]
                Mirror Count .......... [0]
Ref Count : 3 (feature Ref Counts + 1)
IFM Feature Ref Counts
        FID : 59 (AAL_FEATURE_NETFLOW_VACL), Ref Count : 1
        FID : 41 (AAL_FEATURE_QOS), Ref Count : 1
IFM Feature Sub block information
        FID : 59 (AAL_FEATURE_NETFLOW_VACL), Private Data Ptr: 0x7f35648f88b8
        FID : 41 (AAL_FEATURE_QOS), Private Data Ptr: 0x7f356444f448

MHM Cisco World · ‎03-07-2023

total Port L2 sublock is missing, this is L3 not L2 as I think.
also I see there is VACL and QoS policy apply to port ??

please do try the following again for each interface

shut
default address IP <<- if you assign IP to interface.
switchport
no shut

Port L2 Subblock
           Enabled ............. [Yes]
           Allow dot1q ......... [Yes] 
           Allow native ........ [Yes]
           Default VLAN ........ [1]
           Allow priority tag ... [Yes]
           Allow unknown unicast [Yes]
           Allow unknown multicast[Yes]
           Allow unknown broadcast[Yes]
           Allow unknown multicast[Enabled]
           Allow unknown unicast [Enabled]
           IPv4 ARP snoop ....... [No]
           IPv6 ARP snoop ....... [No]
           Jumbo MTU ............ [1500]
           Learning Mode ........ [1]