Not able to SSH or telnet to Switch(2nd) Which is in Port-channel

san ju. · ‎12-08-2023

Hello All,

In my organization, we have two switches, SW1 (Nexus 6001) and SW2 (Catalyst 3750), connected via a Portchannel. Ports 23 and 24 are utilized in the port-channel setup on both switches. Everything was functioning well, but today morning I encountered an issue, I couldn't log in to the second switch (SW2) using telnet or SSH. Surprisingly, I'm receiving ICMP responses from SW2 when pinging it from switch1.

To address this, I've cross-checked the VLAN configuration on the Port-channel and the VTY settings on SW2 using the configuration backup from the previous day. Additionally, I've reviewed the ACL on the VTY, confirming that it permits access to SW2 from my LAN IP.

After running the "show log" command on sw1, I discovered an error message indicating:

 entry number 42647: ETHPORT-3-IF_UNSUPPORTED_TRANSCEIVER
Transceiver on interface Ethernet1/24 is not supported

despite having applied the service unsupported-transceiver globally." Also its been there for a while!.

I'm currently at a loss on how to proceed with troubleshooting this connectivity issue.

I would greatly appreciate any assistance.

Config reference:

LAXSWTCHNX01# sho run int po1
version 7.3(5)N1(1)

interface port-channel1
  description TRUNK to LAXSWTCH02
  switchport mode trunk
  switchport trunk allowed vlan 1,10-13,16-20,92-93,95-96,99-200,300,333,444,500-550,700-701,900
  speed 1000
  duplex full

LAXSWTCHNX01# sho port-channel sum
--------------------------------------------------------------------------------
Group Port-       Type     Protocol  Member Ports
      Channel
--------------------------------------------------------------------------------
1     Po1(SU)     Eth      LACP      Eth1/23(D)   Eth1/24(P)

LAXSWTCHNX01# sho run int Eth1/23-24

version 7.3(5)N1(1)

interface Ethernet1/23
  description TRUNK TO LAXSWTCH02
  switchport mode trunk
  switchport trunk allowed vlan 1,10-13,16-20,92-93,95-96,99-200,300,333,444,500-550,700-701,900
  speed 1000
  duplex full
  channel-group 1 mode active

interface Ethernet1/24
  description TRUNK TO LAXSWTCH02
  switchport mode trunk
  switchport trunk allowed vlan 1,10-13,16-20,92-93,95-96,99-200,300,333,444,500-550,700-701,900
  speed 1000
  duplex full
  channel-group 1 mode active
 
================
SW2

interface Port-channel1
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 1,10-13,16-20,92,93,95,96,99-200,300,333,444
 switchport trunk allowed vlan add 500-550,700,701,900
 switchport mode trunk

Temporarily shut down one member port (ignore it)!!

Thanks,

Menon

MHM Cisco World · ‎12-08-2023

show etherchannel summary
share output of above for 3750

MHM

san ju. · ‎12-08-2023

Hello there,

As mentioned earlier, I'm unable to log in to SW2 (3750) from SW1 or any other devices. All I'm receiving is an ICMP response.

Am trying Vlan92
SW1- 10.5.6.1 & SW2 - 10.5.6.2

Thanks.

MHM Cisco World · ‎12-08-2023

even via Console ?
MHM

san ju. · ‎12-08-2023

The switch is situated remotely, and we haven't attempted a console connection yet. Additionally, these switches are of lower priority. As a final resort, I'm considering a physical visit. I want to know is there anyway that i can fix it remotely from current perspective?

Thanks.

Richard Burts · ‎12-08-2023

Am I correct in assuming that devices connected to SW2 still have connectivity to the resources they need (including through SW1)? That would indicate that you are ok at layer 2 and that the problem is something at layer 3 (or perhaps at layer 4)?

On SW1 what do you get in the output of show cdp neighbor detail? In particular does the output show the same IP address that you are attempting to access?

HTH

Rick

san ju. · ‎12-09-2023

Hello Richard,

The devices linked to SW2, including SW1 and several Dell servers, are operating smoothly. Some of these servers utilize SW2 for ceph traffic, and their ICMP connections are also functioning well. It seems the L2 has no issue.

Furthermore, when using the CDP neighbor command, I'm unable to retrieve details about VLAN 92 and the IP address 10.5.6.2 that I'm attempting to access.

LAXSWTCHNX01# show cdp neighbor detail
----------------------------------------
Device ID:SWTCH02.007rack.net
VTP Management Domain Name: >

Interface address(es):
    IPv4 Address: 10.1.21.2
Platform: WS-C3750G-24T, Capabilities: Router Switch IGMP Filtering 
Interface: mgmt0, Port ID (outgoing port): GigabitEthernet1/0/10
Holdtime: 169 sec

Version:
Cisco IOS Software, C3750 Software (C3750-ADVIPSERVICESK9-M), Version 12.2(46)SE, RELEASE SOFTWARE (fc2)
Copyright (c) 1986-2008 by Cisco Systems, Inc.
Compiled Thu 21-Aug-08 15:43 by nachen

Advertisement Version: 2

Native VLAN: 1
Duplex: full
Mgmt address(es):
    IPv4 Address: 10.1.21.2
----------------------------------------
Device ID:SWTCH02.007rack.net
VTP Management Domain Name: >

Interface address(es):
    IPv4 Address: 10.1.21.2
Platform: WS-C3750G-24T, Capabilities: Router Switch IGMP Filtering 
Interface: Ethernet1/1, Port ID (outgoing port): GigabitEthernet1/0/1
Holdtime: 169 sec

Version:
Cisco IOS Software, C3750 Software (C3750-ADVIPSERVICESK9-M), Version 12.2(46)SE, RELEASE SOFTWARE (fc2)
Copyright (c) 1986-2008 by Cisco Systems, Inc.
Compiled Thu 21-Aug-08 15:43 by nachen

Advertisement Version: 2

Native VLAN: 13
Duplex: full
Mgmt address(es):
    IPv4 Address: 10.1.21.2
----------------------------------------
Device ID:SWTCH02.007rack.net
VTP Management Domain Name: >

Interface address(es):
    IPv4 Address: 10.1.21.2
    IPv6 Address: fe80::222:bdff:fe1d:414f
    IPv6 Address: 2602:ffa6:300::9999:4
Platform: WS-C3750G-24T, Capabilities: Router Switch IGMP Filtering 
Interface: Ethernet1/24, Port ID (outgoing port): GigabitEthernet1/0/24
Holdtime: 165 sec

Version:
Cisco IOS Software, C3750 Software (C3750-ADVIPSERVICESK9-M), Version 12.2(46)SE, RELEASE SOFTWARE (fc2)
Copyright (c) 1986-2008 by Cisco Systems, Inc.
Compiled Thu 21-Aug-08 15:43 by nachen

Advertisement Version: 2

Native VLAN: 1
Duplex: full
Mgmt address(es):
    IPv4 Address: 10.1.21.2

Thanks.

Richard Burts · ‎12-09-2023

Thanks for the output of show cdp neighbor detail. It does confirm that the switches are talking to each other successfully. If you are not able to access the switch using address 10.5.6.2, could you try accessing the switch using address 10.1.21.2? If that is successful it might be helpful to see the output of show ip interface brief from that switch.

HTH

Rick

san ju. · ‎12-09-2023

I attempted to use the IP 10.1.21.2, but the device couldn't find a route. Nonetheless, I'll provide you with the additional details.

LAXSWTCHNX01# ssh 10.1.21.2
ssh: connect to host 10.1.21.2 port 22: No route to host
LAXSWTCHNX01# telnet 10.1.21.2
Trying 10.1.21.2...
telnet: Unable to connect to remote host: No route to host
LAXSWTCHNX01# sho ip int brie
IP Interface Status for VRF "default"(1)
Interface            IP Address      Interface Status
Vlan20               10.1.20.2       protocol-up/link-up/admin-up       
Vlan92               10.5.6.1        protocol-up/link-up/admin-up       
Vlan93               172.16.5.1      protocol-up/link-up/admin-up            
LAXSWTCHNX01# 
LAXSWTCHNX01# sho run int vlan 20

interface Vlan20
  no shutdown
  ip address 10.1.20.2/24
  ipv6 address 2602:ffa6:3ff:ffff::201/127

Thanks.

Richard Burts · ‎12-09-2023

Menon

Thanks for the information. It sheds a little light but I still am puzzled about what is causing this behavior. Would you post the output of the commands show interface trunk show arp and show ip route?

HTH

Rick

san ju. · ‎12-09-2023

I understand it confusing be a bit. Please see the desired outcomes listed below.

LAXSWTCHNX01# show interface trunk 

--------------------------------------------------------------------------------
Port          Native  Status        Port
              Vlan                  Channel
--------------------------------------------------------------------------------
Eth1/3        1       trunking      --
Eth1/7        1       trunking      --
Eth1/9        1       trunking      --
Eth1/10       1       trunking      --
Eth1/12       1       trunking      --
Eth1/14       1       trunking      --
Eth1/15       1       trnk-bndl     Po10
Eth1/16       1       trnk-bndl     Po10
Eth1/17       1       trnk-bndl     Po11
Eth1/18       1       trnk-bndl     Po11
Eth1/19       1       trnk-bndl     Po12
Eth1/20       1       trnk-bndl     Po12
Eth1/21       1       trunking      --
Eth1/22       99      trunking      --
Eth1/23       1       trnk-bndl     Po1
Eth1/24       1       trnk-bndl     Po1
Po1           1       trunking      --
Po10          1       trunking      --
Po11          1       trunking      --
Po12          1       trunking      --

--------------------------------------------------------------------------------
Port          Vlans Allowed on Trunk
--------------------------------------------------------------------------------
Eth1/3        10,12
Eth1/7        92-93,95,122-123,169,444,521,600-603
Eth1/9        92-93,95,178-179,444,505,521,700
Eth1/10       92-93,95,178-179,444,505,521,700
Eth1/12       92-93,444,521
Eth1/14       92-93,178-179,444,505,521,700
Eth1/15       12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Eth1/16       12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Eth1/17       12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Eth1/18       12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Eth1/19       92-93,95,122-123,169,444,521,600-603
Eth1/20       92-93,95,122-123,169,444,521,600-603
Eth1/21       12,92,95,99,122-123,169,177-178,300,444,504,520-521,700-701
Eth1/22       12,92,95,99,122-123,169,177-178,300,444,504,520-521,700-701
Eth1/23       1,10-13,16-20,92-93,95-96,99-200,300,333,444,500-550,700-701,900
Eth1/24       1,10-13,16-20,92-93,95-96,99-200,300,333,444,500-550,700-701,900
Po1           1,10-13,16-20,92-93,95-96,99-200,300,333,444,500-550,700-701,900
Po10          12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Po11          12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Po12          92-93,95,122-123,169,444,521,600-603

--------------------------------------------------------------------------------
Port          Vlans Err-disabled on Trunk
--------------------------------------------------------------------------------
Eth1/3        none
Eth1/7        none
Eth1/9        none
Eth1/10       none
Eth1/12       none
Eth1/14       none
Eth1/15       none
Eth1/16       none
Eth1/17       none
Eth1/18       none
Eth1/19       none
Eth1/20       none
Eth1/21       none
Eth1/22       none
Eth1/23       none
Eth1/24       none
Po1           none
Po10          none
Po11          none
Po12          none

--------------------------------------------------------------------------------
Port          STP Forwarding
--------------------------------------------------------------------------------
Eth1/3        none
Eth1/7        92-93,95,122-123,169,444,521,600-603
Eth1/9        92-93,95,178-179,444,505,521,700
Eth1/10       92-93,95,178-179,444,505,521,700
Eth1/12       92-93,444,521
Eth1/14       92-93,178-179,444,505,521,700
Eth1/15       12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Eth1/16       12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Eth1/17       12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Eth1/18       12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Eth1/19       92-93,95,122-123,169,444,521,600-603
Eth1/20       92-93,95,122-123,169,444,521,600-603
Eth1/21       none
Eth1/22       12,92,95,99,122-123,169,177-178,300,444,504,520-521,700
Eth1/23       1,10,12-13,20,92-93,95,99,122-123,169,177-179,300,333,444,504-505,520-521,700,900
Eth1/24       1,10,12-13,20,92-93,95,99,122-123,169,177-179,300,333,444,504-505,520-521,700,900
Po1           1,10,12-13,20,92-93,95,99,122-123,169,177-179,300,333,444,504-505,520-521,700,900
Po10          12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Po11          12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Po12          92-93,95,122-123,169,444,521,600-603

--------------------------------------------------------------------------------
Port          Vlans in spanning tree forwarding state and not pruned
--------------------------------------------------------------------------------
Eth1/3        none
Eth1/7        92-93,95,122-123,169,444,521,600-603
Eth1/9        92-93,95,178-179,444,505,521,700
Eth1/10       92-93,95,178-179,444,505,521,700
Eth1/12       92-93,444,521
Eth1/14       92-93,178-179,444,505,521,700
Eth1/15       12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Eth1/16       12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Eth1/17       12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Eth1/18       12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Eth1/19       92-93,95,122-123,169,444,521,600-603
Eth1/20       92-93,95,122-123,169,444,521,600-603
Eth1/21       none
Eth1/22       12,92,95,99,122-123,169,177-178,300,444,504,520-521,700
Eth1/23       1,10,12-13,20,92-93,95,99,122-123,169,177-179,300,333,444,504-505,520-521,700,900
Eth1/24       1,10,12-13,20,92-93,95,99,122-123,169,177-179,300,333,444,504-505,520-521,700,900
Po1           1,10,12-13,20,92-93,95,99,122-123,169,177-179,300,333,444,504-505,520-521,700,900
Po10          12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Po11          12-13,92-93,95,122-123,169,177-178,300,444,504,520-521,600-603
Po12          92-93,95,122-123,169,444,521,600-603

====================

LAXSWTCHNX01# show ip route
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.1.20.0/24, ubest/mbest: 1/0, attached
    *via 10.1.20.2, Vlan20, [0/0], 22:51:26, direct
10.1.20.2/32, ubest/mbest: 1/0, attached
    *via 10.1.20.2, Vlan20, [0/0], 22:51:26, local
10.2.6.0/24, ubest/mbest: 1/0
    *via 10.5.6.10, [1/0], 7w5d, static
10.3.6.0/24, ubest/mbest: 1/0
    *via 10.5.6.10, [1/0], 7w5d, static
10.4.6.0/24, ubest/mbest: 1/0
    *via 10.5.6.10, [1/0], 7w5d, static
10.5.6.0/24, ubest/mbest: 1/0, attached
    *via 10.5.6.1, Vlan92, [0/0], 17w6d, direct
10.5.6.1/32, ubest/mbest: 1/0, attached
    *via 10.5.6.1, Vlan92, [0/0], 17w6d, local
10.5.6.3/32, ubest/mbest: 1/0
    *via 10.5.6.3, Vlan92, [0/0], 17w6d, hsrp
10.6.6.0/24, ubest/mbest: 1/0
    *via 10.5.6.10, [1/0], 7w5d, static
10.7.6.0/24, ubest/mbest: 1/0
    *via 10.5.6.10, [1/0], 7w5d, static

=================
LAXSWTCHNX01# sho ip arp

IP ARP Table for context default
Total number of entries: 21
Address         Age       MAC Address     Interface        
10.5.6.2        00:00:23  0022.bd1d.4143  Vlan92          
10.5.6.8        00:00:40  c630.d0bd.3978  Vlan92          
10.5.6.9        00:00:36  6ab3.070a.c510  Vlan92          
10.5.6.10       00:00:14  2e69.b2e1.de32  Vlan92          
10.5.6.28       00:00:35  782b.cb31.656d  Vlan92          
10.5.6.45       00:00:41  ea1a.4205.ec6f  Vlan92          
10.5.6.51       00:00:25  0026.b93a.24c5  Vlan92          
10.5.6.111      00:00:26  4aea.da2b.ac17  Vlan92          
10.5.6.115      00:00:06  c630.d0bd.3978  Vlan92          
10.5.6.240      00:00:34  c658.d5b4.3345  Vlan92          
10.5.6.241      00:00:10  1a0f.0909.cc4c  Vlan92          
10.5.6.242      00:00:39  02a8.c563.c141  Vlan92                   
10.5.6.3           -      0000.0c07.ac5c  Vlan92

I haven't encompassed every outcome. i hope its fine!.

Thanks.

Richard Burts · ‎12-10-2023

Menon

Thank you for the additional information. We continue to look for some explanation for this issue. What we are seeing means pretty clearly that at layer 2 everything is working as expected. And the basics of layer 3 seem ok. I say that particularly based on the fact that arp and ping to the address works. Is SW1 doing anything to manipulate routes like VRF, or perhaps using something like Policy Based Routing that might impact telnet and SSH?

Could you post some of the most recent config that you have for SW2? If you do not want to post the complete config then we would be most interested in all of the aaa config, anything like PBR, any interface acl, and the config of the vty.

Also when you attempt telnet or attempt ssh do you get any response/any prompt?

HTH

Rick

san ju. · ‎12-11-2023

I haven't discovered any information on manipulating routes on SW1. We haven't set up PBR yet. I've attached the recent backup configuration of SW2, and we also possess the SW1 configuration. However, I'm unable to post the complete configuration here due to restrictions. I'd be more than happy to share it via Skype encrypted chat if you support that. Could you please confirm if the provided information is adequate for your needs?

Thanks.

MHM Cisco World · ‎12-11-2023

asymmetric routing <<-
I see there is HSRP, which I think you access the interface you apply HSRP under it ?
this can lead to asymmetric routing (some case) check see if SW is standby
remember the GW of any PC you connect is VIP of HSRP.
MHM

san ju. · ‎12-11-2023

Hi,

Its under HSRP, and the SW1 active,

LAXSWTCHNX01# sho hsrp brie
*:IPv6 group   #:group belongs to a bundle
                     P indicates configured to preempt.
                     |
 Interface   Grp  Prio P State    Active addr      Standby addr     Group addr
  Vlan92      92   200  P Active   local            10.5.6.2         10.5.6.3        (conf)

All private VMs are currently routing through our firewall instead of the Virtual IP (VIP) at 10.5.6.3.

Thanks.