Cisco 2960C management connectivity issues

[J_Rega]

I have a network made up of Cisco 9300, 2960G/C/X/XR and a 3560CX. None of the switches are more than 7 hops away from each other and I can generally ping or SSH into any of the switches without a problem. I have been having issues lately with connecting to the 2960Cs (both are on the same VLAN as the computer I am accessing them on, 1 is one hop from the switch I am connected to and the other is 5). In order to connect, I have to SSH into the switch directly attached to the 2960C I want to connect to, ping it from there, and then I can access it from my computer.

I assume I am having a spanning-tree issue and that the mac entries are being flushed and are causing these issues but I do not know how to figure it out. I am running rapid PVST on all switches and each of the trunk ports is configured as follows:

switchport trunk allowed vlan 5,10-20

switchport trunk native vlan 5

switchport mode trunk

switchport nonegotiate

ip arp inspection trust

Any insight would be greatly appriciated!

[Reza Sharifi]

Do you have redundant uplinks on these switches? STP usually blocks secondary connections/uplinks to the same device to prevent loops on the network. Also, do you have the root bridge configured correctly? It may be helpful if you can provide a diagram.

Also, the trunk port config you posted looks correct. Have you tried removing the below command from the trunk ports and test connectivity?

ip arp inspection trust

HTH

[J_Rega]

Reza,

We do have redundant uplinks on some of the switches. I have checked for inconsistent ports and so far have found none. I need to do some research to see if I have the root bridge configured correctly as I have not modified the configs to change any root bridge settings.

I have removed ip arp inspection trust before and the problem remained.

Rough diagram attached.

[Reza Sharifi]

Hi,

Thanks for providing the diagram. The only place I see 2 links (potential for loop) between any 2 switches is the connectivity from the 9300 to 2960G. All the other switches are singly connected and you should not see any loop. Is STP blocking one of the links between the 9300 and the 2960G? Also, how many VLANs do you have in this network and what device does the routing between them?

HTH

[J_Rega]

STP is blocking one of the loops. I have removed the redundant link in the past to see if that was causing the issue but the problem remained.

We have a total of 20 VLANs and use intervlan routing which is performed by the 9300.

[Reza Sharifi]

Do the switches with this issue have only one IP address configured for management only? Can you post "sh run" from one of them?

HTH

[J_Rega]

Sorry it took so long to respond, here is an edited sh run (I have been setting "switchport block unicast" on the trunk ports as well, but currently have it removed):

version 15.2
no service pad
service tcp-keepalives-in
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
service password-encryption
service unsupported-transceiver
no service dhcp
!
hostname 2960C-8TC-L
!
boot-start-marker
boot-end-marker
!
!
logging userinfo
logging buffered 40960 informational
enable secret 5 XXXXXXXXXXXXXXXXXXXXXXXXX
!
username XXXXXXXXX privilege 15 secret 5 XXXXXXXXXXXXXXXXXXXXXXX
aaa new-model
!
!
aaa authentication login default local
!
aaa common-criteria policy PASSWORDS
 min-length 1
 max-length 127
 numeric-count 1
 upper-case 1
 lower-case 1
 special-case 1
 char-changes 1
!
aaa session-id common
system mtu routing 1500
no ip source-route
!
ip dhcp snooping vlan 10-20
ip dhcp snooping
no ip domain-lookup
ip domain-name XXXXXXXXXXXXXXX
login block-for 1800 attempts 3 within 120
login on-failure log
login on-success log
vtp mode off
!
authentication mac-move permit
!
crypto pki trustpoint 1
 XXXXXXXXXXXXXXX
!
crypto pki certificate chain 1
 certificate XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
  	quit
 certificate ca XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
  	quit
archive
 log config
  logging enable
!
spanning-tree mode rapid-pvst
spanning-tree loopguard default
spanning-tree portfast edge default
spanning-tree extend system-id
spanning-tree vlan 1-4094 priority 61440
no errdisable detect cause gbic-invalid
!
vlan internal allocation policy ascending
!
vlan 5
 name Trunk
!
vlan 10
 name Unused
!
vlan 11
 name XXXXXXXXXXXXXXX
!
vlan 12
 name XXXXXXXXXXXXXXX
!
vlan 13
 name XXXXXXXXXXXXXXX
!
vlan 14
 name XXXXXXXXXXXXXXX
!
vlan 15
 name XXXXXXXXXXXXXXX
!
vlan 16
 name XXXXXXXXXXXXXXX
!
vlan 17
 name XXXXXXXXXXXXXXX
!
vlan 18
 name XXXXXXXXXXXXXXX
!
vlan 19
 name XXXXXXXXXXXXXXX
!
vlan 20
 name Management
!
interface Loopback1
 no ip address
 no ip route-cache
 downshift disable
!
interface FastEthernet0/1
 description Trunk To 2960G
 switchport trunk allowed vlan 5,11-20
 switchport trunk native vlan 5
 switchport mode trunk
 switchport nonegotiate
 ip arp inspection trust
!
interface FastEthernet0/2
 description XXXXXXXXXXXXXXX
 switchport access vlan 12
 switchport mode access
 switchport block unicast
 switchport port-security mac-address sticky
 switchport port-security mac-address sticky XXXXXXXXXXXXXXX
 switchport port-security
 ip device tracking maximum 1
 no cdp enable
 storm-control broadcast level bps 30m
 spanning-tree bpduguard enable
 spanning-tree guard root
 ip verify source tracking port-security
!
interface FastEthernet0/3
 description XXXXXXXXXXXXXXX
 switchport access vlan 14
 switchport mode access
 switchport block unicast
 switchport port-security maximum 5
 switchport port-security mac-address sticky
 ip device tracking maximum 5
 no cdp enable
 storm-control broadcast level bps 30m
 spanning-tree bpduguard enable
 spanning-tree guard root
 ip verify source tracking port-security
!
interface FastEthernet0/4
 description XXXXXXXXXXXXXXX
 switchport access vlan 11
 switchport mode access
 switchport block unicast
 switchport port-security mac-address sticky
 switchport port-security mac-address sticky XXXXXXXXXXXXXXX
 switchport port-security
 ip device tracking maximum 1
 no cdp enable
 storm-control broadcast level bps 30m
 spanning-tree bpduguard enable
 spanning-tree guard root
 ip verify source tracking port-security
!
interface FastEthernet0/5
 description XXXXXXXXXXXXXXX
 switchport access vlan 14
 switchport mode access
 switchport block unicast
 switchport port-security mac-address sticky
 switchport port-security mac-address sticky XXXXXXXXXXXXXXX
 switchport port-security
 ip device tracking maximum 1
 no cdp enable
 storm-control broadcast level bps 30m
 spanning-tree bpduguard enable
 spanning-tree guard root
 ip verify source tracking port-security
!
interface FastEthernet0/6
 description XXXXXXXXXXXXXXX
 switchport access vlan 11
 switchport mode access
 switchport block unicast
 switchport port-security mac-address sticky
 switchport port-security mac-address sticky XXXXXXXXXXXXXXX
 switchport port-security
 ip device tracking maximum 1
 no cdp enable
 storm-control broadcast level bps 30m
 spanning-tree bpduguard enable
 spanning-tree guard root
 ip verify source tracking port-security
!
interface FastEthernet0/7
 description XXXXXXXXXXXXXXX
 switchport access vlan 11
 switchport mode access
 switchport block unicast
 switchport port-security mac-address sticky
 switchport port-security
 ip device tracking maximum 1
 no cdp enable
 storm-control broadcast level bps 30m
 spanning-tree bpduguard enable
 spanning-tree guard root
 ip verify source tracking port-security
!
interface FastEthernet0/8
 description XXXXXXXXXXXXXXX
 switchport access vlan 11
 switchport mode access
 switchport block unicast
 switchport port-security mac-address sticky
 switchport port-security
 ip device tracking maximum 1
 no cdp enable
 storm-control broadcast level bps 30m
 spanning-tree bpduguard enable
 spanning-tree guard root
 ip verify source tracking port-security
!
interface GigabitEthernet0/1
 description Unused
 switchport access vlan 10
 switchport mode access
 switchport block unicast
 switchport port-security mac-address sticky
 switchport port-security
 ip device tracking maximum 1
 shutdown
 no cdp enable
 storm-control broadcast level bps 30m
 spanning-tree bpduguard enable
 spanning-tree guard root
 ip verify source tracking port-security
!
interface GigabitEthernet0/2
 description Unused
 switchport access vlan 10
 switchport mode access
 switchport block unicast
 switchport port-security mac-address sticky
 switchport port-security
 ip device tracking maximum 1
 shutdown
 no cdp enable
 storm-control broadcast level bps 30m
 spanning-tree bpduguard enable
 spanning-tree guard root
 ip verify source tracking port-security
!
interface Vlan1
 description DEFAULT
 no ip address
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 no ip route-cache
 shutdown
!
interface Vlan5
 description Trunk VLAN
 no ip address
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 no ip route-cache
!
interface Vlan10
 description UNUSED
 no ip address
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 no ip route-cache
 shutdown
!
interface Vlan11
 description XXXXXXXXXXXXXXX
 no ip address
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 no ip route-cache
!
interface Vlan12
 description XXXXXXXXXXXXXXX
 no ip address
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 no ip route-cache
!
interface Vlan13
 description XXXXXXXXXXXXXXX
 no ip address
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 no ip route-cache
!
interface Vlan14
 description XXXXXXXXXXXXXXX
 no ip address
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 no ip route-cache
!
interface Vlan20
 description Management VLAN
 ip address X.X.0.101 255.255.254.0
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 no ip route-cache
!
ip default-gateway X.X.1.254
!
no ip ftp passive
ip ftp source-interface Loopback1
ip tcp synwait-time 10
no ip http server
ip http banner
ip http secure-server
ip http secure-ciphersuite aes-256-cbc-sha edche-rsa-aes-256-cbc-sha 
ip http tls-version TLSv1.2 
ip http secure-trustpoint 1
ip http max-connections 15
ip http timeout-policy idle 1200 life 3000 requests 86400 
ip ssh maxstartups 2
ip ssh time-out 60
ip ssh rsa keypair-name XXXXXXXXXXXXXXX
ip ssh version 2
ip ssh server algorithm mac hmac-sha2-256 hmac-sha2-512
!
access-list 22 permit X.X.0.0 0.0.254.255
access-list 22 deny   any log
access-list 23 permit X.X.12.254
access-list 23 deny   any log
access-list 24 deny   any log
!
banner motd 
This is a banner.
!
line con 0
 exec-timeout 10 0
line vty 0 4
 access-class 22 in
 exec-timeout 10 0
 logging synchronous
 transport input ssh
line vty 5 15
 access-class 22 in
 exec-timeout 10 0
 logging synchronous
 transport input ssh
!
ntp authentication-key 1 md5 XXXXXXXXXXXXXXX
ntp trusted-key 1
ntp access-group peer 23
ntp access-group serve-only 24
ntp access-group query-only 24
ntp server X.X.12.254 key 1
end