Hello All,
We've recently moved to a 3rd Party (*non-AnyConnect) Always-On VPN Cloud service. Issue only occurs while I have the VPN service enabled on my PC (*IT Dept can temporarily disable connection).
In the location in question, we have a Cisco 9500 stack that is our "Core switch", and then a stack of 3 x Cisco 9200s. The VPN provider's on-site box connects directly to the 9200s. But, the LAN addresses they are using to route internally have their Gateway configured on the 9500 core (*Int Vlan100).
Now, if I'm connected to the VPN service and I attempt to SSH or even Ping the 9200, I cannot. A traceroute from my PC to that 9200's address just dies at the Vlan Gateway address on the Core that's used for the LAN addresses of the VPN boxes. Hopefully that made sense...
From the 9500 CLI, if I attempt to ping the 9200 and source that Vlan address, the pings failed. So I added a route on the 9200 pointing back to the 9500 address.After that, the ping started working from the 9500 to the 9200. But, I still cannot SSH or ping the 9200 from my PC. I then added a new Vlan on the 9200 on the same subnet as that Vlan gateway on the 9500, and removed that static route and the pings between the switches still work. But still no SSH or ping from my PC.
A lot of hands have been in the configs since our main networking guy left the company so some of the routes configured are a bit confusing... So I'm going to try and only include the relevant configuration parts.
One odd note is that there's a 3rd 9200 switch, called "9200-TOR" in the config below that I have no problem pinging and SSH to. So I'm including some of those bits as well...
9500 Relevant Config:
9500-CORE#show run
Building configuration...
version 17.3
!
!
!...........cut...........
!
!
ip dhcp pool VPNDHCP
network 10.40.100.0 255.255.255.0
default-router 10.40.100.1
dns-server 8.8.8.8 8.8.4.4
!
!
!...........cut...........
!
!
interface Port-channel1
description 9200-Access
switchport mode trunk
!
interface Port-channel2
description 9200-TOR
switchport mode trunk
!
interface GigabitEthernet0/0
vrf forwarding Mgmt-vrf
no ip address
negotiation auto
!
!
!...........cut...........
!
!
interface TwentyFiveGigE1/0/10
description 9200-TOR
switchport mode trunk
channel-group 2 mode active
!
!...........cut...........
!
interface TwentyFiveGigE1/0/23
description 9200-Access
switchport mode trunk
channel-group 1 mode active
!
!...........cut...........
!
interface TwentyFiveGigE2/0/4
description To 9200-Access
switchport access vlan 208
switchport mode access
no keepalive
!
!...........cut...........
!
interface TwentyFiveGigE2/0/10
description 9200-TOR
switchport mode trunk
no keepalive
channel-group 2 mode active
!
!...........cut...........
!
interface TwentyFiveGigE2/0/23
description 9200-Access
switchport mode trunk
no keepalive
channel-group 1 mode active
!
!
!...........cut...........
!
!
interface Vlan100
description On-site VPN Device LAN Gateway
ip address 10.40.100.1 255.255.255.0
!
interface Vlan123
ip address 10.40.123.1 255.255.255.0
!
interface Vlan254
ip address 10.255.254.2 255.255.255.0
!
!
!...........cut...........
!
!
ip route 0.0.0.0 0.0.0.0 10.255.254.1 !--> SD-WAN
ip route 172.21.0.0 255.255.255.0 10.40.100.251 !--> 172.21 are the client addresses while connected to VPN
!
!
!...........cut...........
!
!
!
end
9200 Relevant Config:
9200-Access#show run
Building configuration...
version 16.11
!
!
switch 1 provision c9200-48p
switch 2 provision c9200-48p
switch 3 provision c9200-24t
!
!
!
interface Port-channel1
description Core Uplink
switchport mode trunk
!
interface GigabitEthernet0/0
vrf forwarding Mgmt-vrf
no ip address
shutdown
speed 1000
negotiation auto
!
!
!...........cut...........
!
!
interface TenGigabitEthernet1/1/1
description Core Uplink
switchport mode trunk
channel-group 1 mode active
!
!
!...........cut...........
!
!
interface GigabitEthernet2/0/29
description VPN - LAN
switchport access vlan 100
switchport mode access
!
!...........cut...........
!
interface GigabitEthernet2/0/39
description VPN - WAN
switchport access vlan 100
switchport mode access
!
!
interface TenGigabitEthernet2/1/1
description Core Uplink
switchport mode trunk
channel-group 1 mode active
!
!...........cut...........
!
interface GigabitEthernet3/0/1
description VPN 2 - LAN
switchport access vlan 100
switchport mode access
!
interface GigabitEthernet3/0/2
description VPN 2 - WAN
switchport access vlan 100
switchport mode access
!
!...........cut...........
!
interface Vlan100
description ***THIS IS NEW AS OF TODAY***
ip address 10.40.100.10 255.255.255.0
!
interface Vlan123
ip address 10.40.123.2 255.255.255.0
!
ip route 0.0.0.0 0.0.0.0 10.60.110.2 !-> Our EPL link subnet. Honestly don't know why this default route...
ip route 10.40.100.0 255.255.255.0 10.40.123.1 !-> I've since removed this bc Vlan100 above was added... But having this without Vlan100 above allowed pings to work between switches.
!
!
!...........cut...........
!
!
end
Here's some output from my PC:
C:\Windows\System32>
C:\Windows\System32>ping 10.40.123.1
Pinging 10.40.123.1 with 32 bytes of data:
Reply from 10.40.123.1: bytes=32 time=54ms TTL=254
Reply from 10.40.123.1: bytes=32 time=55ms TTL=254
Reply from 10.40.123.1: bytes=32 time=56ms TTL=254
Reply from 10.40.123.1: bytes=32 time=54ms TTL=254
Ping statistics for 10.40.123.1:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 54ms, Maximum = 56ms, Average = 54ms
C:\Windows\System32>
C:\Windows\System32>
C:\Windows\System32>ping 10.40.123.2
Pinging 10.40.123.2 with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Ping statistics for 10.40.123.2:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
C:\Windows\System32>
C:\Windows\System32>
C:\Windows\System32>tracert 10.40.123.1
Tracing route to 10.40.123.1 over a maximum of 30 hops
1 53 ms 56 ms 54 ms 10.40.123.1
Trace complete.
C:\Windows\System32>
C:\Windows\System32>tracert 10.40.123.2
Tracing route to 10.40.123.2 over a maximum of 30 hops
1 56 ms 55 ms 55 ms 10.40.100.1
2 * * * Request timed out.
3 * * * Request timed out.
4 * * * Request timed out.
5 * * * Request timed out.
............cut...........
27 * * * Request timed out.
28 * * * Request timed out.
29 * * * Request timed out.
30 * * * Request timed out.
Trace complete.
C:\Windows\System32>
Let me know if there's anymore info needed.
Wondering if having the VPN provider change their Gateway to 10.40.100.10 so the GW is on the same device where the VPN boxes terminate... But, don't want to break something else by doing this.
Thanks in Advance,
Matt
