07-08-2011 11:05 AM - edited 03-04-2019 12:55 PM
I have an interesting one here.
I Have a Hub and spoke VPN IPSec VTI Tunnel environment - HQ and Branch in this instance. Beyond the HQ router there is a linux server dishing up SSH connections on port 22. From any server/client at HQ to port 22 on the Linux box connects fine and the connection does not drop.
Over the VPN, clients beyond the Branch router however, can see, ping and indeed connect to port 22 on the linux server at the HQ side - but after a while (and this is not a timeout issue) the connection drops. If I do a netstat on the Linux box I can see that the TCP connection still exists but data is building up in the send-q meaning that the TCP connection isn't receiving the data.
at the HQ I have a 2911 Sec and at the branch I have a 851W advanced IP. The branch is on a clients network so I don't have much control past our cisco box at the branch. I had to create a VLAN1 interface on the 851 and connect one of the switchports to the client networks' switch, thats how its hooked up.
Whats interesting is, if I login into the Branch 851 router and initialize a SSH connection back the linux box from there on port 22, the connection remains up and is totally 100% fine... It's when clients beyond our Cisco branch box connect that there is a problem.
So from the debugging I have done, I can SSH back from the HQ router to the linux box and everything is fine - jumping over the VPN link, I can SSH back from the Branch 851 to the Linux box and everything is fine, but if I SSH from clients beyond the 851 branch router the connection is established but drops intermittently. What methods of debugging the connection are available to me? It's not getting NAT'd at the branch router, and at all times, the client can ping the linux server (even when their SSH drops).
My mind is thinking I need to config the VLAN1 or the Fast Ethernet interface a little more... As Thats where i believe the problem to be. I have dropped all firewall rules (permit ip any any) on the firewall at the HQ. I hope people can hand me a bit of advice, I am really lost with this one!
Any Ideas on debugging?
Config from Branch router:
version 12.4
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname xxxxx
!
boot-start-marker
boot-end-marker
!
logging buffered 10000
enable secret 5 xxxxxxx
!
aaa new-model
!
!
aaa authentication login default local
!
!
aaa session-id common
clock timezone GMT 0
clock summer-time BST recurring last Sun Mar 2:00 last Sun Oct 2:00
clock save interval 8
!
ip cef
ip inspect log drop-pkt
ip inspect name CCP_LOW cuseeme
ip inspect name CCP_LOW dns
ip inspect name CCP_LOW ftp
ip inspect name CCP_LOW h323
ip inspect name CCP_LOW sip
ip inspect name CCP_LOW https
ip inspect name CCP_LOW icmp
ip inspect name CCP_LOW imap
ip inspect name CCP_LOW pop3
ip inspect name CCP_LOW rcmd
ip inspect name CCP_LOW realaudio
ip inspect name CCP_LOW rtsp
ip inspect name CCP_LOW esmtp
ip inspect name CCP_LOW sqlnet
ip inspect name CCP_LOW streamworks
ip inspect name CCP_LOW tftp
ip inspect name CCP_LOW tcp
ip inspect name CCP_LOW udp
ip inspect name CCP_LOW vdolive
ip inspect name CCP_LOW ssh
ip domain lookup source-interface Loopback1
ip domain name xxxxx
ip name-server 208.67.222.222
ip name-server 208.67.220.220
!
!
!
no spanning-tree vlan 1
username cepacadmin privilege 15 password 7 xxxxx
!
!
crypto isakmp policy 10
encr xxx
hash mxxxx
authentication xxxx
group 2
crypto isakmp key xxxxx address xxxxxx
crypto isakmp key xxxxx address xxxxx
crypto isakmp keepalive 120 5
!
!
crypto ipsec transform-set xxxx
crypto ipsec transform-set xxxx
!
crypto ipsec profile IPSecPol_Prim
set transform-set TFS_CepacTransAES-SHA TFS_CepacTrans3DES-MD5
!
interface Loopback1
description Internal Loopback
ip address 192.168.254.251 255.255.255.255
!
interface Tunnel10
description IPSec VPN Tunnel - Primary ADSL > Primary HQ ISP
ip unnumbered Loopback1
tunnel source FastEthernet4
tunnel destination xxxxx
tunnel mode ipsec ipv4
tunnel protection ipsec profile IPSecPol_Prim
!
interface Tunnel20
description IPSec VPN Tunnel - Primary ADSL > Secondary HQ ISP
ip unnumbered Loopback1
tunnel source FastEthernet4
tunnel destination xxxxx
tunnel mode ipsec ipv4
tunnel protection ipsec profile IPSecPol_Prim
!
interface FastEthernet0
description To Client Switch
!
interface FastEthernet1
shutdown
!
interface FastEthernet2
shutdown
!
interface FastEthernet3
shutdown
!
interface FastEthernet4
description TO ADSL ROUTER
ip address xxxxxxxx
ip access-group ACL_FW-Inbound in
ip verify unicast reverse-path
ip inspect CCP_LOW out
ip nat outside
ip virtual-reassembly
duplex auto
speed auto
no cdp enable
!
interface Vlan1
description Client Network
ip nat inside
ip address 192.168.1.200 255.255.255.0
!
router rip
version 2
passive-interface FastEthernet0
passive-interface FastEthernet1
passive-interface FastEthernet2
passive-interface FastEthernet3
passive-interface FastEthernet4
passive-interface Loopback1
offset-list 5 in 10 Tunnel20
offset-list 5 out 10 Tunnel20
network 192.168.1.200
no auto-summary
!
no ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 xxxxxx
07-08-2011 01:54 PM
Just for testing I would suggest to also drop the ip inspect rules on the branch:
interface FastEthernet4
no ip inspect CCP_LOW out
This is a possible cause as this set of rules is sitting between the router and your clients.
Try it and see what happens.
regards,
Leo
07-08-2011 02:17 PM
I am building up a list of things to try for Monday mornign (visiting site straight away).
I have tried this already with no effect if I remember correctly - what gets me is that the SSH from the branch router to the linux box works 100% whilst not from the clients beyond the router.
Would the ip inspect rule on the outgoing interface affect VPN traffic? As VPN traffic traverses through the tunnel interfaces and not fa4
07-10-2011 02:34 PM
Is there anything I can specify on the vlan interface to ensure the flow of tcp packets into the vlan better?
I've tried switching to the bridge group configuration mode but having the same error.
I am physically visiting the device tomorrow to investigate if it's a hardware issue with the switchport but would severely appreciate anybodies experience into debugging this error a little further.
Sent from Cisco Technical Support iPhone App
07-20-2011 08:04 AM
If it is any help to anybody I have solved this issue.
The connection at the other end of the site to site was an ADSL one, connected to via Fast ethernet. The MTU maximum needed to be adjusted to counter the 1500 maximum that passes over ATM. Once I had set the Ip adjust-mss to 1200 (for safety) everything was ok and the connections didnt drop.
07-21-2011 01:21 AM
Tim
I am glad that you resolved this issue. Thanks for posting back with the solution.
HTH
Rick
Sent from Cisco Technical Support iPhone App
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide