Slow downloads/stopped downloads

adrianhenderson · ‎04-17-2012

Not sure if this is the right forum, but I'll post here and take any redirections

I have a customer that has 2800 series router that used to be the hub for two remote branches and the NAT for the internet. They upgraded the service a few months back from DSL to fiber (5mbps). The remote VPN's are no longer in operation in the following config.

The users have been complaining that the internet has been unusable. Browisng seems fine but pretty much any webiste I go to to download a file it drops to a crawl nearly instantly. I've checked the local DNS and its using google 8.8.8.8 and 8.8.4.4. THE CATCH is that I can download from Microsoft downloads full out pretty much each time it test it. Download.com, apple, adobe all are essentially unusable. Is there anything in this config that catches anyones eye? My only though tis this line that may have been from the DSL, on the inside LAN interface "ip tcp adjust-mss 1400" How this doesn't affect microsoft I don't know.....

The rotuer is basically just doing internet NAT now. The rest fo the config is no longer applicable. FYI the ISP says there is no content filtering on their end and the only thing I haven't tried is goign direct in to the fiber transceiver as I am abotu 1.5hours away. Also, no cisco smartnet so a software update isn't in the works. We'd just change them to an ASA or something.

=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2012.04.17 13:51:22 =~=~=~=~=~=~=~=~=~=~=~=

!
version 12.4
no service pad
service timestamps debug uptime
service timestamps log uptime
service password-encryption
!
hostname firewall.LDN
!
boot-start-marker
boot-end-marker
!
logging buffered 8000 debugging
logging console errors
logging monitor errors
enable password 7

!
aaa new-model
!
!
aaa authentication login userauthen group radius
aaa authorization network groupauthor local
!
aaa session-id common
!
resource policy
!
no ip source-route
!
!
ip cef
!
!
no ip domain lookup
ip domain name xxxxxxx

ip name-server 8.8.4.4
ip inspect max-incomplete high 1100
ip inspect max-incomplete low 900
ip inspect one-minute high 1100
ip inspect one-minute low 900
ip inspect name Ethernet_0 tcp
ip inspect name Ethernet_0 udp
ip inspect name Ethernet_0 cuseeme
ip inspect name Ethernet_0 ftp
ip inspect name Ethernet_0 h323
ip inspect name Ethernet_0 rcmd
ip inspect name Ethernet_0 realaudio
ip inspect name Ethernet_0 smtp
ip inspect name Ethernet_0 streamworks
ip inspect name Ethernet_0 vdolive
ip inspect name Ethernet_0 sqlnet
ip inspect name Ethernet_0 tftp
ip inspect name Ethernet_0 http
ip inspect name Ethernet_0 https
ip inspect name Ethernet_1 smtp
ip inspect name Ethernet_1 tcp
ip inspect name Ethernet_1 udp
ip ips name IDS
!
!
voice-card 0
no dspfarm
!
!
!
!
!
--More-- !
!
!
!
!
!
!
!
crypto pki trustpoint TP-self-signed-4067238715
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-4067238715
revocation-check none
rsakeypair TP-self-signed-4067238715
!
!
crypto pki certificate chain TP-self-signed-4067238715
certificate self-signed 01
***deleted for reading this*****

quit
username root privilege 15 secret 5
!
!
!
crypto isakmp policy 5
encr 3des
authentication pre-share
group 2
lifetime 28800
--More-- !
crypto isakmp policy 10
encr 3des
hash md5
authentication pre-share
group 2
crypto isakmp key **deleted** address 216.59.235.160 no-xauth
crypto isakmp key **deleted** address 216.59.235.159 no-xauth
crypto isakmp identity hostname
crypto isakmp keepalive 40 5
crypto isakmp nat keepalive 20
crypto isakmp client configuration address-pool local vpnpool
!
crypto isakmp client configuration group vpngroup
key

dns 192.168.21.20
domain corp.morphycontainers.com
pool vpnpool
acl ACL-SPLIT-TUNNEL
crypto isakmp profile VPNCLIENT
   match identity group vpngroup
   client authentication list userauthen
   isakmp authorization list groupauthor
          client configuration address respond
!
!
crypto ipsec transform-set STRONG esp-3des esp-md5-hmac
crypto ipsec transform-set DYNAMIC esp-3des esp-sha-hmac
crypto ipsec transform-set strong esp-3des esp-sha-hmac
crypto ipsec transform-set dyntrans esp-3des esp-md5-hmac
crypto ipsec fragmentation after-encryption
!
crypto dynamic-map CLIENTS 5
set transform-set DYNAMIC
!
!
crypto map CLIENTS client configuration address initiate
crypto map CLIENTS client configuration address respond
!
crypto map VPN client authentication list userauthen
crypto map VPN isakmp authorization list groupauthor
crypto map VPN client configuration address respond
crypto map VPN 65535 ipsec-isakmp dynamic CLIENTS
!
!
!
--More--                           !
interface Tunnel159
ip address 192.168.252.1 255.255.255.252
tunnel source 209.183.149.70
tunnel destination 216.59.235.159
!
interface Tunnel160
ip address 192.168.252.5 255.255.255.252
tunnel source 209.183.149.70
tunnel destination 216.59.235.160
!
interface Loopback1
ip address 1.1.1.1 255.255.255.252
!
interface FastEthernet0/0
description london private ethernet
ip address 192.168.21.254 255.255.255.0
ip access-group ACL-INSIDE-INBOUND in
ip verify unicast reverse-path
ip nat inside
ip inspect Ethernet_0 in
ip virtual-reassembly
ip route-cache policy
          no ip route-cache cef
ip tcp adjust-mss 1400
ip policy route-map CRYNAT
no ip mroute-cache
duplex auto
speed auto
crypto map VPN
!
interface FastEthernet0/1
description DSL Circuit - 519.681.9369 - GCS VPN 409
ip address 209.183.149.70 255.255.255.252
ip access-group ACL-OUTSIDE-INBOUND in
ip nat outside
ip inspect Ethernet_1 in
ip virtual-reassembly
no ip route-cache cef
no ip route-cache
no ip mroute-cache
duplex full
speed 10
crypto map VPN
!
router eigrp 1
        redistribute static
network 192.168.21.0
network 192.168.252.0
no auto-summary
!
ip local pool vpnpool 192.168.253.1 192.168.253.254
ip route 0.0.0.0 0.0.0.0 209.183.149.69
!
!
ip http server
ip http authentication local
ip http secure-server
ip http timeout-policy idle 600 life 86400 requests 10000
ip nat pool NATPOOL 209.183.153.225 209.183.153.225 netmask 255.255.255.248
ip nat inside source route-map NONAT pool NATPOOL overload
ip nat inside source static 192.168.21.22 209.183.153.226
ip nat inside source static 192.168.21.6 209.183.153.227
ip nat inside source static 192.168.21.24 209.183.153.228
ip nat inside source static 192.168.21.28 209.183.153.229
!
ip access-list standard ACL-SSH-ADMIN
permit 209.183.146.80
permit 192.168.21.0 0.0.0.255
         permit 192.168.253.0 0.0.0.255
ip access-list standard INSIDE_IPS
permit 192.168.21.0 0.0.0.255
ip access-list standard XLAT
!
ip access-list extended ACL-CRY-BRANT
permit gre host 209.183.149.70 host 216.59.235.160
ip access-list extended ACL-CRY-CLIENTSPLIT
permit ip 192.168.21.0 0.0.0.254 192.168.253.0 0.0.0.254
permit ip 192.168.253.0 0.0.0.254 192.168.21.0 0.0.0.254
ip access-list extended ACL-CRY-WHOUSE
permit gre host 209.183.149.70 host 216.59.235.159
ip access-list extended ACL-CRYNAT
permit ip 192.168.21.0 0.0.0.255 192.168.253.0 0.0.0.255
ip access-list extended ACL-INSIDE-INBOUND
permit ip 192.168.21.0 0.0.0.255 192.168.0.0 0.0.255.255
permit tcp host 192.168.21.2 any eq smtp
permit tcp host 192.168.21.6 any eq smtp
permit tcp any any eq smtp
deny   tcp any any eq smtp log
deny   udp any eq netbios-ns any
deny   udp any any eq netbios-ns
deny   tcp any any eq 139
          deny   tcp any eq 139 any
deny   tcp any eq 445 any
permit ip any any
permit tcp any host 209.183.153.229 eq 3389
ip access-list extended ACL-NAT
deny   ip 192.168.0.0 0.0.255.255 192.158.253.0 0.0.0.255
permit ip 192.168.0.0 0.0.255.255 any
ip access-list extended ACL-NONAT
permit ip 192.168.0.0 0.0.255.255 192.168.253.0 0.0.0.255
ip access-list extended ACL-OUTSIDE-INBOUND
permit icmp any any echo
permit icmp any any echo-reply
deny   tcp host 209.183.153.226 eq www any
deny   tcp host 209.183.153.227 eq www any
permit esp any host 209.183.149.70
permit udp any host 209.183.149.70 eq isakmp
permit udp any host 209.183.149.70 eq non500-isakmp
permit gre host 216.59.235.160 host 209.183.149.70
permit gre host 216.59.235.159 host 209.183.149.70
permit esp any host 209.183.153.227
permit ahp any host 209.183.153.227
permit tcp any host 209.183.153.227 eq 1723
permit udp any host 209.183.153.227 eq isakmp
          permit gre any host 209.183.153.227
permit tcp any host 209.183.153.227 eq 1494
permit tcp any host 209.183.153.227 eq www
deny   ip any host 209.183.153.227
permit tcp any host 209.183.153.226 eq www
permit tcp any host 209.183.153.226 eq smtp
deny   udp any eq netbios-ns any
deny   udp any any eq netbios-ns
deny   tcp any any eq 139
deny   tcp any eq 139 any
deny   tcp any any eq 445
permit tcp host 216.59.235.160 host 209.183.149.70 eq 22
permit tcp host 216.59.235.159 host 209.183.149.70 eq 22
permit tcp 192.168.253.0 0.0.0.255 any eq telnet
permit tcp 192.168.253.0 0.0.0.255 any eq 22
permit tcp any host 209.183.153.228 eq www
permit tcp any host 209.183.153.229 eq 3389
deny   ip any any log
ip access-list extended ACL-SPLIT-TUNNEL
permit ip 192.168.21.0 0.0.0.255 192.168.253.0 0.0.0.255
permit ip 192.168.23.0 0.0.0.255 192.168.253.0 0.0.0.255
permit ip 192.168.24.0 0.0.0.255 192.168.253.0 0.0.0.255
!
         logging trap debugging
logging 192.168.21.7
!
route-map CRYNAT permit 10
match ip address ACL-CRYNAT
set ip next-hop 1.1.1.2
!
route-map NONAT permit 10
match ip address ACL-NAT
!
route-map NAT permit 10
match ip address ACL-NAT
set ip next-hop 1.1.1.2
!
!
!
radius-server host 192.168.21.7 auth-port 1645 acct-port 1646 key 7

radius-server timeout 60
radius-server key 7
!
control-plane
!
!
!
!
!
!
!
!
!
!
line con 0
line aux 0
line vty 0 4
access-class ACL-SSH-ADMIN in
exec-timeout 40 0
privilege level 15
transport input all
!
scheduler allocate 20000 1000
!
end

firewall.LDN# quti it

Peter Paluch · ‎04-17-2012

Hi Adrian,

This is probably going to be an issue requiring some more time to get solved.

A couple of suggestions:

The Fa0/0 and Fa0/1 are exempted from CEF, causing the traffic to be fast switched or process switched. Please enter the command ip route-cache cef on both Fa0/0 and Fa0/1.
The policy-map CRYNAT on your Fa0/0 interface seems to have been implemented for performing NAT-on-stick. However, is it still required? If not, please remove the ip policy route-map CRYNAT away from the Fa0/0 interface.
If the IPsec VPN is not being used anymore as you suggested, please remove the crypto map VPN from your Fa0/0 and Fa0/1 interfaces. At least, I do not believe that the crypto map VPN command should be placed on the internal Fa0/0 interface.
You may want trying removing the ip tcp adjust-mss command from the Fa0/0 interface but I have never seen this command creating similar issues - on the contrary, it helped solving them.
The access-list ACL-OUTSIDE-INBOUND is blocking ICMP messages that indicate a packet-too-big condition. This is a serious flaw that prevents the Path MTU Discovery process from working properly. Please add the following entry to this ACL, ideally to the top of the ACL:

permit icmp any any packet-too-big
Should all previous suggestions fail, try removing both the static ACLs and the references to the IP Inspect from both Fa0/0 and Fa0/1. My point is to verify whether it is the IP Inspect feature that has been commonly known to terribly slow down certain TCP transfers. Removing the references to IP Inspect and to static ACLs from your Fa0/0 and Fa0/1 should allow the traffic to pass without filtering and/or inspection.

Please keep us informed!

Best regards,

Peter

adrianhenderson · ‎04-17-2012

Hi Peter,

I have permited the packet too big, and removed the tcp adjust comnmands. I have also enabled the ip-route cache as you mentioned and no change. I tried removing the ip inspect Ethernet_0 in from that interface and I think i lost web access. My RDP session didn't terminate but the web pages i was testing stopped working so I stopped here. I am too far away to brick this config!

for #2 above I am not sure what this is to be truthful so I have left it alone for now as weill.

Peter Paluch · ‎04-17-2012

Hi Adrian,

Oh, are you working remotely? That complicates things.

Regarding the removal of the ip inspect commands, you have to note that you have to remove both the static ACLs (ip access-group) and the IP Inspect commands. Otherwise, if you remove only the IP Inspect, the static ACLs remain in place and because there is no inspection performed on the transit traffic, no additional holes are punched into them, so the router becomes even less traversable than before.

Once again: if you are planning to perform the experiment from Step 6, you first have to remove the ip access-group commands from the Fa0/0 and Fa0/1, and then remove the ip inspect commmands - in this order. Putting things back should be performed in the reverse order.

Best regards,

Peter

adrianhenderson · ‎04-18-2012

Is it possible to remove the inspect http anmd https to see if that helps without having to remove the whole lot?

adrianhenderson · ‎04-18-2012

Solved!

I did a bit more investigating and why this didn't affect microsoft's site is still a mystery but I noticed transfer's over RDP were fine so I looked up the http inspect and noted several people reporting a bug with it so I saved the config and then removed the inspect for http/s/smtp and its good to go! Saved the config and downloaded it for safe keeping.