ISP latency

[Jonathancert_2]

As a network administrator, what is a good tool to measure ISP performance?  Users have been complaining about the latency at the browser level and from normal FTP services.  Sometimes Internet sites will time out while loading.  Some days things appear to be working fine.  Not noticing any congestion on my external WAN interfaces (see below)  but there is definitely a problem.  Before I contact our ISP, I want to conduct my own trend analysis.

WAN-RedRtr#sho int g3/1/0

GigabitEthernet3/1/0 is up, line protocol is up

  Hardware is GigEther SPA, address is c47d.4f6b.7240 (bia c47d.4f6b.7240)

  Description: Connection to DSW1 GI8/1

  Internet address is 172.147.162.122/31

  MTU 9216 bytes, BW 1000000 Kbit, DLY 10 usec,

     reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation ARPA, loopback not set

  Keepalive not supported

  Full Duplex, 1000Mbps, link type is auto, media type is SX

  output flow-control is unsupported, input flow-control is unsupported

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:02, output 00:00:07, output hang never

  Last clearing of "show interface" counters 8w0d

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 1064000 bits/sec, 639 packets/sec

  5 minute output rate 3166000 bits/sec, 837 packets/sec

     2224704181 packets input, 437429288816 bytes, 0 no buffer

     Received 0 broadcasts (0 IP multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 717583 multicast, 0 pause input

     2995898035 packets output, 1415667770032 bytes, 0 underruns

     0 output errors, 0 collisions, 0 interface resets

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 pause output

     0 output buffer failures, 0 output buffers swapped out

WAN-RedRtr#sho int tunnel 10001

Tunnel10001 is up, line protocol is up

  Hardware is Tunnel

  Description: Interface to NE-REG-R7606-1 Tu#1

Internet address is 172.147.166.17/31

  MTU 17868 bytes, BW 622000 Kbit, DLY 50000 usec,

     reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation TUNNEL, loopback not set

  Keepalive not set

  Tunnel source 10.147.166.17, destination 10.147.166.16

  Tunnel protocol/transport GRE/IP

    Key disabled, sequencing disabled

    Checksumming of packets disabled

  Tunnel TTL 255, Fast tunneling enabled

  Tunnel transport MTU 9192 bytes

  Tunnel transmit bandwidth 8000 (kbps)

  Tunnel receive bandwidth 8000 (kbps)

  Last input 00:00:00, output 00:00:00, output hang never

  Last clearing of "show interface" counters 8w0d

  Input queue: 0/75/202/202 (size/max/drops/flushes); Total output drops: 5

  Queueing strategy: fifo

  Output queue: 0/0 (size/max)

  5 minute input rate 3282000 bits/sec, 733 packets/sec

  5 minute output rate 884000 bits/sec, 635 packets/sec

  L2 Switched: ucast: 0 pkt, 0 bytes - mcast: 0 pkt, 0 bytes

  L3 in Switched: ucast: 0 pkt, 0 bytes - mcast: 0 pkt, 0 bytes mcast

  L3 out Switched: ucast: 0 pkt, 0 bytes mcast: 25527318 pkt, 2627237764 bytes

     3781236180 packets input, 2029774550585 bytes, 0 no buffer

     Received 0 broadcasts (0 IP multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort

     2177744442 packets output, 422808095135 bytes, 0 underruns

     0 output errors, 0 collisions, 0 interface resets

     0 output buffer failures, 0 output buffers swapped out

Jonathan,

[John Peek]

I have the same issue with my ISP. They are having a really hard time understanding what I am asking them.

We used to use 2 T1s from sprint for our internet services. Everything worked fine. We then switched to TWTelecom for a 25Mbit fiber connection and I am having the exact same issues you are describing. We have to introduce a delay of up to 30 seconds on our FTP sessions for them to actually connect. We also noticed a higher delay in web traffic. Although, when you ping out, ping times are low and don't vary much.

I am working with my provider, but we can't really take down the internet connection for testing during business hours.

[Jonathancert_2]

So, this most likely is an ISP problem but of course the non IT management that have fast internet at their house can't believe it could be the ISP fault.  Routing table and latency is looking good on the LAN.  Even with 6 hops, on a 21ms delay to reach the ISP.  Will keep looking for suggestions to back my theory.  Thanks.

[John Peek]

21ms seems a little high to reach your isp. If you have another site with a slower connection, you can show management the difference between the two sites.

Here something else that you can do. http://www.cisco.com/en/US/docs/ios/ipsla/configuration/guide/sla_ftp.pdf

IP SLA's are your friend. I use them quite regularly. For instance, I have a sla set up to track jitter between my sites. I will be able to prove to my carrier that the jitter is higher than it should be.

I also use SLA's to detect when a link is starting to have issues by constantly pinging IP addresses within my network. If so many fail it will automatically failover to my backup link and then fail back when the line clears.

Hope this helps.

[Jonathancert_2]

I have not utilized SLA features before.  Completed the configuration and trying to understand how the ouput can be baselined to aid in my troubleshooting.

[John Peek]

You can download a free IP SLA tracker from Solarwinds.

http://www.solarwinds.com/products/freetools/ip_sla_monitor/

There are probably other free sla monitors out there, but I have found that solarwinds usually produces a superior product.

What you can do is perform two installs on seperate machines. One at a remote branch and the other at where you are having problems.

[gerald.suiza]

what i would do is trace to a public IP like 4.2.2.2 for example and see where the latency shoots up. i would also try and ping my public IP or the points where you see a surge in latency using a public gateways(you can source from various points all over the world). you can use the link below. this is to see if your ISP is having problems on their uplinks and peering points.

http://lg.level3.net/ping/ping.cgi?site=gen1&target=172.147.162.122&size=64&count=10

HTH

[Jonathancert_2]

Thanks for the site.  Give me some direction to go.  I'm getting 60ms replies from www.yahoo.com