Too much packet loss between HQ and remote site.

Sandeep Choudhary · ‎08-12-2012

Hello Everyone,

I am facing this issue from long time but still couldnt get the solution.

while transferring the CAD server Data from HQ TO REMOTE location is very slowand full of losses.

see the screenshot:

we are connected with 2 gre tunnels to the remote location.

here are the config from remote router :

interface Tunnel1

description *** Tunnel 1 ***

bandwidth 2000

ip address 10.13.75.2 255.255.255.252

ip mtu 1300

ip tcp adjust-mss 1260

tunnel source 82.99.163.2

tunnel destination 195.243.205.104

tunnel protection ipsec ................... !

!

interface Tunnel2

description *** Tunnel 2 ***

bandwidth 2000

ip address 10.13.175.2 255.255.255.252

ip mtu 1300

ip tcp adjust-mss 1260

qos pre-classify

tunnel source 82.99.163.2

tunnel destination 212.185.41.196

tunnel protection ipsec profile ........................

REgards

paolo bevilacqua · ‎08-13-2012

Beside the fact the the MTU is set worng, you can can have congested or fauly WAN circuits, hence the drops.

Also, please do not use screenshots for simple text.

Alessio Andreoli · ‎08-13-2012

HI,

as Paolo said you should adjust MTU & MSS values and by the way you should consider to run some test on the congestions and also on the tunnel interfaces usage. Why for example you did set the bandwidth statements with so different? I am not totally sure that your issue is congestion because your response time is fine .. Check your router resources usage including proc and memory and it would be an idea to check your broadcast traffic too.

Try also to do an extended ping on the router loopback (not using the tunnel)

ping -l (size packet)

is the one you need on microsoft to check fragmentation too.

Good Luck

Alessio

Sandeep Choudhary · ‎08-13-2012

Hi Alessio,

Here is the sh int tunnels OUTPUT:::MAY BE WE CAN FIND SOMETHING HERE.

HQ side:

HARCVPN1#sh int tunnel175
Tunnel175 is up, line protocol is up
Hardware is Tunnel
Description: *** xyz ***
Internet address is 10.13.75.1/30
MTU 1514 bytes, BW 2000 Kbit, DLY 500000 usec,
     reliability 255/255, txload 1/255, rxload 41/255
Encapsulation TUNNEL, loopback not set
Keepalive not set
Tunnel source 195.243.205.104, destination 82.99.163.2
Tunnel protocol/transport GRE/IP
    Key disabled, sequencing disabled
    Checksumming of packets disabled
Tunnel TTL 255
Fast tunneling enabled
Tunnel transmit bandwidth 8000 (kbps)
Tunnel receive bandwidth 8000 (kbps)
Tunnel protection via IPSec (profile .................s")
Last input 00:00:00, output never, output hang never
Last clearing of "show interface" counters 00:03:49
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/0 (size/max)
5 minute input rate 322000 bits/sec, 246 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
     71176 packets input, 11945701 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     138 packets output, 28222 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 output buffer failures, 0 output buffers swapped out

HARCVPN2#sh int tunnel275
Tunnel275 is up, line protocol is up
Hardware is Tunnel
Description: *** xyz ***
Internet address is 10.13.175.1/30
MTU 1514 bytes, BW 2000 Kbit, DLY 500000 usec,
     reliability 255/255, txload 156/255, rxload 8/255
Encapsulation TUNNEL, loopback not set
Keepalive not set
Tunnel source 212.185.41.196, destination 82.99.163.2
Tunnel protocol/transport GRE/IP
    Key disabled, sequencing disabled
    Checksumming of packets disabled
Tunnel TTL 255
Fast tunneling enabled
Tunnel transmit bandwidth 8000 (kbps)
Tunnel receive bandwidth 8000 (kbps)
Tunnel protection via IPSec (profile...............")
Last input 00:00:03, output never, output hang never
Last clearing of "show interface" counters 00:04:34
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/0 (size/max)
5 minute input rate 70000 bits/sec, 52 packets/sec
5 minute output rate 1225000 bits/sec, 251 packets/sec
     59 packets input, 4956 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     70199 packets output, 43568051 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 output buffer failures, 0 output buffers swapped out

Remote Site:

TARCVPN1#sh int tunnel1
Tunnel1 is up, line protocol is up
Hardware is Tunnel
Description: *** Tunnel 1 ***
Internet address is 10.13.75.2/30
MTU 17916 bytes, BW 2000 Kbit/sec, DLY 50000 usec,
     reliability 255/255, txload 54/255, rxload 1/255
Encapsulation TUNNEL, loopback not set
Keepalive not set
Tunnel source 82.99.163.2, destination 195.243.205.104
Tunnel protocol/transport GRE/IP
    Key disabled, sequencing disabled
    Checksumming of packets disabled
Tunnel TTL 255, Fast tunneling enabled
Tunnel transport MTU 1276 bytes
Tunnel transmit bandwidth 8000 (kbps)
Tunnel receive bandwidth 8000 (kbps)
Tunnel protection via IPSec (profile ..................s")
Last input 00:00:03, output never, output hang never
Last clearing of "show interface" counters 00:04:33
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/0 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 430000 bits/sec, 261 packets/sec
     124 packets input, 25265 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     84440 packets output, 18186518 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out

TARCVPN1#sh int tunnel2
Tunnel2 is up, line protocol is up
Hardware is Tunnel
Description: *** Tunnel 2 ***
Internet address is 10.13.175.2/30
MTU 17916 bytes, BW 2000 Kbit/sec, DLY 50000 usec,
     reliability 255/255, txload 12/255, rxload 117/255
Encapsulation TUNNEL, loopback not set
Keepalive not set
Tunnel source 82.99.163.2, destination 212.185.41.196
Tunnel protocol/transport GRE/IP
    Key disabled, sequencing disabled
    Checksumming of packets disabled
Tunnel TTL 255, Fast tunneling enabled
Tunnel transport MTU 1276 bytes
Tunnel transmit bandwidth 8000 (kbps)
Tunnel receive bandwidth 8000 (kbps)
Tunnel protection via IPSec (profile ...........................")
Last input 00:00:00, output never, output hang never
Last clearing of "show interface" counters 00:04:34
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/0 (size/max)
5 minute input rate 921000 bits/sec, 195 packets/sec
5 minute output rate 98000 bits/sec, 50 packets/sec
     55364 packets input, 32632835 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     59 packets output, 4956 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out

REgards

Joseph W. Doherty · ‎08-13-2012

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Does HQ have more bandwidth then remote? If so, do you shape?

You describe having two tunnels to remote. Do you use both to transfer traffic concurrently?

What are the physical available bandwidths?

Does the physical interface have other than VPN traffic?

Sandeep Choudhary · ‎08-13-2012

Hi Joseph,

1. HQ have bandwidth of 100Mb and remote have 2Mb.....No we are not doing and shape....

2. yes we use both tunnels to transfer traffic concurrently.

3. HQ-100Mb, Remote - 2Mb

4. All the data(FTP, CAD Server......all kind of traffice is handled by these 2 tunnels.) between HQ AND REMOTE SITE:

here is the config from the HQ router:

interface Tunnel175
description *** xyz ***
bandwidth 2000
ip address 10.13.75.1 255.255.255.252
ip mtu 1300
ip tcp adjust-mss 1260
tunnel source 195.243.205.104
tunnel destination 82.99.163.2

tunnel protection ipsec ................... !

interface Tunnel275
description *** xyz ***
bandwidth 2000
ip address 10.13.175.1 255.255.255.252
ip mtu 1300
ip tcp adjust-mss 1260
tunnel source 212.185.41.196
tunnel destination 82.99.163.2

tunnel protection ipsec ................... !

Remote site config:

interface Tunnel1

description *** Tunnel 1 ***

bandwidth 2000

ip address 10.13.75.2 255.255.255.252

ip mtu 1300

ip tcp adjust-mss 1260

tunnel source 82.99.163.2

tunnel destination 195.243.205.104

tunnel protection ipsec ................... !

!

interface Tunnel2

description *** Tunnel 2 ***

bandwidth 2000

ip address 10.13.175.2 255.255.255.252

ip mtu 1300

ip tcp adjust-mss 1260

qos pre-classify

tunnel source 82.99.163.2

tunnel destination 212.185.41.196

tunnel protection ipsec profile ........................

Regards

gogo1982bl · ‎08-14-2012

Hi,

Can you disalble one of the tunnels?

First disable a tunnel (tunnel 2 is enabled) and check whether it is OK.

Also attempt to disable tunnel 2 (tunnel 1 is enabled) and check whether it is OK.

Best Regards, Ognjen

Sandeep Choudhary · ‎08-14-2012

Hi Ognien,

Still the same packet drop.

Regards

Joseph W. Doherty · ‎08-14-2012

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

In that case I would recommend, at the HQ side, you shape for the remote's 2 Mbps and don't use both tunnels concurrently.

On the remote side, assuming the physical interface is 2 Mbps (E1?), enable qos pre-classify on both tunnels and enable FQ on physical interface.

I would also recommend, you consider increasing both your IP MTU and adjust-mss by at least 100 bytes and you include PMTUD on your tunnels.

Sandeep Choudhary · ‎08-14-2012

Hi Joseph,

Thanks for your quick reply.

1.I am not much aware about the shaping ????? can u please tell me little about this?

2.I have enabled qos pre-classify on tunnels but again here i don know about FQ??

3. I have increase IP MTU and TCP adjust-ss by 100 Byte.

Regards

Joseph W. Doherty · ‎08-14-2012

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

On HQ tunnels, try shape average 2000000.

On remote physical try fair-queue. (Again, this assumes physical interface is 2 Mbps.)

Sandeep Choudhary · ‎08-14-2012

Shape average command is not working directly under tunnel interface.

or i have to create a policy???

REgards

Joseph W. Doherty · ‎08-14-2012

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

What's the HQ platform and IOS?