Internet connection Intermittent tunnel IPIP

[jose cortes]

Hi everybody,

I have this issue with an internet connection. The connection to the Far-end is made by a tunnel IPIP I have a monitoring system to check the connectivity from the Outside en it keeps reporting lost of connection after about one hour of service. The connection remains down for about 7min and then it goes up again and so on.

The issue here is when I check the Router log I don't see any information about the interfaces (tunnel and Ethernet) going down and up again. I have a backdoor access to the router, and when the connection is down I use it to check the Interfaces' status and the Tunnel interface and Ethernet (physical) interface are up/up but there is no ping reply from the Tunnel far-end.

I also checked the interfaces' counters looking for any CRC, overrun, collision, but nothing is reported the interfaces are clean.

I'm attaching the router configuration, this configuration was working before I take care of this customer and I nothing has been change but from about a month ago the problem came up.

Please is somebody has an Advice or a debug instruction that could help me to figure out what is happening I will be more than glad.

A BTW, the ISP check the whole circuit and say the only device unreachable when the problem happens is the local router.

Thanks and regards,

Jose

[nkarpysh]

Hi Jose,

I would check on CPU spikes seen on this router. I kinow it running several VPNs but still we need to understand if those correlate to the time of an outage:

Last 72 hours graph

                        1                                                1

    5896469955574899999509578785746386755865579789977583759687647588997790

    2399349685402297899209334286490938799018443219654666431408843688914540

100   *   **      ***** **                       **                 *    * 

90   *   **      ***** **    *               *  **   *   *       ****  ***

80  **   **     ****** **  * *     * *  *    * ****  *   * **    **** ****

70  ***  **   * ****** ** **** *   ***  *   ******** * * * *** * *********

60  *** ***** * ****** ** ****** * ******** ********** * ***** ************

50 **** ******* ****************** ******************* ******* ************

40 ************************************************************************

30 *********************#*********************#****************************

20 ********************####*******************##***************************

10 *********#****#*****####***********#***#***##********##****#******#**#**

   0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.

             0    5    0    5    0    5    0    5    0    5    0    5    0

Then you can try setup EEM script to trigger some show commands whenever spike is happening to see who is culprit.

In regards to the ping which is failing to the far end - I would also try to ping the default GW on your router with different sizes and do traceroute if your provider allowing ISMP on their hosts - to see how it goes.

Nik,

[jose cortes]

Hi Nikolay,

I've never used EEM scripst, I've been reading a little about it but I don't know how to trigger the show commands with the spike counter or threshold. And also, what show commands should I set up and what do I do with them?

At the other hand, the ISP allows ICMP to its router but once the Internet is down I don't have replies with any packet size and the traceroute does not work either.

Could you please help me with the EEM script at least the algorithm to do this:

When the Spikes in CPU utilization reach an XXX threshold then

Execute show "Something" and send the results to "somewhere" to be checked

thanks in advance.

Jose

[nkarpysh]

Hey,

You can try this one:

event manager applet capture_cpu_util

event snmp oid 1.3.6.1.4.1.9.9.109.1.1.1.1.3.1 get-type exact entry-op gt entry-val 85 exit-op lt exit-val 50 poll-interval 0.5

action 1.1 syslog msg "------HIGH CPU DETECTED----, CPU: $_snmp_oid_val%"

action 2.0 cli command "enable"

action 3.0 cli command "show clock | append flash:cpuinfo.txt"

action 4.0 cli command "show proc cpu sort | append flash:cpuinfo.txt"

action 5.0 cli command "show interfaces | append flash:cpuinfo.txt"

action 6.0 cli command "show ptoc cpu hist | append flash:cpuinfo.txt"

This script will monitor for that SNMP OID each laf second and should start when CPU will reach 85%. It will be auto stopped once commands are collected but this part "exit-op lt exit-val 50" will auto restart its monitoring funcrtions once CPU got down below 50% - you can delete it for first time to get just a single capture. Later you can do it continiously.

It will run the commands in the body and store those in flash:cpuinfo.txt

By those commands we first of all will see what is amking CPU to spike and if your internet connection is affected during the spike or these problem are independant.

Nik

[jose cortes]

Hi Nik,

I tried to configure what you send but I could not because the "event manager applet capture_cpu_util" command was not allowed in the router configuration.

I tried to ingress the command in exec mode and in the configure terminal mode and it did not allow me.

Any suggestions?

Jose

[nkarpysh]

I just checked that and it seems that 831 does not support EEM regardless of IOS. Sorry for not checking this before.

So the only was is to monitor the spikes with SNMP monitoring tool and try to repsond to those quickly and get the show commands. Other thing is first of all correlate by SNMP tool if CPU spike accompany tunnel problem. If not smth else is happening but this needs to be rulled out.

Nik