Wan Link up/ down and interface resets

[bapatsubodh]

Hi

We are having CA unicenter spectrum NMS used for wan links monitering. We found that some links were flapping quite frequently over a perion of time. So we monitered the link for 12 hours but we did not see any interface resets in 12 hours. There were CRCs & input error's. So is it so that the link is down and up interface reset count remains the same. How to identify the exact problem. One soluation was to keep the continuous ping on the other end ip address of wan link. This is ok for 10/15 wan links but as we are monitering more that 650 links keeping continuous ping is not practical solution.

Thanks in advance

Subodh

[Pavel Bykov]

I'm not sure what exactly are you trying to achieve. Are you trying to verify connectivity on the higher layer? Like IP?

Here are quick pointers:

1. If there is BGP routing protocol it is possible to setup route-dampening with parameters as to not dumpen line, but keep the statistics. Then you would have a short lived history of flapping.

2. If failure is trackable on subinterface level, you there are syslog and trap possibilities.

3. You could track routing process neighbor relation.

[bapatsubodh]

Hi

It seems I could not put my difficulty in proper format. Really sorry for that !

We are trying to get what is the exact status of links. ( NMS CA Unicenter Spectrun is generating link status reports. ) Is CA unicenter Spectrum is generating faulty link status reports or the link is really flapping as seen in the report.

That is why we checked interface resets to see if link was flapping so frequently. But "Can link go up and down without reseting the interface on either side" ? This point we r still not clear about. There are CRS and input errors on the interfaces.

In a nutshell " If link goes up or goes down is it necessasry that interface reset count will increment or it is not necessary ".

Secondly we are having OSPF as routing protocol as you have suggested we can log the neighbor relationship and it will definitely indicate as on when link flapped and when it was stable.

Thanks a Lot.

Any information on link up/down <---> interface reset count.

Thanx in advance

Subodh

[Pavel Bykov]

>>"Can link go up and down without reseting the interface on either side"

Yes. Definitely.

From Cisco:

interface resets

Number of times an interface has been completely reset. This can happen if packets queued for transmission were not sent within several seconds. On a serial line, this can be caused by a malfunctioning modem that is not supplying the transmit clock signal, or by a cable problem. If the system notices that the carrier detect line of a serial interface is up, but the line protocol is down, it periodically resets the interface in an effort to restart it. Interface resets can also occur when an interface is looped back or shut down.

In practice, that means when you unplug the cable and plug it back in, the reset count stays the same. It refers to the operational reset, that is invoked by software or user.

When you issue "Clear interface XX", it's reset counter will go up. Or when you "SHUT" and then "NO SHUT" the interface, it's counter will go up. But when you unplug the cable and plug it back in, there is no RESET, and the counter stays the same.

What you are trying to monitor is actually availability of the link on IP-level. It is not very straightforward.

1. Unless connectivity is BACK TO BACK (one cable) there could be malfunction in the middle somewhere, and interfaces on both sides could actully stay UP and think that everything works ok. It really depends heavily on how your cabling is done - if it's L1 service (like Dark Fiber), or L2 service (like PPP), or something else (like MPLS).

When it's back-to-back on L1, you can track interface status - they are messages severity 5 (informational) for Layer2 and 3 (error) for link status. Those messages are recorded in log of the device, if logging for the correct severity level is enabled.

2. When you don't have the possibility to track interface status (no real direct connectivity), there is no choice but somehow track upper layers. Neighbor relationships is one way. There is a protocol called BFD that is available on high level platforms that was created for just this scenario, but it's not very available.

3. Remember, monitoring tools use SNMP (mostly). If packets are lost (e.g. due to high load) it will think the device is unavailable, but that does not have to be true. Not that this is your case, but just a reminder.

Hope this helps.

[tomredmond]

We have had CRS and input errors in the past on a link, increasing crc count over a period, this led to some data loss but there were no interface reset errors reported on either side. The problem was traced to clocking errors on the link. Our NMS reported flapping link but the interfaces did not have any resets.

[Pavel Bykov]

There are actually tools available in IOS to monitor this. One of them is IP SLA, or you can use "track" feature, that can track arbitrary objects, including reachability.

http://www.cisco.com/en/US/customer/products/hw/switches/ps5023/products_configuration_guide_chapter09186a00808b01c2.html