Weired behaviour of switches

[khalidsaada]

i have and environment and mpls consists of 2 networks and 8 switches connected to each others,

the last three days we are facing a network cut off on all routers and suddenly all routers restarted at morning every day !

wondering if anyone can help here

best regards

khalid saada

[robertja]

What does the last reload reason show as in a show ver?

[khalidsaada]

thanks for your reply

the last reload reason is that the switches suddenly rebooted , no logs saved even after the reboot now i configured debugging mode on the switches and to copy on a server so i can see what would be the logs and cause of failiur when it will happen tomorrow

any more suggestions ?

[rsimoni]

are all the restarting routers in the same physical location?

are they connected to the same grid?

what is the reason of the reload detected in show version?

i.e:

lan-me3750-1 uptime is 2 weeks, 6 days, 59 minutes

System returned to ROM by power-on    <<<<<<<

Do they always restart at the same time of the day?

Do they restart one by one or all together?

Have you checked if they were complaining for something just before reload?

Which switches are they? (platform and software)

If you can predict that they wil restart tomorrow morning as well I suggest you connect a PC on the console port (remove the auto logout feature if it is enabled) so you can save all the messages the switches are spitting (if any) before reload.

Having said that it seems that you are having power issues occurring in the mornings at the same time. Never heard of bugs or other issues occurring at the same time, every day on multiple devices. It is worth checking your power grid and involve the building administraotrs and ask them if they are doing something at that given time.

Riccardo

[khalidsaada]

thanks for your reply .

the switches are on 2 seperated locations connected to each others by mpls fiber line

the reason of the reload was that the switch rebooted ! but we didnt have any power cut and these switches are in a data center that provides many companies so there is no power cut for sure

im afraid this is because of a network storm could it be ?

and yes they almost restart at the same time of the day almost ! first day was at 8:22 , second day was at 9:00 and today was at 8:15

i didnt check that if they were complaining because i dont check switches that oftenly

and  when the switches rebooted all loogs gone

some of the switches are cisco and some are dell

and again im so sure that the npower is not the reason as im setting at one of the sites and there was no power cut and servers still running while they are ocnnecting to the same power plug !

any suggestions here ?

[rsimoni]

hold on.. I am losing you... what did reboot the switches or the routers?

the reason of the reload was that the switch rebooted

Not sure we are on the same page regarding what a reload is... what do you mean by that? If a switch reboot the router connected to it is not supposed to reload. For you a reload and a reboot are 2 different concepts? Can you elaborate on this?

Also, debugs might not be a good idea as by the time the routers/switches try to export info they could be reloaded already.

As I wrote before

"If you can predict that they wil restart tomorrow morning as well I suggest you connect a PC on the console port (remove the auto logout feature if it is enabled) so you can save all the messages the switches are spitting (if any) before reload."

What about the show version? Can you post them from all the devices (or some of them) which reloaded?

Riccardo

[khalidsaada]

i willl do this regarding connecting a pc for logs thank you

but what i meant when i checked the logs i didnt find any reason why the switch reloaded

well regarding the show version does affect if the switches were different versions ??

[taimoor.jan]

Dear Search teh config for:

"kron occurrence" they can really kick ass the cisco device

!
kron occurrence archive at 2:00 2 recurring
  policy-list archive1
!
kron policy-list archive1
  cli archive config
!