reload/reboot many switches in Prime2.1

[Carstenlymann]

Hi

Have a bunch of switchs (2960 and 3560) that i would like to reload at the same time. Have tried with a restart job in prime. But it only reload 5 at a time. But i have location with 20+ switches. And if i reload the prime switch in that network/topology it could be just as nice to reload all the other switches as well, since they dont do any good as long as the prime switch is down. But havent found out how to do this yet..

Any ideas ?

Thanks in advance :)

/ Carsten

[Vinod Arya · Accepted solution]

You have following options :

1. Use a new interactive template to reboot the devices.

An interactive command contains the input that must be entered following the execution of a command. To enter an interactive command in the CLI Content area, use the following syntax:
CLI Command<IQ>interactive question 1<R>command response 1 <IQ>interactive question 2<R>command response 2
 
where <IQ> and <R> tag are case-sensitive and must be entered as uppercase.
For example:

#INTERACTIVE
reload <IQ>Proceed with reload? [confirm]<R> yes
#ENDS_INTERACTIVE

2. Use SNMP to reboot the devices.

If the devices are configured for the command - snmp-server system-shutdown than you can reboot devices together by sending a simple SNMP-SET command to them. For more details check the document here.

-Thanks

Vinod

[Carstenlymann]

snmp system shutdown command is not on the switches for now. But should be pretty easy to add to them all since snmp is active on all of them. And prime has RW to them.

For option 1. If i make that one and choose 20 switches. Will it then reboot them all at the same time, or only take 5 at a time like it does with everything else ?? Looks like 5 is maximum it can connect and "talk" to at the same time. Thats my experience at least. If it can be changed some place please let me know :)

/ carsten

[Vinod Arya]

I dont think it should be restricted to five. May be the template you made, you might have made it for a specific device type which is why it may not be implementing to other device types.

It can for sure run on more than 5 devices, try to create a new one for all device types and see if it works.

-thanks

Vinod

[Carstenlymann]

Thanks Vinod

Will try it out. Not sure i can here during x-mas but then it will be early january. 

I have tried selectiong 20 switches to be reloaded with the template i made. It only reloaded 5 at a time. And didnt move on to the next 5 switches before the first 5 was back up. And it wasnt because swithces above in topology was the one that was reloaded first. Have also tried it with a 1minute timer so i was sure command was getting out to all swithces before the first was reloaded. But again only 5 at a time. 

/ Carsten

[Carstenlymann]

Okay have tried this now. But with errors. So im waiting to after newyear to do more:

exit
wr
#INTERACTIVE
reload in 1 <IQ>Proceed with reload? [confirm]<R> yes
#ENDS_INTERACTIVE

Have this in feature design..  The exit is because it starts in config mode. Then write so it dont ask back that there has been changes and if it should be saved.

But get this error within 10sec from prime:

Error in running XDE Procedure, Error Message::Failed to invoke java string class method :null

We plan to upgrade to 2.2 early january so might be i wait till that is done before i do more.

/ Carsten

[Carstenlymann]

Did one last try :)

Found this one:

#if ( $configbeforereload eq "true" )
#MODE_ENABLE
#INTERACTIVE
reload in 2<IQ>yes/no<R>no<IQ>Proceed<R>yes <timeout>${waittimeout}</timeout>
#ENDS_INTERACTIVE
#MODE_END_ENABLE
#else
#MODE_ENABLE
#INTERACTIVE
reload in 2<IQ>yes/no<R>yes<IQ>Proceed<R>yes <timeout>${waittimeout}</timeout>
#ENDS_INTERACTIVE
#MODE_END_ENABLE
#end

I selected 6 switches. And deploy it. After 2min (as they should) the first 5 switches reloaded. After 3,5min the 6th switch reloaded. So it looks like it only gives the command to 5 switch at a time. And then 1,5min later to the next. I cant find anything in settings.

Is there anybody that can try out or tell for sure that it can send the command to many switches at the same time. I think the most switches i need to reboot at the same time is aprox 30.

I will put the first answer as correct since it has the right commands. But im still fighting with it to do many switches at the same time.

Thanks :)

[Vinod Arya · Accepted solution]

This is good. PI when working with multiple devices can work with 5 devices in parallel, which is operational design as per engineering. But you can deploy a template on more than 5 devices, so there is no restrictions on how many devices a job can be deployed to. It can be on the total number of devices managed by your PI.

This is because, when a job is executed, the resources and processes used by PI are intense on CPU, so any operation is on max 5 devices at a time and moves to next 5 and so on.

-Thanks

Vinod

[Carstenlymann]

Thanks Vinod.

Was what i was thinking then :)

Its just that i have up to 30 switches on 1 location. Hierachical set up. 1 main switch. 2 switch uplink to that. and 2 to the last 2. and so on.. If i reload with prime and select all 30 switches i might end up it reload the switches farest out in the hierachical setup. And after that close to the main switch and so on.. If switch is 5min to reload i risk the users on the switches farest out will be offline for 20min... If it reloaded ALL switches at the same time they would only be offline for 5min and then that was it..  I know they get reloaded and i can pick all switches, but the time users is offline can be a problem for me. :)

But thanks for the help :)

/ Carsten

[Vinod Arya]

I was checking more details on this, and seems by default PI has 5 thread pools which can be increased. Increasing the thread pool to push any config to more number of devices parallely can be done via :

Administration > System Settings > Configuration

Here you can increase the Deploy CLI Thread Pool Count.

-Thanks

Vinod

[Carstenlymann]

Hi

I already tried that. Tried setting it to 10, 50 and 500.. None of it helped. I even logged out of prime and in Again to make sure it was saved. And it was. But still the 6th switch reloaded 1,5min after the first 5. So that didnt seem to help either :(

/ Carsten