SCP copy of new IOS to ISR 4431 fails

[jowood1412]

I'm trying to upgrade a new ISR 4431 from 3.16.6 to 16.9.2. When I copy the file from my SolarWinds SCP server, I get an error that the file failed to open. Here are the troubleshooting steps I've taken:

-Try on another SCP server. I tried to copy from an SCP server running on my laptop that is connected to the router through a console port and received the same indications. The file fails to open. 

-Try on another router. I have another new router that needs the same upgrade. I tried the transfer on both the remote and local SCP server and got the same failure. 

-Try another IOS. I verified the hash on the 16.9.2 copy and also tried the same transfer with 3.16.8 and 16.3.7 and received the same failure. 

This is the log output:

copy scp://xxx:xxx@172.16.63.42/isr4400-universalk9.16.09.02.SPA.bin flash:
Destination filename [isr4400-universalk9.16.09.02.SPA.bin]?
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
%Error reading scp://*:*@172.16.63.42/isr4400-universalk9.16.09.02.SPA.bin (Transfer aborted)

I've also attached a copy of the SolarWinds log, which shows that the authentication and transfer both appear to have been successful. 

Any ideas?

[fbabashahi]

Hi , i think something interrupting , may it is your firewall

[balaji.bandi]

Hope i  believe you copying from 4431 router prompt #

4431#copy scp://xxx:xxx@172.16.63.42/isr4400-universalk9.16.09.02.SPA flash :  <-- this is correct syntax 

may be worth testing is the connection ok between router and scp server.

test uploading your running config to scp and download for testing

4431#copy run scp://xxx:xxx@172.16.63.42/   <-- see is this works then try other way around as below

4431#copy scp://xxx:xxx@172.16.63.42/running-config flash:/test-cfg   <-- make sure you use test config, do not overwrite with running config.

[jowood1412]

I successfully transferred the running config from the router to the scp server and back to the router. Out of curiosity, I also changed the name of the IOS to isr4400.txt and tried the transfer again to see if there was a firewall issue and got the same failure. 

[balaji.bandi]

So this is rules our SCP Server is ok, and device able to reach SCP Server.

Do you have FW in between these device ? what FW is this ? what is the logs of the FW when this failed.

[jowood1412]

All of our firewall/ips/ids devices connect to Splunk, which doesn't show any denied traffic. 

[johnd2310 · Accepted solution]

Hi,

You can try scp to the router to eliminate the scp server.

• enable scp on the router: "ip scp server enable"
• download scp client: you can get one that comes with putty
• copy ios to router: pscp -v -scp  isr4400-universalk9.16.09.02.SPA.bin  cisco@x.x.x.x:/isr4400-universalk9.16.09.02.SPA.bin

Thanks

John

[jowood1412]

I set up the transfer with pscp and am getting the same error. Running debug ip scp, I can see the authentication complete and the file transfer begin. PSCP then shows that the software terminated the connection and the debug ip scp output shows an error "send not able to open file".

I also tried creating and then transmitting a small text file to see if the file size or type was the issue and received the same error. 

I also, one again, checked Splunk to see if a firewall or ACL was blocking the transfer, but there are no deny logs for the two involved IP addresses. 

[johnd2310]

Hi,

Have you looked at disk space on the router? Try clean out the flash.

Thanks

John

[jowood1412]

I ran an ip scp debug and found the problem in the PSCP transfer (I forgot to use -scp to force scp), so it actually worked the second time. I'm going back and accepting your first post as the solution. Thanks for the suggestion. I never would have thought of PSCP!

[Leo Laohoo]

1. What happens if you try TFTP? You can increase TFTP block size.
2. Have you read the Release Notes? It is not possible to upgrade from 3.X.X to 16.X.X without performing a mandatory ROMmon upgrade.

[iacobansilviu1]

Hi,

Make sure you have AAA (RADIUS) configured on your routers. SCP requires AAA to be configured on the routers so it has a way of knowing if the user is allowed to copy the file.

Best of luck,

Silviu