SSH / Crypto issues on 6500 (ios)

[hsw_networking]

Hi All,

Having a bit of a headache withone of our 6500s.

Hostname and domain names have changed on all devices, and now cannot log in via SSH on this one machine.

• Have tried regenerateing rsa keypairs
• Have tried zeroising keypairs before regenerating
• Have tried changing hostname and domain back to what is was before, zeroising, regenerating.
• Tried removing all reference to SSH from config (in attempt to get ssh service to stop)
• Tried changing ssh version between 1 & 2 (just for the hell of it)

All above fail.

I think that part of the issue is that the <hostname><domain>.server encryption key is missing (possibly due to my over zealous zeroising efforts) but can't figure out how to regenerate it

When attempting to SSH when version 1 active I get the following on terminal monitor:

2w1d: %SSH-3-PRIVATEKEY: Unable to retrieve RSA private key for 6509.domain.com -Process= "SSH Process", ipl= 0, pid= 3

-Traceback= 415FB3E0 415F7D60 415F97E8 41358FBC 41358FA8

With version 2 active:

2w1d: SSH2 1: RSA_sign: private key not found
2w1d: SSH2 1: signature creation failed, status -1

Pertinent info:

6509#sh ver

Cisco IOS Software, s72033_rp Software (s72033_rp-IPSERVICESK9_WAN-M), Version 12.2(33)SXI3, RELEASE SOFTWARE (fc2)

<snip>

6509#sh crypto key mypubkey rsa
% Key pair was generated at: 11:22:03 Summer Apr 7 2010
Key name: 6509.domain.com
Storage Device: not specified
Usage: General Purpose Key
Key is not exportable.
Key Data:
  30819F30 0D06092A 864886F7 0D010101 05000381 8D003081 89028181 00DDAAB6
  D51372C9 53088A7C D3029C3B C3C373CE 9B39B3BC 459A4CA9 2C441C59 1BE2C860
  4F535D76 95FE7782 D5763D44 51E50008 68BFC799 13222334 29EE767D 5457B104
  21A6276B 2E535A39 B4C3B64E 4158D42C 54AD51D5 2794A3DA 1D33A09D 19D65CB2
  E73ABEA0 C1BFDA86 C4B6F903 14AC83B1 DA6E49C8 F269FEEF 94314492 D1020301 0001

(Note lack of .server encryption key)

Anyone got any ideas? As I'm all out!

Many Thanks,

Nick

[Jennifer Halim · Accepted solution]

Yes, it is a known issue in 12.2(33)SXI3.

Check out this bugID: CSCtc41114:

http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCtc41114

[hsw_networking]

Hi,

Thanks for the rapid response.  Unfortunately Cisco seem to have AGAIN messed up my login rights, and I can't get into the bug toolkit (was working 2 weeks ago )

Are any workarounds listed for this?  Have found some info about using named certificates instead, but how do you then tell the SSH process to use the named cert?  the ip ssh rsa command is missing on this IOS.

Also what is the recommeded IOS to upgrade to?  This one was only just installed on the 6500's a week ago as recommended by out support company :-/

Thanks,

Nick

[Collin Clark]

We too are seeing this bug. According to the Bug info it's fixed in the following releases.

Fixed-In
12.2(33.5.47)SXH
12.2(32.8.11)SX311
12.2(32.8.11)XJC273.31
12.2(33)SXH6
12.2(32.8.1)YCA273.35                                                          

It's also working in 12.2(33)SXI2a.

[Mathias.Rufer]

Thank you a lot, I had the exact same problem and got it working again with the workaround described in the BugID.

As a side-note, the only thing I did before hitting the bug was setting ip domain-name again. We changed the DNS server addresses and I figured while I changed that by script on every device I'd put the ip domain-name in again just to ensure all devices had the correct domain name. Note it was the EXACTLY SAME domain name in the case of the 6500s. As example I set the domain-name from "mydomain.com" to "mydomain.com".

PS: What a ¢|§¦@#¦#@ implementation

Greetings

Rufer

[hsw_networking]

Hi,

Thanks for the thread bump - it reminded me to look at this again - I have a new Cisco.com login so was finally able to see the bugtracker post, and had this fixed in a few minutes.

For anyone else stumbling upon this thread here's what you do to fix it.

The bug relates to the SSH process cutting off a character from the the end of the key name of automatically generated keys.

1) Find out what your current key is called

sh crypto key mypubkey rsa | inc name

Find the line that is simply hostname.domain.com - that's your key name (e.g. 6509_1.fred.co.uk)

2) enable ssh debugging and enable logging to your telnet session

debug ip ssh

term mon

3) Generate a new key with one character missing

crypto key gen rsa gen label 6509_1.fred.co.u mod 1024

You should see the ssh process stop then start

4) zeroise the new key you just generated

crypto key zeroize rsa 6509_1.fred.co.u

You should see the ssh service completely stop

5) generate a new named cert, e.g. ssh-key

crypto key gen rsa gen label ssh-key mod 1024

You should then see the ssh service start and you should now be able to connect.

6) make sure you can connect OK, then remove telnet as a valid input transport.

line vty 0 15

transport input ssh

Remember to disable the debug

u all

I hope this helps someone!

Cheers,

Nick

[jeffgreene]

This just worked for us too. We also are running  s72033-advipservicesk9_wan-mz.122-33.SXI3.bin  and had to do it in the order above for it to work.  Just zeroing and adding the key in didn't work. Tried multiple times.  Thanks for posting this