I have had this problem. I opened a TAC case via our support provider for this, who could only point me to this thread!
A few weeks ago when testing the 6807-XL with FEX, I managed to reboot the device before the FEX image downloaded, which wiped off the image from the IA clients, and I had to go through the switch: 'rommon' recovery procedure as others have pointed out, so I won't repeat that
However, here is a another insight. Yesterday, I installed two 6880-X devices in a remote location. I only had console access to the devices, and none of the management interfaces were connected. I found I was in the booting loop
6880-X#sh fex
FEX FEX FEX FEX
Number Description State Model Serial
---------------------------------------------------------------------------
101 FEX0101 img_dnld C6800IA-48TD FCWXXXXXXX
102 FEX0102 img_dnld C6800IA-48TD FCWXXXXXXX
Dec 1 04:01:05.703: fex_chassis: Reg Response - version mismatch
Dec 1 04:01:05.703: fex_chassis (101)189c.5d66.8e80: during state registered, got event 11(version_mismatch)
Dec 1 04:01:05.703: @@@ fex_chassis (101)189c.5d66.8e80: registered -> version_mismatch
Dec 1 04:01:05.703: image incompatible: image to download:tftp://192.1.1.1/c6800ia-universalk9-mz.150-2.EX6.bin
Dec 1 04:01:05.717: fexx_cm:: async svi op 2 (1006, 192.1.1.101, 255.255.255.0) completed, status:success(1)
Dec 1 04:01:05.717: %SATMGR-5-IMAGE_DOWNLOAD_START: Download of image to FEX started. FEX will reboot on completion.
Dec 1 04:02:21.823: fex_chassis (101)189c.5d66.8e80: during state version_mismatch, got event 12(download_fail)
Dec 1 04:02:21.823: @@@ fex_chassis (101)189c.5d66.8e80: version_mismatch -> discovery
Dec 1 04:02:21.823: fex_chassis: Image download fail. 101
From previous experience, I know that if you have CoPP enabled, then you must allow TFTP connectivity for 192.1.1.0/24 as the FEX uses to that communicate with the chassis for the download. I already had this, but removed CoPP just in case.
I thought that I am going to have to upgrade manually, but had no mgmt or console access to the FEX, so I did extracted the FEX image from the microcode as in the rommon recovery procedure, and turned the chassis into a TFTP server for the FEX image. I then telnet'd to the FEX via its management VRF (which is only possible if the FEX is still alive and not in rommon
6880-X(config)#archive tar /xtract microcode:FEX bootdisk:
6880-X(config)#copy bootdisk:/c6800ia-universalk9-mz.150-2.EX6/c6800ia-universalk9-mz.150-2.EX6.bin bootdisk:6800IA-EX6.bin
6880-X(config)#tftp-server bootdisk:6800IA-EX6.bin
6880-X(config)#ip tftp source-interface Vlan1006
6880-X(config)#^Z
6880-X#telnet 192.1.1.101 /vrf __SVI_fex_mgmt_vrf__
Trying 192.1.1.101 ... Open
User Access Verification
Password:
FEX-101>en
Password:
FEX-101#copy tftp://192.1.1.1/6800IA-EX6.bin flash:c6800ia-universalk9-mz.150-2.EX6.bin
Destination filename [c6800ia-universalk9-mz.150-2.EX6.bin]?
Accessing tftp://192.1.1.1/6800IA-EX6.bin...
Loading 6800IA-EX6.bin from 192.1.1.1 (via Vlan1006): !!!!!!!!!!!
The image downloaded, and I was just about to verify it, when the FEX rebooted itself. I thought I had done something wrong, but after a few minutes, the FEX came back on line. I had another FEX which I hadn't even touched yet, and I saw this was also online.
#sh fex
FEX FEX FEX FEX
Number Description State Model Serial
---------------------------------------------------------------------------
101 FEX0101 online C6800IA-48TD FCW1XXXXXXX
102 FEX0102 online C6800IA-48TD FCW1XXXXXXX
From this, I gathered that when I configured 'ip tftp source-interface Vlan1006' which is the FEX vlan interface, the auto download actually worked and beat me to the FEX software upgrade.
I tried this on another chassis which had the same problem, and changed the TFTP source interface to the FEX vlan, and after 20 mins the FEXs were on line
