Bootflash corrupted error in 9396PX leaf node.

[subashchandran]

Hi All,

I am facing Bootflash corrupted error in Leaf switch-9396PX

I am able to boot with USB drive and working minimial commands and not able to format bootflash:-

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++

switch(boot)# format bootflash: check-filesystem
This command is going to erase the contents of your bootflash:.
Do you want to continue? (y/n)  [n] y
Sorry, user root is not allowed to execute '/isanboot/sbin/format-bootflash -c' as root on (none).

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Any workaround for this issue?

!

++ Second  leaf switch Boot flash(sup Local) got corrupted and not loading  with the new code.

 Boot command /Net booting (TFTP)  not working from the Loader>prompt.

USB boot is working but it will work minimal commands and not able to copy the image from tftp to bootflash:

!+++++++++++++++++++++++++++++++++++++++++++++++++

switch(boot)# copy tftp://192.168.10.1/n9000-dk9.6.1.2.I3.3.bin bootflash:n9000-

dk9.6.1.2.I3.3.bin

Destination: error opening bootflash: Device unavailable or corrupted    

!

Board type  2

IOFPGA @ 0xc8000000

SLOT_ID @ 0xf

ERROR ERROR No bootflash 

Autoboot image boot failed. Trying recovery image 

No autoboot or failed autoboot. falling to loader 

!

+++++++++++++++++++++++++++++++++++++++++++++++

[michelvankessel]

Hello,

n9000-dk9.6.1.2.I3.3.bin ​is not an ACI leaf firmware, but it is NX-OS standalone. It should look something like aci-n9000-dk9.11.0.4q.bin.​

​It is probably won't solve the bootflash problem. The only thing I can find is an recovery procedure for NX-OS mode.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-x/troubleshooting/guide/b_Cisco_Nexus_9000_Series_NX-OS_Troubleshooting_Guide_7x/b_Cisco_Nexus_9000_Series_NX-OS_Troubleshooting_Guide_7x_chapter_011.html#reference_7F7ADD86A12B4D2389E478280BE037A0

[subashchandran]

I tried this steps but no luck.

Getting the below error...

switch(boot)# format bootflash: check-filesystem
This command is going to erase the contents of your bootflash:.
Do you want to continue? (y/n)  [n] y
Sorry, user root is not allowed to execute '/isanboot/sbin/format-bootflash -c' as root on (none).

Can we get into the linux mode and try this format command?

[dpita]

Hello

Do you have console access? if so i would recommend breaking into loader and attempting to boot from there. Then we could see the output of the file being decompressed and loaded, maybe we can get a more reasonable error message. 

Are there any other images on the switch besides the 6.1.2? if loader does not work i believe the command init_system from loader should format as well. 

[Tomas de Leon]

subashchandran,

Thank you for using the Cisco Support Community.  In looking at your posts and information that you have provided, i have some additional questions that would help us in assessing your issue.  Also, this information is useful to have to give the Customer Support Engineer when you open a TAC Service Request.

* Switch Model
* New install or Existing Switch performing Upgrade?
* What is the current Operating Mode NXOS or ACI?
* How this switch ever worked and successfully running?
* Description of what are you trying to accomplish?
* What steps were performed prior to issue?
* What troubleshooting tasks already performed and what were the results?
* Other things to gather for this specific issue

At loader> prompt, simply type "dir" and capture the results.
Also, can you successfully boot the desires NXOS from USB?  Does the switch come up ok?

"Destination: error opening bootflash: Device unavailable or corrupted" can possibly indicate and Hardware issue.  To ensure that your problem is addressed promptly. I would recommend to contact the Cisco TAC and open a Service Request to address this issue.

Cheers!

T.

[subashchandran]

Hi Tom,

Switch model is 9396PX.

It was working with  NXOS code and when i convert to ACi after restart it went into loader> prompt.

I tried below steps already:-

+++USB boot with NX-OS code and it is going to switch(boot)# prompt and working with minimal commands...

+++++++++++++++

switch(boot)# format bootflash: check-filesystem
This command is going to erase the contents of your bootflash:.
Do you want to continue? (y/n)  [n] y
Sorry, user root is not allowed to execute '/isanboot/sbin/format-bootflash -c' as root on (none).

++++++++++++++++++++++++++++++++

+++USB boot with ACI code and it going to bash-4.2# prompt.

I am not able to see any space in the bootflash:-

bash-4.2# fdisk -l

Disk /dev/mtdblock0: 0 MB, 524288 bytes
120 heads, 119 sectors/track, 0 cylinders, total 1024 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mtdblock1: 0 MB, 262144 bytes
255 heads, 63 sectors/track, 0 cylinders, total 512 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mtdblock2: 0 MB, 262144 bytes
255 heads, 63 sectors/track, 0 cylinders, total 512 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mtdblock3: 0 MB, 524288 bytes
255 heads, 63 sectors/track, 0 cylinders, total 1024 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mtdblock4: 0 MB, 524288 bytes
255 heads, 63 sectors/track, 0 cylinders, total 1024 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mtdblock5: 12 MB, 12582912 bytes
96 heads, 85 sectors/track, 3 cylinders, total 24576 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mtdblock6: 52 MB, 52428800 bytes
192 heads, 207 sectors/track, 2 cylinders, total 102400 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/sdb: 8004 MB, 8004304896 bytes
35 heads, 21 sectors/track, 21269 cylinders, total 15633408 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1              32    15633407     7816688    b  W95 FAT32----USB

++ Formatted the bootflash with the below linux command:-

bash-4.2# mkfs.ext3 /dev/mtdblock6   5

mke2fs 1.42.1 (17-Feb-2012)

Filesystem label=

OS type: Linux

Block size=1024 (log=0)

Fragment size=1024 (log=0)

Stride=0 blocks, Stripe width=0 blocks

3072 inodes, 12288 blocks

614 blocks (5.00%) reserved for the super user

First data block=1

Maximum filesystem blocks=12582912

2 block groups

8192 blocks per group, 8192 fragments per group

1536 inodes per group

Superblock backups stored on blocks: 

8193

Allocating group tables: 0/21/2   done                            

Writing inode tables: 0/21/2   done                            

Creating journal (1024 blocks): done

Writing superblocks and filesystem accounting information: 0/21/2   [  507.297287] Uhhuh. NMI received for unknown reason 31 on CPU 0.

[  507.367841] Do you have a strange power saving mode enabled?

[  507.435342] Dazed and confused, but trying to continue

done

This filesystem will be automatically checked every 27 mounts or

180 days, whichever comes first.  Use tune2fs -c or -i to override.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Still i am not able to see space in the bootflash

[subashchandran]

Hi Tom,

Any workaround....

[Tomas de Leon]

From the information provided, the problem may require a hardware replacement.  Please open a Cisco TAC Case so that an engineer can assess the issue more thoroughly.

T.

[Joseph Rinehart]

Did the answer end up being a hardware replacement? I am experiencing the exact same issue and need to know if that is the resolution. 

[Tomas de Leon]

Joseph,

Thank you for using the Cisco Support Community. I would gather the following information and provide info to the Customer Support Engineer when you open a TAC Service Request.

* Switch Model
* New install or Existing Switch performing Upgrade?
* What is the current Operating Mode NXOS or ACI?
* How this switch ever worked and successfully running?
* Description of what are you trying to accomplish?
* What steps were performed prior to issue?
* What troubleshooting tasks already performed and what were the results?
* Other things to gather for this specific issue

At loader> prompt, simply type "dir" and capture the results.
Also, can you successfully boot the desires NXOS from USB? Does the switch come up ok?

"Destination: error opening bootflash: Device unavailable or corrupted" can possibly indicate and Hardware issue.

To ensure that your problem is addressed promptly. I would recommend you to contact the Cisco TAC and open a Service Request to address this issue.

Cheers!

T.

[subashchandran]

Joseph,

No workaround for this issue....

We raised RMA for H/W replacement.

Tried all the steps....