VMware has identified a problem where file systems may become read-only after encountering busy
I/O retry or SAN or iSCSI path failover errors. NCS users have also encountered this issue after the
storage has been uncleanly removed, usually brought on by a power outage.
If you can get to the shell prior to a reboot you can try issuing the following. If you don't have access
to a CLI becuase the vm is in a boot loop, proceed to the next section:
mount -o remount /
Recovering an NCS Virtual Machine stuck in a boot loop:
1. Download a live linux distro locally to your machine. Users have reported success with Fedora.
2. In vSphere, left click the NCS VM -> Summary tab -> Storage -> right click the storage -> browse datastore -> click the icon to upload a file -> browse to the ISO.
3. Exit out of the datastore browser.
4. Right click the NCS VM -> edit settings -> CD/DVD drive -> enable 'Connected' and 'Connect at power on' -> select the radio button 'Datastore ISO File' -> browse to the ISO you just uploaded -> save
5. Reload the VM and boot to the ISO
6. Get to the CLI (the exact steps to do so will completely depend on the linux distro)
7. Determine which designation has been given to the volumes we need to repair. In the output below, this
particular linux distro has given the volumes the 'sdb' designation. This can vary. There will be three of them (sdb1, sdb2, sdb3)
# fdisk -l
Disk /dev/sdb: 209.7 GB, 209715200000 bytes
255 heads, 63 sectors/track, 25496 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 * 1 64 512000 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sdb2 64 77 102400 83 Linux
Partition 2 does not end on cylinder boundary.
/dev/sdb3 77 25497 204184576 8e Linux LVM
8. Scan for volume groups:
# lvm vgscan -v
Wiping cache of LVM-capable devices
Wiping internal VG cache
Reading all physical volumes. This may take a while...
Finding all volume groups
Finding volume group "smosvg"
Found volume group "smosvg" using metadata type lvm2
Archiving volume group "smosvg" metadata (seqno 12).
Creating volume group backup "/etc/lvm/backup/smosvg" (seqno 12).
9. Activate all volume groups:
# lvm vgchange -a y
11 logical volume(s) in volume group "smosvg" now active
10. List logical volumes:
# lvm lvs –a
altrootvol smosvg -wi-a- 96.00M
home smosvg -wi-a- 96.00M
localdiskvol smosvg -wi-a- 29.28G
optvol smosvg -wi-a- 123.22G
recvol smosvg -wi-a- 96.00M
rootvol smosvg -wi-a- 3.91G
storeddatavol smosvg -wi-a- 9.75G
swapvol smosvg -wi-a- 15.62G
tmpvol smosvg -wi-a- 1.94G
usrvol smosvg -wi-a- 6.81G
varvol smosvg -wi-a- 3.91G
11. Use fsck to check all the partitions on the drive. It is ok if you receive errors for one of these.
# fsck -t ext3 –y /dev/sdb1
# fsck -t ext3 –y /dev/sdb2
# fsck -t ext3 -y /dev/sdb3
12. Perform the same steps for all of the logical volumes in the group identified with the lvscan command (remember to use the –y flag in all cases).
The first error we get is an error message that states: "SYS-2-MALLOCFAIL - Memory allocation of 962820 bytes failed from 0x8000561D10652750, alignment 8 Pool: Processor Free: 71887584 Cause: Memory fragmentation Alternate Pool: None Free: 0 Cause: N...
How to check Nexus 9K switches can be managed with Dnac current version 184.108.40.206 or not. Is there any possibility of checking compatibility with our existing version ? We want to deploy Nexus switches in DNAC tool with version 220.127.116.11, So please sugge...
Have you ever wondered why pluggable optics exist? Have you ever wondered what acronyms like QSFP, LR4, FEC, and PAM4 actually mean? In this episode I continue my conversation with my colleague Ray Nering. He explains to me how a lot of these came to be, ...
Hello everyone.Im having some tasks to do in my final exam and everything I try seems to fail, can u guys help me please?So, as you see in the image (1), I'm trying to ping 18.104.22.168, 22.214.171.124 and 126.96.36.199 Servers from the higher router ca...
CiscHello, We have a new issue after retrying the upgrade of the DNA center. We can access the web page but not able to login via admin account: We can only ssh via maglev. Steps we performed, Tried to reset the admin account via...