Kernel panic not syncing VFS after updating Ubuntu server 17.10.1
https://askubuntu.com/questions/1026220/kernel-panic-not-syncing-vfs-after-updating-ubuntu-server-17-10-1
I am really struggling to figure out how to fix the boot for my Ubuntu server after running some updates recently.
This is the error on boot:
[ 0.453037] Key type dns_resolver registered
[ 0.455503] RAS: Correctable Errors collector initialized.
[ 0.457334] sched_clock: Marking stable (457271600, 0)->(603354001, -146082401)
[ 0.460908] registered taskstats version 1
[ 0.462694] Loading compiled-in X.509 certificates
[ 0.466969] Loaded X.509 cert 'Build time autogenerated kernel key: 4f0e518fe6e203292fbf35df5fac675bae7fc15a'
[ 0.468915] Couldn't get size: 0x800000000000000e
[ 0.470727] MODSIGN: Couldn't get UEFI db list
[ 0.472590] Couldn't get size: 0x800000000000000e
[ 0.474500] MODSIGN: Couldn't get UEFI MokListRT
[ 0.476391] Couldn't get size: 0x800000000000000e
[ 0.478207] MODSIGN: Couldn't get UEFI dbx list
[ 0.480143] zswap: loaded using pool lzo/zbud
[ 0.485629] Key type big_key registered
[ 0.487484] Key type trusted registered
[ 0.490212] Key type encrypted registered
[ 0.491951] AppArmor: AppArmor sha1 policy hashing enabled
[ 0.493835] ima: No TPM chip found, activating TPM-bypass! (rc=-19)
[ 0.495729] evm: HMAC attrs: 0x1
[ 0.497930] Magic number: 14:354:220
[ 0.499761] dmi id: hash matches
[ 0.501911] rtc_cmos 00:05: setting system clock to 2018-04-18 08:13:36 UTC (1524039216)
[ 0.504184] BIOS EDD facility v0.16 2004-Jun-25, 0 devices found
[ 0.506050] EDD information not available.
[ 0.508035] md: Waiting for all devices to be available before autodetect
[ 0.509873] md: If you don't use raid, use raid=noautodetect
[ 0.512049] md: Autodetecting RAID arrays.
[ 0.513902] md: autorun ...
[ 0.515740] md: ... autorun DONE.
[ 0.517655] VFS: Cannot open root device "mapper/AtlassianServers--vg-root" or unknown-block(0,0): error -6
[ 0.519452] Please append a correct "root=" boot option; here are the available partitions:
[ 0.521424] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[ 0.523268] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 4.13.0-38-generic #43-Ubuntu
[ 0.525136] Hardware name: BHYVE, BIOS 1.00 03/14/2014
[ 0.527005] Call Trace:
[ 0.528833] dump_stack+0x63/0x8b
[ 0.530626] panic+0xe4/0x247
[ 0.532392] mount_block_root+0x1f4/0x2ab
[ 0.534177] ? set_debug_rodata+0x17/0x17
[ 0.535900] mount_root+0x38/0x3a
[ 0.537645] prepare_namespace+0x139/0x18e
[ 0.539455] kernel_init_freeable+0x229/0x256
[ 0.541197] ? rest_init+0xc0/0xc0
[ 0.542846] kernel_init+0xe/0xfc
[ 0.545066] ret_from_fork+0x35/0x40
[ 0.547000] Kernel Offset: 0x29800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 0.548728] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
I can't seem to see any grub loader screen for repair, so I reattached the installation ISO to the VM, and select repair. I've tried a few options, but the result is always the same.
I can get execute a shell on the root of what I think is my installation.
Running fdisk -l /dev/sda gives this output:
root@ubuntu:/# sudo fdisk -l /dev/sda
sudo: unable to resolve host ubuntu: Resource temporarily unavailable
GPT PMBR size mismatch (83886079 != 125829119) will be corrected by w(rite).
Disk /dev/sda: 60 GiB, 64424509440 bytes, 125829120 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 16384 bytes
I/O size (minimum/optimal): 16384 bytes / 16384 bytes
Disklabel type: gpt
Disk identifier: E584DF2E-7A8D-4D1B-A4C5-3AD0818A2CF6
Device Start End Sectors Size Type
/dev/sda1 2048 1050623 1048576 512M EFI System
/dev/sda2 1050624 83884031 82833408 39.5G Linux LVM
root@ubuntu:/# █
uname -r shows that I am on 4.13.0-21-generic, but I have newer installations in the boot directory. I don't know if that is because an upgrade has failed, or because the version running is currently from the CD ISO.
I have tried auto removing to clean, but there were a lot of errors. I thought it might be because there was no space left on the boot partition, but I believe there is plenty.
I have also tried update-initramfs on several of the kernel versions, but I get errors such as:
root@ubuntu:/# sudo update-initramfs -c -k 4.13.0-38-generic
sudo: unable to resolve host ubuntu: Resource temporarily unavailable
update-initramfs: Generating /boot/initrd.img-4.13.0-38-generic
WARNING: Setting CRYPTSETUP in /etc/initramfs-tools/initramfs.conf is deprecated and will stop working in the future. Use /etc/cryptsetup-initramfs/conf-hook instead.
cp: cannot stat '/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerif.ttf': No such file or directory
E: /usr/share/initramfs-tools/hooks/plymouth failed with return 1.
update-initramfs: failed for /boot/initrd.img-4.13.0-38-generic with 1.
root@ubuntu:/# █
I have tried the advice for similar looking issue here including the mount options Kernel Panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) but still get the same error.
The disks/partitions are not encrypted. This is all being run from a Bhyve VM from FreeNAS.
The results of lsblk are:
root@ubuntu:/# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 60G 0 disk
├─sda1 8:1 0 512M 0 part /boot/efi
└─sda2 8:2 0 39.5G 0 part
├─AtlassianServers--vg-root 253:0 0 35.5G 0 lvm /
└─AtlassianServers--vg-swap_1 253:1 0 4G 0 lvm
sr0 11:0 1 754M 0 rom
root@ubuntu:/# █
When I am asked when booting from the original CD, what I want to connect root to, I am selecting the AtlassianServers--vg-root option. It then displays a message saying there is also a boot/efi partition which will also be mounted.
I'm a bit stuck what to try next. Is there a way to repair it that's fairly straightforward, or do I just install again over the top, and keep the underlying files mostly intact?