Unable to boot Ubuntu 24.04

[vitali-fridman]

I used this scrips many times with Ubuntu 22.04, 23.03 and 23.10. I'm now trying it with 24.04 and there is a problem. Script runs without error, but system does not boot. The boot starts (it outputs first few traces) and the host immediately reboots and it's a loop. I can import the rpool into live environment without issues. Also installing from the desktop installer with built-in ZFS option works, but this obviously not what I want as it's only for single disk. How can I troubleshoot this more?

[Sithuk]

There have been a few reported issues with booting. Check the other recent discussion threads, but one user noted that their issue was within the BIOS/UEFI and the resolution involved loading factory default secure boot keys.

[vitali-fridman]

I have default secure boot keys. I resolved the issue by installing 22.04 and upgrading it to 24.04. That worked.

[almightiest]

In my 24.04 initial install post-reboot, I had to go into zfsbootmenu and import the new rpool manually. I also had to export my other pools as they were imported into zfsbootmenu instead (i had backups of other os's with rpools and i think it got confused). It worked from there on out.

[vitali-fridman]

Unfortunately I'm not super familiar with zbm workings. Do you mean that you import correct rpool before the actual boot sequence starts? Can you elaborate how to do that? I imagine you need some sort of command line or maybe in some other way?

[almightiest]

Basically I had to drop into the ZBM command line, and run a few commands (i think these are right):

zpool list to see what's imported
zpool export {anything in that list that's not the new rpool}
zpool import to find a list of pools available to import
zpool import -d /dev/disk/by-id {correct rpool}

[vitali-fridman]

Is dropping to ZBM command shell done with [Control+R]?

[vitali-fridman]

That solution did not work for me. However I saw an error message when I dropped to ZBM command shell. Correct zpool was imported by the error message says that linux kernel was not found.

I realized that this is probably bogus message as I've seen it in many case when the boot proceeds correctly.

[vitali-fridman]

I still don't understand what could be wrong. By now I installed 24.04 on a dozen machines with variety of hardware configurations and I can't boot direct install of 24.04, needed to do an upgrade from 22.04 on all of them.

[Sithuk]

Are you able to post the installation log? Remember to remove any sensitive details such as your password.

[vitali-fridman]

ubuntu_setup_zfs_root.log

Log attached, it's from the latest version of the script. When rebooting after running "initial" target: zfsbootmenu works, boot starts, couple of normal first traces appear on a console (something like "Tearing down USB hub") and immediately system goes into reset/reboot.

[vitali-fridman]

I made some progress. The machine I've been doing this last couple of days had somewhat old BIOS. I updated it to the latest and the problem looks to be resolved. My guess is that the latest greatest Ubuntu build has AMD microcode that is incompatible with some older BIOSes. But the build which is in live environment ISO is compatible. However it also had this problem on some machines with another type of CPU/MB that I believe have the latest BIOS, but I need to double check on that.

[vitali-fridman]

I can confirm that this problem is resolved by updating BIOS to the latest. This seems to only affect AMD based systems but it seems to affect all kinds of AMD CPUs. Also zfsbootmenu that is build while installing 24.04 can't mount pools created in 22.04 installation that has not been upgraded via do-release-upgrade. Also pre-24.04 versions of zfsbootmenu can't mount 24.04 created pools, as they have some new features. I speculate that upgrading pools in upgraded 24.04 system without upgrading zfsbootmenu first will result of unbutable system.

[Sithuk]

"pre-24.04 versions of zfsbootmenu can't mount 24.04 created pools, as they have some new features"
The zfs version and kernel module in the zfsbootmenu image files will determine zfs pool support. The zfs version and kernel module in a user compiled zfsbootmenu version is determined by the zfs version and kernel module on the system that zfsbootmenu was compiled on. I haven't tested it, but you can try updating zfs and the kernel on a 22.04 system and then run "sudo generate-zbm" to update the zfsbootmenu image files. The zfs and kernel module in Zfsbootmenu should then be updated. You can check what zfs version is installed in zfsbootmenu by entering the zfsbootmenu recovery console and entering "zfs --version".

"zfsbootmenu that is build while installing 24.04 can't mount pools created in 22.04 installation"
That's odd. 24.04 currently has zfs version 2.2.2 which is more recent than the zfs version 2.1.5 currently in 22.04. If zfsbootmenu was compiled in 24.04, then it should be able to load a pool created in 22.04.

[vitali-fridman]

Sorry, my mistake. Yes, you can import pools created in 22.04 from 24.04-build zfsbootmenu. However it did not import these pools by default. I had to go into recover console and import them with "-f" force flag.

[almightiest]

Sorry, my mistake. Yes, you can import pools created in 22.04 from 24.04-build zfsbootmenu. However it did not import these pools by default. I had to go into recover console and import them with "-f" force flag.

That's the same thing I had to do after first reboot as well. For what it's worth, I had other zfs data pools and drives that may have confused the installer.