On Friday 13 May 2022 16:02 Rob Brewer wrote:
On Thursday 12 May 2022 14:15 Rob Brewer wrote:
On Thursday 12 May 2022 13:51 Michael Tremer wrote:
Hello Rob,
You seem to already have lost the kernel there which is not part of 168.
Can you extract any log files from /var/log/pakfire?
-Michael
I was upgrading 165-168 so I think Linux 5.15.23-ipfire was from 165. I am trying to build a bootable USB drive so I can mount the ssd and look at the Pakfire logs.
Rob
I have been unable to boot from a usb drive so I re-fitted an old mSATA SSD with an earlier version of IPFIRE and was surprisingly unable to boot from there also.
On investigation the bios is reporting:
SeaBIOS (version rel-1.14.0.1-0-g8610266a)
yet I manually upgraded it to v4.16.0.2 a few weeks ago after a discussion with Bernard.
https://community.ipfire.org/t/entropy-with-new-apu-firmware/7707
Could something have downgraded or corrupted the BIOS?
I'll try to re-flash back to v4.16.0.2 if I can and try to boot again.
Rob
Update..
Sucessfully booted from usb disk.
There appears to be an error in grub.cfg as the menu is looking for the wrong ipfire.img file:
/boot/grub/grub.cfg
# # DO NOT EDIT THIS FILE # # It is automatically generated by grub-mkconfig using templates # from /etc/grub.d and settings from /etc/default/grub #
### BEGIN /etc/grub.d/00_cloud ### ### END /etc/grub.d/00_cloud ###
### BEGIN /etc/grub.d/00_header ### if [ -s $prefix/grubenv ]; then load_env fi if [ "${next_entry}" ] ; then set default="${next_entry}" set next_entry= save_env next_entry set boot_once=true else set default="${saved_entry}" fi
if [ x"${feature_menuentry_id}" = xy ]; then menuentry_id_option="--id" else menuentry_id_option="" fi
export menuentry_id_option
if [ "${prev_saved_entry}" ]; then set saved_entry="${prev_saved_entry}" save_env saved_entry set prev_saved_entry= save_env prev_saved_entry set boot_once=true fi
function savedefault { if [ -z "${boot_once}" ]; then saved_entry="${chosen}" save_env saved_entry fi }
function load_video { if [ x$feature_all_video_module = xy ]; then insmod all_video else insmod efi_gop insmod efi_uga insmod ieee1275_fb insmod vbe insmod vga insmod video_bochs insmod video_cirrus fi }
serial --unit=0 --speed=115200 terminal_input serial terminal_output serial if [ x$feature_timeout_style = xy ] ; then set timeout_style=menu set timeout=5 # Fallback normal timeout code in case the timeout_style feature is # unavailable. else set timeout=5 fi ### END /etc/grub.d/00_header ###
### BEGIN /etc/grub.d/10_linux ### menuentry 'IPFire 2.27 (x86_64) - core166 Development Build: master/8f696f60 GNU/Linux' --class ipfire --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-1c836493-be8b-401c-ad0e-fb9ef6e85328' { load_video set gfxpayload=keep insmod gzio insmod part_msdos insmod ext2 set root='hd0,msdos1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 257d45b3-1483-44ab-815c-029ffa1bc8f6 else search --no-floppy --fs-uuid --set=root 257d45b3-1483-44ab-815c-029ffa1bc8f6 fi echo 'Loading Linux 5.15.23-ipfire ...' linux /vmlinuz-5.15.23-ipfire root=UUID=1c836493-be8b-401c-ad0e-fb9ef6e85328 ro panic=10 console=ttyS0,115200n8 echo 'Loading initial ramdisk ...' initrd /initramfs-5.15.23-ipfire.img } submenu 'Advanced options for IPFire 2.27 (x86_64) - core166 Development Build: master/8f696f60 GNU/Linux' $menuentry_id_option 'gnulinux-advanced-1c836493-be8b-401c-ad0e-fb9ef6e85328' { menuentry 'IPFire 2.27 (x86_64) - core166 Development Build: master/8f696f60 GNU/Linux, with Linux 5.15.23-ipfire' --class ipfire --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.15.23-ipfire-advan load_video set gfxpayload=keep insmod gzio insmod part_msdos insmod ext2 set root='hd0,msdos1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 257d45b3-1483-44ab-815c-029ffa1bc8f6 else search --no-floppy --fs-uuid --set=root 257d45b3-1483-44ab-815c-029ffa1bc8f6 fi echo 'Loading Linux 5.15.23-ipfire ...' linux /vmlinuz-5.15.23-ipfire root=UUID=1c836493-be8b-401c-ad0e-fb9ef6e85328 ro panic=10 console=ttyS0,115200n8 echo 'Loading initial ramdisk ...' initrd /initramfs-5.15.23-ipfire.img } }
### END /etc/grub.d/10_linux ###
### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###
### BEGIN /etc/grub.d/30_os-prober ### ### END /etc/grub.d/30_os-prober ###
### BEGIN /etc/grub.d/30_uefi-firmware ### ### END /etc/grub.d/30_uefi-firmware ###
### BEGIN /etc/grub.d/40_custom ### # This file provides an easy way to add custom menu entries. Simply type the # menu entries you want to add after this comment. Be careful not to change # the 'exec tail' line above. ### END /etc/grub.d/40_custom ###
### BEGIN /etc/grub.d/41_custom ### if [ -f ${config_directory}/custom.cfg ]; then source ${config_directory}/custom.cfg elif [ -z "${config_directory}" -a -f $prefix/custom.cfg ]; then source $prefix/custom.cfg fi
==================================
ls- -l /boot
-rw-r--r-- 1 root root 4869786 May 3 02:02 System.map-5.15.35-ipfire -rw-r--r-- 1 root root 182471 May 3 02:02 config-5.15.35-ipfire drwxr-xr-x 2 root root 4096 Mar 1 12:55 efi drwxr-xr-x 6 root root 4096 May 11 07:59 grub drwx------ 2 root root 16384 Mar 1 12:55 lost+found -rw-r--r-- 1 root root 6993120 May 3 02:02 vmlinuz-5.15.35-ipfire
Grub should be looking for 5.15.35-ipfire but is incorrectly looking for 5.15.23-ipfire
========================================
root@box:~# ls -l /mnt/var/log/pakfire total 1872 -rw-r--r-- 1 root root 422 May 7 09:29 install-cpufrequtils.log -rw-r--r-- 1 root root 73 Mar 29 15:53 install-firmware-update.log -rw-r--r-- 1 root root 66 Mar 29 15:53 install-flashrom.log -rw-r--r-- 1 root root 1033 Mar 10 18:00 install-nano.log -rw-r--r-- 1 root root 380 Mar 29 15:53 install-pcengines-apu-firmware.log -rw-r--r-- 1 root root 79 Mar 29 15:52 install-rsync.log -rw-r--r-- 1 root root 420549 Mar 29 12:14 update-core-upgrade-163.log -rw-r--r-- 1 root root 289507 Mar 29 12:14 update-core-upgrade-164.log -rw-r--r-- 1 root root 841876 May 11 07:59 update-core-upgrade-165.log -rw-r--r-- 1 root root 1718 May 11 07:59 update-core-upgrade-166.log -rw-r--r-- 1 root root 326352 May 11 08:00 update-core-upgrade-167.log -rw-r--r-- 1 root root 2564 Mar 29 12:16 update-nano.log
Rob
On 12 May 2022, at 11:43, Rob Brewer ipfire-devel@grantura.co.uk wrote:
On Thursday 12 May 2022 10:13 Michael Tremer wrote:
Hello,
Thanks for spending so much time on this. We definitely need to improve the general update experience since we sometimes seem to break people’s systems and it is not nice to re-install a firewall from scratch. It will take a while.
So what I can say is that the kernel module issues come from when the running kernel is changed and the kernel is trying to load any modules that now have changed. This fails by design, because we sign our kernel modules. The key is randomly generated at build time and used to sign all modules and it then thrown away. For each build, we are using a different, unique key that is not preserved.
This means that although the kernel modules are of the same version, they cannot be loaded because the signature check fails. That might also explain why you are seeing so many ipset errors, because the kernel cannot load that module any more. However, we use so much ipset now, why isn’t the module loaded from before the update was started?
The same goes for any network drivers. I assume you are using virtio or a generic e1000 network adapter which will have been initialised at boot time. The kernel should never unload the kernel module for that interface and load it again later. I have no idea what could have triggered that.
No matter what though; after you reboot, the new kernel should be booted being able to load all modules it wants and the system should run absolutely fine. Can you confirm that that is at least the case?
My Pakfire upgrade to 168 on my development APU2 board failed during upgrade and I lost ethernet communication with the PC.
The APU2 now fails after the grub prompt with the error:
*IPFire 2.27 (x86_64) - core166 Development Build: master/8f696f60 GNU/Linu
Loading Linux 5.15.23-ipfire ... error: file `/vmlinuz-5.15.23-ipfire' not found. Loading initial ramdisk ... error: you need to load the kernel first.
so it looks like update-initramfs didn't run after the upgrade.
I'll try to boot the box from a usbstick and see if I can access the disk.
Rob