On Friday 13 May 2022 16:02 Rob Brewer wrote: > On Thursday 12 May 2022 14:15 Rob Brewer wrote: > >> On Thursday 12 May 2022 13:51 Michael Tremer wrote: >> >>> Hello Rob, >>> >>> You seem to already have lost the kernel there which is not part of 168. >>> >>> Can you extract any log files from /var/log/pakfire? >>> >>> -Michael >>> >> I was upgrading 165-168 so I think Linux 5.15.23-ipfire was from 165. >> I am trying to build a bootable USB drive so I can mount the ssd and look >> at the Pakfire logs. >> >> >> Rob >> > I have been unable to boot from a usb drive so I re-fitted an old mSATA > SSD with an earlier version of IPFIRE and was surprisingly unable to boot > from there also. > > On investigation the bios is reporting: > > SeaBIOS (version rel-1.14.0.1-0-g8610266a) > > yet I manually upgraded it to v4.16.0.2 a few weeks ago after a > discussion with Bernard. > > https://community.ipfire.org/t/entropy-with-new-apu-firmware/7707 > > Could something have downgraded or corrupted the BIOS? > > I'll try to re-flash back to v4.16.0.2 if I can and try to boot again. > > > Rob > Update.. Sucessfully booted from usb disk. There appears to be an error in grub.cfg as the menu is looking for the wrong ipfire.img file: /boot/grub/grub.cfg # # DO NOT EDIT THIS FILE # # It is automatically generated by grub-mkconfig using templates # from /etc/grub.d and settings from /etc/default/grub # ### BEGIN /etc/grub.d/00_cloud ### ### END /etc/grub.d/00_cloud ### ### BEGIN /etc/grub.d/00_header ### if [ -s $prefix/grubenv ]; then load_env fi if [ "${next_entry}" ] ; then set default="${next_entry}" set next_entry= save_env next_entry set boot_once=true else set default="${saved_entry}" fi if [ x"${feature_menuentry_id}" = xy ]; then menuentry_id_option="--id" else menuentry_id_option="" fi export menuentry_id_option if [ "${prev_saved_entry}" ]; then set saved_entry="${prev_saved_entry}" save_env saved_entry set prev_saved_entry= save_env prev_saved_entry set boot_once=true fi function savedefault { if [ -z "${boot_once}" ]; then saved_entry="${chosen}" save_env saved_entry fi } function load_video { if [ x$feature_all_video_module = xy ]; then insmod all_video else insmod efi_gop insmod efi_uga insmod ieee1275_fb insmod vbe insmod vga insmod video_bochs insmod video_cirrus fi } serial --unit=0 --speed=115200 terminal_input serial terminal_output serial if [ x$feature_timeout_style = xy ] ; then set timeout_style=menu set timeout=5 # Fallback normal timeout code in case the timeout_style feature is # unavailable. else set timeout=5 fi ### END /etc/grub.d/00_header ### ### BEGIN /etc/grub.d/10_linux ### menuentry 'IPFire 2.27 (x86_64) - core166 Development Build: master/8f696f60 GNU/Linux' --class ipfire --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-1c836493-be8b-401c-ad0e-fb9ef6e85328' { load_video set gfxpayload=keep insmod gzio insmod part_msdos insmod ext2 set root='hd0,msdos1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 257d45b3-1483-44ab-815c-029ffa1bc8f6 else search --no-floppy --fs-uuid --set=root 257d45b3-1483-44ab-815c-029ffa1bc8f6 fi echo 'Loading Linux 5.15.23-ipfire ...' linux /vmlinuz-5.15.23-ipfire root=UUID=1c836493-be8b-401c-ad0e-fb9ef6e85328 ro panic=10 console=ttyS0,115200n8 echo 'Loading initial ramdisk ...' initrd /initramfs-5.15.23-ipfire.img } submenu 'Advanced options for IPFire 2.27 (x86_64) - core166 Development Build: master/8f696f60 GNU/Linux' $menuentry_id_option 'gnulinux-advanced-1c836493-be8b-401c-ad0e-fb9ef6e85328' { menuentry 'IPFire 2.27 (x86_64) - core166 Development Build: master/8f696f60 GNU/Linux, with Linux 5.15.23-ipfire' --class ipfire --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.15.23-ipfire-advan load_video set gfxpayload=keep insmod gzio insmod part_msdos insmod ext2 set root='hd0,msdos1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 257d45b3-1483-44ab-815c-029ffa1bc8f6 else search --no-floppy --fs-uuid --set=root 257d45b3-1483-44ab-815c-029ffa1bc8f6 fi echo 'Loading Linux 5.15.23-ipfire ...' linux /vmlinuz-5.15.23-ipfire root=UUID=1c836493-be8b-401c-ad0e-fb9ef6e85328 ro panic=10 console=ttyS0,115200n8 echo 'Loading initial ramdisk ...' initrd /initramfs-5.15.23-ipfire.img } } ### END /etc/grub.d/10_linux ### ### BEGIN /etc/grub.d/20_linux_xen ### ### END /etc/grub.d/20_linux_xen ### ### BEGIN /etc/grub.d/30_os-prober ### ### END /etc/grub.d/30_os-prober ### ### BEGIN /etc/grub.d/30_uefi-firmware ### ### END /etc/grub.d/30_uefi-firmware ### ### BEGIN /etc/grub.d/40_custom ### # This file provides an easy way to add custom menu entries. Simply type the # menu entries you want to add after this comment. Be careful not to change # the 'exec tail' line above. ### END /etc/grub.d/40_custom ### ### BEGIN /etc/grub.d/41_custom ### if [ -f ${config_directory}/custom.cfg ]; then source ${config_directory}/custom.cfg elif [ -z "${config_directory}" -a -f $prefix/custom.cfg ]; then source $prefix/custom.cfg fi ================================== ls- -l /boot -rw-r--r-- 1 root root 4869786 May 3 02:02 System.map-5.15.35-ipfire -rw-r--r-- 1 root root 182471 May 3 02:02 config-5.15.35-ipfire drwxr-xr-x 2 root root 4096 Mar 1 12:55 efi drwxr-xr-x 6 root root 4096 May 11 07:59 grub drwx------ 2 root root 16384 Mar 1 12:55 lost+found -rw-r--r-- 1 root root 6993120 May 3 02:02 vmlinuz-5.15.35-ipfire Grub should be looking for 5.15.35-ipfire but is incorrectly looking for 5.15.23-ipfire ======================================== root(a)box:~# ls -l /mnt/var/log/pakfire total 1872 -rw-r--r-- 1 root root 422 May 7 09:29 install-cpufrequtils.log -rw-r--r-- 1 root root 73 Mar 29 15:53 install-firmware-update.log -rw-r--r-- 1 root root 66 Mar 29 15:53 install-flashrom.log -rw-r--r-- 1 root root 1033 Mar 10 18:00 install-nano.log -rw-r--r-- 1 root root 380 Mar 29 15:53 install-pcengines-apu-firmware.log -rw-r--r-- 1 root root 79 Mar 29 15:52 install-rsync.log -rw-r--r-- 1 root root 420549 Mar 29 12:14 update-core-upgrade-163.log -rw-r--r-- 1 root root 289507 Mar 29 12:14 update-core-upgrade-164.log -rw-r--r-- 1 root root 841876 May 11 07:59 update-core-upgrade-165.log -rw-r--r-- 1 root root 1718 May 11 07:59 update-core-upgrade-166.log -rw-r--r-- 1 root root 326352 May 11 08:00 update-core-upgrade-167.log -rw-r--r-- 1 root root 2564 Mar 29 12:16 update-nano.log Rob >>>> On 12 May 2022, at 11:43, Rob Brewer >>>> wrote: >>>> >>>> On Thursday 12 May 2022 10:13 Michael Tremer wrote: >>>> >>>>> Hello, >>>>> >>>>> Thanks for spending so much time on this. We definitely need to >>>>> improve the general update experience since we sometimes seem to break >>>>> people’s systems and it is not nice to re-install a firewall from >>>>> scratch. It will take a while. >>>>> >>>>> So what I can say is that the kernel module issues come from when the >>>>> running kernel is changed and the kernel is trying to load any modules >>>>> that now have changed. This fails by design, because we sign our >>>>> kernel modules. The key is randomly generated at build time and used >>>>> to sign all modules and it then thrown away. For each build, we are >>>>> using a different, unique key that is not preserved. >>>>> >>>>> This means that although the kernel modules are of the same version, >>>>> they cannot be loaded because the signature check fails. That might >>>>> also explain why you are seeing so many ipset errors, because the >>>>> kernel cannot load that module any more. However, we use so much ipset >>>>> now, why isn’t the module loaded from before the update was started? >>>>> >>>>> The same goes for any network drivers. I assume you are using virtio >>>>> or a generic e1000 network adapter which will have been initialised at >>>>> boot time. The kernel should never unload the kernel module for that >>>>> interface and load it again later. I have no idea what could have >>>>> triggered that. >>>>> >>>>> No matter what though; after you reboot, the new kernel should be >>>>> booted being able to load all modules it wants and the system should >>>>> run absolutely fine. Can you confirm that that is at least the case? >>>>> >>>> >>>> >>>> >>>> My Pakfire upgrade to 168 on my development APU2 board failed during >>>> upgrade and I lost ethernet communication with the PC. >>>> >>>> The APU2 now fails after the grub prompt with the error: >>>> >>>> *IPFire 2.27 (x86_64) - core166 Development Build: master/8f696f60 >>>> GNU/Linu >>>> >>>> Loading Linux 5.15.23-ipfire ... >>>> error: file `/vmlinuz-5.15.23-ipfire' not found. >>>> Loading initial ramdisk ... >>>> error: you need to load the kernel first. >>>> >>>> so it looks like update-initramfs didn't run after the upgrade. >>>> >>>> I'll try to boot the box from a usbstick and see if I can access the >>>> disk. >>>> >>>> Rob