From mboxrd@z Thu Jan 1 00:00:00 1970 From: Rob Brewer To: development@lists.ipfire.org Subject: Re: Feedback on problems with Core Update 168 Testing Date: Fri, 13 May 2022 18:16:18 +0100 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3892997157983518036==" List-Id: --===============3892997157983518036== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On Friday 13 May 2022 16:02 Rob Brewer wrote: > On Thursday 12 May 2022 14:15 Rob Brewer wrote: >=20 >> On Thursday 12 May 2022 13:51 Michael Tremer wrote: >>=20 >>> Hello Rob, >>>=20 >>> You seem to already have lost the kernel there which is not part of 168. >>>=20 >>> Can you extract any log files from /var/log/pakfire? >>>=20 >>> -Michael >>> >> I was upgrading 165-168 so I think Linux 5.15.23-ipfire was from 165. >> I am trying to build a bootable USB drive so I can mount the ssd and look >> at the Pakfire logs. >>=20 >>=20 >> Rob >> > I have been unable to boot from a usb drive so I re-fitted an old mSATA > SSD with an earlier version of IPFIRE and was surprisingly unable to boot > from there also. >=20 > On investigation the bios is reporting: >=20 > SeaBIOS (version rel-1.14.0.1-0-g8610266a) >=20 > yet I manually upgraded it to v4.16.0.2 a few weeks ago after a > discussion with Bernard. >=20 > https://community.ipfire.org/t/entropy-with-new-apu-firmware/7707 >=20 > Could something have downgraded or corrupted the BIOS? >=20 > I'll try to re-flash back to v4.16.0.2 if I can and try to boot again. >=20 >=20 > Rob > Update.. Sucessfully booted from usb disk. There appears to be an error in grub.cfg as the menu is looking for the wrong= ipfire.img file: /boot/grub/grub.cfg # # DO NOT EDIT THIS FILE # # It is automatically generated by grub-mkconfig using templates # from /etc/grub.d and settings from /etc/default/grub # ### BEGIN /etc/grub.d/00_cloud ### ### END /etc/grub.d/00_cloud ### ### BEGIN /etc/grub.d/00_header ### if [ -s $prefix/grubenv ]; then load_env fi if [ "${next_entry}" ] ; then set default=3D"${next_entry}" set next_entry=3D save_env next_entry set boot_once=3Dtrue else set default=3D"${saved_entry}" fi if [ x"${feature_menuentry_id}" =3D xy ]; then menuentry_id_option=3D"--id" else menuentry_id_option=3D"" fi export menuentry_id_option if [ "${prev_saved_entry}" ]; then set saved_entry=3D"${prev_saved_entry}" save_env saved_entry set prev_saved_entry=3D save_env prev_saved_entry set boot_once=3Dtrue fi function savedefault { if [ -z "${boot_once}" ]; then saved_entry=3D"${chosen}" save_env saved_entry fi } function load_video { if [ x$feature_all_video_module =3D xy ]; then insmod all_video else insmod efi_gop insmod efi_uga insmod ieee1275_fb insmod vbe insmod vga insmod video_bochs insmod video_cirrus fi } serial --unit=3D0 --speed=3D115200 terminal_input serial terminal_output serial if [ x$feature_timeout_style =3D xy ] ; then set timeout_style=3Dmenu set timeout=3D5 # Fallback normal timeout code in case the timeout_style feature is # unavailable. =20 else =20 set timeout=3D5 =20 fi =20 ### END /etc/grub.d/00_header ### =20 =20 ### BEGIN /etc/grub.d/10_linux ### =20 menuentry 'IPFire 2.27 (x86_64) - core166 Development Build: master/8f696f60 = GNU/Linux' --class ipfire --class gnu-linux --class gnu --class os $menuentry= _id_option 'gnulinux-simple-1c836493-be8b-401c-ad0e-fb9ef6e85328' { load_video =20 set gfxpayload=3Dkeep =20 insmod gzio = = =20 insmod part_msdos =20 insmod ext2 =20 set root=3D'hd0,msdos1' =20 if [ x$feature_platform_search_hint =3D xy ]; then = = =20 search --no-floppy --fs-uuid --set=3Droot --hint-bios=3Dhd0,msdos1 = --hint-efi=3Dhd0,msdos1 --hint-baremetal=3Dahci0,msdos1 257d45b3-1483-44ab-8= 15c-029ffa1bc8f6 =20 else = = =20 search --no-floppy --fs-uuid --set=3Droot 257d45b3-1483-44ab-815c-0= 29ffa1bc8f6 = =20 fi = = =20 echo 'Loading Linux 5.15.23-ipfire ...' linux /vmlinuz-5.15.23-ipfire root=3DUUID=3D1c836493-be8b-401c-ad0e= -fb9ef6e85328 ro panic=3D10 console=3DttyS0,115200n8 echo 'Loading initial ramdisk ...' =20 initrd /initramfs-5.15.23-ipfire.img = = =20 } = = =20 submenu 'Advanced options for IPFire 2.27 (x86_64) - core166 Development Buil= d: master/8f696f60 GNU/Linux' $menuentry_id_option 'gnulinux-advanced-1c83649= 3-be8b-401c-ad0e-fb9ef6e85328' { =20 menuentry 'IPFire 2.27 (x86_64) - core166 Development Build: master/8= f696f60 GNU/Linux, with Linux 5.15.23-ipfire' --class ipfire --class gnu-linu= x --class gnu --class os $menuentry_id_option 'gnulinux-5.15.23-ipfire-advan load_video = =20 set gfxpayload=3Dkeep = = =20 insmod gzio = = =20 insmod part_msdos = =20 insmod ext2 = = =20 set root=3D'hd0,msdos1' = = =20 if [ x$feature_platform_search_hint =3D xy ]; then = = = =20 search --no-floppy --fs-uuid --set=3Droot --hint-bios=3Dhd0= ,msdos1 --hint-efi=3Dhd0,msdos1 --hint-baremetal=3Dahci0,msdos1 257d45b3-148= 3-44ab-815c-029ffa1bc8f6 = =20 else = = =20 search --no-floppy --fs-uuid --set=3Droot 257d45b3-1483-44a= b-815c-029ffa1bc8f6 = = =20 fi = = =20 echo 'Loading Linux 5.15.23-ipfire ...' = =20 linux /vmlinuz-5.15.23-ipfire root=3DUUID=3D1c836493-be8b-4= 01c-ad0e-fb9ef6e85328 ro panic=3D10 console=3DttyS0,115200n8 echo 'Loading initial ramdisk ...' = = =20 initrd /initramfs-5.15.23-ipfire.img = = =20 } = = =20 } = = =20 = = =20 ### END /etc/grub.d/10_linux ### = =20 = = =20 ### BEGIN /etc/grub.d/20_linux_xen ### = = =20 = =20 ### END /etc/grub.d/20_linux_xen ### = = =20 =20 ### BEGIN /etc/grub.d/30_os-prober ### = = =20 ### END /etc/grub.d/30_os-prober ### = = =20 = =20 ### BEGIN /etc/grub.d/30_uefi-firmware ### = = =20 ### END /etc/grub.d/30_uefi-firmware ### = = =20 = =20 ### BEGIN /etc/grub.d/40_custom ### = =20 # This file provides an easy way to add custom menu entries. Simply type the # menu entries you want to add after this comment. Be careful not to change = = =20 # the 'exec tail' line above. = = =20 ### END /etc/grub.d/40_custom ### = = =20 = = = = =20 ### BEGIN /etc/grub.d/41_custom ### = =20 if [ -f ${config_directory}/custom.cfg ]; then = =20 source ${config_directory}/custom.cfg = elif [ -z "${config_directory}" -a -f $prefix/custom.cfg ]; then = = =20 source $prefix/custom.cfg = = =20 fi =20 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D ls- -l /boot -rw-r--r-- 1 root root 4869786 May 3 02:02 System.map-5.15.35-i= pfire -rw-r--r-- 1 root root 182471 May 3 02:02 config-5.15.35-ipfire drwxr-xr-x 2 root root 4096 Mar 1 12:55 efi drwxr-xr-x 6 root root 4096 May 11 07:59 grub drwx------ 2 root root 16384 Mar 1 12:55 lost+found -rw-r--r-- 1 root root 6993120 May 3 02:02 vmlinuz-5.15.35-ipfi= re Grub should be looking for 5.15.35-ipfire but is incorrectly looking for 5.15= .23-ipfire =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D root(a)box:~# ls -l /mnt/var/log/pakfire total 1872 -rw-r--r-- 1 root root 422 May 7 09:29 install-cpufrequtils= .log -rw-r--r-- 1 root root 73 Mar 29 15:53 install-firmware-upd= ate.log -rw-r--r-- 1 root root 66 Mar 29 15:53 install-flashrom.log -rw-r--r-- 1 root root 1033 Mar 10 18:00 install-nano.log -rw-r--r-- 1 root root 380 Mar 29 15:53 install-pcengines-ap= u-firmware.log -rw-r--r-- 1 root root 79 Mar 29 15:52 install-rsync.log -rw-r--r-- 1 root root 420549 Mar 29 12:14 update-core-upgrade-= 163.log -rw-r--r-- 1 root root 289507 Mar 29 12:14 update-core-upgrade-= 164.log -rw-r--r-- 1 root root 841876 May 11 07:59 update-core-upgrade-= 165.log -rw-r--r-- 1 root root 1718 May 11 07:59 update-core-upgrade-= 166.log -rw-r--r-- 1 root root 326352 May 11 08:00 update-core-upgrade-= 167.log -rw-r--r-- 1 root root 2564 Mar 29 12:16 update-nano.log Rob =20 >>>> On 12 May 2022, at 11:43, Rob Brewer >>>> wrote: >>>>=20 >>>> On Thursday 12 May 2022 10:13 Michael Tremer wrote: >>>>=20 >>>>> Hello, >>>>>=20 >>>>> Thanks for spending so much time on this. We definitely need to >>>>> improve the general update experience since we sometimes seem to break >>>>> people=E2=80=99s systems and it is not nice to re-install a firewall fr= om >>>>> scratch. It will take a while. >>>>>=20 >>>>> So what I can say is that the kernel module issues come from when the >>>>> running kernel is changed and the kernel is trying to load any modules >>>>> that now have changed. This fails by design, because we sign our >>>>> kernel modules. The key is randomly generated at build time and used >>>>> to sign all modules and it then thrown away. For each build, we are >>>>> using a different, unique key that is not preserved. >>>>>=20 >>>>> This means that although the kernel modules are of the same version, >>>>> they cannot be loaded because the signature check fails. That might >>>>> also explain why you are seeing so many ipset errors, because the >>>>> kernel cannot load that module any more. However, we use so much ipset >>>>> now, why isn=E2=80=99t the module loaded from before the update was sta= rted? >>>>>=20 >>>>> The same goes for any network drivers. I assume you are using virtio >>>>> or a generic e1000 network adapter which will have been initialised at >>>>> boot time. The kernel should never unload the kernel module for that >>>>> interface and load it again later. I have no idea what could have >>>>> triggered that. >>>>>=20 >>>>> No matter what though; after you reboot, the new kernel should be >>>>> booted being able to load all modules it wants and the system should >>>>> run absolutely fine. Can you confirm that that is at least the case? >>>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> My Pakfire upgrade to 168 on my development APU2 board failed during >>>> upgrade and I lost ethernet communication with the PC. >>>>=20 >>>> The APU2 now fails after the grub prompt with the error: >>>>=20 >>>> *IPFire 2.27 (x86_64) - core166 Development Build: master/8f696f60 >>>> GNU/Linu >>>>=20 >>>> Loading Linux 5.15.23-ipfire ... >>>> error: file `/vmlinuz-5.15.23-ipfire' not found. >>>> Loading initial ramdisk ... >>>> error: you need to load the kernel first. >>>>=20 >>>> so it looks like update-initramfs didn't run after the upgrade. >>>>=20 >>>> I'll try to boot the box from a usbstick and see if I can access the >>>> disk. >>>>=20 >>>> Rob --===============3892997157983518036==--