From mboxrd@z Thu Jan 1 00:00:00 1970 From: Adolf Belka To: development@lists.ipfire.org Subject: Re: Feedback on problems with Core Update 168 Testing Date: Thu, 19 May 2022 22:10:47 +0200 Message-ID: <4b1d962a-6a1b-65b4-754f-ec7deed4eb35@ipfire.org> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4331578002912941262==" List-Id: --===============4331578002912941262== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Michael, On 19/05/2022 13:33, Michael Tremer wrote: > Hello, >=20 > That would be brilliant. Let me know how it is going. Not a good result I am afraid. >=20 > You would need to run this first: >=20 > https://patchwork.ipfire.org/project/ipfire/patch/20220519085634.197389-1-m= ichael.tremer(a)ipfire.org/ I ran that and rd.auto was added to /etc/default/grub You didn't mention about running grub-mkconfig -o /boot/grub/grub.cfg and I h= ad to run that previously to get rd.auto persistent so I ran that here. Not s= ure if that was correct or not. >=20 > And then just run the script and reboot. The boot process might stall for a= minute. Just let it do its job. And the RAID should come up just fine. I ran the script and cat /proc/mdstat came back with no personalities I then rebooted and after stopping everything the screen went to restart but = then came up with the following message:- FATAL: No bootable medium found! System halted I have a vm clone of the CU167 vm with the failed raid array and with various= changes done with only one disk connected so I can clone from this vm and ru= n whatever further changes you identify are needed. Regards, Adolf. >=20 > The script will also avoid the previous issues with the file system that yo= u have outlined after an unsuccessful resync. >=20 > -Michael >=20 >> On 19 May 2022, at 12:21, Adolf Belka wrote: >> >> Hi, >> >> On 19/05/2022 13:14, Michael Tremer wrote: >>> Hello, >>>> On 19 May 2022, at 11:57, Adolf Belka wrote: >>>> >>>> Hi Michael, >>>> >>>> On 19/05/2022 10:59, Michael Tremer wrote: >>>>> Please note the patch that I just posted to fix any broken RAID arrays. >>>>> Does anyone have any systems that we can use to test this? >>>> Does the script need to be run manually or has it been added to the nigh= tlies together with the rd.auto patch so that I can do a CU168 Testing update= evaluation? >>> My patch includes that it will run automatically with the updater. Howeve= r, that is not part of the build yet, because I would like it to be confirmed= that I do not destroy anything. The whole script is a rather risky operation= :) >> Okay, clear. >> Then I will execute the rd.auto change reboot and then run the script. >> >> Will feedback how things go. >> Adolf. >>>> >>>> Regards, >>>> Adolf. >>>> >>>>> -Michael >>>>>> On 17 May 2022, at 09:09, Adolf Belka wrote: >>>>>> >>>>>> Hi Michael, >>>>>> >>>>>> On 13/05/2022 22:37, Michael Tremer wrote: >>>>>>> Hello, >>>>>>> Okay, this is really bad then if we are breaking RAID installations. >>>>>>> I tried to debug this, but VirtualBox is not my friend today. >>>>>>> I asked some bug reporter to provide me with the logs that I want ove= r here: >>>>>>> https://bugzilla.ipfire.org/show_bug.cgi?id=3D12862 >>>>>> After applying this fix to my CU167 vm and proving that it stayed cons= istent I have now done an upgrade to CU168 Testing. This does a CU167 update = before doing the CU168 update and I am glad to announce that my previous prob= lems with losing all my network interfaces etc are gone. >>>>>> >>>>>> My Core Update 168 Development Build: master/bbd4767f successfully res= tarted with a fully active raid array and I will now restart doing my CU168 T= esting evaluation. >>>>>> >>>>>> Thanks very much for all the help. >>>>>> >>>>>> Regards, >>>>>> Adolf. >>>>>>> -Michael >>>>>>>> On 13 May 2022, at 18:12, Adolf Belka wro= te: >>>>>>>> >>>>>>>> Hi All, >>>>>>>> >>>>>>>> I think I may understand what is happening with my vm now. Of course= I could also be wrong :-) but I will give it a go. >>>>>>>> >>>>>>>> I saw bug#12862 where upgrading from CU166 to CU167 had stopped the = raid array running so that IPFire was only on one disk. >>>>>>>> >>>>>>>> I had missed seeing this in my evaluation of CU167 on my vm which do= es run with a raid array. >>>>>>>> >>>>>>>> The vm shows the following >>>>>>>> >>>>>>>> -bash-5.1$ cat /proc/mdstat >>>>>>>> Personalities : >>>>>>>> md127 : inactive sdb[1](S) >>>>>>>> 52428724 blocks super 1.0 >>>>>>>> >>>>>>>> unused devices: >>>>>>>> >>>>>>>> >>>>>>>> It looks like IPFire was only running on sda. >>>>>>>> >>>>>>>> So when I am doing an upgrade to CU168 it is likely that only one di= sk of the raid pair is being updated. >>>>>>>> >>>>>>>> The CU168 vm that is working has the same as the above setting. >>>>>>>> >>>>>>>> The CU168 vm that is not working has the following setting >>>>>>>> >>>>>>>> -bash-5.1$ cat /proc/mdstat >>>>>>>> Personalities : >>>>>>>> md127 : inactive sda[0](S) >>>>>>>> 52428724 blocks super 1.0 >>>>>>>> >>>>>>>> unused devices: >>>>>>>> >>>>>>>> So it looks like it has booted up on the disk that was not updated f= rom the raid pair. >>>>>>>> >>>>>>>> >>>>>>>> This probably also explains why occasionally I have had a vm that wa= s not working suddenly start working properly when I reboot it, but booting a= gain can put it back into non-working mode. >>>>>>>> >>>>>>>> It is probably dependent on which of the two disks actually get used= to boot from. >>>>>>>> >>>>>>>> So the problem looks like it would only affect people who have raid = setups and the problem originated in the CU167 upgrade. >>>>>>>> >>>>>>>> So raid systems need to have their raid arrays fully working before = doing an upgrade. >>>>>>>> >>>>>>>> At least the above is my interpretation of what I have found. What d= oes everyone else think. >>>>>>>> >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Adolf. >>>>>>>> >>>>>>>> >>>>>>>> On 13/05/2022 15:59, Adolf Belka wrote: >>>>>>>>> Hi Michael, >>>>>>>>> >>>>>>>>> I have found some differences between the vm's that have upgraded w= ith no problem and those that have lost most of the modules. >>>>>>>>> >>>>>>>>> I did another three vm clones and upgrades and the first two all we= nt with no problems. I was starting to think the issue had gone away but on t= he third one the same problem happened after the reboot. >>>>>>>>> >>>>>>>>> However I did copy the boot directory info for these two cases and = found an interesting effect. >>>>>>>>> >>>>>>>>> For the problem vm here was the boot directory layout of the CU167 = base machine >>>>>>>>> >>>>>>>>> drwxr-xr-x 5 root root 4.0K Apr 28 09:47 . >>>>>>>>> drwxr-xr-x 21 root root 4.0K Nov 28 15:49 .. >>>>>>>>> -rw-r--r-- 1 root root 179K Apr 26 13:00 config-5.15.35-ipfire >>>>>>>>> drwxr-xr-x 3 root root 16K Jan 1 1970 efi >>>>>>>>> drwxr-xr-x 6 root root 4.0K Apr 28 09:47 grub >>>>>>>>> -rw------- 1 root root 26M Apr 28 09:47 initramfs-5.15.35-ipfire.= img >>>>>>>>> drwx------ 2 root root 16K Sep 30 2021 lost+found >>>>>>>>> -rw-r--r-- 1 root root 4.7M Apr 26 13:00 System.map-5.15.35-ipfire >>>>>>>>> -rw-r--r-- 1 root root 6.7M Apr 26 13:00 vmlinuz-5.15.35-ipfire >>>>>>>>> >>>>>>>>> After upgrade and before reboot this had changed to >>>>>>>>> >>>>>>>>> drwxr-xr-x 5 root root 4.0K May 13 15:40 . >>>>>>>>> drwxr-xr-x 21 root root 4.0K Nov 28 15:49 .. >>>>>>>>> -rw-r--r-- 1 root root 179K May 3 04:02 config-5.15.35-ipfire >>>>>>>>> drwxr-xr-x 3 root root 16K Jan 1 1970 efi >>>>>>>>> drwxr-xr-x 6 root root 4.0K May 13 15:41 grub >>>>>>>>> -rw------- 1 root root 26M May 13 15:41 initramfs-5.15.35-ipfire.= img >>>>>>>>> drwx------ 2 root root 16K Sep 30 2021 lost+found >>>>>>>>> -rw-r--r-- 1 root root 4.7M May 3 04:02 System.map-5.15.35-ipfire >>>>>>>>> -rw-r--r-- 1 root root 6.7M May 3 04:02 vmlinuz-5.15.35-ipfire >>>>>>>>> >>>>>>>>> Dates and times of some of the files/directories have changed as wo= uld be expected. >>>>>>>>> >>>>>>>>> After Reboot and with the problem the boot directory was now >>>>>>>>> >>>>>>>>> drwxr-xr-x 5 root root 4.0K Apr 28 09:47 . >>>>>>>>> drwxr-xr-x 21 root root 4.0K Nov 28 15:49 .. >>>>>>>>> -rw-r--r-- 1 root root 179K Apr 26 13:00 config-5.15.35-ipfire >>>>>>>>> drwxr-xr-x 3 root root 4.0K Sep 30 2021 efi >>>>>>>>> drwxr-xr-x 6 root root 4.0K Apr 28 09:47 grub >>>>>>>>> -rw------- 1 root root 26M Apr 28 09:47 initramfs-5.15.35-ipfire.= img >>>>>>>>> drwx------ 2 root root 16K Sep 30 2021 lost+found >>>>>>>>> -rw-r--r-- 1 root root 4.7M Apr 26 13:00 System.map-5.15.35-ipfire >>>>>>>>> -rw-r--r-- 1 root root 6.7M Apr 26 13:00 vmlinuz-5.15.35-ipfire >>>>>>>>> >>>>>>>>> Which is the same as for the original CU167 machine except that the= date/time of the efi directory has changed >>>>>>>>> >>>>>>>>> For the vm clone that upgraded with no problems the boot directory = after reboot was the same as after upgrade. >>>>>>>>> >>>>>>>>> I don';t understand why or how this has happened but it is certainl= y a difference between the vm's having the problem and the ones not. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Adolf. >>>>>>>>> >>>>>>>>> On 13/05/2022 12:09, Michael Tremer wrote: >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>>> On 12 May 2022, at 21:10, Adolf Belka = wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Michael, >>>>>>>>>>> >>>>>>>>>>> On 12/05/2022 14:53, Michael Tremer wrote: >>>>>>>>>>>> Hello, >>>>>>>>>>>>> On 12 May 2022, at 12:25, Adolf Belka wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> On 12/05/2022 11:13, Michael Tremer wrote: >>>>>>>>>>>>>> Hello, >>>>>>>>>>>>>> Thanks for spending so much time on this. We definitely need t= o improve the general update experience since we sometimes seem to break peop= le=E2=80=99s systems and it is not nice to re-install a firewall from scratch= . It will take a while. >>>>>>>>>>>>>> So what I can say is that the kernel module issues come from w= hen the running kernel is changed and the kernel is trying to load any module= s that now have changed. This fails by design, because we sign our kernel mod= ules. The key is randomly generated at build time and used to sign all module= s and it then thrown away. For each build, we are using a different, unique k= ey that is not preserved. >>>>>>>>>>>>>> This means that although the kernel modules are of the same ve= rsion, they cannot be loaded because the signature check fails. That might al= so explain why you are seeing so many ipset errors, because the kernel cannot= load that module any more. However, we use so much ipset now, why isn=E2=80= =99t the module loaded from before the update was started? >>>>>>>>>>>>>> The same goes for any network drivers. I assume you are using = virtio or a generic e1000 network adapter which will have been initialised at= boot time. The kernel should never unload the kernel module for that interfa= ce and load it again later. I have no idea what could have triggered that. >>>>>>>>>>>>> It is a generic e1000 network adapter that is being used by Vir= tualBox. Confirmed that on my working CU167 vm. >>>>>>>>>>>>>> No matter what though; after you reboot, the new kernel should= be booted being able to load all modules it wants and the system should run = absolutely fine. Can you confirm that that is at least the case? >>>>>>>>>>>>> No it is not the case. Yesterday when I booted the vm several t= imes the same error messages occurred each time and I always had no network i= nterfaces. >>>>>>>>>>>> Okay. That is indeed quite bad then. >>>>>>>>>>>> Since there is no kernel in 168, is there a chance that we broke= the update to 167? >>>>>>>>>>> I don't know but as far as I have been concerned CU167 has worked= well on my vm. >>>>>>>>>>> Is there something I should look for on my CU167 vm? >>>>>>>>>>>>> Just now I tried booting my third attempt vm again and this tim= e it ran okay and I ended up with all my networks assigned and I had network = connection. An IP was assigned by dhcp to the red interface. >>>>>>>>>>>> Third time lucky isn=E2=80=99t good enough for me :) >>>>>>>>>>> Me neither :-) >>>>>>>>>>>>> I then rebooted again and this time the error messages were bac= k. I rebooted the vm three more times and each time the error messages were t= here and no network interfaces. >>>>>>>>>>>>> >>>>>>>>>>>>> I checked with lsmod and the e1000 driver module was not loaded= . Confirmed the e1000 driver directory was not present in /sys/bus/pci/drivers >>>>>>>>>>>> Have there been any modules loaded? If one loads, they all shoul= d load. >>>>>>>>>>> This is what I get with lsmod >>>>>>>>>>> >>>>>>>>>>> Module Size Used by >>>>>>>>>>> crct10dif_pclmul 16384 1 >>>>>>>>>>> crc32_pclmul 16384 0 >>>>>>>>>>> ghash_clmulni_intel 16384 0 >>>>>>>>>>> serio_raw 20480 0 >>>>>>>>>>> ohci_pci 20480 0 >>>>>>>>>>> ata_generic 16384 0 >>>>>>>>>>> pata_acpi 16384 0 >>>>>>>>>>> video 57344 0 >>>>>>>>>> >>>>>>>>>> It looks like these modules could have come from the initramdisk i= mage. At least that is working well. >>>>>>>>>> >>>>>>>>>>> On my running CU1678 vm lsmod gives the following >>>>>>>>>>> >>>>>>>>>>> Module Size Used by >>>>>>>>>>> tun 61440 2 >>>>>>>>>>> nfnetlink_queue 28672 1 >>>>>>>>>>> xt_NFQUEUE 16384 8 >>>>>>>>>>> xt_MASQUERADE 20480 1 >>>>>>>>>>> cfg80211 1036288 0 >>>>>>>>>>> rfkill 32768 1 cfg80211 >>>>>>>>>>> 8021q 40960 0 >>>>>>>>>>> garp 16384 1 8021q >>>>>>>>>>> xt_set 16384 260 >>>>>>>>>>> ip_set_hash_net 49152 258 >>>>>>>>>>> ip_set 57344 2 xt_set,ip_set_hash_net >>>>>>>>>>> xt_hashlimit 20480 2 >>>>>>>>>>> xt_multiport 20480 4 >>>>>>>>>>> xt_policy 16384 5 >>>>>>>>>>> xt_TCPMSS 16384 1 >>>>>>>>>>> xt_conntrack 16384 7 >>>>>>>>>>> xt_comment 16384 18 >>>>>>>>>>> ipt_REJECT 16384 1 >>>>>>>>>>> nf_reject_ipv4 16384 1 ipt_REJECT >>>>>>>>>>> xt_LOG 20480 26 >>>>>>>>>>> xt_limit 16384 25 >>>>>>>>>>> xt_mark 16384 27 >>>>>>>>>>> xt_connmark 16384 2 >>>>>>>>>>> nf_log_syslog 24576 26 >>>>>>>>>>> iptable_raw 16384 0 >>>>>>>>>>> iptable_mangle 16384 1 >>>>>>>>>>> iptable_filter 16384 1 >>>>>>>>>>> vfat 24576 1 >>>>>>>>>>> fat 90112 1 vfat >>>>>>>>>>> sch_cake 36864 4 >>>>>>>>>>> intel_powerclamp 20480 0 >>>>>>>>>>> psmouse 184320 0 >>>>>>>>>>> pcspkr 16384 0 >>>>>>>>>>> i2c_piix4 28672 0 >>>>>>>>>>> e1000 163840 0 >>>>>>>>>>> i2c_core 106496 2 psmouse,i2c_piix4 >>>>>>>>>>> crct10dif_pclmul 16384 1 >>>>>>>>>>> crc32_pclmul 16384 0 >>>>>>>>>>> ata_generic 16384 0 >>>>>>>>>>> pata_acpi 16384 0 >>>>>>>>>>> ghash_clmulni_intel 16384 0 >>>>>>>>>>> serio_raw 20480 0 >>>>>>>>>>> ohci_pci 20480 0 >>>>>>>>>>> video 57344 0 >>>>>>>>>> >>>>>>>>>> Yes, this looks more like it. >>>>>>>>>> >>>>>>>>>>> The last 8 entries are the same but a whole load of others are mi= ssing. >>>>>>>>>> >>>>>>>>>> Stefan gave me a call this morning telling me that he tried to rep= roduce this but he couldn=E2=80=99t. >>>>>>>>>> >>>>>>>>>> I am not sure what we can do here. We urgently need to fix this si= nce we might break people=E2=80=99s installations. But something seems to wor= k differently for Adolf than for everybody else. >>>>>>>>>> >>>>>>>>>> -Michael >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Adolf. >>>>>>>>>>>>> Tried modprobe e1000 but this came back with the following error >>>>>>>>>>>>> modprobe: ERROR: could not insert 'e1000': Key was rejected by = service >>>>>>>>>>>>> >>>>>>>>>>>>> So out of 7 or 8 reboots the vm booted with the network drivers= loaded once. >>>>>>>>>>>>> >>>>>>>>>>>>> Regards >>>>>>>>>>>>> Adolf >>>>>>>>>>>>>>> On 11 May 2022, at 20:08, Adolf Belka wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 11/05/2022 20:48, Jon Murphy wrote: >>>>>>>>>>>>>>>> I just did an update from CU 167 (stable) to CU 168 (testing= ) and I got the same build info: >>>>>>>>>>>>>>>> *IPFire 2.27 (x86_64) - Core Update 167 Development Build: m= aster/c22d834c* >>>>>>>>>>>>>>>> Shouldn=E2=80=99t this be Core Update 168? >>>>>>>>>>>>>>> That was what I thought but I couldn't precisely remember wha= t it was in the past with Testing Releases. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I then downloaded the Core 168 iso from the nightlies build o= f master/latest/x86_64 and did an install on the First attempt vm. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I got to the stage of entering the root and admin passwords. = Entered what I usually do for the vm's and after pressing OK on the admin pas= sword screen I got as message box saying "Problem setting IPFire 'admin' user= password". I pressed the OK button and got another message box saying "Initi= al setup was not entirely complete. You must ensure that Setup is properly fi= nished by running setup again at the shell.". Pressing the OK button on that = message box causes setup to restart but again after entering a valid admin pa= ssword twice I got the same "Problem setting IPFire 'admin' user password" me= ssage box. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I then tried installing CU168 from the same iso onto a clone = of my running CU167 vm. Same result with "Problem setting IPFire 'admin' user= password" message box. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Then I created a completely new vm from scratch and tried ins= talling CU168 Testing iso and again got "Problem setting IPFire 'admin' user = password" message box. >>>>>>>>>>>>>> That seems to be a bug that should not be there. Did we recent= ly update apache? >>>>>>>>>>>>>> -Michael >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Don't know if this is related to the problem with the missing= interfaces but if not then it is another problem. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I think I am going to raise a bug on this. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>> Adolf. >>>>>>>>>>>>>>>> Jon >>>>>>>>>>>>>>>>> On May 11, 2022, at 9:19 AM, Adolf Belka > wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 11/05/2022 16:00, Adolf Belka wrote: >>>>>>>>>>>>>>>>>> Hi Leo, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 11/05/2022 15:26, Leo Hofmann wrote: >>>>>>>>>>>>>>>>>>> Hi Adolf, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Pakfire always automatically reinstalls the current relea= se before updating if you are in the testing branch. See this function in the= Pakfire code: >>>>>>>>>>>>>>>>>> So that proves that I haven't been looking so closely on m= y upgrades in the past because I have never noticed that. >>>>>>>>>>>>>>>>>>> https://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dblob;f=3Ds= rc/pakfire/lib/functions.pl;h=3Dd4e338f23ae8ae97d6f18c6d8890d13463dc5d30;hb= =3Drefs/heads/next#l762 >>>>>>>>>>>>>>>>>> Now I see it in the code. Thanks very much. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> But unfortunately this is all I can contribute, my test s= ystem updated to 168 without problems! >>>>>>>>>>>>>>>>>> Thanks for your help anyway. It's got rid of one of my que= stions. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Just need to understand now why 2 out of 3 updates resulte= d in a complete loss of the interface assignments. >>>>>>>>>>>>>>>>>> Will wait to hear other inputs on this problem. >>>>>>>>>>>>>>>>>> Maybe will try running setup on the first try I had to con= firm that I can re-assign the interfaces again. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Running setup does not help. None of the interfaces are ava= ilable to be assigned to the red, green etc zones. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I checked the interfaces in the vm setup and they are as pr= eviously defined and previously working on all earlier CU's of my vm. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> When rebooting I did see a fast scrolling message that seem= ed to say something like Invalid Kernel Argument but that is as much as I was= able to see/ >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Adolf. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>> Adolf. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Best regards >>>>>>>>>>>>>>>>>>> Leo >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Am 11.05.2022 um 15:11 schrieb Adolf Belka: >>>>>>>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I have tried to update my Core Update 167 vm machine thr= ee times now (using a clone) and have had several problems so I thought I wou= ld outline what has happened for further thoughts. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Normally my updates go without any real problems. This i= s the first time where this is not the case. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> First try:- >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> After running the update, which went quite quickly, the = bottom of the screen had >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Core Update 167 Development Build: master/c22d834c >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I couldn't remember if this is what is expected or not f= or Testing releases. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Rebooted the vm and on booting a large number of error m= essages scrolled across the console screen. My Virtualbox terminal for IPFire= has no scroll capability so I can only write on things I saw as they flew pa= st or what was on the screen when it stopped. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Lots of ipset restore error but went by too fast to get = more detail. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Tried to access via the WUI and no access. Tried via SSH= and no access. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Checked the red0 directory and there were no files prese= nt. Tried restarting red0 and got the following message >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> starting >>>>>>>>>>>>>>>>>>>> DUID 00:04:70:........ >>>>>>>>>>>>>>>>>>>> red0: interface not found >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> So I ran ip address show and it came up with only the lo= interface. No red0, green0, blue0 or orange0. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Looked in bootlog and this was filled with around 4000 l= ines all the same >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Loading of module with unavailable key is rejected >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Second try:- >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> This time running the update took over 10 mins. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Noticed on the pakfire screen that it first ran a Core U= pdate 167 upgrade then it ran the Core Update 168. My IPFire vm was definitel= y on Core Update 167. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> /opt/pakfire/db/core/mine contained 167 beforehand. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Before doing the reboot I accessed via ssh and saw that = in the /var/log/pakfire/ directory the update-core-upgrade-167 file has been = updated as well as there being a 168. Both files had the same last modified d= ate/time. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Rebooted. This time everything came back up okay but the= bottom of the screen now just said Core Update 167 and there was no update-c= ore-upgrade-168 file in the pakfire directory and the 167 version was back to= its date of Apr 28 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Repository was set back to Stable. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Changed again to Testing and it said again that there wa= s an update from 167 to 168. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Before running the pakfire directory had the following >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 1.8K Mar 31 23:26 update-core-upg= rade-166.log >>>>>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 323K Apr 28 09:47 update-core-upg= rade-167.log >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> After running it had >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 1.8K Mar 31 23:26 update-core-upg= rade-166.log >>>>>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 604K May 11 13:24 update-core-upg= rade-167.log >>>>>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 175 May 11 13:24 update-core-upgr= ade-168.log >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> back to Core Update 167 Development Build: master/c22d83= 4c at bottom of screen. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Rebooted. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> All interfaces present. Pakfire under System Status: say= s Core-Update-Level: 168 and the bottom of the screen showed Core Update 167 = Development Build: master/c22d834c >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> /opt/pakfire/db/core/mine has 168 in it >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I checked the WIO wio.pl file that had a bug fix and it = was as expected for CU168. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Third try:- >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 323K Apr 28 09:47 update-core-upg= rade-167.log >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Selected Testing and ran update from 167 to 168 which oc= curred very quickly. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 604K May 11 14:00 update-core-upg= rade-167.log >>>>>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 63K May 11 14:01 update-core-upgr= ade-168.log >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Here is the log file info for pakfire from messages.1.gz= and messages which shows it downloading and upgrading CU167 first then doing= CU168 as far as I can tell. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: PAKFIRE INFO: IPFire Pak= fire 2.27.1-x86_64 started! >>>>>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: CORE INFO: core-list.db = is 27 seconds old. - DEBUG: noforce >>>>>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: CORE UPGR: Upgrading fro= m release 166 to 168 >>>>>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: DOWNLOAD STARTED: paks/c= ore-upgrade-2.27-167.ipfire >>>>>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: MIRROR INFO: 2 servers f= ound in list >>>>>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: DOWNLOAD INFO: Host: ipf= ire.earl-net.com (HTTPS) - File: pakfire2/2.27.= 1-x86_64/paks/core-upgrade-2.27-167.ipfire >>>>>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: DOWNLOAD INFO: pakfire2/= 2.27.1-x86_64/paks/core-upgrade-2.27-167.ipfire has size of 71914825 bytes >>>>>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD INFO: HTTP-Stat= us-Code: 200 - 200 OK >>>>>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD INFO: File rece= ived. Start checking signature... >>>>>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD INFO: Signature= of core-upgrade-2.27-167.ipfire is fine. >>>>>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD FINISHED: pakfi= re2/2.27.1-x86_64/paks/core-upgrade-2.27-167.ipfire >>>>>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD STARTED: meta/m= eta-core-upgrade-168 >>>>>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: MIRROR INFO: 2 servers f= ound in list >>>>>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD INFO: Host: mir= ror1.ipfire.org (HTTPS) - File: pakfire2/2.27.1-= x86_64/meta/meta-core-upgrade-168 >>>>>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD INFO: pakfire2/= 2.27.1-x86_64/meta/meta-core-upgrade-168 has size of 1804 bytes >>>>>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD INFO: HTTP-Stat= us-Code: 200 - 200 OK >>>>>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD INFO: File rece= ived. Start checking signature... >>>>>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD INFO: Signature= of meta-core-upgrade-168 is fine. >>>>>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD FINISHED: pakfi= re2/2.27.1-x86_64/meta/meta-core-upgrade-168 >>>>>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD STARTED: paks/c= ore-upgrade-2.27-168.ipfire >>>>>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: MIRROR INFO: 2 servers f= ound in list >>>>>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD INFO: Host: ipf= ire.earl-net.com (HTTPS) - File: pakfire2/2.27.= 1-x86_64/paks/core-upgrade-2.27-168.ipfire >>>>>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD INFO: pakfire2/= 2.27.1-x86_64/paks/core-upgrade-2.27-168.ipfire has size of 34355484 bytes >>>>>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: DOWNLOAD INFO: HTTP-Stat= us-Code: 200 - 200 OK >>>>>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: DOWNLOAD INFO: File rece= ived. Start checking signature... >>>>>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: DOWNLOAD INFO: Signature= of core-upgrade-2.27-168.ipfire is fine. >>>>>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: DOWNLOAD FINISHED: pakfi= re2/2.27.1-x86_64/paks/core-upgrade-2.27-168.ipfire >>>>>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: PAKFIRE UPGR: core-upgra= de-167: Decrypting... >>>>>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: DECRYPT STARTED: core-up= grade-167 >>>>>>>>>>>>>>>>>>>> May 11 13:59:35 ipfire pakfire: DECRYPT FINISHED: core-u= pgrade-167 - Status: 0 >>>>>>>>>>>>>>>>>>>> May 11 13:59:35 ipfire pakfire: PAKFIRE UPGR: core-upgra= de-167: Upgrading files and running post-upgrading scripts... >>>>>>>>>>>>>>>>>>>> May 11 14:00:31 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>>>>> May 11 14:00:31 ipfire pakfire: PAKFIRE UPGR: core-upgra= de-167: Finished. >>>>>>>>>>>>>>>>>>>> May 11 14:00:31 ipfire pakfire: PAKFIRE UPGR: core-upgra= de-168: Decrypting... >>>>>>>>>>>>>>>>>>>> May 11 14:00:31 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>>>>> May 11 14:00:31 ipfire pakfire: DECRYPT STARTED: core-up= grade-168 >>>>>>>>>>>>>>>>>>>> May 11 14:00:32 ipfire pakfire: DECRYPT FINISHED: core-u= pgrade-168 - Status: 0 >>>>>>>>>>>>>>>>>>>> May 11 14:00:32 ipfire pakfire: PAKFIRE UPGR: core-upgra= de-168: Upgrading files and running post-upgrading scripts... >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: PAKFIRE UPGR: core-upgra= de-168: Finished. >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: CORE INFO: core-list.db = is 127 seconds old. - DEBUG: noforce >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: PAKFIRE RESV: wio: Resol= ving dependencies... >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: PAKFIRE UPGR: We are goi= ng to install all packages listed above. >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD STARTED: paks/w= io-1.3.2-15.ipfire >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: MIRROR INFO: 2 servers f= ound in list >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD INFO: Host: mir= ror1.ipfire.org (HTTPS) - File: pakfire2/2.27.1-= x86_64/paks/wio-1.3.2-15.ipfire >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD INFO: pakfire2/= 2.27.1-x86_64/paks/wio-1.3.2-15.ipfire has size of 47216 bytes >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD INFO: HTTP-Stat= us-Code: 200 - 200 OK >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD INFO: File rece= ived. Start checking signature... >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD INFO: Signature= of wio-1.3.2-15.ipfire is fine. >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD FINISHED: pakfi= re2/2.27.1-x86_64/paks/wio-1.3.2-15.ipfire >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: PAKFIRE UPGR: wio: Decry= pting... >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DECRYPT STARTED: wio >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DECRYPT FINISHED: wio - = Status: 0 >>>>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: PAKFIRE UPGR: wio: Upgra= ding files and running post-upgrading scripts... >>>>>>>>>>>>>>>>>>>> May 11 14:01:11 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>>>>> May 11 14:01:11 ipfire pakfire: PAKFIRE UPGR: wio: Finis= hed. >>>>>>>>>>>>>>>>>>>> May 11 14:01:11 ipfire pakfire: PAKFIRE INFO: Pakfire ha= s finished. Closing. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Rebooted >>>>>>>>>>>>>>>>>>>> Lots of ipset restore error but went by too fast to get = more detail. >>>>>>>>>>>>>>>>>>>> Several repeats of the following two lines on the consol= e screen:- >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> iptables v1.8.7 (legacy): can't initialize iptables tabl= e `filter' : Table does not exist (do you need to insmod?) >>>>>>>>>>>>>>>>>>>> Perhaps iptables or your kernel needs to be upgraded. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Only lo interface present. All other interfaces red0, gr= een0, blue0, orange0 not present. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I have given up at this point. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I have kept the clones of all three attempts so if you n= eed any additional info from logs or files just let me know. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> If this is something obviously wrong that I am doing, pl= ease let me know. If not then I will probably raise a bug on this. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Adolf. >>>>>>>>>> >=20 --===============4331578002912941262==--