From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer To: development@lists.ipfire.org Subject: Re: Feedback on problems with Core Update 168 Testing Date: Thu, 19 May 2022 12:14:35 +0100 Message-ID: <4AE588DC-F3C0-4551-B46D-40B79C007407@ipfire.org> In-Reply-To: <32a80816-9e7c-b548-4012-8aec7c3a02ca@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4089365680640487074==" List-Id: --===============4089365680640487074== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello, > On 19 May 2022, at 11:57, Adolf Belka wrote: >=20 > Hi Michael, >=20 > On 19/05/2022 10:59, Michael Tremer wrote: >> Please note the patch that I just posted to fix any broken RAID arrays. >> Does anyone have any systems that we can use to test this? > Does the script need to be run manually or has it been added to the nightli= es together with the rd.auto patch so that I can do a CU168 Testing update ev= aluation? My patch includes that it will run automatically with the updater. However, t= hat is not part of the build yet, because I would like it to be confirmed tha= t I do not destroy anything. The whole script is a rather risky operation :) >=20 > Regards, > Adolf. >=20 >> -Michael >>> On 17 May 2022, at 09:09, Adolf Belka wrote: >>>=20 >>> Hi Michael, >>>=20 >>> On 13/05/2022 22:37, Michael Tremer wrote: >>>> Hello, >>>> Okay, this is really bad then if we are breaking RAID installations. >>>> I tried to debug this, but VirtualBox is not my friend today. >>>> I asked some bug reporter to provide me with the logs that I want over h= ere: >>>> https://bugzilla.ipfire.org/show_bug.cgi?id=3D12862 >>> After applying this fix to my CU167 vm and proving that it stayed consist= ent I have now done an upgrade to CU168 Testing. This does a CU167 update bef= ore doing the CU168 update and I am glad to announce that my previous problem= s with losing all my network interfaces etc are gone. >>>=20 >>> My Core Update 168 Development Build: master/bbd4767f successfully restar= ted with a fully active raid array and I will now restart doing my CU168 Test= ing evaluation. >>>=20 >>> Thanks very much for all the help. >>>=20 >>> Regards, >>> Adolf. >>>> -Michael >>>>> On 13 May 2022, at 18:12, Adolf Belka wrote: >>>>>=20 >>>>> Hi All, >>>>>=20 >>>>> I think I may understand what is happening with my vm now. Of course I = could also be wrong :-) but I will give it a go. >>>>>=20 >>>>> I saw bug#12862 where upgrading from CU166 to CU167 had stopped the rai= d array running so that IPFire was only on one disk. >>>>>=20 >>>>> I had missed seeing this in my evaluation of CU167 on my vm which does = run with a raid array. >>>>>=20 >>>>> The vm shows the following >>>>>=20 >>>>> -bash-5.1$ cat /proc/mdstat >>>>> Personalities : >>>>> md127 : inactive sdb[1](S) >>>>> 52428724 blocks super 1.0 >>>>>=20 >>>>> unused devices: >>>>>=20 >>>>>=20 >>>>> It looks like IPFire was only running on sda. >>>>>=20 >>>>> So when I am doing an upgrade to CU168 it is likely that only one disk = of the raid pair is being updated. >>>>>=20 >>>>> The CU168 vm that is working has the same as the above setting. >>>>>=20 >>>>> The CU168 vm that is not working has the following setting >>>>>=20 >>>>> -bash-5.1$ cat /proc/mdstat >>>>> Personalities : >>>>> md127 : inactive sda[0](S) >>>>> 52428724 blocks super 1.0 >>>>>=20 >>>>> unused devices: >>>>>=20 >>>>> So it looks like it has booted up on the disk that was not updated from= the raid pair. >>>>>=20 >>>>>=20 >>>>> This probably also explains why occasionally I have had a vm that was n= ot working suddenly start working properly when I reboot it, but booting agai= n can put it back into non-working mode. >>>>>=20 >>>>> It is probably dependent on which of the two disks actually get used to= boot from. >>>>>=20 >>>>> So the problem looks like it would only affect people who have raid set= ups and the problem originated in the CU167 upgrade. >>>>>=20 >>>>> So raid systems need to have their raid arrays fully working before doi= ng an upgrade. >>>>>=20 >>>>> At least the above is my interpretation of what I have found. What does= everyone else think. >>>>>=20 >>>>>=20 >>>>> Regards, >>>>>=20 >>>>> Adolf. >>>>>=20 >>>>>=20 >>>>> On 13/05/2022 15:59, Adolf Belka wrote: >>>>>> Hi Michael, >>>>>>=20 >>>>>> I have found some differences between the vm's that have upgraded with= no problem and those that have lost most of the modules. >>>>>>=20 >>>>>> I did another three vm clones and upgrades and the first two all went = with no problems. I was starting to think the issue had gone away but on the = third one the same problem happened after the reboot. >>>>>>=20 >>>>>> However I did copy the boot directory info for these two cases and fou= nd an interesting effect. >>>>>>=20 >>>>>> For the problem vm here was the boot directory layout of the CU167 bas= e machine >>>>>>=20 >>>>>> drwxr-xr-x 5 root root 4.0K Apr 28 09:47 . >>>>>> drwxr-xr-x 21 root root 4.0K Nov 28 15:49 .. >>>>>> -rw-r--r-- 1 root root 179K Apr 26 13:00 config-5.15.35-ipfire >>>>>> drwxr-xr-x 3 root root 16K Jan 1 1970 efi >>>>>> drwxr-xr-x 6 root root 4.0K Apr 28 09:47 grub >>>>>> -rw------- 1 root root 26M Apr 28 09:47 initramfs-5.15.35-ipfire.img >>>>>> drwx------ 2 root root 16K Sep 30 2021 lost+found >>>>>> -rw-r--r-- 1 root root 4.7M Apr 26 13:00 System.map-5.15.35-ipfire >>>>>> -rw-r--r-- 1 root root 6.7M Apr 26 13:00 vmlinuz-5.15.35-ipfire >>>>>>=20 >>>>>> After upgrade and before reboot this had changed to >>>>>>=20 >>>>>> drwxr-xr-x 5 root root 4.0K May 13 15:40 . >>>>>> drwxr-xr-x 21 root root 4.0K Nov 28 15:49 .. >>>>>> -rw-r--r-- 1 root root 179K May 3 04:02 config-5.15.35-ipfire >>>>>> drwxr-xr-x 3 root root 16K Jan 1 1970 efi >>>>>> drwxr-xr-x 6 root root 4.0K May 13 15:41 grub >>>>>> -rw------- 1 root root 26M May 13 15:41 initramfs-5.15.35-ipfire.img >>>>>> drwx------ 2 root root 16K Sep 30 2021 lost+found >>>>>> -rw-r--r-- 1 root root 4.7M May 3 04:02 System.map-5.15.35-ipfire >>>>>> -rw-r--r-- 1 root root 6.7M May 3 04:02 vmlinuz-5.15.35-ipfire >>>>>>=20 >>>>>> Dates and times of some of the files/directories have changed as would= be expected. >>>>>>=20 >>>>>> After Reboot and with the problem the boot directory was now >>>>>>=20 >>>>>> drwxr-xr-x 5 root root 4.0K Apr 28 09:47 . >>>>>> drwxr-xr-x 21 root root 4.0K Nov 28 15:49 .. >>>>>> -rw-r--r-- 1 root root 179K Apr 26 13:00 config-5.15.35-ipfire >>>>>> drwxr-xr-x 3 root root 4.0K Sep 30 2021 efi >>>>>> drwxr-xr-x 6 root root 4.0K Apr 28 09:47 grub >>>>>> -rw------- 1 root root 26M Apr 28 09:47 initramfs-5.15.35-ipfire.img >>>>>> drwx------ 2 root root 16K Sep 30 2021 lost+found >>>>>> -rw-r--r-- 1 root root 4.7M Apr 26 13:00 System.map-5.15.35-ipfire >>>>>> -rw-r--r-- 1 root root 6.7M Apr 26 13:00 vmlinuz-5.15.35-ipfire >>>>>>=20 >>>>>> Which is the same as for the original CU167 machine except that the da= te/time of the efi directory has changed >>>>>>=20 >>>>>> For the vm clone that upgraded with no problems the boot directory aft= er reboot was the same as after upgrade. >>>>>>=20 >>>>>> I don';t understand why or how this has happened but it is certainly a= difference between the vm's having the problem and the ones not. >>>>>>=20 >>>>>> Regards, >>>>>>=20 >>>>>> Adolf. >>>>>>=20 >>>>>> On 13/05/2022 12:09, Michael Tremer wrote: >>>>>>> Hello, >>>>>>>=20 >>>>>>>> On 12 May 2022, at 21:10, Adolf Belka wro= te: >>>>>>>>=20 >>>>>>>> Hi Michael, >>>>>>>>=20 >>>>>>>> On 12/05/2022 14:53, Michael Tremer wrote: >>>>>>>>> Hello, >>>>>>>>>> On 12 May 2022, at 12:25, Adolf Belka w= rote: >>>>>>>>>>=20 >>>>>>>>>> Hi, >>>>>>>>>>=20 >>>>>>>>>> On 12/05/2022 11:13, Michael Tremer wrote: >>>>>>>>>>> Hello, >>>>>>>>>>> Thanks for spending so much time on this. We definitely need to i= mprove the general update experience since we sometimes seem to break people= =E2=80=99s systems and it is not nice to re-install a firewall from scratch. = It will take a while. >>>>>>>>>>> So what I can say is that the kernel module issues come from when= the running kernel is changed and the kernel is trying to load any modules t= hat now have changed. This fails by design, because we sign our kernel module= s. The key is randomly generated at build time and used to sign all modules a= nd it then thrown away. For each build, we are using a different, unique key = that is not preserved. >>>>>>>>>>> This means that although the kernel modules are of the same versi= on, they cannot be loaded because the signature check fails. That might also = explain why you are seeing so many ipset errors, because the kernel cannot lo= ad that module any more. However, we use so much ipset now, why isn=E2=80=99t= the module loaded from before the update was started? >>>>>>>>>>> The same goes for any network drivers. I assume you are using vir= tio or a generic e1000 network adapter which will have been initialised at bo= ot time. The kernel should never unload the kernel module for that interface = and load it again later. I have no idea what could have triggered that. >>>>>>>>>> It is a generic e1000 network adapter that is being used by Virtua= lBox. Confirmed that on my working CU167 vm. >>>>>>>>>>> No matter what though; after you reboot, the new kernel should be= booted being able to load all modules it wants and the system should run abs= olutely fine. Can you confirm that that is at least the case? >>>>>>>>>> No it is not the case. Yesterday when I booted the vm several time= s the same error messages occurred each time and I always had no network inte= rfaces. >>>>>>>>> Okay. That is indeed quite bad then. >>>>>>>>> Since there is no kernel in 168, is there a chance that we broke th= e update to 167? >>>>>>>> I don't know but as far as I have been concerned CU167 has worked we= ll on my vm. >>>>>>>> Is there something I should look for on my CU167 vm? >>>>>>>>>> Just now I tried booting my third attempt vm again and this time i= t ran okay and I ended up with all my networks assigned and I had network con= nection. An IP was assigned by dhcp to the red interface. >>>>>>>>> Third time lucky isn=E2=80=99t good enough for me :) >>>>>>>> Me neither :-) >>>>>>>>>> I then rebooted again and this time the error messages were back. = I rebooted the vm three more times and each time the error messages were ther= e and no network interfaces. >>>>>>>>>>=20 >>>>>>>>>> I checked with lsmod and the e1000 driver module was not loaded. C= onfirmed the e1000 driver directory was not present in /sys/bus/pci/drivers >>>>>>>>> Have there been any modules loaded? If one loads, they all should l= oad. >>>>>>>> This is what I get with lsmod >>>>>>>>=20 >>>>>>>> Module Size Used by >>>>>>>> crct10dif_pclmul 16384 1 >>>>>>>> crc32_pclmul 16384 0 >>>>>>>> ghash_clmulni_intel 16384 0 >>>>>>>> serio_raw 20480 0 >>>>>>>> ohci_pci 20480 0 >>>>>>>> ata_generic 16384 0 >>>>>>>> pata_acpi 16384 0 >>>>>>>> video 57344 0 >>>>>>>=20 >>>>>>> It looks like these modules could have come from the initramdisk imag= e. At least that is working well. >>>>>>>=20 >>>>>>>> On my running CU1678 vm lsmod gives the following >>>>>>>>=20 >>>>>>>> Module Size Used by >>>>>>>> tun 61440 2 >>>>>>>> nfnetlink_queue 28672 1 >>>>>>>> xt_NFQUEUE 16384 8 >>>>>>>> xt_MASQUERADE 20480 1 >>>>>>>> cfg80211 1036288 0 >>>>>>>> rfkill 32768 1 cfg80211 >>>>>>>> 8021q 40960 0 >>>>>>>> garp 16384 1 8021q >>>>>>>> xt_set 16384 260 >>>>>>>> ip_set_hash_net 49152 258 >>>>>>>> ip_set 57344 2 xt_set,ip_set_hash_net >>>>>>>> xt_hashlimit 20480 2 >>>>>>>> xt_multiport 20480 4 >>>>>>>> xt_policy 16384 5 >>>>>>>> xt_TCPMSS 16384 1 >>>>>>>> xt_conntrack 16384 7 >>>>>>>> xt_comment 16384 18 >>>>>>>> ipt_REJECT 16384 1 >>>>>>>> nf_reject_ipv4 16384 1 ipt_REJECT >>>>>>>> xt_LOG 20480 26 >>>>>>>> xt_limit 16384 25 >>>>>>>> xt_mark 16384 27 >>>>>>>> xt_connmark 16384 2 >>>>>>>> nf_log_syslog 24576 26 >>>>>>>> iptable_raw 16384 0 >>>>>>>> iptable_mangle 16384 1 >>>>>>>> iptable_filter 16384 1 >>>>>>>> vfat 24576 1 >>>>>>>> fat 90112 1 vfat >>>>>>>> sch_cake 36864 4 >>>>>>>> intel_powerclamp 20480 0 >>>>>>>> psmouse 184320 0 >>>>>>>> pcspkr 16384 0 >>>>>>>> i2c_piix4 28672 0 >>>>>>>> e1000 163840 0 >>>>>>>> i2c_core 106496 2 psmouse,i2c_piix4 >>>>>>>> crct10dif_pclmul 16384 1 >>>>>>>> crc32_pclmul 16384 0 >>>>>>>> ata_generic 16384 0 >>>>>>>> pata_acpi 16384 0 >>>>>>>> ghash_clmulni_intel 16384 0 >>>>>>>> serio_raw 20480 0 >>>>>>>> ohci_pci 20480 0 >>>>>>>> video 57344 0 >>>>>>>=20 >>>>>>> Yes, this looks more like it. >>>>>>>=20 >>>>>>>> The last 8 entries are the same but a whole load of others are missi= ng. >>>>>>>=20 >>>>>>> Stefan gave me a call this morning telling me that he tried to reprod= uce this but he couldn=E2=80=99t. >>>>>>>=20 >>>>>>> I am not sure what we can do here. We urgently need to fix this since= we might break people=E2=80=99s installations. But something seems to work d= ifferently for Adolf than for everybody else. >>>>>>>=20 >>>>>>> -Michael >>>>>>>=20 >>>>>>>>=20 >>>>>>>> Regards, >>>>>>>> Adolf. >>>>>>>>>> Tried modprobe e1000 but this came back with the following error >>>>>>>>>> modprobe: ERROR: could not insert 'e1000': Key was rejected by ser= vice >>>>>>>>>>=20 >>>>>>>>>> So out of 7 or 8 reboots the vm booted with the network drivers lo= aded once. >>>>>>>>>>=20 >>>>>>>>>> Regards >>>>>>>>>> Adolf >>>>>>>>>>>> On 11 May 2022, at 20:08, Adolf Belka = wrote: >>>>>>>>>>>>=20 >>>>>>>>>>>> Hi All, >>>>>>>>>>>>=20 >>>>>>>>>>>> On 11/05/2022 20:48, Jon Murphy wrote: >>>>>>>>>>>>> I just did an update from CU 167 (stable) to CU 168 (testing) a= nd I got the same build info: >>>>>>>>>>>>> *IPFire 2.27 (x86_64) - Core Update 167 Development Build: mast= er/c22d834c* >>>>>>>>>>>>> Shouldn=E2=80=99t this be Core Update 168? >>>>>>>>>>>> That was what I thought but I couldn't precisely remember what i= t was in the past with Testing Releases. >>>>>>>>>>>>=20 >>>>>>>>>>>> I then downloaded the Core 168 iso from the nightlies build of m= aster/latest/x86_64 and did an install on the First attempt vm. >>>>>>>>>>>>=20 >>>>>>>>>>>> I got to the stage of entering the root and admin passwords. Ent= ered what I usually do for the vm's and after pressing OK on the admin passwo= rd screen I got as message box saying "Problem setting IPFire 'admin' user pa= ssword". I pressed the OK button and got another message box saying "Initial = setup was not entirely complete. You must ensure that Setup is properly finis= hed by running setup again at the shell.". Pressing the OK button on that mes= sage box causes setup to restart but again after entering a valid admin passw= ord twice I got the same "Problem setting IPFire 'admin' user password" messa= ge box. >>>>>>>>>>>>=20 >>>>>>>>>>>> I then tried installing CU168 from the same iso onto a clone of = my running CU167 vm. Same result with "Problem setting IPFire 'admin' user pa= ssword" message box. >>>>>>>>>>>>=20 >>>>>>>>>>>> Then I created a completely new vm from scratch and tried instal= ling CU168 Testing iso and again got "Problem setting IPFire 'admin' user pas= sword" message box. >>>>>>>>>>> That seems to be a bug that should not be there. Did we recently = update apache? >>>>>>>>>>> -Michael >>>>>>>>>>>>=20 >>>>>>>>>>>> Don't know if this is related to the problem with the missing in= terfaces but if not then it is another problem. >>>>>>>>>>>>=20 >>>>>>>>>>>> I think I am going to raise a bug on this. >>>>>>>>>>>>=20 >>>>>>>>>>>> Regards, >>>>>>>>>>>> Adolf. >>>>>>>>>>>>> Jon >>>>>>>>>>>>>> On May 11, 2022, at 9:19 AM, Adolf Belka > wrote: >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> On 11/05/2022 16:00, Adolf Belka wrote: >>>>>>>>>>>>>>> Hi Leo, >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>> On 11/05/2022 15:26, Leo Hofmann wrote: >>>>>>>>>>>>>>>> Hi Adolf, >>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>> Pakfire always automatically reinstalls the current release = before updating if you are in the testing branch. See this function in the Pa= kfire code: >>>>>>>>>>>>>>> So that proves that I haven't been looking so closely on my u= pgrades in the past because I have never noticed that. >>>>>>>>>>>>>>>> https://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dblob;f=3Dsrc/= pakfire/lib/functions.pl;h=3Dd4e338f23ae8ae97d6f18c6d8890d13463dc5d30;hb=3Dre= fs/heads/next#l762 >>>>>>>>>>>>>>> Now I see it in the code. Thanks very much. >>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>> But unfortunately this is all I can contribute, my test syst= em updated to 168 without problems! >>>>>>>>>>>>>>> Thanks for your help anyway. It's got rid of one of my questi= ons. >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>> Just need to understand now why 2 out of 3 updates resulted i= n a complete loss of the interface assignments. >>>>>>>>>>>>>>> Will wait to hear other inputs on this problem. >>>>>>>>>>>>>>> Maybe will try running setup on the first try I had to confir= m that I can re-assign the interfaces again. >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Running setup does not help. None of the interfaces are availa= ble to be assigned to the red, green etc zones. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> I checked the interfaces in the vm setup and they are as previ= ously defined and previously working on all earlier CU's of my vm. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> When rebooting I did see a fast scrolling message that seemed = to say something like Invalid Kernel Argument but that is as much as I was ab= le to see/ >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Adolf. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>> Adolf. >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>> Best regards >>>>>>>>>>>>>>>> Leo >>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>> Am 11.05.2022 um 15:11 schrieb Adolf Belka: >>>>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> I have tried to update my Core Update 167 vm machine three = times now (using a clone) and have had several problems so I thought I would = outline what has happened for further thoughts. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Normally my updates go without any real problems. This is t= he first time where this is not the case. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> First try:- >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> After running the update, which went quite quickly, the bot= tom of the screen had >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Core Update 167 Development Build: master/c22d834c >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> I couldn't remember if this is what is expected or not for = Testing releases. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Rebooted the vm and on booting a large number of error mess= ages scrolled across the console screen. My Virtualbox terminal for IPFire ha= s no scroll capability so I can only write on things I saw as they flew past = or what was on the screen when it stopped. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Lots of ipset restore error but went by too fast to get mor= e detail. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Tried to access via the WUI and no access. Tried via SSH an= d no access. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Checked the red0 directory and there were no files present.= Tried restarting red0 and got the following message >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> starting >>>>>>>>>>>>>>>>> DUID 00:04:70:........ >>>>>>>>>>>>>>>>> red0: interface not found >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> So I ran ip address show and it came up with only the lo in= terface. No red0, green0, blue0 or orange0. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Looked in bootlog and this was filled with around 4000 line= s all the same >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Loading of module with unavailable key is rejected >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Second try:- >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> This time running the update took over 10 mins. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Noticed on the pakfire screen that it first ran a Core Upda= te 167 upgrade then it ran the Core Update 168. My IPFire vm was definitely o= n Core Update 167. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> /opt/pakfire/db/core/mine contained 167 beforehand. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Before doing the reboot I accessed via ssh and saw that in = the /var/log/pakfire/ directory the update-core-upgrade-167 file has been upd= ated as well as there being a 168. Both files had the same last modified date= /time. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Rebooted. This time everything came back up okay but the bo= ttom of the screen now just said Core Update 167 and there was no update-core= -upgrade-168 file in the pakfire directory and the 167 version was back to it= s date of Apr 28 >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Repository was set back to Stable. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Changed again to Testing and it said again that there was a= n update from 167 to 168. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Before running the pakfire directory had the following >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 1.8K Mar 31 23:26 update-core-upgrad= e-166.log >>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 323K Apr 28 09:47 update-core-upgrad= e-167.log >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> After running it had >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 1.8K Mar 31 23:26 update-core-upgrad= e-166.log >>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 604K May 11 13:24 update-core-upgrad= e-167.log >>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 175 May 11 13:24 update-core-upgrade= -168.log >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> back to Core Update 167 Development Build: master/c22d834c = at bottom of screen. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Rebooted. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> All interfaces present. Pakfire under System Status: says C= ore-Update-Level: 168 and the bottom of the screen showed Core Update 167 Dev= elopment Build: master/c22d834c >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> /opt/pakfire/db/core/mine has 168 in it >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> I checked the WIO wio.pl file that had a bug fix and it was= as expected for CU168. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Third try:- >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 323K Apr 28 09:47 update-core-upgrad= e-167.log >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Selected Testing and ran update from 167 to 168 which occur= red very quickly. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 604K May 11 14:00 update-core-upgrad= e-167.log >>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 63K May 11 14:01 update-core-upgrade= -168.log >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Here is the log file info for pakfire from messages.1.gz an= d messages which shows it downloading and upgrading CU167 first then doing CU= 168 as far as I can tell. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: PAKFIRE INFO: IPFire Pakfir= e 2.27.1-x86_64 started! >>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: CORE INFO: core-list.db is = 27 seconds old. - DEBUG: noforce >>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: CORE UPGR: Upgrading from r= elease 166 to 168 >>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: DOWNLOAD STARTED: paks/core= -upgrade-2.27-167.ipfire >>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: MIRROR INFO: 2 servers foun= d in list >>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: DOWNLOAD INFO: Host: ipfire= .earl-net.com (HTTPS) - File: pakfire2/2.27.1-x= 86_64/paks/core-upgrade-2.27-167.ipfire >>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: DOWNLOAD INFO: pakfire2/2.2= 7.1-x86_64/paks/core-upgrade-2.27-167.ipfire has size of 71914825 bytes >>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD INFO: HTTP-Status-= Code: 200 - 200 OK >>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD INFO: File receive= d. Start checking signature... >>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD INFO: Signature of= core-upgrade-2.27-167.ipfire is fine. >>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD FINISHED: pakfire2= /2.27.1-x86_64/paks/core-upgrade-2.27-167.ipfire >>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD STARTED: meta/meta= -core-upgrade-168 >>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: MIRROR INFO: 2 servers foun= d in list >>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD INFO: Host: mirror= 1.ipfire.org (HTTPS) - File: pakfire2/2.27.1-x86= _64/meta/meta-core-upgrade-168 >>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD INFO: pakfire2/2.2= 7.1-x86_64/meta/meta-core-upgrade-168 has size of 1804 bytes >>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD INFO: HTTP-Status-= Code: 200 - 200 OK >>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD INFO: File receive= d. Start checking signature... >>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD INFO: Signature of= meta-core-upgrade-168 is fine. >>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD FINISHED: pakfire2= /2.27.1-x86_64/meta/meta-core-upgrade-168 >>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD STARTED: paks/core= -upgrade-2.27-168.ipfire >>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: MIRROR INFO: 2 servers foun= d in list >>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD INFO: Host: ipfire= .earl-net.com (HTTPS) - File: pakfire2/2.27.1-x= 86_64/paks/core-upgrade-2.27-168.ipfire >>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD INFO: pakfire2/2.2= 7.1-x86_64/paks/core-upgrade-2.27-168.ipfire has size of 34355484 bytes >>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: DOWNLOAD INFO: HTTP-Status-= Code: 200 - 200 OK >>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: DOWNLOAD INFO: File receive= d. Start checking signature... >>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: DOWNLOAD INFO: Signature of= core-upgrade-2.27-168.ipfire is fine. >>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: DOWNLOAD FINISHED: pakfire2= /2.27.1-x86_64/paks/core-upgrade-2.27-168.ipfire >>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: PAKFIRE UPGR: core-upgrade-= 167: Decrypting... >>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: DECRYPT STARTED: core-upgra= de-167 >>>>>>>>>>>>>>>>> May 11 13:59:35 ipfire pakfire: DECRYPT FINISHED: core-upgr= ade-167 - Status: 0 >>>>>>>>>>>>>>>>> May 11 13:59:35 ipfire pakfire: PAKFIRE UPGR: core-upgrade-= 167: Upgrading files and running post-upgrading scripts... >>>>>>>>>>>>>>>>> May 11 14:00:31 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>> May 11 14:00:31 ipfire pakfire: PAKFIRE UPGR: core-upgrade-= 167: Finished. >>>>>>>>>>>>>>>>> May 11 14:00:31 ipfire pakfire: PAKFIRE UPGR: core-upgrade-= 168: Decrypting... >>>>>>>>>>>>>>>>> May 11 14:00:31 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>> May 11 14:00:31 ipfire pakfire: DECRYPT STARTED: core-upgra= de-168 >>>>>>>>>>>>>>>>> May 11 14:00:32 ipfire pakfire: DECRYPT FINISHED: core-upgr= ade-168 - Status: 0 >>>>>>>>>>>>>>>>> May 11 14:00:32 ipfire pakfire: PAKFIRE UPGR: core-upgrade-= 168: Upgrading files and running post-upgrading scripts... >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: PAKFIRE UPGR: core-upgrade-= 168: Finished. >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: CORE INFO: core-list.db is = 127 seconds old. - DEBUG: noforce >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: PAKFIRE RESV: wio: Resolvin= g dependencies... >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: PAKFIRE UPGR: We are going = to install all packages listed above. >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD STARTED: paks/wio-= 1.3.2-15.ipfire >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: MIRROR INFO: 2 servers foun= d in list >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD INFO: Host: mirror= 1.ipfire.org (HTTPS) - File: pakfire2/2.27.1-x86= _64/paks/wio-1.3.2-15.ipfire >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD INFO: pakfire2/2.2= 7.1-x86_64/paks/wio-1.3.2-15.ipfire has size of 47216 bytes >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD INFO: HTTP-Status-= Code: 200 - 200 OK >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD INFO: File receive= d. Start checking signature... >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD INFO: Signature of= wio-1.3.2-15.ipfire is fine. >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD FINISHED: pakfire2= /2.27.1-x86_64/paks/wio-1.3.2-15.ipfire >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: PAKFIRE UPGR: wio: Decrypti= ng... >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DECRYPT STARTED: wio >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DECRYPT FINISHED: wio - Sta= tus: 0 >>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: PAKFIRE UPGR: wio: Upgradin= g files and running post-upgrading scripts... >>>>>>>>>>>>>>>>> May 11 14:01:11 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>> May 11 14:01:11 ipfire pakfire: PAKFIRE UPGR: wio: Finished. >>>>>>>>>>>>>>>>> May 11 14:01:11 ipfire pakfire: PAKFIRE INFO: Pakfire has f= inished. Closing. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Rebooted >>>>>>>>>>>>>>>>> Lots of ipset restore error but went by too fast to get mor= e detail. >>>>>>>>>>>>>>>>> Several repeats of the following two lines on the console s= creen:- >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> iptables v1.8.7 (legacy): can't initialize iptables table `= filter' : Table does not exist (do you need to insmod?) >>>>>>>>>>>>>>>>> Perhaps iptables or your kernel needs to be upgraded. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Only lo interface present. All other interfaces red0, green= 0, blue0, orange0 not present. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> I have given up at this point. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> I have kept the clones of all three attempts so if you need= any additional info from logs or files just let me know. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> If this is something obviously wrong that I am doing, pleas= e let me know. If not then I will probably raise a bug on this. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Adolf. >>>>>>>=20 --===============4089365680640487074==--