From mboxrd@z Thu Jan 1 00:00:00 1970 From: Adolf Belka To: development@lists.ipfire.org Subject: Re: Feedback on problems with Core Update 168 Testing Date: Thu, 19 May 2022 13:21:09 +0200 Message-ID: In-Reply-To: <4AE588DC-F3C0-4551-B46D-40B79C007407@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7940496142477443562==" List-Id: --===============7940496142477443562== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi, On 19/05/2022 13:14, Michael Tremer wrote: > Hello, >=20 >> On 19 May 2022, at 11:57, Adolf Belka wrote: >> >> Hi Michael, >> >> On 19/05/2022 10:59, Michael Tremer wrote: >>> Please note the patch that I just posted to fix any broken RAID arrays. >>> Does anyone have any systems that we can use to test this? >> Does the script need to be run manually or has it been added to the nightl= ies together with the rd.auto patch so that I can do a CU168 Testing update e= valuation? >=20 > My patch includes that it will run automatically with the updater. However,= that is not part of the build yet, because I would like it to be confirmed t= hat I do not destroy anything. The whole script is a rather risky operation :) Okay, clear. Then I will execute the rd.auto change reboot and then run the script. Will feedback how things go. Adolf. >=20 >> >> Regards, >> Adolf. >> >>> -Michael >>>> On 17 May 2022, at 09:09, Adolf Belka wrote: >>>> >>>> Hi Michael, >>>> >>>> On 13/05/2022 22:37, Michael Tremer wrote: >>>>> Hello, >>>>> Okay, this is really bad then if we are breaking RAID installations. >>>>> I tried to debug this, but VirtualBox is not my friend today. >>>>> I asked some bug reporter to provide me with the logs that I want over = here: >>>>> https://bugzilla.ipfire.org/show_bug.cgi?id=3D12862 >>>> After applying this fix to my CU167 vm and proving that it stayed consis= tent I have now done an upgrade to CU168 Testing. This does a CU167 update be= fore doing the CU168 update and I am glad to announce that my previous proble= ms with losing all my network interfaces etc are gone. >>>> >>>> My Core Update 168 Development Build: master/bbd4767f successfully resta= rted with a fully active raid array and I will now restart doing my CU168 Tes= ting evaluation. >>>> >>>> Thanks very much for all the help. >>>> >>>> Regards, >>>> Adolf. >>>>> -Michael >>>>>> On 13 May 2022, at 18:12, Adolf Belka wrote: >>>>>> >>>>>> Hi All, >>>>>> >>>>>> I think I may understand what is happening with my vm now. Of course I= could also be wrong :-) but I will give it a go. >>>>>> >>>>>> I saw bug#12862 where upgrading from CU166 to CU167 had stopped the ra= id array running so that IPFire was only on one disk. >>>>>> >>>>>> I had missed seeing this in my evaluation of CU167 on my vm which does= run with a raid array. >>>>>> >>>>>> The vm shows the following >>>>>> >>>>>> -bash-5.1$ cat /proc/mdstat >>>>>> Personalities : >>>>>> md127 : inactive sdb[1](S) >>>>>> 52428724 blocks super 1.0 >>>>>> >>>>>> unused devices: >>>>>> >>>>>> >>>>>> It looks like IPFire was only running on sda. >>>>>> >>>>>> So when I am doing an upgrade to CU168 it is likely that only one disk= of the raid pair is being updated. >>>>>> >>>>>> The CU168 vm that is working has the same as the above setting. >>>>>> >>>>>> The CU168 vm that is not working has the following setting >>>>>> >>>>>> -bash-5.1$ cat /proc/mdstat >>>>>> Personalities : >>>>>> md127 : inactive sda[0](S) >>>>>> 52428724 blocks super 1.0 >>>>>> >>>>>> unused devices: >>>>>> >>>>>> So it looks like it has booted up on the disk that was not updated fro= m the raid pair. >>>>>> >>>>>> >>>>>> This probably also explains why occasionally I have had a vm that was = not working suddenly start working properly when I reboot it, but booting aga= in can put it back into non-working mode. >>>>>> >>>>>> It is probably dependent on which of the two disks actually get used t= o boot from. >>>>>> >>>>>> So the problem looks like it would only affect people who have raid se= tups and the problem originated in the CU167 upgrade. >>>>>> >>>>>> So raid systems need to have their raid arrays fully working before do= ing an upgrade. >>>>>> >>>>>> At least the above is my interpretation of what I have found. What doe= s everyone else think. >>>>>> >>>>>> >>>>>> Regards, >>>>>> >>>>>> Adolf. >>>>>> >>>>>> >>>>>> On 13/05/2022 15:59, Adolf Belka wrote: >>>>>>> Hi Michael, >>>>>>> >>>>>>> I have found some differences between the vm's that have upgraded wit= h no problem and those that have lost most of the modules. >>>>>>> >>>>>>> I did another three vm clones and upgrades and the first two all went= with no problems. I was starting to think the issue had gone away but on the= third one the same problem happened after the reboot. >>>>>>> >>>>>>> However I did copy the boot directory info for these two cases and fo= und an interesting effect. >>>>>>> >>>>>>> For the problem vm here was the boot directory layout of the CU167 ba= se machine >>>>>>> >>>>>>> drwxr-xr-x 5 root root 4.0K Apr 28 09:47 . >>>>>>> drwxr-xr-x 21 root root 4.0K Nov 28 15:49 .. >>>>>>> -rw-r--r-- 1 root root 179K Apr 26 13:00 config-5.15.35-ipfire >>>>>>> drwxr-xr-x 3 root root 16K Jan 1 1970 efi >>>>>>> drwxr-xr-x 6 root root 4.0K Apr 28 09:47 grub >>>>>>> -rw------- 1 root root 26M Apr 28 09:47 initramfs-5.15.35-ipfire.img >>>>>>> drwx------ 2 root root 16K Sep 30 2021 lost+found >>>>>>> -rw-r--r-- 1 root root 4.7M Apr 26 13:00 System.map-5.15.35-ipfire >>>>>>> -rw-r--r-- 1 root root 6.7M Apr 26 13:00 vmlinuz-5.15.35-ipfire >>>>>>> >>>>>>> After upgrade and before reboot this had changed to >>>>>>> >>>>>>> drwxr-xr-x 5 root root 4.0K May 13 15:40 . >>>>>>> drwxr-xr-x 21 root root 4.0K Nov 28 15:49 .. >>>>>>> -rw-r--r-- 1 root root 179K May 3 04:02 config-5.15.35-ipfire >>>>>>> drwxr-xr-x 3 root root 16K Jan 1 1970 efi >>>>>>> drwxr-xr-x 6 root root 4.0K May 13 15:41 grub >>>>>>> -rw------- 1 root root 26M May 13 15:41 initramfs-5.15.35-ipfire.img >>>>>>> drwx------ 2 root root 16K Sep 30 2021 lost+found >>>>>>> -rw-r--r-- 1 root root 4.7M May 3 04:02 System.map-5.15.35-ipfire >>>>>>> -rw-r--r-- 1 root root 6.7M May 3 04:02 vmlinuz-5.15.35-ipfire >>>>>>> >>>>>>> Dates and times of some of the files/directories have changed as woul= d be expected. >>>>>>> >>>>>>> After Reboot and with the problem the boot directory was now >>>>>>> >>>>>>> drwxr-xr-x 5 root root 4.0K Apr 28 09:47 . >>>>>>> drwxr-xr-x 21 root root 4.0K Nov 28 15:49 .. >>>>>>> -rw-r--r-- 1 root root 179K Apr 26 13:00 config-5.15.35-ipfire >>>>>>> drwxr-xr-x 3 root root 4.0K Sep 30 2021 efi >>>>>>> drwxr-xr-x 6 root root 4.0K Apr 28 09:47 grub >>>>>>> -rw------- 1 root root 26M Apr 28 09:47 initramfs-5.15.35-ipfire.img >>>>>>> drwx------ 2 root root 16K Sep 30 2021 lost+found >>>>>>> -rw-r--r-- 1 root root 4.7M Apr 26 13:00 System.map-5.15.35-ipfire >>>>>>> -rw-r--r-- 1 root root 6.7M Apr 26 13:00 vmlinuz-5.15.35-ipfire >>>>>>> >>>>>>> Which is the same as for the original CU167 machine except that the d= ate/time of the efi directory has changed >>>>>>> >>>>>>> For the vm clone that upgraded with no problems the boot directory af= ter reboot was the same as after upgrade. >>>>>>> >>>>>>> I don';t understand why or how this has happened but it is certainly = a difference between the vm's having the problem and the ones not. >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Adolf. >>>>>>> >>>>>>> On 13/05/2022 12:09, Michael Tremer wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>>> On 12 May 2022, at 21:10, Adolf Belka wr= ote: >>>>>>>>> >>>>>>>>> Hi Michael, >>>>>>>>> >>>>>>>>> On 12/05/2022 14:53, Michael Tremer wrote: >>>>>>>>>> Hello, >>>>>>>>>>> On 12 May 2022, at 12:25, Adolf Belka = wrote: >>>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> On 12/05/2022 11:13, Michael Tremer wrote: >>>>>>>>>>>> Hello, >>>>>>>>>>>> Thanks for spending so much time on this. We definitely need to = improve the general update experience since we sometimes seem to break people= =E2=80=99s systems and it is not nice to re-install a firewall from scratch. = It will take a while. >>>>>>>>>>>> So what I can say is that the kernel module issues come from whe= n the running kernel is changed and the kernel is trying to load any modules = that now have changed. This fails by design, because we sign our kernel modul= es. The key is randomly generated at build time and used to sign all modules = and it then thrown away. For each build, we are using a different, unique key= that is not preserved. >>>>>>>>>>>> This means that although the kernel modules are of the same vers= ion, they cannot be loaded because the signature check fails. That might also= explain why you are seeing so many ipset errors, because the kernel cannot l= oad that module any more. However, we use so much ipset now, why isn=E2=80=99= t the module loaded from before the update was started? >>>>>>>>>>>> The same goes for any network drivers. I assume you are using vi= rtio or a generic e1000 network adapter which will have been initialised at b= oot time. The kernel should never unload the kernel module for that interface= and load it again later. I have no idea what could have triggered that. >>>>>>>>>>> It is a generic e1000 network adapter that is being used by Virtu= alBox. Confirmed that on my working CU167 vm. >>>>>>>>>>>> No matter what though; after you reboot, the new kernel should b= e booted being able to load all modules it wants and the system should run ab= solutely fine. Can you confirm that that is at least the case? >>>>>>>>>>> No it is not the case. Yesterday when I booted the vm several tim= es the same error messages occurred each time and I always had no network int= erfaces. >>>>>>>>>> Okay. That is indeed quite bad then. >>>>>>>>>> Since there is no kernel in 168, is there a chance that we broke t= he update to 167? >>>>>>>>> I don't know but as far as I have been concerned CU167 has worked w= ell on my vm. >>>>>>>>> Is there something I should look for on my CU167 vm? >>>>>>>>>>> Just now I tried booting my third attempt vm again and this time = it ran okay and I ended up with all my networks assigned and I had network co= nnection. An IP was assigned by dhcp to the red interface. >>>>>>>>>> Third time lucky isn=E2=80=99t good enough for me :) >>>>>>>>> Me neither :-) >>>>>>>>>>> I then rebooted again and this time the error messages were back.= I rebooted the vm three more times and each time the error messages were the= re and no network interfaces. >>>>>>>>>>> >>>>>>>>>>> I checked with lsmod and the e1000 driver module was not loaded. = Confirmed the e1000 driver directory was not present in /sys/bus/pci/drivers >>>>>>>>>> Have there been any modules loaded? If one loads, they all should = load. >>>>>>>>> This is what I get with lsmod >>>>>>>>> >>>>>>>>> Module Size Used by >>>>>>>>> crct10dif_pclmul 16384 1 >>>>>>>>> crc32_pclmul 16384 0 >>>>>>>>> ghash_clmulni_intel 16384 0 >>>>>>>>> serio_raw 20480 0 >>>>>>>>> ohci_pci 20480 0 >>>>>>>>> ata_generic 16384 0 >>>>>>>>> pata_acpi 16384 0 >>>>>>>>> video 57344 0 >>>>>>>> >>>>>>>> It looks like these modules could have come from the initramdisk ima= ge. At least that is working well. >>>>>>>> >>>>>>>>> On my running CU1678 vm lsmod gives the following >>>>>>>>> >>>>>>>>> Module Size Used by >>>>>>>>> tun 61440 2 >>>>>>>>> nfnetlink_queue 28672 1 >>>>>>>>> xt_NFQUEUE 16384 8 >>>>>>>>> xt_MASQUERADE 20480 1 >>>>>>>>> cfg80211 1036288 0 >>>>>>>>> rfkill 32768 1 cfg80211 >>>>>>>>> 8021q 40960 0 >>>>>>>>> garp 16384 1 8021q >>>>>>>>> xt_set 16384 260 >>>>>>>>> ip_set_hash_net 49152 258 >>>>>>>>> ip_set 57344 2 xt_set,ip_set_hash_net >>>>>>>>> xt_hashlimit 20480 2 >>>>>>>>> xt_multiport 20480 4 >>>>>>>>> xt_policy 16384 5 >>>>>>>>> xt_TCPMSS 16384 1 >>>>>>>>> xt_conntrack 16384 7 >>>>>>>>> xt_comment 16384 18 >>>>>>>>> ipt_REJECT 16384 1 >>>>>>>>> nf_reject_ipv4 16384 1 ipt_REJECT >>>>>>>>> xt_LOG 20480 26 >>>>>>>>> xt_limit 16384 25 >>>>>>>>> xt_mark 16384 27 >>>>>>>>> xt_connmark 16384 2 >>>>>>>>> nf_log_syslog 24576 26 >>>>>>>>> iptable_raw 16384 0 >>>>>>>>> iptable_mangle 16384 1 >>>>>>>>> iptable_filter 16384 1 >>>>>>>>> vfat 24576 1 >>>>>>>>> fat 90112 1 vfat >>>>>>>>> sch_cake 36864 4 >>>>>>>>> intel_powerclamp 20480 0 >>>>>>>>> psmouse 184320 0 >>>>>>>>> pcspkr 16384 0 >>>>>>>>> i2c_piix4 28672 0 >>>>>>>>> e1000 163840 0 >>>>>>>>> i2c_core 106496 2 psmouse,i2c_piix4 >>>>>>>>> crct10dif_pclmul 16384 1 >>>>>>>>> crc32_pclmul 16384 0 >>>>>>>>> ata_generic 16384 0 >>>>>>>>> pata_acpi 16384 0 >>>>>>>>> ghash_clmulni_intel 16384 0 >>>>>>>>> serio_raw 20480 0 >>>>>>>>> ohci_pci 20480 0 >>>>>>>>> video 57344 0 >>>>>>>> >>>>>>>> Yes, this looks more like it. >>>>>>>> >>>>>>>>> The last 8 entries are the same but a whole load of others are miss= ing. >>>>>>>> >>>>>>>> Stefan gave me a call this morning telling me that he tried to repro= duce this but he couldn=E2=80=99t. >>>>>>>> >>>>>>>> I am not sure what we can do here. We urgently need to fix this sinc= e we might break people=E2=80=99s installations. But something seems to work = differently for Adolf than for everybody else. >>>>>>>> >>>>>>>> -Michael >>>>>>>> >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Adolf. >>>>>>>>>>> Tried modprobe e1000 but this came back with the following error >>>>>>>>>>> modprobe: ERROR: could not insert 'e1000': Key was rejected by se= rvice >>>>>>>>>>> >>>>>>>>>>> So out of 7 or 8 reboots the vm booted with the network drivers l= oaded once. >>>>>>>>>>> >>>>>>>>>>> Regards >>>>>>>>>>> Adolf >>>>>>>>>>>>> On 11 May 2022, at 20:08, Adolf Belka wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi All, >>>>>>>>>>>>> >>>>>>>>>>>>> On 11/05/2022 20:48, Jon Murphy wrote: >>>>>>>>>>>>>> I just did an update from CU 167 (stable) to CU 168 (testing) = and I got the same build info: >>>>>>>>>>>>>> *IPFire 2.27 (x86_64) - Core Update 167 Development Build: mas= ter/c22d834c* >>>>>>>>>>>>>> Shouldn=E2=80=99t this be Core Update 168? >>>>>>>>>>>>> That was what I thought but I couldn't precisely remember what = it was in the past with Testing Releases. >>>>>>>>>>>>> >>>>>>>>>>>>> I then downloaded the Core 168 iso from the nightlies build of = master/latest/x86_64 and did an install on the First attempt vm. >>>>>>>>>>>>> >>>>>>>>>>>>> I got to the stage of entering the root and admin passwords. En= tered what I usually do for the vm's and after pressing OK on the admin passw= ord screen I got as message box saying "Problem setting IPFire 'admin' user p= assword". I pressed the OK button and got another message box saying "Initial= setup was not entirely complete. You must ensure that Setup is properly fini= shed by running setup again at the shell.". Pressing the OK button on that me= ssage box causes setup to restart but again after entering a valid admin pass= word twice I got the same "Problem setting IPFire 'admin' user password" mess= age box. >>>>>>>>>>>>> >>>>>>>>>>>>> I then tried installing CU168 from the same iso onto a clone of= my running CU167 vm. Same result with "Problem setting IPFire 'admin' user p= assword" message box. >>>>>>>>>>>>> >>>>>>>>>>>>> Then I created a completely new vm from scratch and tried insta= lling CU168 Testing iso and again got "Problem setting IPFire 'admin' user pa= ssword" message box. >>>>>>>>>>>> That seems to be a bug that should not be there. Did we recently= update apache? >>>>>>>>>>>> -Michael >>>>>>>>>>>>> >>>>>>>>>>>>> Don't know if this is related to the problem with the missing i= nterfaces but if not then it is another problem. >>>>>>>>>>>>> >>>>>>>>>>>>> I think I am going to raise a bug on this. >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Adolf. >>>>>>>>>>>>>> Jon >>>>>>>>>>>>>>> On May 11, 2022, at 9:19 AM, Adolf Belka > wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 11/05/2022 16:00, Adolf Belka wrote: >>>>>>>>>>>>>>>> Hi Leo, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 11/05/2022 15:26, Leo Hofmann wrote: >>>>>>>>>>>>>>>>> Hi Adolf, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Pakfire always automatically reinstalls the current release= before updating if you are in the testing branch. See this function in the P= akfire code: >>>>>>>>>>>>>>>> So that proves that I haven't been looking so closely on my = upgrades in the past because I have never noticed that. >>>>>>>>>>>>>>>>> https://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dblob;f=3Dsrc= /pakfire/lib/functions.pl;h=3Dd4e338f23ae8ae97d6f18c6d8890d13463dc5d30;hb=3Dr= efs/heads/next#l762 >>>>>>>>>>>>>>>> Now I see it in the code. Thanks very much. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> But unfortunately this is all I can contribute, my test sys= tem updated to 168 without problems! >>>>>>>>>>>>>>>> Thanks for your help anyway. It's got rid of one of my quest= ions. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Just need to understand now why 2 out of 3 updates resulted = in a complete loss of the interface assignments. >>>>>>>>>>>>>>>> Will wait to hear other inputs on this problem. >>>>>>>>>>>>>>>> Maybe will try running setup on the first try I had to confi= rm that I can re-assign the interfaces again. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Running setup does not help. None of the interfaces are avail= able to be assigned to the red, green etc zones. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I checked the interfaces in the vm setup and they are as prev= iously defined and previously working on all earlier CU's of my vm. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> When rebooting I did see a fast scrolling message that seemed= to say something like Invalid Kernel Argument but that is as much as I was a= ble to see/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Adolf. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>> Adolf. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Best regards >>>>>>>>>>>>>>>>> Leo >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Am 11.05.2022 um 15:11 schrieb Adolf Belka: >>>>>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I have tried to update my Core Update 167 vm machine three= times now (using a clone) and have had several problems so I thought I would= outline what has happened for further thoughts. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Normally my updates go without any real problems. This is = the first time where this is not the case. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> First try:- >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> After running the update, which went quite quickly, the bo= ttom of the screen had >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Core Update 167 Development Build: master/c22d834c >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I couldn't remember if this is what is expected or not for= Testing releases. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Rebooted the vm and on booting a large number of error mes= sages scrolled across the console screen. My Virtualbox terminal for IPFire h= as no scroll capability so I can only write on things I saw as they flew past= or what was on the screen when it stopped. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Lots of ipset restore error but went by too fast to get mo= re detail. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Tried to access via the WUI and no access. Tried via SSH a= nd no access. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Checked the red0 directory and there were no files present= . Tried restarting red0 and got the following message >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> starting >>>>>>>>>>>>>>>>>> DUID 00:04:70:........ >>>>>>>>>>>>>>>>>> red0: interface not found >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> So I ran ip address show and it came up with only the lo i= nterface. No red0, green0, blue0 or orange0. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Looked in bootlog and this was filled with around 4000 lin= es all the same >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Loading of module with unavailable key is rejected >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Second try:- >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> This time running the update took over 10 mins. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Noticed on the pakfire screen that it first ran a Core Upd= ate 167 upgrade then it ran the Core Update 168. My IPFire vm was definitely = on Core Update 167. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> /opt/pakfire/db/core/mine contained 167 beforehand. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Before doing the reboot I accessed via ssh and saw that in= the /var/log/pakfire/ directory the update-core-upgrade-167 file has been up= dated as well as there being a 168. Both files had the same last modified dat= e/time. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Rebooted. This time everything came back up okay but the b= ottom of the screen now just said Core Update 167 and there was no update-cor= e-upgrade-168 file in the pakfire directory and the 167 version was back to i= ts date of Apr 28 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Repository was set back to Stable. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Changed again to Testing and it said again that there was = an update from 167 to 168. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Before running the pakfire directory had the following >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 1.8K Mar 31 23:26 update-core-upgra= de-166.log >>>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 323K Apr 28 09:47 update-core-upgra= de-167.log >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> After running it had >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 1.8K Mar 31 23:26 update-core-upgra= de-166.log >>>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 604K May 11 13:24 update-core-upgra= de-167.log >>>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 175 May 11 13:24 update-core-upgrad= e-168.log >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> back to Core Update 167 Development Build: master/c22d834c= at bottom of screen. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Rebooted. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> All interfaces present. Pakfire under System Status: says = Core-Update-Level: 168 and the bottom of the screen showed Core Update 167 De= velopment Build: master/c22d834c >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> /opt/pakfire/db/core/mine has 168 in it >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I checked the WIO wio.pl file that had a bug fix and it wa= s as expected for CU168. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Third try:- >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 323K Apr 28 09:47 update-core-upgra= de-167.log >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Selected Testing and ran update from 167 to 168 which occu= rred very quickly. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 604K May 11 14:00 update-core-upgra= de-167.log >>>>>>>>>>>>>>>>>> -rw-r--r-- 1 root root 63K May 11 14:01 update-core-upgrad= e-168.log >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Here is the log file info for pakfire from messages.1.gz a= nd messages which shows it downloading and upgrading CU167 first then doing C= U168 as far as I can tell. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: PAKFIRE INFO: IPFire Pakfi= re 2.27.1-x86_64 started! >>>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: CORE INFO: core-list.db is= 27 seconds old. - DEBUG: noforce >>>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: CORE UPGR: Upgrading from = release 166 to 168 >>>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: DOWNLOAD STARTED: paks/cor= e-upgrade-2.27-167.ipfire >>>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: MIRROR INFO: 2 servers fou= nd in list >>>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: DOWNLOAD INFO: Host: ipfir= e.earl-net.com (HTTPS) - File: pakfire2/2.27.1-= x86_64/paks/core-upgrade-2.27-167.ipfire >>>>>>>>>>>>>>>>>> May 11 13:59:30 ipfire pakfire: DOWNLOAD INFO: pakfire2/2.= 27.1-x86_64/paks/core-upgrade-2.27-167.ipfire has size of 71914825 bytes >>>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD INFO: HTTP-Status= -Code: 200 - 200 OK >>>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD INFO: File receiv= ed. Start checking signature... >>>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD INFO: Signature o= f core-upgrade-2.27-167.ipfire is fine. >>>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD FINISHED: pakfire= 2/2.27.1-x86_64/paks/core-upgrade-2.27-167.ipfire >>>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD STARTED: meta/met= a-core-upgrade-168 >>>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: MIRROR INFO: 2 servers fou= nd in list >>>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD INFO: Host: mirro= r1.ipfire.org (HTTPS) - File: pakfire2/2.27.1-x8= 6_64/meta/meta-core-upgrade-168 >>>>>>>>>>>>>>>>>> May 11 13:59:32 ipfire pakfire: DOWNLOAD INFO: pakfire2/2.= 27.1-x86_64/meta/meta-core-upgrade-168 has size of 1804 bytes >>>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD INFO: HTTP-Status= -Code: 200 - 200 OK >>>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD INFO: File receiv= ed. Start checking signature... >>>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD INFO: Signature o= f meta-core-upgrade-168 is fine. >>>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD FINISHED: pakfire= 2/2.27.1-x86_64/meta/meta-core-upgrade-168 >>>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD STARTED: paks/cor= e-upgrade-2.27-168.ipfire >>>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: MIRROR INFO: 2 servers fou= nd in list >>>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD INFO: Host: ipfir= e.earl-net.com (HTTPS) - File: pakfire2/2.27.1-= x86_64/paks/core-upgrade-2.27-168.ipfire >>>>>>>>>>>>>>>>>> May 11 13:59:33 ipfire pakfire: DOWNLOAD INFO: pakfire2/2.= 27.1-x86_64/paks/core-upgrade-2.27-168.ipfire has size of 34355484 bytes >>>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: DOWNLOAD INFO: HTTP-Status= -Code: 200 - 200 OK >>>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: DOWNLOAD INFO: File receiv= ed. Start checking signature... >>>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: DOWNLOAD INFO: Signature o= f core-upgrade-2.27-168.ipfire is fine. >>>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: DOWNLOAD FINISHED: pakfire= 2/2.27.1-x86_64/paks/core-upgrade-2.27-168.ipfire >>>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: PAKFIRE UPGR: core-upgrade= -167: Decrypting... >>>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>>> May 11 13:59:34 ipfire pakfire: DECRYPT STARTED: core-upgr= ade-167 >>>>>>>>>>>>>>>>>> May 11 13:59:35 ipfire pakfire: DECRYPT FINISHED: core-upg= rade-167 - Status: 0 >>>>>>>>>>>>>>>>>> May 11 13:59:35 ipfire pakfire: PAKFIRE UPGR: core-upgrade= -167: Upgrading files and running post-upgrading scripts... >>>>>>>>>>>>>>>>>> May 11 14:00:31 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>>> May 11 14:00:31 ipfire pakfire: PAKFIRE UPGR: core-upgrade= -167: Finished. >>>>>>>>>>>>>>>>>> May 11 14:00:31 ipfire pakfire: PAKFIRE UPGR: core-upgrade= -168: Decrypting... >>>>>>>>>>>>>>>>>> May 11 14:00:31 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>>> May 11 14:00:31 ipfire pakfire: DECRYPT STARTED: core-upgr= ade-168 >>>>>>>>>>>>>>>>>> May 11 14:00:32 ipfire pakfire: DECRYPT FINISHED: core-upg= rade-168 - Status: 0 >>>>>>>>>>>>>>>>>> May 11 14:00:32 ipfire pakfire: PAKFIRE UPGR: core-upgrade= -168: Upgrading files and running post-upgrading scripts... >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: PAKFIRE UPGR: core-upgrade= -168: Finished. >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: CORE INFO: core-list.db is= 127 seconds old. - DEBUG: noforce >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: PAKFIRE RESV: wio: Resolvi= ng dependencies... >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: PAKFIRE UPGR: We are going= to install all packages listed above. >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD STARTED: paks/wio= -1.3.2-15.ipfire >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: MIRROR INFO: 2 servers fou= nd in list >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD INFO: Host: mirro= r1.ipfire.org (HTTPS) - File: pakfire2/2.27.1-x8= 6_64/paks/wio-1.3.2-15.ipfire >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD INFO: pakfire2/2.= 27.1-x86_64/paks/wio-1.3.2-15.ipfire has size of 47216 bytes >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD INFO: HTTP-Status= -Code: 200 - 200 OK >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD INFO: File receiv= ed. Start checking signature... >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD INFO: Signature o= f wio-1.3.2-15.ipfire is fine. >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DOWNLOAD FINISHED: pakfire= 2/2.27.1-x86_64/paks/wio-1.3.2-15.ipfire >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: PAKFIRE UPGR: wio: Decrypt= ing... >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DECRYPT STARTED: wio >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: DECRYPT FINISHED: wio - St= atus: 0 >>>>>>>>>>>>>>>>>> May 11 14:01:10 ipfire pakfire: PAKFIRE UPGR: wio: Upgradi= ng files and running post-upgrading scripts... >>>>>>>>>>>>>>>>>> May 11 14:01:11 ipfire pakfire: CLEANUP: tmp >>>>>>>>>>>>>>>>>> May 11 14:01:11 ipfire pakfire: PAKFIRE UPGR: wio: Finishe= d. >>>>>>>>>>>>>>>>>> May 11 14:01:11 ipfire pakfire: PAKFIRE INFO: Pakfire has = finished. Closing. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Rebooted >>>>>>>>>>>>>>>>>> Lots of ipset restore error but went by too fast to get mo= re detail. >>>>>>>>>>>>>>>>>> Several repeats of the following two lines on the console = screen:- >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> iptables v1.8.7 (legacy): can't initialize iptables table = `filter' : Table does not exist (do you need to insmod?) >>>>>>>>>>>>>>>>>> Perhaps iptables or your kernel needs to be upgraded. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Only lo interface present. All other interfaces red0, gree= n0, blue0, orange0 not present. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I have given up at this point. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I have kept the clones of all three attempts so if you nee= d any additional info from logs or files just let me know. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> If this is something obviously wrong that I am doing, plea= se let me know. If not then I will probably raise a bug on this. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Adolf. >>>>>>>> >=20 --===============7940496142477443562==--