From mboxrd@z Thu Jan 1 00:00:00 1970 From: Peter =?utf-8?q?M=C3=BCller?= To: development@lists.ipfire.org Subject: Re: [PATCH v2] Core Update 183: Perform housekeeping to keep file lists aligned Date: Thu, 11 Jan 2024 16:33:00 +0000 Message-ID: <90aa990a-77a6-4b57-9956-bc2902a03da9@ipfire.org> In-Reply-To: <254AC305-C5F9-4D24-A363-7EFE60D19F0B@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4558544517880612009==" List-Id: --===============4558544517880612009== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello Michael, thank you for your reply. > Hello, >=20 > I am somewhat concerned about this patch when it comes to the libraries. Given my track record when deleting these, your concerns are perfectly undera= standable. I'll try my best to disperse them below... :-) >=20 > Please make sure that literally nothing is linked against any of those and = that we definitely shipped any binary that might have linked against those li= braries. >=20 > Secondly, we do have a script that should take care of this. Why did the sc= ript not cleanup those files? Could you please investigate on your system why= they did not get deleted? Quite frankly, I don't know. My impression is that some of these libraries we= re in place before we started rolling out the cleanup script. As the IPFire installations in question have never been in testing, I can rul= e out these files to stem from orphaned switches between testing and stable r= eleases of IPFire. As far as other files are concerned, these all stem from insufficient clean u= p or comparison of rootfiles - particularly linux-firmware is prone to these = quirks. >=20 > -Michael >=20 >> On 8 Jan 2024, at 21:48, Peter M=C3=BCller wr= ote: >> >> By comparing the filelist present on a fresh installation of the latest >> Core Update 183 nightly build with various IPFire installations in the >> fields, a number of differences surfaced, of which most are caused by >> erroneous additions or exclusions of certain files while shipping Core >> Updates, first and foremost related to linux-firmware. >> >> In addition, libcap was also updated to 2.69, but never shipped on >> existing installations. >> >> This patch corrects all differences, and aligns the files present and >> absent on existing installations with those freshly shipped with Core >> Update 183. >> >> The second version of this patch does not delete the >> "/etc/rc.d/rc3.d/off" directory, if present (it is used for storing >> initscripts of disabled services), is more explicit about removing >> /usr/lib/grub/x86_64-efi/verify.* (dot omitted in the first version), >> and includes additional files surfacing on yet another IPFire >> installation in the fields. >> >> The changes are cross-checked against linked libraries on the affected >> systems to rule out any instances of binaries being present that are >> still linked against the old libraries. >> >> Cc: Arne Fitzenreiter >> Signed-off-by: Peter M=C3=BCller >> --- >> config/rootfiles/core/183/filelists/files | 45 +++++++++++++++++++ >> config/rootfiles/core/183/filelists/libcap | 1 + >> config/rootfiles/core/183/update.sh | 52 +++++++++++++++++++++- >> 3 files changed, 97 insertions(+), 1 deletion(-) >> create mode 120000 config/rootfiles/core/183/filelists/libcap >> >> diff --git a/config/rootfiles/core/183/filelists/files b/config/rootfiles/= core/183/filelists/files >> index 949b1b2dc..259fc7c37 100644 >> --- a/config/rootfiles/core/183/filelists/files >> +++ b/config/rootfiles/core/183/filelists/files >> @@ -1,3 +1,48 @@ >> +etc/sudoers.d/logwatch-mdadm >> +lib/firmware/brcm/BCM-0a5c-6410.hcd >> +lib/firmware/brcm/brcmfmac43012-sdio.bin >> +lib/firmware/brcm/brcmfmac43012-sdio.clm_blob >> +lib/firmware/brcm/brcmfmac43430-sdio.clm_blob >> +lib/firmware/brcm/brcmfmac43430-sdio.raspberrypi,model-zero-w.txt >> +lib/firmware/brcm/brcmfmac43430-sdio.sinovoip,bpi-m2-plus.txt >> +lib/firmware/brcm/brcmfmac43430-sdio.sinovoip,bpi-m2-ultra.txt >> +lib/firmware/brcm/brcmfmac43430-sdio.sinovoip,bpi-m2-zero.txt >> +lib/firmware/brcm/brcmfmac43430-sdio.sinovoip,bpi-m3.txt >> +lib/firmware/brcm/brcmfmac43455-sdio.clm_blob >> +lib/firmware/brcm/brcmfmac43455-sdio.raspberrypi,3-model-a-plus.txt >> +lib/firmware/brcm/brcmfmac43455-sdio.Raspberry_Pi_Foundation-Raspberry_Pi= _4_Model_B.txt >> +lib/firmware/brcm/brcmfmac43455-sdio.Raspberry_Pi_Foundation-Raspberry_Pi= _Compute_Module_4.txt >> +lib/firmware/brcm/brcmfmac4354-sdio.clm_blob >> +lib/firmware/brcm/brcmfmac4356-pcie.clm_blob >> +lib/firmware/brcm/brcmfmac4356-sdio.clm_blob >> +lib/firmware/brcm/brcmfmac4356-sdio.khadas,vim2.txt >> +lib/firmware/brcm/brcmfmac43570-pcie.clm_blob >> +lib/firmware/brcm/brcmfmac4373-sdio.clm_blob >> +lib/firmware/brcm/brcmfmac54591-pcie.bin >> +lib/firmware/brcm/brcmfmac54591-pcie.clm_blob >> +lib/firmware/cxgb4/t4-config.txt >> +lib/firmware/cxgb4/t5-config.txt >> +lib/firmware/cxgb4/t6-config.txt >> +lib/firmware/intel/ice/ddp/ice.pkg >> +lib/firmware/netronome/flower/nic_AMDA0058-0011_1x100.nffw >> +lib/firmware/netronome/flower/nic_AMDA0058-0011_2x40.nffw >> +lib/firmware/netronome/flower/nic_AMDA0058-0011_4x10_1x40.nffw >> +lib/firmware/netronome/flower/nic_AMDA0058-0011_8x10.nffw >> +lib/firmware/netronome/flower/nic_AMDA0058-0012_1x100.nffw >> +lib/firmware/netronome/flower/nic_AMDA0058-0012_2x40.nffw >> +lib/firmware/netronome/flower/nic_AMDA0058-0012_4x10_1x40.nffw >> +lib/firmware/netronome/flower/nic_AMDA0058-0012_8x10.nffw >> +lib/firmware/netronome/flower/nic_AMDA0078-0011_1x100.nffw >> +lib/firmware/netronome/flower/nic_AMDA0078-0011_2x40.nffw >> +lib/firmware/netronome/flower/nic_AMDA0078-0011_4x10_1x40.nffw >> +lib/firmware/netronome/flower/nic_AMDA0078-0011_8x10.nffw >> +lib/firmware/netronome/flower/nic_AMDA0078-0012_1x100.nffw >> +lib/firmware/netronome/flower/nic_AMDA0078-0012_2x40.nffw >> +lib/firmware/netronome/flower/nic_AMDA0078-0012_4x10_1x40.nffw >> +lib/firmware/netronome/flower/nic_AMDA0078-0012_8x10.nffw >> +lib/firmware/nvidia/tegra124/vic.bin >> +lib/firmware/nvidia/tegra186/vic.bin >> +lib/firmware/nvidia/tegra210/vic.bin >> srv/web/ipfire/cgi-bin/dhcp.cgi >> srv/web/ipfire/cgi-bin/proxy.cgi >> srv/web/ipfire/cgi-bin/logs.cgi/firewalllog.dat >> diff --git a/config/rootfiles/core/183/filelists/libcap b/config/rootfiles= /core/183/filelists/libcap >> new file mode 120000 >> index 000000000..ed67d950a >> --- /dev/null >> +++ b/config/rootfiles/core/183/filelists/libcap >> @@ -0,0 +1 @@ >> +../../../common/libcap >> \ No newline at end of file >> diff --git a/config/rootfiles/core/183/update.sh b/config/rootfiles/core/1= 83/update.sh >> index 6ff84387f..db807c5df 100644 >> --- a/config/rootfiles/core/183/update.sh >> +++ b/config/rootfiles/core/183/update.sh >> @@ -92,15 +92,65 @@ extract_files >> >> # Remove files >> rm -rvf \ >> + /etc/fb.modes \ >> + /etc/pango \ >> /etc/fonts/conf.d/10-sub-pixel-rgb.conf \ >> + /etc/rc.d/init.d/snort \ >> + /lib/libBrokenLocale-2.33.so \ (FYI, your MUA stripped the tabulator here.) This file does not appear in any "ldd" output of any other executable on the = affected system. >> + /lib/libcap.so.2.66 \ For whatever reason, the symlink targets regaring libcap differ on the affect= ed system: > [root(a)firewall ~]# ls -lah /usr/lib64/libcap.so.2 > lrwxrwxrwx 1 root root 14 Jul 10 2023 /usr/lib64/libcap.so.2 -> libcap.so.= 2.69 > [root(a)firewall ~]# ls -lah /lib/libcap.so.2 > lrwxrwxrwx 1 root root 14 Feb 22 2023 /lib/libcap.so.2 -> libcap.so.2.66 The following binaries are still implicitly make use of libcap.so.2.66 on the= affected system: /usr/bin/ntpq /usr/bin/ntpdate /usr/bin/ntp-keygen /usr/bin/ntpdc /usr/bin/sntp /usr/bin/ping /usr/bin/tickadj /usr/bin/ntpd /usr/bin/ntptime /usr/lib/security/pam_cap.so /usr/sbin/arping /lib/security/pam_cap.so /sbin/getcap /sbin/nstat /sbin/setcap /sbin/rtmon /sbin/ss /sbin/capsh /sbin/rtacct /sbin/ifstat /sbin/dcb /sbin/vdpa /sbin/genl /sbin/rdma /sbin/ip /sbin/bridge /sbin/lnstat /sbin/tc /sbin/devlink /sbin/getpcaps However, on another system, libcap.so.2.66 is not present anymore, and /lib/l= ibcap.so.2 points to libcap.so.2.67. Since this patch proposes to ship a new version of = libcap, this should solve the problem for some of the affected binaries, but we might neee= d to manually adjust /lib/libcap.so.2. I am not sure how the current situation (having two = different versions of libcap present under two different symlinks) has happened. >> + /lib/libpsx.so.2.66 \ See above. >> + /lib/firmware/ath10k/WCN3990/hw1.0/notice.txt_wlanmdsp \ >> + /lib/firmware/ath11k/IPQ6018/hw1.0/Notice.txt \ >> + /lib/firmware/ath11k/IPQ8074/hw2.0/Notice.txt \ >> + /lib/firmware/ath11k/QCA6390/hw2.0/Notice.txt \ >> + /lib/firmware/ath11k/QCN9074/hw1.0/Notice.txt \ >> + /lib/firmware/ath11k/WCN6855/hw2.0/Notice.txt \ >> + /lib/firmware/intel-ucode/06-86-04 \ >> + /lib/firmware/intel-ucode/06-86-05 \ >> + /lib/xtables/libebt_802_3.so \ Does not yield any hits in "ldd" outputs anymore. Remnant of former xtables v= ersion. >> + /lib/xtables/libebt_ip.so \ Ditto. >> + /lib/xtables/libebt_log.so \ Ditto. >> + /lib/xtables/libebt_mark_m.so \ Ditto. >> + /lib/xtables/libxt_mangle.so \ Ditto. >> + /sbin/xtables-multi \ >> + /srv/web/ipfire/html/themes/ipfire-rounded \ >> + /usr/lib/crda/pubkeys/linville.key.pub.pem \ >> + /usr/lib/libasan.so.{4,6}* \ Ditto. >> + /usr/lib/libbfd-2.3* \ Only surfaces in "ldd" outputs concerning outdated libraries removed below. >> + /usr/lib/libbfd-2.40.so \ Ditto. >> /usr/lib/libbind9-9.16.44.so \ >> + /usr/lib/libcilkrts.so* \ Remnant of former GCC (?) version. Not used by any binary on the affected sys= tem. >> /usr/lib/libdns-9.16.44.so \ >> + /usr/lib/libdnssec.so.6* \ Ditto. >> + /usr/lib/libhogweed.so.4* \ Ditto. >> + /usr/lib/libipset.so.11* \ Orphaned version, not used by any binary. >> /usr/lib/libirs-9.16.44.so \ >> /usr/lib/libisc-9.16.44.so \ >> /usr/lib/libisccc-9.16.44.so \ >> /usr/lib/libisccfg-9.16.44.so \ >> + /usr/lib/libknot.so.8* \ Ditto. >> + /usr/lib/libknot.so.12* \ Ditto. >> + /usr/lib/libnettle.so.6* \ Ditto. >> /usr/lib/libns-9.16.44.so \ >> - /usr/lib/libxml2.so.2.11* >> + /usr/lib/libopcodes-2.3* \ Ditto, see above. >> + /usr/lib/libopcodes-2.40.so \ Ditto. >> + /usr/lib/libubsan.so.0* \ Does not yield any "ldd" output hits whatsoever. >> + /usr/lib/libxml2.so.2.11* \ Orphaned version, all binaries use newer versions present on the system. >> + /usr/lib/libzscanner.so* \ GCC (?) remnant. Not used. To sum it up, the situation surrounding libcap is the only aspect of this pat= ch that strikes me as potentially problematic. Do you see any other sources for forth= coming trouble? As a general rule of thumb, my impression is that there is a cut-off date bef= ore which IPFire installations may have been a bit messier than they are today, now tha= t we got better at removing outdated files during Core Update installation. Perhaps ru= nning this entire comparison against a long-standing productive system in a future = patch also makes sense to clean up the rest. Thanks, and best regards, Peter M=C3=BCller >> + /usr/lib/grub/i386-pc/efiemu{32,64}.o \ >> + /usr/lib/grub/i386-pc/verifiers.* \ >> + /usr/lib/grub/i386-pc/verify.* \ >> + /usr/lib/grub/x86_64-efi/shim_lock.* \ >> + /usr/lib/grub/x86_64-efi/verifiers.* \ >> + /usr/lib/grub/x86_64-efi/verify.* \ >> + /usr/lib/snort_dynamic* \ >> + /usr/local/bin/snortctrl \ >> + /usr/share/usb_modeswitch/1033:0035 \ >> + /usr/share/vim/vim7* \ >> + /var/ipfire/geoip-functions.pl \ >> + /var/ipfire/dhcpc/dhcpcd-hooks/00-linux \ >> + /var/ipfire/dhcpc/dhcpcd-hooks/02-dump \ >> + /var/lib/location/tmp* >> >> # update linker config >> ldconfig >> --=20 >> 2.35.3 >=20 --===============4558544517880612009==--