Development March 2022

development@lists.ipfire.org

21 participants
110 discussions

[PATCH 1/3] unbound-dhcp-leases-bridge : fix bug 12694 - DHCP hosts not reliably propagated to DNS
by Anthony Heading 30 Mar '22

30 Mar '22

Switch from inotify watching individual files to monitoring the containing directories, as because dhcpd renames its leases file into a backup, monitoring the single inode does not work well. Additionally, python appears to have a bug with replacing expired inotify watches on single files. --- unbound-dhcp-leases-bridge | 47 +++++++++++++++++++++++++------------- 1 file changed, 31 insertions(+), 16 deletions(-) diff --git unbound-dhcp-leases-bridge unbound-dhcp-leases-bridge index a2df5f1..6e22066 100644 --- unbound-dhcp-leases-bridge +++ unbound-dhcp-leases-bridge @@ -72,6 +72,15 @@ class UnboundDHCPLeasesBridge(object): self.fix_leases_file = fix_leases_file self.hosts_file = hosts_file + # base mask for a completed file change + mask = inotify.constants.IN_CLOSE_WRITE | inotify.constants.IN_MOVED_TO + # IN_MODIFY since dhcpd appends lease updates to an open file + self.watches = { + self.leases_file: mask | inotify.constants.IN_MODIFY, + self.fix_leases_file: mask, + self.hosts_file: mask + } + self.unbound = UnboundConfigWriter(unbound_leases_file) self.running = False @@ -80,36 +89,42 @@ class UnboundDHCPLeasesBridge(object): self.running = True # Initial setup - self.hosts = self.read_static_hosts() - self.update_dhcp_leases() + update_hosts = True + update_leases = True + + i = inotify.adapters.Inotify() - i = inotify.adapters.Inotify([ - self.leases_file, - self.fix_leases_file, - self.hosts_file, - ]) + for f in self.watches: + i.add_watch(os.path.dirname(f), self.watches[f]) for event in i.event_gen(): # End if we are requested to terminate if not self.running: break + # Make pending updates once inotify queue is empty if event is None: + if update_hosts: + self.hosts = self.read_static_hosts() + update_hosts = False + if update_leases: + self.update_dhcp_leases() + update_leases = False continue header, type_names, watch_path, filename = event - # Update leases after leases file has been modified - if "IN_MODIFY" in type_names: - # Reload hosts - if watch_path == self.hosts_file: - self.hosts = self.read_static_hosts() + file = os.path.join(watch_path, filename) + + if not file in self.watches: + continue + + log.debug("Inotify %s: %s", file, " ".join(type_names)) - self.update_dhcp_leases() + update_leases = True - # If the file is deleted, we re-add the watcher - if "IN_IGNORED" in type_names: - i.add_watch(watch_path) + if file == self.hosts_file: + update_hosts = True log.info("Unbound DHCP Leases Bridge terminated") -- 2.34.1

3 11

[PATCH 1/3] unbound-dhcp-leases-bridge: Improve logging
by Michael Tremer 29 Mar '22

29 Mar '22

This improves logging and enables logging to the console. Suggested-by: Anthony Heading <ajrh(a)ajrh.net> Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org> --- config/unbound/unbound-dhcp-leases-bridge | 26 ++++++++++++++++------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/config/unbound/unbound-dhcp-leases-bridge b/config/unbound/unbound-dhcp-leases-bridge index a2df5f101..ce0cb8614 100644 --- a/config/unbound/unbound-dhcp-leases-bridge +++ b/config/unbound/unbound-dhcp-leases-bridge @@ -30,27 +30,37 @@ import re import signal import stat import subprocess +import sys import tempfile import inotify.adapters LOCAL_TTL = 60 -def setup_logging(loglevel=logging.INFO): - log = logging.getLogger("dhcp") +log = logging.getLogger("dhcp") +log.setLevel(logging.DEBUG) + +def setup_logging(daemon=True, loglevel=logging.INFO): log.setLevel(loglevel) + # Log to syslog by default handler = logging.handlers.SysLogHandler(address="/dev/log", facility="daemon") - handler.setLevel(loglevel) + log.addHandler(handler) + # Format everything formatter = logging.Formatter("%(name)s[%(process)d]: %(message)s") handler.setFormatter(formatter) - log.addHandler(handler) + handler.setLevel(loglevel) - return log + # If we are running in foreground, we should write everything to the console, too + if not daemon: + handler = logging.StreamHandler() + log.addHandler(handler) -log = logging.getLogger("dhcp") + handler.setLevel(loglevel) + + return log def ip_address_to_reverse_pointer(address): parts = address.split(".") @@ -579,12 +589,12 @@ if __name__ == "__main__": elif args.verbose >= 2: loglevel = logging.DEBUG - setup_logging(loglevel) + setup_logging(daemon=args.daemon, loglevel=loglevel) bridge = UnboundDHCPLeasesBridge(args.dhcp_leases, args.fix_leases, args.unbound_leases, args.hosts) - ctx = daemon.DaemonContext(detach_process=args.daemon) + ctx = daemon.DaemonContext(detach_process=args.daemon, stderr=sys.stderr) ctx.signal_map = { signal.SIGHUP : bridge.update_dhcp_leases, signal.SIGTERM : bridge.terminate, -- 2.30.2

1 2

[PATCH] backup.pl: fix Bug12817 - directories missing in backup after include file changed to relat>
by Adolf Belka 29 Mar '22

29 Mar '22

From: Bernhard Bitsch <bbitsch(a)ipfire.org> - The change of the backup include file from absolute paths to relative paths in CU164 commit https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=c7e0d73e7cfd7be95db9d0a… resulted in 20 core function directories no longer being backed up. Additionally a similar effect occurred with some addon backups - Fix applied here proposed by Bernhard Bitsch and tested on a vm testbed system and confirmed to fix the problem. Backup of all directories again being done. Fixes: Bug12817 Tested-By: Adolf Belka <adolf.belka(a)ipfire.org> Signed-off-by: Adolf Belka <adolf.belka(a)ipfire.org> --- config/backup/backup.pl | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/config/backup/backup.pl b/config/backup/backup.pl index a2337cf23..c7dbc6cae 100644 --- a/config/backup/backup.pl +++ b/config/backup/backup.pl @@ -35,16 +35,18 @@ list_addons() { process_includes() { local include - for include in $@; do - local file - while read -r file; do - for file in ${file}; do - if [ -e "/${file}" ]; then - echo "${file}" - fi - done - done < "${include}" - done | sort -u + pushd / >/dev/null + for include in $@; do + local file + while read -r file; do + for file in ${file}; do + if [ -e "/${file}" ]; then + echo "${file}" + fi + done + done < "${include}" + done | sort -u + popd >/dev/null } make_backup() { -- 2.35.1

3 8

[PATCH 1/5] linux-firmware: Don't ship Marvell Prestera Firmware
by Michael Tremer 29 Mar '22

29 Mar '22

This firmware is required for a switch ASIC which we build the kernel module for, but which is probably not usable with IPFire. This saves about 40 MiB of compressed firmware space. Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org> --- config/rootfiles/common/linux-firmware | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/config/rootfiles/common/linux-firmware b/config/rootfiles/common/linux-firmware index 867c41e9c..0decb5646 100644 --- a/config/rootfiles/common/linux-firmware +++ b/config/rootfiles/common/linux-firmware @@ -1560,10 +1560,10 @@ lib/firmware/mrvl/pcie8897_uapsta.bin.xz lib/firmware/mrvl/pcie8997_wlan_v4.bin.xz lib/firmware/mrvl/pcieuart8997_combo_v4.bin.xz lib/firmware/mrvl/pcieusb8997_combo_v4.bin.xz -lib/firmware/mrvl/prestera -lib/firmware/mrvl/prestera/mvsw_prestera_fw-v2.0.img.xz -lib/firmware/mrvl/prestera/mvsw_prestera_fw-v3.0.img.xz -lib/firmware/mrvl/prestera/mvsw_prestera_fw-v4.0.img.xz +#lib/firmware/mrvl/prestera +#lib/firmware/mrvl/prestera/mvsw_prestera_fw-v2.0.img.xz +#lib/firmware/mrvl/prestera/mvsw_prestera_fw-v3.0.img.xz +#lib/firmware/mrvl/prestera/mvsw_prestera_fw-v4.0.img.xz lib/firmware/mrvl/sd8688.bin.xz lib/firmware/mrvl/sd8688_helper.bin.xz lib/firmware/mrvl/sd8787_uapsta.bin.xz -- 2.30.2

1 4

[PATCH 1/5] ids-functions.pl: Drop downloader code for sourcefire based ruleset.
by Stefan Schantl 28 Mar '22

28 Mar '22

Even if the servers do not support HEAD requests, the remote filesize (content_length) can be obtained from the connection headers. This generic method works for all servers and therefore we do not need the code for handle sourcefire servers in a different way anymore. Signed-off-by: Stefan Schantl <stefan.schantl(a)ipfire.org> --- config/cfgroot/ids-functions.pl | 43 +++++---------------------------- 1 file changed, 6 insertions(+), 37 deletions(-) diff --git a/config/cfgroot/ids-functions.pl b/config/cfgroot/ids-functions.pl index 94dccc8ae..eb276030b 100644 --- a/config/cfgroot/ids-functions.pl +++ b/config/cfgroot/ids-functions.pl @@ -354,43 +354,6 @@ sub downloadruleset ($) { return 1; } - # Variable to store the filesize of the remote object. - my $remote_filesize; - - # The sourcfire (snort rules) does not allow to send "HEAD" requests, so skip this check - # for this webserver. - # - # Check if the ruleset source contains "snort.org". - unless ($url =~ /\.snort\.org/) { - # Pass the requrested url to the downloader. - my $request = HTTP::Request->new(HEAD => $url); - - # Accept the html header. - $request->header('Accept' => 'text/html'); - - # Perform the request and fetch the html header. - my $response = $downloader->request($request); - - # Check if there was any error. - unless ($response->is_success) { - # Obtain error. - my $error = $response->status_line(); - - # Log error message. - &_log_to_syslog("Unable to download the ruleset. $$error$"); - - # Return "1" - false. - return 1; - } - - # Assign the fetched header object. - my $header = $response->headers(); - - # Grab the remote file size from the object and store it in the - # variable. - $remote_filesize = $header->content_length; - } - # Load perl module to deal with temporary files. use File::Temp; @@ -416,6 +379,12 @@ sub downloadruleset ($) { return 1; } + # Obtain the connection headers. + my $headers = $response->headers; + + # Get the remote size of the downloaded file. + my $remote_filesize = $headers->content_length; + # Load perl stat module. use File::stat; -- 2.30.2

2 11

[PATCH] monit: Update to 5.32.0
by Matthias Fischer 25 Mar '22

25 Mar '22

For details see: https://mmonit.com/monit/changes/ "Fixed: Issue #1028: If the Monit statefile was removed, the monit start <service> action for services with onreboot nostart option started the service, but did not enable monitoring of said service. The same problem occurred if a new onreboot nostart service was added, even if the statefile did exist. Fixed: Issue #1029: The generic protocol test truncated received data if the response contained zeros. Fixed: PAM authentication: Users with a valid password for a disabled account could still login to Monit. Thanks to Youssef Rebahi-Gilbert. Fixed: The Monit HTTP interface could be blocked by sending a request with an infinite stream of HTTP headers. Thanks to Youssef Rebahi-Gilbert for report." For more details see: https://bitbucket.org/tildeslash/monit/commits/tag/release-5-32-0 Signed-off-by: Matthias Fischer <matthias.fischer(a)ipfire.org> --- lfs/monit | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/lfs/monit b/lfs/monit index 8d4b6ced5..6af036adb 100644 --- a/lfs/monit +++ b/lfs/monit @@ -24,7 +24,7 @@ include Config -VER = 5.31.0 +VER = 5.32.0 SUMMARY = Utility for monitoring services on a Unix system THISAPP = monit-$(VER) @@ -33,7 +33,7 @@ DL_FROM = $(URL_IPFIRE) DIR_APP = $(DIR_SRC)/$(THISAPP) TARGET = $(DIR_INFO)/$(THISAPP) PROG = monit -PAK_VER = 18 +PAK_VER = 19 DEPS = @@ -47,7 +47,7 @@ objects = $(DL_FILE) $(DL_FILE) = $(DL_FROM)/$(DL_FILE) -$(DL_FILE)_MD5 = 7f2efa2175841573a96209086e4a5a26 +$(DL_FILE)_MD5 = 4e3e9f9dd4a5094101e1cd24a292a4fb install : $(TARGET) -- 2.25.1

1 0

[PATCH] Nano: Move nano editor from packages to core system
by Jon Murphy 24 Mar '22

24 Mar '22

- this will not change the default editor `vim` Signed-off-by: Jon Murphy <jon.murphy(a)ipfire.org> --- config/rootfiles/{packages => common}/nano | 0 lfs/nano | 7 +++---- 2 files changed, 3 insertions(+), 4 deletions(-) rename config/rootfiles/{packages => common}/nano (100%) diff --git a/config/rootfiles/packages/nano b/config/rootfiles/common/nano similarity index 100% rename from config/rootfiles/packages/nano rename to config/rootfiles/common/nano diff --git a/lfs/nano b/lfs/nano index df994364f..85be014ee 100644 --- a/lfs/nano +++ b/lfs/nano @@ -32,12 +32,11 @@ DL_FILE = $(THISAPP).tar.xz DL_FROM = $(URL_IPFIRE) DIR_APP = $(DIR_SRC)/$(THISAPP) TARGET = $(DIR_INFO)/$(THISAPP) -PROG = nano -PAK_VER = 41 -DEPS = "" -SERVICES = ############################################################################### # Top-level Rules -- 2.30.2

8 12

libloc 0.9.12 released
by Michael Tremer 24 Mar '22

24 Mar '22

Hello, This is the release announcement for libloc 0.9.12. It comes with the following changes: * The importer is now able to process any aggregated networks from Bird. This was not implemented before and is now allowing us to parse the global routing table in its entirety. * Various functions to parse and manipulate IP addresses have been moved into their own files (address.h and address.c). This allows us easier re-using the code in various places. * The list of bogons (subnets without a known country) is not generated correctly over the entire IP address ranges. For that, the internal algorithms that summarise and gaps between multiple networks has been refactored to be faster and not use any trial-and-error approach any more. * Various internal data structures have been optimised for performance on the expense of slightly more memory usage when writing the database file. Please refer to the Git change log for more details on this. * The dump of the database now contains all networks again. Thank you to everyone who has contributed to this release. You can find the source here: https://git.ipfire.org/?p=location/libloc.git;a=shortlog;h=refs/tags/0.9.12 https://source.ipfire.org/releases/libloc/libloc-0.9.12.tar.gz Best, -Michael

1 0

Re: [PATCH] linux-firmware: Compress firmware on disk
by Michael Tremer 24 Mar '22

24 Mar '22

Hello, So Peter has merged this patch now and it has resulted in an increase of about 70M of the ISO image. This is expected since we are now compressing everything individually which will give us a worse compression ratio. I assume that the updater would grow in a similar size which is probably acceptable to me. Since we suffer from bloated Core Updates, I wanted to double-check with everybody that this is okay for everybody else, too. Best, -Michael

2 3

[PATCH] Core Update 165: Restart Squid during the update
by Peter Müller 24 Mar '22

24 Mar '22

Fixes: #12810 Signed-off-by: Peter Müller <peter.mueller(a)ipfire.org> --- config/rootfiles/core/165/update.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/config/rootfiles/core/165/update.sh b/config/rootfiles/core/165/update.sh index ffb552c80..2c2e6a10c 100644 --- a/config/rootfiles/core/165/update.sh +++ b/config/rootfiles/core/165/update.sh @@ -128,6 +128,7 @@ ldconfig # Start services telinit u /etc/rc.d/init.d/firewall restart +/etc/rc.d/init.d/squid restart # Fix ownership of classification file. chown nobody:nobody /usr/share/suricata/classification.config -- 2.34.1

3 8

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

Development March 2022