* [RFC PATCH 1/2] unbound: Enable the Python module(s) @ 2025-04-25 12:37 Michael Tremer 2025-04-25 12:37 ` [RFC PATCH 2/2] Add Fast Flux detection in DNS Michael Tremer 2025-08-16 16:23 ` [RFC PATCH 1/2] unbound: Enable the Python module(s) Adolf Belka 0 siblings, 2 replies; 7+ messages in thread From: Michael Tremer @ 2025-04-25 12:37 UTC (permalink / raw) To: development; +Cc: Michael Tremer This can be used to script the resolver and use libunbound from Python. Signed-off-by: Michael Tremer <michael.tremer@ipfire.org> --- config/rootfiles/common/unbound | 4 ++++ lfs/unbound | 3 +++ make.sh | 2 +- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/config/rootfiles/common/unbound b/config/rootfiles/common/unbound index 57390d6d9..03c956503 100644 --- a/config/rootfiles/common/unbound +++ b/config/rootfiles/common/unbound @@ -13,6 +13,10 @@ etc/unbound/unbound.conf usr/lib/libunbound.so.8 usr/lib/libunbound.so.8.1.30 #usr/lib/pkgconfig/libunbound.pc +#usr/lib/python3.10/site-packages/_unbound.la +usr/lib/python3.10/site-packages/_unbound.so +usr/lib/python3.10/site-packages/unbound.py +usr/lib/python3.10/site-packages/unboundmodule.py usr/sbin/unbound usr/sbin/unbound-anchor usr/sbin/unbound-checkconf diff --git a/lfs/unbound b/lfs/unbound index 537ccff7e..8c79125ec 100644 --- a/lfs/unbound +++ b/lfs/unbound @@ -71,6 +71,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) @$(PREBUILD) @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE) cd $(DIR_APP) && \ + PYTHON_VERSION=3 \ ./configure \ --prefix=/usr \ --sysconfdir=/etc \ @@ -83,6 +84,8 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) --enable-tfo-server \ --enable-ipset \ --with-libmnl \ + --with-pythonmodule \ + --with-pyunbound \ ac_cv_func_getentropy=no cd $(DIR_APP) && make $(MAKETUNING) diff --git a/make.sh b/make.sh index 3643b469e..3a673cd2b 100755 --- a/make.sh +++ b/make.sh @@ -1534,6 +1534,7 @@ build_system() { lfsmake2 libevent2 lfsmake2 apr lfsmake2 aprutil + lfsmake2 swig lfsmake2 unbound lfsmake2 gnutls lfsmake2 libuv @@ -1930,7 +1931,6 @@ build_system() { lfsmake2 minicom lfsmake2 ddrescue lfsmake2 parted - lfsmake2 swig lfsmake2 python3-pyelftools lfsmake2 u-boot lfsmake2 wireless-regdb -- 2.39.5 ^ permalink raw reply [flat|nested] 7+ messages in thread
* [RFC PATCH 2/2] Add Fast Flux detection in DNS 2025-04-25 12:37 [RFC PATCH 1/2] unbound: Enable the Python module(s) Michael Tremer @ 2025-04-25 12:37 ` Michael Tremer 2025-08-16 16:23 ` [RFC PATCH 1/2] unbound: Enable the Python module(s) Adolf Belka 1 sibling, 0 replies; 7+ messages in thread From: Michael Tremer @ 2025-04-25 12:37 UTC (permalink / raw) To: development; +Cc: Michael Tremer This has been implemented because of a request on the forum. Since the proxy is generally outgoing technology it makes sense to enable this kind of filtering in DNS. This patch adds a Python script which processes every query and its response and extracts all IP addresses from it. Those IP addresses will then be resolved to their origin AS. If there are more then THRESHOLD different ASes, the request will be blocked. The AS lookups will only be performed when there is enough IP addresses to actually hit the threshold. So there should be next to no performance impact here except the overhead of the Python module itself. Signed-off-by: Michael Tremer <michael.tremer@ipfire.org> --- config/rootfiles/common/unbound | 1 + config/unbound/fastflux-detection.py | 167 +++++++++++++++++++++++++++ config/unbound/unbound.conf | 7 ++ doc/language_issues.en | 1 + doc/language_issues.es | 1 + doc/language_issues.fr | 1 + doc/language_issues.it | 1 + doc/language_issues.nl | 1 + doc/language_issues.pl | 1 + doc/language_issues.ru | 1 + doc/language_issues.tr | 1 + doc/language_missings | 7 ++ html/cgi-bin/dns.cgi | 21 ++++ langs/de/cgi-bin/de.pl | 1 + langs/en/cgi-bin/en.pl | 1 + lfs/unbound | 4 + 16 files changed, 217 insertions(+) create mode 100644 config/unbound/fastflux-detection.py diff --git a/config/rootfiles/common/unbound b/config/rootfiles/common/unbound index 03c956503..9ca84d2d9 100644 --- a/config/rootfiles/common/unbound +++ b/config/rootfiles/common/unbound @@ -1,6 +1,7 @@ etc/rc.d/init.d/unbound #etc/unbound etc/unbound/dhcp-leases.conf +etc/unbound/fastflux-detection.py etc/unbound/forward.conf etc/unbound/icannbundle.pem etc/unbound/local.d diff --git a/config/unbound/fastflux-detection.py b/config/unbound/fastflux-detection.py new file mode 100644 index 000000000..4ad62eed6 --- /dev/null +++ b/config/unbound/fastflux-detection.py @@ -0,0 +1,167 @@ +############################################################################### +# # +# Pakfire - The IPFire package management system # +# Copyright (C) 2025 IPFire Development Team # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see <http://www.gnu.org/licenses/>. # +# # +############################################################################### + +import datetime +import ipaddress +import location +import socket + +DEFAULT_THRESHOLD = 5 + +def read_config(path): + """ + Opens the configuration file and reads it line by line + """ + config = {} + + with open(path) as f: + for line in f: + # Remove any trailing newline + line = line.rstrip() + + # Split by key and value + key, _, val = line.partition("=") + + # Store the line + config[key] = val + + return config + +def init(id, cfg): + global db + global ENABLED + global THRESHOLD + + # Read the configuration + config = read_config("/var/ipfire/dns/settings") + + # Is this module enabled? + ENABLED = config.get("FF_DETECTION", "false") in ("on", "true", "1") + + # Fetch the treshold + if ENABLED: + threshold = config.get("FF_THRESHOLD", 5) + + try: + THRESHOLD = int(threshold) + except (TypeError, ValueError): + log_warning("Failed to parse Fast Flux threshold '%s'." + " Using default of %s" % (threshold, DEFAULT_THRESHOLD)) + THRESHOLD = DEFAULT_THRESHOLD + + # Open the location database + try: + db = location.open() + + # Fail if we could not open the database + except Exception as e: + log_error("Failed to open the location database: %s" % e) + return False + + log_info("Opened Location database") + log_info(" Database Vendor : %s" % db.vendor) + log_info(" Created At : %s" % datetime.datetime.fromtimestamp(db.created_at)) + + # Done! + log_info("FastFlux detection module loaded") + + return True + +def deinit(id): + log_info("FastFlux detection module unloaded") + return True + +def inform_super(id, qstate, superqstate, qdata): + return True + +def operate(id, event, qstate, qdata): + # Execute when everything else is done + if event == MODULE_EVENT_MODDONE: + # Do nothing if this is not enabled + if not ENABLED: + qstate.ext_state[id] = MODULE_FINISHED + return True + + # Extract the qname + qname = qstate.qinfo.qname_str + + # Deny access to the qname? + deny = False + + # Extract the response + rrset = qstate.return_msg.rep.rrsets + + addrs = set() + + # Find all IP addresses in the response + for i in range(qstate.return_msg.rep.rrset_count): + rr = rrset[i] + + # Extract the type + type = socket.ntohs(rr.rk.type) + + # Only process types A and AAAA + if type in (1, 28): + for i in range(rr.entry.data.count): + payload = rr.entry.data.rr_data[i] + + # Parse the IP address + if type == 1: + addr = ipaddress.IPv4Address(payload[2:]) + elif type == 28: + addr = ipaddress.IPv6Address(payload[2:]) + + addrs.add(addr) + + # Only perform any further action if we have at least as many as threshold IP addresses + if len(addrs) >= THRESHOLD: + asns = set() + + # Look up the networks for all addresses + for addr in addrs: + network = db.lookup("%s" % addr) + + # If no network could be found, we add zero to represent an unknown value + asns.add(network.asn if network else 0) + + # Check for selective announements + if 0 in asns: + log_info("Denying access to %s due to suspected selective announcements" % qname) + deny = True + + # Check if the threshold was exceeded + elif len(asns) >= THRESHOLD: + log_info("Denying access to %s due to suspected Fast Flux announcement" % qname) + deny = True + + # Return SERVFAIL? + # XXX It would be nice to send an extended DNS error here (e.g. BLOCKED), but it + # seems that this is currently not supported in the Python module. + if deny: + qstate.ext_state[id] = MODULE_ERROR + return True + + # Otherwise, continue + qstate.ext_state[id] = MODULE_FINISHED + return True + + # Not handling other events + qstate.ext_state[id] = MODULE_WAIT_MODULE + return True diff --git a/config/unbound/unbound.conf b/config/unbound/unbound.conf index 012beab54..b1f2871c4 100644 --- a/config/unbound/unbound.conf +++ b/config/unbound/unbound.conf @@ -12,6 +12,9 @@ server: username: "nobody" do-ip6: no + # Load modules + module-config: "validator python iterator" + # System Tuning include: "/etc/unbound/tuning.conf" @@ -68,6 +71,10 @@ server: # Include any forward zones include: "/etc/unbound/forward.conf" +python: + # Enable Fast Flux Detection + python-script: "/etc/unbound/fastflux-detection.py" + remote-control: control-enable: yes control-use-cert: no diff --git a/doc/language_issues.en b/doc/language_issues.en index 1c1c546f7..a5860a153 100644 --- a/doc/language_issues.en +++ b/doc/language_issues.en @@ -611,6 +611,7 @@ WARNING: untranslated string: dnat address = Firewall Interface WARNING: untranslated string: dns check failed = DNS check failed WARNING: untranslated string: dns check servers = Check DNS Servers WARNING: untranslated string: dns configuration = DNS Configuration +WARNING: untranslated string: dns enable fast flux detection = Fast Flux Detection WARNING: untranslated string: dns enable safe-search = Enable Safe Search WARNING: untranslated string: dns enable safe-search youtube = Include YouTube in Safe Search WARNING: untranslated string: dns forward disable dnssec = Disable DNSSEC (dangerous) diff --git a/doc/language_issues.es b/doc/language_issues.es index cf7237435..2b7937842 100644 --- a/doc/language_issues.es +++ b/doc/language_issues.es @@ -1015,6 +1015,7 @@ WARNING: untranslated string: ca name must only contain characters and spaces = WARNING: untranslated string: cpu frequency = CPU frequency WARNING: untranslated string: data transfer = Data Transfer WARNING: untranslated string: dhcp fixed ip address in dynamic range = Fixed IP Address in dynamic range +WARNING: untranslated string: dns enable fast flux detection = Fast Flux Detection WARNING: untranslated string: dns servers = DNS Servers WARNING: untranslated string: done = Done WARNING: untranslated string: downfall gather data sampling = Downfall/Gather Data Sampling diff --git a/doc/language_issues.fr b/doc/language_issues.fr index 702911061..b9703cd4d 100644 --- a/doc/language_issues.fr +++ b/doc/language_issues.fr @@ -979,6 +979,7 @@ WARNING: untranslated string: bypassed = Bypassed WARNING: untranslated string: ca name must only contain characters and spaces = unknown string WARNING: untranslated string: core notice 3 = available. WARNING: untranslated string: data transfer = Data Transfer +WARNING: untranslated string: dns enable fast flux detection = Fast Flux Detection WARNING: untranslated string: done = Done WARNING: untranslated string: enable disable client = unknown string WARNING: untranslated string: enable disable dyndns = unknown string diff --git a/doc/language_issues.it b/doc/language_issues.it index 3d93239af..c5297c241 100644 --- a/doc/language_issues.it +++ b/doc/language_issues.it @@ -1019,6 +1019,7 @@ WARNING: untranslated string: disconnected = Disconnected WARNING: untranslated string: dl client arch insecure = Download insecure Client Package (zip) WARNING: untranslated string: dns check servers = Check DNS Servers WARNING: untranslated string: dns configuration = DNS Configuration +WARNING: untranslated string: dns enable fast flux detection = Fast Flux Detection WARNING: untranslated string: dns enable safe-search = Enable Safe Search WARNING: untranslated string: dns enable safe-search youtube = Include YouTube in Safe Search WARNING: untranslated string: dns forward disable dnssec = Disable DNSSEC (dangerous) diff --git a/doc/language_issues.nl b/doc/language_issues.nl index f1090fc33..7f16de518 100644 --- a/doc/language_issues.nl +++ b/doc/language_issues.nl @@ -1019,6 +1019,7 @@ WARNING: untranslated string: disable = Disable WARNING: untranslated string: disconnected = Disconnected WARNING: untranslated string: dns check servers = Check DNS Servers WARNING: untranslated string: dns configuration = DNS Configuration +WARNING: untranslated string: dns enable fast flux detection = Fast Flux Detection WARNING: untranslated string: dns enable safe-search = Enable Safe Search WARNING: untranslated string: dns enable safe-search youtube = Include YouTube in Safe Search WARNING: untranslated string: dns forward disable dnssec = Disable DNSSEC (dangerous) diff --git a/doc/language_issues.pl b/doc/language_issues.pl index 1db36fb67..063045ef3 100644 --- a/doc/language_issues.pl +++ b/doc/language_issues.pl @@ -991,6 +991,7 @@ WARNING: untranslated string: dl client arch insecure = Download insecure Client WARNING: untranslated string: dnat address = Firewall Interface WARNING: untranslated string: dns check servers = Check DNS Servers WARNING: untranslated string: dns configuration = DNS Configuration +WARNING: untranslated string: dns enable fast flux detection = Fast Flux Detection WARNING: untranslated string: dns enable safe-search = Enable Safe Search WARNING: untranslated string: dns enable safe-search youtube = Include YouTube in Safe Search WARNING: untranslated string: dns forward disable dnssec = Disable DNSSEC (dangerous) diff --git a/doc/language_issues.ru b/doc/language_issues.ru index 4d29c4f95..5c85a2c40 100644 --- a/doc/language_issues.ru +++ b/doc/language_issues.ru @@ -986,6 +986,7 @@ WARNING: untranslated string: dl client arch insecure = Download insecure Client WARNING: untranslated string: dnat address = Firewall Interface WARNING: untranslated string: dns check servers = Check DNS Servers WARNING: untranslated string: dns configuration = DNS Configuration +WARNING: untranslated string: dns enable fast flux detection = Fast Flux Detection WARNING: untranslated string: dns enable safe-search = Enable Safe Search WARNING: untranslated string: dns enable safe-search youtube = Include YouTube in Safe Search WARNING: untranslated string: dns forward disable dnssec = Disable DNSSEC (dangerous) diff --git a/doc/language_issues.tr b/doc/language_issues.tr index 2da19f276..473250445 100644 --- a/doc/language_issues.tr +++ b/doc/language_issues.tr @@ -998,6 +998,7 @@ WARNING: untranslated string: disable = Disable WARNING: untranslated string: disconnected = Disconnected WARNING: untranslated string: dns check servers = Check DNS Servers WARNING: untranslated string: dns configuration = DNS Configuration +WARNING: untranslated string: dns enable fast flux detection = Fast Flux Detection WARNING: untranslated string: dns enable safe-search = Enable Safe Search WARNING: untranslated string: dns enable safe-search youtube = Include YouTube in Safe Search WARNING: untranslated string: dns forward disable dnssec = Disable DNSSEC (dangerous) diff --git a/doc/language_missings b/doc/language_missings index 48b98ce74..f5c0949b5 100644 --- a/doc/language_missings +++ b/doc/language_missings @@ -173,6 +173,7 @@ < cpu frequency < data transfer < dhcp fixed ip address in dynamic range +< dns enable fast flux detection < dns servers < done < downfall gather data sampling @@ -287,6 +288,7 @@ < bypassed < ca name must only contain characters or spaces < data transfer +< dns enable fast flux detection < done < endpoint < endpoint address @@ -509,6 +511,7 @@ < dns check servers < dns configuration < dns could not add server +< dns enable fast flux detection < dns enable safe-search < dns enable safe-search youtube < dns forward disable dnssec @@ -1110,6 +1113,7 @@ < dns check servers < dns configuration < dns could not add server +< dns enable fast flux detection < dns enable safe-search < dns enable safe-search youtube < dns forward disable dnssec @@ -1807,6 +1811,7 @@ < dns check servers < dns configuration < dns could not add server +< dns enable fast flux detection < dns enable safe-search < dns enable safe-search youtube < dnsforward @@ -2884,6 +2889,7 @@ < dns check servers < dns configuration < dns could not add server +< dns enable fast flux detection < dns enable safe-search < dns enable safe-search youtube < dnsforward @@ -3823,6 +3829,7 @@ < dns check servers < dns configuration < dns could not add server +< dns enable fast flux detection < dns enable safe-search < dns enable safe-search youtube < dns forward disable dnssec diff --git a/html/cgi-bin/dns.cgi b/html/cgi-bin/dns.cgi index 0d3b14797..8cc39f97c 100644 --- a/html/cgi-bin/dns.cgi +++ b/html/cgi-bin/dns.cgi @@ -82,6 +82,11 @@ if ($cgiparams{'GENERAL'} eq $Lang::tr{'save'}) { $cgiparams{'USE_ISP_NAMESERVERS'} = "off"; } + # Add value for non-checked checkbox. + if ($cgiparams{'FF_DETECTION'} ne "on") { + $cgiparams{'FF_DETECTION'} = "off"; + } + # Add value for non-checked checkbox. if ($cgiparams{'ENABLE_SAFE_SEARCH'} ne "on") { $cgiparams{'ENABLE_SAFE_SEARCH'} = "off"; @@ -264,6 +269,7 @@ if (($cgiparams{'SERVERS'} eq $Lang::tr{'save'}) || ($cgiparams{'SERVERS'} eq $L # Hash to store the generic DNS settings. my %settings = (); $settings{"ENABLE_SAFE_SEARCH_YOUTUBE"} = "on"; +$settings{"FF_DETECTION"} = "on"; # Read-in general DNS settings. &General::readhash("$settings_file", \%settings); @@ -311,6 +317,10 @@ $checked{'USE_ISP_NAMESERVERS'}{'off'} = ''; $checked{'USE_ISP_NAMESERVERS'}{'on'} = ''; $checked{'USE_ISP_NAMESERVERS'}{$settings{'USE_ISP_NAMESERVERS'}} = "checked='checked'"; +$checked{'FF_DETECTION'}{'off'} = ''; +$checked{'FF_DETECTION'}{'on'} = ''; +$checked{'FF_DETECTION'}{$settings{'FF_DETECTION'}} = "checked='checked'"; + $checked{'ENABLE_SAFE_SEARCH'}{'off'} = ''; $checked{'ENABLE_SAFE_SEARCH'}{'on'} = ''; $checked{'ENABLE_SAFE_SEARCH'}{$settings{'ENABLE_SAFE_SEARCH'}} = "checked='checked'"; @@ -380,6 +390,17 @@ sub show_general_dns_configuration () { </td> </tr> + <tr> + <td width="33%"> + $Lang::tr{'dns enable fast flux detection'} + </td> + + <td> + <input type="checkbox" name="FF_DETECTION" $checked{'FF_DETECTION'}{'on'}> + </td> + </tr> + + <tr> <td width="33%"> $Lang::tr{'dns enable safe-search'} diff --git a/langs/de/cgi-bin/de.pl b/langs/de/cgi-bin/de.pl index 3ce02b657..aafc180a6 100644 --- a/langs/de/cgi-bin/de.pl +++ b/langs/de/cgi-bin/de.pl @@ -835,6 +835,7 @@ 'dns check servers' => 'DNS-Server prüfen', 'dns configuration' => 'DNS-Konfiguration', 'dns desc' => 'Wenn auf Schnittstelle red0 die IP-Adressinformationen über DHCP vom Provider kommen, werden automatisch die DNS-Server-Adressen des Providers gesetzt. Hier können Sie nun diese mit den eigenen DNS-Server-IP-Adressen überschreiben.', +'dns enable fast flux detection' => 'Fast-Flux-Erkennung', 'dns enable safe-search' => 'Safe Search via DNS aktivieren', 'dns enable safe-search youtube' => 'YouTube in Safe Search einbeziehen', 'dns error 0' => 'Die IP Adresse vom <strong>primären</strong> DNS Server ist nicht gültig, bitte überprüfen Sie Ihre Eingabe!<br />Die eingegebene <strong>sekundären</strong> DNS Server Adresse ist jedoch gültig.<br />', diff --git a/langs/en/cgi-bin/en.pl b/langs/en/cgi-bin/en.pl index 3e647e6e5..fb82fcd08 100644 --- a/langs/en/cgi-bin/en.pl +++ b/langs/en/cgi-bin/en.pl @@ -880,6 +880,7 @@ 'dns configuration' => 'DNS Configuration', 'dns could not add server' => 'Could not add server - Reason:', 'dns desc' => 'If the red0 interface gets the IP address information via DHCP from the provider, the DNS server addresses will be set automatically. Now here you are able to change these DNS server IP addresses with your own ones.', +'dns enable fast flux detection' => 'Fast Flux Detection', 'dns enable safe-search' => 'Enable Safe Search', 'dns enable safe-search youtube' => 'Include YouTube in Safe Search', 'dns error 0' => 'The IP address of the <strong>primary</strong> DNS server is not valid, please check your entries!<br />The entered <strong>secondary</strong> DNS server address is valid.', diff --git a/lfs/unbound b/lfs/unbound index 8c79125ec..00105b691 100644 --- a/lfs/unbound +++ b/lfs/unbound @@ -97,6 +97,10 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) touch /etc/unbound/{dhcp-leases,forward}.conf -mkdir -pv /etc/unbound/local.d + # Install Python scripts + install -v -m 644 $(DIR_SRC)/config/unbound/fastflux-detection.py \ + /etc/unbound/fastflux-detection.py + # Install root hints install -v -m 644 $(DIR_SRC)/config/unbound/root.hints \ /etc/unbound/root.hints -- 2.39.5 ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC PATCH 1/2] unbound: Enable the Python module(s) 2025-04-25 12:37 [RFC PATCH 1/2] unbound: Enable the Python module(s) Michael Tremer 2025-04-25 12:37 ` [RFC PATCH 2/2] Add Fast Flux detection in DNS Michael Tremer @ 2025-08-16 16:23 ` Adolf Belka 2025-08-18 11:50 ` Michael Tremer 1 sibling, 1 reply; 7+ messages in thread From: Adolf Belka @ 2025-08-16 16:23 UTC (permalink / raw) To: Michael Tremer; +Cc: IPFire: Development-List Hi Michael, After some time I thought I should have a go at building with your two patches for the unbound fast flux option but the build fails to complete. I am doing the build with the current next repo. I have done the build twice and it fails at the same point each time at the python3-tomli build. The following is the log output, which I have not been able to interpret to identify what is causing the build to fail. Aug 16 15:11:49: Building python3-tomli make: Entering directory '/home/ahb/sandbox/ipfire-2-builder3/lfs' make: Nothing to be done for 'download'. make: Leaving directory '/home/ahb/sandbox/ipfire-2-builder3/lfs' make: Entering directory '/usr/src/lfs' tomli-2.0.1.tar.gz checksum OK ====================================== Installing tomli-2.0.1 ... Install started; saving file list to /usr/src/lsalr ... cd /usr/src/tomli-2.0.1 && python3 -m pyproject2setuppy.main build Traceback (most recent call last): File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/usr/lib/python3.10/site-packages/pyproject2setuppy/main.py", line 15, in <module> main() File "/usr/lib/python3.10/site-packages/pyproject2setuppy/__main__.py", line 53, in main handler(data) File "/usr/lib/python3.10/site-packages/pyproject2setuppy/flit.py", line 91, in handle_flit setup(**setup_metadata) File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 86, in setup _install_setup_requires(attrs) File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires dist = MinimalDistribution(attrs) File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 57, in __init__ super().__init__(filtered) File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 474, in __init__ for ep in metadata.entry_points(group='distutils.setup_keywords'): File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1009, in entry_points return SelectableGroups.load(eps).select(**params) File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 459, in load ordered = sorted(eps, key=by_group) File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1006, in <genexpr> eps = itertools.chain.from_iterable( File "/usr/lib/python3.10/importlib/metadata/_itertools.py", line 16, in unique_everseen k = key(element) File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 941, in _normalized_name return self._name_from_stem(stem) or super()._normalized_name File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 622, in _normalized_name return Prepared.normalize(self.name) File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 871, in normalize return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_') File "/usr/lib/python3.10/re.py", line 209, in sub return _compile(pattern, flags).sub(repl, string, count) TypeError: expected string or bytes-like object make: *** [python3-tomli:78: /usr/src/log/tomli-2.0.1] Error 1 make: Leaving directory '/usr/src/lfs' Regards, Adolf. On 25/04/2025 14:37, Michael Tremer wrote: > This can be used to script the resolver and use libunbound from Python. > > Signed-off-by: Michael Tremer <michael.tremer@ipfire.org> > --- > config/rootfiles/common/unbound | 4 ++++ > lfs/unbound | 3 +++ > make.sh | 2 +- > 3 files changed, 8 insertions(+), 1 deletion(-) > > diff --git a/config/rootfiles/common/unbound b/config/rootfiles/common/unbound > index 57390d6d9..03c956503 100644 > --- a/config/rootfiles/common/unbound > +++ b/config/rootfiles/common/unbound > @@ -13,6 +13,10 @@ etc/unbound/unbound.conf > usr/lib/libunbound.so.8 > usr/lib/libunbound.so.8.1.30 > #usr/lib/pkgconfig/libunbound.pc > +#usr/lib/python3.10/site-packages/_unbound.la > +usr/lib/python3.10/site-packages/_unbound.so > +usr/lib/python3.10/site-packages/unbound.py > +usr/lib/python3.10/site-packages/unboundmodule.py > usr/sbin/unbound > usr/sbin/unbound-anchor > usr/sbin/unbound-checkconf > diff --git a/lfs/unbound b/lfs/unbound > index 537ccff7e..8c79125ec 100644 > --- a/lfs/unbound > +++ b/lfs/unbound > @@ -71,6 +71,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) > @$(PREBUILD) > @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE) > cd $(DIR_APP) && \ > + PYTHON_VERSION=3 \ > ./configure \ > --prefix=/usr \ > --sysconfdir=/etc \ > @@ -83,6 +84,8 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) > --enable-tfo-server \ > --enable-ipset \ > --with-libmnl \ > + --with-pythonmodule \ > + --with-pyunbound \ > ac_cv_func_getentropy=no > > cd $(DIR_APP) && make $(MAKETUNING) > diff --git a/make.sh b/make.sh > index 3643b469e..3a673cd2b 100755 > --- a/make.sh > +++ b/make.sh > @@ -1534,6 +1534,7 @@ build_system() { > lfsmake2 libevent2 > lfsmake2 apr > lfsmake2 aprutil > + lfsmake2 swig > lfsmake2 unbound > lfsmake2 gnutls > lfsmake2 libuv > @@ -1930,7 +1931,6 @@ build_system() { > lfsmake2 minicom > lfsmake2 ddrescue > lfsmake2 parted > - lfsmake2 swig > lfsmake2 python3-pyelftools > lfsmake2 u-boot > lfsmake2 wireless-regdb ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC PATCH 1/2] unbound: Enable the Python module(s) 2025-08-16 16:23 ` [RFC PATCH 1/2] unbound: Enable the Python module(s) Adolf Belka @ 2025-08-18 11:50 ` Michael Tremer 2025-08-18 12:40 ` Adolf Belka 0 siblings, 1 reply; 7+ messages in thread From: Michael Tremer @ 2025-08-18 11:50 UTC (permalink / raw) To: Adolf Belka; +Cc: IPFire: Development-List Hello, I have the same problem. I have built this from a non-clean build initially and everything was fine. I used that version of Unbound to write and test the code. No problems there whatsoever. But I never managed to run a clean build. It seems that swig messes something up and I cannot tell you at all what that could be. It is not only tomli not compiling. If you comment it out and try to compile the next thing it won’t work either. I don’t know what it could break like this because the problems suddenly seem to occur in the Python standard library. Maybe an update of Swig solves the problem? Would you like to give that a try? -Michael > On 16 Aug 2025, at 17:23, Adolf Belka <adolf.belka@ipfire.org> wrote: > > Hi Michael, > > After some time I thought I should have a go at building with your two patches for the unbound fast flux option but the build fails to complete. > > I am doing the build with the current next repo. > > I have done the build twice and it fails at the same point each time at the python3-tomli build. > > The following is the log output, which I have not been able to interpret to identify what is causing the build to fail. > > > Aug 16 15:11:49: Building python3-tomli make: Entering directory '/home/ahb/sandbox/ipfire-2-builder3/lfs' > make: Nothing to be done for 'download'. > make: Leaving directory '/home/ahb/sandbox/ipfire-2-builder3/lfs' > make: Entering directory '/usr/src/lfs' > tomli-2.0.1.tar.gz checksum OK > ====================================== Installing tomli-2.0.1 ... > Install started; saving file list to /usr/src/lsalr ... > cd /usr/src/tomli-2.0.1 && python3 -m pyproject2setuppy.main build > Traceback (most recent call last): > File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main > return _run_code(code, main_globals, None, > File "/usr/lib/python3.10/runpy.py", line 86, in _run_code > exec(code, run_globals) > File "/usr/lib/python3.10/site-packages/pyproject2setuppy/main.py", line 15, in <module> > main() > File "/usr/lib/python3.10/site-packages/pyproject2setuppy/__main__.py", line 53, in main > handler(data) > File "/usr/lib/python3.10/site-packages/pyproject2setuppy/flit.py", line 91, in handle_flit > setup(**setup_metadata) > File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 86, in setup > _install_setup_requires(attrs) > File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires > dist = MinimalDistribution(attrs) > File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 57, in __init__ > super().__init__(filtered) > File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 474, in __init__ > for ep in metadata.entry_points(group='distutils.setup_keywords'): > File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1009, in entry_points > return SelectableGroups.load(eps).select(**params) > File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 459, in load > ordered = sorted(eps, key=by_group) > File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1006, in <genexpr> > eps = itertools.chain.from_iterable( > File "/usr/lib/python3.10/importlib/metadata/_itertools.py", line 16, in unique_everseen > k = key(element) > File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 941, in _normalized_name > return self._name_from_stem(stem) or super()._normalized_name > File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 622, in _normalized_name > return Prepared.normalize(self.name) > File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 871, in normalize > return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_') > File "/usr/lib/python3.10/re.py", line 209, in sub > return _compile(pattern, flags).sub(repl, string, count) > TypeError: expected string or bytes-like object > make: *** [python3-tomli:78: /usr/src/log/tomli-2.0.1] Error 1 > make: Leaving directory '/usr/src/lfs' > > Regards, > > Adolf. > > On 25/04/2025 14:37, Michael Tremer wrote: >> This can be used to script the resolver and use libunbound from Python. >> Signed-off-by: Michael Tremer <michael.tremer@ipfire.org> >> --- >> config/rootfiles/common/unbound | 4 ++++ >> lfs/unbound | 3 +++ >> make.sh | 2 +- >> 3 files changed, 8 insertions(+), 1 deletion(-) >> diff --git a/config/rootfiles/common/unbound b/config/rootfiles/common/unbound >> index 57390d6d9..03c956503 100644 >> --- a/config/rootfiles/common/unbound >> +++ b/config/rootfiles/common/unbound >> @@ -13,6 +13,10 @@ etc/unbound/unbound.conf >> usr/lib/libunbound.so.8 >> usr/lib/libunbound.so.8.1.30 >> #usr/lib/pkgconfig/libunbound.pc >> +#usr/lib/python3.10/site-packages/_unbound.la >> +usr/lib/python3.10/site-packages/_unbound.so >> +usr/lib/python3.10/site-packages/unbound.py >> +usr/lib/python3.10/site-packages/unboundmodule.py >> usr/sbin/unbound >> usr/sbin/unbound-anchor >> usr/sbin/unbound-checkconf >> diff --git a/lfs/unbound b/lfs/unbound >> index 537ccff7e..8c79125ec 100644 >> --- a/lfs/unbound >> +++ b/lfs/unbound >> @@ -71,6 +71,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) >> @$(PREBUILD) >> @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE) >> cd $(DIR_APP) && \ >> + PYTHON_VERSION=3 \ >> ./configure \ >> --prefix=/usr \ >> --sysconfdir=/etc \ >> @@ -83,6 +84,8 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) >> --enable-tfo-server \ >> --enable-ipset \ >> --with-libmnl \ >> + --with-pythonmodule \ >> + --with-pyunbound \ >> ac_cv_func_getentropy=no >> cd $(DIR_APP) && make $(MAKETUNING) >> diff --git a/make.sh b/make.sh >> index 3643b469e..3a673cd2b 100755 >> --- a/make.sh >> +++ b/make.sh >> @@ -1534,6 +1534,7 @@ build_system() { >> lfsmake2 libevent2 >> lfsmake2 apr >> lfsmake2 aprutil >> + lfsmake2 swig >> lfsmake2 unbound >> lfsmake2 gnutls >> lfsmake2 libuv >> @@ -1930,7 +1931,6 @@ build_system() { >> lfsmake2 minicom >> lfsmake2 ddrescue >> lfsmake2 parted >> - lfsmake2 swig >> lfsmake2 python3-pyelftools >> lfsmake2 u-boot >> lfsmake2 wireless-regdb > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC PATCH 1/2] unbound: Enable the Python module(s) 2025-08-18 11:50 ` Michael Tremer @ 2025-08-18 12:40 ` Adolf Belka 2025-08-18 16:17 ` Adolf Belka 0 siblings, 1 reply; 7+ messages in thread From: Adolf Belka @ 2025-08-18 12:40 UTC (permalink / raw) To: Michael Tremer; +Cc: IPFire: Development-List Hi Michael, On 18/08/2025 13:50, Michael Tremer wrote: > Hello, > > I have the same problem. > > I have built this from a non-clean build initially and everything was fine. I used that version of Unbound to write and test the code. No problems there whatsoever. > > But I never managed to run a clean build. It seems that swig messes something up and I cannot tell you at all what that could be. It is not only tomli not compiling. If you comment it out and try to compile the next thing it won’t work either. I don’t know what it could break like this because the problems suddenly seem to occur in the Python standard library. > > Maybe an update of Swig solves the problem? Would you like to give that a try? The last swig update I provided had to be reverted because uboot would not build with it. However there is a new version out now, 4.3.1 from 2025-04-15, so I will give that a go and see if it helps and also do an aarch64 build and see if uboot builds or not. If it doesn't help or uboot fails to build then I will look at what other packages might need to be moved earlier to be before swig so that all the expected swig bits are built. Currently we have swig-4.1.0 and the reverted version was 4.3.0, so there are also other versions newer than the existing one that might allow uboot to build and help the issue we are having. Will have a go at it, although might take a while as I am cleaning my house up to make it child proof as my nephew and family are visiting from end of this week for 5 days. Regards, Adolf. > > -Michael > >> On 16 Aug 2025, at 17:23, Adolf Belka <adolf.belka@ipfire.org> wrote: >> >> Hi Michael, >> >> After some time I thought I should have a go at building with your two patches for the unbound fast flux option but the build fails to complete. >> >> I am doing the build with the current next repo. >> >> I have done the build twice and it fails at the same point each time at the python3-tomli build. >> >> The following is the log output, which I have not been able to interpret to identify what is causing the build to fail. >> >> >> Aug 16 15:11:49: Building python3-tomli make: Entering directory '/home/ahb/sandbox/ipfire-2-builder3/lfs' >> make: Nothing to be done for 'download'. >> make: Leaving directory '/home/ahb/sandbox/ipfire-2-builder3/lfs' >> make: Entering directory '/usr/src/lfs' >> tomli-2.0.1.tar.gz checksum OK >> ====================================== Installing tomli-2.0.1 ... >> Install started; saving file list to /usr/src/lsalr ... >> cd /usr/src/tomli-2.0.1 && python3 -m pyproject2setuppy.main build >> Traceback (most recent call last): >> File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main >> return _run_code(code, main_globals, None, >> File "/usr/lib/python3.10/runpy.py", line 86, in _run_code >> exec(code, run_globals) >> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/main.py", line 15, in <module> >> main() >> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/__main__.py", line 53, in main >> handler(data) >> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/flit.py", line 91, in handle_flit >> setup(**setup_metadata) >> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 86, in setup >> _install_setup_requires(attrs) >> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires >> dist = MinimalDistribution(attrs) >> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 57, in __init__ >> super().__init__(filtered) >> File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 474, in __init__ >> for ep in metadata.entry_points(group='distutils.setup_keywords'): >> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1009, in entry_points >> return SelectableGroups.load(eps).select(**params) >> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 459, in load >> ordered = sorted(eps, key=by_group) >> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1006, in <genexpr> >> eps = itertools.chain.from_iterable( >> File "/usr/lib/python3.10/importlib/metadata/_itertools.py", line 16, in unique_everseen >> k = key(element) >> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 941, in _normalized_name >> return self._name_from_stem(stem) or super()._normalized_name >> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 622, in _normalized_name >> return Prepared.normalize(self.name) >> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 871, in normalize >> return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_') >> File "/usr/lib/python3.10/re.py", line 209, in sub >> return _compile(pattern, flags).sub(repl, string, count) >> TypeError: expected string or bytes-like object >> make: *** [python3-tomli:78: /usr/src/log/tomli-2.0.1] Error 1 >> make: Leaving directory '/usr/src/lfs' >> >> Regards, >> >> Adolf. >> >> On 25/04/2025 14:37, Michael Tremer wrote: >>> This can be used to script the resolver and use libunbound from Python. >>> Signed-off-by: Michael Tremer <michael.tremer@ipfire.org> >>> --- >>> config/rootfiles/common/unbound | 4 ++++ >>> lfs/unbound | 3 +++ >>> make.sh | 2 +- >>> 3 files changed, 8 insertions(+), 1 deletion(-) >>> diff --git a/config/rootfiles/common/unbound b/config/rootfiles/common/unbound >>> index 57390d6d9..03c956503 100644 >>> --- a/config/rootfiles/common/unbound >>> +++ b/config/rootfiles/common/unbound >>> @@ -13,6 +13,10 @@ etc/unbound/unbound.conf >>> usr/lib/libunbound.so.8 >>> usr/lib/libunbound.so.8.1.30 >>> #usr/lib/pkgconfig/libunbound.pc >>> +#usr/lib/python3.10/site-packages/_unbound.la >>> +usr/lib/python3.10/site-packages/_unbound.so >>> +usr/lib/python3.10/site-packages/unbound.py >>> +usr/lib/python3.10/site-packages/unboundmodule.py >>> usr/sbin/unbound >>> usr/sbin/unbound-anchor >>> usr/sbin/unbound-checkconf >>> diff --git a/lfs/unbound b/lfs/unbound >>> index 537ccff7e..8c79125ec 100644 >>> --- a/lfs/unbound >>> +++ b/lfs/unbound >>> @@ -71,6 +71,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) >>> @$(PREBUILD) >>> @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE) >>> cd $(DIR_APP) && \ >>> + PYTHON_VERSION=3 \ >>> ./configure \ >>> --prefix=/usr \ >>> --sysconfdir=/etc \ >>> @@ -83,6 +84,8 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) >>> --enable-tfo-server \ >>> --enable-ipset \ >>> --with-libmnl \ >>> + --with-pythonmodule \ >>> + --with-pyunbound \ >>> ac_cv_func_getentropy=no >>> cd $(DIR_APP) && make $(MAKETUNING) >>> diff --git a/make.sh b/make.sh >>> index 3643b469e..3a673cd2b 100755 >>> --- a/make.sh >>> +++ b/make.sh >>> @@ -1534,6 +1534,7 @@ build_system() { >>> lfsmake2 libevent2 >>> lfsmake2 apr >>> lfsmake2 aprutil >>> + lfsmake2 swig >>> lfsmake2 unbound >>> lfsmake2 gnutls >>> lfsmake2 libuv >>> @@ -1930,7 +1931,6 @@ build_system() { >>> lfsmake2 minicom >>> lfsmake2 ddrescue >>> lfsmake2 parted >>> - lfsmake2 swig >>> lfsmake2 python3-pyelftools >>> lfsmake2 u-boot >>> lfsmake2 wireless-regdb >> > > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC PATCH 1/2] unbound: Enable the Python module(s) 2025-08-18 12:40 ` Adolf Belka @ 2025-08-18 16:17 ` Adolf Belka 2025-08-18 17:00 ` Adolf Belka 0 siblings, 1 reply; 7+ messages in thread From: Adolf Belka @ 2025-08-18 16:17 UTC (permalink / raw) To: Michael Tremer; +Cc: IPFire: Development-List Hi Michael, So I tried swig-4.3.1 but dtc failed to build. Looking up the failure message I believe it is the same error that u-boot would have had. A function was changed in swig-4.3.x to use 3 parameters instead of 2. The latest version of dtc and u-boot from Jan 2025 both have commits to resolve it. So I will do a build with the newer dtc and test out the building of u-boot on aarch64 with the newer swig and a newer u-boot. Regards, Adolf. On 18/08/2025 14:40, Adolf Belka wrote: > Hi Michael, > > On 18/08/2025 13:50, Michael Tremer wrote: >> Hello, >> >> I have the same problem. >> >> I have built this from a non-clean build initially and everything was fine. I used that version of Unbound to write and test the code. No problems there whatsoever. >> >> But I never managed to run a clean build. It seems that swig messes something up and I cannot tell you at all what that could be. It is not only tomli not compiling. If you comment it out and try to compile the next thing it won’t work either. I don’t know what it could break like this because the problems suddenly seem to occur in the Python standard library. >> >> Maybe an update of Swig solves the problem? Would you like to give that a try? > > The last swig update I provided had to be reverted because uboot would not build with it. > > However there is a new version out now, 4.3.1 from 2025-04-15, so I will give that a go and see if it helps and also do an aarch64 build and see if uboot builds or not. > > If it doesn't help or uboot fails to build then I will look at what other packages might need to be moved earlier to be before swig so that all the expected swig bits are built. > > Currently we have swig-4.1.0 and the reverted version was 4.3.0, so there are also other versions newer than the existing one that might allow uboot to build and help the issue we are having. > > Will have a go at it, although might take a while as I am cleaning my house up to make it child proof as my nephew and family are visiting from end of this week for 5 days. > > Regards, > > Adolf. > >> >> -Michael >> >>> On 16 Aug 2025, at 17:23, Adolf Belka <adolf.belka@ipfire.org> wrote: >>> >>> Hi Michael, >>> >>> After some time I thought I should have a go at building with your two patches for the unbound fast flux option but the build fails to complete. >>> >>> I am doing the build with the current next repo. >>> >>> I have done the build twice and it fails at the same point each time at the python3-tomli build. >>> >>> The following is the log output, which I have not been able to interpret to identify what is causing the build to fail. >>> >>> >>> Aug 16 15:11:49: Building python3-tomli make: Entering directory '/home/ahb/sandbox/ipfire-2-builder3/lfs' >>> make: Nothing to be done for 'download'. >>> make: Leaving directory '/home/ahb/sandbox/ipfire-2-builder3/lfs' >>> make: Entering directory '/usr/src/lfs' >>> tomli-2.0.1.tar.gz checksum OK >>> ====================================== Installing tomli-2.0.1 ... >>> Install started; saving file list to /usr/src/lsalr ... >>> cd /usr/src/tomli-2.0.1 && python3 -m pyproject2setuppy.main build >>> Traceback (most recent call last): >>> File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main >>> return _run_code(code, main_globals, None, >>> File "/usr/lib/python3.10/runpy.py", line 86, in _run_code >>> exec(code, run_globals) >>> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/main.py", line 15, in <module> >>> main() >>> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/__main__.py", line 53, in main >>> handler(data) >>> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/flit.py", line 91, in handle_flit >>> setup(**setup_metadata) >>> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 86, in setup >>> _install_setup_requires(attrs) >>> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires >>> dist = MinimalDistribution(attrs) >>> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 57, in __init__ >>> super().__init__(filtered) >>> File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 474, in __init__ >>> for ep in metadata.entry_points(group='distutils.setup_keywords'): >>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1009, in entry_points >>> return SelectableGroups.load(eps).select(**params) >>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 459, in load >>> ordered = sorted(eps, key=by_group) >>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1006, in <genexpr> >>> eps = itertools.chain.from_iterable( >>> File "/usr/lib/python3.10/importlib/metadata/_itertools.py", line 16, in unique_everseen >>> k = key(element) >>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 941, in _normalized_name >>> return self._name_from_stem(stem) or super()._normalized_name >>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 622, in _normalized_name >>> return Prepared.normalize(self.name) >>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 871, in normalize >>> return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_') >>> File "/usr/lib/python3.10/re.py", line 209, in sub >>> return _compile(pattern, flags).sub(repl, string, count) >>> TypeError: expected string or bytes-like object >>> make: *** [python3-tomli:78: /usr/src/log/tomli-2.0.1] Error 1 >>> make: Leaving directory '/usr/src/lfs' >>> >>> Regards, >>> >>> Adolf. >>> >>> On 25/04/2025 14:37, Michael Tremer wrote: >>>> This can be used to script the resolver and use libunbound from Python. >>>> Signed-off-by: Michael Tremer <michael.tremer@ipfire.org> >>>> --- >>>> config/rootfiles/common/unbound | 4 ++++ >>>> lfs/unbound | 3 +++ >>>> make.sh | 2 +- >>>> 3 files changed, 8 insertions(+), 1 deletion(-) >>>> diff --git a/config/rootfiles/common/unbound b/config/rootfiles/common/unbound >>>> index 57390d6d9..03c956503 100644 >>>> --- a/config/rootfiles/common/unbound >>>> +++ b/config/rootfiles/common/unbound >>>> @@ -13,6 +13,10 @@ etc/unbound/unbound.conf >>>> usr/lib/libunbound.so.8 >>>> usr/lib/libunbound.so.8.1.30 >>>> #usr/lib/pkgconfig/libunbound.pc >>>> +#usr/lib/python3.10/site-packages/_unbound.la >>>> +usr/lib/python3.10/site-packages/_unbound.so >>>> +usr/lib/python3.10/site-packages/unbound.py >>>> +usr/lib/python3.10/site-packages/unboundmodule.py >>>> usr/sbin/unbound >>>> usr/sbin/unbound-anchor >>>> usr/sbin/unbound-checkconf >>>> diff --git a/lfs/unbound b/lfs/unbound >>>> index 537ccff7e..8c79125ec 100644 >>>> --- a/lfs/unbound >>>> +++ b/lfs/unbound >>>> @@ -71,6 +71,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) >>>> @$(PREBUILD) >>>> @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE) >>>> cd $(DIR_APP) && \ >>>> + PYTHON_VERSION=3 \ >>>> ./configure \ >>>> --prefix=/usr \ >>>> --sysconfdir=/etc \ >>>> @@ -83,6 +84,8 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) >>>> --enable-tfo-server \ >>>> --enable-ipset \ >>>> --with-libmnl \ >>>> + --with-pythonmodule \ >>>> + --with-pyunbound \ >>>> ac_cv_func_getentropy=no >>>> cd $(DIR_APP) && make $(MAKETUNING) >>>> diff --git a/make.sh b/make.sh >>>> index 3643b469e..3a673cd2b 100755 >>>> --- a/make.sh >>>> +++ b/make.sh >>>> @@ -1534,6 +1534,7 @@ build_system() { >>>> lfsmake2 libevent2 >>>> lfsmake2 apr >>>> lfsmake2 aprutil >>>> + lfsmake2 swig >>>> lfsmake2 unbound >>>> lfsmake2 gnutls >>>> lfsmake2 libuv >>>> @@ -1930,7 +1931,6 @@ build_system() { >>>> lfsmake2 minicom >>>> lfsmake2 ddrescue >>>> lfsmake2 parted >>>> - lfsmake2 swig >>>> lfsmake2 python3-pyelftools >>>> lfsmake2 u-boot >>>> lfsmake2 wireless-regdb >>> >> >> > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC PATCH 1/2] unbound: Enable the Python module(s) 2025-08-18 16:17 ` Adolf Belka @ 2025-08-18 17:00 ` Adolf Belka 0 siblings, 0 replies; 7+ messages in thread From: Adolf Belka @ 2025-08-18 17:00 UTC (permalink / raw) To: Michael Tremer; +Cc: IPFire: Development-List Hi Michael, On 18/08/2025 18:17, Adolf Belka wrote: > Hi Michael, > > So I tried swig-4.3.1 but dtc failed to build. > > Looking up the failure message I believe it is the same error that u-boot would have had. A function was changed in swig-4.3.x to use 3 parameters instead of 2. The latest version of dtc and u-boot from Jan 2025 both have commits to resolve it. > > So I will do a build with the newer dtc and test out the building of u-boot on aarch64 with the newer swig and a newer u-boot. The newer dtc built without problems but the python3-tomli still failed. So then I thought it might be because the tomli file was too old so I tried a newer version. That also failed to build but with a different message which Seemed to ring a bell for problems with the pyproject2setuppy module as tomli only has pyproject.toml and no setup.py So then did the build for python3-tomli using the newer commands for working with pyproject.toml however then it failed for no build module, so I put that before the python3-tomli file and did the build again but this time it failed for the same error message as we had with tomli originally File "/usr/lib/python3.10/re.py", line 209, in sub return _compile(pattern, flags).sub(repl, string, count) TypeError: expected string or bytes-like object I'll have a further think about these build issues. Regards, Adolf. > > Regards, > > Adolf. > > > On 18/08/2025 14:40, Adolf Belka wrote: >> Hi Michael, >> >> On 18/08/2025 13:50, Michael Tremer wrote: >>> Hello, >>> >>> I have the same problem. >>> >>> I have built this from a non-clean build initially and everything was fine. I used that version of Unbound to write and test the code. No problems there whatsoever. >>> >>> But I never managed to run a clean build. It seems that swig messes something up and I cannot tell you at all what that could be. It is not only tomli not compiling. If you comment it out and try to compile the next thing it won’t work either. I don’t know what it could break like this because the problems suddenly seem to occur in the Python standard library. >>> >>> Maybe an update of Swig solves the problem? Would you like to give that a try? >> >> The last swig update I provided had to be reverted because uboot would not build with it. >> >> However there is a new version out now, 4.3.1 from 2025-04-15, so I will give that a go and see if it helps and also do an aarch64 build and see if uboot builds or not. >> >> If it doesn't help or uboot fails to build then I will look at what other packages might need to be moved earlier to be before swig so that all the expected swig bits are built. >> >> Currently we have swig-4.1.0 and the reverted version was 4.3.0, so there are also other versions newer than the existing one that might allow uboot to build and help the issue we are having. >> >> Will have a go at it, although might take a while as I am cleaning my house up to make it child proof as my nephew and family are visiting from end of this week for 5 days. >> >> Regards, >> >> Adolf. >> >>> >>> -Michael >>> >>>> On 16 Aug 2025, at 17:23, Adolf Belka <adolf.belka@ipfire.org> wrote: >>>> >>>> Hi Michael, >>>> >>>> After some time I thought I should have a go at building with your two patches for the unbound fast flux option but the build fails to complete. >>>> >>>> I am doing the build with the current next repo. >>>> >>>> I have done the build twice and it fails at the same point each time at the python3-tomli build. >>>> >>>> The following is the log output, which I have not been able to interpret to identify what is causing the build to fail. >>>> >>>> >>>> Aug 16 15:11:49: Building python3-tomli make: Entering directory '/home/ahb/sandbox/ipfire-2-builder3/lfs' >>>> make: Nothing to be done for 'download'. >>>> make: Leaving directory '/home/ahb/sandbox/ipfire-2-builder3/lfs' >>>> make: Entering directory '/usr/src/lfs' >>>> tomli-2.0.1.tar.gz checksum OK >>>> ====================================== Installing tomli-2.0.1 ... >>>> Install started; saving file list to /usr/src/lsalr ... >>>> cd /usr/src/tomli-2.0.1 && python3 -m pyproject2setuppy.main build >>>> Traceback (most recent call last): >>>> File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main >>>> return _run_code(code, main_globals, None, >>>> File "/usr/lib/python3.10/runpy.py", line 86, in _run_code >>>> exec(code, run_globals) >>>> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/main.py", line 15, in <module> >>>> main() >>>> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/__main__.py", line 53, in main >>>> handler(data) >>>> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/flit.py", line 91, in handle_flit >>>> setup(**setup_metadata) >>>> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 86, in setup >>>> _install_setup_requires(attrs) >>>> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires >>>> dist = MinimalDistribution(attrs) >>>> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 57, in __init__ >>>> super().__init__(filtered) >>>> File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 474, in __init__ >>>> for ep in metadata.entry_points(group='distutils.setup_keywords'): >>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1009, in entry_points >>>> return SelectableGroups.load(eps).select(**params) >>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 459, in load >>>> ordered = sorted(eps, key=by_group) >>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1006, in <genexpr> >>>> eps = itertools.chain.from_iterable( >>>> File "/usr/lib/python3.10/importlib/metadata/_itertools.py", line 16, in unique_everseen >>>> k = key(element) >>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 941, in _normalized_name >>>> return self._name_from_stem(stem) or super()._normalized_name >>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 622, in _normalized_name >>>> return Prepared.normalize(self.name) >>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 871, in normalize >>>> return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_') >>>> File "/usr/lib/python3.10/re.py", line 209, in sub >>>> return _compile(pattern, flags).sub(repl, string, count) >>>> TypeError: expected string or bytes-like object >>>> make: *** [python3-tomli:78: /usr/src/log/tomli-2.0.1] Error 1 >>>> make: Leaving directory '/usr/src/lfs' >>>> >>>> Regards, >>>> >>>> Adolf. >>>> >>>> On 25/04/2025 14:37, Michael Tremer wrote: >>>>> This can be used to script the resolver and use libunbound from Python. >>>>> Signed-off-by: Michael Tremer <michael.tremer@ipfire.org> >>>>> --- >>>>> config/rootfiles/common/unbound | 4 ++++ >>>>> lfs/unbound | 3 +++ >>>>> make.sh | 2 +- >>>>> 3 files changed, 8 insertions(+), 1 deletion(-) >>>>> diff --git a/config/rootfiles/common/unbound b/config/rootfiles/common/unbound >>>>> index 57390d6d9..03c956503 100644 >>>>> --- a/config/rootfiles/common/unbound >>>>> +++ b/config/rootfiles/common/unbound >>>>> @@ -13,6 +13,10 @@ etc/unbound/unbound.conf >>>>> usr/lib/libunbound.so.8 >>>>> usr/lib/libunbound.so.8.1.30 >>>>> #usr/lib/pkgconfig/libunbound.pc >>>>> +#usr/lib/python3.10/site-packages/_unbound.la >>>>> +usr/lib/python3.10/site-packages/_unbound.so >>>>> +usr/lib/python3.10/site-packages/unbound.py >>>>> +usr/lib/python3.10/site-packages/unboundmodule.py >>>>> usr/sbin/unbound >>>>> usr/sbin/unbound-anchor >>>>> usr/sbin/unbound-checkconf >>>>> diff --git a/lfs/unbound b/lfs/unbound >>>>> index 537ccff7e..8c79125ec 100644 >>>>> --- a/lfs/unbound >>>>> +++ b/lfs/unbound >>>>> @@ -71,6 +71,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) >>>>> @$(PREBUILD) >>>>> @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE) >>>>> cd $(DIR_APP) && \ >>>>> + PYTHON_VERSION=3 \ >>>>> ./configure \ >>>>> --prefix=/usr \ >>>>> --sysconfdir=/etc \ >>>>> @@ -83,6 +84,8 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) >>>>> --enable-tfo-server \ >>>>> --enable-ipset \ >>>>> --with-libmnl \ >>>>> + --with-pythonmodule \ >>>>> + --with-pyunbound \ >>>>> ac_cv_func_getentropy=no >>>>> cd $(DIR_APP) && make $(MAKETUNING) >>>>> diff --git a/make.sh b/make.sh >>>>> index 3643b469e..3a673cd2b 100755 >>>>> --- a/make.sh >>>>> +++ b/make.sh >>>>> @@ -1534,6 +1534,7 @@ build_system() { >>>>> lfsmake2 libevent2 >>>>> lfsmake2 apr >>>>> lfsmake2 aprutil >>>>> + lfsmake2 swig >>>>> lfsmake2 unbound >>>>> lfsmake2 gnutls >>>>> lfsmake2 libuv >>>>> @@ -1930,7 +1931,6 @@ build_system() { >>>>> lfsmake2 minicom >>>>> lfsmake2 ddrescue >>>>> lfsmake2 parted >>>>> - lfsmake2 swig >>>>> lfsmake2 python3-pyelftools >>>>> lfsmake2 u-boot >>>>> lfsmake2 wireless-regdb >>>> >>> >>> >> > ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2025-08-18 17:00 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2025-04-25 12:37 [RFC PATCH 1/2] unbound: Enable the Python module(s) Michael Tremer 2025-04-25 12:37 ` [RFC PATCH 2/2] Add Fast Flux detection in DNS Michael Tremer 2025-08-16 16:23 ` [RFC PATCH 1/2] unbound: Enable the Python module(s) Adolf Belka 2025-08-18 11:50 ` Michael Tremer 2025-08-18 12:40 ` Adolf Belka 2025-08-18 16:17 ` Adolf Belka 2025-08-18 17:00 ` Adolf Belka
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox