* [RFC PATCH 1/2] unbound: Enable the Python module(s)
@ 2025-04-25 12:37 Michael Tremer
2025-04-25 12:37 ` [RFC PATCH 2/2] Add Fast Flux detection in DNS Michael Tremer
2025-08-16 16:23 ` [RFC PATCH 1/2] unbound: Enable the Python module(s) Adolf Belka
0 siblings, 2 replies; 7+ messages in thread
From: Michael Tremer @ 2025-04-25 12:37 UTC (permalink / raw)
To: development; +Cc: Michael Tremer
This can be used to script the resolver and use libunbound from Python.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
---
config/rootfiles/common/unbound | 4 ++++
lfs/unbound | 3 +++
make.sh | 2 +-
3 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/config/rootfiles/common/unbound b/config/rootfiles/common/unbound
index 57390d6d9..03c956503 100644
--- a/config/rootfiles/common/unbound
+++ b/config/rootfiles/common/unbound
@@ -13,6 +13,10 @@ etc/unbound/unbound.conf
usr/lib/libunbound.so.8
usr/lib/libunbound.so.8.1.30
#usr/lib/pkgconfig/libunbound.pc
+#usr/lib/python3.10/site-packages/_unbound.la
+usr/lib/python3.10/site-packages/_unbound.so
+usr/lib/python3.10/site-packages/unbound.py
+usr/lib/python3.10/site-packages/unboundmodule.py
usr/sbin/unbound
usr/sbin/unbound-anchor
usr/sbin/unbound-checkconf
diff --git a/lfs/unbound b/lfs/unbound
index 537ccff7e..8c79125ec 100644
--- a/lfs/unbound
+++ b/lfs/unbound
@@ -71,6 +71,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
@$(PREBUILD)
@rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE)
cd $(DIR_APP) && \
+ PYTHON_VERSION=3 \
./configure \
--prefix=/usr \
--sysconfdir=/etc \
@@ -83,6 +84,8 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
--enable-tfo-server \
--enable-ipset \
--with-libmnl \
+ --with-pythonmodule \
+ --with-pyunbound \
ac_cv_func_getentropy=no
cd $(DIR_APP) && make $(MAKETUNING)
diff --git a/make.sh b/make.sh
index 3643b469e..3a673cd2b 100755
--- a/make.sh
+++ b/make.sh
@@ -1534,6 +1534,7 @@ build_system() {
lfsmake2 libevent2
lfsmake2 apr
lfsmake2 aprutil
+ lfsmake2 swig
lfsmake2 unbound
lfsmake2 gnutls
lfsmake2 libuv
@@ -1930,7 +1931,6 @@ build_system() {
lfsmake2 minicom
lfsmake2 ddrescue
lfsmake2 parted
- lfsmake2 swig
lfsmake2 python3-pyelftools
lfsmake2 u-boot
lfsmake2 wireless-regdb
--
2.39.5
^ permalink raw reply [flat|nested] 7+ messages in thread
* [RFC PATCH 2/2] Add Fast Flux detection in DNS
2025-04-25 12:37 [RFC PATCH 1/2] unbound: Enable the Python module(s) Michael Tremer
@ 2025-04-25 12:37 ` Michael Tremer
2025-08-16 16:23 ` [RFC PATCH 1/2] unbound: Enable the Python module(s) Adolf Belka
1 sibling, 0 replies; 7+ messages in thread
From: Michael Tremer @ 2025-04-25 12:37 UTC (permalink / raw)
To: development; +Cc: Michael Tremer
This has been implemented because of a request on the forum. Since the
proxy is generally outgoing technology it makes sense to enable this
kind of filtering in DNS.
This patch adds a Python script which processes every query and its
response and extracts all IP addresses from it. Those IP addresses will
then be resolved to their origin AS. If there are more then THRESHOLD
different ASes, the request will be blocked.
The AS lookups will only be performed when there is enough IP addresses
to actually hit the threshold. So there should be next to no performance
impact here except the overhead of the Python module itself.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
---
config/rootfiles/common/unbound | 1 +
config/unbound/fastflux-detection.py | 167 +++++++++++++++++++++++++++
config/unbound/unbound.conf | 7 ++
doc/language_issues.en | 1 +
doc/language_issues.es | 1 +
doc/language_issues.fr | 1 +
doc/language_issues.it | 1 +
doc/language_issues.nl | 1 +
doc/language_issues.pl | 1 +
doc/language_issues.ru | 1 +
doc/language_issues.tr | 1 +
doc/language_missings | 7 ++
html/cgi-bin/dns.cgi | 21 ++++
langs/de/cgi-bin/de.pl | 1 +
langs/en/cgi-bin/en.pl | 1 +
lfs/unbound | 4 +
16 files changed, 217 insertions(+)
create mode 100644 config/unbound/fastflux-detection.py
diff --git a/config/rootfiles/common/unbound b/config/rootfiles/common/unbound
index 03c956503..9ca84d2d9 100644
--- a/config/rootfiles/common/unbound
+++ b/config/rootfiles/common/unbound
@@ -1,6 +1,7 @@
etc/rc.d/init.d/unbound
#etc/unbound
etc/unbound/dhcp-leases.conf
+etc/unbound/fastflux-detection.py
etc/unbound/forward.conf
etc/unbound/icannbundle.pem
etc/unbound/local.d
diff --git a/config/unbound/fastflux-detection.py b/config/unbound/fastflux-detection.py
new file mode 100644
index 000000000..4ad62eed6
--- /dev/null
+++ b/config/unbound/fastflux-detection.py
@@ -0,0 +1,167 @@
+###############################################################################
+# #
+# Pakfire - The IPFire package management system #
+# Copyright (C) 2025 IPFire Development Team #
+# #
+# This program is free software: you can redistribute it and/or modify #
+# it under the terms of the GNU General Public License as published by #
+# the Free Software Foundation, either version 3 of the License, or #
+# (at your option) any later version. #
+# #
+# This program is distributed in the hope that it will be useful, #
+# but WITHOUT ANY WARRANTY; without even the implied warranty of #
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
+# GNU General Public License for more details. #
+# #
+# You should have received a copy of the GNU General Public License #
+# along with this program. If not, see <http://www.gnu.org/licenses/>. #
+# #
+###############################################################################
+
+import datetime
+import ipaddress
+import location
+import socket
+
+DEFAULT_THRESHOLD = 5
+
+def read_config(path):
+ """
+ Opens the configuration file and reads it line by line
+ """
+ config = {}
+
+ with open(path) as f:
+ for line in f:
+ # Remove any trailing newline
+ line = line.rstrip()
+
+ # Split by key and value
+ key, _, val = line.partition("=")
+
+ # Store the line
+ config[key] = val
+
+ return config
+
+def init(id, cfg):
+ global db
+ global ENABLED
+ global THRESHOLD
+
+ # Read the configuration
+ config = read_config("/var/ipfire/dns/settings")
+
+ # Is this module enabled?
+ ENABLED = config.get("FF_DETECTION", "false") in ("on", "true", "1")
+
+ # Fetch the treshold
+ if ENABLED:
+ threshold = config.get("FF_THRESHOLD", 5)
+
+ try:
+ THRESHOLD = int(threshold)
+ except (TypeError, ValueError):
+ log_warning("Failed to parse Fast Flux threshold '%s'."
+ " Using default of %s" % (threshold, DEFAULT_THRESHOLD))
+ THRESHOLD = DEFAULT_THRESHOLD
+
+ # Open the location database
+ try:
+ db = location.open()
+
+ # Fail if we could not open the database
+ except Exception as e:
+ log_error("Failed to open the location database: %s" % e)
+ return False
+
+ log_info("Opened Location database")
+ log_info(" Database Vendor : %s" % db.vendor)
+ log_info(" Created At : %s" % datetime.datetime.fromtimestamp(db.created_at))
+
+ # Done!
+ log_info("FastFlux detection module loaded")
+
+ return True
+
+def deinit(id):
+ log_info("FastFlux detection module unloaded")
+ return True
+
+def inform_super(id, qstate, superqstate, qdata):
+ return True
+
+def operate(id, event, qstate, qdata):
+ # Execute when everything else is done
+ if event == MODULE_EVENT_MODDONE:
+ # Do nothing if this is not enabled
+ if not ENABLED:
+ qstate.ext_state[id] = MODULE_FINISHED
+ return True
+
+ # Extract the qname
+ qname = qstate.qinfo.qname_str
+
+ # Deny access to the qname?
+ deny = False
+
+ # Extract the response
+ rrset = qstate.return_msg.rep.rrsets
+
+ addrs = set()
+
+ # Find all IP addresses in the response
+ for i in range(qstate.return_msg.rep.rrset_count):
+ rr = rrset[i]
+
+ # Extract the type
+ type = socket.ntohs(rr.rk.type)
+
+ # Only process types A and AAAA
+ if type in (1, 28):
+ for i in range(rr.entry.data.count):
+ payload = rr.entry.data.rr_data[i]
+
+ # Parse the IP address
+ if type == 1:
+ addr = ipaddress.IPv4Address(payload[2:])
+ elif type == 28:
+ addr = ipaddress.IPv6Address(payload[2:])
+
+ addrs.add(addr)
+
+ # Only perform any further action if we have at least as many as threshold IP addresses
+ if len(addrs) >= THRESHOLD:
+ asns = set()
+
+ # Look up the networks for all addresses
+ for addr in addrs:
+ network = db.lookup("%s" % addr)
+
+ # If no network could be found, we add zero to represent an unknown value
+ asns.add(network.asn if network else 0)
+
+ # Check for selective announements
+ if 0 in asns:
+ log_info("Denying access to %s due to suspected selective announcements" % qname)
+ deny = True
+
+ # Check if the threshold was exceeded
+ elif len(asns) >= THRESHOLD:
+ log_info("Denying access to %s due to suspected Fast Flux announcement" % qname)
+ deny = True
+
+ # Return SERVFAIL?
+ # XXX It would be nice to send an extended DNS error here (e.g. BLOCKED), but it
+ # seems that this is currently not supported in the Python module.
+ if deny:
+ qstate.ext_state[id] = MODULE_ERROR
+ return True
+
+ # Otherwise, continue
+ qstate.ext_state[id] = MODULE_FINISHED
+ return True
+
+ # Not handling other events
+ qstate.ext_state[id] = MODULE_WAIT_MODULE
+ return True
diff --git a/config/unbound/unbound.conf b/config/unbound/unbound.conf
index 012beab54..b1f2871c4 100644
--- a/config/unbound/unbound.conf
+++ b/config/unbound/unbound.conf
@@ -12,6 +12,9 @@ server:
username: "nobody"
do-ip6: no
+ # Load modules
+ module-config: "validator python iterator"
+
# System Tuning
include: "/etc/unbound/tuning.conf"
@@ -68,6 +71,10 @@ server:
# Include any forward zones
include: "/etc/unbound/forward.conf"
+python:
+ # Enable Fast Flux Detection
+ python-script: "/etc/unbound/fastflux-detection.py"
+
remote-control:
control-enable: yes
control-use-cert: no
diff --git a/doc/language_issues.en b/doc/language_issues.en
index 1c1c546f7..a5860a153 100644
--- a/doc/language_issues.en
+++ b/doc/language_issues.en
@@ -611,6 +611,7 @@ WARNING: untranslated string: dnat address = Firewall Interface
WARNING: untranslated string: dns check failed = DNS check failed
WARNING: untranslated string: dns check servers = Check DNS Servers
WARNING: untranslated string: dns configuration = DNS Configuration
+WARNING: untranslated string: dns enable fast flux detection = Fast Flux Detection
WARNING: untranslated string: dns enable safe-search = Enable Safe Search
WARNING: untranslated string: dns enable safe-search youtube = Include YouTube in Safe Search
WARNING: untranslated string: dns forward disable dnssec = Disable DNSSEC (dangerous)
diff --git a/doc/language_issues.es b/doc/language_issues.es
index cf7237435..2b7937842 100644
--- a/doc/language_issues.es
+++ b/doc/language_issues.es
@@ -1015,6 +1015,7 @@ WARNING: untranslated string: ca name must only contain characters and spaces =
WARNING: untranslated string: cpu frequency = CPU frequency
WARNING: untranslated string: data transfer = Data Transfer
WARNING: untranslated string: dhcp fixed ip address in dynamic range = Fixed IP Address in dynamic range
+WARNING: untranslated string: dns enable fast flux detection = Fast Flux Detection
WARNING: untranslated string: dns servers = DNS Servers
WARNING: untranslated string: done = Done
WARNING: untranslated string: downfall gather data sampling = Downfall/Gather Data Sampling
diff --git a/doc/language_issues.fr b/doc/language_issues.fr
index 702911061..b9703cd4d 100644
--- a/doc/language_issues.fr
+++ b/doc/language_issues.fr
@@ -979,6 +979,7 @@ WARNING: untranslated string: bypassed = Bypassed
WARNING: untranslated string: ca name must only contain characters and spaces = unknown string
WARNING: untranslated string: core notice 3 = available.
WARNING: untranslated string: data transfer = Data Transfer
+WARNING: untranslated string: dns enable fast flux detection = Fast Flux Detection
WARNING: untranslated string: done = Done
WARNING: untranslated string: enable disable client = unknown string
WARNING: untranslated string: enable disable dyndns = unknown string
diff --git a/doc/language_issues.it b/doc/language_issues.it
index 3d93239af..c5297c241 100644
--- a/doc/language_issues.it
+++ b/doc/language_issues.it
@@ -1019,6 +1019,7 @@ WARNING: untranslated string: disconnected = Disconnected
WARNING: untranslated string: dl client arch insecure = Download insecure Client Package (zip)
WARNING: untranslated string: dns check servers = Check DNS Servers
WARNING: untranslated string: dns configuration = DNS Configuration
+WARNING: untranslated string: dns enable fast flux detection = Fast Flux Detection
WARNING: untranslated string: dns enable safe-search = Enable Safe Search
WARNING: untranslated string: dns enable safe-search youtube = Include YouTube in Safe Search
WARNING: untranslated string: dns forward disable dnssec = Disable DNSSEC (dangerous)
diff --git a/doc/language_issues.nl b/doc/language_issues.nl
index f1090fc33..7f16de518 100644
--- a/doc/language_issues.nl
+++ b/doc/language_issues.nl
@@ -1019,6 +1019,7 @@ WARNING: untranslated string: disable = Disable
WARNING: untranslated string: disconnected = Disconnected
WARNING: untranslated string: dns check servers = Check DNS Servers
WARNING: untranslated string: dns configuration = DNS Configuration
+WARNING: untranslated string: dns enable fast flux detection = Fast Flux Detection
WARNING: untranslated string: dns enable safe-search = Enable Safe Search
WARNING: untranslated string: dns enable safe-search youtube = Include YouTube in Safe Search
WARNING: untranslated string: dns forward disable dnssec = Disable DNSSEC (dangerous)
diff --git a/doc/language_issues.pl b/doc/language_issues.pl
index 1db36fb67..063045ef3 100644
--- a/doc/language_issues.pl
+++ b/doc/language_issues.pl
@@ -991,6 +991,7 @@ WARNING: untranslated string: dl client arch insecure = Download insecure Client
WARNING: untranslated string: dnat address = Firewall Interface
WARNING: untranslated string: dns check servers = Check DNS Servers
WARNING: untranslated string: dns configuration = DNS Configuration
+WARNING: untranslated string: dns enable fast flux detection = Fast Flux Detection
WARNING: untranslated string: dns enable safe-search = Enable Safe Search
WARNING: untranslated string: dns enable safe-search youtube = Include YouTube in Safe Search
WARNING: untranslated string: dns forward disable dnssec = Disable DNSSEC (dangerous)
diff --git a/doc/language_issues.ru b/doc/language_issues.ru
index 4d29c4f95..5c85a2c40 100644
--- a/doc/language_issues.ru
+++ b/doc/language_issues.ru
@@ -986,6 +986,7 @@ WARNING: untranslated string: dl client arch insecure = Download insecure Client
WARNING: untranslated string: dnat address = Firewall Interface
WARNING: untranslated string: dns check servers = Check DNS Servers
WARNING: untranslated string: dns configuration = DNS Configuration
+WARNING: untranslated string: dns enable fast flux detection = Fast Flux Detection
WARNING: untranslated string: dns enable safe-search = Enable Safe Search
WARNING: untranslated string: dns enable safe-search youtube = Include YouTube in Safe Search
WARNING: untranslated string: dns forward disable dnssec = Disable DNSSEC (dangerous)
diff --git a/doc/language_issues.tr b/doc/language_issues.tr
index 2da19f276..473250445 100644
--- a/doc/language_issues.tr
+++ b/doc/language_issues.tr
@@ -998,6 +998,7 @@ WARNING: untranslated string: disable = Disable
WARNING: untranslated string: disconnected = Disconnected
WARNING: untranslated string: dns check servers = Check DNS Servers
WARNING: untranslated string: dns configuration = DNS Configuration
+WARNING: untranslated string: dns enable fast flux detection = Fast Flux Detection
WARNING: untranslated string: dns enable safe-search = Enable Safe Search
WARNING: untranslated string: dns enable safe-search youtube = Include YouTube in Safe Search
WARNING: untranslated string: dns forward disable dnssec = Disable DNSSEC (dangerous)
diff --git a/doc/language_missings b/doc/language_missings
index 48b98ce74..f5c0949b5 100644
--- a/doc/language_missings
+++ b/doc/language_missings
@@ -173,6 +173,7 @@
< cpu frequency
< data transfer
< dhcp fixed ip address in dynamic range
+< dns enable fast flux detection
< dns servers
< done
< downfall gather data sampling
@@ -287,6 +288,7 @@
< bypassed
< ca name must only contain characters or spaces
< data transfer
+< dns enable fast flux detection
< done
< endpoint
< endpoint address
@@ -509,6 +511,7 @@
< dns check servers
< dns configuration
< dns could not add server
+< dns enable fast flux detection
< dns enable safe-search
< dns enable safe-search youtube
< dns forward disable dnssec
@@ -1110,6 +1113,7 @@
< dns check servers
< dns configuration
< dns could not add server
+< dns enable fast flux detection
< dns enable safe-search
< dns enable safe-search youtube
< dns forward disable dnssec
@@ -1807,6 +1811,7 @@
< dns check servers
< dns configuration
< dns could not add server
+< dns enable fast flux detection
< dns enable safe-search
< dns enable safe-search youtube
< dnsforward
@@ -2884,6 +2889,7 @@
< dns check servers
< dns configuration
< dns could not add server
+< dns enable fast flux detection
< dns enable safe-search
< dns enable safe-search youtube
< dnsforward
@@ -3823,6 +3829,7 @@
< dns check servers
< dns configuration
< dns could not add server
+< dns enable fast flux detection
< dns enable safe-search
< dns enable safe-search youtube
< dns forward disable dnssec
diff --git a/html/cgi-bin/dns.cgi b/html/cgi-bin/dns.cgi
index 0d3b14797..8cc39f97c 100644
--- a/html/cgi-bin/dns.cgi
+++ b/html/cgi-bin/dns.cgi
@@ -82,6 +82,11 @@ if ($cgiparams{'GENERAL'} eq $Lang::tr{'save'}) {
$cgiparams{'USE_ISP_NAMESERVERS'} = "off";
}
+ # Add value for non-checked checkbox.
+ if ($cgiparams{'FF_DETECTION'} ne "on") {
+ $cgiparams{'FF_DETECTION'} = "off";
+ }
+
# Add value for non-checked checkbox.
if ($cgiparams{'ENABLE_SAFE_SEARCH'} ne "on") {
$cgiparams{'ENABLE_SAFE_SEARCH'} = "off";
@@ -264,6 +269,7 @@ if (($cgiparams{'SERVERS'} eq $Lang::tr{'save'}) || ($cgiparams{'SERVERS'} eq $L
# Hash to store the generic DNS settings.
my %settings = ();
$settings{"ENABLE_SAFE_SEARCH_YOUTUBE"} = "on";
+$settings{"FF_DETECTION"} = "on";
# Read-in general DNS settings.
&General::readhash("$settings_file", \%settings);
@@ -311,6 +317,10 @@ $checked{'USE_ISP_NAMESERVERS'}{'off'} = '';
$checked{'USE_ISP_NAMESERVERS'}{'on'} = '';
$checked{'USE_ISP_NAMESERVERS'}{$settings{'USE_ISP_NAMESERVERS'}} = "checked='checked'";
+$checked{'FF_DETECTION'}{'off'} = '';
+$checked{'FF_DETECTION'}{'on'} = '';
+$checked{'FF_DETECTION'}{$settings{'FF_DETECTION'}} = "checked='checked'";
+
$checked{'ENABLE_SAFE_SEARCH'}{'off'} = '';
$checked{'ENABLE_SAFE_SEARCH'}{'on'} = '';
$checked{'ENABLE_SAFE_SEARCH'}{$settings{'ENABLE_SAFE_SEARCH'}} = "checked='checked'";
@@ -380,6 +390,17 @@ sub show_general_dns_configuration () {
</td>
</tr>
+ <tr>
+ <td width="33%">
+ $Lang::tr{'dns enable fast flux detection'}
+ </td>
+
+ <td>
+ <input type="checkbox" name="FF_DETECTION" $checked{'FF_DETECTION'}{'on'}>
+ </td>
+ </tr>
+
+
<tr>
<td width="33%">
$Lang::tr{'dns enable safe-search'}
diff --git a/langs/de/cgi-bin/de.pl b/langs/de/cgi-bin/de.pl
index 3ce02b657..aafc180a6 100644
--- a/langs/de/cgi-bin/de.pl
+++ b/langs/de/cgi-bin/de.pl
@@ -835,6 +835,7 @@
'dns check servers' => 'DNS-Server prüfen',
'dns configuration' => 'DNS-Konfiguration',
'dns desc' => 'Wenn auf Schnittstelle red0 die IP-Adressinformationen über DHCP vom Provider kommen, werden automatisch die DNS-Server-Adressen des Providers gesetzt. Hier können Sie nun diese mit den eigenen DNS-Server-IP-Adressen überschreiben.',
+'dns enable fast flux detection' => 'Fast-Flux-Erkennung',
'dns enable safe-search' => 'Safe Search via DNS aktivieren',
'dns enable safe-search youtube' => 'YouTube in Safe Search einbeziehen',
'dns error 0' => 'Die IP Adresse vom <strong>primären</strong> DNS Server ist nicht gültig, bitte überprüfen Sie Ihre Eingabe!<br />Die eingegebene <strong>sekundären</strong> DNS Server Adresse ist jedoch gültig.<br />',
diff --git a/langs/en/cgi-bin/en.pl b/langs/en/cgi-bin/en.pl
index 3e647e6e5..fb82fcd08 100644
--- a/langs/en/cgi-bin/en.pl
+++ b/langs/en/cgi-bin/en.pl
@@ -880,6 +880,7 @@
'dns configuration' => 'DNS Configuration',
'dns could not add server' => 'Could not add server - Reason:',
'dns desc' => 'If the red0 interface gets the IP address information via DHCP from the provider, the DNS server addresses will be set automatically. Now here you are able to change these DNS server IP addresses with your own ones.',
+'dns enable fast flux detection' => 'Fast Flux Detection',
'dns enable safe-search' => 'Enable Safe Search',
'dns enable safe-search youtube' => 'Include YouTube in Safe Search',
'dns error 0' => 'The IP address of the <strong>primary</strong> DNS server is not valid, please check your entries!<br />The entered <strong>secondary</strong> DNS server address is valid.',
diff --git a/lfs/unbound b/lfs/unbound
index 8c79125ec..00105b691 100644
--- a/lfs/unbound
+++ b/lfs/unbound
@@ -97,6 +97,10 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
touch /etc/unbound/{dhcp-leases,forward}.conf
-mkdir -pv /etc/unbound/local.d
+ # Install Python scripts
+ install -v -m 644 $(DIR_SRC)/config/unbound/fastflux-detection.py \
+ /etc/unbound/fastflux-detection.py
+
# Install root hints
install -v -m 644 $(DIR_SRC)/config/unbound/root.hints \
/etc/unbound/root.hints
--
2.39.5
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC PATCH 1/2] unbound: Enable the Python module(s)
2025-04-25 12:37 [RFC PATCH 1/2] unbound: Enable the Python module(s) Michael Tremer
2025-04-25 12:37 ` [RFC PATCH 2/2] Add Fast Flux detection in DNS Michael Tremer
@ 2025-08-16 16:23 ` Adolf Belka
2025-08-18 11:50 ` Michael Tremer
1 sibling, 1 reply; 7+ messages in thread
From: Adolf Belka @ 2025-08-16 16:23 UTC (permalink / raw)
To: Michael Tremer; +Cc: IPFire: Development-List
Hi Michael,
After some time I thought I should have a go at building with your two patches for the unbound fast flux option but the build fails to complete.
I am doing the build with the current next repo.
I have done the build twice and it fails at the same point each time at the python3-tomli build.
The following is the log output, which I have not been able to interpret to identify what is causing the build to fail.
Aug 16 15:11:49: Building python3-tomli make: Entering directory '/home/ahb/sandbox/ipfire-2-builder3/lfs'
make: Nothing to be done for 'download'.
make: Leaving directory '/home/ahb/sandbox/ipfire-2-builder3/lfs'
make: Entering directory '/usr/src/lfs'
tomli-2.0.1.tar.gz checksum OK
====================================== Installing tomli-2.0.1 ...
Install started; saving file list to /usr/src/lsalr ...
cd /usr/src/tomli-2.0.1 && python3 -m pyproject2setuppy.main build
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/lib/python3.10/site-packages/pyproject2setuppy/main.py", line 15, in <module>
main()
File "/usr/lib/python3.10/site-packages/pyproject2setuppy/__main__.py", line 53, in main
handler(data)
File "/usr/lib/python3.10/site-packages/pyproject2setuppy/flit.py", line 91, in handle_flit
setup(**setup_metadata)
File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 86, in setup
_install_setup_requires(attrs)
File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires
dist = MinimalDistribution(attrs)
File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 57, in __init__
super().__init__(filtered)
File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 474, in __init__
for ep in metadata.entry_points(group='distutils.setup_keywords'):
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1009, in entry_points
return SelectableGroups.load(eps).select(**params)
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 459, in load
ordered = sorted(eps, key=by_group)
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1006, in <genexpr>
eps = itertools.chain.from_iterable(
File "/usr/lib/python3.10/importlib/metadata/_itertools.py", line 16, in unique_everseen
k = key(element)
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 941, in _normalized_name
return self._name_from_stem(stem) or super()._normalized_name
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 622, in _normalized_name
return Prepared.normalize(self.name)
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 871, in normalize
return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_')
File "/usr/lib/python3.10/re.py", line 209, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object
make: *** [python3-tomli:78: /usr/src/log/tomli-2.0.1] Error 1
make: Leaving directory '/usr/src/lfs'
Regards,
Adolf.
On 25/04/2025 14:37, Michael Tremer wrote:
> This can be used to script the resolver and use libunbound from Python.
>
> Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
> ---
> config/rootfiles/common/unbound | 4 ++++
> lfs/unbound | 3 +++
> make.sh | 2 +-
> 3 files changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/config/rootfiles/common/unbound b/config/rootfiles/common/unbound
> index 57390d6d9..03c956503 100644
> --- a/config/rootfiles/common/unbound
> +++ b/config/rootfiles/common/unbound
> @@ -13,6 +13,10 @@ etc/unbound/unbound.conf
> usr/lib/libunbound.so.8
> usr/lib/libunbound.so.8.1.30
> #usr/lib/pkgconfig/libunbound.pc
> +#usr/lib/python3.10/site-packages/_unbound.la
> +usr/lib/python3.10/site-packages/_unbound.so
> +usr/lib/python3.10/site-packages/unbound.py
> +usr/lib/python3.10/site-packages/unboundmodule.py
> usr/sbin/unbound
> usr/sbin/unbound-anchor
> usr/sbin/unbound-checkconf
> diff --git a/lfs/unbound b/lfs/unbound
> index 537ccff7e..8c79125ec 100644
> --- a/lfs/unbound
> +++ b/lfs/unbound
> @@ -71,6 +71,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
> @$(PREBUILD)
> @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE)
> cd $(DIR_APP) && \
> + PYTHON_VERSION=3 \
> ./configure \
> --prefix=/usr \
> --sysconfdir=/etc \
> @@ -83,6 +84,8 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
> --enable-tfo-server \
> --enable-ipset \
> --with-libmnl \
> + --with-pythonmodule \
> + --with-pyunbound \
> ac_cv_func_getentropy=no
>
> cd $(DIR_APP) && make $(MAKETUNING)
> diff --git a/make.sh b/make.sh
> index 3643b469e..3a673cd2b 100755
> --- a/make.sh
> +++ b/make.sh
> @@ -1534,6 +1534,7 @@ build_system() {
> lfsmake2 libevent2
> lfsmake2 apr
> lfsmake2 aprutil
> + lfsmake2 swig
> lfsmake2 unbound
> lfsmake2 gnutls
> lfsmake2 libuv
> @@ -1930,7 +1931,6 @@ build_system() {
> lfsmake2 minicom
> lfsmake2 ddrescue
> lfsmake2 parted
> - lfsmake2 swig
> lfsmake2 python3-pyelftools
> lfsmake2 u-boot
> lfsmake2 wireless-regdb
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC PATCH 1/2] unbound: Enable the Python module(s)
2025-08-16 16:23 ` [RFC PATCH 1/2] unbound: Enable the Python module(s) Adolf Belka
@ 2025-08-18 11:50 ` Michael Tremer
2025-08-18 12:40 ` Adolf Belka
0 siblings, 1 reply; 7+ messages in thread
From: Michael Tremer @ 2025-08-18 11:50 UTC (permalink / raw)
To: Adolf Belka; +Cc: IPFire: Development-List
Hello,
I have the same problem.
I have built this from a non-clean build initially and everything was fine. I used that version of Unbound to write and test the code. No problems there whatsoever.
But I never managed to run a clean build. It seems that swig messes something up and I cannot tell you at all what that could be. It is not only tomli not compiling. If you comment it out and try to compile the next thing it won’t work either. I don’t know what it could break like this because the problems suddenly seem to occur in the Python standard library.
Maybe an update of Swig solves the problem? Would you like to give that a try?
-Michael
> On 16 Aug 2025, at 17:23, Adolf Belka <adolf.belka@ipfire.org> wrote:
>
> Hi Michael,
>
> After some time I thought I should have a go at building with your two patches for the unbound fast flux option but the build fails to complete.
>
> I am doing the build with the current next repo.
>
> I have done the build twice and it fails at the same point each time at the python3-tomli build.
>
> The following is the log output, which I have not been able to interpret to identify what is causing the build to fail.
>
>
> Aug 16 15:11:49: Building python3-tomli make: Entering directory '/home/ahb/sandbox/ipfire-2-builder3/lfs'
> make: Nothing to be done for 'download'.
> make: Leaving directory '/home/ahb/sandbox/ipfire-2-builder3/lfs'
> make: Entering directory '/usr/src/lfs'
> tomli-2.0.1.tar.gz checksum OK
> ====================================== Installing tomli-2.0.1 ...
> Install started; saving file list to /usr/src/lsalr ...
> cd /usr/src/tomli-2.0.1 && python3 -m pyproject2setuppy.main build
> Traceback (most recent call last):
> File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
> return _run_code(code, main_globals, None,
> File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
> exec(code, run_globals)
> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/main.py", line 15, in <module>
> main()
> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/__main__.py", line 53, in main
> handler(data)
> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/flit.py", line 91, in handle_flit
> setup(**setup_metadata)
> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 86, in setup
> _install_setup_requires(attrs)
> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires
> dist = MinimalDistribution(attrs)
> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 57, in __init__
> super().__init__(filtered)
> File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 474, in __init__
> for ep in metadata.entry_points(group='distutils.setup_keywords'):
> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1009, in entry_points
> return SelectableGroups.load(eps).select(**params)
> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 459, in load
> ordered = sorted(eps, key=by_group)
> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1006, in <genexpr>
> eps = itertools.chain.from_iterable(
> File "/usr/lib/python3.10/importlib/metadata/_itertools.py", line 16, in unique_everseen
> k = key(element)
> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 941, in _normalized_name
> return self._name_from_stem(stem) or super()._normalized_name
> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 622, in _normalized_name
> return Prepared.normalize(self.name)
> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 871, in normalize
> return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_')
> File "/usr/lib/python3.10/re.py", line 209, in sub
> return _compile(pattern, flags).sub(repl, string, count)
> TypeError: expected string or bytes-like object
> make: *** [python3-tomli:78: /usr/src/log/tomli-2.0.1] Error 1
> make: Leaving directory '/usr/src/lfs'
>
> Regards,
>
> Adolf.
>
> On 25/04/2025 14:37, Michael Tremer wrote:
>> This can be used to script the resolver and use libunbound from Python.
>> Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
>> ---
>> config/rootfiles/common/unbound | 4 ++++
>> lfs/unbound | 3 +++
>> make.sh | 2 +-
>> 3 files changed, 8 insertions(+), 1 deletion(-)
>> diff --git a/config/rootfiles/common/unbound b/config/rootfiles/common/unbound
>> index 57390d6d9..03c956503 100644
>> --- a/config/rootfiles/common/unbound
>> +++ b/config/rootfiles/common/unbound
>> @@ -13,6 +13,10 @@ etc/unbound/unbound.conf
>> usr/lib/libunbound.so.8
>> usr/lib/libunbound.so.8.1.30
>> #usr/lib/pkgconfig/libunbound.pc
>> +#usr/lib/python3.10/site-packages/_unbound.la
>> +usr/lib/python3.10/site-packages/_unbound.so
>> +usr/lib/python3.10/site-packages/unbound.py
>> +usr/lib/python3.10/site-packages/unboundmodule.py
>> usr/sbin/unbound
>> usr/sbin/unbound-anchor
>> usr/sbin/unbound-checkconf
>> diff --git a/lfs/unbound b/lfs/unbound
>> index 537ccff7e..8c79125ec 100644
>> --- a/lfs/unbound
>> +++ b/lfs/unbound
>> @@ -71,6 +71,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
>> @$(PREBUILD)
>> @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE)
>> cd $(DIR_APP) && \
>> + PYTHON_VERSION=3 \
>> ./configure \
>> --prefix=/usr \
>> --sysconfdir=/etc \
>> @@ -83,6 +84,8 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
>> --enable-tfo-server \
>> --enable-ipset \
>> --with-libmnl \
>> + --with-pythonmodule \
>> + --with-pyunbound \
>> ac_cv_func_getentropy=no
>> cd $(DIR_APP) && make $(MAKETUNING)
>> diff --git a/make.sh b/make.sh
>> index 3643b469e..3a673cd2b 100755
>> --- a/make.sh
>> +++ b/make.sh
>> @@ -1534,6 +1534,7 @@ build_system() {
>> lfsmake2 libevent2
>> lfsmake2 apr
>> lfsmake2 aprutil
>> + lfsmake2 swig
>> lfsmake2 unbound
>> lfsmake2 gnutls
>> lfsmake2 libuv
>> @@ -1930,7 +1931,6 @@ build_system() {
>> lfsmake2 minicom
>> lfsmake2 ddrescue
>> lfsmake2 parted
>> - lfsmake2 swig
>> lfsmake2 python3-pyelftools
>> lfsmake2 u-boot
>> lfsmake2 wireless-regdb
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC PATCH 1/2] unbound: Enable the Python module(s)
2025-08-18 11:50 ` Michael Tremer
@ 2025-08-18 12:40 ` Adolf Belka
2025-08-18 16:17 ` Adolf Belka
0 siblings, 1 reply; 7+ messages in thread
From: Adolf Belka @ 2025-08-18 12:40 UTC (permalink / raw)
To: Michael Tremer; +Cc: IPFire: Development-List
Hi Michael,
On 18/08/2025 13:50, Michael Tremer wrote:
> Hello,
>
> I have the same problem.
>
> I have built this from a non-clean build initially and everything was fine. I used that version of Unbound to write and test the code. No problems there whatsoever.
>
> But I never managed to run a clean build. It seems that swig messes something up and I cannot tell you at all what that could be. It is not only tomli not compiling. If you comment it out and try to compile the next thing it won’t work either. I don’t know what it could break like this because the problems suddenly seem to occur in the Python standard library.
>
> Maybe an update of Swig solves the problem? Would you like to give that a try?
The last swig update I provided had to be reverted because uboot would not build with it.
However there is a new version out now, 4.3.1 from 2025-04-15, so I will give that a go and see if it helps and also do an aarch64 build and see if uboot builds or not.
If it doesn't help or uboot fails to build then I will look at what other packages might need to be moved earlier to be before swig so that all the expected swig bits are built.
Currently we have swig-4.1.0 and the reverted version was 4.3.0, so there are also other versions newer than the existing one that might allow uboot to build and help the issue we are having.
Will have a go at it, although might take a while as I am cleaning my house up to make it child proof as my nephew and family are visiting from end of this week for 5 days.
Regards,
Adolf.
>
> -Michael
>
>> On 16 Aug 2025, at 17:23, Adolf Belka <adolf.belka@ipfire.org> wrote:
>>
>> Hi Michael,
>>
>> After some time I thought I should have a go at building with your two patches for the unbound fast flux option but the build fails to complete.
>>
>> I am doing the build with the current next repo.
>>
>> I have done the build twice and it fails at the same point each time at the python3-tomli build.
>>
>> The following is the log output, which I have not been able to interpret to identify what is causing the build to fail.
>>
>>
>> Aug 16 15:11:49: Building python3-tomli make: Entering directory '/home/ahb/sandbox/ipfire-2-builder3/lfs'
>> make: Nothing to be done for 'download'.
>> make: Leaving directory '/home/ahb/sandbox/ipfire-2-builder3/lfs'
>> make: Entering directory '/usr/src/lfs'
>> tomli-2.0.1.tar.gz checksum OK
>> ====================================== Installing tomli-2.0.1 ...
>> Install started; saving file list to /usr/src/lsalr ...
>> cd /usr/src/tomli-2.0.1 && python3 -m pyproject2setuppy.main build
>> Traceback (most recent call last):
>> File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
>> return _run_code(code, main_globals, None,
>> File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
>> exec(code, run_globals)
>> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/main.py", line 15, in <module>
>> main()
>> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/__main__.py", line 53, in main
>> handler(data)
>> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/flit.py", line 91, in handle_flit
>> setup(**setup_metadata)
>> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 86, in setup
>> _install_setup_requires(attrs)
>> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires
>> dist = MinimalDistribution(attrs)
>> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 57, in __init__
>> super().__init__(filtered)
>> File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 474, in __init__
>> for ep in metadata.entry_points(group='distutils.setup_keywords'):
>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1009, in entry_points
>> return SelectableGroups.load(eps).select(**params)
>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 459, in load
>> ordered = sorted(eps, key=by_group)
>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1006, in <genexpr>
>> eps = itertools.chain.from_iterable(
>> File "/usr/lib/python3.10/importlib/metadata/_itertools.py", line 16, in unique_everseen
>> k = key(element)
>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 941, in _normalized_name
>> return self._name_from_stem(stem) or super()._normalized_name
>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 622, in _normalized_name
>> return Prepared.normalize(self.name)
>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 871, in normalize
>> return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_')
>> File "/usr/lib/python3.10/re.py", line 209, in sub
>> return _compile(pattern, flags).sub(repl, string, count)
>> TypeError: expected string or bytes-like object
>> make: *** [python3-tomli:78: /usr/src/log/tomli-2.0.1] Error 1
>> make: Leaving directory '/usr/src/lfs'
>>
>> Regards,
>>
>> Adolf.
>>
>> On 25/04/2025 14:37, Michael Tremer wrote:
>>> This can be used to script the resolver and use libunbound from Python.
>>> Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
>>> ---
>>> config/rootfiles/common/unbound | 4 ++++
>>> lfs/unbound | 3 +++
>>> make.sh | 2 +-
>>> 3 files changed, 8 insertions(+), 1 deletion(-)
>>> diff --git a/config/rootfiles/common/unbound b/config/rootfiles/common/unbound
>>> index 57390d6d9..03c956503 100644
>>> --- a/config/rootfiles/common/unbound
>>> +++ b/config/rootfiles/common/unbound
>>> @@ -13,6 +13,10 @@ etc/unbound/unbound.conf
>>> usr/lib/libunbound.so.8
>>> usr/lib/libunbound.so.8.1.30
>>> #usr/lib/pkgconfig/libunbound.pc
>>> +#usr/lib/python3.10/site-packages/_unbound.la
>>> +usr/lib/python3.10/site-packages/_unbound.so
>>> +usr/lib/python3.10/site-packages/unbound.py
>>> +usr/lib/python3.10/site-packages/unboundmodule.py
>>> usr/sbin/unbound
>>> usr/sbin/unbound-anchor
>>> usr/sbin/unbound-checkconf
>>> diff --git a/lfs/unbound b/lfs/unbound
>>> index 537ccff7e..8c79125ec 100644
>>> --- a/lfs/unbound
>>> +++ b/lfs/unbound
>>> @@ -71,6 +71,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
>>> @$(PREBUILD)
>>> @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE)
>>> cd $(DIR_APP) && \
>>> + PYTHON_VERSION=3 \
>>> ./configure \
>>> --prefix=/usr \
>>> --sysconfdir=/etc \
>>> @@ -83,6 +84,8 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
>>> --enable-tfo-server \
>>> --enable-ipset \
>>> --with-libmnl \
>>> + --with-pythonmodule \
>>> + --with-pyunbound \
>>> ac_cv_func_getentropy=no
>>> cd $(DIR_APP) && make $(MAKETUNING)
>>> diff --git a/make.sh b/make.sh
>>> index 3643b469e..3a673cd2b 100755
>>> --- a/make.sh
>>> +++ b/make.sh
>>> @@ -1534,6 +1534,7 @@ build_system() {
>>> lfsmake2 libevent2
>>> lfsmake2 apr
>>> lfsmake2 aprutil
>>> + lfsmake2 swig
>>> lfsmake2 unbound
>>> lfsmake2 gnutls
>>> lfsmake2 libuv
>>> @@ -1930,7 +1931,6 @@ build_system() {
>>> lfsmake2 minicom
>>> lfsmake2 ddrescue
>>> lfsmake2 parted
>>> - lfsmake2 swig
>>> lfsmake2 python3-pyelftools
>>> lfsmake2 u-boot
>>> lfsmake2 wireless-regdb
>>
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC PATCH 1/2] unbound: Enable the Python module(s)
2025-08-18 12:40 ` Adolf Belka
@ 2025-08-18 16:17 ` Adolf Belka
2025-08-18 17:00 ` Adolf Belka
0 siblings, 1 reply; 7+ messages in thread
From: Adolf Belka @ 2025-08-18 16:17 UTC (permalink / raw)
To: Michael Tremer; +Cc: IPFire: Development-List
Hi Michael,
So I tried swig-4.3.1 but dtc failed to build.
Looking up the failure message I believe it is the same error that u-boot would have had. A function was changed in swig-4.3.x to use 3 parameters instead of 2. The latest version of dtc and u-boot from Jan 2025 both have commits to resolve it.
So I will do a build with the newer dtc and test out the building of u-boot on aarch64 with the newer swig and a newer u-boot.
Regards,
Adolf.
On 18/08/2025 14:40, Adolf Belka wrote:
> Hi Michael,
>
> On 18/08/2025 13:50, Michael Tremer wrote:
>> Hello,
>>
>> I have the same problem.
>>
>> I have built this from a non-clean build initially and everything was fine. I used that version of Unbound to write and test the code. No problems there whatsoever.
>>
>> But I never managed to run a clean build. It seems that swig messes something up and I cannot tell you at all what that could be. It is not only tomli not compiling. If you comment it out and try to compile the next thing it won’t work either. I don’t know what it could break like this because the problems suddenly seem to occur in the Python standard library.
>>
>> Maybe an update of Swig solves the problem? Would you like to give that a try?
>
> The last swig update I provided had to be reverted because uboot would not build with it.
>
> However there is a new version out now, 4.3.1 from 2025-04-15, so I will give that a go and see if it helps and also do an aarch64 build and see if uboot builds or not.
>
> If it doesn't help or uboot fails to build then I will look at what other packages might need to be moved earlier to be before swig so that all the expected swig bits are built.
>
> Currently we have swig-4.1.0 and the reverted version was 4.3.0, so there are also other versions newer than the existing one that might allow uboot to build and help the issue we are having.
>
> Will have a go at it, although might take a while as I am cleaning my house up to make it child proof as my nephew and family are visiting from end of this week for 5 days.
>
> Regards,
>
> Adolf.
>
>>
>> -Michael
>>
>>> On 16 Aug 2025, at 17:23, Adolf Belka <adolf.belka@ipfire.org> wrote:
>>>
>>> Hi Michael,
>>>
>>> After some time I thought I should have a go at building with your two patches for the unbound fast flux option but the build fails to complete.
>>>
>>> I am doing the build with the current next repo.
>>>
>>> I have done the build twice and it fails at the same point each time at the python3-tomli build.
>>>
>>> The following is the log output, which I have not been able to interpret to identify what is causing the build to fail.
>>>
>>>
>>> Aug 16 15:11:49: Building python3-tomli make: Entering directory '/home/ahb/sandbox/ipfire-2-builder3/lfs'
>>> make: Nothing to be done for 'download'.
>>> make: Leaving directory '/home/ahb/sandbox/ipfire-2-builder3/lfs'
>>> make: Entering directory '/usr/src/lfs'
>>> tomli-2.0.1.tar.gz checksum OK
>>> ====================================== Installing tomli-2.0.1 ...
>>> Install started; saving file list to /usr/src/lsalr ...
>>> cd /usr/src/tomli-2.0.1 && python3 -m pyproject2setuppy.main build
>>> Traceback (most recent call last):
>>> File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
>>> return _run_code(code, main_globals, None,
>>> File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
>>> exec(code, run_globals)
>>> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/main.py", line 15, in <module>
>>> main()
>>> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/__main__.py", line 53, in main
>>> handler(data)
>>> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/flit.py", line 91, in handle_flit
>>> setup(**setup_metadata)
>>> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 86, in setup
>>> _install_setup_requires(attrs)
>>> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires
>>> dist = MinimalDistribution(attrs)
>>> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 57, in __init__
>>> super().__init__(filtered)
>>> File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 474, in __init__
>>> for ep in metadata.entry_points(group='distutils.setup_keywords'):
>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1009, in entry_points
>>> return SelectableGroups.load(eps).select(**params)
>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 459, in load
>>> ordered = sorted(eps, key=by_group)
>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1006, in <genexpr>
>>> eps = itertools.chain.from_iterable(
>>> File "/usr/lib/python3.10/importlib/metadata/_itertools.py", line 16, in unique_everseen
>>> k = key(element)
>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 941, in _normalized_name
>>> return self._name_from_stem(stem) or super()._normalized_name
>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 622, in _normalized_name
>>> return Prepared.normalize(self.name)
>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 871, in normalize
>>> return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_')
>>> File "/usr/lib/python3.10/re.py", line 209, in sub
>>> return _compile(pattern, flags).sub(repl, string, count)
>>> TypeError: expected string or bytes-like object
>>> make: *** [python3-tomli:78: /usr/src/log/tomli-2.0.1] Error 1
>>> make: Leaving directory '/usr/src/lfs'
>>>
>>> Regards,
>>>
>>> Adolf.
>>>
>>> On 25/04/2025 14:37, Michael Tremer wrote:
>>>> This can be used to script the resolver and use libunbound from Python.
>>>> Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
>>>> ---
>>>> config/rootfiles/common/unbound | 4 ++++
>>>> lfs/unbound | 3 +++
>>>> make.sh | 2 +-
>>>> 3 files changed, 8 insertions(+), 1 deletion(-)
>>>> diff --git a/config/rootfiles/common/unbound b/config/rootfiles/common/unbound
>>>> index 57390d6d9..03c956503 100644
>>>> --- a/config/rootfiles/common/unbound
>>>> +++ b/config/rootfiles/common/unbound
>>>> @@ -13,6 +13,10 @@ etc/unbound/unbound.conf
>>>> usr/lib/libunbound.so.8
>>>> usr/lib/libunbound.so.8.1.30
>>>> #usr/lib/pkgconfig/libunbound.pc
>>>> +#usr/lib/python3.10/site-packages/_unbound.la
>>>> +usr/lib/python3.10/site-packages/_unbound.so
>>>> +usr/lib/python3.10/site-packages/unbound.py
>>>> +usr/lib/python3.10/site-packages/unboundmodule.py
>>>> usr/sbin/unbound
>>>> usr/sbin/unbound-anchor
>>>> usr/sbin/unbound-checkconf
>>>> diff --git a/lfs/unbound b/lfs/unbound
>>>> index 537ccff7e..8c79125ec 100644
>>>> --- a/lfs/unbound
>>>> +++ b/lfs/unbound
>>>> @@ -71,6 +71,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
>>>> @$(PREBUILD)
>>>> @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE)
>>>> cd $(DIR_APP) && \
>>>> + PYTHON_VERSION=3 \
>>>> ./configure \
>>>> --prefix=/usr \
>>>> --sysconfdir=/etc \
>>>> @@ -83,6 +84,8 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
>>>> --enable-tfo-server \
>>>> --enable-ipset \
>>>> --with-libmnl \
>>>> + --with-pythonmodule \
>>>> + --with-pyunbound \
>>>> ac_cv_func_getentropy=no
>>>> cd $(DIR_APP) && make $(MAKETUNING)
>>>> diff --git a/make.sh b/make.sh
>>>> index 3643b469e..3a673cd2b 100755
>>>> --- a/make.sh
>>>> +++ b/make.sh
>>>> @@ -1534,6 +1534,7 @@ build_system() {
>>>> lfsmake2 libevent2
>>>> lfsmake2 apr
>>>> lfsmake2 aprutil
>>>> + lfsmake2 swig
>>>> lfsmake2 unbound
>>>> lfsmake2 gnutls
>>>> lfsmake2 libuv
>>>> @@ -1930,7 +1931,6 @@ build_system() {
>>>> lfsmake2 minicom
>>>> lfsmake2 ddrescue
>>>> lfsmake2 parted
>>>> - lfsmake2 swig
>>>> lfsmake2 python3-pyelftools
>>>> lfsmake2 u-boot
>>>> lfsmake2 wireless-regdb
>>>
>>
>>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC PATCH 1/2] unbound: Enable the Python module(s)
2025-08-18 16:17 ` Adolf Belka
@ 2025-08-18 17:00 ` Adolf Belka
0 siblings, 0 replies; 7+ messages in thread
From: Adolf Belka @ 2025-08-18 17:00 UTC (permalink / raw)
To: Michael Tremer; +Cc: IPFire: Development-List
Hi Michael,
On 18/08/2025 18:17, Adolf Belka wrote:
> Hi Michael,
>
> So I tried swig-4.3.1 but dtc failed to build.
>
> Looking up the failure message I believe it is the same error that u-boot would have had. A function was changed in swig-4.3.x to use 3 parameters instead of 2. The latest version of dtc and u-boot from Jan 2025 both have commits to resolve it.
>
> So I will do a build with the newer dtc and test out the building of u-boot on aarch64 with the newer swig and a newer u-boot.
The newer dtc built without problems but the python3-tomli still failed. So then I thought it might be because the tomli file was too old so I tried a newer version.
That also failed to build but with a different message which Seemed to ring a bell for problems with the pyproject2setuppy module as tomli only has pyproject.toml and no setup.py
So then did the build for python3-tomli using the newer commands for working with pyproject.toml however then it failed for no build module, so I put that before the python3-tomli file and did the build again but this time it failed for the same error message as we had with tomli originally
File "/usr/lib/python3.10/re.py", line 209, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object
I'll have a further think about these build issues.
Regards,
Adolf.
>
> Regards,
>
> Adolf.
>
>
> On 18/08/2025 14:40, Adolf Belka wrote:
>> Hi Michael,
>>
>> On 18/08/2025 13:50, Michael Tremer wrote:
>>> Hello,
>>>
>>> I have the same problem.
>>>
>>> I have built this from a non-clean build initially and everything was fine. I used that version of Unbound to write and test the code. No problems there whatsoever.
>>>
>>> But I never managed to run a clean build. It seems that swig messes something up and I cannot tell you at all what that could be. It is not only tomli not compiling. If you comment it out and try to compile the next thing it won’t work either. I don’t know what it could break like this because the problems suddenly seem to occur in the Python standard library.
>>>
>>> Maybe an update of Swig solves the problem? Would you like to give that a try?
>>
>> The last swig update I provided had to be reverted because uboot would not build with it.
>>
>> However there is a new version out now, 4.3.1 from 2025-04-15, so I will give that a go and see if it helps and also do an aarch64 build and see if uboot builds or not.
>>
>> If it doesn't help or uboot fails to build then I will look at what other packages might need to be moved earlier to be before swig so that all the expected swig bits are built.
>>
>> Currently we have swig-4.1.0 and the reverted version was 4.3.0, so there are also other versions newer than the existing one that might allow uboot to build and help the issue we are having.
>>
>> Will have a go at it, although might take a while as I am cleaning my house up to make it child proof as my nephew and family are visiting from end of this week for 5 days.
>>
>> Regards,
>>
>> Adolf.
>>
>>>
>>> -Michael
>>>
>>>> On 16 Aug 2025, at 17:23, Adolf Belka <adolf.belka@ipfire.org> wrote:
>>>>
>>>> Hi Michael,
>>>>
>>>> After some time I thought I should have a go at building with your two patches for the unbound fast flux option but the build fails to complete.
>>>>
>>>> I am doing the build with the current next repo.
>>>>
>>>> I have done the build twice and it fails at the same point each time at the python3-tomli build.
>>>>
>>>> The following is the log output, which I have not been able to interpret to identify what is causing the build to fail.
>>>>
>>>>
>>>> Aug 16 15:11:49: Building python3-tomli make: Entering directory '/home/ahb/sandbox/ipfire-2-builder3/lfs'
>>>> make: Nothing to be done for 'download'.
>>>> make: Leaving directory '/home/ahb/sandbox/ipfire-2-builder3/lfs'
>>>> make: Entering directory '/usr/src/lfs'
>>>> tomli-2.0.1.tar.gz checksum OK
>>>> ====================================== Installing tomli-2.0.1 ...
>>>> Install started; saving file list to /usr/src/lsalr ...
>>>> cd /usr/src/tomli-2.0.1 && python3 -m pyproject2setuppy.main build
>>>> Traceback (most recent call last):
>>>> File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
>>>> return _run_code(code, main_globals, None,
>>>> File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
>>>> exec(code, run_globals)
>>>> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/main.py", line 15, in <module>
>>>> main()
>>>> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/__main__.py", line 53, in main
>>>> handler(data)
>>>> File "/usr/lib/python3.10/site-packages/pyproject2setuppy/flit.py", line 91, in handle_flit
>>>> setup(**setup_metadata)
>>>> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 86, in setup
>>>> _install_setup_requires(attrs)
>>>> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires
>>>> dist = MinimalDistribution(attrs)
>>>> File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 57, in __init__
>>>> super().__init__(filtered)
>>>> File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 474, in __init__
>>>> for ep in metadata.entry_points(group='distutils.setup_keywords'):
>>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1009, in entry_points
>>>> return SelectableGroups.load(eps).select(**params)
>>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 459, in load
>>>> ordered = sorted(eps, key=by_group)
>>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 1006, in <genexpr>
>>>> eps = itertools.chain.from_iterable(
>>>> File "/usr/lib/python3.10/importlib/metadata/_itertools.py", line 16, in unique_everseen
>>>> k = key(element)
>>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 941, in _normalized_name
>>>> return self._name_from_stem(stem) or super()._normalized_name
>>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 622, in _normalized_name
>>>> return Prepared.normalize(self.name)
>>>> File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 871, in normalize
>>>> return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_')
>>>> File "/usr/lib/python3.10/re.py", line 209, in sub
>>>> return _compile(pattern, flags).sub(repl, string, count)
>>>> TypeError: expected string or bytes-like object
>>>> make: *** [python3-tomli:78: /usr/src/log/tomli-2.0.1] Error 1
>>>> make: Leaving directory '/usr/src/lfs'
>>>>
>>>> Regards,
>>>>
>>>> Adolf.
>>>>
>>>> On 25/04/2025 14:37, Michael Tremer wrote:
>>>>> This can be used to script the resolver and use libunbound from Python.
>>>>> Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
>>>>> ---
>>>>> config/rootfiles/common/unbound | 4 ++++
>>>>> lfs/unbound | 3 +++
>>>>> make.sh | 2 +-
>>>>> 3 files changed, 8 insertions(+), 1 deletion(-)
>>>>> diff --git a/config/rootfiles/common/unbound b/config/rootfiles/common/unbound
>>>>> index 57390d6d9..03c956503 100644
>>>>> --- a/config/rootfiles/common/unbound
>>>>> +++ b/config/rootfiles/common/unbound
>>>>> @@ -13,6 +13,10 @@ etc/unbound/unbound.conf
>>>>> usr/lib/libunbound.so.8
>>>>> usr/lib/libunbound.so.8.1.30
>>>>> #usr/lib/pkgconfig/libunbound.pc
>>>>> +#usr/lib/python3.10/site-packages/_unbound.la
>>>>> +usr/lib/python3.10/site-packages/_unbound.so
>>>>> +usr/lib/python3.10/site-packages/unbound.py
>>>>> +usr/lib/python3.10/site-packages/unboundmodule.py
>>>>> usr/sbin/unbound
>>>>> usr/sbin/unbound-anchor
>>>>> usr/sbin/unbound-checkconf
>>>>> diff --git a/lfs/unbound b/lfs/unbound
>>>>> index 537ccff7e..8c79125ec 100644
>>>>> --- a/lfs/unbound
>>>>> +++ b/lfs/unbound
>>>>> @@ -71,6 +71,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
>>>>> @$(PREBUILD)
>>>>> @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE)
>>>>> cd $(DIR_APP) && \
>>>>> + PYTHON_VERSION=3 \
>>>>> ./configure \
>>>>> --prefix=/usr \
>>>>> --sysconfdir=/etc \
>>>>> @@ -83,6 +84,8 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
>>>>> --enable-tfo-server \
>>>>> --enable-ipset \
>>>>> --with-libmnl \
>>>>> + --with-pythonmodule \
>>>>> + --with-pyunbound \
>>>>> ac_cv_func_getentropy=no
>>>>> cd $(DIR_APP) && make $(MAKETUNING)
>>>>> diff --git a/make.sh b/make.sh
>>>>> index 3643b469e..3a673cd2b 100755
>>>>> --- a/make.sh
>>>>> +++ b/make.sh
>>>>> @@ -1534,6 +1534,7 @@ build_system() {
>>>>> lfsmake2 libevent2
>>>>> lfsmake2 apr
>>>>> lfsmake2 aprutil
>>>>> + lfsmake2 swig
>>>>> lfsmake2 unbound
>>>>> lfsmake2 gnutls
>>>>> lfsmake2 libuv
>>>>> @@ -1930,7 +1931,6 @@ build_system() {
>>>>> lfsmake2 minicom
>>>>> lfsmake2 ddrescue
>>>>> lfsmake2 parted
>>>>> - lfsmake2 swig
>>>>> lfsmake2 python3-pyelftools
>>>>> lfsmake2 u-boot
>>>>> lfsmake2 wireless-regdb
>>>>
>>>
>>>
>>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2025-08-18 17:00 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-04-25 12:37 [RFC PATCH 1/2] unbound: Enable the Python module(s) Michael Tremer
2025-04-25 12:37 ` [RFC PATCH 2/2] Add Fast Flux detection in DNS Michael Tremer
2025-08-16 16:23 ` [RFC PATCH 1/2] unbound: Enable the Python module(s) Adolf Belka
2025-08-18 11:50 ` Michael Tremer
2025-08-18 12:40 ` Adolf Belka
2025-08-18 16:17 ` Adolf Belka
2025-08-18 17:00 ` Adolf Belka
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox