Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi Michael,
I am up for helping. I suspect the ones written in python2 that need to be converted to python3 might not be so good for me. I have never programmed in python at all.
I will have a go with some of the packages and see how things go.
I can start with something like fetchmail and see if I run into any problems or not.
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi all,
Looking at fetchmail it looks like I just need to set an environment variable to python3. However, I have also realised that the current version of fetchmail is from 2013 and is marked obsolete so I will also do an update to the current version. Fetchmail from 2013 is definitely not a secure version to be using.
Will let you know how things go.
Regards,
Adolf.
On 05/05/2021 18:49, Adolf Belka wrote:
Hi Michael,
I am up for helping. I suspect the ones written in python2 that need to be converted to python3 might not be so good for me. I have never programmed in python at all.
I will have a go with some of the packages and see how things go.
I can start with something like fetchmail and see if I run into any problems or not.
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi All,
fetchmail looks to have worked with no problems. I have submitted a patch for it.
I have also looked at python-daemon and if it is updated to the latest version then it requires a minimum python of 3.0 so that should also work fine.
I will try that package tomorrow morning.
Regards,
Adolf.
On 05/05/2021 19:12, Adolf Belka wrote:
Hi all,
Looking at fetchmail it looks like I just need to set an environment variable to python3. However, I have also realised that the current version of fetchmail is from 2013 and is marked obsolete so I will also do an update to the current version. Fetchmail from 2013 is definitely not a secure version to be using.
Will let you know how things go.
Regards,
Adolf.
On 05/05/2021 18:49, Adolf Belka wrote:
Hi Michael,
I am up for helping. I suspect the ones written in python2 that need to be converted to python3 might not be so good for me. I have never programmed in python at all.
I will have a go with some of the packages and see how things go.
I can start with something like fetchmail and see if I run into any problems or not.
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hello,
On 5 May 2021, at 17:49, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
I am up for helping. I suspect the ones written in python2 that need to be converted to python3 might not be so good for me. I have never programmed in python at all.
I will have a go with some of the packages and see how things go.
Yeah, so with most of the packages, we would need to disable Python 2 and enable Python 3. That is it. No programming.
I can start with something like fetchmail and see if I run into any problems or not.
Yes! Brilliant :)
-Michael
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
-- Sent from my laptop
Hi Michael,
On 06/05/2021 00:30, Michael Tremer wrote:
Hello,
On 5 May 2021, at 17:49, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
I am up for helping. I suspect the ones written in python2 that need to be converted to python3 might not be so good for me. I have never programmed in python at all.
I will have a go with some of the packages and see how things go.
Yeah, so with most of the packages, we would need to disable Python 2 and enable Python 3. That is it. No programming.
So I am working with boost now and it looks like I have successfully built that using python3
One thing I noticed is that boost also has the static libraries created but then commented out in the rootfile. Should these be disabled. There is a parameter to do that except for two libraries that are only built as static. The BLFS page on boost suggests that most people will not need the static libraries so it would look like I should disable them.
Regards, Adolf.
I can start with something like fetchmail and see if I run into any problems or not.
Yes! Brilliant :)
-Michael
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
-- Sent from my laptop
Hello,
On 6 May 2021, at 20:40, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
On 06/05/2021 00:30, Michael Tremer wrote:
Hello,
On 5 May 2021, at 17:49, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
I am up for helping. I suspect the ones written in python2 that need to be converted to python3 might not be so good for me. I have never programmed in python at all.
I will have a go with some of the packages and see how things go.
Yeah, so with most of the packages, we would need to disable Python 2 and enable Python 3. That is it. No programming.
So I am working with boost now and it looks like I have successfully built that using python3
Excellent.
One thing I noticed is that boost also has the static libraries created but then commented out in the rootfile. Should these be disabled. There is a parameter to do that except for two libraries that are only built as static. The BLFS page on boost suggests that most people will not need the static libraries so it would look like I should disable them.
Yes, you can disable them wherever you stumble over them.
Best, -Michael
Regards, Adolf.
I can start with something like fetchmail and see if I run into any problems or not.
Yes! Brilliant :) -Michael
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
-- Sent from my laptop
Hello,
I went ahead and converted fireinfo to Python 3.
This was a little bit more work than usual because it has a little helper module written in C.
I could mainly use 2to3, but some things needed some manual attention. I split those into individual commits.
I would now like to invite everyone to manually install this version on their systems and let me know if it
a) works
and
b) does it keep the same ID?
The ID is being generated by a number of factors and if one bit in those changes, the system would receive a different ID which is something we would like to avoid.
It worked fine on my system, but I would like to have some larger sample size to be sure.
Best, -Michael
Signed-off-by: Michael Tremer michael.tremer@ipfire.org --- configure.ac | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/configure.ac b/configure.ac index 4696d6b..1ce3329 100644 --- a/configure.ac +++ b/configure.ac @@ -90,7 +90,7 @@ CC_CHECK_FLAGS_APPEND([with_ldflags], [LDFLAGS], [\ AC_SUBST([OUR_LDFLAGS], $with_ldflags)
# Python -AM_PATH_PYTHON([2.7]) +AM_PATH_PYTHON([3.7]) PKG_CHECK_MODULES([PYTHON_DEVEL], [python-${PYTHON_VERSION}])
AC_CONFIG_FILES([
Signed-off-by: Michael Tremer michael.tremer@ipfire.org --- src/_fireinfo/fireinfo.c | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-)
diff --git a/src/_fireinfo/fireinfo.c b/src/_fireinfo/fireinfo.c index 6601c21..58ee7e2 100644 --- a/src/_fireinfo/fireinfo.c +++ b/src/_fireinfo/fireinfo.c @@ -185,7 +185,7 @@ do_detect_hypervisor() { if (!hypervisor_vendor) Py_RETURN_NONE;
- return PyString_FromString(hypervisor_vendor); + return PyUnicode_FromString(hypervisor_vendor); }
Py_RETURN_NONE; @@ -215,22 +215,30 @@ do_get_harddisk_serial(PyObject *o, PyObject *args) { strncpy(serial, (const char *)hd.serial_no, sizeof(serial) - 1);
if (serial[0]) - return PyString_FromString(serial); + return PyUnicode_FromString(serial); }
Py_RETURN_NONE; }
-static PyMethodDef fireinfoModuleMethods[] = { +static PyMethodDef fireinfo_methods[] = { { "detect_hypervisor", (PyCFunction) do_detect_hypervisor, METH_NOARGS, NULL }, { "get_harddisk_serial", (PyCFunction) do_get_harddisk_serial, METH_VARARGS, NULL }, { NULL, NULL, 0, NULL } };
-void init_fireinfo(void) { - PyObject *m; +static struct PyModuleDef fireinfo_module = { + .m_base = PyModuleDef_HEAD_INIT, + .m_name = "_fireinfo", + .m_size = -1, + .m_doc = "Python module for fireinfo", + .m_methods = fireinfo_methods, +}; + +PyMODINIT_FUNC PyInit__fireinfo(void) { + PyObject* m = PyModule_Create(&fireinfo_module); + if (!m) + return NULL;
- m = Py_InitModule("_fireinfo", fireinfoModuleMethods); - if (m == NULL) - return; + return m; }
No functional changes. This only makes the code more similar to what I am writing in other projects and makes it tidier.
Signed-off-by: Michael Tremer michael.tremer@ipfire.org --- src/_fireinfo/fireinfo.c | 35 ++++++++++++++++++----------------- 1 file changed, 18 insertions(+), 17 deletions(-)
diff --git a/src/_fireinfo/fireinfo.c b/src/_fireinfo/fireinfo.c index 58ee7e2..75ccb2c 100644 --- a/src/_fireinfo/fireinfo.c +++ b/src/_fireinfo/fireinfo.c @@ -58,23 +58,22 @@ const char *hypervisor_vendors[] = {
#define NEWLINE "\n\r"
-char *truncate_nl(char *s) { +static void truncate_nl(char *s) { assert(s);
- s[strcspn(s, NEWLINE)] = 0; - return s; + s[strcspn(s, NEWLINE)] = '\0'; }
-int read_one_line_file(const char *filename, char **line) { - assert(filename); - assert(line); +static int read_one_line_file(const char *filename, char **line) { + char t[2048]; + + if (!filename || !line) + return -EINVAL;
- FILE *f = NULL; - f = fopen(filename, "re"); + FILE* f = fopen(filename, "re"); if (!f) return -errno;
- char t[2048]; if (!fgets(t, sizeof(t), f)) { if (ferror(f)) return errno ? -errno : -EIO; @@ -85,6 +84,7 @@ int read_one_line_file(const char *filename, char **line) { char *c = strdup(t); if (!c) return -ENOMEM; + truncate_nl(c);
*line = c; @@ -171,7 +171,6 @@ int detect_hypervisor(int *hypervisor) { return 0; }
- static PyObject * do_detect_hypervisor() { /* @@ -197,27 +196,29 @@ do_get_harddisk_serial(PyObject *o, PyObject *args) { Python wrapper around read_harddisk_serial. */ static struct hd_driveid hd; - int fd; const char *device = NULL; + char serial[21];
if (!PyArg_ParseTuple(args, "s", &device)) return NULL;
- if ((fd = open(device, O_RDONLY | O_NONBLOCK)) < 0) { + int fd = open(device, O_RDONLY | O_NONBLOCK); + if (fd < 0) { PyErr_Format(PyExc_OSError, "Could not open block device: %s", device); return NULL; }
if (!ioctl(fd, HDIO_GET_IDENTITY, &hd)) { - char serial[21]; - memset(serial, 0, sizeof(serial)); - - strncpy(serial, (const char *)hd.serial_no, sizeof(serial) - 1); + snprintf(serial, sizeof(serial) - 1, "%s", (const char *)hd.serial_no);
- if (serial[0]) + if (*serial) { + close(fd); return PyUnicode_FromString(serial); + } }
+ close(fd); + Py_RETURN_NONE; }
This is the result of running the automatic conversion tool and I have not tested this, yet. There might be some bugs being introduced by this.
Signed-off-by: Michael Tremer michael.tremer@ipfire.org --- src/fireinfo/cpu.py | 20 ++++++++++---------- src/fireinfo/hypervisor.py | 6 +++--- src/fireinfo/network.py | 8 ++++---- src/fireinfo/system.py | 38 ++++++++++++++++++-------------------- src/sendprofile | 34 +++++++++++++++++----------------- 5 files changed, 52 insertions(+), 54 deletions(-)
diff --git a/src/fireinfo/cpu.py b/src/fireinfo/cpu.py index dc76caf..e433a04 100644 --- a/src/fireinfo/cpu.py +++ b/src/fireinfo/cpu.py @@ -21,7 +21,7 @@
import os
-import system +from . import system
PROC_CPUINFO = "/proc/cpuinfo"
@@ -188,12 +188,12 @@ class CPU(object): if __name__ == "__main__": c = CPU()
- print "Vendor:", c.vendor - print "Model:", c.model - print "Stepping:", c.stepping - print "Flags:", c.flags - print "Bogomips:", c.bogomips - print "Speed:", c.speed - print "Family:", c.family - print "Count:", c.count - print "Model string:", c.model_string + print("Vendor:", c.vendor) + print("Model:", c.model) + print("Stepping:", c.stepping) + print("Flags:", c.flags) + print("Bogomips:", c.bogomips) + print("Speed:", c.speed) + print("Family:", c.family) + print("Count:", c.count) + print("Model string:", c.model_string) diff --git a/src/fireinfo/hypervisor.py b/src/fireinfo/hypervisor.py index 0c07cfa..e76aad5 100644 --- a/src/fireinfo/hypervisor.py +++ b/src/fireinfo/hypervisor.py @@ -20,7 +20,7 @@ ###############################################################################
import _fireinfo -import system +from . import system
class Hypervisor(object): def __init__(self): @@ -115,5 +115,5 @@ class Hypervisor(object): if __name__ == "__main__": h = Hypervisor()
- print "Vendor:", h.vendor - print "Virtual:", h.virtual + print("Vendor:", h.vendor) + print("Virtual:", h.virtual) diff --git a/src/fireinfo/network.py b/src/fireinfo/network.py index 063e9ec..de58be3 100644 --- a/src/fireinfo/network.py +++ b/src/fireinfo/network.py @@ -47,7 +47,7 @@ class Network(object): if __name__ == "__main__": n = Network()
- print "has_green", n.has_green() - print "has_red", n.has_red() - print "has_blue", n.has_blue() - print "has_orange", n.has_orange() + print("has_green", n.has_green()) + print("has_red", n.has_red()) + print("has_blue", n.has_blue()) + print("has_orange", n.has_orange()) diff --git a/src/fireinfo/system.py b/src/fireinfo/system.py index 73c3882..cfeb171 100644 --- a/src/fireinfo/system.py +++ b/src/fireinfo/system.py @@ -26,11 +26,11 @@ import string
import _fireinfo
-import bios -import cpu -import device -import hypervisor -import network +from . import bios +from . import cpu +from . import device +from . import hypervisor +from . import network
PROFILE_VERSION = 0
@@ -84,9 +84,7 @@ def read_from_file(filename): except IOError: pass
-class System(object): - __metaclass__ = Singleton - +class System(object, metaclass=Singleton): def __init__(self): self.bios = bios.BIOS(self)
@@ -338,7 +336,7 @@ class System(object): if s is None: return
- return filter(lambda x: x in string.printable, s) + return [x for x in s if x in string.printable]
@property def vendor(self): @@ -373,7 +371,7 @@ class System(object): ret = read_from_file("/proc/device-tree/model") if ret: # replace the NULL byte with which the DT string ends - ret = ret.replace(u"\u0000", "") + ret = ret.replace("\u0000", "")
# Fall back to read /proc/cpuinfo if not ret: @@ -485,13 +483,13 @@ class System(object):
if __name__ == "__main__": s=System() - print s.arch - print s.language - print s.release - print s.bios_vendor - print s.memory - print s.kernel - print s.root_disk - print s.root_size - print "------------\n", s.devices, "\n------------\n" - print json.dumps(s.profile(), sort_keys=True, indent=4) + print(s.arch) + print(s.language) + print(s.release) + print(s.bios_vendor) + print(s.memory) + print(s.kernel) + print(s.root_disk) + print(s.root_size) + print("------------\n", s.devices, "\n------------\n") + print(json.dumps(s.profile(), sort_keys=True, indent=4)) diff --git a/src/sendprofile b/src/sendprofile index 3ce68b9..5fc6c86 100755 --- a/src/sendprofile +++ b/src/sendprofile @@ -24,8 +24,8 @@ import logging import logging.handlers import os import sys -import urllib -import urllib2 +import urllib.request, urllib.parse, urllib.error +import urllib.request, urllib.error, urllib.parse
import fireinfo
@@ -67,8 +67,8 @@ def send_profile(profile): for line in json.dumps(profile, sort_keys=True, indent=4).splitlines(): logging.debug(line)
- request = urllib2.Request(PROFILE_URL % profile, - data = urllib.urlencode({"profile" : json.dumps(profile)}), + request = urllib.request.Request(PROFILE_URL % profile, + data = urllib.parse.urlencode({"profile" : json.dumps(profile)}), ) request.add_header("User-Agent", "fireinfo/%s" % fireinfo.__version__)
@@ -80,22 +80,22 @@ def send_profile(profile): if proxy["user"] and proxy["pass"]: prx_auth_string = "http://%s:%s@%s/" % (proxy["user"], proxy["pass"], proxy["host"])
- proxy_handler = urllib2.ProxyHandler({'http': prx_auth_string, 'https': prx_auth_string}) - auth = urllib2.HTTPBasicAuthHandler() - opener = urllib2.build_opener(proxy_handler, auth, urllib2.HTTPHandler) - urllib2.install_opener(opener) + proxy_handler = urllib.request.ProxyHandler({'http': prx_auth_string, 'https': prx_auth_string}) + auth = urllib.request.HTTPBasicAuthHandler() + opener = urllib.request.build_opener(proxy_handler, auth, urllib.request.HTTPHandler) + urllib.request.install_opener(opener) else: request.set_proxy(proxy["host"], "http") request.set_proxy(proxy["host"], "https")
try: - urllib2.urlopen(request, timeout=60) - except (urllib2.HTTPError, urllib2.URLError), e: + urllib.request.urlopen(request, timeout=60) + except (urllib.error.HTTPError, urllib.error.URLError) as e: reason = "Unknown reason"
- if isinstance(e, urllib2.HTTPError): + if isinstance(e, urllib.error.HTTPError): reason = "%s" % e - elif isinstance(e, urllib2.URLError): + elif isinstance(e, urllib.error.URLError): reason = e.reason
logging.error("Profile was not sent propertly: %s" % reason) @@ -114,19 +114,19 @@ def main(): # it contains no information and may confuse people. del profile["private_id"]
- print json.dumps(profile, sort_keys=True, indent=4) + print(json.dumps(profile, sort_keys=True, indent=4)) return 0
if "--secret-id" in sys.argv: - print system.secret_id + print(system.secret_id) return 0
if "--hardware-string" in sys.argv: - print system._unique_id + print(system._unique_id) return 0
if "--public-id" in sys.argv: - print system.public_id + print(system.public_id) return 0
if not os.path.exists(ENABLED_FILE): @@ -134,7 +134,7 @@ def main():
try: send_profile(profile) - except urllib2.URLError: + except urllib.error.URLError: return 1
return 0
Hello,
Obviously there should have been more patches after this one.
Unfortunately the rate-limiting feature of our mail server fucked me over. Please find the rest on here:
https://git.ipfire.org/?p=oddments/fireinfo.git;a=shortlog;h=refs/heads/pyth...
-Michael
On 7 May 2021, at 12:16, Michael Tremer michael.tremer@ipfire.org wrote:
This is the result of running the automatic conversion tool and I have not tested this, yet. There might be some bugs being introduced by this.
Signed-off-by: Michael Tremer michael.tremer@ipfire.org
src/fireinfo/cpu.py | 20 ++++++++++---------- src/fireinfo/hypervisor.py | 6 +++--- src/fireinfo/network.py | 8 ++++---- src/fireinfo/system.py | 38 ++++++++++++++++++-------------------- src/sendprofile | 34 +++++++++++++++++----------------- 5 files changed, 52 insertions(+), 54 deletions(-)
diff --git a/src/fireinfo/cpu.py b/src/fireinfo/cpu.py index dc76caf..e433a04 100644 --- a/src/fireinfo/cpu.py +++ b/src/fireinfo/cpu.py @@ -21,7 +21,7 @@
import os
-import system +from . import system
PROC_CPUINFO = "/proc/cpuinfo"
@@ -188,12 +188,12 @@ class CPU(object): if __name__ == "__main__": c = CPU()
- print "Vendor:", c.vendor
- print "Model:", c.model
- print "Stepping:", c.stepping
- print "Flags:", c.flags
- print "Bogomips:", c.bogomips
- print "Speed:", c.speed
- print "Family:", c.family
- print "Count:", c.count
- print "Model string:", c.model_string
- print("Vendor:", c.vendor)
- print("Model:", c.model)
- print("Stepping:", c.stepping)
- print("Flags:", c.flags)
- print("Bogomips:", c.bogomips)
- print("Speed:", c.speed)
- print("Family:", c.family)
- print("Count:", c.count)
- print("Model string:", c.model_string)
diff --git a/src/fireinfo/hypervisor.py b/src/fireinfo/hypervisor.py index 0c07cfa..e76aad5 100644 --- a/src/fireinfo/hypervisor.py +++ b/src/fireinfo/hypervisor.py @@ -20,7 +20,7 @@ ###############################################################################
import _fireinfo -import system +from . import system
class Hypervisor(object): def __init__(self): @@ -115,5 +115,5 @@ class Hypervisor(object): if __name__ == "__main__": h = Hypervisor()
- print "Vendor:", h.vendor
- print "Virtual:", h.virtual
- print("Vendor:", h.vendor)
- print("Virtual:", h.virtual)
diff --git a/src/fireinfo/network.py b/src/fireinfo/network.py index 063e9ec..de58be3 100644 --- a/src/fireinfo/network.py +++ b/src/fireinfo/network.py @@ -47,7 +47,7 @@ class Network(object): if __name__ == "__main__": n = Network()
- print "has_green", n.has_green()
- print "has_red", n.has_red()
- print "has_blue", n.has_blue()
- print "has_orange", n.has_orange()
- print("has_green", n.has_green())
- print("has_red", n.has_red())
- print("has_blue", n.has_blue())
- print("has_orange", n.has_orange())
diff --git a/src/fireinfo/system.py b/src/fireinfo/system.py index 73c3882..cfeb171 100644 --- a/src/fireinfo/system.py +++ b/src/fireinfo/system.py @@ -26,11 +26,11 @@ import string
import _fireinfo
-import bios -import cpu -import device -import hypervisor -import network +from . import bios +from . import cpu +from . import device +from . import hypervisor +from . import network
PROFILE_VERSION = 0
@@ -84,9 +84,7 @@ def read_from_file(filename): except IOError: pass
-class System(object):
- __metaclass__ = Singleton
+class System(object, metaclass=Singleton): def __init__(self): self.bios = bios.BIOS(self)
@@ -338,7 +336,7 @@ class System(object): if s is None: return
return filter(lambda x: x in string.printable, s)
return [x for x in s if x in string.printable]@property def vendor(self):
@@ -373,7 +371,7 @@ class System(object): ret = read_from_file("/proc/device-tree/model") if ret: # replace the NULL byte with which the DT string ends
ret = ret.replace(u"\u0000", "")
ret = ret.replace("\u0000", "")# Fall back to read /proc/cpuinfo if not ret:
@@ -485,13 +483,13 @@ class System(object):
if __name__ == "__main__": s=System()
- print s.arch
- print s.language
- print s.release
- print s.bios_vendor
- print s.memory
- print s.kernel
- print s.root_disk
- print s.root_size
- print "------------\n", s.devices, "\n------------\n"
- print json.dumps(s.profile(), sort_keys=True, indent=4)
- print(s.arch)
- print(s.language)
- print(s.release)
- print(s.bios_vendor)
- print(s.memory)
- print(s.kernel)
- print(s.root_disk)
- print(s.root_size)
- print("------------\n", s.devices, "\n------------\n")
- print(json.dumps(s.profile(), sort_keys=True, indent=4))
diff --git a/src/sendprofile b/src/sendprofile index 3ce68b9..5fc6c86 100755 --- a/src/sendprofile +++ b/src/sendprofile @@ -24,8 +24,8 @@ import logging import logging.handlers import os import sys -import urllib -import urllib2 +import urllib.request, urllib.parse, urllib.error +import urllib.request, urllib.error, urllib.parse
import fireinfo
@@ -67,8 +67,8 @@ def send_profile(profile): for line in json.dumps(profile, sort_keys=True, indent=4).splitlines(): logging.debug(line)
- request = urllib2.Request(PROFILE_URL % profile,
data = urllib.urlencode({"profile" : json.dumps(profile)}),
- request = urllib.request.Request(PROFILE_URL % profile,
data = urllib.parse.urlencode({"profile" : json.dumps(profile)}),@@ -80,22 +80,22 @@ def send_profile(profile): if proxy["user"] and proxy["pass"]: prx_auth_string = "http://%s:%s@%s/" % (proxy["user"], proxy["pass"], proxy["host"])
proxy_handler = urllib2.ProxyHandler({'http': prx_auth_string, 'https': prx_auth_string})auth = urllib2.HTTPBasicAuthHandler()opener = urllib2.build_opener(proxy_handler, auth, urllib2.HTTPHandler)urllib2.install_opener(opener)
proxy_handler = urllib.request.ProxyHandler({'http': prx_auth_string, 'https': prx_auth_string})auth = urllib.request.HTTPBasicAuthHandler()opener = urllib.request.build_opener(proxy_handler, auth, urllib.request.HTTPHandler)urllib.request.install_opener(opener)else: request.set_proxy(proxy["host"], "http") request.set_proxy(proxy["host"], "https")
try:
urllib2.urlopen(request, timeout=60)- except (urllib2.HTTPError, urllib2.URLError), e:
urllib.request.urlopen(request, timeout=60)- except (urllib.error.HTTPError, urllib.error.URLError) as e: reason = "Unknown reason"
if isinstance(e, urllib2.HTTPError):
if isinstance(e, urllib.error.HTTPError): reason = "%s" % e
elif isinstance(e, urllib2.URLError):
elif isinstance(e, urllib.error.URLError): reason = e.reasonlogging.error("Profile was not sent propertly: %s" % reason)
@@ -114,19 +114,19 @@ def main(): # it contains no information and may confuse people. del profile["private_id"]
print json.dumps(profile, sort_keys=True, indent=4)
print(json.dumps(profile, sort_keys=True, indent=4))return 0
if "--secret-id" in sys.argv:
print system.secret_id
print(system.secret_id)return 0
if "--hardware-string" in sys.argv:
print system._unique_id
print(system._unique_id)return 0
if "--public-id" in sys.argv:
print system.public_id
print(system.public_id)return 0
if not os.path.exists(ENABLED_FILE):
@@ -134,7 +134,7 @@ def main():
try: send_profile(profile)
- except urllib2.URLError:
except urllib.error.URLError: return 1
return 0
-- 2.20.1
Hi All,
On 07/05/2021 13:16, Michael Tremer wrote:
Hello,
I went ahead and converted fireinfo to Python 3.
This was a little bit more work than usual because it has a little helper module written in C.
I could mainly use 2to3, but some things needed some manual attention. I split those into individual commits.
I would now like to invite everyone to manually install this version on their systems and let me know if it
The patches are for application to the build system. I can do the patching and then do a clean build but what do I then need to manually install. Is it just the fireinfo.cgi file that I need to install or are there other files I need to copy/replace into my testbed system?
Regards,
Adolf.
a) works
and
b) does it keep the same ID?
The ID is being generated by a number of factors and if one bit in those changes, the system would receive a different ID which is something we would like to avoid.
It worked fine on my system, but I would like to have some larger sample size to be sure.
Best, -Michael
Hello,
You can use Stefan’s build if you are on x86_64:
https://people.ipfire.org/~stevee/fireinfo_python3_x86_64.tar.gz
He was even very smart and renamed the “sendprofile” command into “sendprofile3” so that you can run both versions and compare.
I didn’t upload a binary version here because I do not want to encourage to download and extract tarballs as root on the firewall. But it should be fine on a development system :)
-Michael
On 14 May 2021, at 13:10, Adolf Belka adolf.belka@ipfire.org wrote:
Hi All,
On 07/05/2021 13:16, Michael Tremer wrote:
Hello,
I went ahead and converted fireinfo to Python 3.
This was a little bit more work than usual because it has a little helper module written in C.
I could mainly use 2to3, but some things needed some manual attention. I split those into individual commits.
I would now like to invite everyone to manually install this version on their systems and let me know if it
The patches are for application to the build system. I can do the patching and then do a clean build but what do I then need to manually install. Is it just the fireinfo.cgi file that I need to install or are there other files I need to copy/replace into my testbed system?
Regards,
Adolf.
a) works
and
b) does it keep the same ID?
The ID is being generated by a number of factors and if one bit in those changes, the system would receive a different ID which is something we would like to avoid.
It worked fine on my system, but I would like to have some larger sample size to be sure.
Best, -Michael
Hi Michael,
I ran the python3 fireinfo on my vm testbed system and can confirm that the id stayed the same as with the original python2 files. Both links also worked and gave me the same web page in terms of system details.
Regards,
Adolf.
On 14/05/2021 14:13, Michael Tremer wrote:
Hello,
You can use Stefan’s build if you are on x86_64:
https://people.ipfire.org/~stevee/fireinfo_python3_x86_64.tar.gz
He was even very smart and renamed the “sendprofile” command into “sendprofile3” so that you can run both versions and compare.
I didn’t upload a binary version here because I do not want to encourage to download and extract tarballs as root on the firewall. But it should be fine on a development system :)
-Michael
On 14 May 2021, at 13:10, Adolf Belka adolf.belka@ipfire.org wrote:
Hi All,
On 07/05/2021 13:16, Michael Tremer wrote:
Hello,
I went ahead and converted fireinfo to Python 3.
This was a little bit more work than usual because it has a little helper module written in C.
I could mainly use 2to3, but some things needed some manual attention. I split those into individual commits.
I would now like to invite everyone to manually install this version on their systems and let me know if it
The patches are for application to the build system. I can do the patching and then do a clean build but what do I then need to manually install. Is it just the fireinfo.cgi file that I need to install or are there other files I need to copy/replace into my testbed system?
Regards,
Adolf.
a) works
and
b) does it keep the same ID?
The ID is being generated by a number of factors and if one bit in those changes, the system would receive a different ID which is something we would like to avoid.
It worked fine on my system, but I would like to have some larger sample size to be sure.
Best, -Michael
Hello,
Thanks for the feedback.
I will tag a release then and schedule it to be included in 158.
Best, -Michael
On 17 May 2021, at 14:59, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
I ran the python3 fireinfo on my vm testbed system and can confirm that the id stayed the same as with the original python2 files. Both links also worked and gave me the same web page in terms of system details.
Regards,
Adolf.
On 14/05/2021 14:13, Michael Tremer wrote:
Hello, You can use Stefan’s build if you are on x86_64: https://people.ipfire.org/~stevee/fireinfo_python3_x86_64.tar.gz He was even very smart and renamed the “sendprofile” command into “sendprofile3” so that you can run both versions and compare. I didn’t upload a binary version here because I do not want to encourage to download and extract tarballs as root on the firewall. But it should be fine on a development system :) -Michael
On 14 May 2021, at 13:10, Adolf Belka adolf.belka@ipfire.org wrote:
Hi All,
On 07/05/2021 13:16, Michael Tremer wrote:
Hello,
I went ahead and converted fireinfo to Python 3.
This was a little bit more work than usual because it has a little helper module written in C.
I could mainly use 2to3, but some things needed some manual attention. I split those into individual commits.
I would now like to invite everyone to manually install this version on their systems and let me know if it
The patches are for application to the build system. I can do the patching and then do a clean build but what do I then need to manually install. Is it just the fireinfo.cgi file that I need to install or are there other files I need to copy/replace into my testbed system?
Regards,
Adolf.
a) works
and
b) does it keep the same ID?
The ID is being generated by a number of factors and if one bit in those changes, the system would receive a different ID which is something we would like to avoid.
It worked fine on my system, but I would like to have some larger sample size to be sure.
Best, -Michael
Hi All,
I have run into a problem with ipaddr that I have not been able to solve myself.
ipaddr.py has a lot of python2 specific code in it so I ran the 2to3 convertor on it. That fixed those issues.
Then I had the problem that the original ipaddr.py had a mixture of spaces and tabs at the line starts. Python2 appears to have accepted this but python3 doesn't.
So I converted all initial line tabs to spaces in the file and reran the build and that problem was solved.
Now I am getting the following error message.
cd /usr/src/ipaddr-1.2 && install -m 0644 ipaddr.py /usr/lib/python3* /usr/bin/python3 -c "import ipaddr" Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib/python3.8/ipaddr.py", line 1287, in <module> broadcast_network = network('255.255.255.255', 32, DEMAND_FILTER) File "/usr/lib/python3.8/ipaddr.py", line 612, in __init__ self.ip = ipaddr(ip) File "/usr/lib/python3.8/ipaddr.py", line 351, in __init__ self.intrep = int(ip) ValueError: invalid literal for int() with base 10: '255.255.255.255' make: *** [ipaddr:75: /usr/src/log/ipaddr-1.2] Error 1
python2 had long variables as well as int. python3 only has the int variables which are also longs.
However python2 long appeared to accept the string '255.255.255.255' as an input and python3 does not.
I don't know how to deal with this problem.
If this issue is a fundamental stop to using ipaddr.py with python3 I have found that python3 has, as standard, ipaddress.py available which I believe is supposed to cover the same things as ipaddr.py does. I just don't know if it does it with the same usage format or not. I could not find where in IPFire, ipaddr.py is being used.
Any help, advice, direction is gladly welcomed.
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hello,
Python3 comes with an own module to handle IP addresses. Therefore I am not sure if there is any need to bundle any of the alternative ones.
What is depending on this? Can we not simply drop this?
-Michael
On 11 May 2021, at 11:37, Adolf Belka adolf.belka@ipfire.org wrote:
Hi All,
I have run into a problem with ipaddr that I have not been able to solve myself.
ipaddr.py has a lot of python2 specific code in it so I ran the 2to3 convertor on it. That fixed those issues.
Then I had the problem that the original ipaddr.py had a mixture of spaces and tabs at the line starts. Python2 appears to have accepted this but python3 doesn't.
So I converted all initial line tabs to spaces in the file and reran the build and that problem was solved.
Now I am getting the following error message.
cd /usr/src/ipaddr-1.2 && install -m 0644 ipaddr.py /usr/lib/python3* /usr/bin/python3 -c "import ipaddr" Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib/python3.8/ipaddr.py", line 1287, in <module> broadcast_network = network('255.255.255.255', 32, DEMAND_FILTER) File "/usr/lib/python3.8/ipaddr.py", line 612, in __init__ self.ip = ipaddr(ip) File "/usr/lib/python3.8/ipaddr.py", line 351, in __init__ self.intrep = int(ip) ValueError: invalid literal for int() with base 10: '255.255.255.255' make: *** [ipaddr:75: /usr/src/log/ipaddr-1.2] Error 1python2 had long variables as well as int. python3 only has the int variables which are also longs.
However python2 long appeared to accept the string '255.255.255.255' as an input and python3 does not.
I don't know how to deal with this problem.
If this issue is a fundamental stop to using ipaddr.py with python3 I have found that python3 has, as standard, ipaddress.py available which I believe is supposed to cover the same things as ipaddr.py does. I just don't know if it does it with the same usage format or not. I could not find where in IPFire, ipaddr.py is being used.
Any help, advice, direction is gladly welcomed.
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi Michael, All
On 11/05/2021 12:40, Michael Tremer wrote:
Hello,
Python3 comes with an own module to handle IP addresses. Therefore I am not sure if there is any need to bundle any of the alternative ones.
What is depending on this? Can we not simply drop this?
There has not been any further response on this. Should I raise a patch to remove this module from IPFire?
Regards, Adolf.
-Michael
On 11 May 2021, at 11:37, Adolf Belka adolf.belka@ipfire.org wrote:
Hi All,
I have run into a problem with ipaddr that I have not been able to solve myself.
ipaddr.py has a lot of python2 specific code in it so I ran the 2to3 convertor on it. That fixed those issues.
Then I had the problem that the original ipaddr.py had a mixture of spaces and tabs at the line starts. Python2 appears to have accepted this but python3 doesn't.
So I converted all initial line tabs to spaces in the file and reran the build and that problem was solved.
Now I am getting the following error message.
cd /usr/src/ipaddr-1.2 && install -m 0644 ipaddr.py /usr/lib/python3* /usr/bin/python3 -c "import ipaddr" Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib/python3.8/ipaddr.py", line 1287, in <module> broadcast_network = network('255.255.255.255', 32, DEMAND_FILTER) File "/usr/lib/python3.8/ipaddr.py", line 612, in __init__ self.ip = ipaddr(ip) File "/usr/lib/python3.8/ipaddr.py", line 351, in __init__ self.intrep = int(ip) ValueError: invalid literal for int() with base 10: '255.255.255.255' make: *** [ipaddr:75: /usr/src/log/ipaddr-1.2] Error 1python2 had long variables as well as int. python3 only has the int variables which are also longs.
However python2 long appeared to accept the string '255.255.255.255' as an input and python3 does not.
I don't know how to deal with this problem.
If this issue is a fundamental stop to using ipaddr.py with python3 I have found that python3 has, as standard, ipaddress.py available which I believe is supposed to cover the same things as ipaddr.py does. I just don't know if it does it with the same usage format or not. I could not find where in IPFire, ipaddr.py is being used.
Any help, advice, direction is gladly welcomed.
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi all,
Three of the python-xxx files listed for python2 already seem to have a python3 versions available in IPFire.
For python-six there is python3-six. Both are addons. Can python3-six replace python-six?
Both python-six and python3-six appear to be the same source file but just a different version number 1.10.0 vs 1.14.0 and they have a different PROG name. Searching in the git repository I found spice refers to needing python-six to be built but it is not mentioned in the spice source install instructions under either mandatory or optional dependencies for building.
For python-docutils there is python3-docutils. They again appear to be the same filename but different versions. python-docutils is version 0.12 and is a core program. python3-docutils is version 0.14 and is an addon. Current version is 0.17.1
Could python3-docutils be made into a core program and updated and python-docutils removed?
There is python-setuptools and python3-setupttols-scm. These do look to be different files. Both are addons and grepping in the git repository neither are dependencies for any other package as far as I can find. Python-setuptools also appears to be a very old version of setuptools at 0.6c11 and the installation is running a .egg file via the bash command.
Can python-setuptools just be removed without creating an equivalent python3 version or is it being used/required somewhere that I have been unable to identify?
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi,
Am 11.05.2021 um 14:18 schrieb Adolf Belka adolf.belka@ipfire.org:
Hi all,
Three of the python-xxx files listed for python2 already seem to have a python3 versions available in IPFire.
For python-six there is python3-six. Both are addons. Can python3-six replace python-six?
Both python-six and python3-six appear to be the same source file but just a different version number 1.10.0 vs 1.14.0 and they have a different PROG name. Searching in the git repository I found spice refers to needing python-six to be built but it is not mentioned in the spice source install instructions under either mandatory or optional dependencies for building.
So the easiest way to try would be to disable python-six and doing a clean build. If nothing needs it it can be dropped. Otherwise you could give the python3 version a try.
For python-docutils there is python3-docutils. They again appear to be the same filename but different versions. python-docutils is version 0.12 and is a core program. python3-docutils is version 0.14 and is an addon. Current version is 0.17.1
Could python3-docutils be made into a core program and updated and python-docutils removed?
Seem to be added because of python daemon, so I would update this package first and then see what happen.
Greetings Jonatan
There is python-setuptools and python3-setupttols-scm. These do look to be different files. Both are addons and grepping in the git repository neither are dependencies for any other package as far as I can find. Python-setuptools also appears to be a very old version of setuptools at 0.6c11 and the installation is running a .egg file via the bash command.
Can python-setuptools just be removed without creating an equivalent python3 version or is it being used/required somewhere that I have been unable to identify?
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote: Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi Jonatan,
On 12/05/2021 20:34, Jonatan Schlag wrote:
Hi,
Am 11.05.2021 um 14:18 schrieb Adolf Belka adolf.belka@ipfire.org:
Hi all,
Three of the python-xxx files listed for python2 already seem to have a python3 versions available in IPFire.
For python-six there is python3-six. Both are addons. Can python3-six replace python-six?
Both python-six and python3-six appear to be the same source file but just a different version number 1.10.0 vs 1.14.0 and they have a different PROG name. Searching in the git repository I found spice refers to needing python-six to be built but it is not mentioned in the spice source install instructions under either mandatory or optional dependencies for building.
So the easiest way to try would be to disable python-six and doing a clean build. If nothing needs it it can be dropped. Otherwise you could give the python3 version a try.
Thanks for the suggestion. It worked well.
I found that python3-dateutil had python3-six defined as a dependency. However python3-dateutil is built with python3 and as we are trying to remove python2 then the use of six "as a compatibility module that can be used to allow projects to be run on either python2 or python3, whatever is available" is no longer needed.
So I removed python3-six as a dependency from python3-dateutil and removed both python3-six and python-six from make.sh and did a clean build and had no problems.
Patch has been submitted.
Thanks for your help, Adolf
For python-docutils there is python3-docutils. They again appear to be the same filename but different versions. python-docutils is version 0.12 and is a core program. python3-docutils is version 0.14 and is an addon. Current version is 0.17.1
Could python3-docutils be made into a core program and updated and python-docutils removed?
Seem to be added because of python daemon, so I would update this package first and then see what happen.
Greetings Jonatan
There is python-setuptools and python3-setupttols-scm. These do look to be different files. Both are addons and grepping in the git repository neither are dependencies for any other package as far as I can find. Python-setuptools also appears to be a very old version of setuptools at 0.6c11 and the installation is running a .egg file via the bash command.
Can python-setuptools just be removed without creating an equivalent python3 version or is it being used/required somewhere that I have been unable to identify?
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote: Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi All,
Problems trying to update libxml2 and libxslt from python2 to python3.
Using --with-python=python3 in the configure section gives the result of unknown command and it builds with python2.
Using PYTHON=python3 before the ./configure command is ignored by the configure process and it builds with python2.
There is a python3 version of libxml2 available in PyPI but it was released in 2018 and nothing since then.
It looks like both libxml2 and libxslt only do their python bits in python2. Not running python at all would still give the libxml2 and libxslt libraries but would miss out the uncommented python lines in the rootfile:-
usr/lib/python2.7/site-packages/drv_libxml2.py usr/lib/python2.7/site-packages/libxml2.py #usr/lib/python2.7/site-packages/libxml2mod.la usr/lib/python2.7/site-packages/libxml2mod.so
usr/lib/python2.7/site-packages/libxml2.pyc #usr/lib/python2.7/site-packages/libxslt.py #usr/lib/python2.7/site-packages/libxsltmod.a #usr/lib/python2.7/site-packages/libxsltmod.la usr/lib/python2.7/site-packages/libxsltmod.so
I can't tell if missing those would be a problem.
I also can't tell if there is any alternative way around libxml2/libxslt with python3 or not and if this means that python 2 needs to stay for these two libraries.
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hello,
Did you try —-disable-python?
I am not aware that we need the Python modules.
-Michael
On 11 May 2021, at 14:24, Adolf Belka adolf.belka@ipfire.org wrote:
Hi All,
Problems trying to update libxml2 and libxslt from python2 to python3.
Using --with-python=python3 in the configure section gives the result of unknown command and it builds with python2.
Using PYTHON=python3 before the ./configure command is ignored by the configure process and it builds with python2.
There is a python3 version of libxml2 available in PyPI but it was released in 2018 and nothing since then.
It looks like both libxml2 and libxslt only do their python bits in python2. Not running python at all would still give the libxml2 and libxslt libraries but would miss out the uncommented python lines in the rootfile:-
usr/lib/python2.7/site-packages/drv_libxml2.py usr/lib/python2.7/site-packages/libxml2.py #usr/lib/python2.7/site-packages/libxml2mod.la usr/lib/python2.7/site-packages/libxml2mod.so
usr/lib/python2.7/site-packages/libxml2.pyc #usr/lib/python2.7/site-packages/libxslt.py #usr/lib/python2.7/site-packages/libxsltmod.a #usr/lib/python2.7/site-packages/libxsltmod.la usr/lib/python2.7/site-packages/libxsltmod.so
I can't tell if missing those would be a problem.
I also can't tell if there is any alternative way around libxml2/libxslt with python3 or not and if this means that python 2 needs to stay for these two libraries.
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi Michael,
On 11/05/2021 15:25, Michael Tremer wrote:
Hello,
Did you try —-disable-python?
I found that command but I was trying to get the python3 versions because the python2 versions existed currently.
I am not aware that we need the Python modules.
If the python modules are not needed then that makes these two modules much easier to deal with.
Thanks for the clarification.
Adolf.
-Michael
On 11 May 2021, at 14:24, Adolf Belka adolf.belka@ipfire.org wrote:
Hi All,
Problems trying to update libxml2 and libxslt from python2 to python3.
Using --with-python=python3 in the configure section gives the result of unknown command and it builds with python2.
Using PYTHON=python3 before the ./configure command is ignored by the configure process and it builds with python2.
There is a python3 version of libxml2 available in PyPI but it was released in 2018 and nothing since then.
It looks like both libxml2 and libxslt only do their python bits in python2. Not running python at all would still give the libxml2 and libxslt libraries but would miss out the uncommented python lines in the rootfile:-
usr/lib/python2.7/site-packages/drv_libxml2.py usr/lib/python2.7/site-packages/libxml2.py #usr/lib/python2.7/site-packages/libxml2mod.la usr/lib/python2.7/site-packages/libxml2mod.so
usr/lib/python2.7/site-packages/libxml2.pyc #usr/lib/python2.7/site-packages/libxslt.py #usr/lib/python2.7/site-packages/libxsltmod.a #usr/lib/python2.7/site-packages/libxsltmod.la usr/lib/python2.7/site-packages/libxsltmod.so
I can't tell if missing those would be a problem.
I also can't tell if there is any alternative way around libxml2/libxslt with python3 or not and if this means that python 2 needs to stay for these two libraries.
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi All,
I was looking at python-distutils and python-distutils-extra as the next files to convert from python2 to python3 but I am wondering if this makes sense.
python-distutils has a lfs file that is missing most of the content. python-distutils is not listed in make.sh so it is not being built. However the rootfile and the addon install are still in place and working.
python-distutils is listed as a dependency for python-distutils-extra. This package is provided by Ubuntu launchpad. The current IPFire version is 2.25 from 2011. The most up to date version available is 2.39 from 2016. There looks to be no more development of this module. This addon package is not listed anywhere in IPFire as a dependency.
Looking up python-distutils in the python.org documentation it says that it has been replaced by python-setuptools.
PyPI indicates that distutils development stopped in 2012 and that setuptools and pip should be used.
The current version of python-setuptools in IPFire is 0.6c11 and the version available in PyPI is 56.2.0
It would seem to me that python-distutils and python-distutils-extra should be removed and python-setuptools should be updated to version 56.2.0 and converted to python3
Is my conclusion reasonable or am I missing things.
Thanks,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hello Adolf,
As far as I am aware, setuptools was merged into the base distribution of Python 3 and neither distutils or setuptools are required any more.
Just dropping this package should be fine.
-Michael
On 12 May 2021, at 12:07, Adolf Belka adolf.belka@ipfire.org wrote:
Hi All,
I was looking at python-distutils and python-distutils-extra as the next files to convert from python2 to python3 but I am wondering if this makes sense.
python-distutils has a lfs file that is missing most of the content. python-distutils is not listed in make.sh so it is not being built. However the rootfile and the addon install are still in place and working.
python-distutils is listed as a dependency for python-distutils-extra. This package is provided by Ubuntu launchpad. The current IPFire version is 2.25 from 2011. The most up to date version available is 2.39 from 2016. There looks to be no more development of this module. This addon package is not listed anywhere in IPFire as a dependency.
Looking up python-distutils in the python.org documentation it says that it has been replaced by python-setuptools.
PyPI indicates that distutils development stopped in 2012 and that setuptools and pip should be used.
The current version of python-setuptools in IPFire is 0.6c11 and the version available in PyPI is 56.2.0
It would seem to me that python-distutils and python-distutils-extra should be removed and python-setuptools should be updated to version 56.2.0 and converted to python3
Is my conclusion reasonable or am I missing things.
Thanks,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi All,
I went to have a look at updating python-mechanize to python3. This can be done, there is a release from end of 2019 which is python3 compatible.
Noticed that it has a dependency of python-clientform. This turns out to have been obsoleted in 2008 and mechanize now provides the same API as clientform did. So clientform can be removed.
Then noticed that python-mechanize is a dependency in python-feedparser and both are dependencies for python-rssdler.
feedparser has a more up to date version which is also python compatible but I noticed that it's purpose is to parse Atom and RSS feeds in python and I wonder if this is really something we want in a firewall.
Looking at python-rssdler it is from the google code archive and the version was released in 2008. The most current version is from 2009 and nothing has been done since then with it. The purpose of this module is to automatically download enclosures and other objects linked to various RSS feeds such as podcasts, videocasts and torrents.
Again this doesn't seem to be the sort of module to have in a firewall and especially when it is 12 years old.
My proposal would be to remove python-rssdler, python-feedparser and python-clientform. python-mechanize could be updated to a python3 version if it is believed to be required but my view would be to also remove this. It's purpose is to provide stateful, programmatic web browsing, including ftp,http and file:URL schemes etc. More can be found at https://pypi.org/project/mechanize/
I would appreciate your feedback before I do anything further here as I may not have the whole picture of what these addons are being used for.
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi Michael,
The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
make[1]: Entering directory '/usr/src/crda-4.14' GEN keys-gcrypt.c Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto Please install the "M2Crypto" Python module. On Debian GNU/Linux the package is called "python-m2crypto".
No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
cd $(DIR_APP) && python3 setup.py build cd $(DIR_APP) && python3 setup.py install --root=/
to
cd $(DIR_APP) && python setup.py build cd $(DIR_APP) && python setup.py install --root=/
then crda builds successfully.
Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
Regards,
Adolf
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi All,
If python-m2crypto has to stay as a python2 module then so does python-typing. python-m2crypto fails to build if python-typing is not present.
python-typing is the python2 backport of the typing capability built in to core python3, so it is only needed for python2.
Regards,
Adolf.
On 14/05/2021 21:28, Adolf Belka wrote:
Hi Michael,
The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
make[1]: Entering directory '/usr/src/crda-4.14' GEN keys-gcrypt.c Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto Please install the "M2Crypto" Python module. On Debian GNU/Linux the package is called "python-m2crypto".
No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
cd $(DIR_APP) && python3 setup.py build cd $(DIR_APP) && python3 setup.py install --root=/
to
cd $(DIR_APP) && python setup.py build cd $(DIR_APP) && python setup.py install --root=/
then crda builds successfully.
Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
Regards,
Adolf
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi All,
So I migrated python-setuptools to python3-setuptools. The python3 version worked with python3-daemon with no problems.
However later on python-m2crypto failed to build successfully because it could not find python-setuptools. So python-m2crypto needs both python-typing and python-setuptools to be present. So this triplet looks to be the barrier to going fully python3.
Regards,
Adolf.
On 15/05/2021 23:03, Adolf Belka wrote:
Hi All,
If python-m2crypto has to stay as a python2 module then so does python-typing. python-m2crypto fails to build if python-typing is not present.
python-typing is the python2 backport of the typing capability built in to core python3, so it is only needed for python2.
Regards,
Adolf.
On 14/05/2021 21:28, Adolf Belka wrote:
Hi Michael,
The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
make[1]: Entering directory '/usr/src/crda-4.14' GEN keys-gcrypt.c Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto Please install the "M2Crypto" Python module. On Debian GNU/Linux the package is called "python-m2crypto".
No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
cd $(DIR_APP) && python3 setup.py build cd $(DIR_APP) && python3 setup.py install --root=/
to
cd $(DIR_APP) && python setup.py build cd $(DIR_APP) && python setup.py install --root=/
then crda builds successfully.
Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
Regards,
Adolf
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hello,
Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules.
Fedora has dropped crda a little while ago and added some scripts to wireless-regdb:
https://src.fedoraproject.org/rpms/wireless-regdb
I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs:
https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomai...
We could then drop crda and all dependencies.
Would you be up for having a look at this?
-Michael
On 14 May 2021, at 20:28, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
make[1]: Entering directory '/usr/src/crda-4.14' GEN keys-gcrypt.c Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto Please install the "M2Crypto" Python module. On Debian GNU/Linux the package is called "python-m2crypto".No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
cd $(DIR_APP) && python3 setup.py build cd $(DIR_APP) && python3 setup.py install --root=/to
cd $(DIR_APP) && python setup.py build cd $(DIR_APP) && python setup.py install --root=/then crda builds successfully.
Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
Regards,
Adolf
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi Michael,
On 20/05/2021 12:15, Michael Tremer wrote:
Hello,
Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules.
Fedora has dropped crda a little while ago and added some scripts to wireless-regdb:
https://src.fedoraproject.org/rpms/wireless-regdb
I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs:
https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomai...
We could then drop crda and all dependencies.
Would you be up for having a look at this?
Absolutely. I will give it a go and see how I get on. Good news about crda because then we should eventually be able to remove python2.
Regards, Adolf.
-Michael
On 14 May 2021, at 20:28, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
make[1]: Entering directory '/usr/src/crda-4.14' GEN keys-gcrypt.c Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto Please install the "M2Crypto" Python module. On Debian GNU/Linux the package is called "python-m2crypto".No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
cd $(DIR_APP) && python3 setup.py build cd $(DIR_APP) && python3 setup.py install --root=/to
cd $(DIR_APP) && python setup.py build cd $(DIR_APP) && python setup.py install --root=/then crda builds successfully.
Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
Regards,
Adolf
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
On 20 May 2021, at 11:30, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
On 20/05/2021 12:15, Michael Tremer wrote:
Hello, Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules. Fedora has dropped crda a little while ago and added some scripts to wireless-regdb: https://src.fedoraproject.org/rpms/wireless-regdb I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs: https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomai... We could then drop crda and all dependencies. Would you be up for having a look at this?
Absolutely. I will give it a go and see how I get on. Good news about crda because then we should eventually be able to remove python2.
That would make me quite happy :)
Regards, Adolf.
-Michael
On 14 May 2021, at 20:28, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
make[1]: Entering directory '/usr/src/crda-4.14' GEN keys-gcrypt.c Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto Please install the "M2Crypto" Python module. On Debian GNU/Linux the package is called "python-m2crypto".No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
cd $(DIR_APP) && python3 setup.py build cd $(DIR_APP) && python3 setup.py install --root=/to
cd $(DIR_APP) && python setup.py build cd $(DIR_APP) && python setup.py install --root=/then crda builds successfully.
Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
Regards,
Adolf
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi Michael and All,
On 20/05/2021 12:15, Michael Tremer wrote:
Hello,
Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules.
Fedora has dropped crda a little while ago and added some scripts to wireless-regdb:
https://src.fedoraproject.org/rpms/wireless-regdb
I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs:
https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomai...
I have been working on this script but I have reached an impasse. The script as it is, presumes that /etc/localtime is a symlink to one of the timezones in /usr/share/zoneinfo/ If localtime is a file and not a symlink then the scripts man page says that the system admin must create a file called regdomain containing the country code in /etc/sysconfig/
IPFire does not use symlinks to the zoneinfo files so the country code can't be created by the script.
localtime can't just be read as it is a binary file. zdump could read it but that is not available in IPFire as it is commented out in the glibc rootfile and anyway it would say that the TZ="/etc/localtime" so that wouldn't help either. I tried it on another system that did have zdump.
I thought I could use the location routine with the local-ipaddress from red but that won't work if IPFire is behind another router as red would then be a private ip address.
Then I thought about doing a similar thing as with DDNS to "Guess the real public IP with help of an external server" but I am not sure if that could be just done without letting people know we are accessing an external server.
So I have reached the point where the script will not work as it is and I have not been able to figure out a way to find the country code that has been selected for the IPFire installation.
Hopefully there is an easy option that someone can point me to that I have missed to recognise.
Looking forward to any suggestions.
Regards,
Adolf.
We could then drop crda and all dependencies.
Would you be up for having a look at this?
-Michael
On 14 May 2021, at 20:28, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
make[1]: Entering directory '/usr/src/crda-4.14' GEN keys-gcrypt.c Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto Please install the "M2Crypto" Python module. On Debian GNU/Linux the package is called "python-m2crypto".No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
cd $(DIR_APP) && python3 setup.py build cd $(DIR_APP) && python3 setup.py install --root=/to
cd $(DIR_APP) && python setup.py build cd $(DIR_APP) && python setup.py install --root=/then crda builds successfully.
Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
Regards,
Adolf
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hello,
On 22 May 2021, at 13:25, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael and All,
On 20/05/2021 12:15, Michael Tremer wrote:
Hello, Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules. Fedora has dropped crda a little while ago and added some scripts to wireless-regdb: https://src.fedoraproject.org/rpms/wireless-regdb I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs: https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomai...
I have been working on this script but I have reached an impasse. The script as it is, presumes that /etc/localtime is a symlink to one of the timezones in /usr/share/zoneinfo/ If localtime is a file and not a symlink then the scripts man page says that the system admin must create a file called regdomain containing the country code in /etc/sysconfig/
I believe we are using a hardlink here. I would therefore drop that part of the code, or change /etc/localtime to a symlink.
IPFire does not use symlinks to the zoneinfo files so the country code can't be created by the script.
localtime can't just be read as it is a binary file. zdump could read it but that is not available in IPFire as it is commented out in the glibc rootfile and anyway it would say that the TZ="/etc/localtime" so that wouldn't help either. I tried it on another system that did have zdump.
I thought I could use the location routine with the local-ipaddress from red but that won't work if IPFire is behind another router as red would then be a private ip address.
Then I thought about doing a similar thing as with DDNS to "Guess the real public IP with help of an external server" but I am not sure if that could be just done without letting people know we are accessing an external server.
So I have reached the point where the script will not work as it is and I have not been able to figure out a way to find the country code that has been selected for the IPFire installation.
Hopefully there is an easy option that someone can point me to that I have missed to recognise.
How about to make it so simply that we simply read the value from /var/ipfire/wlanap/settings? Where did we get this data from before?
Best, -Michael
Looking forward to any suggestions.
Regards,
Adolf.
We could then drop crda and all dependencies. Would you be up for having a look at this? -Michael
On 14 May 2021, at 20:28, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
make[1]: Entering directory '/usr/src/crda-4.14' GEN keys-gcrypt.c Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto Please install the "M2Crypto" Python module. On Debian GNU/Linux the package is called "python-m2crypto".No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
cd $(DIR_APP) && python3 setup.py build cd $(DIR_APP) && python3 setup.py install --root=/to
cd $(DIR_APP) && python setup.py build cd $(DIR_APP) && python setup.py install --root=/then crda builds successfully.
Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
Regards,
Adolf
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
-- Sent from my laptop
Hi,
Am Dienstag, den 25.05.2021, 11:03 +0100 schrieb Michael Tremer:
Hello,
On 22 May 2021, at 13:25, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael and All,
On 20/05/2021 12:15, Michael Tremer wrote:
Hello, Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules. Fedora has dropped crda a little while ago and added some scripts to wireless-regdb: https://src.fedoraproject.org/rpms/wireless-regdb I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs: https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomai...
read the word "crda" today and was reminded of this thread. As crda seems to be deprecated, we could wait for a new kernel, which should make solving this problem easier.
Source is: https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/crda.git/commit/?id=f... and https://wiki.gentoo.org/wiki/Hostapd#Proper_use_of_the_5GHz_band
Jonatan
I have been working on this script but I have reached an impasse. The script as it is, presumes that /etc/localtime is a symlink to one of the timezones in /usr/share/zoneinfo/ If localtime is a file and not a symlink then the scripts man page says that the system admin must create a file called regdomain containing the country code in /etc/sysconfig/
I believe we are using a hardlink here. I would therefore drop that part of the code, or change /etc/localtime to a symlink.
IPFire does not use symlinks to the zoneinfo files so the country code can't be created by the script.
localtime can't just be read as it is a binary file. zdump could read it but that is not available in IPFire as it is commented out in the glibc rootfile and anyway it would say that the TZ="/etc/localtime" so that wouldn't help either. I tried it on another system that did have zdump.
I thought I could use the location routine with the local-ipaddress from red but that won't work if IPFire is behind another router as red would then be a private ip address.
Then I thought about doing a similar thing as with DDNS to "Guess the real public IP with help of an external server" but I am not sure if that could be just done without letting people know we are accessing an external server.
So I have reached the point where the script will not work as it is and I have not been able to figure out a way to find the country code that has been selected for the IPFire installation.
Hopefully there is an easy option that someone can point me to that I have missed to recognise.
How about to make it so simply that we simply read the value from /var/ipfire/wlanap/settings? Where did we get this data from before?
Best, -Michael
Looking forward to any suggestions.
Regards,
Adolf.
We could then drop crda and all dependencies. Would you be up for having a look at this? -Michael
On 14 May 2021, at 20:28, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
make[1]: Entering directory '/usr/src/crda-4.14' GEN keys-gcrypt.c Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto Please install the "M2Crypto" Python module. On Debian GNU/Linux the package is called "python-m2crypto".No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
cd $(DIR_APP) && python3 setup.py build cd $(DIR_APP) && python3 setup.py install --root=/to
cd $(DIR_APP) && python setup.py build cd $(DIR_APP) && python setup.py install --root=/then crda builds successfully.
Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
Regards,
Adolf
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
-- Sent from my laptop
Hi Jonatan and Michael,
On 29/05/2021 16:31, Jonatan Schlag wrote:
Hi,
Am Dienstag, den 25.05.2021, 11:03 +0100 schrieb Michael Tremer:
Hello,
On 22 May 2021, at 13:25, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael and All,
On 20/05/2021 12:15, Michael Tremer wrote:
Hello, Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules. Fedora has dropped crda a little while ago and added some scripts to wireless-regdb: https://src.fedoraproject.org/rpms/wireless-regdb I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs: https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomai...
read the word "crda" today and was reminded of this thread. As crda seems to be deprecated, we could wait for a new kernel, which should make solving this problem easier.
Thanks Jonatan for finding this status. This might be a better option.
Source is: https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/crda.git/commit/?id=f... and https://wiki.gentoo.org/wiki/Hostapd#Proper_use_of_the_5GHz_band
Jonatan
I have been working on this script but I have reached an impasse. The script as it is, presumes that /etc/localtime is a symlink to one of the timezones in /usr/share/zoneinfo/ If localtime is a file and not a symlink then the scripts man page says that the system admin must create a file called regdomain containing the country code in /etc/sysconfig/
I believe we are using a hardlink here. I would therefore drop that part of the code, or change /etc/localtime to a symlink.
Changing etc/localtime to a softlink would work for new installations but the problem would be in upgrading existing installations. The localtime hardlink has no info on the country that was selected only on the timezone which covers multiple countries. So I don't think that changing the hardlink to a softlink for localtime is a viable option here.
IPFire does not use symlinks to the zoneinfo files so the country code can't be created by the script.
localtime can't just be read as it is a binary file. zdump could read it but that is not available in IPFire as it is commented out in the glibc rootfile and anyway it would say that the TZ="/etc/localtime" so that wouldn't help either. I tried it on another system that did have zdump.
I thought I could use the location routine with the local-ipaddress from red but that won't work if IPFire is behind another router as red would then be a private ip address.
Then I thought about doing a similar thing as with DDNS to "Guess the real public IP with help of an external server" but I am not sure if that could be just done without letting people know we are accessing an external server.
So I have reached the point where the script will not work as it is and I have not been able to figure out a way to find the country code that has been selected for the IPFire installation.
Hopefully there is an easy option that someone can point me to that I have missed to recognise.
How about to make it so simply that we simply read the value from /var/ipfire/wlanap/settings? Where did we get this data from before?
Checking the source code files of crda it looks like it just checks the contents of the environment variable COUNTRY. If it is blank or set to 00 then crda just provides the global settings to the kernel, ie those that are in line with every country, and therefore at the lowest applicable level. So using wlanap/settings would be a workable approach as the COUNTRY value could then be used from those settings.
So looks like two options:- 1) wait for kernel 4.15 or later to be implemented in IPFire. 2) use the wlanap/settings value for COUNTRY and then add the udev entry as per the Fedora approach.
Either way I will not be able to test out any change myself as my IPFire system has no built in wireless card. I am using separate Wireless Access Points in my network.
Regards,
Adolf
Best, -Michael
Looking forward to any suggestions.
Regards,
Adolf.
We could then drop crda and all dependencies. Would you be up for having a look at this? -Michael
On 14 May 2021, at 20:28, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
make[1]: Entering directory '/usr/src/crda-4.14' GEN keys-gcrypt.c Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto Please install the "M2Crypto" Python module. On Debian GNU/Linux the package is called "python-m2crypto".No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
cd $(DIR_APP) && python3 setup.py build cd $(DIR_APP) && python3 setup.py install --root=/to
cd $(DIR_APP) && python setup.py build cd $(DIR_APP) && python setup.py install --root=/then crda builds successfully.
Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
Regards,
Adolf
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
-- Sent from my laptop
Hello,
On 29 May 2021, at 17:19, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Jonatan and Michael,
On 29/05/2021 16:31, Jonatan Schlag wrote:
Hi, Am Dienstag, den 25.05.2021, 11:03 +0100 schrieb Michael Tremer:
Hello,
On 22 May 2021, at 13:25, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael and All,
On 20/05/2021 12:15, Michael Tremer wrote:
Hello, Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules. Fedora has dropped crda a little while ago and added some scripts to wireless-regdb: https://src.fedoraproject.org/rpms/wireless-regdb I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs: https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomai...
read the word "crda" today and was reminded of this thread. As crda seems to be deprecated, we could wait for a new kernel, which should make solving this problem easier.
Thanks Jonatan for finding this status. This might be a better option.
Source is: https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/crda.git/commit/?id=f... and https://wiki.gentoo.org/wiki/Hostapd#Proper_use_of_the_5GHz_band Jonatan
I have been working on this script but I have reached an impasse. The script as it is, presumes that /etc/localtime is a symlink to one of the timezones in /usr/share/zoneinfo/ If localtime is a file and not a symlink then the scripts man page says that the system admin must create a file called regdomain containing the country code in /etc/sysconfig/
I believe we are using a hardlink here. I would therefore drop that part of the code, or change /etc/localtime to a symlink.
Changing etc/localtime to a softlink would work for new installations but the problem would be in upgrading existing installations. The localtime hardlink has no info on the country that was selected only on the timezone which covers multiple countries. So I don't think that changing the hardlink to a softlink for localtime is a viable option here.
IPFire does not use symlinks to the zoneinfo files so the country code can't be created by the script.
localtime can't just be read as it is a binary file. zdump could read it but that is not available in IPFire as it is commented out in the glibc rootfile and anyway it would say that the TZ="/etc/localtime" so that wouldn't help either. I tried it on another system that did have zdump.
I thought I could use the location routine with the local-ipaddress from red but that won't work if IPFire is behind another router as red would then be a private ip address.
Then I thought about doing a similar thing as with DDNS to "Guess the real public IP with help of an external server" but I am not sure if that could be just done without letting people know we are accessing an external server.
So I have reached the point where the script will not work as it is and I have not been able to figure out a way to find the country code that has been selected for the IPFire installation.
Hopefully there is an easy option that someone can point me to that I have missed to recognise.
How about to make it so simply that we simply read the value from /var/ipfire/wlanap/settings? Where did we get this data from before?
Checking the source code files of crda it looks like it just checks the contents of the environment variable COUNTRY. If it is blank or set to 00 then crda just provides the global settings to the kernel, ie those that are in line with every country, and therefore at the lowest applicable level. So using wlanap/settings would be a workable approach as the COUNTRY value could then be used from those settings.
So looks like two options:-
- wait for kernel 4.15 or later to be implemented in IPFire.
There is a kernel branch that will land in “next” quite soon. So you can go ahead and create a feature branch that drops CRDA which can be merged after the kernel has been merged.
This is slightly sad because it delays dropping Python 2 before the kernel :(
- use the wlanap/settings value for COUNTRY and then add the udev entry as per the Fedora approach.
That is an option, but hostapd sets that anyways. What we do not set is the country code when running in client mode. That is where CRDA was required.
I suppose it is okay to remain configured as “00” until the client associates with the AP. If the AP pushes its country code, the kernel will be notified and load the settings from the wireless reg db in the future. So we won’t have to do anything.
Either way I will not be able to test out any change myself as my IPFire system has no built in wireless card. I am using separate Wireless Access Points in my network.
No problem. Would you still be up for dropping CRDA and repackaging wireless-reg-db?
Running “iw reg set XX” and “iw reg get” should show the correct results then if you are using the development kernel.
-Michael
Regards,
Adolf
Best, -Michael
Looking forward to any suggestions.
Regards,
Adolf.
We could then drop crda and all dependencies. Would you be up for having a look at this? -Michael
On 14 May 2021, at 20:28, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
make[1]: Entering directory '/usr/src/crda-4.14' GEN keys-gcrypt.c Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto Please install the "M2Crypto" Python module. On Debian GNU/Linux the package is called "python-m2crypto".No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
cd $(DIR_APP) && python3 setup.py build cd $(DIR_APP) && python3 setup.py install --root=/to
cd $(DIR_APP) && python setup.py build cd $(DIR_APP) && python setup.py install --root=/then crda builds successfully.
Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
Regards,
Adolf
On 05/05/2021 16:27, Michael Tremer wrote: > Hello, > > I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2 > > This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible. > > However, there is lots of software that still depends on it which has to be migrated away first. > > There are at least the following packages: > > boost > fetchmail > fireinfo > iotop > ipaddr > libxml2 > libxslt > newt > nmap > python > python-clientform > python-daemon > python-distutils > python-distutils-extra > python-docutils > python-feedparser > python-inotify > python-ipaddress > python-m2crypto > python-mechanize > python-optional-src > python-pyparsing > python-rssdler > python-setuptools > python-six > python-typing > > We also have the following scripts: > > config/ca-certificates/certdata2pem.py > config/unbound/unbound-dhcp-leases-bridge > > Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour > of the current implementation. > > The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force > it to build with Python 3 or we just disable the bindings. > > Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it > from the distribution very soon. Looking at his list, it does not seem to be too difficult. > > Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can > everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away. > > Best, > -Michael
-- Sent from my laptop
Hi Michael,
On 31/05/2021 14:35, Michael Tremer wrote:
Hello,
On 29 May 2021, at 17:19, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Jonatan and Michael,
On 29/05/2021 16:31, Jonatan Schlag wrote:
Hi, Am Dienstag, den 25.05.2021, 11:03 +0100 schrieb Michael Tremer:
Hello,
On 22 May 2021, at 13:25, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael and All,
On 20/05/2021 12:15, Michael Tremer wrote:
Hello, Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules. Fedora has dropped crda a little while ago and added some scripts to wireless-regdb: https://src.fedoraproject.org/rpms/wireless-regdb I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs: https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomai...
read the word "crda" today and was reminded of this thread. As crda seems to be deprecated, we could wait for a new kernel, which should make solving this problem easier.
Thanks Jonatan for finding this status. This might be a better option.
Source is: https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/crda.git/commit/?id=f... and https://wiki.gentoo.org/wiki/Hostapd#Proper_use_of_the_5GHz_band Jonatan
I have been working on this script but I have reached an impasse. The script as it is, presumes that /etc/localtime is a symlink to one of the timezones in /usr/share/zoneinfo/ If localtime is a file and not a symlink then the scripts man page says that the system admin must create a file called regdomain containing the country code in /etc/sysconfig/
I believe we are using a hardlink here. I would therefore drop that part of the code, or change /etc/localtime to a symlink.
Changing etc/localtime to a softlink would work for new installations but the problem would be in upgrading existing installations. The localtime hardlink has no info on the country that was selected only on the timezone which covers multiple countries. So I don't think that changing the hardlink to a softlink for localtime is a viable option here.
IPFire does not use symlinks to the zoneinfo files so the country code can't be created by the script.
localtime can't just be read as it is a binary file. zdump could read it but that is not available in IPFire as it is commented out in the glibc rootfile and anyway it would say that the TZ="/etc/localtime" so that wouldn't help either. I tried it on another system that did have zdump.
I thought I could use the location routine with the local-ipaddress from red but that won't work if IPFire is behind another router as red would then be a private ip address.
Then I thought about doing a similar thing as with DDNS to "Guess the real public IP with help of an external server" but I am not sure if that could be just done without letting people know we are accessing an external server.
So I have reached the point where the script will not work as it is and I have not been able to figure out a way to find the country code that has been selected for the IPFire installation.
Hopefully there is an easy option that someone can point me to that I have missed to recognise.
How about to make it so simply that we simply read the value from /var/ipfire/wlanap/settings? Where did we get this data from before?
Checking the source code files of crda it looks like it just checks the contents of the environment variable COUNTRY. If it is blank or set to 00 then crda just provides the global settings to the kernel, ie those that are in line with every country, and therefore at the lowest applicable level. So using wlanap/settings would be a workable approach as the COUNTRY value could then be used from those settings.
So looks like two options:-
- wait for kernel 4.15 or later to be implemented in IPFire.
There is a kernel branch that will land in “next” quite soon. So you can go ahead and create a feature branch that drops CRDA which can be merged after the kernel has been merged.
This is slightly sad because it delays dropping Python 2 before the kernel :(
- use the wlanap/settings value for COUNTRY and then add the udev entry as per the Fedora approach.
That is an option, but hostapd sets that anyways. What we do not set is the country code when running in client mode. That is where CRDA was required.
I suppose it is okay to remain configured as “00” until the client associates with the AP. If the AP pushes its country code, the kernel will be notified and load the settings from the wireless reg db in the future. So we won’t have to do anything.
Either way I will not be able to test out any change myself as my IPFire system has no built in wireless card. I am using separate Wireless Access Points in my network.
No problem. Would you still be up for dropping CRDA and repackaging wireless-reg-db?
Yes I will give it a go.
Running “iw reg set XX” and “iw reg get” should show the correct results then if you are using the development kernel.
By development kernel do you mean the experimental kernel repo of Arne?
Regards, Adolf.
-Michael
Regards,
Adolf
Best, -Michael
Looking forward to any suggestions.
Regards,
Adolf.
We could then drop crda and all dependencies. Would you be up for having a look at this? -Michael > On 14 May 2021, at 20:28, Adolf Belka adolf.belka@ipfire.org wrote: > > Hi Michael, > > The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve. > > python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2. > > When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error > > > make[1]: Entering directory '/usr/src/crda-4.14' > GEN keys-gcrypt.c > Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem > ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto > Please install the "M2Crypto" Python module. > On Debian GNU/Linux the package is called "python-m2crypto". > > > No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using > > cd $(DIR_APP) && python3 setup.py build > cd $(DIR_APP) && python3 setup.py install --root=/ > > to > > cd $(DIR_APP) && python setup.py build > cd $(DIR_APP) && python setup.py install --root=/ > > then crda builds successfully. > > Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build. > > > Regards, > > Adolf > > > On 05/05/2021 16:27, Michael Tremer wrote: >> Hello, >> >> I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2 >> >> This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible. >> >> However, there is lots of software that still depends on it which has to be migrated away first. >> >> There are at least the following packages: >> >> boost >> fetchmail >> fireinfo >> iotop >> ipaddr >> libxml2 >> libxslt >> newt >> nmap >> python >> python-clientform >> python-daemon >> python-distutils >> python-distutils-extra >> python-docutils >> python-feedparser >> python-inotify >> python-ipaddress >> python-m2crypto >> python-mechanize >> python-optional-src >> python-pyparsing >> python-rssdler >> python-setuptools >> python-six >> python-typing >> >> We also have the following scripts: >> >> config/ca-certificates/certdata2pem.py >> config/unbound/unbound-dhcp-leases-bridge >> >> Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour >> of the current implementation. >> >> The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force >> it to build with Python 3 or we just disable the bindings. >> >> Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it >> from the distribution very soon. Looking at his list, it does not seem to be too difficult. >> >> Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can >> everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away. >> >> Best, >> -Michael
-- Sent from my laptop
On 1 Jun 2021, at 14:57, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
On 31/05/2021 14:35, Michael Tremer wrote:
Hello,
On 29 May 2021, at 17:19, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Jonatan and Michael,
On 29/05/2021 16:31, Jonatan Schlag wrote:
Hi, Am Dienstag, den 25.05.2021, 11:03 +0100 schrieb Michael Tremer:
Hello,
On 22 May 2021, at 13:25, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael and All,
On 20/05/2021 12:15, Michael Tremer wrote: > Hello, > Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules. > Fedora has dropped crda a little while ago and added some scripts to wireless-regdb: > https://src.fedoraproject.org/rpms/wireless-regdb > I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs: > https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomai...
read the word "crda" today and was reminded of this thread. As crda seems to be deprecated, we could wait for a new kernel, which should make solving this problem easier.
Thanks Jonatan for finding this status. This might be a better option.
Source is: https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/crda.git/commit/?id=f... and https://wiki.gentoo.org/wiki/Hostapd#Proper_use_of_the_5GHz_band Jonatan
I have been working on this script but I have reached an impasse. The script as it is, presumes that /etc/localtime is a symlink to one of the timezones in /usr/share/zoneinfo/ If localtime is a file and not a symlink then the scripts man page says that the system admin must create a file called regdomain containing the country code in /etc/sysconfig/
I believe we are using a hardlink here. I would therefore drop that part of the code, or change /etc/localtime to a symlink.
Changing etc/localtime to a softlink would work for new installations but the problem would be in upgrading existing installations. The localtime hardlink has no info on the country that was selected only on the timezone which covers multiple countries. So I don't think that changing the hardlink to a softlink for localtime is a viable option here.
IPFire does not use symlinks to the zoneinfo files so the country code can't be created by the script.
localtime can't just be read as it is a binary file. zdump could read it but that is not available in IPFire as it is commented out in the glibc rootfile and anyway it would say that the TZ="/etc/localtime" so that wouldn't help either. I tried it on another system that did have zdump.
I thought I could use the location routine with the local-ipaddress from red but that won't work if IPFire is behind another router as red would then be a private ip address.
Then I thought about doing a similar thing as with DDNS to "Guess the real public IP with help of an external server" but I am not sure if that could be just done without letting people know we are accessing an external server.
So I have reached the point where the script will not work as it is and I have not been able to figure out a way to find the country code that has been selected for the IPFire installation.
Hopefully there is an easy option that someone can point me to that I have missed to recognise.
How about to make it so simply that we simply read the value from /var/ipfire/wlanap/settings? Where did we get this data from before?
Checking the source code files of crda it looks like it just checks the contents of the environment variable COUNTRY. If it is blank or set to 00 then crda just provides the global settings to the kernel, ie those that are in line with every country, and therefore at the lowest applicable level. So using wlanap/settings would be a workable approach as the COUNTRY value could then be used from those settings.
So looks like two options:-
- wait for kernel 4.15 or later to be implemented in IPFire.
There is a kernel branch that will land in “next” quite soon. So you can go ahead and create a feature branch that drops CRDA which can be merged after the kernel has been merged. This is slightly sad because it delays dropping Python 2 before the kernel :(
- use the wlanap/settings value for COUNTRY and then add the udev entry as per the Fedora approach.
That is an option, but hostapd sets that anyways. What we do not set is the country code when running in client mode. That is where CRDA was required. I suppose it is okay to remain configured as “00” until the client associates with the AP. If the AP pushes its country code, the kernel will be notified and load the settings from the wireless reg db in the future. So we won’t have to do anything.
Either way I will not be able to test out any change myself as my IPFire system has no built in wireless card. I am using separate Wireless Access Points in my network.
No problem. Would you still be up for dropping CRDA and repackaging wireless-reg-db?
Yes I will give it a go.
Running “iw reg set XX” and “iw reg get” should show the correct results then if you are using the development kernel.
By development kernel do you mean the experimental kernel repo of Arne?
Yes. You should be able to build everything based on next as normal and just install the kernel package from here:
https://people.ipfire.org/~arne_f/highly-experimental/kernel-5.10/
-Michael
Regards, Adolf.
-Michael
Regards,
Adolf
Best, -Michael
Looking forward to any suggestions.
Regards,
Adolf. > We could then drop crda and all dependencies. > Would you be up for having a look at this? > -Michael >> On 14 May 2021, at 20:28, Adolf Belka adolf.belka@ipfire.org wrote: >> >> Hi Michael, >> >> The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve. >> >> python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2. >> >> When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error >> >> >> make[1]: Entering directory '/usr/src/crda-4.14' >> GEN keys-gcrypt.c >> Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem >> ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto >> Please install the "M2Crypto" Python module. >> On Debian GNU/Linux the package is called "python-m2crypto". >> >> >> No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using >> >> cd $(DIR_APP) && python3 setup.py build >> cd $(DIR_APP) && python3 setup.py install --root=/ >> >> to >> >> cd $(DIR_APP) && python setup.py build >> cd $(DIR_APP) && python setup.py install --root=/ >> >> then crda builds successfully. >> >> Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build. >> >> >> Regards, >> >> Adolf >> >> >> On 05/05/2021 16:27, Michael Tremer wrote: >>> Hello, >>> >>> I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2 >>> >>> This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible. >>> >>> However, there is lots of software that still depends on it which has to be migrated away first. >>> >>> There are at least the following packages: >>> >>> boost >>> fetchmail >>> fireinfo >>> iotop >>> ipaddr >>> libxml2 >>> libxslt >>> newt >>> nmap >>> python >>> python-clientform >>> python-daemon >>> python-distutils >>> python-distutils-extra >>> python-docutils >>> python-feedparser >>> python-inotify >>> python-ipaddress >>> python-m2crypto >>> python-mechanize >>> python-optional-src >>> python-pyparsing >>> python-rssdler >>> python-setuptools >>> python-six >>> python-typing >>> >>> We also have the following scripts: >>> >>> config/ca-certificates/certdata2pem.py >>> config/unbound/unbound-dhcp-leases-bridge >>> >>> Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour >>> of the current implementation. >>> >>> The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force >>> it to build with Python 3 or we just disable the bindings. >>> >>> Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it >>> from the distribution very soon. Looking at his list, it does not seem to be too difficult. >>> >>> Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can >>> everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away. >>> >>> Best, >>> -Michael
-- Sent from my laptop
-- Sent from my laptop
Hi All,
I just had a look at python-optional-src for converting to python3 but found most of it is not present.
There is an lfs file but it does not mention the source file and there are no build instructions.
python-optional-src is not in make.sh so it is not getting built anyway.
There is no rootfile for python-optional-src
I think this lfs can just be deleted. I will raise a patch for this unless I hear something different.
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
Hi All,
On 15/05/2021 10:28, Adolf Belka wrote:
Hi All,
I just had a look at python-optional-src for converting to python3 but found most of it is not present.
There is an lfs file but it does not mention the source file and there are no build instructions.
python-optional-src is not in make.sh so it is not getting built anyway.
There is no rootfile for python-optional-src
Error in my first message. There is a rootfile in packages. grepping for "import optional-src" or "from optional-src import" in the build doesn't flag anything up. So I still believe it can be removed.
I think this lfs can just be deleted. I will raise a patch for this unless I hear something different.
Regards,
Adolf.
On 05/05/2021 16:27, Michael Tremer wrote:
Hello,
I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
However, there is lots of software that still depends on it which has to be migrated away first.
There are at least the following packages:
boost fetchmail fireinfo iotop ipaddr libxml2 libxslt newt nmap python python-clientform python-daemon python-distutils python-distutils-extra python-docutils python-feedparser python-inotify python-ipaddress python-m2crypto python-mechanize python-optional-src python-pyparsing python-rssdler python-setuptools python-six python-typing
We also have the following scripts:
config/ca-certificates/certdata2pem.py config/unbound/unbound-dhcp-leases-bridge
Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
Best, -Michael
-
Adolf Belka -
Jonatan Schlag -
Michael Tremer