From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer To: development@lists.ipfire.org Subject: Re: Dropping Python 2 (python-m2crypto) Date: Wed, 02 Jun 2021 10:41:53 +0100 Message-ID: In-Reply-To: <1a05dcbd-51dc-1331-088f-8a6619d5d518@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5139969712395763276==" List-Id: --===============5139969712395763276== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable > On 1 Jun 2021, at 14:57, Adolf Belka wrote: >=20 > Hi Michael, >=20 > On 31/05/2021 14:35, Michael Tremer wrote: >> Hello, >>> On 29 May 2021, at 17:19, Adolf Belka wrote: >>>=20 >>> Hi Jonatan and Michael, >>>=20 >>> On 29/05/2021 16:31, Jonatan Schlag wrote: >>>> Hi, >>>> Am Dienstag, den 25.05.2021, 11:03 +0100 schrieb Michael Tremer: >>>>> Hello, >>>>>=20 >>>>>> On 22 May 2021, at 13:25, Adolf Belka wrote: >>>>>>=20 >>>>>> Hi Michael and All, >>>>>>=20 >>>>>> On 20/05/2021 12:15, Michael Tremer wrote: >>>>>>> Hello, >>>>>>> Since Core Update 158 is now open, I would like to finish migrating t= he remaining Python 2 modules. >>>>>>> Fedora has dropped crda a little while ago and added some scripts to = wireless-regdb: >>>>>>> https://src.fedoraproject.org/rpms/wireless-regdb >>>>>>> I think we should follow that path and add the udev rules to lfs/wire= less-regdb and adapt this script for our needs: >>>>>>> https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/se= tregdomain >>>> read the word "crda" today and was reminded of this thread. As crda seem= s to be deprecated, we could wait for a new kernel, which should make solving= this problem easier. >>> Thanks Jonatan for finding this status. This might be a better option. >>>> Source is: https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/crda.g= it/commit/?id=3Df4ef2531698fb9ba006e8b31a223b3269be8bc7c >>>> and https://wiki.gentoo.org/wiki/Hostapd#Proper_use_of_the_5GHz_band >>>> Jonatan >>>>>> I have been working on this script but I have reached an impasse. >>>>>> The script as it is, presumes that /etc/localtime is a symlink to one = of the timezones in /usr/share/zoneinfo/ >>>>>> If localtime is a file and not a symlink then the scripts man page say= s that the system admin must create a file called regdomain containing the co= untry code in /etc/sysconfig/ >>>>>=20 >>>>> I believe we are using a hardlink here. I would therefore drop that par= t of the code, or change /etc/localtime to a symlink. >>>=20 >>> Changing etc/localtime to a softlink would work for new installations but= the problem would be in upgrading existing installations. The localtime hard= link has no info on the country that was selected only on the timezone which = covers multiple countries. So I don't think that changing the hardlink to a s= oftlink for localtime is a viable option here. >>>=20 >>>>>=20 >>>>>> IPFire does not use symlinks to the zoneinfo files so the country code= can't be created by the script. >>>>>>=20 >>>>>> localtime can't just be read as it is a binary file. zdump could read = it but that is not available in IPFire as it is commented out in the glibc ro= otfile and anyway it would >>>>>> say that the TZ=3D"/etc/localtime" so that wouldn't help either. I tri= ed it on another system that did have zdump. >>>>>>=20 >>>>>> I thought I could use the location routine with the local-ipaddress fr= om red but that won't work if IPFire is behind another router as red would th= en be a private ip address. >>>>>>=20 >>>>>> Then I thought about doing a similar thing as with DDNS to "Guess the = real public IP with help of an external server" but I am not sure if that cou= ld be just done without >>>>>> letting people know we are accessing an external server. >>>>>>=20 >>>>>> So I have reached the point where the script will not work as it is an= d I have not been able to figure out a way to find the country code that has = been selected for the IPFire >>>>>> installation. >>>>>>=20 >>>>>> Hopefully there is an easy option that someone can point me to that I = have missed to recognise. >>>>>=20 >>>>> How about to make it so simply that we simply read the value from /var/= ipfire/wlanap/settings? Where did we get this data from before? >>>=20 >>> Checking the source code files of crda it looks like it just checks the c= ontents of the environment variable COUNTRY. If it is blank or set to 00 then= crda just provides the global settings to the kernel, ie those that are in l= ine with every country, and therefore at the lowest applicable level. >>> So using wlanap/settings would be a workable approach as the COUNTRY valu= e could then be used from those settings. >>>=20 >>>=20 >>> So looks like two options:- >>> 1) wait for kernel 4.15 or later to be implemented in IPFire. >> There is a kernel branch that will land in =E2=80=9Cnext=E2=80=9D quite so= on. So you can go ahead and create a feature branch that drops CRDA which can= be merged after the kernel has been merged. >> This is slightly sad because it delays dropping Python 2 before the kernel= :( >>> 2) use the wlanap/settings value for COUNTRY and then add the udev entry = as per the Fedora approach. >> That is an option, but hostapd sets that anyways. What we do not set is th= e country code when running in client mode. That is where CRDA was required. >> I suppose it is okay to remain configured as =E2=80=9C00=E2=80=9D until th= e client associates with the AP. If the AP pushes its country code, the kerne= l will be notified and load the settings from the wireless reg db in the futu= re. So we won=E2=80=99t have to do anything. >>> Either way I will not be able to test out any change myself as my IPFire = system has no built in wireless card. I am using separate Wireless Access Poi= nts in my network. >> No problem. Would you still be up for dropping CRDA and repackaging wirele= ss-reg-db? >=20 > Yes I will give it a go. >=20 >> Running =E2=80=9Ciw reg set XX=E2=80=9D and =E2=80=9Ciw reg get=E2=80=9D s= hould show the correct results then if you are using the development kernel. >=20 > By development kernel do you mean the experimental kernel repo of Arne? Yes. You should be able to build everything based on next as normal and just = install the kernel package from here: https://people.ipfire.org/~arne_f/highly-experimental/kernel-5.10/ -Michael > Regards, > Adolf. >=20 >> -Michael >>>=20 >>> Regards, >>>=20 >>> Adolf >>>>>=20 >>>>> Best, >>>>> -Michael >>>>>=20 >>>>>> Looking forward to any suggestions. >>>>>>=20 >>>>>> Regards, >>>>>>=20 >>>>>> Adolf. >>>>>>> We could then drop crda and all dependencies. >>>>>>> Would you be up for having a look at this? >>>>>>> -Michael >>>>>>>> On 14 May 2021, at 20:28, Adolf Belka wro= te: >>>>>>>>=20 >>>>>>>> Hi Michael, >>>>>>>>=20 >>>>>>>> The blockage of nmap to removing python2 was resolved but I have a n= ew one now that may be more difficult to resolve. >>>>>>>>=20 >>>>>>>> python-m2crypt was able to be successfully built with python3. Howev= er crda uses python-m2crypt and from what I have read it only works with pyth= on2. >>>>>>>>=20 >>>>>>>> When I do the build then M2Crypto is in python3.8/site-packages/ and= crda gives the following error >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> make[1]: Entering directory '/usr/src/crda-4.14' >>>>>>>> GEN keys-gcrypt.c >>>>>>>> Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.k= ey.pub.pem >>>>>>>> ERROR: Failed to import the "M2Crypto" module: No module named _= m2crypto >>>>>>>> Please install the "M2Crypto" Python module. >>>>>>>> On Debian GNU/Linux the package is called "python-m2crypto". >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> No matter what I do crda doesn't find python-m2crypto. If I change t= he python-m2crypto lfs from using >>>>>>>>=20 >>>>>>>> cd $(DIR_APP) && python3 setup.py build >>>>>>>> cd $(DIR_APP) && python3 setup.py install --root=3D/ >>>>>>>>=20 >>>>>>>> to >>>>>>>>=20 >>>>>>>> cd $(DIR_APP) && python setup.py build >>>>>>>> cd $(DIR_APP) && python setup.py install --root=3D/ >>>>>>>>=20 >>>>>>>> then crda builds successfully. >>>>>>>>=20 >>>>>>>> Arch Linux and Gentoo have a python3 patch for crda (the same patch = for both) but it makes no difference for my build. >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> Regards, >>>>>>>>=20 >>>>>>>> Adolf >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> On 05/05/2021 16:27, Michael Tremer wrote: >>>>>>>>> Hello, >>>>>>>>>=20 >>>>>>>>> I would like to talk about what has been discussed at the last deve= loper conference call this week: Dropping Python 2 >>>>>>>>>=20 >>>>>>>>> This version of Python has reached its end of life and will no long= er receive any security fixes. Hence we need to get rid of it as soon as poss= ible. >>>>>>>>>=20 >>>>>>>>> However, there is lots of software that still depends on it which h= as to be migrated away first. >>>>>>>>>=20 >>>>>>>>> There are at least the following packages: >>>>>>>>>=20 >>>>>>>>> boost >>>>>>>>> fetchmail >>>>>>>>> fireinfo >>>>>>>>> iotop >>>>>>>>> ipaddr >>>>>>>>> libxml2 >>>>>>>>> libxslt >>>>>>>>> newt >>>>>>>>> nmap >>>>>>>>> python >>>>>>>>> python-clientform >>>>>>>>> python-daemon >>>>>>>>> python-distutils >>>>>>>>> python-distutils-extra >>>>>>>>> python-docutils >>>>>>>>> python-feedparser >>>>>>>>> python-inotify >>>>>>>>> python-ipaddress >>>>>>>>> python-m2crypto >>>>>>>>> python-mechanize >>>>>>>>> python-optional-src >>>>>>>>> python-pyparsing >>>>>>>>> python-rssdler >>>>>>>>> python-setuptools >>>>>>>>> python-six >>>>>>>>> python-typing >>>>>>>>>=20 >>>>>>>>> We also have the following scripts: >>>>>>>>>=20 >>>>>>>>> config/ca-certificates/certdata2pem.py >>>>>>>>> config/unbound/unbound-dhcp-leases-bridge >>>>>>>>>=20 >>>>>>>>> Fireinfo is written by us and has a lot of C which will make it a l= ittle bit more difficult to migrate. We would also have to be very careful to= not change any behaviour >>>>>>>>> of the current implementation. >>>>>>>>>=20 >>>>>>>>> The rest is probably either software that is entirely written in Py= thon 2 or software that brings bindings for Python. The latter case is easy b= ecause we can either force >>>>>>>>> it to build with Python 3 or we just disable the bindings. >>>>>>>>>=20 >>>>>>>>> Ultimately we might need to keep Python around in the build system = if there are other packages that rely on it. However, it would be great if we= were able to remove it >>>>>>>>> from the distribution very soon. Looking at his list, it does not s= eem to be too difficult. >>>>>>>>>=20 >>>>>>>>> Would anyone be up to help and remove Python from any of those pack= ages above? I would like to aim for Core Update 158 and remove as much stuff = as possible - if we can >>>>>>>>> everything - and then remove Python 2 in the update after that. If = anyone has any custom scripts or applications, people will have some time to = migrate away. >>>>>>>>>=20 >>>>>>>>> Best, >>>>>>>>> -Michael >>>>>>=20 >>>>>> --=20 >>>>>> Sent from my laptop >=20 > --=20 > Sent from my laptop --===============5139969712395763276==--