From mboxrd@z Thu Jan 1 00:00:00 1970 From: Adolf Belka To: development@lists.ipfire.org Subject: Re: Dropping Python 2 (python-m2crypto) Date: Tue, 01 Jun 2021 15:57:42 +0200 Message-ID: <1a05dcbd-51dc-1331-088f-8a6619d5d518@ipfire.org> In-Reply-To: <4E6F7814-66BE-4B68-A1DF-C45377AA2F31@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3377141570614097257==" List-Id: --===============3377141570614097257== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Michael, On 31/05/2021 14:35, Michael Tremer wrote: > Hello, >=20 >> On 29 May 2021, at 17:19, Adolf Belka wrote: >> >> Hi Jonatan and Michael, >> >> On 29/05/2021 16:31, Jonatan Schlag wrote: >>> Hi, >>> Am Dienstag, den 25.05.2021, 11:03 +0100 schrieb Michael Tremer: >>>> Hello, >>>> >>>>> On 22 May 2021, at 13:25, Adolf Belka wrote: >>>>> >>>>> Hi Michael and All, >>>>> >>>>> On 20/05/2021 12:15, Michael Tremer wrote: >>>>>> Hello, >>>>>> Since Core Update 158 is now open, I would like to finish migrating th= e remaining Python 2 modules. >>>>>> Fedora has dropped crda a little while ago and added some scripts to w= ireless-regdb: >>>>>> https://src.fedoraproject.org/rpms/wireless-regdb >>>>>> I think we should follow that path and add the udev rules to lfs/wirel= ess-regdb and adapt this script for our needs: >>>>>> https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/se= tregdomain >>> read the word "crda" today and was reminded of this thread. As crda seems= to be deprecated, we could wait for a new kernel, which should make solving = this problem easier. >> Thanks Jonatan for finding this status. This might be a better option. >>> Source is: https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/crda.gi= t/commit/?id=3Df4ef2531698fb9ba006e8b31a223b3269be8bc7c >>> and https://wiki.gentoo.org/wiki/Hostapd#Proper_use_of_the_5GHz_band >>> Jonatan >>>>> I have been working on this script but I have reached an impasse. >>>>> The script as it is, presumes that /etc/localtime is a symlink to one o= f the timezones in /usr/share/zoneinfo/ >>>>> If localtime is a file and not a symlink then the scripts man page says= that the system admin must create a file called regdomain containing the cou= ntry code in /etc/sysconfig/ >>>> >>>> I believe we are using a hardlink here. I would therefore drop that part= of the code, or change /etc/localtime to a symlink. >> >> Changing etc/localtime to a softlink would work for new installations but = the problem would be in upgrading existing installations. The localtime hardl= ink has no info on the country that was selected only on the timezone which c= overs multiple countries. So I don't think that changing the hardlink to a so= ftlink for localtime is a viable option here. >> >>>> >>>>> IPFire does not use symlinks to the zoneinfo files so the country code = can't be created by the script. >>>>> >>>>> localtime can't just be read as it is a binary file. zdump could read i= t but that is not available in IPFire as it is commented out in the glibc roo= tfile and anyway it would >>>>> say that the TZ=3D"/etc/localtime" so that wouldn't help either. I trie= d it on another system that did have zdump. >>>>> >>>>> I thought I could use the location routine with the local-ipaddress fro= m red but that won't work if IPFire is behind another router as red would the= n be a private ip address. >>>>> >>>>> Then I thought about doing a similar thing as with DDNS to "Guess the r= eal public IP with help of an external server" but I am not sure if that coul= d be just done without >>>>> letting people know we are accessing an external server. >>>>> >>>>> So I have reached the point where the script will not work as it is and= I have not been able to figure out a way to find the country code that has b= een selected for the IPFire >>>>> installation. >>>>> >>>>> Hopefully there is an easy option that someone can point me to that I h= ave missed to recognise. >>>> >>>> How about to make it so simply that we simply read the value from /var/i= pfire/wlanap/settings? Where did we get this data from before? >> >> Checking the source code files of crda it looks like it just checks the co= ntents of the environment variable COUNTRY. If it is blank or set to 00 then = crda just provides the global settings to the kernel, ie those that are in li= ne with every country, and therefore at the lowest applicable level. >> So using wlanap/settings would be a workable approach as the COUNTRY value= could then be used from those settings. >> >> >> So looks like two options:- >> 1) wait for kernel 4.15 or later to be implemented in IPFire. >=20 > There is a kernel branch that will land in =E2=80=9Cnext=E2=80=9D quite soo= n. So you can go ahead and create a feature branch that drops CRDA which can = be merged after the kernel has been merged. >=20 > This is slightly sad because it delays dropping Python 2 before the kernel = :( >=20 >> 2) use the wlanap/settings value for COUNTRY and then add the udev entry a= s per the Fedora approach. >=20 > That is an option, but hostapd sets that anyways. What we do not set is the= country code when running in client mode. That is where CRDA was required. >=20 > I suppose it is okay to remain configured as =E2=80=9C00=E2=80=9D until the= client associates with the AP. If the AP pushes its country code, the kernel= will be notified and load the settings from the wireless reg db in the futur= e. So we won=E2=80=99t have to do anything. >=20 >> Either way I will not be able to test out any change myself as my IPFire s= ystem has no built in wireless card. I am using separate Wireless Access Poin= ts in my network. >=20 > No problem. Would you still be up for dropping CRDA and repackaging wireles= s-reg-db? Yes I will give it a go. >=20 > Running =E2=80=9Ciw reg set XX=E2=80=9D and =E2=80=9Ciw reg get=E2=80=9D sh= ould show the correct results then if you are using the development kernel. By development kernel do you mean the experimental kernel repo of Arne? Regards, Adolf. >=20 > -Michael >=20 >> >> Regards, >> >> Adolf >>>> >>>> Best, >>>> -Michael >>>> >>>>> Looking forward to any suggestions. >>>>> >>>>> Regards, >>>>> >>>>> Adolf. >>>>>> We could then drop crda and all dependencies. >>>>>> Would you be up for having a look at this? >>>>>> -Michael >>>>>>> On 14 May 2021, at 20:28, Adolf Belka wrot= e: >>>>>>> >>>>>>> Hi Michael, >>>>>>> >>>>>>> The blockage of nmap to removing python2 was resolved but I have a ne= w one now that may be more difficult to resolve. >>>>>>> >>>>>>> python-m2crypt was able to be successfully built with python3. Howeve= r crda uses python-m2crypt and from what I have read it only works with pytho= n2. >>>>>>> >>>>>>> When I do the build then M2Crypto is in python3.8/site-packages/ and = crda gives the following error >>>>>>> >>>>>>> >>>>>>> make[1]: Entering directory '/usr/src/crda-4.14' >>>>>>> GEN keys-gcrypt.c >>>>>>> Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.k= ey.pub.pem >>>>>>> ERROR: Failed to import the "M2Crypto" module: No module named _= m2crypto >>>>>>> Please install the "M2Crypto" Python module. >>>>>>> On Debian GNU/Linux the package is called "python-m2crypto". >>>>>>> >>>>>>> >>>>>>> No matter what I do crda doesn't find python-m2crypto. If I change th= e python-m2crypto lfs from using >>>>>>> >>>>>>> cd $(DIR_APP) && python3 setup.py build >>>>>>> cd $(DIR_APP) && python3 setup.py install --root=3D/ >>>>>>> >>>>>>> to >>>>>>> >>>>>>> cd $(DIR_APP) && python setup.py build >>>>>>> cd $(DIR_APP) && python setup.py install --root=3D/ >>>>>>> >>>>>>> then crda builds successfully. >>>>>>> >>>>>>> Arch Linux and Gentoo have a python3 patch for crda (the same patch f= or both) but it makes no difference for my build. >>>>>>> >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Adolf >>>>>>> >>>>>>> >>>>>>> On 05/05/2021 16:27, Michael Tremer wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> I would like to talk about what has been discussed at the last devel= oper conference call this week: Dropping Python 2 >>>>>>>> >>>>>>>> This version of Python has reached its end of life and will no longe= r receive any security fixes. Hence we need to get rid of it as soon as possi= ble. >>>>>>>> >>>>>>>> However, there is lots of software that still depends on it which ha= s to be migrated away first. >>>>>>>> >>>>>>>> There are at least the following packages: >>>>>>>> >>>>>>>> boost >>>>>>>> fetchmail >>>>>>>> fireinfo >>>>>>>> iotop >>>>>>>> ipaddr >>>>>>>> libxml2 >>>>>>>> libxslt >>>>>>>> newt >>>>>>>> nmap >>>>>>>> python >>>>>>>> python-clientform >>>>>>>> python-daemon >>>>>>>> python-distutils >>>>>>>> python-distutils-extra >>>>>>>> python-docutils >>>>>>>> python-feedparser >>>>>>>> python-inotify >>>>>>>> python-ipaddress >>>>>>>> python-m2crypto >>>>>>>> python-mechanize >>>>>>>> python-optional-src >>>>>>>> python-pyparsing >>>>>>>> python-rssdler >>>>>>>> python-setuptools >>>>>>>> python-six >>>>>>>> python-typing >>>>>>>> >>>>>>>> We also have the following scripts: >>>>>>>> >>>>>>>> config/ca-certificates/certdata2pem.py >>>>>>>> config/unbound/unbound-dhcp-leases-bridge >>>>>>>> >>>>>>>> Fireinfo is written by us and has a lot of C which will make it a li= ttle bit more difficult to migrate. We would also have to be very careful to = not change any behaviour >>>>>>>> of the current implementation. >>>>>>>> >>>>>>>> The rest is probably either software that is entirely written in Pyt= hon 2 or software that brings bindings for Python. The latter case is easy be= cause we can either force >>>>>>>> it to build with Python 3 or we just disable the bindings. >>>>>>>> >>>>>>>> Ultimately we might need to keep Python around in the build system i= f there are other packages that rely on it. However, it would be great if we = were able to remove it >>>>>>>> from the distribution very soon. Looking at his list, it does not se= em to be too difficult. >>>>>>>> >>>>>>>> Would anyone be up to help and remove Python from any of those packa= ges above? I would like to aim for Core Update 158 and remove as much stuff a= s possible - if we can >>>>>>>> everything - and then remove Python 2 in the update after that. If a= nyone has any custom scripts or applications, people will have some time to m= igrate away. >>>>>>>> >>>>>>>> Best, >>>>>>>> -Michael >>>>> >>>>> --=20 >>>>> Sent from my laptop >=20 --=20 Sent from my laptop --===============3377141570614097257==--