Re: Dropping Python 2 (python-m2crypto)

Re: Dropping Python 2 (python-m2crypto)

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 10046 bytes --]

Hi Michael,

On 31/05/2021 14:35, Michael Tremer wrote:
> Hello,
> 
>> On 29 May 2021, at 17:19, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>
>> Hi Jonatan and Michael,
>>
>> On 29/05/2021 16:31, Jonatan Schlag wrote:
>>> Hi,
>>> Am Dienstag, den 25.05.2021, 11:03 +0100 schrieb Michael Tremer:
>>>> Hello,
>>>>
>>>>> On 22 May 2021, at 13:25, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>
>>>>> Hi Michael and All,
>>>>>
>>>>> On 20/05/2021 12:15, Michael Tremer wrote:
>>>>>> Hello,
>>>>>> Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules.
>>>>>> Fedora has dropped crda a little while ago and added some scripts to wireless-regdb:
>>>>>>    https://src.fedoraproject.org/rpms/wireless-regdb
>>>>>> I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs:
>>>>>>    https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomain
>>> read the word "crda" today and was reminded of this thread. As crda seems to be deprecated, we could wait for a new kernel, which should make solving this problem easier.
>> Thanks Jonatan for finding this status. This might be a better option.
>>> Source is: https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/crda.git/commit/?id=f4ef2531698fb9ba006e8b31a223b3269be8bc7c
>>> and https://wiki.gentoo.org/wiki/Hostapd#Proper_use_of_the_5GHz_band
>>> Jonatan
>>>>> I have been working on this script but I have reached an impasse.
>>>>> The script as it is, presumes that /etc/localtime is a symlink to one of the timezones in /usr/share/zoneinfo/
>>>>> If localtime is a file and not a symlink then the scripts man page says that the system admin must create a file called regdomain containing the country code in /etc/sysconfig/
>>>>
>>>> I believe we are using a hardlink here. I would therefore drop that part of the code, or change /etc/localtime to a symlink.
>>
>> Changing etc/localtime to a softlink would work for new installations but the problem would be in upgrading existing installations. The localtime hardlink has no info on the country that was selected only on the timezone which covers multiple countries. So I don't think that changing the hardlink to a softlink for localtime is a viable option here.
>>
>>>>
>>>>> IPFire does not use symlinks to the zoneinfo files so the country code can't be created by the script.
>>>>>
>>>>> localtime can't just be read as it is a binary file. zdump could read it but that is not available in IPFire as it is commented out in the glibc rootfile and anyway it would
>>>>> say that the TZ="/etc/localtime" so that wouldn't help either. I tried it on another system that did have zdump.
>>>>>
>>>>> I thought I could use the location routine with the local-ipaddress from red but that won't work if IPFire is behind another router as red would then be a private ip address.
>>>>>
>>>>> Then I thought about doing a similar thing as with DDNS to "Guess the real public IP with help of an external server" but I am not sure if that could be just done without
>>>>> letting people know we are accessing an external server.
>>>>>
>>>>> So I have reached the point where the script will not work as it is and I have not been able to figure out a way to find the country code that has been selected for the IPFire
>>>>> installation.
>>>>>
>>>>> Hopefully there is an easy option that someone can point me to that I have missed to recognise.
>>>>
>>>> How about to make it so simply that we simply read the value from /var/ipfire/wlanap/settings? Where did we get this data from before?
>>
>> Checking the source code files of crda it looks like it just checks the contents of the environment variable COUNTRY. If it is blank or set to 00 then crda just provides the global settings to the kernel, ie those that are in line with every country, and  therefore at the lowest applicable level.
>> So using wlanap/settings would be a workable approach as the COUNTRY value could then be used from those settings.
>>
>>
>> So looks like two options:-
>> 1) wait for kernel 4.15 or later to be implemented in IPFire.
> 
> There is a kernel branch that will land in “next” quite soon. So you can go ahead and create a feature branch that drops CRDA which can be merged after the kernel has been merged.
> 
> This is slightly sad because it delays dropping Python 2 before the kernel :(
> 
>> 2) use the wlanap/settings value for COUNTRY and then add the udev entry as per the Fedora approach.
> 
> That is an option, but hostapd sets that anyways. What we do not set is the country code when running in client mode. That is where CRDA was required.
> 
> I suppose it is okay to remain configured as “00” until the client associates with the AP. If the AP pushes its country code, the kernel will be notified and load the settings from the wireless reg db in the future. So we won’t have to do anything.
> 
>> Either way I will not be able to test out any change myself as my IPFire system has no built in wireless card. I am using separate Wireless Access Points in my network.
> 
> No problem. Would you still be up for dropping CRDA and repackaging wireless-reg-db?

Yes I will give it a go.

> 
> Running “iw reg set XX” and “iw reg get” should show the correct results then if you are using the development kernel.

By development kernel do you mean the experimental kernel repo of Arne?

Regards,
Adolf.

> 
> -Michael
> 
>>
>> Regards,
>>
>> Adolf
>>>>
>>>> Best,
>>>> -Michael
>>>>
>>>>> Looking forward to any suggestions.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Adolf.
>>>>>> We could then drop crda and all dependencies.
>>>>>> Would you be up for having a look at this?
>>>>>> -Michael
>>>>>>> On 14 May 2021, at 20:28, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>
>>>>>>> Hi Michael,
>>>>>>>
>>>>>>> The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
>>>>>>>
>>>>>>> python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
>>>>>>>
>>>>>>> When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
>>>>>>>
>>>>>>>
>>>>>>>      make[1]: Entering directory '/usr/src/crda-4.14'
>>>>>>>      GEN  keys-gcrypt.c
>>>>>>>      Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem
>>>>>>>      ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto
>>>>>>>      Please install the "M2Crypto" Python module.
>>>>>>>      On Debian GNU/Linux the package is called "python-m2crypto".
>>>>>>>
>>>>>>>
>>>>>>> No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
>>>>>>>
>>>>>>>      cd $(DIR_APP) && python3 setup.py build
>>>>>>>      cd $(DIR_APP) && python3 setup.py install --root=/
>>>>>>>
>>>>>>> to
>>>>>>>
>>>>>>>      cd $(DIR_APP) && python setup.py build
>>>>>>>      cd $(DIR_APP) && python setup.py install --root=/
>>>>>>>
>>>>>>> then crda builds successfully.
>>>>>>>
>>>>>>> Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Adolf
>>>>>>>
>>>>>>>
>>>>>>> On 05/05/2021 16:27, Michael Tremer wrote:
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
>>>>>>>>
>>>>>>>> This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
>>>>>>>>
>>>>>>>> However, there is lots of software that still depends on it which has to be migrated away first.
>>>>>>>>
>>>>>>>> There are at least the following packages:
>>>>>>>>
>>>>>>>> boost
>>>>>>>> fetchmail
>>>>>>>> fireinfo
>>>>>>>> iotop
>>>>>>>> ipaddr
>>>>>>>> libxml2
>>>>>>>> libxslt
>>>>>>>> newt
>>>>>>>> nmap
>>>>>>>> python
>>>>>>>> python-clientform
>>>>>>>> python-daemon
>>>>>>>> python-distutils
>>>>>>>> python-distutils-extra
>>>>>>>> python-docutils
>>>>>>>> python-feedparser
>>>>>>>> python-inotify
>>>>>>>> python-ipaddress
>>>>>>>> python-m2crypto
>>>>>>>> python-mechanize
>>>>>>>> python-optional-src
>>>>>>>> python-pyparsing
>>>>>>>> python-rssdler
>>>>>>>> python-setuptools
>>>>>>>> python-six
>>>>>>>> python-typing
>>>>>>>>
>>>>>>>> We also have the following scripts:
>>>>>>>>
>>>>>>>> config/ca-certificates/certdata2pem.py
>>>>>>>> config/unbound/unbound-dhcp-leases-bridge
>>>>>>>>
>>>>>>>> Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour
>>>>>>>> of the current implementation.
>>>>>>>>
>>>>>>>> The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force
>>>>>>>> it to build with Python 3 or we just disable the bindings.
>>>>>>>>
>>>>>>>> Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it
>>>>>>>> from the distribution very soon. Looking at his list, it does not seem to be too difficult.
>>>>>>>>
>>>>>>>> Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can
>>>>>>>> everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> -Michael
>>>>>
>>>>> -- 
>>>>> Sent from my laptop
> 

-- 
Sent from my laptop

Re: Dropping Python 2 (python-m2crypto)

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 10537 bytes --]



> On 1 Jun 2021, at 14:57, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> 
> Hi Michael,
> 
> On 31/05/2021 14:35, Michael Tremer wrote:
>> Hello,
>>> On 29 May 2021, at 17:19, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>> 
>>> Hi Jonatan and Michael,
>>> 
>>> On 29/05/2021 16:31, Jonatan Schlag wrote:
>>>> Hi,
>>>> Am Dienstag, den 25.05.2021, 11:03 +0100 schrieb Michael Tremer:
>>>>> Hello,
>>>>> 
>>>>>> On 22 May 2021, at 13:25, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>> 
>>>>>> Hi Michael and All,
>>>>>> 
>>>>>> On 20/05/2021 12:15, Michael Tremer wrote:
>>>>>>> Hello,
>>>>>>> Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules.
>>>>>>> Fedora has dropped crda a little while ago and added some scripts to wireless-regdb:
>>>>>>>   https://src.fedoraproject.org/rpms/wireless-regdb
>>>>>>> I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs:
>>>>>>>   https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomain
>>>> read the word "crda" today and was reminded of this thread. As crda seems to be deprecated, we could wait for a new kernel, which should make solving this problem easier.
>>> Thanks Jonatan for finding this status. This might be a better option.
>>>> Source is: https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/crda.git/commit/?id=f4ef2531698fb9ba006e8b31a223b3269be8bc7c
>>>> and https://wiki.gentoo.org/wiki/Hostapd#Proper_use_of_the_5GHz_band
>>>> Jonatan
>>>>>> I have been working on this script but I have reached an impasse.
>>>>>> The script as it is, presumes that /etc/localtime is a symlink to one of the timezones in /usr/share/zoneinfo/
>>>>>> If localtime is a file and not a symlink then the scripts man page says that the system admin must create a file called regdomain containing the country code in /etc/sysconfig/
>>>>> 
>>>>> I believe we are using a hardlink here. I would therefore drop that part of the code, or change /etc/localtime to a symlink.
>>> 
>>> Changing etc/localtime to a softlink would work for new installations but the problem would be in upgrading existing installations. The localtime hardlink has no info on the country that was selected only on the timezone which covers multiple countries. So I don't think that changing the hardlink to a softlink for localtime is a viable option here.
>>> 
>>>>> 
>>>>>> IPFire does not use symlinks to the zoneinfo files so the country code can't be created by the script.
>>>>>> 
>>>>>> localtime can't just be read as it is a binary file. zdump could read it but that is not available in IPFire as it is commented out in the glibc rootfile and anyway it would
>>>>>> say that the TZ="/etc/localtime" so that wouldn't help either. I tried it on another system that did have zdump.
>>>>>> 
>>>>>> I thought I could use the location routine with the local-ipaddress from red but that won't work if IPFire is behind another router as red would then be a private ip address.
>>>>>> 
>>>>>> Then I thought about doing a similar thing as with DDNS to "Guess the real public IP with help of an external server" but I am not sure if that could be just done without
>>>>>> letting people know we are accessing an external server.
>>>>>> 
>>>>>> So I have reached the point where the script will not work as it is and I have not been able to figure out a way to find the country code that has been selected for the IPFire
>>>>>> installation.
>>>>>> 
>>>>>> Hopefully there is an easy option that someone can point me to that I have missed to recognise.
>>>>> 
>>>>> How about to make it so simply that we simply read the value from /var/ipfire/wlanap/settings? Where did we get this data from before?
>>> 
>>> Checking the source code files of crda it looks like it just checks the contents of the environment variable COUNTRY. If it is blank or set to 00 then crda just provides the global settings to the kernel, ie those that are in line with every country, and  therefore at the lowest applicable level.
>>> So using wlanap/settings would be a workable approach as the COUNTRY value could then be used from those settings.
>>> 
>>> 
>>> So looks like two options:-
>>> 1) wait for kernel 4.15 or later to be implemented in IPFire.
>> There is a kernel branch that will land in “next” quite soon. So you can go ahead and create a feature branch that drops CRDA which can be merged after the kernel has been merged.
>> This is slightly sad because it delays dropping Python 2 before the kernel :(
>>> 2) use the wlanap/settings value for COUNTRY and then add the udev entry as per the Fedora approach.
>> That is an option, but hostapd sets that anyways. What we do not set is the country code when running in client mode. That is where CRDA was required.
>> I suppose it is okay to remain configured as “00” until the client associates with the AP. If the AP pushes its country code, the kernel will be notified and load the settings from the wireless reg db in the future. So we won’t have to do anything.
>>> Either way I will not be able to test out any change myself as my IPFire system has no built in wireless card. I am using separate Wireless Access Points in my network.
>> No problem. Would you still be up for dropping CRDA and repackaging wireless-reg-db?
> 
> Yes I will give it a go.
> 
>> Running “iw reg set XX” and “iw reg get” should show the correct results then if you are using the development kernel.
> 
> By development kernel do you mean the experimental kernel repo of Arne?

Yes. You should be able to build everything based on next as normal and just install the kernel package from here:

  https://people.ipfire.org/~arne_f/highly-experimental/kernel-5.10/

-Michael

> Regards,
> Adolf.
> 
>> -Michael
>>> 
>>> Regards,
>>> 
>>> Adolf
>>>>> 
>>>>> Best,
>>>>> -Michael
>>>>> 
>>>>>> Looking forward to any suggestions.
>>>>>> 
>>>>>> Regards,
>>>>>> 
>>>>>> Adolf.
>>>>>>> We could then drop crda and all dependencies.
>>>>>>> Would you be up for having a look at this?
>>>>>>> -Michael
>>>>>>>> On 14 May 2021, at 20:28, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>> 
>>>>>>>> Hi Michael,
>>>>>>>> 
>>>>>>>> The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
>>>>>>>> 
>>>>>>>> python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
>>>>>>>> 
>>>>>>>> When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
>>>>>>>> 
>>>>>>>> 
>>>>>>>>     make[1]: Entering directory '/usr/src/crda-4.14'
>>>>>>>>     GEN  keys-gcrypt.c
>>>>>>>>     Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem
>>>>>>>>     ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto
>>>>>>>>     Please install the "M2Crypto" Python module.
>>>>>>>>     On Debian GNU/Linux the package is called "python-m2crypto".
>>>>>>>> 
>>>>>>>> 
>>>>>>>> No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
>>>>>>>> 
>>>>>>>>     cd $(DIR_APP) && python3 setup.py build
>>>>>>>>     cd $(DIR_APP) && python3 setup.py install --root=/
>>>>>>>> 
>>>>>>>> to
>>>>>>>> 
>>>>>>>>     cd $(DIR_APP) && python setup.py build
>>>>>>>>     cd $(DIR_APP) && python setup.py install --root=/
>>>>>>>> 
>>>>>>>> then crda builds successfully.
>>>>>>>> 
>>>>>>>> Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Regards,
>>>>>>>> 
>>>>>>>> Adolf
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On 05/05/2021 16:27, Michael Tremer wrote:
>>>>>>>>> Hello,
>>>>>>>>> 
>>>>>>>>> I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
>>>>>>>>> 
>>>>>>>>> This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
>>>>>>>>> 
>>>>>>>>> However, there is lots of software that still depends on it which has to be migrated away first.
>>>>>>>>> 
>>>>>>>>> There are at least the following packages:
>>>>>>>>> 
>>>>>>>>> boost
>>>>>>>>> fetchmail
>>>>>>>>> fireinfo
>>>>>>>>> iotop
>>>>>>>>> ipaddr
>>>>>>>>> libxml2
>>>>>>>>> libxslt
>>>>>>>>> newt
>>>>>>>>> nmap
>>>>>>>>> python
>>>>>>>>> python-clientform
>>>>>>>>> python-daemon
>>>>>>>>> python-distutils
>>>>>>>>> python-distutils-extra
>>>>>>>>> python-docutils
>>>>>>>>> python-feedparser
>>>>>>>>> python-inotify
>>>>>>>>> python-ipaddress
>>>>>>>>> python-m2crypto
>>>>>>>>> python-mechanize
>>>>>>>>> python-optional-src
>>>>>>>>> python-pyparsing
>>>>>>>>> python-rssdler
>>>>>>>>> python-setuptools
>>>>>>>>> python-six
>>>>>>>>> python-typing
>>>>>>>>> 
>>>>>>>>> We also have the following scripts:
>>>>>>>>> 
>>>>>>>>> config/ca-certificates/certdata2pem.py
>>>>>>>>> config/unbound/unbound-dhcp-leases-bridge
>>>>>>>>> 
>>>>>>>>> Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour
>>>>>>>>> of the current implementation.
>>>>>>>>> 
>>>>>>>>> The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force
>>>>>>>>> it to build with Python 3 or we just disable the bindings.
>>>>>>>>> 
>>>>>>>>> Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it
>>>>>>>>> from the distribution very soon. Looking at his list, it does not seem to be too difficult.
>>>>>>>>> 
>>>>>>>>> Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can
>>>>>>>>> everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
>>>>>>>>> 
>>>>>>>>> Best,
>>>>>>>>> -Michael
>>>>>> 
>>>>>> -- 
>>>>>> Sent from my laptop
> 
> -- 
> Sent from my laptop

Re: Dropping Python 2 (python-m2crypto)

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 8458 bytes --]

Hi Jonatan and Michael,

On 29/05/2021 16:31, Jonatan Schlag wrote:
> Hi,
> 
> Am Dienstag, den 25.05.2021, 11:03 +0100 schrieb Michael Tremer:
>> Hello,
>>
>>> On 22 May 2021, at 13:25, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>
>>> Hi Michael and All,
>>>
>>> On 20/05/2021 12:15, Michael Tremer wrote:
>>>> Hello,
>>>> Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules.
>>>> Fedora has dropped crda a little while ago and added some scripts to wireless-regdb:
>>>>    https://src.fedoraproject.org/rpms/wireless-regdb
>>>> I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs:
>>>>    https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomain
> 
> read the word "crda" today and was reminded of this thread. As crda seems to be deprecated, we could wait for a new kernel, which should make solving this problem easier.
Thanks Jonatan for finding this status. This might be a better option.
> 
> 
> Source is: https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/crda.git/commit/?id=f4ef2531698fb9ba006e8b31a223b3269be8bc7c
> and https://wiki.gentoo.org/wiki/Hostapd#Proper_use_of_the_5GHz_band
> 
> Jonatan
> 
>>> I have been working on this script but I have reached an impasse.
>>> The script as it is, presumes that /etc/localtime is a symlink to one of the timezones in /usr/share/zoneinfo/
>>> If localtime is a file and not a symlink then the scripts man page says that the system admin must create a file called regdomain containing the country code in /etc/sysconfig/
>>
>> I believe we are using a hardlink here. I would therefore drop that part of the code, or change /etc/localtime to a symlink.

Changing etc/localtime to a softlink would work for new installations but the problem would be in upgrading existing installations. The localtime hardlink has no info on the country that was selected only on the timezone which covers multiple countries. So I don't think that changing the hardlink to a softlink for localtime is a viable option here.

>>
>>> IPFire does not use symlinks to the zoneinfo files so the country code can't be created by the script.
>>>
>>> localtime can't just be read as it is a binary file. zdump could read it but that is not available in IPFire as it is commented out in the glibc rootfile and anyway it would
>>> say that the TZ="/etc/localtime" so that wouldn't help either. I tried it on another system that did have zdump.
>>>
>>> I thought I could use the location routine with the local-ipaddress from red but that won't work if IPFire is behind another router as red would then be a private ip address.
>>>
>>> Then I thought about doing a similar thing as with DDNS to "Guess the real public IP with help of an external server" but I am not sure if that could be just done without
>>> letting people know we are accessing an external server.
>>>
>>> So I have reached the point where the script will not work as it is and I have not been able to figure out a way to find the country code that has been selected for the IPFire
>>> installation.
>>>
>>> Hopefully there is an easy option that someone can point me to that I have missed to recognise.
>>
>> How about to make it so simply that we simply read the value from /var/ipfire/wlanap/settings? Where did we get this data from before?

Checking the source code files of crda it looks like it just checks the contents of the environment variable COUNTRY. If it is blank or set to 00 then crda just provides the global settings to the kernel, ie those that are in line with every country, and  therefore at the lowest applicable level.
So using wlanap/settings would be a workable approach as the COUNTRY value could then be used from those settings.


So looks like two options:-
1) wait for kernel 4.15 or later to be implemented in IPFire.
2) use the wlanap/settings value for COUNTRY and then add the udev entry as per the Fedora approach.

Either way I will not be able to test out any change myself as my IPFire system has no built in wireless card. I am using separate Wireless Access Points in my network.

Regards,

Adolf
>>
>> Best,
>> -Michael
>>
>>> Looking forward to any suggestions.
>>>
>>> Regards,
>>>
>>> Adolf.
>>>> We could then drop crda and all dependencies.
>>>> Would you be up for having a look at this?
>>>> -Michael
>>>>> On 14 May 2021, at 20:28, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>
>>>>> Hi Michael,
>>>>>
>>>>> The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
>>>>>
>>>>> python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
>>>>>
>>>>> When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
>>>>>
>>>>>
>>>>>      make[1]: Entering directory '/usr/src/crda-4.14'
>>>>>      GEN  keys-gcrypt.c
>>>>>      Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem
>>>>>      ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto
>>>>>      Please install the "M2Crypto" Python module.
>>>>>      On Debian GNU/Linux the package is called "python-m2crypto".
>>>>>
>>>>>
>>>>> No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
>>>>>
>>>>>      cd $(DIR_APP) && python3 setup.py build
>>>>>      cd $(DIR_APP) && python3 setup.py install --root=/
>>>>>
>>>>> to
>>>>>
>>>>>      cd $(DIR_APP) && python setup.py build
>>>>>      cd $(DIR_APP) && python setup.py install --root=/
>>>>>
>>>>> then crda builds successfully.
>>>>>
>>>>> Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Adolf
>>>>>
>>>>>
>>>>> On 05/05/2021 16:27, Michael Tremer wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
>>>>>>
>>>>>> This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
>>>>>>
>>>>>> However, there is lots of software that still depends on it which has to be migrated away first.
>>>>>>
>>>>>> There are at least the following packages:
>>>>>>
>>>>>> boost
>>>>>> fetchmail
>>>>>> fireinfo
>>>>>> iotop
>>>>>> ipaddr
>>>>>> libxml2
>>>>>> libxslt
>>>>>> newt
>>>>>> nmap
>>>>>> python
>>>>>> python-clientform
>>>>>> python-daemon
>>>>>> python-distutils
>>>>>> python-distutils-extra
>>>>>> python-docutils
>>>>>> python-feedparser
>>>>>> python-inotify
>>>>>> python-ipaddress
>>>>>> python-m2crypto
>>>>>> python-mechanize
>>>>>> python-optional-src
>>>>>> python-pyparsing
>>>>>> python-rssdler
>>>>>> python-setuptools
>>>>>> python-six
>>>>>> python-typing
>>>>>>
>>>>>> We also have the following scripts:
>>>>>>
>>>>>> config/ca-certificates/certdata2pem.py
>>>>>> config/unbound/unbound-dhcp-leases-bridge
>>>>>>
>>>>>> Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour
>>>>>> of the current implementation.
>>>>>>
>>>>>> The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force
>>>>>> it to build with Python 3 or we just disable the bindings.
>>>>>>
>>>>>> Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it
>>>>>> from the distribution very soon. Looking at his list, it does not seem to be too difficult.
>>>>>>
>>>>>> Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can
>>>>>> everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
>>>>>>
>>>>>> Best,
>>>>>> -Michael
>>>
>>> -- 
>>> Sent from my laptop
> 

Re: Dropping Python 2 (python-m2crypto)

[Jonatan Schlag]

[-- Attachment #1: Type: text/plain, Size: 7405 bytes --]

Hi,

Am Dienstag, den 25.05.2021, 11:03 +0100 schrieb Michael Tremer:
> Hello,
> 
> > On 22 May 2021, at 13:25, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> > 
> > Hi Michael and All,
> > 
> > On 20/05/2021 12:15, Michael Tremer wrote:
> > > Hello,
> > > Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules.
> > > Fedora has dropped crda a little while ago and added some scripts to wireless-regdb:
> > >   https://src.fedoraproject.org/rpms/wireless-regdb
> > > I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs:
> > >   https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomain

read the word "crda" today and was reminded of this thread. As crda seems to be deprecated, we could wait for a new kernel, which should make solving this problem easier.


Source is: https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/crda.git/commit/?id=f4ef2531698fb9ba006e8b31a223b3269be8bc7c
and https://wiki.gentoo.org/wiki/Hostapd#Proper_use_of_the_5GHz_band

Jonatan

> > I have been working on this script but I have reached an impasse.
> > The script as it is, presumes that /etc/localtime is a symlink to one of the timezones in /usr/share/zoneinfo/
> > If localtime is a file and not a symlink then the scripts man page says that the system admin must create a file called regdomain containing the country code in /etc/sysconfig/
> 
> I believe we are using a hardlink here. I would therefore drop that part of the code, or change /etc/localtime to a symlink.
> 
> > IPFire does not use symlinks to the zoneinfo files so the country code can't be created by the script.
> > 
> > localtime can't just be read as it is a binary file. zdump could read it but that is not available in IPFire as it is commented out in the glibc rootfile and anyway it would
> > say that the TZ="/etc/localtime" so that wouldn't help either. I tried it on another system that did have zdump.
> > 
> > I thought I could use the location routine with the local-ipaddress from red but that won't work if IPFire is behind another router as red would then be a private ip address.
> > 
> > Then I thought about doing a similar thing as with DDNS to "Guess the real public IP with help of an external server" but I am not sure if that could be just done without
> > letting people know we are accessing an external server.
> > 
> > So I have reached the point where the script will not work as it is and I have not been able to figure out a way to find the country code that has been selected for the IPFire
> > installation.
> > 
> > Hopefully there is an easy option that someone can point me to that I have missed to recognise.
> 
> How about to make it so simply that we simply read the value from /var/ipfire/wlanap/settings? Where did we get this data from before?
> 
> Best,
> -Michael
> 
> > Looking forward to any suggestions.
> > 
> > Regards,
> > 
> > Adolf.
> > > We could then drop crda and all dependencies.
> > > Would you be up for having a look at this?
> > > -Michael
> > > > On 14 May 2021, at 20:28, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> > > > 
> > > > Hi Michael,
> > > > 
> > > > The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
> > > > 
> > > > python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
> > > > 
> > > > When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
> > > > 
> > > > 
> > > >     make[1]: Entering directory '/usr/src/crda-4.14'
> > > >     GEN  keys-gcrypt.c
> > > >     Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem
> > > >     ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto
> > > >     Please install the "M2Crypto" Python module.
> > > >     On Debian GNU/Linux the package is called "python-m2crypto".
> > > > 
> > > > 
> > > > No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
> > > > 
> > > >     cd $(DIR_APP) && python3 setup.py build
> > > >     cd $(DIR_APP) && python3 setup.py install --root=/
> > > > 
> > > > to
> > > > 
> > > >     cd $(DIR_APP) && python setup.py build
> > > >     cd $(DIR_APP) && python setup.py install --root=/
> > > > 
> > > > then crda builds successfully.
> > > > 
> > > > Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
> > > > 
> > > > 
> > > > Regards,
> > > > 
> > > > Adolf
> > > > 
> > > > 
> > > > On 05/05/2021 16:27, Michael Tremer wrote:
> > > > > Hello,
> > > > > 
> > > > > I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
> > > > > 
> > > > > This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
> > > > > 
> > > > > However, there is lots of software that still depends on it which has to be migrated away first.
> > > > > 
> > > > > There are at least the following packages:
> > > > > 
> > > > > boost
> > > > > fetchmail
> > > > > fireinfo
> > > > > iotop
> > > > > ipaddr
> > > > > libxml2
> > > > > libxslt
> > > > > newt
> > > > > nmap
> > > > > python
> > > > > python-clientform
> > > > > python-daemon
> > > > > python-distutils
> > > > > python-distutils-extra
> > > > > python-docutils
> > > > > python-feedparser
> > > > > python-inotify
> > > > > python-ipaddress
> > > > > python-m2crypto
> > > > > python-mechanize
> > > > > python-optional-src
> > > > > python-pyparsing
> > > > > python-rssdler
> > > > > python-setuptools
> > > > > python-six
> > > > > python-typing
> > > > > 
> > > > > We also have the following scripts:
> > > > > 
> > > > > config/ca-certificates/certdata2pem.py
> > > > > config/unbound/unbound-dhcp-leases-bridge
> > > > > 
> > > > > Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour
> > > > > of the current implementation.
> > > > > 
> > > > > The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force
> > > > > it to build with Python 3 or we just disable the bindings.
> > > > > 
> > > > > Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it
> > > > > from the distribution very soon. Looking at his list, it does not seem to be too difficult.
> > > > > 
> > > > > Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can
> > > > > everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
> > > > > 
> > > > > Best,
> > > > > -Michael
> > 
> > -- 
> > Sent from my laptop

Re: Dropping Python 2 (python-m2crypto)

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 6361 bytes --]

Hello,

> On 22 May 2021, at 13:25, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> 
> Hi Michael and All,
> 
> On 20/05/2021 12:15, Michael Tremer wrote:
>> Hello,
>> Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules.
>> Fedora has dropped crda a little while ago and added some scripts to wireless-regdb:
>>   https://src.fedoraproject.org/rpms/wireless-regdb
>> I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs:
>>   https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomain
> I have been working on this script but I have reached an impasse.
> The script as it is, presumes that /etc/localtime is a symlink to one of the timezones in /usr/share/zoneinfo/
> If localtime is a file and not a symlink then the scripts man page says that the system admin must create a file called regdomain containing the country code in /etc/sysconfig/

I believe we are using a hardlink here. I would therefore drop that part of the code, or change /etc/localtime to a symlink.

> IPFire does not use symlinks to the zoneinfo files so the country code can't be created by the script.
> 
> localtime can't just be read as it is a binary file. zdump could read it but that is not available in IPFire as it is commented out in the glibc rootfile and anyway it would say that the TZ="/etc/localtime" so that wouldn't help either. I tried it on another system that did have zdump.
> 
> I thought I could use the location routine with the local-ipaddress from red but that won't work if IPFire is behind another router as red would then be a private ip address.
> 
> Then I thought about doing a similar thing as with DDNS to "Guess the real public IP with help of an external server" but I am not sure if that could be just done without letting people know we are accessing an external server.
> 
> So I have reached the point where the script will not work as it is and I have not been able to figure out a way to find the country code that has been selected for the IPFire installation.
> 
> Hopefully there is an easy option that someone can point me to that I have missed to recognise.

How about to make it so simply that we simply read the value from /var/ipfire/wlanap/settings? Where did we get this data from before?

Best,
-Michael

> Looking forward to any suggestions.
> 
> Regards,
> 
> Adolf.
>> We could then drop crda and all dependencies.
>> Would you be up for having a look at this?
>> -Michael
>>> On 14 May 2021, at 20:28, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>> 
>>> Hi Michael,
>>> 
>>> The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
>>> 
>>> python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
>>> 
>>> When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
>>> 
>>> 
>>>     make[1]: Entering directory '/usr/src/crda-4.14'
>>>     GEN  keys-gcrypt.c
>>>     Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem
>>>     ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto
>>>     Please install the "M2Crypto" Python module.
>>>     On Debian GNU/Linux the package is called "python-m2crypto".
>>> 
>>> 
>>> No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
>>> 
>>>     cd $(DIR_APP) && python3 setup.py build
>>>     cd $(DIR_APP) && python3 setup.py install --root=/
>>> 
>>> to
>>> 
>>>     cd $(DIR_APP) && python setup.py build
>>>     cd $(DIR_APP) && python setup.py install --root=/
>>> 
>>> then crda builds successfully.
>>> 
>>> Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
>>> 
>>> 
>>> Regards,
>>> 
>>> Adolf
>>> 
>>> 
>>> On 05/05/2021 16:27, Michael Tremer wrote:
>>>> Hello,
>>>> 
>>>> I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
>>>> 
>>>> This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
>>>> 
>>>> However, there is lots of software that still depends on it which has to be migrated away first.
>>>> 
>>>> There are at least the following packages:
>>>> 
>>>> boost
>>>> fetchmail
>>>> fireinfo
>>>> iotop
>>>> ipaddr
>>>> libxml2
>>>> libxslt
>>>> newt
>>>> nmap
>>>> python
>>>> python-clientform
>>>> python-daemon
>>>> python-distutils
>>>> python-distutils-extra
>>>> python-docutils
>>>> python-feedparser
>>>> python-inotify
>>>> python-ipaddress
>>>> python-m2crypto
>>>> python-mechanize
>>>> python-optional-src
>>>> python-pyparsing
>>>> python-rssdler
>>>> python-setuptools
>>>> python-six
>>>> python-typing
>>>> 
>>>> We also have the following scripts:
>>>> 
>>>> config/ca-certificates/certdata2pem.py
>>>> config/unbound/unbound-dhcp-leases-bridge
>>>> 
>>>> Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
>>>> 
>>>> The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
>>>> 
>>>> Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
>>>> 
>>>> Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
>>>> 
>>>> Best,
>>>> -Michael
> 
> -- 
> Sent from my laptop

Re: Dropping Python 2 (python-m2crypto)

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 5886 bytes --]

Hi Michael and All,

On 20/05/2021 12:15, Michael Tremer wrote:
> Hello,
> 
> Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules.
> 
> Fedora has dropped crda a little while ago and added some scripts to wireless-regdb:
> 
>    https://src.fedoraproject.org/rpms/wireless-regdb
> 
> I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs:
> 
>    https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomain
I have been working on this script but I have reached an impasse.
The script as it is, presumes that /etc/localtime is a symlink to one of 
the timezones in /usr/share/zoneinfo/
If localtime is a file and not a symlink then the scripts man page says 
that the system admin must create a file called regdomain containing the 
country code in /etc/sysconfig/

IPFire does not use symlinks to the zoneinfo files so the country code 
can't be created by the script.

localtime can't just be read as it is a binary file. zdump could read it 
but that is not available in IPFire as it is commented out in the glibc 
rootfile and anyway it would say that the TZ="/etc/localtime" so that 
wouldn't help either. I tried it on another system that did have zdump.

I thought I could use the location routine with the local-ipaddress from 
red but that won't work if IPFire is behind another router as red would 
then be a private ip address.

Then I thought about doing a similar thing as with DDNS to "Guess the 
real public IP with help of an external server" but I am not sure if 
that could be just done without letting people know we are accessing an 
external server.

So I have reached the point where the script will not work as it is and 
I have not been able to figure out a way to find the country code that 
has been selected for the IPFire installation.

Hopefully there is an easy option that someone can point me to that I 
have missed to recognise.

Looking forward to any suggestions.

Regards,

Adolf.
> 
> We could then drop crda and all dependencies.
> 
> Would you be up for having a look at this?
> 
> -Michael
> 
>> On 14 May 2021, at 20:28, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>
>> Hi Michael,
>>
>> The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
>>
>> python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
>>
>> When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
>>
>>
>>      make[1]: Entering directory '/usr/src/crda-4.14'
>>      GEN  keys-gcrypt.c
>>      Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem
>>      ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto
>>      Please install the "M2Crypto" Python module.
>>      On Debian GNU/Linux the package is called "python-m2crypto".
>>
>>
>> No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
>>
>>      cd $(DIR_APP) && python3 setup.py build
>>      cd $(DIR_APP) && python3 setup.py install --root=/
>>
>> to
>>
>>      cd $(DIR_APP) && python setup.py build
>>      cd $(DIR_APP) && python setup.py install --root=/
>>
>> then crda builds successfully.
>>
>> Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
>>
>>
>> Regards,
>>
>> Adolf
>>
>>
>> On 05/05/2021 16:27, Michael Tremer wrote:
>>> Hello,
>>>
>>> I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
>>>
>>> This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
>>>
>>> However, there is lots of software that still depends on it which has to be migrated away first.
>>>
>>> There are at least the following packages:
>>>
>>> boost
>>> fetchmail
>>> fireinfo
>>> iotop
>>> ipaddr
>>> libxml2
>>> libxslt
>>> newt
>>> nmap
>>> python
>>> python-clientform
>>> python-daemon
>>> python-distutils
>>> python-distutils-extra
>>> python-docutils
>>> python-feedparser
>>> python-inotify
>>> python-ipaddress
>>> python-m2crypto
>>> python-mechanize
>>> python-optional-src
>>> python-pyparsing
>>> python-rssdler
>>> python-setuptools
>>> python-six
>>> python-typing
>>>
>>> We also have the following scripts:
>>>
>>> config/ca-certificates/certdata2pem.py
>>> config/unbound/unbound-dhcp-leases-bridge
>>>
>>> Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
>>>
>>> The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
>>>
>>> Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
>>>
>>> Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
>>>
>>> Best,
>>> -Michael
> 

-- 
Sent from my laptop

Re: Dropping Python 2 (python-m2crypto)

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 4682 bytes --]



> On 20 May 2021, at 11:30, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> 
> Hi Michael,
> 
> On 20/05/2021 12:15, Michael Tremer wrote:
>> Hello,
>> Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules.
>> Fedora has dropped crda a little while ago and added some scripts to wireless-regdb:
>>   https://src.fedoraproject.org/rpms/wireless-regdb
>> I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs:
>>   https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomain
>> We could then drop crda and all dependencies.
>> Would you be up for having a look at this?
> Absolutely. I will give it a go and see how I get on.
> Good news about crda because then we should eventually be able to remove python2.

That would make me quite happy :)

> 
> Regards,
> Adolf.
>> -Michael
>>> On 14 May 2021, at 20:28, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>> 
>>> Hi Michael,
>>> 
>>> The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
>>> 
>>> python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
>>> 
>>> When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
>>> 
>>> 
>>>     make[1]: Entering directory '/usr/src/crda-4.14'
>>>     GEN  keys-gcrypt.c
>>>     Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem
>>>     ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto
>>>     Please install the "M2Crypto" Python module.
>>>     On Debian GNU/Linux the package is called "python-m2crypto".
>>> 
>>> 
>>> No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
>>> 
>>>     cd $(DIR_APP) && python3 setup.py build
>>>     cd $(DIR_APP) && python3 setup.py install --root=/
>>> 
>>> to
>>> 
>>>     cd $(DIR_APP) && python setup.py build
>>>     cd $(DIR_APP) && python setup.py install --root=/
>>> 
>>> then crda builds successfully.
>>> 
>>> Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
>>> 
>>> 
>>> Regards,
>>> 
>>> Adolf
>>> 
>>> 
>>> On 05/05/2021 16:27, Michael Tremer wrote:
>>>> Hello,
>>>> 
>>>> I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
>>>> 
>>>> This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
>>>> 
>>>> However, there is lots of software that still depends on it which has to be migrated away first.
>>>> 
>>>> There are at least the following packages:
>>>> 
>>>> boost
>>>> fetchmail
>>>> fireinfo
>>>> iotop
>>>> ipaddr
>>>> libxml2
>>>> libxslt
>>>> newt
>>>> nmap
>>>> python
>>>> python-clientform
>>>> python-daemon
>>>> python-distutils
>>>> python-distutils-extra
>>>> python-docutils
>>>> python-feedparser
>>>> python-inotify
>>>> python-ipaddress
>>>> python-m2crypto
>>>> python-mechanize
>>>> python-optional-src
>>>> python-pyparsing
>>>> python-rssdler
>>>> python-setuptools
>>>> python-six
>>>> python-typing
>>>> 
>>>> We also have the following scripts:
>>>> 
>>>> config/ca-certificates/certdata2pem.py
>>>> config/unbound/unbound-dhcp-leases-bridge
>>>> 
>>>> Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
>>>> 
>>>> The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
>>>> 
>>>> Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
>>>> 
>>>> Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
>>>> 
>>>> Best,
>>>> -Michael

Re: Dropping Python 2 (python-m2crypto)

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 4463 bytes --]

Hi Michael,

On 20/05/2021 12:15, Michael Tremer wrote:
> Hello,
> 
> Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules.
> 
> Fedora has dropped crda a little while ago and added some scripts to wireless-regdb:
> 
>    https://src.fedoraproject.org/rpms/wireless-regdb
> 
> I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs:
> 
>    https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomain
> 
> We could then drop crda and all dependencies.
> 
> Would you be up for having a look at this?
Absolutely. I will give it a go and see how I get on.
Good news about crda because then we should eventually be able to remove python2.

Regards,
Adolf.
> 
> -Michael
> 
>> On 14 May 2021, at 20:28, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>
>> Hi Michael,
>>
>> The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
>>
>> python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
>>
>> When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
>>
>>
>>      make[1]: Entering directory '/usr/src/crda-4.14'
>>      GEN  keys-gcrypt.c
>>      Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem
>>      ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto
>>      Please install the "M2Crypto" Python module.
>>      On Debian GNU/Linux the package is called "python-m2crypto".
>>
>>
>> No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
>>
>>      cd $(DIR_APP) && python3 setup.py build
>>      cd $(DIR_APP) && python3 setup.py install --root=/
>>
>> to
>>
>>      cd $(DIR_APP) && python setup.py build
>>      cd $(DIR_APP) && python setup.py install --root=/
>>
>> then crda builds successfully.
>>
>> Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
>>
>>
>> Regards,
>>
>> Adolf
>>
>>
>> On 05/05/2021 16:27, Michael Tremer wrote:
>>> Hello,
>>>
>>> I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
>>>
>>> This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
>>>
>>> However, there is lots of software that still depends on it which has to be migrated away first.
>>>
>>> There are at least the following packages:
>>>
>>> boost
>>> fetchmail
>>> fireinfo
>>> iotop
>>> ipaddr
>>> libxml2
>>> libxslt
>>> newt
>>> nmap
>>> python
>>> python-clientform
>>> python-daemon
>>> python-distutils
>>> python-distutils-extra
>>> python-docutils
>>> python-feedparser
>>> python-inotify
>>> python-ipaddress
>>> python-m2crypto
>>> python-mechanize
>>> python-optional-src
>>> python-pyparsing
>>> python-rssdler
>>> python-setuptools
>>> python-six
>>> python-typing
>>>
>>> We also have the following scripts:
>>>
>>> config/ca-certificates/certdata2pem.py
>>> config/unbound/unbound-dhcp-leases-bridge
>>>
>>> Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
>>>
>>> The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
>>>
>>> Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
>>>
>>> Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
>>>
>>> Best,
>>> -Michael
> 

Re: Dropping Python 2 (python-m2crypto)

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 4134 bytes --]

Hello,

Since Core Update 158 is now open, I would like to finish migrating the remaining Python 2 modules.

Fedora has dropped crda a little while ago and added some scripts to wireless-regdb:

  https://src.fedoraproject.org/rpms/wireless-regdb

I think we should follow that path and add the udev rules to lfs/wireless-regdb and adapt this script for our needs:

  https://src.fedoraproject.org/rpms/wireless-regdb/blob/rawhide/f/setregdomain

We could then drop crda and all dependencies.

Would you be up for having a look at this?

-Michael

> On 14 May 2021, at 20:28, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> 
> Hi Michael,
> 
> The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
> 
> python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
> 
> When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
> 
> 
>     make[1]: Entering directory '/usr/src/crda-4.14'
>     GEN  keys-gcrypt.c
>     Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem
>     ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto
>     Please install the "M2Crypto" Python module.
>     On Debian GNU/Linux the package is called "python-m2crypto".
> 
> 
> No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
> 
>     cd $(DIR_APP) && python3 setup.py build
>     cd $(DIR_APP) && python3 setup.py install --root=/
> 
> to
> 
>     cd $(DIR_APP) && python setup.py build
>     cd $(DIR_APP) && python setup.py install --root=/
> 
> then crda builds successfully.
> 
> Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
> 
> 
> Regards,
> 
> Adolf
> 
> 
> On 05/05/2021 16:27, Michael Tremer wrote:
>> Hello,
>> 
>> I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
>> 
>> This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
>> 
>> However, there is lots of software that still depends on it which has to be migrated away first.
>> 
>> There are at least the following packages:
>> 
>> boost
>> fetchmail
>> fireinfo
>> iotop
>> ipaddr
>> libxml2
>> libxslt
>> newt
>> nmap
>> python
>> python-clientform
>> python-daemon
>> python-distutils
>> python-distutils-extra
>> python-docutils
>> python-feedparser
>> python-inotify
>> python-ipaddress
>> python-m2crypto
>> python-mechanize
>> python-optional-src
>> python-pyparsing
>> python-rssdler
>> python-setuptools
>> python-six
>> python-typing
>> 
>> We also have the following scripts:
>> 
>> config/ca-certificates/certdata2pem.py
>> config/unbound/unbound-dhcp-leases-bridge
>> 
>> Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
>> 
>> The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
>> 
>> Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
>> 
>> Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
>> 
>> Best,
>> -Michael

Re: Dropping Python 2 (python-m2crypto)

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 4397 bytes --]

Hi All,

So I migrated python-setuptools to python3-setuptools. The python3 version worked with python3-daemon with no problems.

However later on python-m2crypto failed to build successfully because it could not find python-setuptools. So python-m2crypto needs both python-typing and python-setuptools to be present. So this triplet looks to be the barrier to going fully python3.

Regards,

Adolf.

On 15/05/2021 23:03, Adolf Belka wrote:
> Hi All,
>
> If python-m2crypto has to stay as a python2 module then so does python-typing. python-m2crypto fails to build if python-typing is not present.
>
>
> python-typing is the python2 backport of the typing capability built in to core python3, so it is only needed for python2.
>
> Regards,
>
> Adolf.
>
> On 14/05/2021 21:28, Adolf Belka wrote:
>> Hi Michael,
>>
>> The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
>>
>> python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
>>
>> When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
>>
>>
>>     make[1]: Entering directory '/usr/src/crda-4.14'
>>     GEN  keys-gcrypt.c
>>     Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem
>>     ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto
>>     Please install the "M2Crypto" Python module.
>>     On Debian GNU/Linux the package is called "python-m2crypto".
>>
>>
>> No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
>>
>>     cd $(DIR_APP) && python3 setup.py build
>>     cd $(DIR_APP) && python3 setup.py install --root=/
>>
>> to
>>
>>     cd $(DIR_APP) && python setup.py build
>>     cd $(DIR_APP) && python setup.py install --root=/
>>
>> then crda builds successfully.
>>
>> Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
>>
>>
>> Regards,
>>
>> Adolf
>>
>>
>> On 05/05/2021 16:27, Michael Tremer wrote:
>>> Hello,
>>>
>>> I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
>>>
>>> This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
>>>
>>> However, there is lots of software that still depends on it which has to be migrated away first.
>>>
>>> There are at least the following packages:
>>>
>>> boost
>>> fetchmail
>>> fireinfo
>>> iotop
>>> ipaddr
>>> libxml2
>>> libxslt
>>> newt
>>> nmap
>>> python
>>> python-clientform
>>> python-daemon
>>> python-distutils
>>> python-distutils-extra
>>> python-docutils
>>> python-feedparser
>>> python-inotify
>>> python-ipaddress
>>> python-m2crypto
>>> python-mechanize
>>> python-optional-src
>>> python-pyparsing
>>> python-rssdler
>>> python-setuptools
>>> python-six
>>> python-typing
>>>
>>> We also have the following scripts:
>>>
>>> config/ca-certificates/certdata2pem.py
>>> config/unbound/unbound-dhcp-leases-bridge
>>>
>>> Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
>>>
>>> The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
>>>
>>> Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
>>>
>>> Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
>>>
>>> Best,
>>> -Michael

Re: Dropping Python 2 (python-m2crypto)

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 3837 bytes --]

Hi All,

If python-m2crypto has to stay as a python2 module then so does python-typing. python-m2crypto fails to build if python-typing is not present.


python-typing is the python2 backport of the typing capability built in to core python3, so it is only needed for python2.

Regards,

Adolf.

On 14/05/2021 21:28, Adolf Belka wrote:
> Hi Michael,
>
> The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.
>
> python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.
>
> When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error
>
>
>     make[1]: Entering directory '/usr/src/crda-4.14'
>     GEN  keys-gcrypt.c
>     Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem
>     ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto
>     Please install the "M2Crypto" Python module.
>     On Debian GNU/Linux the package is called "python-m2crypto".
>
>
> No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using
>
>     cd $(DIR_APP) && python3 setup.py build
>     cd $(DIR_APP) && python3 setup.py install --root=/
>
> to
>
>     cd $(DIR_APP) && python setup.py build
>     cd $(DIR_APP) && python setup.py install --root=/
>
> then crda builds successfully.
>
> Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.
>
>
> Regards,
>
> Adolf
>
>
> On 05/05/2021 16:27, Michael Tremer wrote:
>> Hello,
>>
>> I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
>>
>> This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
>>
>> However, there is lots of software that still depends on it which has to be migrated away first.
>>
>> There are at least the following packages:
>>
>> boost
>> fetchmail
>> fireinfo
>> iotop
>> ipaddr
>> libxml2
>> libxslt
>> newt
>> nmap
>> python
>> python-clientform
>> python-daemon
>> python-distutils
>> python-distutils-extra
>> python-docutils
>> python-feedparser
>> python-inotify
>> python-ipaddress
>> python-m2crypto
>> python-mechanize
>> python-optional-src
>> python-pyparsing
>> python-rssdler
>> python-setuptools
>> python-six
>> python-typing
>>
>> We also have the following scripts:
>>
>> config/ca-certificates/certdata2pem.py
>> config/unbound/unbound-dhcp-leases-bridge
>>
>> Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
>>
>> The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
>>
>> Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
>>
>> Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
>>
>> Best,
>> -Michael

Re: Dropping Python 2 (python-m2crypto)

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 3388 bytes --]

Hi Michael,

The blockage of nmap to removing python2 was resolved but I have a new one now that may be more difficult to resolve.

python-m2crypt was able to be successfully built with python3. However crda uses python-m2crypt and from what I have read it only works with python2.

When I do the build then M2Crypto is in python3.8/site-packages/ and crda gives the following error


     make[1]: Entering directory '/usr/src/crda-4.14'
     GEN  keys-gcrypt.c
     Trusted pubkeys: pubkeys/linville.key.pub.pem pubkeys/sforshee.key.pub.pem
     ERROR: Failed to import the "M2Crypto" module: No module named _m2crypto
     Please install the "M2Crypto" Python module.
     On Debian GNU/Linux the package is called "python-m2crypto".


No matter what I do crda doesn't find python-m2crypto. If I change the python-m2crypto lfs from using

     cd $(DIR_APP) && python3 setup.py build
     cd $(DIR_APP) && python3 setup.py install --root=/

to

     cd $(DIR_APP) && python setup.py build
     cd $(DIR_APP) && python setup.py install --root=/

then crda builds successfully.

Arch Linux and Gentoo have a python3 patch for crda (the same patch for both) but it makes no difference for my build.


Regards,

Adolf


On 05/05/2021 16:27, Michael Tremer wrote:
> Hello,
>
> I would like to talk about what has been discussed at the last developer conference call this week: Dropping Python 2
>
> This version of Python has reached its end of life and will no longer receive any security fixes. Hence we need to get rid of it as soon as possible.
>
> However, there is lots of software that still depends on it which has to be migrated away first.
>
> There are at least the following packages:
>
> boost
> fetchmail
> fireinfo
> iotop
> ipaddr
> libxml2
> libxslt
> newt
> nmap
> python
> python-clientform
> python-daemon
> python-distutils
> python-distutils-extra
> python-docutils
> python-feedparser
> python-inotify
> python-ipaddress
> python-m2crypto
> python-mechanize
> python-optional-src
> python-pyparsing
> python-rssdler
> python-setuptools
> python-six
> python-typing
>
> We also have the following scripts:
>
> config/ca-certificates/certdata2pem.py
> config/unbound/unbound-dhcp-leases-bridge
>
> Fireinfo is written by us and has a lot of C which will make it a little bit more difficult to migrate. We would also have to be very careful to not change any behaviour of the current implementation.
>
> The rest is probably either software that is entirely written in Python 2 or software that brings bindings for Python. The latter case is easy because we can either force it to build with Python 3 or we just disable the bindings.
>
> Ultimately we might need to keep Python around in the build system if there are other packages that rely on it. However, it would be great if we were able to remove it from the distribution very soon. Looking at his list, it does not seem to be too difficult.
>
> Would anyone be up to help and remove Python from any of those packages above? I would like to aim for Core Update 158 and remove as much stuff as possible - if we can everything - and then remove Python 2 in the update after that. If anyone has any custom scripts or applications, people will have some time to migrate away.
>
> Best,
> -Michael