First results from running build without python2

First results from running build without python2

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 969 bytes --]

Hi All,

So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.


ca-certificates was flagged up.

There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.

I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)

I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.

TypeError: a bytes-like object is required, not 'str'


I don't know how to further move forward with this as I am totally unfamiliar with the python language.


Regards,

Adolf.

Re: First results from running build without python2

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 1678 bytes --]

Hello,

Thank you for looking into this.

This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.

However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.

If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.

-Michael

> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> 
> Hi All,
> 
> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
> 
> 
> ca-certificates was flagged up.
> 
> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
> 
> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
> 
> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
> 
> TypeError: a bytes-like object is required, not 'str'
> 
> 
> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
> 
> 
> Regards,
> 
> Adolf.
> 
> 

Re: First results from running build without python2

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 3264 bytes --]

Hi Michael & all,


On 03/08/2021 17:11, Michael Tremer wrote:
> Hello,
> 
> Thank you for looking into this.
> 
> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
> 
> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
I then commented ca-certificates out in make.sh and ran the build.
This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
py-compile is python2 based and the build stopped because it could not find python

There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.

I then commented out spice-protocol and ran the build.

It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
It looks like I should be able to tell it to use python3 in the ./configure

I commented out frr and nothing else failed before cdrom was reached.

So the packages that need to be made to work with python3 are
ca-certificates
spice-protocol
frr


I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.

Regards,
Adolf.

> 
> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
> 
> -Michael
> 
>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>
>> Hi All,
>>
>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>
>>
>> ca-certificates was flagged up.
>>
>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>
>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>
>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>
>> TypeError: a bytes-like object is required, not 'str'
>>
>>
>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>
>>
>> Regards,
>>
>> Adolf.
>>
>>
> 

Re: First results from running build without python2

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 3831 bytes --]

Hi All,

I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.

Will now go back and have another go with spice-protocol.

Regards,

Adolf.


On 03/08/2021 23:38, Adolf Belka wrote:
> Hi Michael & all,
>
>
> On 03/08/2021 17:11, Michael Tremer wrote:
>> Hello,
>>
>> Thank you for looking into this.
>>
>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>
>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
> I then commented ca-certificates out in make.sh and ran the build.
> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
> py-compile is python2 based and the build stopped because it could not find python
>
> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>
> I then commented out spice-protocol and ran the build.
>
> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
> It looks like I should be able to tell it to use python3 in the ./configure
>
> I commented out frr and nothing else failed before cdrom was reached.
>
> So the packages that need to be made to work with python3 are
> ca-certificates
> spice-protocol
> frr
>
>
> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>
> Regards,
> Adolf.
>
>>
>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>
>> -Michael
>>
>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>
>>> Hi All,
>>>
>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>
>>>
>>> ca-certificates was flagged up.
>>>
>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>
>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>
>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>
>>> TypeError: a bytes-like object is required, not 'str'
>>>
>>>
>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>
>>>
>>> Regards,
>>>
>>> Adolf.
>>>
>>>
>>

Re: First results from running build without python2

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 4176 bytes --]

Hello,

> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> 
> Hi All,
> 
> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.

Great. This could also resolve Matthias’ problem with building frr.

> Will now go back and have another go with spice-protocol.

Maybe it has a —-disable-python switch?

-Michael

> 
> Regards,
> 
> Adolf.
> 
> 
> On 03/08/2021 23:38, Adolf Belka wrote:
>> Hi Michael & all,
>> 
>> 
>> On 03/08/2021 17:11, Michael Tremer wrote:
>>> Hello,
>>> 
>>> Thank you for looking into this.
>>> 
>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>> 
>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>> I then commented ca-certificates out in make.sh and ran the build.
>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>> py-compile is python2 based and the build stopped because it could not find python
>> 
>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>> 
>> I then commented out spice-protocol and ran the build.
>> 
>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>> It looks like I should be able to tell it to use python3 in the ./configure
>> 
>> I commented out frr and nothing else failed before cdrom was reached.
>> 
>> So the packages that need to be made to work with python3 are
>> ca-certificates
>> spice-protocol
>> frr
>> 
>> 
>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>> 
>> Regards,
>> Adolf.
>> 
>>> 
>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>> 
>>> -Michael
>>> 
>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>> 
>>>> Hi All,
>>>> 
>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>> 
>>>> 
>>>> ca-certificates was flagged up.
>>>> 
>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>> 
>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>> 
>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>> 
>>>> TypeError: a bytes-like object is required, not 'str'
>>>> 
>>>> 
>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>> 
>>>> 
>>>> Regards,
>>>> 
>>>> Adolf.
>>>> 
>>>> 
>>> 

Re: First results from running build without python2

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 4770 bytes --]

Hi All,

On 04/08/2021 16:45, Michael Tremer wrote:
> Hello,
> 
>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>
>> Hi All,
>>
>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
> 
> Great. This could also resolve Matthias’ problem with building frr.
> 
>> Will now go back and have another go with spice-protocol.
> 
> Maybe it has a —-disable-python switch?
I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.

This now only leaves the ca-certificates script that needs to be updated to work with python3.

Regards,
Adolf.
> 
> -Michael
> 
>>
>> Regards,
>>
>> Adolf.
>>
>>
>> On 03/08/2021 23:38, Adolf Belka wrote:
>>> Hi Michael & all,
>>>
>>>
>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>> Hello,
>>>>
>>>> Thank you for looking into this.
>>>>
>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>
>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>> I then commented ca-certificates out in make.sh and ran the build.
>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>> py-compile is python2 based and the build stopped because it could not find python
>>>
>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>
>>> I then commented out spice-protocol and ran the build.
>>>
>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>
>>> I commented out frr and nothing else failed before cdrom was reached.
>>>
>>> So the packages that need to be made to work with python3 are
>>> ca-certificates
>>> spice-protocol
>>> frr
>>>
>>>
>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>
>>> Regards,
>>> Adolf.
>>>
>>>>
>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>
>>>> -Michael
>>>>
>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>
>>>>> Hi All,
>>>>>
>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>
>>>>>
>>>>> ca-certificates was flagged up.
>>>>>
>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>
>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>
>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>
>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>
>>>>>
>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Adolf.
>>>>>
>>>>>
>>>>
> 

Re: First results from running build without python2

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 7550 bytes --]

Hi All,

I had another go at the ca-certificates problem, the last barrier to getting rid of python2.

I found certdata2pem.py files from fedora and 2 from suse. I created build subdirectories for each version so I could just test running the build.sh file with each version of certdata2pem.py, including the IPFire current version after running through the 2to3 convertor.

fedora

The fedora certdata2pem.py file runs successfully with python3 but has sed commands built into it which fail to find certain files. The sed commands are not in the IPFire version.

The error message is

  -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', trust = ['CKA_TRUST_SERVER_AUTH', 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = ['serverAuth', 'emailProtection'], distrust = [], openssl-distrust = []
sed: can't read certs/*.crt: No such file or directory


suse

The first suse version runs successfully with python3 but also has the sed commands in it with the same error message.

The second suse version runs successfully with python3, does not have the sed commands and completes the build.sh script with no errors. However this certdata2pem.py file has a section that is in the IPFire version completely missing.


IPfire version after running through the 2to3 convertor

The following error message occurs

producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148
Traceback (most recent call last):
   File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in <module>
f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64)))
   File "/usr/lib/python3.9/base64.py", line 58, in b64encode
     encoded = binascii.b2a_base64(s, newline=False)
TypeError: a bytes-like object is required, not 'str'

The section that is failing is the section that is missing in the 2nd suse version. There is an identical fwrite line at line 206 but that does not seem to flag up the same TypeError message.


As the certdata2pem.py files from the other distributions vary significantly in content, with some having nearly double the number of lines of code, I think the best alternative is to fix the IPFire version so we stay consistent but I am unable to figure out how to fix the python code that is causing the "  TypeError: a bytes-like object is required, not 'str' " error message and need someone's help with that.

Let me know if there is any other information that I need to provide.


Regards,

Adolf.


On 07/08/2021 15:54, Adolf Belka wrote:
> Hi All,
>
> On 04/08/2021 16:45, Michael Tremer wrote:
>> Hello,
>>
>>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>
>>> Hi All,
>>>
>>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
>>
>> Great. This could also resolve Matthias’ problem with building frr.
>>
>>> Will now go back and have another go with spice-protocol.
>>
>> Maybe it has a —-disable-python switch?
> I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
> Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.
>
> This now only leaves the ca-certificates script that needs to be updated to work with python3.
>
> Regards,
> Adolf.
>>
>> -Michael
>>
>>>
>>> Regards,
>>>
>>> Adolf.
>>>
>>>
>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>> Hi Michael & all,
>>>>
>>>>
>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>> Hello,
>>>>>
>>>>> Thank you for looking into this.
>>>>>
>>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>>
>>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>>> I then commented ca-certificates out in make.sh and ran the build.
>>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>>> py-compile is python2 based and the build stopped because it could not find python
>>>>
>>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>>
>>>> I then commented out spice-protocol and ran the build.
>>>>
>>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>>
>>>> I commented out frr and nothing else failed before cdrom was reached.
>>>>
>>>> So the packages that need to be made to work with python3 are
>>>> ca-certificates
>>>> spice-protocol
>>>> frr
>>>>
>>>>
>>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>>
>>>> Regards,
>>>> Adolf.
>>>>
>>>>>
>>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>>
>>>>> -Michael
>>>>>
>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>>
>>>>>>
>>>>>> ca-certificates was flagged up.
>>>>>>
>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>>
>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>>
>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>>
>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>
>>>>>>
>>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Adolf.
>>>>>>
>>>>>>
>>>>>
>>

Re: First results from running build without python2

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 8224 bytes --]

Hello,

Is this the one with the broken sed command?

  https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2pem.py

This should run if you execute it in the right directory:

pushd %{name}/certs
 pwd
 cp certdata.txt .
 python3 certdata2pem.py
popd

The fedora version no longer has the build.sh script.

-Michael

> On 8 Aug 2021, at 14:47, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> 
> Hi All,
> 
> I had another go at the ca-certificates problem, the last barrier to getting rid of python2.
> 
> I found certdata2pem.py files from fedora and 2 from suse. I created build subdirectories for each version so I could just test running the build.sh file with each version of certdata2pem.py, including the IPFire current version after running through the 2to3 convertor.
> 
> fedora
> 
> The fedora certdata2pem.py file runs successfully with python3 but has sed commands built into it which fail to find certain files. The sed commands are not in the IPFire version.
> 
> The error message is
> 
>  -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', trust = ['CKA_TRUST_SERVER_AUTH', 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = ['serverAuth', 'emailProtection'], distrust = [], openssl-distrust = []
> sed: can't read certs/*.crt: No such file or directory
> 
> 
> suse
> 
> The first suse version runs successfully with python3 but also has the sed commands in it with the same error message.
> 
> The second suse version runs successfully with python3, does not have the sed commands and completes the build.sh script with no errors. However this certdata2pem.py file has a section that is in the IPFire version completely missing.
> 
> 
> IPfire version after running through the 2to3 convertor
> 
> The following error message occurs
> 
> producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148
> Traceback (most recent call last):
>   File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in <module>
> f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64)))
>   File "/usr/lib/python3.9/base64.py", line 58, in b64encode
>     encoded = binascii.b2a_base64(s, newline=False)
> TypeError: a bytes-like object is required, not 'str'
> 
> The section that is failing is the section that is missing in the 2nd suse version. There is an identical fwrite line at line 206 but that does not seem to flag up the same TypeError message.
> 
> 
> As the certdata2pem.py files from the other distributions vary significantly in content, with some having nearly double the number of lines of code, I think the best alternative is to fix the IPFire version so we stay consistent but I am unable to figure out how to fix the python code that is causing the "  TypeError: a bytes-like object is required, not 'str' " error message and need someone's help with that.
> 
> Let me know if there is any other information that I need to provide.
> 
> 
> Regards,
> 
> Adolf.
> 
> 
> On 07/08/2021 15:54, Adolf Belka wrote:
>> Hi All,
>> 
>> On 04/08/2021 16:45, Michael Tremer wrote:
>>> Hello,
>>> 
>>>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>> 
>>>> Hi All,
>>>> 
>>>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
>>> 
>>> Great. This could also resolve Matthias’ problem with building frr.
>>> 
>>>> Will now go back and have another go with spice-protocol.
>>> 
>>> Maybe it has a —-disable-python switch?
>> I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
>> Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.
>> 
>> This now only leaves the ca-certificates script that needs to be updated to work with python3.
>> 
>> Regards,
>> Adolf.
>>> 
>>> -Michael
>>> 
>>>> 
>>>> Regards,
>>>> 
>>>> Adolf.
>>>> 
>>>> 
>>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>>> Hi Michael & all,
>>>>> 
>>>>> 
>>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>>> Hello,
>>>>>> 
>>>>>> Thank you for looking into this.
>>>>>> 
>>>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>>> 
>>>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>>>> I then commented ca-certificates out in make.sh and ran the build.
>>>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>>>> py-compile is python2 based and the build stopped because it could not find python
>>>>> 
>>>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>>> 
>>>>> I then commented out spice-protocol and ran the build.
>>>>> 
>>>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>>> 
>>>>> I commented out frr and nothing else failed before cdrom was reached.
>>>>> 
>>>>> So the packages that need to be made to work with python3 are
>>>>> ca-certificates
>>>>> spice-protocol
>>>>> frr
>>>>> 
>>>>> 
>>>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>>> 
>>>>> Regards,
>>>>> Adolf.
>>>>> 
>>>>>> 
>>>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>>> 
>>>>>> -Michael
>>>>>> 
>>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>> 
>>>>>>> Hi All,
>>>>>>> 
>>>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>>> 
>>>>>>> 
>>>>>>> ca-certificates was flagged up.
>>>>>>> 
>>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>>> 
>>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>>> 
>>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>>> 
>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>> 
>>>>>>> 
>>>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>>> 
>>>>>>> 
>>>>>>> Regards,
>>>>>>> 
>>>>>>> Adolf.
>>>>>>> 
>>>>>>> 
>>>>>> 
>>> 

Re: First results from running build without python2

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 8620 bytes --]

Hi Michael,

On 11/08/2021 12:43, Michael Tremer wrote:
> Hello,
> 
> Is this the one with the broken sed command?
> 
>    https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2pem.py
Yes, this is that one. Confirmed with a diff.
> 
> This should run if you execute it in the right directory:
> 
> pushd %{name}/certs
>   pwd
>   cp certdata.txt .
>   python3 certdata2pem.py
> popd
I have just learnt about the pushd and popd commands. by doing a quick search. Never heard of them before.
> 
> The fedora version no longer has the build.sh script.
That was the bit I didn't realise.

Regards,
Adolf.
> 
> -Michael
> 
>> On 8 Aug 2021, at 14:47, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>
>> Hi All,
>>
>> I had another go at the ca-certificates problem, the last barrier to getting rid of python2.
>>
>> I found certdata2pem.py files from fedora and 2 from suse. I created build subdirectories for each version so I could just test running the build.sh file with each version of certdata2pem.py, including the IPFire current version after running through the 2to3 convertor.
>>
>> fedora
>>
>> The fedora certdata2pem.py file runs successfully with python3 but has sed commands built into it which fail to find certain files. The sed commands are not in the IPFire version.
>>
>> The error message is
>>
>>   -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', trust = ['CKA_TRUST_SERVER_AUTH', 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = ['serverAuth', 'emailProtection'], distrust = [], openssl-distrust = []
>> sed: can't read certs/*.crt: No such file or directory
>>
>>
>> suse
>>
>> The first suse version runs successfully with python3 but also has the sed commands in it with the same error message.
>>
>> The second suse version runs successfully with python3, does not have the sed commands and completes the build.sh script with no errors. However this certdata2pem.py file has a section that is in the IPFire version completely missing.
>>
>>
>> IPfire version after running through the 2to3 convertor
>>
>> The following error message occurs
>>
>> producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148
>> Traceback (most recent call last):
>>    File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in <module>
>> f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64)))
>>    File "/usr/lib/python3.9/base64.py", line 58, in b64encode
>>      encoded = binascii.b2a_base64(s, newline=False)
>> TypeError: a bytes-like object is required, not 'str'
>>
>> The section that is failing is the section that is missing in the 2nd suse version. There is an identical fwrite line at line 206 but that does not seem to flag up the same TypeError message.
>>
>>
>> As the certdata2pem.py files from the other distributions vary significantly in content, with some having nearly double the number of lines of code, I think the best alternative is to fix the IPFire version so we stay consistent but I am unable to figure out how to fix the python code that is causing the "  TypeError: a bytes-like object is required, not 'str' " error message and need someone's help with that.
>>
>> Let me know if there is any other information that I need to provide.
>>
>>
>> Regards,
>>
>> Adolf.
>>
>>
>> On 07/08/2021 15:54, Adolf Belka wrote:
>>> Hi All,
>>>
>>> On 04/08/2021 16:45, Michael Tremer wrote:
>>>> Hello,
>>>>
>>>>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>
>>>>> Hi All,
>>>>>
>>>>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
>>>>
>>>> Great. This could also resolve Matthias’ problem with building frr.
>>>>
>>>>> Will now go back and have another go with spice-protocol.
>>>>
>>>> Maybe it has a —-disable-python switch?
>>> I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
>>> Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.
>>>
>>> This now only leaves the ca-certificates script that needs to be updated to work with python3.
>>>
>>> Regards,
>>> Adolf.
>>>>
>>>> -Michael
>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Adolf.
>>>>>
>>>>>
>>>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>>>> Hi Michael & all,
>>>>>>
>>>>>>
>>>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> Thank you for looking into this.
>>>>>>>
>>>>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>>>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>>>>
>>>>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>>>>> I then commented ca-certificates out in make.sh and ran the build.
>>>>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>>>>> py-compile is python2 based and the build stopped because it could not find python
>>>>>>
>>>>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>>>>
>>>>>> I then commented out spice-protocol and ran the build.
>>>>>>
>>>>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>>>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>>>>
>>>>>> I commented out frr and nothing else failed before cdrom was reached.
>>>>>>
>>>>>> So the packages that need to be made to work with python3 are
>>>>>> ca-certificates
>>>>>> spice-protocol
>>>>>> frr
>>>>>>
>>>>>>
>>>>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>>>>
>>>>>> Regards,
>>>>>> Adolf.
>>>>>>
>>>>>>>
>>>>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>>>>
>>>>>>> -Michael
>>>>>>>
>>>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>>>>
>>>>>>>>
>>>>>>>> ca-certificates was flagged up.
>>>>>>>>
>>>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>>>>
>>>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>>>>
>>>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>>>>
>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>
>>>>>>>>
>>>>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Adolf.
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>
> 

Re: First results from running build without python2

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 9177 bytes --]

Hello,

> On 11 Aug 2021, at 15:03, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> 
> Hi Michael,
> 
> On 11/08/2021 12:43, Michael Tremer wrote:
>> Hello,
>> Is this the one with the broken sed command?
>>   https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2pem.py
> Yes, this is that one. Confirmed with a diff.
>> This should run if you execute it in the right directory:
>> pushd %{name}/certs
>>  pwd
>>  cp certdata.txt .
>>  python3 certdata2pem.py
>> popd
> I have just learnt about the pushd and popd commands. by doing a quick search. Never heard of them before.

It is just a version of “cd” that remembers where it has been.

So if you call “pushd some-directory”, then “popd” will bring you back to where you have been before.

“cd -“ does the same as popd now.

>> The fedora version no longer has the build.sh script.
> That was the bit I didn't realise.

No problem.

-Michael

> Regards,
> Adolf.
>> -Michael
>>> On 8 Aug 2021, at 14:47, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>> 
>>> Hi All,
>>> 
>>> I had another go at the ca-certificates problem, the last barrier to getting rid of python2.
>>> 
>>> I found certdata2pem.py files from fedora and 2 from suse. I created build subdirectories for each version so I could just test running the build.sh file with each version of certdata2pem.py, including the IPFire current version after running through the 2to3 convertor.
>>> 
>>> fedora
>>> 
>>> The fedora certdata2pem.py file runs successfully with python3 but has sed commands built into it which fail to find certain files. The sed commands are not in the IPFire version.
>>> 
>>> The error message is
>>> 
>>>  -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', trust = ['CKA_TRUST_SERVER_AUTH', 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = ['serverAuth', 'emailProtection'], distrust = [], openssl-distrust = []
>>> sed: can't read certs/*.crt: No such file or directory
>>> 
>>> 
>>> suse
>>> 
>>> The first suse version runs successfully with python3 but also has the sed commands in it with the same error message.
>>> 
>>> The second suse version runs successfully with python3, does not have the sed commands and completes the build.sh script with no errors. However this certdata2pem.py file has a section that is in the IPFire version completely missing.
>>> 
>>> 
>>> IPfire version after running through the 2to3 convertor
>>> 
>>> The following error message occurs
>>> 
>>> producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148
>>> Traceback (most recent call last):
>>>   File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in <module>
>>> f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64)))
>>>   File "/usr/lib/python3.9/base64.py", line 58, in b64encode
>>>     encoded = binascii.b2a_base64(s, newline=False)
>>> TypeError: a bytes-like object is required, not 'str'
>>> 
>>> The section that is failing is the section that is missing in the 2nd suse version. There is an identical fwrite line at line 206 but that does not seem to flag up the same TypeError message.
>>> 
>>> 
>>> As the certdata2pem.py files from the other distributions vary significantly in content, with some having nearly double the number of lines of code, I think the best alternative is to fix the IPFire version so we stay consistent but I am unable to figure out how to fix the python code that is causing the "  TypeError: a bytes-like object is required, not 'str' " error message and need someone's help with that.
>>> 
>>> Let me know if there is any other information that I need to provide.
>>> 
>>> 
>>> Regards,
>>> 
>>> Adolf.
>>> 
>>> 
>>> On 07/08/2021 15:54, Adolf Belka wrote:
>>>> Hi All,
>>>> 
>>>> On 04/08/2021 16:45, Michael Tremer wrote:
>>>>> Hello,
>>>>> 
>>>>>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>> 
>>>>>> Hi All,
>>>>>> 
>>>>>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
>>>>> 
>>>>> Great. This could also resolve Matthias’ problem with building frr.
>>>>> 
>>>>>> Will now go back and have another go with spice-protocol.
>>>>> 
>>>>> Maybe it has a —-disable-python switch?
>>>> I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
>>>> Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.
>>>> 
>>>> This now only leaves the ca-certificates script that needs to be updated to work with python3.
>>>> 
>>>> Regards,
>>>> Adolf.
>>>>> 
>>>>> -Michael
>>>>> 
>>>>>> 
>>>>>> Regards,
>>>>>> 
>>>>>> Adolf.
>>>>>> 
>>>>>> 
>>>>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>>>>> Hi Michael & all,
>>>>>>> 
>>>>>>> 
>>>>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>>>>> Hello,
>>>>>>>> 
>>>>>>>> Thank you for looking into this.
>>>>>>>> 
>>>>>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>>>>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>>>>> 
>>>>>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>>>>>> I then commented ca-certificates out in make.sh and ran the build.
>>>>>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>>>>>> py-compile is python2 based and the build stopped because it could not find python
>>>>>>> 
>>>>>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>>>>> 
>>>>>>> I then commented out spice-protocol and ran the build.
>>>>>>> 
>>>>>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>>>>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>>>>> 
>>>>>>> I commented out frr and nothing else failed before cdrom was reached.
>>>>>>> 
>>>>>>> So the packages that need to be made to work with python3 are
>>>>>>> ca-certificates
>>>>>>> spice-protocol
>>>>>>> frr
>>>>>>> 
>>>>>>> 
>>>>>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>>>>> 
>>>>>>> Regards,
>>>>>>> Adolf.
>>>>>>> 
>>>>>>>> 
>>>>>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>>>>> 
>>>>>>>> -Michael
>>>>>>>> 
>>>>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>> 
>>>>>>>>> Hi All,
>>>>>>>>> 
>>>>>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> ca-certificates was flagged up.
>>>>>>>>> 
>>>>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>>>>> 
>>>>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>>>>> 
>>>>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>>>>> 
>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Regards,
>>>>>>>>> 
>>>>>>>>> Adolf.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>> 

Re: First results from running build without python2

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 10644 bytes --]

Hi Michael,

On 12/08/2021 11:17, Michael Tremer wrote:
> Hello,
> 
>> On 11 Aug 2021, at 15:03, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>
>> Hi Michael,
>>
>> On 11/08/2021 12:43, Michael Tremer wrote:
>>> Hello,
>>> Is this the one with the broken sed command?
>>>    https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2pem.py
>> Yes, this is that one. Confirmed with a diff.
>>> This should run if you execute it in the right directory:
>>> pushd %{name}/certs
>>>   pwd
>>>   cp certdata.txt .
>>>   python3 certdata2pem.py
>>> popd
>> I have just learnt about the pushd and popd commands. by doing a quick search. Never heard of them before.
> 
> It is just a version of “cd” that remembers where it has been.
> 
> So if you call “pushd some-directory”, then “popd” will bring you back to where you have been before.
> 
> “cd -“ does the same as popd now.
> 
>>> The fedora version no longer has the build.sh script.
>> That was the bit I didn't realise.
> 
> No problem.

Not as simple as I hoped.

The new certdata2pem.py script no longer creates .crt files but .tmp-p11-kit files but that is as far as that script goes.

There is an update-ca-trust file in fedora which splits the various certs to their respective locations, with the openssl ones ending up in ca-bundle.trust.crt but this uses p11-kit. I suspect that it uses ca-bundle.trust.p11-kit generated later on in the %build section of the ca-certificate.spec file from the tmp-p11-kit files generated by the certdata2pem.py script. So it looks like p11-kit needs to be installed to make this work.

Alternatively I have found the following in LFS.
https://www.linuxfromscratch.org/blfs/view/svn/postlfs/make-ca.html
which seems to also create the bundled cert file but also requiring p11-kit but then talking about different certdata.txt versions that have various tunings. The one from LFS comes from the mozilla release branch but is modified to something called the Mercurial revision, or the different ones shipped by RedHat or OpenSUSE which use the version that comes with NSS.

All in all I am not sure which approach to use or how to actually build it in IPFire.

Definitely help required to know the correct way to go further with this.

Regards,
Adolf.
  
> -Michael
> 
>> Regards,
>> Adolf.
>>> -Michael
>>>> On 8 Aug 2021, at 14:47, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>
>>>> Hi All,
>>>>
>>>> I had another go at the ca-certificates problem, the last barrier to getting rid of python2.
>>>>
>>>> I found certdata2pem.py files from fedora and 2 from suse. I created build subdirectories for each version so I could just test running the build.sh file with each version of certdata2pem.py, including the IPFire current version after running through the 2to3 convertor.
>>>>
>>>> fedora
>>>>
>>>> The fedora certdata2pem.py file runs successfully with python3 but has sed commands built into it which fail to find certain files. The sed commands are not in the IPFire version.
>>>>
>>>> The error message is
>>>>
>>>>   -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', trust = ['CKA_TRUST_SERVER_AUTH', 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = ['serverAuth', 'emailProtection'], distrust = [], openssl-distrust = []
>>>> sed: can't read certs/*.crt: No such file or directory
>>>>
>>>>
>>>> suse
>>>>
>>>> The first suse version runs successfully with python3 but also has the sed commands in it with the same error message.
>>>>
>>>> The second suse version runs successfully with python3, does not have the sed commands and completes the build.sh script with no errors. However this certdata2pem.py file has a section that is in the IPFire version completely missing.
>>>>
>>>>
>>>> IPfire version after running through the 2to3 convertor
>>>>
>>>> The following error message occurs
>>>>
>>>> producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148
>>>> Traceback (most recent call last):
>>>>    File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in <module>
>>>> f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64)))
>>>>    File "/usr/lib/python3.9/base64.py", line 58, in b64encode
>>>>      encoded = binascii.b2a_base64(s, newline=False)
>>>> TypeError: a bytes-like object is required, not 'str'
>>>>
>>>> The section that is failing is the section that is missing in the 2nd suse version. There is an identical fwrite line at line 206 but that does not seem to flag up the same TypeError message.
>>>>
>>>>
>>>> As the certdata2pem.py files from the other distributions vary significantly in content, with some having nearly double the number of lines of code, I think the best alternative is to fix the IPFire version so we stay consistent but I am unable to figure out how to fix the python code that is causing the "  TypeError: a bytes-like object is required, not 'str' " error message and need someone's help with that.
>>>>
>>>> Let me know if there is any other information that I need to provide.
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Adolf.
>>>>
>>>>
>>>> On 07/08/2021 15:54, Adolf Belka wrote:
>>>>> Hi All,
>>>>>
>>>>> On 04/08/2021 16:45, Michael Tremer wrote:
>>>>>> Hello,
>>>>>>
>>>>>>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
>>>>>>
>>>>>> Great. This could also resolve Matthias’ problem with building frr.
>>>>>>
>>>>>>> Will now go back and have another go with spice-protocol.
>>>>>>
>>>>>> Maybe it has a —-disable-python switch?
>>>>> I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
>>>>> Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.
>>>>>
>>>>> This now only leaves the ca-certificates script that needs to be updated to work with python3.
>>>>>
>>>>> Regards,
>>>>> Adolf.
>>>>>>
>>>>>> -Michael
>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Adolf.
>>>>>>>
>>>>>>>
>>>>>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>>>>>> Hi Michael & all,
>>>>>>>>
>>>>>>>>
>>>>>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> Thank you for looking into this.
>>>>>>>>>
>>>>>>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>>>>>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>>>>>>
>>>>>>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>>>>>>> I then commented ca-certificates out in make.sh and ran the build.
>>>>>>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>>>>>>> py-compile is python2 based and the build stopped because it could not find python
>>>>>>>>
>>>>>>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>>>>>>
>>>>>>>> I then commented out spice-protocol and ran the build.
>>>>>>>>
>>>>>>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>>>>>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>>>>>>
>>>>>>>> I commented out frr and nothing else failed before cdrom was reached.
>>>>>>>>
>>>>>>>> So the packages that need to be made to work with python3 are
>>>>>>>> ca-certificates
>>>>>>>> spice-protocol
>>>>>>>> frr
>>>>>>>>
>>>>>>>>
>>>>>>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Adolf.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>>>>>>
>>>>>>>>> -Michael
>>>>>>>>>
>>>>>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi All,
>>>>>>>>>>
>>>>>>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ca-certificates was flagged up.
>>>>>>>>>>
>>>>>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>>>>>>
>>>>>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>>>>>>
>>>>>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>>>>>>
>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>>
>>>>>>>>>> Adolf.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>
> 

Re: First results from running build without python2

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 11590 bytes --]

Hi Michael,

On 12/08/2021 13:36, Adolf Belka wrote:
> Hi Michael,
>
> On 12/08/2021 11:17, Michael Tremer wrote:
>> Hello,
>>
>>> On 11 Aug 2021, at 15:03, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>
>>> Hi Michael,
>>>
>>> On 11/08/2021 12:43, Michael Tremer wrote:
>>>> Hello,
>>>> Is this the one with the broken sed command?
>>>> https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2pem.py
>>> Yes, this is that one. Confirmed with a diff.
>>>> This should run if you execute it in the right directory:
>>>> pushd %{name}/certs
>>>>   pwd
>>>>   cp certdata.txt .
>>>>   python3 certdata2pem.py
>>>> popd
>>> I have just learnt about the pushd and popd commands. by doing a quick search. Never heard of them before.
>>
>> It is just a version of “cd” that remembers where it has been.
>>
>> So if you call “pushd some-directory”, then “popd” will bring you back to where you have been before.
>>
>> “cd -“ does the same as popd now.
>>
>>>> The fedora version no longer has the build.sh script.
>>> That was the bit I didn't realise.
>>
>> No problem.
>
> Not as simple as I hoped.
>
> The new certdata2pem.py script no longer creates .crt files but .tmp-p11-kit files but that is as far as that script goes.
>
> There is an update-ca-trust file in fedora which splits the various certs to their respective locations, with the openssl ones ending up in ca-bundle.trust.crt but this uses p11-kit. I suspect that it uses ca-bundle.trust.p11-kit generated later on in the %build section of the ca-certificate.spec file from the tmp-p11-kit files generated by the certdata2pem.py script. So it looks like p11-kit needs to be installed to make this work.
>
> Alternatively I have found the following in LFS.
> https://www.linuxfromscratch.org/blfs/view/svn/postlfs/make-ca.html
> which seems to also create the bundled cert file but also requiring p11-kit but then talking about different certdata.txt versions that have various tunings. The one from LFS comes from the mozilla release branch but is modified to something called the Mercurial revision, or the different ones shipped by RedHat or OpenSUSE which use the version that comes with NSS.
>
> All in all I am not sure which approach to use or how to actually build it in IPFire.
>
> Definitely help required to know the correct way to go further with this.
>
Having said I needed help, I thought I would have another go and copied the lines from the %build section of the fedora ca-certificate.spec that created the ca-bundle.trust.p11.kit

As I have p11-kit installed on my Arch Linux system I then ran the command

p11-kit extract --format=openssl-bundle --filter=certificates --overwrite --comment ~/openssl/ca-bundle.trust.crt

and I successfully created the ca-bundle.trust.crt which is needed by the IPFire ca-certificates lfs file.

I will now try and create a new build script that will do all the new type stuff and also install the p11-kit library files and see how things go.

:crossed_fingers:

Adolf

> Regards,
> Adolf.
>
>> -Michael
>>
>>> Regards,
>>> Adolf.
>>>> -Michael
>>>>> On 8 Aug 2021, at 14:47, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>
>>>>> Hi All,
>>>>>
>>>>> I had another go at the ca-certificates problem, the last barrier to getting rid of python2.
>>>>>
>>>>> I found certdata2pem.py files from fedora and 2 from suse. I created build subdirectories for each version so I could just test running the build.sh file with each version of certdata2pem.py, including the IPFire current version after running through the 2to3 convertor.
>>>>>
>>>>> fedora
>>>>>
>>>>> The fedora certdata2pem.py file runs successfully with python3 but has sed commands built into it which fail to find certain files. The sed commands are not in the IPFire version.
>>>>>
>>>>> The error message is
>>>>>
>>>>>   -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', trust = ['CKA_TRUST_SERVER_AUTH', 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = ['serverAuth', 'emailProtection'], distrust = [], openssl-distrust = []
>>>>> sed: can't read certs/*.crt: No such file or directory
>>>>>
>>>>>
>>>>> suse
>>>>>
>>>>> The first suse version runs successfully with python3 but also has the sed commands in it with the same error message.
>>>>>
>>>>> The second suse version runs successfully with python3, does not have the sed commands and completes the build.sh script with no errors. However this certdata2pem.py file has a section that is in the IPFire version completely missing.
>>>>>
>>>>>
>>>>> IPfire version after running through the 2to3 convertor
>>>>>
>>>>> The following error message occurs
>>>>>
>>>>> producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148
>>>>> Traceback (most recent call last):
>>>>>    File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in <module>
>>>>> f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64)))
>>>>>    File "/usr/lib/python3.9/base64.py", line 58, in b64encode
>>>>>      encoded = binascii.b2a_base64(s, newline=False)
>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>
>>>>> The section that is failing is the section that is missing in the 2nd suse version. There is an identical fwrite line at line 206 but that does not seem to flag up the same TypeError message.
>>>>>
>>>>>
>>>>> As the certdata2pem.py files from the other distributions vary significantly in content, with some having nearly double the number of lines of code, I think the best alternative is to fix the IPFire version so we stay consistent but I am unable to figure out how to fix the python code that is causing the "  TypeError: a bytes-like object is required, not 'str' " error message and need someone's help with that.
>>>>>
>>>>> Let me know if there is any other information that I need to provide.
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Adolf.
>>>>>
>>>>>
>>>>> On 07/08/2021 15:54, Adolf Belka wrote:
>>>>>> Hi All,
>>>>>>
>>>>>> On 04/08/2021 16:45, Michael Tremer wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
>>>>>>>
>>>>>>> Great. This could also resolve Matthias’ problem with building frr.
>>>>>>>
>>>>>>>> Will now go back and have another go with spice-protocol.
>>>>>>>
>>>>>>> Maybe it has a —-disable-python switch?
>>>>>> I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
>>>>>> Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.
>>>>>>
>>>>>> This now only leaves the ca-certificates script that needs to be updated to work with python3.
>>>>>>
>>>>>> Regards,
>>>>>> Adolf.
>>>>>>>
>>>>>>> -Michael
>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Adolf.
>>>>>>>>
>>>>>>>>
>>>>>>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>>>>>>> Hi Michael & all,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> Thank you for looking into this.
>>>>>>>>>>
>>>>>>>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>>>>>>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>>>>>>>
>>>>>>>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>>>>>>>> I then commented ca-certificates out in make.sh and ran the build.
>>>>>>>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>>>>>>>> py-compile is python2 based and the build stopped because it could not find python
>>>>>>>>>
>>>>>>>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>>>>>>>
>>>>>>>>> I then commented out spice-protocol and ran the build.
>>>>>>>>>
>>>>>>>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>>>>>>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>>>>>>>
>>>>>>>>> I commented out frr and nothing else failed before cdrom was reached.
>>>>>>>>>
>>>>>>>>> So the packages that need to be made to work with python3 are
>>>>>>>>> ca-certificates
>>>>>>>>> spice-protocol
>>>>>>>>> frr
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Adolf.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>>>>>>>
>>>>>>>>>> -Michael
>>>>>>>>>>
>>>>>>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi All,
>>>>>>>>>>>
>>>>>>>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ca-certificates was flagged up.
>>>>>>>>>>>
>>>>>>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>>>>>>>
>>>>>>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>>>>>>>
>>>>>>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>>>>>>>
>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>
>>>>>>>>>>> Adolf.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>

Re: First results from running build without python2

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 12027 bytes --]

Hello,

Yes, this is the way to go :)

Reach out here if you need any help.

-Michael

> On 12 Aug 2021, at 13:10, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> 
> Hi Michael,
> 
> On 12/08/2021 13:36, Adolf Belka wrote:
>> Hi Michael,
>> 
>> On 12/08/2021 11:17, Michael Tremer wrote:
>>> Hello,
>>> 
>>>> On 11 Aug 2021, at 15:03, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>> 
>>>> Hi Michael,
>>>> 
>>>> On 11/08/2021 12:43, Michael Tremer wrote:
>>>>> Hello,
>>>>> Is this the one with the broken sed command?
>>>>> https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2pem.py
>>>> Yes, this is that one. Confirmed with a diff.
>>>>> This should run if you execute it in the right directory:
>>>>> pushd %{name}/certs
>>>>>   pwd
>>>>>   cp certdata.txt .
>>>>>   python3 certdata2pem.py
>>>>> popd
>>>> I have just learnt about the pushd and popd commands. by doing a quick search. Never heard of them before.
>>> 
>>> It is just a version of “cd” that remembers where it has been.
>>> 
>>> So if you call “pushd some-directory”, then “popd” will bring you back to where you have been before.
>>> 
>>> “cd -“ does the same as popd now.
>>> 
>>>>> The fedora version no longer has the build.sh script.
>>>> That was the bit I didn't realise.
>>> 
>>> No problem.
>> 
>> Not as simple as I hoped.
>> 
>> The new certdata2pem.py script no longer creates .crt files but .tmp-p11-kit files but that is as far as that script goes.
>> 
>> There is an update-ca-trust file in fedora which splits the various certs to their respective locations, with the openssl ones ending up in ca-bundle.trust.crt but this uses p11-kit. I suspect that it uses ca-bundle.trust.p11-kit generated later on in the %build section of the ca-certificate.spec file from the tmp-p11-kit files generated by the certdata2pem.py script. So it looks like p11-kit needs to be installed to make this work.
>> 
>> Alternatively I have found the following in LFS.
>> https://www.linuxfromscratch.org/blfs/view/svn/postlfs/make-ca.html
>> which seems to also create the bundled cert file but also requiring p11-kit but then talking about different certdata.txt versions that have various tunings. The one from LFS comes from the mozilla release branch but is modified to something called the Mercurial revision, or the different ones shipped by RedHat or OpenSUSE which use the version that comes with NSS.
>> 
>> All in all I am not sure which approach to use or how to actually build it in IPFire.
>> 
>> Definitely help required to know the correct way to go further with this.
>> 
> Having said I needed help, I thought I would have another go and copied the lines from the %build section of the fedora ca-certificate.spec that created the ca-bundle.trust.p11.kit
> 
> As I have p11-kit installed on my Arch Linux system I then ran the command
> 
> p11-kit extract --format=openssl-bundle --filter=certificates --overwrite --comment ~/openssl/ca-bundle.trust.crt
> 
> and I successfully created the ca-bundle.trust.crt which is needed by the IPFire ca-certificates lfs file.
> 
> I will now try and create a new build script that will do all the new type stuff and also install the p11-kit library files and see how things go.
> 
> :crossed_fingers:
> 
> Adolf
> 
>> Regards,
>> Adolf.
>> 
>>> -Michael
>>> 
>>>> Regards,
>>>> Adolf.
>>>>> -Michael
>>>>>> On 8 Aug 2021, at 14:47, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>> 
>>>>>> Hi All,
>>>>>> 
>>>>>> I had another go at the ca-certificates problem, the last barrier to getting rid of python2.
>>>>>> 
>>>>>> I found certdata2pem.py files from fedora and 2 from suse. I created build subdirectories for each version so I could just test running the build.sh file with each version of certdata2pem.py, including the IPFire current version after running through the 2to3 convertor.
>>>>>> 
>>>>>> fedora
>>>>>> 
>>>>>> The fedora certdata2pem.py file runs successfully with python3 but has sed commands built into it which fail to find certain files. The sed commands are not in the IPFire version.
>>>>>> 
>>>>>> The error message is
>>>>>> 
>>>>>>   -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', trust = ['CKA_TRUST_SERVER_AUTH', 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = ['serverAuth', 'emailProtection'], distrust = [], openssl-distrust = []
>>>>>> sed: can't read certs/*.crt: No such file or directory
>>>>>> 
>>>>>> 
>>>>>> suse
>>>>>> 
>>>>>> The first suse version runs successfully with python3 but also has the sed commands in it with the same error message.
>>>>>> 
>>>>>> The second suse version runs successfully with python3, does not have the sed commands and completes the build.sh script with no errors. However this certdata2pem.py file has a section that is in the IPFire version completely missing.
>>>>>> 
>>>>>> 
>>>>>> IPfire version after running through the 2to3 convertor
>>>>>> 
>>>>>> The following error message occurs
>>>>>> 
>>>>>> producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148
>>>>>> Traceback (most recent call last):
>>>>>>    File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in <module>
>>>>>> f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64)))
>>>>>>    File "/usr/lib/python3.9/base64.py", line 58, in b64encode
>>>>>>      encoded = binascii.b2a_base64(s, newline=False)
>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>> 
>>>>>> The section that is failing is the section that is missing in the 2nd suse version. There is an identical fwrite line at line 206 but that does not seem to flag up the same TypeError message.
>>>>>> 
>>>>>> 
>>>>>> As the certdata2pem.py files from the other distributions vary significantly in content, with some having nearly double the number of lines of code, I think the best alternative is to fix the IPFire version so we stay consistent but I am unable to figure out how to fix the python code that is causing the "  TypeError: a bytes-like object is required, not 'str' " error message and need someone's help with that.
>>>>>> 
>>>>>> Let me know if there is any other information that I need to provide.
>>>>>> 
>>>>>> 
>>>>>> Regards,
>>>>>> 
>>>>>> Adolf.
>>>>>> 
>>>>>> 
>>>>>> On 07/08/2021 15:54, Adolf Belka wrote:
>>>>>>> Hi All,
>>>>>>> 
>>>>>>> On 04/08/2021 16:45, Michael Tremer wrote:
>>>>>>>> Hello,
>>>>>>>> 
>>>>>>>>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>> 
>>>>>>>>> Hi All,
>>>>>>>>> 
>>>>>>>>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
>>>>>>>> 
>>>>>>>> Great. This could also resolve Matthias’ problem with building frr.
>>>>>>>> 
>>>>>>>>> Will now go back and have another go with spice-protocol.
>>>>>>>> 
>>>>>>>> Maybe it has a —-disable-python switch?
>>>>>>> I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
>>>>>>> Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.
>>>>>>> 
>>>>>>> This now only leaves the ca-certificates script that needs to be updated to work with python3.
>>>>>>> 
>>>>>>> Regards,
>>>>>>> Adolf.
>>>>>>>> 
>>>>>>>> -Michael
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Regards,
>>>>>>>>> 
>>>>>>>>> Adolf.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>>>>>>>> Hi Michael & all,
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>>>>>>>> Hello,
>>>>>>>>>>> 
>>>>>>>>>>> Thank you for looking into this.
>>>>>>>>>>> 
>>>>>>>>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>>>>>>>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>>>>>>>> 
>>>>>>>>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>>>>>>>>> I then commented ca-certificates out in make.sh and ran the build.
>>>>>>>>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>>>>>>>>> py-compile is python2 based and the build stopped because it could not find python
>>>>>>>>>> 
>>>>>>>>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>>>>>>>> 
>>>>>>>>>> I then commented out spice-protocol and ran the build.
>>>>>>>>>> 
>>>>>>>>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>>>>>>>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>>>>>>>> 
>>>>>>>>>> I commented out frr and nothing else failed before cdrom was reached.
>>>>>>>>>> 
>>>>>>>>>> So the packages that need to be made to work with python3 are
>>>>>>>>>> ca-certificates
>>>>>>>>>> spice-protocol
>>>>>>>>>> frr
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>>>>>>>> 
>>>>>>>>>> Regards,
>>>>>>>>>> Adolf.
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>>>>>>>> 
>>>>>>>>>>> -Michael
>>>>>>>>>>> 
>>>>>>>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Hi All,
>>>>>>>>>>>> 
>>>>>>>>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> ca-certificates was flagged up.
>>>>>>>>>>>> 
>>>>>>>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>>>>>>>> 
>>>>>>>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>>>>>>>> 
>>>>>>>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>>>>>>>> 
>>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> 
>>>>>>>>>>>> Adolf.

Re: First results from running build without python2

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 12668 bytes --]

Hi Michael,

On 12/08/2021 15:38, Michael Tremer wrote:
> Hello,
> 
> Yes, this is the way to go :)
> 
> Reach out here if you need any help.
I am trying to build p11-kit and it has flagged up libtasn1. It is an optional dependency and I can disable it but I would like to check if that is the correct thing to do or not.

libtasn1 is said to be required to allow the trust policy module to be built as well as other code that interacts with certificates.
Is this something that would be needed for IPFire or can I disable it in the p11-kit build.

Regards,
Adolf.
> 
> -Michael
> 
>> On 12 Aug 2021, at 13:10, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>
>> Hi Michael,
>>
>> On 12/08/2021 13:36, Adolf Belka wrote:
>>> Hi Michael,
>>>
>>> On 12/08/2021 11:17, Michael Tremer wrote:
>>>> Hello,
>>>>
>>>>> On 11 Aug 2021, at 15:03, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>
>>>>> Hi Michael,
>>>>>
>>>>> On 11/08/2021 12:43, Michael Tremer wrote:
>>>>>> Hello,
>>>>>> Is this the one with the broken sed command?
>>>>>> https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2pem.py
>>>>> Yes, this is that one. Confirmed with a diff.
>>>>>> This should run if you execute it in the right directory:
>>>>>> pushd %{name}/certs
>>>>>>    pwd
>>>>>>    cp certdata.txt .
>>>>>>    python3 certdata2pem.py
>>>>>> popd
>>>>> I have just learnt about the pushd and popd commands. by doing a quick search. Never heard of them before.
>>>>
>>>> It is just a version of “cd” that remembers where it has been.
>>>>
>>>> So if you call “pushd some-directory”, then “popd” will bring you back to where you have been before.
>>>>
>>>> “cd -“ does the same as popd now.
>>>>
>>>>>> The fedora version no longer has the build.sh script.
>>>>> That was the bit I didn't realise.
>>>>
>>>> No problem.
>>>
>>> Not as simple as I hoped.
>>>
>>> The new certdata2pem.py script no longer creates .crt files but .tmp-p11-kit files but that is as far as that script goes.
>>>
>>> There is an update-ca-trust file in fedora which splits the various certs to their respective locations, with the openssl ones ending up in ca-bundle.trust.crt but this uses p11-kit. I suspect that it uses ca-bundle.trust.p11-kit generated later on in the %build section of the ca-certificate.spec file from the tmp-p11-kit files generated by the certdata2pem.py script. So it looks like p11-kit needs to be installed to make this work.
>>>
>>> Alternatively I have found the following in LFS.
>>> https://www.linuxfromscratch.org/blfs/view/svn/postlfs/make-ca.html
>>> which seems to also create the bundled cert file but also requiring p11-kit but then talking about different certdata.txt versions that have various tunings. The one from LFS comes from the mozilla release branch but is modified to something called the Mercurial revision, or the different ones shipped by RedHat or OpenSUSE which use the version that comes with NSS.
>>>
>>> All in all I am not sure which approach to use or how to actually build it in IPFire.
>>>
>>> Definitely help required to know the correct way to go further with this.
>>>
>> Having said I needed help, I thought I would have another go and copied the lines from the %build section of the fedora ca-certificate.spec that created the ca-bundle.trust.p11.kit
>>
>> As I have p11-kit installed on my Arch Linux system I then ran the command
>>
>> p11-kit extract --format=openssl-bundle --filter=certificates --overwrite --comment ~/openssl/ca-bundle.trust.crt
>>
>> and I successfully created the ca-bundle.trust.crt which is needed by the IPFire ca-certificates lfs file.
>>
>> I will now try and create a new build script that will do all the new type stuff and also install the p11-kit library files and see how things go.
>>
>> :crossed_fingers:
>>
>> Adolf
>>
>>> Regards,
>>> Adolf.
>>>
>>>> -Michael
>>>>
>>>>> Regards,
>>>>> Adolf.
>>>>>> -Michael
>>>>>>> On 8 Aug 2021, at 14:47, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> I had another go at the ca-certificates problem, the last barrier to getting rid of python2.
>>>>>>>
>>>>>>> I found certdata2pem.py files from fedora and 2 from suse. I created build subdirectories for each version so I could just test running the build.sh file with each version of certdata2pem.py, including the IPFire current version after running through the 2to3 convertor.
>>>>>>>
>>>>>>> fedora
>>>>>>>
>>>>>>> The fedora certdata2pem.py file runs successfully with python3 but has sed commands built into it which fail to find certain files. The sed commands are not in the IPFire version.
>>>>>>>
>>>>>>> The error message is
>>>>>>>
>>>>>>>    -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', trust = ['CKA_TRUST_SERVER_AUTH', 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = ['serverAuth', 'emailProtection'], distrust = [], openssl-distrust = []
>>>>>>> sed: can't read certs/*.crt: No such file or directory
>>>>>>>
>>>>>>>
>>>>>>> suse
>>>>>>>
>>>>>>> The first suse version runs successfully with python3 but also has the sed commands in it with the same error message.
>>>>>>>
>>>>>>> The second suse version runs successfully with python3, does not have the sed commands and completes the build.sh script with no errors. However this certdata2pem.py file has a section that is in the IPFire version completely missing.
>>>>>>>
>>>>>>>
>>>>>>> IPfire version after running through the 2to3 convertor
>>>>>>>
>>>>>>> The following error message occurs
>>>>>>>
>>>>>>> producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148
>>>>>>> Traceback (most recent call last):
>>>>>>>     File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in <module>
>>>>>>> f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64)))
>>>>>>>     File "/usr/lib/python3.9/base64.py", line 58, in b64encode
>>>>>>>       encoded = binascii.b2a_base64(s, newline=False)
>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>
>>>>>>> The section that is failing is the section that is missing in the 2nd suse version. There is an identical fwrite line at line 206 but that does not seem to flag up the same TypeError message.
>>>>>>>
>>>>>>>
>>>>>>> As the certdata2pem.py files from the other distributions vary significantly in content, with some having nearly double the number of lines of code, I think the best alternative is to fix the IPFire version so we stay consistent but I am unable to figure out how to fix the python code that is causing the "  TypeError: a bytes-like object is required, not 'str' " error message and need someone's help with that.
>>>>>>>
>>>>>>> Let me know if there is any other information that I need to provide.
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Adolf.
>>>>>>>
>>>>>>>
>>>>>>> On 07/08/2021 15:54, Adolf Belka wrote:
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> On 04/08/2021 16:45, Michael Tremer wrote:
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi All,
>>>>>>>>>>
>>>>>>>>>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
>>>>>>>>>
>>>>>>>>> Great. This could also resolve Matthias’ problem with building frr.
>>>>>>>>>
>>>>>>>>>> Will now go back and have another go with spice-protocol.
>>>>>>>>>
>>>>>>>>> Maybe it has a —-disable-python switch?
>>>>>>>> I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
>>>>>>>> Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.
>>>>>>>>
>>>>>>>> This now only leaves the ca-certificates script that needs to be updated to work with python3.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Adolf.
>>>>>>>>>
>>>>>>>>> -Michael
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>>
>>>>>>>>>> Adolf.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>>>>>>>>> Hi Michael & all,
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>>>>>>>>> Hello,
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you for looking into this.
>>>>>>>>>>>>
>>>>>>>>>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>>>>>>>>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>>>>>>>>>
>>>>>>>>>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>>>>>>>>>> I then commented ca-certificates out in make.sh and ran the build.
>>>>>>>>>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>>>>>>>>>> py-compile is python2 based and the build stopped because it could not find python
>>>>>>>>>>>
>>>>>>>>>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>>>>>>>>>
>>>>>>>>>>> I then commented out spice-protocol and ran the build.
>>>>>>>>>>>
>>>>>>>>>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>>>>>>>>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>>>>>>>>>
>>>>>>>>>>> I commented out frr and nothing else failed before cdrom was reached.
>>>>>>>>>>>
>>>>>>>>>>> So the packages that need to be made to work with python3 are
>>>>>>>>>>> ca-certificates
>>>>>>>>>>> spice-protocol
>>>>>>>>>>> frr
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Adolf.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>>>>>>>>>
>>>>>>>>>>>> -Michael
>>>>>>>>>>>>
>>>>>>>>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>
>>>>>>>>>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> ca-certificates was flagged up.
>>>>>>>>>>>>>
>>>>>>>>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>>>>>>>>>
>>>>>>>>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>>>>>>>>>
>>>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Adolf.
> 

Re: First results from running build without python2

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 13278 bytes --]

Hi Michael,

On 12/08/2021 21:23, Adolf Belka wrote:
> Hi Michael,
>
> On 12/08/2021 15:38, Michael Tremer wrote:
>> Hello,
>>
>> Yes, this is the way to go :)
>>
>> Reach out here if you need any help.
> I am trying to build p11-kit and it has flagged up libtasn1. It is an optional dependency and I can disable it but I would like to check if that is the correct thing to do or not.
>
> libtasn1 is said to be required to allow the trust policy module to be built as well as other code that interacts with certificates.
> Is this something that would be needed for IPFire or can I disable it in the p11-kit build.
>
Ignore this question. I just did a quick test of disabling libtasn1 and p11-kit built ok but then my ca-certificates build failed due to the lack of the trust tool, so that makes it clear that it is required. I will add it in to the build.

Slowly getting there.

Regards,

Adolf.

> Regards,
> Adolf.
>>
>> -Michael
>>
>>> On 12 Aug 2021, at 13:10, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>
>>> Hi Michael,
>>>
>>> On 12/08/2021 13:36, Adolf Belka wrote:
>>>> Hi Michael,
>>>>
>>>> On 12/08/2021 11:17, Michael Tremer wrote:
>>>>> Hello,
>>>>>
>>>>>> On 11 Aug 2021, at 15:03, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>
>>>>>> Hi Michael,
>>>>>>
>>>>>> On 11/08/2021 12:43, Michael Tremer wrote:
>>>>>>> Hello,
>>>>>>> Is this the one with the broken sed command?
>>>>>>> https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2pem.py
>>>>>> Yes, this is that one. Confirmed with a diff.
>>>>>>> This should run if you execute it in the right directory:
>>>>>>> pushd %{name}/certs
>>>>>>>    pwd
>>>>>>>    cp certdata.txt .
>>>>>>>    python3 certdata2pem.py
>>>>>>> popd
>>>>>> I have just learnt about the pushd and popd commands. by doing a quick search. Never heard of them before.
>>>>>
>>>>> It is just a version of “cd” that remembers where it has been.
>>>>>
>>>>> So if you call “pushd some-directory”, then “popd” will bring you back to where you have been before.
>>>>>
>>>>> “cd -“ does the same as popd now.
>>>>>
>>>>>>> The fedora version no longer has the build.sh script.
>>>>>> That was the bit I didn't realise.
>>>>>
>>>>> No problem.
>>>>
>>>> Not as simple as I hoped.
>>>>
>>>> The new certdata2pem.py script no longer creates .crt files but .tmp-p11-kit files but that is as far as that script goes.
>>>>
>>>> There is an update-ca-trust file in fedora which splits the various certs to their respective locations, with the openssl ones ending up in ca-bundle.trust.crt but this uses p11-kit. I suspect that it uses ca-bundle.trust.p11-kit generated later on in the %build section of the ca-certificate.spec file from the tmp-p11-kit files generated by the certdata2pem.py script. So it looks like p11-kit needs to be installed to make this work.
>>>>
>>>> Alternatively I have found the following in LFS.
>>>> https://www.linuxfromscratch.org/blfs/view/svn/postlfs/make-ca.html
>>>> which seems to also create the bundled cert file but also requiring p11-kit but then talking about different certdata.txt versions that have various tunings. The one from LFS comes from the mozilla release branch but is modified to something called the Mercurial revision, or the different ones shipped by RedHat or OpenSUSE which use the version that comes with NSS.
>>>>
>>>> All in all I am not sure which approach to use or how to actually build it in IPFire.
>>>>
>>>> Definitely help required to know the correct way to go further with this.
>>>>
>>> Having said I needed help, I thought I would have another go and copied the lines from the %build section of the fedora ca-certificate.spec that created the ca-bundle.trust.p11.kit
>>>
>>> As I have p11-kit installed on my Arch Linux system I then ran the command
>>>
>>> p11-kit extract --format=openssl-bundle --filter=certificates --overwrite --comment ~/openssl/ca-bundle.trust.crt
>>>
>>> and I successfully created the ca-bundle.trust.crt which is needed by the IPFire ca-certificates lfs file.
>>>
>>> I will now try and create a new build script that will do all the new type stuff and also install the p11-kit library files and see how things go.
>>>
>>> :crossed_fingers:
>>>
>>> Adolf
>>>
>>>> Regards,
>>>> Adolf.
>>>>
>>>>> -Michael
>>>>>
>>>>>> Regards,
>>>>>> Adolf.
>>>>>>> -Michael
>>>>>>>> On 8 Aug 2021, at 14:47, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> I had another go at the ca-certificates problem, the last barrier to getting rid of python2.
>>>>>>>>
>>>>>>>> I found certdata2pem.py files from fedora and 2 from suse. I created build subdirectories for each version so I could just test running the build.sh file with each version of certdata2pem.py, including the IPFire current version after running through the 2to3 convertor.
>>>>>>>>
>>>>>>>> fedora
>>>>>>>>
>>>>>>>> The fedora certdata2pem.py file runs successfully with python3 but has sed commands built into it which fail to find certain files. The sed commands are not in the IPFire version.
>>>>>>>>
>>>>>>>> The error message is
>>>>>>>>
>>>>>>>>    -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', trust = ['CKA_TRUST_SERVER_AUTH', 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = ['serverAuth', 'emailProtection'], distrust = [], openssl-distrust = []
>>>>>>>> sed: can't read certs/*.crt: No such file or directory
>>>>>>>>
>>>>>>>>
>>>>>>>> suse
>>>>>>>>
>>>>>>>> The first suse version runs successfully with python3 but also has the sed commands in it with the same error message.
>>>>>>>>
>>>>>>>> The second suse version runs successfully with python3, does not have the sed commands and completes the build.sh script with no errors. However this certdata2pem.py file has a section that is in the IPFire version completely missing.
>>>>>>>>
>>>>>>>>
>>>>>>>> IPfire version after running through the 2to3 convertor
>>>>>>>>
>>>>>>>> The following error message occurs
>>>>>>>>
>>>>>>>> producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148
>>>>>>>> Traceback (most recent call last):
>>>>>>>>     File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in <module>
>>>>>>>> f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64)))
>>>>>>>>     File "/usr/lib/python3.9/base64.py", line 58, in b64encode
>>>>>>>>       encoded = binascii.b2a_base64(s, newline=False)
>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>
>>>>>>>> The section that is failing is the section that is missing in the 2nd suse version. There is an identical fwrite line at line 206 but that does not seem to flag up the same TypeError message.
>>>>>>>>
>>>>>>>>
>>>>>>>> As the certdata2pem.py files from the other distributions vary significantly in content, with some having nearly double the number of lines of code, I think the best alternative is to fix the IPFire version so we stay consistent but I am unable to figure out how to fix the python code that is causing the "  TypeError: a bytes-like object is required, not 'str' " error message and need someone's help with that.
>>>>>>>>
>>>>>>>> Let me know if there is any other information that I need to provide.
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Adolf.
>>>>>>>>
>>>>>>>>
>>>>>>>> On 07/08/2021 15:54, Adolf Belka wrote:
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> On 04/08/2021 16:45, Michael Tremer wrote:
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi All,
>>>>>>>>>>>
>>>>>>>>>>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
>>>>>>>>>>
>>>>>>>>>> Great. This could also resolve Matthias’ problem with building frr.
>>>>>>>>>>
>>>>>>>>>>> Will now go back and have another go with spice-protocol.
>>>>>>>>>>
>>>>>>>>>> Maybe it has a —-disable-python switch?
>>>>>>>>> I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
>>>>>>>>> Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.
>>>>>>>>>
>>>>>>>>> This now only leaves the ca-certificates script that needs to be updated to work with python3.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Adolf.
>>>>>>>>>>
>>>>>>>>>> -Michael
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>
>>>>>>>>>>> Adolf.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>>>>>>>>>> Hi Michael & all,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thank you for looking into this.
>>>>>>>>>>>>>
>>>>>>>>>>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>>>>>>>>>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>>>>>>>>>>
>>>>>>>>>>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>>>>>>>>>>> I then commented ca-certificates out in make.sh and ran the build.
>>>>>>>>>>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>>>>>>>>>>> py-compile is python2 based and the build stopped because it could not find python
>>>>>>>>>>>>
>>>>>>>>>>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>>>>>>>>>>
>>>>>>>>>>>> I then commented out spice-protocol and ran the build.
>>>>>>>>>>>>
>>>>>>>>>>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>>>>>>>>>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>>>>>>>>>>
>>>>>>>>>>>> I commented out frr and nothing else failed before cdrom was reached.
>>>>>>>>>>>>
>>>>>>>>>>>> So the packages that need to be made to work with python3 are
>>>>>>>>>>>> ca-certificates
>>>>>>>>>>>> spice-protocol
>>>>>>>>>>>> frr
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ca-certificates was flagged up.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Adolf.
>>

Re: First results from running build without python2

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 16197 bytes --]

Hi All,

I have successfully built ca-certificates with the new p11-kit approach. The only difference is that the certificate data such as Version, Serial Number, Signature Algorithm etc is no longer included in the created bundle files. There is only the actual certificate itself.

This is also the case for the bundles that Arch Linux has created on my systems, that also uses the p11-kit approach.

I did a diff on the current ca-bundle.crt/ca-bundle.trust.crt files and the ones created by the new approach as used by Fedora and all the same certificates are present, it is only the certificate data that is missed.

There is a --comment option in the trust extract command that looks like it should add the comments that are in the .p11-kit file into the extracted bundle files but nothing changes if it is used or not. I will ask into the p11-kit git repository whether this should work or not.


I did a build just with the new ca-certificate files into Core Update 160 and then installed the created iso into a vm on my testbed system.

Everything worked that I tested out but I am not 100% sure what needs to be exercised to test the ca-bundle.crt and ca-bundle.trust.crt files. If someone can give me some guidance on this then I can do that final test and submit a patch for this change.


I also then did a test build with all python2 programs removed, including python2 itself and with the python 3 fixes for spice, frr and ca-certificates installed. The build for every program in the ipfire build section completed successfully but then I had a stop in the cdrom stage for collectd. It has a python2 plugin module defined which was of course missing. The module could be disabled in the rootfile or in the build config if the python plugin is not required for IPFire - I could not find any python modules defined.

I have been able to successfully define the path to the python interpreter in the ./configure statement to point to the python3 program and collectd successfully built with this so if the python plugin is required then a python3 version can be created. I have not yet tested this in my vm but will do so.

I also found that freeradius has a python2 module defined. This can be disabled. There is a python3 module available since 2.0.20. To use it with 3.0.21 (current version) you have to have --with-experimental-modules defined. I am looking at latest version 2.0.23 to see if that also needs the experimental modules enabled or if it has become part of the normal modules now and will test the build to see if it works successfully or not.

Regards,

Adolf.

On 12/08/2021 21:30, Adolf Belka wrote:
> Hi Michael,
>
> On 12/08/2021 21:23, Adolf Belka wrote:
>> Hi Michael,
>>
>> On 12/08/2021 15:38, Michael Tremer wrote:
>>> Hello,
>>>
>>> Yes, this is the way to go :)
>>>
>>> Reach out here if you need any help.
>> I am trying to build p11-kit and it has flagged up libtasn1. It is an optional dependency and I can disable it but I would like to check if that is the correct thing to do or not.
>>
>> libtasn1 is said to be required to allow the trust policy module to be built as well as other code that interacts with certificates.
>> Is this something that would be needed for IPFire or can I disable it in the p11-kit build.
>>
> Ignore this question. I just did a quick test of disabling libtasn1 and p11-kit built ok but then my ca-certificates build failed due to the lack of the trust tool, so that makes it clear that it is required. I will add it in to the build.
>
> Slowly getting there.
>
> Regards,
>
> Adolf.
>
>> Regards,
>> Adolf.
>>>
>>> -Michael
>>>
>>>> On 12 Aug 2021, at 13:10, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>
>>>> Hi Michael,
>>>>
>>>> On 12/08/2021 13:36, Adolf Belka wrote:
>>>>> Hi Michael,
>>>>>
>>>>> On 12/08/2021 11:17, Michael Tremer wrote:
>>>>>> Hello,
>>>>>>
>>>>>>> On 11 Aug 2021, at 15:03, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>
>>>>>>> Hi Michael,
>>>>>>>
>>>>>>> On 11/08/2021 12:43, Michael Tremer wrote:
>>>>>>>> Hello,
>>>>>>>> Is this the one with the broken sed command?
>>>>>>>> https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2pem.py
>>>>>>> Yes, this is that one. Confirmed with a diff.
>>>>>>>> This should run if you execute it in the right directory:
>>>>>>>> pushd %{name}/certs
>>>>>>>>    pwd
>>>>>>>>    cp certdata.txt .
>>>>>>>>    python3 certdata2pem.py
>>>>>>>> popd
>>>>>>> I have just learnt about the pushd and popd commands. by doing a quick search. Never heard of them before.
>>>>>>
>>>>>> It is just a version of “cd” that remembers where it has been.
>>>>>>
>>>>>> So if you call “pushd some-directory”, then “popd” will bring you back to where you have been before.
>>>>>>
>>>>>> “cd -“ does the same as popd now.
>>>>>>
>>>>>>>> The fedora version no longer has the build.sh script.
>>>>>>> That was the bit I didn't realise.
>>>>>>
>>>>>> No problem.
>>>>>
>>>>> Not as simple as I hoped.
>>>>>
>>>>> The new certdata2pem.py script no longer creates .crt files but .tmp-p11-kit files but that is as far as that script goes.
>>>>>
>>>>> There is an update-ca-trust file in fedora which splits the various certs to their respective locations, with the openssl ones ending up in ca-bundle.trust.crt but this uses p11-kit. I suspect that it uses ca-bundle.trust.p11-kit generated later on in the %build section of the ca-certificate.spec file from the tmp-p11-kit files generated by the certdata2pem.py script. So it looks like p11-kit needs to be installed to make this work.
>>>>>
>>>>> Alternatively I have found the following in LFS.
>>>>> https://www.linuxfromscratch.org/blfs/view/svn/postlfs/make-ca.html
>>>>> which seems to also create the bundled cert file but also requiring p11-kit but then talking about different certdata.txt versions that have various tunings. The one from LFS comes from the mozilla release branch but is modified to something called the Mercurial revision, or the different ones shipped by RedHat or OpenSUSE which use the version that comes with NSS.
>>>>>
>>>>> All in all I am not sure which approach to use or how to actually build it in IPFire.
>>>>>
>>>>> Definitely help required to know the correct way to go further with this.
>>>>>
>>>> Having said I needed help, I thought I would have another go and copied the lines from the %build section of the fedora ca-certificate.spec that created the ca-bundle.trust.p11.kit
>>>>
>>>> As I have p11-kit installed on my Arch Linux system I then ran the command
>>>>
>>>> p11-kit extract --format=openssl-bundle --filter=certificates --overwrite --comment ~/openssl/ca-bundle.trust.crt
>>>>
>>>> and I successfully created the ca-bundle.trust.crt which is needed by the IPFire ca-certificates lfs file.
>>>>
>>>> I will now try and create a new build script that will do all the new type stuff and also install the p11-kit library files and see how things go.
>>>>
>>>> :crossed_fingers:
>>>>
>>>> Adolf
>>>>
>>>>> Regards,
>>>>> Adolf.
>>>>>
>>>>>> -Michael
>>>>>>
>>>>>>> Regards,
>>>>>>> Adolf.
>>>>>>>> -Michael
>>>>>>>>> On 8 Aug 2021, at 14:47, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> I had another go at the ca-certificates problem, the last barrier to getting rid of python2.
>>>>>>>>>
>>>>>>>>> I found certdata2pem.py files from fedora and 2 from suse. I created build subdirectories for each version so I could just test running the build.sh file with each version of certdata2pem.py, including the IPFire current version after running through the 2to3 convertor.
>>>>>>>>>
>>>>>>>>> fedora
>>>>>>>>>
>>>>>>>>> The fedora certdata2pem.py file runs successfully with python3 but has sed commands built into it which fail to find certain files. The sed commands are not in the IPFire version.
>>>>>>>>>
>>>>>>>>> The error message is
>>>>>>>>>
>>>>>>>>>    -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', trust = ['CKA_TRUST_SERVER_AUTH', 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = ['serverAuth', 'emailProtection'], distrust = [], openssl-distrust = []
>>>>>>>>> sed: can't read certs/*.crt: No such file or directory
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> suse
>>>>>>>>>
>>>>>>>>> The first suse version runs successfully with python3 but also has the sed commands in it with the same error message.
>>>>>>>>>
>>>>>>>>> The second suse version runs successfully with python3, does not have the sed commands and completes the build.sh script with no errors. However this certdata2pem.py file has a section that is in the IPFire version completely missing.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> IPfire version after running through the 2to3 convertor
>>>>>>>>>
>>>>>>>>> The following error message occurs
>>>>>>>>>
>>>>>>>>> producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>     File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in <module>
>>>>>>>>> f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64)))
>>>>>>>>>     File "/usr/lib/python3.9/base64.py", line 58, in b64encode
>>>>>>>>>       encoded = binascii.b2a_base64(s, newline=False)
>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>
>>>>>>>>> The section that is failing is the section that is missing in the 2nd suse version. There is an identical fwrite line at line 206 but that does not seem to flag up the same TypeError message.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> As the certdata2pem.py files from the other distributions vary significantly in content, with some having nearly double the number of lines of code, I think the best alternative is to fix the IPFire version so we stay consistent but I am unable to figure out how to fix the python code that is causing the "  TypeError: a bytes-like object is required, not 'str' " error message and need someone's help with that.
>>>>>>>>>
>>>>>>>>> Let me know if there is any other information that I need to provide.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> Adolf.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 07/08/2021 15:54, Adolf Belka wrote:
>>>>>>>>>> Hi All,
>>>>>>>>>>
>>>>>>>>>> On 04/08/2021 16:45, Michael Tremer wrote:
>>>>>>>>>>> Hello,
>>>>>>>>>>>
>>>>>>>>>>>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>
>>>>>>>>>>>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
>>>>>>>>>>>
>>>>>>>>>>> Great. This could also resolve Matthias’ problem with building frr.
>>>>>>>>>>>
>>>>>>>>>>>> Will now go back and have another go with spice-protocol.
>>>>>>>>>>>
>>>>>>>>>>> Maybe it has a —-disable-python switch?
>>>>>>>>>> I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
>>>>>>>>>> Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.
>>>>>>>>>>
>>>>>>>>>> This now only leaves the ca-certificates script that needs to be updated to work with python3.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Adolf.
>>>>>>>>>>>
>>>>>>>>>>> -Michael
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>>
>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>>>>>>>>>>> Hi Michael & all,
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thank you for looking into this.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>>>>>>>>>>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>>>>>>>>>>>> I then commented ca-certificates out in make.sh and ran the build.
>>>>>>>>>>>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>>>>>>>>>>>> py-compile is python2 based and the build stopped because it could not find python
>>>>>>>>>>>>>
>>>>>>>>>>>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I then commented out spice-protocol and ran the build.
>>>>>>>>>>>>>
>>>>>>>>>>>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>>>>>>>>>>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>>>>>>>>>>>
>>>>>>>>>>>>> I commented out frr and nothing else failed before cdrom was reached.
>>>>>>>>>>>>>
>>>>>>>>>>>>> So the packages that need to be made to work with python3 are
>>>>>>>>>>>>> ca-certificates
>>>>>>>>>>>>> spice-protocol
>>>>>>>>>>>>> frr
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ca-certificates was flagged up.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Adolf.
>>>

Re: First results from running build without python2

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 17366 bytes --]

Hello Adolf,

This sounds very good.

> On 18 Aug 2021, at 22:08, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> 
> Hi All,
> 
> I have successfully built ca-certificates with the new p11-kit approach. The only difference is that the certificate data such as Version, Serial Number, Signature Algorithm etc is no longer included in the created bundle files. There is only the actual certificate itself.

I do not exactly understand what you mean. Can you post a file system listing or something that would help me?

> This is also the case for the bundles that Arch Linux has created on my systems, that also uses the p11-kit approach.
> 
> I did a diff on the current ca-bundle.crt/ca-bundle.trust.crt files and the ones created by the new approach as used by Fedora and all the same certificates are present, it is only the certificate data that is missed.
> 
> There is a --comment option in the trust extract command that looks like it should add the comments that are in the .p11-kit file into the extracted bundle files but nothing changes if it is used or not. I will ask into the p11-kit git repository whether this should work or not.
> 
> 
> I did a build just with the new ca-certificate files into Core Update 160 and then installed the created iso into a vm on my testbed system.
> 
> Everything worked that I tested out but I am not 100% sure what needs to be exercised to test the ca-bundle.crt and ca-bundle.trust.crt files. If someone can give me some guidance on this then I can do that final test and submit a patch for this change.

If you can download something with wget using HTTPS and the certificate can be validated, things should be working fine.

> I also then did a test build with all python2 programs removed, including python2 itself and with the python 3 fixes for spice, frr and ca-certificates installed. The build for every program in the ipfire build section completed successfully but then I had a stop in the cdrom stage for collectd. It has a python2 plugin module defined which was of course missing. The module could be disabled in the rootfile or in the build config if the python plugin is not required for IPFire - I could not find any python modules defined.

You can just pass “—-without-python” or “—-disable-python” and then the module won’t be built any more.

> I have been able to successfully define the path to the python interpreter in the ./configure statement to point to the python3 program and collectd successfully built with this so if the python plugin is required then a python3 version can be created. I have not yet tested this in my vm but will do so.

We do not use this module, so I suppose it isn’t worth building it in the first place.

> I also found that freeradius has a python2 module defined. This can be disabled.

Please do that. If someone needs the Python 3 module, we can still build it.

> There is a python3 module available since 2.0.20. To use it with 3.0.21 (current version) you have to have --with-experimental-modules defined. I am looking at latest version 2.0.23 to see if that also needs the experimental modules enabled or if it has become part of the normal modules now and will test the build to see if it works successfully or not.

Thank you for all the research that you invested into this. Looks like we are not far away to drop Python 2.

-Michael

> 
> Regards,
> 
> Adolf.
> 
> On 12/08/2021 21:30, Adolf Belka wrote:
>> Hi Michael,
>> 
>> On 12/08/2021 21:23, Adolf Belka wrote:
>>> Hi Michael,
>>> 
>>> On 12/08/2021 15:38, Michael Tremer wrote:
>>>> Hello,
>>>> 
>>>> Yes, this is the way to go :)
>>>> 
>>>> Reach out here if you need any help.
>>> I am trying to build p11-kit and it has flagged up libtasn1. It is an optional dependency and I can disable it but I would like to check if that is the correct thing to do or not.
>>> 
>>> libtasn1 is said to be required to allow the trust policy module to be built as well as other code that interacts with certificates.
>>> Is this something that would be needed for IPFire or can I disable it in the p11-kit build.
>>> 
>> Ignore this question. I just did a quick test of disabling libtasn1 and p11-kit built ok but then my ca-certificates build failed due to the lack of the trust tool, so that makes it clear that it is required. I will add it in to the build.
>> 
>> Slowly getting there.
>> 
>> Regards,
>> 
>> Adolf.
>> 
>>> Regards,
>>> Adolf.
>>>> 
>>>> -Michael
>>>> 
>>>>> On 12 Aug 2021, at 13:10, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>> 
>>>>> Hi Michael,
>>>>> 
>>>>> On 12/08/2021 13:36, Adolf Belka wrote:
>>>>>> Hi Michael,
>>>>>> 
>>>>>> On 12/08/2021 11:17, Michael Tremer wrote:
>>>>>>> Hello,
>>>>>>> 
>>>>>>>> On 11 Aug 2021, at 15:03, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>> 
>>>>>>>> Hi Michael,
>>>>>>>> 
>>>>>>>> On 11/08/2021 12:43, Michael Tremer wrote:
>>>>>>>>> Hello,
>>>>>>>>> Is this the one with the broken sed command?
>>>>>>>>> https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2pem.py
>>>>>>>> Yes, this is that one. Confirmed with a diff.
>>>>>>>>> This should run if you execute it in the right directory:
>>>>>>>>> pushd %{name}/certs
>>>>>>>>>    pwd
>>>>>>>>>    cp certdata.txt .
>>>>>>>>>    python3 certdata2pem.py
>>>>>>>>> popd
>>>>>>>> I have just learnt about the pushd and popd commands. by doing a quick search. Never heard of them before.
>>>>>>> 
>>>>>>> It is just a version of “cd” that remembers where it has been.
>>>>>>> 
>>>>>>> So if you call “pushd some-directory”, then “popd” will bring you back to where you have been before.
>>>>>>> 
>>>>>>> “cd -“ does the same as popd now.
>>>>>>> 
>>>>>>>>> The fedora version no longer has the build.sh script.
>>>>>>>> That was the bit I didn't realise.
>>>>>>> 
>>>>>>> No problem.
>>>>>> 
>>>>>> Not as simple as I hoped.
>>>>>> 
>>>>>> The new certdata2pem.py script no longer creates .crt files but .tmp-p11-kit files but that is as far as that script goes.
>>>>>> 
>>>>>> There is an update-ca-trust file in fedora which splits the various certs to their respective locations, with the openssl ones ending up in ca-bundle.trust.crt but this uses p11-kit. I suspect that it uses ca-bundle.trust.p11-kit generated later on in the %build section of the ca-certificate.spec file from the tmp-p11-kit files generated by the certdata2pem.py script. So it looks like p11-kit needs to be installed to make this work.
>>>>>> 
>>>>>> Alternatively I have found the following in LFS.
>>>>>> https://www.linuxfromscratch.org/blfs/view/svn/postlfs/make-ca.html
>>>>>> which seems to also create the bundled cert file but also requiring p11-kit but then talking about different certdata.txt versions that have various tunings. The one from LFS comes from the mozilla release branch but is modified to something called the Mercurial revision, or the different ones shipped by RedHat or OpenSUSE which use the version that comes with NSS.
>>>>>> 
>>>>>> All in all I am not sure which approach to use or how to actually build it in IPFire.
>>>>>> 
>>>>>> Definitely help required to know the correct way to go further with this.
>>>>>> 
>>>>> Having said I needed help, I thought I would have another go and copied the lines from the %build section of the fedora ca-certificate.spec that created the ca-bundle.trust.p11.kit
>>>>> 
>>>>> As I have p11-kit installed on my Arch Linux system I then ran the command
>>>>> 
>>>>> p11-kit extract --format=openssl-bundle --filter=certificates --overwrite --comment ~/openssl/ca-bundle.trust.crt
>>>>> 
>>>>> and I successfully created the ca-bundle.trust.crt which is needed by the IPFire ca-certificates lfs file.
>>>>> 
>>>>> I will now try and create a new build script that will do all the new type stuff and also install the p11-kit library files and see how things go.
>>>>> 
>>>>> :crossed_fingers:
>>>>> 
>>>>> Adolf
>>>>> 
>>>>>> Regards,
>>>>>> Adolf.
>>>>>> 
>>>>>>> -Michael
>>>>>>> 
>>>>>>>> Regards,
>>>>>>>> Adolf.
>>>>>>>>> -Michael
>>>>>>>>>> On 8 Aug 2021, at 14:47, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>> 
>>>>>>>>>> Hi All,
>>>>>>>>>> 
>>>>>>>>>> I had another go at the ca-certificates problem, the last barrier to getting rid of python2.
>>>>>>>>>> 
>>>>>>>>>> I found certdata2pem.py files from fedora and 2 from suse. I created build subdirectories for each version so I could just test running the build.sh file with each version of certdata2pem.py, including the IPFire current version after running through the 2to3 convertor.
>>>>>>>>>> 
>>>>>>>>>> fedora
>>>>>>>>>> 
>>>>>>>>>> The fedora certdata2pem.py file runs successfully with python3 but has sed commands built into it which fail to find certain files. The sed commands are not in the IPFire version.
>>>>>>>>>> 
>>>>>>>>>> The error message is
>>>>>>>>>> 
>>>>>>>>>>    -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', trust = ['CKA_TRUST_SERVER_AUTH', 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = ['serverAuth', 'emailProtection'], distrust = [], openssl-distrust = []
>>>>>>>>>> sed: can't read certs/*.crt: No such file or directory
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> suse
>>>>>>>>>> 
>>>>>>>>>> The first suse version runs successfully with python3 but also has the sed commands in it with the same error message.
>>>>>>>>>> 
>>>>>>>>>> The second suse version runs successfully with python3, does not have the sed commands and completes the build.sh script with no errors. However this certdata2pem.py file has a section that is in the IPFire version completely missing.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> IPfire version after running through the 2to3 convertor
>>>>>>>>>> 
>>>>>>>>>> The following error message occurs
>>>>>>>>>> 
>>>>>>>>>> producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148
>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>     File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in <module>
>>>>>>>>>> f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64)))
>>>>>>>>>>     File "/usr/lib/python3.9/base64.py", line 58, in b64encode
>>>>>>>>>>       encoded = binascii.b2a_base64(s, newline=False)
>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>> 
>>>>>>>>>> The section that is failing is the section that is missing in the 2nd suse version. There is an identical fwrite line at line 206 but that does not seem to flag up the same TypeError message.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> As the certdata2pem.py files from the other distributions vary significantly in content, with some having nearly double the number of lines of code, I think the best alternative is to fix the IPFire version so we stay consistent but I am unable to figure out how to fix the python code that is causing the "  TypeError: a bytes-like object is required, not 'str' " error message and need someone's help with that.
>>>>>>>>>> 
>>>>>>>>>> Let me know if there is any other information that I need to provide.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Regards,
>>>>>>>>>> 
>>>>>>>>>> Adolf.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On 07/08/2021 15:54, Adolf Belka wrote:
>>>>>>>>>>> Hi All,
>>>>>>>>>>> 
>>>>>>>>>>> On 04/08/2021 16:45, Michael Tremer wrote:
>>>>>>>>>>>> Hello,
>>>>>>>>>>>> 
>>>>>>>>>>>>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
>>>>>>>>>>>> 
>>>>>>>>>>>> Great. This could also resolve Matthias’ problem with building frr.
>>>>>>>>>>>> 
>>>>>>>>>>>>> Will now go back and have another go with spice-protocol.
>>>>>>>>>>>> 
>>>>>>>>>>>> Maybe it has a —-disable-python switch?
>>>>>>>>>>> I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
>>>>>>>>>>> Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.
>>>>>>>>>>> 
>>>>>>>>>>> This now only leaves the ca-certificates script that needs to be updated to work with python3.
>>>>>>>>>>> 
>>>>>>>>>>> Regards,
>>>>>>>>>>> Adolf.
>>>>>>>>>>>> 
>>>>>>>>>>>> -Michael
>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>>>>>>>>>>>> Hi Michael & all,
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thank you for looking into this.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>>>>>>>>>>>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>>>>>>>>>>>>> I then commented ca-certificates out in make.sh and ran the build.
>>>>>>>>>>>>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>>>>>>>>>>>>> py-compile is python2 based and the build stopped because it could not find python
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I then commented out spice-protocol and ran the build.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>>>>>>>>>>>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I commented out frr and nothing else failed before cdrom was reached.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> So the packages that need to be made to work with python3 are
>>>>>>>>>>>>>> ca-certificates
>>>>>>>>>>>>>> spice-protocol
>>>>>>>>>>>>>> frr
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> ca-certificates was flagged up.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Adolf.
>>>> 

Re: First results from running build without python2

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 29884 bytes --]

Hi Michael,

On 19/08/2021 15:54, Michael Tremer wrote:
> Hello Adolf,
> 
> This sounds very good.
> 
>> On 18 Aug 2021, at 22:08, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>
>> Hi All,
>>
>> I have successfully built ca-certificates with the new p11-kit approach. The only difference is that the certificate data such as Version, Serial Number, Signature Algorithm etc is no longer included in the created bundle files. There is only the actual certificate itself.
> 
> I do not exactly understand what you mean. Can you post a file system listing or something that would help me?The first certificate in the new ca-bundle.crt looks like the following:-

# ACCVRAIZ1
-----BEGIN CERTIFICATE-----
MIIH0zCCBbugAwIBAgIIXsO3pkN/pOAwDQYJKoZIhvcNAQEFBQAwQjESMBAGA1UE
AwwJQUNDVlJBSVoxMRAwDgYDVQQLDAdQS0lBQ0NWMQ0wCwYDVQQKDARBQ0NWMQsw
CQYDVQQGEwJFUzAeFw0xMTA1MDUwOTM3MzdaFw0zMDEyMzEwOTM3MzdaMEIxEjAQ
BgNVBAMMCUFDQ1ZSQUlaMTEQMA4GA1UECwwHUEtJQUNDVjENMAsGA1UECgwEQUND
VjELMAkGA1UEBhMCRVMwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCb
qau/YUqXry+XZpp0X9DZlv3P4uRm7x8fRzPCRKPfmt4ftVTdFXxpNRFvu8gMjmoY
HtiP2Ra8EEg2XPBjs5BaXCQ316PWywlxufEBcoSwfdtNgM3802/J+Nq2DoLSRYWo
G2ioPej0RGy9ocLLA76MPhMAhN9KSMDjIgro6TenGEyxCQ0jVn8ETdkXhBilyNpA
lHPrzg5XPAOBOp0KoVdDaaxXbXmQeOW1tDvYvEyNKKGno6e6Ak4l0Squ7a4DIrhr
IA8wKFSVf+DuzgpmndFALW4ir50awQUZ0m/A8p/4e7MCQvtQqR0tkw8jq8bBD5L/
0KIV9VMJcRz/RROE5iZe+OCIHAr8Fraocwa48GOEAqDGWuzndN9wrqODJerWx5eH
k6fGioozl2A3ED6XPm4pFdahD9GILBKfb6qkxkLrQaLjlUPTAYVtjrs78yM2x/47
4KElB0iryYl0/wiPgL/AlmXz7uxLaL2diMMxs0Dx6M/2OLuc5NF/1OVYm3z61PMO
m3WR5LpSLhl+0fXNWhn8ugb2+1KoS5kE3fj5tItQo05iifCHJPqDQsGH+tUtKSpa
cXpkatcnYGMN285J9Y0fkIkyF/hzQ7jSWpOGYdbhdQrqeWZ2iE9x6wQl1gpaepPl
uUsXQA+xtrn13k/c4LOsOxFwYIRKQ26ZIMApcQrAZQIDAQABo4ICyzCCAscwfQYI
KwYBBQUHAQEEcTBvMEwGCCsGAQUFBzAChkBodHRwOi8vd3d3LmFjY3YuZXMvZmls
ZWFkbWluL0FyY2hpdm9zL2NlcnRpZmljYWRvcy9yYWl6YWNjdjEuY3J0MB8GCCsG
AQUFBzABhhNodHRwOi8vb2NzcC5hY2N2LmVzMB0GA1UdDgQWBBTSh7Tj3zcnk1X2
VuqB5TbMjB4/vTAPBgNVHRMBAf8EBTADAQH/MB8GA1UdIwQYMBaAFNKHtOPfNyeT
VfZW6oHlNsyMHj+9MIIBcwYDVR0gBIIBajCCAWYwggFiBgRVHSAAMIIBWDCCASIG
CCsGAQUFBwICMIIBFB6CARAAQQB1AHQAbwByAGkAZABhAGQAIABkAGUAIABDAGUA
cgB0AGkAZgBpAGMAYQBjAGkA8wBuACAAUgBhAO0AegAgAGQAZQAgAGwAYQAgAEEA
QwBDAFYAIAAoAEEAZwBlAG4AYwBpAGEAIABkAGUAIABUAGUAYwBuAG8AbABvAGcA
7QBhACAAeQAgAEMAZQByAHQAaQBmAGkAYwBhAGMAaQDzAG4AIABFAGwAZQBjAHQA
cgDzAG4AaQBjAGEALAAgAEMASQBGACAAUQA0ADYAMAAxADEANQA2AEUAKQAuACAA
QwBQAFMAIABlAG4AIABoAHQAdABwADoALwAvAHcAdwB3AC4AYQBjAGMAdgAuAGUA
czAwBggrBgEFBQcCARYkaHR0cDovL3d3dy5hY2N2LmVzL2xlZ2lzbGFjaW9uX2Mu
aHRtMFUGA1UdHwROMEwwSqBIoEaGRGh0dHA6Ly93d3cuYWNjdi5lcy9maWxlYWRt
aW4vQXJjaGl2b3MvY2VydGlmaWNhZG9zL3JhaXphY2N2MV9kZXIuY3JsMA4GA1Ud
DwEB/wQEAwIBBjAXBgNVHREEEDAOgQxhY2N2QGFjY3YuZXMwDQYJKoZIhvcNAQEF
BQADggIBAJcxAp/n/UNnSEQU5CmH7UwoZtCPNdpNYbdKl02125DgBS4OxnnQ8pdp
D70ER9m+27Up2pvZrqmZ1dM8MJP1jaGo/AaNRPTKFpV8M9xii6g3+CfYCS0b78gU
JyCpZET/LtZ1qmxNYEAZSUNUY9rizLpm5U9EelvZaoErQNV/+QEnWCzI7UiRfD+m
AM/EKXMRNt6GGT6d7hmKG9Ww7Y49nCrADdg9ZuM8Db3VlFzi4qc1GwQA9j9ajepD
vV+JHanBsMyZ4k0ACtrJJ1vnE5Bc5PUzolVt3OAJTS+xJlsndQAJxGJ3KQhfnlms
tn6tn1QwIgPBHnFk/vk4CpYY3QIUrCPLBhwepH2NDd4nQeit2hW3sCPdK6jT2iWH
7ehVRE2I9DZ+hJp4rPcOVkkO1jMl1oRQQmwgEh0q1b688nCBpHBgvgW1m54ERL5h
I6zppSSMEYCUWqKiuUnSwdzRp+0xESyeGabu4VXhwOrPDYTkF7eifKXeVSUG7szA
h1xA2syVP1XgNce4hL60Xc16gwFy7ofmXx2utYXGJt/mwZrpHgJHnyqobalbz+xF
d3+YJ5oyXSrjhO7FmGYvliAd3djDJ9ew+f7Zfc3Qn48LFFhRny+Lwzgt3uiP1o2H
pPVWQxaZLPSkVrQ0uGE3ycJYgBugl6H8WY3pEfbRD0tVNEYqi4Y7
-----END CERTIFICATE-----

The version currently in place has the following content for the first certificate:-

Certificate:
     Data:
         Version: 3 (0x2)
         Serial Number: 6828503384748696800 (0x5ec3b7a6437fa4e0)
         Signature Algorithm: sha1WithRSAEncryption
         Issuer: CN = ACCVRAIZ1, OU = PKIACCV, O = ACCV, C = ES
         Validity
             Not Before: May  5 09:37:37 2011 GMT
             Not After : Dec 31 09:37:37 2030 GMT
         Subject: CN = ACCVRAIZ1, OU = PKIACCV, O = ACCV, C = ES
         Subject Public Key Info:
             Public Key Algorithm: rsaEncryption
                 RSA Public-Key: (4096 bit)
                 Modulus:
                     00:9b:a9:ab:bf:61:4a:97:af:2f:97:66:9a:74:5f:
                     d0:d9:96:fd:cf:e2:e4:66:ef:1f:1f:47:33:c2:44:
                     a3:df:9a:de:1f:b5:54:dd:15:7c:69:35:11:6f:bb:
                     c8:0c:8e:6a:18:1e:d8:8f:d9:16:bc:10:48:36:5c:
                     f0:63:b3:90:5a:5c:24:37:d7:a3:d6:cb:09:71:b9:
                     f1:01:72:84:b0:7d:db:4d:80:cd:fc:d3:6f:c9:f8:
                     da:b6:0e:82:d2:45:85:a8:1b:68:a8:3d:e8:f4:44:
                     6c:bd:a1:c2:cb:03:be:8c:3e:13:00:84:df:4a:48:
                     c0:e3:22:0a:e8:e9:37:a7:18:4c:b1:09:0d:23:56:
                     7f:04:4d:d9:17:84:18:a5:c8:da:40:94:73:eb:ce:
                     0e:57:3c:03:81:3a:9d:0a:a1:57:43:69:ac:57:6d:
                     79:90:78:e5:b5:b4:3b:d8:bc:4c:8d:28:a1:a7:a3:
                     a7:ba:02:4e:25:d1:2a:ae:ed:ae:03:22:b8:6b:20:
                     0f:30:28:54:95:7f:e0:ee:ce:0a:66:9d:d1:40:2d:
                     6e:22:af:9d:1a:c1:05:19:d2:6f:c0:f2:9f:f8:7b:
                     b3:02:42:fb:50:a9:1d:2d:93:0f:23:ab:c6:c1:0f:
                     92:ff:d0:a2:15:f5:53:09:71:1c:ff:45:13:84:e6:
                     26:5e:f8:e0:88:1c:0a:fc:16:b6:a8:73:06:b8:f0:
                     63:84:02:a0:c6:5a:ec:e7:74:df:70:ae:a3:83:25:
                     ea:d6:c7:97:87:93:a7:c6:8a:8a:33:97:60:37:10:
                     3e:97:3e:6e:29:15:d6:a1:0f:d1:88:2c:12:9f:6f:
                     aa:a4:c6:42:eb:41:a2:e3:95:43:d3:01:85:6d:8e:
                     bb:3b:f3:23:36:c7:fe:3b:e0:a1:25:07:48:ab:c9:
                     89:74:ff:08:8f:80:bf:c0:96:65:f3:ee:ec:4b:68:
                     bd:9d:88:c3:31:b3:40:f1:e8:cf:f6:38:bb:9c:e4:
                     d1:7f:d4:e5:58:9b:7c:fa:d4:f3:0e:9b:75:91:e4:
                     ba:52:2e:19:7e:d1:f5:cd:5a:19:fc:ba:06:f6:fb:
                     52:a8:4b:99:04:dd:f8:f9:b4:8b:50:a3:4e:62:89:
                     f0:87:24:fa:83:42:c1:87:fa:d5:2d:29:2a:5a:71:
                     7a:64:6a:d7:27:60:63:0d:db:ce:49:f5:8d:1f:90:
                     89:32:17:f8:73:43:b8:d2:5a:93:86:61:d6:e1:75:
                     0a:ea:79:66:76:88:4f:71:eb:04:25:d6:0a:5a:7a:
                     93:e5:b9:4b:17:40:0f:b1:b6:b9:f5:de:4f:dc:e0:
                     b3:ac:3b:11:70:60:84:4a:43:6e:99:20:c0:29:71:
                     0a:c0:65
                 Exponent: 65537 (0x10001)
         X509v3 extensions:
             Authority Information Access:
                 CA Issuers - URI:http://www.accv.es/fileadmin/Archivos/certificados/raizaccv1.crt
                 OCSP - URI:http://ocsp.accv.es

             X509v3 Subject Key Identifier:
                 D2:87:B4:E3:DF:37:27:93:55:F6:56:EA:81:E5:36:CC:8C:1E:3F:BD
             X509v3 Basic Constraints: critical
                 CA:TRUE
             X509v3 Authority Key Identifier:
                 keyid:D2:87:B4:E3:DF:37:27:93:55:F6:56:EA:81:E5:36:CC:8C:1E:3F:BD

             X509v3 Certificate Policies:
                 Policy: X509v3 Any Policy
                   User Notice:
                     Explicit Text:
                   CPS: http://www.accv.es/legislacion_c.htm

             X509v3 CRL Distribution Points:

                 Full Name:
                   URI:http://www.accv.es/fileadmin/Archivos/certificados/raizaccv1_der.crl

             X509v3 Key Usage: critical
                 Certificate Sign, CRL Sign
             X509v3 Subject Alternative Name:
                 email:accv(a)accv.es
     Signature Algorithm: sha1WithRSAEncryption
          97:31:02:9f:e7:fd:43:67:48:44:14:e4:29:87:ed:4c:28:66:
          d0:8f:35:da:4d:61:b7:4a:97:4d:b5:db:90:e0:05:2e:0e:c6:
          79:d0:f2:97:69:0f:bd:04:47:d9:be:db:b5:29:da:9b:d9:ae:
          a9:99:d5:d3:3c:30:93:f5:8d:a1:a8:fc:06:8d:44:f4:ca:16:
          95:7c:33:dc:62:8b:a8:37:f8:27:d8:09:2d:1b:ef:c8:14:27:
          20:a9:64:44:ff:2e:d6:75:aa:6c:4d:60:40:19:49:43:54:63:
          da:e2:cc:ba:66:e5:4f:44:7a:5b:d9:6a:81:2b:40:d5:7f:f9:
          01:27:58:2c:c8:ed:48:91:7c:3f:a6:00:cf:c4:29:73:11:36:
          de:86:19:3e:9d:ee:19:8a:1b:d5:b0:ed:8e:3d:9c:2a:c0:0d:
          d8:3d:66:e3:3c:0d:bd:d5:94:5c:e2:e2:a7:35:1b:04:00:f6:
          3f:5a:8d:ea:43:bd:5f:89:1d:a9:c1:b0:cc:99:e2:4d:00:0a:
          da:c9:27:5b:e7:13:90:5c:e4:f5:33:a2:55:6d:dc:e0:09:4d:
          2f:b1:26:5b:27:75:00:09:c4:62:77:29:08:5f:9e:59:ac:b6:
          7e:ad:9f:54:30:22:03:c1:1e:71:64:fe:f9:38:0a:96:18:dd:
          02:14:ac:23:cb:06:1c:1e:a4:7d:8d:0d:de:27:41:e8:ad:da:
          15:b7:b0:23:dd:2b:a8:d3:da:25:87:ed:e8:55:44:4d:88:f4:
          36:7e:84:9a:78:ac:f7:0e:56:49:0e:d6:33:25:d6:84:50:42:
          6c:20:12:1d:2a:d5:be:bc:f2:70:81:a4:70:60:be:05:b5:9b:
          9e:04:44:be:61:23:ac:e9:a5:24:8c:11:80:94:5a:a2:a2:b9:
          49:d2:c1:dc:d1:a7:ed:31:11:2c:9e:19:a6:ee:e1:55:e1:c0:
          ea:cf:0d:84:e4:17:b7:a2:7c:a5:de:55:25:06:ee:cc:c0:87:
          5c:40:da:cc:95:3f:55:e0:35:c7:b8:84:be:b4:5d:cd:7a:83:
          01:72:ee:87:e6:5f:1d:ae:b5:85:c6:26:df:e6:c1:9a:e9:1e:
          02:47:9f:2a:a8:6d:a9:5b:cf:ec:45:77:7f:98:27:9a:32:5d:
          2a:e3:84:ee:c5:98:66:2f:96:20:1d:dd:d8:c3:27:d7:b0:f9:
          fe:d9:7d:cd:d0:9f:8f:0b:14:58:51:9f:2f:8b:c3:38:2d:de:
          e8:8f:d6:8d:87:a4:f5:56:43:16:99:2c:f4:a4:56:b4:34:b8:
          61:37:c9:c2:58:80:1b:a0:97:a1:fc:59:8d:e9:11:f6:d1:0f:
          4b:55:34:46:2a:8b:86:3b
-----BEGIN CERTIFICATE-----
MIIH0zCCBbugAwIBAgIIXsO3pkN/pOAwDQYJKoZIhvcNAQEFBQAwQjESMBAGA1UE
AwwJQUNDVlJBSVoxMRAwDgYDVQQLDAdQS0lBQ0NWMQ0wCwYDVQQKDARBQ0NWMQsw
CQYDVQQGEwJFUzAeFw0xMTA1MDUwOTM3MzdaFw0zMDEyMzEwOTM3MzdaMEIxEjAQ
BgNVBAMMCUFDQ1ZSQUlaMTEQMA4GA1UECwwHUEtJQUNDVjENMAsGA1UECgwEQUND
VjELMAkGA1UEBhMCRVMwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCb
qau/YUqXry+XZpp0X9DZlv3P4uRm7x8fRzPCRKPfmt4ftVTdFXxpNRFvu8gMjmoY
HtiP2Ra8EEg2XPBjs5BaXCQ316PWywlxufEBcoSwfdtNgM3802/J+Nq2DoLSRYWo
G2ioPej0RGy9ocLLA76MPhMAhN9KSMDjIgro6TenGEyxCQ0jVn8ETdkXhBilyNpA
lHPrzg5XPAOBOp0KoVdDaaxXbXmQeOW1tDvYvEyNKKGno6e6Ak4l0Squ7a4DIrhr
IA8wKFSVf+DuzgpmndFALW4ir50awQUZ0m/A8p/4e7MCQvtQqR0tkw8jq8bBD5L/
0KIV9VMJcRz/RROE5iZe+OCIHAr8Fraocwa48GOEAqDGWuzndN9wrqODJerWx5eH
k6fGioozl2A3ED6XPm4pFdahD9GILBKfb6qkxkLrQaLjlUPTAYVtjrs78yM2x/47
4KElB0iryYl0/wiPgL/AlmXz7uxLaL2diMMxs0Dx6M/2OLuc5NF/1OVYm3z61PMO
m3WR5LpSLhl+0fXNWhn8ugb2+1KoS5kE3fj5tItQo05iifCHJPqDQsGH+tUtKSpa
cXpkatcnYGMN285J9Y0fkIkyF/hzQ7jSWpOGYdbhdQrqeWZ2iE9x6wQl1gpaepPl
uUsXQA+xtrn13k/c4LOsOxFwYIRKQ26ZIMApcQrAZQIDAQABo4ICyzCCAscwfQYI
KwYBBQUHAQEEcTBvMEwGCCsGAQUFBzAChkBodHRwOi8vd3d3LmFjY3YuZXMvZmls
ZWFkbWluL0FyY2hpdm9zL2NlcnRpZmljYWRvcy9yYWl6YWNjdjEuY3J0MB8GCCsG
AQUFBzABhhNodHRwOi8vb2NzcC5hY2N2LmVzMB0GA1UdDgQWBBTSh7Tj3zcnk1X2
VuqB5TbMjB4/vTAPBgNVHRMBAf8EBTADAQH/MB8GA1UdIwQYMBaAFNKHtOPfNyeT
VfZW6oHlNsyMHj+9MIIBcwYDVR0gBIIBajCCAWYwggFiBgRVHSAAMIIBWDCCASIG
CCsGAQUFBwICMIIBFB6CARAAQQB1AHQAbwByAGkAZABhAGQAIABkAGUAIABDAGUA
cgB0AGkAZgBpAGMAYQBjAGkA8wBuACAAUgBhAO0AegAgAGQAZQAgAGwAYQAgAEEA
QwBDAFYAIAAoAEEAZwBlAG4AYwBpAGEAIABkAGUAIABUAGUAYwBuAG8AbABvAGcA
7QBhACAAeQAgAEMAZQByAHQAaQBmAGkAYwBhAGMAaQDzAG4AIABFAGwAZQBjAHQA
cgDzAG4AaQBjAGEALAAgAEMASQBGACAAUQA0ADYAMAAxADEANQA2AEUAKQAuACAA
QwBQAFMAIABlAG4AIABoAHQAdABwADoALwAvAHcAdwB3AC4AYQBjAGMAdgAuAGUA
czAwBggrBgEFBQcCARYkaHR0cDovL3d3dy5hY2N2LmVzL2xlZ2lzbGFjaW9uX2Mu
aHRtMFUGA1UdHwROMEwwSqBIoEaGRGh0dHA6Ly93d3cuYWNjdi5lcy9maWxlYWRt
aW4vQXJjaGl2b3MvY2VydGlmaWNhZG9zL3JhaXphY2N2MV9kZXIuY3JsMA4GA1Ud
DwEB/wQEAwIBBjAXBgNVHREEEDAOgQxhY2N2QGFjY3YuZXMwDQYJKoZIhvcNAQEF
BQADggIBAJcxAp/n/UNnSEQU5CmH7UwoZtCPNdpNYbdKl02125DgBS4OxnnQ8pdp
D70ER9m+27Up2pvZrqmZ1dM8MJP1jaGo/AaNRPTKFpV8M9xii6g3+CfYCS0b78gU
JyCpZET/LtZ1qmxNYEAZSUNUY9rizLpm5U9EelvZaoErQNV/+QEnWCzI7UiRfD+m
AM/EKXMRNt6GGT6d7hmKG9Ww7Y49nCrADdg9ZuM8Db3VlFzi4qc1GwQA9j9ajepD
vV+JHanBsMyZ4k0ACtrJJ1vnE5Bc5PUzolVt3OAJTS+xJlsndQAJxGJ3KQhfnlms
tn6tn1QwIgPBHnFk/vk4CpYY3QIUrCPLBhwepH2NDd4nQeit2hW3sCPdK6jT2iWH
7ehVRE2I9DZ+hJp4rPcOVkkO1jMl1oRQQmwgEh0q1b688nCBpHBgvgW1m54ERL5h
I6zppSSMEYCUWqKiuUnSwdzRp+0xESyeGabu4VXhwOrPDYTkF7eifKXeVSUG7szA
h1xA2syVP1XgNce4hL60Xc16gwFy7ofmXx2utYXGJt/mwZrpHgJHnyqobalbz+xF
d3+YJ5oyXSrjhO7FmGYvliAd3djDJ9ew+f7Zfc3Qn48LFFhRny+Lwzgt3uiP1o2H
pPVWQxaZLPSkVrQ0uGE3ycJYgBugl6H8WY3pEfbRD0tVNEYqi4Y7
-----END CERTIFICATE-----

The bit between -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- is the same in both cases, it is all the other information that is missing.
> 
>> This is also the case for the bundles that Arch Linux has created on my systems, that also uses the p11-kit approach.
>>
>> I did a diff on the current ca-bundle.crt/ca-bundle.trust.crt files and the ones created by the new approach as used by Fedora and all the same certificates are present, it is only the certificate data that is missed.
>>
>> There is a --comment option in the trust extract command that looks like it should add the comments that are in the .p11-kit file into the extracted bundle files but nothing changes if it is used or not. I will ask into the p11-kit git repository whether this should work or not.
>>
>>
>> I did a build just with the new ca-certificate files into Core Update 160 and then installed the created iso into a vm on my testbed system.
>>
>> Everything worked that I tested out but I am not 100% sure what needs to be exercised to test the ca-bundle.crt and ca-bundle.trust.crt files. If someone can give me some guidance on this then I can do that final test and submit a patch for this change.
> 
> If you can download something with wget using HTTPS and the certificate can be validated, things should be working fine.
I will give that a try.
> 
>> I also then did a test build with all python2 programs removed, including python2 itself and with the python 3 fixes for spice, frr and ca-certificates installed. The build for every program in the ipfire build section completed successfully but then I had a stop in the cdrom stage for collectd. It has a python2 plugin module defined which was of course missing. The module could be disabled in the rootfile or in the build config if the python plugin is not required for IPFire - I could not find any python modules defined.
> 
> You can just pass “—-without-python” or “—-disable-python” and then the module won’t be built any more.
> 
>> I have been able to successfully define the path to the python interpreter in the ./configure statement to point to the python3 program and collectd successfully built with this so if the python plugin is required then a python3 version can be created. I have not yet tested this in my vm but will do so.
> 
> We do not use this module, so I suppose it isn’t worth building it in the first place.
> 
>> I also found that freeradius has a python2 module defined. This can be disabled.
I will do that for collectd and freeradius then.

I will also do it for alsa as that is the next one I found with a python2 library entry.
> 
> Please do that. If someone needs the Python 3 module, we can still build it> 
>> There is a python3 module available since 2.0.20. To use it with 3.0.21 (current version) you have to have --with-experimental-modules defined. I am looking at latest version 2.0.23 to see if that also needs the experimental modules enabled or if it has become part of the normal modules now and will test the build to see if it works successfully or not.
> 
> Thank you for all the research that you invested into this. Looks like we are not far away to drop Python 2.
I also think we are very close now. Hopefully in time for Core Update 160.

Regards,
Adolf.
> 
> -Michael
> 
>>
>> Regards,
>>
>> Adolf.
>>
>> On 12/08/2021 21:30, Adolf Belka wrote:
>>> Hi Michael,
>>>
>>> On 12/08/2021 21:23, Adolf Belka wrote:
>>>> Hi Michael,
>>>>
>>>> On 12/08/2021 15:38, Michael Tremer wrote:
>>>>> Hello,
>>>>>
>>>>> Yes, this is the way to go :)
>>>>>
>>>>> Reach out here if you need any help.
>>>> I am trying to build p11-kit and it has flagged up libtasn1. It is an optional dependency and I can disable it but I would like to check if that is the correct thing to do or not.
>>>>
>>>> libtasn1 is said to be required to allow the trust policy module to be built as well as other code that interacts with certificates.
>>>> Is this something that would be needed for IPFire or can I disable it in the p11-kit build.
>>>>
>>> Ignore this question. I just did a quick test of disabling libtasn1 and p11-kit built ok but then my ca-certificates build failed due to the lack of the trust tool, so that makes it clear that it is required. I will add it in to the build.
>>>
>>> Slowly getting there.
>>>
>>> Regards,
>>>
>>> Adolf.
>>>
>>>> Regards,
>>>> Adolf.
>>>>>
>>>>> -Michael
>>>>>
>>>>>> On 12 Aug 2021, at 13:10, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>
>>>>>> Hi Michael,
>>>>>>
>>>>>> On 12/08/2021 13:36, Adolf Belka wrote:
>>>>>>> Hi Michael,
>>>>>>>
>>>>>>> On 12/08/2021 11:17, Michael Tremer wrote:
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>>> On 11 Aug 2021, at 15:03, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>
>>>>>>>>> Hi Michael,
>>>>>>>>>
>>>>>>>>> On 11/08/2021 12:43, Michael Tremer wrote:
>>>>>>>>>> Hello,
>>>>>>>>>> Is this the one with the broken sed command?
>>>>>>>>>> https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2pem.py
>>>>>>>>> Yes, this is that one. Confirmed with a diff.
>>>>>>>>>> This should run if you execute it in the right directory:
>>>>>>>>>> pushd %{name}/certs
>>>>>>>>>>     pwd
>>>>>>>>>>     cp certdata.txt .
>>>>>>>>>>     python3 certdata2pem.py
>>>>>>>>>> popd
>>>>>>>>> I have just learnt about the pushd and popd commands. by doing a quick search. Never heard of them before.
>>>>>>>>
>>>>>>>> It is just a version of “cd” that remembers where it has been.
>>>>>>>>
>>>>>>>> So if you call “pushd some-directory”, then “popd” will bring you back to where you have been before.
>>>>>>>>
>>>>>>>> “cd -“ does the same as popd now.
>>>>>>>>
>>>>>>>>>> The fedora version no longer has the build.sh script.
>>>>>>>>> That was the bit I didn't realise.
>>>>>>>>
>>>>>>>> No problem.
>>>>>>>
>>>>>>> Not as simple as I hoped.
>>>>>>>
>>>>>>> The new certdata2pem.py script no longer creates .crt files but .tmp-p11-kit files but that is as far as that script goes.
>>>>>>>
>>>>>>> There is an update-ca-trust file in fedora which splits the various certs to their respective locations, with the openssl ones ending up in ca-bundle.trust.crt but this uses p11-kit. I suspect that it uses ca-bundle.trust.p11-kit generated later on in the %build section of the ca-certificate.spec file from the tmp-p11-kit files generated by the certdata2pem.py script. So it looks like p11-kit needs to be installed to make this work.
>>>>>>>
>>>>>>> Alternatively I have found the following in LFS.
>>>>>>> https://www.linuxfromscratch.org/blfs/view/svn/postlfs/make-ca.html
>>>>>>> which seems to also create the bundled cert file but also requiring p11-kit but then talking about different certdata.txt versions that have various tunings. The one from LFS comes from the mozilla release branch but is modified to something called the Mercurial revision, or the different ones shipped by RedHat or OpenSUSE which use the version that comes with NSS.
>>>>>>>
>>>>>>> All in all I am not sure which approach to use or how to actually build it in IPFire.
>>>>>>>
>>>>>>> Definitely help required to know the correct way to go further with this.
>>>>>>>
>>>>>> Having said I needed help, I thought I would have another go and copied the lines from the %build section of the fedora ca-certificate.spec that created the ca-bundle.trust.p11.kit
>>>>>>
>>>>>> As I have p11-kit installed on my Arch Linux system I then ran the command
>>>>>>
>>>>>> p11-kit extract --format=openssl-bundle --filter=certificates --overwrite --comment ~/openssl/ca-bundle.trust.crt
>>>>>>
>>>>>> and I successfully created the ca-bundle.trust.crt which is needed by the IPFire ca-certificates lfs file.
>>>>>>
>>>>>> I will now try and create a new build script that will do all the new type stuff and also install the p11-kit library files and see how things go.
>>>>>>
>>>>>> :crossed_fingers:
>>>>>>
>>>>>> Adolf
>>>>>>
>>>>>>> Regards,
>>>>>>> Adolf.
>>>>>>>
>>>>>>>> -Michael
>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Adolf.
>>>>>>>>>> -Michael
>>>>>>>>>>> On 8 Aug 2021, at 14:47, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi All,
>>>>>>>>>>>
>>>>>>>>>>> I had another go at the ca-certificates problem, the last barrier to getting rid of python2.
>>>>>>>>>>>
>>>>>>>>>>> I found certdata2pem.py files from fedora and 2 from suse. I created build subdirectories for each version so I could just test running the build.sh file with each version of certdata2pem.py, including the IPFire current version after running through the 2to3 convertor.
>>>>>>>>>>>
>>>>>>>>>>> fedora
>>>>>>>>>>>
>>>>>>>>>>> The fedora certdata2pem.py file runs successfully with python3 but has sed commands built into it which fail to find certain files. The sed commands are not in the IPFire version.
>>>>>>>>>>>
>>>>>>>>>>> The error message is
>>>>>>>>>>>
>>>>>>>>>>>     -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', trust = ['CKA_TRUST_SERVER_AUTH', 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = ['serverAuth', 'emailProtection'], distrust = [], openssl-distrust = []
>>>>>>>>>>> sed: can't read certs/*.crt: No such file or directory
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> suse
>>>>>>>>>>>
>>>>>>>>>>> The first suse version runs successfully with python3 but also has the sed commands in it with the same error message.
>>>>>>>>>>>
>>>>>>>>>>> The second suse version runs successfully with python3, does not have the sed commands and completes the build.sh script with no errors. However this certdata2pem.py file has a section that is in the IPFire version completely missing.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> IPfire version after running through the 2to3 convertor
>>>>>>>>>>>
>>>>>>>>>>> The following error message occurs
>>>>>>>>>>>
>>>>>>>>>>> producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148
>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>      File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in <module>
>>>>>>>>>>> f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64)))
>>>>>>>>>>>      File "/usr/lib/python3.9/base64.py", line 58, in b64encode
>>>>>>>>>>>        encoded = binascii.b2a_base64(s, newline=False)
>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>
>>>>>>>>>>> The section that is failing is the section that is missing in the 2nd suse version. There is an identical fwrite line at line 206 but that does not seem to flag up the same TypeError message.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> As the certdata2pem.py files from the other distributions vary significantly in content, with some having nearly double the number of lines of code, I think the best alternative is to fix the IPFire version so we stay consistent but I am unable to figure out how to fix the python code that is causing the "  TypeError: a bytes-like object is required, not 'str' " error message and need someone's help with that.
>>>>>>>>>>>
>>>>>>>>>>> Let me know if there is any other information that I need to provide.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>
>>>>>>>>>>> Adolf.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 07/08/2021 15:54, Adolf Belka wrote:
>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>
>>>>>>>>>>>> On 04/08/2021 16:45, Michael Tremer wrote:
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Great. This could also resolve Matthias’ problem with building frr.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Will now go back and have another go with spice-protocol.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Maybe it has a —-disable-python switch?
>>>>>>>>>>>> I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
>>>>>>>>>>>> Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.
>>>>>>>>>>>>
>>>>>>>>>>>> This now only leaves the ca-certificates script that needs to be updated to work with python3.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>>>>>>>>>>>>> Hi Michael & all,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thank you for looking into this.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>>>>>>>>>>>>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>>>>>>>>>>>>>> I then commented ca-certificates out in make.sh and ran the build.
>>>>>>>>>>>>>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>>>>>>>>>>>>>> py-compile is python2 based and the build stopped because it could not find python
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I then commented out spice-protocol and ran the build.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>>>>>>>>>>>>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I commented out frr and nothing else failed before cdrom was reached.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> So the packages that need to be made to work with python3 are
>>>>>>>>>>>>>>> ca-certificates
>>>>>>>>>>>>>>> spice-protocol
>>>>>>>>>>>>>>> frr
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> ca-certificates was flagged up.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Adolf.
>>>>>
> 

Re: First results from running build without python2

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 33652 bytes --]

Hi Michael,

On 19/08/2021 18:28, Adolf Belka wrote:
> Hi Michael,
>
> On 19/08/2021 15:54, Michael Tremer wrote:
>> Hello Adolf,
>>
>> This sounds very good.
>>
>>> On 18 Aug 2021, at 22:08, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>
>>> Hi All,
>>>
>>> I have successfully built ca-certificates with the new p11-kit approach. The only difference is that the certificate data such as Version, Serial Number, Signature Algorithm etc is no longer included in the created bundle files. There is only the actual certificate itself.
>>
>> I do not exactly understand what you mean. Can you post a file system listing or something that would help me?The first certificate in the new ca-bundle.crt looks like the following:-
>
> # ACCVRAIZ1
> -----BEGIN CERTIFICATE-----
> MIIH0zCCBbugAwIBAgIIXsO3pkN/pOAwDQYJKoZIhvcNAQEFBQAwQjESMBAGA1UE
> AwwJQUNDVlJBSVoxMRAwDgYDVQQLDAdQS0lBQ0NWMQ0wCwYDVQQKDARBQ0NWMQsw
> CQYDVQQGEwJFUzAeFw0xMTA1MDUwOTM3MzdaFw0zMDEyMzEwOTM3MzdaMEIxEjAQ
> BgNVBAMMCUFDQ1ZSQUlaMTEQMA4GA1UECwwHUEtJQUNDVjENMAsGA1UECgwEQUND
> VjELMAkGA1UEBhMCRVMwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCb
> qau/YUqXry+XZpp0X9DZlv3P4uRm7x8fRzPCRKPfmt4ftVTdFXxpNRFvu8gMjmoY
> HtiP2Ra8EEg2XPBjs5BaXCQ316PWywlxufEBcoSwfdtNgM3802/J+Nq2DoLSRYWo
> G2ioPej0RGy9ocLLA76MPhMAhN9KSMDjIgro6TenGEyxCQ0jVn8ETdkXhBilyNpA
> lHPrzg5XPAOBOp0KoVdDaaxXbXmQeOW1tDvYvEyNKKGno6e6Ak4l0Squ7a4DIrhr
> IA8wKFSVf+DuzgpmndFALW4ir50awQUZ0m/A8p/4e7MCQvtQqR0tkw8jq8bBD5L/
> 0KIV9VMJcRz/RROE5iZe+OCIHAr8Fraocwa48GOEAqDGWuzndN9wrqODJerWx5eH
> k6fGioozl2A3ED6XPm4pFdahD9GILBKfb6qkxkLrQaLjlUPTAYVtjrs78yM2x/47
> 4KElB0iryYl0/wiPgL/AlmXz7uxLaL2diMMxs0Dx6M/2OLuc5NF/1OVYm3z61PMO
> m3WR5LpSLhl+0fXNWhn8ugb2+1KoS5kE3fj5tItQo05iifCHJPqDQsGH+tUtKSpa
> cXpkatcnYGMN285J9Y0fkIkyF/hzQ7jSWpOGYdbhdQrqeWZ2iE9x6wQl1gpaepPl
> uUsXQA+xtrn13k/c4LOsOxFwYIRKQ26ZIMApcQrAZQIDAQABo4ICyzCCAscwfQYI
> KwYBBQUHAQEEcTBvMEwGCCsGAQUFBzAChkBodHRwOi8vd3d3LmFjY3YuZXMvZmls
> ZWFkbWluL0FyY2hpdm9zL2NlcnRpZmljYWRvcy9yYWl6YWNjdjEuY3J0MB8GCCsG
> AQUFBzABhhNodHRwOi8vb2NzcC5hY2N2LmVzMB0GA1UdDgQWBBTSh7Tj3zcnk1X2
> VuqB5TbMjB4/vTAPBgNVHRMBAf8EBTADAQH/MB8GA1UdIwQYMBaAFNKHtOPfNyeT
> VfZW6oHlNsyMHj+9MIIBcwYDVR0gBIIBajCCAWYwggFiBgRVHSAAMIIBWDCCASIG
> CCsGAQUFBwICMIIBFB6CARAAQQB1AHQAbwByAGkAZABhAGQAIABkAGUAIABDAGUA
> cgB0AGkAZgBpAGMAYQBjAGkA8wBuACAAUgBhAO0AegAgAGQAZQAgAGwAYQAgAEEA
> QwBDAFYAIAAoAEEAZwBlAG4AYwBpAGEAIABkAGUAIABUAGUAYwBuAG8AbABvAGcA
> 7QBhACAAeQAgAEMAZQByAHQAaQBmAGkAYwBhAGMAaQDzAG4AIABFAGwAZQBjAHQA
> cgDzAG4AaQBjAGEALAAgAEMASQBGACAAUQA0ADYAMAAxADEANQA2AEUAKQAuACAA
> QwBQAFMAIABlAG4AIABoAHQAdABwADoALwAvAHcAdwB3AC4AYQBjAGMAdgAuAGUA
> czAwBggrBgEFBQcCARYkaHR0cDovL3d3dy5hY2N2LmVzL2xlZ2lzbGFjaW9uX2Mu
> aHRtMFUGA1UdHwROMEwwSqBIoEaGRGh0dHA6Ly93d3cuYWNjdi5lcy9maWxlYWRt
> aW4vQXJjaGl2b3MvY2VydGlmaWNhZG9zL3JhaXphY2N2MV9kZXIuY3JsMA4GA1Ud
> DwEB/wQEAwIBBjAXBgNVHREEEDAOgQxhY2N2QGFjY3YuZXMwDQYJKoZIhvcNAQEF
> BQADggIBAJcxAp/n/UNnSEQU5CmH7UwoZtCPNdpNYbdKl02125DgBS4OxnnQ8pdp
> D70ER9m+27Up2pvZrqmZ1dM8MJP1jaGo/AaNRPTKFpV8M9xii6g3+CfYCS0b78gU
> JyCpZET/LtZ1qmxNYEAZSUNUY9rizLpm5U9EelvZaoErQNV/+QEnWCzI7UiRfD+m
> AM/EKXMRNt6GGT6d7hmKG9Ww7Y49nCrADdg9ZuM8Db3VlFzi4qc1GwQA9j9ajepD
> vV+JHanBsMyZ4k0ACtrJJ1vnE5Bc5PUzolVt3OAJTS+xJlsndQAJxGJ3KQhfnlms
> tn6tn1QwIgPBHnFk/vk4CpYY3QIUrCPLBhwepH2NDd4nQeit2hW3sCPdK6jT2iWH
> 7ehVRE2I9DZ+hJp4rPcOVkkO1jMl1oRQQmwgEh0q1b688nCBpHBgvgW1m54ERL5h
> I6zppSSMEYCUWqKiuUnSwdzRp+0xESyeGabu4VXhwOrPDYTkF7eifKXeVSUG7szA
> h1xA2syVP1XgNce4hL60Xc16gwFy7ofmXx2utYXGJt/mwZrpHgJHnyqobalbz+xF
> d3+YJ5oyXSrjhO7FmGYvliAd3djDJ9ew+f7Zfc3Qn48LFFhRny+Lwzgt3uiP1o2H
> pPVWQxaZLPSkVrQ0uGE3ycJYgBugl6H8WY3pEfbRD0tVNEYqi4Y7
> -----END CERTIFICATE-----
>
> The version currently in place has the following content for the first certificate:-
>
> Certificate:
>     Data:
>         Version: 3 (0x2)
>         Serial Number: 6828503384748696800 (0x5ec3b7a6437fa4e0)
>         Signature Algorithm: sha1WithRSAEncryption
>         Issuer: CN = ACCVRAIZ1, OU = PKIACCV, O = ACCV, C = ES
>         Validity
>             Not Before: May  5 09:37:37 2011 GMT
>             Not After : Dec 31 09:37:37 2030 GMT
>         Subject: CN = ACCVRAIZ1, OU = PKIACCV, O = ACCV, C = ES
>         Subject Public Key Info:
>             Public Key Algorithm: rsaEncryption
>                 RSA Public-Key: (4096 bit)
>                 Modulus:
>                     00:9b:a9:ab:bf:61:4a:97:af:2f:97:66:9a:74:5f:
>                     d0:d9:96:fd:cf:e2:e4:66:ef:1f:1f:47:33:c2:44:
>                     a3:df:9a:de:1f:b5:54:dd:15:7c:69:35:11:6f:bb:
>                     c8:0c:8e:6a:18:1e:d8:8f:d9:16:bc:10:48:36:5c:
>                     f0:63:b3:90:5a:5c:24:37:d7:a3:d6:cb:09:71:b9:
>                     f1:01:72:84:b0:7d:db:4d:80:cd:fc:d3:6f:c9:f8:
>                     da:b6:0e:82:d2:45:85:a8:1b:68:a8:3d:e8:f4:44:
>                     6c:bd:a1:c2:cb:03:be:8c:3e:13:00:84:df:4a:48:
>                     c0:e3:22:0a:e8:e9:37:a7:18:4c:b1:09:0d:23:56:
>                     7f:04:4d:d9:17:84:18:a5:c8:da:40:94:73:eb:ce:
>                     0e:57:3c:03:81:3a:9d:0a:a1:57:43:69:ac:57:6d:
>                     79:90:78:e5:b5:b4:3b:d8:bc:4c:8d:28:a1:a7:a3:
>                     a7:ba:02:4e:25:d1:2a:ae:ed:ae:03:22:b8:6b:20:
>                     0f:30:28:54:95:7f:e0:ee:ce:0a:66:9d:d1:40:2d:
>                     6e:22:af:9d:1a:c1:05:19:d2:6f:c0:f2:9f:f8:7b:
>                     b3:02:42:fb:50:a9:1d:2d:93:0f:23:ab:c6:c1:0f:
>                     92:ff:d0:a2:15:f5:53:09:71:1c:ff:45:13:84:e6:
>                     26:5e:f8:e0:88:1c:0a:fc:16:b6:a8:73:06:b8:f0:
>                     63:84:02:a0:c6:5a:ec:e7:74:df:70:ae:a3:83:25:
>                     ea:d6:c7:97:87:93:a7:c6:8a:8a:33:97:60:37:10:
>                     3e:97:3e:6e:29:15:d6:a1:0f:d1:88:2c:12:9f:6f:
>                     aa:a4:c6:42:eb:41:a2:e3:95:43:d3:01:85:6d:8e:
>                     bb:3b:f3:23:36:c7:fe:3b:e0:a1:25:07:48:ab:c9:
>                     89:74:ff:08:8f:80:bf:c0:96:65:f3:ee:ec:4b:68:
>                     bd:9d:88:c3:31:b3:40:f1:e8:cf:f6:38:bb:9c:e4:
>                     d1:7f:d4:e5:58:9b:7c:fa:d4:f3:0e:9b:75:91:e4:
>                     ba:52:2e:19:7e:d1:f5:cd:5a:19:fc:ba:06:f6:fb:
>                     52:a8:4b:99:04:dd:f8:f9:b4:8b:50:a3:4e:62:89:
>                     f0:87:24:fa:83:42:c1:87:fa:d5:2d:29:2a:5a:71:
>                     7a:64:6a:d7:27:60:63:0d:db:ce:49:f5:8d:1f:90:
>                     89:32:17:f8:73:43:b8:d2:5a:93:86:61:d6:e1:75:
>                     0a:ea:79:66:76:88:4f:71:eb:04:25:d6:0a:5a:7a:
>                     93:e5:b9:4b:17:40:0f:b1:b6:b9:f5:de:4f:dc:e0:
>                     b3:ac:3b:11:70:60:84:4a:43:6e:99:20:c0:29:71:
>                     0a:c0:65
>                 Exponent: 65537 (0x10001)
>         X509v3 extensions:
>             Authority Information Access:
>                 CA Issuers - URI:http://www.accv.es/fileadmin/Archivos/certificados/raizaccv1.crt
>                 OCSP - URI:http://ocsp.accv.es
>
>             X509v3 Subject Key Identifier:
> D2:87:B4:E3:DF:37:27:93:55:F6:56:EA:81:E5:36:CC:8C:1E:3F:BD
>             X509v3 Basic Constraints: critical
>                 CA:TRUE
>             X509v3 Authority Key Identifier:
> keyid:D2:87:B4:E3:DF:37:27:93:55:F6:56:EA:81:E5:36:CC:8C:1E:3F:BD
>
>             X509v3 Certificate Policies:
>                 Policy: X509v3 Any Policy
>                   User Notice:
>                     Explicit Text:
>                   CPS: http://www.accv.es/legislacion_c.htm
>
>             X509v3 CRL Distribution Points:
>
>                 Full Name:
> URI:http://www.accv.es/fileadmin/Archivos/certificados/raizaccv1_der.crl
>
>             X509v3 Key Usage: critical
>                 Certificate Sign, CRL Sign
>             X509v3 Subject Alternative Name:
>                 email:accv(a)accv.es
>     Signature Algorithm: sha1WithRSAEncryption
>          97:31:02:9f:e7:fd:43:67:48:44:14:e4:29:87:ed:4c:28:66:
>          d0:8f:35:da:4d:61:b7:4a:97:4d:b5:db:90:e0:05:2e:0e:c6:
>          79:d0:f2:97:69:0f:bd:04:47:d9:be:db:b5:29:da:9b:d9:ae:
>          a9:99:d5:d3:3c:30:93:f5:8d:a1:a8:fc:06:8d:44:f4:ca:16:
>          95:7c:33:dc:62:8b:a8:37:f8:27:d8:09:2d:1b:ef:c8:14:27:
>          20:a9:64:44:ff:2e:d6:75:aa:6c:4d:60:40:19:49:43:54:63:
>          da:e2:cc:ba:66:e5:4f:44:7a:5b:d9:6a:81:2b:40:d5:7f:f9:
>          01:27:58:2c:c8:ed:48:91:7c:3f:a6:00:cf:c4:29:73:11:36:
>          de:86:19:3e:9d:ee:19:8a:1b:d5:b0:ed:8e:3d:9c:2a:c0:0d:
>          d8:3d:66:e3:3c:0d:bd:d5:94:5c:e2:e2:a7:35:1b:04:00:f6:
>          3f:5a:8d:ea:43:bd:5f:89:1d:a9:c1:b0:cc:99:e2:4d:00:0a:
>          da:c9:27:5b:e7:13:90:5c:e4:f5:33:a2:55:6d:dc:e0:09:4d:
>          2f:b1:26:5b:27:75:00:09:c4:62:77:29:08:5f:9e:59:ac:b6:
>          7e:ad:9f:54:30:22:03:c1:1e:71:64:fe:f9:38:0a:96:18:dd:
>          02:14:ac:23:cb:06:1c:1e:a4:7d:8d:0d:de:27:41:e8:ad:da:
>          15:b7:b0:23:dd:2b:a8:d3:da:25:87:ed:e8:55:44:4d:88:f4:
>          36:7e:84:9a:78:ac:f7:0e:56:49:0e:d6:33:25:d6:84:50:42:
>          6c:20:12:1d:2a:d5:be:bc:f2:70:81:a4:70:60:be:05:b5:9b:
>          9e:04:44:be:61:23:ac:e9:a5:24:8c:11:80:94:5a:a2:a2:b9:
>          49:d2:c1:dc:d1:a7:ed:31:11:2c:9e:19:a6:ee:e1:55:e1:c0:
>          ea:cf:0d:84:e4:17:b7:a2:7c:a5:de:55:25:06:ee:cc:c0:87:
>          5c:40:da:cc:95:3f:55:e0:35:c7:b8:84:be:b4:5d:cd:7a:83:
>          01:72:ee:87:e6:5f:1d:ae:b5:85:c6:26:df:e6:c1:9a:e9:1e:
>          02:47:9f:2a:a8:6d:a9:5b:cf:ec:45:77:7f:98:27:9a:32:5d:
>          2a:e3:84:ee:c5:98:66:2f:96:20:1d:dd:d8:c3:27:d7:b0:f9:
>          fe:d9:7d:cd:d0:9f:8f:0b:14:58:51:9f:2f:8b:c3:38:2d:de:
>          e8:8f:d6:8d:87:a4:f5:56:43:16:99:2c:f4:a4:56:b4:34:b8:
>          61:37:c9:c2:58:80:1b:a0:97:a1:fc:59:8d:e9:11:f6:d1:0f:
>          4b:55:34:46:2a:8b:86:3b
> -----BEGIN CERTIFICATE-----
> MIIH0zCCBbugAwIBAgIIXsO3pkN/pOAwDQYJKoZIhvcNAQEFBQAwQjESMBAGA1UE
> AwwJQUNDVlJBSVoxMRAwDgYDVQQLDAdQS0lBQ0NWMQ0wCwYDVQQKDARBQ0NWMQsw
> CQYDVQQGEwJFUzAeFw0xMTA1MDUwOTM3MzdaFw0zMDEyMzEwOTM3MzdaMEIxEjAQ
> BgNVBAMMCUFDQ1ZSQUlaMTEQMA4GA1UECwwHUEtJQUNDVjENMAsGA1UECgwEQUND
> VjELMAkGA1UEBhMCRVMwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCb
> qau/YUqXry+XZpp0X9DZlv3P4uRm7x8fRzPCRKPfmt4ftVTdFXxpNRFvu8gMjmoY
> HtiP2Ra8EEg2XPBjs5BaXCQ316PWywlxufEBcoSwfdtNgM3802/J+Nq2DoLSRYWo
> G2ioPej0RGy9ocLLA76MPhMAhN9KSMDjIgro6TenGEyxCQ0jVn8ETdkXhBilyNpA
> lHPrzg5XPAOBOp0KoVdDaaxXbXmQeOW1tDvYvEyNKKGno6e6Ak4l0Squ7a4DIrhr
> IA8wKFSVf+DuzgpmndFALW4ir50awQUZ0m/A8p/4e7MCQvtQqR0tkw8jq8bBD5L/
> 0KIV9VMJcRz/RROE5iZe+OCIHAr8Fraocwa48GOEAqDGWuzndN9wrqODJerWx5eH
> k6fGioozl2A3ED6XPm4pFdahD9GILBKfb6qkxkLrQaLjlUPTAYVtjrs78yM2x/47
> 4KElB0iryYl0/wiPgL/AlmXz7uxLaL2diMMxs0Dx6M/2OLuc5NF/1OVYm3z61PMO
> m3WR5LpSLhl+0fXNWhn8ugb2+1KoS5kE3fj5tItQo05iifCHJPqDQsGH+tUtKSpa
> cXpkatcnYGMN285J9Y0fkIkyF/hzQ7jSWpOGYdbhdQrqeWZ2iE9x6wQl1gpaepPl
> uUsXQA+xtrn13k/c4LOsOxFwYIRKQ26ZIMApcQrAZQIDAQABo4ICyzCCAscwfQYI
> KwYBBQUHAQEEcTBvMEwGCCsGAQUFBzAChkBodHRwOi8vd3d3LmFjY3YuZXMvZmls
> ZWFkbWluL0FyY2hpdm9zL2NlcnRpZmljYWRvcy9yYWl6YWNjdjEuY3J0MB8GCCsG
> AQUFBzABhhNodHRwOi8vb2NzcC5hY2N2LmVzMB0GA1UdDgQWBBTSh7Tj3zcnk1X2
> VuqB5TbMjB4/vTAPBgNVHRMBAf8EBTADAQH/MB8GA1UdIwQYMBaAFNKHtOPfNyeT
> VfZW6oHlNsyMHj+9MIIBcwYDVR0gBIIBajCCAWYwggFiBgRVHSAAMIIBWDCCASIG
> CCsGAQUFBwICMIIBFB6CARAAQQB1AHQAbwByAGkAZABhAGQAIABkAGUAIABDAGUA
> cgB0AGkAZgBpAGMAYQBjAGkA8wBuACAAUgBhAO0AegAgAGQAZQAgAGwAYQAgAEEA
> QwBDAFYAIAAoAEEAZwBlAG4AYwBpAGEAIABkAGUAIABUAGUAYwBuAG8AbABvAGcA
> 7QBhACAAeQAgAEMAZQByAHQAaQBmAGkAYwBhAGMAaQDzAG4AIABFAGwAZQBjAHQA
> cgDzAG4AaQBjAGEALAAgAEMASQBGACAAUQA0ADYAMAAxADEANQA2AEUAKQAuACAA
> QwBQAFMAIABlAG4AIABoAHQAdABwADoALwAvAHcAdwB3AC4AYQBjAGMAdgAuAGUA
> czAwBggrBgEFBQcCARYkaHR0cDovL3d3dy5hY2N2LmVzL2xlZ2lzbGFjaW9uX2Mu
> aHRtMFUGA1UdHwROMEwwSqBIoEaGRGh0dHA6Ly93d3cuYWNjdi5lcy9maWxlYWRt
> aW4vQXJjaGl2b3MvY2VydGlmaWNhZG9zL3JhaXphY2N2MV9kZXIuY3JsMA4GA1Ud
> DwEB/wQEAwIBBjAXBgNVHREEEDAOgQxhY2N2QGFjY3YuZXMwDQYJKoZIhvcNAQEF
> BQADggIBAJcxAp/n/UNnSEQU5CmH7UwoZtCPNdpNYbdKl02125DgBS4OxnnQ8pdp
> D70ER9m+27Up2pvZrqmZ1dM8MJP1jaGo/AaNRPTKFpV8M9xii6g3+CfYCS0b78gU
> JyCpZET/LtZ1qmxNYEAZSUNUY9rizLpm5U9EelvZaoErQNV/+QEnWCzI7UiRfD+m
> AM/EKXMRNt6GGT6d7hmKG9Ww7Y49nCrADdg9ZuM8Db3VlFzi4qc1GwQA9j9ajepD
> vV+JHanBsMyZ4k0ACtrJJ1vnE5Bc5PUzolVt3OAJTS+xJlsndQAJxGJ3KQhfnlms
> tn6tn1QwIgPBHnFk/vk4CpYY3QIUrCPLBhwepH2NDd4nQeit2hW3sCPdK6jT2iWH
> 7ehVRE2I9DZ+hJp4rPcOVkkO1jMl1oRQQmwgEh0q1b688nCBpHBgvgW1m54ERL5h
> I6zppSSMEYCUWqKiuUnSwdzRp+0xESyeGabu4VXhwOrPDYTkF7eifKXeVSUG7szA
> h1xA2syVP1XgNce4hL60Xc16gwFy7ofmXx2utYXGJt/mwZrpHgJHnyqobalbz+xF
> d3+YJ5oyXSrjhO7FmGYvliAd3djDJ9ew+f7Zfc3Qn48LFFhRny+Lwzgt3uiP1o2H
> pPVWQxaZLPSkVrQ0uGE3ycJYgBugl6H8WY3pEfbRD0tVNEYqi4Y7
> -----END CERTIFICATE-----
>
> The bit between -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- is the same in both cases, it is all the other information that is missing.
>>
>>> This is also the case for the bundles that Arch Linux has created on my systems, that also uses the p11-kit approach.
>>>
>>> I did a diff on the current ca-bundle.crt/ca-bundle.trust.crt files and the ones created by the new approach as used by Fedora and all the same certificates are present, it is only the certificate data that is missed.
>>>
>>> There is a --comment option in the trust extract command that looks like it should add the comments that are in the .p11-kit file into the extracted bundle files but nothing changes if it is used or not. I will ask into the p11-kit git repository whether this should work or not.
>>>
>>>
>>> I did a build just with the new ca-certificate files into Core Update 160 and then installed the created iso into a vm on my testbed system.
>>>
>>> Everything worked that I tested out but I am not 100% sure what needs to be exercised to test the ca-bundle.crt and ca-bundle.trust.crt files. If someone can give me some guidance on this then I can do that final test and submit a patch for this change.
>>
>> If you can download something with wget using HTTPS and the certificate can be validated, things should be working fine.
> I will give that a try.
>>
I successfully downloaded clamav with wgat from an https url. I got the following messages which I think indicate it worked but there is also mention of http as well as https. Can you confirm if what I did was a good test result?


wget https://www.clamav.net/downloads/production/clamav-0.103.3.tar.gz
--2021-08-19 22:53:03-- https://www.clamav.net/downloads/production/clamav-0.103.3.tar.gz
Resolving www.clamav.net... 104.16.218.84, 104.16.219.84
Connecting to www.clamav.net|104.16.218.84|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://clamav-site.s3.amazonaws.com/production/release_files/files/000/000/612/original/clamav-0.103.3.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIY6OSGQFGUNJQ7GQ%2F20210819%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210819T205304Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=22909d2fe084a9cfcc50b41c0e59ed049089d26c42745801415c9cf697635051 [following]
--2021-08-19 22:53:04-- https://clamav-site.s3.amazonaws.com/production/release_files/files/000/000/612/original/clamav-0.103.3.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIY6OSGQFGUNJQ7GQ%2F20210819%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210819T205304Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=22909d2fe084a9cfcc50b41c0e59ed049089d26c42745801415c9cf697635051
Resolving clamav-site.s3.amazonaws.com... 52.216.224.232
Connecting to clamav-site.s3.amazonaws.com|52.216.224.232|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 13389239 (13M) [binary/octet-stream]
Saving to: 'clamav-0.103.3.tar.gz'

clamav-0.103.3.tar.gz   100%[==============================>] 12.77M  11.6MB/s    in 1.1s

2021-08-19 22:53:05 (11.6 MB/s) - 'clamav-0.103.3.tar.gz' saved [13389239/13389239]

Regards,

Adolf.


>>> I also then did a test build with all python2 programs removed, including python2 itself and with the python 3 fixes for spice, frr and ca-certificates installed. The build for every program in the ipfire build section completed successfully but then I had a stop in the cdrom stage for collectd. It has a python2 plugin module defined which was of course missing. The module could be disabled in the rootfile or in the build config if the python plugin is not required for IPFire - I could not find any python modules defined.
>>
>> You can just pass “—-without-python” or “—-disable-python” and then the module won’t be built any more.
>>
>>> I have been able to successfully define the path to the python interpreter in the ./configure statement to point to the python3 program and collectd successfully built with this so if the python plugin is required then a python3 version can be created. I have not yet tested this in my vm but will do so.
>>
>> We do not use this module, so I suppose it isn’t worth building it in the first place.
>>
>>> I also found that freeradius has a python2 module defined. This can be disabled.
> I will do that for collectd and freeradius then.
>
> I will also do it for alsa as that is the next one I found with a python2 library entry.
>>
>> Please do that. If someone needs the Python 3 module, we can still build it>
>>> There is a python3 module available since 2.0.20. To use it with 3.0.21 (current version) you have to have --with-experimental-modules defined. I am looking at latest version 2.0.23 to see if that also needs the experimental modules enabled or if it has become part of the normal modules now and will test the build to see if it works successfully or not.
>>
>> Thank you for all the research that you invested into this. Looks like we are not far away to drop Python 2.
> I also think we are very close now. Hopefully in time for Core Update 160.
>
> Regards,
> Adolf.
>>
>> -Michael
>>
>>>
>>> Regards,
>>>
>>> Adolf.
>>>
>>> On 12/08/2021 21:30, Adolf Belka wrote:
>>>> Hi Michael,
>>>>
>>>> On 12/08/2021 21:23, Adolf Belka wrote:
>>>>> Hi Michael,
>>>>>
>>>>> On 12/08/2021 15:38, Michael Tremer wrote:
>>>>>> Hello,
>>>>>>
>>>>>> Yes, this is the way to go :)
>>>>>>
>>>>>> Reach out here if you need any help.
>>>>> I am trying to build p11-kit and it has flagged up libtasn1. It is an optional dependency and I can disable it but I would like to check if that is the correct thing to do or not.
>>>>>
>>>>> libtasn1 is said to be required to allow the trust policy module to be built as well as other code that interacts with certificates.
>>>>> Is this something that would be needed for IPFire or can I disable it in the p11-kit build.
>>>>>
>>>> Ignore this question. I just did a quick test of disabling libtasn1 and p11-kit built ok but then my ca-certificates build failed due to the lack of the trust tool, so that makes it clear that it is required. I will add it in to the build.
>>>>
>>>> Slowly getting there.
>>>>
>>>> Regards,
>>>>
>>>> Adolf.
>>>>
>>>>> Regards,
>>>>> Adolf.
>>>>>>
>>>>>> -Michael
>>>>>>
>>>>>>> On 12 Aug 2021, at 13:10, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>
>>>>>>> Hi Michael,
>>>>>>>
>>>>>>> On 12/08/2021 13:36, Adolf Belka wrote:
>>>>>>>> Hi Michael,
>>>>>>>>
>>>>>>>> On 12/08/2021 11:17, Michael Tremer wrote:
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>>> On 11 Aug 2021, at 15:03, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Michael,
>>>>>>>>>>
>>>>>>>>>> On 11/08/2021 12:43, Michael Tremer wrote:
>>>>>>>>>>> Hello,
>>>>>>>>>>> Is this the one with the broken sed command?
>>>>>>>>>>> https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2pem.py
>>>>>>>>>> Yes, this is that one. Confirmed with a diff.
>>>>>>>>>>> This should run if you execute it in the right directory:
>>>>>>>>>>> pushd %{name}/certs
>>>>>>>>>>>     pwd
>>>>>>>>>>>     cp certdata.txt .
>>>>>>>>>>>     python3 certdata2pem.py
>>>>>>>>>>> popd
>>>>>>>>>> I have just learnt about the pushd and popd commands. by doing a quick search. Never heard of them before.
>>>>>>>>>
>>>>>>>>> It is just a version of “cd” that remembers where it has been.
>>>>>>>>>
>>>>>>>>> So if you call “pushd some-directory”, then “popd” will bring you back to where you have been before.
>>>>>>>>>
>>>>>>>>> “cd -“ does the same as popd now.
>>>>>>>>>
>>>>>>>>>>> The fedora version no longer has the build.sh script.
>>>>>>>>>> That was the bit I didn't realise.
>>>>>>>>>
>>>>>>>>> No problem.
>>>>>>>>
>>>>>>>> Not as simple as I hoped.
>>>>>>>>
>>>>>>>> The new certdata2pem.py script no longer creates .crt files but .tmp-p11-kit files but that is as far as that script goes.
>>>>>>>>
>>>>>>>> There is an update-ca-trust file in fedora which splits the various certs to their respective locations, with the openssl ones ending up in ca-bundle.trust.crt but this uses p11-kit. I suspect that it uses ca-bundle.trust.p11-kit generated later on in the %build section of the ca-certificate.spec file from the tmp-p11-kit files generated by the certdata2pem.py script. So it looks like p11-kit needs to be installed to make this work.
>>>>>>>>
>>>>>>>> Alternatively I have found the following in LFS.
>>>>>>>> https://www.linuxfromscratch.org/blfs/view/svn/postlfs/make-ca.html
>>>>>>>> which seems to also create the bundled cert file but also requiring p11-kit but then talking about different certdata.txt versions that have various tunings. The one from LFS comes from the mozilla release branch but is modified to something called the Mercurial revision, or the different ones shipped by RedHat or OpenSUSE which use the version that comes with NSS.
>>>>>>>>
>>>>>>>> All in all I am not sure which approach to use or how to actually build it in IPFire.
>>>>>>>>
>>>>>>>> Definitely help required to know the correct way to go further with this.
>>>>>>>>
>>>>>>> Having said I needed help, I thought I would have another go and copied the lines from the %build section of the fedora ca-certificate.spec that created the ca-bundle.trust.p11.kit
>>>>>>>
>>>>>>> As I have p11-kit installed on my Arch Linux system I then ran the command
>>>>>>>
>>>>>>> p11-kit extract --format=openssl-bundle --filter=certificates --overwrite --comment ~/openssl/ca-bundle.trust.crt
>>>>>>>
>>>>>>> and I successfully created the ca-bundle.trust.crt which is needed by the IPFire ca-certificates lfs file.
>>>>>>>
>>>>>>> I will now try and create a new build script that will do all the new type stuff and also install the p11-kit library files and see how things go.
>>>>>>>
>>>>>>> :crossed_fingers:
>>>>>>>
>>>>>>> Adolf
>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Adolf.
>>>>>>>>
>>>>>>>>> -Michael
>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Adolf.
>>>>>>>>>>> -Michael
>>>>>>>>>>>> On 8 Aug 2021, at 14:47, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>
>>>>>>>>>>>> I had another go at the ca-certificates problem, the last barrier to getting rid of python2.
>>>>>>>>>>>>
>>>>>>>>>>>> I found certdata2pem.py files from fedora and 2 from suse. I created build subdirectories for each version so I could just test running the build.sh file with each version of certdata2pem.py, including the IPFire current version after running through the 2to3 convertor.
>>>>>>>>>>>>
>>>>>>>>>>>> fedora
>>>>>>>>>>>>
>>>>>>>>>>>> The fedora certdata2pem.py file runs successfully with python3 but has sed commands built into it which fail to find certain files. The sed commands are not in the IPFire version.
>>>>>>>>>>>>
>>>>>>>>>>>> The error message is
>>>>>>>>>>>>
>>>>>>>>>>>>     -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', trust = ['CKA_TRUST_SERVER_AUTH', 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = ['serverAuth', 'emailProtection'], distrust = [], openssl-distrust = []
>>>>>>>>>>>> sed: can't read certs/*.crt: No such file or directory
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> suse
>>>>>>>>>>>>
>>>>>>>>>>>> The first suse version runs successfully with python3 but also has the sed commands in it with the same error message.
>>>>>>>>>>>>
>>>>>>>>>>>> The second suse version runs successfully with python3, does not have the sed commands and completes the build.sh script with no errors. However this certdata2pem.py file has a section that is in the IPFire version completely missing.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> IPfire version after running through the 2to3 convertor
>>>>>>>>>>>>
>>>>>>>>>>>> The following error message occurs
>>>>>>>>>>>>
>>>>>>>>>>>> producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148
>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>      File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in <module>
>>>>>>>>>>>> f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64)))
>>>>>>>>>>>>      File "/usr/lib/python3.9/base64.py", line 58, in b64encode
>>>>>>>>>>>>        encoded = binascii.b2a_base64(s, newline=False)
>>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>>
>>>>>>>>>>>> The section that is failing is the section that is missing in the 2nd suse version. There is an identical fwrite line at line 206 but that does not seem to flag up the same TypeError message.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> As the certdata2pem.py files from the other distributions vary significantly in content, with some having nearly double the number of lines of code, I think the best alternative is to fix the IPFire version so we stay consistent but I am unable to figure out how to fix the python code that is causing the "  TypeError: a bytes-like object is required, not 'str' " error message and need someone's help with that.
>>>>>>>>>>>>
>>>>>>>>>>>> Let me know if there is any other information that I need to provide.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>>
>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 07/08/2021 15:54, Adolf Belka wrote:
>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 04/08/2021 16:45, Michael Tremer wrote:
>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Great. This could also resolve Matthias’ problem with building frr.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Will now go back and have another go with spice-protocol.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Maybe it has a —-disable-python switch?
>>>>>>>>>>>>> I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
>>>>>>>>>>>>> Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.
>>>>>>>>>>>>>
>>>>>>>>>>>>> This now only leaves the ca-certificates script that needs to be updated to work with python3.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>>>>>>>>>>>>>> Hi Michael & all,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thank you for looking into this.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>>>>>>>>>>>>>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>>>>>>>>>>>>>>> I then commented ca-certificates out in make.sh and ran the build.
>>>>>>>>>>>>>>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>>>>>>>>>>>>>>> py-compile is python2 based and the build stopped because it could not find python
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I then commented out spice-protocol and ran the build.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>>>>>>>>>>>>>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I commented out frr and nothing else failed before cdrom was reached.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> So the packages that need to be made to work with python3 are
>>>>>>>>>>>>>>>> ca-certificates
>>>>>>>>>>>>>>>> spice-protocol
>>>>>>>>>>>>>>>> frr
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> ca-certificates was flagged up.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Adolf.
>>>>>>
>>

Re: First results from running build without python2

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 35854 bytes --]

Hi Michael and everyone,

After disabling python on collectd, freeradius and alsa I have been able 
to successfully do a complete build with next with python2 removed.

I will now create patches for the above three programs and also for the 
fix for ca-certificates. With the other patches already submitted we can 
then decide in which Core Update to remove python2.


Regards,

Adolf.


On 19/08/2021 22:54, Adolf Belka wrote:
> Hi Michael,
>
> On 19/08/2021 18:28, Adolf Belka wrote:
>> Hi Michael,
>>
>> On 19/08/2021 15:54, Michael Tremer wrote:
>>> Hello Adolf,
>>>
>>> This sounds very good.
>>>
>>>> On 18 Aug 2021, at 22:08, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>
>>>> Hi All,
>>>>
>>>> I have successfully built ca-certificates with the new p11-kit 
>>>> approach. The only difference is that the certificate data such as 
>>>> Version, Serial Number, Signature Algorithm etc is no longer 
>>>> included in the created bundle files. There is only the actual 
>>>> certificate itself.
>>>
>>> I do not exactly understand what you mean. Can you post a file 
>>> system listing or something that would help me?The first certificate 
>>> in the new ca-bundle.crt looks like the following:-
>>
>> # ACCVRAIZ1
>> -----BEGIN CERTIFICATE-----
>> MIIH0zCCBbugAwIBAgIIXsO3pkN/pOAwDQYJKoZIhvcNAQEFBQAwQjESMBAGA1UE
>> AwwJQUNDVlJBSVoxMRAwDgYDVQQLDAdQS0lBQ0NWMQ0wCwYDVQQKDARBQ0NWMQsw
>> CQYDVQQGEwJFUzAeFw0xMTA1MDUwOTM3MzdaFw0zMDEyMzEwOTM3MzdaMEIxEjAQ
>> BgNVBAMMCUFDQ1ZSQUlaMTEQMA4GA1UECwwHUEtJQUNDVjENMAsGA1UECgwEQUND
>> VjELMAkGA1UEBhMCRVMwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCb
>> qau/YUqXry+XZpp0X9DZlv3P4uRm7x8fRzPCRKPfmt4ftVTdFXxpNRFvu8gMjmoY
>> HtiP2Ra8EEg2XPBjs5BaXCQ316PWywlxufEBcoSwfdtNgM3802/J+Nq2DoLSRYWo
>> G2ioPej0RGy9ocLLA76MPhMAhN9KSMDjIgro6TenGEyxCQ0jVn8ETdkXhBilyNpA
>> lHPrzg5XPAOBOp0KoVdDaaxXbXmQeOW1tDvYvEyNKKGno6e6Ak4l0Squ7a4DIrhr
>> IA8wKFSVf+DuzgpmndFALW4ir50awQUZ0m/A8p/4e7MCQvtQqR0tkw8jq8bBD5L/
>> 0KIV9VMJcRz/RROE5iZe+OCIHAr8Fraocwa48GOEAqDGWuzndN9wrqODJerWx5eH
>> k6fGioozl2A3ED6XPm4pFdahD9GILBKfb6qkxkLrQaLjlUPTAYVtjrs78yM2x/47
>> 4KElB0iryYl0/wiPgL/AlmXz7uxLaL2diMMxs0Dx6M/2OLuc5NF/1OVYm3z61PMO
>> m3WR5LpSLhl+0fXNWhn8ugb2+1KoS5kE3fj5tItQo05iifCHJPqDQsGH+tUtKSpa
>> cXpkatcnYGMN285J9Y0fkIkyF/hzQ7jSWpOGYdbhdQrqeWZ2iE9x6wQl1gpaepPl
>> uUsXQA+xtrn13k/c4LOsOxFwYIRKQ26ZIMApcQrAZQIDAQABo4ICyzCCAscwfQYI
>> KwYBBQUHAQEEcTBvMEwGCCsGAQUFBzAChkBodHRwOi8vd3d3LmFjY3YuZXMvZmls
>> ZWFkbWluL0FyY2hpdm9zL2NlcnRpZmljYWRvcy9yYWl6YWNjdjEuY3J0MB8GCCsG
>> AQUFBzABhhNodHRwOi8vb2NzcC5hY2N2LmVzMB0GA1UdDgQWBBTSh7Tj3zcnk1X2
>> VuqB5TbMjB4/vTAPBgNVHRMBAf8EBTADAQH/MB8GA1UdIwQYMBaAFNKHtOPfNyeT
>> VfZW6oHlNsyMHj+9MIIBcwYDVR0gBIIBajCCAWYwggFiBgRVHSAAMIIBWDCCASIG
>> CCsGAQUFBwICMIIBFB6CARAAQQB1AHQAbwByAGkAZABhAGQAIABkAGUAIABDAGUA
>> cgB0AGkAZgBpAGMAYQBjAGkA8wBuACAAUgBhAO0AegAgAGQAZQAgAGwAYQAgAEEA
>> QwBDAFYAIAAoAEEAZwBlAG4AYwBpAGEAIABkAGUAIABUAGUAYwBuAG8AbABvAGcA
>> 7QBhACAAeQAgAEMAZQByAHQAaQBmAGkAYwBhAGMAaQDzAG4AIABFAGwAZQBjAHQA
>> cgDzAG4AaQBjAGEALAAgAEMASQBGACAAUQA0ADYAMAAxADEANQA2AEUAKQAuACAA
>> QwBQAFMAIABlAG4AIABoAHQAdABwADoALwAvAHcAdwB3AC4AYQBjAGMAdgAuAGUA
>> czAwBggrBgEFBQcCARYkaHR0cDovL3d3dy5hY2N2LmVzL2xlZ2lzbGFjaW9uX2Mu
>> aHRtMFUGA1UdHwROMEwwSqBIoEaGRGh0dHA6Ly93d3cuYWNjdi5lcy9maWxlYWRt
>> aW4vQXJjaGl2b3MvY2VydGlmaWNhZG9zL3JhaXphY2N2MV9kZXIuY3JsMA4GA1Ud
>> DwEB/wQEAwIBBjAXBgNVHREEEDAOgQxhY2N2QGFjY3YuZXMwDQYJKoZIhvcNAQEF
>> BQADggIBAJcxAp/n/UNnSEQU5CmH7UwoZtCPNdpNYbdKl02125DgBS4OxnnQ8pdp
>> D70ER9m+27Up2pvZrqmZ1dM8MJP1jaGo/AaNRPTKFpV8M9xii6g3+CfYCS0b78gU
>> JyCpZET/LtZ1qmxNYEAZSUNUY9rizLpm5U9EelvZaoErQNV/+QEnWCzI7UiRfD+m
>> AM/EKXMRNt6GGT6d7hmKG9Ww7Y49nCrADdg9ZuM8Db3VlFzi4qc1GwQA9j9ajepD
>> vV+JHanBsMyZ4k0ACtrJJ1vnE5Bc5PUzolVt3OAJTS+xJlsndQAJxGJ3KQhfnlms
>> tn6tn1QwIgPBHnFk/vk4CpYY3QIUrCPLBhwepH2NDd4nQeit2hW3sCPdK6jT2iWH
>> 7ehVRE2I9DZ+hJp4rPcOVkkO1jMl1oRQQmwgEh0q1b688nCBpHBgvgW1m54ERL5h
>> I6zppSSMEYCUWqKiuUnSwdzRp+0xESyeGabu4VXhwOrPDYTkF7eifKXeVSUG7szA
>> h1xA2syVP1XgNce4hL60Xc16gwFy7ofmXx2utYXGJt/mwZrpHgJHnyqobalbz+xF
>> d3+YJ5oyXSrjhO7FmGYvliAd3djDJ9ew+f7Zfc3Qn48LFFhRny+Lwzgt3uiP1o2H
>> pPVWQxaZLPSkVrQ0uGE3ycJYgBugl6H8WY3pEfbRD0tVNEYqi4Y7
>> -----END CERTIFICATE-----
>>
>> The version currently in place has the following content for the 
>> first certificate:-
>>
>> Certificate:
>>     Data:
>>         Version: 3 (0x2)
>>         Serial Number: 6828503384748696800 (0x5ec3b7a6437fa4e0)
>>         Signature Algorithm: sha1WithRSAEncryption
>>         Issuer: CN = ACCVRAIZ1, OU = PKIACCV, O = ACCV, C = ES
>>         Validity
>>             Not Before: May  5 09:37:37 2011 GMT
>>             Not After : Dec 31 09:37:37 2030 GMT
>>         Subject: CN = ACCVRAIZ1, OU = PKIACCV, O = ACCV, C = ES
>>         Subject Public Key Info:
>>             Public Key Algorithm: rsaEncryption
>>                 RSA Public-Key: (4096 bit)
>>                 Modulus:
>> 00:9b:a9:ab:bf:61:4a:97:af:2f:97:66:9a:74:5f:
>> d0:d9:96:fd:cf:e2:e4:66:ef:1f:1f:47:33:c2:44:
>> a3:df:9a:de:1f:b5:54:dd:15:7c:69:35:11:6f:bb:
>> c8:0c:8e:6a:18:1e:d8:8f:d9:16:bc:10:48:36:5c:
>> f0:63:b3:90:5a:5c:24:37:d7:a3:d6:cb:09:71:b9:
>> f1:01:72:84:b0:7d:db:4d:80:cd:fc:d3:6f:c9:f8:
>> da:b6:0e:82:d2:45:85:a8:1b:68:a8:3d:e8:f4:44:
>> 6c:bd:a1:c2:cb:03:be:8c:3e:13:00:84:df:4a:48:
>> c0:e3:22:0a:e8:e9:37:a7:18:4c:b1:09:0d:23:56:
>> 7f:04:4d:d9:17:84:18:a5:c8:da:40:94:73:eb:ce:
>> 0e:57:3c:03:81:3a:9d:0a:a1:57:43:69:ac:57:6d:
>> 79:90:78:e5:b5:b4:3b:d8:bc:4c:8d:28:a1:a7:a3:
>> a7:ba:02:4e:25:d1:2a:ae:ed:ae:03:22:b8:6b:20:
>> 0f:30:28:54:95:7f:e0:ee:ce:0a:66:9d:d1:40:2d:
>> 6e:22:af:9d:1a:c1:05:19:d2:6f:c0:f2:9f:f8:7b:
>> b3:02:42:fb:50:a9:1d:2d:93:0f:23:ab:c6:c1:0f:
>> 92:ff:d0:a2:15:f5:53:09:71:1c:ff:45:13:84:e6:
>> 26:5e:f8:e0:88:1c:0a:fc:16:b6:a8:73:06:b8:f0:
>> 63:84:02:a0:c6:5a:ec:e7:74:df:70:ae:a3:83:25:
>> ea:d6:c7:97:87:93:a7:c6:8a:8a:33:97:60:37:10:
>> 3e:97:3e:6e:29:15:d6:a1:0f:d1:88:2c:12:9f:6f:
>> aa:a4:c6:42:eb:41:a2:e3:95:43:d3:01:85:6d:8e:
>> bb:3b:f3:23:36:c7:fe:3b:e0:a1:25:07:48:ab:c9:
>> 89:74:ff:08:8f:80:bf:c0:96:65:f3:ee:ec:4b:68:
>> bd:9d:88:c3:31:b3:40:f1:e8:cf:f6:38:bb:9c:e4:
>> d1:7f:d4:e5:58:9b:7c:fa:d4:f3:0e:9b:75:91:e4:
>> ba:52:2e:19:7e:d1:f5:cd:5a:19:fc:ba:06:f6:fb:
>> 52:a8:4b:99:04:dd:f8:f9:b4:8b:50:a3:4e:62:89:
>> f0:87:24:fa:83:42:c1:87:fa:d5:2d:29:2a:5a:71:
>> 7a:64:6a:d7:27:60:63:0d:db:ce:49:f5:8d:1f:90:
>> 89:32:17:f8:73:43:b8:d2:5a:93:86:61:d6:e1:75:
>> 0a:ea:79:66:76:88:4f:71:eb:04:25:d6:0a:5a:7a:
>> 93:e5:b9:4b:17:40:0f:b1:b6:b9:f5:de:4f:dc:e0:
>> b3:ac:3b:11:70:60:84:4a:43:6e:99:20:c0:29:71:
>>                     0a:c0:65
>>                 Exponent: 65537 (0x10001)
>>         X509v3 extensions:
>>             Authority Information Access:
>>                 CA Issuers - 
>> URI:http://www.accv.es/fileadmin/Archivos/certificados/raizaccv1.crt
>>                 OCSP - URI:http://ocsp.accv.es
>>
>>             X509v3 Subject Key Identifier:
>> D2:87:B4:E3:DF:37:27:93:55:F6:56:EA:81:E5:36:CC:8C:1E:3F:BD
>>             X509v3 Basic Constraints: critical
>>                 CA:TRUE
>>             X509v3 Authority Key Identifier:
>> keyid:D2:87:B4:E3:DF:37:27:93:55:F6:56:EA:81:E5:36:CC:8C:1E:3F:BD
>>
>>             X509v3 Certificate Policies:
>>                 Policy: X509v3 Any Policy
>>                   User Notice:
>>                     Explicit Text:
>>                   CPS: http://www.accv.es/legislacion_c.htm
>>
>>             X509v3 CRL Distribution Points:
>>
>>                 Full Name:
>> URI:http://www.accv.es/fileadmin/Archivos/certificados/raizaccv1_der.crl
>>
>>             X509v3 Key Usage: critical
>>                 Certificate Sign, CRL Sign
>>             X509v3 Subject Alternative Name:
>>                 email:accv(a)accv.es
>>     Signature Algorithm: sha1WithRSAEncryption
>>          97:31:02:9f:e7:fd:43:67:48:44:14:e4:29:87:ed:4c:28:66:
>>          d0:8f:35:da:4d:61:b7:4a:97:4d:b5:db:90:e0:05:2e:0e:c6:
>>          79:d0:f2:97:69:0f:bd:04:47:d9:be:db:b5:29:da:9b:d9:ae:
>>          a9:99:d5:d3:3c:30:93:f5:8d:a1:a8:fc:06:8d:44:f4:ca:16:
>>          95:7c:33:dc:62:8b:a8:37:f8:27:d8:09:2d:1b:ef:c8:14:27:
>>          20:a9:64:44:ff:2e:d6:75:aa:6c:4d:60:40:19:49:43:54:63:
>>          da:e2:cc:ba:66:e5:4f:44:7a:5b:d9:6a:81:2b:40:d5:7f:f9:
>>          01:27:58:2c:c8:ed:48:91:7c:3f:a6:00:cf:c4:29:73:11:36:
>>          de:86:19:3e:9d:ee:19:8a:1b:d5:b0:ed:8e:3d:9c:2a:c0:0d:
>>          d8:3d:66:e3:3c:0d:bd:d5:94:5c:e2:e2:a7:35:1b:04:00:f6:
>>          3f:5a:8d:ea:43:bd:5f:89:1d:a9:c1:b0:cc:99:e2:4d:00:0a:
>>          da:c9:27:5b:e7:13:90:5c:e4:f5:33:a2:55:6d:dc:e0:09:4d:
>>          2f:b1:26:5b:27:75:00:09:c4:62:77:29:08:5f:9e:59:ac:b6:
>>          7e:ad:9f:54:30:22:03:c1:1e:71:64:fe:f9:38:0a:96:18:dd:
>>          02:14:ac:23:cb:06:1c:1e:a4:7d:8d:0d:de:27:41:e8:ad:da:
>>          15:b7:b0:23:dd:2b:a8:d3:da:25:87:ed:e8:55:44:4d:88:f4:
>>          36:7e:84:9a:78:ac:f7:0e:56:49:0e:d6:33:25:d6:84:50:42:
>>          6c:20:12:1d:2a:d5:be:bc:f2:70:81:a4:70:60:be:05:b5:9b:
>>          9e:04:44:be:61:23:ac:e9:a5:24:8c:11:80:94:5a:a2:a2:b9:
>>          49:d2:c1:dc:d1:a7:ed:31:11:2c:9e:19:a6:ee:e1:55:e1:c0:
>>          ea:cf:0d:84:e4:17:b7:a2:7c:a5:de:55:25:06:ee:cc:c0:87:
>>          5c:40:da:cc:95:3f:55:e0:35:c7:b8:84:be:b4:5d:cd:7a:83:
>>          01:72:ee:87:e6:5f:1d:ae:b5:85:c6:26:df:e6:c1:9a:e9:1e:
>>          02:47:9f:2a:a8:6d:a9:5b:cf:ec:45:77:7f:98:27:9a:32:5d:
>>          2a:e3:84:ee:c5:98:66:2f:96:20:1d:dd:d8:c3:27:d7:b0:f9:
>>          fe:d9:7d:cd:d0:9f:8f:0b:14:58:51:9f:2f:8b:c3:38:2d:de:
>>          e8:8f:d6:8d:87:a4:f5:56:43:16:99:2c:f4:a4:56:b4:34:b8:
>>          61:37:c9:c2:58:80:1b:a0:97:a1:fc:59:8d:e9:11:f6:d1:0f:
>>          4b:55:34:46:2a:8b:86:3b
>> -----BEGIN CERTIFICATE-----
>> MIIH0zCCBbugAwIBAgIIXsO3pkN/pOAwDQYJKoZIhvcNAQEFBQAwQjESMBAGA1UE
>> AwwJQUNDVlJBSVoxMRAwDgYDVQQLDAdQS0lBQ0NWMQ0wCwYDVQQKDARBQ0NWMQsw
>> CQYDVQQGEwJFUzAeFw0xMTA1MDUwOTM3MzdaFw0zMDEyMzEwOTM3MzdaMEIxEjAQ
>> BgNVBAMMCUFDQ1ZSQUlaMTEQMA4GA1UECwwHUEtJQUNDVjENMAsGA1UECgwEQUND
>> VjELMAkGA1UEBhMCRVMwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCb
>> qau/YUqXry+XZpp0X9DZlv3P4uRm7x8fRzPCRKPfmt4ftVTdFXxpNRFvu8gMjmoY
>> HtiP2Ra8EEg2XPBjs5BaXCQ316PWywlxufEBcoSwfdtNgM3802/J+Nq2DoLSRYWo
>> G2ioPej0RGy9ocLLA76MPhMAhN9KSMDjIgro6TenGEyxCQ0jVn8ETdkXhBilyNpA
>> lHPrzg5XPAOBOp0KoVdDaaxXbXmQeOW1tDvYvEyNKKGno6e6Ak4l0Squ7a4DIrhr
>> IA8wKFSVf+DuzgpmndFALW4ir50awQUZ0m/A8p/4e7MCQvtQqR0tkw8jq8bBD5L/
>> 0KIV9VMJcRz/RROE5iZe+OCIHAr8Fraocwa48GOEAqDGWuzndN9wrqODJerWx5eH
>> k6fGioozl2A3ED6XPm4pFdahD9GILBKfb6qkxkLrQaLjlUPTAYVtjrs78yM2x/47
>> 4KElB0iryYl0/wiPgL/AlmXz7uxLaL2diMMxs0Dx6M/2OLuc5NF/1OVYm3z61PMO
>> m3WR5LpSLhl+0fXNWhn8ugb2+1KoS5kE3fj5tItQo05iifCHJPqDQsGH+tUtKSpa
>> cXpkatcnYGMN285J9Y0fkIkyF/hzQ7jSWpOGYdbhdQrqeWZ2iE9x6wQl1gpaepPl
>> uUsXQA+xtrn13k/c4LOsOxFwYIRKQ26ZIMApcQrAZQIDAQABo4ICyzCCAscwfQYI
>> KwYBBQUHAQEEcTBvMEwGCCsGAQUFBzAChkBodHRwOi8vd3d3LmFjY3YuZXMvZmls
>> ZWFkbWluL0FyY2hpdm9zL2NlcnRpZmljYWRvcy9yYWl6YWNjdjEuY3J0MB8GCCsG
>> AQUFBzABhhNodHRwOi8vb2NzcC5hY2N2LmVzMB0GA1UdDgQWBBTSh7Tj3zcnk1X2
>> VuqB5TbMjB4/vTAPBgNVHRMBAf8EBTADAQH/MB8GA1UdIwQYMBaAFNKHtOPfNyeT
>> VfZW6oHlNsyMHj+9MIIBcwYDVR0gBIIBajCCAWYwggFiBgRVHSAAMIIBWDCCASIG
>> CCsGAQUFBwICMIIBFB6CARAAQQB1AHQAbwByAGkAZABhAGQAIABkAGUAIABDAGUA
>> cgB0AGkAZgBpAGMAYQBjAGkA8wBuACAAUgBhAO0AegAgAGQAZQAgAGwAYQAgAEEA
>> QwBDAFYAIAAoAEEAZwBlAG4AYwBpAGEAIABkAGUAIABUAGUAYwBuAG8AbABvAGcA
>> 7QBhACAAeQAgAEMAZQByAHQAaQBmAGkAYwBhAGMAaQDzAG4AIABFAGwAZQBjAHQA
>> cgDzAG4AaQBjAGEALAAgAEMASQBGACAAUQA0ADYAMAAxADEANQA2AEUAKQAuACAA
>> QwBQAFMAIABlAG4AIABoAHQAdABwADoALwAvAHcAdwB3AC4AYQBjAGMAdgAuAGUA
>> czAwBggrBgEFBQcCARYkaHR0cDovL3d3dy5hY2N2LmVzL2xlZ2lzbGFjaW9uX2Mu
>> aHRtMFUGA1UdHwROMEwwSqBIoEaGRGh0dHA6Ly93d3cuYWNjdi5lcy9maWxlYWRt
>> aW4vQXJjaGl2b3MvY2VydGlmaWNhZG9zL3JhaXphY2N2MV9kZXIuY3JsMA4GA1Ud
>> DwEB/wQEAwIBBjAXBgNVHREEEDAOgQxhY2N2QGFjY3YuZXMwDQYJKoZIhvcNAQEF
>> BQADggIBAJcxAp/n/UNnSEQU5CmH7UwoZtCPNdpNYbdKl02125DgBS4OxnnQ8pdp
>> D70ER9m+27Up2pvZrqmZ1dM8MJP1jaGo/AaNRPTKFpV8M9xii6g3+CfYCS0b78gU
>> JyCpZET/LtZ1qmxNYEAZSUNUY9rizLpm5U9EelvZaoErQNV/+QEnWCzI7UiRfD+m
>> AM/EKXMRNt6GGT6d7hmKG9Ww7Y49nCrADdg9ZuM8Db3VlFzi4qc1GwQA9j9ajepD
>> vV+JHanBsMyZ4k0ACtrJJ1vnE5Bc5PUzolVt3OAJTS+xJlsndQAJxGJ3KQhfnlms
>> tn6tn1QwIgPBHnFk/vk4CpYY3QIUrCPLBhwepH2NDd4nQeit2hW3sCPdK6jT2iWH
>> 7ehVRE2I9DZ+hJp4rPcOVkkO1jMl1oRQQmwgEh0q1b688nCBpHBgvgW1m54ERL5h
>> I6zppSSMEYCUWqKiuUnSwdzRp+0xESyeGabu4VXhwOrPDYTkF7eifKXeVSUG7szA
>> h1xA2syVP1XgNce4hL60Xc16gwFy7ofmXx2utYXGJt/mwZrpHgJHnyqobalbz+xF
>> d3+YJ5oyXSrjhO7FmGYvliAd3djDJ9ew+f7Zfc3Qn48LFFhRny+Lwzgt3uiP1o2H
>> pPVWQxaZLPSkVrQ0uGE3ycJYgBugl6H8WY3pEfbRD0tVNEYqi4Y7
>> -----END CERTIFICATE-----
>>
>> The bit between -----BEGIN CERTIFICATE----- and -----END 
>> CERTIFICATE----- is the same in both cases, it is all the other 
>> information that is missing.
>>>
>>>> This is also the case for the bundles that Arch Linux has created 
>>>> on my systems, that also uses the p11-kit approach.
>>>>
>>>> I did a diff on the current ca-bundle.crt/ca-bundle.trust.crt files 
>>>> and the ones created by the new approach as used by Fedora and all 
>>>> the same certificates are present, it is only the certificate data 
>>>> that is missed.
>>>>
>>>> There is a --comment option in the trust extract command that looks 
>>>> like it should add the comments that are in the .p11-kit file into 
>>>> the extracted bundle files but nothing changes if it is used or 
>>>> not. I will ask into the p11-kit git repository whether this should 
>>>> work or not.
>>>>
>>>>
>>>> I did a build just with the new ca-certificate files into Core 
>>>> Update 160 and then installed the created iso into a vm on my 
>>>> testbed system.
>>>>
>>>> Everything worked that I tested out but I am not 100% sure what 
>>>> needs to be exercised to test the ca-bundle.crt and 
>>>> ca-bundle.trust.crt files. If someone can give me some guidance on 
>>>> this then I can do that final test and submit a patch for this change.
>>>
>>> If you can download something with wget using HTTPS and the 
>>> certificate can be validated, things should be working fine.
>> I will give that a try.
>>>
> I successfully downloaded clamav with wgat from an https url. I got 
> the following messages which I think indicate it worked but there is 
> also mention of http as well as https. Can you confirm if what I did 
> was a good test result?
>
>
> wget https://www.clamav.net/downloads/production/clamav-0.103.3.tar.gz
> --2021-08-19 22:53:03-- 
> https://www.clamav.net/downloads/production/clamav-0.103.3.tar.gz
> Resolving www.clamav.net... 104.16.218.84, 104.16.219.84
> Connecting to www.clamav.net|104.16.218.84|:443... connected.
> HTTP request sent, awaiting response... 302 Found
> Location: 
> https://clamav-site.s3.amazonaws.com/production/release_files/files/000/000/612/original/clamav-0.103.3.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIY6OSGQFGUNJQ7GQ%2F20210819%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210819T205304Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=22909d2fe084a9cfcc50b41c0e59ed049089d26c42745801415c9cf697635051 
> [following]
> --2021-08-19 22:53:04-- 
> https://clamav-site.s3.amazonaws.com/production/release_files/files/000/000/612/original/clamav-0.103.3.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIY6OSGQFGUNJQ7GQ%2F20210819%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210819T205304Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=22909d2fe084a9cfcc50b41c0e59ed049089d26c42745801415c9cf697635051
> Resolving clamav-site.s3.amazonaws.com... 52.216.224.232
> Connecting to clamav-site.s3.amazonaws.com|52.216.224.232|:443... 
> connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 13389239 (13M) [binary/octet-stream]
> Saving to: 'clamav-0.103.3.tar.gz'
>
> clamav-0.103.3.tar.gz   100%[==============================>] 12.77M  
> 11.6MB/s    in 1.1s
>
> 2021-08-19 22:53:05 (11.6 MB/s) - 'clamav-0.103.3.tar.gz' saved 
> [13389239/13389239]
>
> Regards,
>
> Adolf.
>
>
>>>> I also then did a test build with all python2 programs removed, 
>>>> including python2 itself and with the python 3 fixes for spice, frr 
>>>> and ca-certificates installed. The build for every program in the 
>>>> ipfire build section completed successfully but then I had a stop 
>>>> in the cdrom stage for collectd. It has a python2 plugin module 
>>>> defined which was of course missing. The module could be disabled 
>>>> in the rootfile or in the build config if the python plugin is not 
>>>> required for IPFire - I could not find any python modules defined.
>>>
>>> You can just pass “—-without-python” or “—-disable-python” and then 
>>> the module won’t be built any more.
>>>
>>>> I have been able to successfully define the path to the python 
>>>> interpreter in the ./configure statement to point to the python3 
>>>> program and collectd successfully built with this so if the python 
>>>> plugin is required then a python3 version can be created. I have 
>>>> not yet tested this in my vm but will do so.
>>>
>>> We do not use this module, so I suppose it isn’t worth building it 
>>> in the first place.
>>>
>>>> I also found that freeradius has a python2 module defined. This can 
>>>> be disabled.
>> I will do that for collectd and freeradius then.
>>
>> I will also do it for alsa as that is the next one I found with a 
>> python2 library entry.
>>>
>>> Please do that. If someone needs the Python 3 module, we can still 
>>> build it>
>>>> There is a python3 module available since 2.0.20. To use it with 
>>>> 3.0.21 (current version) you have to have 
>>>> --with-experimental-modules defined. I am looking at latest version 
>>>> 2.0.23 to see if that also needs the experimental modules enabled 
>>>> or if it has become part of the normal modules now and will test 
>>>> the build to see if it works successfully or not.
>>>
>>> Thank you for all the research that you invested into this. Looks 
>>> like we are not far away to drop Python 2.
>> I also think we are very close now. Hopefully in time for Core Update 
>> 160.
>>
>> Regards,
>> Adolf.
>>>
>>> -Michael
>>>
>>>>
>>>> Regards,
>>>>
>>>> Adolf.
>>>>
>>>> On 12/08/2021 21:30, Adolf Belka wrote:
>>>>> Hi Michael,
>>>>>
>>>>> On 12/08/2021 21:23, Adolf Belka wrote:
>>>>>> Hi Michael,
>>>>>>
>>>>>> On 12/08/2021 15:38, Michael Tremer wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> Yes, this is the way to go :)
>>>>>>>
>>>>>>> Reach out here if you need any help.
>>>>>> I am trying to build p11-kit and it has flagged up libtasn1. It 
>>>>>> is an optional dependency and I can disable it but I would like 
>>>>>> to check if that is the correct thing to do or not.
>>>>>>
>>>>>> libtasn1 is said to be required to allow the trust policy module 
>>>>>> to be built as well as other code that interacts with certificates.
>>>>>> Is this something that would be needed for IPFire or can I 
>>>>>> disable it in the p11-kit build.
>>>>>>
>>>>> Ignore this question. I just did a quick test of disabling 
>>>>> libtasn1 and p11-kit built ok but then my ca-certificates build 
>>>>> failed due to the lack of the trust tool, so that makes it clear 
>>>>> that it is required. I will add it in to the build.
>>>>>
>>>>> Slowly getting there.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Adolf.
>>>>>
>>>>>> Regards,
>>>>>> Adolf.
>>>>>>>
>>>>>>> -Michael
>>>>>>>
>>>>>>>> On 12 Aug 2021, at 13:10, Adolf Belka <adolf.belka(a)ipfire.org> 
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi Michael,
>>>>>>>>
>>>>>>>> On 12/08/2021 13:36, Adolf Belka wrote:
>>>>>>>>> Hi Michael,
>>>>>>>>>
>>>>>>>>> On 12/08/2021 11:17, Michael Tremer wrote:
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>>> On 11 Aug 2021, at 15:03, Adolf Belka 
>>>>>>>>>>> <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi Michael,
>>>>>>>>>>>
>>>>>>>>>>> On 11/08/2021 12:43, Michael Tremer wrote:
>>>>>>>>>>>> Hello,
>>>>>>>>>>>> Is this the one with the broken sed command?
>>>>>>>>>>>> https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2pem.py 
>>>>>>>>>>>>
>>>>>>>>>>> Yes, this is that one. Confirmed with a diff.
>>>>>>>>>>>> This should run if you execute it in the right directory:
>>>>>>>>>>>> pushd %{name}/certs
>>>>>>>>>>>>     pwd
>>>>>>>>>>>>     cp certdata.txt .
>>>>>>>>>>>>     python3 certdata2pem.py
>>>>>>>>>>>> popd
>>>>>>>>>>> I have just learnt about the pushd and popd commands. by 
>>>>>>>>>>> doing a quick search. Never heard of them before.
>>>>>>>>>>
>>>>>>>>>> It is just a version of “cd” that remembers where it has been.
>>>>>>>>>>
>>>>>>>>>> So if you call “pushd some-directory”, then “popd” will bring 
>>>>>>>>>> you back to where you have been before.
>>>>>>>>>>
>>>>>>>>>> “cd -“ does the same as popd now.
>>>>>>>>>>
>>>>>>>>>>>> The fedora version no longer has the build.sh script.
>>>>>>>>>>> That was the bit I didn't realise.
>>>>>>>>>>
>>>>>>>>>> No problem.
>>>>>>>>>
>>>>>>>>> Not as simple as I hoped.
>>>>>>>>>
>>>>>>>>> The new certdata2pem.py script no longer creates .crt files 
>>>>>>>>> but .tmp-p11-kit files but that is as far as that script goes.
>>>>>>>>>
>>>>>>>>> There is an update-ca-trust file in fedora which splits the 
>>>>>>>>> various certs to their respective locations, with the openssl 
>>>>>>>>> ones ending up in ca-bundle.trust.crt but this uses p11-kit. I 
>>>>>>>>> suspect that it uses ca-bundle.trust.p11-kit generated later 
>>>>>>>>> on in the %build section of the ca-certificate.spec file from 
>>>>>>>>> the tmp-p11-kit files generated by the certdata2pem.py script. 
>>>>>>>>> So it looks like p11-kit needs to be installed to make this work.
>>>>>>>>>
>>>>>>>>> Alternatively I have found the following in LFS.
>>>>>>>>> https://www.linuxfromscratch.org/blfs/view/svn/postlfs/make-ca.html 
>>>>>>>>>
>>>>>>>>> which seems to also create the bundled cert file but also 
>>>>>>>>> requiring p11-kit but then talking about different 
>>>>>>>>> certdata.txt versions that have various tunings. The one from 
>>>>>>>>> LFS comes from the mozilla release branch but is modified to 
>>>>>>>>> something called the Mercurial revision, or the different ones 
>>>>>>>>> shipped by RedHat or OpenSUSE which use the version that comes 
>>>>>>>>> with NSS.
>>>>>>>>>
>>>>>>>>> All in all I am not sure which approach to use or how to 
>>>>>>>>> actually build it in IPFire.
>>>>>>>>>
>>>>>>>>> Definitely help required to know the correct way to go further 
>>>>>>>>> with this.
>>>>>>>>>
>>>>>>>> Having said I needed help, I thought I would have another go 
>>>>>>>> and copied the lines from the %build section of the fedora 
>>>>>>>> ca-certificate.spec that created the ca-bundle.trust.p11.kit
>>>>>>>>
>>>>>>>> As I have p11-kit installed on my Arch Linux system I then ran 
>>>>>>>> the command
>>>>>>>>
>>>>>>>> p11-kit extract --format=openssl-bundle --filter=certificates 
>>>>>>>> --overwrite --comment ~/openssl/ca-bundle.trust.crt
>>>>>>>>
>>>>>>>> and I successfully created the ca-bundle.trust.crt which is 
>>>>>>>> needed by the IPFire ca-certificates lfs file.
>>>>>>>>
>>>>>>>> I will now try and create a new build script that will do all 
>>>>>>>> the new type stuff and also install the p11-kit library files 
>>>>>>>> and see how things go.
>>>>>>>>
>>>>>>>> :crossed_fingers:
>>>>>>>>
>>>>>>>> Adolf
>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Adolf.
>>>>>>>>>
>>>>>>>>>> -Michael
>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Adolf.
>>>>>>>>>>>> -Michael
>>>>>>>>>>>>> On 8 Aug 2021, at 14:47, Adolf Belka 
>>>>>>>>>>>>> <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I had another go at the ca-certificates problem, the last 
>>>>>>>>>>>>> barrier to getting rid of python2.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I found certdata2pem.py files from fedora and 2 from suse. 
>>>>>>>>>>>>> I created build subdirectories for each version so I could 
>>>>>>>>>>>>> just test running the build.sh file with each version of 
>>>>>>>>>>>>> certdata2pem.py, including the IPFire current version 
>>>>>>>>>>>>> after running through the 2to3 convertor.
>>>>>>>>>>>>>
>>>>>>>>>>>>> fedora
>>>>>>>>>>>>>
>>>>>>>>>>>>> The fedora certdata2pem.py file runs successfully with 
>>>>>>>>>>>>> python3 but has sed commands built into it which fail to 
>>>>>>>>>>>>> find certain files. The sed commands are not in the IPFire 
>>>>>>>>>>>>> version.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The error message is
>>>>>>>>>>>>>
>>>>>>>>>>>>>     -> written as 
>>>>>>>>>>>>> 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', 
>>>>>>>>>>>>> trust = ['CKA_TRUST_SERVER_AUTH', 
>>>>>>>>>>>>> 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = 
>>>>>>>>>>>>> ['serverAuth', 'emailProtection'], distrust = [], 
>>>>>>>>>>>>> openssl-distrust = []
>>>>>>>>>>>>> sed: can't read certs/*.crt: No such file or directory
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> suse
>>>>>>>>>>>>>
>>>>>>>>>>>>> The first suse version runs successfully with python3 but 
>>>>>>>>>>>>> also has the sed commands in it with the same error message.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The second suse version runs successfully with python3, 
>>>>>>>>>>>>> does not have the sed commands and completes the build.sh 
>>>>>>>>>>>>> script with no errors. However this certdata2pem.py file 
>>>>>>>>>>>>> has a section that is in the IPFire version completely 
>>>>>>>>>>>>> missing.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> IPfire version after running through the 2to3 convertor
>>>>>>>>>>>>>
>>>>>>>>>>>>> The following error message occurs
>>>>>>>>>>>>>
>>>>>>>>>>>>> producing trust for "GlobalSign Root 
>>>>>>>>>>>>> CA"2.11.4.0.0.0.0.1.21.75.90.195.148
>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>      File 
>>>>>>>>>>>>> "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in 
>>>>>>>>>>>>> progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", 
>>>>>>>>>>>>> line 224, in <module>
>>>>>>>>>>>>> f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 
>>>>>>>>>>>>> 64)))
>>>>>>>>>>>>>      File "/usr/lib/python3.9/base64.py", line 58, in 
>>>>>>>>>>>>> b64encode
>>>>>>>>>>>>>        encoded = binascii.b2a_base64(s, newline=False)
>>>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>>>
>>>>>>>>>>>>> The section that is failing is the section that is missing 
>>>>>>>>>>>>> in the 2nd suse version. There is an identical fwrite line 
>>>>>>>>>>>>> at line 206 but that does not seem to flag up the same 
>>>>>>>>>>>>> TypeError message.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> As the certdata2pem.py files from the other distributions 
>>>>>>>>>>>>> vary significantly in content, with some having nearly 
>>>>>>>>>>>>> double the number of lines of code, I think the best 
>>>>>>>>>>>>> alternative is to fix the IPFire version so we stay 
>>>>>>>>>>>>> consistent but I am unable to figure out how to fix the 
>>>>>>>>>>>>> python code that is causing the "  TypeError: a bytes-like 
>>>>>>>>>>>>> object is required, not 'str' " error message and need 
>>>>>>>>>>>>> someone's help with that.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Let me know if there is any other information that I need 
>>>>>>>>>>>>> to provide.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 07/08/2021 15:54, Adolf Belka wrote:
>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 04/08/2021 16:45, Michael Tremer wrote:
>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 4 Aug 2021, at 13:40, Adolf Belka 
>>>>>>>>>>>>>>>> <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have resolved the frr program build. The version 
>>>>>>>>>>>>>>>> currently in IPFire (6.0) only works with python2. 
>>>>>>>>>>>>>>>> Python3 support came in with version 7.4. I have now 
>>>>>>>>>>>>>>>> built frr with version 8.0 including libyang as a new 
>>>>>>>>>>>>>>>> dependency but only for the build, so nothing installed 
>>>>>>>>>>>>>>>> into IPFire itself, and that has successfully built 
>>>>>>>>>>>>>>>> without python2 being present.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Great. This could also resolve Matthias’ problem with 
>>>>>>>>>>>>>>> building frr.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Will now go back and have another go with spice-protocol.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Maybe it has a —-disable-python switch?
>>>>>>>>>>>>>> I just removed the line in the spice-protocol lfs that 
>>>>>>>>>>>>>> ran automake/py-compile on the python modules from spice.
>>>>>>>>>>>>>> Spice and spice-protocol are present for qemu and with 
>>>>>>>>>>>>>> the py-compile line removed all three successfully built 
>>>>>>>>>>>>>> without python2 being present. I have submitted a patch 
>>>>>>>>>>>>>> for this combined with updating spice and spice-protocol, 
>>>>>>>>>>>>>> both from 2017.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This now only leaves the ca-certificates script that 
>>>>>>>>>>>>>> needs to be updated to work with python3.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>>>>>>>>>>>>>>> Hi Michael & all,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thank you for looking into this.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> This is a third-party script that came from either 
>>>>>>>>>>>>>>>>>> Mozilla or RedHat. Maybe they have ported it. If not, 
>>>>>>>>>>>>>>>>>> it should not be rocket science to do it ourselves. 
>>>>>>>>>>>>>>>>>> If we do it, we should of course upstream it.
>>>>>>>>>>>>>>>>> I found an updated script from fedora and gave that a 
>>>>>>>>>>>>>>>>> try. This time the script went all the way through but 
>>>>>>>>>>>>>>>>> then the build.sh script failed at the point where it 
>>>>>>>>>>>>>>>>> should find all the .crt files in the certs directory 
>>>>>>>>>>>>>>>>> and it came back and said there weren't any.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> However, can you comment out this package and 
>>>>>>>>>>>>>>>>>> continue the build? This should be required until you 
>>>>>>>>>>>>>>>>>> reach the cdrom stage.
>>>>>>>>>>>>>>>>> I then commented ca-certificates out in make.sh and 
>>>>>>>>>>>>>>>>> ran the build.
>>>>>>>>>>>>>>>>> This time it stopped at spice-protocol which is an 
>>>>>>>>>>>>>>>>> addon and uses the py-compile script that is in 
>>>>>>>>>>>>>>>>> automake to compile some python modules.
>>>>>>>>>>>>>>>>> py-compile is python2 based and the build stopped 
>>>>>>>>>>>>>>>>> because it could not find python
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> There is a py_compile.py script that is python3 based 
>>>>>>>>>>>>>>>>> but when I ran that in place of the py-compile script 
>>>>>>>>>>>>>>>>> I got a Permission denied error when it tried to carry 
>>>>>>>>>>>>>>>>> out the compile.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I then commented out spice-protocol and ran the build.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> It then failed on frr which did look for 
>>>>>>>>>>>>>>>>> python3-config but then failed due to not finding 
>>>>>>>>>>>>>>>>> python-config or pkg-config python
>>>>>>>>>>>>>>>>> It looks like I should be able to tell it to use 
>>>>>>>>>>>>>>>>> python3 in the ./configure
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I commented out frr and nothing else failed before 
>>>>>>>>>>>>>>>>> cdrom was reached.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> So the packages that need to be made to work with 
>>>>>>>>>>>>>>>>> python3 are
>>>>>>>>>>>>>>>>> ca-certificates
>>>>>>>>>>>>>>>>> spice-protocol
>>>>>>>>>>>>>>>>> frr
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I also converted client175 with 2to3 converter and 
>>>>>>>>>>>>>>>>> built it and installed the .ipfire package into a vm 
>>>>>>>>>>>>>>>>> and successfully got the WUI page for Media Player 
>>>>>>>>>>>>>>>>> IPFire to render. What I haven't tested yet is if the 
>>>>>>>>>>>>>>>>> audio works. I will need to get audio set up in my vm 
>>>>>>>>>>>>>>>>> to try that.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> If this is the only thing that flags up, we should 
>>>>>>>>>>>>>>>>>> port the script. If we find another, stronger reason 
>>>>>>>>>>>>>>>>>> to keep Python 2 around, we do not need to bother and 
>>>>>>>>>>>>>>>>>> can keep the script this way.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka 
>>>>>>>>>>>>>>>>>>> <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> So with crda and the remaining python2 modules 
>>>>>>>>>>>>>>>>>>> removed the question was if removing python2 from 
>>>>>>>>>>>>>>>>>>> the build ran without any problem or if something 
>>>>>>>>>>>>>>>>>>> was flagged up.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> ca-certificates was flagged up.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> There is a python2 script, certdata2pem.py, which 
>>>>>>>>>>>>>>>>>>> fails if python2 is not present. Running that script 
>>>>>>>>>>>>>>>>>>> with python3 flags up some invalid syntax, 
>>>>>>>>>>>>>>>>>>> unsurprisingly.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I found some patches in Debian from 2015 for 
>>>>>>>>>>>>>>>>>>> certdata2pem.py to provide python3 compatibility. 
>>>>>>>>>>>>>>>>>>> Unfortunately looking at the patch approx half could 
>>>>>>>>>>>>>>>>>>> not be applied because the lines don't exist in the 
>>>>>>>>>>>>>>>>>>> IPFire version of certdata2pem.py (sections to do 
>>>>>>>>>>>>>>>>>>> with blacklisted certs)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I then ran the 2to3 converter on certdata2pem.py and 
>>>>>>>>>>>>>>>>>>> tried that in the build but it came up with the 
>>>>>>>>>>>>>>>>>>> following error.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I don't know how to further move forward with this 
>>>>>>>>>>>>>>>>>>> as I am totally unfamiliar with the python language.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Adolf.
>>>>>>>
>>>
-- 
Sent from my laptop

Re: First results from running build without python2

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 33979 bytes --]

Hello,

> On 20 Aug 2021, at 11:17, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> 
> Hi Michael and everyone,
> 
> After disabling python on collectd, freeradius and alsa I have been able to successfully do a complete build with next with python2 removed.

Very good news!

I think you can submit those patches already because we can merge them straight away.

> I will now create patches for the above three programs and also for the fix for ca-certificates. With the other patches already submitted we can then decide in which Core Update to remove python2.

Agreed. If you like you can submit a patch that removes Python 2 from the build which we will merge at a later time.

I would suggest Core Update 161.

This would be a great milestone and make the distribution smaller, and most importantly, remove another large piece of software that is unmaintained upstream. Thanks for your hard work on this!

Best,
-Michael

> Regards,
> 
> Adolf.
> 
> 
> On 19/08/2021 22:54, Adolf Belka wrote:
>> Hi Michael,
>> 
>> On 19/08/2021 18:28, Adolf Belka wrote:
>>> Hi Michael,
>>> 
>>> On 19/08/2021 15:54, Michael Tremer wrote:
>>>> Hello Adolf,
>>>> 
>>>> This sounds very good.
>>>> 
>>>>> On 18 Aug 2021, at 22:08, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>> 
>>>>> Hi All,
>>>>> 
>>>>> I have successfully built ca-certificates with the new p11-kit approach. The only difference is that the certificate data such as Version, Serial Number, Signature Algorithm etc is no longer included in the created bundle files. There is only the actual certificate itself.
>>>> 
>>>> I do not exactly understand what you mean. Can you post a file system listing or something that would help me?The first certificate in the new ca-bundle.crt looks like the following:-
>>> 
>>> # ACCVRAIZ1
>>> -----BEGIN CERTIFICATE-----
>>> MIIH0zCCBbugAwIBAgIIXsO3pkN/pOAwDQYJKoZIhvcNAQEFBQAwQjESMBAGA1UE
>>> AwwJQUNDVlJBSVoxMRAwDgYDVQQLDAdQS0lBQ0NWMQ0wCwYDVQQKDARBQ0NWMQsw
>>> CQYDVQQGEwJFUzAeFw0xMTA1MDUwOTM3MzdaFw0zMDEyMzEwOTM3MzdaMEIxEjAQ
>>> BgNVBAMMCUFDQ1ZSQUlaMTEQMA4GA1UECwwHUEtJQUNDVjENMAsGA1UECgwEQUND
>>> VjELMAkGA1UEBhMCRVMwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCb
>>> qau/YUqXry+XZpp0X9DZlv3P4uRm7x8fRzPCRKPfmt4ftVTdFXxpNRFvu8gMjmoY
>>> HtiP2Ra8EEg2XPBjs5BaXCQ316PWywlxufEBcoSwfdtNgM3802/J+Nq2DoLSRYWo
>>> G2ioPej0RGy9ocLLA76MPhMAhN9KSMDjIgro6TenGEyxCQ0jVn8ETdkXhBilyNpA
>>> lHPrzg5XPAOBOp0KoVdDaaxXbXmQeOW1tDvYvEyNKKGno6e6Ak4l0Squ7a4DIrhr
>>> IA8wKFSVf+DuzgpmndFALW4ir50awQUZ0m/A8p/4e7MCQvtQqR0tkw8jq8bBD5L/
>>> 0KIV9VMJcRz/RROE5iZe+OCIHAr8Fraocwa48GOEAqDGWuzndN9wrqODJerWx5eH
>>> k6fGioozl2A3ED6XPm4pFdahD9GILBKfb6qkxkLrQaLjlUPTAYVtjrs78yM2x/47
>>> 4KElB0iryYl0/wiPgL/AlmXz7uxLaL2diMMxs0Dx6M/2OLuc5NF/1OVYm3z61PMO
>>> m3WR5LpSLhl+0fXNWhn8ugb2+1KoS5kE3fj5tItQo05iifCHJPqDQsGH+tUtKSpa
>>> cXpkatcnYGMN285J9Y0fkIkyF/hzQ7jSWpOGYdbhdQrqeWZ2iE9x6wQl1gpaepPl
>>> uUsXQA+xtrn13k/c4LOsOxFwYIRKQ26ZIMApcQrAZQIDAQABo4ICyzCCAscwfQYI
>>> KwYBBQUHAQEEcTBvMEwGCCsGAQUFBzAChkBodHRwOi8vd3d3LmFjY3YuZXMvZmls
>>> ZWFkbWluL0FyY2hpdm9zL2NlcnRpZmljYWRvcy9yYWl6YWNjdjEuY3J0MB8GCCsG
>>> AQUFBzABhhNodHRwOi8vb2NzcC5hY2N2LmVzMB0GA1UdDgQWBBTSh7Tj3zcnk1X2
>>> VuqB5TbMjB4/vTAPBgNVHRMBAf8EBTADAQH/MB8GA1UdIwQYMBaAFNKHtOPfNyeT
>>> VfZW6oHlNsyMHj+9MIIBcwYDVR0gBIIBajCCAWYwggFiBgRVHSAAMIIBWDCCASIG
>>> CCsGAQUFBwICMIIBFB6CARAAQQB1AHQAbwByAGkAZABhAGQAIABkAGUAIABDAGUA
>>> cgB0AGkAZgBpAGMAYQBjAGkA8wBuACAAUgBhAO0AegAgAGQAZQAgAGwAYQAgAEEA
>>> QwBDAFYAIAAoAEEAZwBlAG4AYwBpAGEAIABkAGUAIABUAGUAYwBuAG8AbABvAGcA
>>> 7QBhACAAeQAgAEMAZQByAHQAaQBmAGkAYwBhAGMAaQDzAG4AIABFAGwAZQBjAHQA
>>> cgDzAG4AaQBjAGEALAAgAEMASQBGACAAUQA0ADYAMAAxADEANQA2AEUAKQAuACAA
>>> QwBQAFMAIABlAG4AIABoAHQAdABwADoALwAvAHcAdwB3AC4AYQBjAGMAdgAuAGUA
>>> czAwBggrBgEFBQcCARYkaHR0cDovL3d3dy5hY2N2LmVzL2xlZ2lzbGFjaW9uX2Mu
>>> aHRtMFUGA1UdHwROMEwwSqBIoEaGRGh0dHA6Ly93d3cuYWNjdi5lcy9maWxlYWRt
>>> aW4vQXJjaGl2b3MvY2VydGlmaWNhZG9zL3JhaXphY2N2MV9kZXIuY3JsMA4GA1Ud
>>> DwEB/wQEAwIBBjAXBgNVHREEEDAOgQxhY2N2QGFjY3YuZXMwDQYJKoZIhvcNAQEF
>>> BQADggIBAJcxAp/n/UNnSEQU5CmH7UwoZtCPNdpNYbdKl02125DgBS4OxnnQ8pdp
>>> D70ER9m+27Up2pvZrqmZ1dM8MJP1jaGo/AaNRPTKFpV8M9xii6g3+CfYCS0b78gU
>>> JyCpZET/LtZ1qmxNYEAZSUNUY9rizLpm5U9EelvZaoErQNV/+QEnWCzI7UiRfD+m
>>> AM/EKXMRNt6GGT6d7hmKG9Ww7Y49nCrADdg9ZuM8Db3VlFzi4qc1GwQA9j9ajepD
>>> vV+JHanBsMyZ4k0ACtrJJ1vnE5Bc5PUzolVt3OAJTS+xJlsndQAJxGJ3KQhfnlms
>>> tn6tn1QwIgPBHnFk/vk4CpYY3QIUrCPLBhwepH2NDd4nQeit2hW3sCPdK6jT2iWH
>>> 7ehVRE2I9DZ+hJp4rPcOVkkO1jMl1oRQQmwgEh0q1b688nCBpHBgvgW1m54ERL5h
>>> I6zppSSMEYCUWqKiuUnSwdzRp+0xESyeGabu4VXhwOrPDYTkF7eifKXeVSUG7szA
>>> h1xA2syVP1XgNce4hL60Xc16gwFy7ofmXx2utYXGJt/mwZrpHgJHnyqobalbz+xF
>>> d3+YJ5oyXSrjhO7FmGYvliAd3djDJ9ew+f7Zfc3Qn48LFFhRny+Lwzgt3uiP1o2H
>>> pPVWQxaZLPSkVrQ0uGE3ycJYgBugl6H8WY3pEfbRD0tVNEYqi4Y7
>>> -----END CERTIFICATE-----
>>> 
>>> The version currently in place has the following content for the first certificate:-
>>> 
>>> Certificate:
>>>     Data:
>>>         Version: 3 (0x2)
>>>         Serial Number: 6828503384748696800 (0x5ec3b7a6437fa4e0)
>>>         Signature Algorithm: sha1WithRSAEncryption
>>>         Issuer: CN = ACCVRAIZ1, OU = PKIACCV, O = ACCV, C = ES
>>>         Validity
>>>             Not Before: May  5 09:37:37 2011 GMT
>>>             Not After : Dec 31 09:37:37 2030 GMT
>>>         Subject: CN = ACCVRAIZ1, OU = PKIACCV, O = ACCV, C = ES
>>>         Subject Public Key Info:
>>>             Public Key Algorithm: rsaEncryption
>>>                 RSA Public-Key: (4096 bit)
>>>                 Modulus:
>>> 00:9b:a9:ab:bf:61:4a:97:af:2f:97:66:9a:74:5f:
>>> d0:d9:96:fd:cf:e2:e4:66:ef:1f:1f:47:33:c2:44:
>>> a3:df:9a:de:1f:b5:54:dd:15:7c:69:35:11:6f:bb:
>>> c8:0c:8e:6a:18:1e:d8:8f:d9:16:bc:10:48:36:5c:
>>> f0:63:b3:90:5a:5c:24:37:d7:a3:d6:cb:09:71:b9:
>>> f1:01:72:84:b0:7d:db:4d:80:cd:fc:d3:6f:c9:f8:
>>> da:b6:0e:82:d2:45:85:a8:1b:68:a8:3d:e8:f4:44:
>>> 6c:bd:a1:c2:cb:03:be:8c:3e:13:00:84:df:4a:48:
>>> c0:e3:22:0a:e8:e9:37:a7:18:4c:b1:09:0d:23:56:
>>> 7f:04:4d:d9:17:84:18:a5:c8:da:40:94:73:eb:ce:
>>> 0e:57:3c:03:81:3a:9d:0a:a1:57:43:69:ac:57:6d:
>>> 79:90:78:e5:b5:b4:3b:d8:bc:4c:8d:28:a1:a7:a3:
>>> a7:ba:02:4e:25:d1:2a:ae:ed:ae:03:22:b8:6b:20:
>>> 0f:30:28:54:95:7f:e0:ee:ce:0a:66:9d:d1:40:2d:
>>> 6e:22:af:9d:1a:c1:05:19:d2:6f:c0:f2:9f:f8:7b:
>>> b3:02:42:fb:50:a9:1d:2d:93:0f:23:ab:c6:c1:0f:
>>> 92:ff:d0:a2:15:f5:53:09:71:1c:ff:45:13:84:e6:
>>> 26:5e:f8:e0:88:1c:0a:fc:16:b6:a8:73:06:b8:f0:
>>> 63:84:02:a0:c6:5a:ec:e7:74:df:70:ae:a3:83:25:
>>> ea:d6:c7:97:87:93:a7:c6:8a:8a:33:97:60:37:10:
>>> 3e:97:3e:6e:29:15:d6:a1:0f:d1:88:2c:12:9f:6f:
>>> aa:a4:c6:42:eb:41:a2:e3:95:43:d3:01:85:6d:8e:
>>> bb:3b:f3:23:36:c7:fe:3b:e0:a1:25:07:48:ab:c9:
>>> 89:74:ff:08:8f:80:bf:c0:96:65:f3:ee:ec:4b:68:
>>> bd:9d:88:c3:31:b3:40:f1:e8:cf:f6:38:bb:9c:e4:
>>> d1:7f:d4:e5:58:9b:7c:fa:d4:f3:0e:9b:75:91:e4:
>>> ba:52:2e:19:7e:d1:f5:cd:5a:19:fc:ba:06:f6:fb:
>>> 52:a8:4b:99:04:dd:f8:f9:b4:8b:50:a3:4e:62:89:
>>> f0:87:24:fa:83:42:c1:87:fa:d5:2d:29:2a:5a:71:
>>> 7a:64:6a:d7:27:60:63:0d:db:ce:49:f5:8d:1f:90:
>>> 89:32:17:f8:73:43:b8:d2:5a:93:86:61:d6:e1:75:
>>> 0a:ea:79:66:76:88:4f:71:eb:04:25:d6:0a:5a:7a:
>>> 93:e5:b9:4b:17:40:0f:b1:b6:b9:f5:de:4f:dc:e0:
>>> b3:ac:3b:11:70:60:84:4a:43:6e:99:20:c0:29:71:
>>>                     0a:c0:65
>>>                 Exponent: 65537 (0x10001)
>>>         X509v3 extensions:
>>>             Authority Information Access:
>>>                 CA Issuers - URI:http://www.accv.es/fileadmin/Archivos/certificados/raizaccv1.crt
>>>                 OCSP - URI:http://ocsp.accv.es
>>> 
>>>             X509v3 Subject Key Identifier:
>>> D2:87:B4:E3:DF:37:27:93:55:F6:56:EA:81:E5:36:CC:8C:1E:3F:BD
>>>             X509v3 Basic Constraints: critical
>>>                 CA:TRUE
>>>             X509v3 Authority Key Identifier:
>>> keyid:D2:87:B4:E3:DF:37:27:93:55:F6:56:EA:81:E5:36:CC:8C:1E:3F:BD
>>> 
>>>             X509v3 Certificate Policies:
>>>                 Policy: X509v3 Any Policy
>>>                   User Notice:
>>>                     Explicit Text:
>>>                   CPS: http://www.accv.es/legislacion_c.htm
>>> 
>>>             X509v3 CRL Distribution Points:
>>> 
>>>                 Full Name:
>>> URI:http://www.accv.es/fileadmin/Archivos/certificados/raizaccv1_der.crl
>>> 
>>>             X509v3 Key Usage: critical
>>>                 Certificate Sign, CRL Sign
>>>             X509v3 Subject Alternative Name:
>>>                 email:accv(a)accv.es
>>>     Signature Algorithm: sha1WithRSAEncryption
>>>          97:31:02:9f:e7:fd:43:67:48:44:14:e4:29:87:ed:4c:28:66:
>>>          d0:8f:35:da:4d:61:b7:4a:97:4d:b5:db:90:e0:05:2e:0e:c6:
>>>          79:d0:f2:97:69:0f:bd:04:47:d9:be:db:b5:29:da:9b:d9:ae:
>>>          a9:99:d5:d3:3c:30:93:f5:8d:a1:a8:fc:06:8d:44:f4:ca:16:
>>>          95:7c:33:dc:62:8b:a8:37:f8:27:d8:09:2d:1b:ef:c8:14:27:
>>>          20:a9:64:44:ff:2e:d6:75:aa:6c:4d:60:40:19:49:43:54:63:
>>>          da:e2:cc:ba:66:e5:4f:44:7a:5b:d9:6a:81:2b:40:d5:7f:f9:
>>>          01:27:58:2c:c8:ed:48:91:7c:3f:a6:00:cf:c4:29:73:11:36:
>>>          de:86:19:3e:9d:ee:19:8a:1b:d5:b0:ed:8e:3d:9c:2a:c0:0d:
>>>          d8:3d:66:e3:3c:0d:bd:d5:94:5c:e2:e2:a7:35:1b:04:00:f6:
>>>          3f:5a:8d:ea:43:bd:5f:89:1d:a9:c1:b0:cc:99:e2:4d:00:0a:
>>>          da:c9:27:5b:e7:13:90:5c:e4:f5:33:a2:55:6d:dc:e0:09:4d:
>>>          2f:b1:26:5b:27:75:00:09:c4:62:77:29:08:5f:9e:59:ac:b6:
>>>          7e:ad:9f:54:30:22:03:c1:1e:71:64:fe:f9:38:0a:96:18:dd:
>>>          02:14:ac:23:cb:06:1c:1e:a4:7d:8d:0d:de:27:41:e8:ad:da:
>>>          15:b7:b0:23:dd:2b:a8:d3:da:25:87:ed:e8:55:44:4d:88:f4:
>>>          36:7e:84:9a:78:ac:f7:0e:56:49:0e:d6:33:25:d6:84:50:42:
>>>          6c:20:12:1d:2a:d5:be:bc:f2:70:81:a4:70:60:be:05:b5:9b:
>>>          9e:04:44:be:61:23:ac:e9:a5:24:8c:11:80:94:5a:a2:a2:b9:
>>>          49:d2:c1:dc:d1:a7:ed:31:11:2c:9e:19:a6:ee:e1:55:e1:c0:
>>>          ea:cf:0d:84:e4:17:b7:a2:7c:a5:de:55:25:06:ee:cc:c0:87:
>>>          5c:40:da:cc:95:3f:55:e0:35:c7:b8:84:be:b4:5d:cd:7a:83:
>>>          01:72:ee:87:e6:5f:1d:ae:b5:85:c6:26:df:e6:c1:9a:e9:1e:
>>>          02:47:9f:2a:a8:6d:a9:5b:cf:ec:45:77:7f:98:27:9a:32:5d:
>>>          2a:e3:84:ee:c5:98:66:2f:96:20:1d:dd:d8:c3:27:d7:b0:f9:
>>>          fe:d9:7d:cd:d0:9f:8f:0b:14:58:51:9f:2f:8b:c3:38:2d:de:
>>>          e8:8f:d6:8d:87:a4:f5:56:43:16:99:2c:f4:a4:56:b4:34:b8:
>>>          61:37:c9:c2:58:80:1b:a0:97:a1:fc:59:8d:e9:11:f6:d1:0f:
>>>          4b:55:34:46:2a:8b:86:3b
>>> -----BEGIN CERTIFICATE-----
>>> MIIH0zCCBbugAwIBAgIIXsO3pkN/pOAwDQYJKoZIhvcNAQEFBQAwQjESMBAGA1UE
>>> AwwJQUNDVlJBSVoxMRAwDgYDVQQLDAdQS0lBQ0NWMQ0wCwYDVQQKDARBQ0NWMQsw
>>> CQYDVQQGEwJFUzAeFw0xMTA1MDUwOTM3MzdaFw0zMDEyMzEwOTM3MzdaMEIxEjAQ
>>> BgNVBAMMCUFDQ1ZSQUlaMTEQMA4GA1UECwwHUEtJQUNDVjENMAsGA1UECgwEQUND
>>> VjELMAkGA1UEBhMCRVMwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCb
>>> qau/YUqXry+XZpp0X9DZlv3P4uRm7x8fRzPCRKPfmt4ftVTdFXxpNRFvu8gMjmoY
>>> HtiP2Ra8EEg2XPBjs5BaXCQ316PWywlxufEBcoSwfdtNgM3802/J+Nq2DoLSRYWo
>>> G2ioPej0RGy9ocLLA76MPhMAhN9KSMDjIgro6TenGEyxCQ0jVn8ETdkXhBilyNpA
>>> lHPrzg5XPAOBOp0KoVdDaaxXbXmQeOW1tDvYvEyNKKGno6e6Ak4l0Squ7a4DIrhr
>>> IA8wKFSVf+DuzgpmndFALW4ir50awQUZ0m/A8p/4e7MCQvtQqR0tkw8jq8bBD5L/
>>> 0KIV9VMJcRz/RROE5iZe+OCIHAr8Fraocwa48GOEAqDGWuzndN9wrqODJerWx5eH
>>> k6fGioozl2A3ED6XPm4pFdahD9GILBKfb6qkxkLrQaLjlUPTAYVtjrs78yM2x/47
>>> 4KElB0iryYl0/wiPgL/AlmXz7uxLaL2diMMxs0Dx6M/2OLuc5NF/1OVYm3z61PMO
>>> m3WR5LpSLhl+0fXNWhn8ugb2+1KoS5kE3fj5tItQo05iifCHJPqDQsGH+tUtKSpa
>>> cXpkatcnYGMN285J9Y0fkIkyF/hzQ7jSWpOGYdbhdQrqeWZ2iE9x6wQl1gpaepPl
>>> uUsXQA+xtrn13k/c4LOsOxFwYIRKQ26ZIMApcQrAZQIDAQABo4ICyzCCAscwfQYI
>>> KwYBBQUHAQEEcTBvMEwGCCsGAQUFBzAChkBodHRwOi8vd3d3LmFjY3YuZXMvZmls
>>> ZWFkbWluL0FyY2hpdm9zL2NlcnRpZmljYWRvcy9yYWl6YWNjdjEuY3J0MB8GCCsG
>>> AQUFBzABhhNodHRwOi8vb2NzcC5hY2N2LmVzMB0GA1UdDgQWBBTSh7Tj3zcnk1X2
>>> VuqB5TbMjB4/vTAPBgNVHRMBAf8EBTADAQH/MB8GA1UdIwQYMBaAFNKHtOPfNyeT
>>> VfZW6oHlNsyMHj+9MIIBcwYDVR0gBIIBajCCAWYwggFiBgRVHSAAMIIBWDCCASIG
>>> CCsGAQUFBwICMIIBFB6CARAAQQB1AHQAbwByAGkAZABhAGQAIABkAGUAIABDAGUA
>>> cgB0AGkAZgBpAGMAYQBjAGkA8wBuACAAUgBhAO0AegAgAGQAZQAgAGwAYQAgAEEA
>>> QwBDAFYAIAAoAEEAZwBlAG4AYwBpAGEAIABkAGUAIABUAGUAYwBuAG8AbABvAGcA
>>> 7QBhACAAeQAgAEMAZQByAHQAaQBmAGkAYwBhAGMAaQDzAG4AIABFAGwAZQBjAHQA
>>> cgDzAG4AaQBjAGEALAAgAEMASQBGACAAUQA0ADYAMAAxADEANQA2AEUAKQAuACAA
>>> QwBQAFMAIABlAG4AIABoAHQAdABwADoALwAvAHcAdwB3AC4AYQBjAGMAdgAuAGUA
>>> czAwBggrBgEFBQcCARYkaHR0cDovL3d3dy5hY2N2LmVzL2xlZ2lzbGFjaW9uX2Mu
>>> aHRtMFUGA1UdHwROMEwwSqBIoEaGRGh0dHA6Ly93d3cuYWNjdi5lcy9maWxlYWRt
>>> aW4vQXJjaGl2b3MvY2VydGlmaWNhZG9zL3JhaXphY2N2MV9kZXIuY3JsMA4GA1Ud
>>> DwEB/wQEAwIBBjAXBgNVHREEEDAOgQxhY2N2QGFjY3YuZXMwDQYJKoZIhvcNAQEF
>>> BQADggIBAJcxAp/n/UNnSEQU5CmH7UwoZtCPNdpNYbdKl02125DgBS4OxnnQ8pdp
>>> D70ER9m+27Up2pvZrqmZ1dM8MJP1jaGo/AaNRPTKFpV8M9xii6g3+CfYCS0b78gU
>>> JyCpZET/LtZ1qmxNYEAZSUNUY9rizLpm5U9EelvZaoErQNV/+QEnWCzI7UiRfD+m
>>> AM/EKXMRNt6GGT6d7hmKG9Ww7Y49nCrADdg9ZuM8Db3VlFzi4qc1GwQA9j9ajepD
>>> vV+JHanBsMyZ4k0ACtrJJ1vnE5Bc5PUzolVt3OAJTS+xJlsndQAJxGJ3KQhfnlms
>>> tn6tn1QwIgPBHnFk/vk4CpYY3QIUrCPLBhwepH2NDd4nQeit2hW3sCPdK6jT2iWH
>>> 7ehVRE2I9DZ+hJp4rPcOVkkO1jMl1oRQQmwgEh0q1b688nCBpHBgvgW1m54ERL5h
>>> I6zppSSMEYCUWqKiuUnSwdzRp+0xESyeGabu4VXhwOrPDYTkF7eifKXeVSUG7szA
>>> h1xA2syVP1XgNce4hL60Xc16gwFy7ofmXx2utYXGJt/mwZrpHgJHnyqobalbz+xF
>>> d3+YJ5oyXSrjhO7FmGYvliAd3djDJ9ew+f7Zfc3Qn48LFFhRny+Lwzgt3uiP1o2H
>>> pPVWQxaZLPSkVrQ0uGE3ycJYgBugl6H8WY3pEfbRD0tVNEYqi4Y7
>>> -----END CERTIFICATE-----
>>> 
>>> The bit between -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- is the same in both cases, it is all the other information that is missing.
>>>> 
>>>>> This is also the case for the bundles that Arch Linux has created on my systems, that also uses the p11-kit approach.
>>>>> 
>>>>> I did a diff on the current ca-bundle.crt/ca-bundle.trust.crt files and the ones created by the new approach as used by Fedora and all the same certificates are present, it is only the certificate data that is missed.
>>>>> 
>>>>> There is a --comment option in the trust extract command that looks like it should add the comments that are in the .p11-kit file into the extracted bundle files but nothing changes if it is used or not. I will ask into the p11-kit git repository whether this should work or not.
>>>>> 
>>>>> 
>>>>> I did a build just with the new ca-certificate files into Core Update 160 and then installed the created iso into a vm on my testbed system.
>>>>> 
>>>>> Everything worked that I tested out but I am not 100% sure what needs to be exercised to test the ca-bundle.crt and ca-bundle.trust.crt files. If someone can give me some guidance on this then I can do that final test and submit a patch for this change.
>>>> 
>>>> If you can download something with wget using HTTPS and the certificate can be validated, things should be working fine.
>>> I will give that a try.
>>>> 
>> I successfully downloaded clamav with wgat from an https url. I got the following messages which I think indicate it worked but there is also mention of http as well as https. Can you confirm if what I did was a good test result?
>> 
>> 
>> wget https://www.clamav.net/downloads/production/clamav-0.103.3.tar.gz
>> --2021-08-19 22:53:03-- https://www.clamav.net/downloads/production/clamav-0.103.3.tar.gz
>> Resolving www.clamav.net... 104.16.218.84, 104.16.219.84
>> Connecting to www.clamav.net|104.16.218.84|:443... connected.
>> HTTP request sent, awaiting response... 302 Found
>> Location: https://clamav-site.s3.amazonaws.com/production/release_files/files/000/000/612/original/clamav-0.103.3.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIY6OSGQFGUNJQ7GQ%2F20210819%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210819T205304Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=22909d2fe084a9cfcc50b41c0e59ed049089d26c42745801415c9cf697635051 [following]
>> --2021-08-19 22:53:04-- https://clamav-site.s3.amazonaws.com/production/release_files/files/000/000/612/original/clamav-0.103.3.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIY6OSGQFGUNJQ7GQ%2F20210819%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210819T205304Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=22909d2fe084a9cfcc50b41c0e59ed049089d26c42745801415c9cf697635051
>> Resolving clamav-site.s3.amazonaws.com... 52.216.224.232
>> Connecting to clamav-site.s3.amazonaws.com|52.216.224.232|:443... connected.
>> HTTP request sent, awaiting response... 200 OK
>> Length: 13389239 (13M) [binary/octet-stream]
>> Saving to: 'clamav-0.103.3.tar.gz'
>> 
>> clamav-0.103.3.tar.gz   100%[==============================>] 12.77M  11.6MB/s    in 1.1s
>> 
>> 2021-08-19 22:53:05 (11.6 MB/s) - 'clamav-0.103.3.tar.gz' saved [13389239/13389239]
>> 
>> Regards,
>> 
>> Adolf.
>> 
>> 
>>>>> I also then did a test build with all python2 programs removed, including python2 itself and with the python 3 fixes for spice, frr and ca-certificates installed. The build for every program in the ipfire build section completed successfully but then I had a stop in the cdrom stage for collectd. It has a python2 plugin module defined which was of course missing. The module could be disabled in the rootfile or in the build config if the python plugin is not required for IPFire - I could not find any python modules defined.
>>>> 
>>>> You can just pass “—-without-python” or “—-disable-python” and then the module won’t be built any more.
>>>> 
>>>>> I have been able to successfully define the path to the python interpreter in the ./configure statement to point to the python3 program and collectd successfully built with this so if the python plugin is required then a python3 version can be created. I have not yet tested this in my vm but will do so.
>>>> 
>>>> We do not use this module, so I suppose it isn’t worth building it in the first place.
>>>> 
>>>>> I also found that freeradius has a python2 module defined. This can be disabled.
>>> I will do that for collectd and freeradius then.
>>> 
>>> I will also do it for alsa as that is the next one I found with a python2 library entry.
>>>> 
>>>> Please do that. If someone needs the Python 3 module, we can still build it>
>>>>> There is a python3 module available since 2.0.20. To use it with 3.0.21 (current version) you have to have --with-experimental-modules defined. I am looking at latest version 2.0.23 to see if that also needs the experimental modules enabled or if it has become part of the normal modules now and will test the build to see if it works successfully or not.
>>>> 
>>>> Thank you for all the research that you invested into this. Looks like we are not far away to drop Python 2.
>>> I also think we are very close now. Hopefully in time for Core Update 160.
>>> 
>>> Regards,
>>> Adolf.
>>>> 
>>>> -Michael
>>>> 
>>>>> 
>>>>> Regards,
>>>>> 
>>>>> Adolf.
>>>>> 
>>>>> On 12/08/2021 21:30, Adolf Belka wrote:
>>>>>> Hi Michael,
>>>>>> 
>>>>>> On 12/08/2021 21:23, Adolf Belka wrote:
>>>>>>> Hi Michael,
>>>>>>> 
>>>>>>> On 12/08/2021 15:38, Michael Tremer wrote:
>>>>>>>> Hello,
>>>>>>>> 
>>>>>>>> Yes, this is the way to go :)
>>>>>>>> 
>>>>>>>> Reach out here if you need any help.
>>>>>>> I am trying to build p11-kit and it has flagged up libtasn1. It is an optional dependency and I can disable it but I would like to check if that is the correct thing to do or not.
>>>>>>> 
>>>>>>> libtasn1 is said to be required to allow the trust policy module to be built as well as other code that interacts with certificates.
>>>>>>> Is this something that would be needed for IPFire or can I disable it in the p11-kit build.
>>>>>>> 
>>>>>> Ignore this question. I just did a quick test of disabling libtasn1 and p11-kit built ok but then my ca-certificates build failed due to the lack of the trust tool, so that makes it clear that it is required. I will add it in to the build.
>>>>>> 
>>>>>> Slowly getting there.
>>>>>> 
>>>>>> Regards,
>>>>>> 
>>>>>> Adolf.
>>>>>> 
>>>>>>> Regards,
>>>>>>> Adolf.
>>>>>>>> 
>>>>>>>> -Michael
>>>>>>>> 
>>>>>>>>> On 12 Aug 2021, at 13:10, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>> 
>>>>>>>>> Hi Michael,
>>>>>>>>> 
>>>>>>>>> On 12/08/2021 13:36, Adolf Belka wrote:
>>>>>>>>>> Hi Michael,
>>>>>>>>>> 
>>>>>>>>>> On 12/08/2021 11:17, Michael Tremer wrote:
>>>>>>>>>>> Hello,
>>>>>>>>>>> 
>>>>>>>>>>>> On 11 Aug 2021, at 15:03, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Hi Michael,
>>>>>>>>>>>> 
>>>>>>>>>>>> On 11/08/2021 12:43, Michael Tremer wrote:
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>> Is this the one with the broken sed command?
>>>>>>>>>>>>> https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2pem.py 
>>>>>>>>>>>> Yes, this is that one. Confirmed with a diff.
>>>>>>>>>>>>> This should run if you execute it in the right directory:
>>>>>>>>>>>>> pushd %{name}/certs
>>>>>>>>>>>>>     pwd
>>>>>>>>>>>>>     cp certdata.txt .
>>>>>>>>>>>>>     python3 certdata2pem.py
>>>>>>>>>>>>> popd
>>>>>>>>>>>> I have just learnt about the pushd and popd commands. by doing a quick search. Never heard of them before.
>>>>>>>>>>> 
>>>>>>>>>>> It is just a version of “cd” that remembers where it has been.
>>>>>>>>>>> 
>>>>>>>>>>> So if you call “pushd some-directory”, then “popd” will bring you back to where you have been before.
>>>>>>>>>>> 
>>>>>>>>>>> “cd -“ does the same as popd now.
>>>>>>>>>>> 
>>>>>>>>>>>>> The fedora version no longer has the build.sh script.
>>>>>>>>>>>> That was the bit I didn't realise.
>>>>>>>>>>> 
>>>>>>>>>>> No problem.
>>>>>>>>>> 
>>>>>>>>>> Not as simple as I hoped.
>>>>>>>>>> 
>>>>>>>>>> The new certdata2pem.py script no longer creates .crt files but .tmp-p11-kit files but that is as far as that script goes.
>>>>>>>>>> 
>>>>>>>>>> There is an update-ca-trust file in fedora which splits the various certs to their respective locations, with the openssl ones ending up in ca-bundle.trust.crt but this uses p11-kit. I suspect that it uses ca-bundle.trust.p11-kit generated later on in the %build section of the ca-certificate.spec file from the tmp-p11-kit files generated by the certdata2pem.py script. So it looks like p11-kit needs to be installed to make this work.
>>>>>>>>>> 
>>>>>>>>>> Alternatively I have found the following in LFS.
>>>>>>>>>> https://www.linuxfromscratch.org/blfs/view/svn/postlfs/make-ca.html 
>>>>>>>>>> which seems to also create the bundled cert file but also requiring p11-kit but then talking about different certdata.txt versions that have various tunings. The one from LFS comes from the mozilla release branch but is modified to something called the Mercurial revision, or the different ones shipped by RedHat or OpenSUSE which use the version that comes with NSS.
>>>>>>>>>> 
>>>>>>>>>> All in all I am not sure which approach to use or how to actually build it in IPFire.
>>>>>>>>>> 
>>>>>>>>>> Definitely help required to know the correct way to go further with this.
>>>>>>>>>> 
>>>>>>>>> Having said I needed help, I thought I would have another go and copied the lines from the %build section of the fedora ca-certificate.spec that created the ca-bundle.trust.p11.kit
>>>>>>>>> 
>>>>>>>>> As I have p11-kit installed on my Arch Linux system I then ran the command
>>>>>>>>> 
>>>>>>>>> p11-kit extract --format=openssl-bundle --filter=certificates --overwrite --comment ~/openssl/ca-bundle.trust.crt
>>>>>>>>> 
>>>>>>>>> and I successfully created the ca-bundle.trust.crt which is needed by the IPFire ca-certificates lfs file.
>>>>>>>>> 
>>>>>>>>> I will now try and create a new build script that will do all the new type stuff and also install the p11-kit library files and see how things go.
>>>>>>>>> 
>>>>>>>>> :crossed_fingers:
>>>>>>>>> 
>>>>>>>>> Adolf
>>>>>>>>> 
>>>>>>>>>> Regards,
>>>>>>>>>> Adolf.
>>>>>>>>>> 
>>>>>>>>>>> -Michael
>>>>>>>>>>> 
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>> On 8 Aug 2021, at 14:47, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I had another go at the ca-certificates problem, the last barrier to getting rid of python2.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I found certdata2pem.py files from fedora and 2 from suse. I created build subdirectories for each version so I could just test running the build.sh file with each version of certdata2pem.py, including the IPFire current version after running through the 2to3 convertor.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> fedora
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> The fedora certdata2pem.py file runs successfully with python3 but has sed commands built into it which fail to find certain files. The sed commands are not in the IPFire version.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> The error message is
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>     -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', trust = ['CKA_TRUST_SERVER_AUTH', 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = ['serverAuth', 'emailProtection'], distrust = [], openssl-distrust = []
>>>>>>>>>>>>>> sed: can't read certs/*.crt: No such file or directory
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> suse
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> The first suse version runs successfully with python3 but also has the sed commands in it with the same error message.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> The second suse version runs successfully with python3, does not have the sed commands and completes the build.sh script with no errors. However this certdata2pem.py file has a section that is in the IPFire version completely missing.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> IPfire version after running through the 2to3 convertor
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> The following error message occurs
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148
>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>      File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in <module>
>>>>>>>>>>>>>> f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64)))
>>>>>>>>>>>>>>      File "/usr/lib/python3.9/base64.py", line 58, in b64encode
>>>>>>>>>>>>>>        encoded = binascii.b2a_base64(s, newline=False)
>>>>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> The section that is failing is the section that is missing in the 2nd suse version. There is an identical fwrite line at line 206 but that does not seem to flag up the same TypeError message.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> As the certdata2pem.py files from the other distributions vary significantly in content, with some having nearly double the number of lines of code, I think the best alternative is to fix the IPFire version so we stay consistent but I am unable to figure out how to fix the python code that is causing the "  TypeError: a bytes-like object is required, not 'str' " error message and need someone's help with that.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Let me know if there is any other information that I need to provide.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On 07/08/2021 15:54, Adolf Belka wrote:
>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On 04/08/2021 16:45, Michael Tremer wrote:
>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Great. This could also resolve Matthias’ problem with building frr.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Will now go back and have another go with spice-protocol.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Maybe it has a —-disable-python switch?
>>>>>>>>>>>>>>> I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
>>>>>>>>>>>>>>> Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> This now only leaves the ca-certificates script that needs to be updated to work with python3.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>>>>>>>>>>>>>>>> Hi Michael & all,
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Thank you for looking into this.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>>>>>>>>>>>>>>>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>>>>>>>>>>>>>>>>> I then commented ca-certificates out in make.sh and ran the build.
>>>>>>>>>>>>>>>>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>>>>>>>>>>>>>>>>> py-compile is python2 based and the build stopped because it could not find python
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I then commented out spice-protocol and ran the build.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>>>>>>>>>>>>>>>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I commented out frr and nothing else failed before cdrom was reached.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> So the packages that need to be made to work with python3 are
>>>>>>>>>>>>>>>>>> ca-certificates
>>>>>>>>>>>>>>>>>> spice-protocol
>>>>>>>>>>>>>>>>>> frr
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> ca-certificates was flagged up.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Adolf.
>>>>>>>> 
>>>> 
> -- 
> Sent from my laptop
> 

Re: First results from running build without python2

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 34913 bytes --]

Hi Michael,

On 20/08/2021 18:38, Michael Tremer wrote:
> Hello,
> 
>> On 20 Aug 2021, at 11:17, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>
>> Hi Michael and everyone,
>>
>> After disabling python on collectd, freeradius and alsa I have been able to successfully do a complete build with next with python2 removed.
> 
> Very good news!
> 
> I think you can submit those patches already because we can merge them straight away.
Patches submitted.
> 
>> I will now create patches for the above three programs and also for the fix for ca-certificates. With the other patches already submitted we can then decide in which Core Update to remove python2.
> 
> Agreed. If you like you can submit a patch that removes Python 2 from the build which we will merge at a later time.
I will submit a patch for this.
> 
> I would suggest Core Update 161.
> 
> This would be a great milestone and make the distribution smaller, and most importantly, remove another large piece of software that is unmaintained upstream. Thanks for your hard work on this!
I have enjoyed working on this, meeting the challenges and overcoming them with the support given to me. I have learnt a lot while doing this as well.

Now my fingers are crossed that what worked in my builds will work when merged but if not then I will work to fix any problems that might arise.

Regards,
Adolf.
> 
> Best,
> -Michael
> 
>> Regards,
>>
>> Adolf.
>>
>>
>> On 19/08/2021 22:54, Adolf Belka wrote:
>>> Hi Michael,
>>>
>>> On 19/08/2021 18:28, Adolf Belka wrote:
>>>> Hi Michael,
>>>>
>>>> On 19/08/2021 15:54, Michael Tremer wrote:
>>>>> Hello Adolf,
>>>>>
>>>>> This sounds very good.
>>>>>
>>>>>> On 18 Aug 2021, at 22:08, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> I have successfully built ca-certificates with the new p11-kit approach. The only difference is that the certificate data such as Version, Serial Number, Signature Algorithm etc is no longer included in the created bundle files. There is only the actual certificate itself.
>>>>>
>>>>> I do not exactly understand what you mean. Can you post a file system listing or something that would help me?The first certificate in the new ca-bundle.crt looks like the following:-
>>>>
>>>> # ACCVRAIZ1
>>>> -----BEGIN CERTIFICATE-----
>>>> MIIH0zCCBbugAwIBAgIIXsO3pkN/pOAwDQYJKoZIhvcNAQEFBQAwQjESMBAGA1UE
>>>> AwwJQUNDVlJBSVoxMRAwDgYDVQQLDAdQS0lBQ0NWMQ0wCwYDVQQKDARBQ0NWMQsw
>>>> CQYDVQQGEwJFUzAeFw0xMTA1MDUwOTM3MzdaFw0zMDEyMzEwOTM3MzdaMEIxEjAQ
>>>> BgNVBAMMCUFDQ1ZSQUlaMTEQMA4GA1UECwwHUEtJQUNDVjENMAsGA1UECgwEQUND
>>>> VjELMAkGA1UEBhMCRVMwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCb
>>>> qau/YUqXry+XZpp0X9DZlv3P4uRm7x8fRzPCRKPfmt4ftVTdFXxpNRFvu8gMjmoY
>>>> HtiP2Ra8EEg2XPBjs5BaXCQ316PWywlxufEBcoSwfdtNgM3802/J+Nq2DoLSRYWo
>>>> G2ioPej0RGy9ocLLA76MPhMAhN9KSMDjIgro6TenGEyxCQ0jVn8ETdkXhBilyNpA
>>>> lHPrzg5XPAOBOp0KoVdDaaxXbXmQeOW1tDvYvEyNKKGno6e6Ak4l0Squ7a4DIrhr
>>>> IA8wKFSVf+DuzgpmndFALW4ir50awQUZ0m/A8p/4e7MCQvtQqR0tkw8jq8bBD5L/
>>>> 0KIV9VMJcRz/RROE5iZe+OCIHAr8Fraocwa48GOEAqDGWuzndN9wrqODJerWx5eH
>>>> k6fGioozl2A3ED6XPm4pFdahD9GILBKfb6qkxkLrQaLjlUPTAYVtjrs78yM2x/47
>>>> 4KElB0iryYl0/wiPgL/AlmXz7uxLaL2diMMxs0Dx6M/2OLuc5NF/1OVYm3z61PMO
>>>> m3WR5LpSLhl+0fXNWhn8ugb2+1KoS5kE3fj5tItQo05iifCHJPqDQsGH+tUtKSpa
>>>> cXpkatcnYGMN285J9Y0fkIkyF/hzQ7jSWpOGYdbhdQrqeWZ2iE9x6wQl1gpaepPl
>>>> uUsXQA+xtrn13k/c4LOsOxFwYIRKQ26ZIMApcQrAZQIDAQABo4ICyzCCAscwfQYI
>>>> KwYBBQUHAQEEcTBvMEwGCCsGAQUFBzAChkBodHRwOi8vd3d3LmFjY3YuZXMvZmls
>>>> ZWFkbWluL0FyY2hpdm9zL2NlcnRpZmljYWRvcy9yYWl6YWNjdjEuY3J0MB8GCCsG
>>>> AQUFBzABhhNodHRwOi8vb2NzcC5hY2N2LmVzMB0GA1UdDgQWBBTSh7Tj3zcnk1X2
>>>> VuqB5TbMjB4/vTAPBgNVHRMBAf8EBTADAQH/MB8GA1UdIwQYMBaAFNKHtOPfNyeT
>>>> VfZW6oHlNsyMHj+9MIIBcwYDVR0gBIIBajCCAWYwggFiBgRVHSAAMIIBWDCCASIG
>>>> CCsGAQUFBwICMIIBFB6CARAAQQB1AHQAbwByAGkAZABhAGQAIABkAGUAIABDAGUA
>>>> cgB0AGkAZgBpAGMAYQBjAGkA8wBuACAAUgBhAO0AegAgAGQAZQAgAGwAYQAgAEEA
>>>> QwBDAFYAIAAoAEEAZwBlAG4AYwBpAGEAIABkAGUAIABUAGUAYwBuAG8AbABvAGcA
>>>> 7QBhACAAeQAgAEMAZQByAHQAaQBmAGkAYwBhAGMAaQDzAG4AIABFAGwAZQBjAHQA
>>>> cgDzAG4AaQBjAGEALAAgAEMASQBGACAAUQA0ADYAMAAxADEANQA2AEUAKQAuACAA
>>>> QwBQAFMAIABlAG4AIABoAHQAdABwADoALwAvAHcAdwB3AC4AYQBjAGMAdgAuAGUA
>>>> czAwBggrBgEFBQcCARYkaHR0cDovL3d3dy5hY2N2LmVzL2xlZ2lzbGFjaW9uX2Mu
>>>> aHRtMFUGA1UdHwROMEwwSqBIoEaGRGh0dHA6Ly93d3cuYWNjdi5lcy9maWxlYWRt
>>>> aW4vQXJjaGl2b3MvY2VydGlmaWNhZG9zL3JhaXphY2N2MV9kZXIuY3JsMA4GA1Ud
>>>> DwEB/wQEAwIBBjAXBgNVHREEEDAOgQxhY2N2QGFjY3YuZXMwDQYJKoZIhvcNAQEF
>>>> BQADggIBAJcxAp/n/UNnSEQU5CmH7UwoZtCPNdpNYbdKl02125DgBS4OxnnQ8pdp
>>>> D70ER9m+27Up2pvZrqmZ1dM8MJP1jaGo/AaNRPTKFpV8M9xii6g3+CfYCS0b78gU
>>>> JyCpZET/LtZ1qmxNYEAZSUNUY9rizLpm5U9EelvZaoErQNV/+QEnWCzI7UiRfD+m
>>>> AM/EKXMRNt6GGT6d7hmKG9Ww7Y49nCrADdg9ZuM8Db3VlFzi4qc1GwQA9j9ajepD
>>>> vV+JHanBsMyZ4k0ACtrJJ1vnE5Bc5PUzolVt3OAJTS+xJlsndQAJxGJ3KQhfnlms
>>>> tn6tn1QwIgPBHnFk/vk4CpYY3QIUrCPLBhwepH2NDd4nQeit2hW3sCPdK6jT2iWH
>>>> 7ehVRE2I9DZ+hJp4rPcOVkkO1jMl1oRQQmwgEh0q1b688nCBpHBgvgW1m54ERL5h
>>>> I6zppSSMEYCUWqKiuUnSwdzRp+0xESyeGabu4VXhwOrPDYTkF7eifKXeVSUG7szA
>>>> h1xA2syVP1XgNce4hL60Xc16gwFy7ofmXx2utYXGJt/mwZrpHgJHnyqobalbz+xF
>>>> d3+YJ5oyXSrjhO7FmGYvliAd3djDJ9ew+f7Zfc3Qn48LFFhRny+Lwzgt3uiP1o2H
>>>> pPVWQxaZLPSkVrQ0uGE3ycJYgBugl6H8WY3pEfbRD0tVNEYqi4Y7
>>>> -----END CERTIFICATE-----
>>>>
>>>> The version currently in place has the following content for the first certificate:-
>>>>
>>>> Certificate:
>>>>      Data:
>>>>          Version: 3 (0x2)
>>>>          Serial Number: 6828503384748696800 (0x5ec3b7a6437fa4e0)
>>>>          Signature Algorithm: sha1WithRSAEncryption
>>>>          Issuer: CN = ACCVRAIZ1, OU = PKIACCV, O = ACCV, C = ES
>>>>          Validity
>>>>              Not Before: May  5 09:37:37 2011 GMT
>>>>              Not After : Dec 31 09:37:37 2030 GMT
>>>>          Subject: CN = ACCVRAIZ1, OU = PKIACCV, O = ACCV, C = ES
>>>>          Subject Public Key Info:
>>>>              Public Key Algorithm: rsaEncryption
>>>>                  RSA Public-Key: (4096 bit)
>>>>                  Modulus:
>>>> 00:9b:a9:ab:bf:61:4a:97:af:2f:97:66:9a:74:5f:
>>>> d0:d9:96:fd:cf:e2:e4:66:ef:1f:1f:47:33:c2:44:
>>>> a3:df:9a:de:1f:b5:54:dd:15:7c:69:35:11:6f:bb:
>>>> c8:0c:8e:6a:18:1e:d8:8f:d9:16:bc:10:48:36:5c:
>>>> f0:63:b3:90:5a:5c:24:37:d7:a3:d6:cb:09:71:b9:
>>>> f1:01:72:84:b0:7d:db:4d:80:cd:fc:d3:6f:c9:f8:
>>>> da:b6:0e:82:d2:45:85:a8:1b:68:a8:3d:e8:f4:44:
>>>> 6c:bd:a1:c2:cb:03:be:8c:3e:13:00:84:df:4a:48:
>>>> c0:e3:22:0a:e8:e9:37:a7:18:4c:b1:09:0d:23:56:
>>>> 7f:04:4d:d9:17:84:18:a5:c8:da:40:94:73:eb:ce:
>>>> 0e:57:3c:03:81:3a:9d:0a:a1:57:43:69:ac:57:6d:
>>>> 79:90:78:e5:b5:b4:3b:d8:bc:4c:8d:28:a1:a7:a3:
>>>> a7:ba:02:4e:25:d1:2a:ae:ed:ae:03:22:b8:6b:20:
>>>> 0f:30:28:54:95:7f:e0:ee:ce:0a:66:9d:d1:40:2d:
>>>> 6e:22:af:9d:1a:c1:05:19:d2:6f:c0:f2:9f:f8:7b:
>>>> b3:02:42:fb:50:a9:1d:2d:93:0f:23:ab:c6:c1:0f:
>>>> 92:ff:d0:a2:15:f5:53:09:71:1c:ff:45:13:84:e6:
>>>> 26:5e:f8:e0:88:1c:0a:fc:16:b6:a8:73:06:b8:f0:
>>>> 63:84:02:a0:c6:5a:ec:e7:74:df:70:ae:a3:83:25:
>>>> ea:d6:c7:97:87:93:a7:c6:8a:8a:33:97:60:37:10:
>>>> 3e:97:3e:6e:29:15:d6:a1:0f:d1:88:2c:12:9f:6f:
>>>> aa:a4:c6:42:eb:41:a2:e3:95:43:d3:01:85:6d:8e:
>>>> bb:3b:f3:23:36:c7:fe:3b:e0:a1:25:07:48:ab:c9:
>>>> 89:74:ff:08:8f:80:bf:c0:96:65:f3:ee:ec:4b:68:
>>>> bd:9d:88:c3:31:b3:40:f1:e8:cf:f6:38:bb:9c:e4:
>>>> d1:7f:d4:e5:58:9b:7c:fa:d4:f3:0e:9b:75:91:e4:
>>>> ba:52:2e:19:7e:d1:f5:cd:5a:19:fc:ba:06:f6:fb:
>>>> 52:a8:4b:99:04:dd:f8:f9:b4:8b:50:a3:4e:62:89:
>>>> f0:87:24:fa:83:42:c1:87:fa:d5:2d:29:2a:5a:71:
>>>> 7a:64:6a:d7:27:60:63:0d:db:ce:49:f5:8d:1f:90:
>>>> 89:32:17:f8:73:43:b8:d2:5a:93:86:61:d6:e1:75:
>>>> 0a:ea:79:66:76:88:4f:71:eb:04:25:d6:0a:5a:7a:
>>>> 93:e5:b9:4b:17:40:0f:b1:b6:b9:f5:de:4f:dc:e0:
>>>> b3:ac:3b:11:70:60:84:4a:43:6e:99:20:c0:29:71:
>>>>                      0a:c0:65
>>>>                  Exponent: 65537 (0x10001)
>>>>          X509v3 extensions:
>>>>              Authority Information Access:
>>>>                  CA Issuers - URI:http://www.accv.es/fileadmin/Archivos/certificados/raizaccv1.crt
>>>>                  OCSP - URI:http://ocsp.accv.es
>>>>
>>>>              X509v3 Subject Key Identifier:
>>>> D2:87:B4:E3:DF:37:27:93:55:F6:56:EA:81:E5:36:CC:8C:1E:3F:BD
>>>>              X509v3 Basic Constraints: critical
>>>>                  CA:TRUE
>>>>              X509v3 Authority Key Identifier:
>>>> keyid:D2:87:B4:E3:DF:37:27:93:55:F6:56:EA:81:E5:36:CC:8C:1E:3F:BD
>>>>
>>>>              X509v3 Certificate Policies:
>>>>                  Policy: X509v3 Any Policy
>>>>                    User Notice:
>>>>                      Explicit Text:
>>>>                    CPS: http://www.accv.es/legislacion_c.htm
>>>>
>>>>              X509v3 CRL Distribution Points:
>>>>
>>>>                  Full Name:
>>>> URI:http://www.accv.es/fileadmin/Archivos/certificados/raizaccv1_der.crl
>>>>
>>>>              X509v3 Key Usage: critical
>>>>                  Certificate Sign, CRL Sign
>>>>              X509v3 Subject Alternative Name:
>>>>                  email:accv(a)accv.es
>>>>      Signature Algorithm: sha1WithRSAEncryption
>>>>           97:31:02:9f:e7:fd:43:67:48:44:14:e4:29:87:ed:4c:28:66:
>>>>           d0:8f:35:da:4d:61:b7:4a:97:4d:b5:db:90:e0:05:2e:0e:c6:
>>>>           79:d0:f2:97:69:0f:bd:04:47:d9:be:db:b5:29:da:9b:d9:ae:
>>>>           a9:99:d5:d3:3c:30:93:f5:8d:a1:a8:fc:06:8d:44:f4:ca:16:
>>>>           95:7c:33:dc:62:8b:a8:37:f8:27:d8:09:2d:1b:ef:c8:14:27:
>>>>           20:a9:64:44:ff:2e:d6:75:aa:6c:4d:60:40:19:49:43:54:63:
>>>>           da:e2:cc:ba:66:e5:4f:44:7a:5b:d9:6a:81:2b:40:d5:7f:f9:
>>>>           01:27:58:2c:c8:ed:48:91:7c:3f:a6:00:cf:c4:29:73:11:36:
>>>>           de:86:19:3e:9d:ee:19:8a:1b:d5:b0:ed:8e:3d:9c:2a:c0:0d:
>>>>           d8:3d:66:e3:3c:0d:bd:d5:94:5c:e2:e2:a7:35:1b:04:00:f6:
>>>>           3f:5a:8d:ea:43:bd:5f:89:1d:a9:c1:b0:cc:99:e2:4d:00:0a:
>>>>           da:c9:27:5b:e7:13:90:5c:e4:f5:33:a2:55:6d:dc:e0:09:4d:
>>>>           2f:b1:26:5b:27:75:00:09:c4:62:77:29:08:5f:9e:59:ac:b6:
>>>>           7e:ad:9f:54:30:22:03:c1:1e:71:64:fe:f9:38:0a:96:18:dd:
>>>>           02:14:ac:23:cb:06:1c:1e:a4:7d:8d:0d:de:27:41:e8:ad:da:
>>>>           15:b7:b0:23:dd:2b:a8:d3:da:25:87:ed:e8:55:44:4d:88:f4:
>>>>           36:7e:84:9a:78:ac:f7:0e:56:49:0e:d6:33:25:d6:84:50:42:
>>>>           6c:20:12:1d:2a:d5:be:bc:f2:70:81:a4:70:60:be:05:b5:9b:
>>>>           9e:04:44:be:61:23:ac:e9:a5:24:8c:11:80:94:5a:a2:a2:b9:
>>>>           49:d2:c1:dc:d1:a7:ed:31:11:2c:9e:19:a6:ee:e1:55:e1:c0:
>>>>           ea:cf:0d:84:e4:17:b7:a2:7c:a5:de:55:25:06:ee:cc:c0:87:
>>>>           5c:40:da:cc:95:3f:55:e0:35:c7:b8:84:be:b4:5d:cd:7a:83:
>>>>           01:72:ee:87:e6:5f:1d:ae:b5:85:c6:26:df:e6:c1:9a:e9:1e:
>>>>           02:47:9f:2a:a8:6d:a9:5b:cf:ec:45:77:7f:98:27:9a:32:5d:
>>>>           2a:e3:84:ee:c5:98:66:2f:96:20:1d:dd:d8:c3:27:d7:b0:f9:
>>>>           fe:d9:7d:cd:d0:9f:8f:0b:14:58:51:9f:2f:8b:c3:38:2d:de:
>>>>           e8:8f:d6:8d:87:a4:f5:56:43:16:99:2c:f4:a4:56:b4:34:b8:
>>>>           61:37:c9:c2:58:80:1b:a0:97:a1:fc:59:8d:e9:11:f6:d1:0f:
>>>>           4b:55:34:46:2a:8b:86:3b
>>>> -----BEGIN CERTIFICATE-----
>>>> MIIH0zCCBbugAwIBAgIIXsO3pkN/pOAwDQYJKoZIhvcNAQEFBQAwQjESMBAGA1UE
>>>> AwwJQUNDVlJBSVoxMRAwDgYDVQQLDAdQS0lBQ0NWMQ0wCwYDVQQKDARBQ0NWMQsw
>>>> CQYDVQQGEwJFUzAeFw0xMTA1MDUwOTM3MzdaFw0zMDEyMzEwOTM3MzdaMEIxEjAQ
>>>> BgNVBAMMCUFDQ1ZSQUlaMTEQMA4GA1UECwwHUEtJQUNDVjENMAsGA1UECgwEQUND
>>>> VjELMAkGA1UEBhMCRVMwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCb
>>>> qau/YUqXry+XZpp0X9DZlv3P4uRm7x8fRzPCRKPfmt4ftVTdFXxpNRFvu8gMjmoY
>>>> HtiP2Ra8EEg2XPBjs5BaXCQ316PWywlxufEBcoSwfdtNgM3802/J+Nq2DoLSRYWo
>>>> G2ioPej0RGy9ocLLA76MPhMAhN9KSMDjIgro6TenGEyxCQ0jVn8ETdkXhBilyNpA
>>>> lHPrzg5XPAOBOp0KoVdDaaxXbXmQeOW1tDvYvEyNKKGno6e6Ak4l0Squ7a4DIrhr
>>>> IA8wKFSVf+DuzgpmndFALW4ir50awQUZ0m/A8p/4e7MCQvtQqR0tkw8jq8bBD5L/
>>>> 0KIV9VMJcRz/RROE5iZe+OCIHAr8Fraocwa48GOEAqDGWuzndN9wrqODJerWx5eH
>>>> k6fGioozl2A3ED6XPm4pFdahD9GILBKfb6qkxkLrQaLjlUPTAYVtjrs78yM2x/47
>>>> 4KElB0iryYl0/wiPgL/AlmXz7uxLaL2diMMxs0Dx6M/2OLuc5NF/1OVYm3z61PMO
>>>> m3WR5LpSLhl+0fXNWhn8ugb2+1KoS5kE3fj5tItQo05iifCHJPqDQsGH+tUtKSpa
>>>> cXpkatcnYGMN285J9Y0fkIkyF/hzQ7jSWpOGYdbhdQrqeWZ2iE9x6wQl1gpaepPl
>>>> uUsXQA+xtrn13k/c4LOsOxFwYIRKQ26ZIMApcQrAZQIDAQABo4ICyzCCAscwfQYI
>>>> KwYBBQUHAQEEcTBvMEwGCCsGAQUFBzAChkBodHRwOi8vd3d3LmFjY3YuZXMvZmls
>>>> ZWFkbWluL0FyY2hpdm9zL2NlcnRpZmljYWRvcy9yYWl6YWNjdjEuY3J0MB8GCCsG
>>>> AQUFBzABhhNodHRwOi8vb2NzcC5hY2N2LmVzMB0GA1UdDgQWBBTSh7Tj3zcnk1X2
>>>> VuqB5TbMjB4/vTAPBgNVHRMBAf8EBTADAQH/MB8GA1UdIwQYMBaAFNKHtOPfNyeT
>>>> VfZW6oHlNsyMHj+9MIIBcwYDVR0gBIIBajCCAWYwggFiBgRVHSAAMIIBWDCCASIG
>>>> CCsGAQUFBwICMIIBFB6CARAAQQB1AHQAbwByAGkAZABhAGQAIABkAGUAIABDAGUA
>>>> cgB0AGkAZgBpAGMAYQBjAGkA8wBuACAAUgBhAO0AegAgAGQAZQAgAGwAYQAgAEEA
>>>> QwBDAFYAIAAoAEEAZwBlAG4AYwBpAGEAIABkAGUAIABUAGUAYwBuAG8AbABvAGcA
>>>> 7QBhACAAeQAgAEMAZQByAHQAaQBmAGkAYwBhAGMAaQDzAG4AIABFAGwAZQBjAHQA
>>>> cgDzAG4AaQBjAGEALAAgAEMASQBGACAAUQA0ADYAMAAxADEANQA2AEUAKQAuACAA
>>>> QwBQAFMAIABlAG4AIABoAHQAdABwADoALwAvAHcAdwB3AC4AYQBjAGMAdgAuAGUA
>>>> czAwBggrBgEFBQcCARYkaHR0cDovL3d3dy5hY2N2LmVzL2xlZ2lzbGFjaW9uX2Mu
>>>> aHRtMFUGA1UdHwROMEwwSqBIoEaGRGh0dHA6Ly93d3cuYWNjdi5lcy9maWxlYWRt
>>>> aW4vQXJjaGl2b3MvY2VydGlmaWNhZG9zL3JhaXphY2N2MV9kZXIuY3JsMA4GA1Ud
>>>> DwEB/wQEAwIBBjAXBgNVHREEEDAOgQxhY2N2QGFjY3YuZXMwDQYJKoZIhvcNAQEF
>>>> BQADggIBAJcxAp/n/UNnSEQU5CmH7UwoZtCPNdpNYbdKl02125DgBS4OxnnQ8pdp
>>>> D70ER9m+27Up2pvZrqmZ1dM8MJP1jaGo/AaNRPTKFpV8M9xii6g3+CfYCS0b78gU
>>>> JyCpZET/LtZ1qmxNYEAZSUNUY9rizLpm5U9EelvZaoErQNV/+QEnWCzI7UiRfD+m
>>>> AM/EKXMRNt6GGT6d7hmKG9Ww7Y49nCrADdg9ZuM8Db3VlFzi4qc1GwQA9j9ajepD
>>>> vV+JHanBsMyZ4k0ACtrJJ1vnE5Bc5PUzolVt3OAJTS+xJlsndQAJxGJ3KQhfnlms
>>>> tn6tn1QwIgPBHnFk/vk4CpYY3QIUrCPLBhwepH2NDd4nQeit2hW3sCPdK6jT2iWH
>>>> 7ehVRE2I9DZ+hJp4rPcOVkkO1jMl1oRQQmwgEh0q1b688nCBpHBgvgW1m54ERL5h
>>>> I6zppSSMEYCUWqKiuUnSwdzRp+0xESyeGabu4VXhwOrPDYTkF7eifKXeVSUG7szA
>>>> h1xA2syVP1XgNce4hL60Xc16gwFy7ofmXx2utYXGJt/mwZrpHgJHnyqobalbz+xF
>>>> d3+YJ5oyXSrjhO7FmGYvliAd3djDJ9ew+f7Zfc3Qn48LFFhRny+Lwzgt3uiP1o2H
>>>> pPVWQxaZLPSkVrQ0uGE3ycJYgBugl6H8WY3pEfbRD0tVNEYqi4Y7
>>>> -----END CERTIFICATE-----
>>>>
>>>> The bit between -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- is the same in both cases, it is all the other information that is missing.
>>>>>
>>>>>> This is also the case for the bundles that Arch Linux has created on my systems, that also uses the p11-kit approach.
>>>>>>
>>>>>> I did a diff on the current ca-bundle.crt/ca-bundle.trust.crt files and the ones created by the new approach as used by Fedora and all the same certificates are present, it is only the certificate data that is missed.
>>>>>>
>>>>>> There is a --comment option in the trust extract command that looks like it should add the comments that are in the .p11-kit file into the extracted bundle files but nothing changes if it is used or not. I will ask into the p11-kit git repository whether this should work or not.
>>>>>>
>>>>>>
>>>>>> I did a build just with the new ca-certificate files into Core Update 160 and then installed the created iso into a vm on my testbed system.
>>>>>>
>>>>>> Everything worked that I tested out but I am not 100% sure what needs to be exercised to test the ca-bundle.crt and ca-bundle.trust.crt files. If someone can give me some guidance on this then I can do that final test and submit a patch for this change.
>>>>>
>>>>> If you can download something with wget using HTTPS and the certificate can be validated, things should be working fine.
>>>> I will give that a try.
>>>>>
>>> I successfully downloaded clamav with wgat from an https url. I got the following messages which I think indicate it worked but there is also mention of http as well as https. Can you confirm if what I did was a good test result?
>>>
>>>
>>> wget https://www.clamav.net/downloads/production/clamav-0.103.3.tar.gz
>>> --2021-08-19 22:53:03-- https://www.clamav.net/downloads/production/clamav-0.103.3.tar.gz
>>> Resolving www.clamav.net... 104.16.218.84, 104.16.219.84
>>> Connecting to www.clamav.net|104.16.218.84|:443... connected.
>>> HTTP request sent, awaiting response... 302 Found
>>> Location: https://clamav-site.s3.amazonaws.com/production/release_files/files/000/000/612/original/clamav-0.103.3.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIY6OSGQFGUNJQ7GQ%2F20210819%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210819T205304Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=22909d2fe084a9cfcc50b41c0e59ed049089d26c42745801415c9cf697635051 [following]
>>> --2021-08-19 22:53:04-- https://clamav-site.s3.amazonaws.com/production/release_files/files/000/000/612/original/clamav-0.103.3.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIY6OSGQFGUNJQ7GQ%2F20210819%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210819T205304Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=22909d2fe084a9cfcc50b41c0e59ed049089d26c42745801415c9cf697635051
>>> Resolving clamav-site.s3.amazonaws.com... 52.216.224.232
>>> Connecting to clamav-site.s3.amazonaws.com|52.216.224.232|:443... connected.
>>> HTTP request sent, awaiting response... 200 OK
>>> Length: 13389239 (13M) [binary/octet-stream]
>>> Saving to: 'clamav-0.103.3.tar.gz'
>>>
>>> clamav-0.103.3.tar.gz   100%[==============================>] 12.77M  11.6MB/s    in 1.1s
>>>
>>> 2021-08-19 22:53:05 (11.6 MB/s) - 'clamav-0.103.3.tar.gz' saved [13389239/13389239]
>>>
>>> Regards,
>>>
>>> Adolf.
>>>
>>>
>>>>>> I also then did a test build with all python2 programs removed, including python2 itself and with the python 3 fixes for spice, frr and ca-certificates installed. The build for every program in the ipfire build section completed successfully but then I had a stop in the cdrom stage for collectd. It has a python2 plugin module defined which was of course missing. The module could be disabled in the rootfile or in the build config if the python plugin is not required for IPFire - I could not find any python modules defined.
>>>>>
>>>>> You can just pass “—-without-python” or “—-disable-python” and then the module won’t be built any more.
>>>>>
>>>>>> I have been able to successfully define the path to the python interpreter in the ./configure statement to point to the python3 program and collectd successfully built with this so if the python plugin is required then a python3 version can be created. I have not yet tested this in my vm but will do so.
>>>>>
>>>>> We do not use this module, so I suppose it isn’t worth building it in the first place.
>>>>>
>>>>>> I also found that freeradius has a python2 module defined. This can be disabled.
>>>> I will do that for collectd and freeradius then.
>>>>
>>>> I will also do it for alsa as that is the next one I found with a python2 library entry.
>>>>>
>>>>> Please do that. If someone needs the Python 3 module, we can still build it>
>>>>>> There is a python3 module available since 2.0.20. To use it with 3.0.21 (current version) you have to have --with-experimental-modules defined. I am looking at latest version 2.0.23 to see if that also needs the experimental modules enabled or if it has become part of the normal modules now and will test the build to see if it works successfully or not.
>>>>>
>>>>> Thank you for all the research that you invested into this. Looks like we are not far away to drop Python 2.
>>>> I also think we are very close now. Hopefully in time for Core Update 160.
>>>>
>>>> Regards,
>>>> Adolf.
>>>>>
>>>>> -Michael
>>>>>
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Adolf.
>>>>>>
>>>>>> On 12/08/2021 21:30, Adolf Belka wrote:
>>>>>>> Hi Michael,
>>>>>>>
>>>>>>> On 12/08/2021 21:23, Adolf Belka wrote:
>>>>>>>> Hi Michael,
>>>>>>>>
>>>>>>>> On 12/08/2021 15:38, Michael Tremer wrote:
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> Yes, this is the way to go :)
>>>>>>>>>
>>>>>>>>> Reach out here if you need any help.
>>>>>>>> I am trying to build p11-kit and it has flagged up libtasn1. It is an optional dependency and I can disable it but I would like to check if that is the correct thing to do or not.
>>>>>>>>
>>>>>>>> libtasn1 is said to be required to allow the trust policy module to be built as well as other code that interacts with certificates.
>>>>>>>> Is this something that would be needed for IPFire or can I disable it in the p11-kit build.
>>>>>>>>
>>>>>>> Ignore this question. I just did a quick test of disabling libtasn1 and p11-kit built ok but then my ca-certificates build failed due to the lack of the trust tool, so that makes it clear that it is required. I will add it in to the build.
>>>>>>>
>>>>>>> Slowly getting there.
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Adolf.
>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Adolf.
>>>>>>>>>
>>>>>>>>> -Michael
>>>>>>>>>
>>>>>>>>>> On 12 Aug 2021, at 13:10, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Michael,
>>>>>>>>>>
>>>>>>>>>> On 12/08/2021 13:36, Adolf Belka wrote:
>>>>>>>>>>> Hi Michael,
>>>>>>>>>>>
>>>>>>>>>>> On 12/08/2021 11:17, Michael Tremer wrote:
>>>>>>>>>>>> Hello,
>>>>>>>>>>>>
>>>>>>>>>>>>> On 11 Aug 2021, at 15:03, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Michael,
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 11/08/2021 12:43, Michael Tremer wrote:
>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>> Is this the one with the broken sed command?
>>>>>>>>>>>>>> https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2pem.py
>>>>>>>>>>>>> Yes, this is that one. Confirmed with a diff.
>>>>>>>>>>>>>> This should run if you execute it in the right directory:
>>>>>>>>>>>>>> pushd %{name}/certs
>>>>>>>>>>>>>>      pwd
>>>>>>>>>>>>>>      cp certdata.txt .
>>>>>>>>>>>>>>      python3 certdata2pem.py
>>>>>>>>>>>>>> popd
>>>>>>>>>>>>> I have just learnt about the pushd and popd commands. by doing a quick search. Never heard of them before.
>>>>>>>>>>>>
>>>>>>>>>>>> It is just a version of “cd” that remembers where it has been.
>>>>>>>>>>>>
>>>>>>>>>>>> So if you call “pushd some-directory”, then “popd” will bring you back to where you have been before.
>>>>>>>>>>>>
>>>>>>>>>>>> “cd -“ does the same as popd now.
>>>>>>>>>>>>
>>>>>>>>>>>>>> The fedora version no longer has the build.sh script.
>>>>>>>>>>>>> That was the bit I didn't realise.
>>>>>>>>>>>>
>>>>>>>>>>>> No problem.
>>>>>>>>>>>
>>>>>>>>>>> Not as simple as I hoped.
>>>>>>>>>>>
>>>>>>>>>>> The new certdata2pem.py script no longer creates .crt files but .tmp-p11-kit files but that is as far as that script goes.
>>>>>>>>>>>
>>>>>>>>>>> There is an update-ca-trust file in fedora which splits the various certs to their respective locations, with the openssl ones ending up in ca-bundle.trust.crt but this uses p11-kit. I suspect that it uses ca-bundle.trust.p11-kit generated later on in the %build section of the ca-certificate.spec file from the tmp-p11-kit files generated by the certdata2pem.py script. So it looks like p11-kit needs to be installed to make this work.
>>>>>>>>>>>
>>>>>>>>>>> Alternatively I have found the following in LFS.
>>>>>>>>>>> https://www.linuxfromscratch.org/blfs/view/svn/postlfs/make-ca.html
>>>>>>>>>>> which seems to also create the bundled cert file but also requiring p11-kit but then talking about different certdata.txt versions that have various tunings. The one from LFS comes from the mozilla release branch but is modified to something called the Mercurial revision, or the different ones shipped by RedHat or OpenSUSE which use the version that comes with NSS.
>>>>>>>>>>>
>>>>>>>>>>> All in all I am not sure which approach to use or how to actually build it in IPFire.
>>>>>>>>>>>
>>>>>>>>>>> Definitely help required to know the correct way to go further with this.
>>>>>>>>>>>
>>>>>>>>>> Having said I needed help, I thought I would have another go and copied the lines from the %build section of the fedora ca-certificate.spec that created the ca-bundle.trust.p11.kit
>>>>>>>>>>
>>>>>>>>>> As I have p11-kit installed on my Arch Linux system I then ran the command
>>>>>>>>>>
>>>>>>>>>> p11-kit extract --format=openssl-bundle --filter=certificates --overwrite --comment ~/openssl/ca-bundle.trust.crt
>>>>>>>>>>
>>>>>>>>>> and I successfully created the ca-bundle.trust.crt which is needed by the IPFire ca-certificates lfs file.
>>>>>>>>>>
>>>>>>>>>> I will now try and create a new build script that will do all the new type stuff and also install the p11-kit library files and see how things go.
>>>>>>>>>>
>>>>>>>>>> :crossed_fingers:
>>>>>>>>>>
>>>>>>>>>> Adolf
>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Adolf.
>>>>>>>>>>>
>>>>>>>>>>>> -Michael
>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>>> On 8 Aug 2021, at 14:47, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I had another go at the ca-certificates problem, the last barrier to getting rid of python2.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I found certdata2pem.py files from fedora and 2 from suse. I created build subdirectories for each version so I could just test running the build.sh file with each version of certdata2pem.py, including the IPFire current version after running through the 2to3 convertor.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> fedora
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The fedora certdata2pem.py file runs successfully with python3 but has sed commands built into it which fail to find certain files. The sed commands are not in the IPFire version.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The error message is
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>      -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', trust = ['CKA_TRUST_SERVER_AUTH', 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = ['serverAuth', 'emailProtection'], distrust = [], openssl-distrust = []
>>>>>>>>>>>>>>> sed: can't read certs/*.crt: No such file or directory
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> suse
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The first suse version runs successfully with python3 but also has the sed commands in it with the same error message.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The second suse version runs successfully with python3, does not have the sed commands and completes the build.sh script with no errors. However this certdata2pem.py file has a section that is in the IPFire version completely missing.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> IPfire version after running through the 2to3 convertor
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The following error message occurs
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148
>>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>>       File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in <module>
>>>>>>>>>>>>>>> f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64)))
>>>>>>>>>>>>>>>       File "/usr/lib/python3.9/base64.py", line 58, in b64encode
>>>>>>>>>>>>>>>         encoded = binascii.b2a_base64(s, newline=False)
>>>>>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The section that is failing is the section that is missing in the 2nd suse version. There is an identical fwrite line at line 206 but that does not seem to flag up the same TypeError message.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As the certdata2pem.py files from the other distributions vary significantly in content, with some having nearly double the number of lines of code, I think the best alternative is to fix the IPFire version so we stay consistent but I am unable to figure out how to fix the python code that is causing the "  TypeError: a bytes-like object is required, not 'str' " error message and need someone's help with that.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Let me know if there is any other information that I need to provide.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 07/08/2021 15:54, Adolf Belka wrote:
>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 04/08/2021 16:45, Michael Tremer wrote:
>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Great. This could also resolve Matthias’ problem with building frr.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Will now go back and have another go with spice-protocol.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Maybe it has a —-disable-python switch?
>>>>>>>>>>>>>>>> I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
>>>>>>>>>>>>>>>> Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This now only leaves the ca-certificates script that needs to be updated to work with python3.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>>>>>>>>>>>>>>>>> Hi Michael & all,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thank you for looking into this.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>>>>>>>>>>>>>>>>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>>>>>>>>>>>>>>>>>> I then commented ca-certificates out in make.sh and ran the build.
>>>>>>>>>>>>>>>>>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>>>>>>>>>>>>>>>>>> py-compile is python2 based and the build stopped because it could not find python
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I then commented out spice-protocol and ran the build.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>>>>>>>>>>>>>>>>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I commented out frr and nothing else failed before cdrom was reached.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> So the packages that need to be made to work with python3 are
>>>>>>>>>>>>>>>>>>> ca-certificates
>>>>>>>>>>>>>>>>>>> spice-protocol
>>>>>>>>>>>>>>>>>>> frr
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> ca-certificates was flagged up.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>
>>>>>
>> -- 
>> Sent from my laptop
>>
> 

Re: First results from running build without python2

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 36049 bytes --]

Hello,

> On 20 Aug 2021, at 21:17, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> 
> Hi Michael,
> 
> On 20/08/2021 18:38, Michael Tremer wrote:
>> Hello,
>>> On 20 Aug 2021, at 11:17, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>> 
>>> Hi Michael and everyone,
>>> 
>>> After disabling python on collectd, freeradius and alsa I have been able to successfully do a complete build with next with python2 removed.
>> Very good news!
>> I think you can submit those patches already because we can merge them straight away.
> Patches submitted.

Very good. I had a look through them and they look fine. It has made me very happy that we are now ready to remove Python 2.

>>> I will now create patches for the above three programs and also for the fix for ca-certificates. With the other patches already submitted we can then decide in which Core Update to remove python2.
>> Agreed. If you like you can submit a patch that removes Python 2 from the build which we will merge at a later time.
> I will submit a patch for this.

Cool. No hurry on this one.

>> I would suggest Core Update 161.
>> This would be a great milestone and make the distribution smaller, and most importantly, remove another large piece of software that is unmaintained upstream. Thanks for your hard work on this!
> I have enjoyed working on this, meeting the challenges and overcoming them with the support given to me. I have learnt a lot while doing this as well.

Judging from the patches this was definitely the most complicated stuff that you have submitted so far, but you did it absolutely fine. Cannot complain about a thing. I owe you a beer for this :)

> Now my fingers are crossed that what worked in my builds will work when merged but if not then I will work to fix any problems that might arise.

Hopefully not. Fingers crossed :)

-Michael

> Regards,
> Adolf.
>> Best,
>> -Michael
>>> Regards,
>>> 
>>> Adolf.
>>> 
>>> 
>>> On 19/08/2021 22:54, Adolf Belka wrote:
>>>> Hi Michael,
>>>> 
>>>> On 19/08/2021 18:28, Adolf Belka wrote:
>>>>> Hi Michael,
>>>>> 
>>>>> On 19/08/2021 15:54, Michael Tremer wrote:
>>>>>> Hello Adolf,
>>>>>> 
>>>>>> This sounds very good.
>>>>>> 
>>>>>>> On 18 Aug 2021, at 22:08, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>> 
>>>>>>> Hi All,
>>>>>>> 
>>>>>>> I have successfully built ca-certificates with the new p11-kit approach. The only difference is that the certificate data such as Version, Serial Number, Signature Algorithm etc is no longer included in the created bundle files. There is only the actual certificate itself.
>>>>>> 
>>>>>> I do not exactly understand what you mean. Can you post a file system listing or something that would help me?The first certificate in the new ca-bundle.crt looks like the following:-
>>>>> 
>>>>> # ACCVRAIZ1
>>>>> -----BEGIN CERTIFICATE-----
>>>>> MIIH0zCCBbugAwIBAgIIXsO3pkN/pOAwDQYJKoZIhvcNAQEFBQAwQjESMBAGA1UE
>>>>> AwwJQUNDVlJBSVoxMRAwDgYDVQQLDAdQS0lBQ0NWMQ0wCwYDVQQKDARBQ0NWMQsw
>>>>> CQYDVQQGEwJFUzAeFw0xMTA1MDUwOTM3MzdaFw0zMDEyMzEwOTM3MzdaMEIxEjAQ
>>>>> BgNVBAMMCUFDQ1ZSQUlaMTEQMA4GA1UECwwHUEtJQUNDVjENMAsGA1UECgwEQUND
>>>>> VjELMAkGA1UEBhMCRVMwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCb
>>>>> qau/YUqXry+XZpp0X9DZlv3P4uRm7x8fRzPCRKPfmt4ftVTdFXxpNRFvu8gMjmoY
>>>>> HtiP2Ra8EEg2XPBjs5BaXCQ316PWywlxufEBcoSwfdtNgM3802/J+Nq2DoLSRYWo
>>>>> G2ioPej0RGy9ocLLA76MPhMAhN9KSMDjIgro6TenGEyxCQ0jVn8ETdkXhBilyNpA
>>>>> lHPrzg5XPAOBOp0KoVdDaaxXbXmQeOW1tDvYvEyNKKGno6e6Ak4l0Squ7a4DIrhr
>>>>> IA8wKFSVf+DuzgpmndFALW4ir50awQUZ0m/A8p/4e7MCQvtQqR0tkw8jq8bBD5L/
>>>>> 0KIV9VMJcRz/RROE5iZe+OCIHAr8Fraocwa48GOEAqDGWuzndN9wrqODJerWx5eH
>>>>> k6fGioozl2A3ED6XPm4pFdahD9GILBKfb6qkxkLrQaLjlUPTAYVtjrs78yM2x/47
>>>>> 4KElB0iryYl0/wiPgL/AlmXz7uxLaL2diMMxs0Dx6M/2OLuc5NF/1OVYm3z61PMO
>>>>> m3WR5LpSLhl+0fXNWhn8ugb2+1KoS5kE3fj5tItQo05iifCHJPqDQsGH+tUtKSpa
>>>>> cXpkatcnYGMN285J9Y0fkIkyF/hzQ7jSWpOGYdbhdQrqeWZ2iE9x6wQl1gpaepPl
>>>>> uUsXQA+xtrn13k/c4LOsOxFwYIRKQ26ZIMApcQrAZQIDAQABo4ICyzCCAscwfQYI
>>>>> KwYBBQUHAQEEcTBvMEwGCCsGAQUFBzAChkBodHRwOi8vd3d3LmFjY3YuZXMvZmls
>>>>> ZWFkbWluL0FyY2hpdm9zL2NlcnRpZmljYWRvcy9yYWl6YWNjdjEuY3J0MB8GCCsG
>>>>> AQUFBzABhhNodHRwOi8vb2NzcC5hY2N2LmVzMB0GA1UdDgQWBBTSh7Tj3zcnk1X2
>>>>> VuqB5TbMjB4/vTAPBgNVHRMBAf8EBTADAQH/MB8GA1UdIwQYMBaAFNKHtOPfNyeT
>>>>> VfZW6oHlNsyMHj+9MIIBcwYDVR0gBIIBajCCAWYwggFiBgRVHSAAMIIBWDCCASIG
>>>>> CCsGAQUFBwICMIIBFB6CARAAQQB1AHQAbwByAGkAZABhAGQAIABkAGUAIABDAGUA
>>>>> cgB0AGkAZgBpAGMAYQBjAGkA8wBuACAAUgBhAO0AegAgAGQAZQAgAGwAYQAgAEEA
>>>>> QwBDAFYAIAAoAEEAZwBlAG4AYwBpAGEAIABkAGUAIABUAGUAYwBuAG8AbABvAGcA
>>>>> 7QBhACAAeQAgAEMAZQByAHQAaQBmAGkAYwBhAGMAaQDzAG4AIABFAGwAZQBjAHQA
>>>>> cgDzAG4AaQBjAGEALAAgAEMASQBGACAAUQA0ADYAMAAxADEANQA2AEUAKQAuACAA
>>>>> QwBQAFMAIABlAG4AIABoAHQAdABwADoALwAvAHcAdwB3AC4AYQBjAGMAdgAuAGUA
>>>>> czAwBggrBgEFBQcCARYkaHR0cDovL3d3dy5hY2N2LmVzL2xlZ2lzbGFjaW9uX2Mu
>>>>> aHRtMFUGA1UdHwROMEwwSqBIoEaGRGh0dHA6Ly93d3cuYWNjdi5lcy9maWxlYWRt
>>>>> aW4vQXJjaGl2b3MvY2VydGlmaWNhZG9zL3JhaXphY2N2MV9kZXIuY3JsMA4GA1Ud
>>>>> DwEB/wQEAwIBBjAXBgNVHREEEDAOgQxhY2N2QGFjY3YuZXMwDQYJKoZIhvcNAQEF
>>>>> BQADggIBAJcxAp/n/UNnSEQU5CmH7UwoZtCPNdpNYbdKl02125DgBS4OxnnQ8pdp
>>>>> D70ER9m+27Up2pvZrqmZ1dM8MJP1jaGo/AaNRPTKFpV8M9xii6g3+CfYCS0b78gU
>>>>> JyCpZET/LtZ1qmxNYEAZSUNUY9rizLpm5U9EelvZaoErQNV/+QEnWCzI7UiRfD+m
>>>>> AM/EKXMRNt6GGT6d7hmKG9Ww7Y49nCrADdg9ZuM8Db3VlFzi4qc1GwQA9j9ajepD
>>>>> vV+JHanBsMyZ4k0ACtrJJ1vnE5Bc5PUzolVt3OAJTS+xJlsndQAJxGJ3KQhfnlms
>>>>> tn6tn1QwIgPBHnFk/vk4CpYY3QIUrCPLBhwepH2NDd4nQeit2hW3sCPdK6jT2iWH
>>>>> 7ehVRE2I9DZ+hJp4rPcOVkkO1jMl1oRQQmwgEh0q1b688nCBpHBgvgW1m54ERL5h
>>>>> I6zppSSMEYCUWqKiuUnSwdzRp+0xESyeGabu4VXhwOrPDYTkF7eifKXeVSUG7szA
>>>>> h1xA2syVP1XgNce4hL60Xc16gwFy7ofmXx2utYXGJt/mwZrpHgJHnyqobalbz+xF
>>>>> d3+YJ5oyXSrjhO7FmGYvliAd3djDJ9ew+f7Zfc3Qn48LFFhRny+Lwzgt3uiP1o2H
>>>>> pPVWQxaZLPSkVrQ0uGE3ycJYgBugl6H8WY3pEfbRD0tVNEYqi4Y7
>>>>> -----END CERTIFICATE-----
>>>>> 
>>>>> The version currently in place has the following content for the first certificate:-
>>>>> 
>>>>> Certificate:
>>>>>     Data:
>>>>>         Version: 3 (0x2)
>>>>>         Serial Number: 6828503384748696800 (0x5ec3b7a6437fa4e0)
>>>>>         Signature Algorithm: sha1WithRSAEncryption
>>>>>         Issuer: CN = ACCVRAIZ1, OU = PKIACCV, O = ACCV, C = ES
>>>>>         Validity
>>>>>             Not Before: May  5 09:37:37 2011 GMT
>>>>>             Not After : Dec 31 09:37:37 2030 GMT
>>>>>         Subject: CN = ACCVRAIZ1, OU = PKIACCV, O = ACCV, C = ES
>>>>>         Subject Public Key Info:
>>>>>             Public Key Algorithm: rsaEncryption
>>>>>                 RSA Public-Key: (4096 bit)
>>>>>                 Modulus:
>>>>> 00:9b:a9:ab:bf:61:4a:97:af:2f:97:66:9a:74:5f:
>>>>> d0:d9:96:fd:cf:e2:e4:66:ef:1f:1f:47:33:c2:44:
>>>>> a3:df:9a:de:1f:b5:54:dd:15:7c:69:35:11:6f:bb:
>>>>> c8:0c:8e:6a:18:1e:d8:8f:d9:16:bc:10:48:36:5c:
>>>>> f0:63:b3:90:5a:5c:24:37:d7:a3:d6:cb:09:71:b9:
>>>>> f1:01:72:84:b0:7d:db:4d:80:cd:fc:d3:6f:c9:f8:
>>>>> da:b6:0e:82:d2:45:85:a8:1b:68:a8:3d:e8:f4:44:
>>>>> 6c:bd:a1:c2:cb:03:be:8c:3e:13:00:84:df:4a:48:
>>>>> c0:e3:22:0a:e8:e9:37:a7:18:4c:b1:09:0d:23:56:
>>>>> 7f:04:4d:d9:17:84:18:a5:c8:da:40:94:73:eb:ce:
>>>>> 0e:57:3c:03:81:3a:9d:0a:a1:57:43:69:ac:57:6d:
>>>>> 79:90:78:e5:b5:b4:3b:d8:bc:4c:8d:28:a1:a7:a3:
>>>>> a7:ba:02:4e:25:d1:2a:ae:ed:ae:03:22:b8:6b:20:
>>>>> 0f:30:28:54:95:7f:e0:ee:ce:0a:66:9d:d1:40:2d:
>>>>> 6e:22:af:9d:1a:c1:05:19:d2:6f:c0:f2:9f:f8:7b:
>>>>> b3:02:42:fb:50:a9:1d:2d:93:0f:23:ab:c6:c1:0f:
>>>>> 92:ff:d0:a2:15:f5:53:09:71:1c:ff:45:13:84:e6:
>>>>> 26:5e:f8:e0:88:1c:0a:fc:16:b6:a8:73:06:b8:f0:
>>>>> 63:84:02:a0:c6:5a:ec:e7:74:df:70:ae:a3:83:25:
>>>>> ea:d6:c7:97:87:93:a7:c6:8a:8a:33:97:60:37:10:
>>>>> 3e:97:3e:6e:29:15:d6:a1:0f:d1:88:2c:12:9f:6f:
>>>>> aa:a4:c6:42:eb:41:a2:e3:95:43:d3:01:85:6d:8e:
>>>>> bb:3b:f3:23:36:c7:fe:3b:e0:a1:25:07:48:ab:c9:
>>>>> 89:74:ff:08:8f:80:bf:c0:96:65:f3:ee:ec:4b:68:
>>>>> bd:9d:88:c3:31:b3:40:f1:e8:cf:f6:38:bb:9c:e4:
>>>>> d1:7f:d4:e5:58:9b:7c:fa:d4:f3:0e:9b:75:91:e4:
>>>>> ba:52:2e:19:7e:d1:f5:cd:5a:19:fc:ba:06:f6:fb:
>>>>> 52:a8:4b:99:04:dd:f8:f9:b4:8b:50:a3:4e:62:89:
>>>>> f0:87:24:fa:83:42:c1:87:fa:d5:2d:29:2a:5a:71:
>>>>> 7a:64:6a:d7:27:60:63:0d:db:ce:49:f5:8d:1f:90:
>>>>> 89:32:17:f8:73:43:b8:d2:5a:93:86:61:d6:e1:75:
>>>>> 0a:ea:79:66:76:88:4f:71:eb:04:25:d6:0a:5a:7a:
>>>>> 93:e5:b9:4b:17:40:0f:b1:b6:b9:f5:de:4f:dc:e0:
>>>>> b3:ac:3b:11:70:60:84:4a:43:6e:99:20:c0:29:71:
>>>>>                     0a:c0:65
>>>>>                 Exponent: 65537 (0x10001)
>>>>>         X509v3 extensions:
>>>>>             Authority Information Access:
>>>>>                 CA Issuers - URI:http://www.accv.es/fileadmin/Archivos/certificados/raizaccv1.crt
>>>>>                 OCSP - URI:http://ocsp.accv.es
>>>>> 
>>>>>             X509v3 Subject Key Identifier:
>>>>> D2:87:B4:E3:DF:37:27:93:55:F6:56:EA:81:E5:36:CC:8C:1E:3F:BD
>>>>>             X509v3 Basic Constraints: critical
>>>>>                 CA:TRUE
>>>>>             X509v3 Authority Key Identifier:
>>>>> keyid:D2:87:B4:E3:DF:37:27:93:55:F6:56:EA:81:E5:36:CC:8C:1E:3F:BD
>>>>> 
>>>>>             X509v3 Certificate Policies:
>>>>>                 Policy: X509v3 Any Policy
>>>>>                   User Notice:
>>>>>                     Explicit Text:
>>>>>                   CPS: http://www.accv.es/legislacion_c.htm
>>>>> 
>>>>>             X509v3 CRL Distribution Points:
>>>>> 
>>>>>                 Full Name:
>>>>> URI:http://www.accv.es/fileadmin/Archivos/certificados/raizaccv1_der.crl
>>>>> 
>>>>>             X509v3 Key Usage: critical
>>>>>                 Certificate Sign, CRL Sign
>>>>>             X509v3 Subject Alternative Name:
>>>>>                 email:accv(a)accv.es
>>>>>     Signature Algorithm: sha1WithRSAEncryption
>>>>>          97:31:02:9f:e7:fd:43:67:48:44:14:e4:29:87:ed:4c:28:66:
>>>>>          d0:8f:35:da:4d:61:b7:4a:97:4d:b5:db:90:e0:05:2e:0e:c6:
>>>>>          79:d0:f2:97:69:0f:bd:04:47:d9:be:db:b5:29:da:9b:d9:ae:
>>>>>          a9:99:d5:d3:3c:30:93:f5:8d:a1:a8:fc:06:8d:44:f4:ca:16:
>>>>>          95:7c:33:dc:62:8b:a8:37:f8:27:d8:09:2d:1b:ef:c8:14:27:
>>>>>          20:a9:64:44:ff:2e:d6:75:aa:6c:4d:60:40:19:49:43:54:63:
>>>>>          da:e2:cc:ba:66:e5:4f:44:7a:5b:d9:6a:81:2b:40:d5:7f:f9:
>>>>>          01:27:58:2c:c8:ed:48:91:7c:3f:a6:00:cf:c4:29:73:11:36:
>>>>>          de:86:19:3e:9d:ee:19:8a:1b:d5:b0:ed:8e:3d:9c:2a:c0:0d:
>>>>>          d8:3d:66:e3:3c:0d:bd:d5:94:5c:e2:e2:a7:35:1b:04:00:f6:
>>>>>          3f:5a:8d:ea:43:bd:5f:89:1d:a9:c1:b0:cc:99:e2:4d:00:0a:
>>>>>          da:c9:27:5b:e7:13:90:5c:e4:f5:33:a2:55:6d:dc:e0:09:4d:
>>>>>          2f:b1:26:5b:27:75:00:09:c4:62:77:29:08:5f:9e:59:ac:b6:
>>>>>          7e:ad:9f:54:30:22:03:c1:1e:71:64:fe:f9:38:0a:96:18:dd:
>>>>>          02:14:ac:23:cb:06:1c:1e:a4:7d:8d:0d:de:27:41:e8:ad:da:
>>>>>          15:b7:b0:23:dd:2b:a8:d3:da:25:87:ed:e8:55:44:4d:88:f4:
>>>>>          36:7e:84:9a:78:ac:f7:0e:56:49:0e:d6:33:25:d6:84:50:42:
>>>>>          6c:20:12:1d:2a:d5:be:bc:f2:70:81:a4:70:60:be:05:b5:9b:
>>>>>          9e:04:44:be:61:23:ac:e9:a5:24:8c:11:80:94:5a:a2:a2:b9:
>>>>>          49:d2:c1:dc:d1:a7:ed:31:11:2c:9e:19:a6:ee:e1:55:e1:c0:
>>>>>          ea:cf:0d:84:e4:17:b7:a2:7c:a5:de:55:25:06:ee:cc:c0:87:
>>>>>          5c:40:da:cc:95:3f:55:e0:35:c7:b8:84:be:b4:5d:cd:7a:83:
>>>>>          01:72:ee:87:e6:5f:1d:ae:b5:85:c6:26:df:e6:c1:9a:e9:1e:
>>>>>          02:47:9f:2a:a8:6d:a9:5b:cf:ec:45:77:7f:98:27:9a:32:5d:
>>>>>          2a:e3:84:ee:c5:98:66:2f:96:20:1d:dd:d8:c3:27:d7:b0:f9:
>>>>>          fe:d9:7d:cd:d0:9f:8f:0b:14:58:51:9f:2f:8b:c3:38:2d:de:
>>>>>          e8:8f:d6:8d:87:a4:f5:56:43:16:99:2c:f4:a4:56:b4:34:b8:
>>>>>          61:37:c9:c2:58:80:1b:a0:97:a1:fc:59:8d:e9:11:f6:d1:0f:
>>>>>          4b:55:34:46:2a:8b:86:3b
>>>>> -----BEGIN CERTIFICATE-----
>>>>> MIIH0zCCBbugAwIBAgIIXsO3pkN/pOAwDQYJKoZIhvcNAQEFBQAwQjESMBAGA1UE
>>>>> AwwJQUNDVlJBSVoxMRAwDgYDVQQLDAdQS0lBQ0NWMQ0wCwYDVQQKDARBQ0NWMQsw
>>>>> CQYDVQQGEwJFUzAeFw0xMTA1MDUwOTM3MzdaFw0zMDEyMzEwOTM3MzdaMEIxEjAQ
>>>>> BgNVBAMMCUFDQ1ZSQUlaMTEQMA4GA1UECwwHUEtJQUNDVjENMAsGA1UECgwEQUND
>>>>> VjELMAkGA1UEBhMCRVMwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCb
>>>>> qau/YUqXry+XZpp0X9DZlv3P4uRm7x8fRzPCRKPfmt4ftVTdFXxpNRFvu8gMjmoY
>>>>> HtiP2Ra8EEg2XPBjs5BaXCQ316PWywlxufEBcoSwfdtNgM3802/J+Nq2DoLSRYWo
>>>>> G2ioPej0RGy9ocLLA76MPhMAhN9KSMDjIgro6TenGEyxCQ0jVn8ETdkXhBilyNpA
>>>>> lHPrzg5XPAOBOp0KoVdDaaxXbXmQeOW1tDvYvEyNKKGno6e6Ak4l0Squ7a4DIrhr
>>>>> IA8wKFSVf+DuzgpmndFALW4ir50awQUZ0m/A8p/4e7MCQvtQqR0tkw8jq8bBD5L/
>>>>> 0KIV9VMJcRz/RROE5iZe+OCIHAr8Fraocwa48GOEAqDGWuzndN9wrqODJerWx5eH
>>>>> k6fGioozl2A3ED6XPm4pFdahD9GILBKfb6qkxkLrQaLjlUPTAYVtjrs78yM2x/47
>>>>> 4KElB0iryYl0/wiPgL/AlmXz7uxLaL2diMMxs0Dx6M/2OLuc5NF/1OVYm3z61PMO
>>>>> m3WR5LpSLhl+0fXNWhn8ugb2+1KoS5kE3fj5tItQo05iifCHJPqDQsGH+tUtKSpa
>>>>> cXpkatcnYGMN285J9Y0fkIkyF/hzQ7jSWpOGYdbhdQrqeWZ2iE9x6wQl1gpaepPl
>>>>> uUsXQA+xtrn13k/c4LOsOxFwYIRKQ26ZIMApcQrAZQIDAQABo4ICyzCCAscwfQYI
>>>>> KwYBBQUHAQEEcTBvMEwGCCsGAQUFBzAChkBodHRwOi8vd3d3LmFjY3YuZXMvZmls
>>>>> ZWFkbWluL0FyY2hpdm9zL2NlcnRpZmljYWRvcy9yYWl6YWNjdjEuY3J0MB8GCCsG
>>>>> AQUFBzABhhNodHRwOi8vb2NzcC5hY2N2LmVzMB0GA1UdDgQWBBTSh7Tj3zcnk1X2
>>>>> VuqB5TbMjB4/vTAPBgNVHRMBAf8EBTADAQH/MB8GA1UdIwQYMBaAFNKHtOPfNyeT
>>>>> VfZW6oHlNsyMHj+9MIIBcwYDVR0gBIIBajCCAWYwggFiBgRVHSAAMIIBWDCCASIG
>>>>> CCsGAQUFBwICMIIBFB6CARAAQQB1AHQAbwByAGkAZABhAGQAIABkAGUAIABDAGUA
>>>>> cgB0AGkAZgBpAGMAYQBjAGkA8wBuACAAUgBhAO0AegAgAGQAZQAgAGwAYQAgAEEA
>>>>> QwBDAFYAIAAoAEEAZwBlAG4AYwBpAGEAIABkAGUAIABUAGUAYwBuAG8AbABvAGcA
>>>>> 7QBhACAAeQAgAEMAZQByAHQAaQBmAGkAYwBhAGMAaQDzAG4AIABFAGwAZQBjAHQA
>>>>> cgDzAG4AaQBjAGEALAAgAEMASQBGACAAUQA0ADYAMAAxADEANQA2AEUAKQAuACAA
>>>>> QwBQAFMAIABlAG4AIABoAHQAdABwADoALwAvAHcAdwB3AC4AYQBjAGMAdgAuAGUA
>>>>> czAwBggrBgEFBQcCARYkaHR0cDovL3d3dy5hY2N2LmVzL2xlZ2lzbGFjaW9uX2Mu
>>>>> aHRtMFUGA1UdHwROMEwwSqBIoEaGRGh0dHA6Ly93d3cuYWNjdi5lcy9maWxlYWRt
>>>>> aW4vQXJjaGl2b3MvY2VydGlmaWNhZG9zL3JhaXphY2N2MV9kZXIuY3JsMA4GA1Ud
>>>>> DwEB/wQEAwIBBjAXBgNVHREEEDAOgQxhY2N2QGFjY3YuZXMwDQYJKoZIhvcNAQEF
>>>>> BQADggIBAJcxAp/n/UNnSEQU5CmH7UwoZtCPNdpNYbdKl02125DgBS4OxnnQ8pdp
>>>>> D70ER9m+27Up2pvZrqmZ1dM8MJP1jaGo/AaNRPTKFpV8M9xii6g3+CfYCS0b78gU
>>>>> JyCpZET/LtZ1qmxNYEAZSUNUY9rizLpm5U9EelvZaoErQNV/+QEnWCzI7UiRfD+m
>>>>> AM/EKXMRNt6GGT6d7hmKG9Ww7Y49nCrADdg9ZuM8Db3VlFzi4qc1GwQA9j9ajepD
>>>>> vV+JHanBsMyZ4k0ACtrJJ1vnE5Bc5PUzolVt3OAJTS+xJlsndQAJxGJ3KQhfnlms
>>>>> tn6tn1QwIgPBHnFk/vk4CpYY3QIUrCPLBhwepH2NDd4nQeit2hW3sCPdK6jT2iWH
>>>>> 7ehVRE2I9DZ+hJp4rPcOVkkO1jMl1oRQQmwgEh0q1b688nCBpHBgvgW1m54ERL5h
>>>>> I6zppSSMEYCUWqKiuUnSwdzRp+0xESyeGabu4VXhwOrPDYTkF7eifKXeVSUG7szA
>>>>> h1xA2syVP1XgNce4hL60Xc16gwFy7ofmXx2utYXGJt/mwZrpHgJHnyqobalbz+xF
>>>>> d3+YJ5oyXSrjhO7FmGYvliAd3djDJ9ew+f7Zfc3Qn48LFFhRny+Lwzgt3uiP1o2H
>>>>> pPVWQxaZLPSkVrQ0uGE3ycJYgBugl6H8WY3pEfbRD0tVNEYqi4Y7
>>>>> -----END CERTIFICATE-----
>>>>> 
>>>>> The bit between -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- is the same in both cases, it is all the other information that is missing.
>>>>>> 
>>>>>>> This is also the case for the bundles that Arch Linux has created on my systems, that also uses the p11-kit approach.
>>>>>>> 
>>>>>>> I did a diff on the current ca-bundle.crt/ca-bundle.trust.crt files and the ones created by the new approach as used by Fedora and all the same certificates are present, it is only the certificate data that is missed.
>>>>>>> 
>>>>>>> There is a --comment option in the trust extract command that looks like it should add the comments that are in the .p11-kit file into the extracted bundle files but nothing changes if it is used or not. I will ask into the p11-kit git repository whether this should work or not.
>>>>>>> 
>>>>>>> 
>>>>>>> I did a build just with the new ca-certificate files into Core Update 160 and then installed the created iso into a vm on my testbed system.
>>>>>>> 
>>>>>>> Everything worked that I tested out but I am not 100% sure what needs to be exercised to test the ca-bundle.crt and ca-bundle.trust.crt files. If someone can give me some guidance on this then I can do that final test and submit a patch for this change.
>>>>>> 
>>>>>> If you can download something with wget using HTTPS and the certificate can be validated, things should be working fine.
>>>>> I will give that a try.
>>>>>> 
>>>> I successfully downloaded clamav with wgat from an https url. I got the following messages which I think indicate it worked but there is also mention of http as well as https. Can you confirm if what I did was a good test result?
>>>> 
>>>> 
>>>> wget https://www.clamav.net/downloads/production/clamav-0.103.3.tar.gz
>>>> --2021-08-19 22:53:03-- https://www.clamav.net/downloads/production/clamav-0.103.3.tar.gz
>>>> Resolving www.clamav.net... 104.16.218.84, 104.16.219.84
>>>> Connecting to www.clamav.net|104.16.218.84|:443... connected.
>>>> HTTP request sent, awaiting response... 302 Found
>>>> Location: https://clamav-site.s3.amazonaws.com/production/release_files/files/000/000/612/original/clamav-0.103.3.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIY6OSGQFGUNJQ7GQ%2F20210819%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210819T205304Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=22909d2fe084a9cfcc50b41c0e59ed049089d26c42745801415c9cf697635051 [following]
>>>> --2021-08-19 22:53:04-- https://clamav-site.s3.amazonaws.com/production/release_files/files/000/000/612/original/clamav-0.103.3.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIY6OSGQFGUNJQ7GQ%2F20210819%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210819T205304Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=22909d2fe084a9cfcc50b41c0e59ed049089d26c42745801415c9cf697635051
>>>> Resolving clamav-site.s3.amazonaws.com... 52.216.224.232
>>>> Connecting to clamav-site.s3.amazonaws.com|52.216.224.232|:443... connected.
>>>> HTTP request sent, awaiting response... 200 OK
>>>> Length: 13389239 (13M) [binary/octet-stream]
>>>> Saving to: 'clamav-0.103.3.tar.gz'
>>>> 
>>>> clamav-0.103.3.tar.gz   100%[==============================>] 12.77M  11.6MB/s    in 1.1s
>>>> 
>>>> 2021-08-19 22:53:05 (11.6 MB/s) - 'clamav-0.103.3.tar.gz' saved [13389239/13389239]
>>>> 
>>>> Regards,
>>>> 
>>>> Adolf.
>>>> 
>>>> 
>>>>>>> I also then did a test build with all python2 programs removed, including python2 itself and with the python 3 fixes for spice, frr and ca-certificates installed. The build for every program in the ipfire build section completed successfully but then I had a stop in the cdrom stage for collectd. It has a python2 plugin module defined which was of course missing. The module could be disabled in the rootfile or in the build config if the python plugin is not required for IPFire - I could not find any python modules defined.
>>>>>> 
>>>>>> You can just pass “—-without-python” or “—-disable-python” and then the module won’t be built any more.
>>>>>> 
>>>>>>> I have been able to successfully define the path to the python interpreter in the ./configure statement to point to the python3 program and collectd successfully built with this so if the python plugin is required then a python3 version can be created. I have not yet tested this in my vm but will do so.
>>>>>> 
>>>>>> We do not use this module, so I suppose it isn’t worth building it in the first place.
>>>>>> 
>>>>>>> I also found that freeradius has a python2 module defined. This can be disabled.
>>>>> I will do that for collectd and freeradius then.
>>>>> 
>>>>> I will also do it for alsa as that is the next one I found with a python2 library entry.
>>>>>> 
>>>>>> Please do that. If someone needs the Python 3 module, we can still build it>
>>>>>>> There is a python3 module available since 2.0.20. To use it with 3.0.21 (current version) you have to have --with-experimental-modules defined. I am looking at latest version 2.0.23 to see if that also needs the experimental modules enabled or if it has become part of the normal modules now and will test the build to see if it works successfully or not.
>>>>>> 
>>>>>> Thank you for all the research that you invested into this. Looks like we are not far away to drop Python 2.
>>>>> I also think we are very close now. Hopefully in time for Core Update 160.
>>>>> 
>>>>> Regards,
>>>>> Adolf.
>>>>>> 
>>>>>> -Michael
>>>>>> 
>>>>>>> 
>>>>>>> Regards,
>>>>>>> 
>>>>>>> Adolf.
>>>>>>> 
>>>>>>> On 12/08/2021 21:30, Adolf Belka wrote:
>>>>>>>> Hi Michael,
>>>>>>>> 
>>>>>>>> On 12/08/2021 21:23, Adolf Belka wrote:
>>>>>>>>> Hi Michael,
>>>>>>>>> 
>>>>>>>>> On 12/08/2021 15:38, Michael Tremer wrote:
>>>>>>>>>> Hello,
>>>>>>>>>> 
>>>>>>>>>> Yes, this is the way to go :)
>>>>>>>>>> 
>>>>>>>>>> Reach out here if you need any help.
>>>>>>>>> I am trying to build p11-kit and it has flagged up libtasn1. It is an optional dependency and I can disable it but I would like to check if that is the correct thing to do or not.
>>>>>>>>> 
>>>>>>>>> libtasn1 is said to be required to allow the trust policy module to be built as well as other code that interacts with certificates.
>>>>>>>>> Is this something that would be needed for IPFire or can I disable it in the p11-kit build.
>>>>>>>>> 
>>>>>>>> Ignore this question. I just did a quick test of disabling libtasn1 and p11-kit built ok but then my ca-certificates build failed due to the lack of the trust tool, so that makes it clear that it is required. I will add it in to the build.
>>>>>>>> 
>>>>>>>> Slowly getting there.
>>>>>>>> 
>>>>>>>> Regards,
>>>>>>>> 
>>>>>>>> Adolf.
>>>>>>>> 
>>>>>>>>> Regards,
>>>>>>>>> Adolf.
>>>>>>>>>> 
>>>>>>>>>> -Michael
>>>>>>>>>> 
>>>>>>>>>>> On 12 Aug 2021, at 13:10, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hi Michael,
>>>>>>>>>>> 
>>>>>>>>>>> On 12/08/2021 13:36, Adolf Belka wrote:
>>>>>>>>>>>> Hi Michael,
>>>>>>>>>>>> 
>>>>>>>>>>>> On 12/08/2021 11:17, Michael Tremer wrote:
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On 11 Aug 2021, at 15:03, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Hi Michael,
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On 11/08/2021 12:43, Michael Tremer wrote:
>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>> Is this the one with the broken sed command?
>>>>>>>>>>>>>>> https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2pem.py
>>>>>>>>>>>>>> Yes, this is that one. Confirmed with a diff.
>>>>>>>>>>>>>>> This should run if you execute it in the right directory:
>>>>>>>>>>>>>>> pushd %{name}/certs
>>>>>>>>>>>>>>>     pwd
>>>>>>>>>>>>>>>     cp certdata.txt .
>>>>>>>>>>>>>>>     python3 certdata2pem.py
>>>>>>>>>>>>>>> popd
>>>>>>>>>>>>>> I have just learnt about the pushd and popd commands. by doing a quick search. Never heard of them before.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> It is just a version of “cd” that remembers where it has been.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> So if you call “pushd some-directory”, then “popd” will bring you back to where you have been before.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> “cd -“ does the same as popd now.
>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> The fedora version no longer has the build.sh script.
>>>>>>>>>>>>>> That was the bit I didn't realise.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> No problem.
>>>>>>>>>>>> 
>>>>>>>>>>>> Not as simple as I hoped.
>>>>>>>>>>>> 
>>>>>>>>>>>> The new certdata2pem.py script no longer creates .crt files but .tmp-p11-kit files but that is as far as that script goes.
>>>>>>>>>>>> 
>>>>>>>>>>>> There is an update-ca-trust file in fedora which splits the various certs to their respective locations, with the openssl ones ending up in ca-bundle.trust.crt but this uses p11-kit. I suspect that it uses ca-bundle.trust.p11-kit generated later on in the %build section of the ca-certificate.spec file from the tmp-p11-kit files generated by the certdata2pem.py script. So it looks like p11-kit needs to be installed to make this work.
>>>>>>>>>>>> 
>>>>>>>>>>>> Alternatively I have found the following in LFS.
>>>>>>>>>>>> https://www.linuxfromscratch.org/blfs/view/svn/postlfs/make-ca.html
>>>>>>>>>>>> which seems to also create the bundled cert file but also requiring p11-kit but then talking about different certdata.txt versions that have various tunings. The one from LFS comes from the mozilla release branch but is modified to something called the Mercurial revision, or the different ones shipped by RedHat or OpenSUSE which use the version that comes with NSS.
>>>>>>>>>>>> 
>>>>>>>>>>>> All in all I am not sure which approach to use or how to actually build it in IPFire.
>>>>>>>>>>>> 
>>>>>>>>>>>> Definitely help required to know the correct way to go further with this.
>>>>>>>>>>>> 
>>>>>>>>>>> Having said I needed help, I thought I would have another go and copied the lines from the %build section of the fedora ca-certificate.spec that created the ca-bundle.trust.p11.kit
>>>>>>>>>>> 
>>>>>>>>>>> As I have p11-kit installed on my Arch Linux system I then ran the command
>>>>>>>>>>> 
>>>>>>>>>>> p11-kit extract --format=openssl-bundle --filter=certificates --overwrite --comment ~/openssl/ca-bundle.trust.crt
>>>>>>>>>>> 
>>>>>>>>>>> and I successfully created the ca-bundle.trust.crt which is needed by the IPFire ca-certificates lfs file.
>>>>>>>>>>> 
>>>>>>>>>>> I will now try and create a new build script that will do all the new type stuff and also install the p11-kit library files and see how things go.
>>>>>>>>>>> 
>>>>>>>>>>> :crossed_fingers:
>>>>>>>>>>> 
>>>>>>>>>>> Adolf
>>>>>>>>>>> 
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Adolf.
>>>>>>>>>>>> 
>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>>>> On 8 Aug 2021, at 14:47, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I had another go at the ca-certificates problem, the last barrier to getting rid of python2.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I found certdata2pem.py files from fedora and 2 from suse. I created build subdirectories for each version so I could just test running the build.sh file with each version of certdata2pem.py, including the IPFire current version after running through the 2to3 convertor.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> fedora
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> The fedora certdata2pem.py file runs successfully with python3 but has sed commands built into it which fail to find certain files. The sed commands are not in the IPFire version.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> The error message is
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>     -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.6.247.235.85.79.181.237.tmp-p11-kit', trust = ['CKA_TRUST_SERVER_AUTH', 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust = ['serverAuth', 'emailProtection'], distrust = [], openssl-distrust = []
>>>>>>>>>>>>>>>> sed: can't read certs/*.crt: No such file or directory
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> suse
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> The first suse version runs successfully with python3 but also has the sed commands in it with the same error message.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> The second suse version runs successfully with python3, does not have the sed commands and completes the build.sh script with no errors. However this certdata2pem.py file has a section that is in the IPFire version completely missing.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> IPfire version after running through the 2to3 convertor
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> The following error message occurs
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148
>>>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>>>      File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in <module>
>>>>>>>>>>>>>>>> f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64)))
>>>>>>>>>>>>>>>>      File "/usr/lib/python3.9/base64.py", line 58, in b64encode
>>>>>>>>>>>>>>>>        encoded = binascii.b2a_base64(s, newline=False)
>>>>>>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> The section that is failing is the section that is missing in the 2nd suse version. There is an identical fwrite line at line 206 but that does not seem to flag up the same TypeError message.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> As the certdata2pem.py files from the other distributions vary significantly in content, with some having nearly double the number of lines of code, I think the best alternative is to fix the IPFire version so we stay consistent but I am unable to figure out how to fix the python code that is causing the "  TypeError: a bytes-like object is required, not 'str' " error message and need someone's help with that.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Let me know if there is any other information that I need to provide.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On 07/08/2021 15:54, Adolf Belka wrote:
>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On 04/08/2021 16:45, Michael Tremer wrote:
>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On 4 Aug 2021, at 13:40, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I have resolved the frr program build. The version currently in IPFire (6.0) only works with python2. Python3 support came in with version 7.4. I have now built frr with version 8.0 including libyang as a new dependency but only for the build, so nothing installed into IPFire itself, and that has successfully built without python2 being present.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Great. This could also resolve Matthias’ problem with building frr.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Will now go back and have another go with spice-protocol.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Maybe it has a —-disable-python switch?
>>>>>>>>>>>>>>>>> I just removed the line in the spice-protocol lfs that ran automake/py-compile on the python modules from spice.
>>>>>>>>>>>>>>>>> Spice and spice-protocol are present for qemu and with the py-compile line removed all three successfully built without python2 being present. I have submitted a patch for this combined with updating spice and spice-protocol, both from 2017.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> This now only leaves the ca-certificates script that needs to be updated to work with python3.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On 03/08/2021 23:38, Adolf Belka wrote:
>>>>>>>>>>>>>>>>>>>> Hi Michael & all,
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On 03/08/2021 17:11, Michael Tremer wrote:
>>>>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Thank you for looking into this.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> This is a third-party script that came from either Mozilla or RedHat. Maybe they have ported it. If not, it should not be rocket science to do it ourselves. If we do it, we should of course upstream it.
>>>>>>>>>>>>>>>>>>>> I found an updated script from fedora and gave that a try. This time the script went all the way through but then the build.sh script failed at the point where it should find all the .crt files in the certs directory and it came back and said there weren't any.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> However, can you comment out this package and continue the build? This should be required until you reach the cdrom stage.
>>>>>>>>>>>>>>>>>>>> I then commented ca-certificates out in make.sh and ran the build.
>>>>>>>>>>>>>>>>>>>> This time it stopped at spice-protocol which is an addon and uses the py-compile script that is in automake to compile some python modules.
>>>>>>>>>>>>>>>>>>>> py-compile is python2 based and the build stopped because it could not find python
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> There is a py_compile.py script that is python3 based but when I ran that in place of the py-compile script I got a Permission denied error when it tried to carry out the compile.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I then commented out spice-protocol and ran the build.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> It then failed on frr which did look for python3-config but then failed due to not finding python-config or pkg-config python
>>>>>>>>>>>>>>>>>>>> It looks like I should be able to tell it to use python3 in the ./configure
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I commented out frr and nothing else failed before cdrom was reached.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> So the packages that need to be made to work with python3 are
>>>>>>>>>>>>>>>>>>>> ca-certificates
>>>>>>>>>>>>>>>>>>>> spice-protocol
>>>>>>>>>>>>>>>>>>>> frr
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I also converted client175 with 2to3 converter and built it and installed the .ipfire package into a vm and successfully got the WUI page for Media Player IPFire to render. What I haven't tested yet is if the audio works. I will need to get audio set up in my vm to try that.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> If this is the only thing that flags up, we should port the script. If we find another, stronger reason to keep Python 2 around, we do not need to bother and can keep the script this way.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> -Michael
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> So with crda and the remaining python2 modules removed the question was if removing python2 from the build ran without any problem or if something was flagged up.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> ca-certificates was flagged up.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is not present. Running that script with python3 flags up some invalid syntax, unsurprisingly.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provide python3 compatibility. Unfortunately looking at the patch approx half could not be applied because the lines don't exist in the IPFire version of certdata2pem.py (sections to do with blacklisted certs)
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the build but it came up with the following error.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> TypeError: a bytes-like object is required, not 'str'
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> I don't know how to further move forward with this as I am totally unfamiliar with the python language.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Adolf.
>>>>>>>>>> 
>>>>>> 
>>> -- 
>>> Sent from my laptop
>>>