From mboxrd@z Thu Jan 1 00:00:00 1970 From: Adolf Belka To: development@lists.ipfire.org Subject: Re: First results from running build without python2 Date: Sun, 08 Aug 2021 14:47:12 +0200 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6272055113155405585==" List-Id: --===============6272055113155405585== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi All, I had another go at the ca-certificates problem, the last barrier to getting = rid of python2. I found certdata2pem.py files from fedora and 2 from suse. I created build su= bdirectories for each version so I could just test running the build.sh file = with each version of certdata2pem.py, including the IPFire current version af= ter running through the 2to3 convertor. fedora The fedora certdata2pem.py file runs successfully with python3 but has sed co= mmands built into it which fail to find certain files. The sed commands are n= ot in the IPFire version. The error message is =C2=A0-> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55= .76.6.247.235.85.79.181.237.tmp-p11-kit', trust =3D ['CKA_TRUST_SERVER_AUTH',= 'CKA_TRUST_EMAIL_PROTECTION'], openssl-trust =3D ['serverAuth', 'emailProtec= tion'], distrust =3D [], openssl-distrust =3D [] sed: can't read certs/*.crt: No such file or directory suse The first suse version runs successfully with python3 but also has the sed co= mmands in it with the same error message. The second suse version runs successfully with python3, does not have the sed= commands and completes the build.sh script with no errors. However this cert= data2pem.py file has a section that is in the IPFire version completely missi= ng. IPfire version after running through the 2to3 convertor The following error message occurs producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148 Traceback (most recent call last): =C2=A0 File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progre= ss/python/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224= , in f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64))) =C2=A0 File "/usr/lib/python3.9/base64.py", line 58, in b64encode =C2=A0=C2=A0=C2=A0 encoded =3D binascii.b2a_base64(s, newline=3DFalse) TypeError: a bytes-like object is required, not 'str' The section that is failing is the section that is missing in the 2nd suse ve= rsion. There is an identical fwrite line at line 206 but that does not seem t= o flag up the same TypeError message. As the certdata2pem.py files from the other distributions vary significantly = in content, with some having nearly double the number of lines of code, I thi= nk the best alternative is to fix the IPFire version so we stay consistent bu= t I am unable to figure out how to fix the python code that is causing the "= =C2=A0 TypeError: a bytes-like object is required, not 'str' " error message = and need someone's help with that. Let me know if there is any other information that I need to provide. Regards, Adolf. On 07/08/2021 15:54, Adolf Belka wrote: > Hi All, > > On 04/08/2021 16:45, Michael Tremer wrote: >> Hello, >> >>> On 4 Aug 2021, at 13:40, Adolf Belka wrote: >>> >>> Hi All, >>> >>> I have resolved the frr program build. The version currently in IPFire (6= .0) only works with python2. Python3 support came in with version 7.4. I have= now built frr with version 8.0 including libyang as a new dependency but onl= y for the build, so nothing installed into IPFire itself, and that has succes= sfully built without python2 being present. >> >> Great. This could also resolve Matthias=E2=80=99 problem with building frr. >> >>> Will now go back and have another go with spice-protocol. >> >> Maybe it has a =E2=80=94-disable-python switch? > I just removed the line in the spice-protocol lfs that ran automake/py-comp= ile on the python modules from spice. > Spice and spice-protocol are present for qemu and with the py-compile line = removed all three successfully built without python2 being present. I have su= bmitted a patch for this combined with updating spice and spice-protocol, bot= h from 2017. > > This now only leaves the ca-certificates script that needs to be updated to= work with python3. > > Regards, > Adolf. >> >> -Michael >> >>> >>> Regards, >>> >>> Adolf. >>> >>> >>> On 03/08/2021 23:38, Adolf Belka wrote: >>>> Hi Michael & all, >>>> >>>> >>>> On 03/08/2021 17:11, Michael Tremer wrote: >>>>> Hello, >>>>> >>>>> Thank you for looking into this. >>>>> >>>>> This is a third-party script that came from either Mozilla or RedHat. M= aybe they have ported it. If not, it should not be rocket science to do it ou= rselves. If we do it, we should of course upstream it. >>>> I found an updated script from fedora and gave that a try. This time the= script went all the way through but then the build.sh script failed at the p= oint where it should find all the .crt files in the certs directory and it ca= me back and said there weren't any. >>>>> >>>>> However, can you comment out this package and continue the build? This = should be required until you reach the cdrom stage. >>>> I then commented ca-certificates out in make.sh and ran the build. >>>> This time it stopped at spice-protocol which is an addon and uses the py= -compile script that is in automake to compile some python modules. >>>> py-compile is python2 based and the build stopped because it could not f= ind python >>>> >>>> There is a py_compile.py script that is python3 based but when I ran tha= t in place of the py-compile script I got a Permission denied error when it t= ried to carry out the compile. >>>> >>>> I then commented out spice-protocol and ran the build. >>>> >>>> It then failed on frr which did look for python3-config but then failed = due to not finding python-config or pkg-config python >>>> It looks like I should be able to tell it to use python3 in the ./config= ure >>>> >>>> I commented out frr and nothing else failed before cdrom was reached. >>>> >>>> So the packages that need to be made to work with python3 are >>>> ca-certificates >>>> spice-protocol >>>> frr >>>> >>>> >>>> I also converted client175 with 2to3 converter and built it and installe= d the .ipfire package into a vm and successfully got the WUI page for Media P= layer IPFire to render. What I haven't tested yet is if the audio works. I wi= ll need to get audio set up in my vm to try that. >>>> >>>> Regards, >>>> Adolf. >>>> >>>>> >>>>> If this is the only thing that flags up, we should port the script. If = we find another, stronger reason to keep Python 2 around, we do not need to b= other and can keep the script this way. >>>>> >>>>> -Michael >>>>> >>>>>> On 3 Aug 2021, at 13:31, Adolf Belka wrote: >>>>>> >>>>>> Hi All, >>>>>> >>>>>> So with crda and the remaining python2 modules removed the question wa= s if removing python2 from the build ran without any problem or if something = was flagged up. >>>>>> >>>>>> >>>>>> ca-certificates was flagged up. >>>>>> >>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is = not present. Running that script with python3 flags up some invalid syntax, u= nsurprisingly. >>>>>> >>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provid= e python3 compatibility. Unfortunately looking at the patch approx half could= not be applied because the lines don't exist in the IPFire version of certda= ta2pem.py (sections to do with blacklisted certs) >>>>>> >>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in the= build but it came up with the following error. >>>>>> >>>>>> TypeError: a bytes-like object is required, not 'str' >>>>>> >>>>>> >>>>>> I don't know how to further move forward with this as I am totally unf= amiliar with the python language. >>>>>> >>>>>> >>>>>> Regards, >>>>>> >>>>>> Adolf. >>>>>> >>>>>> >>>>> >> --===============6272055113155405585==--