From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer To: development@lists.ipfire.org Subject: Re: First results from running build without python2 Date: Wed, 11 Aug 2021 12:43:18 +0200 Message-ID: <15B4DB6B-5FD9-4256-9026-A59E936CDF8B@ipfire.org> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7622099684261002413==" List-Id: --===============7622099684261002413== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello, Is this the one with the broken sed command? https://src.fedoraproject.org/rpms/ca-certificates/blob/rawhide/f/certdata2= pem.py This should run if you execute it in the right directory: pushd %{name}/certs pwd cp certdata.txt . python3 certdata2pem.py popd The fedora version no longer has the build.sh script. -Michael > On 8 Aug 2021, at 14:47, Adolf Belka wrote: >=20 > Hi All, >=20 > I had another go at the ca-certificates problem, the last barrier to gettin= g rid of python2. >=20 > I found certdata2pem.py files from fedora and 2 from suse. I created build = subdirectories for each version so I could just test running the build.sh fil= e with each version of certdata2pem.py, including the IPFire current version = after running through the 2to3 convertor. >=20 > fedora >=20 > The fedora certdata2pem.py file runs successfully with python3 but has sed = commands built into it which fail to find certain files. The sed commands are= not in the IPFire version. >=20 > The error message is >=20 > -> written as 'Certum_Trusted_Root_CA:2.16.30.191.89.80.184.201.128.55.76.= 6.247.235.85.79.181.237.tmp-p11-kit', trust =3D ['CKA_TRUST_SERVER_AUTH', 'CK= A_TRUST_EMAIL_PROTECTION'], openssl-trust =3D ['serverAuth', 'emailProtection= '], distrust =3D [], openssl-distrust =3D [] > sed: can't read certs/*.crt: No such file or directory >=20 >=20 > suse >=20 > The first suse version runs successfully with python3 but also has the sed = commands in it with the same error message. >=20 > The second suse version runs successfully with python3, does not have the s= ed commands and completes the build.sh script with no errors. However this ce= rtdata2pem.py file has a section that is in the IPFire version completely mis= sing. >=20 >=20 > IPfire version after running through the 2to3 convertor >=20 > The following error message occurs >=20 > producing trust for "GlobalSign Root CA"2.11.4.0.0.0.0.1.21.75.90.195.148 > Traceback (most recent call last): > File "/mnt/File_Server/Computers/Linux/ipfire/sandbox/patch in progress/p= ython/ca-certificates/orig-2to3-build/certs/../certdata2pem.py", line 224, in= > f.write("\n".join(textwrap.wrap(base64.b64encode(obj['CKA_VALUE']), 64))) > File "/usr/lib/python3.9/base64.py", line 58, in b64encode > encoded =3D binascii.b2a_base64(s, newline=3DFalse) > TypeError: a bytes-like object is required, not 'str' >=20 > The section that is failing is the section that is missing in the 2nd suse = version. There is an identical fwrite line at line 206 but that does not seem= to flag up the same TypeError message. >=20 >=20 > As the certdata2pem.py files from the other distributions vary significantl= y in content, with some having nearly double the number of lines of code, I t= hink the best alternative is to fix the IPFire version so we stay consistent = but I am unable to figure out how to fix the python code that is causing the = " TypeError: a bytes-like object is required, not 'str' " error message and = need someone's help with that. >=20 > Let me know if there is any other information that I need to provide. >=20 >=20 > Regards, >=20 > Adolf. >=20 >=20 > On 07/08/2021 15:54, Adolf Belka wrote: >> Hi All, >>=20 >> On 04/08/2021 16:45, Michael Tremer wrote: >>> Hello, >>>=20 >>>> On 4 Aug 2021, at 13:40, Adolf Belka wrote: >>>>=20 >>>> Hi All, >>>>=20 >>>> I have resolved the frr program build. The version currently in IPFire (= 6.0) only works with python2. Python3 support came in with version 7.4. I hav= e now built frr with version 8.0 including libyang as a new dependency but on= ly for the build, so nothing installed into IPFire itself, and that has succe= ssfully built without python2 being present. >>>=20 >>> Great. This could also resolve Matthias=E2=80=99 problem with building fr= r. >>>=20 >>>> Will now go back and have another go with spice-protocol. >>>=20 >>> Maybe it has a =E2=80=94-disable-python switch? >> I just removed the line in the spice-protocol lfs that ran automake/py-com= pile on the python modules from spice. >> Spice and spice-protocol are present for qemu and with the py-compile line= removed all three successfully built without python2 being present. I have s= ubmitted a patch for this combined with updating spice and spice-protocol, bo= th from 2017. >>=20 >> This now only leaves the ca-certificates script that needs to be updated t= o work with python3. >>=20 >> Regards, >> Adolf. >>>=20 >>> -Michael >>>=20 >>>>=20 >>>> Regards, >>>>=20 >>>> Adolf. >>>>=20 >>>>=20 >>>> On 03/08/2021 23:38, Adolf Belka wrote: >>>>> Hi Michael & all, >>>>>=20 >>>>>=20 >>>>> On 03/08/2021 17:11, Michael Tremer wrote: >>>>>> Hello, >>>>>>=20 >>>>>> Thank you for looking into this. >>>>>>=20 >>>>>> This is a third-party script that came from either Mozilla or RedHat. = Maybe they have ported it. If not, it should not be rocket science to do it o= urselves. If we do it, we should of course upstream it. >>>>> I found an updated script from fedora and gave that a try. This time th= e script went all the way through but then the build.sh script failed at the = point where it should find all the .crt files in the certs directory and it c= ame back and said there weren't any. >>>>>>=20 >>>>>> However, can you comment out this package and continue the build? This= should be required until you reach the cdrom stage. >>>>> I then commented ca-certificates out in make.sh and ran the build. >>>>> This time it stopped at spice-protocol which is an addon and uses the p= y-compile script that is in automake to compile some python modules. >>>>> py-compile is python2 based and the build stopped because it could not = find python >>>>>=20 >>>>> There is a py_compile.py script that is python3 based but when I ran th= at in place of the py-compile script I got a Permission denied error when it = tried to carry out the compile. >>>>>=20 >>>>> I then commented out spice-protocol and ran the build. >>>>>=20 >>>>> It then failed on frr which did look for python3-config but then failed= due to not finding python-config or pkg-config python >>>>> It looks like I should be able to tell it to use python3 in the ./confi= gure >>>>>=20 >>>>> I commented out frr and nothing else failed before cdrom was reached. >>>>>=20 >>>>> So the packages that need to be made to work with python3 are >>>>> ca-certificates >>>>> spice-protocol >>>>> frr >>>>>=20 >>>>>=20 >>>>> I also converted client175 with 2to3 converter and built it and install= ed the .ipfire package into a vm and successfully got the WUI page for Media = Player IPFire to render. What I haven't tested yet is if the audio works. I w= ill need to get audio set up in my vm to try that. >>>>>=20 >>>>> Regards, >>>>> Adolf. >>>>>=20 >>>>>>=20 >>>>>> If this is the only thing that flags up, we should port the script. If= we find another, stronger reason to keep Python 2 around, we do not need to = bother and can keep the script this way. >>>>>>=20 >>>>>> -Michael >>>>>>=20 >>>>>>> On 3 Aug 2021, at 13:31, Adolf Belka wrote: >>>>>>>=20 >>>>>>> Hi All, >>>>>>>=20 >>>>>>> So with crda and the remaining python2 modules removed the question w= as if removing python2 from the build ran without any problem or if something= was flagged up. >>>>>>>=20 >>>>>>>=20 >>>>>>> ca-certificates was flagged up. >>>>>>>=20 >>>>>>> There is a python2 script, certdata2pem.py, which fails if python2 is= not present. Running that script with python3 flags up some invalid syntax, = unsurprisingly. >>>>>>>=20 >>>>>>> I found some patches in Debian from 2015 for certdata2pem.py to provi= de python3 compatibility. Unfortunately looking at the patch approx half coul= d not be applied because the lines don't exist in the IPFire version of certd= ata2pem.py (sections to do with blacklisted certs) >>>>>>>=20 >>>>>>> I then ran the 2to3 converter on certdata2pem.py and tried that in th= e build but it came up with the following error. >>>>>>>=20 >>>>>>> TypeError: a bytes-like object is required, not 'str' >>>>>>>=20 >>>>>>>=20 >>>>>>> I don't know how to further move forward with this as I am totally un= familiar with the python language. >>>>>>>=20 >>>>>>>=20 >>>>>>> Regards, >>>>>>>=20 >>>>>>> Adolf. >>>>>>>=20 >>>>>>>=20 >>>>>>=20 >>>=20 --===============7622099684261002413==--