From mboxrd@z Thu Jan 1 00:00:00 1970 From: Adolf Belka To: development@lists.ipfire.org Subject: Re: Question about updating python to 3.10 Date: Sun, 02 Jan 2022 14:44:11 +0100 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1264312327388955434==" List-Id: --===============1264312327388955434== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Michael and All, On 26/12/2021 21:55, Michael Tremer wrote: > Merry Christmas, >=20 >> On 26 Dec 2021, at 15:38, Adolf Belka wrote: >> >> Hi Michael, >> >> On 17/12/2021 12:41, Michael Tremer wrote: >>> Hello Adolf, >>>> On 17 Dec 2021, at 11:55, Adolf Belka wrote: >>>> >>>> Hi All, >>>> >>>> >>>> I am working on updating python from 3.8 to 3.10. Python itself is okay = after some other python related programs were also updated due to their older= versions not working with 3.10 >>> Thank you for working on this. >>>> I am working through all the programs that reference python-3.8 in their= rootfiles, or wherever, to change them to 3.10 >>>> >>>> My question comes with regard to the changes to the rootfiles to ensure = they will be included properly in an update. >>> We will have to ship everything that puts anything into /usr/lib*/python3= .*/site-packages again with the release of Python 3.10. >>> This isn=E2=80=99t fun, but not as much of a pain in the rear as shipping= a new Perl release. >>>> For the addons then when I change the rootfile I am also incrementing th= e PAK_VER number in the lfs files and that ensures that they will be properly= updated. >>> Yes. That is required for Pakfire to find a new version. >>>> I don't know what to do for the core programs where I have updated the r= ootfile. The lfs files for these programs don't have a PAK_VER number to incr= ement and the actual package version number is not being changed so I don't k= now what I need to change in the lfs to ensure that the rootfile changes will= be properly included in the update? >>> I would say committing everything into a =E2=80=9Cpython-3.10=E2=80=9D br= anch and submitting it all as a patchset is fine. All packages that have been= touched in that branch will have to be shipped again which should be easy to= identify if it all comes in one large branch. >> Okay, that is clear then.Will follow that approach. >>>> I would appreciate any help on what I need to do for these programs that= have had changes only in their rootfiles. >>> Absolutely. Did you find anything that didn=E2=80=99t build yet? >> >> I have not had any problem with python3.10 itself but I found some of the = older modules would not work with 3.10 so I decided to do a combined update t= o 3.10 plus update modules to latest versions. >> >> I found that after python3-setuptools-scm had run that the next module pyt= hon3-six failed to run due to missing the tomli library. I commented out six = and reran the build and then python3-dateutil had the same problem and then = python3-jmespath >=20 > I completely lost track of what the status of setuptools/distutils and so o= n is. This has changed multiple times and become independent/part of the dist= ribution again and so on. >=20 >> After some investigation I have found that since version 6.0.0 of python3-= setuptools-scm, tomli has been a required library when it is being used by ot= her modules. So running version 6.0.0 of setuptools-scm worked with python3.1= 0 and subsequent modules built successfully as tomli not needed. >> >> So the obvious fix is to install tomli but that is when I ran into a probl= em. tomli has no setup.py or setup.cfg files in its tarball. The only way to = build it is using pip. >=20 > Err. Great. >=20 >> Apparently, after some searching, setup.py has been deprecated as the reco= mmended install method and if setup.py doesn't work the usual fix suggestion = is to run pip install. >=20 > Yes. But apparently the new system is deprecated, too. I really dislike the= state of packaging systems that come with any language these days. >=20 >> Having searched on pip for a while, it looks like builds using it can stil= l be done from tarballs locally without needing to access the internet to dow= nload the modules. >=20 > That is good! >=20 >> As pip is installed as part of python3 i tried using that. >> >> I used "python3 -m pip install --root=3D/ ." in place of >> "python3 setup.py build >> python3 setup.py install --root=3D/" >> for the tomli source. >> >> This failed because tomli requires flit_core to be available. >> >> flit_core also has no setup.py and so I tried to build it with pip install= . This failed with the error message that tomli was required as a dependency.= So I added --no-deps to the flit_core pip install line and this time flit_co= re successfully built but the tomli build still came back with not finding fl= it_core. >> >> I have tried a variety of changes to the pip install line but none of them= has been successful and it has become clear that I am out of my depth here a= nd need guidance rather than just trying anything I find from a google search= . I am not even really clear if pip install is the correct approach, except i= f not then I have no idea how to progress. >> >> Help urgently needed on how I should be approaching this. >=20 > Circular dependencies can happen. This won=E2=80=99t be the first time and = it always is an utter nightmare with our buildsystem. >=20 > This page states that pip comes by default with all binary packages: >=20 > https://docs.python.org/3/installing/index.html >=20 > Is there a chance that it is part of the distribution and can be enabled wi= th a configure switch? I wasn't able to find any way to get it enabled that way. I did find that pip install will not install if the package is already=20 present. You have to add --upgrade to make it update any package already=20 present. So I tried that. Didn't work. I found a section about needing to use bootstrapping if you want to=20 install tomli. Followed the information for creating a wheel file for=20 flit_core and then extracting that file into the site-packages=20 directory. Tried that. Didn't work. Rather than doing a complete build every time I changed to trying to=20 build flit_core followed by tomli in a ./make.sh shell environment. Doing that I found that I had a pip install command that was creating=20 the flit_core directory in site-packages and also including the=20 .dist-info directory. The info I found said that the .dist-info=20 directory had to be there for pip to be able to find the flit_core package. However the pip install of tomli still ends up not finding any flit_core=20 package. In the bootstrapping information it mentioned about changing=20 the PYTHONPATH environment variable for find the flit_core package so I=20 tried that in the shell but it also made no difference. I then tried running python3 -c 'import flit_core' in the tomli build=20 shell before doing the tomli pip install. The import command completed=20 with no error but tomli still could not find any flit_core package. End result of all my work is that flit_core has been built and installed=20 into python3.10/site-packages but nothing seems to want to make the=20 tomli build find it. Does anyone have any ideas on what command(s) I need to use in the tomli=20 pip install to have the installed flit_core package visible. Regards and a Happy and Healthy New Year to all, Adolf. >=20 > -Michael >=20 >> >> Regards, >> Adolf. >> >> PS: I hope everyone had an enjoyable Christmas. >=20 > I hope you did, too! >=20 >=20 >> >> >>> -Michael >>>> >>>> Regards, >>>> >>>> Adolf. >>>> >>>> --=20 >>>> Sent from my laptop >>>> >> >> --=20 >> Sent from my laptop >=20 --=20 Sent from my laptop --===============1264312327388955434==--