From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer To: development@lists.ipfire.org Subject: Re: Question about updating python to 3.10 Date: Mon, 03 Jan 2022 10:52:29 +0000 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6731064674484360533==" List-Id: --===============6731064674484360533== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello, I am not sure if you already looked at what Arch is doing, but as far as I un= derstand it works like this: They build Python 3 without pip and setuptools (https://github.com/archlinux/= svntogit-packages/blob/912617035af94b1466595eedd6e6637676e2e733/trunk/PKGBUIL= D#L59). At this stage we might need to build setuptools separately. They use a tool called pyproject2setuppy (great name!) to build tomli (https:= //github.com/archlinux/svntogit-packages/blob/packages/python-tomli/trunk/PKG= BUILD). Setuptools-scm can then be built with tomli: https://github.com/archlinux/svn= togit-community/blob/packages/python-setuptools-scm/trunk/PKGBUILD Would this work or did I overlook anything? -Michael > On 2 Jan 2022, at 13:44, Adolf Belka wrote: >=20 > Hi Michael and All, >=20 > On 26/12/2021 21:55, Michael Tremer wrote: >> Merry Christmas, >>> On 26 Dec 2021, at 15:38, Adolf Belka wrote: >>>=20 >>> Hi Michael, >>>=20 >>> On 17/12/2021 12:41, Michael Tremer wrote: >>>> Hello Adolf, >>>>> On 17 Dec 2021, at 11:55, Adolf Belka wrote: >>>>>=20 >>>>> Hi All, >>>>>=20 >>>>>=20 >>>>> I am working on updating python from 3.8 to 3.10. Python itself is okay= after some other python related programs were also updated due to their olde= r versions not working with 3.10 >>>> Thank you for working on this. >>>>> I am working through all the programs that reference python-3.8 in thei= r rootfiles, or wherever, to change them to 3.10 >>>>>=20 >>>>> My question comes with regard to the changes to the rootfiles to ensure= they will be included properly in an update. >>>> We will have to ship everything that puts anything into /usr/lib*/python= 3.*/site-packages again with the release of Python 3.10. >>>> This isn=E2=80=99t fun, but not as much of a pain in the rear as shippin= g a new Perl release. >>>>> For the addons then when I change the rootfile I am also incrementing t= he PAK_VER number in the lfs files and that ensures that they will be properl= y updated. >>>> Yes. That is required for Pakfire to find a new version. >>>>> I don't know what to do for the core programs where I have updated the = rootfile. The lfs files for these programs don't have a PAK_VER number to inc= rement and the actual package version number is not being changed so I don't = know what I need to change in the lfs to ensure that the rootfile changes wil= l be properly included in the update? >>>> I would say committing everything into a =E2=80=9Cpython-3.10=E2=80=9D b= ranch and submitting it all as a patchset is fine. All packages that have bee= n touched in that branch will have to be shipped again which should be easy t= o identify if it all comes in one large branch. >>> Okay, that is clear then.Will follow that approach. >>>>> I would appreciate any help on what I need to do for these programs tha= t have had changes only in their rootfiles. >>>> Absolutely. Did you find anything that didn=E2=80=99t build yet? >>>=20 >>> I have not had any problem with python3.10 itself but I found some of the= older modules would not work with 3.10 so I decided to do a combined update = to 3.10 plus update modules to latest versions. >>>=20 >>> I found that after python3-setuptools-scm had run that the next module py= thon3-six failed to run due to missing the tomli library. I commented out six= and reran the build and then python3-dateutil had the same problem and then= python3-jmespath >> I completely lost track of what the status of setuptools/distutils and so = on is. This has changed multiple times and become independent/part of the dis= tribution again and so on. >>> After some investigation I have found that since version 6.0.0 of python3= -setuptools-scm, tomli has been a required library when it is being used by o= ther modules. So running version 6.0.0 of setuptools-scm worked with python3.= 10 and subsequent modules built successfully as tomli not needed. >>>=20 >>> So the obvious fix is to install tomli but that is when I ran into a prob= lem. tomli has no setup.py or setup.cfg files in its tarball. The only way to= build it is using pip. >> Err. Great. >>> Apparently, after some searching, setup.py has been deprecated as the rec= ommended install method and if setup.py doesn't work the usual fix suggestion= is to run pip install. >> Yes. But apparently the new system is deprecated, too. I really dislike th= e state of packaging systems that come with any language these days. >>> Having searched on pip for a while, it looks like builds using it can sti= ll be done from tarballs locally without needing to access the internet to do= wnload the modules. >> That is good! >>> As pip is installed as part of python3 i tried using that. >>>=20 >>> I used "python3 -m pip install --root=3D/ ." in place of >>> "python3 setup.py build >>> python3 setup.py install --root=3D/" >>> for the tomli source. >>>=20 >>> This failed because tomli requires flit_core to be available. >>>=20 >>> flit_core also has no setup.py and so I tried to build it with pip instal= l. This failed with the error message that tomli was required as a dependency= . So I added --no-deps to the flit_core pip install line and this time flit_c= ore successfully built but the tomli build still came back with not finding f= lit_core. >>>=20 >>> I have tried a variety of changes to the pip install line but none of the= m has been successful and it has become clear that I am out of my depth here = and need guidance rather than just trying anything I find from a google searc= h. I am not even really clear if pip install is the correct approach, except = if not then I have no idea how to progress. >>>=20 >>> Help urgently needed on how I should be approaching this. >> Circular dependencies can happen. This won=E2=80=99t be the first time and= it always is an utter nightmare with our buildsystem. >> This page states that pip comes by default with all binary packages: >> https://docs.python.org/3/installing/index.html >> Is there a chance that it is part of the distribution and can be enabled w= ith a configure switch? > I wasn't able to find any way to get it enabled that way. >=20 > I did find that pip install will not install if the package is already pres= ent. You have to add --upgrade to make it update any package already present.= So I tried that. Didn't work. >=20 > I found a section about needing to use bootstrapping if you want to install= tomli. Followed the information for creating a wheel file for flit_core and = then extracting that file into the site-packages directory. Tried that. Didn'= t work. >=20 > Rather than doing a complete build every time I changed to trying to build = flit_core followed by tomli in a ./make.sh shell environment. >=20 > Doing that I found that I had a pip install command that was creating the f= lit_core directory in site-packages and also including the .dist-info directo= ry. The info I found said that the .dist-info directory had to be there for p= ip to be able to find the flit_core package. >=20 > However the pip install of tomli still ends up not finding any flit_core pa= ckage. In the bootstrapping information it mentioned about changing the PYTHO= NPATH environment variable for find the flit_core package so I tried that in = the shell but it also made no difference. >=20 > I then tried running python3 -c 'import flit_core' in the tomli build shell= before doing the tomli pip install. The import command completed with no err= or but tomli still could not find any flit_core package. >=20 > End result of all my work is that flit_core has been built and installed in= to python3.10/site-packages but nothing seems to want to make the tomli build= find it. >=20 > Does anyone have any ideas on what command(s) I need to use in the tomli pi= p install to have the installed flit_core package visible. >=20 > Regards and a Happy and Healthy New Year to all, >=20 > Adolf. >=20 >> -Michael >>>=20 >>> Regards, >>> Adolf. >>>=20 >>> PS: I hope everyone had an enjoyable Christmas. >> I hope you did, too! >>>=20 >>>=20 >>>> -Michael >>>>>=20 >>>>> Regards, >>>>>=20 >>>>> Adolf. >>>>>=20 >>>>> --=20 >>>>> Sent from my laptop >>>>>=20 >>>=20 >>> --=20 >>> Sent from my laptop >=20 > --=20 > Sent from my laptop --===============6731064674484360533==--