Problem building a python3 package in IPFire3

Problem building a python3 package in IPFire3

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 1646 bytes --]

Hi All,

I was looking at updating python3-markdown in IPFire3. However that 
version no longer has a setup.py file. It only has a pyproject.toml file.

In IPFire2 we dealt with that by installing python3-build which had a 
setup.py file and then using

python3 -m build \
               --no-isolation \
               --wheel

for the build part and

pip3 install    \
                 --no-build-isolation \
                 --root="/" \
                 --no-deps \
                 --ignore-installed \
                 dist/*.whl

for the install part.

However build is now version 1.0.3 and since version 1.0.0 it no longer 
has a setup.py file in it, only a pyproject.toml file and you can't use 
build to build build until build has been installed!!!!!

The instructions for build says that you have to use pip to install 
build, although it is not clear if that will work without internet.
The above build and install worked for IPFire2 because the build stage 
used python3-build to create a wheel from the source file and then pip 
to install the generated wheel file and none of that needed to load 
things from the internet.

I am not sure how to deal with this now. This will also be a problem for 
IPFire2 if we want to bring the various python3 packages up to date.

I had a look at what Arch Linux were doing to build the python3-build 
package and they have a makedepends of python-build to build 
python-build, which I don't understand how it would work.

Looking forward to any suggestions.

Regards,

Adolf.

-- 
Sent from my laptop

Re: Problem building a python3 package in IPFire3

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 3205 bytes --]

Hello,

> On 22 Sep 2023, at 15:00, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> 
> Hi All,
> 
> I was looking at updating python3-markdown in IPFire3. However that version no longer has a setup.py file. It only has a pyproject.toml file.
> 
> In IPFire2 we dealt with that by installing python3-build which had a setup.py file and then using
> 
> python3 -m build \
>               --no-isolation \
>               --wheel
> 
> for the build part and
> 
> pip3 install    \
>                 --no-build-isolation \
>                 --root="/" \
>                 --no-deps \
>                 --ignore-installed \
>                 dist/*.whl
> 
> for the install part.
> 
> However build is now version 1.0.3 and since version 1.0.0 it no longer has a setup.py file in it, only a pyproject.toml file and you can't use build to build build until build has been installed!!!!!
> 
> The instructions for build says that you have to use pip to install build, although it is not clear if that will work without internet.
> The above build and install worked for IPFire2 because the build stage used python3-build to create a wheel from the source file and then pip to install the generated wheel file and none of that needed to load things from the internet.

Well, this is a classic bootstrapping dilemma. We have this in many places: For example you will need a C compiler to build GCC, which is our C compiler. The Rust toolchain also has this problem and can not be bootstrapped without a rust compiler.

In IPFire 2 this is usually a big disaster and we used to build some packages with only some minimal parts, then build something else that depended on it, and the built the first package in all its full glory again. This is obviously complicated and breaks all the time - as well as wasting CPU cycles. Currently we only do this with libstdc++:

  https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=make.sh;h=acb6921403bf868128f1810a7e9bbafae13fc10e;hb=HEAD#l987

It is the GCC being built with PASS=L (L being short for libs).

Back to the original problem… This might be fun, but IPFire 3 should make this easier. You could start by building the older version of python3-build that still comes with setup.py. The update the package to its later version and make it depend on itself. That way, you have all the tools available to build the next version for python3-build. This normally works well except that we will have a nightmare when we bootstrap IPFire for yet another architecture, but as things stand right now, that won’t happen anytime soon.

> I am not sure how to deal with this now. This will also be a problem for IPFire2 if we want to bring the various python3 packages up to date.
> 
> I had a look at what Arch Linux were doing to build the python3-build package and they have a makedepends of python-build to build python-build, which I don't understand how it would work.

See above :) If this works, please submit two patches with one for each version so that we can recreate this in the build service.

-Michael

> Looking forward to any suggestions.
> 
> Regards,
> 
> Adolf.
> 
> -- 
> Sent from my laptop
>