Re: Upload libloc to Debian

[Michael Tremer <michael.tremer@ipfire.org>]

Hello Valters,

> On 27 Mar 2025, at 00:25, Valters Jansons <valter.jansons@gmail.com> wrote:
> 
> On Wed, Mar 26, 2025 at 10:30 AM Hans-Christoph Steiner
> <hans@guardianproject.info> wrote:
>> 
>> Valters Jansons:
>> 
>> This is great!  One workflow we could use is adding you to the libloc and
>> libloc-location projects on salsa, then you could review and merge changes
>> there.  Then you'd just need Jochen and/or I for reviewing and uploading
>> releases to Debian.  How does that sound?
> 
> Hi Hans-Christoph,
> 
> I'm not a part of the IPFire/libloc team, however I would be happy to
> join as a maintainer for the Debian package.
> 
> I have historically maintained an Ubuntu PPA for use in my old
> company, and in other projects afterwards on Ubuntu 22.04 and older
> (before the Debian Universe package). I may have gaps in how packaging
> is expected, but overall I feel comfortable in moving the process
> forward.
> 
>> Towards that end, I made a merge request to review:
>> https://salsa.debian.org/debian/libloc-database/-/merge_requests/3
> 
> I am wondering if the d/watch mangle rule should be changed to follow
> the same sed logic as in d/rules. Left a comment on the PR and can be
> discussed there too. Overall, it looks like a good, helpful change.
> 
> On Wed, Mar 26, 2025 at 12:17 PM Michael Tremer
> <michael.tremer@ipfire.org> wrote:
>> 
>> Hello Valters,
>> 
>> Thank you for helping to get the upstream version nice and shiny.
>> 
>>> Michael, I haven't looked further into it, but mips64el architecture
>>> failed to build due to undefined _ABIO32 and _ABIN32. If you have any
>>> proposals, they would presumably be good to include in the "upstream"
>>> project directly. The build log is visible at
>>> https://buildd.debian.org/status/fetch.php?pkg=libloc&arch=mips64el&ver=0.9.18-1&stamp=1742656796&raw=0
>> 
>> Yes, this is the error that I got. I did not look further into it because I am not aware of any users of MIPS for libloc, or even as a whole.
>> 
>> I assumed this was a problem in Python. Looking for the two constants, I am not sure which one to set, because hardcoding this would break the build for other MIPS architectures.
>> 
>> What would you suggest to fix this?
> 
> Hi Michael,
> 
> Debian doesn't support MIPS 32-bit variants (mips or mipsel), but it
> supports mips64el (64-bit little-endian). Even more so, for packages
> to automatically transition out of unstable, mips64el is one of the
> architectures expected to successfully build. I haven't seen MIPS in
> real life myself, but Debian tooling having this automation check is a
> strong argument for patching the issue.

Generally I am all in favour of having the library build for as many things as possible. Usually any problems (e.g. like the recent endianness issue) propagate across multiple architectures.

Sometimes however, some niche problem exists in some niche architecture. I am currently incredibly stretched with my own time, and therefore there is not a very good idea to spend it all on some niche problem where we probably won’t have any users anyways. I am trying to find a happy balance here.

> And you are correct about this being unexpected behavior in CPython.
> The reason it bubbled up now is the addition of -Werror=undef as a
> compiler flag. Previously the undefined identifiers were silently
> converted to zero, and _MIPS_SIM would never be zero so it was
> "working as expected".

I agree that this would abort the build. However, the problem is not exactly in my code :)

I would have expected the compiler to define those values depending on the output architecture.

> I am not yet 100% sure how the CPython internals get prepared, and
> need to investigate a bit more before contributing upstream there.
> What I have found is that the logic matches what is defined twice in
> [Misc/platform_triplet.c] as "if _MIPS_SIM == _ABIO32, elif _MIPS_SIM
> == _ABIN32, elif _MIPS_SIM == _ABI64". Debian's
> [ArchitectureSpecificsMemo] recommends using _MIPS_SIM for evaluating
> what the architecture is.
> 
> _MIPS_SIM naming seems to be sourced from a legacy constant from
> decades ago, and overall seems reasonable.The _ABIO32 name however
> specifically seems to stem in gcc codebase from year 2003. There were
> libffi changes happening, and people were trying to avoid pulling in
> legacy headers, so gcc had a patch ["Consistently use _ABIO32 for
> _MIPS_SIM"] which defined _ABIO32=1 and switched to _MIPS_SIM=_ABIO32
> (from some other inherited magic constant) for ABI_32. Over time as
> other MIPS architectures were added, it transformed into a switch/case
> in [config/mips/mips.h] also defining  _ABIN32 for ABI_N32 (as 2),
> _ABI64 for ABI_64 (as 3) and _ABIO64 for ABI_O64 (as 4).
> 
> The current issue comes from the fact that these are defined as
> compiler `builtin_define`. That means, mips64el knows about _ABI64 but
> doesn't know about _ABIO32 and _ABIN32. When CPython tries to use all
> three, two of them are undefined, so CPython should be checking
> whether the values are defined before trying to use them.
> 
> Two solutions I can suggest now: Either downgrading -Werror=undef to
> -Wundef so that it raises a warning (visibility) but doesn't fail the
> build, or adding the magic numbers based on gcc to libloc headers,
> 
> #ifndef _ABIO32
> #  define _ABIO32 1
> #endif
> #ifndef _ABIN32
> #  define _ABIN32 2
> #endif
> #ifndef _ABI64
> #  define _ABI64 3
> #endif
> #ifndef _ABIO64
> #  define _ABIO64 4
> #endif

So it seems that all those values are somewhat defined, because if I add it to the compiler command line, I am getting this:

In file included from <command-line>:
././config.h:224: warning: "_ABI64" redefined
224 | #define _ABI64 1

> Both options have pros and cons, and I haven't been able to convince
> myself which direction I believe to be more preferable as it stands.
> 
> [Misc/platform_triplet.c]:
> https://github.com/python/cpython/blob/v3.13.2/Misc/platform_triplet.c#L60
> [ArchitectureSpecificsMemo]: https://wiki.debian.org/ArchitectureSpecificsMemo
> ["Consistently use _ABIO32 for _MIPS_SIM"]:
> https://gcc.gnu.org/legacy-ml/gcc-patches/2003-10/msg00607.html
> [config/mips/mips.h]:
> https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/config/mips/mips.h;h=616a275b918c34caf87f333f40e00866504156d7;hb=04696df09633baf97cdbbdd6e9929b9d472161d3#l560

On trixie, there is an additional problem where the build dependencies cannot be installed:

Setting up python3.13-minimal:mips64el (3.13.2-2) ...
/var/lib/dpkg/info/python3.13-minimal.postinst: 51: /usr/bin/python3.13: Exec format error
dpkg: error processing package python3.13-minimal:mips64el (--configure):
installed python3.13-minimal:mips64el package post-installation script subprocess returned error exit status 126
Errors were encountered while processing:
python3.13-minimal:mips64el
E: Sub-process /usr/bin/dpkg returned an error code (1)
E: Failed to process build dependencies
script returned exit code 100

Obviously this is emulated, but qemu-user-static is generally available for mips64el.

Given all these problems, I would be happy to receive patches, but I don’t think I have enough experience and time to poke around here. I am happy to assist though.

Best,
-Michael

> --Valters Jansons