Re: Upload libloc to Debian

[Michael Tremer <michael.tremer@ipfire.org>]

Hello HC,

> On 27 Mar 2025, at 17:22, Hans-Christoph Steiner <hans@guardianproject.info> wrote:
> 
> Michael, if you think libloc on mips64el has no real userbase, then we can update the package to have looser requirements there.  I have done this on other packages, its kosher to do in this kind of situation.

I think I was talking bollocks here. I assumed that in bookworm, we did not have a package for mips64el, but we do. If we didn’t I would have expected that sooner or later someone would come and ask for it.

Preferably we support all architectures that Debian supports. As a user of non-amd64, it was always super annoying if some package was missing. So I don’t want to be *that guy*.

However, as stated before, sometimes those problems are not the easiest to solve...

> Valters, if everyone is OK with it, I'd add you as a "Developer" on the projects on Salsa.  Then you can review and merge things to the packages.
> 
> 
>>> Towards that end, I made a merge request to review:
>>> https://salsa.debian.org/debian/libloc-database/-/merge_requests/3
>> I am wondering if the d/watch mangle rule should be changed to follow
>> the same sed logic as in d/rules. Left a comment on the PR and can be
>> discussed there too. Overall, it looks like a good, helpful change.
> 
> I had the same thought, if that works for you, I think that would be a better approach.
> 
> .hc
> 
> Michael Tremer:
>> Hello Valters,
>>> On 27 Mar 2025, at 00:25, Valters Jansons <valter.jansons@gmail.com> wrote:
>>> 
>>> On Wed, Mar 26, 2025 at 10:30 AM Hans-Christoph Steiner
>>> <hans@guardianproject.info> wrote:
>>>> 
>>>> Valters Jansons:
>>>> 
>>>> This is great!  One workflow we could use is adding you to the libloc and
>>>> libloc-location projects on salsa, then you could review and merge changes
>>>> there.  Then you'd just need Jochen and/or I for reviewing and uploading
>>>> releases to Debian.  How does that sound?
>>> 
>>> Hi Hans-Christoph,
>>> 
>>> I'm not a part of the IPFire/libloc team, however I would be happy to
>>> join as a maintainer for the Debian package.
>>> 
>>> I have historically maintained an Ubuntu PPA for use in my old
>>> company, and in other projects afterwards on Ubuntu 22.04 and older
>>> (before the Debian Universe package). I may have gaps in how packaging
>>> is expected, but overall I feel comfortable in moving the process
>>> forward.
>>> 
>>>> Towards that end, I made a merge request to review:
>>>> https://salsa.debian.org/debian/libloc-database/-/merge_requests/3
>>> 
>>> I am wondering if the d/watch mangle rule should be changed to follow
>>> the same sed logic as in d/rules. Left a comment on the PR and can be
>>> discussed there too. Overall, it looks like a good, helpful change.
>>> 
>>> On Wed, Mar 26, 2025 at 12:17 PM Michael Tremer
>>> <michael.tremer@ipfire.org> wrote:
>>>> 
>>>> Hello Valters,
>>>> 
>>>> Thank you for helping to get the upstream version nice and shiny.
>>>> 
>>>>> Michael, I haven't looked further into it, but mips64el architecture
>>>>> failed to build due to undefined _ABIO32 and _ABIN32. If you have any
>>>>> proposals, they would presumably be good to include in the "upstream"
>>>>> project directly. The build log is visible at
>>>>> https://buildd.debian.org/status/fetch.php?pkg=libloc&arch=mips64el&ver=0.9.18-1&stamp=1742656796&raw=0
>>>> 
>>>> Yes, this is the error that I got. I did not look further into it because I am not aware of any users of MIPS for libloc, or even as a whole.
>>>> 
>>>> I assumed this was a problem in Python. Looking for the two constants, I am not sure which one to set, because hardcoding this would break the build for other MIPS architectures.
>>>> 
>>>> What would you suggest to fix this?
>>> 
>>> Hi Michael,
>>> 
>>> Debian doesn't support MIPS 32-bit variants (mips or mipsel), but it
>>> supports mips64el (64-bit little-endian). Even more so, for packages
>>> to automatically transition out of unstable, mips64el is one of the
>>> architectures expected to successfully build. I haven't seen MIPS in
>>> real life myself, but Debian tooling having this automation check is a
>>> strong argument for patching the issue.
>> Generally I am all in favour of having the library build for as many things as possible. Usually any problems (e.g. like the recent endianness issue) propagate across multiple architectures.
>> Sometimes however, some niche problem exists in some niche architecture. I am currently incredibly stretched with my own time, and therefore there is not a very good idea to spend it all on some niche problem where we probably won’t have any users anyways. I am trying to find a happy balance here.
>>> And you are correct about this being unexpected behavior in CPython.
>>> The reason it bubbled up now is the addition of -Werror=undef as a
>>> compiler flag. Previously the undefined identifiers were silently
>>> converted to zero, and _MIPS_SIM would never be zero so it was
>>> "working as expected".
>> I agree that this would abort the build. However, the problem is not exactly in my code :)
>> I would have expected the compiler to define those values depending on the output architecture.
>>> I am not yet 100% sure how the CPython internals get prepared, and
>>> need to investigate a bit more before contributing upstream there.
>>> What I have found is that the logic matches what is defined twice in
>>> [Misc/platform_triplet.c] as "if _MIPS_SIM == _ABIO32, elif _MIPS_SIM
>>> == _ABIN32, elif _MIPS_SIM == _ABI64". Debian's
>>> [ArchitectureSpecificsMemo] recommends using _MIPS_SIM for evaluating
>>> what the architecture is.
>>> 
>>> _MIPS_SIM naming seems to be sourced from a legacy constant from
>>> decades ago, and overall seems reasonable.The _ABIO32 name however
>>> specifically seems to stem in gcc codebase from year 2003. There were
>>> libffi changes happening, and people were trying to avoid pulling in
>>> legacy headers, so gcc had a patch ["Consistently use _ABIO32 for
>>> _MIPS_SIM"] which defined _ABIO32=1 and switched to _MIPS_SIM=_ABIO32
>>> (from some other inherited magic constant) for ABI_32. Over time as
>>> other MIPS architectures were added, it transformed into a switch/case
>>> in [config/mips/mips.h] also defining  _ABIN32 for ABI_N32 (as 2),
>>> _ABI64 for ABI_64 (as 3) and _ABIO64 for ABI_O64 (as 4).
>>> 
>>> The current issue comes from the fact that these are defined as
>>> compiler `builtin_define`. That means, mips64el knows about _ABI64 but
>>> doesn't know about _ABIO32 and _ABIN32. When CPython tries to use all
>>> three, two of them are undefined, so CPython should be checking
>>> whether the values are defined before trying to use them.
>>> 
>>> Two solutions I can suggest now: Either downgrading -Werror=undef to
>>> -Wundef so that it raises a warning (visibility) but doesn't fail the
>>> build, or adding the magic numbers based on gcc to libloc headers,
>>> 
>>> #ifndef _ABIO32
>>> #  define _ABIO32 1
>>> #endif
>>> #ifndef _ABIN32
>>> #  define _ABIN32 2
>>> #endif
>>> #ifndef _ABI64
>>> #  define _ABI64 3
>>> #endif
>>> #ifndef _ABIO64
>>> #  define _ABIO64 4
>>> #endif
>> So it seems that all those values are somewhat defined, because if I add it to the compiler command line, I am getting this:
>> In file included from <command-line>:
>> ././config.h:224: warning: "_ABI64" redefined
>> 224 | #define _ABI64 1
>>> Both options have pros and cons, and I haven't been able to convince
>>> myself which direction I believe to be more preferable as it stands.
>>> 
>>> [Misc/platform_triplet.c]:
>>> https://github.com/python/cpython/blob/v3.13.2/Misc/platform_triplet.c#L60
>>> [ArchitectureSpecificsMemo]: https://wiki.debian.org/ArchitectureSpecificsMemo
>>> ["Consistently use _ABIO32 for _MIPS_SIM"]:
>>> https://gcc.gnu.org/legacy-ml/gcc-patches/2003-10/msg00607.html
>>> [config/mips/mips.h]:
>>> https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/config/mips/mips.h;h=616a275b918c34caf87f333f40e00866504156d7;hb=04696df09633baf97cdbbdd6e9929b9d472161d3#l560
>> On trixie, there is an additional problem where the build dependencies cannot be installed:
>> Setting up python3.13-minimal:mips64el (3.13.2-2) ...
>> /var/lib/dpkg/info/python3.13-minimal.postinst: 51: /usr/bin/python3.13: Exec format error
>> dpkg: error processing package python3.13-minimal:mips64el (--configure):
>> installed python3.13-minimal:mips64el package post-installation script subprocess returned error exit status 126
>> Errors were encountered while processing:
>> python3.13-minimal:mips64el
>> E: Sub-process /usr/bin/dpkg returned an error code (1)
>> E: Failed to process build dependencies
>> script returned exit code 100
>> Obviously this is emulated, but qemu-user-static is generally available for mips64el.
>> Given all these problems, I would be happy to receive patches, but I don’t think I have enough experience and time to poke around here. I am happy to assist though.
>> Best,
>> -Michael
>>> --Valters Jansons
> 
> -- 
> Signal: +13478504872
> PGP fingerprint: EE66 20C7 136B 0D2C 456C  0A4D E9E2 8DEA 00AA 5556