Re: The unshared changes give a problem with build after doing git pull on next

[Michael Tremer <michael.tremer@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 11109 bytes --]

Hello,

> On 24 Aug 2024, at 09:33, Matthias Fischer <matthias.fischer(a)ipfire.org> wrote:
> 
> On 21.08.2024 16:28, Michael Tremer wrote:
>> Hello Matthias,
> 
> Hi Michael,
> 
>> As you might have seen I played around with this all more… There is either a kernel bug, or a bug in unshare which causes these problems…
>> 
>> So if we mount /proc manually later this works for me on Ubuntu 22.04 LTS:
>> 
>>  https://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=8ea702f3f853c4c28a2885dafdcae2e737c0a49b
>> 
>> However, there are now some packages which don’t want to build any more.
>> 
>> So I would simply suggest to upgrade the kernel on your Ubuntu release like this:
>> 
>>  apt install linux-generic-hwe-22.04
> 
> Done. Ran without problems.
> 
> I'm now on "Linux Devel64-1 6.8.0-40-generic #40~22.04.3-Ubuntu SMP
> PREEMPT_DYNAMIC..."
> 
>> That should allow you to build with the stock make.sh again.
> 
> Sorry, but no chance...
> 
> Today I tested the hint from "siosios" found in
> https://community.ipfire.org/t/trying-to-build-188/11999/8

Are you referring to this?

   mount -o bind ${BUILD_DIR}/proc ${BUILD_DIR}/proc

He put this into an unfortunate place and will mount this over and over and over again... but making the empty directory a mount point is something that would acceptable as a workaround.

If you can confirm, I can add this to the code.

> Works. Build is running. I'm building 'clamav 1.4.0' right now.
> 
> One glitch I found: the new build still has problems identifying the
> need for customizing new rootfiles(?).

This might be fixed here:

  https://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=d7ee801712705c97fda658bb71209d814d1db841

> Best
> Matthias
> 
>> Best,
>> -Michael
>> 
>>> On 16 Aug 2024, at 17:43, Matthias Fischer <matthias.fischer(a)ipfire.org> wrote:
>>> 
>>> On 16.08.2024 17:35, Michael Tremer wrote:
>>>> Hello,
>>> 
>>> Hi,
>>> 
>>>>> On 8 Aug 2024, at 13:06, Matthias Fischer <matthias.fischer(a)ipfire.org> wrote:
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> ok - please don't laugh.
>>>>> 
>>>>> I did something completely different...
>>>>> 
>>>>> I added the following to 'make.sh':
>>>>> 
>>>>> ***SNIP***
>>>>> diff --git a/make.sh b/make.sh
>>>>> index 922fc4b4c..212256ab1 100755
>>>>> --- a/make.sh
>>>>> +++ b/make.sh
>>>>> @@ -662,6 +662,11 @@ execute() {
>>>>> # If unshare is asked to terminate, terminate all child processes
>>>>> "--kill-child"
>>>>> )
>>>>> +
>>>>> +mount --bind ${BUILD_DIR}/proc ${BUILD_DIR}/proc
>>>>> +
>>>>> +mount --make-rslave ${BUILD_DIR}/proc
>>>>> +
>>>>> fi
>>>> 
>>>> I don’t know why this would fix anything.
>>> 
>>> I don't know either. If I remove these lines, I get the 'invalid
>>> argument' errors again. Immediately.
>>> 
>>>> Could you once again try to swap the slave/private parameters of the —-propagation option for both calls of unshare?
>>> 
>>> Ok. I'll try and keep you informed, lets see. By the way: "Devel 1" just
>>> built 'next' in 3:09:49, updating 'rust 1.80.1' and 'clamav 1.4.0'.
>>> 
>>> 1. I commented my two lines above and I get:
>>> 
>>> ...
>>> Aug 16 16:34:12: Building stage2 unshare: cannot change
>>> /git/ipfire-2.x/build_x86_64/proc filesystem propagation: Invalid argument
>>> ...
>>> 
>>> 2. I swap slave/private parameters
>>> => change line 646 to "--propagation=private" and line 2146 to
>>> "--propagation=slave" and get the same error: "...Invalid argument"
>>> 
>>> 3. I swap parameters back and uncomment my two lines:
>>> => Build is running again.
>>> 
>>>> 
>>>> Also: Why is this working on my freshly installed system?
>>> 
>>> Good question...this is something really strange...
>>> 
>>>>> 
>>>>> while [ $# -gt 0 ]; do
>>>>> ***SNAP***
>>>>> 
>>>>> No 'invalid argument' error anymore.
>>>>> 
>>>>> I don't know why - but with this patch the build is running...
>>>>> 
>>>>> Any comments?
>>>>> 
>>>>> Best
>>>>> Matthias
>>>>> 
>>>>> On 06.08.2024 17:40, Michael Tremer wrote:
>>>>>> Hello,
>>>>>> 
>>>>>> We should not touch the mount propagation of the host’s namespace.
>>>>>> 
>>>>>> Instead, we should create our own mount namespace and that should be private.
>>>>>> 
>>>>>> You can however try to see what happens when you add —-propagation=slave to the first unshare command.
>>>>>> 
>>>>>> I just installed the plain Ubuntu Server, installed git, checkout out the repository, downloaded the toolchain and ran the build.
>>>>>> 
>>>>>> -Michael
>>>>>> 
>>>>>>> On 3 Aug 2024, at 12:22, Matthias Fischer <matthias.fischer(a)ipfire.org> wrote:
>>>>>>> 
>>>>>>> On 03.08.2024 10:54, Michael Tremer wrote:
>>>>>>>> Hello Matthias,
>>>>>>> 
>>>>>>> Hi Michael,
>>>>>>> 
>>>>>>> just looked through https://man7.org/linux/man-pages/man1/unshare.1.html
>>>>>>> and ran 'findmnt -o+PROPAGATION' (see attachment).
>>>>>>> 
>>>>>>> I don't know if this could help but does this differ from your test
>>>>>>> installation?
>>>>>>> 
>>>>>>> Best
>>>>>>> Matthias
>>>>>>> 
>>>>>>>>> On 3 Aug 2024, at 08:47, Matthias Fischer <matthias.fischer(a)ipfire.org> wrote:
>>>>>>>>> 
>>>>>>>>> Hi Michael,
>>>>>>>>> 
>>>>>>>>> [shortened some stuff]
>>>>>>>>> 
>>>>>>>>> ...
>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Being curious I tried to build 'next', but I always get the same error:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> ***SNIP***
>>>>>>>>>>>>> root(a)Devel64-1: /git/ipfire-2.x # ./make.sh build
>>>>>>>>>>>>> Packaged toolchain compilation
>>>>>>>>>>>>> Building IPFire
>>>>>>>>>>>>> stage2
>>>>>>>>>>>>> Jul 26 13:32:59: Building stage2 unshare: cannot change
>>>>>>>>>>>>> /git/ipfire-2.x/build_x86_64/proc filesystem propagation: Invalid argument
>>>>>>>>> 
>>>>>>>>>>>> ...
>>>>>>>>> 
>>>>>>>>>> It looks like you can simply update the kernel staying on the same release:
>>>>>>>>>> 
>>>>>>>>>> https://ubuntu.com/security/livepatch/docs/livepatch/reference/kernels
>>>>>>>>>> 
>>>>>>>>>> For 22.04 LTS, there is a Linux 6.8 image available.
>>>>>>>>>> 
>>>>>>>>>> Could you check that and confirm that it fixes the mount propagation problem?
>>>>>>>>> 
>>>>>>>>> Done.
>>>>>>>>> 
>>>>>>>>> Current state is as follows:
>>>>>>>>> 
>>>>>>>>> ***SNIP***
>>>>>>>>> ...
>>>>>>>>> root(a)Devel64-1: /git/ipfire-2.x # lsb_release -a
>>>>>>>>> No LSB modules are available.
>>>>>>>>> Distributor ID: Ubuntu
>>>>>>>>> Description:    Ubuntu 22.04.4 LTS
>>>>>>>>> Release:        22.04
>>>>>>>>> Codename:       jammy
>>>>>>>>> ...
>>>>>>>>> root(a)Devel64-1: /git/ipfire-2.x # uname -mrs
>>>>>>>>> Linux 6.8.0-39-generic x86_64
>>>>>>>>> ...
>>>>>>>>> ***SNAP***
>>>>>>>>> 
>>>>>>>>> But when I try to build 'next' I get exactly the same error as
>>>>>>>>> before:
>>>>>>>>> 
>>>>>>>>> ***SNIP***
>>>>>>>>> root(a)Devel64-1: /git/ipfire-2.x # ./make.sh build
>>>>>>>>> Packaged toolchain compilation
>>>>>>>>> Building IPFire
>>>>>>>>> stage2
>>>>>>>>> Aug  2 21:21:15: Building stage2 unshare: cannot change
>>>>>>>>> /git/ipfire-2.x/build_x86_64/proc filesystem propagation: Invalid argument
>>>>>>>> 
>>>>>>>> Ah, this is good information. So it is not the kernel, it rather is Ubuntu handling something differently.
>>>>>>>> 
>>>>>>>> I will have a look at this and get back to you.
>>>>>>>> 
>>>>>>>>> ERROR: Downloading stage2
>>>>>>>>>                                   [ FAIL ]
>>>>>>>>> Check /git/ipfire-2.x/log_x86_64/_build.ipfire.log for errors if
>>>>>>>>> applicable                                 [ FAIL ]
>>>>>>>>> ***SNAP***
>>>>>>>>> 
>>>>>>>>> Being curious, I commented line line 633 in 'make.sh'
>>>>>>>>> ("--mount-proc=${BUILD_DIR}/proc") => Building starts but fails during
>>>>>>>>> 'glib 2.77.0':
>>>>>>>>> 
>>>>>>>>> ***SNIP***
>>>>>>>>> ...
>>>>>>>>> glib (2.77.0)
>>>>>>>>>                       [     1:14 ][0/1011]
>>>>>>>>> 
>>>>>>>>> [951/1374] Compiling C object gio/gio.p/gio-tool-tree.c.o
>>>>>>>>> [952/1374] Linking target gio/gio
>>>>>>>>> [953/1374] Compiling C object
>>>>>>>>> gio/gio-querymodules.p/gio-querymodules.c.o
>>>>>>>>> [954/1374] Linking target gio/gio-querymodules
>>>>>>>>> [955/1374] Compiling C object gio/gresource.p/gresource-tool.c.o
>>>>>>>>> [956/1374] Compiling C object
>>>>>>>>> gio/glib-compile-schemas.p/.._subprojects_gvdb_gvdb_gvdb-reader.c.o
>>>>>>>>> [957/1374] Linking target gio/gresource
>>>>>>>>> [958/1374] Compiling C object
>>>>>>>>> gio/glib-compile-schemas.p/.._subprojects_gvdb_gvdb_gvdb-builder.c.o
>>>>>>>>> [959/1374] Compiling C object
>>>>>>>>> gio/glib-compile-resources.p/.._subprojects_gvdb_gvdb_gvdb-reader.c.o
>>>>>>>>> [960/1374] Compiling C object
>>>>>>>>> gio/glib-compile-resources.p/.._subprojects_gvdb_gvdb_gvdb-builder.c.o
>>>>>>>>> [961/1374] Compiling C object
>>>>>>>>> gio/glib-compile-resources.p/glib-compile-resources.c.o
>>>>>>>>> [962/1374] Linking target gio/glib-compile-resources
>>>>>>>>> [963/1374] Compiling C object
>>>>>>>>> gio/tests/modules/libtestmodulea.so.p/test-module-a.c.o
>>>>>>>>> [964/1374] Compiling C object gio/gapplication.p/gapplication-tool.c.o
>>>>>>>>> [965/1374] Compiling C object gio/gsettings.p/gsettings-tool.c.o
>>>>>>>>> [966/1374] Linking target gio/tests/modules/libtestmodulea.so
>>>>>>>>> [967/1374] Generating gio/tests/plugin-resources.c with a custom command
>>>>>>>>> FAILED: gio/tests/plugin-resources.c
>>>>>>>>> /usr/src/glib-2.77.0/builddir/gio/glib-compile-resources
>>>>>>>>> --compiler=gcc --target=gio/tests/plugin-resources.c --sour
>>>>>>>>> cedir=/usr/src/glib-2.77.0/gio/tests --internal --generate-source
>>>>>>>>> --c-name _g_plugin ../gio/tests/test4.gresource.xml
>>>>>>>>> /usr/src/glib-2.77.0/builddir/gio/glib-compile-resources: error
>>>>>>>>> while loading shared libraries: libgio-2.0.so.0: can
>>>>>>>>> not open shared object file: No such file or directory
>>>>>>>>> [968/1374] Linking target gio/gapplication
>>>>>>>>> [969/1374] Compiling C object
>>>>>>>>> gio/glib-compile-schemas.p/glib-compile-schemas.c.o
>>>>>>>>> [970/1374] Linking target gio/gsettings
>>>>>>>>> [971/1374] Compiling C object
>>>>>>>>> gio/tests/modules/libtestmoduleb.so.p/test-module-b.c.o
>>>>>>>>> [972/1374] Compiling C object
>>>>>>>>> gio/tests/gdbus-overflow.p/gdbus-overflow.c.o
>>>>>>>>> [973/1374] Compiling C object gio/gdbus.p/gdbus-tool.c.o
>>>>>>>>> [974/1374] Compiling C object
>>>>>>>>> gio/tests/gdbus-object-manager-example/libgdbus-example-objectmanager.so.p/meson-gener
>>>>>>>>> ated_.._objectmanager-gen.c.o
>>>>>>>>> ninja: build stopped: subcommand failed.
>>>>>>>>> make: *** [glib:75: /usr/src/log/glib-2.77.0] Error 1
>>>>>>>>> make: Leaving directory '/usr/src/lfs'
>>>>>>>>> 
>>>>>>>>> ERROR: Building glib
>>>>>>>>>                                   [ FAIL ]
>>>>>>>>> Check /git/ipfire-2.x/log_x86_64/_build.ipfire.log for errors if
>>>>>>>>> applicable                                 [ FAIL ]
>>>>>>>>> ***SNAP***
>>>>>>>>> 
>>>>>>>>> Best
>>>>>>>>> Matthias
>> 
>> 
>