Fwd: The unshared changes give a problem with build after doing git pull on next

Fwd: The unshared changes give a problem with build after doing git pull on next

[Matthias Fischer]

[-- Attachment #1: Type: text/plain, Size: 8020 bytes --]

Sorry, I forgot to send this to the list...

-------- Forwarded Message --------
Subject: Re: The unshared changes give a problem with build after doing
git pull on next
Date: Wed, 7 Aug 2024 13:45:28 +0200
From: Matthias Fischer <matthias.fischer(a)ipfire.org>
To: Michael Tremer <michael.tremer(a)ipfire.org>

On 06.08.2024 17:40, Michael Tremer wrote:
> Hello,

Hi,

> We should not touch the mount propagation of the host’s namespace.
> 
> Instead, we should create our own mount namespace and that should be private.
> 
> You can however try to see what happens when you add —-propagation=slave to the first unshare command.

[Where exactly is the "first unshare command"? Looking at the code, the
first occurence is at line 639ff.
But line 646 already reads "--propagation=slave" so I didn't change it...]

I then tried several things. I did not pay attention to whether these
tests were useful in this context - I only tested:

First: both machines are still running without any problems as long as
don't try to build 'next'. ;-) Hm.

So I tested the 'unshare' commands that are executed from line 642
onwards step by step.

Interesting: no seen problems, cursor just jumps to the next line...

Before and after I ran that "findmnt -o+PROPAGATION" command - see
attachments. Useful? I don't know...

But even the complete command ran without any errors:

unshare --cgroup --mount --propagation=slave --pid --fork --time --uts
--mount-proc=${BUILD_DIR}/proc --kill-child

But as soon as I start './make.sh build' the "invalid argument" error is
back again.

I then played with two commands found in
https://github.com/util-linux/util-linux/issues/718

mount --bind ${BUILD_DIR}/proc ${BUILD_DIR}/proc

and

mount --make-rprivate ${BUILD_DIR}/proc ${BUILD_DIR}/proc

And for the first time I see that dir with 'findmnt...' (see 3rd
attachment). But I don't know how to go on...

Is there anything left I could try?

Best
Matthias

> I just installed the plain Ubuntu Server, installed git, checkout out the repository, downloaded the toolchain and ran the build.
> 
> -Michael
> 
>> On 3 Aug 2024, at 12:22, Matthias Fischer <matthias.fischer(a)ipfire.org> wrote:
>> 
>> On 03.08.2024 10:54, Michael Tremer wrote:
>>> Hello Matthias,
>> 
>> Hi Michael,
>> 
>> just looked through https://man7.org/linux/man-pages/man1/unshare.1.html
>> and ran 'findmnt -o+PROPAGATION' (see attachment).
>> 
>> I don't know if this could help but does this differ from your test
>> installation?
>> 
>> Best
>> Matthias
>> 
>>>> On 3 Aug 2024, at 08:47, Matthias Fischer <matthias.fischer(a)ipfire.org> wrote:
>>>> 
>>>> Hi Michael,
>>>> 
>>>> [shortened some stuff]
>>>> 
>>>> ...
>>>> 
>>>>>>>> 
>>>>>>>> Being curious I tried to build 'next', but I always get the same error:
>>>>>>>> 
>>>>>>>> ***SNIP***
>>>>>>>> root(a)Devel64-1: /git/ipfire-2.x # ./make.sh build
>>>>>>>> Packaged toolchain compilation
>>>>>>>> Building IPFire
>>>>>>>> stage2
>>>>>>>> Jul 26 13:32:59: Building stage2 unshare: cannot change
>>>>>>>> /git/ipfire-2.x/build_x86_64/proc filesystem propagation: Invalid argument
>>>> 
>>>>>>> ...
>>>> 
>>>>> It looks like you can simply update the kernel staying on the same release:
>>>>> 
>>>>> https://ubuntu.com/security/livepatch/docs/livepatch/reference/kernels
>>>>> 
>>>>> For 22.04 LTS, there is a Linux 6.8 image available.
>>>>> 
>>>>> Could you check that and confirm that it fixes the mount propagation problem?
>>>> 
>>>> Done.
>>>> 
>>>> Current state is as follows:
>>>> 
>>>> ***SNIP***
>>>> ...
>>>> root(a)Devel64-1: /git/ipfire-2.x # lsb_release -a
>>>> No LSB modules are available.
>>>> Distributor ID: Ubuntu
>>>> Description:    Ubuntu 22.04.4 LTS
>>>> Release:        22.04
>>>> Codename:       jammy
>>>> ...
>>>> root(a)Devel64-1: /git/ipfire-2.x # uname -mrs
>>>> Linux 6.8.0-39-generic x86_64
>>>> ...
>>>> ***SNAP***
>>>> 
>>>> But when I try to build 'next' I get exactly the same error as
>>>> before:
>>>> 
>>>> ***SNIP***
>>>> root(a)Devel64-1: /git/ipfire-2.x # ./make.sh build
>>>> Packaged toolchain compilation
>>>> Building IPFire
>>>> stage2
>>>>   Aug  2 21:21:15: Building stage2 unshare: cannot change
>>>> /git/ipfire-2.x/build_x86_64/proc filesystem propagation: Invalid argument
>>> 
>>> Ah, this is good information. So it is not the kernel, it rather is Ubuntu handling something differently.
>>> 
>>> I will have a look at this and get back to you.
>>> 
>>>> ERROR: Downloading stage2
>>>>                                      [ FAIL ]
>>>>   Check /git/ipfire-2.x/log_x86_64/_build.ipfire.log for errors if
>>>> applicable                                 [ FAIL ]
>>>> ***SNAP***
>>>> 
>>>> Being curious, I commented line line 633 in 'make.sh'
>>>> ("--mount-proc=${BUILD_DIR}/proc") => Building starts but fails during
>>>> 'glib 2.77.0':
>>>> 
>>>> ***SNIP***
>>>> ...
>>>> glib (2.77.0)
>>>>                          [     1:14 ][0/1011]
>>>> 
>>>>   [951/1374] Compiling C object gio/gio.p/gio-tool-tree.c.o
>>>>   [952/1374] Linking target gio/gio
>>>>   [953/1374] Compiling C object
>>>> gio/gio-querymodules.p/gio-querymodules.c.o
>>>>   [954/1374] Linking target gio/gio-querymodules
>>>>   [955/1374] Compiling C object gio/gresource.p/gresource-tool.c.o
>>>>   [956/1374] Compiling C object
>>>> gio/glib-compile-schemas.p/.._subprojects_gvdb_gvdb_gvdb-reader.c.o
>>>>   [957/1374] Linking target gio/gresource
>>>>   [958/1374] Compiling C object
>>>> gio/glib-compile-schemas.p/.._subprojects_gvdb_gvdb_gvdb-builder.c.o
>>>>   [959/1374] Compiling C object
>>>> gio/glib-compile-resources.p/.._subprojects_gvdb_gvdb_gvdb-reader.c.o
>>>>   [960/1374] Compiling C object
>>>> gio/glib-compile-resources.p/.._subprojects_gvdb_gvdb_gvdb-builder.c.o
>>>>   [961/1374] Compiling C object
>>>> gio/glib-compile-resources.p/glib-compile-resources.c.o
>>>>   [962/1374] Linking target gio/glib-compile-resources
>>>>   [963/1374] Compiling C object
>>>> gio/tests/modules/libtestmodulea.so.p/test-module-a.c.o
>>>>   [964/1374] Compiling C object gio/gapplication.p/gapplication-tool.c.o
>>>>   [965/1374] Compiling C object gio/gsettings.p/gsettings-tool.c.o
>>>>   [966/1374] Linking target gio/tests/modules/libtestmodulea.so
>>>>   [967/1374] Generating gio/tests/plugin-resources.c with a custom command
>>>>   FAILED: gio/tests/plugin-resources.c
>>>>   /usr/src/glib-2.77.0/builddir/gio/glib-compile-resources
>>>> --compiler=gcc --target=gio/tests/plugin-resources.c --sour
>>>> cedir=/usr/src/glib-2.77.0/gio/tests --internal --generate-source
>>>> --c-name _g_plugin ../gio/tests/test4.gresource.xml
>>>>   /usr/src/glib-2.77.0/builddir/gio/glib-compile-resources: error
>>>> while loading shared libraries: libgio-2.0.so.0: can
>>>> not open shared object file: No such file or directory
>>>>   [968/1374] Linking target gio/gapplication
>>>>   [969/1374] Compiling C object
>>>> gio/glib-compile-schemas.p/glib-compile-schemas.c.o
>>>>   [970/1374] Linking target gio/gsettings
>>>>   [971/1374] Compiling C object
>>>> gio/tests/modules/libtestmoduleb.so.p/test-module-b.c.o
>>>>   [972/1374] Compiling C object
>>>> gio/tests/gdbus-overflow.p/gdbus-overflow.c.o
>>>>   [973/1374] Compiling C object gio/gdbus.p/gdbus-tool.c.o
>>>>   [974/1374] Compiling C object
>>>> gio/tests/gdbus-object-manager-example/libgdbus-example-objectmanager.so.p/meson-gener
>>>> ated_.._objectmanager-gen.c.o
>>>>   ninja: build stopped: subcommand failed.
>>>>   make: *** [glib:75: /usr/src/log/glib-2.77.0] Error 1
>>>>   make: Leaving directory '/usr/src/lfs'
>>>> 
>>>> ERROR: Building glib
>>>>                                      [ FAIL ]
>>>>   Check /git/ipfire-2.x/log_x86_64/_build.ipfire.log for errors if
>>>> applicable                                 [ FAIL ]
>>>> ***SNAP***
>>>> 
>>>> Best
>>>> Matthias
>>> 
> 
> 
> 

[-- Attachment #2: 01-stepbystep.pdf --]
[-- Type: application/pdf, Size: 37721 bytes --]

[-- Attachment #3: 02-stepbystep.pdf --]
[-- Type: application/pdf, Size: 34891 bytes --]

[-- Attachment #4: 03-stepbystep.pdf --]
[-- Type: application/pdf, Size: 15224 bytes --]

Fwd: The unshared changes give a problem with build after doing git pull on next

[Matthias Fischer]

[-- Attachment #1: Type: text/plain, Size: 7320 bytes --]

Again - I only hit "Reply" not "Reply all"...

-------- Forwarded Message --------
Subject: Re: The unshared changes give a problem with build after doing
git pull on next
Date: Thu, 8 Aug 2024 14:06:24 +0200
From: Matthias Fischer <matthias.fischer(a)ipfire.org>
To: Michael Tremer <michael.tremer(a)ipfire.org>

Hi,

ok - please don't laugh.

I did something completely different...

I added the following to 'make.sh':

***SNIP***
diff --git a/make.sh b/make.sh
index 922fc4b4c..212256ab1 100755
--- a/make.sh
+++ b/make.sh
@@ -662,6 +662,11 @@ execute() {
 			# If unshare is asked to terminate, terminate all child processes
 			"--kill-child"
 		)
+
+mount --bind ${BUILD_DIR}/proc ${BUILD_DIR}/proc
+
+mount --make-rslave ${BUILD_DIR}/proc
+
 	fi

 	while [ $# -gt 0 ]; do
***SNAP***

No 'invalid argument' error anymore.

I don't know why - but with this patch the build is running...

Any comments?

Best
Matthias

 On 06.08.2024 17:40, Michael Tremer wrote:
> Hello,
> 
> We should not touch the mount propagation of the host’s namespace.
> 
> Instead, we should create our own mount namespace and that should be private.
> 
> You can however try to see what happens when you add —-propagation=slave to the first unshare command.
> 
> I just installed the plain Ubuntu Server, installed git, checkout out the repository, downloaded the toolchain and ran the build.
> 
> -Michael
> 
>> On 3 Aug 2024, at 12:22, Matthias Fischer <matthias.fischer(a)ipfire.org> wrote:
>> 
>> On 03.08.2024 10:54, Michael Tremer wrote:
>>> Hello Matthias,
>> 
>> Hi Michael,
>> 
>> just looked through https://man7.org/linux/man-pages/man1/unshare.1.html
>> and ran 'findmnt -o+PROPAGATION' (see attachment).
>> 
>> I don't know if this could help but does this differ from your test
>> installation?
>> 
>> Best
>> Matthias
>> 
>>>> On 3 Aug 2024, at 08:47, Matthias Fischer <matthias.fischer(a)ipfire.org> wrote:
>>>> 
>>>> Hi Michael,
>>>> 
>>>> [shortened some stuff]
>>>> 
>>>> ...
>>>> 
>>>>>>>> 
>>>>>>>> Being curious I tried to build 'next', but I always get the same error:
>>>>>>>> 
>>>>>>>> ***SNIP***
>>>>>>>> root(a)Devel64-1: /git/ipfire-2.x # ./make.sh build
>>>>>>>> Packaged toolchain compilation
>>>>>>>> Building IPFire
>>>>>>>> stage2
>>>>>>>> Jul 26 13:32:59: Building stage2 unshare: cannot change
>>>>>>>> /git/ipfire-2.x/build_x86_64/proc filesystem propagation: Invalid argument
>>>> 
>>>>>>> ...
>>>> 
>>>>> It looks like you can simply update the kernel staying on the same release:
>>>>> 
>>>>> https://ubuntu.com/security/livepatch/docs/livepatch/reference/kernels
>>>>> 
>>>>> For 22.04 LTS, there is a Linux 6.8 image available.
>>>>> 
>>>>> Could you check that and confirm that it fixes the mount propagation problem?
>>>> 
>>>> Done.
>>>> 
>>>> Current state is as follows:
>>>> 
>>>> ***SNIP***
>>>> ...
>>>> root(a)Devel64-1: /git/ipfire-2.x # lsb_release -a
>>>> No LSB modules are available.
>>>> Distributor ID: Ubuntu
>>>> Description:    Ubuntu 22.04.4 LTS
>>>> Release:        22.04
>>>> Codename:       jammy
>>>> ...
>>>> root(a)Devel64-1: /git/ipfire-2.x # uname -mrs
>>>> Linux 6.8.0-39-generic x86_64
>>>> ...
>>>> ***SNAP***
>>>> 
>>>> But when I try to build 'next' I get exactly the same error as
>>>> before:
>>>> 
>>>> ***SNIP***
>>>> root(a)Devel64-1: /git/ipfire-2.x # ./make.sh build
>>>> Packaged toolchain compilation
>>>> Building IPFire
>>>> stage2
>>>>   Aug  2 21:21:15: Building stage2 unshare: cannot change
>>>> /git/ipfire-2.x/build_x86_64/proc filesystem propagation: Invalid argument
>>> 
>>> Ah, this is good information. So it is not the kernel, it rather is Ubuntu handling something differently.
>>> 
>>> I will have a look at this and get back to you.
>>> 
>>>> ERROR: Downloading stage2
>>>>                                      [ FAIL ]
>>>>   Check /git/ipfire-2.x/log_x86_64/_build.ipfire.log for errors if
>>>> applicable                                 [ FAIL ]
>>>> ***SNAP***
>>>> 
>>>> Being curious, I commented line line 633 in 'make.sh'
>>>> ("--mount-proc=${BUILD_DIR}/proc") => Building starts but fails during
>>>> 'glib 2.77.0':
>>>> 
>>>> ***SNIP***
>>>> ...
>>>> glib (2.77.0)
>>>>                          [     1:14 ][0/1011]
>>>> 
>>>>   [951/1374] Compiling C object gio/gio.p/gio-tool-tree.c.o
>>>>   [952/1374] Linking target gio/gio
>>>>   [953/1374] Compiling C object
>>>> gio/gio-querymodules.p/gio-querymodules.c.o
>>>>   [954/1374] Linking target gio/gio-querymodules
>>>>   [955/1374] Compiling C object gio/gresource.p/gresource-tool.c.o
>>>>   [956/1374] Compiling C object
>>>> gio/glib-compile-schemas.p/.._subprojects_gvdb_gvdb_gvdb-reader.c.o
>>>>   [957/1374] Linking target gio/gresource
>>>>   [958/1374] Compiling C object
>>>> gio/glib-compile-schemas.p/.._subprojects_gvdb_gvdb_gvdb-builder.c.o
>>>>   [959/1374] Compiling C object
>>>> gio/glib-compile-resources.p/.._subprojects_gvdb_gvdb_gvdb-reader.c.o
>>>>   [960/1374] Compiling C object
>>>> gio/glib-compile-resources.p/.._subprojects_gvdb_gvdb_gvdb-builder.c.o
>>>>   [961/1374] Compiling C object
>>>> gio/glib-compile-resources.p/glib-compile-resources.c.o
>>>>   [962/1374] Linking target gio/glib-compile-resources
>>>>   [963/1374] Compiling C object
>>>> gio/tests/modules/libtestmodulea.so.p/test-module-a.c.o
>>>>   [964/1374] Compiling C object gio/gapplication.p/gapplication-tool.c.o
>>>>   [965/1374] Compiling C object gio/gsettings.p/gsettings-tool.c.o
>>>>   [966/1374] Linking target gio/tests/modules/libtestmodulea.so
>>>>   [967/1374] Generating gio/tests/plugin-resources.c with a custom command
>>>>   FAILED: gio/tests/plugin-resources.c
>>>>   /usr/src/glib-2.77.0/builddir/gio/glib-compile-resources
>>>> --compiler=gcc --target=gio/tests/plugin-resources.c --sour
>>>> cedir=/usr/src/glib-2.77.0/gio/tests --internal --generate-source
>>>> --c-name _g_plugin ../gio/tests/test4.gresource.xml
>>>>   /usr/src/glib-2.77.0/builddir/gio/glib-compile-resources: error
>>>> while loading shared libraries: libgio-2.0.so.0: can
>>>> not open shared object file: No such file or directory
>>>>   [968/1374] Linking target gio/gapplication
>>>>   [969/1374] Compiling C object
>>>> gio/glib-compile-schemas.p/glib-compile-schemas.c.o
>>>>   [970/1374] Linking target gio/gsettings
>>>>   [971/1374] Compiling C object
>>>> gio/tests/modules/libtestmoduleb.so.p/test-module-b.c.o
>>>>   [972/1374] Compiling C object
>>>> gio/tests/gdbus-overflow.p/gdbus-overflow.c.o
>>>>   [973/1374] Compiling C object gio/gdbus.p/gdbus-tool.c.o
>>>>   [974/1374] Compiling C object
>>>> gio/tests/gdbus-object-manager-example/libgdbus-example-objectmanager.so.p/meson-gener
>>>> ated_.._objectmanager-gen.c.o
>>>>   ninja: build stopped: subcommand failed.
>>>>   make: *** [glib:75: /usr/src/log/glib-2.77.0] Error 1
>>>>   make: Leaving directory '/usr/src/lfs'
>>>> 
>>>> ERROR: Building glib
>>>>                                      [ FAIL ]
>>>>   Check /git/ipfire-2.x/log_x86_64/_build.ipfire.log for errors if
>>>> applicable                                 [ FAIL ]
>>>> ***SNAP***
>>>> 
>>>> Best
>>>> Matthias
>>> 
> 
> 
>