HI,
On 11.05.2018 22:27, Michael Tremer wrote:
Hey Matthias,
are you still working on this?
Its still on my list, but I must confess: I didn't get a grip on this yet.
I am asking because I just noticed that the new next build is about 50MB larger than the old one due to the larger kernel and loads more of firmware files.
Yes, I saw this too. I was waiting for you to ask... ;-)
I'll try to explain what I did and where I'm stuck:
...
On 06.02.2018 02:05, Michael Tremer wrote:
... ((XZ_MEM=(($HOST_MEM)/10)*7)) && XZ_MEM=${XZ_MEM}MiB
You can write this even short:
XZ_MEM="$(( HOST_MEM * 7 / 10 ))MiB"
Oneliner! Yes. I was looking for something like that... ;-)
This did work, no problem here.
... # Host memory is ok, calculating XZ memory XZ_MEM="$(( HOST_MEM * 7 / 10 ))MiB" XZ_OPT="--threads=0 -8 $XZ_MEM" echo XZ-Memory: $XZ_MEM echo XZ-Options: $XZ_OPT echo "XZ memory size is OK (must be at least 700MiB), starting build..." ... ["echoes" for debugging!]
This is working. '$XZ_OPT' now gives me: --threads=0 -8 5450MiB
Will there be some checks in make.sh now that check if at least 1024MB of memory are available for build? ...
For this, I changed the above ("read -p..."-lines are just for debugging):
***SNIP*** ... # Get the amount of memory in this build system HOST_MEM=$(system_memory)
# XZ-TUNING BEGIN # Checking host memory
if [ $HOST_MEM -ge 1024 ]; then print_build_stage "Host-Memory: $HOST_MEM MiB" print_build_stage "Host memory is OK (must be at least 1024 MiB), calculating XZ memory..."
# Host memory is ok, calculating XZ memory echo
XZ_MEM="$(( HOST_MEM * 7 / 10 ))MiB" XZ_OPT="--threads=0 -8 $XZ_MEM" print_build_stage "XZ-Memory:" $XZ_MEM print_build_stage "XZ-Options:" $XZ_OPT print_build_stage "XZ memory size is OK (must be at least 700 MiB), moving on..."
echo read -p "Press [Enter] key to continue..."
else
print_build_stage "Host-Memory: $HOST_MEM MiB" print_build_stage "Not enough host memory (less than 1024 MiB, consider upgrading)," print_build_stage "building will use standard XZ options." XZ_OPT="-T0" # alte Standardparameter einsetzen
echo read -p "Press [Enter] key to continue..."
fi ... # XZ-TUNING END
***SNAP***
... I think there should be a warning if not enough memory is available, but we should still try to build.
As far as I can see, this could be done by replacing the above 'echoes' with 'print_build_stage'(?).
E.g.: print_build_stage "Not enough host memory (less than 1024MiB, consider upgrading)," print_build_stage "building will use standard XZ options."
...
But. ;-)
You can put this probably next under "HOST_MEM=$(system_memory)" and then export this to the internal shell by adding the variable to lfsmake2 for the disk image and ipfiredist for the packages.
Sorry, I must confess that I don't know exactly how to do this. As I wrote: "How do I pass the 'XZ_MEM'-variable to 'cdrom' and 'Config'?" Or in this case, XZ_OPT!?
And this is still my problem! I don't get this to work. How do I "pass" this variable?
Then in lfs/cdrom and lfs/Config for the dist target, you just need to add the parameters to the command line.
In 'Config' I changed:
... cd /install/packages/package/tmp/ && XZ_OPT=-T0 tar -c -p --numeric-owner -J -f /install/packages/package/files.tar.xz * ...
To: ... cd /install/packages/package/tmp/ && $XZ_OPT tar -c -p --numeric-owner -J -f /install/packages/package/files.tar.xz * ...
Maybe it is a good idea to have a variable XZ_OPT that adds everything together and is then used in both places.
I tried that with no luck. I added the variable '$XZ_OPT' to 'lfs/Config' in line 212:
cd /install/packages/package/tmp/ && $XZ_OPT tar -c -p --numeric-owner...
But this usually ends - like today - in '_build_packages.log' with:
***SNIP*** ... #Change xxxKVERxxx to Kernelversion sed -e "s/xxxKVERxxx/4.14.40/g" -i /install/packages/package/update.sh cd /install/packages/package && tar cf ../core-upgrade-2.19-$(basename core/121).ipfire \ update.sh files.tar.xz ROOTFILES rm -rf /install/packages/package sed -e "s/NAME/core-upgrade/g" \ -e "s/VER/2.19/g" \ -e "s/RELEASE/$(basename core/121)/g" \ -e "s/DEPS//g" \ -e "s/SIZE/`ls -l /install/packages/core-upgrade-2.19-$(basename core/121).ipfire | awk '{ print $5 }'`/g" \ < /usr/src/src/pakfire/meta > /install/packages/meta-core-upgrade-$(basename core/121) May 12 13:39:49: Building directfb DirectFB-1.7.7.tar.gz checksum OK + cd /usr/src/lfs + make -f directfb LFS_BASEDIR=/usr/src dist '/usr/src/config/rootfiles/packages/i586/directfb' -> '/install/packages/package/ROOTFILES' /bin/sh: Z_OPT: command not found make: *** [directfb:60: dist] Error 127 ... ***SNAP***
Why "Z_OPT"?
I added the other xz-options and created XZ_OPT as shown above, but in 'cdrom' there is an 'export XZ_OPT = ...'-line and I don't know how to handle these line. Is it still necessary?
No, that is no longer necessary and can be removed. We should just add it to the tar command just like it is done in lfs/Config.
Understanding problem: What do mean with "adding it to the tar command"? For me, this sounds as if we need an "export $XZ_OPT"-line here...
Do I have to 'export' the XZ_OPT'-variable in 'make.sh'?
No, because the chroot environment will throw away all environment variables from the host system and then set them again. That's why we have this enterchroot() function that does this with the env command.
And how do I "add" this variable to 'lfsmake2' and 'ipfiredist'(-function)?
Just have a look at the functions. There should be loads of examples where we pass other variables and you just need to add this one, too.
I think I've found these "examples" and tried several "local XZ_OPT='$XZ_OPT'" placements and versions in 'lfsmake2' and 'ipfiredist' but none seemed to work as I wanted.
Sorry if these questions sound simple, but I'm guessing a bit too much while trying to get a grip on this.
This is absolutely fine to sort this out first and then send the patch :) That's what this list is for.
After several attempts I stopped guessing and concentrated on something I could handle. Somehow I've got the feeling I'm making some really simple mistakes - for reviewing I added my current test patches as attachments.
Please advise... ;-)
Best, Matthias