Hello,
any progress on this?
-Michael
On Tue, 2017-11-07 at 22:47 +0000, Michael Tremer wrote:
Hi,
On Tue, 2017-11-07 at 19:41 +0100, Matthias Fischer wrote:
Hi,
On 07.11.2017 16:05, Michael Tremer wrote:
Hi,
maybe this function helps:
https://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=5190eea24f9822a63d c5d06d214b48f973b14f29
Thanks. I'll take a look...
On Tue, 2017-11-07 at 10:46 +0000, Michael Tremer wrote:
On Mon, 2017-11-06 at 22:09 +0100, Matthias Fischer wrote: ...
I forgot: Tuning the XZ-parameters... ;-)
Good that you are bringing this up. So I think what we should do here is bringing back XZ compression with -8 so that we do not waste too much space when extracting the image again.
Right now I'm testing how much RAM I need.
Current 'next'-build is running with:
'lfs/cdrom': ... export XZ_OPT = --threads=0 -8 --memory=500MiB ...
'lfs/Config': ... cd /install/packages/package/tmp/ && XZ_OPT="-T0 -8" ... ...
I'm waiting for the results. Takes a while.
You can just run it like this and it will tell you:
[ms@hughes ~]$ xz -vv8 -T0 xz: Filter chain: -- lzma2=dict=32MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0 xz: Using up to 4 threads. xz: 2,629 MiB of memory is required. The limiter is disabled. xz: Decompression will need 33 MiB of memory. xz: Compressed data cannot be written to a terminal xz: Try `xz --help' for more information.
To not run into memory limits we should detect how much memory the build host has and set the memory limit to maybe 80% of that. xz will then automatically scale down to a number of parallel processes that it can fit into memory.
I'm a bit puzzled about this: I tried using '--memory=70%' as mentioned in the xz man pages, but is seems to have no effect. It looks as if the percent parameter isn't recognized and ignored. It always ends up in "cannot allocate memory". Any hints on this? Do both 'lfs/cdrom' AND 'lfs/Config' need the '--memory'-parameter?
Yes, but it would be good to have one place where the parameters are being generated and then we just use them from a variable.
... We will at least need about half a GB which should be a sensible requirement for a build system any ways.
Agreed. Because of this I'm testing with '500Mib'.
Will you work on this bringing it into the build system?!
"I'll do my very best!" :-))
Very well Miss Sophie...
Best™, Matthias