From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer To: development@lists.ipfire.org Subject: Re: [PATCH 0/1] Parallelized build for several packages Date: Wed, 06 Mar 2019 16:36:44 +0000 Message-ID: In-Reply-To: <00aa01d4ca0a$d4cf8300$7e6e8900$@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5785870771394066936==" List-Id: --===============5785870771394066936== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi, Sorry for not being able to get around to this earlier, but I have invested s= ome time on this: First of all, I merged Wolfgang=E2=80=99s patch. He is right, the build is to= o slow. It sometimes looks a bit like it is not taking enough advantage of fa= st systems. On the other hand with have small systems, which are (in their ow= n right) quite fast. Those are the ARM systems. They usually have 1G of RAM. = 2GB when you are lucky. So I adjusted make.sh a little bit and removed composing the MAKETUNING varia= ble from it. Now, make.sh only figures out the value that usually comes after the -j. It i= s set in DEFAULT_PARALLELISM. In the individual LFS files, we can now set a new variable called MAX_PARALLE= LISM which is optional. In boost, I now calculate this from the amount of mem= ory that is available. That will make sure that we have a cap on this that wi= ll be high on systems that have the memory and low on those that don=E2=80=99= t. Before, we hardcoded -j2 which of course runs very slow on large systems. The code is here: https://git.ipfire.org/?p=3Dpeople/ms/ipfire-2.x.git;a=3Dshortlog;h=3Drefs/= heads/faster-build=20 Please review. I am now running this on a couple of builders and will see wha= t comes out. Please do the same and send me feedback. Best, -Michael > On 21 Feb 2019, at 17:28, Wolfgang Apolinarski wrote: >=20 > Hi Arne, >=20 > of course I did delete the ccache between my multiple builds to verify the = commit. To play it safe, I hard reset using git, checked that all folders (bu= ild, ccache, etc.) are gone and restarted the build after gettoolchain and do= wnloadsrc. >=20 > Sorry, I did not know that there are still a lot of machines with low ram t= hat we are using to build ipfire. My hope was that these machines have disapp= eared over the last years. > Additionally, I do not have the ability to test all ipfire arches, as menti= oned, the patch needs testing by others. >=20 > I also thought of adding a manual parameter for developers that somehow man= ually sets the parameters for parallel build, but since the MAKETUNING parame= ter already exists and is used with other packages this would just somehow me= ss everything up. >=20 > For my builds, I can just manually apply the patch to speed up the full bui= ld. My thought was that sharing the patch would make sense for others. >=20 >> -----Urspr=C3=BCngliche Nachricht----- >> Von: Arne Fitzenreiter >> Gesendet: Donnerstag, 21. Februar 2019 09:44 >> An: Wolfgang Apolinarski >> Cc: development(a)lists.ipfire.org >> Betreff: Re: [PATCH 0/1] Parallelized build for several packages >>=20 >> On the most of this packages (eg. boot and cmake) i have disabled >> parallel build because it fails on machines with low ram (less than 2GB) >> or on arm. >> So such changes need tested on an 1GB machine, on all architectures and >> build two times after deleting ccache. Because the build need much lower >> resources if it has the file already prebuilt in cache. >=20 --===============5785870771394066936==--