Improving build performance (on slow storage)

Improving build performance (on slow storage)

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 955 bytes --]

Hello,

I just wanted to post you an update about the build system which we have recently merged.

The whole build process is primarily CPU bound and secondarily IO bound. Some builders have by definition slow storage (the ARM SBCs for example), but also for other reason it does not make sense to run the build on persistent storage. We can keep it in RAM instead which makes accessing files a lot faster and saves write cycles on the media.

Therefore, the make.sh build script is mounting /usr/src in the build environment as a ramdisk. All source will be extracted there and the build will be performed. After the build is done, the whole ramdisk will be destroyed again.

This feature is automatically enabled on all build systems which have 4GB of RAM or more. You can manually disable it with the ENABLE_RAMDISK=off switch in .config.

I hope this brief email makes sense. If you have any questions, feel free to ask.

Best,
-Michael

Re: Improving build performance (on slow storage)

[Matthias Fischer]

[-- Attachment #1: Type: text/plain, Size: 578 bytes --]

On 13.06.2019 17:17, Michael Tremer wrote:
> Hello,
> 
> I just wanted to post you an update about the build system which we have recently merged.
> ...

> Therefore, the make.sh build script is mounting /usr/src in the build environment as a ramdisk. All source will be extracted there and the build will be performed. After the build is done, the whole ramdisk will be destroyed again.
> ...
> I hope this brief email makes sense...

Yes it does. ;-)

FYI:
Last build (Core 133) here completed in 4:22:14.

Significantly faster than before.

Best,
Matthias

Re: Improving build performance (on slow storage)

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 736 bytes --]

Cool. What was the time before?

> On 15 Jun 2019, at 11:03, Matthias Fischer <matthias.fischer(a)ipfire.org> wrote:
> 
>> On 13.06.2019 17:17, Michael Tremer wrote:
>> Hello,
>> 
>> I just wanted to post you an update about the build system which we have recently merged.
>> ...
> 
>> Therefore, the make.sh build script is mounting /usr/src in the build environment as a ramdisk. All source will be extracted there and the build will be performed. After the build is done, the whole ramdisk will be destroyed again.
>> ...
>> I hope this brief email makes sense...
> 
> Yes it does. ;-)
> 
> FYI:
> Last build (Core 133) here completed in 4:22:14.
> 
> Significantly faster than before.
> 
> Best,
> Matthias

Re: Improving build performance (on slow storage)

[Matthias Fischer]

[-- Attachment #1: Type: text/plain, Size: 265 bytes --]

On 15.06.2019 15:51, Michael Tremer wrote:
> Cool. What was the time before?
> 

Normally between 6-8 hours, depending on branch and cache condition.

This was a "clean" build with Core 133 - we'll see if it stays that way.
I'm happy, so far. ;-)

Best,
Matthias

Re: Improving build performance (on slow storage)

[Matthias Fischer]

[-- Attachment #1: Type: text/plain, Size: 515 bytes --]

Hi,

On 13.06.2019 17:17, Michael Tremer wrote:
> Hello,
> 
> I just wanted to post you an update about the build system which we have recently merged.
> 
> ...

I'm bringing this thread up again because beginning with Core 136, I
notice a significant deceleration.

Last - clean - build from tonight:

...
Checking Logfiles for new Files
*** Build finished in 12:28:46
...

That's a new record. Build time with Core 135 was about 7-8 hours.

Is it the same for others?

Best,
Matthias

Re: Improving build performance (on slow storage)

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 811 bytes --]

Hi,

Is that a repeated build?

Do you know which package took extremely long?

The nightly builds seem to be fine at around 4:30 hrs.

-Michael

> On 17 Sep 2019, at 12:18, Matthias Fischer <matthias.fischer(a)ipfire.org> wrote:
> 
> Hi,
> 
> On 13.06.2019 17:17, Michael Tremer wrote:
>> Hello,
>> 
>> I just wanted to post you an update about the build system which we have recently merged.
>> 
>> ...
> 
> I'm bringing this thread up again because beginning with Core 136, I
> notice a significant deceleration.
> 
> Last - clean - build from tonight:
> 
> ...
> Checking Logfiles for new Files
> *** Build finished in 12:28:46
> ...
> 
> That's a new record. Build time with Core 135 was about 7-8 hours.
> 
> Is it the same for others?
> 
> Best,
> Matthias
> 
> 

Re: Improving build performance (on slow storage)

[Arne Fitzenreiter]

[-- Attachment #1: Type: text/plain, Size: 1038 bytes --]

If this was the first core136 build it is normal because it has a new 
toolchain package and this invalidate the ccache so
all has to rebuild without cache.

Arne


Am 2019-09-17 22:30, schrieb Michael Tremer:
> Hi,
> 
> Is that a repeated build?
> 
> Do you know which package took extremely long?
> 
> The nightly builds seem to be fine at around 4:30 hrs.
> 
> -Michael
> 
>> On 17 Sep 2019, at 12:18, Matthias Fischer 
>> <matthias.fischer(a)ipfire.org> wrote:
>> 
>> Hi,
>> 
>> On 13.06.2019 17:17, Michael Tremer wrote:
>>> Hello,
>>> 
>>> I just wanted to post you an update about the build system which we 
>>> have recently merged.
>>> 
>>> ...
>> 
>> I'm bringing this thread up again because beginning with Core 136, I
>> notice a significant deceleration.
>> 
>> Last - clean - build from tonight:
>> 
>> ...
>> Checking Logfiles for new Files
>> *** Build finished in 12:28:46
>> ...
>> 
>> That's a new record. Build time with Core 135 was about 7-8 hours.
>> 
>> Is it the same for others?
>> 
>> Best,
>> Matthias
>> 
>> 

Re: Improving build performance (on slow storage)

[Matthias Fischer]

[-- Attachment #1: Type: text/plain, Size: 1045 bytes --]

On 17.09.2019 22:30, Michael Tremer wrote:
> Hi,
> 
> Is that a repeated build?

Why did I know you would ask this question... ;-))

No, it was the first.

> Do you know which package took extremely long?

No, but I'll take a look - I kept the logs.

> The nightly builds seem to be fine at around 4:30 hrs.

I'll see.

> -Michael
> 
>> On 17 Sep 2019, at 12:18, Matthias Fischer <matthias.fischer(a)ipfire.org> wrote:
>> 
>> Hi,
>> 
>> On 13.06.2019 17:17, Michael Tremer wrote:
>>> Hello,
>>> 
>>> I just wanted to post you an update about the build system which we have recently merged.
>>> 
>>> ...
>> 
>> I'm bringing this thread up again because beginning with Core 136, I
>> notice a significant deceleration.
>> 
>> Last - clean - build from tonight:
>> 
>> ...
>> Checking Logfiles for new Files
>> *** Build finished in 12:28:46
>> ...
>> 
>> That's a new record. Build time with Core 135 was about 7-8 hours.
>> 
>> Is it the same for others?
>> 
>> Best,
>> Matthias
>> 
>> 
> 
> 

Re: Improving build performance (on slow storage)

[Matthias Fischer]

[-- Attachment #1: Type: text/plain, Size: 255 bytes --]

On 18.09.2019 16:31, Arne Fitzenreiter wrote:
> If this was the first core136 build it is normal because it has a new 
> toolchain package and this invalidate the ccache so
> all has to rebuild without cache.

Ok, I'll watch the next one.

Best,
Matthias