Problem with ipfire build system

Problem with ipfire build system

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 1570 bytes --]

Hi Michael,

I have experienced a problem yesterday and today with the IPFire build system. Not a major problem but it only seems to have started recently.

Today when I tried to run the build command with the standard IPFire-2.x git clone I got the following error message.

./make.sh build

ERROR: Not enough temporary space available, need at least 8192MiB, but only have 491 [ FAIL ]
     Check /home/ahb/sandbox/ipfire-2.x/log_x86_64/_build.preparation.log for errors if applicable [ FAIL ]

I was able to overcome it by running the ./make.sh clean command on the openvpn-2.6-meetup-rebased branch git repo that I also have on my system.

After running the clean command the normal ipfire-2.x git repo build command is working fine and no longer gives the above message.


Yesterday it was the other way around. I tried to run the build command on the openvpn-2.6-meetup-rebased branch and got the same sort of message as above just the amount available was a bit different. Something like 670 instead of 491 today. Yesterday, when I ran the clean command on my normal ipfire-2.x git repo then the openvpn-2.6-meetup-rebased git repo had no problem with the build. Just doing a reboot on my desktop machine that has the git repo's on it made no difference.

I have not had this problem in the past. Although not a major problem, I have definitely run a build on one repo when the other still had its build results in place so something has changed. Maybe something in Arch Linux with one of the recent updates I ran.

Regards,

Adolf.

Re: Problem with ipfire build system

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 2688 bytes --]

Hello Adolf,

> On 8 Dec 2024, at 12:04, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> 
> Hi Michael,
> 
> I have experienced a problem yesterday and today with the IPFire build system. Not a major problem but it only seems to have started recently.
> 
> Today when I tried to run the build command with the standard IPFire-2.x git clone I got the following error message.
> 
> ./make.sh build
> 
> ERROR: Not enough temporary space available, need at least 8192MiB, but only have 491 [ FAIL ]
>     Check /home/ahb/sandbox/ipfire-2.x/log_x86_64/_build.preparation.log for errors if applicable [ FAIL ]

So, there is a check that will ensure that there is 8 GiB available for the build directory so that (ideally) the build does not abort because of low disk space.

This is however a little bit of a gamble because we don’t always know exactly how much (temporary) space the build environment might need and it is a little bit difficult if a build is being restarted (because we might have consumed a chunk of the 8 GiB already).

Could you send me the output of “df -h” and “du -csh build_x86_64” when this problem happens?

> I was able to overcome it by running the ./make.sh clean command on the openvpn-2.6-meetup-rebased branch git repo that I also have on my system.
> 
> After running the clean command the normal ipfire-2.x git repo build command is working fine and no longer gives the above message.

It sounds like the calculation works well when there is no build directory. That is the easy case.

> Yesterday it was the other way around. I tried to run the build command on the openvpn-2.6-meetup-rebased branch and got the same sort of message as above just the amount available was a bit different. Something like 670 instead of 491 today. Yesterday, when I ran the clean command on my normal ipfire-2.x git repo then the openvpn-2.6-meetup-rebased git repo had no problem with the build. Just doing a reboot on my desktop machine that has the git repo's on it made no difference.

Hmm, is it possible that the build environment has gone so large that 8 GiB is exceeded and my calculation goes into negative numbers? That might break things hard.

> I have not had this problem in the past. Although not a major problem, I have definitely run a build on one repo when the other still had its build results in place so something has changed. Maybe something in Arch Linux with one of the recent updates I ran.

It shouldn’t. This code was changed when we introduced the containerised build system earlier this year. So I am quite confident that I broke this myself.

Best,
-Michael

> Regards,
> 
> Adolf.
> 
> 

Re: Problem with ipfire build system

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 3612 bytes --]

Hi Michael,

On 09/12/2024 12:48, Michael Tremer wrote:
> Hello Adolf,
> 
>> On 8 Dec 2024, at 12:04, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>
>> Hi Michael,
>>
>> I have experienced a problem yesterday and today with the IPFire build system. Not a major problem but it only seems to have started recently.
>>
>> Today when I tried to run the build command with the standard IPFire-2.x git clone I got the following error message.
>>
>> ./make.sh build
>>
>> ERROR: Not enough temporary space available, need at least 8192MiB, but only have 491 [ FAIL ]
>>      Check /home/ahb/sandbox/ipfire-2.x/log_x86_64/_build.preparation.log for errors if applicable [ FAIL ]
> 
> So, there is a check that will ensure that there is 8 GiB available for the build directory so that (ideally) the build does not abort because of low disk space.
> 
> This is however a little bit of a gamble because we don’t always know exactly how much (temporary) space the build environment might need and it is a little bit difficult if a build is being restarted (because we might have consumed a chunk of the 8 GiB already).
> 
> Could you send me the output of “df -h” and “du -csh build_x86_64” when this problem happens?

I managed to get the same problem to occur. Then running df -h told me where my problem was and it is self made.

My home directory, where the git repo's are in is nearly full. With one repo with a full build directory I only had 4.9G left (99% full) not enough space for the 8G.

When I cleaned the other build directory then my home directory had 14G available (97% full) and so enough space for the 8G.

So I don't currently have enough space for both of the git repos to be getting built.

I will need to search out what is using up the space as normally most things are on my file server, which includes most of my vm definitions. I will need to figure out what has filled up the space over time and either get rid of it or move it elsewhere.

Sorry for the noise.

Regards,
Adolf.

> 
>> I was able to overcome it by running the ./make.sh clean command on the openvpn-2.6-meetup-rebased branch git repo that I also have on my system.
>>
>> After running the clean command the normal ipfire-2.x git repo build command is working fine and no longer gives the above message.
> 
> It sounds like the calculation works well when there is no build directory. That is the easy case.
> 
>> Yesterday it was the other way around. I tried to run the build command on the openvpn-2.6-meetup-rebased branch and got the same sort of message as above just the amount available was a bit different. Something like 670 instead of 491 today. Yesterday, when I ran the clean command on my normal ipfire-2.x git repo then the openvpn-2.6-meetup-rebased git repo had no problem with the build. Just doing a reboot on my desktop machine that has the git repo's on it made no difference.
> 
> Hmm, is it possible that the build environment has gone so large that 8 GiB is exceeded and my calculation goes into negative numbers? That might break things hard.
> 
>> I have not had this problem in the past. Although not a major problem, I have definitely run a build on one repo when the other still had its build results in place so something has changed. Maybe something in Arch Linux with one of the recent updates I ran.
> 
> It shouldn’t. This code was changed when we introduced the containerised build system earlier this year. So I am quite confident that I broke this myself.
> 
> Best,
> -Michael
> 
>> Regards,
>>
>> Adolf.
>>
>>
> 

Re: Problem with ipfire build system

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 4475 bytes --]

Hi Michael,

I found what had been using up all the space. The Expunged directory in my Trash folder for my home system was full of all the files from running the ./make clean command.

As those files are owned by root they get put in the Trash but clearing the Trash Icon only clears all the trash that is owned by my user but it doesn't show that there is a lot of trash still there owned by root.

It turned out that all those clean commands had built up to 108GB.

So I have now cleared all the directories in the expunged directory and I now have free space of 108G and can run two builds at the same time without any problem.

I just need to remember to check the expunged directory in the Trash every now and then.

Regards,

Adolf.

On 09/12/2024 13:43, Adolf Belka wrote:
> Hi Michael,
>
> On 09/12/2024 12:48, Michael Tremer wrote:
>> Hello Adolf,
>>
>>> On 8 Dec 2024, at 12:04, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>
>>> Hi Michael,
>>>
>>> I have experienced a problem yesterday and today with the IPFire build system. Not a major problem but it only seems to have started recently.
>>>
>>> Today when I tried to run the build command with the standard IPFire-2.x git clone I got the following error message.
>>>
>>> ./make.sh build
>>>
>>> ERROR: Not enough temporary space available, need at least 8192MiB, but only have 491 [ FAIL ]
>>>      Check /home/ahb/sandbox/ipfire-2.x/log_x86_64/_build.preparation.log for errors if applicable [ FAIL ]
>>
>> So, there is a check that will ensure that there is 8 GiB available for the build directory so that (ideally) the build does not abort because of low disk space.
>>
>> This is however a little bit of a gamble because we don’t always know exactly how much (temporary) space the build environment might need and it is a little bit difficult if a build is being restarted (because we might have consumed a chunk of the 8 GiB already).
>>
>> Could you send me the output of “df -h” and “du -csh build_x86_64” when this problem happens?
>
> I managed to get the same problem to occur. Then running df -h told me where my problem was and it is self made.
>
> My home directory, where the git repo's are in is nearly full. With one repo with a full build directory I only had 4.9G left (99% full) not enough space for the 8G.
>
> When I cleaned the other build directory then my home directory had 14G available (97% full) and so enough space for the 8G.
>
> So I don't currently have enough space for both of the git repos to be getting built.
>
> I will need to search out what is using up the space as normally most things are on my file server, which includes most of my vm definitions. I will need to figure out what has filled up the space over time and either get rid of it or move it elsewhere.
>
> Sorry for the noise.
>
> Regards,
> Adolf.
>
>>
>>> I was able to overcome it by running the ./make.sh clean command on the openvpn-2.6-meetup-rebased branch git repo that I also have on my system.
>>>
>>> After running the clean command the normal ipfire-2.x git repo build command is working fine and no longer gives the above message.
>>
>> It sounds like the calculation works well when there is no build directory. That is the easy case.
>>
>>> Yesterday it was the other way around. I tried to run the build command on the openvpn-2.6-meetup-rebased branch and got the same sort of message as above just the amount available was a bit different. Something like 670 instead of 491 today. Yesterday, when I ran the clean command on my normal ipfire-2.x git repo then the openvpn-2.6-meetup-rebased git repo had no problem with the build. Just doing a reboot on my desktop machine that has the git repo's on it made no difference.
>>
>> Hmm, is it possible that the build environment has gone so large that 8 GiB is exceeded and my calculation goes into negative numbers? That might break things hard.
>>
>>> I have not had this problem in the past. Although not a major problem, I have definitely run a build on one repo when the other still had its build results in place so something has changed. Maybe something in Arch Linux with one of the recent updates I ran.
>>
>> It shouldn’t. This code was changed when we introduced the containerised build system earlier this year. So I am quite confident that I broke this myself.
>>
>> Best,
>> -Michael
>>
>>> Regards,
>>>
>>> Adolf.
>>>
>>>
>>
>