Re: Errors during update to Core165 found in '/var/log/httpd/error_log'

Re: Errors during update to Core165 found in '/var/log/httpd/error_log'

[Matthias Fischer]

[-- Attachment #1: Type: text/plain, Size: 2596 bytes --]

Hi,

just did the update to Core 166.

Checked the logs: no more errors: everything's clean.

Best,
Matthias

On 31.03.2022 21:33, Bernhard Bitsch wrote:
> Hi,
> 
> I had some 'gimmicks' during the update process also.
> But I regarded them as a 'price' of my early update try ( the mirrors 
> didn't show the CU first).
> 
> Looking at Matthias' observation lets guess me, that the Perl modules 
> were installed with the update first. Perl doesn't reread the include 
> namespace. Therefore the modules are know not earlier than after a 
> restart/reboot.
> 
> Regards,
> Bernhard
> 
> Am 31.03.2022 um 20:52 schrieb Matthias Fischer:
>> Hi,
>> 
>> I've found the missing files:
>> 
>> ...
>> root(a)ipfire: / # find -name Ping.pm
>> ./usr/lib/perl5/5.32.1/Net/Ping.pm
>> root(a)ipfire: / # find -name Copy.pm
>> ./usr/lib/perl5/5.32.1/File/Copy.pm
>> ...
>> 
>> It seems, 'pakfire.cgi' searches in the wrong directory...
>> 
>> But I must confess - I have no idea how to fix this.
>> 
>> Best,
>> Matthias
>> 
>> On 31.03.2022 19:22, Adolf Belka wrote:
>>> Hi Matthias,
>>>
>>> I have found similar messages on my system but some of them are from around when I did the upgrade from CU164 to CU165.
>>>
>>>
>>> Additionally I have the same messages with earlier time stamps, previous day, which would have been from when it was still running as CU164.
>>>
>>> I have just checked the timestamp for my CU165 log and it is after the messages. So all those messages were from when IPFire was running as CU164.
>>>
>>>
>>> Regards,
>>>
>>> Adolf
>>>
>>> On 31/03/2022 19:09, Matthias Fischer wrote:
>>>> Hi,
>>>>
>>>> first: update to Core 165 ran through - no seen errors. Pakfire looked
>>>> good, system is running with .
>>>>
>>>> But after digging deeper in the logs I found some errors that made me
>>>> ...nervous...:
>>>>
>>>> Looking at '/var/log/httpd/error_log' I found a bunch of "tac: write
>>>> error: Broken pipe", some "Permission denied:"-messages regarding
>>>> 'pakfire.cgi', some Perl errors and a "Could not open password file:
>>>> /var/ipfire/auth/users"-error. I wouldn't have noticed if I hadn't take
>>>> a closer look.
>>>>
>>>> In detail:
>>>> Perl couldn't locate two files - "Net/Ping.pm" and "File/Copy.pm" needed
>>>> by '/opt/pakfire/lib/functions.pl'. These doesn't exist.
>>>>
>>>> Current log file ('/var/log/httpd/error_log') is attached.
>>>>
>>>> Nevertheless - system is up and running. WebGui is running.
>>>>
>>>> Can anyone confirm?
>>>>
>>>> Best,
>>>> Matthias
>> 

Re: Errors during update to Core165 found in '/var/log/httpd/error_log'

[Leo Hofmann]

[-- Attachment #1: Type: text/plain, Size: 4566 bytes --]

Hi all,

Jon and I have been working on a patchset to fix a bug he found in the UI.
Unfortunately we are now facing the same problems you discovered earlier. For me, this happens often while installing bigger packages like 'samba' or 'epson-inkjet-printer-escpr'.

Everything works fine, until at some point Perl randomly fails with errors like this:

Can't locate CGI.pm: /usr/lib/perl5/site_perl/5.32.1/x86_64-linux-thread-multi/CGI.pm: Permission denied at /var/ipfire/header.pl line 14.
Can't locate URI/_generic.pm: /usr/lib/perl5/site_perl/5.32.1/x86_64-linux-thread-multi/URI/_generic.pm: Permission denied at /usr/lib/perl5/site_perl/5.32.1/URI.pm line 88.
sh: error while loading shared libraries: libreadline.so.8: cannot open shared object file: Permission denied

This problem is not limited to pakfire.cgi either. I have seen the simple speed.cgi fail as well while I was running "pakfire install samba" on the console:

[cgid:error] [pid 17848:tid 137777085163392] (13)Permission denied: AH01241: exec of '/srv/web/ipfire/cgi-bin/speed.cgi' failed
[cgid:error] [pid 18396:tid 137776871712320] [client 192.168.144.41:5726] End of script output before headers: speed.cgi, referer: ...

Luckily this only affects the web interface, the Pakfire process won't be disrupted.

I have to admit that this problem has been bothering me for a long time and I'm all out of ideas. The modules and paths are all there and work fine 99.999% of the time.
Logging the most common environment variables (ENV, INC, UID) from within pakfire.cgi didn't reveal anything obvious, in fact they are unchanged every time.

Has anyone seen this before, is this a known issue? Is this to be expected because Pakfire is modifying the system?
I want to make sure that this isn't a hidden bug in my pakfire.cgi changes!

Best regards,
Leo

Am 01.04.2022 um 17:55 schrieb Matthias Fischer:
> Hi,
>
> just did the update to Core 166.
>
> Checked the logs: no more errors: everything's clean.
>
> Best,
> Matthias
>
> On 31.03.2022 21:33, Bernhard Bitsch wrote:
>> Hi,
>>
>> I had some 'gimmicks' during the update process also.
>> But I regarded them as a 'price' of my early update try ( the mirrors
>> didn't show the CU first).
>>
>> Looking at Matthias' observation lets guess me, that the Perl modules
>> were installed with the update first. Perl doesn't reread the include
>> namespace. Therefore the modules are know not earlier than after a
>> restart/reboot.
>>
>> Regards,
>> Bernhard
>>
>> Am 31.03.2022 um 20:52 schrieb Matthias Fischer:
>>> Hi,
>>>
>>> I've found the missing files:
>>>
>>> ...
>>> root(a)ipfire: / # find -name Ping.pm
>>> ./usr/lib/perl5/5.32.1/Net/Ping.pm
>>> root(a)ipfire: / # find -name Copy.pm
>>> ./usr/lib/perl5/5.32.1/File/Copy.pm
>>> ...
>>>
>>> It seems, 'pakfire.cgi' searches in the wrong directory...
>>>
>>> But I must confess - I have no idea how to fix this.
>>>
>>> Best,
>>> Matthias
>>>
>>> On 31.03.2022 19:22, Adolf Belka wrote:
>>>> Hi Matthias,
>>>>
>>>> I have found similar messages on my system but some of them are from around when I did the upgrade from CU164 to CU165.
>>>>
>>>>
>>>> Additionally I have the same messages with earlier time stamps, previous day, which would have been from when it was still running as CU164.
>>>>
>>>> I have just checked the timestamp for my CU165 log and it is after the messages. So all those messages were from when IPFire was running as CU164.
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Adolf
>>>>
>>>> On 31/03/2022 19:09, Matthias Fischer wrote:
>>>>> Hi,
>>>>>
>>>>> first: update to Core 165 ran through - no seen errors. Pakfire looked
>>>>> good, system is running with .
>>>>>
>>>>> But after digging deeper in the logs I found some errors that made me
>>>>> ...nervous...:
>>>>>
>>>>> Looking at '/var/log/httpd/error_log' I found a bunch of "tac: write
>>>>> error: Broken pipe", some "Permission denied:"-messages regarding
>>>>> 'pakfire.cgi', some Perl errors and a "Could not open password file:
>>>>> /var/ipfire/auth/users"-error. I wouldn't have noticed if I hadn't take
>>>>> a closer look.
>>>>>
>>>>> In detail:
>>>>> Perl couldn't locate two files - "Net/Ping.pm" and "File/Copy.pm" needed
>>>>> by '/opt/pakfire/lib/functions.pl'. These doesn't exist.
>>>>>
>>>>> Current log file ('/var/log/httpd/error_log') is attached.
>>>>>
>>>>> Nevertheless - system is up and running. WebGui is running.
>>>>>
>>>>> Can anyone confirm?
>>>>>
>>>>> Best,
>>>>> Matthias

Re: Errors during update to Core165 found in '/var/log/httpd/error_log'

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 5084 bytes --]

Hello,

I have been writing on this list what I think the problem is.

This was supposed to fix it: https://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=c639ab1d31d51b534539ea27516c672cb9f3ec41

I was however not able to easily reproduce this.

-Michael

> On 7 May 2022, at 13:50, Leo Hofmann <hofmann(a)leo-andres.de> wrote:
> 
> Hi all,
> 
> Jon and I have been working on a patchset to fix a bug he found in the UI.
> Unfortunately we are now facing the same problems you discovered earlier. For me, this happens often while installing bigger packages like 'samba' or 'epson-inkjet-printer-escpr'.
> 
> Everything works fine, until at some point Perl randomly fails with errors like this:
> 
> Can't locate CGI.pm: /usr/lib/perl5/site_perl/5.32.1/x86_64-linux-thread-multi/CGI.pm: Permission denied at /var/ipfire/header.pl line 14.
> Can't locate URI/_generic.pm: /usr/lib/perl5/site_perl/5.32.1/x86_64-linux-thread-multi/URI/_generic.pm: Permission denied at /usr/lib/perl5/site_perl/5.32.1/URI.pm line 88.
> sh: error while loading shared libraries: libreadline.so.8: cannot open shared object file: Permission denied
> 
> This problem is not limited to pakfire.cgi either. I have seen the simple speed.cgi fail as well while I was running "pakfire install samba" on the console:
> 
> [cgid:error] [pid 17848:tid 137777085163392] (13)Permission denied: AH01241: exec of '/srv/web/ipfire/cgi-bin/speed.cgi' failed
> [cgid:error] [pid 18396:tid 137776871712320] [client 192.168.144.41:5726] End of script output before headers: speed.cgi, referer: ...
> 
> Luckily this only affects the web interface, the Pakfire process won't be disrupted.
> 
> I have to admit that this problem has been bothering me for a long time and I'm all out of ideas. The modules and paths are all there and work fine 99.999% of the time.
> Logging the most common environment variables (ENV, INC, UID) from within pakfire.cgi didn't reveal anything obvious, in fact they are unchanged every time.
> 
> Has anyone seen this before, is this a known issue? Is this to be expected because Pakfire is modifying the system?
> I want to make sure that this isn't a hidden bug in my pakfire.cgi changes!
> 
> Best regards,
> Leo
> 
> Am 01.04.2022 um 17:55 schrieb Matthias Fischer:
>> Hi,
>> 
>> just did the update to Core 166.
>> 
>> Checked the logs: no more errors: everything's clean.
>> 
>> Best,
>> Matthias
>> 
>> On 31.03.2022 21:33, Bernhard Bitsch wrote:
>>> Hi,
>>> 
>>> I had some 'gimmicks' during the update process also.
>>> But I regarded them as a 'price' of my early update try ( the mirrors
>>> didn't show the CU first).
>>> 
>>> Looking at Matthias' observation lets guess me, that the Perl modules
>>> were installed with the update first. Perl doesn't reread the include
>>> namespace. Therefore the modules are know not earlier than after a
>>> restart/reboot.
>>> 
>>> Regards,
>>> Bernhard
>>> 
>>> Am 31.03.2022 um 20:52 schrieb Matthias Fischer:
>>>> Hi,
>>>> 
>>>> I've found the missing files:
>>>> 
>>>> ...
>>>> root(a)ipfire: / # find -name Ping.pm
>>>> ./usr/lib/perl5/5.32.1/Net/Ping.pm
>>>> root(a)ipfire: / # find -name Copy.pm
>>>> ./usr/lib/perl5/5.32.1/File/Copy.pm
>>>> ...
>>>> 
>>>> It seems, 'pakfire.cgi' searches in the wrong directory...
>>>> 
>>>> But I must confess - I have no idea how to fix this.
>>>> 
>>>> Best,
>>>> Matthias
>>>> 
>>>> On 31.03.2022 19:22, Adolf Belka wrote:
>>>>> Hi Matthias,
>>>>> 
>>>>> I have found similar messages on my system but some of them are from around when I did the upgrade from CU164 to CU165.
>>>>> 
>>>>> 
>>>>> Additionally I have the same messages with earlier time stamps, previous day, which would have been from when it was still running as CU164.
>>>>> 
>>>>> I have just checked the timestamp for my CU165 log and it is after the messages. So all those messages were from when IPFire was running as CU164.
>>>>> 
>>>>> 
>>>>> Regards,
>>>>> 
>>>>> Adolf
>>>>> 
>>>>> On 31/03/2022 19:09, Matthias Fischer wrote:
>>>>>> Hi,
>>>>>> 
>>>>>> first: update to Core 165 ran through - no seen errors. Pakfire looked
>>>>>> good, system is running with .
>>>>>> 
>>>>>> But after digging deeper in the logs I found some errors that made me
>>>>>> ...nervous...:
>>>>>> 
>>>>>> Looking at '/var/log/httpd/error_log' I found a bunch of "tac: write
>>>>>> error: Broken pipe", some "Permission denied:"-messages regarding
>>>>>> 'pakfire.cgi', some Perl errors and a "Could not open password file:
>>>>>> /var/ipfire/auth/users"-error. I wouldn't have noticed if I hadn't take
>>>>>> a closer look.
>>>>>> 
>>>>>> In detail:
>>>>>> Perl couldn't locate two files - "Net/Ping.pm" and "File/Copy.pm" needed
>>>>>> by '/opt/pakfire/lib/functions.pl'. These doesn't exist.
>>>>>> 
>>>>>> Current log file ('/var/log/httpd/error_log') is attached.
>>>>>> 
>>>>>> Nevertheless - system is up and running. WebGui is running.
>>>>>> 
>>>>>> Can anyone confirm?
>>>>>> 
>>>>>> Best,
>>>>>> Matthias