Re: [PATCH 3/3] rrdimage: Switch graphs.pl to the new graph scripts

15 Apr 2021


      Hello,
...
On 13 Apr 2021, at 16:47, Tom Rymes tom@rymes.net wrote:
Would it be possible to dump the key-generation task into the background in those instances and have the WUI report on the status? That way, the timeout does not have to be long enough for the entire process, just for however long it takes to submit the task for processing.
This is quite difficult to do because the CGI script would lose control of the forked process.
We no longer know the PID at the next call of the CGI script unless we write that to some temporary file system and have a wrapper script run that removed the temporary data after the process is done. It would come with a lot of overhead.
But I generally agree that we need a solution. Keys are getting longer and longer and a reasonable RSA key takes a long time to generate. Especially with some people using SBCs, there is no chance to generate a cryptographically secure key in reasonable time.
...
Kind of like Pakfire does, or even just submit the changes and have the WUI show that the process is still running if you, for example, navigate to the IPSec page, where the keys are displayed.
Pakfire is just launched in the background and we hope for the best. In the meantime we show the log. Not ideal, but the best we can do :)
-Michael
...
Tom
...
On Apr 13, 2021, at 11:24 AM, Peter Müller peter.mueller@ipfire.org wrote:
Hello Jon,
thanks for your reply.
Indeed, generating key files for OpenVPN or IPsec - and, worse, DH parameters for OpenVPN - was something
I did not have had in mind yesterday. These can take ages indeed, especially on slower hardware. As far as
I am aware, we are not there yet when it comes to ECC cryptography for both VPN subsystems.
In an ideal world, Apache would support two distinct types of timeouts: One for network interaction to the
client, where 60 seconds would be absolutely satisfying. The other one would be for server side applications
such as CGIs which may take longer.
At the moment, I would estimate a relatively small amount of connections to be sufficient to launch a DoS
against Apache as it comes in IPFire. Ultimately, tuning won't help here, if the offender is physically
present in the GREEN or BLUE network. So, in the end, my timeout thought probably does not really matter... :-)
Thanks, and best regards,
Peter Müller
...
...
Is there any particular reason why we have a extremely long (5 minutes) timeout set? The Apache
documentation recommends 60 seconds, and I cannot think of an application on IPFire taking five
minutes to execute...
What about when running openssl to generate a new key?  That one times out for me.
I believe its purpose is to generate as many periods as possible.   ;-)
...
...
On Apr 12, 2021, at 4:06 PM, Peter Müller peter.mueller@ipfire.org wrote:
Good evening Michael,
good evening Leo,
...
@Peter: Would you be up for rewriting the apache configuration?
voila: https://patchwork.ipfire.org/project/ipfire/list/?series=1941
This configuration has been tested and audited using NSE, Nikto, and a bunch of other wonderful
penetration testing tools. To my surprise, HTTP TRACE is mandatory in HTTP 1.1 nowadays, and clients
can expect it to be enabled - I used to read the advice to keep it disabled, but that seems to be
obsolete meanwhile.
Is there any particular reason why we have a extremely long (5 minutes) timeout set? The Apache
documentation recommends 60 seconds, and I cannot think of an application on IPFire taking five
minutes to execute...
Thanks, and best regards,
Peter Müller
...
Hello,
Sorry for my late response. This is probably a little bit more urgent…
Our Apache configuration is here:
https://git.ipfire.org/?p=ipfire-2.x.git;a=tree;f=config/httpd;hb=HEAD
And it does not have any aggressive caching enabled.
The ETag header is precisely there for validating content without transferring it again. We probably should overhaul the entire apache configuration and come up with something that guarantees that we are using modern features of the browser and Apache. Currently the configuration is full of directives for MS Internet Explorer and Java-based browsers. We wouldn’t support any of them - not even sure if they exist any more.
@Peter: Would you be up for rewriting the apache configuration?
Best,
-Michael
...
On 9 Apr 2021, at 18:45, Leo Hofmann hofmann@leo-andres.de wrote:
Hi,
I found that disabling the HTTP ETag header solved the problem for me. Are there any Cache-Control headers configured for these static files?
I'll change the CSS and submit a patch soon!
Leo
Am 09.04.2021 um 12:57 schrieb Michael Tremer:
> Hello,
> 
> That was it. For some reason my browser did not validate the CSS file and used a cached version. This is probably something we should look into.
> 
> Apart from that it works.
> 
> Can we maybe change the background colour to something less yellow when a button is selected. Maybe just underlining the word is enough?
> 
> -Michael
> 
>> On 7 Apr 2021, at 23:18, Leo Hofmann hofmann@leo-andres.de wrote:
>> 
>> Hello Michael,
>> 
>> No, of course not, that looks terrible!
>> Could you please check/CTRL+F5 the CSS file /themes/ipfire/include/css/style.css? Patch 2 of this series should have added ~30 lines at the end.
>> Please have a look at the attached screenshot, there you can see how it is supposed to be. I have tested this with Firefox and Chrome.
>> 
>> Best regards,
>> Leo
>> 
>> Am 07.04.2021 um 23:31 schrieb Michael Tremer:
>>> Hello,
>>> 
>>> Is this meant to look like this?
>>> 
>>> 
>>> 
>>>> On 1 Apr 2021, at 14:35, Leo-Andres Hofmann <hofmann@leo-andres.de mailto:hofmann@leo-andres.de> wrote:
>>>> 
>>>> "makegraphbox" is modified to remove the old iframe method and output
>>>> a modern div container instead.
>>>> Graph errors are now returned, to be displayed by getrrdimage.cgi.
>>>> 
>>>> entropy.cgi and netovpnsrv.cgi are modified to ensure compatibility.
>>>> 
>>>> Add cache control HTTP header to image output.
>>>> 
>>>> Signed-off-by: Leo-Andres Hofmann <hofmann@leo-andres.de mailto:hofmann@leo-andres.de>
>>>> ---
>>>> config/cfgroot/graphs.pl    | 86 +++++++++++++++++++++----------------
>>>> html/cgi-bin/entropy.cgi    |  2 +-
>>>> html/cgi-bin/netovpnsrv.cgi |  2 +-
>>>> 3 files changed, 50 insertions(+), 40 deletions(-)
>>>> 
>>>> diff --git a/config/cfgroot/graphs.pl b/config/cfgroot/graphs.pl
>>>> index beddff032..cf4a30de3 100644
>>>> --- a/config/cfgroot/graphs.pl
>>>> +++ b/config/cfgroot/graphs.pl
>>>> @@ -24,6 +24,7 @@ package Graphs;
>>>> 
>>>> use strict;
>>>> use RRDs;
>>>> +use experimental 'smartmatch';
>>>> 
>>>> require '/var/ipfire/general-functions.pl';
>>>> require "${General::swroot}/lang.pl";
>>>> @@ -99,26 +100,35 @@ foreach (@sensorsdir){
>>>> &General::readhash("${General::swroot}/sensors/settings", %sensorsettings);
>>>> 
>>>> # Generate a nice box for selection of time range in graphs
>>>> -# this will generate a nice iframe for the cgi every klick for
>>>> -# the graph will be handled inside the iframe
>>>> +# this will generate a nice div box for the cgi every klick for
>>>> +# the graph will be handled by javascript
>>>> # 0 is the cgi refering to
>>>> # 1 is the graph name
>>>> -# 2 is the time range for the graph
>>>> -# 3 if given is the height of the iframe default if nothing is given
>>>> +# 2 is the time range for the graph (optional)
>>>> 
>>>> sub makegraphbox {
>>>> -print "<center>";
>>>> -print "<a href='".$_[0]."?".$_[1]."?hour' target='".$_[1]."box'><b>".$Lang::tr{'hour'}."</b></a>";
>>>> -print " - ";
>>>> -print "<a href='".$_[0]."?".$_[1]."?day' target='".$_[1]."box'><b>".$Lang::tr{'day'}."</b></a>";
>>>> -print " - ";
>>>> -print "<a href='".$_[0]."?".$_[1]."?week' target='".$_[1]."box'><b>".$Lang::tr{'week'}."</b></a>";
>>>> -print " - ";
>>>> -print "<a href='".$_[0]."?".$_[1]."?month' target='".$_[1]."box'><b>".$Lang::tr{'month'}."</b></a>";
>>>> -print " - ";
>>>> -print "<a href='".$_[0]."?".$_[1]."?year' target='".$_[1]."box'><b>".$Lang::tr{'year'}."</b></a>";
>>>> -print "<br></center>";
>>>> -print "<iframe class='graph' src='".$_[0]."?".$_[1]."?".$_[2]."' scrolling='no' frameborder='no' marginheight='0' name='".$_[1]."box'></iframe>";
>>>> +my ($origin, $name, $default_range) = @_;
>>>> +
>>>> +# Optional time range: Default to "day" unless otherwise specified
>>>> +$default_range = "day" unless ($default_range ~~ @time_ranges);
>>>> +
>>>> +print <<END;
>>>> +<div class="rrdimage" id="rrdimg-$name" data-origin="$origin" data-graph="$name" data-default-range="$default_range">
>>>> +<ul>
>>>> +END
>>>> +
>>>> +# Print range select buttons
>>>> +foreach my $range (@time_ranges) {
>>>> +print <<END;
>>>> +<li><button data-range="$range" onclick="rrdimage_selectRange(this)">$Lang::tr{$range}</button></li>
>>>> +END
>>>> +}
>>>> +
>>>> +print <<END;
>>>> +</ul>
>>>> +<img src="/cgi-bin/getrrdimage.cgi?origin=${origin}&graph=${name}&range=${default_range}" alt="$Lang::tr{'graph'} ($name)">
>>>> +</div>
>>>> +END
>>>> }
>>>> 
>>>> # Generate the CPU Graph for the current period of time for values given by
>>>> @@ -248,7 +258,7 @@ sub updatecpugraph {
>>>> 
>>>> RRDs::graph (@command);
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for cpu: ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for cpu: ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> # Generate the Load Graph for the current period of time for values given by collecd
>>>> @@ -280,7 +290,7 @@ sub updateloadgraph {
>>>> "LINE1:load1".$color{"color18"},
>>>> );
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for load: ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for load: ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> # Generate the Memory Graph for the current period of time for values given by collecd
>>>> @@ -336,7 +346,7 @@ sub updatememorygraph {
>>>> "GPRINT:freepct:LAST:%3.2lf%%\j",
>>>> );
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for memory: ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for memory: ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> # Generate the Swap Graph for the current period of time for values given by collecd
>>>> @@ -385,7 +395,7 @@ sub updateswapgraph {
>>>> "GPRINT:freepct:LAST:%3.2lf%%\j",
>>>> );
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for memory: ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for memory: ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> # Generate the Process Cpu Graph for the current period of time for values given by collecd
>>>> @@ -432,7 +442,7 @@ sub updateprocessescpugraph {
>>>> 
>>>> RRDs::graph (@command);
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for processes: ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for processes: ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> # Generate the Process Memory Graph for the current period of time for values given by collecd
>>>> @@ -478,7 +488,7 @@ sub updateprocessesmemorygraph {
>>>> 
>>>> RRDs::graph (@command);
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for processesmemory: ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for processesmemory: ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> # Generate the Disk Graph for the current period of time for values given by collecd
>>>> @@ -522,7 +532,7 @@ sub updatediskgraph {
>>>> "GPRINT:write:LAST:%8.1lf %sBps\j",
>>>> );
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for ".$disk.": ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for ".$disk.": ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> # Generate the Interface Graph for the current period of time for values given by collecd
>>>> @@ -561,7 +571,7 @@ sub updateifgraph {
>>>> "GPRINT:outgoing:LAST:%8.1lf %sBps\j",
>>>> );
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for ".$interface.": ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for ".$interface.": ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> sub updatevpngraph {
>>>> @@ -598,7 +608,7 @@ sub updatevpngraph {
>>>> "GPRINT:outgoing:LAST:%8.1lf %sBps\j",
>>>> );
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for ".$interface.": ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for ".$interface.": ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> sub updatevpnn2ngraph {
>>>> @@ -661,7 +671,7 @@ sub updatevpnn2ngraph {
>>>> "GPRINT:compression_out:LAST:%8.1lf %sBps\j",
>>>> );
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for ".$interface.": ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for ".$interface.": ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> # Generate the Firewall Graph for the current period of time for values given by collecd
>>>> @@ -716,7 +726,7 @@ sub updatefwhitsgraph {
>>>> "GPRINT:portscan:LAST:%8.1lf %sBps\j",
>>>> );
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for firewallhits: ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for firewallhits: ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> # Generate the Line Quality Graph for the current period of time for values given by collecd
>>>> @@ -758,7 +768,7 @@ sub updatepinggraph {
>>>> "GPRINT:roundtrip:LAST:%3.2lf ms\j",
>>>> );
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for link quality: ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for link quality: ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> sub updatewirelessgraph {
>>>> @@ -793,7 +803,7 @@ sub updatewirelessgraph {
>>>> "GPRINT:power:LAST:%5.1lf %sdBm\j",
>>>> );
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for wireless: ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for wireless: ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> # Generate the HDD Temp Graph for the current period of time for values given by collecd and lm_sensors
>>>> @@ -827,7 +837,7 @@ sub updatehddgraph {
>>>> "GPRINT:temperature:LAST:%3.0lf °C\j",
>>>> );
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for hdd-".$disk.": ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for hdd-".$disk.": ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> # Generate the Temp Graph for the current period of time for values given by collecd and lm_sensors
>>>> @@ -875,7 +885,7 @@ sub updatehwtempgraph {
>>>> 
>>>> RRDs::graph (@command);
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for HDD Temp: ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for HDD Temp: ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> # Generate the Fan Graph for the current period of time for values given by collecd and lm_sensors
>>>> @@ -922,7 +932,7 @@ sub updatehwfangraph {
>>>> 
>>>> RRDs::graph (@command);
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for Fan Speed: ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for Fan Speed: ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> # Generate the Voltage Graph for the current period of time for values given by collecd and lm_sensors
>>>> @@ -969,7 +979,7 @@ sub updatehwvoltgraph {
>>>> 
>>>> RRDs::graph (@command);
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for Voltage: ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for Voltage: ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> 
>>>> @@ -1051,7 +1061,7 @@ sub updateqosgraph {
>>>> }
>>>> RRDs::graph (@command);
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for qos device ".$qossettings{'DEV'}.": ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for qos device ".$qossettings{'DEV'}.": ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> # Generate the CPU Frequency Graph for the current period of time for values given by collectd an lm_sensors
>>>> @@ -1090,7 +1100,7 @@ sub updatecpufreqgraph {
>>>> 
>>>> RRDs::graph (@command);
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for cpu freq: ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for cpu freq: ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> # Generate the Thermal Zone Temp CPU Graph
>>>> @@ -1129,7 +1139,7 @@ sub updatethermaltempgraph {
>>>> 
>>>> RRDs::graph (@command);
>>>> $ERROR = RRDs::error;
>>>> -print "Error in RRD::graph for thermal temp: ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for thermal temp: ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> 
>>>> @@ -1174,7 +1184,7 @@ sub updateentropygraph {
>>>> RRDs::graph (@command);
>>>> $ERROR = RRDs::error;
>>>> 
>>>> -print "Error in RRD::graph for entropy: ".$ERROR."\n" if $ERROR;
>>>> +return "Error in RRD::graph for entropy: ".$ERROR."\n" if $ERROR;
>>>> }
>>>> 
>>>> sub updateconntrackgraph {
>>>> @@ -1202,5 +1212,5 @@ sub updateconntrackgraph {
>>>> RRDs::graph(@command);
>>>> $ERROR = RRDs::error;
>>>> 
>>>> -print STDERR "Error in RRD::Graph for conntrack: " . $ERROR . "\n" if $ERROR;
>>>> +return "Error in RRD::Graph for conntrack: " . $ERROR . "\n" if $ERROR;
>>>> }
>>>> diff --git a/html/cgi-bin/entropy.cgi b/html/cgi-bin/entropy.cgi
>>>> index d7a9ca5d8..f8045db5a 100644
>>>> --- a/html/cgi-bin/entropy.cgi
>>>> +++ b/html/cgi-bin/entropy.cgi
>>>> @@ -45,7 +45,7 @@ if ( $querry[0] ne~ "") {
>>>> &Header::openbigbox('100%', 'left');
>>>> 
>>>> &Header::openbox('100%', 'center', $Lang::tr{'entropy'});
>>>> -&Graphs::makegraphbox("entropy.cgi", "day");
>>>> +&Graphs::makegraphbox("entropy.cgi", "entropy", "day");
>>>> &Header::closebox();
>>>> 
>>>> # Check for hardware support.
>>>> diff --git a/html/cgi-bin/netovpnsrv.cgi b/html/cgi-bin/netovpnsrv.cgi
>>>> index 77c69cddb..ab3548713 100755
>>>> --- a/html/cgi-bin/netovpnsrv.cgi
>>>> +++ b/html/cgi-bin/netovpnsrv.cgi
>>>> @@ -75,7 +75,7 @@ if ( $querry[0] ne ""){
>>>> if (@vpns || %ipsecgraphs) {
>>>> foreach my $name (sort keys %ipsecgraphs) {
>>>> &Header::openbox('100%', 'center', "$Lang::tr{'ipsec connection'}: $name");
>>>> -&Graphs::makegraphbox("netovpnsrv.cgi", $ipsecgraphs{$name}, "day");
>>>> +&Graphs::makegraphbox("netovpnsrv.cgi", "ipsec-$ipsecgraphs{$name}", "day");
>>>> &Header::closebox();
>>>> }
>>>> 
>>>> -- 
>>>> 2.27.0.windows.1
>>>> 
>> <rrdimg-css.png>

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

Re: [PATCH 3/3] rrdimage: Switch graphs.pl to the new graph scripts