From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer To: development@lists.ipfire.org Subject: Re: [PATCH] squid-graph 3.2: Proxy Graphs rebuilt (V2) Date: Mon, 09 Oct 2017 15:28:49 +0100 Message-ID: <1507559329.4045.18.camel@ipfire.org> In-Reply-To: <62cb9412-59bd-233e-b9fd-f60d46bc5013@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5134919172537042234==" List-Id: --===============5134919172537042234== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On Wed, 2017-10-04 at 21:01 +0200, Matthias Fischer wrote: > Hi, >=20 > On 04.10.2017 18:36, Michael Tremer wrote: > > Any screenshots? >=20 > See attachment. Hope this helps. Yes looks good. But why is it half German and half English? Any chance we can localise the sq= uid graph sections?! > Please note: > I'd like to handle this addon as "under review", because during my > testing of the last week I found that analysing rather LARGE squid > access logs can last a while (here: ~15sec) and produce system loads of > about ~15% while working. Well, that is probably okay. > Because of this I moved this job from 'makegraphs' (which runs every 5 > minutes) to a hourly cron job. This fits my needs and the load is ok. How do you handle it then that the access log file is being rotated? You will miss log lines on a busy system which isn't really what we want. -Michael >=20 > Original comment about this from squid-graphs-README: >=20 > "RUNNING SQUID-GRAPH IN REAL TIME VIA THE WEB > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > As of v3, Squid Graph does not support this feature anymore. If you > would still like to do this, simply create a shell script to execute > Squid Graph. >=20 > We do not recommend this unless your log files are really small. Also, > we did not audit Squid Graph for any possible security issues that might > arise in an online CGI enviroment. YOU HAVE BEEN WARNED." >=20 > Best, > Matthias >=20 > > Tester's feedback wanted! > >=20 > > On Fri, 2017-09-29 at 08:50 +0200, Matthias Fischer wrote: > > > This is a rather old fellow. > > >=20 > > > For details see: > > > https://sourceforge.net/projects/squid-graph/ > > >=20 > > > "Squid Graph is a free, simple, yet powerful Squid v2 native logfile > > > analysis > > > tool > > > that generates reports with graphical representation of the proxy serve= r's > > > traffic, > > > somewhat like the popular MRTG tool." > > >=20 > > > I found by chance that the corresponding translation strings were still > > > present, > > > so I thought "Why not?". ;-) > > >=20 > > > Here, it works without problems and 'squid 3.5.27'. > > >=20 > > > The original 'sgraph.cgi' was slightly tuned - like in IPCop - and > > > polished > > > for better readability. > > >=20 > > > If someone likes it, here it is - again... > > >=20 > > > Best, > > > Matthias > > >=20 > > > Signed-off-by: Matthias Fischer > > > --- > > > config/menu/20-status.menu | 10 +++- > > > config/rootfiles/common/squid-graph | 3 ++ > > > config/rootfiles/common/web-user-interface | 1 + > > > html/cgi-bin/sgraph.cgi | 72 > > > ++++++++++++++++++++++++++ > > > lfs/squid-graph | 81 > > > ++++++++++++++++++++++++++++++ > > > make.sh | 1 + > > > src/scripts/makegraphs | 9 +++- > > > 7 files changed, 175 insertions(+), 2 deletions(-) > > > create mode 100644 config/rootfiles/common/squid-graph > > > create mode 100644 html/cgi-bin/sgraph.cgi > > > create mode 100644 lfs/squid-graph > > >=20 > > > diff --git a/config/menu/20-status.menu b/config/menu/20-status.menu > > > index 2bcf0d5e8..dfc6dae9b 100644 > > > --- a/config/menu/20-status.menu > > > +++ b/config/menu/20-status.menu > > > @@ -58,7 +58,15 @@ > > > 'title' =3D> "$Lang::tr{'hardware > > > graphs'}", > > > 'enabled' =3D> 1, > > > }; > > > - $substatus->{'61.entropy'} =3D { > > > + > > > + $substatus->{'61.proxygraphs'} =3D { > > > + 'caption' =3D> "$Lang::tr{'ssproxy > > > graphs'}", > > > + 'uri' =3D> '/cgi-bin/sgraph.cgi', > > > + 'title' =3D> "$Lang::tr{'ssproxy graphs'}", > > > + 'enabled' =3D> 1, > > > + }; > > > + > > > + $substatus->{'62.entropy'} =3D { > > > 'caption' =3D> "$Lang::tr{'entropy'}", > > > 'uri' =3D> '/cgi-bin/entropy.cgi', > > > 'title' =3D> "$Lang::tr{'entropy graphs'}", > > > diff --git a/config/rootfiles/common/squid-graph > > > b/config/rootfiles/common/squid-graph > > > new file mode 100644 > > > index 000000000..0b558d87a > > > --- /dev/null > > > +++ b/config/rootfiles/common/squid-graph > > > @@ -0,0 +1,3 @@ > > > +srv/web/ipfire/html/sgraph > > > +srv/web/ipfire/html/sgraph/logo.png > > > +usr/bin/squid-graph > > > diff --git a/config/rootfiles/common/web-user-interface > > > b/config/rootfiles/common/web-user-interface > > > index f204f1657..b14efb402 100644 > > > --- a/config/rootfiles/common/web-user-interface > > > +++ b/config/rootfiles/common/web-user-interface > > > @@ -72,6 +72,7 @@ srv/web/ipfire/cgi-bin/routing.cgi > > > #srv/web/ipfire/cgi-bin/samba.cgi > > > #srv/web/ipfire/cgi-bin/sambahlp.cgi > > > srv/web/ipfire/cgi-bin/services.cgi > > > +srv/web/ipfire/cgi-bin/sgraph.cgi > > > srv/web/ipfire/cgi-bin/shutdown.cgi > > > srv/web/ipfire/cgi-bin/speed.cgi > > > srv/web/ipfire/cgi-bin/system.cgi > > > diff --git a/html/cgi-bin/sgraph.cgi b/html/cgi-bin/sgraph.cgi > > > new file mode 100644 > > > index 000000000..dea4c068a > > > --- /dev/null > > > +++ b/html/cgi-bin/sgraph.cgi > > > @@ -0,0 +1,72 @@ > > > +#!/usr/bin/perl > > > +######################################################################= ### > > > #### > > > ## > > > +# = =20 > > > =20 > > > # > > > +# IPFire.org - A linux based > > > firewall # > > > +# Copyright (C) 2016 IPFire Team = =20 > > > =20 > > > # > > > +# = =20 > > > =20 > > > # > > > +# This program is free software: you can redistribute it and/or > > > modify # > > > +# it under the terms of the GNU General Public License as published > > > by # > > > +# the Free Software Foundation, either version 3 of the License, > > > or # > > > +# (at your option) any later > > > version. # > > > +# = =20 > > > =20 > > > # > > > +# This program is distributed in the hope that it will be > > > useful, # > > > +# but WITHOUT ANY WARRANTY; without even the implied warranty > > > of # > > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See > > > the # > > > +# GNU General Public License for more > > > details. # > > > +# = =20 > > > =20 > > > # > > > +# You should have received a copy of the GNU General Public > > > License # > > > +# along with this program. If not, see = . =20 > > > =20 > > > # > > > +# = =20 > > > =20 > > > # > > > +######################################################################= ### > > > #### > > > ## > > > + > > > +use strict; > > > + > > > +# enable only the following on debugging purpose > > > +#use warnings; > > > +#use CGI::Carp 'fatalsToBrowser'; > > > + > > > +require '/var/ipfire/general-functions.pl'; > > > +require "${General::swroot}/lang.pl"; > > > +require "${General::swroot}/header.pl"; > > > + > > > +my @graphs =3D (); > > > + > > > +&Header::showhttpheaders(); > > > + > > > +my $dir =3D "/srv/web/ipfire/html/sgraph"; > > > +my $sgraphdir =3D "/srv/web/ipfire/html/sgraph"; > > > + > > > +&Header::openpage($Lang::tr{'ssproxy graphs'}, 1, ''); > > > + > > > +&Header::openbigbox('100%', 'left'); > > > + > > > +&Header::openbox('100%', 'left', $Lang::tr{'proxy access graphs'} . ":" > > > ); > > > + > > > +if (open(IPACHTML, "$sgraphdir/index.html")) > > > +{ > > > + my $skip =3D 1; > > > + while () > > > + { > > > + $skip =3D 1 if /^
$/; > > > + if ($skip) > > > + { > > > + $skip =3D 0 if /

/; > > > + next; > > > + } > > > + s/]+)>/3D'Graph' > > \/>/; > > > + s/
/
/g; > > > + s/
/
/g; > > > + s/<([^>]*)>/\L<$1>\E/g; > > > + s/(size|align|border|color)=3D([^'"> ]+)/$1=3D'$2'/g; > > > + print; > > > + } > > > + close(IPACHTML); > > > +} > > > +else { > > > + print $Lang::tr{'no information available'}; } > > > + > > > +&Header::closebox(); > > > + > > > +&Header::closebigbox(); > > > + > > > +&Header::closepage(); > > > diff --git a/lfs/squid-graph b/lfs/squid-graph > > > new file mode 100644 > > > index 000000000..e94c0c130 > > > --- /dev/null > > > +++ b/lfs/squid-graph > > > @@ -0,0 +1,81 @@ > > > +######################################################################= ### > > > #### > > > ## > > > +# = =20 > > > =20 > > > # > > > +# IPFire.org - A linux based > > > firewall # > > > +# Copyright (C) 2007-2015 IPFire Team = =20 > > > =20 > > > # > > > +# = =20 > > > =20 > > > # > > > +# This program is free software: you can redistribute it and/or > > > modify # > > > +# it under the terms of the GNU General Public License as published > > > by # > > > +# the Free Software Foundation, either version 3 of the License, > > > or # > > > +# (at your option) any later > > > version. # > > > +# = =20 > > > =20 > > > # > > > +# This program is distributed in the hope that it will be > > > useful, # > > > +# but WITHOUT ANY WARRANTY; without even the implied warranty > > > of # > > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See > > > the # > > > +# GNU General Public License for more > > > details. # > > > +# = =20 > > > =20 > > > # > > > +# You should have received a copy of the GNU General Public > > > License # > > > +# along with this program. If not, see = . =20 > > > =20 > > > # > > > +# = =20 > > > =20 > > > # > > > +######################################################################= ### > > > #### > > > ## > > > + > > > +######################################################################= ### > > > #### > > > ## > > > +# Definitions > > > +######################################################################= ### > > > #### > > > ## > > > + > > > +include Config > > > + > > > +VER =3D 3.2 > > > + > > > +THISAPP =3D squid-graph-$(VER) > > > +DL_FILE =3D $(THISAPP).tar.gz > > > +DL_FROM =3D $(URL_IPFIRE) > > > +DIR_APP =3D $(DIR_SRC)/$(THISAPP) > > > +TARGET =3D $(DIR_INFO)/$(THISAPP) > > > + > > > +######################################################################= ### > > > #### > > > ## > > > +# Top-level Rules > > > +######################################################################= ### > > > #### > > > ## > > > + > > > +objects =3D $(DL_FILE) > > > + > > > +$(DL_FILE) =3D $(DL_FROM)/$(DL_FILE) > > > + > > > +$(DL_FILE)_MD5 =3D 89ac122a99dc2db67efbdcd2467b9939 > > > + > > > +install : $(TARGET) > > > + > > > +check : $(patsubst %,$(DIR_CHK)/%,$(objects)) > > > + > > > +download :$(patsubst %,$(DIR_DL)/%,$(objects)) > > > + > > > +md5 : $(subst %,%_MD5,$(objects)) > > > + > > > +######################################################################= ### > > > #### > > > ## > > > +# Downloading, checking, md5sum > > > +######################################################################= ### > > > #### > > > ## > > > + > > > +$(patsubst %,$(DIR_CHK)/%,$(objects)) : > > > + @$(CHECK) > > > + > > > +$(patsubst %,$(DIR_DL)/%,$(objects)) : > > > + @$(LOAD) > > > + > > > +$(subst %,%_MD5,$(objects)) : > > > + @$(MD5) > > > + > > > +######################################################################= ### > > > #### > > > ## > > > +# Installation Details > > > +######################################################################= ### > > > #### > > > ## > > > + > > > +$(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) > > > + @$(PREBUILD) > > > + @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar zxf > > > $(DIR_DL)/$(DL_FILE) > > > + > > > + cp -f $(DIR_SRC)/squid-graph/squid-graph /usr/bin/squid-graph > > > + chmod 755 /usr/bin/squid-graph > > > + > > > + -mkdir -p /srv/web/ipfire/html/sgraph > > > + cp -f $(DIR_SRC)/squid-graph/logo.png /srv/web/ipfire/html/sgraph > > > + > > > + @rm -rf $(DIR_APP) > > > + @$(POSTBUILD) > > > diff --git a/make.sh b/make.sh > > > index 815b25aca..eac4b229f 100755 > > > --- a/make.sh > > > +++ b/make.sh > > > @@ -638,6 +638,7 @@ buildipfire() { > > > lfsmake2 snort > > > lfsmake2 oinkmaster > > > lfsmake2 squid > > > + lfsmake2 squid-graph > > > lfsmake2 squidguard > > > lfsmake2 calamaris > > > lfsmake2 tcpdump > > > diff --git a/src/scripts/makegraphs b/src/scripts/makegraphs > > > index 0b56b9325..0191ef70b 100644 > > > --- a/src/scripts/makegraphs > > > +++ b/src/scripts/makegraphs > > > @@ -79,7 +79,7 @@ sub updatehdddata{ > > > print "Error in RRD::create for hdd-".$array[$#array].": > > > $ERROR\n" if $ERROR; > > > } > > > =20 > > > - # Temperaturlesen w=EF=BF=BDrde die Platte aufwecken!!! > > > + # Temperaturlesen w=C3=BCrde die Platte aufwecken!!! > > > if (!$standby){ > > > $temp =3D 0; > > > my $smart_output =3D ''; > > > @@ -134,3 +134,10 @@ foreach (@disks){ > > > =20 > > > updatehdddata($disk); > > > } > > > + > > > +### > > > +### Squid Graphs > > > +### > > > +if ( -e "/var/log/squid/access.log" ) { > > > + system ("/usr/bin/squid-graph --no-console-log --tcp-only -- > > > output- > > > dir=3D/srv/web/ipfire/html/sgraph < /var/log/squid/access.log >/dev/null > > > 2>&1"); > > > +} >=20 >=20 --===============5134919172537042234== Content-Type: application/pgp-signature Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="signature.asc" MIME-Version: 1.0 LS0tLS1CRUdJTiBQR1AgU0lHTkFUVVJFLS0tLS0KCmlRSXpCQUFCQ2dBZEZpRUU1L3JXNWwzR0dl Mnlwa3R4Z0hudy8yK1FDUWNGQWxuYmg2RUFDZ2tRZ0hudy8yK1EKQ1FjY0JSQUF0U3ZIM0p0emQx cWRSTmM3MnVnS0I1QUNOQjdOZ2MwcmpuMGR6eERHeTJPWkk0OXNLVVVSVG1pUgpLU0FrK2swWGs0 S1laSU9LalNtVHNmMWhwdVNDc05jS1U2WnlzdkhyR1llRWp6UXhLZXVOWlBNMTBVTmE1bTQrCk94 S1J0UzdBYVhlYlM1NG1lekhUcEgvMXFTSnFuMVlFTm55WjFxeGlrZCtNVllYUkZmeXFETHErWDQy d2xiUlMKTkh5MnZCRXVUSU5jRG5XM0Rmc282NTlwUEE5MnBWN1ZaN3k3ZkFPRm9GTS9VZStzb0lZ WmJEN1QxRmlGRndycwpVNUFmTXRkOEhhZDdnd3ZCQzQ3czNaajBLUVNoMUppeE10cktDcGJwWm00 a2k1Ykgzc1Q0a1lrYURzWDVFTVhiClErODRLVit3eHBtWVFuU2xsTHpvQjNoRXRBaUlWbDlvWkR5 aVUwSlptMEFKd2w4YzB5TzY3d1VjWVdjK2p6dFgKajRicnZMOVU0RjBqMTBORlBVdlRkcG5NajU2 N2poWW1iQUJjNXl3TDhDQVRBVWttbDl1Q3pLSUgvTWZRdjJkRwp5Sk5LRTZSUWlBaWhBNkdtMlNH amJzQi8reEphbFh6QkhoTjFkRzM3V2wzemIvalhvTnNqcWVlMXFOT2pUcy9WClQ2ZWRwQVZTMThY aTkreTAwSldJb3E2U0UrMGpWcklrQ3VjTWNWczB3UjZzYTFCMlZoMGZSbnpkN2UybHlkNzYKM2kv RlFWallLN1M4Q0E3UGVyUnVYUnVtdnlEK2xJNU5SbWJkRk1lMUU2V2FlWnMxdStnWlA0WlFlOXUr a1ZCdAp3S2VVRXc3ZW9ad0ZlMFFYbTR6Nk9rdEtTT0RkN21uOVNLYS9jZlkwalh2Vm55T1ZSSlU9 Cj1QZjEvCi0tLS0tRU5EIFBHUCBTSUdOQVRVUkUtLS0tLQo= --===============5134919172537042234==--