Re: [PATCH] BUG 11786 - squid: Remove setting for filter processes the number of Squid processes

[Matthias Fischer <matthias.fischer@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 5923 bytes --]

On 30.10.2018 15:03, Michael Tremer wrote:
> Hey,

Hello,

> do you have any idea why the redirector is busy?

Sorry: no. Long time ago, I searched for an optimal value but didn't
find any useful links. Right now, I'm testing 'squid 4.4" and its always
the same. Too few redirectors => warning.

I only found something like "If you see this warning, you should
increase the number of child processes and restart (or reconfigure)
Squid. If the queue size becomes twice the number of redirectors, Squid
aborts with a fatal message."

Taken from:
http://etutorials.org/Server+Administration/Squid.+The+definitive+guide/Chapter+11.+Redirectors/11.4+Configuring+Squid/

So my current "40"-value was found through "try and look if the warning
comes again"...

Best,
Matthias


> -Michael
> 
> On Tue, 2018-10-30 at 14:54 +0100, Matthias Fischer wrote:
>> Hi,
>> 
>> I tested this patch today and I fear the resulting value is too low.
>> 
>> For me, its leads to:
>> 
>> ***SNIP***
>> ...
>> 2018/10/30 14:42:43 kid1| WARNING: All 2/2 redirector processes are busy.
>> 2018/10/30 14:42:43 kid1| WARNING: 4 pending requests queued
>> 2018/10/30 14:42:43 kid1| WARNING: Consider increasing the number of
>> redirector processes in your config file.
>> ...
>> 
>> Current setting is "40" (Duo box, 2 Cores, 'squidclamav, squidguard)
>> 
>> Best,
>> Matthias
>> 
>> On 30.10.2018 12:06, Daniel Weismüller wrote:
>> > I added a function to determine the number of cores.
>> > Now the number of squid processes will be equal to the number of logical
>> > cores.
>> > Further I removed the possibility of changing the number
>> > of squid processes in the proxy.cgi
>> > 
>> > Signed-off-by: Daniel Weismüller <daniel.weismueller(a)ipfire.org>
>> > ---
>> >  config/cfgroot/general-functions.pl |  7 +++++++
>> >  html/cgi-bin/proxy.cgi              | 16 ++--------------
>> >  2 files changed, 9 insertions(+), 14 deletions(-)
>> > 
>> > diff --git a/config/cfgroot/general-functions.pl b/config/cfgroot/general-
>> > functions.pl
>> > index 0577afe28..e8495e885 100644
>> > --- a/config/cfgroot/general-functions.pl
>> > +++ b/config/cfgroot/general-functions.pl
>> > @@ -1165,5 +1165,12 @@ sub dnssec_status() {
>> >  
>> >  	return $status;
>> >  }
>> > +sub number_cpu_cores() {
>> > +	open my $cpuinfo, "/proc/cpuinfo" or die "Can't open cpuinfo: $!\n";
>> > +	my $cores = scalar (map /^processor/, <$cpuinfo>);
>> > +	close $cpuinfo;
>> > +
>> > +	return $cores;
>> > +}
>> >  
>> >  1;
>> > diff --git a/html/cgi-bin/proxy.cgi b/html/cgi-bin/proxy.cgi
>> > index 738425b9a..92bebfe18 100644
>> > --- a/html/cgi-bin/proxy.cgi
>> > +++ b/html/cgi-bin/proxy.cgi
>> > @@ -287,7 +287,6 @@ $proxysettings{'IDENT_USER_ACL'} = 'positive';
>> >  $proxysettings{'ENABLE_FILTER'} = 'off';
>> >  $proxysettings{'ENABLE_UPDXLRATOR'} = 'off';
>> >  $proxysettings{'ENABLE_CLAMAV'} = 'off';
>> > -$proxysettings{'CHILDREN'} = '10';
>> >  
>> >  $ncsa_buttontext = $Lang::tr{'advproxy NCSA create user'};
>> >  
>> > @@ -437,11 +436,6 @@ if (($proxysettings{'ACTION'} eq $Lang::tr{'save'}) ||
>> > ($proxysettings{'ACTION'}
>> >  		$errormessage = $Lang::tr{'invalid maximum incoming size'};
>> >  		goto ERROR;
>> >  	}
>> > -		if (!($proxysettings{'CHILDREN'} =~ /^\d+$/) ||
>> > ($proxysettings{'CHILDREN'} < 1))
>> > -	{
>> > -		$errormessage = $Lang::tr{'advproxy invalid num of children'};
>> > -		goto ERROR;
>> > -	}
>> >  	if ($proxysettings{'ENABLE_BROWSER_CHECK'} eq 'on')
>> >  	{
>> >  		$browser_regexp = '';
>> > @@ -1034,12 +1028,8 @@ print <<END
>> >  </table>
>> >  <hr size='1'>
>> >  <table width='100%'>
>> > -<tr><td class='base' colspan='4'><b>$Lang::tr{'advproxy redirector
>> > children'}</b></td></tr>
>> > -<tr><td class='base' >$Lang::tr{'processes'}:&nbsp;<img src='/blob.gif'
>> > alt='*' /><input type='text' name='CHILDREN'
>> > value='$proxysettings{'CHILDREN'}' size='5' /></td>
>> >  END
>> >  ;
>> > -my $count = `ip n| wc -l`;
>> > -if ( $count < 1 ){$count = 1;}
>> >  if ( -e "/usr/bin/squidclamav" ) {
>> >  	print "<td class='base'><b>".$Lang::tr{'advproxy squidclamav'}."</b><br
>> > />";
>> >  	if ( ! -e "/var/run/clamav/clamd.pid" ){
>> > @@ -1048,18 +1038,16 @@ if ( -e "/usr/bin/squidclamav" ) {
>> >  		}
>> >  	else {
>> >  		print $Lang::tr{'advproxy enabled'}."<input type='checkbox'
>> > name='ENABLE_CLAMAV' ".$checked{'ENABLE_CLAMAV'}{'on'}." /><br />";
>> > -		print "+ ".int(( $count**(1/3)) * 8);}
>> > +}
>> >  	print "</td>";
>> >  } else {
>> >  	print "<td></td>";
>> >  }
>> >  print "<td class='base'><a href='/cgi-
>> > bin/urlfilter.cgi'><b>".$Lang::tr{'advproxy url filter'}."</a></b><br />";
>> >  print $Lang::tr{'advproxy enabled'}."<input type='checkbox'
>> > name='ENABLE_FILTER' ".$checked{'ENABLE_FILTER'}{'on'}." /><br />";
>> > -print "+ ".int(($count**(1/3)) * 6);
>> >  print "</td>";
>> >  print "<td class='base'><a href='/cgi-
>> > bin/updatexlrator.cgi'><b>".$Lang::tr{'advproxy update
>> > accelerator'}."</a></b><br />";
>> >  print $Lang::tr{'advproxy enabled'}."<input type='checkbox'
>> > name='ENABLE_UPDXLRATOR' ".$checked{'ENABLE_UPDXLRATOR'}{'on'}." /><br />";
>> > -print "+ ".int(($count**(1/3)) * 5);
>> >  print "</td></tr>";
>> >  print <<END
>> >  </table>
>> > @@ -4095,7 +4083,7 @@ END
>> >  	if (($proxysettings{'ENABLE_FILTER'} eq 'on') ||
>> > ($proxysettings{'ENABLE_UPDXLRATOR'} eq 'on') ||
>> > ($proxysettings{'ENABLE_CLAMAV'} eq 'on'))
>> >  	{
>> >  		print FILE "url_rewrite_program /usr/sbin/redirect_wrapper\n";
>> > -		print FILE "url_rewrite_children
>> > $proxysettings{'CHILDREN'}\n\n";
>> > +		print FILE "url_rewrite_children ",
>> > &General::number_cpu_cores(), "\n\n";
>> >  	}
>> >  
>> >  	# Include file with user defined settings.
>> > 
>> 
>> 
> 
>