Hi Matthias I've tested your changes. But for me it didn't work. 14:24:48 squid: Bungled /etc/squid/squid.conf line 136: url_rewrite_children 8 startup=2 idle=2 queue-size=64 According to the squid pages the "queue-size" argument isn't availlable for squid 3.5. http://www.squid-cache.org/Versions/v3/3.5/cfgman/url_rewrite_children.html I changed the code to: print FILE "url_rewrite_children ",&General::number_cpu_cores()," startup=",&General::number_cpu_cores()," idle=",&General::number_cpu_cores()," \n\n"; This seems to work with squid 3.5 and leads to this line in 'squid.conf': url_rewrite_children 8 startup=8 idle=8 As you can see, my Test-ipfire has 8 virtual cores, so I couldn't see the above problems. Thanks for testing it! - Daniel Matthias Fischer – Tue, 30. October 2018 18:12 > On 30.10.2018 17:05, Michael Tremer wrote: > > On Tue, 2018-10-30 at 15:37 +0100, Matthias Fischer wrote: > >> On 30.10.2018 15:03, Michael Tremer wrote: > >> > Hey, > >> > >> Hello, > >> > >> > do you have any idea why the redirector is busy? > >> > >> Sorry: no. Long time ago, I searched for an optimal value but didn't > >> find any useful links. Right now, I'm testing 'squid 4.4" and its always > >> the same. Too few redirectors => warning. > >> > >> I only found something like "If you see this warning, you should > >> increase the number of child processes and restart (or reconfigure) > >> Squid. If the queue size becomes twice the number of redirectors, Squid > >> aborts with a fatal message." > > > > The last bit is not very good. Why does squid not queue the requests long > > enough? That means even CPU cores * 2 or 4 won't be enough. On a single-core > > machine that would only be 2 requests. LOL > > > > Looking at the options that we have ( > > www.squid-cache.org/Doc/config/url_rewrite_children/), could you please > > test this: > > > > url_rewrite_children X startup=N idle=N queue-size=64 > > > > With N being the number of cores that your system has. I do not think that > there > > is any point in trying to run more than N queries at a time. > > > > But squid doesn't start anything when it starts up. startup= changes that > and > > idle= makes sure that all processes remain active. > > Yep. I had this already in my 'proxy.cgi'. > > I applied the patches again and changed the 'url_rewrite_children'-line > in 'proxy.cgi' to read: > > print FILE "url_rewrite_children ", &General::number_cpu_cores()," > startup=2 idle=2 queue-size=64\n\n"; > > This leads to this line in 'squid.conf': > url_rewrite_children 2 startup=2 idle=2 queue-size=64 > > No warnings so far. Testing ... > > Best, > Matthias > > > A queue size of 64 seems sensible to me because that is probably what the > > average tabloid newspaper page has. Maybe it should be 16 * N or so... > > > > -Michael > > > >> > >> Taken from: > >> > > > etutorials.org/Server+Administration/Squid.+The+definitive+guide/Chapter+11.+Redirectors/11.4+Configuring+Squid/ > >> > >> So my current "40"-value was found through "try and look if the warning > >> comes again"... > >> > >> Best, > >> Matthias > >> > >> > >> > -Michael > >> > > >> > On Tue, 2018-10-30 at 14:54 +0100, Matthias Fischer wrote: > >> > > Hi, > >> > > > >> > > I tested this patch today and I fear the resulting value is too low. > >> > > > >> > > For me, its leads to: > >> > > > >> > > ***SNIP*** > >> > > ... > >> > > 2018/10/30 14:42:43 kid1| WARNING: All 2/2 redirector processes are > busy. > >> > > 2018/10/30 14:42:43 kid1| WARNING: 4 pending requests queued > >> > > 2018/10/30 14:42:43 kid1| WARNING: Consider increasing the number of > >> > > redirector processes in your config file. > >> > > ... > >> > > > >> > > Current setting is "40" (Duo box, 2 Cores, 'squidclamav, squidguard) > >> > > > >> > > Best, > >> > > Matthias > >> > > > >> > > On 30.10.2018 12:06, Daniel Weismüller wrote: > >> > > > I added a function to determine the number of cores. > >> > > > Now the number of squid processes will be equal to the number of > logical > >> > > > cores. > >> > > > Further I removed the possibility of changing the number > >> > > > of squid processes in the proxy.cgi > >> > > > > >> > > > Signed-off-by: Daniel Weismüller > >> > > > --- > >> > > > config/cfgroot/general-functions.pl | 7 +++++++ > >> > > > html/cgi-bin/proxy.cgi | 16 ++-------------- > >> > > > 2 files changed, 9 insertions(+), 14 deletions(-) > >> > > > > >> > > > diff --git a/config/cfgroot/general-functions.pl > >> > > > b/config/cfgroot/general- > >> > > > functions.pl > >> > > > index 0577afe28..e8495e885 100644 > >> > > > --- a/config/cfgroot/general-functions.pl > >> > > > +++ b/config/cfgroot/general-functions.pl > >> > > > @@ -1165,5 +1165,12 @@ sub dnssec_status() { > >> > > > > >> > > > return $status; > >> > > > } > >> > > > +sub number_cpu_cores() { > >> > > > + open my $cpuinfo, "/proc/cpuinfo" or die "Can't open cpuinfo: > >> > > > $!\n"; > >> > > > + my $cores = scalar (map /^processor/, <$cpuinfo>); > >> > > > + close $cpuinfo; > >> > > > + > >> > > > + return $cores; > >> > > > +} > >> > > > > >> > > > 1; > >> > > > diff --git a/html/cgi-bin/proxy.cgi b/html/cgi-bin/proxy.cgi > >> > > > index 738425b9a..92bebfe18 100644 > >> > > > --- a/html/cgi-bin/proxy.cgi > >> > > > +++ b/html/cgi-bin/proxy.cgi > >> > > > @@ -287,7 +287,6 @@ $proxysettings{'IDENT_USER_ACL'} = 'positive'; > >> > > > $proxysettings{'ENABLE_FILTER'} = 'off'; > >> > > > $proxysettings{'ENABLE_UPDXLRATOR'} = 'off'; > >> > > > $proxysettings{'ENABLE_CLAMAV'} = 'off'; > >> > > > -$proxysettings{'CHILDREN'} = '10'; > >> > > > > >> > > > $ncsa_buttontext = $Lang::tr{'advproxy NCSA create user'}; > >> > > > > >> > > > @@ -437,11 +436,6 @@ if (($proxysettings{'ACTION'} eq > $Lang::tr{'save'}) > >> > > > || > >> > > > ($proxysettings{'ACTION'} > >> > > > $errormessage = $Lang::tr{'invalid maximum incoming > >> > > > size'}; > >> > > > goto ERROR; > >> > > > } > >> > > > - if (!($proxysettings{'CHILDREN'} =~ /^\d+$/) || > >> > > > ($proxysettings{'CHILDREN'} < 1)) > >> > > > - { > >> > > > - $errormessage = $Lang::tr{'advproxy invalid num of > >> > > > children'}; > >> > > > - goto ERROR; > >> > > > - } > >> > > > if ($proxysettings{'ENABLE_BROWSER_CHECK'} eq 'on') > >> > > > { > >> > > > $browser_regexp = ''; > >> > > > @@ -1034,12 +1028,8 @@ print < >> > > > > >> > > >
> >> > > > > >> > > > - > >> > > > - > >> > > > END > >> > > > ; > >> > > > -my $count = `ip n| wc -l`; > >> > > > -if ( $count < 1 ){$count = 1;} > >> > > > if ( -e "/usr/bin/squidclamav" ) { > >> > > > print ""; > >> > > > } else { > >> > > > print ""; > >> > > > } > >> > > > print ""; > >> > > > print ""; > >> > > > print < >> > > >
$Lang::tr{'advproxy redirector > >> > > > children'}
$Lang::tr{'processes'}:  src='/blob.gif' > >> > > > alt='*' /> >> > > > value='$proxysettings{'CHILDREN'}' size='5' />".$Lang::tr{'advproxy > >> > > > squidclamav'}."
>> > > > />"; > >> > > > if ( ! -e "/var/run/clamav/clamd.pid" ){ > >> > > > @@ -1048,18 +1038,16 @@ if ( -e "/usr/bin/squidclamav" ) { > >> > > > } > >> > > > else { > >> > > > print $Lang::tr{'advproxy enabled'}." >> > > > type='checkbox' > >> > > > name='ENABLE_CLAMAV' ".$checked{'ENABLE_CLAMAV'}{'on'}." />
"; > >> > > > - print "+ ".int(( $count**(1/3)) * 8);} > >> > > > +} > >> > > > print "		".$Lang::tr{'advproxy url filter'}."
>> > > > />"; > >> > > > print $Lang::tr{'advproxy enabled'}." >> > > > name='ENABLE_FILTER' ".$checked{'ENABLE_FILTER'}{'on'}." />
"; > >> > > > -print "+ ".int(($count**(1/3)) * 6); > >> > > > print "		".$Lang::tr{'advproxy update > >> > > > accelerator'}."
"; > >> > > > print $Lang::tr{'advproxy enabled'}." >> > > > name='ENABLE_UPDXLRATOR' ".$checked{'ENABLE_UPDXLRATOR'}{'on'}." > />
>> > > > />"; > >> > > > -print "+ ".int(($count**(1/3)) * 5); > >> > > > print "
> >> > > > @@ -4095,7 +4083,7 @@ END > >> > > > if (($proxysettings{'ENABLE_FILTER'} eq 'on') || > >> > > > ($proxysettings{'ENABLE_UPDXLRATOR'} eq 'on') || > >> > > > ($proxysettings{'ENABLE_CLAMAV'} eq 'on')) > >> > > > { > >> > > > print FILE "url_rewrite_program > >> > > > /usr/sbin/redirect_wrapper\n"; > >> > > > - print FILE "url_rewrite_children > >> > > > $proxysettings{'CHILDREN'}\n\n"; > >> > > > + print FILE "url_rewrite_children ", > >> > > > &General::number_cpu_cores(), "\n\n"; > >> > > > } > >> > > > > >> > > > # Include file with user defined settings.