From mboxrd@z Thu Jan 1 00:00:00 1970 From: Matthias Fischer To: development@lists.ipfire.org Subject: Re: [PATCH] squid init v_final Date: Thu, 19 May 2016 21:03:21 +0200 Message-ID: In-Reply-To: <1463606559.19435.25.camel@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1978458549182477861==" List-Id: --===============1978458549182477861== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi, first feedback from here: Works. ;-) No squidclamav-crashes, clean log. Max Mem size: 32768 KB Max Swap size: 262144 KB ~3000 Objects (not much in cache) Took about 8 seconds for stopping '(squid-1)'. I'll watch this during the next days. Best, Matthias On 18.05.2016 23:22, Michael Tremer wrote: > Hi, >=20 > On Wed, 2016-05-18 at 21:03 +0200, Matthias Fischer wrote: >> Just coming back from work... >>=20 >> On 18.05.2016 20:15, Michael Tremer wrote: >> >=20 >> > Hi, >> Hi, >> =20 >> >=20 >> > thanks. I finally merged this with a few changes: >> >=20 >> > http://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dcommitdiff;h=3D6113575d8= 1201591b2e >> > 8a76520579a4e7b7c5d46 >> >=20 >> > and added some more changes: >> >=20 >> > http://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dcommitdiff;h=3D691b2836c= 0ffa54b3f7 >> > fc8d6f3b98377b3c9a470 >> > http://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dcommitdiff;h=3D3fcc9b67f= c7ab188624 >> > 4a647dd7cf98f00e5c555 >> >=20 >> > I used boot_mesg instead of echo in some situations. Refrased the warning >> > message. Removed -r from rm. And finally if squid shut down gracefully t= he >> > shut >> > down time is logged to syslog instead of being written to the console. >> I took a quick look. For me, the warning message goes across the screen. >> Nevertheless, message will clearly be seen. >=20 > Yes, it is kind of long, but will line-break then. >=20 >> That 'while'-counters look as if you got some cats running across your >> keyboard!? ;-) >> Such things are "a bit" beyond my programming skills. Nice. >=20 > That is just modulus. It calculates the remainder of a division of n by 6. = If > that is 0, then a dot will be printed. That way every 6 cycles only one dot= is > printed. >=20 >> And I never thought of moving that d*** 'killproc'/'wait'-block to the bot= tom. >> I'm really curious what 'squidclamav' will make of this. >=20 > Actually this should never do anything because squid should take care of ki= lling > all processes. However we had issues with that in the past so that we are > killing those manually. >=20 >> >=20 >> > I also added some more comments to the script so that we know what is go= ing >> > on >> > in a few months down from here. >> >=20 >> > This is working fine for me. Even with a small and almost empty cache sq= uid >> > takes about 14 seconds to shut down on my IPFire Prime Box. >> 14 seconds! Not bad. >> I thought that it would take *some* time, but not that much. >> One problem remains - what happens to really BIG caches. >> I don't have a solution for that by now. >=20 > Well, they should be closed within 6 minutes. If not they will rebuild every > time. >=20 > But I think if it doesn't close within in 6 minutes your cache is way too b= ig > for your hardware and should be downsized. >=20 > We might have to wait for some feedback. >=20 >>=20 >> >=20 >> > Please test everything again. I am going to deploy this on a few more te= st >> > systems this week. >> Testmachine didn't complain - it did everything right while testing >> with "360" and "3" seconds. >>=20 >> Now running on production machine. >> Tomorrow I'll see what the logs really think of this. ;-) >=20 > I installed it on one of the big test machines. Waiting for feedback... >=20 > Best, > -Michael >=20 >>=20 >> Best, Matthias >>=20 >> >=20 >> >=20 >> > Best, >> > -Michael >> >=20 >> > On Tue, 2016-05-17 at 21:33 +0200, Matthias Fischer wrote: >> > >=20 >> > > Last update suggestions for (3.5.xx) initscript, awaiting feedback. ;-) >> > >=20 >> > > Changelog: >> > >=20 >> > > - Raised 'while'-loop-time for stopping squid to 360 seconds until >> > > '/var/log/cache/swap.state' is deleted. NOT the whole cache structur= e! >> > > I know, this timeout could not be enough for really BIG caches. >> > > The only other choice I know would be to leave this loop alone witho= ut >> > > the "xxx seconds"-counter and wait how 'squid -k shutdown' handles t= his. >> > > But if it fails, I fear that we could end up in an endless loop. >> > > Suggestions are welcome! >> > >=20 >> > > - Had to delete 'wait' after killing squidguard, updxlrator, squidclam= av >> > > and redirect_wrappers - 'while'-loop counter wouldn't work because of >> > > this line. >> > > Suggestions for better handling? >> > >=20 >> > > - Process detection looks for leftover '(squid-1)'-process using 'pgre= p'. >> > >=20 >> > > - Cosmetic changes to some 'boot_mesg' lines. Added a few. Is it >> > > (still) too much? I'd prefer to get a warning (an announcement) if >> > > '/var/log/cache/swap.state' was deleted. >> > >=20 >> > > - Changed the 'flush'-command to really delete the entire >> > > '/var/log/cache'-structure, it will automatically be rebuild >> > > during the next start. >> > > Reason: a CLEAR cache command should really CLEAR the cache! >> > >=20 >> > > Best, >> > > Matthias >> > >=20 >> > > Signed-off-by: Matthias Fischer >> > > --- >> > > src/initscripts/init.d/squid | 55 ++++++++++++++++++++++++++++-------= ---- >> > > ---- >> > > - >> > > 1 file changed, 35 insertions(+), 20 deletions(-) >> > >=20 >> > > diff --git a/src/initscripts/init.d/squid b/src/initscripts/init.d/squ= id >> > > index abed90a..1b369ea 100644 >> > > --- a/src/initscripts/init.d/squid >> > > +++ b/src/initscripts/init.d/squid >> > > @@ -94,9 +94,9 @@ case "$1" in >> > > stop) >> > > iptables -t nat -F SQUID >> > > if [ -e /var/run/squid.pid ]; then >> > > - boot_mesg "Stopping Squid Proxy Server..." >> > > - squid -k shutdown >/dev/null 2>&1 >> > > - evaluate_retval >> > > + boot_mesg -n "Stopping Squid Proxy Server...\n" >> > > + boot_mesg "(this may take up to a few minutes)" >> > > + /usr/sbin/squid -k shutdown >/dev/null 2>&1 >> > > =20 >> > > # Stop squidGuard, updxlrator, squidclamav >> > > # and redirect_wrappers. >> > > @@ -105,25 +105,41 @@ case "$1" in >> > > killproc /usr/bin/squidclamav >/dev/null & >> > > killproc /usr/sbin/redirect_wrapper >/dev/null & >> > > =20 >> > > - # Wait until all redirectors have been stopped. >> > > - wait >> > > - >> > > - # If squid is still running, wait up to 30 >> > > seconds >> > > - # before we go on to kill it. >> > > - counter=3D30 >> > > - >> > > - while [ ${counter} -gt 0 ]; do >> > > - statusproc /usr/sbin/squid >/dev/null && >> > > break; >> > > - sleep 1 >> > > - counter=3D$(( ${counter} - 1)) >> > > + # If some squid processes are still running, wait >> > > up >> > > to 360 seconds >> > > + # before we go on to kill the remaining >> > > process(es) >> > > and delete damaged >> > > + # '/var/log/cache/swap.state'. >> > > + n=3D0 >> > > + while squid -k check > /dev/null 2>&1 && [ $n -lt >> > > 360 >> > > ]; do >> > > + sleep 2 >> > > + n=3D$(( ${n} + 2 )) >> > > + echo -n . >> > > done >> > > - >> > > - # Kill squid service, if still running. >> > > - killproc /usr/sbin/squid >/dev/null >> > > + echo "" >> > > + echo "Done." >> > > + echo "Shutdown time:" "$n" "seconds" >> > > + >> > > + # If (squid-1) is still running after 360 >> > > seconds, >> > > + # kill all squid processes and delete >> > > '/var/log/cache/swap.state'. >> > > + if ( pgrep -fl "(squid-1)" > /dev/null 2>&1 ); >> > > then >> > > + killproc /usr/sbin/squid >/dev/null >> > > + rm -rf /var/log/cache/swap.state >> > > + echo "" >> > > + boot_mesg -n "You should not be reading >> > > this >> > > warning.\n" >> > > + boot_mesg -n "Some squid-processes had to >> > > be >> > > killed after 360 seconds,\n" >> > > + boot_mesg -n "so the cache index file was >> > > damaged and had to be deleted.\n" >> > > + boot_mesg -n "This file will be rebuild >> > > during the next start." >> > > + echo_warning >> > > + echo "" >> > > + else >> > > + boot_mesg "All squid processes exited >> > > normally." >> > > + echo_ok >> > > + echo "" >> > > + fi >> > > + fi >> > > =20 >> > > # Trash remain pid file from squid. >> > > rm -rf /var/run/squid.pid >> > > - fi >> > > + >> > > ;; >> > > =20 >> > > restart) >> > > @@ -143,8 +159,7 @@ case "$1" in >> > > =20 >> > > flush) >> > > $0 stop >> > > - echo > /var/log/cache/swap.state >> > > - chown squid.squid /var/log/cache/swap.state >> > > + rm -rf /var/log/cache/* >> > > sleep 1 >> > > $0 start >> > > ;; >=20 --===============1978458549182477861==--