From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer To: development@lists.ipfire.org Subject: Re: [PATCH] squid init v_final Date: Wed, 18 May 2016 22:22:39 +0100 Message-ID: <1463606559.19435.25.camel@ipfire.org> In-Reply-To: <396235c5-8626-a645-9953-dfd430d8d404@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1815047835894497844==" List-Id: --===============1815047835894497844== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi, On Wed, 2016-05-18 at 21:03 +0200, Matthias Fischer wrote: > Just coming back from work... >=20 > On 18.05.2016 20:15, Michael Tremer wrote: > >=20 > > Hi, > Hi, > =C2=A0 > >=20 > > thanks. I finally merged this with a few changes: > >=20 > > =C2=A0 http://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dcommitdiff;h=3D61135= 75d81201591b2e > > 8a76520579a4e7b7c5d46 > >=20 > > and added some more changes: > >=20 > > =C2=A0 http://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dcommitdiff;h=3D691b2= 836c0ffa54b3f7 > > fc8d6f3b98377b3c9a470 > > =C2=A0 http://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dcommitdiff;h=3D3fcc9= b67fc7ab188624 > > 4a647dd7cf98f00e5c555 > >=20 > > I used boot_mesg instead of echo in some situations. Refrased the warning > > message. Removed -r from rm. And finally if squid shut down gracefully the > > shut > > down time is logged to syslog instead of being written to the console. > I took a quick look. For me, the warning message goes across the screen. > Nevertheless, message will clearly be seen. Yes, it is kind of long, but will line-break then. > That 'while'-counters look as if you got some cats running across your > keyboard!? ;-) > Such things are "a bit" beyond my programming skills. Nice. That is just modulus. It calculates the remainder of a division of n by 6. If that is 0, then a dot will be printed. That way every 6 cycles only one dot is printed. > And I never thought of moving that d*** 'killproc'/'wait'-block to the bott= om. > I'm really curious what 'squidclamav' will make of this. Actually this should never do anything because squid should take care of kill= ing all processes. However we had issues with that in the past so that we are killing those manually. > >=20 > > I also added some more comments to the script so that we know what is goi= ng > > on > > in a few months down from here. > >=20 > > This is working fine for me. Even with a small and almost empty cache squ= id > > takes about 14 seconds to shut down on my IPFire Prime Box. > 14 seconds! Not bad. > I thought that it would take *some* time, but not that much. > One problem remains - what happens to really BIG caches. > I don't have a solution for that by now. Well, they should be closed within 6 minutes. If not they will rebuild every time. But I think if it doesn't close within in 6 minutes your cache is way too big for your hardware and should be downsized. We might have to wait for some feedback. >=20 > >=20 > > Please test everything again. I am going to deploy this on a few more test > > systems this week. > Testmachine didn't complain - it did everything right while testing > with "360" and "3" seconds. >=20 > Now running on production machine. > Tomorrow I'll see what the logs really think of this. ;-) I installed it on one of the big test machines. Waiting for feedback... Best, -Michael >=20 > Best, Matthias >=20 > >=20 > >=20 > > Best, > > -Michael > >=20 > > On Tue, 2016-05-17 at 21:33 +0200, Matthias Fischer wrote: > > >=20 > > > Last update suggestions for (3.5.xx) initscript, awaiting feedback. ;-) > > >=20 > > > Changelog: > > >=20 > > > - Raised 'while'-loop-time for stopping squid to 360 seconds until > > > =C2=A0 '/var/log/cache/swap.state' is deleted. NOT the whole cache stru= cture! > > > =C2=A0 I know, this timeout could not be enough for really BIG caches. > > > =C2=A0 The only other choice I know would be to leave this loop alone w= ithout > > > =C2=A0 the "xxx seconds"-counter and wait how 'squid -k shutdown' handl= es this. > > > =C2=A0 But if it fails, I fear that we could end up in an endless loop. > > > =C2=A0 Suggestions are welcome! > > >=20 > > > - Had to delete 'wait' after killing squidguard, updxlrator, squidclamav > > > =C2=A0 and redirect_wrappers - 'while'-loop counter wouldn't work becau= se of > > > =C2=A0 this line. > > > =C2=A0 Suggestions for better handling? > > >=20 > > > - Process detection looks for leftover '(squid-1)'-process using 'pgrep= '. > > >=20 > > > - Cosmetic changes to some 'boot_mesg' lines. Added a few. Is it > > > =C2=A0 (still) too much? I'd prefer to get a warning (an announcement) = if > > > =C2=A0 '/var/log/cache/swap.state' was deleted. > > >=20 > > > - Changed the 'flush'-command to really delete the entire > > > =C2=A0 '/var/log/cache'-structure, it will automatically be rebuild > > > =C2=A0 during the next start. > > > =C2=A0 Reason: a CLEAR cache command should really CLEAR the cache! > > >=20 > > > Best, > > > Matthias > > >=20 > > > Signed-off-by: Matthias Fischer > > > --- > > > =C2=A0src/initscripts/init.d/squid | 55 ++++++++++++++++++++++++++++---= -------- > > > ---- > > > - > > > =C2=A01 file changed, 35 insertions(+), 20 deletions(-) > > >=20 > > > diff --git a/src/initscripts/init.d/squid b/src/initscripts/init.d/squid > > > index abed90a..1b369ea 100644 > > > --- a/src/initscripts/init.d/squid > > > +++ b/src/initscripts/init.d/squid > > > @@ -94,9 +94,9 @@ case "$1" in > > > =C2=A0 stop) > > > =C2=A0 iptables -t nat -F SQUID > > > =C2=A0 if [ -e /var/run/squid.pid ]; then > > > - boot_mesg "Stopping Squid Proxy Server..." > > > - squid -k shutdown >/dev/null 2>&1 > > > - evaluate_retval > > > + boot_mesg -n "Stopping Squid Proxy Server...\n" > > > + boot_mesg "(this may take up to a few minutes)" > > > + /usr/sbin/squid -k shutdown >/dev/null 2>&1 > > > =C2=A0 > > > =C2=A0 # Stop squidGuard, updxlrator, squidclamav > > > =C2=A0 # and redirect_wrappers. > > > @@ -105,25 +105,41 @@ case "$1" in > > > =C2=A0 killproc /usr/bin/squidclamav >/dev/null & > > > =C2=A0 killproc /usr/sbin/redirect_wrapper >/dev/null & > > > =C2=A0 > > > - # Wait until all redirectors have been stopped. > > > - wait > > > - > > > - # If squid is still running, wait up to 30 > > > seconds > > > - # before we go on to kill it. > > > - counter=3D30 > > > - > > > - while [ ${counter} -gt 0 ]; do > > > - statusproc /usr/sbin/squid >/dev/null && > > > break; > > > - sleep 1 > > > - counter=3D$(( ${counter} - 1)) > > > + # If some squid processes are still running, wait > > > up > > > to 360 seconds > > > + # before we go on to kill the remaining > > > process(es) > > > and delete damaged > > > + # '/var/log/cache/swap.state'. > > > + n=3D0 > > > + while squid -k check > /dev/null 2>&1 && [ $n -lt > > > 360 > > > ]; do > > > + sleep 2 > > > + n=3D$(( ${n} + 2 )) > > > + echo -n . > > > =C2=A0 done > > > - > > > - # Kill squid service, if still running. > > > - killproc /usr/sbin/squid >/dev/null > > > + echo "" > > > + echo "Done." > > > + echo "Shutdown time:" "$n" "seconds" > > > + > > > + # If (squid-1) is still running after 360 > > > seconds, > > > + # kill all squid processes and delete > > > '/var/log/cache/swap.state'. > > > + if ( pgrep -fl "(squid-1)" > /dev/null 2>&1 ); > > > then > > > + killproc /usr/sbin/squid >/dev/null > > > + rm -rf /var/log/cache/swap.state > > > + echo "" > > > + boot_mesg -n "You should not be reading > > > this > > > warning.\n" > > > + boot_mesg -n "Some squid-processes had to > > > be > > > killed after 360 seconds,\n" > > > + boot_mesg -n "so the cache index file was > > > damaged and had to be deleted.\n" > > > + boot_mesg -n "This file will be rebuild > > > during the next start." > > > + echo_warning > > > + echo "" > > > + else > > > + boot_mesg "All squid processes exited > > > normally." > > > + echo_ok > > > + echo "" > > > + fi > > > + fi > > > =C2=A0 > > > =C2=A0 # Trash remain pid file from squid. > > > =C2=A0 rm -rf /var/run/squid.pid > > > - fi > > > + > > > =C2=A0 ;; > > > =C2=A0 > > > =C2=A0 restart) > > > @@ -143,8 +159,7 @@ case "$1" in > > > =C2=A0 > > > =C2=A0 flush) > > > =C2=A0 $0 stop > > > - echo > /var/log/cache/swap.state > > > - chown squid.squid /var/log/cache/swap.state > > > + rm -rf /var/log/cache/* > > > =C2=A0 sleep 1 > > > =C2=A0 $0 start > > > =C2=A0 ;; --===============1815047835894497844== Content-Type: application/pgp-signature Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="signature.asc" MIME-Version: 1.0 LS0tLS1CRUdJTiBQR1AgU0lHTkFUVVJFLS0tLS0KVmVyc2lvbjogR251UEcgdjIKCmlRSWNCQUFC Q2dBR0JRSlhQTjBmQUFvSkVJQjU4UDl2a0FrSEVzUVFBSWpuMDZMYVVaSFZOcWVIa3ZFdDlVbG8K Vi9hamx0WGJGeGZ4OUZzbGtmekN5QnRjVnlIVVNCVnRFR3FIVUxTVUxjSmxtRlNSYTFwcWprOU5H bDRjemhwSQo1TGhyMFVZeGJPS09Ha3NITXRma3JIQW1RM3FYdW1FZFhUQW5mM3hQaHlDcEg5UkF3 ZS83cDQzUnNnYitteGdsCitrQ0tFS29aNzJvZnFSNlk4eXlvRStXZ0tqSTFoVVlKU1ZuL1ZnTXdI a01JQWRwTkpGMlN1cXl2NytNek9ncjcKS0tsZmNOYXB2aW55MVBpTzRyWENFNFhtdTM5akZmTlE4 VEJzV3NlR3lMVmg4MHVwZFo4WXZPeVdtZERkNzhxeApMc2FaV1c1N1FpelZrYTdhQnVkbjZBakIy NysvK005cDUzU2R0L0hUUjhVWjNzQzhhejhaUzhuOUVISk9Wdm5rCitrZFkxeVBnYS9QL2dkZ1cy emNUS0trTUFHSzkvWTJaZFkrNmRkNVNiaEQ2bEtQOGs1U1hwRFB2ZDVXdzFsOVMKeEU5TFV1amJx d1M2TDF4QWRaZUJiMU1EekRXUVd5cWxGczlUMGdqdU9JTHhXV3I3bHByWmNzV1F4NXptb2UxWgpS cW9SakI5SWZDcGoyMXpPUG1QVE5JU0hKTG9pNHFwclpRbVVUUGU3c1VZcXNzMi9YaHBHcTFhWGhP K0dFb3dUCk5DdjQ4ajB2cWxjZEdpeDR6QmxLamwyTGpOWVkrb0NRSW1hb1pURGpOTlN2MEJXdTBY VzJKOVFkU25rclhNSzMKV3oydDNxQm1qaCt6RjVRTDhQaUxWTDNOcnhldW54RkZkbmVOS3I5Z1Bv Mno3VXliWkF1MDl1czlVbnRwUkZxagpDOVdXOEtzVjA1OHJPQ2xxSjJPTQo9V3JDQgotLS0tLUVO RCBQR1AgU0lHTkFUVVJFLS0tLS0K --===============1815047835894497844==--