From mboxrd@z Thu Jan 1 00:00:00 1970
From: Michael Tremer <michael.tremer@ipfire.org>
To: development@lists.ipfire.org
Subject: Re: [PATCH] squid init v_final
Date: Fri, 20 May 2016 11:06:06 +0100
Message-ID: <1463738766.19435.40.camel@ipfire.org>
In-Reply-To: <e2aa1cdd-aa64-f687-7222-eb5937cf9663@ipfire.org>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============2230758017706074210=="
List-Id: <development.lists.ipfire.org>
--===============2230758017706074210==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Looking good on my end, too.
We will have to wait for Monday when all the students are coming back.
-Michael
On Thu, 2016-05-19 at 21:03 +0200, Matthias Fischer wrote:
> Hi,
>=20
> first feedback from here:
>=20
> Works. ;-)
>=20
> No squidclamav-crashes, clean log.
>=20
> Max Mem size: 32768 KB
> Max Swap size: 262144 KB
> ~3000 Objects (not much in cache)
>=20
> Took about 8 seconds for stopping '(squid-1)'.
>=20
> I'll watch this during the next days.
>=20
> Best,
> Matthias
>=20
> On 18.05.2016 23:22, Michael Tremer wrote:
> >=20
> > Hi,
> >=20
> > On Wed, 2016-05-18 at 21:03 +0200, Matthias Fischer wrote:
> > >=20
> > > Just coming back from work...
> > >=20
> > > On 18.05.2016 20:15, Michael Tremer wrote:
> > > >=20
> > > >=20
> > > > Hi,
> > > Hi,
> > > =C2=A0
> > > >=20
> > > >=20
> > > > thanks. I finally merged this with a few changes:
> > > >=20
> > > > =C2=A0 http://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dcommitdiff;h=3D6=
113575d8120159
> > > > 1b2e
> > > > 8a76520579a4e7b7c5d46
> > > >=20
> > > > and added some more changes:
> > > >=20
> > > > =C2=A0 http://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dcommitdiff;h=3D6=
91b2836c0ffa54
> > > > b3f7
> > > > fc8d6f3b98377b3c9a470
> > > > =C2=A0 http://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dcommitdiff;h=3D3=
fcc9b67fc7ab18
> > > > 8624
> > > > 4a647dd7cf98f00e5c555
> > > >=20
> > > > I used boot_mesg instead of echo in some situations. Refrased the
> > > > warning
> > > > message. Removed -r from rm. And finally if squid shut down gracefully
> > > > the
> > > > shut
> > > > down time is logged to syslog instead of being written to the console.
> > > I took a quick look. For me, the warning message goes across the screen.
> > > Nevertheless, message will clearly be seen.
> > Yes, it is kind of long, but will line-break then.
> >=20
> > >=20
> > > That 'while'-counters look as if you got some cats running across your
> > > keyboard!? ;-)
> > > Such things are "a bit" beyond my programming skills. Nice.
> > That is just modulus. It calculates the remainder of a division of n by 6.
> > If
> > that is 0, then a dot will be printed. That way every 6 cycles only one d=
ot
> > is
> > printed.
> >=20
> > >=20
> > > And I never thought of moving that d*** 'killproc'/'wait'-block to the
> > > bottom.
> > > I'm really curious what 'squidclamav' will make of this.
> > Actually this should never do anything because squid should take care of
> > killing
> > all processes. However we had issues with that in the past so that we are
> > killing those manually.
> >=20
> > >=20
> > > >=20
> > > >=20
> > > > I also added some more comments to the script so that we know what is
> > > > going
> > > > on
> > > > in a few months down from here.
> > > >=20
> > > > This is working fine for me. Even with a small and almost empty cache
> > > > squid
> > > > takes about 14 seconds to shut down on my IPFire Prime Box.
> > > 14 seconds! Not bad.
> > > I thought that it would take *some* time, but not that much.
> > > One problem remains - what happens to really BIG caches.
> > > I don't have a solution for that by now.
> > Well, they should be closed within 6 minutes. If not they will rebuild ev=
ery
> > time.
> >=20
> > But I think if it doesn't close within in 6 minutes your cache is way too
> > big
> > for your hardware and should be downsized.
> >=20
> > We might have to wait for some feedback.
> >=20
> > >=20
> > >=20
> > > >=20
> > > >=20
> > > > Please test everything again. I am going to deploy this on a few more
> > > > test
> > > > systems this week.
> > > Testmachine didn't complain - it did everything right while testing
> > > with "360" and "3" seconds.
> > >=20
> > > Now running on production machine.
> > > Tomorrow I'll see what the logs really think of this. ;-)
> > I installed it on one of the big test machines. Waiting for feedback...
> >=20
> > Best,
> > -Michael
> >=20
> > >=20
> > >=20
> > > Best, Matthias
> > >=20
> > > >=20
> > > >=20
> > > >=20
> > > > Best,
> > > > -Michael
> > > >=20
> > > > On Tue, 2016-05-17 at 21:33 +0200, Matthias Fischer wrote:
> > > > >=20
> > > > >=20
> > > > > Last update suggestions for (3.5.xx) initscript, awaiting feedback.=
;-
> > > > > )
> > > > >=20
> > > > > Changelog:
> > > > >=20
> > > > > - Raised 'while'-loop-time for stopping squid to 360 seconds until
> > > > > =C2=A0 '/var/log/cache/swap.state' is deleted. NOT the whole cache
> > > > > structure!
> > > > > =C2=A0 I know, this timeout could not be enough for really BIG cach=
es.
> > > > > =C2=A0 The only other choice I know would be to leave this loop alo=
ne
> > > > > without
> > > > > =C2=A0 the "xxx seconds"-counter and wait how 'squid -k shutdown' h=
andles
> > > > > this.
> > > > > =C2=A0 But if it fails, I fear that we could end up in an endless l=
oop.
> > > > > =C2=A0 Suggestions are welcome!
> > > > >=20
> > > > > - Had to delete 'wait' after killing squidguard, updxlrator,
> > > > > squidclamav
> > > > > =C2=A0 and redirect_wrappers - 'while'-loop counter wouldn't work b=
ecause
> > > > > of
> > > > > =C2=A0 this line.
> > > > > =C2=A0 Suggestions for better handling?
> > > > >=20
> > > > > - Process detection looks for leftover '(squid-1)'-process using
> > > > > 'pgrep'.
> > > > >=20
> > > > > - Cosmetic changes to some 'boot_mesg' lines. Added a few. Is it
> > > > > =C2=A0 (still) too much? I'd prefer to get a warning (an announceme=
nt) if
> > > > > =C2=A0 '/var/log/cache/swap.state' was deleted.
> > > > >=20
> > > > > - Changed the 'flush'-command to really delete the entire
> > > > > =C2=A0 '/var/log/cache'-structure, it will automatically be rebuild
> > > > > =C2=A0 during the next start.
> > > > > =C2=A0 Reason: a CLEAR cache command should really CLEAR the cache!
> > > > >=20
> > > > > Best,
> > > > > Matthias
> > > > >=20
> > > > > Signed-off-by: Matthias Fischer <matthias.fischer(a)ipfire.org>
> > > > > ---
> > > > > =C2=A0src/initscripts/init.d/squid | 55 +++++++++++++++++++++++++++=
+-------
> > > > > ----
> > > > > ----
> > > > > -
> > > > > =C2=A01 file changed, 35 insertions(+), 20 deletions(-)
> > > > >=20
> > > > > diff --git a/src/initscripts/init.d/squid
> > > > > b/src/initscripts/init.d/squid
> > > > > index abed90a..1b369ea 100644
> > > > > --- a/src/initscripts/init.d/squid
> > > > > +++ b/src/initscripts/init.d/squid
> > > > > @@ -94,9 +94,9 @@ case "$1" in
> > > > > =C2=A0 stop)
> > > > > =C2=A0 iptables -t nat -F SQUID
> > > > > =C2=A0 if [ -e /var/run/squid.pid ]; then
> > > > > - boot_mesg "Stopping Squid Proxy Server..."
> > > > > - squid -k shutdown >/dev/null 2>&1
> > > > > - evaluate_retval
> > > > > + boot_mesg -n "Stopping Squid Proxy
> > > > > Server...\n"
> > > > > + boot_mesg "(this may take up to a few
> > > > > minutes)"
> > > > > + /usr/sbin/squid -k shutdown >/dev/null 2>&1
> > > > > =C2=A0
> > > > > =C2=A0 # Stop squidGuard, updxlrator, squidclamav
> > > > > =C2=A0 # and redirect_wrappers.
> > > > > @@ -105,25 +105,41 @@ case "$1" in
> > > > > =C2=A0 killproc /usr/bin/squidclamav >/dev/null &
> > > > > =C2=A0 killproc /usr/sbin/redirect_wrapper
> > > > > >/dev/null &
> > > > > =C2=A0
> > > > > - # Wait until all redirectors have been
> > > > > stopped.
> > > > > - wait
> > > > > -
> > > > > - # If squid is still running, wait up to 30
> > > > > seconds
> > > > > - # before we go on to kill it.
> > > > > - counter=3D30
> > > > > -
> > > > > - while [ ${counter} -gt 0 ]; do
> > > > > - statusproc /usr/sbin/squid >/dev/null
> > > > > &&
> > > > > break;
> > > > > - sleep 1
> > > > > - counter=3D$(( ${counter} - 1))
> > > > > + # If some squid processes are still running,
> > > > > wait
> > > > > up
> > > > > to 360 seconds
> > > > > + # before we go on to kill the remaining
> > > > > process(es)
> > > > > and delete damaged
> > > > > + # '/var/log/cache/swap.state'.
> > > > > + n=3D0
> > > > > + while squid -k check > /dev/null 2>&1 && [ $n
> > > > > -lt
> > > > > 360
> > > > > ]; do
> > > > > + sleep 2
> > > > > + n=3D$(( ${n} + 2 ))
> > > > > + echo -n .
> > > > > =C2=A0 done
> > > > > -
> > > > > - # Kill squid service, if still running.
> > > > > - killproc /usr/sbin/squid >/dev/null
> > > > > + echo ""
> > > > > + echo "Done."
> > > > > + echo "Shutdown time:" "$n" "seconds"
> > > > > +
> > > > > + # If (squid-1) is still running after 360
> > > > > seconds,
> > > > > + # kill all squid processes and delete
> > > > > '/var/log/cache/swap.state'.
> > > > > + if ( pgrep -fl "(squid-1)" > /dev/null 2>&1
> > > > > );
> > > > > then
> > > > > + killproc /usr/sbin/squid >/dev/null
> > > > > + rm -rf /var/log/cache/swap.state
> > > > > + echo ""
> > > > > + boot_mesg -n "You should not be
> > > > > reading
> > > > > this
> > > > > warning.\n"
> > > > > + boot_mesg -n "Some squid-processes
> > > > > had to
> > > > > be
> > > > > killed after 360 seconds,\n"
> > > > > + boot_mesg -n "so the cache index file
> > > > > was
> > > > > damaged and had to be deleted.\n"
> > > > > + boot_mesg -n "This file will be
> > > > > rebuild
> > > > > during the next start."
> > > > > + echo_warning
> > > > > + echo ""
> > > > > + else
> > > > > + boot_mesg "All squid processes exited
> > > > > normally."
> > > > > + echo_ok
> > > > > + echo ""
> > > > > + fi
> > > > > + fi
> > > > > =C2=A0
> > > > > =C2=A0 # Trash remain pid file from squid.
> > > > > =C2=A0 rm -rf /var/run/squid.pid
> > > > > - fi
> > > > > +
> > > > > =C2=A0 ;;
> > > > > =C2=A0
> > > > > =C2=A0 restart)
> > > > > @@ -143,8 +159,7 @@ case "$1" in
> > > > > =C2=A0
> > > > > =C2=A0 flush)
> > > > > =C2=A0 $0 stop
> > > > > - echo > /var/log/cache/swap.state
> > > > > - chown squid.squid /var/log/cache/swap.state
> > > > > + rm -rf /var/log/cache/*
> > > > > =C2=A0 sleep 1
> > > > > =C2=A0 $0 start
> > > > > =C2=A0 ;;
--===============2230758017706074210==
Content-Type: application/pgp-signature
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="signature.asc"
MIME-Version: 1.0
LS0tLS1CRUdJTiBQR1AgU0lHTkFUVVJFLS0tLS0KVmVyc2lvbjogR251UEcgdjIKCmlRSWNCQUFC
Q2dBR0JRSlhQdUdPQUFvSkVJQjU4UDl2a0FrSGU4a1Ard1hWV3REWmpGZk9tVjJSS1FEV2h0R3gK
QVB0bTJNOTU1b2U1UUYzUjN5NUtkaWk1Sm1zY250bG9YekRtcG5UWjBaWEt5elJMRDV6VjdMTWxP
SVVodm16QgpwTXA1RU1WdmdpTHYyTWt4TUpHMkZTa3lITzNQMG5ySmtPVWhjT282RkVpbEhRRGJi
NXNKU3pkaC9RTi8zdmh5Ckl1T3V6bUZENGNNRTUrUW9WN3haN216V0U1c1Rsb3NYZW5SaGE2TFda
MHdPVEd4bGl0c212aFRKN0V4VXJBbkgKMm1YL09Pc1UrYTJTTm9PK2xaN2RQWTNlNmdPczlsWVFw
cWZRNHIxTXhpTWYyakw4aEZWUzV3RGdVK0QzVnhYaQo0T1A5SzYzVytWRnhRNjZPT1E2VUozT1RI
dGlNQUpONm9nQ1VuRW0wYnJIUTlrZi9PYTY0R1BDOVJ0MlMxZ3I5CndlQytVK2dUb2ZyWlhMSnRY
aWU2QXRRS2lGWi9jc1JRSHg5NDBUZjNmYUhLT2VuVmFITEtyK0YvWkU1ai9pdkUKdTNDK2pZdGxn
Vko3TytwY2J2QTVvYVErZjMzc1lEYlBoSVNUbkpSaUV5ZjI3Y2lTM3RBalZNNFh1TElIQU5iUwpu
Y0pIZjhoWU9BS043S2dvSTMwalk4L1V1bWxIbDJwTGNnOENpamkyZWlEeFFST2F5QXVHRm1hcWE1
clpicElLCjNsN251UXJBd1JFRzlhV1BJOWlHZlVYOHFGOUdvYmMyc3hHdnBpbkNrcStWVlg2UFV1
bkpJUTUwbWhsWUVEdWIKdi9DWXZDV1VGVHhTSWJheFNBSHJvNDJGblBueVVROUFQRWlhbGpHUTh5
SmxhWGZTalgvWWVnZVhIdlA4WXVpVQpwRURFZi84Uy90WUVlQ1FvcEEvZAo9NFo4SwotLS0tLUVO
RCBQR1AgU0lHTkFUVVJFLS0tLS0K
--===============2230758017706074210==--