Re: [PATCH] squid init v_final

[Michael Tremer <michael.tremer@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 9838 bytes --]

Looking good on my end, too.

We will have to wait for Monday when all the students are coming back.

-Michael

On Thu, 2016-05-19 at 21:03 +0200, Matthias Fischer wrote:
> Hi,
> 
> first feedback from here:
> 
> Works. ;-)
> 
> No squidclamav-crashes, clean log.
> 
> Max Mem size: 32768 KB
> Max Swap size: 262144 KB
> ~3000 Objects (not much in cache)
> 
> Took about 8 seconds for stopping '(squid-1)'.
> 
> I'll watch this during the next days.
> 
> Best,
> Matthias
> 
> On 18.05.2016 23:22, Michael Tremer wrote:
> > 
> > Hi,
> > 
> > On Wed, 2016-05-18 at 21:03 +0200, Matthias Fischer wrote:
> > > 
> > > Just coming back from work...
> > > 
> > > On 18.05.2016 20:15, Michael Tremer wrote:
> > > > 
> > > > 
> > > > Hi,
> > > Hi,
> > >  
> > > > 
> > > > 
> > > > thanks. I finally merged this with a few changes:
> > > > 
> > > >   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=6113575d8120159
> > > > 1b2e
> > > > 8a76520579a4e7b7c5d46
> > > > 
> > > > and added some more changes:
> > > > 
> > > >   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=691b2836c0ffa54
> > > > b3f7
> > > > fc8d6f3b98377b3c9a470
> > > >   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=3fcc9b67fc7ab18
> > > > 8624
> > > > 4a647dd7cf98f00e5c555
> > > > 
> > > > I used boot_mesg instead of echo in some situations. Refrased the
> > > > warning
> > > > message. Removed -r from rm. And finally if squid shut down gracefully
> > > > the
> > > > shut
> > > > down time is logged to syslog instead of being written to the console.
> > > I took a quick look. For me, the warning message goes across the screen.
> > > Nevertheless, message will clearly be seen.
> > Yes, it is kind of long, but will line-break then.
> > 
> > > 
> > > That 'while'-counters look as if you got some cats running across your
> > > keyboard!? ;-)
> > > Such things are "a bit" beyond my programming skills. Nice.
> > That is just modulus. It calculates the remainder of a division of n by 6.
> > If
> > that is 0, then a dot will be printed. That way every 6 cycles only one dot
> > is
> > printed.
> > 
> > > 
> > > And I never thought of moving that d*** 'killproc'/'wait'-block to the
> > > bottom.
> > > I'm really curious what 'squidclamav' will make of this.
> > Actually this should never do anything because squid should take care of
> > killing
> > all processes. However we had issues with that in the past so that we are
> > killing those manually.
> > 
> > > 
> > > > 
> > > > 
> > > > I also added some more comments to the script so that we know what is
> > > > going
> > > > on
> > > > in a few months down from here.
> > > > 
> > > > This is working fine for me. Even with a small and almost empty cache
> > > > squid
> > > > takes about 14 seconds to shut down on my IPFire Prime Box.
> > > 14 seconds! Not bad.
> > > I thought that it would take *some* time, but not that much.
> > > One problem remains - what happens to really BIG caches.
> > > I don't have a solution for that by now.
> > Well, they should be closed within 6 minutes. If not they will rebuild every
> > time.
> > 
> > But I think if it doesn't close within in 6 minutes your cache is way too
> > big
> > for your hardware and should be downsized.
> > 
> > We might have to wait for some feedback.
> > 
> > > 
> > > 
> > > > 
> > > > 
> > > > Please test everything again. I am going to deploy this on a few more
> > > > test
> > > > systems this week.
> > > Testmachine didn't complain - it did everything right while testing
> > > with "360" and "3" seconds.
> > > 
> > > Now running on production machine.
> > > Tomorrow I'll see what the logs really think of this. ;-)
> > I installed it on one of the big test machines. Waiting for feedback...
> > 
> > Best,
> > -Michael
> > 
> > > 
> > > 
> > > Best, Matthias
> > > 
> > > > 
> > > > 
> > > > 
> > > > Best,
> > > > -Michael
> > > > 
> > > > On Tue, 2016-05-17 at 21:33 +0200, Matthias Fischer wrote:
> > > > > 
> > > > > 
> > > > > Last update suggestions for (3.5.xx) initscript, awaiting feedback. ;-
> > > > > )
> > > > > 
> > > > > Changelog:
> > > > > 
> > > > > - Raised 'while'-loop-time for stopping squid to 360 seconds until
> > > > >   '/var/log/cache/swap.state' is deleted. NOT the whole cache
> > > > > structure!
> > > > >   I know, this timeout could not be enough for really BIG caches.
> > > > >   The only other choice I know would be to leave this loop alone
> > > > > without
> > > > >   the "xxx seconds"-counter and wait how 'squid -k shutdown' handles
> > > > > this.
> > > > >   But if it fails, I fear that we could end up in an endless loop.
> > > > >   Suggestions are welcome!
> > > > > 
> > > > > - Had to delete 'wait' after killing squidguard, updxlrator,
> > > > > squidclamav
> > > > >   and redirect_wrappers - 'while'-loop counter wouldn't work because
> > > > > of
> > > > >   this line.
> > > > >   Suggestions for better handling?
> > > > > 
> > > > > - Process detection looks for leftover '(squid-1)'-process using
> > > > > 'pgrep'.
> > > > > 
> > > > > - Cosmetic changes to some 'boot_mesg' lines. Added a few. Is it
> > > > >   (still) too much? I'd prefer to get a warning (an announcement) if
> > > > >   '/var/log/cache/swap.state' was deleted.
> > > > > 
> > > > > - Changed the 'flush'-command to really delete the entire
> > > > >   '/var/log/cache'-structure, it will automatically be rebuild
> > > > >   during the next start.
> > > > >   Reason: a CLEAR cache command should really CLEAR the cache!
> > > > > 
> > > > > Best,
> > > > > Matthias
> > > > > 
> > > > > Signed-off-by: Matthias Fischer <matthias.fischer(a)ipfire.org>
> > > > > ---
> > > > >  src/initscripts/init.d/squid | 55 ++++++++++++++++++++++++++++-------
> > > > > ----
> > > > > ----
> > > > > -
> > > > >  1 file changed, 35 insertions(+), 20 deletions(-)
> > > > > 
> > > > > diff --git a/src/initscripts/init.d/squid
> > > > > b/src/initscripts/init.d/squid
> > > > > index abed90a..1b369ea 100644
> > > > > --- a/src/initscripts/init.d/squid
> > > > > +++ b/src/initscripts/init.d/squid
> > > > > @@ -94,9 +94,9 @@ case "$1" in
> > > > >  	stop)
> > > > >  			iptables -t nat -F SQUID
> > > > >  		if [ -e /var/run/squid.pid ]; then
> > > > > -			boot_mesg "Stopping Squid Proxy Server..."
> > > > > -			squid -k shutdown >/dev/null 2>&1
> > > > > -			evaluate_retval
> > > > > +			boot_mesg -n "Stopping Squid Proxy
> > > > > Server...\n"
> > > > > +			boot_mesg "(this may take up to a few
> > > > > minutes)"
> > > > > +			/usr/sbin/squid -k shutdown >/dev/null 2>&1
> > > > >  
> > > > >  			# Stop squidGuard, updxlrator, squidclamav
> > > > >  			# and redirect_wrappers.
> > > > > @@ -105,25 +105,41 @@ case "$1" in
> > > > >  			killproc /usr/bin/squidclamav >/dev/null &
> > > > >  			killproc /usr/sbin/redirect_wrapper
> > > > > >/dev/null &
> > > > >  
> > > > > -			# Wait until all redirectors have been
> > > > > stopped.
> > > > > -			wait
> > > > > -
> > > > > -			# If squid is still running, wait up to 30
> > > > > seconds
> > > > > -			# before we go on to kill it.
> > > > > -			counter=30
> > > > > -
> > > > > -			while [ ${counter} -gt 0 ]; do
> > > > > -				statusproc /usr/sbin/squid >/dev/null
> > > > > &&
> > > > > break;
> > > > > -				sleep 1
> > > > > -				counter=$(( ${counter} - 1))
> > > > > +			# If some squid processes are still running,
> > > > > wait
> > > > > up
> > > > > to 360 seconds
> > > > > +			# before we go on to kill the remaining
> > > > > process(es)
> > > > > and delete damaged
> > > > > +			# '/var/log/cache/swap.state'.
> > > > > +			n=0
> > > > > +			while squid -k check > /dev/null 2>&1 && [ $n
> > > > > -lt
> > > > > 360
> > > > > ]; do
> > > > > +				sleep 2
> > > > > +				n=$(( ${n} + 2 ))
> > > > > +				echo -n .
> > > > >  			done
> > > > > -
> > > > > -			# Kill squid service, if still running.
> > > > > -			killproc /usr/sbin/squid >/dev/null
> > > > > +			echo ""
> > > > > +			echo "Done."
> > > > > +			echo "Shutdown time:" "$n" "seconds"
> > > > > +
> > > > > +			# If (squid-1) is still running after 360
> > > > > seconds,
> > > > > +			# kill all squid processes and delete
> > > > > '/var/log/cache/swap.state'.
> > > > > +			if ( pgrep -fl "(squid-1)" > /dev/null 2>&1
> > > > > );
> > > > > then
> > > > > +				killproc /usr/sbin/squid >/dev/null
> > > > > +				rm -rf /var/log/cache/swap.state
> > > > > +				echo ""
> > > > > +				boot_mesg -n "You should not be
> > > > > reading
> > > > > this
> > > > > warning.\n"
> > > > > +				boot_mesg -n "Some squid-processes
> > > > > had to
> > > > > be
> > > > > killed after 360 seconds,\n"
> > > > > +				boot_mesg -n "so the cache index file
> > > > > was
> > > > > damaged and had to be deleted.\n"
> > > > > +				boot_mesg -n "This file will be
> > > > > rebuild
> > > > > during the next start."
> > > > > +				echo_warning
> > > > > +				echo ""
> > > > > +			else
> > > > > +				boot_mesg "All squid processes exited
> > > > > normally."
> > > > > +				echo_ok
> > > > > +				echo ""
> > > > > +			fi
> > > > > +		fi
> > > > >  
> > > > >  			# Trash remain pid file from squid.
> > > > >  			rm -rf /var/run/squid.pid
> > > > > -		fi
> > > > > +
> > > > >  		;;
> > > > >  
> > > > >  	restart)
> > > > > @@ -143,8 +159,7 @@ case "$1" in
> > > > >  
> > > > >  	flush)
> > > > >  		$0 stop
> > > > > -		echo > /var/log/cache/swap.state
> > > > > -		chown squid.squid /var/log/cache/swap.state
> > > > > +		rm -rf /var/log/cache/*
> > > > >  		sleep 1
> > > > >  		$0 start
> > > > >  		;;

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]