Hi,

On Wed, 2016-05-18 at 21:03 +0200, Matthias Fischer wrote:
> Just coming back from work...
> 
> On 18.05.2016 20:15, Michael Tremer wrote:
> > 
> > Hi,
> Hi,
>  
> > 
> > thanks. I finally merged this with a few changes:
> > 
> >   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=6113575d81201591b2e
> > 8a76520579a4e7b7c5d46
> > 
> > and added some more changes:
> > 
> >   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=691b2836c0ffa54b3f7
> > fc8d6f3b98377b3c9a470
> >   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=3fcc9b67fc7ab188624
> > 4a647dd7cf98f00e5c555
> > 
> > I used boot_mesg instead of echo in some situations. Refrased the warning
> > message. Removed -r from rm. And finally if squid shut down gracefully the
> > shut
> > down time is logged to syslog instead of being written to the console.
> I took a quick look. For me, the warning message goes across the screen.
> Nevertheless, message will clearly be seen.

Yes, it is kind of long, but will line-break then.

> That 'while'-counters look as if you got some cats running across your
> keyboard!? ;-)
> Such things are "a bit" beyond my programming skills. Nice.

That is just modulus. It calculates the remainder of a division of n by 6. If
that is 0, then a dot will be printed. That way every 6 cycles only one dot is
printed.

> And I never thought of moving that d*** 'killproc'/'wait'-block to the bottom.
> I'm really curious what 'squidclamav' will make of this.

Actually this should never do anything because squid should take care of killing
all processes. However we had issues with that in the past so that we are
killing those manually.

> > 
> > I also added some more comments to the script so that we know what is going
> > on
> > in a few months down from here.
> > 
> > This is working fine for me. Even with a small and almost empty cache squid
> > takes about 14 seconds to shut down on my IPFire Prime Box.
> 14 seconds! Not bad.
> I thought that it would take *some* time, but not that much.
> One problem remains - what happens to really BIG caches.
> I don't have a solution for that by now.

Well, they should be closed within 6 minutes. If not they will rebuild every
time.

But I think if it doesn't close within in 6 minutes your cache is way too big
for your hardware and should be downsized.

We might have to wait for some feedback.

> 
> > 
> > Please test everything again. I am going to deploy this on a few more test
> > systems this week.
> Testmachine didn't complain - it did everything right while testing
> with "360" and "3" seconds.
> 
> Now running on production machine.
> Tomorrow I'll see what the logs really think of this. ;-)

I installed it on one of the big test machines. Waiting for feedback...

Best,
-Michael

> 
> Best, Matthias
> 
> > 
> > 
> > Best,
> > -Michael
> > 
> > On Tue, 2016-05-17 at 21:33 +0200, Matthias Fischer wrote:
> > > 
> > > Last update suggestions for (3.5.xx) initscript, awaiting feedback. ;-)
> > > 
> > > Changelog:
> > > 
> > > - Raised 'while'-loop-time for stopping squid to 360 seconds until
> > >   '/var/log/cache/swap.state' is deleted. NOT the whole cache structure!
> > >   I know, this timeout could not be enough for really BIG caches.
> > >   The only other choice I know would be to leave this loop alone without
> > >   the "xxx seconds"-counter and wait how 'squid -k shutdown' handles this.
> > >   But if it fails, I fear that we could end up in an endless loop.
> > >   Suggestions are welcome!
> > > 
> > > - Had to delete 'wait' after killing squidguard, updxlrator, squidclamav
> > >   and redirect_wrappers - 'while'-loop counter wouldn't work because of
> > >   this line.
> > >   Suggestions for better handling?
> > > 
> > > - Process detection looks for leftover '(squid-1)'-process using 'pgrep'.
> > > 
> > > - Cosmetic changes to some 'boot_mesg' lines. Added a few. Is it
> > >   (still) too much? I'd prefer to get a warning (an announcement) if
> > >   '/var/log/cache/swap.state' was deleted.
> > > 
> > > - Changed the 'flush'-command to really delete the entire
> > >   '/var/log/cache'-structure, it will automatically be rebuild
> > >   during the next start.
> > >   Reason: a CLEAR cache command should really CLEAR the cache!
> > > 
> > > Best,
> > > Matthias
> > > 
> > > Signed-off-by: Matthias Fischer <matthias.fischer(a)ipfire.org>
> > > ---
> > >  src/initscripts/init.d/squid | 55 ++++++++++++++++++++++++++++-----------
> > > ----
> > > -
> > >  1 file changed, 35 insertions(+), 20 deletions(-)
> > > 
> > > diff --git a/src/initscripts/init.d/squid b/src/initscripts/init.d/squid
> > > index abed90a..1b369ea 100644
> > > --- a/src/initscripts/init.d/squid
> > > +++ b/src/initscripts/init.d/squid
> > > @@ -94,9 +94,9 @@ case "$1" in
> > >  	stop)
> > >  			iptables -t nat -F SQUID
> > >  		if [ -e /var/run/squid.pid ]; then
> > > -			boot_mesg "Stopping Squid Proxy Server..."
> > > -			squid -k shutdown >/dev/null 2>&1
> > > -			evaluate_retval
> > > +			boot_mesg -n "Stopping Squid Proxy Server...\n"
> > > +			boot_mesg "(this may take up to a few minutes)"
> > > +			/usr/sbin/squid -k shutdown >/dev/null 2>&1
> > >  
> > >  			# Stop squidGuard, updxlrator, squidclamav
> > >  			# and redirect_wrappers.
> > > @@ -105,25 +105,41 @@ case "$1" in
> > >  			killproc /usr/bin/squidclamav >/dev/null &
> > >  			killproc /usr/sbin/redirect_wrapper >/dev/null &
> > >  
> > > -			# Wait until all redirectors have been stopped.
> > > -			wait
> > > -
> > > -			# If squid is still running, wait up to 30
> > > seconds
> > > -			# before we go on to kill it.
> > > -			counter=30
> > > -
> > > -			while [ ${counter} -gt 0 ]; do
> > > -				statusproc /usr/sbin/squid >/dev/null &&
> > > break;
> > > -				sleep 1
> > > -				counter=$(( ${counter} - 1))
> > > +			# If some squid processes are still running, wait
> > > up
> > > to 360 seconds
> > > +			# before we go on to kill the remaining
> > > process(es)
> > > and delete damaged
> > > +			# '/var/log/cache/swap.state'.
> > > +			n=0
> > > +			while squid -k check > /dev/null 2>&1 && [ $n -lt
> > > 360
> > > ]; do
> > > +				sleep 2
> > > +				n=$(( ${n} + 2 ))
> > > +				echo -n .
> > >  			done
> > > -
> > > -			# Kill squid service, if still running.
> > > -			killproc /usr/sbin/squid >/dev/null
> > > +			echo ""
> > > +			echo "Done."
> > > +			echo "Shutdown time:" "$n" "seconds"
> > > +
> > > +			# If (squid-1) is still running after 360
> > > seconds,
> > > +			# kill all squid processes and delete
> > > '/var/log/cache/swap.state'.
> > > +			if ( pgrep -fl "(squid-1)" > /dev/null 2>&1 );
> > > then
> > > +				killproc /usr/sbin/squid >/dev/null
> > > +				rm -rf /var/log/cache/swap.state
> > > +				echo ""
> > > +				boot_mesg -n "You should not be reading
> > > this
> > > warning.\n"
> > > +				boot_mesg -n "Some squid-processes had to
> > > be
> > > killed after 360 seconds,\n"
> > > +				boot_mesg -n "so the cache index file was
> > > damaged and had to be deleted.\n"
> > > +				boot_mesg -n "This file will be rebuild
> > > during the next start."
> > > +				echo_warning
> > > +				echo ""
> > > +			else
> > > +				boot_mesg "All squid processes exited
> > > normally."
> > > +				echo_ok
> > > +				echo ""
> > > +			fi
> > > +		fi
> > >  
> > >  			# Trash remain pid file from squid.
> > >  			rm -rf /var/run/squid.pid
> > > -		fi
> > > +
> > >  		;;
> > >  
> > >  	restart)
> > > @@ -143,8 +159,7 @@ case "$1" in
> > >  
> > >  	flush)
> > >  		$0 stop
> > > -		echo > /var/log/cache/swap.state
> > > -		chown squid.squid /var/log/cache/swap.state
> > > +		rm -rf /var/log/cache/*
> > >  		sleep 1
> > >  		$0 start
> > >  		;;