Hello guys,
Upon request, I would like to announce that on Saturday, 12th Jan, we are going to branch Core Update 127.
This means that we no longer accept any patches that introduce any new features and only accept patches that fix any bugs introduced. Nothing new here, you all know how this works :)
This announcement is just coming now so that you all know this. It is a bit short notice this time, but I hope I will be better with this next time round. In the past some patches have been submitted after the merge window closed and had to be deferred to the next Core Update to the frustration of the submitters. Therefore, we now try this and hope that we can make the process better for everyone who is contributing!
We have an update with many new features here. Please help testing as soon as the first build is available.
Best, -Michael
Hello Michael,
thank you for the notification. :-)
Currently, I do not have anything for Core Update 127 left and unfortunately, things are very busy.
I will try to get something done around Suricata, IPFire 3.x and so on over the weekend.
Thank you for the patience, and best regards, Peter Müller
Hello guys,
Upon request, I would like to announce that on Saturday, 12th Jan, we are going to branch Core Update 127.
This means that we no longer accept any patches that introduce any new features and only accept patches that fix any bugs introduced. Nothing new here, you all know how this works :)
This announcement is just coming now so that you all know this. It is a bit short notice this time, but I hope I will be better with this next time round. In the past some patches have been submitted after the merge window closed and had to be deferred to the next Core Update to the frustration of the submitters. Therefore, we now try this and hope that we can make the process better for everyone who is contributing!
We have an update with many new features here. Please help testing as soon as the first build is available.
Best, -Michael
On 10.01.2019 15:22, Michael Tremer wrote:
Hello guys,
Hi,
Upon request, I would like to announce that on Saturday, 12th Jan, we are going to branch Core Update 127.
This means that we no longer accept any patches that introduce any new features and only accept patches that fix any bugs introduced. Nothing new here, you all know how this works :)
This announcement is just coming now so that you all know this. It is a bit short notice this time, but I hope I will be better with this next time round. In the past some patches have been submitted after the merge window closed and had to be deferred to the next Core Update to the frustration of the submitters. Therefore, we now try this and hope that we can make the process better for everyone who is contributing!
We have an update with many new features here. Please help testing as soon as the first build is available. ...
Thanks for the reminder, but regarding this Core update, one question is bothering me:
Did anyone had the chance to test 'squid 4.5' - especially along with this commit (https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=eedca6e36c1131ce5542da5c... => "squid: Run as many redirectors as we have CPU cores") - in a productive environment?
During my first tests, I got occasional warnings like "idnsSendQuery FD 8: sendto: (1) Operation not permitted" or "WARNING: All 2/2 redirector processes are busy" in squid 'cache.log' and I'd like to clarify possible mistakes in advance.
By now, I'm running the untuned parameters from current 'next' and these errors haven't appeared for some time now. Furthermore, 'testing tree' exists, ( ;-) ) but did anyone had a chance to try this?
Best, Matthias
Hi,
On 13 Jan 2019, at 10:04, Matthias Fischer matthias.fischer@ipfire.org wrote:
On 10.01.2019 15:22, Michael Tremer wrote:
Hello guys,
Hi,
Upon request, I would like to announce that on Saturday, 12th Jan, we are going to branch Core Update 127.
This means that we no longer accept any patches that introduce any new features and only accept patches that fix any bugs introduced. Nothing new here, you all know how this works :)
This announcement is just coming now so that you all know this. It is a bit short notice this time, but I hope I will be better with this next time round. In the past some patches have been submitted after the merge window closed and had to be deferred to the next Core Update to the frustration of the submitters. Therefore, we now try this and hope that we can make the process better for everyone who is contributing!
We have an update with many new features here. Please help testing as soon as the first build is available. ...
Thanks for the reminder, but regarding this Core update, one question is bothering me:
Did anyone had the chance to test 'squid 4.5' - especially along with this commit (https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=eedca6e36c1131ce5542da5c... => "squid: Run as many redirectors as we have CPU cores") - in a productive environment?
During my first tests, I got occasional warnings like "idnsSendQuery FD 8: sendto: (1) Operation not permitted" or "WARNING: All 2/2 redirector processes are busy" in squid 'cache.log' and I'd like to clarify possible mistakes in advance.
By now, I'm running the untuned parameters from current 'next' and these errors haven't appeared for some time now. Furthermore, 'testing tree' exists, ( ;-) ) but did anyone had a chance to try this?
I sent squid thousands of requests with ab (Apache Benchmark) and that worked fine with only one redirector.
Best, -Michael
Best, Matthias
Gesendet: Sonntag, 13. Januar 2019 um 13:43 Uhr Von: "Matthias Fischer" matthias.fischer@ipfire.org An: "Michael Tremer" michael.tremer@ipfire.org Cc: "IPFire: Development-List" development@lists.ipfire.org Betreff: Re: Merge Window Closing Announcement for Core Update 127
Hi,
On 13.01.2019 13:14, Michael Tremer wrote:
I sent squid thousands of requests with ab (Apache Benchmark) and that worked fine with only one redirector.
Thanks for the feedback! Merged. ;-)
Nevertheless it's worth reasoning about multiprocessing ( #redirectors = #processors ) versus multitasking ( #redirectors = #task needed by the environment ). Even with only one processor it is possible to serve multiple requests. Using wait phases for other processes is a basic feature of multitasking.
- Bernhard ( aka BeBiMa )
Hello,
On 13 Jan 2019, at 14:24, Bernhard Bitsch Bernhard.Bitsch@gmx.de wrote:
Gesendet: Sonntag, 13. Januar 2019 um 13:43 Uhr Von: "Matthias Fischer" matthias.fischer@ipfire.org An: "Michael Tremer" michael.tremer@ipfire.org Cc: "IPFire: Development-List" development@lists.ipfire.org Betreff: Re: Merge Window Closing Announcement for Core Update 127
Hi,
On 13.01.2019 13:14, Michael Tremer wrote:
I sent squid thousands of requests with ab (Apache Benchmark) and that worked fine with only one redirector.
Thanks for the feedback! Merged. ;-)
Nevertheless it's worth reasoning about multiprocessing ( #redirectors = #processors ) versus multitasking ( #redirectors = #task needed by the environment ). Even with only one processor it is possible to serve multiple requests. Using wait phases for other processes is a basic feature of multitasking.
This is not true for the redirectors though.
Our version of squidGuard takes only one request at a time. While processing that, it performs a database lookup (which is a btree) which is held in memory. So at no time it is reading anything from disk which would allow the kernel to interrupt the process and wait until the hard drive has responded.
Therefore it does not make any sense to run more than one process per CPU core.
I guess update accelerator might suffer a little bit in the instance that it finds a match. Squidclamav is a different thing. It makes absolutely no sense to use that any more.
-Michael
- Bernhard ( aka BeBiMa )
Hi,
Gesendet: Montag, 14. Januar 2019 um 11:43 Uhr Von: "Michael Tremer" michael.tremer@ipfire.org An: "Bernhard Bitsch" Bernhard.Bitsch@gmx.de Cc: "IPFire: Development-List" development@lists.ipfire.org Betreff: Re: Merge Window Closing Announcement for Core Update 127
Hello,
On 13 Jan 2019, at 14:24, Bernhard Bitsch Bernhard.Bitsch@gmx.de wrote:
Gesendet: Sonntag, 13. Januar 2019 um 13:43 Uhr Von: "Matthias Fischer" matthias.fischer@ipfire.org An: "Michael Tremer" michael.tremer@ipfire.org Cc: "IPFire: Development-List" development@lists.ipfire.org Betreff: Re: Merge Window Closing Announcement for Core Update 127
Hi,
On 13.01.2019 13:14, Michael Tremer wrote:
I sent squid thousands of requests with ab (Apache Benchmark) and that worked fine with only one redirector.
Thanks for the feedback! Merged. ;-)
Nevertheless it's worth reasoning about multiprocessing ( #redirectors = #processors ) versus multitasking ( #redirectors = #task needed by the environment ). Even with only one processor it is possible to serve multiple requests. Using wait phases for other processes is a basic feature of multitasking.
This is not true for the redirectors though.
Our version of squidGuard takes only one request at a time. While processing that, it performs a database lookup (which is a btree) which is held in memory. So at no time it is reading anything from disk which would allow the kernel to interrupt the process and wait until the hard drive has responded.
Therefore it does not make any sense to run more than one process per CPU core.
But multitasking isn't accomplished by I/O-waits only. Task switches occur through time slicing and signals also.
I guess update accelerator might suffer a little bit in the instance that it finds a match. Squidclamav is a different thing. It makes absolutely no sense to use that any more.
-Michael
- Bernhard ( aka BeBiMa )
- Bernhard
On 14 Jan 2019, at 13:32, Bernhard Bitsch Bernhard.Bitsch@gmx.de wrote:
Hi,
Gesendet: Montag, 14. Januar 2019 um 11:43 Uhr Von: "Michael Tremer" michael.tremer@ipfire.org An: "Bernhard Bitsch" Bernhard.Bitsch@gmx.de Cc: "IPFire: Development-List" development@lists.ipfire.org Betreff: Re: Merge Window Closing Announcement for Core Update 127
Hello,
On 13 Jan 2019, at 14:24, Bernhard Bitsch Bernhard.Bitsch@gmx.de wrote:
Gesendet: Sonntag, 13. Januar 2019 um 13:43 Uhr Von: "Matthias Fischer" matthias.fischer@ipfire.org An: "Michael Tremer" michael.tremer@ipfire.org Cc: "IPFire: Development-List" development@lists.ipfire.org Betreff: Re: Merge Window Closing Announcement for Core Update 127
Hi,
On 13.01.2019 13:14, Michael Tremer wrote:
I sent squid thousands of requests with ab (Apache Benchmark) and that worked fine with only one redirector.
Thanks for the feedback! Merged. ;-)
Nevertheless it's worth reasoning about multiprocessing ( #redirectors = #processors ) versus multitasking ( #redirectors = #task needed by the environment ). Even with only one processor it is possible to serve multiple requests. Using wait phases for other processes is a basic feature of multitasking.
This is not true for the redirectors though.
Our version of squidGuard takes only one request at a time. While processing that, it performs a database lookup (which is a btree) which is held in memory. So at no time it is reading anything from disk which would allow the kernel to interrupt the process and wait until the hard drive has responded.
Therefore it does not make any sense to run more than one process per CPU core.
But multitasking isn't accomplished by I/O-waits only. Task switches occur through time slicing and signals also.
Yes, but that is not a case squidguard is likely to run into.
Processing a request is fast and won’t take minutes or even seconds.
If the process is being paused and something else is running then there is no point in having multiple processes waiting. One process waiting is enough.
I guess update accelerator might suffer a little bit in the instance that it finds a match. Squidclamav is a different thing. It makes absolutely no sense to use that any more.
-Michael
- Bernhard ( aka BeBiMa )
- Bernhard
-
Bernhard Bitsch -
Matthias Fischer -
Michael Tremer -
Peter Müller