Hello,
On 19 Aug 2021, at 17:32, Matthias Fischer matthias.fischer@ipfire.org wrote:
Hi,
please comments below...
On 19.08.2021 15:57, Michael Tremer wrote:
Hello,
On 18 Aug 2021, at 17:42, Matthias Fischer matthias.fischer@ipfire.org wrote:
Hi,
On 16.08.2021 11:40, Michael Tremer wrote: ...
What makes me wonder: during build, 'squid' says it can open '32768', during start its '4096'. If someone knows why, please enlighten me... ;-)
4096 is the default maximum number of files any process can open at the same.
This is to protect the system from going crazy by having too many open files (because I think the file descriptor table used to be of a static size in older versions of the kernel).
I suppose this is enough and I can live with 32k. We should remove the field from the UI then.
Me too, but are 4096 enough?
No. I don’t know why the squid team isn’t handling this better. We are hitting this problem every time we update to a new version.
I suppose this is fine for testing.
You can try adding “ulimit -n 32768” to the squid init script and then it should be able to open up to 32k files.
...
Thanks for the clarification - I tested this with 'squid 5.1'. It seems to work:
... case "1$" in start) ulimit -n 32768 getpids "squid" ...
What is “getpids” good for?
Its a function call from '/etc/init.d/functions', but please don't ask me what its good for: ;-)
***SNIP*** # This will ensure compatibility with previous LFS Bootscripts getpids() { if [ -z "${PIDFILE}" ]; then pidofproc -s -p "${PIDFILE}" $@ else pidofproc -s $@ fi base="${1##*/}" } ***SNAP***
It simply seems to print all PIDs of the squid processes.
Adding the ulimit call to the initscript and removing the configuration option from the CGI script is fine with me.
I'm testing. Works.
***SNIP*** 2021/08/19 18:14:58 kid1| Logfile: closing log stdio:/var/log/squid/access.log 2021/08/19 18:14:58 kid1| Open FD READ/WRITE 8 redirect_wrapper #1 2021/08/19 18:14:58 kid1| Open FD READ/WRITE 10 redirect_wrapper #2 2021/08/19 18:14:58 kid1| Squid Cache (Version 5.1): Exiting normally. 2021/08/19 18:14:58| Removing PID file (/var/run/squid.pid) 2021/08/19 18:15:08 kid1| Current Directory is /srv/web/ipfire/cgi-bin 2021/08/19 18:15:08 kid1| Creating missing swap directories 2021/08/19 18:15:08 kid1| No cache_dir stores are configured. 2021/08/19 18:15:08| Removing PID file (/var/run/squid.pid) 2021/08/19 18:15:09 kid1| Current Directory is /srv/web/ipfire/cgi-bin 2021/08/19 18:15:09 kid1| Starting Squid Cache version 5.1 for x86_64-pc-linux-gnu... 2021/08/19 18:15:09 kid1| Service Name: squid 2021/08/19 18:15:09 kid1| Process ID 27102 2021/08/19 18:15:09 kid1| Process Roles: worker 2021/08/19 18:15:09 kid1| With 32768 file descriptors available 2021/08/19 18:15:09 kid1| Initializing IP Cache... 2021/08/19 18:15:09 kid1| DNS Socket created at 0.0.0.0, FD 7 2021/08/19 18:15:09 kid1| Adding domain localdomain from /etc/resolv.conf 2021/08/19 18:15:09 kid1| Adding nameserver 127.0.0.1 from /etc/resolv.conf 2021/08/19 18:15:09 kid1| helperOpenServers: Starting 2/2 'redirect_wrapper' processes 2021/08/19 18:15:09 kid1| Logfile: opening log stdio:/var/log/squid/access.log 2021/08/19 18:15:09 kid1| Store logging disabled 2021/08/19 18:15:09 kid1| Swap maxSize 0 + 262144 KB, estimated 20164 objects 2021/08/19 18:15:09 kid1| Target number of buckets: 1008 2021/08/19 18:15:09 kid1| Using 8192 Store buckets 2021/08/19 18:15:09 kid1| Max Mem size: 262144 KB 2021/08/19 18:15:09 kid1| Max Swap size: 0 KB 2021/08/19 18:15:09 kid1| Using Least Load store dir selection 2021/08/19 18:15:09 kid1| Current Directory is /srv/web/ipfire/cgi-bin 2021/08/19 18:15:09 kid1| Finished loading MIME types and icons. 2021/08/19 18:15:09 kid1| HTCP Disabled. 2021/08/19 18:15:09 kid1| Squid plugin modules loaded: 0 2021/08/19 18:15:09 kid1| Adaptation support is off. 2021/08/19 18:15:09 kid1| Accepting HTTP Socket connections at conn6 local=192.168.100.254:8080 remote=[::] FD 13 flags=9 2021/08/19 18:15:09 kid1| Accepting HTTP Socket connections at conn8 local=192.168.101.254:8080 remote=[::] FD 14 flags=9 2021/08/19 18:15:09 kid1| Accepting HTTP Socket connections at conn10 local=127.0.0.1:8080 remote=[::] FD 15 flags=9 2021/08/19 18:15:10 kid1| storeLateRelease: released 0 objects ***SNAP***
For my 'cache_peer' problem I opened a bug report (https://bugs.squid-cache.org/show_bug.cgi?id=5147). Work in progress.
I didn’t get it, but I am sure you know what you are doing :)
Despite being "only" a "non-caching web proxy with advanced filtering capabilities for enhancing privacy", 'privoxy' still threw away up to 15-20% of unnecessary or unwanted ads or internet junk. I wanted to keep it.
Okay. Does that still work with all this HTTPS traffic?
It is good to work together with upstream.
Yep. And the 'squids' are friendly... ;-)
As I read the last answers (really fast reaction!), these messages about "failed" TCP connections are triggered by the rejected CONNECTs from 'privoxy'. "Squid v4 did not consider rejected CONNECTs a problem worth marking the peer DEAD for." They fixed this bug and now 'squid 5.1' does... And "the bad header field is sent by privoxy".
For now, I've disabled 'privoxy' to see how things are going without it.
Best, Matthias