From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer To: development@lists.ipfire.org Subject: Re: squid 5.1 gone stable Date: Mon, 23 Aug 2021 14:29:33 +0100 Message-ID: <7A008831-7AAD-4A48-ABB9-3CC17F5C1CED@ipfire.org> In-Reply-To: <3a5ed28e-cec7-3d52-ed0b-6c7e00b66fc7@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4042217537113700240==" List-Id: --===============4042217537113700240== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello, > On 19 Aug 2021, at 17:32, Matthias Fischer = wrote: >=20 > Hi, >=20 > please comments below... >=20 > On 19.08.2021 15:57, Michael Tremer wrote: >> Hello, >>=20 >>> On 18 Aug 2021, at 17:42, Matthias Fischer wrote: >>>=20 >>> Hi, >>>=20 >>> On 16.08.2021 11:40, Michael Tremer wrote: >>> ... >>>>> What makes me wonder: during build, 'squid' says it can open '32768', >>>>> during start its '4096'. If someone knows why, please enlighten me... ;= -) >>>> 4096 is the default maximum number of files any process can open at the = same. >>>>=20 >>>> This is to protect the system from going crazy by having too many open f= iles (because I think the file descriptor table used to be of a static size i= n older versions of the kernel). >>>>=20 >>>>>> I suppose this is enough and I can live with 32k. We should remove the= field from the UI then. >>>>> Me too, but are 4096 enough? >>>> No. I don=E2=80=99t know why the squid team isn=E2=80=99t handling this = better. We are hitting this problem every time we update to a new version. >>>>=20 >>>> I suppose this is fine for testing. >>>>=20 >>>> You can try adding =E2=80=9Culimit -n 32768=E2=80=9D to the squid init s= cript and then it should be able to open up to 32k files. >>> ... >>>=20 >>> Thanks for the clarification - I tested this with 'squid 5.1'. It seems >>> to work: >>>=20 >>> ... >>> case "1$" in >>> start) >>> ulimit -n 32768 >>> getpids "squid" >>> ... >>=20 >> What is =E2=80=9Cgetpids=E2=80=9D good for? >=20 > Its a function call from '/etc/init.d/functions', but please don't ask > me what its good for: ;-) >=20 > ***SNIP*** > # This will ensure compatibility with previous LFS Bootscripts > getpids() > { > if [ -z "${PIDFILE}" ]; then > pidofproc -s -p "${PIDFILE}" $@ > else > pidofproc -s $@ > fi > base=3D"${1##*/}" > } > ***SNAP*** It simply seems to print all PIDs of the squid processes. >> Adding the ulimit call to the initscript and removing the configuration op= tion from the CGI script is fine with me. >=20 > I'm testing. Works. >=20 > ***SNIP*** > 2021/08/19 18:14:58 kid1| Logfile: closing log > stdio:/var/log/squid/access.log > 2021/08/19 18:14:58 kid1| Open FD READ/WRITE 8 redirect_wrapper #1 > 2021/08/19 18:14:58 kid1| Open FD READ/WRITE 10 redirect_wrapper #2 > 2021/08/19 18:14:58 kid1| Squid Cache (Version 5.1): Exiting normally. > 2021/08/19 18:14:58| Removing PID file (/var/run/squid.pid) > 2021/08/19 18:15:08 kid1| Current Directory is /srv/web/ipfire/cgi-bin > 2021/08/19 18:15:08 kid1| Creating missing swap directories > 2021/08/19 18:15:08 kid1| No cache_dir stores are configured. > 2021/08/19 18:15:08| Removing PID file (/var/run/squid.pid) > 2021/08/19 18:15:09 kid1| Current Directory is /srv/web/ipfire/cgi-bin > 2021/08/19 18:15:09 kid1| Starting Squid Cache version 5.1 for > x86_64-pc-linux-gnu... > 2021/08/19 18:15:09 kid1| Service Name: squid > 2021/08/19 18:15:09 kid1| Process ID 27102 > 2021/08/19 18:15:09 kid1| Process Roles: worker > 2021/08/19 18:15:09 kid1| With 32768 file descriptors available > 2021/08/19 18:15:09 kid1| Initializing IP Cache... > 2021/08/19 18:15:09 kid1| DNS Socket created at 0.0.0.0, FD 7 > 2021/08/19 18:15:09 kid1| Adding domain localdomain from /etc/resolv.conf > 2021/08/19 18:15:09 kid1| Adding nameserver 127.0.0.1 from /etc/resolv.conf > 2021/08/19 18:15:09 kid1| helperOpenServers: Starting 2/2 > 'redirect_wrapper' processes > 2021/08/19 18:15:09 kid1| Logfile: opening log > stdio:/var/log/squid/access.log > 2021/08/19 18:15:09 kid1| Store logging disabled > 2021/08/19 18:15:09 kid1| Swap maxSize 0 + 262144 KB, estimated 20164 > objects > 2021/08/19 18:15:09 kid1| Target number of buckets: 1008 > 2021/08/19 18:15:09 kid1| Using 8192 Store buckets > 2021/08/19 18:15:09 kid1| Max Mem size: 262144 KB > 2021/08/19 18:15:09 kid1| Max Swap size: 0 KB > 2021/08/19 18:15:09 kid1| Using Least Load store dir selection > 2021/08/19 18:15:09 kid1| Current Directory is /srv/web/ipfire/cgi-bin > 2021/08/19 18:15:09 kid1| Finished loading MIME types and icons. > 2021/08/19 18:15:09 kid1| HTCP Disabled. > 2021/08/19 18:15:09 kid1| Squid plugin modules loaded: 0 > 2021/08/19 18:15:09 kid1| Adaptation support is off. > 2021/08/19 18:15:09 kid1| Accepting HTTP Socket connections at conn6 > local=3D192.168.100.254:8080 remote=3D[::] FD 13 flags=3D9 > 2021/08/19 18:15:09 kid1| Accepting HTTP Socket connections at conn8 > local=3D192.168.101.254:8080 remote=3D[::] FD 14 flags=3D9 > 2021/08/19 18:15:09 kid1| Accepting HTTP Socket connections at conn10 > local=3D127.0.0.1:8080 remote=3D[::] FD 15 flags=3D9 > 2021/08/19 18:15:10 kid1| storeLateRelease: released 0 objects > ***SNAP*** >=20 >>> For my 'cache_peer' problem I opened a bug report >>> (https://bugs.squid-cache.org/show_bug.cgi?id=3D5147). Work in progress. >>=20 >> I didn=E2=80=99t get it, but I am sure you know what you are doing :) >=20 > Despite being "only" a "non-caching web proxy with advanced filtering > capabilities for enhancing privacy", 'privoxy' still threw away up to > 15-20% of unnecessary or unwanted ads or internet junk. I wanted to keep it. Okay. Does that still work with all this HTTPS traffic? >> It is good to work together with upstream. >=20 > Yep. And the 'squids' are friendly... ;-) >=20 > As I read the last answers (really fast reaction!), these messages about > "failed" TCP connections are triggered by the rejected CONNECTs from > 'privoxy'. "Squid v4 did not consider rejected CONNECTs a problem worth > marking the peer DEAD for." They fixed this bug and now 'squid 5.1' > does... And "the bad header field is sent by privoxy". >=20 > For now, I've disabled 'privoxy' to see how things are going without it. >=20 > Best, > Matthias --===============4042217537113700240==--