From mboxrd@z Thu Jan 1 00:00:00 1970 From: Peter =?utf-8?q?M=C3=BCller?= <peter.mueller@link38.eu> To: development@lists.ipfire.org Subject: Re: [PATCH] Allow kernel to swap memory on high demand Date: Thu, 04 Oct 2018 17:46:44 +0200 Message-ID: <8b5ff4d1-382d-72b1-04d1-4c9196d3d957@link38.eu> In-Reply-To: <45ff156d8510d6e5fd1acacdd42e206be552c43d.camel@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5381184794528736498==" List-Id: <development.lists.ipfire.org> --===============5381184794528736498== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello Michael, me too. Thanks for the clarification. Best regards, Peter M=C3=BCller > Hello, >=20 > do we have objections to this still? I am fine with it. >=20 > Best, > -Michael >=20 > Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org> >=20 > On Wed, 2018-09-26 at 20:55 +0100, Tim FitzGeorge wrote: >> On 25/09/2018 21:50, Michael Tremer wrote: >>> Hello Tim, >>> >>> welcome to the list and thanks for the patch. >>> >>> On Tue, 2018-09-25 at 20:29 +0200, Peter M=C3=BCller wrote: >>>> Hello Tim, >>>> >>>> I am not sure about the side-effects introduced with this patch. >>>> >>>> As far as I am aware, values for "vm.swappiness" range from 0 >>>> (disable swapping entirely) to 100 (swap aggressively). Value 1 >>>> means swapping is only used to avoid OOM situations. >>> >>> No, 0 does not disable swapping entirely. >>> >>> https://git.ipfire.org/?p=3Dthirdparty/kernel/linux.git;a=3Dblob;f=3DDo= cumentation/sysctl/vm.txt;hb=3D02214bfc89c71bcc5167f653994cfa5c57f10ff1#l808 >> >> Yes, with swappiness=3D0 the system will start swapping when the amount of= free memory drops below a calculated value, but it will not handle a demand = for more than the available amount of free memory. In this case it calls the= OOM killer. >> >> >> With swappiness=3D1 it will start swapping memory out to make room rather = than OOM. >> >> >> With swappiness>=3D2 the system will start pre-emptively swapping out memo= ry pages that it doesn't think are needed any more. In principle this makes = for a faster system because it doesn't have to stop to swap out memory when i= t needs to find some, but obviously there's a problem if it's swapped out som= ething that's actually needed. >> >> It's worth noting that while the kernel doesn't load pages of an executabl= e until they're actually used, it has to load any initialisation code. This = initialisation code is left in memory when the program is running. There is = therefore an argument that _under nominal conditions_ a slightly larger value= of swappiness (for example 10) would give the best results, since it would a= llow initialisation and other code to be swapped out, while still keeping mem= ory that's being used. >> >> Of course, as Peter has said we also need to consider non-nominal conditio= ns. >> >>>> This raises three questions: >>>> (a) On some systems, I observer swap usage > 0% indeed, which >>>> should not happen if value 0 for this setting _disables_ swapping. >>> >>> See above. >>> >>>> (b) Since disk I/O is much slower than RAM access, I fear it >>>> might be a DoS vector to enable this (infected program running >>>> amok). On the other hand, if a systems might avoid running out >>>> of memory, this sounds good too. >>> >>> Not swapping would cause the OOM killer to kill random processes. That al= so is a >>> DoS attack vector. Every OOM situation is. There is no way to recover fro= m this. >> >> I think it's really a trade-off. With swappiness=3D0 a random process is >> killed, which could well reduce the amount of protection that IPFire is >> giving you. With swappiness=3D1 there's a longer pause while swapping is >> carried out (the OOM killer also takes time), but the amount of >> protection stays the same. Note that with swappiness=3D0 we could also >> set oom_kill_allocating_task to kill the task that triggers the >> event;this would be quicker but still potentially leads to a loss of >> protection. >> >> Which is preferable probably depends on what you're protecting. If you're= protecting the IPFire source, the nightmare scenario is someone installing a= backdoor in the code under the cover of a DOS attack - so you'd likely prefe= r swappiness=3D1. If you're protecting a home network that's mainly used for= gaming, you may well decide that increased risk with swappiness=3D0 is accep= table. >> >> Obviously, this only applies to a one off instance of swapping if your sys= tem is swapping continually, then you need more memory as Michael says below. >> >>>> (c) How does the kernel treat anonymous pages after changing >>>> this setting? >>> >>> Anonymous pages? >> >> The system can swap out pages of executable code - this just requires >> marking the page as unused since if it's needed again it can just be >> reloaded from the file on disk, or data. Data pages are known as >> anonymous whereas program pages can be considered to have the name of >> the executable. >> >> I don't think that setting swappiness=3D1 affects this. The kernel will t= ry to swap out pages that it thinks are least likely to be needed; if these a= re anonymous pages they get written to the swap file, otherwise they're just = freed. >> >>>> However, these might be academic threats since the overall >>>> issue is already discussed in=20 >>>> https://bugzilla.ipfire.org/show_bug.cgi?id=3D11839 . >>>> Just some comments from my side... :-) >>> >>> Not really. I guess what we really want is 1 here. There is usually no re= ason >>> for the firewall to swap. That only happens for the proxy or IDS (or an >>> application with a massive memory leak). Usually that can be configured a= way or >>> the amount of RAM in the machine has to be upgraded. >>> >>> As long as there is enough memory available, we want to keep everything in >>> memory. There is no point in swapping out the IDS ruleset or proxy cache = in >>> memory. We only will do this to keep the system alive if there is a peak = in >>> memory usage and that is what vm.swappiness=3D1 is doing from my POV. >>> >>> Best, >>> -Michael >> >> In my scenario I've got a script that downloads IDS rules, assesses and >> the applies the changes, runs snort -T to check that the updated rule >> files are OK and then tells the running instances of Snort to re-read >> the rules. Its peak memory usage (during snort -T) is around 500MB.=20 >> I'm happy for the system to swap when it's doing this, even if it does >> slow things down (mind you, the downloading of a almost 100MB set of >> rules will slow traffic over the red interface anyway). >> >> With swappiness=3D0 I would get the OOM killer triggered (it would usually= kill one of the snort instances). Since setting swappiness=3D1 this doesn't= happen any more, and my swap partition usage has stayed at 0. From my point = of view, everything is behaving correctly. >> >> It's a pity that the WUI doesn't include a graph of swap rate alongside th= e swap usage graph - it would be useful for this discussion. >> >> Best regards, >> >> Tim >> >>>> Thanks, and best regards, >>>> Peter M=C3=BCller >>>> >>>> >>>>> Signed-off-by: Tim FitzGeorge <ipfr at tfitzgeorge.me.uk> >>>>> Fixes: Bug 11839 >>>>> --- >>>>> config/etc/sysctl.conf | 2 +- >>>>> 1 file changed, 1 insertion(+), 1 deletion(-) >>>>> >>>>> diff --git a/config/etc/sysctl.conf b/config/etc/sysctl.conf >>>>> index 345f8f52a..4066af767 100644 >>>>> --- a/config/etc/sysctl.conf >>>>> +++ b/config/etc/sysctl.conf >>>>> @@ -27,7 +27,7 @@ net.ipv4.conf.all.accept_source_route =3D 0 >>>>> net.ipv4.conf.all.log_martians =3D 1 >>>>> =20 >>>>> kernel.printk =3D 1 4 1 7 >>>>> -vm.swappiness=3D0 >>>>> +vm.swappiness=3D1 >>>>> vm.mmap_min_addr =3D 4096 >>>>> vm.min_free_kbytes =3D 8192 >>>>> =20 >>>>> >> >> >=20 --=20 Microsoft DNS service terminates abnormally when it recieves a response to a DNS query that was never made. Fix Information: Run your DNS service on a different platform. -- bugtraq --===============5381184794528736498==--