Re: [PATCH] Allow kernel to swap memory on high demand

[Michael Tremer <michael.tremer@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 8205 bytes --]

Please don't forget the Git tags...

On Thu, 2018-10-04 at 17:46 +0200, Peter Müller wrote:
> Hello Michael,
> 
> me too. Thanks for the clarification.
> 
> Best regards,
> Peter Müller
> 
> > Hello,
> > 
> > do we have objections to this still? I am fine with it.
> > 
> > Best,
> > -Michael
> > 
> > Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org>
> > 
> > On Wed, 2018-09-26 at 20:55 +0100, Tim FitzGeorge wrote:
> > > On 25/09/2018 21:50, Michael Tremer wrote:
> > > > Hello Tim,
> > > > 
> > > > welcome to the list and thanks for the patch.
> > > > 
> > > > On Tue, 2018-09-25 at 20:29 +0200, Peter Müller wrote:
> > > > > Hello Tim,
> > > > > 
> > > > > I am not sure about the side-effects introduced with this patch.
> > > > > 
> > > > > As far as I am aware, values for "vm.swappiness" range from 0
> > > > > (disable swapping entirely) to 100 (swap aggressively). Value 1
> > > > > means swapping is only used to avoid OOM situations.
> > > > 
> > > > No, 0 does not disable swapping entirely.
> > > > 
> > > >   
> > > > https://git.ipfire.org/?p=thirdparty/kernel/linux.git;a=blob;f=Documentation/sysctl/vm.txt;hb=02214bfc89c71bcc5167f653994cfa5c57f10ff1#l808
> > > 
> > > Yes, with swappiness=0 the system will start swapping when the amount of
> > > free memory drops below a calculated value, but it will not handle a
> > > demand for more than the available amount of free memory.  In this case it
> > > calls the OOM killer.
> > > 
> > > 
> > > With swappiness=1 it will start swapping memory out to make room rather
> > > than OOM.
> > > 
> > > 
> > > With swappiness>=2 the system will start pre-emptively swapping out memory
> > > pages that it doesn't think are needed any more.  In principle this makes
> > > for a faster system because it doesn't have to stop to swap out memory
> > > when it needs to find some, but obviously there's a problem if it's
> > > swapped out something that's actually needed.
> > > 
> > > It's worth noting that while the kernel doesn't load pages of an
> > > executable until they're actually used, it has to load any initialisation
> > > code.  This initialisation code is left in memory when the program is
> > > running.  There is therefore an argument that _under nominal conditions_ a
> > > slightly larger value of swappiness (for example 10) would give the best
> > > results, since it would allow initialisation and other code to be swapped
> > > out, while still keeping memory that's being used.
> > > 
> > > Of course, as Peter has said we also need to consider non-nominal
> > > conditions.
> > > 
> > > > > This raises three questions:
> > > > > (a) On some systems, I observer swap usage > 0% indeed, which
> > > > > should not happen if value 0 for this setting _disables_ swapping.
> > > > 
> > > > See above.
> > > > 
> > > > > (b) Since disk I/O is much slower than RAM access, I fear it
> > > > > might be a DoS vector to enable this (infected program running
> > > > > amok). On the other hand, if a systems might avoid running out
> > > > > of memory, this sounds good too.
> > > > 
> > > > Not swapping would cause the OOM killer to kill random processes. That
> > > > also is a
> > > > DoS attack vector. Every OOM situation is. There is no way to recover
> > > > from this.
> > > 
> > > I think it's really a trade-off.  With swappiness=0 a random process is
> > > killed, which could well reduce the amount of protection that IPFire is
> > > giving you.  With swappiness=1 there's a longer pause while swapping is
> > > carried out (the OOM killer also takes time), but the amount of
> > > protection stays the same.  Note that with swappiness=0 we could also
> > > set oom_kill_allocating_task to kill the task that triggers the
> > > event;this would be quicker but still potentially leads to a loss of
> > > protection.
> > > 
> > > Which is preferable probably depends on what you're protecting.  If you're
> > > protecting the IPFire source, the nightmare scenario is someone installing
> > > a backdoor in the code under the cover of a DOS attack - so you'd likely
> > > prefer swappiness=1.  If you're protecting a home network that's mainly
> > > used for gaming, you may well decide that increased risk with swappiness=0
> > > is acceptable.
> > > 
> > > Obviously, this only applies to a one off instance of swapping if your
> > > system is swapping continually, then you need more memory as Michael says
> > > below.
> > > 
> > > > > (c) How does the kernel treat anonymous pages after changing
> > > > > this setting?
> > > > 
> > > > Anonymous pages?
> > > 
> > > The system can swap out pages of executable code - this just requires
> > > marking the page as unused since if it's needed again it can just be
> > > reloaded from the file on disk, or data.  Data pages are known as
> > > anonymous whereas program pages can be considered to have the name of
> > > the executable.
> > > 
> > > I don't think that setting swappiness=1 affects this.  The kernel will try
> > > to swap out pages that it thinks are least likely to be needed; if these
> > > are anonymous pages they get written to the swap file, otherwise they're
> > > just freed.
> > > 
> > > > > However, these might be academic threats since the overall
> > > > > issue is already discussed in 
> > > > > https://bugzilla.ipfire.org/show_bug.cgi?id=11839 .
> > > > > Just some comments from my side... :-)
> > > > 
> > > > Not really. I guess what we really want is 1 here. There is usually no
> > > > reason
> > > > for the firewall to swap. That only happens for the proxy or IDS (or an
> > > > application with a massive memory leak). Usually that can be configured
> > > > away or
> > > > the amount of RAM in the machine has to be upgraded.
> > > > 
> > > > As long as there is enough memory available, we want to keep everything
> > > > in
> > > > memory. There is no point in swapping out the IDS ruleset or proxy cache
> > > > in
> > > > memory. We only will do this to keep the system alive if there is a peak
> > > > in
> > > > memory usage and that is what vm.swappiness=1 is doing from my POV.
> > > > 
> > > > Best,
> > > > -Michael
> > > 
> > > In my scenario I've got a script that downloads IDS rules, assesses and
> > > the applies the changes, runs snort -T to check that the updated rule
> > > files are OK and then tells the running instances of Snort to re-read
> > > the rules.  Its peak memory usage (during snort -T) is around 500MB. 
> > > I'm happy for the system to swap when it's doing this, even if it does
> > > slow things down (mind you, the downloading of a almost 100MB set of
> > > rules will slow traffic over the red interface anyway).
> > > 
> > > With swappiness=0 I would get the OOM killer triggered (it would usually
> > > kill one of the snort instances).  Since setting swappiness=1 this doesn't
> > > happen any more, and my swap partition usage has stayed at 0. From my
> > > point of view, everything is behaving correctly.
> > > 
> > > It's a pity that the WUI doesn't include a graph of swap rate alongside
> > > the swap usage graph -  it would be useful for this discussion.
> > > 
> > > Best regards,
> > > 
> > > Tim
> > > 
> > > > > Thanks, and best regards,
> > > > > Peter Müller
> > > > > 
> > > > > 
> > > > > > Signed-off-by: Tim FitzGeorge <ipfr at tfitzgeorge.me.uk>
> > > > > > Fixes: Bug 11839
> > > > > > ---
> > > > > >  config/etc/sysctl.conf | 2 +-
> > > > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > > 
> > > > > > diff --git a/config/etc/sysctl.conf b/config/etc/sysctl.conf
> > > > > > index 345f8f52a..4066af767 100644
> > > > > > --- a/config/etc/sysctl.conf
> > > > > > +++ b/config/etc/sysctl.conf
> > > > > > @@ -27,7 +27,7 @@ net.ipv4.conf.all.accept_source_route = 0
> > > > > >  net.ipv4.conf.all.log_martians = 1
> > > > > >  
> > > > > >  kernel.printk = 1 4 1 7
> > > > > > -vm.swappiness=0
> > > > > > +vm.swappiness=1
> > > > > >  vm.mmap_min_addr = 4096
> > > > > >  vm.min_free_kbytes = 8192
> > > > > >  
> > > > > > 
> > > 
> > > 
> 
>