Re: Question regarding IPsec N2N throughput with and without IPS

["Peter Müller" <peter.mueller@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 5038 bytes --]

Hello Michael,

> Hi,
> 
>> On 22 Jan 2020, at 16:24, Peter Müller <peter.mueller(a)ipfire.org> wrote:
>>
>> Hello Stefan, hello list (CC'ed),
>>
>> are you aware of any IPS bottlenecks regarding IPsec N2N throughput?
> 
> No, not at all.
> 
> There is actually not much the IPS can do with the ESP packets. I would assume it just passes them through.
I am not sure about this as the decapsulated clear text packages show up on
ppp0/red0, too - perhaps these pass Suricata twice?
> 
>> I finally (!) managed to get an IPsec connection between OpenBSD 6.6 (OpenIKED)
>> and IPFire 2.23 Core Update 139 working.
> 
> Yay \o/.
Well, actually not. PSK works, but certificate based authentication does not
due to bug #12276 (OpenSSL does not use subjectAltNames from CSRs and is unable
to be safely forced to do so - you will have to supply an additional configuration
file to make CSR signing with subjectAltNames work which is a _very_ ugly thing
to do if no CSR configuration is available on the signing machine).

Furthermore, Dead Peer Detection is tricky as OpenIKED does not support it
- the OpenBSD people use ifstated instead, but restarting single IPsec connections
is likely impossible at the moment -, so we can only rely on Strongswan here. :-/

Side note: OpenIKED does not support AES-GCM for ESP (see configuration below).
> 
> Please do not forget to add the relevant documentation to the IPFire wiki.
I will do so as soon this thing works stable and all corresponding IPFire bugs
have been fixed. :-)
> 
>> During throughput tests, where
>> I downloaded a 1 GByte test file from the machine via the IPsec tunnel,
>> a rather large throughput difference with and without IPS enabled on RED
>> has come to my attention:
>>
>> With IPS enabled on RED, the download starts at ~ 2.5 MByte/sec. and
>> continually decreases to ~ 580 kByte/sec. - 800 kByte/sec., which is
>> even lower than OpenVPN performance. Without IPS enabled on RED, throughput
>> is 4.0 MByte/sek. on average - running the IPS on other interfaces does
>> not change this behaviour, neither does enabling monitoring mode.
> 
> How is CPU load?
>From the collectd graphs I can recall, load average (1 minute) was about 1.1 .
> 
> Did you see any retransmissions?
I am currently unable to test this with IPsec but transferring the same test file
via SCP (which results approximately in the same throughput) shows some duplicate
ACKs, but much less than those we have seen on the similar bug investigated last
year.
> 
>> This suspiciously sounds like the issue we have had with Suricata last
>> year - as far as I am concerned that was fully fixed. Are you aware of
>> any other similar issue that could cause this massive throughput loss?
> 
> No. You can try Suricata 5 which has been posted to the list today.
I would like to do so - is there a nightly/pre-testing ISO available?
> 
>> Anyway, thank you in advance for any help and hints. :-)
>>
>> Kernel and CPU information of the OpenBSD machine:
>>> openbsd# uname -a
>>> OpenBSD openbsd 6.6 GENERIC.MP#4 amd64
>>> openbsd# sysctl hw.model hw.machine hw.ncpu
>>> hw.model=Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
>>> hw.machine=amd64
>>> hw.ncpu=2
>>
>> Content of /etc/iked.conf on the OpenBSD machine:
>>> set fragmentation
>>>
>>> ikev2 "[REDACTED]" active esp \
>>>   from 10.xxx.xxx.2/24 to 10.xxx.xxx.0/24 \
>>>   local [REDACTED] peer [REDACTED] \
>>>   ikesa auth hmac-sha2-512 enc aes-256 prf hmac-sha2-512 group curve25519 \
>>>   childsa enc aes-256-gcm group curve25519 \
>>>   srcid [REDACTED] dstid [REDACTED] \
>>>   ikelifetime 3h \
>>>   lifetime 1h
>>
>> Kernel and CPU information of the IPFire machine:
>>> [root(a)maverick ~]# uname -a
>>> Linux maverick 4.14.154-ipfire #1 SMP Fri Nov 15 07:27:41 GMT 2019 x86_64 Intel(R) Celeron(R) CPU N3150 @ 1.60GHz GenuineIntel GNU/Linux
> 
> This is a really small Atom processor AFAIK. Could we struggle with Meltdown/Spectre mitigations here? Just to rule it out, can you boot the kernel with them disabled?
Hm, I do not think we need to worry about them too much as the N3150 is not vulnerable
to some CPU vulnerabilities:
> [root(a)maverick ~]# grep . /sys/devices/system/cpu/vulnerabilities/*
> /sys/devices/system/cpu/vulnerabilities/itlb_multihit:Not affected
> /sys/devices/system/cpu/vulnerabilities/l1tf:Not affected
> /sys/devices/system/cpu/vulnerabilities/mds:Mitigation: Clear CPU buffers; SMT disabled
> /sys/devices/system/cpu/vulnerabilities/meltdown:Mitigation: PTI
> /sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Not affected
> /sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: usercopy/swapgs barriers and __user pointer sanitization
> /sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full generic retpoline, IBPB: conditional, IBRS_FW, STIBP: disabled, RSB filling
> /sys/devices/system/cpu/vulnerabilities/tsx_async_abort:Not affected

Thanks, and best regards,
Peter Müller