Hello Michael,
Hi,
On 22 Jan 2020, at 16:24, Peter Müller peter.mueller@ipfire.org wrote:
Hello Stefan, hello list (CC'ed),
are you aware of any IPS bottlenecks regarding IPsec N2N throughput?
No, not at all.
There is actually not much the IPS can do with the ESP packets. I would assume it just passes them through.
I am not sure about this as the decapsulated clear text packages show up on ppp0/red0, too - perhaps these pass Suricata twice?
I finally (!) managed to get an IPsec connection between OpenBSD 6.6 (OpenIKED) and IPFire 2.23 Core Update 139 working.
Yay \o/.
Well, actually not. PSK works, but certificate based authentication does not due to bug #12276 (OpenSSL does not use subjectAltNames from CSRs and is unable to be safely forced to do so - you will have to supply an additional configuration file to make CSR signing with subjectAltNames work which is a _very_ ugly thing to do if no CSR configuration is available on the signing machine).
Furthermore, Dead Peer Detection is tricky as OpenIKED does not support it - the OpenBSD people use ifstated instead, but restarting single IPsec connections is likely impossible at the moment -, so we can only rely on Strongswan here. :-/
Side note: OpenIKED does not support AES-GCM for ESP (see configuration below).
Please do not forget to add the relevant documentation to the IPFire wiki.
I will do so as soon this thing works stable and all corresponding IPFire bugs have been fixed. :-)
During throughput tests, where I downloaded a 1 GByte test file from the machine via the IPsec tunnel, a rather large throughput difference with and without IPS enabled on RED has come to my attention:
With IPS enabled on RED, the download starts at ~ 2.5 MByte/sec. and continually decreases to ~ 580 kByte/sec. - 800 kByte/sec., which is even lower than OpenVPN performance. Without IPS enabled on RED, throughput is 4.0 MByte/sek. on average - running the IPS on other interfaces does not change this behaviour, neither does enabling monitoring mode.
How is CPU load? From the collectd graphs I can recall, load average (1 minute) was about 1.1 .
Did you see any retransmissions?
I am currently unable to test this with IPsec but transferring the same test file via SCP (which results approximately in the same throughput) shows some duplicate ACKs, but much less than those we have seen on the similar bug investigated last year.
This suspiciously sounds like the issue we have had with Suricata last year - as far as I am concerned that was fully fixed. Are you aware of any other similar issue that could cause this massive throughput loss?
No. You can try Suricata 5 which has been posted to the list today.
I would like to do so - is there a nightly/pre-testing ISO available?
Anyway, thank you in advance for any help and hints. :-)
Kernel and CPU information of the OpenBSD machine:
openbsd# uname -a OpenBSD openbsd 6.6 GENERIC.MP#4 amd64 openbsd# sysctl hw.model hw.machine hw.ncpu hw.model=Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz hw.machine=amd64 hw.ncpu=2
Content of /etc/iked.conf on the OpenBSD machine:
set fragmentation
ikev2 "[REDACTED]" active esp \ from 10.xxx.xxx.2/24 to 10.xxx.xxx.0/24 \ local [REDACTED] peer [REDACTED] \ ikesa auth hmac-sha2-512 enc aes-256 prf hmac-sha2-512 group curve25519 \ childsa enc aes-256-gcm group curve25519 \ srcid [REDACTED] dstid [REDACTED] \ ikelifetime 3h \ lifetime 1h
Kernel and CPU information of the IPFire machine:
[root@maverick ~]# uname -a Linux maverick 4.14.154-ipfire #1 SMP Fri Nov 15 07:27:41 GMT 2019 x86_64 Intel(R) Celeron(R) CPU N3150 @ 1.60GHz GenuineIntel GNU/Linux
This is a really small Atom processor AFAIK. Could we struggle with Meltdown/Spectre mitigations here? Just to rule it out, can you boot the kernel with them disabled?
Hm, I do not think we need to worry about them too much as the N3150 is not vulnerable to some CPU vulnerabilities:
[root@maverick ~]# grep . /sys/devices/system/cpu/vulnerabilities/* /sys/devices/system/cpu/vulnerabilities/itlb_multihit:Not affected /sys/devices/system/cpu/vulnerabilities/l1tf:Not affected /sys/devices/system/cpu/vulnerabilities/mds:Mitigation: Clear CPU buffers; SMT disabled /sys/devices/system/cpu/vulnerabilities/meltdown:Mitigation: PTI /sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Not affected /sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: usercopy/swapgs barriers and __user pointer sanitization /sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full generic retpoline, IBPB: conditional, IBRS_FW, STIBP: disabled, RSB filling /sys/devices/system/cpu/vulnerabilities/tsx_async_abort:Not affected
Thanks, and best regards, Peter Müller