From mboxrd@z Thu Jan 1 00:00:00 1970 From: Peter =?utf-8?q?M=C3=BCller?= To: development@lists.ipfire.org Subject: Re: Question regarding IPsec N2N throughput with and without IPS Date: Fri, 24 Jan 2020 14:32:00 +0000 Message-ID: In-Reply-To: <81BA3FDB-766B-403F-B5B0-20E4F5FBF6A1@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5743676965810257222==" List-Id: --===============5743676965810257222== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello Michael, > Hi, >=20 >> On 22 Jan 2020, at 16:24, Peter M=C3=BCller w= rote: >> >> Hello Stefan, hello list (CC'ed), >> >> are you aware of any IPS bottlenecks regarding IPsec N2N throughput? >=20 > No, not at all. >=20 > There is actually not much the IPS can do with the ESP packets. I would ass= ume it just passes them through. I am not sure about this as the decapsulated clear text packages show up on ppp0/red0, too - perhaps these pass Suricata twice? >=20 >> I finally (!) managed to get an IPsec connection between OpenBSD 6.6 (Open= IKED) >> and IPFire 2.23 Core Update 139 working. >=20 > Yay \o/. Well, actually not. PSK works, but certificate based authentication does not due to bug #12276 (OpenSSL does not use subjectAltNames from CSRs and is unab= le to be safely forced to do so - you will have to supply an additional configur= ation file to make CSR signing with subjectAltNames work which is a _very_ ugly thi= ng to do if no CSR configuration is available on the signing machine). Furthermore, Dead Peer Detection is tricky as OpenIKED does not support it - the OpenBSD people use ifstated instead, but restarting single IPsec connec= tions is likely impossible at the moment -, so we can only rely on Strongswan here.= :-/ Side note: OpenIKED does not support AES-GCM for ESP (see configuration below= ). >=20 > Please do not forget to add the relevant documentation to the IPFire wiki. I will do so as soon this thing works stable and all corresponding IPFire bugs have been fixed. :-) >=20 >> During throughput tests, where >> I downloaded a 1 GByte test file from the machine via the IPsec tunnel, >> a rather large throughput difference with and without IPS enabled on RED >> has come to my attention: >> >> With IPS enabled on RED, the download starts at ~ 2.5 MByte/sec. and >> continually decreases to ~ 580 kByte/sec. - 800 kByte/sec., which is >> even lower than OpenVPN performance. Without IPS enabled on RED, throughput >> is 4.0 MByte/sek. on average - running the IPS on other interfaces does >> not change this behaviour, neither does enabling monitoring mode. >=20 > How is CPU load? >>From the collectd graphs I can recall, load average (1 minute) was about 1.1= . >=20 > Did you see any retransmissions? I am currently unable to test this with IPsec but transferring the same test = file via SCP (which results approximately in the same throughput) shows some dupli= cate ACKs, but much less than those we have seen on the similar bug investigated l= ast year. >=20 >> This suspiciously sounds like the issue we have had with Suricata last >> year - as far as I am concerned that was fully fixed. Are you aware of >> any other similar issue that could cause this massive throughput loss? >=20 > No. You can try Suricata 5 which has been posted to the list today. I would like to do so - is there a nightly/pre-testing ISO available? >=20 >> Anyway, thank you in advance for any help and hints. :-) >> >> Kernel and CPU information of the OpenBSD machine: >>> openbsd# uname -a >>> OpenBSD openbsd 6.6 GENERIC.MP#4 amd64 >>> openbsd# sysctl hw.model hw.machine hw.ncpu >>> hw.model=3DIntel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz >>> hw.machine=3Damd64 >>> hw.ncpu=3D2 >> >> Content of /etc/iked.conf on the OpenBSD machine: >>> set fragmentation >>> >>> ikev2 "[REDACTED]" active esp \ >>> from 10.xxx.xxx.2/24 to 10.xxx.xxx.0/24 \ >>> local [REDACTED] peer [REDACTED] \ >>> ikesa auth hmac-sha2-512 enc aes-256 prf hmac-sha2-512 group curve25519= \ >>> childsa enc aes-256-gcm group curve25519 \ >>> srcid [REDACTED] dstid [REDACTED] \ >>> ikelifetime 3h \ >>> lifetime 1h >> >> Kernel and CPU information of the IPFire machine: >>> [root(a)maverick ~]# uname -a >>> Linux maverick 4.14.154-ipfire #1 SMP Fri Nov 15 07:27:41 GMT 2019 x86_64= Intel(R) Celeron(R) CPU N3150 @ 1.60GHz GenuineIntel GNU/Linux >=20 > This is a really small Atom processor AFAIK. Could we struggle with Meltdow= n/Spectre mitigations here? Just to rule it out, can you boot the kernel with= them disabled? Hm, I do not think we need to worry about them too much as the N3150 is not v= ulnerable to some CPU vulnerabilities: > [root(a)maverick ~]# grep . /sys/devices/system/cpu/vulnerabilities/* > /sys/devices/system/cpu/vulnerabilities/itlb_multihit:Not affected > /sys/devices/system/cpu/vulnerabilities/l1tf:Not affected > /sys/devices/system/cpu/vulnerabilities/mds:Mitigation: Clear CPU buffers; = SMT disabled > /sys/devices/system/cpu/vulnerabilities/meltdown:Mitigation: PTI > /sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Not affected > /sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: usercopy/swa= pgs barriers and __user pointer sanitization > /sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full generic= retpoline, IBPB: conditional, IBRS_FW, STIBP: disabled, RSB filling > /sys/devices/system/cpu/vulnerabilities/tsx_async_abort:Not affected Thanks, and best regards, Peter M=C3=BCller --===============5743676965810257222==--