Hello Stefan, sorry for my late reply on this topic. Finally, I was able to test your new ISO, next-suricata-rust/d22217e1-dirty . New connections, regardless of SSH, TLS, DNS or whatever, are now established instantly. No packets are dropped anymore, this issue seems to be solved by the changes included in your ISO. OpenVPN RW performance, however, is worse than before: Now around ~ 400 kB/sec. if Suricata is enabled, and somewhere around 900 kB/sec. if it is not. I am getting the feeling this is an OpenVPN bug or performance issue, but as mentioned several times before, I am not satisfied with OpenVPN anyway. A few minutes after booting, kernel emits these log lines: > Sep 19 22:01:51 maverick kernel: refcount_t: increment on 0; use-after-free. > Sep 19 22:01:51 maverick kernel: ------------[ cut here ]------------ > Sep 19 22:01:51 maverick kernel: WARNING: CPU: 2 PID: 2510 at lib/refcount.c:153 refcount_inc.cold.12+0x13/0x16 > Sep 19 22:01:51 maverick kernel: Modules linked in: xt_IMQ imq xt_length xt_DSCP xt_layer7 cls_fw sch_htb chacha20_x86_64 chacha20_generic poly1305_x86_64 poly1305_generic chacha20poly1305 esp4 xfrm6_mode_tunnel xfrm4_mode_tunnel tun nfnetlink_queue xt_NFQUEUE ipt_MASQUERADE nf_nat_masquerade_ipv4 pppoe pppox ppp_generic slhc xt_geoip(O) xt_connlimit xt_multiport xt_hashlimit xt_mark xt_policy xt_TCPMSS nf_nat_irc nf_conntrack_irc nf_nat_tftp nf_conntrack_tftp nf_nat_ftp nf_conntrack_ftp xt_CT xt_helper nf_nat_h323 nf_conntrack_h323 xt_conntrack xt_comment ipt_REJECT nf_reject_ipv4 nf_log_ipv4 nf_log_common xt_LOG xt_limit iptable_raw iptable_mangle iptable_filter vfat fat snd_hda_codec_hdmi sch_fq_codel snd_hda_codec_realtek snd_hda_codec_generic intel_powerclamp coretemp kvm_intel i2c_algo_bit fb_sys_fops syscopyarea > Sep 19 22:01:51 maverick kernel: sysfillrect sysimgblt snd_hda_intel kvm snd_hda_codec snd_hda_core iTCO_wdt iTCO_vendor_support irqbypass snd_hwdep crct10dif_pclmul mcs7830 crc32_pclmul snd_pcm ghash_clmulni_intel snd_timer usbnet r8169 pcspkr snd i2c_i801 lpc_ich mfd_core mii soundcore i2c_hid i2c_core pcc_cpufreq rfkill_gpio rfkill intel_int0002_vgpio lp parport_pc parport video > Sep 19 22:01:51 maverick kernel: CPU: 2 PID: 2510 Comm: W-Q3 Tainted: G O 4.14.138-ipfire #1 > Sep 19 22:01:51 maverick kernel: Hardware name: Gigabyte Technology Co., Ltd. Default string/N3150ND3V, BIOS F5a 01/19/2018 > Sep 19 22:01:51 maverick kernel: task: ffffa393374fb200 task.stack: ffffa98f00290000 > Sep 19 22:01:51 maverick kernel: RIP: 0010:refcount_inc.cold.12+0x13/0x16 > Sep 19 22:01:51 maverick kernel: RSP: 0018:ffffa98f00293798 EFLAGS: 00010246 > Sep 19 22:01:51 maverick kernel: RAX: 000000000000002b RBX: ffffa39338e7ed00 RCX: 0000000000000000 > Sep 19 22:01:51 maverick kernel: RDX: 0000000000000000 RSI: ffffa3933fd163f8 RDI: ffffa3933fd163f8 > Sep 19 22:01:51 maverick kernel: RBP: ffffffffb3e9e220 R08: 000000000000003c R09: 000000000000027c > Sep 19 22:01:51 maverick kernel: R10: 0000000000000000 R11: 0000000000000001 R12: ffffa39338c81100 > Sep 19 22:01:51 maverick kernel: R13: ffffffffb44c35c0 R14: 0000000000028003 R15: ffffffffc03fc3e0 > Sep 19 22:01:51 maverick kernel: FS: 00007309737fe700(0000) GS:ffffa3933fd00000(0000) knlGS:0000000000000000 > Sep 19 22:01:51 maverick kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > Sep 19 22:01:51 maverick kernel: CR2: 00007eb62b581728 CR3: 0000000177e30000 CR4: 00000000001006e0 > Sep 19 22:01:51 maverick kernel: Call Trace: > Sep 19 22:01:51 maverick kernel: nf_queue_entry_get_refs+0x41/0x90 > Sep 19 22:01:51 maverick kernel: nf_queue+0xf9/0x220 > Sep 19 22:01:51 maverick kernel: nf_hook_slow+0x9f/0xf0 > Sep 19 22:01:51 maverick kernel: __ip_local_out+0xe4/0x150 > Sep 19 22:01:51 maverick kernel: ? ip_forward_options.cold.7+0x27/0x27 > Sep 19 22:01:51 maverick kernel: xfrm_output_resume+0x21e/0x540 > Sep 19 22:01:51 maverick kernel: ? ipv4_confirm+0x3f/0xd0 > Sep 19 22:01:51 maverick kernel: xfrm4_output+0x3a/0xe0 > Sep 19 22:01:51 maverick kernel: ? xfrm4_udp_encap_rcv+0x1a0/0x1a0 > Sep 19 22:01:51 maverick kernel: nf_reinject+0x176/0x190 > Sep 19 22:01:51 maverick kernel: nfqnl_recv_verdict+0x293/0x4a0 [nfnetlink_queue] > Sep 19 22:01:51 maverick kernel: ? nla_parse+0xb5/0xe0 > Sep 19 22:01:51 maverick kernel: nfnetlink_rcv_msg+0x14e/0x260 > Sep 19 22:01:51 maverick kernel: ? nfnetlink_net_exit_batch+0x60/0x60 > Sep 19 22:01:51 maverick kernel: netlink_rcv_skb+0x78/0x150 > Sep 19 22:01:51 maverick kernel: nfnetlink_rcv+0x70/0x760 > Sep 19 22:01:51 maverick kernel: ? lock_timer_base+0x67/0x80 > Sep 19 22:01:51 maverick kernel: ? try_to_del_timer_sync+0x4d/0x80 > Sep 19 22:01:51 maverick kernel: ? __netlink_lookup+0xe1/0x140 > Sep 19 22:01:51 maverick kernel: netlink_unicast+0x183/0x230 > Sep 19 22:01:51 maverick kernel: netlink_sendmsg+0x204/0x3d0 > Sep 19 22:01:51 maverick kernel: sock_sendmsg+0x36/0x40 > Sep 19 22:01:51 maverick kernel: ___sys_sendmsg+0x2a7/0x300 > Sep 19 22:01:51 maverick kernel: ? netlink_recvmsg+0x398/0x460 > Sep 19 22:01:51 maverick kernel: ? __switch_to_asm+0x41/0x70 > Sep 19 22:01:51 maverick kernel: __sys_sendmsg+0x67/0xb0 > Sep 19 22:01:51 maverick kernel: do_syscall_64+0x67/0x100 > Sep 19 22:01:51 maverick kernel: entry_SYSCALL_64_after_hwframe+0x3d/0xa2 > Sep 19 22:01:51 maverick kernel: RIP: 0033:0x73097af4a8c7 > Sep 19 22:01:51 maverick kernel: RSP: 002b:00007309737fbfc0 EFLAGS: 00000293 ORIG_RAX: 000000000000002e > Sep 19 22:01:51 maverick kernel: RAX: ffffffffffffffda RBX: 0000000000000008 RCX: 000073097af4a8c7 > Sep 19 22:01:51 maverick kernel: RDX: 0000000000000000 RSI: 00007309737fc000 RDI: 0000000000000008 > Sep 19 22:01:51 maverick kernel: RBP: 00007309737fc000 R08: 0000000000000000 R09: 0000000000000301 > Sep 19 22:01:51 maverick kernel: R10: 0000730974409e20 R11: 0000000000000293 R12: 0000000000000000 > Sep 19 22:01:51 maverick kernel: R13: 000073096026cd98 R14: 0000000000000070 R15: 0000000000000001 > Sep 19 22:01:51 maverick kernel: Code: 08 90 67 b4 c6 05 5e ca d8 00 01 e8 f7 e5 d7 ff 0f 0b b8 01 00 00 00 c3 48 c7 c7 60 90 67 b4 c6 05 42 ca d8 00 01 e8 dc e5 d7 ff <0f> 0b c3 48 c7 c7 90 90 67 b4 c6 05 2b ca d8 00 01 e8 c6 e5 d7 > Sep 19 22:01:51 maverick kernel: ---[ end trace dc2e33bbb9167d28 ]--- > Sep 19 22:01:51 maverick kernel: refcount_t: underflow; use-after-free. > Sep 19 22:01:51 maverick kernel: ------------[ cut here ]------------ > Sep 19 22:01:51 maverick kernel: WARNING: CPU: 3 PID: 2507 at lib/refcount.c:187 refcount_sub_and_test.cold.13+0x13/0x1a > Sep 19 22:01:51 maverick kernel: Modules linked in: xt_IMQ imq xt_length xt_DSCP xt_layer7 cls_fw sch_htb chacha20_x86_64 chacha20_generic poly1305_x86_64 poly1305_generic chacha20poly1305 esp4 xfrm6_mode_tunnel xfrm4_mode_tunnel tun nfnetlink_queue xt_NFQUEUE ipt_MASQUERADE nf_nat_masquerade_ipv4 pppoe pppox ppp_generic slhc xt_geoip(O) xt_connlimit xt_multiport xt_hashlimit xt_mark xt_policy xt_TCPMSS nf_nat_irc nf_conntrack_irc nf_nat_tftp nf_conntrack_tftp nf_nat_ftp nf_conntrack_ftp xt_CT xt_helper nf_nat_h323 nf_conntrack_h323 xt_conntrack xt_comment ipt_REJECT nf_reject_ipv4 nf_log_ipv4 nf_log_common xt_LOG xt_limit iptable_raw iptable_mangle iptable_filter vfat fat snd_hda_codec_hdmi sch_fq_codel snd_hda_codec_realtek snd_hda_codec_generic intel_powerclamp coretemp kvm_intel i2c_algo_bit fb_sys_fops syscopyarea > Sep 19 22:01:51 maverick kernel: sysfillrect sysimgblt snd_hda_intel kvm snd_hda_codec snd_hda_core iTCO_wdt iTCO_vendor_support irqbypass snd_hwdep crct10dif_pclmul mcs7830 crc32_pclmul snd_pcm ghash_clmulni_intel snd_timer usbnet r8169 pcspkr snd i2c_i801 lpc_ich mfd_core mii soundcore i2c_hid i2c_core pcc_cpufreq rfkill_gpio rfkill intel_int0002_vgpio lp parport_pc parport video > Sep 19 22:01:51 maverick kernel: CPU: 3 PID: 2507 Comm: W-Q2 Tainted: G W O 4.14.138-ipfire #1 > Sep 19 22:01:51 maverick kernel: Hardware name: Gigabyte Technology Co., Ltd. Default string/N3150ND3V, BIOS F5a 01/19/2018 > Sep 19 22:01:51 maverick kernel: task: ffffa393395ea580 task.stack: ffffa98f00250000 > Sep 19 22:01:51 maverick kernel: RIP: 0010:refcount_sub_and_test.cold.13+0x13/0x1a > Sep 19 22:01:51 maverick kernel: RSP: 0018:ffffa98f00253928 EFLAGS: 00010246 > Sep 19 22:01:51 maverick kernel: RAX: 0000000000000026 RBX: 0000000000000000 RCX: 0000000000000000 > Sep 19 22:01:51 maverick kernel: RDX: 0000000000000000 RSI: ffffa3933fd963f8 RDI: ffffa3933fd963f8 > Sep 19 22:01:51 maverick kernel: RBP: ffffa39339bc4400 R08: 0000000000000038 R09: 00000000000002b4 > Sep 19 22:01:51 maverick kernel: R10: 0000000000000000 R11: 0000000000000001 R12: ffffa393383ab480 > Sep 19 22:01:51 maverick kernel: R13: ffffa39337f5da00 R14: ffffa39339cd9840 R15: 0000000000000000 > Sep 19 22:01:51 maverick kernel: FS: 0000730973fff700(0000) GS:ffffa3933fd80000(0000) knlGS:0000000000000000 > Sep 19 22:01:51 maverick kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > Sep 19 22:01:51 maverick kernel: CR2: 0000000002218828 CR3: 0000000177e30000 CR4: 00000000001006e0 > Sep 19 22:01:51 maverick kernel: Call Trace: > Sep 19 22:01:51 maverick kernel: nf_queue_entry_release_refs+0x45/0xa0 > Sep 19 22:01:51 maverick kernel: nf_reinject+0x3d/0x190 > Sep 19 22:01:51 maverick kernel: nfqnl_recv_verdict+0x293/0x4a0 [nfnetlink_queue] > Sep 19 22:01:51 maverick kernel: ? nla_parse+0xb5/0xe0 > Sep 19 22:01:51 maverick kernel: nfnetlink_rcv_msg+0x14e/0x260 > Sep 19 22:01:51 maverick kernel: ? nfnetlink_net_exit_batch+0x60/0x60 > Sep 19 22:01:51 maverick kernel: netlink_rcv_skb+0x78/0x150 > Sep 19 22:01:51 maverick kernel: nfnetlink_rcv+0x70/0x760 > Sep 19 22:01:51 maverick kernel: ? lock_timer_base+0x67/0x80 > Sep 19 22:01:51 maverick kernel: ? try_to_del_timer_sync+0x4d/0x80 > Sep 19 22:01:51 maverick kernel: ? __netlink_lookup+0xe1/0x140 > Sep 19 22:01:51 maverick kernel: netlink_unicast+0x183/0x230 > Sep 19 22:01:51 maverick kernel: netlink_sendmsg+0x204/0x3d0 > Sep 19 22:01:51 maverick kernel: sock_sendmsg+0x36/0x40 > Sep 19 22:01:51 maverick kernel: ___sys_sendmsg+0x2a7/0x300 > Sep 19 22:01:51 maverick kernel: ? netlink_recvmsg+0x398/0x460 > Sep 19 22:01:51 maverick kernel: ? __switch_to_asm+0x41/0x70 > Sep 19 22:01:51 maverick kernel: __sys_sendmsg+0x67/0xb0 > Sep 19 22:01:51 maverick kernel: do_syscall_64+0x67/0x100 > Sep 19 22:01:51 maverick kernel: entry_SYSCALL_64_after_hwframe+0x3d/0xa2 > Sep 19 22:01:51 maverick kernel: RIP: 0033:0x73097af4a8c7 > Sep 19 22:01:51 maverick kernel: RSP: 002b:0000730973ffcfc0 EFLAGS: 00000293 ORIG_RAX: 000000000000002e > Sep 19 22:01:51 maverick kernel: RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 000073097af4a8c7 > Sep 19 22:01:51 maverick kernel: RDX: 0000000000000000 RSI: 0000730973ffd000 RDI: 0000000000000007 > Sep 19 22:01:51 maverick kernel: RBP: 0000730973ffd000 R08: 0000000000000000 R09: 0000000000000301 > Sep 19 22:01:51 maverick kernel: R10: 000073096826c188 R11: 0000000000000293 R12: 0000000000000000 > Sep 19 22:01:51 maverick kernel: R13: 000073096826cd98 R14: 0000000065000070 R15: 0000000000000001 > Sep 19 22:01:51 maverick kernel: Code: 00 c3 48 c7 c7 60 90 67 b4 c6 05 42 ca d8 00 01 e8 dc e5 d7 ff 0f 0b c3 48 c7 c7 90 90 67 b4 c6 05 2b ca d8 00 01 e8 c6 e5 d7 ff <0f> 0b e9 86 fe ff ff 48 c7 c7 b8 90 67 b4 c6 05 10 ca d8 00 01 > Sep 19 22:01:51 maverick kernel: ---[ end trace dc2e33bbb9167d29 ]--- Suricata works correctly and detects attacks as expected. Starting sequence took about 83 seconds on my testing hardware. In my point of view, the ISO includes all necessary fixes for curing our Suricata problems - slow establish of new connections, DNS trouble, et cetera - and I would like them to be included in the next Core Update (testing) so we can give it a bigger and more extensive test. What do you think about this? OpenVPN performance is poor, but I guess that is because of something else. Command outputs for reference: > [root(a)maverick ~]# uname -a > Linux maverick 4.14.138-ipfire #1 SMP Mon Sep 9 07:55:34 GMT 2019 x86_64 Intel(R) Celeron(R) CPU N3150 @ 1.60GHz GenuineIntel GNU/Linux > [root(a)maverick ~]# suricata -V > This is Suricata version 4.1.4 RELEASE > [root(a)maverick ~]# suricata --build-info > This is Suricata version 4.1.4 RELEASE > Features: NFQ PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_LIBJANSSON TLS MAGIC RUST > SIMD support: none > Atomic intrisics: 1 2 4 8 byte(s) > 64-bits, Little-endian architecture > GCC version 8.3.0, C version 199901 > compiled with _FORTIFY_SOURCE=2 > L1 cache line size (CLS)=64 > thread local storage method: __thread > compiled with LibHTP v0.5.30, linked against LibHTP v0.5.30 > > Suricata Configuration: > AF_PACKET support: yes > eBPF support: no > XDP support: no > PF_RING support: no > NFQueue support: yes > NFLOG support: no > IPFW support: no > Netmap support: no > DAG enabled: no > Napatech enabled: no > WinDivert enabled: no > > Unix socket enabled: yes > Detection enabled: yes > > Libmagic support: yes > libnss support: no > libnspr support: no > libjansson support: yes > liblzma support: yes > hiredis support: no > hiredis async with libevent: no > Prelude support: no > PCRE jit: yes > LUA support: no > libluajit: no > libgeoip: no > Non-bundled htp: yes > Old barnyard2 support: no > Hyperscan support: yes > Libnet support: yes > liblz4 support: no > > Rust support: yes > Rust strict mode: no > Rust debug mode: no > Rust compiler: rustc 1.37.0 (eae3437df 2019-08-13) > Rust cargo: cargo 1.37.0 (9edd08916 2019-08-02) > > Install suricatasc: no > Install suricata-update: no > > Profiling enabled: no > Profiling locks enabled: no > > Development settings: > Coccinelle / spatch: no > Unit tests enabled: no > Debug output enabled: no > Debug validation enabled: no > > Generic build parameters: > Installation prefix: /usr > Configuration directory: /etc/suricata/ > Log directory: /var/log/suricata/ > > --prefix /usr > --sysconfdir /etc > --localstatedir /var > --datarootdir /usr/share > > Host: x86_64-pc-linux-gnu > Compiler: gcc (exec name) / gcc (real) > GCC Protect enabled: yes > GCC march native enabled: no > GCC Profile enabled: no > Position Independent Executable enabled: no > CFLAGS -O2 -pipe -Wall -fexceptions -fPIC -m64 -mindirect-branch=thunk -mfunction-return=thunk -mtune=generic -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -I${srcdir}/../rust/gen/c-headers > PCAP_CFLAGS -I/usr/include > SECCFLAGS -fstack-protector -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security Thanks, and best regards, Peter Müller