From: Michael Tremer <michael.tremer@ipfire.org>
To: development@lists.ipfire.org
Subject: Re: Testing report regarding image: next-suricata-rust/d22217e1-dirty
Date: Mon, 23 Sep 2019 23:17:15 +0100 [thread overview]
Message-ID: <81CE3C2B-203D-4E12-8241-4B9C363B57D4@ipfire.org> (raw)
In-Reply-To: <36e6c823-99eb-dea5-b949-d5aa8919c8b0@ipfire.org>
[-- Attachment #1: Type: text/plain, Size: 16770 bytes --]
What are we doing about these kernel traces?
They don’t look like we can ignore them.
-Michael
> On 19 Sep 2019, at 21:16, peter.mueller(a)ipfire.org wrote:
>
> Hello Stefan,
>
> sorry for my late reply on this topic. Finally, I was able to
> test your new ISO, next-suricata-rust/d22217e1-dirty .
>
> New connections, regardless of SSH, TLS, DNS or whatever, are
> now established instantly. No packets are dropped anymore, this
> issue seems to be solved by the changes included in your ISO.
>
> OpenVPN RW performance, however, is worse than before: Now around
> ~ 400 kB/sec. if Suricata is enabled, and somewhere around 900 kB/sec.
> if it is not. I am getting the feeling this is an OpenVPN bug or
> performance issue, but as mentioned several times before, I am not
> satisfied with OpenVPN anyway.
>
> A few minutes after booting, kernel emits these log lines:
>> Sep 19 22:01:51 maverick kernel: refcount_t: increment on 0; use-after-free.
>> Sep 19 22:01:51 maverick kernel: ------------[ cut here ]------------
>> Sep 19 22:01:51 maverick kernel: WARNING: CPU: 2 PID: 2510 at lib/refcount.c:153 refcount_inc.cold.12+0x13/0x16
>> Sep 19 22:01:51 maverick kernel: Modules linked in: xt_IMQ imq xt_length xt_DSCP xt_layer7 cls_fw sch_htb chacha20_x86_64 chacha20_generic poly1305_x86_64 poly1305_generic chacha20poly1305 esp4 xfrm6_mode_tunnel xfrm4_mode_tunnel tun nfnetlink_queue xt_NFQUEUE ipt_MASQUERADE nf_nat_masquerade_ipv4 pppoe pppox ppp_generic slhc xt_geoip(O) xt_connlimit xt_multiport xt_hashlimit xt_mark xt_policy xt_TCPMSS nf_nat_irc nf_conntrack_irc nf_nat_tftp nf_conntrack_tftp nf_nat_ftp nf_conntrack_ftp xt_CT xt_helper nf_nat_h323 nf_conntrack_h323 xt_conntrack xt_comment ipt_REJECT nf_reject_ipv4 nf_log_ipv4 nf_log_common xt_LOG xt_limit iptable_raw iptable_mangle iptable_filter vfat fat snd_hda_codec_hdmi sch_fq_codel snd_hda_codec_realtek snd_hda_codec_generic intel_powerclamp coretemp kvm_intel i2c_algo_bit fb_sys_fops syscopyarea
>> Sep 19 22:01:51 maverick kernel: sysfillrect sysimgblt snd_hda_intel kvm snd_hda_codec snd_hda_core iTCO_wdt iTCO_vendor_support irqbypass snd_hwdep crct10dif_pclmul mcs7830 crc32_pclmul snd_pcm ghash_clmulni_intel snd_timer usbnet r8169 pcspkr snd i2c_i801 lpc_ich mfd_core mii soundcore i2c_hid i2c_core pcc_cpufreq rfkill_gpio rfkill intel_int0002_vgpio lp parport_pc parport video
>> Sep 19 22:01:51 maverick kernel: CPU: 2 PID: 2510 Comm: W-Q3 Tainted: G O 4.14.138-ipfire #1
>> Sep 19 22:01:51 maverick kernel: Hardware name: Gigabyte Technology Co., Ltd. Default string/N3150ND3V, BIOS F5a 01/19/2018
>> Sep 19 22:01:51 maverick kernel: task: ffffa393374fb200 task.stack: ffffa98f00290000
>> Sep 19 22:01:51 maverick kernel: RIP: 0010:refcount_inc.cold.12+0x13/0x16
>> Sep 19 22:01:51 maverick kernel: RSP: 0018:ffffa98f00293798 EFLAGS: 00010246
>> Sep 19 22:01:51 maverick kernel: RAX: 000000000000002b RBX: ffffa39338e7ed00 RCX: 0000000000000000
>> Sep 19 22:01:51 maverick kernel: RDX: 0000000000000000 RSI: ffffa3933fd163f8 RDI: ffffa3933fd163f8
>> Sep 19 22:01:51 maverick kernel: RBP: ffffffffb3e9e220 R08: 000000000000003c R09: 000000000000027c
>> Sep 19 22:01:51 maverick kernel: R10: 0000000000000000 R11: 0000000000000001 R12: ffffa39338c81100
>> Sep 19 22:01:51 maverick kernel: R13: ffffffffb44c35c0 R14: 0000000000028003 R15: ffffffffc03fc3e0
>> Sep 19 22:01:51 maverick kernel: FS: 00007309737fe700(0000) GS:ffffa3933fd00000(0000) knlGS:0000000000000000
>> Sep 19 22:01:51 maverick kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> Sep 19 22:01:51 maverick kernel: CR2: 00007eb62b581728 CR3: 0000000177e30000 CR4: 00000000001006e0
>> Sep 19 22:01:51 maverick kernel: Call Trace:
>> Sep 19 22:01:51 maverick kernel: nf_queue_entry_get_refs+0x41/0x90
>> Sep 19 22:01:51 maverick kernel: nf_queue+0xf9/0x220
>> Sep 19 22:01:51 maverick kernel: nf_hook_slow+0x9f/0xf0
>> Sep 19 22:01:51 maverick kernel: __ip_local_out+0xe4/0x150
>> Sep 19 22:01:51 maverick kernel: ? ip_forward_options.cold.7+0x27/0x27
>> Sep 19 22:01:51 maverick kernel: xfrm_output_resume+0x21e/0x540
>> Sep 19 22:01:51 maverick kernel: ? ipv4_confirm+0x3f/0xd0
>> Sep 19 22:01:51 maverick kernel: xfrm4_output+0x3a/0xe0
>> Sep 19 22:01:51 maverick kernel: ? xfrm4_udp_encap_rcv+0x1a0/0x1a0
>> Sep 19 22:01:51 maverick kernel: nf_reinject+0x176/0x190
>> Sep 19 22:01:51 maverick kernel: nfqnl_recv_verdict+0x293/0x4a0 [nfnetlink_queue]
>> Sep 19 22:01:51 maverick kernel: ? nla_parse+0xb5/0xe0
>> Sep 19 22:01:51 maverick kernel: nfnetlink_rcv_msg+0x14e/0x260
>> Sep 19 22:01:51 maverick kernel: ? nfnetlink_net_exit_batch+0x60/0x60
>> Sep 19 22:01:51 maverick kernel: netlink_rcv_skb+0x78/0x150
>> Sep 19 22:01:51 maverick kernel: nfnetlink_rcv+0x70/0x760
>> Sep 19 22:01:51 maverick kernel: ? lock_timer_base+0x67/0x80
>> Sep 19 22:01:51 maverick kernel: ? try_to_del_timer_sync+0x4d/0x80
>> Sep 19 22:01:51 maverick kernel: ? __netlink_lookup+0xe1/0x140
>> Sep 19 22:01:51 maverick kernel: netlink_unicast+0x183/0x230
>> Sep 19 22:01:51 maverick kernel: netlink_sendmsg+0x204/0x3d0
>> Sep 19 22:01:51 maverick kernel: sock_sendmsg+0x36/0x40
>> Sep 19 22:01:51 maverick kernel: ___sys_sendmsg+0x2a7/0x300
>> Sep 19 22:01:51 maverick kernel: ? netlink_recvmsg+0x398/0x460
>> Sep 19 22:01:51 maverick kernel: ? __switch_to_asm+0x41/0x70
>> Sep 19 22:01:51 maverick kernel: __sys_sendmsg+0x67/0xb0
>> Sep 19 22:01:51 maverick kernel: do_syscall_64+0x67/0x100
>> Sep 19 22:01:51 maverick kernel: entry_SYSCALL_64_after_hwframe+0x3d/0xa2
>> Sep 19 22:01:51 maverick kernel: RIP: 0033:0x73097af4a8c7
>> Sep 19 22:01:51 maverick kernel: RSP: 002b:00007309737fbfc0 EFLAGS: 00000293 ORIG_RAX: 000000000000002e
>> Sep 19 22:01:51 maverick kernel: RAX: ffffffffffffffda RBX: 0000000000000008 RCX: 000073097af4a8c7
>> Sep 19 22:01:51 maverick kernel: RDX: 0000000000000000 RSI: 00007309737fc000 RDI: 0000000000000008
>> Sep 19 22:01:51 maverick kernel: RBP: 00007309737fc000 R08: 0000000000000000 R09: 0000000000000301
>> Sep 19 22:01:51 maverick kernel: R10: 0000730974409e20 R11: 0000000000000293 R12: 0000000000000000
>> Sep 19 22:01:51 maverick kernel: R13: 000073096026cd98 R14: 0000000000000070 R15: 0000000000000001
>> Sep 19 22:01:51 maverick kernel: Code: 08 90 67 b4 c6 05 5e ca d8 00 01 e8 f7 e5 d7 ff 0f 0b b8 01 00 00 00 c3 48 c7 c7 60 90 67 b4 c6 05 42 ca d8 00 01 e8 dc e5 d7 ff <0f> 0b c3 48 c7 c7 90 90 67 b4 c6 05 2b ca d8 00 01 e8 c6 e5 d7
>> Sep 19 22:01:51 maverick kernel: ---[ end trace dc2e33bbb9167d28 ]---
>> Sep 19 22:01:51 maverick kernel: refcount_t: underflow; use-after-free.
>> Sep 19 22:01:51 maverick kernel: ------------[ cut here ]------------
>> Sep 19 22:01:51 maverick kernel: WARNING: CPU: 3 PID: 2507 at lib/refcount.c:187 refcount_sub_and_test.cold.13+0x13/0x1a
>> Sep 19 22:01:51 maverick kernel: Modules linked in: xt_IMQ imq xt_length xt_DSCP xt_layer7 cls_fw sch_htb chacha20_x86_64 chacha20_generic poly1305_x86_64 poly1305_generic chacha20poly1305 esp4 xfrm6_mode_tunnel xfrm4_mode_tunnel tun nfnetlink_queue xt_NFQUEUE ipt_MASQUERADE nf_nat_masquerade_ipv4 pppoe pppox ppp_generic slhc xt_geoip(O) xt_connlimit xt_multiport xt_hashlimit xt_mark xt_policy xt_TCPMSS nf_nat_irc nf_conntrack_irc nf_nat_tftp nf_conntrack_tftp nf_nat_ftp nf_conntrack_ftp xt_CT xt_helper nf_nat_h323 nf_conntrack_h323 xt_conntrack xt_comment ipt_REJECT nf_reject_ipv4 nf_log_ipv4 nf_log_common xt_LOG xt_limit iptable_raw iptable_mangle iptable_filter vfat fat snd_hda_codec_hdmi sch_fq_codel snd_hda_codec_realtek snd_hda_codec_generic intel_powerclamp coretemp kvm_intel i2c_algo_bit fb_sys_fops syscopyarea
>> Sep 19 22:01:51 maverick kernel: sysfillrect sysimgblt snd_hda_intel kvm snd_hda_codec snd_hda_core iTCO_wdt iTCO_vendor_support irqbypass snd_hwdep crct10dif_pclmul mcs7830 crc32_pclmul snd_pcm ghash_clmulni_intel snd_timer usbnet r8169 pcspkr snd i2c_i801 lpc_ich mfd_core mii soundcore i2c_hid i2c_core pcc_cpufreq rfkill_gpio rfkill intel_int0002_vgpio lp parport_pc parport video
>> Sep 19 22:01:51 maverick kernel: CPU: 3 PID: 2507 Comm: W-Q2 Tainted: G W O 4.14.138-ipfire #1
>> Sep 19 22:01:51 maverick kernel: Hardware name: Gigabyte Technology Co., Ltd. Default string/N3150ND3V, BIOS F5a 01/19/2018
>> Sep 19 22:01:51 maverick kernel: task: ffffa393395ea580 task.stack: ffffa98f00250000
>> Sep 19 22:01:51 maverick kernel: RIP: 0010:refcount_sub_and_test.cold.13+0x13/0x1a
>> Sep 19 22:01:51 maverick kernel: RSP: 0018:ffffa98f00253928 EFLAGS: 00010246
>> Sep 19 22:01:51 maverick kernel: RAX: 0000000000000026 RBX: 0000000000000000 RCX: 0000000000000000
>> Sep 19 22:01:51 maverick kernel: RDX: 0000000000000000 RSI: ffffa3933fd963f8 RDI: ffffa3933fd963f8
>> Sep 19 22:01:51 maverick kernel: RBP: ffffa39339bc4400 R08: 0000000000000038 R09: 00000000000002b4
>> Sep 19 22:01:51 maverick kernel: R10: 0000000000000000 R11: 0000000000000001 R12: ffffa393383ab480
>> Sep 19 22:01:51 maverick kernel: R13: ffffa39337f5da00 R14: ffffa39339cd9840 R15: 0000000000000000
>> Sep 19 22:01:51 maverick kernel: FS: 0000730973fff700(0000) GS:ffffa3933fd80000(0000) knlGS:0000000000000000
>> Sep 19 22:01:51 maverick kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> Sep 19 22:01:51 maverick kernel: CR2: 0000000002218828 CR3: 0000000177e30000 CR4: 00000000001006e0
>> Sep 19 22:01:51 maverick kernel: Call Trace:
>> Sep 19 22:01:51 maverick kernel: nf_queue_entry_release_refs+0x45/0xa0
>> Sep 19 22:01:51 maverick kernel: nf_reinject+0x3d/0x190
>> Sep 19 22:01:51 maverick kernel: nfqnl_recv_verdict+0x293/0x4a0 [nfnetlink_queue]
>> Sep 19 22:01:51 maverick kernel: ? nla_parse+0xb5/0xe0
>> Sep 19 22:01:51 maverick kernel: nfnetlink_rcv_msg+0x14e/0x260
>> Sep 19 22:01:51 maverick kernel: ? nfnetlink_net_exit_batch+0x60/0x60
>> Sep 19 22:01:51 maverick kernel: netlink_rcv_skb+0x78/0x150
>> Sep 19 22:01:51 maverick kernel: nfnetlink_rcv+0x70/0x760
>> Sep 19 22:01:51 maverick kernel: ? lock_timer_base+0x67/0x80
>> Sep 19 22:01:51 maverick kernel: ? try_to_del_timer_sync+0x4d/0x80
>> Sep 19 22:01:51 maverick kernel: ? __netlink_lookup+0xe1/0x140
>> Sep 19 22:01:51 maverick kernel: netlink_unicast+0x183/0x230
>> Sep 19 22:01:51 maverick kernel: netlink_sendmsg+0x204/0x3d0
>> Sep 19 22:01:51 maverick kernel: sock_sendmsg+0x36/0x40
>> Sep 19 22:01:51 maverick kernel: ___sys_sendmsg+0x2a7/0x300
>> Sep 19 22:01:51 maverick kernel: ? netlink_recvmsg+0x398/0x460
>> Sep 19 22:01:51 maverick kernel: ? __switch_to_asm+0x41/0x70
>> Sep 19 22:01:51 maverick kernel: __sys_sendmsg+0x67/0xb0
>> Sep 19 22:01:51 maverick kernel: do_syscall_64+0x67/0x100
>> Sep 19 22:01:51 maverick kernel: entry_SYSCALL_64_after_hwframe+0x3d/0xa2
>> Sep 19 22:01:51 maverick kernel: RIP: 0033:0x73097af4a8c7
>> Sep 19 22:01:51 maverick kernel: RSP: 002b:0000730973ffcfc0 EFLAGS: 00000293 ORIG_RAX: 000000000000002e
>> Sep 19 22:01:51 maverick kernel: RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 000073097af4a8c7
>> Sep 19 22:01:51 maverick kernel: RDX: 0000000000000000 RSI: 0000730973ffd000 RDI: 0000000000000007
>> Sep 19 22:01:51 maverick kernel: RBP: 0000730973ffd000 R08: 0000000000000000 R09: 0000000000000301
>> Sep 19 22:01:51 maverick kernel: R10: 000073096826c188 R11: 0000000000000293 R12: 0000000000000000
>> Sep 19 22:01:51 maverick kernel: R13: 000073096826cd98 R14: 0000000065000070 R15: 0000000000000001
>> Sep 19 22:01:51 maverick kernel: Code: 00 c3 48 c7 c7 60 90 67 b4 c6 05 42 ca d8 00 01 e8 dc e5 d7 ff 0f 0b c3 48 c7 c7 90 90 67 b4 c6 05 2b ca d8 00 01 e8 c6 e5 d7 ff <0f> 0b e9 86 fe ff ff 48 c7 c7 b8 90 67 b4 c6 05 10 ca d8 00 01
>> Sep 19 22:01:51 maverick kernel: ---[ end trace dc2e33bbb9167d29 ]---
>
> Suricata works correctly and detects attacks as expected. Starting
> sequence took about 83 seconds on my testing hardware.
>
> In my point of view, the ISO includes all necessary fixes for curing
> our Suricata problems - slow establish of new connections, DNS trouble,
> et cetera - and I would like them to be included in the next Core Update
> (testing) so we can give it a bigger and more extensive test. What
> do you think about this?
>
> OpenVPN performance is poor, but I guess that is because of something
> else.
>
> Command outputs for reference:
>> [root(a)maverick ~]# uname -a
>> Linux maverick 4.14.138-ipfire #1 SMP Mon Sep 9 07:55:34 GMT 2019 x86_64 Intel(R) Celeron(R) CPU N3150 @ 1.60GHz GenuineIntel GNU/Linux
>
>> [root(a)maverick ~]# suricata -V
>> This is Suricata version 4.1.4 RELEASE
>
>> [root(a)maverick ~]# suricata --build-info
>> This is Suricata version 4.1.4 RELEASE
>> Features: NFQ PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_LIBJANSSON TLS MAGIC RUST
>> SIMD support: none
>> Atomic intrisics: 1 2 4 8 byte(s)
>> 64-bits, Little-endian architecture
>> GCC version 8.3.0, C version 199901
>> compiled with _FORTIFY_SOURCE=2
>> L1 cache line size (CLS)=64
>> thread local storage method: __thread
>> compiled with LibHTP v0.5.30, linked against LibHTP v0.5.30
>>
>> Suricata Configuration:
>> AF_PACKET support: yes
>> eBPF support: no
>> XDP support: no
>> PF_RING support: no
>> NFQueue support: yes
>> NFLOG support: no
>> IPFW support: no
>> Netmap support: no
>> DAG enabled: no
>> Napatech enabled: no
>> WinDivert enabled: no
>>
>> Unix socket enabled: yes
>> Detection enabled: yes
>>
>> Libmagic support: yes
>> libnss support: no
>> libnspr support: no
>> libjansson support: yes
>> liblzma support: yes
>> hiredis support: no
>> hiredis async with libevent: no
>> Prelude support: no
>> PCRE jit: yes
>> LUA support: no
>> libluajit: no
>> libgeoip: no
>> Non-bundled htp: yes
>> Old barnyard2 support: no
>> Hyperscan support: yes
>> Libnet support: yes
>> liblz4 support: no
>>
>> Rust support: yes
>> Rust strict mode: no
>> Rust debug mode: no
>> Rust compiler: rustc 1.37.0 (eae3437df 2019-08-13)
>> Rust cargo: cargo 1.37.0 (9edd08916 2019-08-02)
>>
>> Install suricatasc: no
>> Install suricata-update: no
>>
>> Profiling enabled: no
>> Profiling locks enabled: no
>>
>> Development settings:
>> Coccinelle / spatch: no
>> Unit tests enabled: no
>> Debug output enabled: no
>> Debug validation enabled: no
>>
>> Generic build parameters:
>> Installation prefix: /usr
>> Configuration directory: /etc/suricata/
>> Log directory: /var/log/suricata/
>>
>> --prefix /usr
>> --sysconfdir /etc
>> --localstatedir /var
>> --datarootdir /usr/share
>>
>> Host: x86_64-pc-linux-gnu
>> Compiler: gcc (exec name) / gcc (real)
>> GCC Protect enabled: yes
>> GCC march native enabled: no
>> GCC Profile enabled: no
>> Position Independent Executable enabled: no
>> CFLAGS -O2 -pipe -Wall -fexceptions -fPIC -m64 -mindirect-branch=thunk -mfunction-return=thunk -mtune=generic -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -I${srcdir}/../rust/gen/c-headers
>> PCAP_CFLAGS -I/usr/include
>> SECCFLAGS -fstack-protector -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security
>
> Thanks, and best regards,
> Peter Müller
next prev parent reply other threads:[~2019-09-23 22:17 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-19 20:16 peter.mueller
2019-09-23 22:17 ` Michael Tremer [this message]
2019-09-24 7:24 ` Arne Fitzenreiter
2019-09-24 10:15 ` Michael Tremer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=81CE3C2B-203D-4E12-8241-4B9C363B57D4@ipfire.org \
--to=michael.tremer@ipfire.org \
--cc=development@lists.ipfire.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox