Development September 2019

development@lists.ipfire.org

13 participants
80 discussions

Re: [Oisf-users] Suricata causes massive packet loss
by Stefan Schantl 07 Sep '19

07 Sep '19

Hello Peter, Hello Coop, Hello *, > Hi again Peter, > > I am absolutely not recommending shipping a beta release! Rather, > I'm suggesting you try it to see if it's an issue that has already > been resolved as a sanity check. I've build and uploaded an custom install-able ISO image, containing the latest libhtp (0.5.30) and suricata 5.0.0-beta1. @Peter, please test and report if any of these changes affects your packet lost rate. > > > Have you tried using libpcap instead of AF_PACKET as the capture > mechanism? Currently we are using IPtables to delegate the packets via netfilter queue to suricata and to drop or re-inject them after scanning. Best regards, -Stefan > > > -Coop > > -----Original Message----- > From: peter.mueller(a)ipfire.org <peter.mueller(a)ipfire.org> > Sent: Thursday, September 5, 2019 11:56 AM > To: Nelson, Cooper <cnelson(a)ucsd.edu>; Peter Manev < > petermanev(a)gmail.com> > Cc: oisf-users(a)lists.openinfosecfoundation.org; IPFire: Development- > List <development(a)lists.ipfire.org>; Stefan Schantl < > stefan.schantl(a)ipfire.org> > Subject: Re: [Oisf-users] Suricata causes massive packet loss > > Hello Nelson, hello Peter, hello *, > > thank you for your replies. > > Upgrading to Suricata 5.0-beta is a difficult task, as we cannot > simply ship beta releases in our firewall distribution. Personally, I > rather doubt this is an issue due to a kernel/library/... > combination, as we use Suricata for quite a while now and are > upgrading IPFire's distribution kernel on a regular basis. > > Anyway, Stefan (see CC) is currently working on Rust for the > distribution, so we hope to take advantage of some more features > soon. But since our issue is regarding packet loss for at least DNS > and TLS traffic, I rather doubt Rust will make a big difference here. > > Changing from "workers" to "autofp" mode unfortunately did not solve > the problem. It is good to know the latter is recommended for inline > deployments, "workers" was about 0.5 % faster in our benchmarks. > > In IPFire, Suricata is started by a custom init script (please refer > to > https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=src/initscripts/system/su… > for its content) and appears like this in the process list: > > [root@maverick ~]# ps aux | grep suricata > > suricata 4882 10.9 7.3 1419868 289192 ? Ssl 20:38 1:37 > > /usr/bin/suricata -c /etc/suricata/suricata.yaml -D -q 0 -q 1 -q 2 > > -q 3 > > I am not sure what the number behind "tcp.pkt_on_wrong_thread" should > read like normally. @Peter: Is it too low or too high? > > We will ship an update for libhtp as soon as possible, thank you for > catching this. > > Thanks, and best regards, > Peter Müller

2 1

Re: [Oisf-users] Suricata causes massive packet loss
by Stefan Schantl 07 Sep '19

07 Sep '19

Hello Peter, Hello Coop, Hello *, > Hi again Peter, > > I am absolutely not recommending shipping a beta release! Rather, > I'm suggesting you try it to see if it's an issue that has already > been resolved as a sanity check. I've build and uploaded an custom install-able ISO image, containing the latest libhtp (0.5.30) and suricata 5.0.0-beta1. https://people.ipfire.org/~stevee/IPFire_2.x_suricata_rust/ipfire-2.23.x86_… @Peter, please test and report if any of these changes affects your packet lost rate. > > > Have you tried using libpcap instead of AF_PACKET as the capture > mechanism? Currently we are using IPtables to delegate the packets via netfilter queue to suricata and to drop or re-inject them after scanning. Best regards, -Stefan PS: Sorry for re-sending, I lacked to subscribe to the oisf-users mailing list... > > > -Coop > > -----Original Message----- > From: peter.mueller(a)ipfire.org <peter.mueller(a)ipfire.org> > Sent: Thursday, September 5, 2019 11:56 AM > To: Nelson, Cooper <cnelson(a)ucsd.edu>; Peter Manev < > petermanev(a)gmail.com> > Cc: oisf-users(a)lists.openinfosecfoundation.org; IPFire: Development- > List <development(a)lists.ipfire.org>; Stefan Schantl < > stefan.schantl(a)ipfire.org> > Subject: Re: [Oisf-users] Suricata causes massive packet loss > > Hello Nelson, hello Peter, hello *, > > thank you for your replies. > > Upgrading to Suricata 5.0-beta is a difficult task, as we cannot > simply ship beta releases in our firewall distribution. Personally, I > rather doubt this is an issue due to a kernel/library/... > combination, as we use Suricata for quite a while now and are > upgrading IPFire's distribution kernel on a regular basis. > > Anyway, Stefan (see CC) is currently working on Rust for the > distribution, so we hope to take advantage of some more features > soon. But since our issue is regarding packet loss for at least DNS > and TLS traffic, I rather doubt Rust will make a big difference here. > > Changing from "workers" to "autofp" mode unfortunately did not solve > the problem. It is good to know the latter is recommended for inline > deployments, "workers" was about 0.5 % faster in our benchmarks. > > In IPFire, Suricata is started by a custom init script (please refer > to > https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=src/initscripts/system/su… > for its content) and appears like this in the process list: > > [root@maverick ~]# ps aux | grep suricata > > suricata 4882 10.9 7.3 1419868 289192 ? Ssl 20:38 1:37 > > /usr/bin/suricata -c /etc/suricata/suricata.yaml -D -q 0 -q 1 -q 2 > > -q 3 > > I am not sure what the number behind "tcp.pkt_on_wrong_thread" should > read like normally. @Peter: Is it too low or too high? > > We will ship an update for libhtp as soon as possible, thank you for > catching this. > > Thanks, and best regards, > Peter Müller

1 0

[PATCH] libhtp: Update to 0.5.30
by Stefan Schantl 07 Sep '19

07 Sep '19

Fixes #12170 Signed-off-by: Stefan Schantl <stefan.schantl(a)ipfire.org> --- lfs/libhtp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lfs/libhtp b/lfs/libhtp index 212514dfb..bec93a935 100644 --- a/lfs/libhtp +++ b/lfs/libhtp @@ -24,7 +24,7 @@ include Config -VER = 0.5.29 +VER = 0.5.30 THISAPP = libhtp-$(VER) DL_FILE = $(THISAPP).tar.gz @@ -40,7 +40,7 @@ objects = $(DL_FILE) $(DL_FILE) = $(DL_FROM)/$(DL_FILE) -$(DL_FILE)_MD5 = 5feb73647723db5b458d00faddb30954 +$(DL_FILE)_MD5 = ec0d8c57f9c611719e07668bbad7e685 install : $(TARGET) -- 2.20.1

3 2

[PATCH] libhtp: Update to 0.5.30
by Stefan Schantl 06 Sep '19

06 Sep '19

This is a minor release to the latest stable version of libhtp. Signed-off-by: Stefan Schantl <stefan.schantl(a)ipfire.org> --- libhtp/libhtp.nm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libhtp/libhtp.nm b/libhtp/libhtp.nm index 8bb4ebf9c..5a7317bdb 100644 --- a/libhtp/libhtp.nm +++ b/libhtp/libhtp.nm @@ -4,7 +4,7 @@ ############################################################################### name = libhtp -version = 0.5.29 +version = 0.5.30 release = 1 groups = System/Libraries -- 2.20.1

1 0

Re: [Oisf-users] Suricata causes massive packet loss
by peter.mueller＠ipfire.org 05 Sep '19

05 Sep '19

Hello Nelson, hello Peter, hello *, thank you for your replies. Upgrading to Suricata 5.0-beta is a difficult task, as we cannot simply ship beta releases in our firewall distribution. Personally, I rather doubt this is an issue due to a kernel/library/... combination, as we use Suricata for quite a while now and are upgrading IPFire's distribution kernel on a regular basis. Anyway, Stefan (see CC) is currently working on Rust for the distribution, so we hope to take advantage of some more features soon. But since our issue is regarding packet loss for at least DNS and TLS traffic, I rather doubt Rust will make a big difference here. Changing from "workers" to "autofp" mode unfortunately did not solve the problem. It is good to know the latter is recommended for inline deployments, "workers" was about 0.5 % faster in our benchmarks. In IPFire, Suricata is started by a custom init script (please refer to https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=src/initscripts/system/su… for its content) and appears like this in the process list: > [root@maverick ~]# ps aux | grep suricata > suricata 4882 10.9 7.3 1419868 289192 ? Ssl 20:38 1:37 /usr/bin/suricata -c /etc/suricata/suricata.yaml -D -q 0 -q 1 -q 2 -q 3 I am not sure what the number behind "tcp.pkt_on_wrong_thread" should read like normally. @Peter: Is it too low or too high? We will ship an update for libhtp as soon as possible, thank you for catching this. Thanks, and best regards, Peter Müller

1 0

Please change your MUA settings for ipfire.org
by peter.mueller＠ipfire.org 05 Sep '19

05 Sep '19

Hello *, this is for people with an ipfire.org mail account only: The new mail infrastructure is now up and running for several days. Please make sure to change your MUAs' settings for ipfire.org according to https://wiki.ipfire.org/accounts , if necessary. Please let me know if there are any problems or questions. Thanks, and best regards, Peter Müller

2 1

[PATCH] zoneconf: reduce the width of inputs for vlanid
by Alex Koch 05 Sep '19

05 Sep '19

The inputs for the vlanids are overlapping the borders of their cells (using a recent Firefox on Linux Mint, Android or Windows 7). This patch fixes this by limiting the width to a fixed value. Signed-off-by: Alex Koch <ipfire(a)starkstromkonsument.de> --- html/cgi-bin/zoneconf.cgi | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/html/cgi-bin/zoneconf.cgi b/html/cgi-bin/zoneconf.cgi index 40239c14e..6b8642818 100644 --- a/html/cgi-bin/zoneconf.cgi +++ b/html/cgi-bin/zoneconf.cgi @@ -93,6 +93,10 @@ my $css = <<END text-align: center; } + input.vlanid { + width: 4em; + } + #submit-container { width: 100%; padding-top: 20px; @@ -432,7 +436,7 @@ foreach (@nics) { <option value="NATIVE" $access_selected{"NATIVE"}>$Lang::tr{"zoneconf access native"}</option> <option value="VLAN" $access_selected{"VLAN"} $vlan_disabled>$Lang::tr{"zoneconf access vlan"}</option> </select> - <input type="number" id="TAG $uc $mac" name="TAG $uc $mac" min="1" max="4095" value="$zone_vlan_id" $field_disabled> + <input type="number" class="vlanid" id="TAG $uc $mac" name="TAG $uc $mac" min="1" max="4095" value="$zone_vlan_id" $field_disabled> </td> END ; -- 2.17.1

3 3

[PATCH 1/5] BUG12156: changed wio.cgi to fix broken Web GUI
by sfeddersen 05 Sep '19

05 Sep '19

--- src/wio/wio.cgi | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/wio/wio.cgi b/src/wio/wio.cgi index c51910f4d..1645aa54e 100644 --- a/src/wio/wio.cgi +++ b/src/wio/wio.cgi @@ -1504,7 +1504,7 @@ my $dotip = length($ipaddresses[$a]) - rindex($ipaddresses[$a],'.'); $ic = "${_}"; foreach $interface (@devs_net) { - next if ( $netsettings{"$ic"."_DEV"} eq 'red0' && ($netsettings{"RED_TYPE"} eq 'DHCP' || $netsettings{"RED_TYPE"} eq 'PPPOE')); + next if ( $netsettings{"$ic"."_DEV"} eq 'red0' && $netsettings{"RED_TYPE"} eq 'PPPOE'); if ( $netsettings{"$ic"."_DEV"} eq $interface ) { if ( &General::IpInSubnet($ipaddresses[$a], $netsettings{"$ic"."_NETADDRESS"}, $netsettings{"$ic"."_NETMASK"}) ) { print"<td align='center' height='20'><img src='$imgstatic/$devs_img[$in]' alt='$Lang::tr{$devs_alt[$in]}' title='$Lang::tr{$devs_alt[$in]}' /></td>"; @@ -1558,7 +1558,7 @@ my $dotip = length($ipaddresses[$a]) - rindex($ipaddresses[$a],'.'); } } - if ($netsettings{"RED_TYPE"} eq 'DHCP' || $netsettings{"RED_TYPE"} eq 'PPPOE') { + if ( $netsettings{"RED_TYPE"} eq 'PPPOE' ) { my $redipadr = qx'ip addr | grep red0 | grep inet | awk "{print \$2}"'; my @rednet = split ("/", $redipadr); chomp ($rednet[1]); @@ -1569,10 +1569,10 @@ my $dotip = length($ipaddresses[$a]) - rindex($ipaddresses[$a],'.'); print"<td align='center' height='20'><img src='$imgstatic/red.png' alt='$Lang::tr{'wio_red_lan'}' title='$Lang::tr{'wio_red_lan'}' /></td>"; last SWITCH; } - else { + } + else { print"<td align='center'><img align='middle' src='$imgstatic/white.png' alt='$Lang::tr{'wio_unknown_lan'}' title='$Lang::tr{'wio_unknown_lan'}' /></td>"; last SWITCH; - } } } @@ -1872,7 +1872,7 @@ SWITCH: { } } - if ($netsettings{"RED_TYPE"} eq 'DHCP' || $netsettings{"RED_TYPE"} eq 'PPPOE') { + if ($netsettings{"RED_TYPE"} eq 'DHCP' || $netsettings{"RED_TYPE"} eq 'PPPOE' || $netsettings{"RED_TYPE"} eq 'STATIC') { my $redipadr = qx'ip addr | grep red0 | grep inet | awk "{print \$2}"'; my @rednet = split ("/", $redipadr); chomp ($rednet[1]); -- 2.17.1

2 5

[PATCH] haproxy: Update to 2.0.5
by Michael Tremer 03 Sep '19

03 Sep '19

This release brings a couple of new features and this patch enables using the PCRE Jit. Signed-off-by: Michael Tremer <michael.tremer(a)ipfire.org> --- lfs/haproxy | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/lfs/haproxy b/lfs/haproxy index 8917d30c9..308d96edf 100644 --- a/lfs/haproxy +++ b/lfs/haproxy @@ -24,7 +24,7 @@ include Config -VER = 1.8.21 +VER = 2.0.5 THISAPP = haproxy-$(VER) DL_FILE = $(THISAPP).tar.gz @@ -32,7 +32,7 @@ DL_FROM = $(URL_IPFIRE) DIR_APP = $(DIR_SRC)/$(THISAPP) TARGET = $(DIR_INFO)/$(THISAPP) PROG = haproxy -PAK_VER = 11 +PAK_VER = 12 DEPS = "" @@ -48,7 +48,7 @@ objects = $(DL_FILE) $(DL_FILE) = $(DL_FROM)/$(DL_FILE) -$(DL_FILE)_MD5 = 6f86f1272c7daf5ea1d1bae5c9192183 +$(DL_FILE)_MD5 = 497c716adf4b056484601a887f34d152 install : $(TARGET) @@ -82,8 +82,8 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) @$(PREBUILD) @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE) - cd $(DIR_APP) && make $(MAKETUNING) CPU="generic" TARGET="linux2628" \ - USE_OPENSSL=1 USE_PCRE=1 USE_ZLIB=1 USE_REGPARM=1 \ + cd $(DIR_APP) && make $(MAKETUNING) CPU="generic" TARGET="linux-glibc" \ + USE_OPENSSL=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_ZLIB=1 USE_REGPARM=1 \ ADDINC="$(CFLAGS)" USE_LINUX_TPROXY=1 ADDLIB="$(LDFLAGS)" cd $(DIR_APP) && make install-bin install-man PREFIX=/usr -- 2.20.1

1 0

Suricata causes massive packet loss
by peter.mueller＠ipfire.org 03 Sep '19

03 Sep '19

Dear OSIF/Suricata users, earlier this year, the linux-based open source firewall distribution IPFire has migrated from Snort to Suricata for a number of reasons (further information is available at https://blog.ipfire.org/post/introducing-ipfire-s-new-intrusion-prevention-…). While we are quite pleased with some of its features (multi-threading, ability to monitor several interfaces per process, etc.), we experienced some problems ever since we are running it. Not being reproducible everywhere, we initially thought they were corner cases in obscure network scenarios. Ultimately, they were not. Even worse, no dropped packets were logged although we can tell for sure there were some. For example, several IPFire users - including myself - report very slow DNS resolution when trying to access a website, while "normal" lookups using dig or host commands perform fine. Another issue is reduced OpenVPN tunnel throughput, which seems to be caused by massive packet loss when Suricata is enabled (~ 800 kB/s, ~ 2 MB/s if Suricata is turned off). In order to get closer to its origin, we spend a lot of time on testing and debugging, eventually left without any idea what the solution might be. Both issues - possibly being related to each other - can be reproduced using Suricata 4.1.4 without any rules or packet decoders enabled. Unfortunately, our setup, where Suricata runs inline via Netfilter queue, does not seem to be documented very well. That's why I am asking here if anybody is able to tell us what we are doing wrong. Perhaps this just might be a configuration problem, but we are out of ideas where to look for it. Please find our suricata.yaml (decoders enabled, but disabling it does not matter) and the stats.log file enclosed. Details regarding a testing machine: > [root@maverick ~]# suricata -V > This is Suricata version 4.1.4 RELEASE > [root@maverick ~]# uname -a > Linux maverick 4.14.138-ipfire #1 SMP Sat Aug 10 00:53:30 GMT 2019 x86_64 Intel(R) Celeron(R) CPU N3150 @ 1.60GHz GenuineIntel GNU/Linux > [root@maverick ~]# ldd /usr/bin/suricata > linux-vdso.so.1 (0x00007ffdd77df000) > libdl.so.2 => /lib/libdl.so.2 (0x00007ba2d3e7f000) > librt.so.1 => /lib/librt.so.1 (0x00007ba2d3e75000) > libm.so.6 => /lib/libm.so.6 (0x00007ba2d3d26000) > libmagic.so.1 => /usr/lib/libmagic.so.1 (0x00007ba2d3afc000) > libcap-ng.so.0 => /usr/lib/libcap-ng.so.0 (0x00007ba2d38f6000) > libpcap.so.1 => /usr/lib/libpcap.so.1 (0x00007ba2d36b5000) > libnet.so.1 => /usr/lib/libnet.so.1 (0x00007ba2d3498000) > libnetfilter_queue.so.1 => /usr/lib/libnetfilter_queue.so.1 (0x00007ba2d3291000) > libnfnetlink.so.0 => /usr/lib/libnfnetlink.so.0 (0x00007ba2d308a000) > libjansson.so.4 => /usr/lib/libjansson.so.4 (0x00007ba2d307b000) > libpthread.so.0 => /lib/libpthread.so.0 (0x00007ba2d305a000) > libyaml-0.so.2 => /usr/lib/libyaml-0.so.2 (0x00007ba2d2e3c000) > libpcre.so.1 => /usr/lib/libpcre.so.1 (0x00007ba2d2bc7000) > liblzma.so.5 => /usr/lib/liblzma.so.5 (0x00007ba2d29a0000) > libhs.so.5 => /usr/lib/libhs.so.5 (0x00007ba2d221a000) > libhtp.so.2 => /usr/lib/libhtp.so.2 (0x00007ba2d1ff2000) > libc.so.6 => /lib/libc.so.6 (0x00007ba2d1e0d000) > /lib64/ld-linux-x86-64.so.2 (0x00007ba2d3e8f000) > libmnl.so.0 => /usr/lib/libmnl.so.0 (0x00007ba2d1c07000) > libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007ba2d1a6c000) > libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007ba2d1855000) > [root@maverick ~]# suricata --build-info > This is Suricata version 4.1.4 RELEASE > Features: NFQ PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_LIBJANSSON TLS MAGIC > SIMD support: none > Atomic intrisics: 1 2 4 8 byte(s) > 64-bits, Little-endian architecture > GCC version 8.3.0, C version 199901 > compiled with _FORTIFY_SOURCE=2 > L1 cache line size (CLS)=64 > thread local storage method: __thread > compiled with LibHTP v0.5.28, linked against LibHTP v0.5.28 > > Suricata Configuration: > AF_PACKET support: yes > eBPF support: no > XDP support: no > PF_RING support: no > NFQueue support: yes > NFLOG support: no > IPFW support: no > Netmap support: no > DAG enabled: no > Napatech enabled: no > WinDivert enabled: no > > Unix socket enabled: yes > Detection enabled: yes > > Libmagic support: yes > libnss support: no > libnspr support: no > libjansson support: yes > liblzma support: yes > hiredis support: no > hiredis async with libevent: no > Prelude support: no > PCRE jit: yes > LUA support: no > libluajit: no > libgeoip: no > Non-bundled htp: yes > Old barnyard2 support: no > Hyperscan support: yes > Libnet support: yes > liblz4 support: no > > Rust support: no > Rust strict mode: no > Rust debug mode: no > Rust compiler: not set > Rust cargo: not set > > Install suricatasc: no > Install suricata-update: no > > Profiling enabled: no > Profiling locks enabled: no > > Development settings: > Coccinelle / spatch: no > Unit tests enabled: no > Debug output enabled: no > Debug validation enabled: no > > Generic build parameters: > Installation prefix: /usr > Configuration directory: /etc/suricata/ > Log directory: /var/log/suricata/ > > --prefix /usr > --sysconfdir /etc > --localstatedir /var > --datarootdir /usr/share > > Host: x86_64-pc-linux-gnu > Compiler: gcc (exec name) / gcc (real) > GCC Protect enabled: yes > GCC march native enabled: no > GCC Profile enabled: no > Position Independent Executable enabled: no > CFLAGS -O2 -pipe -Wall -fexceptions -fPIC -m64 -mindirect-branch=thunk -mfunction-return=thunk -mtune=generic -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong > PCAP_CFLAGS -I/usr/include > SECCFLAGS -fstack-protector -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security Please let me know if further information is needed. Any help is highly appreciated. Thanks, and best regards, Peter Müller

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

Development September 2019