Re: Testing report regarding image: next-suricata-rust/7808a8a2-dirty

[Stefan Schantl <stefan.schantl@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 8636 bytes --]

I've forgot to post the link to the ISO:

https://people.ipfire.org/~stevee/IPFire_2.x_suricata_rust/ipfire-2.23.x86_64-full-core136.iso

Best regards,

-Stefan
> Hello Peter, Hello Victor, hello *,
> 
> @Victor, thanks for pointing to the bug at your bugtracker.
> 
> https://redmine.openinfosecfoundation.org/issues/2806#note-19
> 
> I think we are affected exactly by this issue.
> 
> @Peter, thanks for testing the image and sharing your feedback.
> 
> Regarding to Victor's link to redmine, I've built and uploaded a
> new image which contains both kernel fixes and the latest netfilter
> updates which have been sent to our development mailing list.
> 
> Please test once more if your DNS  and VPN issues still are existant.
> 
> Best regards,
> 
> -Stefan
> 
> peter.mueller(a)ipfire.org:
> > Hello Stefan, hello Peter, hello *,
> > 
> > @Stefan: Thank you again for building the ISO with Suricata 5.0.0-
> > beta1,
> > Rust and current libhtp.
> > 
> > @Peter: Sorry for having not answered your question: The problem is
> > not
> > only related to DNS traffic, but to new connections in general (no
> > matter
> > if they are encrypted or plain text) - see below for details.
> > 
> > Initially, Suricata refuses to start:
> > > Sep  8 11:59:43 maverick suricata: [ERRCODE:
> > > SC_WARN_NO_STATS_LOGGERS(261)] - stats are enabled but no loggers
> > > are active
> > > Sep  8 11:59:43 maverick suricata: [ERRCODE: SC_ERR_NFQ_OPEN(68)]
> > > -
> > > no queue for given index
> > > Sep  8 11:59:43 maverick suricata: [ERRCODE:
> > > SC_ERR_NFQ_THREAD_INIT(78)] - nfq thread failed to initialize
> > 
> > Although statistics are disabled in /etc/suricata/suricata.yaml ,
> > enabling
> > the statistics logger was necessary. Perhaps a glitch in the beta
> > version,
> > as the statistics log file is empty.
> > 
> > Instead of passing the NFQ indices one by one ("-q 0 -q 1 -q 2 -q
> > 3"),
> > Suricata now likes them as a range: "-q 0:3" After changing this in
> > the
> > initscript and deleting orphaned PID file, Suricata starts
> > correctly.
> > 
> > Initialisation procedure takes 87 seconds on my testing hardware,
> > which
> > is approximately two times faster compared to Suricata 4.x. Rule
> > parsing
> > works, all tested attacks were successfully detected.
> > 
> > Resource consumption of Suricata 5.x is a bit lower compared to 4.x
> > .
> > 
> > Unfortunately, both of my problems can be reproduced with that
> > image:
> > (a) Poor OpenVPN throughput.
> >     This has improved a bit to 1.2 MB/sec peak, but still is lower
> > than
> >     the 2.1 MB/sec I observe on another productive machine.
> > (b) Establishing connections and DNS resolutions takes age
> >     Regardless of SSH, HTTP, HTTPS, SMTPS or IMAPS, establishing a
> > new
> >     connection takes 1-2 seconds due to massive packet loss.
> > Resolving
> >     DNS records using "dig" or "host" is fast, but using "wget" or
> > "curl"
> >     is slow.
> > 
> > Increasing memory allocations or "max-pending-packets" did not help
> > again.
> > That being said, I think we can be more generous regarding our
> > memory
> > allocations, as most RAM of my testing hardware stayed unallocated.
> > :-)
> > 
> > As Eric Leblond already mentioned on the OISF mailing list, the
> > actual
> > problem seems to be something else (netfilter/iptables/?).
> > 
> > Version commands for reference:
> > 
> > > [root(a)maverick ~]# suricata -V
> > > This is Suricata version 5.0.0-beta1 RELEASE
> > > [root(a)maverick ~]# uname -a
> > > Linux maverick 4.14.138-ipfire #1 SMP Sat Sep 7 06:27:36 GMT 2019
> > > x86_64 Intel(R) Celeron(R) CPU N3150 @ 1.60GHz GenuineIntel
> > > GNU/Linux
> > > [root(a)maverick ~]# suricata --build-info
> > > This is Suricata version 5.0.0-beta1 RELEASE
> > > Features: NFQ PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT
> > > LIBCAP_NG
> > > LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_LIBJANSSON
> > > TLS
> > > MAGIC RUST 
> > > SIMD support: none
> > > Atomic intrisics: 1 2 4 8 byte(s)
> > > 64-bits, Little-endian architecture
> > > GCC version 8.3.0, C version 199901
> > > compiled with _FORTIFY_SOURCE=2
> > > L1 cache line size (CLS)=64
> > > thread local storage method: __thread
> > > compiled with LibHTP v0.5.30, linked against LibHTP v0.5.30
> > > 
> > > Suricata Configuration:
> > >   AF_PACKET support:                       yes
> > >   eBPF support:                            no
> > >   XDP support:                             no
> > >   PF_RING support:                         no
> > >   NFQueue support:                         yes
> > >   NFLOG support:                           no
> > >   IPFW support:                            no
> > >   Netmap support:                          no 
> > >   DAG enabled:                             no
> > >   Napatech enabled:                        no
> > >   WinDivert enabled:                       no
> > > 
> > >   Unix socket enabled:                     yes
> > >   Detection enabled:                       yes
> > > 
> > >   Libmagic support:                        yes
> > >   libnss support:                          yes
> > >   libnspr support:                         yes
> > >   libjansson support:                      yes
> > >   liblzma support:                         yes
> > >   hiredis support:                         no
> > >   hiredis async with libevent:             no
> > >   Prelude support:                         no
> > >   PCRE jit:                                yes
> > >   LUA support:                             no
> > >   libluajit:                               no
> > >   libgeoip:                                no
> > >   Non-bundled htp:                         yes
> > >   Old barnyard2 support:                   no
> > >   Hyperscan support:                       yes
> > >   Libnet support:                          yes
> > >   liblz4 support:                          no
> > > 
> > >   Rust support:                            yes
> > >   Rust strict mode:                        no
> > >   Rust debug mode:                         no
> > >   Rust compiler:                           rustc 1.37.0
> > > (eae3437df
> > > 2019-08-13)
> > >   Rust cargo:                              cargo 1.37.0
> > > (9edd08916
> > > 2019-08-02)
> > > 
> > >   Python support:                          no
> > >   Python path:                             not set
> > >   Python version:                          not set
> > >   Python distutils                         no
> > >   Python yaml                              no
> > >   Install suricatactl:                     requires python
> > >   Install suricatasc:                      requires python
> > >   Install suricata-update:                 not bundled
> > > 
> > >   Profiling enabled:                       no
> > >   Profiling locks enabled:                 no
> > > 
> > > Development settings:
> > >   Coccinelle / spatch:                     no
> > >   Unit tests enabled:                      no
> > >   Debug output enabled:                    no
> > >   Debug validation enabled:                no
> > > 
> > > Generic build parameters:
> > >   Installation prefix:                     /usr
> > >   Configuration directory:                 /etc/suricata/
> > >   Log directory:                           /var/log/suricata/
> > > 
> > >   --prefix                                 /usr
> > >   --sysconfdir                             /etc
> > >   --localstatedir                          /var
> > >   --datarootdir                            /usr/share
> > > 
> > >   Host:                                    x86_64-pc-linux-gnu
> > >   Compiler:                                gcc (exec name) / gcc
> > > (real)
> > >   GCC Protect enabled:                     yes
> > >   GCC march native enabled:                no
> > >   GCC Profile enabled:                     no
> > >   Position Independent Executable enabled: no
> > >   CFLAGS                                   -O2 -pipe -Wall
> > > -fexceptions -fPIC -m64 -mindirect-branch=thunk -mfunction-
> > > return=thunk -mtune=generic -Wp,-D_FORTIFY_SOURCE=2 -Wp,-
> > > D_GLIBCXX_ASSERTIONS -fstack-protector-strong
> > > -I${srcdir}/../rust/gen/c-headers
> > >   PCAP_CFLAGS                               -I/usr/include
> > >   SECCFLAGS                                -fstack-protector
> > > -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security
> > 
> > Thanks, and best regards,
> > Peter Müller