Re: [Oisf-users] Suricata causes massive packet loss

[Stefan Schantl <stefan.schantl@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 3046 bytes --]

Hello Peter, Hello Coop, Hello *,
> Hi again Peter,
> 
> I am absolutely not recommending shipping a beta release!  Rather,
> I'm suggesting you try it to see if it's an issue that has already
> been resolved as a sanity check.

I've build and uploaded an custom install-able ISO image, containing
the latest libhtp (0.5.30) and suricata 5.0.0-beta1.

https://people.ipfire.org/~stevee/IPFire_2.x_suricata_rust/ipfire-2.23.x86_64-full-core136.iso 

@Peter, please test and report if any of these changes affects your
packet lost rate.
>   
> 
> Have you tried using libpcap instead of AF_PACKET as the capture
> mechanism?

Currently we are using IPtables to delegate the packets via netfilter
queue to suricata and to drop or re-inject them after scanning.


Best regards,

-Stefan

PS: Sorry for re-sending, I lacked to subscribe to the oisf-users
mailing list...
>   
> 
> -Coop
> 
> -----Original Message-----
> From: peter.mueller(a)ipfire.org <peter.mueller(a)ipfire.org> 
> Sent: Thursday, September 5, 2019 11:56 AM
> To: Nelson, Cooper <cnelson(a)ucsd.edu>; Peter Manev <
> petermanev(a)gmail.com>
> Cc: oisf-users(a)lists.openinfosecfoundation.org; IPFire: Development-
> List <development(a)lists.ipfire.org>; Stefan Schantl <
> stefan.schantl(a)ipfire.org>
> Subject: Re: [Oisf-users] Suricata causes massive packet loss
> 
> Hello Nelson, hello Peter, hello *,
> 
> thank you for your replies.
> 
> Upgrading to Suricata 5.0-beta is a difficult task, as we cannot
> simply ship beta releases in our firewall distribution. Personally, I
> rather doubt this is an issue due to a kernel/library/...
> combination, as we use Suricata for quite a while now and are
> upgrading IPFire's distribution kernel on a regular basis.
> 
> Anyway, Stefan (see CC) is currently working on Rust for the
> distribution, so we hope to take advantage of some more features
> soon. But since our issue is regarding packet loss for at least DNS
> and TLS traffic, I rather doubt Rust will make a big difference here.
> 
> Changing from "workers" to "autofp" mode unfortunately did not solve
> the problem. It is good to know the latter is recommended for inline
> deployments, "workers" was about 0.5 % faster in our benchmarks.
> 
> In IPFire, Suricata is started by a custom init script (please refer
> to 
> https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=src/initscripts/system/suricata;h=5a567f2d7f4bfef90fabb11438bc5065e731f21c;hb=HEAD
> for its content) and appears like this in the process list:
> > [root(a)maverick ~]# ps aux | grep suricata
> > suricata  4882 10.9  7.3 1419868 289192 ?      Ssl  20:38   1:37
> > /usr/bin/suricata -c /etc/suricata/suricata.yaml -D -q 0 -q 1 -q 2
> > -q 3
> 
> I am not sure what the number behind "tcp.pkt_on_wrong_thread" should
> read like normally. @Peter: Is it too low or too high?
> 
> We will ship an update for libhtp as soon as possible, thank you for
> catching this.
> 
> Thanks, and best regards,
> Peter Müller