From mboxrd@z Thu Jan  1 00:00:00 1970
From: Stefan Schantl <stefan.schantl@ipfire.org>
To: development@lists.ipfire.org
Subject: Re: [Oisf-users] Suricata causes massive packet loss
Date: Sat, 07 Sep 2019 14:19:31 +0200
Message-ID: <ea5d6ed6cf97e022f4d212a6bc6959f744192dbd.camel@ipfire.org>
In-Reply-To: =?utf-8?q?=3CCY1PR04MB2268A9A19841B0CCBCCD9AE5D7BB0=40CY1PR04MB?=
 =?utf-8?q?2268=2Enamprd04=2Eprod=2Eoutlook=2Ecom=3E?=
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============2300209487675767604=="
List-Id: <development.lists.ipfire.org>

--===============2300209487675767604==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Hello Peter, Hello Coop, Hello *,
> Hi again Peter,
>=20
> I am absolutely not recommending shipping a beta release!  Rather,
> I'm suggesting you try it to see if it's an issue that has already
> been resolved as a sanity check.

I've build and uploaded an custom install-able ISO image, containing
the latest libhtp (0.5.30) and suricata 5.0.0-beta1.=20

@Peter, please test and report if any of these changes affects your
packet lost rate.
>  =20
>=20
> Have you tried using libpcap instead of AF_PACKET as the capture
> mechanism?

Currently we are using IPtables to delegate the packets via netfilter
queue to suricata and to drop or re-inject them after scanning.


Best regards,

-Stefan
>  =20
>=20
> -Coop
>=20
> -----Original Message-----
> From: peter.mueller(a)ipfire.org <peter.mueller(a)ipfire.org>=20
> Sent: Thursday, September 5, 2019 11:56 AM
> To: Nelson, Cooper <cnelson(a)ucsd.edu>; Peter Manev <
> petermanev(a)gmail.com>
> Cc: oisf-users(a)lists.openinfosecfoundation.org; IPFire: Development-
> List <development(a)lists.ipfire.org>; Stefan Schantl <
> stefan.schantl(a)ipfire.org>
> Subject: Re: [Oisf-users] Suricata causes massive packet loss
>=20
> Hello Nelson, hello Peter, hello *,
>=20
> thank you for your replies.
>=20
> Upgrading to Suricata 5.0-beta is a difficult task, as we cannot
> simply ship beta releases in our firewall distribution. Personally, I
> rather doubt this is an issue due to a kernel/library/...
> combination, as we use Suricata for quite a while now and are
> upgrading IPFire's distribution kernel on a regular basis.
>=20
> Anyway, Stefan (see CC) is currently working on Rust for the
> distribution, so we hope to take advantage of some more features
> soon. But since our issue is regarding packet loss for at least DNS
> and TLS traffic, I rather doubt Rust will make a big difference here.
>=20
> Changing from "workers" to "autofp" mode unfortunately did not solve
> the problem. It is good to know the latter is recommended for inline
> deployments, "workers" was about 0.5 % faster in our benchmarks.
>=20
> In IPFire, Suricata is started by a custom init script (please refer
> to=20
> https://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dblob;f=3Dsrc/initscripts/sys=
tem/suricata;h=3D5a567f2d7f4bfef90fabb11438bc5065e731f21c;hb=3DHEAD
> for its content) and appears like this in the process list:
> > [root(a)maverick ~]# ps aux | grep suricata
> > suricata  4882 10.9  7.3 1419868 289192 ?      Ssl  20:38   1:37
> > /usr/bin/suricata -c /etc/suricata/suricata.yaml -D -q 0 -q 1 -q 2
> > -q 3
>=20
> I am not sure what the number behind "tcp.pkt_on_wrong_thread" should
> read like normally. @Peter: Is it too low or too high?
>=20
> We will ship an update for libhtp as soon as possible, thank you for
> catching this.
>=20
> Thanks, and best regards,
> Peter M=C3=BCller


--===============2300209487675767604==--