From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.haj.ipfire.org (localhost [IPv6:::1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4cf2Rx2p1Vz2xph for ; Sat, 04 Oct 2025 10:52:45 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail01.haj.ipfire.org", Issuer "R13" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4cf2Rs6tbNz2xM3 for ; Sat, 04 Oct 2025 10:52:41 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail01.ipfire.org (Postfix) with ESMTPSA id 4cf2Rr6Q85zBY; Sat, 04 Oct 2025 10:52:40 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1759575161; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2VnscMUwsthLaGkKYwgqNewT62+jRLknFTfSqrAq2eo=; b=8/2hDZVyKdx//QqFJ19FXJMMIQZoeoK5H8g0RXmP0j5ZuD4gyX8lLKoq9TLig/A7D3jYBJ 2mQMJqzn+sanwSAw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1759575161; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2VnscMUwsthLaGkKYwgqNewT62+jRLknFTfSqrAq2eo=; b=vgU1UM4Q08RgJcBoSztNWp1wY2HKf25vVmG+ZRJqL+N0QhZaFLuF5ZbSNgSpGrIh8gWOJu h4iVBbGcIxYwd7YgTJthGcc9pJ7b6ky9WsZAap6Tv9/Og9CGIB8ohonfrvQf6zwuxy4VS5 PsuJZd18M4Q3+Pu98hT1FGTBNlxmaIjbSgGoRNq/ds3eT/WXxGKY7hPopXUvq1wQHvlDzm yV77eP8GfOl5LUmyV5gNHqpAbV3QZ/OkjUqoZJVu22qIkSC24g4Va2sjQDNlpk0DbM6xDY 4zhHhxmnNOg+nd67LtFgyEdekfof1n4wodktctedAv0OUs9DaWVW0tbsN40tmQ== Content-Type: text/plain; charset=utf-8 Precedence: list List-Id: List-Subscribe: , List-Unsubscribe: , List-Post: List-Help: Sender: Mail-Followup-To: Mime-Version: 1.0 Subject: Re: Zabbix support for suricata-reporter From: Michael Tremer In-Reply-To: <56ad0ff847bb0b5b5808084321ac52efd63cd16f.camel@disroot.org> Date: Sat, 4 Oct 2025 11:52:40 +0100 Cc: "IPFire: Development-List" Content-Transfer-Encoding: quoted-printable Message-Id: <708978AA-034A-4533-BF40-257A843AFF8E@ipfire.org> References: <56ad0ff847bb0b5b5808084321ac52efd63cd16f.camel@disroot.org> To: Robin Roevens Hello Robin, > On 2 Oct 2025, at 22:04, Robin Roevens = wrote: >=20 > Hi Michael >=20 > I saw there is a suricata-reporter in the upcoming CU. And I was > wondering if I could add an additional reporter into it for sending > alerts straight to Zabbix, next to syslog and email. I think that is a great idea! > I have already been experimenting with parsing fast.log using the > zabbix_agentd, which seems to work quite well. But since there is now = a > reporter, it would be nice to have it support sending alerts to zabbix > directly instead of zabbix separately monitoring the fast.log file. Parsing log files like that is never fun. I also think that I remember = that suricata has deprecated support for writing fast.log, but then it = does not seem to be deprecated any more. Looking at the alternative EVE = format, fast.log is just there to have something that is somewhat = human-readable, but at the same time does not carry a lot of = information. > If that would be ok for you. There are 2 possible ways to do this: > - using the zabbix_utils python library: > https://blog.zabbix.com/python-zabbix-utils/27056/ > - or using the zabbix_sender command utility that currently gets > installed when installing zabbix_agentd The IPS could generate a lot of alerts. Especially when there is bursts = in (malicious) traffic and I am already unhappy with calling sendmail - = but there I have no other choice. The daemon initially had some worker daemons so that reading from = Suricata would be entirely decoupled from processing any messages. = However, Python=E2=80=99s multiprocessing capabilities are really bad = and so now it is a threaded implementation. Python isn=E2=80=99t good at = that either, but when we are calling a subprocess the GIL is being = released and so we should not actually spend too much time in the Python = code itself. This has to be considered when adding yet another output so that the = process does not get stalled when Zabbix does not respond, is = overwhelmed or anything else. We could simply combat that by adding a = couple more worker threads if necessary - but I am getting ahead of = myself. So the Python module would be by far preferable. It seems to have a nice = API, but we will have to check how it works underneath to reach the = objectives outlined above. > I assume, using the python library will probably be the most = performant > option; But then I should also create a zabbix_utils python library > pak-file? Hmm, this is where it is getting interesting. The suricata-reporter stuff is supposed to be rather distro agnostic. = Besides Python itself, it only has a dependency to sqlite3 (which = normally comes with the Python standard library) and reportlab to = generate the PDFs. The goal is to run it on IPFire 2 for now, but it = would also be pretty much ready as is to run in IPFire 3 some day. Of = course it runs on any other distro - although I don=E2=80=99t know what = it would do on a non-Linux OS. I would like to keep it lightweight and hopefully some more people would = adopt it. I believe adding zabbix should be done as an optional dependency. We = could simply try to import the Zabbix modules. If they are there, the = process has support for Zabbix. If not, then not. That could simply be = handled at runtime. That way, we could have zabbix as an add-on in IPFire instead of adding = it to the core distribution and if other people want to run the whole = thing, they can enjoy a minimum amount of dependencies and don=E2=80=99t = have to build/install any zabbix packages if they don=E2=80=99t need = them. > Both the python module and the commandline cli have the possibility to > get zabbix server connection info from the zabbix_agentd configfile so > config of the reporter would be something like: > [zabbix] > enabled =3D true > zabbix_agentd_config =3D /etc/zabbix_agentd/zabbix_agentd.conf > alert_item_key =3D ipfire.suricata.event.get I mean, this is all up to you. I would definitely do this bit: [zabbix] enabled =3D true That way it is consistent with the remainder. Any other options, well I = don=E2=80=99t know what you would need :) I do enjoy a daemon that does = not need a lot of configuration though and therefore can be run with = only a very small configuration as it relies on sensible defaults. > Then the reporter can format the incoming suricata alert/event as json > and send it to the configured alert_item_key on the zabbix server as > configured in the zabbix_agentd.conf We do store the entire JSON object as it comes from Suricata in the = sqlite3 database. That way we have all the information that could = possible be available to us even though we don=E2=80=99t need everything = right now. The fast.log format definitely lacks a lot of important = information, but it is nicely readable. Depending on what processing Zabbix can do later and what information it = permanently-ish stores, I would send the whole lot. The JSON objects are = small enough and over the wire they would compress nicely, too. On the = daemon side I wouldn=E2=80=99t implement this to be configurable because = that only makes everything more complicated. > Is this something you are open to? Then I can try to create a patch = for > suricata-reporter. (where should I then submit it? Also on this list?) Yes, very much so. Patches can go on this list. Let=E2=80=99s talk about any implementation details in advance. I feel a = little bit precious about this program because it is nice, shiny and new = and the code is very simple. I would like to keep it that way. > If not I will have to continue working on the fast.log parsing. Don=E2=80=99t. That will only make you unhappy. > And while on the topic of monitoring suricata; I would like to get = some > extra stats from it, which, for as far as I currently know, can be > retrieved using the suricata unix-socket that is currently disabled by > default on ipfire. Many seem to use a 'suricatasc' tool to query > suricata using that socket, but that tool is not available on ipfire. What stats do you want/need? I am happy to turn on the socket and built the tool. Does it come with = the suricata package? Please submit patches :) I think this could be done first before diving = into Zabbix because it should be a very simple change. > Is it possible to have it on ipfire?, or should I start experimenting > using socat? No, if there are tools built already, use the code that is available. > And if succesful, is it then allowed for a future zabbix_agentd addon > pak to enable that socket in the suricata config? Enable the socket by default. It won=E2=80=99t hurt anyone. > If you dislike the idea of enabling and querying the socket, another > possibility is having suricata dump stats in a seperate stats.log = which > I should then be able to parse using Zabbix. Hmm, but then we are back to parsing logs. That is never fun. > Before I start any implementations, what are your thoughts about all > this ? Very very happy with this here. Let me know how I can help. -Michael > Regards > Robin >=20 > --=20 > Dit bericht is gescanned op virussen en andere gevaarlijke > inhoud door MailScanner en lijkt schoon te zijn. >=20