From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.haj.ipfire.org (localhost [IPv6:::1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4hB10y0Mgzz36Vr for ; Thu, 30 Jul 2026 20:25:38 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519) (Client CN "mail01.haj.ipfire.org", Issuer "YR2" (not verified)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4hB10r59g1z2xqt for ; Thu, 30 Jul 2026 20:25:32 +0000 (UTC) Received: from layka.disroot.org (layka.disroot.org [178.21.23.139]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519) (Client did not present a certificate) by mail01.ipfire.org (Postfix) with ESMTPS id 4hB10g694tznl for ; Thu, 30 Jul 2026 20:25:23 +0000 (UTC) Authentication-Results: mail01.ipfire.org; dkim=pass header.d=disroot.org header.s=mail header.b=Qua2wtQz; spf=pass (mail01.ipfire.org: domain of robin.roevens@disroot.org designates 178.21.23.139 as permitted sender) smtp.mailfrom=robin.roevens@disroot.org; dmarc=pass (policy=reject) header.from=disroot.org ARC-Seal: i=1; a=rsa-sha256; d=lists.ipfire.org; s=202003rsa; cv=none; t=1785443123; b=e6Ai6KNo3AtOpvkXov/i0zYRP9UR89o+COmzsjYh5DKVpgAON6RZPiNQURaciuxYhKDQtX H5vXkmQSQltS0PMIaT1nIeC2ccWciWtG2O/KC1kNXEwcj9Bc++xT3Rnt0RvjdlXXCJlBcC CI5PYvQoEbcQ0I1CdJL/2YvJ9jm8p/mlEOnPl/+aCNgVcOr02X0W+Xk2VUvHOx1BGwwgk4 hxaV7MBenJ3THl/p0gjvwft29Q/TLD8pG+Gqr+f0SZgmn6VjG34lu4qkFfP93/xBlZCRxL 8tUCwTsNb7P0BDWLwURfDqxcJg9k/caHpQlvtOk8BbUqqSmPB8SjyYX4eUtwyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=lists.ipfire.org; s=202003rsa; t=1785443123; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=X+A41N3ys0ZMmxPEIv0OUwGaBtaT7amRcvrRBAQ+UXk=; b=gz8yEqlsuqi1xhZeZYKIjMATJirlP3n+bBtMeLoeIRJQjGBtP1uCaqYdso/vGCWOOgObFt uTpb6mAKtrRaCzxNK6LnOFDoPD7mGUBZ8lFzVGsvGnuwvTIXtMMAvPhIBWUkfQUdyxpWqk kzSIVZ3KnOYfC0ftKqWPcDwplTG03+JV8mYI6ixpQpffuhJXiJfYidBExiRhiowCEdljRD eZBahPRgmjkzmnIBJE6mCJpAyqi7CeJzOCCNrP83aeGMPxPCcgtOnA5IJm0stJ7nzJmRZO wSUSiXeBWaJk7DtheYZG1Kor2fh+mzGIMmW0YVid9msKP19Z80St+2XEi6ZcKQ== ARC-Authentication-Results: i=1; mail01.ipfire.org; dkim=pass header.d=disroot.org header.s=mail header.b=Qua2wtQz; spf=pass (mail01.ipfire.org: domain of robin.roevens@disroot.org designates 178.21.23.139 as permitted sender) smtp.mailfrom=robin.roevens@disroot.org; dmarc=pass (policy=reject) header.from=disroot.org Received: from mail01.layka.lan (localhost [127.0.0.1]) by disroot.org (Postfix) with ESMTP id 851F541B98 for ; Thu, 30 Jul 2026 22:25:23 +0200 (CEST) X-Virus-Scanned: SPAM Filter at disroot.org Received: from layka.disroot.org ([127.0.0.1]) by localhost (disroot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id dAs4fT99y8P5 for ; Thu, 30 Jul 2026 22:25:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=disroot.org; s=mail; t=1785443122; bh=X+A41N3ys0ZMmxPEIv0OUwGaBtaT7amRcvrRBAQ+UXk=; h=Subject:From:To:Date:In-Reply-To:References; b=Qua2wtQzk0rAK+EjT5ks7c/zb9Im0zzp+/Auoc0cWdpq1SlOvTloNabAvABNNdVhR JH4/Wz+pss0QXivdFiZ1vkJKnB7MPwxTPRz9uDQ33fQ75Ic4HdqTj7jQ2Q35B02DfD OpitBUqR8+iaALTgttrYpYeync0NX+mztisH9nQnOcZmrNrI0X08fcTyntIXOjRhmp MKFPNLfyl3uItBH45GfwuCj8XX4enesPWfc3fw5vsfKvonYHvMS9NKmeWiKCSiqlof vc1ty+QpjNP26E30i9H0ZhLEz3cfk91i6PWGTlo4BZAUBlQYL3Pvi8ZlbALQXG92ca sADU2e0G+TUqg== Received: from chojin.roevenslambrechts.be (chojin.roevenslambrechts.be [192.168.0.50]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (no client certificate requested) (Authenticated sender) by hachiman (MailScanner Milter) with SMTP id 243C9586132 for ; Thu, 30 Jul 2026 22:25:17 +0200 (CEST) Message-ID: <2a1ee7a806c0ad40cbf7683edf667b998428def4.camel@disroot.org> Subject: Re: [PATCH 0/5] Add Zabbix functionality to suricata-reporter From: Robin Roevens To: development@lists.ipfire.org Date: Thu, 30 Jul 2026 22:25:16 +0200 In-Reply-To: <20260730195148.3278295-1-robin.roevens@disroot.org> References: <20260730195148.3278295-1-robin.roevens@disroot.org> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.60.2 Precedence: list List-Id: List-Subscribe: , List-Unsubscribe: , List-Post: List-Help: Sender: Mail-Followup-To: MIME-Version: 1.0 X-RoevensLambrechts-MailScanner-ID: 243C9586132.A4A0D X-RoevensLambrechts-MailScanner: Found to be clean X-RoevensLambrechts-MailScanner-From: robin.roevens@disroot.org X-RoevensLambrechts-MailScanner-Watermark: 1786047921.02893@swNdRPECrkJkkp7qh764VQ X-Rspamd-Server: mail01.haj.ipfire.org X-Rspamd-Queue-Id: 4hB10g694tznl X-Rspamd-Action: no action X-Spamd-Result: default: False [-7.13 / 11.00]; BAYES_HAM(-3.00)[100.00%]; R_DKIM_ALLOW(-1.65)[disroot.org:s=mail]; DKIM_REPUTATION(-0.92)[-0.92157203836436]; SPF_REPUTATION_HAM(-0.66)[-0.6568552202107]; DMARC_POLICY_ALLOW(-0.50)[disroot.org,reject]; R_SPF_ALLOW(-0.20)[+a:c]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.10)[disroot.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; IP_REPUTATION_HAM(0.00)[asn: 50673(0.00), country: NL(-0.01), ip: 178.21.23.139(0.00)]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[disroot.org:+]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; PREVIOUSLY_DELIVERED(0.00)[development@lists.ipfire.org]; ASN(0.00)[asn:50673, ipnet:178.21.23.0/24, country:NL]; MID_RHS_MATCH_FROM(0.00)[]; ARC_SIGNED(0.00)[lists.ipfire.org:s=202003rsa:i=1] Small correction to my explanation, the statement I made about sending older events from the time when zabbix functionality is not yet enabled, is not true, as when zabbix functionality is disabled, events recorded in the database are not marked as pending, so they won't be sent to zabbix when that functionality is enabled on a later time. It does however give the user the implicit functionality of augmenting the max_age in case zabbix server was unreachable for longer than current max_age to recover events it missed due to previous max_age setting. Regards Robin Robin Roevens schreef op do 30-07-2026 om 21:15 [+0200]: > Hi all, >=20 > As discussed here earlier, I've worked on implementing sending > Suricata alerts straight to Zabbix from within suricata-reporter > instead > of trying to parse the suricata logging separately using the Zabbix > agent. >=20 > For this I use the zabbix-utils python library, which I submited here > also as a separate pak (but meanwhile already requires an update, > which > I will post soon). This set of patches makes suricata-reporter able > to > directly communicate to a Zabbix server without having the > zabbix_agentd > pak installed, sending suricata alerts in real-time. >=20 > As Zabbix supports sending items in bulk, I have opted to create an > async background task that will send all events from last 1 second in > bulk so that even in the case that there are hundreds of incoming > alerts, Zabbix server is only contacted once per second. >=20 > When for some reason sending to Zabbix server fails, it will be > retried > 3 times and then the background task will be suspended until a new > suricata event comes in. That will wake the task again and retry to > send all > pending events. In environments with many events, that may actually > not > have that much of an effect. But in the average environment, this > will > give the Zabbix Server some breathing space as it failing to receive > our > events, may indicate a Zabbix server overload.=20 >=20 > For this I have to keep track which events are sent and which are > pending. So I added a column in the database that keeps track of > that. >=20 > I have also added an alert_max_age config parameter that allows the > user > to set how long suricata-reporter should retry to send events to > Zabbix. > Events older than that set age, will no longer be sent to Zabbix. > This also give the user the implicit option to send older events when > only just enabling the zabbix sending functionality, since the DB > column > exists and no event was ever sent to Zabbix, all events will be > 'pending". At first run with zabbix functionality enabled, all events > up > to alert_max_age that are in the database will be sent to zabbix > immediatly. >=20 > All events sent to Zabbix contain the timestamp of retrieval by > suricata-reporter, so Zabbix will register and order them as received > on that > timestamp independently of the actual time Zabbix itself received the > event. >=20 > This is my first adventure in Python async programming, so I hope I > did > not make any flagrant mistakes. But the code has been running here > for > weeks now without any problem. I have not actually tested large > bursts > of events, as I could not simulate that.. But I did make Zabbix > server > slow, unavailable and finally replaced it with netcat (to accept the > connection, but > not react on it) and I had the connection with the server off for a > few > hours to then re-establish the connection to see hundereds of pending > events=20 > being registered in only a few milliseconds. > I did not notice any problems with suricata-reporter in any of these > cases. >=20 > Regards >=20 > Robin --=20 Dit bericht is gescanned op virussen en andere gevaarlijke inhoud door MailScanner en lijkt schoon te zijn.