From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.haj.ipfire.org (localhost [IPv6:::1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4dVRNz5Wrlz30VZ for ; Mon, 15 Dec 2025 17:09:07 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [IPv6:2001:678:b28::25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail01.haj.ipfire.org", Issuer "R12" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4dVRNv69dDz2xkP for ; Mon, 15 Dec 2025 17:09:03 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519) (No client certificate requested) by mail01.ipfire.org (Postfix) with ESMTPSA id 4dVRNt1rCbz2LH; Mon, 15 Dec 2025 17:09:02 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1765818542; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mOZS0fA1EUMqBEAsEOvT7f0uo9w8LPTe7eyAb/8VERM=; b=8nqdn3t4F9Tn2i7KZx9s9dLGMm+hLRvYaPzt+RMoYZ/Cum1lsBqNxa2+ALJVwS1ForD6nl bcUd8tZCLl1ZiGBQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1765818542; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mOZS0fA1EUMqBEAsEOvT7f0uo9w8LPTe7eyAb/8VERM=; b=kqZmHJ7+s33j1w4PuYzHzBSS18akBgFz+j1A5HMNnaU7ynCXkFWWXGFg90Rh7PR31+S2Ji qiRGxub+AGdQNg1v+WMfUsSGIxRe1KEicQxfh3ErSNoXUttNMZfMaea6ylNK0iniJQM5FR Kco0UarIK+q0mOwuFuy39jkMoodictnAfgfpcUvUNen/HT/4EXH3MKm6vjMETWQ2X+0hSc 2yeisVt5Pb8Pcg0eIx2uKhI+DYte0AZnaMR3HrOgV6gA6qv9OzAUxDrK/JmUQQffrzamc8 hgb6CRhwrU83tLUOU5KEdZ/LavYi0zWSTjx/LmPl95tU5HSyvQ+UdVPYrE2KbQ== Message-ID: <335eb24d-230d-45fa-ae97-11c520c46445@ipfire.org> Date: Mon, 15 Dec 2025 18:09:02 +0100 Precedence: list List-Id: List-Subscribe: , List-Unsubscribe: , List-Post: List-Help: Sender: Mail-Followup-To: MIME-Version: 1.0 Subject: Re: Large Suricata cache directory. To: Michael Tremer , Adam Gibbons References: <8ac70e7aa3d72cf2de5cda09a52f1cf6@ipfire.org> <8BFE2D8F-E49A-4823-8795-0A6B54D6A1B5@ipfire.org> Content-Language: en-GB Cc: "IPFire: Development-List" From: Adolf Belka In-Reply-To: <8BFE2D8F-E49A-4823-8795-0A6B54D6A1B5@ipfire.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi Michael & Adam, On 15/12/2025 17:54, Michael Tremer wrote: > Hello Adam, > > Thank you for raising this here. > > We seem to have two different issues as far as I can see: > > 1) The directory just keeps growing > > 2) It is being backed up and completely blows up the size of the backup > > No. 2 has been fixed by Adolf. It is however quite interesting that we already have something in /var/cache in the backup. The intention was to have a valid set of rules available as soon as a backup is being restored, but I think there is very little value in this. The rules are probably long expires and will be re-downloaded again. I did wonder about why those cached rules tarballs are backed up but wasn't sure if there was a reasonable explanation I was not aware of. Hence my patch only dealt with the new issue of the sgh cache. However, I don't have a problem with removing the whole suricata cache directory from the backup include list. > > We also have a large number of other (also large in disk size) lists around that we are not backing up, so I would propose that we remove /var/cache/suricata from the backup entirely. Until we have made a decision on this, I have merged Adolf’s patch. > > Regarding No. 1, this is indeed a problem that Suricata does not clean this up itself. We could add a simple command that looks a bit like this: > > find /var/cache/suricata/ -type f -atime +7 -delete > > This would delete all of the cached files that have not been accessed in the last seven days. > > On my system, the entire directory is 1.4 GiB in size and the command would remove 500 MiB. > > Happy to read your thoughts. Seems a reasonable thing to do. We just need to remember to remove it again when suricata issue the fix to clean out the cache. The current plan is to have it released in 8.0.3 and 9.0.0-beta1 It was originally planned for 8.0.2 but this got changed to 8.0.3 as the fix was still marked as "In Progress". As of 6 days ago the status of the fix was changed to "In Review" so hopefully it will be available relatively quickly. However there is no estimate provided of when 8.0.3 should get released. Regards, Adolf. > > -Michael > >> On 12 Dec 2025, at 16:49, Adam Gibbons wrote: >> >> Hi all, >> >> As discussed on the forum >> https://community.ipfire.org/t/re-large-backupfile/15346 >> it appears that Suricata’s new cache optimisation feature is creating a large number of files under >> `/var/cache/suricata/sgh/`, which in some cases causes backup files to grow to 800+ MB. >> >> @Adolf has confirmed that this directory probably should not be included in backups, as it is automatically regenerated, and I believe he mentioned he is working on a patch to exclude it from the backup. >> >> However, in the meantime, this directory continues to grow over time. The upstream Suricata patches to automatically clean or maintain the cache have not yet been merged, although they may be soon: >> >> https://github.com/OISF/suricata/pull/13850 >> https://github.com/OISF/suricata/pull/14400 >> >> To me this represents a disk-space exhaustion risk on systems with limited storage. Perhaps we should consider disabling Suricata’s new cache optimisation feature until automatic cache cleanup/maintenance is available upstream and included. >> >> Thanks, >> Adam >> > >