Hi,

I wanted to report that after updating 'update-ids-ruleset' the problems
are gone:

- GUI doesn't hang anymore.
- After the daily rules update, '/tmp' doesn't contain any leftovers.
- Reloading the rules did work and was logged properly.

Just in case someone wanted to know... ;-)

Best,
Matthias

On 14.03.2022 19:01, Matthias Fischer wrote:
> Hi,
> 
> after updating the old '/usr/local/bin/update-ids-ruleset' to one from
> Core164, I manually started "Force ruleset update" through IDS GUI for
> one of my lists.
> 
> Result:
> GUI changes to "Downloading and unpacking new ruleset. Please wait until
> all operations have completed successfully...". Then it stays there
> forever until I reload it - which works after some time.
> 
> But in '/var/tmp' there are remains of a 'ids_tmp'-directory, containing
> a 'conf' and a 'rules' directory. These seem to contain the updated,
> unpacked config- and rule-files(?). For me it looks as if in
> '/var/lib/suricata' there are still the *old* rule files.
> 
> 'var/lib/suricata' and '/tmp/ids_tmp/rules' contain identically named
> *.rules-files, but the rules-files in '/var/lib/suricata' start with
> "drop tcp ..." and the ones in '/var/ids_tmp/rules' with "alert tcp ...".
> 
> And 'suricata' isn't restarted, either. No update entries in the logs.
> 
> Besides that, the GUI shows the updated date and time stamps from the
> downloaded archives in '/var/tmp'. At first look, everything seems ok,
> but it isn't. Weird...
> 
> On 14.03.2022 18:04, Matthias Fischer wrote:
>> On 14.03.2022 06:08, Stefan Schantl wrote:
>>> Hello Mathias,
>> 
>> Hi Stefan,
>> 
>>> thanks for reporting your issue here.
>> 
>> No problem - I should have taken a closer look: 'update-ids-ruleset'
>> seems to be missing in the Core update for Core164, I was still using
>> the old version from Core163. ;-)
>> 
>> See:
>> https://git.ipfire.org/?p=ipfire-2.x.git;a=tree;f=config/rootfiles/core/164/filelists;h=c81a06294cd3527b9b68d1352d487bf1c8f95c9e;hb=refs/heads/core164
>> 
>>> The IDS page locking issue should be fixed and shipped with the final
>>> C164 release.
>>> 
>>> See:
>>> https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=85b1d83b2a6fe2beb8169f3e810e915c4ad54036
>>> 
>>> Please report if you are affected again by this issue.
>> 
>> I'll test and report.
>> 
>> Best,
>> Matthias
>> 
>>> You safely can drop those folder and files from "/tmp".
>>> 
>>> Best regards,
>>> 
>>> -Stefan
>>>> Hi,
>>>> 
>>>> ..I'd like to have *two* problems with the IDS/IPS... ;-)
>>>> 
>>>> 
>>>> 1. After upgrading to Core164 yesterday I took a look at the IDS GUI
>>>> today and found it locked with a spinning wheel at "Ruleset update in
>>>> progress. Please wait until all operations have completed
>>>> successfully..."
>>>> 
>>>> Despite this message IPS seemed to work.
>>>> 
>>>> So I searched and found that the file "/tmp/ids_page_locked" still
>>>> existed. After deleting this file manually the web GUI opened,
>>>> everything seems normal.
>>>> 
>>>> Three activated rulesets were running, see attachment "01".
>>>> 
>>>> But the Web GUI stopped because of lines 234ff in 'ids.cgi':
>>>> 
>>>> ...
>>>> # Check if the page is locked, in this case, the ids_page_lock_file
>>>> exists.
>>>> if (-e $IDS::ids_page_lock_file) {
>>>>         # Lock the webpage and print notice about autoupgrade of the
>>>> ruleset
>>>>         # is in progess.
>>>>         &working_notice("$Lang::tr{'ids ruleset autoupdate in
>>>> progress'}");
>>>> 
>>>>         # Loop and check if the file still exists.
>>>>         while(-e $IDS::ids_page_lock_file) {
>>>>                 # Sleep for a second and re-check.
>>>>                 sleep 1;
>>>>         }
>>>> ...
>>>> 
>>>> I searched again and found that there are two subroutines in
>>>> '/var/ipfire/ids-functions.pl', lines 1920ff, which are responsible
>>>> for
>>>> locking and unlocking this file:
>>>> 
>>>> ...
>>>> ## Function to write the lock file for locking the WUI, while
>>>> ## the autoupdate script runs.
>>>> #
>>>> sub lock_ids_page() {
>>>>         # Call subfunction to create the file.
>>>>         &create_empty_file($ids_page_lock_file);
>>>> }
>>>> 
>>>> #
>>>> ## Function to release the lock of the WUI, again.
>>>> #
>>>> sub unlock_ids_page() {
>>>>         # Delete lock file.
>>>>         unlink($ids_page_lock_file);
>>>> }
>>>> ...
>>>> 
>>>> Somehow lines 103ff in "/usr/local/bin/update-ids-ruleset" didn't
>>>> work/weren't executed(?):
>>>> 
>>>> ...
>>>> # Lock the IDS page.
>>>> &IDS::lock_ids_page();
>>>> 
>>>> # The script has requested a lock, so set locket to "1".
>>>> $locked = "1";
>>>> 
>>>> # Grab the configured providers.
>>>> &General::readhasharray("$IDS::providers_settings_file",
>>>> \%providers);
>>>> 
>>>> # Loop through the array of available providers.
>>>> foreach my $id (keys %providers) {
>>>>    # Assign some nice variabled.
>>>>    my $provider = $providers{$id}[0];
>>>>    my $autoupdate_status = $providers{$id}[3];
>>>> 
>>>>    # Skip the provider if autoupdate is not enabled.
>>>>    next unless($autoupdate_status eq "enabled");
>>>> 
>>>>    # Call the download function and gather the new ruleset for the
>>>> current processed provider.
>>>>    if(&IDS::downloadruleset($provider)) {
>>>>       # Store error message for displaying in the WUI.
>>>>       &IDS::_store_error_message("$provider: $Lang::tr{'could not
>>>> download latest updates'}");
>>>> 
>>>>       # Unlock the IDS page.
>>>>       &IDS::unlock_ids_page();
>>>> 
>>>>       # Exit.
>>>>       exit 0;
>>>>    }
>>>> ...
>>>> 
>>>> Manual updates - yesterday - were ok, but the automatic updates
>>>> tonight
>>>> didn't unlink the lock file: IDS GUI stopped at the next call today.
>>>> 
>>>> ##########
>>>> 
>>>> 2. There still exist a directory '/tmp/isd_tmp' containing 'conf'
>>>> ('config' and 'map' files from emerging-threats) and a 'rules'-dir
>>>> containing all '*.rules'-files. Leftovers!? Can I safely delete these
>>>> dirs?
>>>> 
>>>> And: can anyone confirm these two problems - or reproduce?
>>>> 
>>>> Best,
>>>> Matthias
>>> 
>>> 
>> 
>