Hi,
..I'd like to have *two* problems with the IDS/IPS... ;-)
1. After upgrading to Core164 yesterday I took a look at the IDS GUI today and found it locked with a spinning wheel at "Ruleset update in progress. Please wait until all operations have completed successfully..."
Despite this message IPS seemed to work.
So I searched and found that the file "/tmp/ids_page_locked" still existed. After deleting this file manually the web GUI opened, everything seems normal.
Three activated rulesets were running, see attachment "01".
But the Web GUI stopped because of lines 234ff in 'ids.cgi':
... # Check if the page is locked, in this case, the ids_page_lock_file exists. if (-e $IDS::ids_page_lock_file) { # Lock the webpage and print notice about autoupgrade of the ruleset # is in progess. &working_notice("$Lang::tr{'ids ruleset autoupdate in progress'}");
# Loop and check if the file still exists. while(-e $IDS::ids_page_lock_file) { # Sleep for a second and re-check. sleep 1; } ...
I searched again and found that there are two subroutines in '/var/ipfire/ids-functions.pl', lines 1920ff, which are responsible for locking and unlocking this file:
... ## Function to write the lock file for locking the WUI, while ## the autoupdate script runs. # sub lock_ids_page() { # Call subfunction to create the file. &create_empty_file($ids_page_lock_file); }
# ## Function to release the lock of the WUI, again. # sub unlock_ids_page() { # Delete lock file. unlink($ids_page_lock_file); } ...
Somehow lines 103ff in "/usr/local/bin/update-ids-ruleset" didn't work/weren't executed(?):
... # Lock the IDS page. &IDS::lock_ids_page();
# The script has requested a lock, so set locket to "1". $locked = "1";
# Grab the configured providers. &General::readhasharray("$IDS::providers_settings_file", %providers);
# Loop through the array of available providers. foreach my $id (keys %providers) { # Assign some nice variabled. my $provider = $providers{$id}[0]; my $autoupdate_status = $providers{$id}[3];
# Skip the provider if autoupdate is not enabled. next unless($autoupdate_status eq "enabled");
# Call the download function and gather the new ruleset for the current processed provider. if(&IDS::downloadruleset($provider)) { # Store error message for displaying in the WUI. &IDS::_store_error_message("$provider: $Lang::tr{'could not download latest updates'}");
# Unlock the IDS page. &IDS::unlock_ids_page();
# Exit. exit 0; } ...
Manual updates - yesterday - were ok, but the automatic updates tonight didn't unlink the lock file: IDS GUI stopped at the next call today.
##########
2. There still exist a directory '/tmp/isd_tmp' containing 'conf' ('config' and 'map' files from emerging-threats) and a 'rules'-dir containing all '*.rules'-files. Leftovers!? Can I safely delete these dirs?
And: can anyone confirm these two problems - or reproduce?
Best, Matthias
Hello Mathias,
thanks for reporting your issue here.
The IDS page locking issue should be fixed and shipped with the final C164 release.
See: https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=85b1d83b2a6fe2beb8169f3e...
Please report if you are affected again by this issue.
You safely can drop those folder and files from "/tmp".
Best regards,
-Stefan
Hi,
..I'd like to have *two* problems with the IDS/IPS... ;-)
- After upgrading to Core164 yesterday I took a look at the IDS GUI
today and found it locked with a spinning wheel at "Ruleset update in progress. Please wait until all operations have completed successfully..."
Despite this message IPS seemed to work.
So I searched and found that the file "/tmp/ids_page_locked" still existed. After deleting this file manually the web GUI opened, everything seems normal.
Three activated rulesets were running, see attachment "01".
But the Web GUI stopped because of lines 234ff in 'ids.cgi':
... # Check if the page is locked, in this case, the ids_page_lock_file exists. if (-e $IDS::ids_page_lock_file) { # Lock the webpage and print notice about autoupgrade of the ruleset # is in progess. &working_notice("$Lang::tr{'ids ruleset autoupdate in progress'}");
# Loop and check if the file still exists. while(-e $IDS::ids_page_lock_file) { # Sleep for a second and re-check. sleep 1; } ...
I searched again and found that there are two subroutines in '/var/ipfire/ids-functions.pl', lines 1920ff, which are responsible for locking and unlocking this file:
... ## Function to write the lock file for locking the WUI, while ## the autoupdate script runs. # sub lock_ids_page() { # Call subfunction to create the file. &create_empty_file($ids_page_lock_file); }
# ## Function to release the lock of the WUI, again. # sub unlock_ids_page() { # Delete lock file. unlink($ids_page_lock_file); } ...
Somehow lines 103ff in "/usr/local/bin/update-ids-ruleset" didn't work/weren't executed(?):
... # Lock the IDS page. &IDS::lock_ids_page();
# The script has requested a lock, so set locket to "1". $locked = "1";
# Grab the configured providers. &General::readhasharray("$IDS::providers_settings_file", %providers);
# Loop through the array of available providers. foreach my $id (keys %providers) { # Assign some nice variabled. my $provider = $providers{$id}[0]; my $autoupdate_status = $providers{$id}[3];
# Skip the provider if autoupdate is not enabled. next unless($autoupdate_status eq "enabled");
# Call the download function and gather the new ruleset for the current processed provider. if(&IDS::downloadruleset($provider)) { # Store error message for displaying in the WUI. &IDS::_store_error_message("$provider: $Lang::tr{'could not download latest updates'}");
# Unlock the IDS page. &IDS::unlock_ids_page();
# Exit. exit 0; } ...
Manual updates - yesterday - were ok, but the automatic updates tonight didn't unlink the lock file: IDS GUI stopped at the next call today.
##########
- There still exist a directory '/tmp/isd_tmp' containing 'conf'
('config' and 'map' files from emerging-threats) and a 'rules'-dir containing all '*.rules'-files. Leftovers!? Can I safely delete these dirs?
And: can anyone confirm these two problems - or reproduce?
Best, Matthias
On 14.03.2022 06:08, Stefan Schantl wrote:
Hello Mathias,
Hi Stefan,
thanks for reporting your issue here.
No problem - I should have taken a closer look: 'update-ids-ruleset' seems to be missing in the Core update for Core164, I was still using the old version from Core163. ;-)
See: https://git.ipfire.org/?p=ipfire-2.x.git;a=tree;f=config/rootfiles/core/164/...
The IDS page locking issue should be fixed and shipped with the final C164 release.
See: https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=85b1d83b2a6fe2beb8169f3e...
Please report if you are affected again by this issue.
I'll test and report.
Best, Matthias
You safely can drop those folder and files from "/tmp".
Best regards,
-Stefan
Hi,
..I'd like to have *two* problems with the IDS/IPS... ;-)
- After upgrading to Core164 yesterday I took a look at the IDS GUI
today and found it locked with a spinning wheel at "Ruleset update in progress. Please wait until all operations have completed successfully..."
Despite this message IPS seemed to work.
So I searched and found that the file "/tmp/ids_page_locked" still existed. After deleting this file manually the web GUI opened, everything seems normal.
Three activated rulesets were running, see attachment "01".
But the Web GUI stopped because of lines 234ff in 'ids.cgi':
... # Check if the page is locked, in this case, the ids_page_lock_file exists. if (-e $IDS::ids_page_lock_file) { # Lock the webpage and print notice about autoupgrade of the ruleset # is in progess. &working_notice("$Lang::tr{'ids ruleset autoupdate in progress'}");
# Loop and check if the file still exists. while(-e $IDS::ids_page_lock_file) { # Sleep for a second and re-check. sleep 1; } ...
I searched again and found that there are two subroutines in '/var/ipfire/ids-functions.pl', lines 1920ff, which are responsible for locking and unlocking this file:
... ## Function to write the lock file for locking the WUI, while ## the autoupdate script runs. # sub lock_ids_page() { # Call subfunction to create the file. &create_empty_file($ids_page_lock_file); }
# ## Function to release the lock of the WUI, again. # sub unlock_ids_page() { # Delete lock file. unlink($ids_page_lock_file); } ...
Somehow lines 103ff in "/usr/local/bin/update-ids-ruleset" didn't work/weren't executed(?):
... # Lock the IDS page. &IDS::lock_ids_page();
# The script has requested a lock, so set locket to "1". $locked = "1";
# Grab the configured providers. &General::readhasharray("$IDS::providers_settings_file", %providers);
# Loop through the array of available providers. foreach my $id (keys %providers) { # Assign some nice variabled. my $provider = $providers{$id}[0]; my $autoupdate_status = $providers{$id}[3];
# Skip the provider if autoupdate is not enabled. next unless($autoupdate_status eq "enabled");
# Call the download function and gather the new ruleset for the current processed provider. if(&IDS::downloadruleset($provider)) { # Store error message for displaying in the WUI. &IDS::_store_error_message("$provider: $Lang::tr{'could not download latest updates'}");
# Unlock the IDS page. &IDS::unlock_ids_page();
# Exit. exit 0; } ...
Manual updates - yesterday - were ok, but the automatic updates tonight didn't unlink the lock file: IDS GUI stopped at the next call today.
##########
- There still exist a directory '/tmp/isd_tmp' containing 'conf'
('config' and 'map' files from emerging-threats) and a 'rules'-dir containing all '*.rules'-files. Leftovers!? Can I safely delete these dirs?
And: can anyone confirm these two problems - or reproduce?
Best, Matthias
Hi,
after updating the old '/usr/local/bin/update-ids-ruleset' to one from Core164, I manually started "Force ruleset update" through IDS GUI for one of my lists.
Result: GUI changes to "Downloading and unpacking new ruleset. Please wait until all operations have completed successfully...". Then it stays there forever until I reload it - which works after some time.
But in '/var/tmp' there are remains of a 'ids_tmp'-directory, containing a 'conf' and a 'rules' directory. These seem to contain the updated, unpacked config- and rule-files(?). For me it looks as if in '/var/lib/suricata' there are still the *old* rule files.
'var/lib/suricata' and '/tmp/ids_tmp/rules' contain identically named *.rules-files, but the rules-files in '/var/lib/suricata' start with "drop tcp ..." and the ones in '/var/ids_tmp/rules' with "alert tcp ...".
And 'suricata' isn't restarted, either. No update entries in the logs.
Besides that, the GUI shows the updated date and time stamps from the downloaded archives in '/var/tmp'. At first look, everything seems ok, but it isn't. Weird...
On 14.03.2022 18:04, Matthias Fischer wrote:
On 14.03.2022 06:08, Stefan Schantl wrote:
Hello Mathias,
Hi Stefan,
thanks for reporting your issue here.
No problem - I should have taken a closer look: 'update-ids-ruleset' seems to be missing in the Core update for Core164, I was still using the old version from Core163. ;-)
See: https://git.ipfire.org/?p=ipfire-2.x.git;a=tree;f=config/rootfiles/core/164/...
The IDS page locking issue should be fixed and shipped with the final C164 release.
See: https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=85b1d83b2a6fe2beb8169f3e...
Please report if you are affected again by this issue.
I'll test and report.
Best, Matthias
You safely can drop those folder and files from "/tmp".
Best regards,
-Stefan
Hi,
..I'd like to have *two* problems with the IDS/IPS... ;-)
- After upgrading to Core164 yesterday I took a look at the IDS GUI
today and found it locked with a spinning wheel at "Ruleset update in progress. Please wait until all operations have completed successfully..."
Despite this message IPS seemed to work.
So I searched and found that the file "/tmp/ids_page_locked" still existed. After deleting this file manually the web GUI opened, everything seems normal.
Three activated rulesets were running, see attachment "01".
But the Web GUI stopped because of lines 234ff in 'ids.cgi':
... # Check if the page is locked, in this case, the ids_page_lock_file exists. if (-e $IDS::ids_page_lock_file) { # Lock the webpage and print notice about autoupgrade of the ruleset # is in progess. &working_notice("$Lang::tr{'ids ruleset autoupdate in progress'}");
# Loop and check if the file still exists. while(-e $IDS::ids_page_lock_file) { # Sleep for a second and re-check. sleep 1; } ...
I searched again and found that there are two subroutines in '/var/ipfire/ids-functions.pl', lines 1920ff, which are responsible for locking and unlocking this file:
... ## Function to write the lock file for locking the WUI, while ## the autoupdate script runs. # sub lock_ids_page() { # Call subfunction to create the file. &create_empty_file($ids_page_lock_file); }
# ## Function to release the lock of the WUI, again. # sub unlock_ids_page() { # Delete lock file. unlink($ids_page_lock_file); } ...
Somehow lines 103ff in "/usr/local/bin/update-ids-ruleset" didn't work/weren't executed(?):
... # Lock the IDS page. &IDS::lock_ids_page();
# The script has requested a lock, so set locket to "1". $locked = "1";
# Grab the configured providers. &General::readhasharray("$IDS::providers_settings_file", %providers);
# Loop through the array of available providers. foreach my $id (keys %providers) { # Assign some nice variabled. my $provider = $providers{$id}[0]; my $autoupdate_status = $providers{$id}[3];
# Skip the provider if autoupdate is not enabled. next unless($autoupdate_status eq "enabled");
# Call the download function and gather the new ruleset for the current processed provider. if(&IDS::downloadruleset($provider)) { # Store error message for displaying in the WUI. &IDS::_store_error_message("$provider: $Lang::tr{'could not download latest updates'}");
# Unlock the IDS page. &IDS::unlock_ids_page();
# Exit. exit 0; } ...
Manual updates - yesterday - were ok, but the automatic updates tonight didn't unlink the lock file: IDS GUI stopped at the next call today.
##########
- There still exist a directory '/tmp/isd_tmp' containing 'conf'
('config' and 'map' files from emerging-threats) and a 'rules'-dir containing all '*.rules'-files. Leftovers!? Can I safely delete these dirs?
And: can anyone confirm these two problems - or reproduce?
Best, Matthias
Hi,
I wanted to report that after updating 'update-ids-ruleset' the problems are gone:
- GUI doesn't hang anymore. - After the daily rules update, '/tmp' doesn't contain any leftovers. - Reloading the rules did work and was logged properly.
Just in case someone wanted to know... ;-)
Best, Matthias
On 14.03.2022 19:01, Matthias Fischer wrote:
Hi,
after updating the old '/usr/local/bin/update-ids-ruleset' to one from Core164, I manually started "Force ruleset update" through IDS GUI for one of my lists.
Result: GUI changes to "Downloading and unpacking new ruleset. Please wait until all operations have completed successfully...". Then it stays there forever until I reload it - which works after some time.
But in '/var/tmp' there are remains of a 'ids_tmp'-directory, containing a 'conf' and a 'rules' directory. These seem to contain the updated, unpacked config- and rule-files(?). For me it looks as if in '/var/lib/suricata' there are still the *old* rule files.
'var/lib/suricata' and '/tmp/ids_tmp/rules' contain identically named *.rules-files, but the rules-files in '/var/lib/suricata' start with "drop tcp ..." and the ones in '/var/ids_tmp/rules' with "alert tcp ...".
And 'suricata' isn't restarted, either. No update entries in the logs.
Besides that, the GUI shows the updated date and time stamps from the downloaded archives in '/var/tmp'. At first look, everything seems ok, but it isn't. Weird...
On 14.03.2022 18:04, Matthias Fischer wrote:
On 14.03.2022 06:08, Stefan Schantl wrote:
Hello Mathias,
Hi Stefan,
thanks for reporting your issue here.
No problem - I should have taken a closer look: 'update-ids-ruleset' seems to be missing in the Core update for Core164, I was still using the old version from Core163. ;-)
See: https://git.ipfire.org/?p=ipfire-2.x.git;a=tree;f=config/rootfiles/core/164/...
The IDS page locking issue should be fixed and shipped with the final C164 release.
See: https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=85b1d83b2a6fe2beb8169f3e...
Please report if you are affected again by this issue.
I'll test and report.
Best, Matthias
You safely can drop those folder and files from "/tmp".
Best regards,
-Stefan
Hi,
..I'd like to have *two* problems with the IDS/IPS... ;-)
- After upgrading to Core164 yesterday I took a look at the IDS GUI
today and found it locked with a spinning wheel at "Ruleset update in progress. Please wait until all operations have completed successfully..."
Despite this message IPS seemed to work.
So I searched and found that the file "/tmp/ids_page_locked" still existed. After deleting this file manually the web GUI opened, everything seems normal.
Three activated rulesets were running, see attachment "01".
But the Web GUI stopped because of lines 234ff in 'ids.cgi':
... # Check if the page is locked, in this case, the ids_page_lock_file exists. if (-e $IDS::ids_page_lock_file) { # Lock the webpage and print notice about autoupgrade of the ruleset # is in progess. &working_notice("$Lang::tr{'ids ruleset autoupdate in progress'}");
# Loop and check if the file still exists. while(-e $IDS::ids_page_lock_file) { # Sleep for a second and re-check. sleep 1; } ...
I searched again and found that there are two subroutines in '/var/ipfire/ids-functions.pl', lines 1920ff, which are responsible for locking and unlocking this file:
... ## Function to write the lock file for locking the WUI, while ## the autoupdate script runs. # sub lock_ids_page() { # Call subfunction to create the file. &create_empty_file($ids_page_lock_file); }
# ## Function to release the lock of the WUI, again. # sub unlock_ids_page() { # Delete lock file. unlink($ids_page_lock_file); } ...
Somehow lines 103ff in "/usr/local/bin/update-ids-ruleset" didn't work/weren't executed(?):
... # Lock the IDS page. &IDS::lock_ids_page();
# The script has requested a lock, so set locket to "1". $locked = "1";
# Grab the configured providers. &General::readhasharray("$IDS::providers_settings_file", %providers);
# Loop through the array of available providers. foreach my $id (keys %providers) { # Assign some nice variabled. my $provider = $providers{$id}[0]; my $autoupdate_status = $providers{$id}[3];
# Skip the provider if autoupdate is not enabled. next unless($autoupdate_status eq "enabled");
# Call the download function and gather the new ruleset for the current processed provider. if(&IDS::downloadruleset($provider)) { # Store error message for displaying in the WUI. &IDS::_store_error_message("$provider: $Lang::tr{'could not download latest updates'}");
# Unlock the IDS page. &IDS::unlock_ids_page();
# Exit. exit 0; } ...
Manual updates - yesterday - were ok, but the automatic updates tonight didn't unlink the lock file: IDS GUI stopped at the next call today.
##########
- There still exist a directory '/tmp/isd_tmp' containing 'conf'
('config' and 'map' files from emerging-threats) and a 'rules'-dir containing all '*.rules'-files. Leftovers!? Can I safely delete these dirs?
And: can anyone confirm these two problems - or reproduce?
Best, Matthias
-
Matthias Fischer -
Stefan Schantl