From mboxrd@z Thu Jan 1 00:00:00 1970 From: Matthias Fischer To: development@lists.ipfire.org Subject: Re: New IDS won't unlink "/tmp/ids_page_locked" Date: Wed, 16 Mar 2022 17:19:13 +0100 Message-ID: <9089558d-1204-44ef-7a8e-7743170e9892@ipfire.org> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4677092723796630771==" List-Id: --===============4677092723796630771== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi, I wanted to report that after updating 'update-ids-ruleset' the problems are gone: - GUI doesn't hang anymore. - After the daily rules update, '/tmp' doesn't contain any leftovers. - Reloading the rules did work and was logged properly. Just in case someone wanted to know... ;-) Best, Matthias On 14.03.2022 19:01, Matthias Fischer wrote: > Hi, >=20 > after updating the old '/usr/local/bin/update-ids-ruleset' to one from > Core164, I manually started "Force ruleset update" through IDS GUI for > one of my lists. >=20 > Result: > GUI changes to "Downloading and unpacking new ruleset. Please wait until > all operations have completed successfully...". Then it stays there > forever until I reload it - which works after some time. >=20 > But in '/var/tmp' there are remains of a 'ids_tmp'-directory, containing > a 'conf' and a 'rules' directory. These seem to contain the updated, > unpacked config- and rule-files(?). For me it looks as if in > '/var/lib/suricata' there are still the *old* rule files. >=20 > 'var/lib/suricata' and '/tmp/ids_tmp/rules' contain identically named > *.rules-files, but the rules-files in '/var/lib/suricata' start with > "drop tcp ..." and the ones in '/var/ids_tmp/rules' with "alert tcp ...". >=20 > And 'suricata' isn't restarted, either. No update entries in the logs. >=20 > Besides that, the GUI shows the updated date and time stamps from the > downloaded archives in '/var/tmp'. At first look, everything seems ok, > but it isn't. Weird... >=20 > On 14.03.2022 18:04, Matthias Fischer wrote: >> On 14.03.2022 06:08, Stefan Schantl wrote: >>> Hello Mathias, >>=20 >> Hi Stefan, >>=20 >>> thanks for reporting your issue here. >>=20 >> No problem - I should have taken a closer look: 'update-ids-ruleset' >> seems to be missing in the Core update for Core164, I was still using >> the old version from Core163. ;-) >>=20 >> See: >> https://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dtree;f=3Dconfig/rootfiles/c= ore/164/filelists;h=3Dc81a06294cd3527b9b68d1352d487bf1c8f95c9e;hb=3Drefs/head= s/core164 >>=20 >>> The IDS page locking issue should be fixed and shipped with the final >>> C164 release. >>>=20 >>> See: >>> https://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dcommit;h=3D85b1d83b2a6fe2b= eb8169f3e810e915c4ad54036 >>>=20 >>> Please report if you are affected again by this issue. >>=20 >> I'll test and report. >>=20 >> Best, >> Matthias >>=20 >>> You safely can drop those folder and files from "/tmp". >>>=20 >>> Best regards, >>>=20 >>> -Stefan >>>> Hi, >>>>=20 >>>> ..I'd like to have *two* problems with the IDS/IPS... ;-) >>>>=20 >>>>=20 >>>> 1. After upgrading to Core164 yesterday I took a look at the IDS GUI >>>> today and found it locked with a spinning wheel at "Ruleset update in >>>> progress. Please wait until all operations have completed >>>> successfully..." >>>>=20 >>>> Despite this message IPS seemed to work. >>>>=20 >>>> So I searched and found that the file "/tmp/ids_page_locked" still >>>> existed. After deleting this file manually the web GUI opened, >>>> everything seems normal. >>>>=20 >>>> Three activated rulesets were running, see attachment "01". >>>>=20 >>>> But the Web GUI stopped because of lines 234ff in 'ids.cgi': >>>>=20 >>>> ... >>>> # Check if the page is locked, in this case, the ids_page_lock_file >>>> exists. >>>> if (-e $IDS::ids_page_lock_file) { >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0# Lock the webpage and p= rint notice about autoupgrade of the >>>> ruleset >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0# is in progess. >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0&working_notice("$Lang::= tr{'ids ruleset autoupdate in >>>> progress'}"); >>>>=20 >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0# Loop and check if the = file still exists. >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0while(-e $IDS::ids_page_= lock_file) { >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0# Sleep for a second and re-check. >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0sleep 1; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0} >>>> ... >>>>=20 >>>> I searched again and found that there are two subroutines in >>>> '/var/ipfire/ids-functions.pl', lines 1920ff, which are responsible >>>> for >>>> locking and unlocking this file: >>>>=20 >>>> ... >>>> ## Function to write the lock file for locking the WUI, while >>>> ## the autoupdate script runs. >>>> # >>>> sub lock_ids_page() { >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0# Call subfunction to cr= eate the file. >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0&create_empty_file($ids_= page_lock_file); >>>> } >>>>=20 >>>> # >>>> ## Function to release the lock of the WUI, again. >>>> # >>>> sub unlock_ids_page() { >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0# Delete lock file. >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0unlink($ids_page_lock_fi= le); >>>> } >>>> ... >>>>=20 >>>> Somehow lines 103ff in "/usr/local/bin/update-ids-ruleset" didn't >>>> work/weren't executed(?): >>>>=20 >>>> ... >>>> # Lock the IDS page. >>>> &IDS::lock_ids_page(); >>>>=20 >>>> # The script has requested a lock, so set locket to "1". >>>> $locked =3D "1"; >>>>=20 >>>> # Grab the configured providers. >>>> &General::readhasharray("$IDS::providers_settings_file", >>>> \%providers); >>>>=20 >>>> # Loop through the array of available providers. >>>> foreach my $id (keys %providers) { >>>> =C2=A0=C2=A0 # Assign some nice variabled. >>>> =C2=A0=C2=A0 my $provider =3D $providers{$id}[0]; >>>> =C2=A0=C2=A0 my $autoupdate_status =3D $providers{$id}[3]; >>>>=20 >>>> =C2=A0=C2=A0 # Skip the provider if autoupdate is not enabled. >>>> =C2=A0=C2=A0 next unless($autoupdate_status eq "enabled"); >>>>=20 >>>> =C2=A0=C2=A0 # Call the download function and gather the new ruleset for= the >>>> current processed provider. >>>> =C2=A0=C2=A0 if(&IDS::downloadruleset($provider)) { >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Store error message for displaying in t= he WUI. >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 &IDS::_store_error_message("$provider: $L= ang::tr{'could not >>>> download latest updates'}"); >>>>=20 >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Unlock the IDS page. >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 &IDS::unlock_ids_page(); >>>>=20 >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # Exit. >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 exit 0; >>>> =C2=A0=C2=A0 } >>>> ... >>>>=20 >>>> Manual updates - yesterday - were ok, but the automatic updates >>>> tonight >>>> didn't unlink the lock file: IDS GUI stopped at the next call today. >>>>=20 >>>> ########## >>>>=20 >>>> 2. There still exist a directory '/tmp/isd_tmp' containing 'conf' >>>> ('config' and 'map' files from emerging-threats) and a 'rules'-dir >>>> containing all '*.rules'-files. Leftovers!? Can I safely delete these >>>> dirs? >>>>=20 >>>> And: can anyone confirm these two problems - or reproduce? >>>>=20 >>>> Best, >>>> Matthias >>>=20 >>>=20 >>=20 >=20 --===============4677092723796630771==--