Re: IDS with support for multiple ruleset providers

[Michael Tremer <michael.tremer@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 8142 bytes --]



> On 13 Apr 2021, at 19:57, Stefan Schantl <stefan.schantl(a)ipfire.org> wrote:
> 
> Hello Adolf,
> 
> thanks you very much for your huge effort in testing this and providing
> this very detailed feedback.
> 
> While reading through your single steps it feels sometimes near to get
> a knot inside my brain....
> 
>> Hi Stefan,
>> 
>> I did a bit more testing.
>> 
>> I added the snort community rules set. I then went to customise and
>> left the snort rules unchecked then pressed apply.
>> 
>> I then disabled the snort rules from the main page and on the
>> customise page the snort rules were no longer showing.
> 
> Workes as designed.
> 
>> 
>> I then enabled the snort rules on the first page and then went to
>> customise but the snort rules still were not showing.
> 
> Very good catch - Fixed.
> 
>> 
>> I deleted the snort ruleset provider on the first page and then added
>> them back and now the snort ruleset was shown again on the customise
>> page.
> 
> OK.
> 
>> 
>> I then checked the snort ruleset and applied it and then entered
>> customise again and unchecked the snort ruleset and applied it. When
>> I went back into customise the snort ruleset was checked again. So
>> once checked I could not uncheck it and keep it that why by pressing
>> apply.
>> 
> 
> Confirmed. Thanks for finding this.
> 
>> I then deleted the snort ruleset provider from the first page. Then
>> the ruleset was gone from the customise page.
>> 
>> Then I added the snort ruleset provider back in but then got an error
>> message saying that the snort ruleset provider was already selected.
>> I then pressed back and came back to the main page with no snort
>> ruleset provider but also with the page  only showing down to the
>> Ruleset Settings table. There was nothing else after that.
>> 
>> The httpd/error_log showed the following
>> 
>> Smartmatch is experimental at /srv/web/ipfire/cgi-bin/ids.cgi line
>> 288.
>> Smartmatch is experimental at /srv/web/ipfire/cgi-bin/ids.cgi line
>> 288.
>> Can't call method "mtime" on an undefined value at /var/ipfire/ids-
>> functions.pl line 1512
>> 
>> Reloading the IPFire browser page and going back to the IDS main page
>> gives the same result with the additional two lines in the log
>> 
>> Smartmatch is experimental at /srv/web/ipfire/cgi-bin/ids.cgi line
>> 288.
>> Can't call method "mtime" on an undefined value at /var/ipfire/ids-
>> functions.pl line 1512.
>> 
> 
> Sorry I'm unable to reproduce this - maybe a download error?

Might be. You do not check if stat() was successful and continue working with the result:

  https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=blob;f=config/cfgroot/ids-functions.pl;h=7e7ad46b53ee1481d1c56f436ff0ee2636e767ce;hb=de30329f3b089302969d5f79709855b57605df57#l1498

Just check if stat() returned something useful before continuing.

>> 
>> Sorry for breaking it again. If any of my steps are not clear let me
>> know and I will clarify where necessary.
> 
> Hey, this is why we do testing - each found bug until release is a good
> bug!

:)

-Michael

> 
>> 
>> 
>> Regards,
>> 
>> Adolf.
>> 
>> 
>> On 11/04/2021 11:49, Adolf Belka wrote:
>>> Hi Stefan,
>>> 
>>> I have installed the new version from scratch in my ipfire vm
>>> testbed. I followed "all" the instructions this time :-)
>>> 
>>> I was able to add additional providers and then go and select the
>>> rules I wanted and had no problems at all.
>>> 
>>> Looks like all fixed. I will do further evaluation of it over the
>>> next few days and let you know how things go for me.
>>> 
>>> Regards,
>>> 
>>> Adolf.
>>> 
>>> On 11/04/2021 10:46, Stefan Schantl wrote:
>>>> Hello again,
>>>> 
>>>> I've tested and uploaded the fourth test verstion.
>>>> 
>>>> https://people.ipfire.org/~stevee/ids-multiple-providers/ids-multiple-providers-004.tar.gz
>>>> 
>>>> This time the ownership of all files are correct at my test
>>>> system.
>>>> 
>>>> (Tested with ruleset changes and without)
>>>> 
>>>> Best regards,
>>>> 
>>>> -Stefan
>>>> 
>>>>> Best regards,
>>>>> 
>>>>> -Stefan
>>>>> 
>>>>>> Hi Stefan,
>>>>>> 
>>>>>> I copied the new tarfile to my ipfire vm testbed machine and
>>>>>> extracted it and ran the converter script. No errors. I then
>>>>>> used
>>>>>> the
>>>>>> wui page to add a new provider to the list then selected to
>>>>>> customize
>>>>>> the rules and ticked the box for the added rules. Then I
>>>>>> pressed
>>>>>> apply and got a blank white screen again.
>>>>>> 
>>>>>> 
>>>>>> The error log has the following:-
>>>>>> 
>>>>>> Smartmatch is experimental at /srv/web/ipfire/cgi-bin/ids.cgi 
>>>>>> line
>>>>>> 288.
>>>>>> Smartmatch is experimental at /srv/web/ipfire/cgi-bin/ids.cgi 
>>>>>> line
>>>>>> 288.
>>>>>> Smartmatch is experimental at /srv/web/ipfire/cgi-bin/ids.cgi 
>>>>>> line
>>>>>> 288.
>>>>>> Smartmatch is experimental at /srv/web/ipfire/cgi-bin/ids.cgi 
>>>>>> line
>>>>>> 288.
>>>>>> Smartmatch is experimental at /srv/web/ipfire/cgi-bin/ids.cgi 
>>>>>> line
>>>>>> 288.
>>>>>> Smartmatch is experimental at /srv/web/ipfire/cgi-bin/ids.cgi 
>>>>>> line
>>>>>> 288.
>>>>>> Could not open /var/ipfire/suricata/oinkmaster-provider-
>>>>>> includes.conf. Permission denied
>>>>>> 
>>>>>> 
>>>>>> ls- hal of /var/ipfire/suricata shows the following
>>>>>> 
>>>>>> drwxr-xr-x  2 nobody nobody 4.0K Apr 10 22:47 .
>>>>>> drwxr-xr-x 49 root   root   4.0K Apr  5 08:20 ..
>>>>>> -rw-r--r--  1 nobody nobody    0 Dec 14 19:05 ignored
>>>>>> -rw-r--r--  1 root   root    21K Apr  1 20:00 oinkmaster.conf
>>>>>> -rw-r--r--  1 nobody nobody   61 Apr 10 14:40 oinkmaster-
>>>>>> modify-
>>>>>> sids.conf
>>>>>> -rw-r--r--  1 root   root      0 Apr 10 14:54 oinkmaster-
>>>>>> provider-
>>>>>> includes.conf
>>>>>> -rw-r--r--  1 nobody nobody   55 Apr 10 22:47 providers-
>>>>>> settings
>>>>>> -rw-r--r--  1 root   root   6.0K Apr  5 07:13 ruleset-sources
>>>>>> -rw-r--r--  1 nobody nobody  102 Apr 10 14:54 settings
>>>>>> -rw-r--r--  1 nobody nobody  140 Apr 10 22:41 suricata-dns-
>>>>>> servers.yaml
>>>>>> -rw-r--r--  1 nobody nobody  125 Apr 10 14:54 suricata-
>>>>>> emerging-
>>>>>> used-
>>>>>> rulefiles.yaml
>>>>>> -rw-r--r--  1 nobody nobody  159 Apr 10 22:41 suricata-
>>>>>> homenet.yaml
>>>>>> -rw-r--r--  1 nobody nobody   98 Apr 10 14:40 suricata-http-
>>>>>> ports.yaml
>>>>>> -rw-r--r--  1 nobody nobody   95 Apr 10 14:54 suricata-
>>>>>> static-
>>>>>> included-rulefiles.yaml
>>>>>> -rw-r--r--  1 nobody nobody   76 Apr 10 22:47 suricata-
>>>>>> urlhaus-
>>>>>> used-
>>>>>> rulefiles.yaml
>>>>>> -rw-r--r--  1 nobody nobody  214 Apr 10 14:54 suricata-used-
>>>>>> providers.yaml
>>>>>> 
>>>>>> Three of the files are owned root:root while all the others
>>>>>> are
>>>>>> nobody:nobody
>>>>>> 
>>>>>> 
>>>>>> The above was with extracting and applying the updated tar
>>>>>> file on
>>>>>> top of IPFire after running the last version.
>>>>>> 
>>>>>> I will do a fresh clone of my IPFire vm and then repeat the
>>>>>> tar
>>>>>> extraction and convert and see if that gives any difference.
>>>>>> 
>>>>>> 
>>>>>> Regards,
>>>>>> 
>>>>>> Adolf
>>>>>> 
>>>>>> On 10/04/2021 20:25, Stefan Schantl wrote:
>>>>>>> Hello list followers,
>>>>>>> 
>>>>>>> after getting a lot of feedback and bug reports I'm happy
>>>>>>> to
>>>>>>> announce the third test version for the new IDS system.
>>>>>>> 
>>>>>>> https://people.ipfire.org/~stevee/ids-multiple-providers/ids-multiple-providers-003.tar.gz
>>>>>>> 
>>>>>>> If you just join testing, please omit the installation
>>>>>>> instructions
>>>>>>> from the initial Mail from this list.
>>>>>>> 
>>>>>>> The converter script now works as expected and runs very
>>>>>>> smooth.
>>>>>>> 
>>>>>>> As usual please post your feedback and opinions to this
>>>>>>> list and
>>>>>>> any
>>>>>>> remain bugs to our bugtracker. (https://bugzilla.ipfire.org
>>>>>>> )
>>>>>>> 
>>>>>>> A big thanks in advance,
>>>>>>> 
>>>>>>> -Stefan