Re: IDS with support for multiple ruleset providers

[Stefan Schantl <stefan.schantl@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 9099 bytes --]

Am Dienstag, den 13.04.2021, 20:57 +0200 schrieb Stefan Schantl:
> Hello Adolf,
> 
> thanks you very much for your huge effort in testing this and
> providing
> this very detailed feedback.
> 
> While reading through your single steps it feels sometimes near to
> get
> a knot inside my brain....
> 
> > Hi Stefan,
> > 
> > I did a bit more testing.
> > 
> > I added the snort community rules set. I then went to customise and
> > left the snort rules unchecked then pressed apply.
> > 
> > I then disabled the snort rules from the main page and on the
> > customise page the snort rules were no longer showing.
> 
> Workes as designed.
> 
> > I then enabled the snort rules on the first page and then went to
> > customise but the snort rules still were not showing.
> 
> Very good catch - Fixed.
> 
> > I deleted the snort ruleset provider on the first page and then
> > added
> > them back and now the snort ruleset was shown again on the
> > customise
> > page.
> 
> OK.
> 
> > I then checked the snort ruleset and applied it and then entered
> > customise again and unchecked the snort ruleset and applied it.
> > When
> > I went back into customise the snort ruleset was checked again. So
> > once checked I could not uncheck it and keep it that why by
> > pressing
> > apply.
> > 
> 
> Confirmed. Thanks for finding this.

Fixed by commit:

https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=79cc92267f1811beab84ae190fc1c3724a67e5f4

> 
> > I then deleted the snort ruleset provider from the first page. Then
> > the ruleset was gone from the customise page.
> > 
> > Then I added the snort ruleset provider back in but then got an
> > error
> > message saying that the snort ruleset provider was already
> > selected.
> > I then pressed back and came back to the main page with no snort
> > ruleset provider but also with the page  only showing down to the
> > Ruleset Settings table. There was nothing else after that.
> > 
> > The httpd/error_log showed the following
> > 
> > Smartmatch is experimental at /srv/web/ipfire/cgi-bin/ids.cgi line
> > 288.
> > Smartmatch is experimental at /srv/web/ipfire/cgi-bin/ids.cgi line
> > 288.
> > Can't call method "mtime" on an undefined value at /var/ipfire/ids-
> > functions.pl line 1512
> > 
> > Reloading the IPFire browser page and going back to the IDS main
> > page
> > gives the same result with the additional two lines in the log
> > 
> > Smartmatch is experimental at /srv/web/ipfire/cgi-bin/ids.cgi line
> > 288.
> > Can't call method "mtime" on an undefined value at /var/ipfire/ids-
> > functions.pl line 1512.
> > 
> 
> Sorry I'm unable to reproduce this - maybe a download error?

Fixed by 
https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=7cf0ecadc14c2a8f6a711ff3ff3dfa2c0a516fb5
and
https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=e59b8178e0cb4098904a8c0f591639d92a1f625e

> 
> > Sorry for breaking it again. If any of my steps are not clear let
> > me
> > know and I will clarify where necessary.
> 
> Hey, this is why we do testing - each found bug until release is a
> good
> bug!
> 
> > 
> > Regards,
> > 
> > Adolf.
> > 
> > 
> > On 11/04/2021 11:49, Adolf Belka wrote:
> > > Hi Stefan,
> > > 
> > > I have installed the new version from scratch in my ipfire vm
> > > testbed. I followed "all" the instructions this time :-)
> > > 
> > > I was able to add additional providers and then go and select the
> > > rules I wanted and had no problems at all.
> > > 
> > > Looks like all fixed. I will do further evaluation of it over the
> > > next few days and let you know how things go for me.
> > > 
> > > Regards,
> > > 
> > > Adolf.
> > > 
> > > On 11/04/2021 10:46, Stefan Schantl wrote:
> > > > Hello again,
> > > > 
> > > > I've tested and uploaded the fourth test verstion.
> > > > 
> > > > https://people.ipfire.org/~stevee/ids-multiple-providers/ids-multiple-providers-004.tar.gz
> > > > 
> > > > This time the ownership of all files are correct at my test
> > > > system.
> > > > 
> > > > (Tested with ruleset changes and without)
> > > > 
> > > > Best regards,
> > > > 
> > > > -Stefan
> > > > 
> > > > > Best regards,
> > > > > 
> > > > > -Stefan
> > > > > 
> > > > > > Hi Stefan,
> > > > > > 
> > > > > > I copied the new tarfile to my ipfire vm testbed machine
> > > > > > and
> > > > > > extracted it and ran the converter script. No errors. I
> > > > > > then
> > > > > > used
> > > > > > the
> > > > > > wui page to add a new provider to the list then selected to
> > > > > > customize
> > > > > > the rules and ticked the box for the added rules. Then I
> > > > > > pressed
> > > > > > apply and got a blank white screen again.
> > > > > > 
> > > > > > 
> > > > > > The error log has the following:-
> > > > > > 
> > > > > > Smartmatch is experimental at /srv/web/ipfire/cgi-
> > > > > > bin/ids.cgi 
> > > > > > line
> > > > > > 288.
> > > > > > Smartmatch is experimental at /srv/web/ipfire/cgi-
> > > > > > bin/ids.cgi 
> > > > > > line
> > > > > > 288.
> > > > > > Smartmatch is experimental at /srv/web/ipfire/cgi-
> > > > > > bin/ids.cgi 
> > > > > > line
> > > > > > 288.
> > > > > > Smartmatch is experimental at /srv/web/ipfire/cgi-
> > > > > > bin/ids.cgi 
> > > > > > line
> > > > > > 288.
> > > > > > Smartmatch is experimental at /srv/web/ipfire/cgi-
> > > > > > bin/ids.cgi 
> > > > > > line
> > > > > > 288.
> > > > > > Smartmatch is experimental at /srv/web/ipfire/cgi-
> > > > > > bin/ids.cgi 
> > > > > > line
> > > > > > 288.
> > > > > > Could not open /var/ipfire/suricata/oinkmaster-provider-
> > > > > > includes.conf. Permission denied
> > > > > > 
> > > > > > 
> > > > > > ls- hal of /var/ipfire/suricata shows the following
> > > > > > 
> > > > > > drwxr-xr-x  2 nobody nobody 4.0K Apr 10 22:47 .
> > > > > > drwxr-xr-x 49 root   root   4.0K Apr  5 08:20 ..
> > > > > > -rw-r--r--  1 nobody nobody    0 Dec 14 19:05 ignored
> > > > > > -rw-r--r--  1 root   root    21K Apr  1 20:00
> > > > > > oinkmaster.conf
> > > > > > -rw-r--r--  1 nobody nobody   61 Apr 10 14:40 oinkmaster-
> > > > > > modify-
> > > > > > sids.conf
> > > > > > -rw-r--r--  1 root   root      0 Apr 10 14:54 oinkmaster-
> > > > > > provider-
> > > > > > includes.conf
> > > > > > -rw-r--r--  1 nobody nobody   55 Apr 10 22:47 providers-
> > > > > > settings
> > > > > > -rw-r--r--  1 root   root   6.0K Apr  5 07:13 ruleset-
> > > > > > sources
> > > > > > -rw-r--r--  1 nobody nobody  102 Apr 10 14:54 settings
> > > > > > -rw-r--r--  1 nobody nobody  140 Apr 10 22:41 suricata-dns-
> > > > > > servers.yaml
> > > > > > -rw-r--r--  1 nobody nobody  125 Apr 10 14:54 suricata-
> > > > > > emerging-
> > > > > > used-
> > > > > > rulefiles.yaml
> > > > > > -rw-r--r--  1 nobody nobody  159 Apr 10 22:41 suricata-
> > > > > > homenet.yaml
> > > > > > -rw-r--r--  1 nobody nobody   98 Apr 10 14:40 suricata-
> > > > > > http-
> > > > > > ports.yaml
> > > > > > -rw-r--r--  1 nobody nobody   95 Apr 10 14:54 suricata-
> > > > > > static-
> > > > > > included-rulefiles.yaml
> > > > > > -rw-r--r--  1 nobody nobody   76 Apr 10 22:47 suricata-
> > > > > > urlhaus-
> > > > > > used-
> > > > > > rulefiles.yaml
> > > > > > -rw-r--r--  1 nobody nobody  214 Apr 10 14:54 suricata-
> > > > > > used-
> > > > > > providers.yaml
> > > > > > 
> > > > > > Three of the files are owned root:root while all the others
> > > > > > are
> > > > > > nobody:nobody
> > > > > > 
> > > > > > 
> > > > > > The above was with extracting and applying the updated tar
> > > > > > file on
> > > > > > top of IPFire after running the last version.
> > > > > > 
> > > > > > I will do a fresh clone of my IPFire vm and then repeat the
> > > > > > tar
> > > > > > extraction and convert and see if that gives any
> > > > > > difference.
> > > > > > 
> > > > > > 
> > > > > > Regards,
> > > > > > 
> > > > > > Adolf
> > > > > > 
> > > > > > On 10/04/2021 20:25, Stefan Schantl wrote:
> > > > > > > Hello list followers,
> > > > > > > 
> > > > > > > after getting a lot of feedback and bug reports I'm happy
> > > > > > > to
> > > > > > > announce the third test version for the new IDS system.
> > > > > > > 
> > > > > > > https://people.ipfire.org/~stevee/ids-multiple-providers/ids-multiple-providers-003.tar.gz
> > > > > > > 
> > > > > > > If you just join testing, please omit the installation
> > > > > > > instructions
> > > > > > > from the initial Mail from this list.
> > > > > > > 
> > > > > > > The converter script now works as expected and runs very
> > > > > > > smooth.
> > > > > > > 
> > > > > > > As usual please post your feedback and opinions to this
> > > > > > > list and
> > > > > > > any
> > > > > > > remain bugs to our bugtracker. (
> > > > > > > https://bugzilla.ipfire.org
> > > > > > > )
> > > > > > > 
> > > > > > > A big thanks in advance,
> > > > > > > 
> > > > > > > -Stefan
> > > > > > >