Re: IDS with support for multiple ruleset providers

[Stefan Schantl <stefan.schantl@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 5848 bytes --]

Hello Michael,

a big thanks for testing and providing a lot of feedback and finding
all this bugs.

> Hello,
> 
> I just tried to install this on my c156 test system and I have found
> the following issues:
> 
> The converter script was not executable:
> 
> [root(a)fw01 ~]# convert-ids-multiple-providers
> -bash: /usr/sbin/convert-ids-multiple-providers: Permission denied
> 
> But that could be easily fixed:

Thanks, I've fixed this in the 003 release.
> 
> [root(a)fw01 ~]# chmod a+x /usr/sbin/convert-ids-multiple-providers
> 
> But then it fails with another error; probably a typo:
> 
> [root(a)fw01 ~]# convert-ids-multiple-providers
> Can't locate /var/ipfire/ids-functions.pl1 at /usr/sbin/convert-ids-
> multiple-providers line 25.

Shame on me, this was during testing while developing to have the old
and the changed ids-functions.pl on the same system. Accidently I've
commited the file with this hack and shipped it.

Because both libs were present on my development system I didn't note
about this problem.

> 
> Fixing that gives me this:
> 
> [root(a)fw01 ~]# convert-ids-multiple-providers
> Could not write to /var/ipfire/suricata/oinkmaster-emerging-modified-
> sids.conf. No such file or directory
> 
> Creating that file makes the converter exit without any errors:
> 

Yea, this happened, because I didn't proper opened the file for writing
in the converter script. So It has not been created and the old ruleset
changes also are not written to it.

> [root(a)fw01 ~]# touch /var/ipfire/suricata/oinkmaster-emerging-
> modified-sids.conf
> [root(a)fw01 ~]# convert-ids-multiple-providers
> [root(a)fw01 ~]#
> 
> But it didn’t convert anything and the file is empty.
> 
> However, the CGI loads and shows the new functionality.
> 
> Clicking the checkbox to disable a ruleset shows this:
> 
> Smartmatch is experimental at /srv/web/ipfire/cgi-bin/ids.cgi line
> 288.
> Smartmatch is experimental at /srv/web/ipfire/cgi-bin/ids.cgi line
> 288.
> Smartmatch is experimental at /srv/web/ipfire/cgi-bin/ids.cgi line
> 288.
> Could not write to /var/ipfire/suricata/suricata-used-providers.yaml. 
> Permission denied
> 
> That file exists but is owned by root:
> 
> -rw-r--r-- 1 root   root       0 Apr 10 13:42 suricata-used-
> providers.yaml

Strange, the ownerships of these files should be changed to the correct
one, by the converter script. I'll have a look on it. 

> 
> Changing permissions to nobody:nobody brings me one step further:
> 
> Could not write to /var/ipfire/suricata/oinkmaster-provider-
> includes.conf. Permission denied
> 
> After fixing the ownership, I can enable and disable the rule
> provider.
> 
> However, when I click “customise ruleset”, no rules are selected
> which suggests that the converter did not work correctly.

See above.

> 
> I could also add a new provider and found it confusing that no
> categories are enabled automatically. It could happen, that you add a
> couple of providers, but then there are still no rules enabled. That
> would be potentially dangerous. Thoughts on this?

The easiest solution would be to check if at least one ruleset provider
is enabled and has at least one rulefile enabled.

This would prevent from starting suricata without any rules.

> 
> Apart from that, this seems to work solidly even with the new perl
> release.
> 
> -Michael
> 
> > On 9 Apr 2021, at 20:27, Stefan Schantl <stefan.schantl(a)ipfire.org>
> > wrote:
> > 
> > Hello Development Team and list followers,
> > 
> > there are a lot of different vendors out there which offers
> > different
> > IDS rules for suricata. Some of them offers a complete set of rules
> > and
> > other ones some very specialized rules for different tasks.
> > 
> > Unfortunately it only was possible to select only one ruleset
> > provider
> > at the same time, so it usually wasn't an option to use one of them
> > and
> > keep a lot of traffic uninspected by the IDS.
> > 
> > Today I'm very happy to announce a testing version of a reworked
> > Intrusion Detection System which supports the usage of multiple
> > different providers and rulesets at the same time.
> > 
> > In total up to 15 different ruleset providers now can be used and
> > mixed
> > together to fit your personal requirements. They easily can be
> > managed
> > and configured via the WUI. Of course each one individually can be
> > disabled or re-enabled at each time.
> > 
> > The section for customizing the entire ruleset has been moved to a
> > subpage, which allows to enable a certain amount of ruleset files
> > or
> > enabling / disabling single rules inside them.
> > 
> > This helps to speed up the CGI if you want to mange your whitelist,
> > manage your ruleset providers or change basic settings of your
> > IDS. 
> > 
> > If you liked this short introduction, please help us testing to get
> > this cool stuff as soon as possible into the core distribution and
> > to
> > find bugs or other improvements.
> > 
> > The test versions and some screenshots can be found here:
> > 
> > https://people.ipfire.org/~stevee/ids-multiple-providers/
> > 
> > To join testing, please download the latest tarball and place it on
> > your IPFire test machine.
> > 
> > Execute the archive by using "tar -xvf ids-multiple-providers-
> > XXX.tar.gz - C /" on your local console or via SSH remote session.
> > 
> > The next steps would be to regenerate the language cache by
> > executing
> > "update-langs-cache" and to launch "convert-ids-multiple-
> > providers".
> > 
> > The converter will convert all your existing settings into the new
> > format and also will take care about your used rules and their
> > settings.
> > 
> > As usual, please report back any kind of feedback on this list and
> > submit any found bugs to our bugtracker (https://bugs.ipfire.org).
> > 
> > Thanks in advance,
> > 
> > -Stefan
> > 
> >