Re: IPFire meets Suricata - Call for tester

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 3637 bytes --]

Hello Ville,

thanks for joining the testing team.

There is currently no update from an existing IPFire installation to
the suricata containing one. So you have to create backups, do a fresh
installation and then restore your backups.

If possible and some test hardware is available, please install the
image on a different machine than your productive one.

Best regards,

-Stefan


> Hello,
> 
> Thx for bringing this update!
> 
> Is it ok to update my stable version of ipfire core 126 or should I
> install this suricata version and then use backups to get my data and
> settings back?
> 
> -Ville-
> 
> 
> > On 6 Feb 2019, at 10.58, Stefan Schantl <stefan.schantl(a)ipfire.org>
> > wrote:
> > 
> > Hello list,
> > 
> > today im very happy to announce a new test image with the latest
> > snapshot of the process bringing suricata to all of you.
> > 
> > The image is now hosted and provided by the nightly build feature
> > (a
> > big thanks to Michael for providing this) of IPFire, so if
> > development
> > goes on, every time a new image will be generated and easily can be
> > downloaded.
> > 
> > The latest image always can be grabbed from here:
> > 
> > https://nightly.ipfire.org/next-suricata/latest/x86_64/
> > 
> > Direct link for downloading the ISO image:
> > 
> > https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.isof
> > 
> > There is currently one known issue, that any kind of snort rules
> > (sourcefire) currenty can not be downloaded, so you have to use the
> > rulesets from emergingthreads for testing. This issue will be fixed
> > with the next image provided by the nightly build service.
> > 
> > Thanks for downloading and testing, as usual please file any bugs
> > to
> > our bugtracker (https://bugzilla.ipfire.org) and share your
> > feedback on
> > this list.
> > 
> > Best regards,
> > 
> > -Stefan
> > 
> > 
> > > Hello list followers,
> > > 
> > > some time ago development for the new implementation of the
> > > Intrusion
> > > Detection functionality in IPFire has been started.
> > > 
> > > The main goal, in a nutshell, was to give IPFire a modern,
> > > feature-
> > > rich 
> > > and user-friendly Intrusion Detection Engine. During this
> > > progress,
> > > the
> > > detection framework has been replaced - now suricata is used
> > > instead
> > > of
> > > snort.
> > > 
> > > Suricata uses a very modern and multi-threaded detection engine
> > > with
> > > support to perform actions on malicious traffic. So it provides
> > > the
> > > functionality of detecting any kind of intrusion attempts and the
> > > ability of guardian to block them under the same hood.
> > > 
> > > It was a lot of work, but finaly I'm happy to announce the first
> > > test
> > > version. It is almost feature complete and without any kind of
> > > bigger
> > > issues.
> > > 
> > > Because Intrusion Detection is a key feature of a firewall
> > > system, a
> > > lot of testing is required until the new implementation can
> > > become
> > > part
> > > of IPFire - therefore we need your help!
> > > 
> > > Download the test image (
> > > https://people.ipfire.org/~stevee/suricata/Images/), do a lot of
> > > hard
> > > testing and provide your feedback or suggestions on the
> > > develoment
> > > mailing list (
> > > https://lists.ipfire.org/mailman/listinfo/development).
> > > 
> > > If you find any bugs please file them in the IPFire Bugtracker (
> > > https://bugzilla.ipfire.org/).
> > > 
> > > Many thanks in advance,
> > > 
> > > -Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 4218 bytes --]

Hello guys,

Great news Stefan! I will have a look at the image as soon as I can.

You have an extra f at the end of the download URL. I am sure that everyone has noticed that already, but here is the correct url:

  https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso

What are the next steps for development here? I guess fixing the bugs that the testers find and the few outstanding ones. When is an updater going to be worked on?

Best,
-Michael

> On 6 Feb 2019, at 09:57, Stefan Schantl <stefan.schantl(a)ipfire.org> wrote:
> 
> Hello Ville,
> 
> thanks for joining the testing team.
> 
> There is currently no update from an existing IPFire installation to
> the suricata containing one. So you have to create backups, do a fresh
> installation and then restore your backups.
> 
> If possible and some test hardware is available, please install the
> image on a different machine than your productive one.
> 
> Best regards,
> 
> -Stefan
> 
> 
>> Hello,
>> 
>> Thx for bringing this update!
>> 
>> Is it ok to update my stable version of ipfire core 126 or should I
>> install this suricata version and then use backups to get my data and
>> settings back?
>> 
>> -Ville-
>> 
>> 
>>> On 6 Feb 2019, at 10.58, Stefan Schantl <stefan.schantl(a)ipfire.org>
>>> wrote:
>>> 
>>> Hello list,
>>> 
>>> today im very happy to announce a new test image with the latest
>>> snapshot of the process bringing suricata to all of you.
>>> 
>>> The image is now hosted and provided by the nightly build feature
>>> (a
>>> big thanks to Michael for providing this) of IPFire, so if
>>> development
>>> goes on, every time a new image will be generated and easily can be
>>> downloaded.
>>> 
>>> The latest image always can be grabbed from here:
>>> 
>>> https://nightly.ipfire.org/next-suricata/latest/x86_64/
>>> 
>>> Direct link for downloading the ISO image:
>>> 
>>> https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.isof
>>> 
>>> There is currently one known issue, that any kind of snort rules
>>> (sourcefire) currenty can not be downloaded, so you have to use the
>>> rulesets from emergingthreads for testing. This issue will be fixed
>>> with the next image provided by the nightly build service.
>>> 
>>> Thanks for downloading and testing, as usual please file any bugs
>>> to
>>> our bugtracker (https://bugzilla.ipfire.org) and share your
>>> feedback on
>>> this list.
>>> 
>>> Best regards,
>>> 
>>> -Stefan
>>> 
>>> 
>>>> Hello list followers,
>>>> 
>>>> some time ago development for the new implementation of the
>>>> Intrusion
>>>> Detection functionality in IPFire has been started.
>>>> 
>>>> The main goal, in a nutshell, was to give IPFire a modern,
>>>> feature-
>>>> rich 
>>>> and user-friendly Intrusion Detection Engine. During this
>>>> progress,
>>>> the
>>>> detection framework has been replaced - now suricata is used
>>>> instead
>>>> of
>>>> snort.
>>>> 
>>>> Suricata uses a very modern and multi-threaded detection engine
>>>> with
>>>> support to perform actions on malicious traffic. So it provides
>>>> the
>>>> functionality of detecting any kind of intrusion attempts and the
>>>> ability of guardian to block them under the same hood.
>>>> 
>>>> It was a lot of work, but finaly I'm happy to announce the first
>>>> test
>>>> version. It is almost feature complete and without any kind of
>>>> bigger
>>>> issues.
>>>> 
>>>> Because Intrusion Detection is a key feature of a firewall
>>>> system, a
>>>> lot of testing is required until the new implementation can
>>>> become
>>>> part
>>>> of IPFire - therefore we need your help!
>>>> 
>>>> Download the test image (
>>>> https://people.ipfire.org/~stevee/suricata/Images/), do a lot of
>>>> hard
>>>> testing and provide your feedback or suggestions on the
>>>> develoment
>>>> mailing list (
>>>> https://lists.ipfire.org/mailman/listinfo/development).
>>>> 
>>>> If you find any bugs please file them in the IPFire Bugtracker (
>>>> https://bugzilla.ipfire.org/).
>>>> 
>>>> Many thanks in advance,
>>>> 
>>>> -Stefan

Re: IPFire meets Suricata - Call for tester

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 3858 bytes --]

Hi,

Thank you for testing and your helpful feedback, Mentalic.

@Stefan: Is the Suricata branch based on the latest Core Update 128 branch? If not, that would explain the GeoIP issues.

-Michael

> On 4 Mar 2019, at 19:54, Mentalic <mentalic(a)cox.net> wrote:
> 
> Ran three different installs of tarball over image 5c861701e52ead2620df36049c242255 ipfire-2.x-suricata-rc4_x86_64.tar.gz using a couple different backups. All three had these two snort errors.
> 
> ./install.sh: line 4: /etc/init.d/snort: No such file or directory
> /var/ipfire/snort/settings not found - Nothing to do. Exiting!
> 
> 
> 1)Built without internet access, restored core 128 backup from suricata edition.  Tarball installs with allot of geoip errors apparently because file structure and data did not yet exist due to being offline. Backup had Geoip and GeoIP Groups in use. 
> Repeated error:
> "Could not open /usr/share/xt_geoip/CN.iv4: No such file or directory"
> After giving internet access and rebooting it cleared up these messages.
> 
> 2)Built with internet access, restored core 128 backup from suricata edition. Tarball installs with only the two snort errors.
> ./install.sh: line 4: /etc/init.d/snort: No such file or directory
> /var/ipfire/snort/settings not found - Nothing to do. Exiting!
> 
> 3) Built with internet access, restored core 127 backup from guardian install. IDS had this error:
> Setting up firewall                                                    [  OK  ]
> Stopping Collection daemon...                                          [  OK  ]
> Starting Collection daemon...                                          [  OK  ]
> Starting Intrusion Detection System...                                 [ FAIL ]
> chmod: cannot access '/var/run/suricata.pid': No such file or directory
> 
> From IPS interface was able to do a save and IPS then service started.
> 
> Regards
> Wayne
> 
> 
> 
> -----Original Message-----
> From: Development [mailto:development-bounces(a)lists.ipfire.org] On Behalf Of Mentalic
> Sent: Sunday, March 03, 2019 11:33 AM
> To: 'Stefan Schantl'; development(a)lists.ipfire.org
> Subject: RE: IPFire meets Suricata - Call for tester
> 
> Loaded up the Tarball, reports build 5d04cfe7. Running Blue and orange.
> 
> Noticed that the Blue network no longer requires a firewall rule to enable internet access. Only had to add device in Blue Access interface. I like this change.
> 
> Regards
> Wayne
> 
> -----Original Message-----
> From: Development [mailto:development-bounces(a)lists.ipfire.org] On Behalf Of Stefan Schantl
> Sent: Sunday, March 03, 2019 8:39 AM
> To: development(a)lists.ipfire.org
> Subject: Re: IPFire meets Suricata - Call for tester
> 
> Hello list,
> 
> Recently I've uploaded the fourth release candidate.
> 
> It fixes the issue of non working IPSec tunnels and tunes the main suricata configuration to better use available system resources.
> 
> The new tarball (currently x86_64 only) can be found here:
> 
> https://people.ipfire.org/~stevee/suricata/
> 
> To start testing download the tarball and place it on your IPFire system. Extract the tarball and launch the install (install.sh) script.
> 
> If you already have installed a previous test version or image, with the same steps as noted above you can update the the new version.
> 
> As always, if you prefer a fresh installation, the latest image can be grabbed from here (Please note the delay of at least one day until the new ISO is built by the service):
> 
> https://nightly.ipfire.org/next-suricata/latest/x86_64/
> 
> 
> Thanks for downloading and testing. There are no known bugs so far, as usual please file any bugs to our bugtracker (
> https://bugzilla.ipfire.org) and share your feedback on the list.
> 
> Best regards,
> 
> -Stefan
> 

RE: IPFire meets Suricata - Call for tester

[Mentalic]

[-- Attachment #1: Type: text/plain, Size: 3435 bytes --]

Ran three different installs of tarball over image 5c861701e52ead2620df36049c242255 ipfire-2.x-suricata-rc4_x86_64.tar.gz using a couple different backups. All three had these two snort errors.

./install.sh: line 4: /etc/init.d/snort: No such file or directory
/var/ipfire/snort/settings not found - Nothing to do. Exiting!


1)Built without internet access, restored core 128 backup from suricata edition.  Tarball installs with allot of geoip errors apparently because file structure and data did not yet exist due to being offline. Backup had Geoip and GeoIP Groups in use. 
Repeated error:
"Could not open /usr/share/xt_geoip/CN.iv4: No such file or directory"
After giving internet access and rebooting it cleared up these messages.

2)Built with internet access, restored core 128 backup from suricata edition. Tarball installs with only the two snort errors.
./install.sh: line 4: /etc/init.d/snort: No such file or directory
/var/ipfire/snort/settings not found - Nothing to do. Exiting!

3) Built with internet access, restored core 127 backup from guardian install. IDS had this error:
Setting up firewall                                                    [  OK  ]
Stopping Collection daemon...                                          [  OK  ]
Starting Collection daemon...                                          [  OK  ]
Starting Intrusion Detection System...                                 [ FAIL ]
chmod: cannot access '/var/run/suricata.pid': No such file or directory

>From IPS interface was able to do a save and IPS then service started.

Regards
Wayne



-----Original Message-----
From: Development [mailto:development-bounces(a)lists.ipfire.org] On Behalf Of Mentalic
Sent: Sunday, March 03, 2019 11:33 AM
To: 'Stefan Schantl'; development(a)lists.ipfire.org
Subject: RE: IPFire meets Suricata - Call for tester

Loaded up the Tarball, reports build 5d04cfe7. Running Blue and orange.

Noticed that the Blue network no longer requires a firewall rule to enable internet access. Only had to add device in Blue Access interface. I like this change.

Regards
Wayne

-----Original Message-----
From: Development [mailto:development-bounces(a)lists.ipfire.org] On Behalf Of Stefan Schantl
Sent: Sunday, March 03, 2019 8:39 AM
To: development(a)lists.ipfire.org
Subject: Re: IPFire meets Suricata - Call for tester

Hello list,

Recently I've uploaded the fourth release candidate.

It fixes the issue of non working IPSec tunnels and tunes the main suricata configuration to better use available system resources.

The new tarball (currently x86_64 only) can be found here:

https://people.ipfire.org/~stevee/suricata/

To start testing download the tarball and place it on your IPFire system. Extract the tarball and launch the install (install.sh) script.

If you already have installed a previous test version or image, with the same steps as noted above you can update the the new version.

As always, if you prefer a fresh installation, the latest image can be grabbed from here (Please note the delay of at least one day until the new ISO is built by the service):

https://nightly.ipfire.org/next-suricata/latest/x86_64/


Thanks for downloading and testing. There are no known bugs so far, as usual please file any bugs to our bugtracker (
https://bugzilla.ipfire.org) and share your feedback on the list.

Best regards,

-Stefan

Re: IPFire meets Suricata - Call for tester

[ummeegge]

[-- Attachment #1: Type: text/plain, Size: 5005 bytes --]

Hello Stefan,

On So, 2019-03-03 at 19:48 +0100, Stefan Schantl wrote:
> Hello Erik,
> 
> thank you for testing and your feedback.
You are welcome.

> 
> Sadly the RC3 release is broken - this happened because of wrong
> packed
> files.
I see.

> 
> I've release a fixed RC4 version with the latest development state -
> please use this one.
Have installed it and the WUI is back, Emergingthreads (Community
Rules) are set and downloaded. All seems to be working at the first
glance.

But wanted to report that while installation process the following
appears:

/var/ipfire/snort/settings is empty - Nothing to do. Exiting!
Setting up firewall
/etc/init.d/firewall: line 370: /usr/lib/firewall/ipsec-block: No such file or directory
Could not determine transfer net address: vetest                                                               [  OK  ]

so possibly it has nothing to do with you develoment but i wanted to
report it nevertheless.

Thanks for your hard work on that.

> 
> Best regards,

Best regards,

Erik

> 
> -Stefan
> > Hi Stefan,
> > have tried to install your provided tarball of RC3 from your
> > peoples
> > directory but something went really wrong. After executing
> > install.sh
> > the following appears:
> > 
> > Can't locate CONFIG_ROOT/network-functions.pl in @INC (@INC
> > contains:
> > /usr/lib/perl5/site_perl/5.12.3/x86_64-linux-thread-multi
> > /usr/lib/perl5/site_perl/5.12.3 /usr/lib/perl5/5.12.3/x86_64-linux-
> > thread-multi /usr/lib/perl5/5.12.3 .) at /var/ipfire/general-
> > functions.pl line 30.
> > Compilation failed in require at //var/ipfire/lang.pl line 12.
> > Compilation failed in require at -e line 1.
> > ./install.sh: line 13: /usr/sbin/convert-snort: Permission denied
> > Setting up firewall
> > /etc/init.d/firewall: line 370: /usr/lib/firewall/ipsec-block: No
> > such file or directory
> > Could not determine transfer net address: Va102
> > Can't locate CONFIG_ROOT/network-functions.pl in @INC (@INC
> > contains:
> > /usr/lib/perl5/site_perl/5.12.3/x86_64-linux-thread-multi
> > /usr/lib/perl5/site_perl/5.12.3 /usr/lib/perl5/5.12.3/x86_64-linux-
> > thread-multi /usr/lib/perl5/5.12.3 .) at /var/ipfire/general-
> > functions.pl line 30.
> > Compilation failed in require at /usr/lib/firewall/rules.pl line
> > 24.                                           [  OK  ]
> > Stopping Collection
> > daemon...                                                          
> >   
> >                       [  OK  ]
> > Starting Collection
> > daemon...                                                          
> >   
> >                       [  OK  ]
> > ./install.sh: line 22: /etc/init.d/suricata: Permission denied
> > 
> > Try to login into the the webuserinterface ended in a "Internal
> > server error".
> > error_log:
> > 
> > Can't locate CONFIG_ROOT/network-functions.pl in @INC (@INC
> > contains:
> > /usr/lib/perl5/site_perl/5.12.3/x86_64-linux-thread-multi
> > /usr/lib/perl5/site_perl/5.12.3 /usr/lib/perl5/5.12.3/x86_64-linux-
> > thread-multi /usr/lib/perl5/5.12.3 .) at /var/ipfire/general-
> > functions.pl line 30.
> > Compilation failed in require at /srv/web/ipfire/cgi-bin/index.cgi
> > line 30.
> > 
> > 
> > 'network-functions.pl' was indeed not on his place anymore.
> > Downloaded it again
> > but this didn´t solved it.
> > 
> > 
> > May you have an idea ?
> > 
> > Best,
> > 
> > Erik
> > 
> > 
> > On So, 2019-02-17 at 12:58 +0100, Stefan Schantl wrote:
> > > Hello list,
> > > 
> > > a short note from suricata development. I've uploaded the second
> > > release candidate, which fixes several issues and bugs.
> > > 
> > > Now, the "services.cgi" will correctly show the IPS as running,
> > > and
> > > logrotate and collectd will handle the correct service.
> > > 
> > > The new tarball (i586 for 32bit-systems, and x86_64) can be found
> > > here:
> > > 
> > > https://people.ipfire.org/~stevee/suricata/
> > > 
> > > To start testing download the tarball and place it on your IPFire
> > > system. Extract the tarball and launch the install (install.sh)
> > > script.
> > > 
> > > If you already have installed a previous test version or image,
> > > with
> > > the same steps as noted above you can update the the new version.
> > > 
> > > As always, if you prefer a fresh installation, the latest image
> > > can
> > > be
> > > grabbed
> > > from here:
> > > 
> > > https://nightly.ipfire.org/next-suricata/latest/x86_64/
> > > 
> > > Direct link for downloading the ISO image:
> > > 
> > > 
> > 
> > 
https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso
> > > Thanks for downloading and testing. There are no known bugs so
> > > far,
> > > as
> > > usual please file any bugs to our bugtracker (
> > > https://bugzilla.ipfire.org) and share your feedback on the list.
> > > 
> > > Best regards,
> > > 
> > > -Stefan

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 3922 bytes --]

Hello Erik,

thank you for testing and your feedback.

Sadly the RC3 release is broken - this happened because of wrong packed
files.

I've release a fixed RC4 version with the latest development state -
please use this one.

Best regards,

-Stefan
> Hi Stefan,
> have tried to install your provided tarball of RC3 from your peoples
> directory but something went really wrong. After executing install.sh
> the following appears:
> 
> Can't locate CONFIG_ROOT/network-functions.pl in @INC (@INC contains:
> /usr/lib/perl5/site_perl/5.12.3/x86_64-linux-thread-multi
> /usr/lib/perl5/site_perl/5.12.3 /usr/lib/perl5/5.12.3/x86_64-linux-
> thread-multi /usr/lib/perl5/5.12.3 .) at /var/ipfire/general-
> functions.pl line 30.
> Compilation failed in require at //var/ipfire/lang.pl line 12.
> Compilation failed in require at -e line 1.
> ./install.sh: line 13: /usr/sbin/convert-snort: Permission denied
> Setting up firewall
> /etc/init.d/firewall: line 370: /usr/lib/firewall/ipsec-block: No
> such file or directory
> Could not determine transfer net address: Va102
> Can't locate CONFIG_ROOT/network-functions.pl in @INC (@INC contains:
> /usr/lib/perl5/site_perl/5.12.3/x86_64-linux-thread-multi
> /usr/lib/perl5/site_perl/5.12.3 /usr/lib/perl5/5.12.3/x86_64-linux-
> thread-multi /usr/lib/perl5/5.12.3 .) at /var/ipfire/general-
> functions.pl line 30.
> Compilation failed in require at /usr/lib/firewall/rules.pl line
> 24.                                           [  OK  ]
> Stopping Collection
> daemon...                                                            
>                       [  OK  ]
> Starting Collection
> daemon...                                                            
>                       [  OK  ]
> ./install.sh: line 22: /etc/init.d/suricata: Permission denied
> 
> Try to login into the the webuserinterface ended in a "Internal
> server error".
> error_log:
> 
> Can't locate CONFIG_ROOT/network-functions.pl in @INC (@INC contains:
> /usr/lib/perl5/site_perl/5.12.3/x86_64-linux-thread-multi
> /usr/lib/perl5/site_perl/5.12.3 /usr/lib/perl5/5.12.3/x86_64-linux-
> thread-multi /usr/lib/perl5/5.12.3 .) at /var/ipfire/general-
> functions.pl line 30.
> Compilation failed in require at /srv/web/ipfire/cgi-bin/index.cgi
> line 30.
> 
> 
> 'network-functions.pl' was indeed not on his place anymore.
> Downloaded it again
> but this didn´t solved it.
> 
> 
> May you have an idea ?
> 
> Best,
> 
> Erik
> 
> 
> On So, 2019-02-17 at 12:58 +0100, Stefan Schantl wrote:
> > Hello list,
> > 
> > a short note from suricata development. I've uploaded the second
> > release candidate, which fixes several issues and bugs.
> > 
> > Now, the "services.cgi" will correctly show the IPS as running, and
> > logrotate and collectd will handle the correct service.
> > 
> > The new tarball (i586 for 32bit-systems, and x86_64) can be found
> > here:
> > 
> > https://people.ipfire.org/~stevee/suricata/
> > 
> > To start testing download the tarball and place it on your IPFire
> > system. Extract the tarball and launch the install (install.sh)
> > script.
> > 
> > If you already have installed a previous test version or image,
> > with
> > the same steps as noted above you can update the the new version.
> > 
> > As always, if you prefer a fresh installation, the latest image can
> > be
> > grabbed
> > from here:
> > 
> > https://nightly.ipfire.org/next-suricata/latest/x86_64/
> > 
> > Direct link for downloading the ISO image:
> > 
> > 
> https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso
> > Thanks for downloading and testing. There are no known bugs so far,
> > as
> > usual please file any bugs to our bugtracker (
> > https://bugzilla.ipfire.org) and share your feedback on the list.
> > 
> > Best regards,
> > 
> > -Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

RE: IPFire meets Suricata - Call for tester

[Mentalic]

[-- Attachment #1: Type: text/plain, Size: 1560 bytes --]

Loaded up the Tarball, reports build 5d04cfe7. Running Blue and orange.

Noticed that the Blue network no longer requires a firewall rule to enable internet access. Only had to add device in Blue Access interface. I like this change.

Regards
Wayne

-----Original Message-----
From: Development [mailto:development-bounces(a)lists.ipfire.org] On Behalf Of Stefan Schantl
Sent: Sunday, March 03, 2019 8:39 AM
To: development(a)lists.ipfire.org
Subject: Re: IPFire meets Suricata - Call for tester

Hello list,

Recently I've uploaded the fourth release candidate.

It fixes the issue of non working IPSec tunnels and tunes the main suricata configuration to better use available system resources.

The new tarball (currently x86_64 only) can be found here:

https://people.ipfire.org/~stevee/suricata/

To start testing download the tarball and place it on your IPFire system. Extract the tarball and launch the install (install.sh) script.

If you already have installed a previous test version or image, with the same steps as noted above you can update the the new version.

As always, if you prefer a fresh installation, the latest image can be grabbed from here (Please note the delay of at least one day until the new ISO is built by the service):

https://nightly.ipfire.org/next-suricata/latest/x86_64/


Thanks for downloading and testing. There are no known bugs so far, as usual please file any bugs to our bugtracker (
https://bugzilla.ipfire.org) and share your feedback on the list.

Best regards,

-Stefan

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 1023 bytes --]

Hello list,

Recently I've uploaded the fourth release candidate.

It fixes the issue of non working IPSec tunnels and tunes
the main suricata configuration to better use available system
resources.

The new tarball (currently x86_64 only) can be found here:

https://people.ipfire.org/~stevee/suricata/

To start testing download the tarball and place it on your IPFire
system. Extract the tarball and launch the install (install.sh) script.

If you already have installed a previous test version or image, with
the same steps as noted above you can update the the new version.

As always, if you prefer a fresh installation, the latest image can be
grabbed from here (Please note the delay of at least one day until the
new ISO is built by the service):

https://nightly.ipfire.org/next-suricata/latest/x86_64/


Thanks for downloading and testing. There are no known bugs so far, as
usual please file any bugs to our bugtracker (
https://bugzilla.ipfire.org) and share your feedback on the list.

Best regards,

-Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[ummeegge]

[-- Attachment #1: Type: text/plain, Size: 3438 bytes --]

Hi Stefan,
have tried to install your provided tarball of RC3 from your peoples
directory but something went really wrong. After executing install.sh
the following appears:

Can't locate CONFIG_ROOT/network-functions.pl in @INC (@INC contains: /usr/lib/perl5/site_perl/5.12.3/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.12.3 /usr/lib/perl5/5.12.3/x86_64-linux-thread-multi /usr/lib/perl5/5.12.3 .) at /var/ipfire/general-functions.pl line 30.
Compilation failed in require at //var/ipfire/lang.pl line 12.
Compilation failed in require at -e line 1.
./install.sh: line 13: /usr/sbin/convert-snort: Permission denied
Setting up firewall
/etc/init.d/firewall: line 370: /usr/lib/firewall/ipsec-block: No such file or directory
Could not determine transfer net address: Va102
Can't locate CONFIG_ROOT/network-functions.pl in @INC (@INC contains: /usr/lib/perl5/site_perl/5.12.3/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.12.3 /usr/lib/perl5/5.12.3/x86_64-linux-thread-multi /usr/lib/perl5/5.12.3 .) at /var/ipfire/general-functions.pl line 30.
Compilation failed in require at /usr/lib/firewall/rules.pl line 24.                                           [  OK  ]
Stopping Collection daemon...                                                                                  [  OK  ]
Starting Collection daemon...                                                                                  [  OK  ]
./install.sh: line 22: /etc/init.d/suricata: Permission denied

Try to login into the the webuserinterface ended in a "Internal server error".
error_log:

Can't locate CONFIG_ROOT/network-functions.pl in @INC (@INC contains: /usr/lib/perl5/site_perl/5.12.3/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.12.3 /usr/lib/perl5/5.12.3/x86_64-linux-thread-multi /usr/lib/perl5/5.12.3 .) at /var/ipfire/general-functions.pl line 30.
Compilation failed in require at /srv/web/ipfire/cgi-bin/index.cgi line 30.


'network-functions.pl' was indeed not on his place anymore. Downloaded it again
but this didn´t solved it.


May you have an idea ?

Best,

Erik


On So, 2019-02-17 at 12:58 +0100, Stefan Schantl wrote:
> Hello list,
> 
> a short note from suricata development. I've uploaded the second
> release candidate, which fixes several issues and bugs.
> 
> Now, the "services.cgi" will correctly show the IPS as running, and
> logrotate and collectd will handle the correct service.
> 
> The new tarball (i586 for 32bit-systems, and x86_64) can be found
> here:
> 
> https://people.ipfire.org/~stevee/suricata/
> 
> To start testing download the tarball and place it on your IPFire
> system. Extract the tarball and launch the install (install.sh)
> script.
> 
> If you already have installed a previous test version or image, with
> the same steps as noted above you can update the the new version.
> 
> As always, if you prefer a fresh installation, the latest image can
> be
> grabbed
> from here:
> 
> https://nightly.ipfire.org/next-suricata/latest/x86_64/
> 
> Direct link for downloading the ISO image:
> 
> 
https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso
> 
> Thanks for downloading and testing. There are no known bugs so far,
> as
> usual please file any bugs to our bugtracker (
> https://bugzilla.ipfire.org) and share your feedback on the list.
> 
> Best regards,
> 
> -Stefan

RE: IPFire meets Suricata - Call for tester

[Mentalic]

[-- Attachment #1: Type: text/plain, Size: 8586 bytes --]

Stafan

File I downloaded:
26fbdacc7a7ef6f4ce612c85a87788b9 ipfire-2.x-suricata-rc3_x86_64.tar.gz
Yours from below:
300943d7e6fa667b7c33843124182b4f  ipfire-2.x-suricata-rc3_x86_64.tar.gz

Well, clearly something wrong somewhere.

Regards
Wayne


-----Original Message-----
From: Development [mailto:development-bounces(a)lists.ipfire.org] On Behalf Of Stefan Schantl
Sent: Friday, February 22, 2019 4:59 AM
To: development(a)lists.ipfire.org
Subject: Re: IPFire meets Suricata - Call for tester

Hello Wayne,

once more thanks for your testing and providing such detailed feedback.
> Restored backup from core127 with guardian against a fresh suricata 
> iso install.
> Found:
> IPS service was running with no rules downloaded. Box's checked were 
> enabled, monitor, red and blue.
> It did import the oinkcode but no rules sets downloaded automatically.

Thats correct, there was no code, which called the downloader in such a case. I've added it now to the "convert-snort" script, so in future if a snort config has got imported and no ruleset is present it automatically will be downloaded.

> Since I had two rule sources in guardian I wondered how that would be 
> handled here were one set is all that's possible. Picked a set and 
> downloaded them, had to manually pick the rules. Not a big deal to 
> setup. Made an attempt at importing the suricata tarball into current 
> release core127 with guardian installed but no luck. Tar command exits 
> with "This does not look like a tar archive". Had used "tar -xvf" to 
> extract... Could be doing something wrong here?

Mhm, I've checked the archives they work here, may an incomplete download?

The md5 checksums should be:

c5a6830336fdb75cc1c59e735a0fa655  ipfire-2.x-suricata-rc3_i586.tar.gz
300943d7e6fa667b7c33843124182b4f  ipfire-2.x-suricata-rc3_x86_64.tar.gz

Please only extract the "outer" tarball and call the "install.sh"
script.


Thanks in advance,

-Stefan
> 
> Regards
> Wayne
> 
> 
> -----Original Message-----
> From: Development [mailto:development-bounces(a)lists.ipfire.org] On 
> Behalf Of Stefan Schantl
> Sent: Wednesday, February 20, 2019 1:55 AM
> To: development(a)lists.ipfire.org
> Subject: Re: IPFire meets Suricata - Call for tester
> 
> Hello again and thanks for the feedback.
> > Exposed my test setup directly to my cable modem and noticed a 
> > couple of things.
> > 
> > -The Firewall log seems to only list items that match my firewall 
> > rules. Gone was the typical several a minute "drop_input" entry 
> > noise, there was zero drop_input's in 15min or so. Possible logging 
> > issue?
> 
> The IDS/IPS events are not logged to the firewall log. They only can 
> be accessed in the "Logs"->"IPS Logs" section.
> 
> > -Suricata placed entries into IPS log, but what is done with them?
> > Don't see a block list like Guardian generated.
> 
> Thats exactly how suricata works and the main benefit why we choose to 
> switch to suricata.
> 
> The old snort/guardian solution worked like this:
> 
> Snort detected (based on it's ruleset) an event and logged it to it's 
> logfile. Guardian read this event from the file, parsed it again and 
> if the configured block count for the matching IP-address was reached, 
> the host was blocked by an iptables rule (block list).
> 
> The new suricata-based solution works like this:
> 
> Suricata detects (also based on the ruleset) an event and directly 
> drops the bad package. There is no additional software involved 
> anymore .
> 
> So one of the benefits of the new approach is to reduce the amount of 
> time an attack has been recognized until it's blocked immediately.
> 
> > -Are there any incompatibility issues with using the backup function 
> > to restore to this version? I had made a backup from my core 127 
> > system with the old intrusion detection/guardian not active just in 
> > case.
> 
> There is a converter-script available, which will move the old 
> snort/guardian and rules settings to be used by suricata. This script 
> automatically will be called if a backup gets restored, which contains 
> such settings files.
> 
> Therefore I would ask you to test this feature by restoring such a 
> backup on a fresh installed nightly machine and if possible to install 
> the "update tarball" on a regular machine with configured snort and/or 
> guardian.
> 
> In both ways, all your taken settings should be the same for suricata 
> as before for snort.
> 
> A big thanks in advance and best regards,
> 
> -Stefan
> 
> > Regards
> > Wayne
> > 
> > -----Original Message-----
> > From: Development [mailto:development-bounces(a)lists.ipfire.org] On 
> > Behalf Of Mentalic
> > Sent: Tuesday, February 19, 2019 4:12 PM
> > To: 'Stefan Schantl'; development(a)lists.ipfire.org
> > Subject: RE: IPFire meets Suricata - Call for tester
> > 
> > Stefan
> > 
> > Yep I had downloaded the nightly and suspected is was not current, 
> > and so posted the build number.
> > 
> > With the 5d7d8749 loaded I have not seen any of the previous issues 
> > nor any others thus far.
> > 
> > Regards
> > Wayne
> > 
> > -----Original Message-----
> > From: Development [mailto:development-bounces(a)lists.ipfire.org] On 
> > Behalf Of Stefan Schantl
> > Sent: Tuesday, February 19, 2019 5:34 AM
> > To: development(a)lists.ipfire.org
> > Subject: Re: IPFire meets Suricata - Call for tester
> > 
> > Hello Wayne,
> > 
> > it seems you accidentally downloaded and tested the wrong image.
> > 
> > The latest one is 5d7d8749 were you downloaded one is an older 
> > release.
> > 
> > Sadly the nightly build service and therefore the images are one day 
> > later than the upgrade tarballs....
> > 
> > You simply can update to this release by using the RC3 tarball or 
> > download the available "5d7d8749" ISO.
> > 
> > Best regards,
> > 
> > -Stefan
> > > Loaded the new iso, reports build 77c07352. Still having 
> > > connection issues with suricata as soon as its activated where 
> > > existing connections would continue to work, no new connections 
> > > were possible.
> > > Reboot results in no connection timeouts. Disable suricata, 
> > > reboot, connections work.
> > > 
> > > Any graphical data trend under Status tab reports errors and 
> > > remains blank. Typically on new installs the trends at least show 
> > > the chart even though data had not been collected.
> > > 
> > > Configured options:
> > > Geoip
> > > Proxy on green and blue
> > > URL filter
> > > suricata on red/blue Running a number of emerging threats rule 
> > > sets.
> > > 
> > > Regards
> > > Wayne
> > > 
> > > 
> > > 
> > > -----Original Message-----
> > > From: Development [mailto:development-bounces(a)lists.ipfire.org]
> > > On
> > > Behalf Of Stefan Schantl
> > > Sent: Monday, February 18, 2019 7:16 AM
> > > To: development(a)lists.ipfire.org
> > > Subject: Re: IPFire meets Suricata - Call for tester
> > > 
> > > Hello list,
> > > 
> > > I've uploaded the third release candidate, which hopefully would 
> > > be the last one.
> > > 
> > > It fixes the issue that no traffic could be passed through the 
> > > firewall when suricata was running on some machines and no graphs 
> > > could be displayed anymore. Thanks to Wayne for reporting and 
> > > Michael Tremer for testing and fixing.
> > > 
> > > The new tarball (i586 for 32bit-systems, and x86_64) can be found
> > > here:
> > > 
> > > https://people.ipfire.org/~stevee/suricata/
> > > 
> > > To start testing download the tarball and place it on your IPFire 
> > > system. Extract the tarball and launch the install (install.sh) 
> > > script.
> > > 
> > > If you already have installed a previous test version or image, 
> > > with the same steps as noted above you can update the the new 
> > > version.
> > > 
> > > As always, if you prefer a fresh installation, the latest image 
> > > can be grabbed from here:
> > > 
> > > https://nightly.ipfire.org/next-suricata/latest/x86_64/
> > > 
> > > Direct link for downloading the ISO image:
> > > 
> > > https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21
> > > .x
> > > 86
> > > _64-full-core128.iso
> > > 
> > > Thanks for downloading and testing. There are no known bugs so 
> > > far, as usual please file any bugs to our bugtracker (
> > > https://bugzilla.ipfire.org) and share your feedback on the list.
> > > 
> > > Best regards,
> > > 
> > > -Stefan
> > > 

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 7836 bytes --]

> > On 21 Feb 2019, at 21:56, Mentalic <mentalic(a)cox.net> wrote:
> > 
> > Made an attempt at importing the suricata tarball into current
> > release core127 with guardian installed but no luck. Tar command
> > exits with "This does not look like a tar archive". Had used "tar
> > -xvf" to extract... Could be doing something wrong here?
> 
> Good point actually.
> 
> @Stevee: What is the migration path away from Guardian? This kind of
> needs to be uninstalled or the CGI needs to be stripped to remove all
> IPS features from it.

I did not think much about this until yet. There are two possible
solutions:

1.) Keep guardian, and remove the option so monitor the snort events,
which limits guardian to be a simple brute-force detection tool against
SSH and the WUI-login on IPFire.

2.) Entirely drop guardian.

What do you think? 

@All Any comments ?

Best regards,

-Stefan

> 
> -Michael
> 
> > Regards
> > Wayne
> > 
> > 
> > -----Original Message-----
> > From: Development [mailto:development-bounces(a)lists.ipfire.org] On
> > Behalf Of Stefan Schantl
> > Sent: Wednesday, February 20, 2019 1:55 AM
> > To: development(a)lists.ipfire.org
> > Subject: Re: IPFire meets Suricata - Call for tester
> > 
> > Hello again and thanks for the feedback.
> > > Exposed my test setup directly to my cable modem and noticed a
> > > couple 
> > > of things.
> > > 
> > > -The Firewall log seems to only list items that match my
> > > firewall 
> > > rules. Gone was the typical several a minute "drop_input" entry
> > > noise, 
> > > there was zero drop_input's in 15min or so. Possible logging
> > > issue?
> > 
> > The IDS/IPS events are not logged to the firewall log. They only
> > can be accessed in the "Logs"->"IPS Logs" section.
> > 
> > > -Suricata placed entries into IPS log, but what is done with
> > > them?
> > > Don't see a block list like Guardian generated.
> > 
> > Thats exactly how suricata works and the main benefit why we choose
> > to switch to suricata.
> > 
> > The old snort/guardian solution worked like this:
> > 
> > Snort detected (based on it's ruleset) an event and logged it to
> > it's logfile. Guardian read this event from the file, parsed it
> > again and if the configured block count for the matching IP-address 
> > was reached, the host was blocked by an iptables rule (block list).
> > 
> > The new suricata-based solution works like this:
> > 
> > Suricata detects (also based on the ruleset) an event and directly
> > drops the bad package. There is no additional software involved
> > anymore .
> > 
> > So one of the benefits of the new approach is to reduce the amount
> > of time an attack has been recognized until it's blocked
> > immediately.
> > 
> > > -Are there any incompatibility issues with using the backup
> > > function 
> > > to restore to this version? I had made a backup from my core 127 
> > > system with the old intrusion detection/guardian not active just
> > > in 
> > > case.
> > 
> > There is a converter-script available, which will move the old
> > snort/guardian and rules settings to be used by suricata. This
> > script automatically will be called if a backup gets restored,
> > which contains such settings files.
> > 
> > Therefore I would ask you to test this feature by restoring such a
> > backup on a fresh installed nightly machine and if possible to
> > install the "update tarball" on a regular machine with configured
> > snort and/or guardian.
> > 
> > In both ways, all your taken settings should be the same for
> > suricata as before for snort.
> > 
> > A big thanks in advance and best regards,
> > 
> > -Stefan 
> > 
> > > Regards
> > > Wayne
> > > 
> > > -----Original Message-----
> > > From: Development [mailto:development-bounces(a)lists.ipfire.org]
> > > On 
> > > Behalf Of Mentalic
> > > Sent: Tuesday, February 19, 2019 4:12 PM
> > > To: 'Stefan Schantl'; development(a)lists.ipfire.org
> > > Subject: RE: IPFire meets Suricata - Call for tester
> > > 
> > > Stefan
> > > 
> > > Yep I had downloaded the nightly and suspected is was not
> > > current, and 
> > > so posted the build number.
> > > 
> > > With the 5d7d8749 loaded I have not seen any of the previous
> > > issues 
> > > nor any others thus far.
> > > 
> > > Regards
> > > Wayne
> > > 
> > > -----Original Message-----
> > > From: Development [mailto:development-bounces(a)lists.ipfire.org]
> > > On 
> > > Behalf Of Stefan Schantl
> > > Sent: Tuesday, February 19, 2019 5:34 AM
> > > To: development(a)lists.ipfire.org
> > > Subject: Re: IPFire meets Suricata - Call for tester
> > > 
> > > Hello Wayne,
> > > 
> > > it seems you accidentally downloaded and tested the wrong image.
> > > 
> > > The latest one is 5d7d8749 were you downloaded one is an older 
> > > release.
> > > 
> > > Sadly the nightly build service and therefore the images are one
> > > day 
> > > later than the upgrade tarballs....
> > > 
> > > You simply can update to this release by using the RC3 tarball
> > > or 
> > > download the available "5d7d8749" ISO.
> > > 
> > > Best regards,
> > > 
> > > -Stefan
> > > > Loaded the new iso, reports build 77c07352. Still having
> > > > connection 
> > > > issues with suricata as soon as its activated where existing 
> > > > connections would continue to work, no new connections were 
> > > > possible.
> > > > Reboot results in no connection timeouts. Disable suricata,
> > > > reboot, 
> > > > connections work.
> > > > 
> > > > Any graphical data trend under Status tab reports errors and
> > > > remains 
> > > > blank. Typically on new installs the trends at least show the
> > > > chart 
> > > > even though data had not been collected.
> > > > 
> > > > Configured options:
> > > > Geoip
> > > > Proxy on green and blue
> > > > URL filter
> > > > suricata on red/blue Running a number of emerging threats rule
> > > > sets.
> > > > 
> > > > Regards
> > > > Wayne
> > > > 
> > > > 
> > > > 
> > > > -----Original Message-----
> > > > From: Development [mailto:development-bounces(a)lists.ipfire.org]
> > > > On 
> > > > Behalf Of Stefan Schantl
> > > > Sent: Monday, February 18, 2019 7:16 AM
> > > > To: development(a)lists.ipfire.org
> > > > Subject: Re: IPFire meets Suricata - Call for tester
> > > > 
> > > > Hello list,
> > > > 
> > > > I've uploaded the third release candidate, which hopefully
> > > > would be 
> > > > the last one.
> > > > 
> > > > It fixes the issue that no traffic could be passed through the 
> > > > firewall when suricata was running on some machines and no
> > > > graphs 
> > > > could be displayed anymore. Thanks to Wayne for reporting and 
> > > > Michael Tremer for testing and fixing.
> > > > 
> > > > The new tarball (i586 for 32bit-systems, and x86_64) can be
> > > > found
> > > > here:
> > > > 
> > > > https://people.ipfire.org/~stevee/suricata/
> > > > 
> > > > To start testing download the tarball and place it on your
> > > > IPFire 
> > > > system. Extract the tarball and launch the install
> > > > (install.sh) 
> > > > script.
> > > > 
> > > > If you already have installed a previous test version or image,
> > > > with 
> > > > the same steps as noted above you can update the the new
> > > > version.
> > > > 
> > > > As always, if you prefer a fresh installation, the latest image
> > > > can 
> > > > be grabbed from here:
> > > > 
> > > > https://nightly.ipfire.org/next-suricata/latest/x86_64/
> > > > 
> > > > Direct link for downloading the ISO image:
> > > > 
> > > > https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x
> > > > 86
> > > > _64-full-core128.iso
> > > > 
> > > > Thanks for downloading and testing. There are no known bugs so
> > > > far, 
> > > > as usual please file any bugs to our bugtracker (
> > > > https://bugzilla.ipfire.org) and share your feedback on the
> > > > list.
> > > > 
> > > > Best regards,
> > > > 
> > > > -Stefan
> > > > 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 7898 bytes --]

Hello Wayne,

once more thanks for your testing and providing such detailed feedback.
> Restored backup from core127 with guardian against a fresh suricata
> iso install. 
> Found:
> IPS service was running with no rules downloaded. Box's checked were
> enabled, monitor, red and blue.
> It did import the oinkcode but no rules sets downloaded
> automatically. 

Thats correct, there was no code, which called the downloader in such a
case. I've added it now to the "convert-snort" script, so in future if
a snort config has got imported and no ruleset is present it
automatically will be downloaded.

> Since I had two rule sources in guardian I wondered how that would be
> handled here were one set is all that's possible. Picked a set and
> downloaded them, had to manually pick the rules. Not a big deal to
> setup. Made an attempt at importing the suricata tarball into current
> release core127 with guardian installed but no luck. Tar command
> exits with "This does not look like a tar archive". Had used "tar
> -xvf" to extract... Could be doing something wrong here?

Mhm, I've checked the archives they work here, may an incomplete
download?

The md5 checksums should be:

c5a6830336fdb75cc1c59e735a0fa655  ipfire-2.x-suricata-rc3_i586.tar.gz
300943d7e6fa667b7c33843124182b4f  ipfire-2.x-suricata-rc3_x86_64.tar.gz

Please only extract the "outer" tarball and call the "install.sh"
script.


Thanks in advance,

-Stefan
> 
> Regards
> Wayne
> 
> 
> -----Original Message-----
> From: Development [mailto:development-bounces(a)lists.ipfire.org] On
> Behalf Of Stefan Schantl
> Sent: Wednesday, February 20, 2019 1:55 AM
> To: development(a)lists.ipfire.org
> Subject: Re: IPFire meets Suricata - Call for tester
> 
> Hello again and thanks for the feedback.
> > Exposed my test setup directly to my cable modem and noticed a
> > couple 
> > of things.
> > 
> > -The Firewall log seems to only list items that match my firewall 
> > rules. Gone was the typical several a minute "drop_input" entry
> > noise, 
> > there was zero drop_input's in 15min or so. Possible logging issue?
> 
> The IDS/IPS events are not logged to the firewall log. They only can
> be accessed in the "Logs"->"IPS Logs" section.
> 
> > -Suricata placed entries into IPS log, but what is done with them?
> > Don't see a block list like Guardian generated.
> 
> Thats exactly how suricata works and the main benefit why we choose
> to switch to suricata.
> 
> The old snort/guardian solution worked like this:
> 
> Snort detected (based on it's ruleset) an event and logged it to it's
> logfile. Guardian read this event from the file, parsed it again and
> if the configured block count for the matching IP-address was
> reached, the host was blocked by an iptables rule (block list).
> 
> The new suricata-based solution works like this:
> 
> Suricata detects (also based on the ruleset) an event and directly
> drops the bad package. There is no additional software involved
> anymore .
> 
> So one of the benefits of the new approach is to reduce the amount of
> time an attack has been recognized until it's blocked immediately.
> 
> > -Are there any incompatibility issues with using the backup
> > function 
> > to restore to this version? I had made a backup from my core 127 
> > system with the old intrusion detection/guardian not active just
> > in 
> > case.
> 
> There is a converter-script available, which will move the old
> snort/guardian and rules settings to be used by suricata. This script
> automatically will be called if a backup gets restored, which
> contains such settings files.
> 
> Therefore I would ask you to test this feature by restoring such a
> backup on a fresh installed nightly machine and if possible to
> install the "update tarball" on a regular machine with configured
> snort and/or guardian.
> 
> In both ways, all your taken settings should be the same for suricata
> as before for snort.
> 
> A big thanks in advance and best regards,
> 
> -Stefan 
> 
> > Regards
> > Wayne
> > 
> > -----Original Message-----
> > From: Development [mailto:development-bounces(a)lists.ipfire.org] On 
> > Behalf Of Mentalic
> > Sent: Tuesday, February 19, 2019 4:12 PM
> > To: 'Stefan Schantl'; development(a)lists.ipfire.org
> > Subject: RE: IPFire meets Suricata - Call for tester
> > 
> > Stefan
> > 
> > Yep I had downloaded the nightly and suspected is was not current,
> > and 
> > so posted the build number.
> > 
> > With the 5d7d8749 loaded I have not seen any of the previous
> > issues 
> > nor any others thus far.
> > 
> > Regards
> > Wayne
> > 
> > -----Original Message-----
> > From: Development [mailto:development-bounces(a)lists.ipfire.org] On 
> > Behalf Of Stefan Schantl
> > Sent: Tuesday, February 19, 2019 5:34 AM
> > To: development(a)lists.ipfire.org
> > Subject: Re: IPFire meets Suricata - Call for tester
> > 
> > Hello Wayne,
> > 
> > it seems you accidentally downloaded and tested the wrong image.
> > 
> > The latest one is 5d7d8749 were you downloaded one is an older 
> > release.
> > 
> > Sadly the nightly build service and therefore the images are one
> > day 
> > later than the upgrade tarballs....
> > 
> > You simply can update to this release by using the RC3 tarball or 
> > download the available "5d7d8749" ISO.
> > 
> > Best regards,
> > 
> > -Stefan
> > > Loaded the new iso, reports build 77c07352. Still having
> > > connection 
> > > issues with suricata as soon as its activated where existing 
> > > connections would continue to work, no new connections were 
> > > possible.
> > > Reboot results in no connection timeouts. Disable suricata,
> > > reboot, 
> > > connections work.
> > > 
> > > Any graphical data trend under Status tab reports errors and
> > > remains 
> > > blank. Typically on new installs the trends at least show the
> > > chart 
> > > even though data had not been collected.
> > > 
> > > Configured options:
> > > Geoip
> > > Proxy on green and blue
> > > URL filter
> > > suricata on red/blue Running a number of emerging threats rule
> > > sets.
> > > 
> > > Regards
> > > Wayne
> > > 
> > > 
> > > 
> > > -----Original Message-----
> > > From: Development [mailto:development-bounces(a)lists.ipfire.org]
> > > On 
> > > Behalf Of Stefan Schantl
> > > Sent: Monday, February 18, 2019 7:16 AM
> > > To: development(a)lists.ipfire.org
> > > Subject: Re: IPFire meets Suricata - Call for tester
> > > 
> > > Hello list,
> > > 
> > > I've uploaded the third release candidate, which hopefully would
> > > be 
> > > the last one.
> > > 
> > > It fixes the issue that no traffic could be passed through the 
> > > firewall when suricata was running on some machines and no
> > > graphs 
> > > could be displayed anymore. Thanks to Wayne for reporting and 
> > > Michael Tremer for testing and fixing.
> > > 
> > > The new tarball (i586 for 32bit-systems, and x86_64) can be found
> > > here:
> > > 
> > > https://people.ipfire.org/~stevee/suricata/
> > > 
> > > To start testing download the tarball and place it on your
> > > IPFire 
> > > system. Extract the tarball and launch the install (install.sh) 
> > > script.
> > > 
> > > If you already have installed a previous test version or image,
> > > with 
> > > the same steps as noted above you can update the the new version.
> > > 
> > > As always, if you prefer a fresh installation, the latest image
> > > can 
> > > be grabbed from here:
> > > 
> > > https://nightly.ipfire.org/next-suricata/latest/x86_64/
> > > 
> > > Direct link for downloading the ISO image:
> > > 
> > > https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x
> > > 86
> > > _64-full-core128.iso
> > > 
> > > Thanks for downloading and testing. There are no known bugs so
> > > far, 
> > > as usual please file any bugs to our bugtracker (
> > > https://bugzilla.ipfire.org) and share your feedback on the list.
> > > 
> > > Best regards,
> > > 
> > > -Stefan
> > > 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 6852 bytes --]


> On 21 Feb 2019, at 21:56, Mentalic <mentalic(a)cox.net> wrote:
> 
> Made an attempt at importing the suricata tarball into current release core127 with guardian installed but no luck. Tar command exits with "This does not look like a tar archive". Had used "tar -xvf" to extract... Could be doing something wrong here?

Good point actually.

@Stevee: What is the migration path away from Guardian? This kind of needs to be uninstalled or the CGI needs to be stripped to remove all IPS features from it.

-Michael

> 
> Regards
> Wayne
> 
> 
> -----Original Message-----
> From: Development [mailto:development-bounces(a)lists.ipfire.org] On Behalf Of Stefan Schantl
> Sent: Wednesday, February 20, 2019 1:55 AM
> To: development(a)lists.ipfire.org
> Subject: Re: IPFire meets Suricata - Call for tester
> 
> Hello again and thanks for the feedback.
>> Exposed my test setup directly to my cable modem and noticed a couple 
>> of things.
>> 
>> -The Firewall log seems to only list items that match my firewall 
>> rules. Gone was the typical several a minute "drop_input" entry noise, 
>> there was zero drop_input's in 15min or so. Possible logging issue?
> 
> The IDS/IPS events are not logged to the firewall log. They only can be accessed in the "Logs"->"IPS Logs" section.
> 
>> 
>> -Suricata placed entries into IPS log, but what is done with them?
>> Don't see a block list like Guardian generated.
> 
> Thats exactly how suricata works and the main benefit why we choose to switch to suricata.
> 
> The old snort/guardian solution worked like this:
> 
> Snort detected (based on it's ruleset) an event and logged it to it's logfile. Guardian read this event from the file, parsed it again and if the configured block count for the matching IP-address was reached, the host was blocked by an iptables rule (block list).
> 
> The new suricata-based solution works like this:
> 
> Suricata detects (also based on the ruleset) an event and directly drops the bad package. There is no additional software involved anymore .
> 
> So one of the benefits of the new approach is to reduce the amount of time an attack has been recognized until it's blocked immediately.
> 
>> 
>> -Are there any incompatibility issues with using the backup function 
>> to restore to this version? I had made a backup from my core 127 
>> system with the old intrusion detection/guardian not active just in 
>> case.
> 
> There is a converter-script available, which will move the old snort/guardian and rules settings to be used by suricata. This script automatically will be called if a backup gets restored, which contains such settings files.
> 
> Therefore I would ask you to test this feature by restoring such a backup on a fresh installed nightly machine and if possible to install the "update tarball" on a regular machine with configured snort and/or guardian.
> 
> In both ways, all your taken settings should be the same for suricata as before for snort.
> 
> A big thanks in advance and best regards,
> 
> -Stefan 
> 
>> 
>> Regards
>> Wayne
>> 
>> -----Original Message-----
>> From: Development [mailto:development-bounces(a)lists.ipfire.org] On 
>> Behalf Of Mentalic
>> Sent: Tuesday, February 19, 2019 4:12 PM
>> To: 'Stefan Schantl'; development(a)lists.ipfire.org
>> Subject: RE: IPFire meets Suricata - Call for tester
>> 
>> Stefan
>> 
>> Yep I had downloaded the nightly and suspected is was not current, and 
>> so posted the build number.
>> 
>> With the 5d7d8749 loaded I have not seen any of the previous issues 
>> nor any others thus far.
>> 
>> Regards
>> Wayne
>> 
>> -----Original Message-----
>> From: Development [mailto:development-bounces(a)lists.ipfire.org] On 
>> Behalf Of Stefan Schantl
>> Sent: Tuesday, February 19, 2019 5:34 AM
>> To: development(a)lists.ipfire.org
>> Subject: Re: IPFire meets Suricata - Call for tester
>> 
>> Hello Wayne,
>> 
>> it seems you accidentally downloaded and tested the wrong image.
>> 
>> The latest one is 5d7d8749 were you downloaded one is an older 
>> release.
>> 
>> Sadly the nightly build service and therefore the images are one day 
>> later than the upgrade tarballs....
>> 
>> You simply can update to this release by using the RC3 tarball or 
>> download the available "5d7d8749" ISO.
>> 
>> Best regards,
>> 
>> -Stefan
>>> Loaded the new iso, reports build 77c07352. Still having connection 
>>> issues with suricata as soon as its activated where existing 
>>> connections would continue to work, no new connections were 
>>> possible.
>>> Reboot results in no connection timeouts. Disable suricata, reboot, 
>>> connections work.
>>> 
>>> Any graphical data trend under Status tab reports errors and remains 
>>> blank. Typically on new installs the trends at least show the chart 
>>> even though data had not been collected.
>>> 
>>> Configured options:
>>> Geoip
>>> Proxy on green and blue
>>> URL filter
>>> suricata on red/blue Running a number of emerging threats rule sets.
>>> 
>>> Regards
>>> Wayne
>>> 
>>> 
>>> 
>>> -----Original Message-----
>>> From: Development [mailto:development-bounces(a)lists.ipfire.org] On 
>>> Behalf Of Stefan Schantl
>>> Sent: Monday, February 18, 2019 7:16 AM
>>> To: development(a)lists.ipfire.org
>>> Subject: Re: IPFire meets Suricata - Call for tester
>>> 
>>> Hello list,
>>> 
>>> I've uploaded the third release candidate, which hopefully would be 
>>> the last one.
>>> 
>>> It fixes the issue that no traffic could be passed through the 
>>> firewall when suricata was running on some machines and no graphs 
>>> could be displayed anymore. Thanks to Wayne for reporting and 
>>> Michael Tremer for testing and fixing.
>>> 
>>> The new tarball (i586 for 32bit-systems, and x86_64) can be found
>>> here:
>>> 
>>> https://people.ipfire.org/~stevee/suricata/
>>> 
>>> To start testing download the tarball and place it on your IPFire 
>>> system. Extract the tarball and launch the install (install.sh) 
>>> script.
>>> 
>>> If you already have installed a previous test version or image, with 
>>> the same steps as noted above you can update the the new version.
>>> 
>>> As always, if you prefer a fresh installation, the latest image can 
>>> be grabbed from here:
>>> 
>>> https://nightly.ipfire.org/next-suricata/latest/x86_64/
>>> 
>>> Direct link for downloading the ISO image:
>>> 
>>> https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x
>>> 86
>>> _64-full-core128.iso
>>> 
>>> Thanks for downloading and testing. There are no known bugs so far, 
>>> as usual please file any bugs to our bugtracker (
>>> https://bugzilla.ipfire.org) and share your feedback on the list.
>>> 
>>> Best regards,
>>> 
>>> -Stefan
>>> 
> 

RE: IPFire meets Suricata - Call for tester

[Mentalic]

[-- Attachment #1: Type: text/plain, Size: 6913 bytes --]

Restored backup from core127 with guardian against a fresh suricata iso install. 
Found:
IPS service was running with no rules downloaded. Box's checked were enabled, monitor, red and blue.
It did import the oinkcode but no rules sets downloaded automatically. Since I had two rule sources in guardian I wondered how that would be handled here were one set is all that's possible. Picked a set and downloaded them, had to manually pick the rules. Not a big deal to setup.

Made an attempt at importing the suricata tarball into current release core127 with guardian installed but no luck. Tar command exits with "This does not look like a tar archive". Had used "tar -xvf" to extract... Could be doing something wrong here?

Regards
Wayne


-----Original Message-----
From: Development [mailto:development-bounces(a)lists.ipfire.org] On Behalf Of Stefan Schantl
Sent: Wednesday, February 20, 2019 1:55 AM
To: development(a)lists.ipfire.org
Subject: Re: IPFire meets Suricata - Call for tester

Hello again and thanks for the feedback.
> Exposed my test setup directly to my cable modem and noticed a couple 
> of things.
> 
> -The Firewall log seems to only list items that match my firewall 
> rules. Gone was the typical several a minute "drop_input" entry noise, 
> there was zero drop_input's in 15min or so. Possible logging issue?

The IDS/IPS events are not logged to the firewall log. They only can be accessed in the "Logs"->"IPS Logs" section.

> 
> -Suricata placed entries into IPS log, but what is done with them?
> Don't see a block list like Guardian generated.

Thats exactly how suricata works and the main benefit why we choose to switch to suricata.

The old snort/guardian solution worked like this:

Snort detected (based on it's ruleset) an event and logged it to it's logfile. Guardian read this event from the file, parsed it again and if the configured block count for the matching IP-address was reached, the host was blocked by an iptables rule (block list).

The new suricata-based solution works like this:

Suricata detects (also based on the ruleset) an event and directly drops the bad package. There is no additional software involved anymore .

So one of the benefits of the new approach is to reduce the amount of time an attack has been recognized until it's blocked immediately.

> 
> -Are there any incompatibility issues with using the backup function 
> to restore to this version? I had made a backup from my core 127 
> system with the old intrusion detection/guardian not active just in 
> case.

There is a converter-script available, which will move the old snort/guardian and rules settings to be used by suricata. This script automatically will be called if a backup gets restored, which contains such settings files.

Therefore I would ask you to test this feature by restoring such a backup on a fresh installed nightly machine and if possible to install the "update tarball" on a regular machine with configured snort and/or guardian.

In both ways, all your taken settings should be the same for suricata as before for snort.

A big thanks in advance and best regards,

-Stefan 

> 
> Regards
> Wayne
> 
> -----Original Message-----
> From: Development [mailto:development-bounces(a)lists.ipfire.org] On 
> Behalf Of Mentalic
> Sent: Tuesday, February 19, 2019 4:12 PM
> To: 'Stefan Schantl'; development(a)lists.ipfire.org
> Subject: RE: IPFire meets Suricata - Call for tester
> 
> Stefan
> 
> Yep I had downloaded the nightly and suspected is was not current, and 
> so posted the build number.
> 
> With the 5d7d8749 loaded I have not seen any of the previous issues 
> nor any others thus far.
> 
> Regards
> Wayne
> 
> -----Original Message-----
> From: Development [mailto:development-bounces(a)lists.ipfire.org] On 
> Behalf Of Stefan Schantl
> Sent: Tuesday, February 19, 2019 5:34 AM
> To: development(a)lists.ipfire.org
> Subject: Re: IPFire meets Suricata - Call for tester
> 
> Hello Wayne,
> 
> it seems you accidentally downloaded and tested the wrong image.
> 
> The latest one is 5d7d8749 were you downloaded one is an older 
> release.
> 
> Sadly the nightly build service and therefore the images are one day 
> later than the upgrade tarballs....
> 
> You simply can update to this release by using the RC3 tarball or 
> download the available "5d7d8749" ISO.
> 
> Best regards,
> 
> -Stefan
> > Loaded the new iso, reports build 77c07352. Still having connection 
> > issues with suricata as soon as its activated where existing 
> > connections would continue to work, no new connections were 
> > possible.
> > Reboot results in no connection timeouts. Disable suricata, reboot, 
> > connections work.
> > 
> > Any graphical data trend under Status tab reports errors and remains 
> > blank. Typically on new installs the trends at least show the chart 
> > even though data had not been collected.
> > 
> > Configured options:
> > Geoip
> > Proxy on green and blue
> > URL filter
> > suricata on red/blue Running a number of emerging threats rule sets.
> > 
> > Regards
> > Wayne
> > 
> > 
> > 
> > -----Original Message-----
> > From: Development [mailto:development-bounces(a)lists.ipfire.org] On 
> > Behalf Of Stefan Schantl
> > Sent: Monday, February 18, 2019 7:16 AM
> > To: development(a)lists.ipfire.org
> > Subject: Re: IPFire meets Suricata - Call for tester
> > 
> > Hello list,
> > 
> > I've uploaded the third release candidate, which hopefully would be 
> > the last one.
> > 
> > It fixes the issue that no traffic could be passed through the 
> > firewall when suricata was running on some machines and no graphs 
> > could be displayed anymore. Thanks to Wayne for reporting and 
> > Michael Tremer for testing and fixing.
> > 
> > The new tarball (i586 for 32bit-systems, and x86_64) can be found
> > here:
> > 
> > https://people.ipfire.org/~stevee/suricata/
> > 
> > To start testing download the tarball and place it on your IPFire 
> > system. Extract the tarball and launch the install (install.sh) 
> > script.
> > 
> > If you already have installed a previous test version or image, with 
> > the same steps as noted above you can update the the new version.
> > 
> > As always, if you prefer a fresh installation, the latest image can 
> > be grabbed from here:
> > 
> > https://nightly.ipfire.org/next-suricata/latest/x86_64/
> > 
> > Direct link for downloading the ISO image:
> > 
> > https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x
> > 86
> > _64-full-core128.iso
> > 
> > Thanks for downloading and testing. There are no known bugs so far, 
> > as usual please file any bugs to our bugtracker (
> > https://bugzilla.ipfire.org) and share your feedback on the list.
> > 
> > Best regards,
> > 
> > -Stefan
> > 

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 5767 bytes --]

Hello again and thanks for the feedback.
> Exposed my test setup directly to my cable modem and noticed a couple
> of things.
> 
> -The Firewall log seems to only list items that match my firewall
> rules. Gone was the typical several a minute "drop_input" entry
> noise, there was zero drop_input's in 15min or so. Possible logging
> issue?

The IDS/IPS events are not logged to the firewall log. They only can be
accessed in the "Logs"->"IPS Logs" section.

> 
> -Suricata placed entries into IPS log, but what is done with them?
> Don't see a block list like Guardian generated.

Thats exactly how suricata works and the main benefit why we choose to
switch to suricata.

The old snort/guardian solution worked like this:

Snort detected (based on it's ruleset) an event and logged it to it's
logfile. Guardian read this event from the file, parsed it again and if
the configured block count for the matching IP-address was reached, the
host was blocked by an iptables rule (block list).

The new suricata-based solution works like this:

Suricata detects (also based on the ruleset) an event and directly
drops the bad package. There is no additional software involved anymore
.

So one of the benefits of the new approach is to reduce the amount of
time an attack has been recognized until it's blocked immediately.

> 
> -Are there any incompatibility issues with using the backup function
> to restore to this version? I had made a backup from my core 127
> system with the old intrusion detection/guardian not active just in
> case.

There is a converter-script available, which will move the old
snort/guardian and rules settings to be used by suricata. This script
automatically will be called if a backup gets restored, which contains
such settings files.

Therefore I would ask you to test this feature by restoring such a
backup on a fresh installed nightly machine and if possible to install
the "update tarball" on a regular machine with configured snort and/or
guardian.

In both ways, all your taken settings should be the same for suricata
as before for snort.

A big thanks in advance and best regards,

-Stefan 

> 
> Regards
> Wayne
> 
> -----Original Message-----
> From: Development [mailto:development-bounces(a)lists.ipfire.org] On
> Behalf Of Mentalic
> Sent: Tuesday, February 19, 2019 4:12 PM
> To: 'Stefan Schantl'; development(a)lists.ipfire.org
> Subject: RE: IPFire meets Suricata - Call for tester
> 
> Stefan
> 
> Yep I had downloaded the nightly and suspected is was not current,
> and so posted the build number.
> 
> With the 5d7d8749 loaded I have not seen any of the previous issues
> nor any others thus far.
> 
> Regards
> Wayne 
> 
> -----Original Message-----
> From: Development [mailto:development-bounces(a)lists.ipfire.org] On
> Behalf Of Stefan Schantl
> Sent: Tuesday, February 19, 2019 5:34 AM
> To: development(a)lists.ipfire.org
> Subject: Re: IPFire meets Suricata - Call for tester
> 
> Hello Wayne,
> 
> it seems you accidentally downloaded and tested the wrong image.
> 
> The latest one is 5d7d8749 were you downloaded one is an older
> release.
> 
> Sadly the nightly build service and therefore the images are one day
> later than the upgrade tarballs....
> 
> You simply can update to this release by using the RC3 tarball or
> download the available "5d7d8749" ISO.
> 
> Best regards,
> 
> -Stefan
> > Loaded the new iso, reports build 77c07352. Still having
> > connection 
> > issues with suricata as soon as its activated where existing 
> > connections would continue to work, no new connections were
> > possible.
> > Reboot results in no connection timeouts. Disable suricata,
> > reboot, 
> > connections work.
> > 
> > Any graphical data trend under Status tab reports errors and
> > remains 
> > blank. Typically on new installs the trends at least show the
> > chart 
> > even though data had not been collected.
> > 
> > Configured options:
> > Geoip
> > Proxy on green and blue
> > URL filter
> > suricata on red/blue Running a number of emerging threats rule
> > sets.
> > 
> > Regards
> > Wayne
> > 
> > 
> > 
> > -----Original Message-----
> > From: Development [mailto:development-bounces(a)lists.ipfire.org] On 
> > Behalf Of Stefan Schantl
> > Sent: Monday, February 18, 2019 7:16 AM
> > To: development(a)lists.ipfire.org
> > Subject: Re: IPFire meets Suricata - Call for tester
> > 
> > Hello list,
> > 
> > I've uploaded the third release candidate, which hopefully would
> > be 
> > the last one.
> > 
> > It fixes the issue that no traffic could be passed through the 
> > firewall when suricata was running on some machines and no graphs 
> > could be displayed anymore. Thanks to Wayne for reporting and
> > Michael 
> > Tremer for testing and fixing.
> > 
> > The new tarball (i586 for 32bit-systems, and x86_64) can be found
> > here:
> > 
> > https://people.ipfire.org/~stevee/suricata/
> > 
> > To start testing download the tarball and place it on your IPFire 
> > system. Extract the tarball and launch the install (install.sh) 
> > script.
> > 
> > If you already have installed a previous test version or image,
> > with 
> > the same steps as noted above you can update the the new version.
> > 
> > As always, if you prefer a fresh installation, the latest image can
> > be 
> > grabbed from here:
> > 
> > https://nightly.ipfire.org/next-suricata/latest/x86_64/
> > 
> > Direct link for downloading the ISO image:
> > 
> > https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86
> > _64-full-core128.iso
> > 
> > Thanks for downloading and testing. There are no known bugs so far,
> > as 
> > usual please file any bugs to our bugtracker (
> > https://bugzilla.ipfire.org) and share your feedback on the list.
> > 
> > Best regards,
> > 
> > -Stefan
> > 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 3462 bytes --]

Hello Wayne,

thanks for the positive feedback - I'm happy for you, that everything
is working fine now.

Best regards,

-Stefan
> Stefan
> 
> Yep I had downloaded the nightly and suspected is was not current,
> and so posted the build number.
> 
> With the 5d7d8749 loaded I have not seen any of the previous issues
> nor any others thus far.
> 
> Regards
> Wayne 
> 
> -----Original Message-----
> From: Development [mailto:development-bounces(a)lists.ipfire.org] On
> Behalf Of Stefan Schantl
> Sent: Tuesday, February 19, 2019 5:34 AM
> To: development(a)lists.ipfire.org
> Subject: Re: IPFire meets Suricata - Call for tester
> 
> Hello Wayne,
> 
> it seems you accidentally downloaded and tested the wrong image.
> 
> The latest one is 5d7d8749 were you downloaded one is an older
> release.
> 
> Sadly the nightly build service and therefore the images are one day
> later than the upgrade tarballs....
> 
> You simply can update to this release by using the RC3 tarball or
> download the available "5d7d8749" ISO.
> 
> Best regards,
> 
> -Stefan
> > Loaded the new iso, reports build 77c07352. Still having
> > connection 
> > issues with suricata as soon as its activated where existing 
> > connections would continue to work, no new connections were
> > possible.
> > Reboot results in no connection timeouts. Disable suricata,
> > reboot, 
> > connections work.
> > 
> > Any graphical data trend under Status tab reports errors and
> > remains 
> > blank. Typically on new installs the trends at least show the
> > chart 
> > even though data had not been collected.
> > 
> > Configured options:
> > Geoip
> > Proxy on green and blue
> > URL filter
> > suricata on red/blue Running a number of emerging threats rule
> > sets.
> > 
> > Regards
> > Wayne
> > 
> > 
> > 
> > -----Original Message-----
> > From: Development [mailto:development-bounces(a)lists.ipfire.org] On 
> > Behalf Of Stefan Schantl
> > Sent: Monday, February 18, 2019 7:16 AM
> > To: development(a)lists.ipfire.org
> > Subject: Re: IPFire meets Suricata - Call for tester
> > 
> > Hello list,
> > 
> > I've uploaded the third release candidate, which hopefully would
> > be 
> > the last one.
> > 
> > It fixes the issue that no traffic could be passed through the 
> > firewall when suricata was running on some machines and no graphs 
> > could be displayed anymore. Thanks to Wayne for reporting and
> > Michael 
> > Tremer for testing and fixing.
> > 
> > The new tarball (i586 for 32bit-systems, and x86_64) can be found
> > here:
> > 
> > https://people.ipfire.org/~stevee/suricata/
> > 
> > To start testing download the tarball and place it on your IPFire 
> > system. Extract the tarball and launch the install (install.sh) 
> > script.
> > 
> > If you already have installed a previous test version or image,
> > with 
> > the same steps as noted above you can update the the new version.
> > 
> > As always, if you prefer a fresh installation, the latest image can
> > be 
> > grabbed from here:
> > 
> > https://nightly.ipfire.org/next-suricata/latest/x86_64/
> > 
> > Direct link for downloading the ISO image:
> > 
> > https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86
> > _64-full-core128.iso
> > 
> > Thanks for downloading and testing. There are no known bugs so far,
> > as 
> > usual please file any bugs to our bugtracker (
> > https://bugzilla.ipfire.org) and share your feedback on the list.
> > 
> > Best regards,
> > 
> > -Stefan
> > 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

RE: IPFire meets Suricata - Call for tester

[Mentalic]

[-- Attachment #1: Type: text/plain, Size: 4091 bytes --]

Exposed my test setup directly to my cable modem and noticed a couple of things.

-The Firewall log seems to only list items that match my firewall rules. Gone was the typical several a minute "drop_input" entry noise, there was zero drop_input's in 15min or so. Possible logging issue?

-Suricata placed entries into IPS log, but what is done with them? Don't see a block list like Guardian generated.

-Are there any incompatibility issues with using the backup function to restore to this version? I had made a backup from my core 127 system with the old intrusion detection/guardian not active just in case.

Regards
Wayne

-----Original Message-----
From: Development [mailto:development-bounces(a)lists.ipfire.org] On Behalf Of Mentalic
Sent: Tuesday, February 19, 2019 4:12 PM
To: 'Stefan Schantl'; development(a)lists.ipfire.org
Subject: RE: IPFire meets Suricata - Call for tester

Stefan

Yep I had downloaded the nightly and suspected is was not current, and so posted the build number.

With the 5d7d8749 loaded I have not seen any of the previous issues nor any others thus far.

Regards
Wayne 

-----Original Message-----
From: Development [mailto:development-bounces(a)lists.ipfire.org] On Behalf Of Stefan Schantl
Sent: Tuesday, February 19, 2019 5:34 AM
To: development(a)lists.ipfire.org
Subject: Re: IPFire meets Suricata - Call for tester

Hello Wayne,

it seems you accidentally downloaded and tested the wrong image.

The latest one is 5d7d8749 were you downloaded one is an older release.

Sadly the nightly build service and therefore the images are one day later than the upgrade tarballs....

You simply can update to this release by using the RC3 tarball or download the available "5d7d8749" ISO.

Best regards,

-Stefan
> Loaded the new iso, reports build 77c07352. Still having connection 
> issues with suricata as soon as its activated where existing 
> connections would continue to work, no new connections were possible.
> Reboot results in no connection timeouts. Disable suricata, reboot, 
> connections work.
> 
> Any graphical data trend under Status tab reports errors and remains 
> blank. Typically on new installs the trends at least show the chart 
> even though data had not been collected.
> 
> Configured options:
> Geoip
> Proxy on green and blue
> URL filter
> suricata on red/blue Running a number of emerging threats rule sets.
> 
> Regards
> Wayne
> 
> 
> 
> -----Original Message-----
> From: Development [mailto:development-bounces(a)lists.ipfire.org] On 
> Behalf Of Stefan Schantl
> Sent: Monday, February 18, 2019 7:16 AM
> To: development(a)lists.ipfire.org
> Subject: Re: IPFire meets Suricata - Call for tester
> 
> Hello list,
> 
> I've uploaded the third release candidate, which hopefully would be 
> the last one.
> 
> It fixes the issue that no traffic could be passed through the 
> firewall when suricata was running on some machines and no graphs 
> could be displayed anymore. Thanks to Wayne for reporting and Michael 
> Tremer for testing and fixing.
> 
> The new tarball (i586 for 32bit-systems, and x86_64) can be found
> here:
> 
> https://people.ipfire.org/~stevee/suricata/
> 
> To start testing download the tarball and place it on your IPFire 
> system. Extract the tarball and launch the install (install.sh) 
> script.
> 
> If you already have installed a previous test version or image, with 
> the same steps as noted above you can update the the new version.
> 
> As always, if you prefer a fresh installation, the latest image can be 
> grabbed from here:
> 
> https://nightly.ipfire.org/next-suricata/latest/x86_64/
> 
> Direct link for downloading the ISO image:
> 
> https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86
> _64-full-core128.iso
> 
> Thanks for downloading and testing. There are no known bugs so far, as 
> usual please file any bugs to our bugtracker (
> https://bugzilla.ipfire.org) and share your feedback on the list.
> 
> Best regards,
> 
> -Stefan
> 

RE: IPFire meets Suricata - Call for tester

[Mentalic]

[-- Attachment #1: Type: text/plain, Size: 3183 bytes --]

Stefan

Yep I had downloaded the nightly and suspected is was not current, and so posted the build number.

With the 5d7d8749 loaded I have not seen any of the previous issues nor any others thus far.

Regards
Wayne 

-----Original Message-----
From: Development [mailto:development-bounces(a)lists.ipfire.org] On Behalf Of Stefan Schantl
Sent: Tuesday, February 19, 2019 5:34 AM
To: development(a)lists.ipfire.org
Subject: Re: IPFire meets Suricata - Call for tester

Hello Wayne,

it seems you accidentally downloaded and tested the wrong image.

The latest one is 5d7d8749 were you downloaded one is an older release.

Sadly the nightly build service and therefore the images are one day later than the upgrade tarballs....

You simply can update to this release by using the RC3 tarball or download the available "5d7d8749" ISO.

Best regards,

-Stefan
> Loaded the new iso, reports build 77c07352. Still having connection 
> issues with suricata as soon as its activated where existing 
> connections would continue to work, no new connections were possible.
> Reboot results in no connection timeouts. Disable suricata, reboot, 
> connections work.
> 
> Any graphical data trend under Status tab reports errors and remains 
> blank. Typically on new installs the trends at least show the chart 
> even though data had not been collected.
> 
> Configured options:
> Geoip
> Proxy on green and blue
> URL filter
> suricata on red/blue Running a number of emerging threats rule sets.
> 
> Regards
> Wayne
> 
> 
> 
> -----Original Message-----
> From: Development [mailto:development-bounces(a)lists.ipfire.org] On 
> Behalf Of Stefan Schantl
> Sent: Monday, February 18, 2019 7:16 AM
> To: development(a)lists.ipfire.org
> Subject: Re: IPFire meets Suricata - Call for tester
> 
> Hello list,
> 
> I've uploaded the third release candidate, which hopefully would be 
> the last one.
> 
> It fixes the issue that no traffic could be passed through the 
> firewall when suricata was running on some machines and no graphs 
> could be displayed anymore. Thanks to Wayne for reporting and Michael 
> Tremer for testing and fixing.
> 
> The new tarball (i586 for 32bit-systems, and x86_64) can be found
> here:
> 
> https://people.ipfire.org/~stevee/suricata/
> 
> To start testing download the tarball and place it on your IPFire 
> system. Extract the tarball and launch the install (install.sh) 
> script.
> 
> If you already have installed a previous test version or image, with 
> the same steps as noted above you can update the the new version.
> 
> As always, if you prefer a fresh installation, the latest image can be 
> grabbed from here:
> 
> https://nightly.ipfire.org/next-suricata/latest/x86_64/
> 
> Direct link for downloading the ISO image:
> 
> https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86
> _64-full-core128.iso
> 
> Thanks for downloading and testing. There are no known bugs so far, as 
> usual please file any bugs to our bugtracker (
> https://bugzilla.ipfire.org) and share your feedback on the list.
> 
> Best regards,
> 
> -Stefan
> 

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 2680 bytes --]

Hello Wayne,

it seems you accidentally downloaded and tested the wrong image.

The latest one is 5d7d8749 were you downloaded one is an older release.

Sadly the nightly build service and therefore the images are one day
later than the upgrade tarballs....

You simply can update to this release by using the RC3 tarball or
download the available "5d7d8749" ISO.

Best regards,

-Stefan
> Loaded the new iso, reports build 77c07352. Still having connection
> issues with suricata as soon as its activated where existing
> connections would continue to work, no new connections were possible.
> Reboot results in no connection timeouts. Disable suricata, reboot,
> connections work.
> 
> Any graphical data trend under Status tab reports errors and remains
> blank. Typically on new installs the trends at least show the chart
> even though data had not been collected.
> 
> Configured options:
> Geoip
> Proxy on green and blue
> URL filter
> suricata on red/blue Running a number of emerging threats rule sets.
> 
> Regards
> Wayne
> 
> 
> 
> -----Original Message-----
> From: Development [mailto:development-bounces(a)lists.ipfire.org] On
> Behalf Of Stefan Schantl
> Sent: Monday, February 18, 2019 7:16 AM
> To: development(a)lists.ipfire.org
> Subject: Re: IPFire meets Suricata - Call for tester
> 
> Hello list,
> 
> I've uploaded the third release candidate, which hopefully would be
> the last one.
> 
> It fixes the issue that no traffic could be passed through the
> firewall when suricata was running on some machines and no graphs
> could be displayed anymore. Thanks to Wayne for reporting and Michael
> Tremer for testing and fixing.
> 
> The new tarball (i586 for 32bit-systems, and x86_64) can be found
> here:
> 
> https://people.ipfire.org/~stevee/suricata/
> 
> To start testing download the tarball and place it on your IPFire
> system. Extract the tarball and launch the install (install.sh)
> script.
> 
> If you already have installed a previous test version or image, with
> the same steps as noted above you can update the the new version.
> 
> As always, if you prefer a fresh installation, the latest image can
> be grabbed from here:
> 
> https://nightly.ipfire.org/next-suricata/latest/x86_64/
> 
> Direct link for downloading the ISO image:
> 
> https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso
> 
> Thanks for downloading and testing. There are no known bugs so far,
> as usual please file any bugs to our bugtracker (
> https://bugzilla.ipfire.org) and share your feedback on the list.
> 
> Best regards,
> 
> -Stefan
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

RE: IPFire meets Suricata - Call for tester

[Mentalic]

[-- Attachment #1: Type: text/plain, Size: 2126 bytes --]

Loaded the new iso, reports build 77c07352. Still having connection issues with suricata as soon as its activated where existing connections would continue to work, no new connections were possible. Reboot results in no connection timeouts. Disable suricata, reboot, connections work.

Any graphical data trend under Status tab reports errors and remains blank. Typically on new installs the trends at least show the chart even though data had not been collected.

Configured options:
Geoip
Proxy on green and blue
URL filter
suricata on red/blue Running a number of emerging threats rule sets.

Regards
Wayne



-----Original Message-----
From: Development [mailto:development-bounces(a)lists.ipfire.org] On Behalf Of Stefan Schantl
Sent: Monday, February 18, 2019 7:16 AM
To: development(a)lists.ipfire.org
Subject: Re: IPFire meets Suricata - Call for tester

Hello list,

I've uploaded the third release candidate, which hopefully would be the last one.

It fixes the issue that no traffic could be passed through the firewall when suricata was running on some machines and no graphs could be displayed anymore. Thanks to Wayne for reporting and Michael Tremer for testing and fixing.

The new tarball (i586 for 32bit-systems, and x86_64) can be found here:

https://people.ipfire.org/~stevee/suricata/

To start testing download the tarball and place it on your IPFire system. Extract the tarball and launch the install (install.sh) script.

If you already have installed a previous test version or image, with the same steps as noted above you can update the the new version.

As always, if you prefer a fresh installation, the latest image can be grabbed from here:

https://nightly.ipfire.org/next-suricata/latest/x86_64/

Direct link for downloading the ISO image:

https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso

Thanks for downloading and testing. There are no known bugs so far, as usual please file any bugs to our bugtracker (
https://bugzilla.ipfire.org) and share your feedback on the list.

Best regards,

-Stefan

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 1249 bytes --]

Hello list,

I've uploaded the third release candidate, which hopefully would be the
last one.

It fixes the issue that no traffic could be passed through the firewall
when suricata was running on some machines and no graphs could be
displayed anymore. Thanks to Wayne for reporting and Michael Tremer for
testing and fixing.

The new tarball (i586 for 32bit-systems, and x86_64) can be found here:

https://people.ipfire.org/~stevee/suricata/

To start testing download the tarball and place it on your IPFire
system. Extract the tarball and launch the install (install.sh) script.

If you already have installed a previous test version or image, with
the same steps as noted above you can update the the new version.

As always, if you prefer a fresh installation, the latest image can be
grabbed
from here:

https://nightly.ipfire.org/next-suricata/latest/x86_64/

Direct link for downloading the ISO image:

https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso

Thanks for downloading and testing. There are no known bugs so far, as
usual please file any bugs to our bugtracker (
https://bugzilla.ipfire.org) and share your feedback on the list.

Best regards,

-Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 4395 bytes --]

Hello Michael,

thanks for the feedback, nice to hear, that this issue is fixed now.

Best regards,

-Stefan
> Can confirm that this works now for me...
> 
> > On 17 Feb 2019, at 19:57, Stefan Schantl <stefan.schantl(a)ipfire.org
> > > wrote:
> > 
> > Hello Michael,
> > 
> > thanks for your feedback.
> > > Hello Suricata Testing Community,
> > > Hello Stefan,
> > > 
> > > I just installed the “rc2” image on my production system on my
> > > desk.
> > > 
> > > I am afraid that I can confirm that no new connections are
> > > possible
> > > any more after Suricata is being started. I suppose this is due
> > > to
> > > some of the latest changes to the suricata configuration file.
> > > The
> > > iptables chains look fine and some other traffic continues to
> > > pass.
> > > 
> > > Not sure what I can do about this now.
> > 
> > Finally I figured out, why this happened to you and Wayne (which
> > also
> > reported this issue) and I was not able to reproduce that.
> > 
> > During development we have got this issue one, because of SNAT used
> > the
> > default mark of "1" to mark it's packets. This internal mark will
> > be
> > increased by each interface, so if you are using blue or orange
> > too,
> > the mark "2" (which currently is used by suricata) also is in use.
> > 
> > If all 4 possible interfaces are present, the mark "3" also is in
> > use
> > for SNAT.
> > 
> > Snip from "iptables -L -v -n -t nat"
> > 
> > Chain NAT_DESTINATION_FIX (1 references)
> > pkts bytes target     prot opt
> > in     out     source               destination         
> >    0     0 SNAT       all  
> > --  *      *       0.0.0.0/0            0.0.0.0/0            mark
> > match
> > 0x1 to:192.168.xxx.xxx
> >  728 83711 SNAT       all  
> > --  *      *       0.0.0.0/0            0.0.0.0/0            mark
> > match
> > 0x2 to:192.168.xxx.xxx
> >    0     0 SNAT       all  
> > --  *      *       0.0.0.0/0            0.0.0.0/0            mark
> > match
> > 0x3 to:192.168.xxx.xxx
> > 
> > So we have at least to use "4" to mark the packets which are
> > inspected
> > by suricata - which worked on the testmachine with full interface
> > configuration.
> > 
> > But what happened if OpenVPN or IPsec is also in use and clients
> > are
> > connected ? Will there be spawned any additional rules with marks
> > and
> > we ran into the same issue again ? What would be a good mark
> > default
> > for suricata to prevent from this ?
> > 
> > Thanks in advance,
> > 
> > -Stefan
> > 
> > > I found that this is a bug: 
> > > https://bugzilla.ipfire.org/show_bug.cgi?id=12002
> > > 
> > > -Michael
> > > 
> > > > On 17 Feb 2019, at 11:58, Stefan Schantl <
> > > > stefan.schantl(a)ipfire.org
> > > > > wrote:
> > > > 
> > > > Hello list,
> > > > 
> > > > a short note from suricata development. I've uploaded the
> > > > second
> > > > release candidate, which fixes several issues and bugs.
> > > > 
> > > > Now, the "services.cgi" will correctly show the IPS as running,
> > > > and
> > > > logrotate and collectd will handle the correct service.
> > > > 
> > > > The new tarball (i586 for 32bit-systems, and x86_64) can be
> > > > found
> > > > here:
> > > > 
> > > > https://people.ipfire.org/~stevee/suricata/
> > > > 
> > > > To start testing download the tarball and place it on your
> > > > IPFire
> > > > system. Extract the tarball and launch the install (install.sh)
> > > > script.
> > > > 
> > > > If you already have installed a previous test version or image,
> > > > with
> > > > the same steps as noted above you can update the the new
> > > > version.
> > > > 
> > > > As always, if you prefer a fresh installation, the latest image
> > > > can
> > > > be
> > > > grabbed
> > > > from here:
> > > > 
> > > > https://nightly.ipfire.org/next-suricata/latest/x86_64/
> > > > 
> > > > Direct link for downloading the ISO image:
> > > > 
> > > > https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso
> > > > 
> > > > Thanks for downloading and testing. There are no known bugs so
> > > > far,
> > > > as
> > > > usual please file any bugs to our bugtracker (
> > > > https://bugzilla.ipfire.org) and share your feedback on the
> > > > list.
> > > > 
> > > > Best regards,
> > > > 
> > > > -Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 3788 bytes --]

Can confirm that this works now for me...

> On 17 Feb 2019, at 19:57, Stefan Schantl <stefan.schantl(a)ipfire.org> wrote:
> 
> Hello Michael,
> 
> thanks for your feedback.
>> Hello Suricata Testing Community,
>> Hello Stefan,
>> 
>> I just installed the “rc2” image on my production system on my desk.
>> 
>> I am afraid that I can confirm that no new connections are possible
>> any more after Suricata is being started. I suppose this is due to
>> some of the latest changes to the suricata configuration file. The
>> iptables chains look fine and some other traffic continues to pass.
>> 
>> Not sure what I can do about this now.
> 
> Finally I figured out, why this happened to you and Wayne (which also
> reported this issue) and I was not able to reproduce that.
> 
> During development we have got this issue one, because of SNAT used the
> default mark of "1" to mark it's packets. This internal mark will be
> increased by each interface, so if you are using blue or orange too,
> the mark "2" (which currently is used by suricata) also is in use.
> 
> If all 4 possible interfaces are present, the mark "3" also is in use
> for SNAT.
> 
> Snip from "iptables -L -v -n -t nat"
> 
> Chain NAT_DESTINATION_FIX (1 references)
> pkts bytes target     prot opt
> in     out     source               destination         
>    0     0 SNAT       all  
> --  *      *       0.0.0.0/0            0.0.0.0/0            mark match
> 0x1 to:192.168.xxx.xxx
>  728 83711 SNAT       all  
> --  *      *       0.0.0.0/0            0.0.0.0/0            mark match
> 0x2 to:192.168.xxx.xxx
>    0     0 SNAT       all  
> --  *      *       0.0.0.0/0            0.0.0.0/0            mark match
> 0x3 to:192.168.xxx.xxx
> 
> So we have at least to use "4" to mark the packets which are inspected
> by suricata - which worked on the testmachine with full interface
> configuration.
> 
> But what happened if OpenVPN or IPsec is also in use and clients are
> connected ? Will there be spawned any additional rules with marks and
> we ran into the same issue again ? What would be a good mark default
> for suricata to prevent from this ?
> 
> Thanks in advance,
> 
> -Stefan
> 
>> 
>> I found that this is a bug: 
>> https://bugzilla.ipfire.org/show_bug.cgi?id=12002
>> 
>> -Michael
>> 
>>> On 17 Feb 2019, at 11:58, Stefan Schantl <stefan.schantl(a)ipfire.org
>>>> wrote:
>>> 
>>> Hello list,
>>> 
>>> a short note from suricata development. I've uploaded the second
>>> release candidate, which fixes several issues and bugs.
>>> 
>>> Now, the "services.cgi" will correctly show the IPS as running, and
>>> logrotate and collectd will handle the correct service.
>>> 
>>> The new tarball (i586 for 32bit-systems, and x86_64) can be found
>>> here:
>>> 
>>> https://people.ipfire.org/~stevee/suricata/
>>> 
>>> To start testing download the tarball and place it on your IPFire
>>> system. Extract the tarball and launch the install (install.sh)
>>> script.
>>> 
>>> If you already have installed a previous test version or image,
>>> with
>>> the same steps as noted above you can update the the new version.
>>> 
>>> As always, if you prefer a fresh installation, the latest image can
>>> be
>>> grabbed
>>> from here:
>>> 
>>> https://nightly.ipfire.org/next-suricata/latest/x86_64/
>>> 
>>> Direct link for downloading the ISO image:
>>> 
>>> https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso
>>> 
>>> Thanks for downloading and testing. There are no known bugs so far,
>>> as
>>> usual please file any bugs to our bugtracker (
>>> https://bugzilla.ipfire.org) and share your feedback on the list.
>>> 
>>> Best regards,
>>> 
>>> -Stefan

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 3558 bytes --]

Hello Michael,

thanks for your feedback.
> Hello Suricata Testing Community,
> Hello Stefan,
> 
> I just installed the “rc2” image on my production system on my desk.
> 
> I am afraid that I can confirm that no new connections are possible
> any more after Suricata is being started. I suppose this is due to
> some of the latest changes to the suricata configuration file. The
> iptables chains look fine and some other traffic continues to pass.
> 
> Not sure what I can do about this now.

Finally I figured out, why this happened to you and Wayne (which also
reported this issue) and I was not able to reproduce that.

During development we have got this issue one, because of SNAT used the
default mark of "1" to mark it's packets. This internal mark will be
increased by each interface, so if you are using blue or orange too,
the mark "2" (which currently is used by suricata) also is in use.

If all 4 possible interfaces are present, the mark "3" also is in use
for SNAT.

Snip from "iptables -L -v -n -t nat"

Chain NAT_DESTINATION_FIX (1 references)
 pkts bytes target     prot opt
in     out     source               destination         
    0     0 SNAT       all  
--  *      *       0.0.0.0/0            0.0.0.0/0            mark match
0x1 to:192.168.xxx.xxx
  728 83711 SNAT       all  
--  *      *       0.0.0.0/0            0.0.0.0/0            mark match
0x2 to:192.168.xxx.xxx
    0     0 SNAT       all  
--  *      *       0.0.0.0/0            0.0.0.0/0            mark match
0x3 to:192.168.xxx.xxx

So we have at least to use "4" to mark the packets which are inspected
by suricata - which worked on the testmachine with full interface
configuration.

But what happened if OpenVPN or IPsec is also in use and clients are
connected ? Will there be spawned any additional rules with marks and
we ran into the same issue again ? What would be a good mark default
for suricata to prevent from this ?

Thanks in advance,

-Stefan

> 
> I found that this is a bug: 
> https://bugzilla.ipfire.org/show_bug.cgi?id=12002
> 
> -Michael
> 
> > On 17 Feb 2019, at 11:58, Stefan Schantl <stefan.schantl(a)ipfire.org
> > > wrote:
> > 
> > Hello list,
> > 
> > a short note from suricata development. I've uploaded the second
> > release candidate, which fixes several issues and bugs.
> > 
> > Now, the "services.cgi" will correctly show the IPS as running, and
> > logrotate and collectd will handle the correct service.
> > 
> > The new tarball (i586 for 32bit-systems, and x86_64) can be found
> > here:
> > 
> > https://people.ipfire.org/~stevee/suricata/
> > 
> > To start testing download the tarball and place it on your IPFire
> > system. Extract the tarball and launch the install (install.sh)
> > script.
> > 
> > If you already have installed a previous test version or image,
> > with
> > the same steps as noted above you can update the the new version.
> > 
> > As always, if you prefer a fresh installation, the latest image can
> > be
> > grabbed
> > from here:
> > 
> > https://nightly.ipfire.org/next-suricata/latest/x86_64/
> > 
> > Direct link for downloading the ISO image:
> > 
> > https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso
> > 
> > Thanks for downloading and testing. There are no known bugs so far,
> > as
> > usual please file any bugs to our bugtracker (
> > https://bugzilla.ipfire.org) and share your feedback on the list.
> > 
> > Best regards,
> > 
> > -Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 1873 bytes --]

Hello Suricata Testing Community,
Hello Stefan,

I just installed the “rc2” image on my production system on my desk.

I am afraid that I can confirm that no new connections are possible any more after Suricata is being started. I suppose this is due to some of the latest changes to the suricata configuration file. The iptables chains look fine and some other traffic continues to pass.

Not sure what I can do about this now.

I found that this is a bug: https://bugzilla.ipfire.org/show_bug.cgi?id=12002

-Michael

> On 17 Feb 2019, at 11:58, Stefan Schantl <stefan.schantl(a)ipfire.org> wrote:
> 
> Hello list,
> 
> a short note from suricata development. I've uploaded the second
> release candidate, which fixes several issues and bugs.
> 
> Now, the "services.cgi" will correctly show the IPS as running, and
> logrotate and collectd will handle the correct service.
> 
> The new tarball (i586 for 32bit-systems, and x86_64) can be found here:
> 
> https://people.ipfire.org/~stevee/suricata/
> 
> To start testing download the tarball and place it on your IPFire
> system. Extract the tarball and launch the install (install.sh) script.
> 
> If you already have installed a previous test version or image, with
> the same steps as noted above you can update the the new version.
> 
> As always, if you prefer a fresh installation, the latest image can be
> grabbed
> from here:
> 
> https://nightly.ipfire.org/next-suricata/latest/x86_64/
> 
> Direct link for downloading the ISO image:
> 
> https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso
> 
> Thanks for downloading and testing. There are no known bugs so far, as
> usual please file any bugs to our bugtracker (
> https://bugzilla.ipfire.org) and share your feedback on the list.
> 
> Best regards,
> 
> -Stefan

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 1180 bytes --]

Hello list,

a short note from suricata development. I've uploaded the second
release candidate, which fixes several issues and bugs.

Now, the "services.cgi" will correctly show the IPS as running, and
logrotate and collectd will handle the correct service.

The new tarball (i586 for 32bit-systems, and x86_64) can be found here:

https://people.ipfire.org/~stevee/suricata/

To start testing download the tarball and place it on your IPFire
system. Extract the tarball and launch the install (install.sh) script.

If you already have installed a previous test version or image, with
the same steps as noted above you can update the the new version.

As always, if you prefer a fresh installation, the latest image can be
grabbed
from here:

https://nightly.ipfire.org/next-suricata/latest/x86_64/

Direct link for downloading the ISO image:

https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso

Thanks for downloading and testing. There are no known bugs so far, as
usual please file any bugs to our bugtracker (
https://bugzilla.ipfire.org) and share your feedback on the list.

Best regards,

-Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 2956 bytes --]

Hello Wayne,

thanks for the additional feedback, I answered the first part in the
previous mail.

> Reinstalled, IPS ran until I poked a few rule sets active then it
> blocked all red regardless if I unchecked the rules. Un-checking IPS
> box's no good, had to reboot.
> IPS service listings under Status>services never go green.
> 

Of course, this is a bug, because the CGI script needs to be adjusted
too, to work with suricata instead of snort. Thanks for reporting it.

Best regards,

-Stefan

> Regards
> Wayne
> 
> -----Original Message-----
> From: Mentalic [mailto:mentalic(a)cox.net] 
> Sent: Thursday, February 14, 2019 5:37 PM
> To: 'Stefan Schantl'; 'development(a)lists.ipfire.org'
> Subject: RE: IPFire meets Suricata - Call for tester
> 
> Used the download image link and loaded on my test box, noticed a few
> issues with it.  
> 
> -When Intrusion monitoring is enabled on red interface all traffic to
> red stops with empty IDS log.
> -Intrusion detection Seems to be limited to one rules list source at
> a time. Maybe feature of Suricata?
> -Under "status" tab, any graphic trend display fails to load with
> "contains errors"
> 
> Regards
> Wayne
> 
> -----Original Message-----
> From: Development [mailto:development-bounces(a)lists.ipfire.org] On
> Behalf Of Stefan Schantl
> Sent: Thursday, February 14, 2019 8:29 AM
> To: development(a)lists.ipfire.org
> Subject: Re: IPFire meets Suricata - Call for tester
> 
> Hello list,
> 
> suricata development goes on, so I'm happy to announce the first
> release candidate on this list.
> 
> The biggest difference to any previous versions is, this time an
> update tarball for any IPFire installations is available. So anybody
> now easily can download and install it on an existing system without
> doing a fresh installation.
> 
> The tarball (currently only x86_64) can be grabbed from here:
> 
> https://people.ipfire.org/~stevee/suricata/
> 
> To start testing download the tarball and place it on your IPFire
> system. Extract the tarball and launch the install (install.sh)
> script.
> 
> During installation snort will be stopped (if running), new files
> will be placed on the system, your taken snort settings will be
> converted to get used by suricata and finally suricata will be
> started (if snort was used before) to start protecting the configured
> networks.
> 
> If you prefer a fresh installation, the latest image can be grabbed
> from here:
> 
> https://nightly.ipfire.org/next-suricata/latest/x86_64/
> 
> Direct link for downloading the ISO image:
> 
> https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso
> 
> Thanks for downloading and testing. There are no known bugs so far,
> as usual please file any bugs to our bugtracker (
> https://bugzilla.ipfire.org) and share your feedback on the list.
> 
> Best regards,
> 
> -Stefan
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 2754 bytes --]

Hello Wayne,

thanks for joining testing and sharing your feedback with this list.
> Used the download image link and loaded on my test box, noticed a few
> issues with it.  
> 
> -When Intrusion monitoring is enabled on red interface all traffic to
> red stops with empty IDS log.

Indeed this is not an expected behavior. Please provide some more
details about the used version (did you updated the ISO-release with
the tarball-based RC1 release? ) configuration etc. to figure out what
happened here. 

> -Intrusion detection Seems to be limited to one rules list source at
> a time. Maybe feature of Suricata?

This works as designed, only one Ruleset-Vendor at the same time can be
used.

> -Under "status" tab, any graphic trend display fails to load with
> "contains errors"

This take some time to collect data and generate the graphs. Please be
patient, if after one day still no graphs a generated please report
back.

Best regards,

-Stefan

> 
> Regards
> Wayne
> 
> -----Original Message-----
> From: Development [mailto:development-bounces(a)lists.ipfire.org] On
> Behalf Of Stefan Schantl
> Sent: Thursday, February 14, 2019 8:29 AM
> To: development(a)lists.ipfire.org
> Subject: Re: IPFire meets Suricata - Call for tester
> 
> Hello list,
> 
> suricata development goes on, so I'm happy to announce the first
> release candidate on this list.
> 
> The biggest difference to any previous versions is, this time an
> update tarball for any IPFire installations is available. So anybody
> now easily can download and install it on an existing system without
> doing a fresh installation.
> 
> The tarball (currently only x86_64) can be grabbed from here:
> 
> https://people.ipfire.org/~stevee/suricata/
> 
> To start testing download the tarball and place it on your IPFire
> system. Extract the tarball and launch the install (install.sh)
> script.
> 
> During installation snort will be stopped (if running), new files
> will be placed on the system, your taken snort settings will be
> converted to get used by suricata and finally suricata will be
> started (if snort was used before) to start protecting the configured
> networks.
> 
> If you prefer a fresh installation, the latest image can be grabbed
> from here:
> 
> https://nightly.ipfire.org/next-suricata/latest/x86_64/
> 
> Direct link for downloading the ISO image:
> 
> https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso
> 
> Thanks for downloading and testing. There are no known bugs so far,
> as usual please file any bugs to our bugtracker (
> https://bugzilla.ipfire.org) and share your feedback on the list.
> 
> Best regards,
> 
> -Stefan
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

RE: IPFire meets Suricata - Call for tester

[Mentalic]

[-- Attachment #1: Type: text/plain, Size: 2509 bytes --]

Reinstalled, IPS ran until I poked a few rule sets active then it blocked all red regardless if I unchecked the rules. Un-checking IPS box's no good, had to reboot.
IPS service listings under Status>services never go green.

Regards
Wayne

-----Original Message-----
From: Mentalic [mailto:mentalic(a)cox.net] 
Sent: Thursday, February 14, 2019 5:37 PM
To: 'Stefan Schantl'; 'development(a)lists.ipfire.org'
Subject: RE: IPFire meets Suricata - Call for tester

Used the download image link and loaded on my test box, noticed a few issues with it.  

-When Intrusion monitoring is enabled on red interface all traffic to red stops with empty IDS log.
-Intrusion detection Seems to be limited to one rules list source at a time. Maybe feature of Suricata?
-Under "status" tab, any graphic trend display fails to load with "contains errors"

Regards
Wayne

-----Original Message-----
From: Development [mailto:development-bounces(a)lists.ipfire.org] On Behalf Of Stefan Schantl
Sent: Thursday, February 14, 2019 8:29 AM
To: development(a)lists.ipfire.org
Subject: Re: IPFire meets Suricata - Call for tester

Hello list,

suricata development goes on, so I'm happy to announce the first release candidate on this list.

The biggest difference to any previous versions is, this time an update tarball for any IPFire installations is available. So anybody now easily can download and install it on an existing system without doing a fresh installation.

The tarball (currently only x86_64) can be grabbed from here:

https://people.ipfire.org/~stevee/suricata/

To start testing download the tarball and place it on your IPFire system. Extract the tarball and launch the install (install.sh) script.

During installation snort will be stopped (if running), new files will be placed on the system, your taken snort settings will be converted to get used by suricata and finally suricata will be started (if snort was used before) to start protecting the configured networks.

If you prefer a fresh installation, the latest image can be grabbed from here:

https://nightly.ipfire.org/next-suricata/latest/x86_64/

Direct link for downloading the ISO image:

https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso

Thanks for downloading and testing. There are no known bugs so far, as usual please file any bugs to our bugtracker (
https://bugzilla.ipfire.org) and share your feedback on the list.

Best regards,

-Stefan

RE: IPFire meets Suricata - Call for tester

[Mentalic]

[-- Attachment #1: Type: text/plain, Size: 2035 bytes --]

Used the download image link and loaded on my test box, noticed a few issues with it.  

-When Intrusion monitoring is enabled on red interface all traffic to red stops with empty IDS log.
-Intrusion detection Seems to be limited to one rules list source at a time. Maybe feature of Suricata?
-Under "status" tab, any graphic trend display fails to load with "contains errors"

Regards
Wayne

-----Original Message-----
From: Development [mailto:development-bounces(a)lists.ipfire.org] On Behalf Of Stefan Schantl
Sent: Thursday, February 14, 2019 8:29 AM
To: development(a)lists.ipfire.org
Subject: Re: IPFire meets Suricata - Call for tester

Hello list,

suricata development goes on, so I'm happy to announce the first release candidate on this list.

The biggest difference to any previous versions is, this time an update tarball for any IPFire installations is available. So anybody now easily can download and install it on an existing system without doing a fresh installation.

The tarball (currently only x86_64) can be grabbed from here:

https://people.ipfire.org/~stevee/suricata/

To start testing download the tarball and place it on your IPFire system. Extract the tarball and launch the install (install.sh) script.

During installation snort will be stopped (if running), new files will be placed on the system, your taken snort settings will be converted to get used by suricata and finally suricata will be started (if snort was used before) to start protecting the configured networks.

If you prefer a fresh installation, the latest image can be grabbed from here:

https://nightly.ipfire.org/next-suricata/latest/x86_64/

Direct link for downloading the ISO image:

https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso

Thanks for downloading and testing. There are no known bugs so far, as usual please file any bugs to our bugtracker (
https://bugzilla.ipfire.org) and share your feedback on the list.

Best regards,

-Stefan

Re: IPFire meets Suricata - Call for tester

[Matthias Fischer]

[-- Attachment #1: Type: text/plain, Size: 538 bytes --]

On 14.02.2019 22:49, Stefan Schantl wrote:
> Hello Matthias,
>> ...
> 
> Unfortunately I created this tarball by hand, so there is not build
> script available.

OK.

> If you extract the "inner" tarball by hand, replace the libraries,
> located in usr/lib/, the suricata and suricatctrl binaries and repack
> the tarball you should be fine.

No problem - that's what I'm planning then.

> Tomorrow I'll start a 32 Bit build and do this, to provide a tarball
> for 32bit installations too.

Thanks - that would be great.

Best,
Matthias

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 1234 bytes --]

Hello Matthias,
> Hi Stefan,
> 
> On 14.02.2019 15:28, Stefan Schantl wrote:
> > suricata development goes on, so I'm happy to announce the first
> > release candidate on this list.
> > 
> > The biggest difference to any previous versions is, this time an
> > update
> > tarball for any IPFire installations is available. So anybody now
> > easily can download and install it on an existing system without
> > doing
> > a fresh installation.> The tarball (currently only x86_64)...
> 
> Since I'm still on 32bit, I got a problem here.
> 
> Building should be easy. I already created a - fetched -
> 'stevee_suricata'-branch here. 'Devel' is running...
> 
> But creating the tarball could. Did you build this tarball 'by hand'
> or
> script? If this was scripted, could you send me the script, than
> perhaps
> I could join in...
> 


Unfortunately I created this tarball by hand, so there is not build
script available.

If you extract the "inner" tarball by hand, replace the libraries,
located in usr/lib/, the suricata and suricatctrl binaries and repack
the tarball you should be fine.

Tomorrow I'll start a 32 Bit build and do this, to provide a tarball
for 32bit installations too.
 
> Best,
> Matthias

Best regards,

-Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Matthias Fischer]

[-- Attachment #1: Type: text/plain, Size: 779 bytes --]

Hi Stefan,

On 14.02.2019 15:28, Stefan Schantl wrote:
> suricata development goes on, so I'm happy to announce the first
> release candidate on this list.
> 
> The biggest difference to any previous versions is, this time an update
> tarball for any IPFire installations is available. So anybody now
> easily can download and install it on an existing system without doing
> a fresh installation.> The tarball (currently only x86_64)...

Since I'm still on 32bit, I got a problem here.

Building should be easy. I already created a - fetched -
'stevee_suricata'-branch here. 'Devel' is running...

But creating the tarball could. Did you build this tarball 'by hand' or
script? If this was scripted, could you send me the script, than perhaps
I could join in...

Best,
Matthias

Re: IPFire meets Suricata - Call for tester

[ummeegge]

[-- Attachment #1: Type: text/plain, Size: 119 bytes --]

Great work !

Big thanks to you and all the other helping hands and minds :-).

Am in testing round now.

Best,

Erik

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 1383 bytes --]

Hello list,

suricata development goes on, so I'm happy to announce the first
release candidate on this list.

The biggest difference to any previous versions is, this time an update
tarball for any IPFire installations is available. So anybody now
easily can download and install it on an existing system without doing
a fresh installation.

The tarball (currently only x86_64) can be grabbed from here:

https://people.ipfire.org/~stevee/suricata/

To start testing download the tarball and place it on your IPFire
system. Extract the tarball and launch the install (install.sh) script.

During installation snort will be stopped (if running), new files will
be placed on the system, your taken snort settings will be converted to
get used by suricata and finally suricata will be started (if snort was
used before) to start protecting the configured networks.

If you prefer a fresh installation, the latest image can be grabbed
from here:

https://nightly.ipfire.org/next-suricata/latest/x86_64/

Direct link for downloading the ISO image:

https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.iso

Thanks for downloading and testing. There are no known bugs so far, as
usual please file any bugs to our bugtracker (
https://bugzilla.ipfire.org) and share your feedback on the list.

Best regards,

-Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 2515 bytes --]

Hello list,

today im very happy to announce a new test image with the latest
snapshot of the process bringing suricata to all of you.

The image is now hosted and provided by the nightly build feature (a
big thanks to Michael for providing this) of IPFire, so if development
goes on, every time a new image will be generated and easily can be
downloaded.

The latest image always can be grabbed from here:

https://nightly.ipfire.org/next-suricata/latest/x86_64/

Direct link for downloading the ISO image:

https://nightly.ipfire.org/next-suricata/latest/x86_64/ipfire-2.21.x86_64-full-core128.isof

There is currently one known issue, that any kind of snort rules
(sourcefire) currenty can not be downloaded, so you have to use the
rulesets from emergingthreads for testing. This issue will be fixed
with the next image provided by the nightly build service.

Thanks for downloading and testing, as usual please file any bugs to
our bugtracker (https://bugzilla.ipfire.org) and share your feedback on
this list.

Best regards,

-Stefan

 
> Hello list followers,
> 
> some time ago development for the new implementation of the Intrusion
> Detection functionality in IPFire has been started.
> 
> The main goal, in a nutshell, was to give IPFire a modern, feature-
> rich 
> and user-friendly Intrusion Detection Engine. During this progress,
> the
> detection framework has been replaced - now suricata is used instead
> of
> snort.
> 
> Suricata uses a very modern and multi-threaded detection engine with
> support to perform actions on malicious traffic. So it provides the
> functionality of detecting any kind of intrusion attempts and the
> ability of guardian to block them under the same hood.
> 
> It was a lot of work, but finaly I'm happy to announce the first test
> version. It is almost feature complete and without any kind of bigger
> issues.
> 
> Because Intrusion Detection is a key feature of a firewall system, a
> lot of testing is required until the new implementation can become
> part
> of IPFire - therefore we need your help!
> 
> Download the test image (
> https://people.ipfire.org/~stevee/suricata/Images/), do a lot of hard
> testing and provide your feedback or suggestions on the develoment
> mailing list (https://lists.ipfire.org/mailman/listinfo/development).
> 
> If you find any bugs please file them in the IPFire Bugtracker (
> https://bugzilla.ipfire.org/).
> 
> Many thanks in advance,
> 
> -Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for Tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 737 bytes --]

Hello Dirk,

thanks for testing and providing your feedback.
> Hello Stefan,
> 
> I just downloaded your new image and installed it in a new VM.
> Unfortunately I have to report the same problems as Michael had -
> downloading of rules doesn’t work.
> 
> Any clue what is the problem ?

The problem was introduced, during develoment from Beta 3 to 4.

The fix can be found here:

https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=b749416ad71126d6a05eb92b1409f097cc127617

I'll release a new testing image as soon as possible. In the meantime,
simple grab the fixed "ids-functions.pl" from GIT and replace the
existing one in "/var/ipfire/".

Best regards,

-Stefan

> 
> Best regards
> Dirk

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 3203 bytes --]

Hello Stefan,

Thanks for baking a new image. I just installed this and gave it a little test.

I will start with some comments on the UI:

* The error messages are a bit shouty. “No ruleset available, PLEASE DOWNLOAD ONE!!!111!!!” Is not really how these usually go.

The is two ways how to solve this: a) rephrase the messages and b) avoid that the user is shown this anyways. For example: grey out the enable checkbox when no ruleset has been downloaded and show a small note that this is a requirement.

* I think the top box can be merged with the settings box. There is too many headlines and this is rather confusing. It is one box in all other add-ons.

* It is not possible to click the Download button for rules when Save hasn’t been clicked before. That doesn’t make much sense.

* Downloading rules does not work. The log file says that no ruleset was selected although it was. Doesn’t work with any.

* In the German translation the word “Regelset” and “Regelsatz” are being used. I think the latter is the correct version.

So, I could not get this to work. What should I do?

-Michael

> On 1 Jan 2019, at 13:32, Stefan Schantl <stefan.schantl(a)ipfire.org> wrote:
> 
> Hello list followers,
> 
> a very happy new year to all of you!
> 
> I had some free time during the christmas days and backed a new testing
> image for suricata:
> 
> https://people.ipfire.org/~stevee/suricata/Images/ipfire-2.21.x86_64-suricata-beta4-full-core127.iso
> 
> It contains a lot of bug fixes and improvements suggested on this list.
> 
> As usual please do a lot of testing and report back any issues and
> suggestions.
> 
> Thanks in advance,
> 
> -Stefan
> 
> 
>> Hello list followers,
>> 
>> some time ago development for the new implementation of the Intrusion
>> Detection functionality in IPFire has been started.
>> 
>> The main goal, in a nutshell, was to give IPFire a modern, feature-
>> rich 
>> and user-friendly Intrusion Detection Engine. During this progress,
>> the
>> detection framework has been replaced - now suricata is used instead
>> of
>> snort.
>> 
>> Suricata uses a very modern and multi-threaded detection engine with
>> support to perform actions on malicious traffic. So it provides the
>> functionality of detecting any kind of intrusion attempts and the
>> ability of guardian to block them under the same hood.
>> 
>> It was a lot of work, but finaly I'm happy to announce the first test
>> version. It is almost feature complete and without any kind of bigger
>> issues.
>> 
>> Because Intrusion Detection is a key feature of a firewall system, a
>> lot of testing is required until the new implementation can become
>> part
>> of IPFire - therefore we need your help!
>> 
>> Download the test image (
>> https://people.ipfire.org/~stevee/suricata/Images/), do a lot of hard
>> testing and provide your feedback or suggestions on the develoment
>> mailing list (https://lists.ipfire.org/mailman/listinfo/development).
>> 
>> If you find any bugs please file them in the IPFire Bugtracker (
>> https://bugzilla.ipfire.org/).
>> 
>> Many thanks in advance,
>> 
>> -Stefan

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 1900 bytes --]

Hello list followers,

a very happy new year to all of you!

I had some free time during the christmas days and backed a new testing
image for suricata:

https://people.ipfire.org/~stevee/suricata/Images/ipfire-2.21.x86_64-suricata-beta4-full-core127.iso

It contains a lot of bug fixes and improvements suggested on this list.

As usual please do a lot of testing and report back any issues and
suggestions.

Thanks in advance,

-Stefan


> Hello list followers,
> 
> some time ago development for the new implementation of the Intrusion
> Detection functionality in IPFire has been started.
> 
> The main goal, in a nutshell, was to give IPFire a modern, feature-
> rich 
> and user-friendly Intrusion Detection Engine. During this progress,
> the
> detection framework has been replaced - now suricata is used instead
> of
> snort.
> 
> Suricata uses a very modern and multi-threaded detection engine with
> support to perform actions on malicious traffic. So it provides the
> functionality of detecting any kind of intrusion attempts and the
> ability of guardian to block them under the same hood.
> 
> It was a lot of work, but finaly I'm happy to announce the first test
> version. It is almost feature complete and without any kind of bigger
> issues.
> 
> Because Intrusion Detection is a key feature of a firewall system, a
> lot of testing is required until the new implementation can become
> part
> of IPFire - therefore we need your help!
> 
> Download the test image (
> https://people.ipfire.org/~stevee/suricata/Images/), do a lot of hard
> testing and provide your feedback or suggestions on the develoment
> mailing list (https://lists.ipfire.org/mailman/listinfo/development).
> 
> If you find any bugs please file them in the IPFire Bugtracker (
> https://bugzilla.ipfire.org/).
> 
> Many thanks in advance,
> 
> -Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 16254 bytes --]

Hello,

Good job.

Stefan, are you planning to release another image very soon?

-Michael

> On 25 Dec 2018, at 20:17, Stefan Schantl <stefan.schantl(a)ipfire.org> wrote:
> 
> Hello Tim,
> 
> thanks for joining and providing your feedback on this list.
>> Hi,
>> 
>> I've loaded a system up with the test image.  I've not had much time
>> to
>> actually use it (hopefully after Christmas).  I would like to know
>> when
>> there'll be an image with the marking fixed.
> 
> I'll generate and upload a new one very soon.
> 
>> 
>> I agree with Michael over the GUI; splitting the system settings and
>> ruleset settings seems like a good idea.  The thing that caught me
>> originally was that I selected a ruleset and then clicked on
>> 'Download
>> new ruleset' and nothing happened because I hadn't clicked on 'Save'.
> 
> I splitted the section into two parts. One for configuring the IDS and
> one which contains the rules settings for now.
> 
>> 
>> A couple of points on automatic updates:
>> 
>> 1 - The updates shouldn't run exactly on the hour since this means
>> the
>> load on the servers will be very bursty.  It would be better to
>> spread
>> the load by running at some point past the hour and ensuring that
>> this
>> is different for every system (generate a random offset during
>> installation?).  Since the rulesets are sizeable (100 MiB for Talos
>> VRT)
>> and they're given away for free, I think it's a good idea to do
>> something about this.
> 
> Thanks for that, I don't know if "fcron" does any kind of
> randomization. I'll have a closer look on it.
> 
> I'm reading you are using the ruleset provided by the Talos team. I'm
> intrested in adding support for this. Do you have some more details
> about this ruleset and a download URL?
> 
>> 
>> 2 - There's a warning in the oinkmaster documentation:
>> 
>> Oinkmaster can be run manually or automatically as a cron job. But
>> of 
>> course, be very careful when doing the latter. Things could easily
>> get 
>> messed up from one update to another because of different layout in
>> the 
>> rules archive, typo in a rule, bugs in the script, and so on. At
>> least 
>> run snort -T on the new rules before restarting your Snort process.
>> 
>> Since this is about the rulefiles I'm sure it applies to Suricata as
>> well.  I have actually seen one occasion when the updated rules
>> failed
>> snort -T with my updater - and was after the download passed an
>> md5sum
>> check.  So the downloaded ruleset can be corrupt, if not very often. 
>> Should probably do something like:
>> 
>> download update
>> backup old rules
>> run oinkmaster
>> if (suricata -T is OK) then
>>  tell suricata to re-read rules
>> else
>>  restore backup
>> end if
> 
> Thanks for the hint. I have to dig into this, if suricata offers a
> mechanism to check the ruleset before doing a reload of it.
> 
>> 
>> Otherwise, I'm encouraged by the reduction in memory usage over
>> snort,
>> although it'll be interesting to see how this holds up with a larger
>> ruleset and under heavier load.
> 
> Best regards,
> 
> -Stefan
>> 
>> Regards,
>> Tim
>> 
>> On 20/12/2018 14:05, Michael Tremer wrote:
>>> Hi,
>>> 
>>>> On 20 Dec 2018, at 13:03, Stefan Schantl <
>>>> stefan.schantl(a)ipfire.org> wrote:
>>>> 
>>>> Hello Michael,
>>>> 
>>>> thanks for having a look at suricata and your feedback.
>>>>> Hi,
>>>>> 
>>>>> So I finally had a closer look at this, too. I have a couple of
>>>>> questions and would like to propose some changes to the UI:
>>>>> 
>>>>> Questions:
>>>>> 
>>>>> * Is it a good idea to have “No” rules as default and also “no”
>>>>> automatic rule updates? I guess we can configure at least
>>>>> Emerging
>>>>> Threats as a good default because they are free. I also think
>>>>> that
>>>>> when IDS is enabled, the system should update the ruleset by
>>>>> default
>>>>> once a day. When IDS is enabled, there should be no rule
>>>>> update.
>>>>> 
>>>>> * That also leads me to: Should no automatic updates even be an
>>>>> option?
>>>>> 
>>>> Making the free ruleset of emergingthreads as default is okay for
>>>> me -
>>>> ACK.
>>>> 
>>>> I'm also fine with doing a weekly ruleset update as default
>>>> setting.
>>>> 
>>>> My personal opinion is, it should be possible to disable the
>>>> automatic
>>>> rule update. There are some situations where this feature could
>>>> be
>>>> quite usefull, for example if you have only volume limited
>>>> internet
>>>> access or even other (which we currently do not have in
>>>> mind....).
>>> I guess if you don’t update the rules there is no point in running
>>> the IPS. It’s just like a virus scanner with outdated signatures:
>>> detecting even less.
>>> 
>>> Do we have more people with an opinion on this?
>>> 
>>> The rulesets are indeed quite large. But the only people who might
>>> want to disable it are people who run IPFire on an IoT appliance
>>> and those are too slow to run the IPS anyways. So it’s not a very
>>> strong argument in my opinion.
>>> 
>>> The best argument I can make for it is only that this is probably
>>> unwanted behaviour and therefore can easily be misconfugured.
>>> 
>>>>> Bugs:
>>>>> 
>>>>> * When I enabled IDS in IPS mode, I checked the box to activate
>>>>> it
>>>>> and selected the ET ruleset and hit save. Suricata starts but
>>>>> without
>>>>> rules which is definitely not what I wanted. Either the box
>>>>> should be
>>>>> greyed out or the downloader is launched when there are no
>>>>> rules,
>>>>> yet.
>>>>> 
>>>>> * It is possible to enable suricata without selecting any
>>>>> interfaces.
>>>>> That just wastes RAM and makes it easy to enable it without
>>>>> actually
>>>>> having the protection.
>>>>> 
>>>>> If you agree, I will create tickets for this.
>>>> Thanks, fixed.
>>>> 
>>>> https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=ebdd0f9a90da800cc6173f6f30fb0621dddc354b
>>> I am on a plane now, so I cannot see this, but thanks!
>>> 
>>>>> UI Stuff:
>>>>> 
>>>>> * I think that the settings to enable suricata up to and
>>>>> including
>>>>> “Traffic analysing” should be in the top box. The second box
>>>>> should
>>>>> only care about the rules. That might even solve the problem
>>>>> described above.
>>>> During development I had a very similar idea, of grouping all
>>>> suricata 
>>>> settings and all which are for the ruleset together into seperate
>>>> boxes. I dropped this idea, because of some big restrictions of
>>>> the
>>>> code and I was afraid of how many work this would be to re-write
>>>> huge
>>>> parts of it to get the desired result.
>>> I think that this is very little work.
>>> 
>>>>> * I find it quite confusing that one has to “Activate Intrusion
>>>>> Detection System” and then again select IDS or IPS. It doesn’t
>>>>> seem
>>>>> to make much sense even to select the top box when IPS is
>>>>> checked. I
>>>>> would therefore prefer the following: one box that just says
>>>>> “Activate” and that switches on or off the whole thing, and
>>>>> then a
>>>>> second box that says “monitor only” or something along these
>>>>> lines.
>>>>> That box should be disabled by default which will run suricata
>>>>> in IPS
>>>>> mode (which is what everybody wants!) and when checked will
>>>>> downgrade
>>>>> it to IDS mode. Right now it looks like this is an equal level
>>>>> of
>>>>> both options but I think that there is a risk here that people
>>>>> enable
>>>>> IDS and think that they are protected when they are in fact
>>>>> not. I
>>>>> think the “monitor only” checkbox makes this a lot clearer.
>>>> Fair enough - I'm going now to rework the current section for
>>>> choosing
>>>> the runmode and move from the radio buttons to a select box which
>>>> will
>>>> be called "Only monitor traffic" as you suggested and will place
>>>> it
>>>> right next to the box where you can enable the whole IPS/IDS
>>>> system.
>>> I think the best wording is: Monitor traffic only
>>> 
>>>>> * The Traffic Analyzing headline doesn’t make any sense. It
>>>>> should
>>>>> rather be “Monitored zones” or “Monitored interfaces”. Internet
>>>>> should be RED because it is called RED everywhere else.
>>>> ACK
>>>> 
>>>>> * We will need to grep for Snort everywhere and replace that
>>>>> phrase
>>>>> with “Intrusion Prevention System”. I do not think that we have
>>>>> to
>>>>> mention suricata here instead. It is confusing when the menu
>>>>> option
>>>>> is called IDS.
>>>> Most work on this already should be done, I'll remove the last
>>>> existing
>>>> snort entries if I'll found one. 
>>> Cool.
>>> 
>>>>> * The menu option should be called IPS instead of IDS I think
>>>>> since
>>>>> IPS mode is default. Spelled out obviously.
>>>> ACK - It should be a very easy task to switch the language string
>>>> from
>>>> "Intrusion Detection System" to the desired "Intrusion Prevention
>>>> System".
>>> Cool. I am just saying that it needs to be done :) don’t forget the
>>> logging section.
>>> 
>>>>> Apart from that I have to say that this is really really good
>>>>> work,
>>>>> Stefan. The main thing is: it works. I think this is a huge
>>>>> improvement over what we had in the past and that this is
>>>>> bringing
>>>>> IPFire to the next level. It solves many problems that we
>>>>> didn’t dare
>>>>> to touch because snort was simply badly integrated into IPFire
>>>>> and
>>>>> did not really do much instead of wasting CPU cycles and
>>>>> memory.
>>>>> 
>>>>> I hope that we can continue working in this direction and maybe
>>>>> add
>>>>> some nice reports and so on and obviously carry over this work
>>>>> into
>>>>> IPFire 3 as soon as possible.
>>>>> 
>>>>> Please feel free to disagree with me above, but please do that
>>>>> via
>>>>> email if possible.
>>>>> 
>>>>> I want to target merging this into next as soon as possible in
>>>>> January. I hope that we can get the things above sorted out
>>>>> over the
>>>>> holidays since it should not be much work.
>>>>> 
>>>> Best regards,
>>>> 
>>>> -Stefan
>>>>>> On 17 Dec 2018, at 19:08, Stefan Schantl <
>>>>>> stefan.schantl(a)ipfire.org
>>>>>>> wrote:
>>>>>>> What is the bypass mark for? At least that should be set to
>>>>>>> nothing
>>>>>>> instead of one, because that will again conflict with the
>>>>>>> SNAT
>>>>>>> fix
>>>>>>> rule.
>>>>>>> 
>>>>>>> Best,
>>>>>>> -Michael
>>>>>>> 
>>>>>> Hello Michael,
>>>>>> 
>>>>>> taken from the official suricata.yaml documentation:
>>>>>> 
>>>>>> bypass mark and mask can be used to implement NFQ bypass. If
>>>>>> bypass
>>>>>> mark is set then the NFQ bypass is activated. Suricata will
>>>>>> set the
>>>>>> bypass mark/mask on packet of a flow that need to be
>>>>>> bypassed. The
>>>>>> Nefilter ruleset has to directly accept all packets of a flow
>>>>>> once
>>>>>> a
>>>>>> packet has been marked.
>>>>> Weird. Why would you even pass this to the userspace process
>>>>> then?
>>>>> 
>>>>> Whatever, we don’t need it then...
>>>>> 
>>>>>> In my commit, I've disabled the bypass options to exactly
>>>>>> prevent
>>>>>> from
>>>>>> any issuse again with the SNAT packet mark.
>>>>>> 
>>>>>> Best regards,
>>>>>> 
>>>>>> -Stefan
>>>>> Best,
>>>>> -Michael
>>>>> 
>>>>>>>> On 17 Dec 2018, at 14:21, Stefan Schantl <
>>>>>>>> stefan.schantl(a)ipfire.org
>>>>>>>>> wrote:
>>>>>>>> Am Sonntag, den 16.12.2018, 21:28 +0100 schrieb Peter
>>>>>>>> Müller:
>>>>>>>>> Hello Stefan,
>>>>>>>>> 
>>>>>>>>> to be a bit more precise about the NAT issue:
>>>>>>>>> 
>>>>>>>>> My setup is the IPFire Suricata test VM running in KVM,
>>>>>>>>> with
>>>>>>>>> two clients (Debian and OpenBSD) directly attached to
>>>>>>>>> it.
>>>>>>>>> 
>>>>>>>>> The Debian machine is located in GREEN, OpenBSD in
>>>>>>>>> ORANGE.
>>>>>>>>> RED interface is connected via bridge to my actual
>>>>>>>>> testing
>>>>>>>>> LAN;
>>>>>>>>> for the first testing, any outgoing traffic to the
>>>>>>>>> internet
>>>>>>>>> was allowed (I will test upstream proxy behaviour
>>>>>>>>> later).
>>>>>>>>> 
>>>>>>>>> While GREEN was using IPv4 range 192.168.100.0/24, with
>>>>>>>>> IPFire
>>>>>>>>> as 192.168.100.1, enabling Suricata caused packets
>>>>>>>>> coming
>>>>>>>>> from
>>>>>>>>> GREEN not to be NATted anymore: Instead of using the
>>>>>>>>> firewall's
>>>>>>>>> RED IP for destination, it was the internal GREEN IP.
>>>>>>>>> 
>>>>>>>>> Let me know whether is is useful or not. :-)
>>>>>>>>> 
>>>>>>>>> Thanks, and best regards,
>>>>>>>>> Peter Müller
>>>>>>>>> 
>>>>>>>>>> Hello Stefan,
>>>>>>>>>> 
>>>>>>>>>> back again. :-)
>>>>>>>>>> 
>>>>>>>>>> The new IDS WebUI looks quite good so far -
>>>>>>>>>> enabling/disabling
>>>>>>>>>> Suricata works as well as selecting the rule source
>>>>>>>>>> and the
>>>>>>>>>> operation mode (IDS/IPS).
>>>>>>>>>> 
>>>>>>>>>> I was also able to download the "Snort/VRT Community"
>>>>>>>>>> ruleset.
>>>>>>>>>> Trying to switch to the "Emerging Threats" ruleset is
>>>>>>>>>> possible,
>>>>>>>>>> but downloading its ruleset afterwards is not: The
>>>>>>>>>> GUI
>>>>>>>>>> simply
>>>>>>>>>> stalls, printing a message that "Snort (!) is
>>>>>>>>>> performing a
>>>>>>>>>> task".
>>>>>>>>>> 
>>>>>>>>>> The WebUI services page still shows IDS status for
>>>>>>>>>> each
>>>>>>>>>> interface,
>>>>>>>>>> which does not seem to work anymore (everything is
>>>>>>>>>> stopped,
>>>>>>>>>> but
>>>>>>>>>> Suricata was active on RED and GREEN).
>>>>>>>>>> 
>>>>>>>>>> Further, a client located in GREEN behind the test
>>>>>>>>>> firewall
>>>>>>>>>> instance is unable to browse the internet as soon as
>>>>>>>>>> Suricata
>>>>>>>>>> is
>>>>>>>>>> enabled. If disabled, downloading ET rulesets work as
>>>>>>>>>> well
>>>>>>>>>> as
>>>>>>>>>> internet traffic. At the moment, I am flying blind
>>>>>>>>>> here,
>>>>>>>>>> but it
>>>>>>>>>> looks
>>>>>>>>>> like packets are not NATted anymore if Suricata is
>>>>>>>>>> active.
>>>>>>>>>> 
>>>>>>>>>> Any outgoing connection is in state "SYN_SENT" if
>>>>>>>>>> Suricata
>>>>>>>>>> is
>>>>>>>>>> active.
>>>>>>>>>> 
>>>>>>>>>> A portscan against the firewall (GREEN interface) is
>>>>>>>>>> not
>>>>>>>>>> detected,
>>>>>>>>>> even though ET SCAN ruleset is enabled (used nmap
>>>>>>>>>> with NSE
>>>>>>>>>> active).
>>>>>>>>>> 
>>>>>>>>>> Especially the outgoing connection/NAT/? issue
>>>>>>>>>> mentioned
>>>>>>>>>> above
>>>>>>>>>> breaks things in my scenario. Anything else are minor
>>>>>>>>>> issues
>>>>>>>>>> (of
>>>>>>>>>> course, a portscan should be detected, this needs
>>>>>>>>>> further
>>>>>>>>>> investigation
>>>>>>>>>> indeed). WebUI works fine so far.
>>>>>>>>>> 
>>>>>>>>>> Thanks again for your work; I hope the feedback can
>>>>>>>>>> appreciate
>>>>>>>>>> it
>>>>>>>>>> somehow. :-)
>>>>>>>>>> 
>>>>>>>>>> Let me know if there are questions.
>>>>>>>>>> 
>>>>>>>>>> Thanks, and best regards,
>>>>>>>>>> Peter Müller
>>>>>>>>>> 
>>>>>>>> Hello Peter,
>>>>>>>> 
>>>>>>>> thanks for testing and your feedback.
>>>>>>>> 
>>>>>>>> I've setup a test environment and was able to re-produce
>>>>>>>> your
>>>>>>>> NAT
>>>>>>>> issue
>>>>>>>> very easy. After some research and help of Michael, we
>>>>>>>> were
>>>>>>>> able to
>>>>>>>> find the real issue.
>>>>>>>> 
>>>>>>>> It was located in a identical mark on the packets which
>>>>>>>> already
>>>>>>>> have
>>>>>>>> been scanned by suricata and packets which should be
>>>>>>>> modified
>>>>>>>> for
>>>>>>>> SNAT
>>>>>>>> in the "Mangle" table of the routing logic by the kernel.
>>>>>>>> 
>>>>>>>> The changes can be found in my git repository:
>>>>>>>> 
>>>>>>>> https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=f5ad510e3c0f416a1507999f5ad20ab171df9c07
>>>>>>>> 
>>>>>>>> I'll upload a fixed image very soon - please keep on
>>>>>>>> testing.
>>>>>>>> 
>>>>>>>> Best regards,
>>>>>>>> 
>>>>>>>> -Stefan

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 17285 bytes --]

Hello Tim,

thanks for joining and providing your feedback on this list.
> Hi,
> 
> I've loaded a system up with the test image.  I've not had much time
> to
> actually use it (hopefully after Christmas).  I would like to know
> when
> there'll be an image with the marking fixed.

I'll generate and upload a new one very soon.

> 
> I agree with Michael over the GUI; splitting the system settings and
> ruleset settings seems like a good idea.  The thing that caught me
> originally was that I selected a ruleset and then clicked on
> 'Download
> new ruleset' and nothing happened because I hadn't clicked on 'Save'.

I splitted the section into two parts. One for configuring the IDS and
one which contains the rules settings for now.

> 
> A couple of points on automatic updates:
> 
> 1 - The updates shouldn't run exactly on the hour since this means
> the
> load on the servers will be very bursty.  It would be better to
> spread
> the load by running at some point past the hour and ensuring that
> this
> is different for every system (generate a random offset during
> installation?).  Since the rulesets are sizeable (100 MiB for Talos
> VRT)
> and they're given away for free, I think it's a good idea to do
> something about this.

Thanks for that, I don't know if "fcron" does any kind of
randomization. I'll have a closer look on it.

I'm reading you are using the ruleset provided by the Talos team. I'm
intrested in adding support for this. Do you have some more details
about this ruleset and a download URL?

> 
> 2 - There's a warning in the oinkmaster documentation:
> 
> Oinkmaster can be run manually or automatically as a cron job. But
> of 
> course, be very careful when doing the latter. Things could easily
> get 
> messed up from one update to another because of different layout in
> the 
> rules archive, typo in a rule, bugs in the script, and so on. At
> least 
> run snort -T on the new rules before restarting your Snort process.
> 
> Since this is about the rulefiles I'm sure it applies to Suricata as
> well.  I have actually seen one occasion when the updated rules
> failed
> snort -T with my updater - and was after the download passed an
> md5sum
> check.  So the downloaded ruleset can be corrupt, if not very often. 
> Should probably do something like:
> 
> download update
> backup old rules
> run oinkmaster
> if (suricata -T is OK) then
>   tell suricata to re-read rules
> else
>   restore backup
> end if

Thanks for the hint. I have to dig into this, if suricata offers a
mechanism to check the ruleset before doing a reload of it.

> 
> Otherwise, I'm encouraged by the reduction in memory usage over
> snort,
> although it'll be interesting to see how this holds up with a larger
> ruleset and under heavier load.

Best regards,

-Stefan
> 
> Regards,
> Tim
> 
> On 20/12/2018 14:05, Michael Tremer wrote:
> > Hi,
> > 
> > > On 20 Dec 2018, at 13:03, Stefan Schantl <
> > > stefan.schantl(a)ipfire.org> wrote:
> > > 
> > > Hello Michael,
> > > 
> > > thanks for having a look at suricata and your feedback.
> > > > Hi,
> > > > 
> > > > So I finally had a closer look at this, too. I have a couple of
> > > > questions and would like to propose some changes to the UI:
> > > > 
> > > > Questions:
> > > > 
> > > > * Is it a good idea to have “No” rules as default and also “no”
> > > > automatic rule updates? I guess we can configure at least
> > > > Emerging
> > > > Threats as a good default because they are free. I also think
> > > > that
> > > > when IDS is enabled, the system should update the ruleset by
> > > > default
> > > > once a day. When IDS is enabled, there should be no rule
> > > > update.
> > > > 
> > > > * That also leads me to: Should no automatic updates even be an
> > > > option?
> > > > 
> > > Making the free ruleset of emergingthreads as default is okay for
> > > me -
> > > ACK.
> > > 
> > > I'm also fine with doing a weekly ruleset update as default
> > > setting.
> > > 
> > > My personal opinion is, it should be possible to disable the
> > > automatic
> > > rule update. There are some situations where this feature could
> > > be
> > > quite usefull, for example if you have only volume limited
> > > internet
> > > access or even other (which we currently do not have in
> > > mind....).
> > I guess if you don’t update the rules there is no point in running
> > the IPS. It’s just like a virus scanner with outdated signatures:
> > detecting even less.
> > 
> > Do we have more people with an opinion on this?
> > 
> > The rulesets are indeed quite large. But the only people who might
> > want to disable it are people who run IPFire on an IoT appliance
> > and those are too slow to run the IPS anyways. So it’s not a very
> > strong argument in my opinion.
> > 
> > The best argument I can make for it is only that this is probably
> > unwanted behaviour and therefore can easily be misconfugured.
> > 
> > > > Bugs:
> > > > 
> > > > * When I enabled IDS in IPS mode, I checked the box to activate
> > > > it
> > > > and selected the ET ruleset and hit save. Suricata starts but
> > > > without
> > > > rules which is definitely not what I wanted. Either the box
> > > > should be
> > > > greyed out or the downloader is launched when there are no
> > > > rules,
> > > > yet.
> > > > 
> > > > * It is possible to enable suricata without selecting any
> > > > interfaces.
> > > > That just wastes RAM and makes it easy to enable it without
> > > > actually
> > > > having the protection.
> > > > 
> > > > If you agree, I will create tickets for this.
> > > Thanks, fixed.
> > > 
> > > https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=ebdd0f9a90da800cc6173f6f30fb0621dddc354b
> > I am on a plane now, so I cannot see this, but thanks!
> > 
> > > > UI Stuff:
> > > > 
> > > > * I think that the settings to enable suricata up to and
> > > > including
> > > > “Traffic analysing” should be in the top box. The second box
> > > > should
> > > > only care about the rules. That might even solve the problem
> > > > described above.
> > > During development I had a very similar idea, of grouping all
> > > suricata 
> > > settings and all which are for the ruleset together into seperate
> > > boxes. I dropped this idea, because of some big restrictions of
> > > the
> > > code and I was afraid of how many work this would be to re-write
> > > huge
> > > parts of it to get the desired result.
> > I think that this is very little work.
> > 
> > > > * I find it quite confusing that one has to “Activate Intrusion
> > > > Detection System” and then again select IDS or IPS. It doesn’t
> > > > seem
> > > > to make much sense even to select the top box when IPS is
> > > > checked. I
> > > > would therefore prefer the following: one box that just says
> > > > “Activate” and that switches on or off the whole thing, and
> > > > then a
> > > > second box that says “monitor only” or something along these
> > > > lines.
> > > > That box should be disabled by default which will run suricata
> > > > in IPS
> > > > mode (which is what everybody wants!) and when checked will
> > > > downgrade
> > > > it to IDS mode. Right now it looks like this is an equal level
> > > > of
> > > > both options but I think that there is a risk here that people
> > > > enable
> > > > IDS and think that they are protected when they are in fact
> > > > not. I
> > > > think the “monitor only” checkbox makes this a lot clearer.
> > > Fair enough - I'm going now to rework the current section for
> > > choosing
> > > the runmode and move from the radio buttons to a select box which
> > > will
> > > be called "Only monitor traffic" as you suggested and will place
> > > it
> > > right next to the box where you can enable the whole IPS/IDS
> > > system.
> > I think the best wording is: Monitor traffic only
> > 
> > > > * The Traffic Analyzing headline doesn’t make any sense. It
> > > > should
> > > > rather be “Monitored zones” or “Monitored interfaces”. Internet
> > > > should be RED because it is called RED everywhere else.
> > > ACK
> > > 
> > > > * We will need to grep for Snort everywhere and replace that
> > > > phrase
> > > > with “Intrusion Prevention System”. I do not think that we have
> > > > to
> > > > mention suricata here instead. It is confusing when the menu
> > > > option
> > > > is called IDS.
> > > Most work on this already should be done, I'll remove the last
> > > existing
> > > snort entries if I'll found one. 
> > Cool.
> > 
> > > > * The menu option should be called IPS instead of IDS I think
> > > > since
> > > > IPS mode is default. Spelled out obviously.
> > > ACK - It should be a very easy task to switch the language string
> > > from
> > > "Intrusion Detection System" to the desired "Intrusion Prevention
> > > System".
> > Cool. I am just saying that it needs to be done :) don’t forget the
> > logging section.
> > 
> > > > Apart from that I have to say that this is really really good
> > > > work,
> > > > Stefan. The main thing is: it works. I think this is a huge
> > > > improvement over what we had in the past and that this is
> > > > bringing
> > > > IPFire to the next level. It solves many problems that we
> > > > didn’t dare
> > > > to touch because snort was simply badly integrated into IPFire
> > > > and
> > > > did not really do much instead of wasting CPU cycles and
> > > > memory.
> > > > 
> > > > I hope that we can continue working in this direction and maybe
> > > > add
> > > > some nice reports and so on and obviously carry over this work
> > > > into
> > > > IPFire 3 as soon as possible.
> > > > 
> > > > Please feel free to disagree with me above, but please do that
> > > > via
> > > > email if possible.
> > > > 
> > > > I want to target merging this into next as soon as possible in
> > > > January. I hope that we can get the things above sorted out
> > > > over the
> > > > holidays since it should not be much work.
> > > > 
> > > Best regards,
> > > 
> > > -Stefan
> > > > > On 17 Dec 2018, at 19:08, Stefan Schantl <
> > > > > stefan.schantl(a)ipfire.org
> > > > > > wrote:
> > > > > > What is the bypass mark for? At least that should be set to
> > > > > > nothing
> > > > > > instead of one, because that will again conflict with the
> > > > > > SNAT
> > > > > > fix
> > > > > > rule.
> > > > > > 
> > > > > > Best,
> > > > > > -Michael
> > > > > > 
> > > > > Hello Michael,
> > > > > 
> > > > > taken from the official suricata.yaml documentation:
> > > > > 
> > > > > bypass mark and mask can be used to implement NFQ bypass. If
> > > > > bypass
> > > > > mark is set then the NFQ bypass is activated. Suricata will
> > > > > set the
> > > > > bypass mark/mask on packet of a flow that need to be
> > > > > bypassed. The
> > > > > Nefilter ruleset has to directly accept all packets of a flow
> > > > > once
> > > > > a
> > > > > packet has been marked.
> > > > Weird. Why would you even pass this to the userspace process
> > > > then?
> > > > 
> > > > Whatever, we don’t need it then...
> > > > 
> > > > > In my commit, I've disabled the bypass options to exactly
> > > > > prevent
> > > > > from
> > > > > any issuse again with the SNAT packet mark.
> > > > > 
> > > > > Best regards,
> > > > > 
> > > > > -Stefan
> > > > Best,
> > > > -Michael
> > > > 
> > > > > > > On 17 Dec 2018, at 14:21, Stefan Schantl <
> > > > > > > stefan.schantl(a)ipfire.org
> > > > > > > > wrote:
> > > > > > > Am Sonntag, den 16.12.2018, 21:28 +0100 schrieb Peter
> > > > > > > Müller:
> > > > > > > > Hello Stefan,
> > > > > > > > 
> > > > > > > > to be a bit more precise about the NAT issue:
> > > > > > > > 
> > > > > > > > My setup is the IPFire Suricata test VM running in KVM,
> > > > > > > > with
> > > > > > > > two clients (Debian and OpenBSD) directly attached to
> > > > > > > > it.
> > > > > > > > 
> > > > > > > > The Debian machine is located in GREEN, OpenBSD in
> > > > > > > > ORANGE.
> > > > > > > > RED interface is connected via bridge to my actual
> > > > > > > > testing
> > > > > > > > LAN;
> > > > > > > > for the first testing, any outgoing traffic to the
> > > > > > > > internet
> > > > > > > > was allowed (I will test upstream proxy behaviour
> > > > > > > > later).
> > > > > > > > 
> > > > > > > > While GREEN was using IPv4 range 192.168.100.0/24, with
> > > > > > > > IPFire
> > > > > > > > as 192.168.100.1, enabling Suricata caused packets
> > > > > > > > coming
> > > > > > > > from
> > > > > > > > GREEN not to be NATted anymore: Instead of using the
> > > > > > > > firewall's
> > > > > > > > RED IP for destination, it was the internal GREEN IP.
> > > > > > > > 
> > > > > > > > Let me know whether is is useful or not. :-)
> > > > > > > > 
> > > > > > > > Thanks, and best regards,
> > > > > > > > Peter Müller
> > > > > > > > 
> > > > > > > > > Hello Stefan,
> > > > > > > > > 
> > > > > > > > > back again. :-)
> > > > > > > > > 
> > > > > > > > > The new IDS WebUI looks quite good so far -
> > > > > > > > > enabling/disabling
> > > > > > > > > Suricata works as well as selecting the rule source
> > > > > > > > > and the
> > > > > > > > > operation mode (IDS/IPS).
> > > > > > > > > 
> > > > > > > > > I was also able to download the "Snort/VRT Community"
> > > > > > > > > ruleset.
> > > > > > > > > Trying to switch to the "Emerging Threats" ruleset is
> > > > > > > > > possible,
> > > > > > > > > but downloading its ruleset afterwards is not: The
> > > > > > > > > GUI
> > > > > > > > > simply
> > > > > > > > > stalls, printing a message that "Snort (!) is
> > > > > > > > > performing a
> > > > > > > > > task".
> > > > > > > > > 
> > > > > > > > > The WebUI services page still shows IDS status for
> > > > > > > > > each
> > > > > > > > > interface,
> > > > > > > > > which does not seem to work anymore (everything is
> > > > > > > > > stopped,
> > > > > > > > > but
> > > > > > > > > Suricata was active on RED and GREEN).
> > > > > > > > > 
> > > > > > > > > Further, a client located in GREEN behind the test
> > > > > > > > > firewall
> > > > > > > > > instance is unable to browse the internet as soon as
> > > > > > > > > Suricata
> > > > > > > > > is
> > > > > > > > > enabled. If disabled, downloading ET rulesets work as
> > > > > > > > > well
> > > > > > > > > as
> > > > > > > > > internet traffic. At the moment, I am flying blind
> > > > > > > > > here,
> > > > > > > > > but it
> > > > > > > > > looks
> > > > > > > > > like packets are not NATted anymore if Suricata is
> > > > > > > > > active.
> > > > > > > > > 
> > > > > > > > > Any outgoing connection is in state "SYN_SENT" if
> > > > > > > > > Suricata
> > > > > > > > > is
> > > > > > > > > active.
> > > > > > > > > 
> > > > > > > > > A portscan against the firewall (GREEN interface) is
> > > > > > > > > not
> > > > > > > > > detected,
> > > > > > > > > even though ET SCAN ruleset is enabled (used nmap
> > > > > > > > > with NSE
> > > > > > > > > active).
> > > > > > > > > 
> > > > > > > > > Especially the outgoing connection/NAT/? issue
> > > > > > > > > mentioned
> > > > > > > > > above
> > > > > > > > > breaks things in my scenario. Anything else are minor
> > > > > > > > > issues
> > > > > > > > > (of
> > > > > > > > > course, a portscan should be detected, this needs
> > > > > > > > > further
> > > > > > > > > investigation
> > > > > > > > > indeed). WebUI works fine so far.
> > > > > > > > > 
> > > > > > > > > Thanks again for your work; I hope the feedback can
> > > > > > > > > appreciate
> > > > > > > > > it
> > > > > > > > > somehow. :-)
> > > > > > > > > 
> > > > > > > > > Let me know if there are questions.
> > > > > > > > > 
> > > > > > > > > Thanks, and best regards,
> > > > > > > > > Peter Müller
> > > > > > > > > 
> > > > > > > Hello Peter,
> > > > > > > 
> > > > > > > thanks for testing and your feedback.
> > > > > > > 
> > > > > > > I've setup a test environment and was able to re-produce
> > > > > > > your
> > > > > > > NAT
> > > > > > > issue
> > > > > > > very easy. After some research and help of Michael, we
> > > > > > > were
> > > > > > > able to
> > > > > > > find the real issue.
> > > > > > > 
> > > > > > > It was located in a identical mark on the packets which
> > > > > > > already
> > > > > > > have
> > > > > > > been scanned by suricata and packets which should be
> > > > > > > modified
> > > > > > > for
> > > > > > > SNAT
> > > > > > > in the "Mangle" table of the routing logic by the kernel.
> > > > > > > 
> > > > > > > The changes can be found in my git repository:
> > > > > > > 
> > > > > > > https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=f5ad510e3c0f416a1507999f5ad20ab171df9c07
> > > > > > > 
> > > > > > > I'll upload a fixed image very soon - please keep on
> > > > > > > testing.
> > > > > > > 
> > > > > > > Best regards,
> > > > > > > 
> > > > > > > -Stefan
> 
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 14539 bytes --]

Hello Michael,
> Hi,
> 
> > On 20 Dec 2018, at 13:03, Stefan Schantl <stefan.schantl(a)ipfire.org
> > > wrote:
> > 
> > Hello Michael,
> > 
> > thanks for having a look at suricata and your feedback.
> > > Hi,
> > > 
> > > So I finally had a closer look at this, too. I have a couple of
> > > questions and would like to propose some changes to the UI:
> > > 
> > > Questions:
> > > 
> > > * Is it a good idea to have “No” rules as default and also “no”
> > > automatic rule updates? I guess we can configure at least
> > > Emerging
> > > Threats as a good default because they are free. I also think
> > > that
> > > when IDS is enabled, the system should update the ruleset by
> > > default
> > > once a day. When IDS is enabled, there should be no rule update.
> > > 
> > > * That also leads me to: Should no automatic updates even be an
> > > option?
> > > 
> > Making the free ruleset of emergingthreads as default is okay for
> > me -
> > ACK.

DONE - now the default selection, if no ruleset source has been choosen
yet is the free emergingthreads ruleset.

> > 
> > I'm also fine with doing a weekly ruleset update as default
> > setting.
> > 
> > My personal opinion is, it should be possible to disable the
> > automatic
> > rule update. There are some situations where this feature could be
> > quite usefull, for example if you have only volume limited internet
> > access or even other (which we currently do not have in mind....).
> 
> I guess if you don’t update the rules there is no point in running
> the IPS. It’s just like a virus scanner with outdated signatures:
> detecting even less.
> 
> Do we have more people with an opinion on this?
> 
> The rulesets are indeed quite large. But the only people who might
> want to disable it are people who run IPFire on an IoT appliance and
> those are too slow to run the IPS anyways. So it’s not a very strong
> argument in my opinion.
> 
> The best argument I can make for it is only that this is probably
> unwanted behaviour and therefore can easily be misconfugured.
> 

Default now is "weekly" until anything else has been selected.

I keept the feature for disabling the automatic update for now until we
have more opinions on this topic.

> > 
> > > Bugs:
> > > 
> > > * When I enabled IDS in IPS mode, I checked the box to activate
> > > it
> > > and selected the ET ruleset and hit save. Suricata starts but
> > > without
> > > rules which is definitely not what I wanted. Either the box
> > > should be
> > > greyed out or the downloader is launched when there are no rules,
> > > yet.
> > > 
> > > * It is possible to enable suricata without selecting any
> > > interfaces.
> > > That just wastes RAM and makes it easy to enable it without
> > > actually
> > > having the protection.
> > > 
> > > If you agree, I will create tickets for this.
> > 
> > Thanks, fixed.
> > 
> > https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=ebdd0f9a90da800cc6173f6f30fb0621dddc354b
> 
> I am on a plane now, so I cannot see this, but thanks!
> 
> > > UI Stuff:
> > > 
> > > * I think that the settings to enable suricata up to and
> > > including
> > > “Traffic analysing” should be in the top box. The second box
> > > should
> > > only care about the rules. That might even solve the problem
> > > described above.
> > 
> > During development I had a very similar idea, of grouping all
> > suricata 
> > settings and all which are for the ruleset together into seperate
> > boxes. I dropped this idea, because of some big restrictions of the
> > code and I was afraid of how many work this would be to re-write
> > huge
> > parts of it to get the desired result.
> 
> I think that this is very little work.

DONE - Technically it was just to splitt the HTML <form> from a single
one into two seperate ones. But under the hood more code had to be
changed.

> 
> > > * I find it quite confusing that one has to “Activate Intrusion
> > > Detection System” and then again select IDS or IPS. It doesn’t
> > > seem
> > > to make much sense even to select the top box when IPS is
> > > checked. I
> > > would therefore prefer the following: one box that just says
> > > “Activate” and that switches on or off the whole thing, and then
> > > a
> > > second box that says “monitor only” or something along these
> > > lines.
> > > That box should be disabled by default which will run suricata in
> > > IPS
> > > mode (which is what everybody wants!) and when checked will
> > > downgrade
> > > it to IDS mode. Right now it looks like this is an equal level of
> > > both options but I think that there is a risk here that people
> > > enable
> > > IDS and think that they are protected when they are in fact not.
> > > I
> > > think the “monitor only” checkbox makes this a lot clearer.
> > 
> > Fair enough - I'm going now to rework the current section for
> > choosing
> > the runmode and move from the radio buttons to a select box which
> > will
> > be called "Only monitor traffic" as you suggested and will place it
> > right next to the box where you can enable the whole IPS/IDS
> > system.
> 
> I think the best wording is: Monitor traffic only

DONE - As promised, I've dropped the radio button selection for the
runmode and placed a checkbox for this.

> 
> > > * The Traffic Analyzing headline doesn’t make any sense. It
> > > should
> > > rather be “Monitored zones” or “Monitored interfaces”. Internet
> > > should be RED because it is called RED everywhere else.
> > 
> > ACK

DONE

> > 
> > > * We will need to grep for Snort everywhere and replace that
> > > phrase
> > > with “Intrusion Prevention System”. I do not think that we have
> > > to
> > > mention suricata here instead. It is confusing when the menu
> > > option
> > > is called IDS.
> > 
> > Most work on this already should be done, I'll remove the last
> > existing
> > snort entries if I'll found one. 
> 
> Cool.

This still needs to be done.

> 
> > > * The menu option should be called IPS instead of IDS I think
> > > since
> > > IPS mode is default. Spelled out obviously.
> > 
> > ACK - It should be a very easy task to switch the language string
> > from
> > "Intrusion Detection System" to the desired "Intrusion Prevention
> > System".

This also still needs to be done.

I've spent a lot of time in the past days to rework some code and
moving stuff to the "ids-functions.pl". The final goal was to get a
nice code-base for the "snort to suricata" converter on which I'm
currently working.

I hope I will finish this very soon, so I can go on fixing the last
minor issues and filed bugs.

Best regards,

-Stefan 

> 
> Cool. I am just saying that it needs to be done :) don’t forget the
> logging section.
> 
> > > Apart from that I have to say that this is really really good
> > > work,
> > > Stefan. The main thing is: it works. I think this is a huge
> > > improvement over what we had in the past and that this is
> > > bringing
> > > IPFire to the next level. It solves many problems that we didn’t
> > > dare
> > > to touch because snort was simply badly integrated into IPFire
> > > and
> > > did not really do much instead of wasting CPU cycles and memory.
> > > 
> > > I hope that we can continue working in this direction and maybe
> > > add
> > > some nice reports and so on and obviously carry over this work
> > > into
> > > IPFire 3 as soon as possible.
> > > 
> > > Please feel free to disagree with me above, but please do that
> > > via
> > > email if possible.
> > > 
> > > I want to target merging this into next as soon as possible in
> > > January. I hope that we can get the things above sorted out over
> > > the
> > > holidays since it should not be much work.
> > > 
> > Best regards,
> > 
> > -Stefan
> > > > On 17 Dec 2018, at 19:08, Stefan Schantl <
> > > > stefan.schantl(a)ipfire.org
> > > > > wrote:
> > > > > What is the bypass mark for? At least that should be set to
> > > > > nothing
> > > > > instead of one, because that will again conflict with the
> > > > > SNAT
> > > > > fix
> > > > > rule.
> > > > > 
> > > > > Best,
> > > > > -Michael
> > > > > 
> > > > Hello Michael,
> > > > 
> > > > taken from the official suricata.yaml documentation:
> > > > 
> > > > bypass mark and mask can be used to implement NFQ bypass. If
> > > > bypass
> > > > mark is set then the NFQ bypass is activated. Suricata will set
> > > > the
> > > > bypass mark/mask on packet of a flow that need to be bypassed.
> > > > The
> > > > Nefilter ruleset has to directly accept all packets of a flow
> > > > once
> > > > a
> > > > packet has been marked.
> > > 
> > > Weird. Why would you even pass this to the userspace process
> > > then?
> > > 
> > > Whatever, we don’t need it then...
> > > 
> > > > In my commit, I've disabled the bypass options to exactly
> > > > prevent
> > > > from
> > > > any issuse again with the SNAT packet mark.
> > > > 
> > > > Best regards,
> > > > 
> > > > -Stefan
> > > 
> > > Best,
> > > -Michael
> > > 
> > > > > > On 17 Dec 2018, at 14:21, Stefan Schantl <
> > > > > > stefan.schantl(a)ipfire.org
> > > > > > > wrote:
> > > > > > 
> > > > > > Am Sonntag, den 16.12.2018, 21:28 +0100 schrieb Peter
> > > > > > Müller:
> > > > > > > Hello Stefan,
> > > > > > > 
> > > > > > > to be a bit more precise about the NAT issue:
> > > > > > > 
> > > > > > > My setup is the IPFire Suricata test VM running in KVM,
> > > > > > > with
> > > > > > > two clients (Debian and OpenBSD) directly attached to it.
> > > > > > > 
> > > > > > > The Debian machine is located in GREEN, OpenBSD in
> > > > > > > ORANGE.
> > > > > > > RED interface is connected via bridge to my actual
> > > > > > > testing
> > > > > > > LAN;
> > > > > > > for the first testing, any outgoing traffic to the
> > > > > > > internet
> > > > > > > was allowed (I will test upstream proxy behaviour later).
> > > > > > > 
> > > > > > > While GREEN was using IPv4 range 192.168.100.0/24, with
> > > > > > > IPFire
> > > > > > > as 192.168.100.1, enabling Suricata caused packets coming
> > > > > > > from
> > > > > > > GREEN not to be NATted anymore: Instead of using the
> > > > > > > firewall's
> > > > > > > RED IP for destination, it was the internal GREEN IP.
> > > > > > > 
> > > > > > > Let me know whether is is useful or not. :-)
> > > > > > > 
> > > > > > > Thanks, and best regards,
> > > > > > > Peter Müller
> > > > > > > 
> > > > > > > > Hello Stefan,
> > > > > > > > 
> > > > > > > > back again. :-)
> > > > > > > > 
> > > > > > > > The new IDS WebUI looks quite good so far -
> > > > > > > > enabling/disabling
> > > > > > > > Suricata works as well as selecting the rule source and
> > > > > > > > the
> > > > > > > > operation mode (IDS/IPS).
> > > > > > > > 
> > > > > > > > I was also able to download the "Snort/VRT Community"
> > > > > > > > ruleset.
> > > > > > > > Trying to switch to the "Emerging Threats" ruleset is
> > > > > > > > possible,
> > > > > > > > but downloading its ruleset afterwards is not: The GUI
> > > > > > > > simply
> > > > > > > > stalls, printing a message that "Snort (!) is
> > > > > > > > performing a
> > > > > > > > task".
> > > > > > > > 
> > > > > > > > The WebUI services page still shows IDS status for each
> > > > > > > > interface,
> > > > > > > > which does not seem to work anymore (everything is
> > > > > > > > stopped,
> > > > > > > > but
> > > > > > > > Suricata was active on RED and GREEN).
> > > > > > > > 
> > > > > > > > Further, a client located in GREEN behind the test
> > > > > > > > firewall
> > > > > > > > instance is unable to browse the internet as soon as
> > > > > > > > Suricata
> > > > > > > > is
> > > > > > > > enabled. If disabled, downloading ET rulesets work as
> > > > > > > > well
> > > > > > > > as
> > > > > > > > internet traffic. At the moment, I am flying blind
> > > > > > > > here,
> > > > > > > > but it
> > > > > > > > looks
> > > > > > > > like packets are not NATted anymore if Suricata is
> > > > > > > > active.
> > > > > > > > 
> > > > > > > > Any outgoing connection is in state "SYN_SENT" if
> > > > > > > > Suricata
> > > > > > > > is
> > > > > > > > active.
> > > > > > > > 
> > > > > > > > A portscan against the firewall (GREEN interface) is
> > > > > > > > not
> > > > > > > > detected,
> > > > > > > > even though ET SCAN ruleset is enabled (used nmap with
> > > > > > > > NSE
> > > > > > > > active).
> > > > > > > > 
> > > > > > > > Especially the outgoing connection/NAT/? issue
> > > > > > > > mentioned
> > > > > > > > above
> > > > > > > > breaks things in my scenario. Anything else are minor
> > > > > > > > issues
> > > > > > > > (of
> > > > > > > > course, a portscan should be detected, this needs
> > > > > > > > further
> > > > > > > > investigation
> > > > > > > > indeed). WebUI works fine so far.
> > > > > > > > 
> > > > > > > > Thanks again for your work; I hope the feedback can
> > > > > > > > appreciate
> > > > > > > > it
> > > > > > > > somehow. :-)
> > > > > > > > 
> > > > > > > > Let me know if there are questions.
> > > > > > > > 
> > > > > > > > Thanks, and best regards,
> > > > > > > > Peter Müller
> > > > > > > > 
> > > > > > Hello Peter,
> > > > > > 
> > > > > > thanks for testing and your feedback.
> > > > > > 
> > > > > > I've setup a test environment and was able to re-produce
> > > > > > your
> > > > > > NAT
> > > > > > issue
> > > > > > very easy. After some research and help of Michael, we were
> > > > > > able to
> > > > > > find the real issue.
> > > > > > 
> > > > > > It was located in a identical mark on the packets which
> > > > > > already
> > > > > > have
> > > > > > been scanned by suricata and packets which should be
> > > > > > modified
> > > > > > for
> > > > > > SNAT
> > > > > > in the "Mangle" table of the routing logic by the kernel.
> > > > > > 
> > > > > > The changes can be found in my git repository:
> > > > > > 
> > > > > > https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=f5ad510e3c0f416a1507999f5ad20ab171df9c07
> > > > > > 
> > > > > > I'll upload a fixed image very soon - please keep on
> > > > > > testing.
> > > > > > 
> > > > > > Best regards,
> > > > > > 
> > > > > > -Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Tim FitzGeorge]

[-- Attachment #1: Type: text/plain, Size: 13740 bytes --]

Hi,

I've loaded a system up with the test image.  I've not had much time to
actually use it (hopefully after Christmas).  I would like to know when
there'll be an image with the marking fixed.

I agree with Michael over the GUI; splitting the system settings and
ruleset settings seems like a good idea.  The thing that caught me
originally was that I selected a ruleset and then clicked on 'Download
new ruleset' and nothing happened because I hadn't clicked on 'Save'.

A couple of points on automatic updates:

1 - The updates shouldn't run exactly on the hour since this means the
load on the servers will be very bursty.  It would be better to spread
the load by running at some point past the hour and ensuring that this
is different for every system (generate a random offset during
installation?).  Since the rulesets are sizeable (100 MiB for Talos VRT)
and they're given away for free, I think it's a good idea to do
something about this.

2 - There's a warning in the oinkmaster documentation:

Oinkmaster can be run manually or automatically as a cron job. But of 
course, be very careful when doing the latter. Things could easily get 
messed up from one update to another because of different layout in the 
rules archive, typo in a rule, bugs in the script, and so on. At least 
run snort -T on the new rules before restarting your Snort process.

Since this is about the rulefiles I'm sure it applies to Suricata as
well.  I have actually seen one occasion when the updated rules failed
snort -T with my updater - and was after the download passed an md5sum
check.  So the downloaded ruleset can be corrupt, if not very often. 
Should probably do something like:

download update
backup old rules
run oinkmaster
if (suricata -T is OK) then
  tell suricata to re-read rules
else
  restore backup
end if

Otherwise, I'm encouraged by the reduction in memory usage over snort,
although it'll be interesting to see how this holds up with a larger
ruleset and under heavier load.

Regards,
Tim

On 20/12/2018 14:05, Michael Tremer wrote:
> Hi,
>
>> On 20 Dec 2018, at 13:03, Stefan Schantl <stefan.schantl(a)ipfire.org> wrote:
>>
>> Hello Michael,
>>
>> thanks for having a look at suricata and your feedback.
>>> Hi,
>>>
>>> So I finally had a closer look at this, too. I have a couple of
>>> questions and would like to propose some changes to the UI:
>>>
>>> Questions:
>>>
>>> * Is it a good idea to have “No” rules as default and also “no”
>>> automatic rule updates? I guess we can configure at least Emerging
>>> Threats as a good default because they are free. I also think that
>>> when IDS is enabled, the system should update the ruleset by default
>>> once a day. When IDS is enabled, there should be no rule update.
>>>
>>> * That also leads me to: Should no automatic updates even be an
>>> option?
>>>
>> Making the free ruleset of emergingthreads as default is okay for me -
>> ACK.
>>
>> I'm also fine with doing a weekly ruleset update as default setting.
>>
>> My personal opinion is, it should be possible to disable the automatic
>> rule update. There are some situations where this feature could be
>> quite usefull, for example if you have only volume limited internet
>> access or even other (which we currently do not have in mind....).
> I guess if you don’t update the rules there is no point in running the IPS. It’s just like a virus scanner with outdated signatures: detecting even less.
>
> Do we have more people with an opinion on this?
>
> The rulesets are indeed quite large. But the only people who might want to disable it are people who run IPFire on an IoT appliance and those are too slow to run the IPS anyways. So it’s not a very strong argument in my opinion.
>
> The best argument I can make for it is only that this is probably unwanted behaviour and therefore can easily be misconfugured.
>
>>
>>> Bugs:
>>>
>>> * When I enabled IDS in IPS mode, I checked the box to activate it
>>> and selected the ET ruleset and hit save. Suricata starts but without
>>> rules which is definitely not what I wanted. Either the box should be
>>> greyed out or the downloader is launched when there are no rules,
>>> yet.
>>>
>>> * It is possible to enable suricata without selecting any interfaces.
>>> That just wastes RAM and makes it easy to enable it without actually
>>> having the protection.
>>>
>>> If you agree, I will create tickets for this.
>> Thanks, fixed.
>>
>> https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=ebdd0f9a90da800cc6173f6f30fb0621dddc354b
> I am on a plane now, so I cannot see this, but thanks!
>
>>> UI Stuff:
>>>
>>> * I think that the settings to enable suricata up to and including
>>> “Traffic analysing” should be in the top box. The second box should
>>> only care about the rules. That might even solve the problem
>>> described above.
>> During development I had a very similar idea, of grouping all suricata 
>> settings and all which are for the ruleset together into seperate
>> boxes. I dropped this idea, because of some big restrictions of the
>> code and I was afraid of how many work this would be to re-write huge
>> parts of it to get the desired result.
> I think that this is very little work.
>
>>> * I find it quite confusing that one has to “Activate Intrusion
>>> Detection System” and then again select IDS or IPS. It doesn’t seem
>>> to make much sense even to select the top box when IPS is checked. I
>>> would therefore prefer the following: one box that just says
>>> “Activate” and that switches on or off the whole thing, and then a
>>> second box that says “monitor only” or something along these lines.
>>> That box should be disabled by default which will run suricata in IPS
>>> mode (which is what everybody wants!) and when checked will downgrade
>>> it to IDS mode. Right now it looks like this is an equal level of
>>> both options but I think that there is a risk here that people enable
>>> IDS and think that they are protected when they are in fact not. I
>>> think the “monitor only” checkbox makes this a lot clearer.
>> Fair enough - I'm going now to rework the current section for choosing
>> the runmode and move from the radio buttons to a select box which will
>> be called "Only monitor traffic" as you suggested and will place it
>> right next to the box where you can enable the whole IPS/IDS system.
> I think the best wording is: Monitor traffic only
>
>>> * The Traffic Analyzing headline doesn’t make any sense. It should
>>> rather be “Monitored zones” or “Monitored interfaces”. Internet
>>> should be RED because it is called RED everywhere else.
>> ACK
>>
>>> * We will need to grep for Snort everywhere and replace that phrase
>>> with “Intrusion Prevention System”. I do not think that we have to
>>> mention suricata here instead. It is confusing when the menu option
>>> is called IDS.
>> Most work on this already should be done, I'll remove the last existing
>> snort entries if I'll found one. 
> Cool.
>
>>> * The menu option should be called IPS instead of IDS I think since
>>> IPS mode is default. Spelled out obviously.
>> ACK - It should be a very easy task to switch the language string from
>> "Intrusion Detection System" to the desired "Intrusion Prevention
>> System".
> Cool. I am just saying that it needs to be done :) don’t forget the logging section.
>
>>> Apart from that I have to say that this is really really good work,
>>> Stefan. The main thing is: it works. I think this is a huge
>>> improvement over what we had in the past and that this is bringing
>>> IPFire to the next level. It solves many problems that we didn’t dare
>>> to touch because snort was simply badly integrated into IPFire and
>>> did not really do much instead of wasting CPU cycles and memory.
>>>
>>> I hope that we can continue working in this direction and maybe add
>>> some nice reports and so on and obviously carry over this work into
>>> IPFire 3 as soon as possible.
>>>
>>> Please feel free to disagree with me above, but please do that via
>>> email if possible.
>>>
>>> I want to target merging this into next as soon as possible in
>>> January. I hope that we can get the things above sorted out over the
>>> holidays since it should not be much work.
>>>
>> Best regards,
>>
>> -Stefan
>>>> On 17 Dec 2018, at 19:08, Stefan Schantl <stefan.schantl(a)ipfire.org
>>>>> wrote:
>>>>> What is the bypass mark for? At least that should be set to
>>>>> nothing
>>>>> instead of one, because that will again conflict with the SNAT
>>>>> fix
>>>>> rule.
>>>>>
>>>>> Best,
>>>>> -Michael
>>>>>
>>>> Hello Michael,
>>>>
>>>> taken from the official suricata.yaml documentation:
>>>>
>>>> bypass mark and mask can be used to implement NFQ bypass. If bypass
>>>> mark is set then the NFQ bypass is activated. Suricata will set the
>>>> bypass mark/mask on packet of a flow that need to be bypassed. The
>>>> Nefilter ruleset has to directly accept all packets of a flow once
>>>> a
>>>> packet has been marked.
>>> Weird. Why would you even pass this to the userspace process then?
>>>
>>> Whatever, we don’t need it then...
>>>
>>>> In my commit, I've disabled the bypass options to exactly prevent
>>>> from
>>>> any issuse again with the SNAT packet mark.
>>>>
>>>> Best regards,
>>>>
>>>> -Stefan
>>> Best,
>>> -Michael
>>>
>>>>>> On 17 Dec 2018, at 14:21, Stefan Schantl <
>>>>>> stefan.schantl(a)ipfire.org
>>>>>>> wrote:
>>>>>> Am Sonntag, den 16.12.2018, 21:28 +0100 schrieb Peter Müller:
>>>>>>> Hello Stefan,
>>>>>>>
>>>>>>> to be a bit more precise about the NAT issue:
>>>>>>>
>>>>>>> My setup is the IPFire Suricata test VM running in KVM, with
>>>>>>> two clients (Debian and OpenBSD) directly attached to it.
>>>>>>>
>>>>>>> The Debian machine is located in GREEN, OpenBSD in ORANGE.
>>>>>>> RED interface is connected via bridge to my actual testing
>>>>>>> LAN;
>>>>>>> for the first testing, any outgoing traffic to the internet
>>>>>>> was allowed (I will test upstream proxy behaviour later).
>>>>>>>
>>>>>>> While GREEN was using IPv4 range 192.168.100.0/24, with
>>>>>>> IPFire
>>>>>>> as 192.168.100.1, enabling Suricata caused packets coming
>>>>>>> from
>>>>>>> GREEN not to be NATted anymore: Instead of using the
>>>>>>> firewall's
>>>>>>> RED IP for destination, it was the internal GREEN IP.
>>>>>>>
>>>>>>> Let me know whether is is useful or not. :-)
>>>>>>>
>>>>>>> Thanks, and best regards,
>>>>>>> Peter Müller
>>>>>>>
>>>>>>>> Hello Stefan,
>>>>>>>>
>>>>>>>> back again. :-)
>>>>>>>>
>>>>>>>> The new IDS WebUI looks quite good so far -
>>>>>>>> enabling/disabling
>>>>>>>> Suricata works as well as selecting the rule source and the
>>>>>>>> operation mode (IDS/IPS).
>>>>>>>>
>>>>>>>> I was also able to download the "Snort/VRT Community"
>>>>>>>> ruleset.
>>>>>>>> Trying to switch to the "Emerging Threats" ruleset is
>>>>>>>> possible,
>>>>>>>> but downloading its ruleset afterwards is not: The GUI
>>>>>>>> simply
>>>>>>>> stalls, printing a message that "Snort (!) is performing a
>>>>>>>> task".
>>>>>>>>
>>>>>>>> The WebUI services page still shows IDS status for each
>>>>>>>> interface,
>>>>>>>> which does not seem to work anymore (everything is stopped,
>>>>>>>> but
>>>>>>>> Suricata was active on RED and GREEN).
>>>>>>>>
>>>>>>>> Further, a client located in GREEN behind the test firewall
>>>>>>>> instance is unable to browse the internet as soon as
>>>>>>>> Suricata
>>>>>>>> is
>>>>>>>> enabled. If disabled, downloading ET rulesets work as well
>>>>>>>> as
>>>>>>>> internet traffic. At the moment, I am flying blind here,
>>>>>>>> but it
>>>>>>>> looks
>>>>>>>> like packets are not NATted anymore if Suricata is active.
>>>>>>>>
>>>>>>>> Any outgoing connection is in state "SYN_SENT" if Suricata
>>>>>>>> is
>>>>>>>> active.
>>>>>>>>
>>>>>>>> A portscan against the firewall (GREEN interface) is not
>>>>>>>> detected,
>>>>>>>> even though ET SCAN ruleset is enabled (used nmap with NSE
>>>>>>>> active).
>>>>>>>>
>>>>>>>> Especially the outgoing connection/NAT/? issue mentioned
>>>>>>>> above
>>>>>>>> breaks things in my scenario. Anything else are minor
>>>>>>>> issues
>>>>>>>> (of
>>>>>>>> course, a portscan should be detected, this needs further
>>>>>>>> investigation
>>>>>>>> indeed). WebUI works fine so far.
>>>>>>>>
>>>>>>>> Thanks again for your work; I hope the feedback can
>>>>>>>> appreciate
>>>>>>>> it
>>>>>>>> somehow. :-)
>>>>>>>>
>>>>>>>> Let me know if there are questions.
>>>>>>>>
>>>>>>>> Thanks, and best regards,
>>>>>>>> Peter Müller
>>>>>>>>
>>>>>> Hello Peter,
>>>>>>
>>>>>> thanks for testing and your feedback.
>>>>>>
>>>>>> I've setup a test environment and was able to re-produce your
>>>>>> NAT
>>>>>> issue
>>>>>> very easy. After some research and help of Michael, we were
>>>>>> able to
>>>>>> find the real issue.
>>>>>>
>>>>>> It was located in a identical mark on the packets which already
>>>>>> have
>>>>>> been scanned by suricata and packets which should be modified
>>>>>> for
>>>>>> SNAT
>>>>>> in the "Mangle" table of the routing logic by the kernel.
>>>>>>
>>>>>> The changes can be found in my git repository:
>>>>>>
>>>>>> https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=f5ad510e3c0f416a1507999f5ad20ab171df9c07
>>>>>>
>>>>>> I'll upload a fixed image very soon - please keep on testing.
>>>>>>
>>>>>> Best regards,
>>>>>>
>>>>>> -Stefan
>

Re: IPFire meets Suricata - Call for tester

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 11494 bytes --]

Hi,

> On 20 Dec 2018, at 13:03, Stefan Schantl <stefan.schantl(a)ipfire.org> wrote:
> 
> Hello Michael,
> 
> thanks for having a look at suricata and your feedback.
>> Hi,
>> 
>> So I finally had a closer look at this, too. I have a couple of
>> questions and would like to propose some changes to the UI:
>> 
>> Questions:
>> 
>> * Is it a good idea to have “No” rules as default and also “no”
>> automatic rule updates? I guess we can configure at least Emerging
>> Threats as a good default because they are free. I also think that
>> when IDS is enabled, the system should update the ruleset by default
>> once a day. When IDS is enabled, there should be no rule update.
>> 
>> * That also leads me to: Should no automatic updates even be an
>> option?
>> 
> Making the free ruleset of emergingthreads as default is okay for me -
> ACK.
> 
> I'm also fine with doing a weekly ruleset update as default setting.
> 
> My personal opinion is, it should be possible to disable the automatic
> rule update. There are some situations where this feature could be
> quite usefull, for example if you have only volume limited internet
> access or even other (which we currently do not have in mind....).

I guess if you don’t update the rules there is no point in running the IPS. It’s just like a virus scanner with outdated signatures: detecting even less.

Do we have more people with an opinion on this?

The rulesets are indeed quite large. But the only people who might want to disable it are people who run IPFire on an IoT appliance and those are too slow to run the IPS anyways. So it’s not a very strong argument in my opinion.

The best argument I can make for it is only that this is probably unwanted behaviour and therefore can easily be misconfugured.

> 
> 
>> Bugs:
>> 
>> * When I enabled IDS in IPS mode, I checked the box to activate it
>> and selected the ET ruleset and hit save. Suricata starts but without
>> rules which is definitely not what I wanted. Either the box should be
>> greyed out or the downloader is launched when there are no rules,
>> yet.
>> 
>> * It is possible to enable suricata without selecting any interfaces.
>> That just wastes RAM and makes it easy to enable it without actually
>> having the protection.
>> 
>> If you agree, I will create tickets for this.
> 
> Thanks, fixed.
> 
> https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=ebdd0f9a90da800cc6173f6f30fb0621dddc354b

I am on a plane now, so I cannot see this, but thanks!

>> 
>> UI Stuff:
>> 
>> * I think that the settings to enable suricata up to and including
>> “Traffic analysing” should be in the top box. The second box should
>> only care about the rules. That might even solve the problem
>> described above.
> 
> During development I had a very similar idea, of grouping all suricata 
> settings and all which are for the ruleset together into seperate
> boxes. I dropped this idea, because of some big restrictions of the
> code and I was afraid of how many work this would be to re-write huge
> parts of it to get the desired result.

I think that this is very little work.

> 
>> 
>> * I find it quite confusing that one has to “Activate Intrusion
>> Detection System” and then again select IDS or IPS. It doesn’t seem
>> to make much sense even to select the top box when IPS is checked. I
>> would therefore prefer the following: one box that just says
>> “Activate” and that switches on or off the whole thing, and then a
>> second box that says “monitor only” or something along these lines.
>> That box should be disabled by default which will run suricata in IPS
>> mode (which is what everybody wants!) and when checked will downgrade
>> it to IDS mode. Right now it looks like this is an equal level of
>> both options but I think that there is a risk here that people enable
>> IDS and think that they are protected when they are in fact not. I
>> think the “monitor only” checkbox makes this a lot clearer.
> 
> Fair enough - I'm going now to rework the current section for choosing
> the runmode and move from the radio buttons to a select box which will
> be called "Only monitor traffic" as you suggested and will place it
> right next to the box where you can enable the whole IPS/IDS system.

I think the best wording is: Monitor traffic only

> 
>> 
>> * The Traffic Analyzing headline doesn’t make any sense. It should
>> rather be “Monitored zones” or “Monitored interfaces”. Internet
>> should be RED because it is called RED everywhere else.
> 
> ACK
> 
>> 
>> * We will need to grep for Snort everywhere and replace that phrase
>> with “Intrusion Prevention System”. I do not think that we have to
>> mention suricata here instead. It is confusing when the menu option
>> is called IDS.
> 
> Most work on this already should be done, I'll remove the last existing
> snort entries if I'll found one. 

Cool.

> 
>> 
>> * The menu option should be called IPS instead of IDS I think since
>> IPS mode is default. Spelled out obviously.
> 
> ACK - It should be a very easy task to switch the language string from
> "Intrusion Detection System" to the desired "Intrusion Prevention
> System".

Cool. I am just saying that it needs to be done :) don’t forget the logging section.

> 
>> 
>> Apart from that I have to say that this is really really good work,
>> Stefan. The main thing is: it works. I think this is a huge
>> improvement over what we had in the past and that this is bringing
>> IPFire to the next level. It solves many problems that we didn’t dare
>> to touch because snort was simply badly integrated into IPFire and
>> did not really do much instead of wasting CPU cycles and memory.
>> 
>> I hope that we can continue working in this direction and maybe add
>> some nice reports and so on and obviously carry over this work into
>> IPFire 3 as soon as possible.
>> 
>> Please feel free to disagree with me above, but please do that via
>> email if possible.
>> 
>> I want to target merging this into next as soon as possible in
>> January. I hope that we can get the things above sorted out over the
>> holidays since it should not be much work.
>> 
> Best regards,
> 
> -Stefan
>>> On 17 Dec 2018, at 19:08, Stefan Schantl <stefan.schantl(a)ipfire.org
>>>> wrote:
>>> 
>>>> What is the bypass mark for? At least that should be set to
>>>> nothing
>>>> instead of one, because that will again conflict with the SNAT
>>>> fix
>>>> rule.
>>>> 
>>>> Best,
>>>> -Michael
>>>> 
>>> Hello Michael,
>>> 
>>> taken from the official suricata.yaml documentation:
>>> 
>>> bypass mark and mask can be used to implement NFQ bypass. If bypass
>>> mark is set then the NFQ bypass is activated. Suricata will set the
>>> bypass mark/mask on packet of a flow that need to be bypassed. The
>>> Nefilter ruleset has to directly accept all packets of a flow once
>>> a
>>> packet has been marked.
>> 
>> Weird. Why would you even pass this to the userspace process then?
>> 
>> Whatever, we don’t need it then...
>> 
>>> In my commit, I've disabled the bypass options to exactly prevent
>>> from
>>> any issuse again with the SNAT packet mark.
>>> 
>>> Best regards,
>>> 
>>> -Stefan
>> 
>> Best,
>> -Michael
>> 
>>>>> On 17 Dec 2018, at 14:21, Stefan Schantl <
>>>>> stefan.schantl(a)ipfire.org
>>>>>> wrote:
>>>>> 
>>>>> Am Sonntag, den 16.12.2018, 21:28 +0100 schrieb Peter Müller:
>>>>>> Hello Stefan,
>>>>>> 
>>>>>> to be a bit more precise about the NAT issue:
>>>>>> 
>>>>>> My setup is the IPFire Suricata test VM running in KVM, with
>>>>>> two clients (Debian and OpenBSD) directly attached to it.
>>>>>> 
>>>>>> The Debian machine is located in GREEN, OpenBSD in ORANGE.
>>>>>> RED interface is connected via bridge to my actual testing
>>>>>> LAN;
>>>>>> for the first testing, any outgoing traffic to the internet
>>>>>> was allowed (I will test upstream proxy behaviour later).
>>>>>> 
>>>>>> While GREEN was using IPv4 range 192.168.100.0/24, with
>>>>>> IPFire
>>>>>> as 192.168.100.1, enabling Suricata caused packets coming
>>>>>> from
>>>>>> GREEN not to be NATted anymore: Instead of using the
>>>>>> firewall's
>>>>>> RED IP for destination, it was the internal GREEN IP.
>>>>>> 
>>>>>> Let me know whether is is useful or not. :-)
>>>>>> 
>>>>>> Thanks, and best regards,
>>>>>> Peter Müller
>>>>>> 
>>>>>>> Hello Stefan,
>>>>>>> 
>>>>>>> back again. :-)
>>>>>>> 
>>>>>>> The new IDS WebUI looks quite good so far -
>>>>>>> enabling/disabling
>>>>>>> Suricata works as well as selecting the rule source and the
>>>>>>> operation mode (IDS/IPS).
>>>>>>> 
>>>>>>> I was also able to download the "Snort/VRT Community"
>>>>>>> ruleset.
>>>>>>> Trying to switch to the "Emerging Threats" ruleset is
>>>>>>> possible,
>>>>>>> but downloading its ruleset afterwards is not: The GUI
>>>>>>> simply
>>>>>>> stalls, printing a message that "Snort (!) is performing a
>>>>>>> task".
>>>>>>> 
>>>>>>> The WebUI services page still shows IDS status for each
>>>>>>> interface,
>>>>>>> which does not seem to work anymore (everything is stopped,
>>>>>>> but
>>>>>>> Suricata was active on RED and GREEN).
>>>>>>> 
>>>>>>> Further, a client located in GREEN behind the test firewall
>>>>>>> instance is unable to browse the internet as soon as
>>>>>>> Suricata
>>>>>>> is
>>>>>>> enabled. If disabled, downloading ET rulesets work as well
>>>>>>> as
>>>>>>> internet traffic. At the moment, I am flying blind here,
>>>>>>> but it
>>>>>>> looks
>>>>>>> like packets are not NATted anymore if Suricata is active.
>>>>>>> 
>>>>>>> Any outgoing connection is in state "SYN_SENT" if Suricata
>>>>>>> is
>>>>>>> active.
>>>>>>> 
>>>>>>> A portscan against the firewall (GREEN interface) is not
>>>>>>> detected,
>>>>>>> even though ET SCAN ruleset is enabled (used nmap with NSE
>>>>>>> active).
>>>>>>> 
>>>>>>> Especially the outgoing connection/NAT/? issue mentioned
>>>>>>> above
>>>>>>> breaks things in my scenario. Anything else are minor
>>>>>>> issues
>>>>>>> (of
>>>>>>> course, a portscan should be detected, this needs further
>>>>>>> investigation
>>>>>>> indeed). WebUI works fine so far.
>>>>>>> 
>>>>>>> Thanks again for your work; I hope the feedback can
>>>>>>> appreciate
>>>>>>> it
>>>>>>> somehow. :-)
>>>>>>> 
>>>>>>> Let me know if there are questions.
>>>>>>> 
>>>>>>> Thanks, and best regards,
>>>>>>> Peter Müller
>>>>>>> 
>>>>> Hello Peter,
>>>>> 
>>>>> thanks for testing and your feedback.
>>>>> 
>>>>> I've setup a test environment and was able to re-produce your
>>>>> NAT
>>>>> issue
>>>>> very easy. After some research and help of Michael, we were
>>>>> able to
>>>>> find the real issue.
>>>>> 
>>>>> It was located in a identical mark on the packets which already
>>>>> have
>>>>> been scanned by suricata and packets which should be modified
>>>>> for
>>>>> SNAT
>>>>> in the "Mangle" table of the routing logic by the kernel.
>>>>> 
>>>>> The changes can be found in my git repository:
>>>>> 
>>>>> https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=f5ad510e3c0f416a1507999f5ad20ab171df9c07
>>>>> 
>>>>> I'll upload a fixed image very soon - please keep on testing.
>>>>> 
>>>>> Best regards,
>>>>> 
>>>>> -Stefan

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 10783 bytes --]

Hello Michael,

thanks for having a look at suricata and your feedback.
> Hi,
> 
> So I finally had a closer look at this, too. I have a couple of
> questions and would like to propose some changes to the UI:
> 
> Questions:
> 
> * Is it a good idea to have “No” rules as default and also “no”
> automatic rule updates? I guess we can configure at least Emerging
> Threats as a good default because they are free. I also think that
> when IDS is enabled, the system should update the ruleset by default
> once a day. When IDS is enabled, there should be no rule update.
> 
> * That also leads me to: Should no automatic updates even be an
> option?
> 
Making the free ruleset of emergingthreads as default is okay for me -
ACK.

I'm also fine with doing a weekly ruleset update as default setting.

My personal opinion is, it should be possible to disable the automatic
rule update. There are some situations where this feature could be
quite usefull, for example if you have only volume limited internet
access or even other (which we currently do not have in mind....).


> Bugs:
> 
> * When I enabled IDS in IPS mode, I checked the box to activate it
> and selected the ET ruleset and hit save. Suricata starts but without
> rules which is definitely not what I wanted. Either the box should be
> greyed out or the downloader is launched when there are no rules,
> yet.
> 
> * It is possible to enable suricata without selecting any interfaces.
> That just wastes RAM and makes it easy to enable it without actually
> having the protection.
> 
> If you agree, I will create tickets for this.

Thanks, fixed.

https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=ebdd0f9a90da800cc6173f6f30fb0621dddc354b
> 
> UI Stuff:
> 
> * I think that the settings to enable suricata up to and including
> “Traffic analysing” should be in the top box. The second box should
> only care about the rules. That might even solve the problem
> described above.

During development I had a very similar idea, of grouping all suricata 
settings and all which are for the ruleset together into seperate
boxes. I dropped this idea, because of some big restrictions of the
code and I was afraid of how many work this would be to re-write huge
parts of it to get the desired result.

> 
> * I find it quite confusing that one has to “Activate Intrusion
> Detection System” and then again select IDS or IPS. It doesn’t seem
> to make much sense even to select the top box when IPS is checked. I
> would therefore prefer the following: one box that just says
> “Activate” and that switches on or off the whole thing, and then a
> second box that says “monitor only” or something along these lines.
> That box should be disabled by default which will run suricata in IPS
> mode (which is what everybody wants!) and when checked will downgrade
> it to IDS mode. Right now it looks like this is an equal level of
> both options but I think that there is a risk here that people enable
> IDS and think that they are protected when they are in fact not. I
> think the “monitor only” checkbox makes this a lot clearer.

Fair enough - I'm going now to rework the current section for choosing
the runmode and move from the radio buttons to a select box which will
be called "Only monitor traffic" as you suggested and will place it
right next to the box where you can enable the whole IPS/IDS system.

> 
> * The Traffic Analyzing headline doesn’t make any sense. It should
> rather be “Monitored zones” or “Monitored interfaces”. Internet
> should be RED because it is called RED everywhere else.

ACK

> 
> * We will need to grep for Snort everywhere and replace that phrase
> with “Intrusion Prevention System”. I do not think that we have to
> mention suricata here instead. It is confusing when the menu option
> is called IDS.

Most work on this already should be done, I'll remove the last existing
snort entries if I'll found one. 

> 
> * The menu option should be called IPS instead of IDS I think since
> IPS mode is default. Spelled out obviously.

ACK - It should be a very easy task to switch the language string from
"Intrusion Detection System" to the desired "Intrusion Prevention
System".

> 
> Apart from that I have to say that this is really really good work,
> Stefan. The main thing is: it works. I think this is a huge
> improvement over what we had in the past and that this is bringing
> IPFire to the next level. It solves many problems that we didn’t dare
> to touch because snort was simply badly integrated into IPFire and
> did not really do much instead of wasting CPU cycles and memory.
> 
> I hope that we can continue working in this direction and maybe add
> some nice reports and so on and obviously carry over this work into
> IPFire 3 as soon as possible.
> 
> Please feel free to disagree with me above, but please do that via
> email if possible.
> 
> I want to target merging this into next as soon as possible in
> January. I hope that we can get the things above sorted out over the
> holidays since it should not be much work.
> 
Best regards,

-Stefan
> > On 17 Dec 2018, at 19:08, Stefan Schantl <stefan.schantl(a)ipfire.org
> > > wrote:
> > 
> > > What is the bypass mark for? At least that should be set to
> > > nothing
> > > instead of one, because that will again conflict with the SNAT
> > > fix
> > > rule.
> > > 
> > > Best,
> > > -Michael
> > > 
> > Hello Michael,
> > 
> > taken from the official suricata.yaml documentation:
> > 
> > bypass mark and mask can be used to implement NFQ bypass. If bypass
> > mark is set then the NFQ bypass is activated. Suricata will set the
> > bypass mark/mask on packet of a flow that need to be bypassed. The
> > Nefilter ruleset has to directly accept all packets of a flow once
> > a
> > packet has been marked.
> 
> Weird. Why would you even pass this to the userspace process then?
> 
> Whatever, we don’t need it then...
> 
> > In my commit, I've disabled the bypass options to exactly prevent
> > from
> > any issuse again with the SNAT packet mark.
> > 
> > Best regards,
> > 
> > -Stefan
> 
> Best,
> -Michael
> 
> > > > On 17 Dec 2018, at 14:21, Stefan Schantl <
> > > > stefan.schantl(a)ipfire.org
> > > > > wrote:
> > > > 
> > > > Am Sonntag, den 16.12.2018, 21:28 +0100 schrieb Peter Müller:
> > > > > Hello Stefan,
> > > > > 
> > > > > to be a bit more precise about the NAT issue:
> > > > > 
> > > > > My setup is the IPFire Suricata test VM running in KVM, with
> > > > > two clients (Debian and OpenBSD) directly attached to it.
> > > > > 
> > > > > The Debian machine is located in GREEN, OpenBSD in ORANGE.
> > > > > RED interface is connected via bridge to my actual testing
> > > > > LAN;
> > > > > for the first testing, any outgoing traffic to the internet
> > > > > was allowed (I will test upstream proxy behaviour later).
> > > > > 
> > > > > While GREEN was using IPv4 range 192.168.100.0/24, with
> > > > > IPFire
> > > > > as 192.168.100.1, enabling Suricata caused packets coming
> > > > > from
> > > > > GREEN not to be NATted anymore: Instead of using the
> > > > > firewall's
> > > > > RED IP for destination, it was the internal GREEN IP.
> > > > > 
> > > > > Let me know whether is is useful or not. :-)
> > > > > 
> > > > > Thanks, and best regards,
> > > > > Peter Müller
> > > > > 
> > > > > > Hello Stefan,
> > > > > > 
> > > > > > back again. :-)
> > > > > > 
> > > > > > The new IDS WebUI looks quite good so far -
> > > > > > enabling/disabling
> > > > > > Suricata works as well as selecting the rule source and the
> > > > > > operation mode (IDS/IPS).
> > > > > > 
> > > > > > I was also able to download the "Snort/VRT Community"
> > > > > > ruleset.
> > > > > > Trying to switch to the "Emerging Threats" ruleset is
> > > > > > possible,
> > > > > > but downloading its ruleset afterwards is not: The GUI
> > > > > > simply
> > > > > > stalls, printing a message that "Snort (!) is performing a
> > > > > > task".
> > > > > > 
> > > > > > The WebUI services page still shows IDS status for each
> > > > > > interface,
> > > > > > which does not seem to work anymore (everything is stopped,
> > > > > > but
> > > > > > Suricata was active on RED and GREEN).
> > > > > > 
> > > > > > Further, a client located in GREEN behind the test firewall
> > > > > > instance is unable to browse the internet as soon as
> > > > > > Suricata
> > > > > > is
> > > > > > enabled. If disabled, downloading ET rulesets work as well
> > > > > > as
> > > > > > internet traffic. At the moment, I am flying blind here,
> > > > > > but it
> > > > > > looks
> > > > > > like packets are not NATted anymore if Suricata is active.
> > > > > > 
> > > > > > Any outgoing connection is in state "SYN_SENT" if Suricata
> > > > > > is
> > > > > > active.
> > > > > > 
> > > > > > A portscan against the firewall (GREEN interface) is not
> > > > > > detected,
> > > > > > even though ET SCAN ruleset is enabled (used nmap with NSE
> > > > > > active).
> > > > > > 
> > > > > > Especially the outgoing connection/NAT/? issue mentioned
> > > > > > above
> > > > > > breaks things in my scenario. Anything else are minor
> > > > > > issues
> > > > > > (of
> > > > > > course, a portscan should be detected, this needs further
> > > > > > investigation
> > > > > > indeed). WebUI works fine so far.
> > > > > > 
> > > > > > Thanks again for your work; I hope the feedback can
> > > > > > appreciate
> > > > > > it
> > > > > > somehow. :-)
> > > > > > 
> > > > > > Let me know if there are questions.
> > > > > > 
> > > > > > Thanks, and best regards,
> > > > > > Peter Müller
> > > > > > 
> > > > Hello Peter,
> > > > 
> > > > thanks for testing and your feedback.
> > > > 
> > > > I've setup a test environment and was able to re-produce your
> > > > NAT
> > > > issue
> > > > very easy. After some research and help of Michael, we were
> > > > able to
> > > > find the real issue.
> > > > 
> > > > It was located in a identical mark on the packets which already
> > > > have
> > > > been scanned by suricata and packets which should be modified
> > > > for
> > > > SNAT
> > > > in the "Mangle" table of the routing logic by the kernel.
> > > > 
> > > > The changes can be found in my git repository:
> > > > 
> > > > https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=f5ad510e3c0f416a1507999f5ad20ab171df9c07
> > > > 
> > > > I'll upload a fixed image very soon - please keep on testing.
> > > > 
> > > > Best regards,
> > > > 
> > > > -Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 8175 bytes --]

Hi,

So I finally had a closer look at this, too. I have a couple of questions and would like to propose some changes to the UI:

Questions:

* Is it a good idea to have “No” rules as default and also “no” automatic rule updates? I guess we can configure at least Emerging Threats as a good default because they are free. I also think that when IDS is enabled, the system should update the ruleset by default once a day. When IDS is enabled, there should be no rule update.

* That also leads me to: Should no automatic updates even be an option?

Bugs:

* When I enabled IDS in IPS mode, I checked the box to activate it and selected the ET ruleset and hit save. Suricata starts but without rules which is definitely not what I wanted. Either the box should be greyed out or the downloader is launched when there are no rules, yet.

* It is possible to enable suricata without selecting any interfaces. That just wastes RAM and makes it easy to enable it without actually having the protection.

If you agree, I will create tickets for this.

UI Stuff:

* I think that the settings to enable suricata up to and including “Traffic analysing” should be in the top box. The second box should only care about the rules. That might even solve the problem described above.

* I find it quite confusing that one has to “Activate Intrusion Detection System” and then again select IDS or IPS. It doesn’t seem to make much sense even to select the top box when IPS is checked. I would therefore prefer the following: one box that just says “Activate” and that switches on or off the whole thing, and then a second box that says “monitor only” or something along these lines. That box should be disabled by default which will run suricata in IPS mode (which is what everybody wants!) and when checked will downgrade it to IDS mode. Right now it looks like this is an equal level of both options but I think that there is a risk here that people enable IDS and think that they are protected when they are in fact not. I think the “monitor only” checkbox makes this a lot clearer.

* The Traffic Analyzing headline doesn’t make any sense. It should rather be “Monitored zones” or “Monitored interfaces”. Internet should be RED because it is called RED everywhere else.

* We will need to grep for Snort everywhere and replace that phrase with “Intrusion Prevention System”. I do not think that we have to mention suricata here instead. It is confusing when the menu option is called IDS.

* The menu option should be called IPS instead of IDS I think since IPS mode is default. Spelled out obviously.

Apart from that I have to say that this is really really good work, Stefan. The main thing is: it works. I think this is a huge improvement over what we had in the past and that this is bringing IPFire to the next level. It solves many problems that we didn’t dare to touch because snort was simply badly integrated into IPFire and did not really do much instead of wasting CPU cycles and memory.

I hope that we can continue working in this direction and maybe add some nice reports and so on and obviously carry over this work into IPFire 3 as soon as possible.

Please feel free to disagree with me above, but please do that via email if possible.

I want to target merging this into next as soon as possible in January. I hope that we can get the things above sorted out over the holidays since it should not be much work.

> On 17 Dec 2018, at 19:08, Stefan Schantl <stefan.schantl(a)ipfire.org> wrote:
> 
>> What is the bypass mark for? At least that should be set to nothing
>> instead of one, because that will again conflict with the SNAT fix
>> rule.
>> 
>> Best,
>> -Michael
>> 
> Hello Michael,
> 
> taken from the official suricata.yaml documentation:
> 
> bypass mark and mask can be used to implement NFQ bypass. If bypass
> mark is set then the NFQ bypass is activated. Suricata will set the
> bypass mark/mask on packet of a flow that need to be bypassed. The
> Nefilter ruleset has to directly accept all packets of a flow once a
> packet has been marked.

Weird. Why would you even pass this to the userspace process then?

Whatever, we don’t need it then...

> 
> In my commit, I've disabled the bypass options to exactly prevent from
> any issuse again with the SNAT packet mark.
> 
> Best regards,
> 
> -Stefan

Best,
-Michael

> 
>>> On 17 Dec 2018, at 14:21, Stefan Schantl <stefan.schantl(a)ipfire.org
>>>> wrote:
>>> 
>>> Am Sonntag, den 16.12.2018, 21:28 +0100 schrieb Peter Müller:
>>>> Hello Stefan,
>>>> 
>>>> to be a bit more precise about the NAT issue:
>>>> 
>>>> My setup is the IPFire Suricata test VM running in KVM, with
>>>> two clients (Debian and OpenBSD) directly attached to it.
>>>> 
>>>> The Debian machine is located in GREEN, OpenBSD in ORANGE.
>>>> RED interface is connected via bridge to my actual testing LAN;
>>>> for the first testing, any outgoing traffic to the internet
>>>> was allowed (I will test upstream proxy behaviour later).
>>>> 
>>>> While GREEN was using IPv4 range 192.168.100.0/24, with IPFire
>>>> as 192.168.100.1, enabling Suricata caused packets coming from
>>>> GREEN not to be NATted anymore: Instead of using the firewall's
>>>> RED IP for destination, it was the internal GREEN IP.
>>>> 
>>>> Let me know whether is is useful or not. :-)
>>>> 
>>>> Thanks, and best regards,
>>>> Peter Müller
>>>> 
>>>>> Hello Stefan,
>>>>> 
>>>>> back again. :-)
>>>>> 
>>>>> The new IDS WebUI looks quite good so far - enabling/disabling
>>>>> Suricata works as well as selecting the rule source and the
>>>>> operation mode (IDS/IPS).
>>>>> 
>>>>> I was also able to download the "Snort/VRT Community" ruleset.
>>>>> Trying to switch to the "Emerging Threats" ruleset is possible,
>>>>> but downloading its ruleset afterwards is not: The GUI simply
>>>>> stalls, printing a message that "Snort (!) is performing a
>>>>> task".
>>>>> 
>>>>> The WebUI services page still shows IDS status for each
>>>>> interface,
>>>>> which does not seem to work anymore (everything is stopped, but
>>>>> Suricata was active on RED and GREEN).
>>>>> 
>>>>> Further, a client located in GREEN behind the test firewall
>>>>> instance is unable to browse the internet as soon as Suricata
>>>>> is
>>>>> enabled. If disabled, downloading ET rulesets work as well as
>>>>> internet traffic. At the moment, I am flying blind here, but it
>>>>> looks
>>>>> like packets are not NATted anymore if Suricata is active.
>>>>> 
>>>>> Any outgoing connection is in state "SYN_SENT" if Suricata is
>>>>> active.
>>>>> 
>>>>> A portscan against the firewall (GREEN interface) is not
>>>>> detected,
>>>>> even though ET SCAN ruleset is enabled (used nmap with NSE
>>>>> active).
>>>>> 
>>>>> Especially the outgoing connection/NAT/? issue mentioned above
>>>>> breaks things in my scenario. Anything else are minor issues
>>>>> (of
>>>>> course, a portscan should be detected, this needs further
>>>>> investigation
>>>>> indeed). WebUI works fine so far.
>>>>> 
>>>>> Thanks again for your work; I hope the feedback can appreciate
>>>>> it
>>>>> somehow. :-)
>>>>> 
>>>>> Let me know if there are questions.
>>>>> 
>>>>> Thanks, and best regards,
>>>>> Peter Müller
>>>>> 
>>> Hello Peter,
>>> 
>>> thanks for testing and your feedback.
>>> 
>>> I've setup a test environment and was able to re-produce your NAT
>>> issue
>>> very easy. After some research and help of Michael, we were able to
>>> find the real issue.
>>> 
>>> It was located in a identical mark on the packets which already
>>> have
>>> been scanned by suricata and packets which should be modified for
>>> SNAT
>>> in the "Mangle" table of the routing logic by the kernel.
>>> 
>>> The changes can be found in my git repository:
>>> 
>>> https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=f5ad510e3c0f416a1507999f5ad20ab171df9c07
>>> 
>>> I'll upload a fixed image very soon - please keep on testing.
>>> 
>>> Best regards,
>>> 
>>> -Stefan

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 4537 bytes --]

> What is the bypass mark for? At least that should be set to nothing
> instead of one, because that will again conflict with the SNAT fix
> rule.
> 
> Best,
> -Michael
> 
Hello Michael,

taken from the official suricata.yaml documentation:

bypass mark and mask can be used to implement NFQ bypass. If bypass
mark is set then the NFQ bypass is activated. Suricata will set the
bypass mark/mask on packet of a flow that need to be bypassed. The
Nefilter ruleset has to directly accept all packets of a flow once a
packet has been marked.

In my commit, I've disabled the bypass options to exactly prevent from
any issuse again with the SNAT packet mark.

Best regards,

-Stefan

> > On 17 Dec 2018, at 14:21, Stefan Schantl <stefan.schantl(a)ipfire.org
> > > wrote:
> > 
> > Am Sonntag, den 16.12.2018, 21:28 +0100 schrieb Peter Müller:
> > > Hello Stefan,
> > > 
> > > to be a bit more precise about the NAT issue:
> > > 
> > > My setup is the IPFire Suricata test VM running in KVM, with
> > > two clients (Debian and OpenBSD) directly attached to it.
> > > 
> > > The Debian machine is located in GREEN, OpenBSD in ORANGE.
> > > RED interface is connected via bridge to my actual testing LAN;
> > > for the first testing, any outgoing traffic to the internet
> > > was allowed (I will test upstream proxy behaviour later).
> > > 
> > > While GREEN was using IPv4 range 192.168.100.0/24, with IPFire
> > > as 192.168.100.1, enabling Suricata caused packets coming from
> > > GREEN not to be NATted anymore: Instead of using the firewall's
> > > RED IP for destination, it was the internal GREEN IP.
> > > 
> > > Let me know whether is is useful or not. :-)
> > > 
> > > Thanks, and best regards,
> > > Peter Müller
> > > 
> > > > Hello Stefan,
> > > > 
> > > > back again. :-)
> > > > 
> > > > The new IDS WebUI looks quite good so far - enabling/disabling
> > > > Suricata works as well as selecting the rule source and the
> > > > operation mode (IDS/IPS).
> > > > 
> > > > I was also able to download the "Snort/VRT Community" ruleset.
> > > > Trying to switch to the "Emerging Threats" ruleset is possible,
> > > > but downloading its ruleset afterwards is not: The GUI simply
> > > > stalls, printing a message that "Snort (!) is performing a
> > > > task".
> > > > 
> > > > The WebUI services page still shows IDS status for each
> > > > interface,
> > > > which does not seem to work anymore (everything is stopped, but
> > > > Suricata was active on RED and GREEN).
> > > > 
> > > > Further, a client located in GREEN behind the test firewall
> > > > instance is unable to browse the internet as soon as Suricata
> > > > is
> > > > enabled. If disabled, downloading ET rulesets work as well as
> > > > internet traffic. At the moment, I am flying blind here, but it
> > > > looks
> > > > like packets are not NATted anymore if Suricata is active.
> > > > 
> > > > Any outgoing connection is in state "SYN_SENT" if Suricata is
> > > > active.
> > > > 
> > > > A portscan against the firewall (GREEN interface) is not
> > > > detected,
> > > > even though ET SCAN ruleset is enabled (used nmap with NSE
> > > > active).
> > > > 
> > > > Especially the outgoing connection/NAT/? issue mentioned above
> > > > breaks things in my scenario. Anything else are minor issues
> > > > (of
> > > > course, a portscan should be detected, this needs further
> > > > investigation
> > > > indeed). WebUI works fine so far.
> > > > 
> > > > Thanks again for your work; I hope the feedback can appreciate
> > > > it
> > > > somehow. :-)
> > > > 
> > > > Let me know if there are questions.
> > > > 
> > > > Thanks, and best regards,
> > > > Peter Müller
> > > > 
> > Hello Peter,
> > 
> > thanks for testing and your feedback.
> > 
> > I've setup a test environment and was able to re-produce your NAT
> > issue
> > very easy. After some research and help of Michael, we were able to
> > find the real issue.
> > 
> > It was located in a identical mark on the packets which already
> > have
> > been scanned by suricata and packets which should be modified for
> > SNAT
> > in the "Mangle" table of the routing logic by the kernel.
> > 
> > The changes can be found in my git repository:
> > 
> > https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=f5ad510e3c0f416a1507999f5ad20ab171df9c07
> > 
> > I'll upload a fixed image very soon - please keep on testing.
> > 
> > Best regards,
> > 
> > -Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 3625 bytes --]

What is the bypass mark for? At least that should be set to nothing instead of one, because that will again conflict with the SNAT fix rule.

Best,
-Michael

> On 17 Dec 2018, at 14:21, Stefan Schantl <stefan.schantl(a)ipfire.org> wrote:
> 
> Am Sonntag, den 16.12.2018, 21:28 +0100 schrieb Peter Müller:
>> Hello Stefan,
>> 
>> to be a bit more precise about the NAT issue:
>> 
>> My setup is the IPFire Suricata test VM running in KVM, with
>> two clients (Debian and OpenBSD) directly attached to it.
>> 
>> The Debian machine is located in GREEN, OpenBSD in ORANGE.
>> RED interface is connected via bridge to my actual testing LAN;
>> for the first testing, any outgoing traffic to the internet
>> was allowed (I will test upstream proxy behaviour later).
>> 
>> While GREEN was using IPv4 range 192.168.100.0/24, with IPFire
>> as 192.168.100.1, enabling Suricata caused packets coming from
>> GREEN not to be NATted anymore: Instead of using the firewall's
>> RED IP for destination, it was the internal GREEN IP.
>> 
>> Let me know whether is is useful or not. :-)
>> 
>> Thanks, and best regards,
>> Peter Müller
>> 
>>> Hello Stefan,
>>> 
>>> back again. :-)
>>> 
>>> The new IDS WebUI looks quite good so far - enabling/disabling
>>> Suricata works as well as selecting the rule source and the
>>> operation mode (IDS/IPS).
>>> 
>>> I was also able to download the "Snort/VRT Community" ruleset.
>>> Trying to switch to the "Emerging Threats" ruleset is possible,
>>> but downloading its ruleset afterwards is not: The GUI simply
>>> stalls, printing a message that "Snort (!) is performing a task".
>>> 
>>> The WebUI services page still shows IDS status for each interface,
>>> which does not seem to work anymore (everything is stopped, but
>>> Suricata was active on RED and GREEN).
>>> 
>>> Further, a client located in GREEN behind the test firewall
>>> instance is unable to browse the internet as soon as Suricata is
>>> enabled. If disabled, downloading ET rulesets work as well as
>>> internet traffic. At the moment, I am flying blind here, but it
>>> looks
>>> like packets are not NATted anymore if Suricata is active.
>>> 
>>> Any outgoing connection is in state "SYN_SENT" if Suricata is
>>> active.
>>> 
>>> A portscan against the firewall (GREEN interface) is not detected,
>>> even though ET SCAN ruleset is enabled (used nmap with NSE active).
>>> 
>>> Especially the outgoing connection/NAT/? issue mentioned above
>>> breaks things in my scenario. Anything else are minor issues (of
>>> course, a portscan should be detected, this needs further
>>> investigation
>>> indeed). WebUI works fine so far.
>>> 
>>> Thanks again for your work; I hope the feedback can appreciate it
>>> somehow. :-)
>>> 
>>> Let me know if there are questions.
>>> 
>>> Thanks, and best regards,
>>> Peter Müller
>>> 
> Hello Peter,
> 
> thanks for testing and your feedback.
> 
> I've setup a test environment and was able to re-produce your NAT issue
> very easy. After some research and help of Michael, we were able to
> find the real issue.
> 
> It was located in a identical mark on the packets which already have
> been scanned by suricata and packets which should be modified for SNAT
> in the "Mangle" table of the routing logic by the kernel.
> 
> The changes can be found in my git repository:
> 
> https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=f5ad510e3c0f416a1507999f5ad20ab171df9c07
> 
> I'll upload a fixed image very soon - please keep on testing.
> 
> Best regards,
> 
> -Stefan

Re: IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 3309 bytes --]

Am Sonntag, den 16.12.2018, 21:28 +0100 schrieb Peter Müller:
> Hello Stefan,
> 
> to be a bit more precise about the NAT issue:
> 
> My setup is the IPFire Suricata test VM running in KVM, with
> two clients (Debian and OpenBSD) directly attached to it.
> 
> The Debian machine is located in GREEN, OpenBSD in ORANGE.
> RED interface is connected via bridge to my actual testing LAN;
> for the first testing, any outgoing traffic to the internet
> was allowed (I will test upstream proxy behaviour later).
> 
> While GREEN was using IPv4 range 192.168.100.0/24, with IPFire
> as 192.168.100.1, enabling Suricata caused packets coming from
> GREEN not to be NATted anymore: Instead of using the firewall's
> RED IP for destination, it was the internal GREEN IP.
> 
> Let me know whether is is useful or not. :-)
> 
> Thanks, and best regards,
> Peter Müller
> 
> > Hello Stefan,
> > 
> > back again. :-)
> > 
> > The new IDS WebUI looks quite good so far - enabling/disabling
> > Suricata works as well as selecting the rule source and the
> > operation mode (IDS/IPS).
> > 
> > I was also able to download the "Snort/VRT Community" ruleset.
> > Trying to switch to the "Emerging Threats" ruleset is possible,
> > but downloading its ruleset afterwards is not: The GUI simply
> > stalls, printing a message that "Snort (!) is performing a task".
> > 
> > The WebUI services page still shows IDS status for each interface,
> > which does not seem to work anymore (everything is stopped, but
> > Suricata was active on RED and GREEN).
> > 
> > Further, a client located in GREEN behind the test firewall
> > instance is unable to browse the internet as soon as Suricata is
> > enabled. If disabled, downloading ET rulesets work as well as
> > internet traffic. At the moment, I am flying blind here, but it
> > looks
> > like packets are not NATted anymore if Suricata is active.
> > 
> > Any outgoing connection is in state "SYN_SENT" if Suricata is
> > active.
> > 
> > A portscan against the firewall (GREEN interface) is not detected,
> > even though ET SCAN ruleset is enabled (used nmap with NSE active).
> > 
> > Especially the outgoing connection/NAT/? issue mentioned above
> > breaks things in my scenario. Anything else are minor issues (of
> > course, a portscan should be detected, this needs further
> > investigation
> > indeed). WebUI works fine so far.
> > 
> > Thanks again for your work; I hope the feedback can appreciate it
> > somehow. :-)
> > 
> > Let me know if there are questions.
> > 
> > Thanks, and best regards,
> > Peter Müller
> > 
Hello Peter,

thanks for testing and your feedback.

I've setup a test environment and was able to re-produce your NAT issue
very easy. After some research and help of Michael, we were able to
find the real issue.

It was located in a identical mark on the packets which already have
been scanned by suricata and packets which should be modified for SNAT
in the "Mangle" table of the routing logic by the kernel.

The changes can be found in my git repository:

https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=f5ad510e3c0f416a1507999f5ad20ab171df9c07

I'll upload a fixed image very soon - please keep on testing.

Best regards,

-Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: IPFire meets Suricata - Call for tester

[Peter Müller]

[-- Attachment #1: Type: text/plain, Size: 2620 bytes --]

Hello Stefan,

to be a bit more precise about the NAT issue:

My setup is the IPFire Suricata test VM running in KVM, with
two clients (Debian and OpenBSD) directly attached to it.

The Debian machine is located in GREEN, OpenBSD in ORANGE.
RED interface is connected via bridge to my actual testing LAN;
for the first testing, any outgoing traffic to the internet
was allowed (I will test upstream proxy behaviour later).

While GREEN was using IPv4 range 192.168.100.0/24, with IPFire
as 192.168.100.1, enabling Suricata caused packets coming from
GREEN not to be NATted anymore: Instead of using the firewall's
RED IP for destination, it was the internal GREEN IP.

Let me know whether is is useful or not. :-)

Thanks, and best regards,
Peter Müller

> Hello Stefan,
> 
> back again. :-)
> 
> The new IDS WebUI looks quite good so far - enabling/disabling
> Suricata works as well as selecting the rule source and the
> operation mode (IDS/IPS).
> 
> I was also able to download the "Snort/VRT Community" ruleset.
> Trying to switch to the "Emerging Threats" ruleset is possible,
> but downloading its ruleset afterwards is not: The GUI simply
> stalls, printing a message that "Snort (!) is performing a task".
> 
> The WebUI services page still shows IDS status for each interface,
> which does not seem to work anymore (everything is stopped, but
> Suricata was active on RED and GREEN).
> 
> Further, a client located in GREEN behind the test firewall
> instance is unable to browse the internet as soon as Suricata is
> enabled. If disabled, downloading ET rulesets work as well as
> internet traffic. At the moment, I am flying blind here, but it looks
> like packets are not NATted anymore if Suricata is active.
> 
> Any outgoing connection is in state "SYN_SENT" if Suricata is active.
> 
> A portscan against the firewall (GREEN interface) is not detected,
> even though ET SCAN ruleset is enabled (used nmap with NSE active).
> 
> Especially the outgoing connection/NAT/? issue mentioned above
> breaks things in my scenario. Anything else are minor issues (of
> course, a portscan should be detected, this needs further investigation
> indeed). WebUI works fine so far.
> 
> Thanks again for your work; I hope the feedback can appreciate it somehow. :-)
> 
> Let me know if there are questions.
> 
> Thanks, and best regards,
> Peter Müller
> 

-- 
Microsoft DNS service terminates abnormally when it recieves a response
to a DNS query that was never made.  Fix Information: Run your DNS
service on a different platform.
		-- bugtraq

Re: IPFire meets Suricata - Call for tester

[Peter Müller]

[-- Attachment #1: Type: text/plain, Size: 1714 bytes --]

Hello Stefan,

back again. :-)

The new IDS WebUI looks quite good so far - enabling/disabling
Suricata works as well as selecting the rule source and the
operation mode (IDS/IPS).

I was also able to download the "Snort/VRT Community" ruleset.
Trying to switch to the "Emerging Threats" ruleset is possible,
but downloading its ruleset afterwards is not: The GUI simply
stalls, printing a message that "Snort (!) is performing a task".

The WebUI services page still shows IDS status for each interface,
which does not seem to work anymore (everything is stopped, but
Suricata was active on RED and GREEN).

Further, a client located in GREEN behind the test firewall
instance is unable to browse the internet as soon as Suricata is
enabled. If disabled, downloading ET rulesets work as well as
internet traffic. At the moment, I am flying blind here, but it looks
like packets are not NATted anymore if Suricata is active.

Any outgoing connection is in state "SYN_SENT" if Suricata is active.

A portscan against the firewall (GREEN interface) is not detected,
even though ET SCAN ruleset is enabled (used nmap with NSE active).

Especially the outgoing connection/NAT/? issue mentioned above
breaks things in my scenario. Anything else are minor issues (of
course, a portscan should be detected, this needs further investigation
indeed). WebUI works fine so far.

Thanks again for your work; I hope the feedback can appreciate it somehow. :-)

Let me know if there are questions.

Thanks, and best regards,
Peter Müller
-- 
Microsoft DNS service terminates abnormally when it recieves a response
to a DNS query that was never made.  Fix Information: Run your DNS
service on a different platform.
		-- bugtraq

Re: IPFire meets Suricata - Call for tester

[Peter Müller]

[-- Attachment #1: Type: text/plain, Size: 903 bytes --]

Hello Stefan,

I am sorry for my late reply. Hope you are not too frustrated by little
feedback volume. :-)

Thank you very much for putting so much effort into this. I installed the
ISO in a virtual machine, but are unfortunately experiencing network troubles,
so I did not get any IDS rules downloaded, yet.

The redesigned user interface looks quite nice and is intuitive (as far
as I can tell). At the moment, only one minor issue showed up: When downloading
IDS rules (or attempting to do so), "snort" is still displayed in the
message instead of "IDS" or "suricata".

I will write more details as soon I got the network to work properly.

Thanks again, and best regards,
Peter Müller
-- 
Microsoft DNS service terminates abnormally when it recieves a response
to a DNS query that was never made.  Fix Information: Run your DNS
service on a different platform.
		-- bugtraq

IPFire meets Suricata - Call for tester

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 1326 bytes --]

Hello list followers,

some time ago development for the new implementation of the Intrusion
Detection functionality in IPFire has been started.

The main goal, in a nutshell, was to give IPFire a modern, feature-rich 
and user-friendly Intrusion Detection Engine. During this progress, the
detection framework has been replaced - now suricata is used instead of
snort.

Suricata uses a very modern and multi-threaded detection engine with
support to perform actions on malicious traffic. So it provides the
functionality of detecting any kind of intrusion attempts and the
ability of guardian to block them under the same hood.

It was a lot of work, but finaly I'm happy to announce the first test
version. It is almost feature complete and without any kind of bigger
issues.

Because Intrusion Detection is a key feature of a firewall system, a
lot of testing is required until the new implementation can become part
of IPFire - therefore we need your help!

Download the test image (
https://people.ipfire.org/~stevee/suricata/Images/), do a lot of hard
testing and provide your feedback or suggestions on the develoment
mailing list (https://lists.ipfire.org/mailman/listinfo/development).

If you find any bugs please file them in the IPFire Bugtracker (
https://bugzilla.ipfire.org/).

Many thanks in advance,

-Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]