Hello,
Good job.
Stefan, are you planning to release another image very soon?
-Michael
On 25 Dec 2018, at 20:17, Stefan Schantl stefan.schantl@ipfire.org wrote:
Hello Tim,
thanks for joining and providing your feedback on this list.
Hi,
I've loaded a system up with the test image. I've not had much time to actually use it (hopefully after Christmas). I would like to know when there'll be an image with the marking fixed.
I'll generate and upload a new one very soon.
I agree with Michael over the GUI; splitting the system settings and ruleset settings seems like a good idea. The thing that caught me originally was that I selected a ruleset and then clicked on 'Download new ruleset' and nothing happened because I hadn't clicked on 'Save'.
I splitted the section into two parts. One for configuring the IDS and one which contains the rules settings for now.
A couple of points on automatic updates:
1 - The updates shouldn't run exactly on the hour since this means the load on the servers will be very bursty. It would be better to spread the load by running at some point past the hour and ensuring that this is different for every system (generate a random offset during installation?). Since the rulesets are sizeable (100 MiB for Talos VRT) and they're given away for free, I think it's a good idea to do something about this.
Thanks for that, I don't know if "fcron" does any kind of randomization. I'll have a closer look on it.
I'm reading you are using the ruleset provided by the Talos team. I'm intrested in adding support for this. Do you have some more details about this ruleset and a download URL?
2 - There's a warning in the oinkmaster documentation:
Oinkmaster can be run manually or automatically as a cron job. But of course, be very careful when doing the latter. Things could easily get messed up from one update to another because of different layout in the rules archive, typo in a rule, bugs in the script, and so on. At least run snort -T on the new rules before restarting your Snort process.
Since this is about the rulefiles I'm sure it applies to Suricata as well. I have actually seen one occasion when the updated rules failed snort -T with my updater - and was after the download passed an md5sum check. So the downloaded ruleset can be corrupt, if not very often. Should probably do something like:
download update backup old rules run oinkmaster if (suricata -T is OK) then tell suricata to re-read rules else restore backup end if
Thanks for the hint. I have to dig into this, if suricata offers a mechanism to check the ruleset before doing a reload of it.
Otherwise, I'm encouraged by the reduction in memory usage over snort, although it'll be interesting to see how this holds up with a larger ruleset and under heavier load.
Best regards,
-Stefan
Regards, Tim
On 20/12/2018 14:05, Michael Tremer wrote:
Hi,
On 20 Dec 2018, at 13:03, Stefan Schantl < stefan.schantl@ipfire.org> wrote:
Hello Michael,
thanks for having a look at suricata and your feedback.
Hi,
So I finally had a closer look at this, too. I have a couple of questions and would like to propose some changes to the UI:
Questions:
- Is it a good idea to have “No” rules as default and also “no”
automatic rule updates? I guess we can configure at least Emerging Threats as a good default because they are free. I also think that when IDS is enabled, the system should update the ruleset by default once a day. When IDS is enabled, there should be no rule update.
- That also leads me to: Should no automatic updates even be an
option?
Making the free ruleset of emergingthreads as default is okay for me - ACK.
I'm also fine with doing a weekly ruleset update as default setting.
My personal opinion is, it should be possible to disable the automatic rule update. There are some situations where this feature could be quite usefull, for example if you have only volume limited internet access or even other (which we currently do not have in mind....).
I guess if you don’t update the rules there is no point in running the IPS. It’s just like a virus scanner with outdated signatures: detecting even less.
Do we have more people with an opinion on this?
The rulesets are indeed quite large. But the only people who might want to disable it are people who run IPFire on an IoT appliance and those are too slow to run the IPS anyways. So it’s not a very strong argument in my opinion.
The best argument I can make for it is only that this is probably unwanted behaviour and therefore can easily be misconfugured.
Bugs:
- When I enabled IDS in IPS mode, I checked the box to activate
it and selected the ET ruleset and hit save. Suricata starts but without rules which is definitely not what I wanted. Either the box should be greyed out or the downloader is launched when there are no rules, yet.
- It is possible to enable suricata without selecting any
interfaces. That just wastes RAM and makes it easy to enable it without actually having the protection.
If you agree, I will create tickets for this.
Thanks, fixed.
https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=ebdd0f9a90...
I am on a plane now, so I cannot see this, but thanks!
UI Stuff:
- I think that the settings to enable suricata up to and
including “Traffic analysing” should be in the top box. The second box should only care about the rules. That might even solve the problem described above.
During development I had a very similar idea, of grouping all suricata settings and all which are for the ruleset together into seperate boxes. I dropped this idea, because of some big restrictions of the code and I was afraid of how many work this would be to re-write huge parts of it to get the desired result.
I think that this is very little work.
- I find it quite confusing that one has to “Activate Intrusion
Detection System” and then again select IDS or IPS. It doesn’t seem to make much sense even to select the top box when IPS is checked. I would therefore prefer the following: one box that just says “Activate” and that switches on or off the whole thing, and then a second box that says “monitor only” or something along these lines. That box should be disabled by default which will run suricata in IPS mode (which is what everybody wants!) and when checked will downgrade it to IDS mode. Right now it looks like this is an equal level of both options but I think that there is a risk here that people enable IDS and think that they are protected when they are in fact not. I think the “monitor only” checkbox makes this a lot clearer.
Fair enough - I'm going now to rework the current section for choosing the runmode and move from the radio buttons to a select box which will be called "Only monitor traffic" as you suggested and will place it right next to the box where you can enable the whole IPS/IDS system.
I think the best wording is: Monitor traffic only
- The Traffic Analyzing headline doesn’t make any sense. It
should rather be “Monitored zones” or “Monitored interfaces”. Internet should be RED because it is called RED everywhere else.
ACK
- We will need to grep for Snort everywhere and replace that
phrase with “Intrusion Prevention System”. I do not think that we have to mention suricata here instead. It is confusing when the menu option is called IDS.
Most work on this already should be done, I'll remove the last existing snort entries if I'll found one.
Cool.
- The menu option should be called IPS instead of IDS I think
since IPS mode is default. Spelled out obviously.
ACK - It should be a very easy task to switch the language string from "Intrusion Detection System" to the desired "Intrusion Prevention System".
Cool. I am just saying that it needs to be done :) don’t forget the logging section.
Apart from that I have to say that this is really really good work, Stefan. The main thing is: it works. I think this is a huge improvement over what we had in the past and that this is bringing IPFire to the next level. It solves many problems that we didn’t dare to touch because snort was simply badly integrated into IPFire and did not really do much instead of wasting CPU cycles and memory.
I hope that we can continue working in this direction and maybe add some nice reports and so on and obviously carry over this work into IPFire 3 as soon as possible.
Please feel free to disagree with me above, but please do that via email if possible.
I want to target merging this into next as soon as possible in January. I hope that we can get the things above sorted out over the holidays since it should not be much work.
Best regards,
-Stefan
On 17 Dec 2018, at 19:08, Stefan Schantl < stefan.schantl@ipfire.org > wrote: > What is the bypass mark for? At least that should be set to > nothing > instead of one, because that will again conflict with the > SNAT > fix > rule. > > Best, > -Michael > Hello Michael,
taken from the official suricata.yaml documentation:
bypass mark and mask can be used to implement NFQ bypass. If bypass mark is set then the NFQ bypass is activated. Suricata will set the bypass mark/mask on packet of a flow that need to be bypassed. The Nefilter ruleset has to directly accept all packets of a flow once a packet has been marked.
Weird. Why would you even pass this to the userspace process then?
Whatever, we don’t need it then...
In my commit, I've disabled the bypass options to exactly prevent from any issuse again with the SNAT packet mark.
Best regards,
-Stefan
Best, -Michael
>> On 17 Dec 2018, at 14:21, Stefan Schantl < >> stefan.schantl@ipfire.org >>> wrote: >> Am Sonntag, den 16.12.2018, 21:28 +0100 schrieb Peter >> Müller: >>> Hello Stefan, >>> >>> to be a bit more precise about the NAT issue: >>> >>> My setup is the IPFire Suricata test VM running in KVM, >>> with >>> two clients (Debian and OpenBSD) directly attached to >>> it. >>> >>> The Debian machine is located in GREEN, OpenBSD in >>> ORANGE. >>> RED interface is connected via bridge to my actual >>> testing >>> LAN; >>> for the first testing, any outgoing traffic to the >>> internet >>> was allowed (I will test upstream proxy behaviour >>> later). >>> >>> While GREEN was using IPv4 range 192.168.100.0/24, with >>> IPFire >>> as 192.168.100.1, enabling Suricata caused packets >>> coming >>> from >>> GREEN not to be NATted anymore: Instead of using the >>> firewall's >>> RED IP for destination, it was the internal GREEN IP. >>> >>> Let me know whether is is useful or not. :-) >>> >>> Thanks, and best regards, >>> Peter Müller >>> >>>> Hello Stefan, >>>> >>>> back again. :-) >>>> >>>> The new IDS WebUI looks quite good so far - >>>> enabling/disabling >>>> Suricata works as well as selecting the rule source >>>> and the >>>> operation mode (IDS/IPS). >>>> >>>> I was also able to download the "Snort/VRT Community" >>>> ruleset. >>>> Trying to switch to the "Emerging Threats" ruleset is >>>> possible, >>>> but downloading its ruleset afterwards is not: The >>>> GUI >>>> simply >>>> stalls, printing a message that "Snort (!) is >>>> performing a >>>> task". >>>> >>>> The WebUI services page still shows IDS status for >>>> each >>>> interface, >>>> which does not seem to work anymore (everything is >>>> stopped, >>>> but >>>> Suricata was active on RED and GREEN). >>>> >>>> Further, a client located in GREEN behind the test >>>> firewall >>>> instance is unable to browse the internet as soon as >>>> Suricata >>>> is >>>> enabled. If disabled, downloading ET rulesets work as >>>> well >>>> as >>>> internet traffic. At the moment, I am flying blind >>>> here, >>>> but it >>>> looks >>>> like packets are not NATted anymore if Suricata is >>>> active. >>>> >>>> Any outgoing connection is in state "SYN_SENT" if >>>> Suricata >>>> is >>>> active. >>>> >>>> A portscan against the firewall (GREEN interface) is >>>> not >>>> detected, >>>> even though ET SCAN ruleset is enabled (used nmap >>>> with NSE >>>> active). >>>> >>>> Especially the outgoing connection/NAT/? issue >>>> mentioned >>>> above >>>> breaks things in my scenario. Anything else are minor >>>> issues >>>> (of >>>> course, a portscan should be detected, this needs >>>> further >>>> investigation >>>> indeed). WebUI works fine so far. >>>> >>>> Thanks again for your work; I hope the feedback can >>>> appreciate >>>> it >>>> somehow. :-) >>>> >>>> Let me know if there are questions. >>>> >>>> Thanks, and best regards, >>>> Peter Müller >>>> >> Hello Peter, >> >> thanks for testing and your feedback. >> >> I've setup a test environment and was able to re-produce >> your >> NAT >> issue >> very easy. After some research and help of Michael, we >> were >> able to >> find the real issue. >> >> It was located in a identical mark on the packets which >> already >> have >> been scanned by suricata and packets which should be >> modified >> for >> SNAT >> in the "Mangle" table of the routing logic by the kernel. >> >> The changes can be found in my git repository: >> >> https://git.ipfire.org/?p=people/stevee/ipfire-2.x.git;a=commit;h=f5ad510e3c... >> >> I'll upload a fixed image very soon - please keep on >> testing. >> >> Best regards, >> >> -Stefan