From mboxrd@z Thu Jan 1 00:00:00 1970 From: Stefan Schantl To: development@lists.ipfire.org Subject: Re: IPFire meets Suricata - Call for tester Date: Tue, 25 Dec 2018 20:03:41 +0100 Message-ID: In-Reply-To: <3F5A20DB-12BB-4E78-A174-4740A7E9E6FD@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3075145354736319752==" List-Id: --===============3075145354736319752== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello Michael, > Hi, >=20 > > On 20 Dec 2018, at 13:03, Stefan Schantl > > wrote: > >=20 > > Hello Michael, > >=20 > > thanks for having a look at suricata and your feedback. > > > Hi, > > >=20 > > > So I finally had a closer look at this, too. I have a couple of > > > questions and would like to propose some changes to the UI: > > >=20 > > > Questions: > > >=20 > > > * Is it a good idea to have =E2=80=9CNo=E2=80=9D rules as default and a= lso =E2=80=9Cno=E2=80=9D > > > automatic rule updates? I guess we can configure at least > > > Emerging > > > Threats as a good default because they are free. I also think > > > that > > > when IDS is enabled, the system should update the ruleset by > > > default > > > once a day. When IDS is enabled, there should be no rule update. > > >=20 > > > * That also leads me to: Should no automatic updates even be an > > > option? > > >=20 > > Making the free ruleset of emergingthreads as default is okay for > > me - > > ACK. DONE - now the default selection, if no ruleset source has been choosen yet is the free emergingthreads ruleset. > >=20 > > I'm also fine with doing a weekly ruleset update as default > > setting. > >=20 > > My personal opinion is, it should be possible to disable the > > automatic > > rule update. There are some situations where this feature could be > > quite usefull, for example if you have only volume limited internet > > access or even other (which we currently do not have in mind....). >=20 > I guess if you don=E2=80=99t update the rules there is no point in running > the IPS. It=E2=80=99s just like a virus scanner with outdated signatures: > detecting even less. >=20 > Do we have more people with an opinion on this? >=20 > The rulesets are indeed quite large. But the only people who might > want to disable it are people who run IPFire on an IoT appliance and > those are too slow to run the IPS anyways. So it=E2=80=99s not a very strong > argument in my opinion. >=20 > The best argument I can make for it is only that this is probably > unwanted behaviour and therefore can easily be misconfugured. >=20 Default now is "weekly" until anything else has been selected. I keept the feature for disabling the automatic update for now until we have more opinions on this topic. > >=20 > > > Bugs: > > >=20 > > > * When I enabled IDS in IPS mode, I checked the box to activate > > > it > > > and selected the ET ruleset and hit save. Suricata starts but > > > without > > > rules which is definitely not what I wanted. Either the box > > > should be > > > greyed out or the downloader is launched when there are no rules, > > > yet. > > >=20 > > > * It is possible to enable suricata without selecting any > > > interfaces. > > > That just wastes RAM and makes it easy to enable it without > > > actually > > > having the protection. > > >=20 > > > If you agree, I will create tickets for this. > >=20 > > Thanks, fixed. > >=20 > > https://git.ipfire.org/?p=3Dpeople/stevee/ipfire-2.x.git;a=3Dcommit;h=3De= bdd0f9a90da800cc6173f6f30fb0621dddc354b >=20 > I am on a plane now, so I cannot see this, but thanks! >=20 > > > UI Stuff: > > >=20 > > > * I think that the settings to enable suricata up to and > > > including > > > =E2=80=9CTraffic analysing=E2=80=9D should be in the top box. The secon= d box > > > should > > > only care about the rules. That might even solve the problem > > > described above. > >=20 > > During development I had a very similar idea, of grouping all > > suricata=20 > > settings and all which are for the ruleset together into seperate > > boxes. I dropped this idea, because of some big restrictions of the > > code and I was afraid of how many work this would be to re-write > > huge > > parts of it to get the desired result. >=20 > I think that this is very little work. DONE - Technically it was just to splitt the HTML
from a single one into two seperate ones. But under the hood more code had to be changed. >=20 > > > * I find it quite confusing that one has to =E2=80=9CActivate Intrusion > > > Detection System=E2=80=9D and then again select IDS or IPS. It doesn=E2= =80=99t > > > seem > > > to make much sense even to select the top box when IPS is > > > checked. I > > > would therefore prefer the following: one box that just says > > > =E2=80=9CActivate=E2=80=9D and that switches on or off the whole thing,= and then > > > a > > > second box that says =E2=80=9Cmonitor only=E2=80=9D or something along = these > > > lines. > > > That box should be disabled by default which will run suricata in > > > IPS > > > mode (which is what everybody wants!) and when checked will > > > downgrade > > > it to IDS mode. Right now it looks like this is an equal level of > > > both options but I think that there is a risk here that people > > > enable > > > IDS and think that they are protected when they are in fact not. > > > I > > > think the =E2=80=9Cmonitor only=E2=80=9D checkbox makes this a lot clea= rer. > >=20 > > Fair enough - I'm going now to rework the current section for > > choosing > > the runmode and move from the radio buttons to a select box which > > will > > be called "Only monitor traffic" as you suggested and will place it > > right next to the box where you can enable the whole IPS/IDS > > system. >=20 > I think the best wording is: Monitor traffic only DONE - As promised, I've dropped the radio button selection for the runmode and placed a checkbox for this. >=20 > > > * The Traffic Analyzing headline doesn=E2=80=99t make any sense. It > > > should > > > rather be =E2=80=9CMonitored zones=E2=80=9D or =E2=80=9CMonitored inter= faces=E2=80=9D. Internet > > > should be RED because it is called RED everywhere else. > >=20 > > ACK DONE > >=20 > > > * We will need to grep for Snort everywhere and replace that > > > phrase > > > with =E2=80=9CIntrusion Prevention System=E2=80=9D. I do not think that= we have > > > to > > > mention suricata here instead. It is confusing when the menu > > > option > > > is called IDS. > >=20 > > Most work on this already should be done, I'll remove the last > > existing > > snort entries if I'll found one.=20 >=20 > Cool. This still needs to be done. >=20 > > > * The menu option should be called IPS instead of IDS I think > > > since > > > IPS mode is default. Spelled out obviously. > >=20 > > ACK - It should be a very easy task to switch the language string > > from > > "Intrusion Detection System" to the desired "Intrusion Prevention > > System". This also still needs to be done. I've spent a lot of time in the past days to rework some code and moving stuff to the "ids-functions.pl". The final goal was to get a nice code-base for the "snort to suricata" converter on which I'm currently working. I hope I will finish this very soon, so I can go on fixing the last minor issues and filed bugs. Best regards, -Stefan=20 >=20 > Cool. I am just saying that it needs to be done :) don=E2=80=99t forget the > logging section. >=20 > > > Apart from that I have to say that this is really really good > > > work, > > > Stefan. The main thing is: it works. I think this is a huge > > > improvement over what we had in the past and that this is > > > bringing > > > IPFire to the next level. It solves many problems that we didn=E2=80=99t > > > dare > > > to touch because snort was simply badly integrated into IPFire > > > and > > > did not really do much instead of wasting CPU cycles and memory. > > >=20 > > > I hope that we can continue working in this direction and maybe > > > add > > > some nice reports and so on and obviously carry over this work > > > into > > > IPFire 3 as soon as possible. > > >=20 > > > Please feel free to disagree with me above, but please do that > > > via > > > email if possible. > > >=20 > > > I want to target merging this into next as soon as possible in > > > January. I hope that we can get the things above sorted out over > > > the > > > holidays since it should not be much work. > > >=20 > > Best regards, > >=20 > > -Stefan > > > > On 17 Dec 2018, at 19:08, Stefan Schantl < > > > > stefan.schantl(a)ipfire.org > > > > > wrote: > > > > > What is the bypass mark for? At least that should be set to > > > > > nothing > > > > > instead of one, because that will again conflict with the > > > > > SNAT > > > > > fix > > > > > rule. > > > > >=20 > > > > > Best, > > > > > -Michael > > > > >=20 > > > > Hello Michael, > > > >=20 > > > > taken from the official suricata.yaml documentation: > > > >=20 > > > > bypass mark and mask can be used to implement NFQ bypass. If > > > > bypass > > > > mark is set then the NFQ bypass is activated. Suricata will set > > > > the > > > > bypass mark/mask on packet of a flow that need to be bypassed. > > > > The > > > > Nefilter ruleset has to directly accept all packets of a flow > > > > once > > > > a > > > > packet has been marked. > > >=20 > > > Weird. Why would you even pass this to the userspace process > > > then? > > >=20 > > > Whatever, we don=E2=80=99t need it then... > > >=20 > > > > In my commit, I've disabled the bypass options to exactly > > > > prevent > > > > from > > > > any issuse again with the SNAT packet mark. > > > >=20 > > > > Best regards, > > > >=20 > > > > -Stefan > > >=20 > > > Best, > > > -Michael > > >=20 > > > > > > On 17 Dec 2018, at 14:21, Stefan Schantl < > > > > > > stefan.schantl(a)ipfire.org > > > > > > > wrote: > > > > > >=20 > > > > > > Am Sonntag, den 16.12.2018, 21:28 +0100 schrieb Peter > > > > > > M=C3=BCller: > > > > > > > Hello Stefan, > > > > > > >=20 > > > > > > > to be a bit more precise about the NAT issue: > > > > > > >=20 > > > > > > > My setup is the IPFire Suricata test VM running in KVM, > > > > > > > with > > > > > > > two clients (Debian and OpenBSD) directly attached to it. > > > > > > >=20 > > > > > > > The Debian machine is located in GREEN, OpenBSD in > > > > > > > ORANGE. > > > > > > > RED interface is connected via bridge to my actual > > > > > > > testing > > > > > > > LAN; > > > > > > > for the first testing, any outgoing traffic to the > > > > > > > internet > > > > > > > was allowed (I will test upstream proxy behaviour later). > > > > > > >=20 > > > > > > > While GREEN was using IPv4 range 192.168.100.0/24, with > > > > > > > IPFire > > > > > > > as 192.168.100.1, enabling Suricata caused packets coming > > > > > > > from > > > > > > > GREEN not to be NATted anymore: Instead of using the > > > > > > > firewall's > > > > > > > RED IP for destination, it was the internal GREEN IP. > > > > > > >=20 > > > > > > > Let me know whether is is useful or not. :-) > > > > > > >=20 > > > > > > > Thanks, and best regards, > > > > > > > Peter M=C3=BCller > > > > > > >=20 > > > > > > > > Hello Stefan, > > > > > > > >=20 > > > > > > > > back again. :-) > > > > > > > >=20 > > > > > > > > The new IDS WebUI looks quite good so far - > > > > > > > > enabling/disabling > > > > > > > > Suricata works as well as selecting the rule source and > > > > > > > > the > > > > > > > > operation mode (IDS/IPS). > > > > > > > >=20 > > > > > > > > I was also able to download the "Snort/VRT Community" > > > > > > > > ruleset. > > > > > > > > Trying to switch to the "Emerging Threats" ruleset is > > > > > > > > possible, > > > > > > > > but downloading its ruleset afterwards is not: The GUI > > > > > > > > simply > > > > > > > > stalls, printing a message that "Snort (!) is > > > > > > > > performing a > > > > > > > > task". > > > > > > > >=20 > > > > > > > > The WebUI services page still shows IDS status for each > > > > > > > > interface, > > > > > > > > which does not seem to work anymore (everything is > > > > > > > > stopped, > > > > > > > > but > > > > > > > > Suricata was active on RED and GREEN). > > > > > > > >=20 > > > > > > > > Further, a client located in GREEN behind the test > > > > > > > > firewall > > > > > > > > instance is unable to browse the internet as soon as > > > > > > > > Suricata > > > > > > > > is > > > > > > > > enabled. If disabled, downloading ET rulesets work as > > > > > > > > well > > > > > > > > as > > > > > > > > internet traffic. At the moment, I am flying blind > > > > > > > > here, > > > > > > > > but it > > > > > > > > looks > > > > > > > > like packets are not NATted anymore if Suricata is > > > > > > > > active. > > > > > > > >=20 > > > > > > > > Any outgoing connection is in state "SYN_SENT" if > > > > > > > > Suricata > > > > > > > > is > > > > > > > > active. > > > > > > > >=20 > > > > > > > > A portscan against the firewall (GREEN interface) is > > > > > > > > not > > > > > > > > detected, > > > > > > > > even though ET SCAN ruleset is enabled (used nmap with > > > > > > > > NSE > > > > > > > > active). > > > > > > > >=20 > > > > > > > > Especially the outgoing connection/NAT/? issue > > > > > > > > mentioned > > > > > > > > above > > > > > > > > breaks things in my scenario. Anything else are minor > > > > > > > > issues > > > > > > > > (of > > > > > > > > course, a portscan should be detected, this needs > > > > > > > > further > > > > > > > > investigation > > > > > > > > indeed). WebUI works fine so far. > > > > > > > >=20 > > > > > > > > Thanks again for your work; I hope the feedback can > > > > > > > > appreciate > > > > > > > > it > > > > > > > > somehow. :-) > > > > > > > >=20 > > > > > > > > Let me know if there are questions. > > > > > > > >=20 > > > > > > > > Thanks, and best regards, > > > > > > > > Peter M=C3=BCller > > > > > > > >=20 > > > > > > Hello Peter, > > > > > >=20 > > > > > > thanks for testing and your feedback. > > > > > >=20 > > > > > > I've setup a test environment and was able to re-produce > > > > > > your > > > > > > NAT > > > > > > issue > > > > > > very easy. After some research and help of Michael, we were > > > > > > able to > > > > > > find the real issue. > > > > > >=20 > > > > > > It was located in a identical mark on the packets which > > > > > > already > > > > > > have > > > > > > been scanned by suricata and packets which should be > > > > > > modified > > > > > > for > > > > > > SNAT > > > > > > in the "Mangle" table of the routing logic by the kernel. > > > > > >=20 > > > > > > The changes can be found in my git repository: > > > > > >=20 > > > > > > https://git.ipfire.org/?p=3Dpeople/stevee/ipfire-2.x.git;a=3Dcomm= it;h=3Df5ad510e3c0f416a1507999f5ad20ab171df9c07 > > > > > >=20 > > > > > > I'll upload a fixed image very soon - please keep on > > > > > > testing. > > > > > >=20 > > > > > > Best regards, > > > > > >=20 > > > > > > -Stefan --===============3075145354736319752== Content-Type: application/pgp-signature Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="signature.asc" MIME-Version: 1.0 LS0tLS1CRUdJTiBQR1AgU0lHTkFUVVJFLS0tLS0KCmlRSXpCQUFCQ2dBZEZpRUVXTzBOWHRTcnZo YXN5dERuVHRkT0ZZK1RzdDRGQWx3aWZ3MEFDZ2tRVHRkT0ZZK1QKc3Q1UFRoQUF4eFhuSDFTTWVs d3dIU1BzSnZBZmpnZkZ3bmEzNmJLbHU5bkhzMDZUaGo5ZmJSYWZ4VGN4S0NUQQpGVTJrUFk3THg3 c2dLa05mVkY2U3RYa2I0YTVkT0dmZVhiS2ZxdWZjUWsxUExzbi9qQ2NyTUxCQkpWcEwwekswCkYy UVBmeHpVMDJ4ektVek95YjVHdXpnbEpPOUV0T1h2NVNMRTdwTGJ1bUdJRjU3TTAraFltcHM4UGlu U05lU20KTHhHQWl0Y3EvdW0zcWhBallLL2l6cTFyOUd6N3dJbkVSQ2FhZGw1bit4ZTJBc3ZBSzFj SXovRy9oUVdRNEtXVAp0QjZ6ZlVELzBMS2ZqN2FqY0ZiZ1VYa3F5a2l3Y0xNOWhaMnlTcHBQU0JZ SStoZFRJeHJxdW91RC9ENnlnVjVBCjN0KzNMdFdPYmJmUUlNMWNNNUZCdWZlQ1BTbWdtNXRKNG9U Rjdtam9JT205Wmh3elo1SEQzTC8yUEZtakg2c3cKNEd1NjErZGVHVUxZRUliSG53cEtYR0lMUDRN S3ZnM3RreGVUNTJKeEZKWEFCYThrWlNEcmsrWGtDdGF0c1ZxUgpMSjd3bFpxdnNYbkZERlFKa2Nh ZDhCN3MyckdqVHp4OHh3RHpuRUJDcUtHQ09hczA4YUdFeVZJMlpCRUJPWkg3ClFpc3hnc241YVg3 K25pWmVnSzJWMTIwNU1vT0svU2ozUXQrY3VYK3VYUTJIV1dOa1pzQW1DVzJBT2pFUFBWNTAKVXJC TTc5NGR3ZmNOVDgwcmg5WGROUVgvdEt4ODhBTXBvd0lJbE1Ra0RoNHpDTk5mT29ZaldwM0E4Tm84 Q0dFSQpkLzlGb05mMFRKVEJrVVBCeEhBbkQ1QW9vanp0by90N1NURE00bjl2bmo2eWpsanhUWG89 Cj0zY0c2Ci0tLS0tRU5EIFBHUCBTSUdOQVRVUkUtLS0tLQo= --===============3075145354736319752==--