From mboxrd@z Thu Jan 1 00:00:00 1970 From: Stefan Schantl To: development@lists.ipfire.org Subject: Re: Guardian 2.0 Testversion 011 Date: Thu, 05 Mar 2015 19:37:08 +0100 Message-ID: <1425580628.8345.35.camel@ipfire.org> In-Reply-To: <934634537.3431881425296755846.JavaMail.root@ip-10-41-150-66.ec2.internal> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2401278231416593722==" List-Id: --===============2401278231416593722== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello H&M, sorry for my late response, the last few days have been very stressful for me. > Hi Stefan, >=20 > I do hope this time my e-mail reaches you... >=20 > I've accumulated some experience with Guardian, but my main experience is w= ith Snort, snort used in different SIEMs. >=20 > Here are my recomandations for IPS/Guardian Thanks for testing and providing your feedback, I really don't know why your messages are not accepted by the mailing list. I've send a copy of my answer, via CC to the development mailing list. >=20 > Recommendation 1: > Please add to the ignore list the ability to add a comment to that IP being= ignored. Reason: I do want to know why I've ignored all Snort alerts from th= at IP. (instead of doing a nice snort tuning and modify threshold.conf from s= nort for a specific list of SIDs). >=20 Accepted, I will schedule this for guardian 2.1. > Note: for the record : ignoring an IP in Guardian does NOT reduce the load = on CPU. Snort will continue to trigger alerts and log them. If IPFire is desi= gned to work for small resource systems, I do recommend to move the ignore ru= le at snort level. I can provide snort logic for accomplishing that.=20 >=20 Sadly this is not possible, because the brute-force detection mechanisms are not provided by snort. > Recommendation 2: > As a best practice before unblocking an IP (an offender) I do following ste= ps that require additional data (data that will be nice to be provided by IPF= ire WUI / guardian-2.0-010 somehow). I am talking about information related t= o the IP that I want to unblock: > 1. What was the reason in the first place for IP being blocked. Recommendat= ion: please add back to guardian.log as much as possible from the information= generated in snort/alert file. Currently (guardian-2.0-010) the message in g= uardian.log is not helping in that direction >=20 > Current guardian-2.0-010 guardian.log message:=20 >=20 > Wed Feb 18 22:46:17 2015: Blocking 222.186.31.208: An active snort rule has= matched and gained an alert. > Wed Feb 18 23:18:15 2015: Blocking 123.249.24.160: An active snort rule has= matched and gained an alert. >=20 > Old Guardian (based on perl) logging style was much better, please use the = old logging style if possible: >=20 > Wed Feb 18 22:15:34 2015: 123.249.24.160 [1:2403344:1577] ET CINS Ac= tive Threat Intelligence Poor Reputation IP TCP group 23 > Wed Feb 18 22:18:55 2015: 218.77.79.43 [1:2402000:3604] ET DROP Dshield Bl= ock Listed Source group 1 > Wed Feb 18 22:23:32 2015: expiring block of 141.212.121.203 > Wed Feb 18 22:26:31 2015: 61.240.144.66 [1:2402000:3604] ET DROP Dshield Bl= ock Listed Source group 1 Accepted, I will include this really important suggestion into the next testing version. >=20 > 2. If the reason is is a Policy EXE or DLL download this means that probabl= y one of my laptops/desktops got spearfished and a malware is starting to bri= ng additional files to the victim. Or, is just me downloading the new version= of AdobleFlash. No matter the case, before unblocking the IP, I do check if = the IP blocked by Guardian =E2=80=9Ekind of=E2=80=9D safe. For that I do use = / rely on following tools, tools that can be easily added to the Guardian sta= tus window as links. Here are the links format that can be added near each bl= ocked IP: >=20 > a. Virustotal. The format of the link is : https://www.virustotal.com/en/ip= -address/a.b.c.d/information/ where a.b.c.d is the IP from GUARDIAN iptables= chain. Example: https://www.virustotal.com/en/ip-address/61.240.144.66/infor= mation/ . Yes, even the mighty Akamai is listed as distributing malware.=20 > b. IPVOID (collection of RBL databases =E2=80=93 most important one are rel= ated to TROJAN monitoring =E2=80=93 like ZeusTracker). Format of the link is = http://www.ipvoid.com/scan/a.b.c.d/ where a.b.c.d is the IP from GUARDIAN ipt= ables chain. Example: http://www.ipvoid.com/scan/61.240.144.66/=20 >=20 > 3. Another awesome tool (but also heavy one!) I use before unblocking an of= fender is the DNS blacklist aggregator from http://multirbl.valli.org/. This = tools checks the IP in a whopping 228 RBL databases! Yes, lots of false posit= ives=E2=80=A6 but when you see the IP in more than 50% of the RBL databases y= ou start to reconsider unblocking it. The format of the link is http://multir= bl.valli.org/lookup/a.b.c.d.html where a.b.c.d is the IP from GUARDIAN iptabl= es chain..Example http://multirbl.valli.org/lookup/61.240.144.66.html=20 > 4. Another tool I use to check for Phishing sites (IPs) is http://blog.gman= e.org/gmane.comp.security.phishings . Here is one example of a site that ste= als PayPals credentials, site that is also listed in VirusTotal but is not cl= ear what the site is doing (the phishing part): Quaery: http://search.gmane.o= rg/?query=3D218.208.237.174&group=3Dgmane.comp.security.phishings=20 >=20 > From my experience, allowing a user to unblock/flush the GUARDIAN is bad be= cause people do not posses so much knowledge about the reason embedded in the= ET rules, rules that were used to trigger alerts and therefore Guardian bloc= k action. > Giving to the user links to sites like VirusTotal & IPVoid will probably ma= ke them wonder themselves why the links are there (in the WUI) and use them b= efore clearing an IP from the offenders list.=20 > Second: the links will help them to better understand the ET Rule and why i= n the first place snort triggered an alert. >=20 > Another use case for the links for above sites, but this times for the IPs = listed in Status - Connection: if a user sees tons of traffic to suspect site= s it is good to be able to check the IPs to a collection of public available = tools. I saw in last few months articles about legit sites being injected wit= h redirect codes =E2=80=93 see Forbes being attacked last year. Such attacks = takes place daily - and unfortunately are successful. An advanced used will t= ake a peak to the active connection and will check why his machines are openi= ng so many connections and will try to see if some of them are legit (using a= ny of the available sites listed above) =20 >=20 > Bottom line: if the IP that a user wants to unblock is listed on one of the= above sites as distributing malware/Trojan, doing phishing, or having +2000= 0 Domains attached to it (yes, I=E2=80=99ve saw one with more than 20000 doma= in attached to the IP!), maybe they will leave that IP blocked. I've seen your feature request on the bugtracker and your posts in the forum thread. I thinks your suggestion would be a nice and handy feature, but I don't agree in putting the icons and links into the "Currently blocked Hosts" section inside the "guardian.cgi".=20 I'm considering in extending the "ipinfo.cgi" and locating the icons + links to the external sources on that page. The "ipinfo.cgi" is linked from various logging pages, so this would offers the ability to use those information services also for firewall hits, snort alerts etc. >=20 >=20 > Recommendation 3: >=20 > As I wrote in the forum - please allow a user to unblock an IP for a limite= d period of time. The old guardian did that. > The reason is this: I do prefer a limited hole in my security rather than a= permanent one. > Here is the use case: all of us does downloads/out machines does tons of do= wnloads (Adobe- so many, Antivirus, Windows updates, XUbuntu updates, etc).=20 > Snort will block them for various reasons: there are so mane RT rules for t= hat. > Me, as user, I do need that download, so I will be forced to add the IP to = the ignore list.=20 >=20 > This is so bad: for a single time download (one update!) I've created a per= manent hole in my IPS. And this holeis most of the time toward an IP that is = a local cache (local mirror)! And the local cache provider does reutilize tha= t Ip for so many domains: I've checked that! Many domains being dynamic, or l= isted in different blacklists. But if I've ignored all alerts for that IP...I= am somehow working against my IPS.=20 >=20 > And, as I said above, the domains behind one IP can be thousands, IPs are r= eutilized (See Dynamic cloud servers services - like Amazon, 1&1, or even the= provider IpFire Uses) - all these providers reutilize IPs for so many sites.= ..=20 >=20 > Bottom line: a permanent hole is totally against IPS concept in my opinion.= I rather allow the traffic for a limited time, but after that IPS should bl= ock any further attempts for doing traffic with that IP if snort triggers ale= rts for IT. >=20 >=20 Accepted, I will add an option for this in guardian 2.1. > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D > Errors - how to replicate them >=20 > I=E2=80=99ve managed to replicate Blago=E2=80=99s error like this: create 2= guardian processes in memory. If this is happening, I get same error message: >=20 > [root(a)dmx ~]#/etc/init.d/guardian status > guardian is running with Process ID(s) 2928 2299. > [root(a)dmx ~]# /etc/init.d/guardian stop > /etc/init.d/guardian: line 33: [: 2928: binary operator expected >=20 > Here is a solution to have 2 guardian processes in memory: >=20 > [root(a)dmx ~]#guardian status >=20 >=20 > There is an executable file named guardian besides the /etc/init.d/script. = Making a call to that generates 2 guardian processes in memory. This has noth= ing to do with an old guardian version on IPFire. Is the same guardian... >=20 > Here are the 2 processes: >=20 > [root(a)dmx ~]#ps aux |grep guard > root 2299 0.9 0.0 9296 3216 ? S 20:31 0:01 /usr/bin/p= erl /usr/bin/guardian > root 2928 0.0 0.0 10864 3328 pts/0 S 20:33 0:00 /usr/bin/p= erl /usr/bin/guardian status >=20 > After this, /etc/init.d/guardian fails to stop the guardian processes. > Other impact observed: the WUI works OK and display as PID number the one f= rom the line with =E2=80=9Cstatus=E2=80=9D (=3D 2928), but Currently blocked = hosts list is empty. > [root(a)dmx ~]# cat /run/guardian.pid > 2928 >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > Note: updating to 2.17 core 87 somehow put down my guardian. > I need some time to rebuild it - the Web interface looks funny - lot of mis= sing elements. Including the Guardian menu! (but the cgi is still on the disk= and can be accessed manually). >=20 >=20 > Best, > H&M Best regards, -Stefan >=20 > -----Original Message----- > From: Stefan Schantl [Masked] [mailto:FWD_eRROOjHG(a)opayq.com]=20 > Sent: 28 februarie 2015 15:51 > To: 64f853bf(a)opayq.com > Subject: Guardian 2.0 Testversion 011 >=20 > Hello followers, >=20 > at first I have to thanks to all of you which have joined the guardian > 2.0 testing team. I've got a lot of positive feedback but also has been not= iced about several tiny issues. >=20 > I've uploaded an updated test version to http://people.ipfire.org/~stevee/g= uardian-2.0/. >=20 > The installation / update works in the same way as described in the planet = post: > http://planet.ipfire.org/post/introducing-guardian-2-0-for-ipfire >=20 > IMPORTANT: Please create a backup of your files stored in "/var/ipfire/guar= dian/", otherwise the will be overwritten by the update! >=20 > Changelog: >=20 > * Matthias Fischer detected some small problems in the "guardian.cgi" > and provided some patches for them. > http://git.ipfire.org/?p=3Dpeople/stevee/ipfire-2.x.git;a=3Dcommit;h=3D19d6= abcce57be35c3bd43ebf45e37d69776f081e http://git.ipfire.org/?p=3Dpeople/stevee= /ipfire-2.x.git;a=3Dcommit;h=3D0a6c3cb89642e2ff567993d810757425cf9ccce7 http:= //git.ipfire.org/?p=3Dpeople/stevee/ipfire-2.x.git;a=3Dcommit;h=3D82208c83c70= 9da1f1c24fd9396e5d351f833fd91 >=20 > * Blago Culjak informed me about problems on starting guardian and displayi= ng it's status in the web interface. This behaviour only happens when the leg= acy version of guardian has been un-installed and not just replaced by the gu= ardian. In the updated version contains the required file which is used by va= rious IPFire scripts to detect if an addon is installed or not. >=20 > * The CLI switch "-d" called "debug mode" has been replaced by "-f" to laun= ch guardian and run it in the foreground. > http://git.ipfire.org/?p=3Dpeople/stevee/ipfire-2.x.git;a=3Dcommit;h=3Da58b= d674863e1c4fd3cff457f1bd51e105c3eb2b >=20 > * Some new code has been added to prevent from starting multiple instances = of guardian. > http://git.ipfire.org/?p=3Dpeople/stevee/ipfire-2.x.git;a=3Dcommit;h=3D9d44= c0d9952e67f6afad15e2940a5be6f1fe9094 >=20 > Best regards, >=20 > -Stefan >=20 >=20 >=20 >=20 --===============2401278231416593722== Content-Type: application/pgp-signature Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="signature.asc" MIME-Version: 1.0 LS0tLS1CRUdJTiBQR1AgU0lHTkFUVVJFLS0tLS0KVmVyc2lvbjogR251UEcgdjIKCmlRSWNCQUFC Q2dBR0JRSlUrS0pVQUFvSkVFN1hUaFdQazdMZTc5OFFBSTVoVmFpSk42dzgzcExJQkR1Z1NQcjUK RDNaUjUrNnJERi94OFhKTjFUblFnRlpxQ0sxb3JrNmtHYnFKeHEvV3N0TndyTkhiWFNZYmNOYlJl ZWdveW5ubwo5djkvY0pyNHQwbk44ZDlSVG9vNzlXSS90aWtvVTlwT05CQzJuQ0h0V3MvZ3NTam42 aUNqUjM1SDBXdkxYaFBXCjZUeS9WekdaMHpydTVLUEp2U1psalZLMXV0VXhvbkpKQXFyU01HKzJ6 RnFDWFMwSmhiR0Vkbk1reW5Edlp3WHQKNVVuYzA1NlBYb2JXL2dFSXJGaFdqdFMxSlNsUUVPZENK QzVEWGR2andkVW9ZdGpoQ2FZNzJIaXR5MkJOdWVucAphUGttU0RKbTBPRUV3WUJTRjRzWUE0ZC9I VjRBM3hFNmlFOUtiNXlPU1RIeWcxUXJGY3hMMFVRUVpBRmhWYmUyCmo5Q05JdHI1ZUYzNUl5MDFy cTFEQjdRWVNIZGZqYVBWV2hGa09mNzQ4eVhoT2wrV2tGQng4Slc3VnVGWnp4SFQKU3dhVzByMDhk WGRRZ2hVVWJqUm5oS0J6NHBDcnpFa0VPZUgyWlNvQTlsS053Nk5hSSt1M1l3cnVFbk9DYkprZgpC SkFGNFVOVm4zQ3hYdWVzS0JnVFpiSk9NMkJKVGg5M3ExUTFXT1BFb2d4RDBaamNQNll6N1g2WU9i dzB2S2FhClF0VjVtU0RXWjZKLzdOUVJSR3RJaUNSWGJFV0JLeWd1RVEwc2FGNDFpL3lxMTJmWTJH VlZ1blBuamlpM1VKSEcKb2s4b1RqeEx5QitJMnNwSERWcmV0UEhtcEp6TEtaR3Y1c1M1YmxZa0RL SlVoWVFTbm9FRVZrbk93WklQckw1agpxSkhVNmViak5Wb2diSUhPWS8zOQo9aE0vZgotLS0tLUVO RCBQR1AgU0lHTkFUVVJFLS0tLS0K --===============2401278231416593722==--