From mboxrd@z Thu Jan 1 00:00:00 1970 From: Peter =?utf-8?q?M=C3=BCller?= To: development@lists.ipfire.org Subject: Re: [PATCH 0/3] Add ASN-based anomaly detections to IPFire's web proxy: Proactive Fast Flux detection and detection for selectively announced networks Date: Mon, 06 Sep 2021 18:35:49 +0200 Message-ID: In-Reply-To: <18ce6cea-a141-c91e-61ca-8fd1b9c4ab01@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6014001928667426769==" List-Id: --===============6014001928667426769== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello *, by accident, I just stumbled across a false positive related to the Fast Flux= detection: > [root(a)maverick ~]# su squid -s /bin/bash > bash-5.1$ /usr/bin/asnbl-helper.py /var/ipfire/proxy/asnbl-helper.conf > Sep 06 18:28:21 squid-asnbl-helper[9945] WARN: No ASNBL configured. This is= acceptable as long as this script is configured to do anything, you just hav= e been warned... > Sep 06 18:28:21 squid-asnbl-helper[9945] INFO: Configuation sanity tests pa= ssed, good, processing... > Sep 06 18:28:21 squid-asnbl-helper[9945] INFO: Successfully loaded location= database from /var/lib/location/database.db generated 'Mon Sep 6 05:52:56 2= 021' (UTC/GMT) by 'IPFire Project' - good > Sep 06 18:28:21 squid-asnbl-helper[9945] INFO: Running ASN database respons= e tests... > Sep 06 18:28:21 squid-asnbl-helper[9945] INFO: ASN database operational - e= xcellent. Waiting for input... > fedoraproject.org > Sep 06 18:28:26 squid-asnbl-helper[9945] WARN: Destination 'fedoraproject.o= rg' exceeds ASN diversity threshold (9 > 5), possibly Fast Flux: [81, 3701, 1= 5456, 16509, 21785, 22753, 36850, 54455, 61317] > Sep 06 18:28:26 squid-asnbl-helper[9945] INFO: Denying access to possible F= ast Flux destination 'fedoraproject.org' > OK Apparently, the Fedora folks think it is a good idea to use round-robin for l= oad balancing: > $ dig +short a fedoraproject.org > 140.211.169.206 > 67.219.144.68 > 85.236.55.6 > 38.145.60.20 > 152.19.134.198 > 209.132.190.2 > 18.133.140.134 > 18.185.136.17 > 185.141.165.254 > 152.19.134.142 > 38.145.60.21 > 18.159.254.57 At the first glance, using the URL filter (by adding fedoraproject.org to the= list of always allowed domains) seems to be a straight-forward solution to this problem. However, it= does not work, as the ASNBL script is executed in the context of an ACL, while the URL filter comes= as a redirect/wrapper. Therefore, it is never reached if a "deny" ACL matches in the first place. This is the only false positive I observed so far. Unfortunately, it is a rat= her bad one. :-/ Any thoughts on what to do now? Thanks, and best regards, Peter M=C3=BCller > Hello Michael, >=20 > thank you for your reply. >=20 >> Hello Peter, >> >> I love this feature. I think it is a one-of-a-kind thing and hopefully man= y more people will think the same. >=20 > Yes, I like the idea, too. Sometimes, security can be simple _and_ effectiv= e... :-) >=20 >> However, it will need a lot of documentation and explaining. >=20 > Indeed. I was thinking about a blog post for it; we probably need to explai= n Fast Flux in the > first place, and I am not sure if all of our users are aware of the existen= ce of autonomous > systems. >=20 >> I have a couple of high-level questions: >> >> * Does it make sense to give the user the choice for the threshold? >> >> It seems to be a difficult question because it requires exact knowledge wh= at this feature actually does. My fears are that people just set this to some= thing like =E2=80=9C9=E2=80=9D and the feature would become ineffective. What= use-case is there to change this? >=20 > One size never fits all, I guess. >=20 > Indeed, the range of useful threshold values is pretty small: Anything belo= w 4 causes _way_ too > much false positives in productive environment, whereas even 7 appears to b= e too ineffective. >=20 > At the moment, the CGI catches values the ASNBL helper would treat itself a= s being invalid. Do > you think narrowing down this range to 4 to 7 makes sense? Or should we rep= lace it by a dropdown > for adjusting sensitivity? >=20 > Either way, it is a good idea to tell users to leave the default where it i= s unless they truly > understand what they are doing. >=20 >> * Selective announcements: Should this necessarily live in the proxy? Why = do we not generate a filter for the firewall? >=20 > We can do so as well, and I would love to see such a feature landing in IPF= ire. >=20 > Given our current state of libloc, I doubt this is possible: We would need = a function that returns > all networks we do not have an AS for - to my knowledge, the libloc (bindin= gs) do not support this > at the moment. >=20 > Apart from that: On a packet filter level, we lack the FQDN of a destinatio= n, which might be useful > to have for debugging or forensic reasons. >=20 > Also, the users will experience a timeout after n seconds. Having selective= announcement detection > turned on, they'll get their error message straight away. I was told this i= mproves UX... :-) >=20 > Thanks, and best regards, > Peter M=C3=BCller >=20 >> >> -Michael >> >>> On 18 Jun 2021, at 18:24, Peter M=C3=BCller = wrote: >>> >>> This patchset adds two new features to IPFire's web proxy, taking advanta= ge >>> of the Autonomous System information we have at hand by using libloc. >>> >>> The proactive Fast Flux detection is especially worth noticing, as even m= ost >>> expensive (=3D advanced?) security suites do not provide similar protecti= on, >>> especially not in a proactive manner. >>> >>> By simply enumerating the distinct amount of Autonomous System Numbers a = FQDN >>> ultimately resolves to, we are able to deny access to malware distribution >>> sites, phishing sites, C&C servers, and other cybercrime stuff hosted on = Fast >>> Flux setups abusing cracked machines around the world - even before the F= QDN >>> or any IP address involved is flagged as malicious by any security vendor. >>> >>> Peter M=C3=BCller (3): >>> squid-asnbl: New package >>> proxy.cgi: Implement proactive Fast Flux detection and detection for >>> selectively announced destinations >>> langs: Add English and German translations for newly added web proxy >>> features >>> >>> config/rootfiles/common/squid-asnbl | 1 + >>> html/cgi-bin/proxy.cgi | 89 +++++++++++++++++++++++++++++ >>> langs/de/cgi-bin/de.pl | 7 +++ >>> langs/en/cgi-bin/en.pl | 7 +++ >>> lfs/squid-asnbl | 83 +++++++++++++++++++++++++++ >>> make.sh | 1 + >>> 6 files changed, 188 insertions(+) >>> create mode 100644 config/rootfiles/common/squid-asnbl >>> create mode 100644 lfs/squid-asnbl >>> >>> --=20 >>> 2.26.2 >> --===============6014001928667426769==--