Hello,
It's my turn to apologise for being slow to respond - I've had a busy week, but I should have plenty of time over the next couple of weeks.
I've made most of the comments inline, however I think Michael had a question (which I can't find now) about what happens if someone enables all the lists. One thing which would perhaps make this less likely is that the WUI tags the available lists with whether they're safe or not, with a footnote that safe means that the list only blocks malicious traffic. This won't guarantee that a user won't still try to enable all the lists, but it should make them realise that they should think first.
I have considered replacing this tag with a risk high/medium/low and maybe adding a category (invalid/application/scanner/C&C or something like that), but that may provide too much information and dissuade them from actually following the links to checkout what the list actually does.
Tim
On 05/12/2019 22:25, Michael Tremer wrote:
Hello,
On 4 Dec 2019, at 17:05, Peter Müller peter.mueller@ipfire.org wrote:
Hello Tim, hello Michael,
please see my responses inline...
We could periodically update the blacklists on our main mirror (and wait for the network to sync it), make sure it is signed and write a small downloader that fetches, validates and installs them.
@All: Thoughts on this?
I think there are a number of points here.
Firstly, from the point of a third party using IPFire, is this really solving the privacy disclosure problem? There's no way round disclosing your public IP address to someone you're downloading from; all this does is change who that information is being disclosed to. For the user there's no way of knowing whether the source is more or less protective of the user's privacy than the blacklist provider. Indeed it won't be possible to know who the lists are being downloaded from until the download starts.
There is a way: Tor. But that is a totally different story.
Well, I see a third option on this: Use the mirror infrastructure we already have. Every IPFire installation discloses its public IP address to one of these servers sooner or later, so we do not disclose additional data if the blacklists were fetched from these.
Needless to say, Tor (hidden services) would be better, but that is a different story indeed. :-)
The point is rather that a forget list can be sent instead of the “real” one.
I did not get this. Forget? Forged?? ???
Yes, I meant to write forged, but auto-correct didn’t let me.
Secondly, latency; some of the lists are updated every 5 minutes. While I've limited the maximum check rate to hourly, will the updates propagate quickly enough. For reference on my main system the 24 updates on the CIARMY list made 143 498 changes (additions or deletions). I've seem it do over 200 000.
Yes, I observe that behaviour for CINS/CIArmy too. Unfortunately, they do not document a recommended update interval anywhere, so we can only guess.
Personally, more static lists seem to be preferable for packet filtering. Highly dynamic ones such as CIArmy should be done via DNSBL queries or something similar
- do we really want to have that list here?
It is not really an option to implement a DNSBL into a packet filter, but I get your point.
One of the 'selling points' for an IP address blacklist is that it can respond quickly to new threats - or rather new attackers. While a new IDS/IPS rule needs time to analyse the threat, generate a rule and check it, it's easy to add an address to a list. So, I think the CIArmy list is potentially useful for protecting home systems etc. with budget hardware, but I would be very careful about using it for a protecting a general access website.
How did you come up with the hour? Will it be retried more often if the download was not successful?
One hour is the most common interval indeed, but adding some random time might be useful in order to reduce load on the servers providing a blacklist.
Yes, definitely. Otherwise we will shoot down our mirrors.
When I implemented that section of code, specifying the minimum check period in hours seemed to provide a convenient way of allowing a check period covering a wide range, with an hour as the fastest and a week as the slowest. I didn't looked at the CIArmy list until much later. Most of the lists don't change nearly as much, but the CIArmy list is described as one that deliberately responds quickly.
From my production system, for yesterday:
The following block lists were updated: BLOCKLIST_DE: 24 Time(s) - 9341 change(s) BOGON_FULL: 1 Time(s) - 10 change(s) CIARMY: 24 Time(s) - 159134 change(s) DSHIELD: 7 Time(s) - 18 change(s) EMERGING_FWRULE: 1 Time(s) - 50 change(s) FEODO_AGGRESIVE: 24 Time(s) - 13 change(s) SHODAN: 1 Time(s) - 0 change(s) SPAMHAUS_DROP: 1 Time(s) - 0 change(s) TOR_EXIT: 24 Time(s) - 162 change(s)
and my test system:
The following block lists were updated: ALIENVAULT: 19 Time(s) - 5331 change(s) EMERGING_COMPROMISED: 1 Time(s) - 26 change(s) TALOS_MALICIOUS: 1 Time(s) - 36 change(s)
That covers most of the lists. From the WUI, since 1 Dec:
Blacklist Entries pkts bytes Last updated in in AUTOBLACKLIST 0 731 51144 Sun Dec 8 18:40:02 2019 BLOCKLIST_DE 28020 857 46735 Sun Dec 8 18:40:02 2019 BOGON_FULL 214 5255 189K Sun Dec 8 16:50:04 2019 CIARMY 15000 19774 976K Sun Dec 8 18:04:01 2019 DSHIELD 20 7992 321K Sun Dec 8 16:45:13 2019 EMERGING_FWRULE 1647 197 8383 Fri Dec 6 05:29:07 2019 FEODO_AGGRESIVE 7169 0 0 Sun Dec 8 18:50:07 2019 SHODAN 32 34 1530 Sun Dec 8 17:54:09 2019 SPAMHAUS_DROP 823 0 0 Fri Dec 6 18:22:35 2019 SPAMHAUS_EDROP 111 82 16433 Thu Dec 5 17:27:09 2019 TOR_EXIT 1055 0 0 Sun Dec 8 18:31:02 2019
(I've left out the pkts/bytes out fields which were all 0)
Note that where possible I do a HEAD request first and then only download the list if the modification time has changed since the last check. For dynamically generated lists this isn't possible.
If the download isn't successful it just gives up and waits for the next attempt (apart from the usual retries in the library). I probably should to change that so that it only applies the per list minimum update period in this case (specified in the sources file) rather than the user specified value as well.
I already use a time offset on the downloads - when it's started from boot, backup restore or WUI enable, it checks to see if it's installed in the fcrontab, and if not adds itself at a randomly generated offset in the hour.
Third, bandwidth; while the downloads are fairly small (ALIENVAULT is the largest at a few MB), there are going to be a lot of them. How will this affect the willingness of people to mirror IPFire?
I do not consider this being a problem as we do not generate that much traffic to them. Of course, that depends on the update interval again.
That depends on your point of view.
I do not have a problem with this at all in my data center, but there are plenty of people with a volume-based LTE plan or simply a 128 kBit/s connection. It will take a longer time to download the lists for them. We need to mind that.
Talking about the preference of packet filter and IPS, I prefer to use the latter as well as it gains more insight in what kind of malicious traffic tried to pass a firewall machine. On systems with low resources, this might be problematic and removing load from the IPS can be preferred (make this configurable?!), on others, people might want to have both results.
You're only going to get one result for a packet whichever way round the IP blacklist and IPS are since whichever comes first will drop the packet before it reaches the second (well it would be possible to put the IP blacklist first and get it to log and mark packets which are then dropped after the IPS, but I think that's getting a little complicated. In addition I've seen the messages about the trouble marking was causing in the QoS).
I think it's a 50/50 choice as to which is more valuable first; it's probably going to differ from packet to packet. For me the possibility of reducing the IPS load means I prefer putting the IP blacklist first
It should be fairly easy to add the choice of where to put the IP blacklist. I think it'll have to be in the main firewall script, so it'll require a firewall restart, but it's not something that'll be changed often.
I do not think that the user should choose this. If we cannot easily make a decision, how can our users do this? Not saying they are stupid here, we are just giving them something so that they do not have to put the thought and research into things themselves and make their jobs easier.
Agreed.
I think performance matters. And if the IPS comes first, the most likely case would be that we are seeing a SYN packet that is being scanned and after that being dropped by the blacklist. It was pointless to even scan the empty packet (TCP fast open aside). This is only different for other packets with payloads.
We would protect the IPS from a SYN flooding attack here at least and from scanning more packets unnecessarily.
So dropping packets from blacklisted IP addresses/networks before IPS is it, then.
I do not even think it makes sense to swap the order in the outgoing direction.
Me too.
What IPFire is lacking is a statistical analysis for those logs. Collecting more and more data isn’t helpful from my point of view. Only if you are looking at a very specific thing.This is true, but I am not sure if it makes sense to spend too much work on this.
Based on my personal experience, firewall hits observed on a single machine exposed to the internet are interesting, but the overall situation across multiple machines is even more interesting. Very quickly, you'll end on something like a centralised logging server and custom statistical analysis here...
Probably a project for IPFire 4.0 :)
Or use one of the existing services, like the DSHIELD client https://dshield.org/howto.html (subject to privacy concerns again).
Personally, I consider Spamhaus DROP/EDROP can be safely enabled by default. I would love to see the bogon ruleset here, too (think about 8chan successor hosted at unallocated RIPE space in Saint Petersburg), but that will likely cause interference if RED does not have a public IP address assigned.
I can add a field to the options file that controls whether a list is enabled by default.
Thank you. :-)
To stress the point from above again: We would then share all public IP addresses of all IPFire systems in the world with Spamhaus and who is hosting their infrastructure. That can be considered a threat.
This is my only objection against this patchset. Now, what can we do about it? One possibility is to apply the patchset now and implement a custom download source thing later on, or do that before releasing Core Update 139 (or which version the patchset will be to) after we agreed on something.
I do not see this being merged for 139. But that is not important. We need to get it right first and then release it.
As far as I know, nobody has tested this, yet.
There are a number of people who have been running an earlier version which I shared on GitHub. There were a few early issues, but it seems to be OK now.
https://forum.ipfire.org/viewtopic.php?f=27&t=21845
This version wasn't integrated into IPFire, so (for example) it inserted itself into the INPUT IPTables chain rather than having it's chains created as part of the firewall start-up script.
I have huge concerns about the automatic blacklist. @Peter: What is your opinion on this?
While I implemented it, I'm aware of its potential to cause problems, which is why it has to be separately enabled. It's not caused me any issues at the default settings (blocks at over 10 packets per hour until 1 hour has passed without seeing packets from the address), but I've not used it on a site with publicly announced services. If I was going to use it on a web site I would want to, at the very minimum, drop the block period drastically.
On the other hand, it's good at responding quickly. Usually I see only 1-2% of blocks from the automatic list:
Reason Count % First Last CIARMY 2416 45 Dec 6 00:00 Dec 6 23:59 DSHIELD 1353 25 Dec 6 00:00 Dec 6 23:59 INPUT 1294 24 Dec 6 00:00 Dec 6 23:59 AUTOBLACKLIST 122 2 Dec 6 00:20 Dec 6 16:28 BLOCKLIST_DE 89 2 Dec 6 00:20 Dec 6 23:46
and sometimes none at all, but one one occasion it blocked over 8000 packets. Again I'm aware this is for a home system, which is rather different than from a Web server.
If we do, I will have a look at the licensing stuff (DShield and Spamhaus do not seem to be problematic, as they are hosted on 3rd party servers, too).
One of them will be, sooner or later. And one is enough I suppose.
DShield (https://dshield.org/api/#threatfeeds) and firehol (http://iplists.firehol.org/) seem to host copies of most of the lists as well.
I do not really want to overthink this - we didn’t do this with clamav for example either. But it probably is a decision to be made by the user what they want to enable and we should not enable anything by default. So no data will be leaked as long as the user does not consent.
-Michael
Thanks, and best regards, Peter Müller