From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer <michael.tremer@ipfire.org> To: development@lists.ipfire.org Subject: Re: [PATCH 0/5] ipblacklist: IP Address Blacklists Date: Mon, 06 Jan 2020 11:27:26 +0000 Message-ID: <28307984-A289-4568-8B89-50A7641003CD@ipfire.org> In-Reply-To: <aeb93668-a4b6-5735-1f68-fd53cafa2210@tfitzgeorge.me.uk> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5889496251395065599==" List-Id: <development.lists.ipfire.org> --===============5889496251395065599== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello all, > On 4 Jan 2020, at 19:02, Tim FitzGeorge <ipfr(a)tfitzgeorge.me.uk> wrote: >=20 > Hello, >=20 > I've now updated the code - screenshots are attached. >=20 > IP_Address_Blacklists.png >=20 > This is the settings page. I don't think there's anything unexpected here. Yes, that looks a lot like the other IPFire pages :) > IP_Address_Blacklist_Logs.png >=20 > The log page, replacing the status information that was previously at > the top of the settings page. Looks great! > I've sorted the lists alphabetically, rather than by the number of hits > since the worst threat is not necessary the largest number of blocked > packets (which could implied by sorting it by number). Yes, that is indeed a very good idea. You could add a % sign in the percentage column, so it cannot be confused so = easily with the other column. > Firewall_log_blacklist.png >=20 > Details page accessed from the log. Almost identical to the other > similar pages. *thumbs up* > Log_Summary.png >=20 > This is an extract from the Log Summary page produced by logwatch. >=20 > I still need to do some tidying, remove unused language strings etc., > and then update and test the patches, so it'll be some time before I'm > ready to submit updated patches. Feel free to send some RFC to the list. It might be slightly noisy, but it is= easier to review changes bit after bit. Great great work! -Michael >=20 > Tim >=20 > On 28/12/2019 20:59, Tim FitzGeorge wrote: >> Hi, >>=20 >> I'll code and test what we've agreed on. In the meantime I'm going to >> reply on the sub-thread started by Tom Rymes as to what we call the list >> categories, so that we can keep that discussion separate from further >> functionality etc. discussion. >>=20 >> Tim >>=20 >> On 21/12/2019 18:34, Tim FitzGeorge wrote: >>> Hi, >>>=20 >>> I've got my systems checking for downloads at 15 minute intervals and >>> using If-Modified-Since. This all seems to be working OK. >>>=20 >>> Responses to comments below: >>>=20 >>> On 18/12/2019 12:07, Michael Tremer wrote: >>>> Hi, >>>>=20 >>>>> On 17 Dec 2019, at 18:21, Tim FitzGeorge <ipfr(a)tfitzgeorge.me.uk> wro= te: >>>>>=20 >>>>> Hi, >>>>>=20 >>>>> I think we've agreed on the following: >>>>>=20 >>>>> - Checks to be made every 15 minutes, subject to per-list minimum rate. >>>>> - Modify download to use If-Modified-Since rather than separate HEAD >>>>> request. >>>>> - Remove automatic blacklist. >>>>=20 >>>> I thought someone wants to counter my argument. This is not a dictatorsh= ip :) >>>>=20 >>>>>=20 >>>>> other comments are below. >>>>>=20 >>>>> Tim >>>>>=20 >>>>> On 16/12/2019 22:20, Michael Tremer wrote: >>>>>> Hi, >>>>>>=20 >>>>>>> On 16 Dec 2019, at 20:06, Tim FitzGeorge <lists(a)tfitzgeorge.me.uk> = wrote: >>>>>>>=20 >>>>>>> Hi, >>>>>>>=20 >>>>>>> I've attached the current GUI screenshot. >>>>>>=20 >>>>>> Thanks for that. >>>>>>=20 >>>>>> I have a couple of suggestions/concerns about it: >>>>>>=20 >>>>>> a) I am not sure what the value is about the top box that is called = =E2=80=9CStatus=E2=80=9D. It is basically a summary of the iptables output, b= ut will it help the user? A blacklist with more or fewer entries is not bette= r or worse, blocking more packets isn=E2=80=99t better than blocking fewer. I= t is about the quality of the blocks. >>>>>=20 >>>>> I agree that the size of the lists is not all that useful, but the >>>>> information about the number of blocked packets can be an indication of >>>>> compromise. It depends on the type of the list, but a list that targets >>>>> C&C channels (e.g. the FEODO lists) should not show any input packets >>>>> being blocked, unless one of the computers being protected is infected. >>>>> Apart from the BOGON lists, none of the lists should show any output >>>>> packets being blocked; again it's a potential indication of infection if >>>>> otherwise. >>>>=20 >>>> The Shodan list will definitely show some activity unless you are behind= CG NAT. >>>>=20 >>>> Should we not have this rather in the logging section? >>>>=20 >>>> And these counters here will be reset when the firewall is being reloade= d. Should this data therefore not be collected from the logs so that you can = go back in time? That would be very important to identify any start of compro= mise. >>>>=20 >>>=20 >>> That makes sense. I'll remove the status from this page and create a >>> log page for the information. If anyone actually wants the iptables >>> information it's available in from Firewall > iptables anyway. >>>=20 >>> Do you think it makes more sense to show packets blocked in/out or >>> packets blocked per source interface? The latter would of (limited) >>> help in tracking down potential compromise. >>>=20 >>>>>>=20 >>>>>> b) I would prefer to move the settings box to the top. If you like you= can show the size of the blacklists there and when they have been updated, t= oo. >>>>>=20 >>>>> I put the status at the top since it's usually what I want to see when I >>>>> go to the page. Under normal circumstances I don't change the settings, >>>>> so I'd rather not have to scroll past them to get to the information I'm >>>>> actually interested in. The update times are just useful to show that >>>>> the lists are actually being updated. The size is interesting to me, >>>>> but as you say it probably won't help the average user, especially if >>>>> you don't know which lists block individual addresses and which block >>>>> nets (BOGON, DSHIELD and EMERGING_COMPROMISED). >>>>=20 >>>> See my comment on the order above. Splitting the page might make sense. >>>>=20 >>>> When you show size, does that mean lines in the blacklists? So it could = be networks or single IP addresses? Does it not make sense to count IP addres= ses (i.e. 256 for a /24 network and so on) and put it in the table? >>> It's lines in the blacklist. >>>=20 >>>>=20 >>>> I think I would only be interested in the size for two things: >>>>=20 >>>> a) Is there any chance for overblocking? i.e. does the list have 2 billi= on entries which would be half the public IPv4 address space or so. I cannot = really come up with a good example here. >>>>=20 >>>> Or >>>>=20 >>>> b) What is the performance impact/memory consumption of the list? >>>=20 >>> It would be interesting to show the type (single address or net), number >>> of entries and number of addresses; the first two affect performance, >>> the second memory use and the latter the number of blocked addresses, >>> but that's really only going to be of interest to a very small number of >>> people, who can always do 'ipset list -t' to get the information. So >>> I'll remove the size column. >>>=20 >>>>=20 >>>>> I originally had the status and settings in one list, but I decided that >>>>> it was getting a bit messy and difficult to follow. I've attached a >>>>> screenshot of a quickly hacked page. (Note that the status area only >>>>> appears once the blacklists have been enabled.) >>>>=20 >>>> You are right. This is getting quite tight there. >>>>=20 >>>> You could write packets in/out into on column like 2k/1023, but see my c= oncerns about this above. >>>=20 >>> Actually I've come to the conclusion that the number of bytes blocked >>> isn't all that important; it's sufficient to show the number of packets. >>>=20 >>>>=20 >>>>>>=20 >>>>>> c) I would suggest to remove the =E2=80=9Csafe=E2=80=9D column because= that is a very hard summary of what the lists do. We should explain that on = the wiki. I guess this is too complicated to explain to our users in one sent= ence and it needs at least a page of text. People who do not read that have y= ou just lost out. >>>>>=20 >>>>> I agree that it's it's an over simplistic summary of the lists. I note >>>>> that Tom Rymes has asked to keep it, but I think its only value is if it >>>>> suggests to users that things are more complicated than they appear and >>>>> therefore prompts them to read the Wiki. >>>>=20 >>>> I will reply to that separately. >>>=20 >>> Looking at that reply, I think it makes sense to include a category >>> column in both the settings and the log summary page - the former to >>> help selection of which lists to use, and the latter as a help when >>> deciding whether the information could indicate compromise or not. >>>=20 >>>>=20 >>>>>>=20 >>>>>> d) I do not understand what the =E2=80=9Cupdate check rate=E2=80=9D me= ans. I suppose we should show the update interval of the lists in the table a= nd just refresh them according to that. >>>>>=20 >>>>> It's the minimum check period for updates. The system checks for >>>>> updates at either the rate specific to the list or this rate, whichever >>>>> is slower. This is a hangover from the early days, copied from the IPS >>>>> updates. I suppose it could have a value for some people who may wish >>>>> to lower the check rate, but I'm not sure that it has sufficient value >>>>> to be worth keeping. >>>>=20 >>>> No, I would not let the user decide when and how often to update the lis= ts. >>>=20 >>> OK. I'll remove it. >>>=20 >>>>=20 >>>> Generally there is this argument internally how often some databases sho= uld be updated. People with slow internet connections or volume based data pl= ans will find this rather expensive to update those lists too often. >>>>=20 >>>> If we ever want to go ahead with that in IPFire 2, there should be a glo= bal switch for a =E2=80=9Clow data mode=E2=80=9D which then delays updating t= hese things. >>>=20 >>> That makes sense. Maybe also the ability to set when downloads take >>> place - but as you say that's for the future. >>>=20 >>>>=20 >>>>>>=20 >>>>>> e) Do we want to keep the automatic blacklists now? I do not think it = will actually improve the security of IPFire. >>>>>>=20 >>>>>=20 >>>>> I'll remove them. I think they do have a use, in that they can detect >>>>> someone doing a port scan and block them before they find an open port, >>>>> but as you say there is also the possibility of a DOS. >>>>=20 >>>> How would this block a DOS? >>>=20 >>> Sorry, I didn't make myself clear - there's the possibility that someone >>> could cause a DOS with the automatic list by faking the source IP >>> address and sending a few packets on a random port. >>>=20 >>>>=20 >>>> -Michael >>>=20 >>> Tim >>>>=20 >>>>>>>=20 >>>>>>> I'll update the code based on our discussions, and submit an updated = set >>>>>>> of patches - I imagine there will have to be at least one more iterat= ion. >>>>>>=20 >>>>>> Let=E2=80=99s wait until we have come to decisions :) >>>>>>=20 >>>>>> -Michael >>>>>>=20 >>>>>>>=20 >>>>>>> Tim >>>>>>>=20 >>>>>>> (No additional comments below) >>>>>>>=20 >>>>>>> On 13/12/2019 23:11, Michael Tremer wrote: >>>>>>>> Hi, >>>>>>>>=20 >>>>>>>> Again my apologies for my late reply. Busy busy weeks. >>>>>>>>=20 >>>>>>>>> On 8 Dec 2019, at 20:50, Tim FitzGeorge <ipfr(a)tfitzgeorge.me.uk> = wrote: >>>>>>>>>=20 >>>>>>>>> Hello, >>>>>>>>>=20 >>>>>>>>> It's my turn to apologise for being slow to respond - I've had a bu= sy >>>>>>>>> week, but I should have plenty of time over the next couple of week= s. >>>>>>>>=20 >>>>>>>> No worries. Turns out we all do :) >>>>>>>>=20 >>>>>>>>> I've made most of the comments inline, however I think Michael had a >>>>>>>>> question (which I can't find now) about what happens if someone ena= bles >>>>>>>>> all the lists. One thing which would perhaps make this less likely= is >>>>>>>>> that the WUI tags the available lists with whether they're safe or = not, >>>>>>>>> with a footnote that safe means that the list only blocks malicious >>>>>>>>> traffic. This won't guarantee that a user won't still try to enabl= e all >>>>>>>>> the lists, but it should make them realise that they should think f= irst. >>>>>>>>=20 >>>>>>>> We had a couple of features going slightly wrong or being =E2=80=9Cm= isunderstood=E2=80=9D by some users. People still seem to panic when they see= =E2=80=9Clocal recursor=E2=80=9D. To this day I do not know why. >>>>>>>>=20 >>>>>>>> We cannot make everything idiot-proof. And when some user if of that= category, they probably should shutdown their IPFire box, educate themselves= and then come back again. So I do not want to limit people, but make things = as easy as possible. >>>>>>>>=20 >>>>>>>> If someone enables all the lists, good luck with passing packets :) >>>>>>>>=20 >>>>>>>>> I have considered replacing this tag with a risk high/medium/low and >>>>>>>>> maybe adding a category (invalid/application/scanner/C&C or somethi= ng >>>>>>>>> like that), but that may provide too much information and dissuade = them >>>>>>>>> from actually following the links to checkout what the list actuall= y does. >>>>>>>>=20 >>>>>>>> Can we have a screenshot of the GUI right now? I didn=E2=80=99t run = the code, yet. >>>>>>>>=20 >>>>>>>> We should document the lists like we do it with the rulesets of the = IPS. People might ignore this, but that is on them. >>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> Tim >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> On 05/12/2019 22:25, Michael Tremer wrote: >>>>>>>>>> Hello, >>>>>>>>>>=20 >>>>>>>>>>> On 4 Dec 2019, at 17:05, Peter M=C3=BCller <peter.mueller(a)ipfir= e.org> wrote: >>>>>>>>>>>=20 >>>>>>>>>>> Hello Tim, hello Michael, >>>>>>>>>>>=20 >>>>>>>>>>> please see my responses inline... >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> We could periodically update the blacklists on our main mirror= (and >>>>>>>>>>>>>> wait for the network to sync it), make sure it is signed and w= rite >>>>>>>>>>>>>> a small downloader that fetches, validates and installs them. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> @All: Thoughts on this? >>>>>>>>>>>>>=20 >>>>>>>>>>>>> I think there are a number of points here. >>>>>>>>>>>>>=20 >>>>>>>>>>>>> Firstly, from the point of a third party using IPFire, is this = really >>>>>>>>>>>>> solving the privacy disclosure problem? There's no way round d= isclosing >>>>>>>>>>>>> your public IP address to someone you're downloading from; all = this does >>>>>>>>>>>>> is change who that information is being disclosed to. For the = user >>>>>>>>>>>>> there's no way of knowing whether the source is more or less pr= otective >>>>>>>>>>>>> of the user's privacy than the blacklist provider. Indeed it w= on't be >>>>>>>>>>>>> possible to know who the lists are being downloaded from until = the >>>>>>>>>>>>> download starts. >>>>>>>>>>>>=20 >>>>>>>>>>>> There is a way: Tor. But that is a totally different story. >>>>>>>>>>> Well, I see a third option on this: Use the mirror infrastructure= we already >>>>>>>>>>> have. Every IPFire installation discloses its public IP address t= o one of these >>>>>>>>>>> servers sooner or later, so we do not disclose additional data if= the blacklists >>>>>>>>>>> were fetched from these. >>>>>>>>>>>=20 >>>>>>>>>>> Needless to say, Tor (hidden services) would be better, but that = is a different >>>>>>>>>>> story indeed. :-) >>>>>>>>>>>>=20 >>>>>>>>>>>> The point is rather that a forget list can be sent instead of th= e =E2=80=9Creal=E2=80=9D one. >>>>>>>>>>> I did not get this. Forget? Forged?? ??? >>>>>>>>>>=20 >>>>>>>>>> Yes, I meant to write forged, but auto-correct didn=E2=80=99t let = me. >>>>>>>>>>=20 >>>>>>>>>>>>> Secondly, latency; some of the lists are updated every 5 minute= s. While >>>>>>>>>>>>> I've limited the maximum check rate to hourly, will the updates >>>>>>>>>>>>> propagate quickly enough. For reference on my main system the = 24 >>>>>>>>>>>>> updates on the CIARMY list made 143 498 changes (additions or >>>>>>>>>>>>> deletions). I've seem it do over 200 000. >>>>>>>>>>> Yes, I observe that behaviour for CINS/CIArmy too. Unfortunately,= they do not >>>>>>>>>>> document a recommended update interval anywhere, so we can only g= uess. >>>>>>>>>>>=20 >>>>>>>>>>> Personally, more static lists seem to be preferable for packet fi= ltering. Highly >>>>>>>>>>> dynamic ones such as CIArmy should be done via DNSBL queries or s= omething similar >>>>>>>>>>> - do we really want to have that list here? >>>>>>>>>>=20 >>>>>>>>>> It is not really an option to implement a DNSBL into a packet filt= er, but I get your point. >>>>>>>>>=20 >>>>>>>>> One of the 'selling points' for an IP address blacklist is that it = can >>>>>>>>> respond quickly to new threats - or rather new attackers. While a = new >>>>>>>>> IDS/IPS rule needs time to analyse the threat, generate a rule and = check >>>>>>>>> it, it's easy to add an address to a list. So, I think the CIArmy = list >>>>>>>>> is potentially useful for protecting home systems etc. with budget >>>>>>>>> hardware, but I would be very careful about using it for a protecti= ng a >>>>>>>>> general access website. >>>>>>>>=20 >>>>>>>> If they are very volatile, we should honour that and update them oft= en, too. >>>>>>>>=20 >>>>>>>> It probably is more about false-positives being removed very quickly= instead of adding threats very quickly. The average IPFire user is probably = not under threats like these to need to react very quickly. >>>>>>>>=20 >>>>>>>> So I do not see much value in adding those lists and then updating o= nce a day. Does it have to be every 5 min? No. I would suggest 15 which shoul= d be good enough for everyone. >>>>>>>>=20 >>>>>>>> Other lists should of course not be updated every 15 minutes when no= t needed. >>>>>>>>=20 >>>>>>>> Running every 15 minutes would allow us to retry downloading lists t= hat are on an hourly schedule if the download failed. >>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>>>> How did you come up with the hour? Will it be retried more often= if the download was not successful? >>>>>>>>>>> One hour is the most common interval indeed, but adding some rand= om time might >>>>>>>>>>> be useful in order to reduce load on the servers providing a blac= klist. >>>>>>>>>>=20 >>>>>>>>>> Yes, definitely. Otherwise we will shoot down our mirrors. >>>>>>>>>=20 >>>>>>>>> When I implemented that section of code, specifying the minimum che= ck >>>>>>>>> period in hours seemed to provide a convenient way of allowing a ch= eck >>>>>>>>> period covering a wide range, with an hour as the fastest and a wee= k as >>>>>>>>> the slowest. I didn't looked at the CIArmy list until much later. = Most >>>>>>>>> of the lists don't change nearly as much, but the CIArmy list is >>>>>>>>> described as one that deliberately responds quickly. >>>>>>>>>=20 >>>>>>>>> From my production system, for yesterday: >>>>>>>>>=20 >>>>>>>>> The following block lists were updated: >>>>>>>>> BLOCKLIST_DE: 24 Time(s) - 9341 change(s) >>>>>>>>> BOGON_FULL: 1 Time(s) - 10 change(s) >>>>>>>>> CIARMY: 24 Time(s) - 159134 change(s) >>>>>>>>> DSHIELD: 7 Time(s) - 18 change(s) >>>>>>>>> EMERGING_FWRULE: 1 Time(s) - 50 change(s) >>>>>>>>> FEODO_AGGRESIVE: 24 Time(s) - 13 change(s) >>>>>>>>> SHODAN: 1 Time(s) - 0 change(s) >>>>>>>>> SPAMHAUS_DROP: 1 Time(s) - 0 change(s) >>>>>>>>> TOR_EXIT: 24 Time(s) - 162 change(s) >>>>>>>>=20 >>>>>>>> Very interesting statistics. >>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> and my test system: >>>>>>>>>=20 >>>>>>>>> The following block lists were updated: >>>>>>>>> ALIENVAULT: 19 Time(s) - 5331 change(s) >>>>>>>>> EMERGING_COMPROMISED: 1 Time(s) - 26 change(s) >>>>>>>>> TALOS_MALICIOUS: 1 Time(s) - 36 change(s) >>>>>>>>>=20 >>>>>>>>> That covers most of the lists. From the WUI, since 1 Dec: >>>>>>>>>=20 >>>>>>>>> Blacklist Entries pkts bytes Last updated >>>>>>>>> in in >>>>>>>>> AUTOBLACKLIST 0 731 51144 Sun Dec 8 18:40:02 2019 >>>>>>>>> BLOCKLIST_DE 28020 857 46735 Sun Dec 8 18:40:02 2019 >>>>>>>>> BOGON_FULL 214 5255 189K Sun Dec 8 16:50:04 2019 >>>>>>>>> CIARMY 15000 19774 976K Sun Dec 8 18:04:01 2019 >>>>>>>>> DSHIELD 20 7992 321K Sun Dec 8 16:45:13 2019 >>>>>>>>> EMERGING_FWRULE 1647 197 8383 Fri Dec 6 05:29:07 2019 >>>>>>>>> FEODO_AGGRESIVE 7169 0 0 Sun Dec 8 18:50:07 2019 >>>>>>>>> SHODAN 32 34 1530 Sun Dec 8 17:54:09 2019 >>>>>>>>> SPAMHAUS_DROP 823 0 0 Fri Dec 6 18:22:35 2019 >>>>>>>>> SPAMHAUS_EDROP 111 82 16433 Thu Dec 5 17:27:09 2019 >>>>>>>>> TOR_EXIT 1055 0 0 Sun Dec 8 18:31:02 2019 >>>>>>>>=20 >>>>>>>> This as well. >>>>>>>>=20 >>>>>>>> Those are more packets than I would have expected. >>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> (I've left out the pkts/bytes out fields which were all 0) >>>>>>>>>=20 >>>>>>>>> Note that where possible I do a HEAD request first and then only >>>>>>>>> download the list if the modification time has changed since the la= st >>>>>>>>> check. For dynamically generated lists this isn't possible. >>>>>>>>=20 >>>>>>>> You won=E2=80=99t need a HEAD request for it. You can include it in = the GET request. >>>>>>>>=20 >>>>>>>> Have a look at location downloader where I use that. The server will= respond with 304 and not send any payload. >>>>>>>>=20 >>>>>>>> https://git.ipfire.org/?p=3Dlocation/libloc.git;a=3Dblob;f=3Dsrc/pyt= hon/location-downloader.in;h=3D1b5932d5822e03737d32b2a27816815b2f7e74dd;hb=3D= HEAD#l151 >>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> If the download isn't successful it just gives up and waits for the= next >>>>>>>>> attempt (apart from the usual retries in the library). I probably >>>>>>>>> should to change that so that it only applies the per list minimum >>>>>>>>> update period in this case (specified in the sources file) rather t= han >>>>>>>>> the user specified value as well. >>>>>>>>=20 >>>>>>>> I think it is not the worst if an update fails. It might just happen= every once in a while. >>>>>>>>=20 >>>>>>>> So I would suggest to just re-run the script more often and when the= mtime of the file is older than the threshold, a download is attempted. You = can use that timestamp for the GET request. >>>>>>>>=20 >>>>>>>>> I already use a time offset on the downloads - when it's started fr= om >>>>>>>>> boot, backup restore or WUI enable, it checks to see if it's instal= led >>>>>>>>> in the fcrontab, and if not adds itself at a randomly generated off= set >>>>>>>>> in the hour. >>>>>>>>=20 >>>>>>>> That should potentially go to red.up, or if we can settle on 15 minu= tes, I would consider that often enough to quickly update all lists after a r= eboot. >>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>>>>> Third, bandwidth; while the downloads are fairly small (ALIENVA= ULT is >>>>>>>>>>>>> the largest at a few MB), there are going to be a lot of them. = How will >>>>>>>>>>>>> this affect the willingness of people to mirror IPFire? >>>>>>>>>>> I do not consider this being a problem as we do not generate that= much traffic >>>>>>>>>>> to them. Of course, that depends on the update interval again. >>>>>>>>>>=20 >>>>>>>>>> That depends on your point of view. >>>>>>>>>>=20 >>>>>>>>>> I do not have a problem with this at all in my data center, but th= ere are plenty of people with a volume-based LTE plan or simply a 128 kBit/s = connection. It will take a longer time to download the lists for them. We nee= d to mind that. >>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Talking about the preference of packet filter and IPS, I prefe= r to >>>>>>>>>>>>>> use the latter as well as it gains more insight in what kind o= f malicious >>>>>>>>>>>>>> traffic tried to pass a firewall machine. On systems with low = resources, >>>>>>>>>>>>>> this might be problematic and removing load from the IPS can b= e preferred >>>>>>>>>>>>>> (make this configurable?!), on others, people might want to ha= ve both >>>>>>>>>>>>>> results. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>> You're only going to get one result for a packet whichever way = round the >>>>>>>>>>>>> IP blacklist and IPS are since whichever comes first will drop = the >>>>>>>>>>>>> packet before it reaches the second (well it would be possible = to put >>>>>>>>>>>>> the IP blacklist first and get it to log and mark packets which= are then >>>>>>>>>>>>> dropped after the IPS, but I think that's getting a little comp= licated. >>>>>>>>>>>>> In addition I've seen the messages about the trouble marking was >>>>>>>>>>>>> causing in the QoS). >>>>>>>>>>>>>=20 >>>>>>>>>>>>> I think it's a 50/50 choice as to which is more valuable first;= it's >>>>>>>>>>>>> probably going to differ from packet to packet. For me the pos= sibility >>>>>>>>>>>>> of reducing the IPS load means I prefer putting the IP blacklis= t first >>>>>>>>>>>>>=20 >>>>>>>>>>>>> It should be fairly easy to add the choice of where to put the = IP >>>>>>>>>>>>> blacklist. I think it'll have to be in the main firewall scrip= t, so >>>>>>>>>>>>> it'll require a firewall restart, but it's not something that'l= l be >>>>>>>>>>>>> changed often. >>>>>>>>>>>>=20 >>>>>>>>>>>> I do not think that the user should choose this. If we cannot ea= sily make a decision, how can our users do this? Not saying they are stupid h= ere, we are just giving them something so that they do not have to put the th= ought and research into things themselves and make their jobs easier. >>>>>>>>>>> Agreed. >>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>> I think performance matters. And if the IPS comes first, the mos= t likely case would be that we are seeing a SYN packet that is being scanned = and after that being dropped by the blacklist. It was pointless to even scan = the empty packet (TCP fast open aside). This is only different for other pack= ets with payloads. >>>>>>>>>>>>=20 >>>>>>>>>>>> We would protect the IPS from a SYN flooding attack here at leas= t and from scanning more packets unnecessarily. >>>>>>>>>>> So dropping packets from blacklisted IP addresses/networks before= IPS is it, then. >>>>>>>>>>>>=20 >>>>>>>>>>>> I do not even think it makes sense to swap the order in the outg= oing direction. >>>>>>>>>>> Me too. >>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>> What IPFire is lacking is a statistical analysis for those logs.= Collecting more and more data isn=E2=80=99t helpful from my point of view. O= nly if you are looking at a very specific thing.This is true, but I am not su= re if it makes sense to spend too much work on this. >>>>>>>>>>> Based on my personal experience, firewall hits observed on a sing= le machine exposed >>>>>>>>>>> to the internet are interesting, but the overall situation across= multiple machines >>>>>>>>>>> is even more interesting. Very quickly, you'll end on something l= ike a centralised >>>>>>>>>>> logging server and custom statistical analysis here... >>>>>>>>>>=20 >>>>>>>>>> Probably a project for IPFire 4.0 :) >>>>>>>>>>=20 >>>>>>>>> Or use one of the existing services, like the DSHIELD client >>>>>>>>> https://dshield.org/howto.html (subject to privacy concerns again). >>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Personally, I consider Spamhaus DROP/EDROP can be safely enabl= ed by default. >>>>>>>>>>>>>> I would love to see the bogon ruleset here, too (think about 8= chan successor >>>>>>>>>>>>>> hosted at unallocated RIPE space in Saint Petersburg), but tha= t will likely cause >>>>>>>>>>>>>> interference if RED does not have a public IP address assigned. >>>>>>>>>>>>>=20 >>>>>>>>>>>>> I can add a field to the options file that controls whether a l= ist is >>>>>>>>>>>>> enabled by default. >>>>>>>>>>> Thank you. :-) >>>>>>>>>>>>=20 >>>>>>>>>>>> To stress the point from above again: We would then share all pu= blic IP addresses of all IPFire systems in the world with Spamhaus and who is= hosting their infrastructure. That can be considered a threat. >>>>>>>>>>> This is my only objection against this patchset. Now, what can we= do about it? >>>>>>>>>>> One possibility is to apply the patchset now and implement a cust= om download >>>>>>>>>>> source thing later on, or do that before releasing Core Update 13= 9 (or which version >>>>>>>>>>> the patchset will be to) after we agreed on something. >>>>>>>>>>=20 >>>>>>>>>> I do not see this being merged for 139. But that is not important.= We need to get it right first and then release it. >>>>>>>>>>=20 >>>>>>>>>> As far as I know, nobody has tested this, yet. >>>>>>>>>=20 >>>>>>>>> There are a number of people who have been running an earlier versi= on >>>>>>>>> which I shared on GitHub. There were a few early issues, but it se= ems >>>>>>>>> to be OK now. >>>>>>>>>=20 >>>>>>>>> https://forum.ipfire.org/viewtopic.php?f=3D27&t=3D21845 >>>>>>>>>=20 >>>>>>>>> This version wasn't integrated into IPFire, so (for example) it ins= erted >>>>>>>>> itself into the INPUT IPTables chain rather than having it's chains >>>>>>>>> created as part of the firewall start-up script. >>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> I have huge concerns about the automatic blacklist. @Peter: What i= s your opinion on this? >>>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> While I implemented it, I'm aware of its potential to cause problem= s, >>>>>>>>> which is why it has to be separately enabled. It's not caused me a= ny >>>>>>>>> issues at the default settings (blocks at over 10 packets per hour = until >>>>>>>>> 1 hour has passed without seeing packets from the address), but I'v= e not >>>>>>>>> used it on a site with publicly announced services. If I was going= to >>>>>>>>> use it on a web site I would want to, at the very minimum, drop the >>>>>>>>> block period drastically. >>>>>>>>=20 >>>>>>>> I suppose this is entirely unusable on an IPFire box in a data cente= r that hosts things. Let=E2=80=99s say our rack in Hanover. >>>>>>>>=20 >>>>>>>> You will have hits from some broken IP stacks and people might just = end up on this without doing anything wrong. >>>>>>>>=20 >>>>>>>> You can denial-of-service easily as well and I suppose without aggre= gation of data from many many systems, it does not make sense to instantly bl= ock addresses. And even then you probably would block an entire subnet. >>>>>>>>=20 >>>>>>>> So I am not sure how I should feel about it. I do not think that it = adds anything because the packets would have been blocked anyways. >>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> On the other hand, it's good at responding quickly. Usually I see = only >>>>>>>>> 1-2% of blocks from the automatic list: >>>>>>>>>=20 >>>>>>>>> Reason Count % First Last >>>>>>>>> CIARMY 2416 45 Dec 6 00:00 Dec 6 23:59 >>>>>>>>> DSHIELD 1353 25 Dec 6 00:00 Dec 6 23:59 >>>>>>>>> INPUT 1294 24 Dec 6 00:00 Dec 6 23:59 >>>>>>>>> AUTOBLACKLIST 122 2 Dec 6 00:20 Dec 6 16:28 >>>>>>>>> BLOCKLIST_DE 89 2 Dec 6 00:20 Dec 6 23:46 >>>>>>>>>=20 >>>>>>>>> and sometimes none at all, but one one occasion it blocked over 8000 >>>>>>>>> packets. Again I'm aware this is for a home system, which is rather >>>>>>>>> different than from a Web server. >>>>>>>>>=20 >>>>>>>>>>> If we do, I will have a look at the licensing stuff (DShield and = Spamhaus do not >>>>>>>>>>> seem to be problematic, as they are hosted on 3rd party servers, = too). >>>>>>>>>>=20 >>>>>>>>>> One of them will be, sooner or later. And one is enough I suppose. >>>>>>>>>=20 >>>>>>>>> DShield (https://dshield.org/api/#threatfeeds) and firehol >>>>>>>>> (http://iplists.firehol.org/) seem to host copies of most of the li= sts >>>>>>>>> as well. >>>>>>>>>>=20 >>>>>>>>>> I do not really want to overthink this - we didn=E2=80=99t do this= with clamav for example either. But it probably is a decision to be made by = the user what they want to enable and we should not enable anything by defaul= t. So no data will be leaked as long as the user does not consent. >>>>>>>>>>=20 >>>>>>>>>> -Michael >>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>> Thanks, and best regards, >>>>>>>>>>> Peter M=C3=BCller >>>>>>>>=20 >>>>>>>=20 >>>>>>> <Screenshot_2019-12-16_IP_Address_Blacklists.png> >>>>>>=20 >>>>>=20 >>>>> <Screenshot_2019-12-17_IP_Address_Blacklists.png> >>>>=20 >>>=20 >>=20 >=20 > <IP_Address_Blacklists.png><IP_Address_Blacklist_Logs.png><Firewall_log_bla= cklist.png><Log_Summary.png> --===============5889496251395065599==--