Hi Michael and Adolf,
I'll combine my reply to both of you.
On Wed, 20 Nov 2024 15:21:28 +0000, Michael Tremer wrote:
Hello,
On 20 Nov 2024, at 13:36, Rob Brewer ipfire-devel@grantura.co.uk wrote:
On Tue, 19 Nov 2024 17:10:24 +0000, Michael Tremer wrote:
Hi Michael, thank for your response, I was beginning to think there was little interest in this topic.
I was traveling and so have a little bit of a back log as usual :)
I can’t speak for the others, though.
Not a problem, I should have said that I previously raised this a topic on the forum and Abolf suggested that I raised it on the mailing list as well.
See below...
Hello Rob,
Good to hear from you again.
On 9 Nov 2024, at 14:00, Rob Brewer ipfire-devel@grantura.co.uk wrote:
As a contributor to the SANS ISC, I email my IPTABLES logs hourly to the Dshield database, however the default IPFire IPTABLES logs are incomplete since packets dropped by Locationblock are not logged to syslog.
This is correct.
As a workaround I edit the locationblock function of rules.pl to add a IPTABLES rule to log the dropped packets. This edit does not of course survive any Core-Updates that upgrade rules.pl and the edit needs to be re-applied.
In addition, unless the Locationblock packets are logged to syslog the data displayed in the Firewall logs sections are incomplete as the packets dropped by Location Block are not displayed.
I understand that the original intension of the Location Block feature was to reduce the amount of log messages on installations running on extremely cheap flash storage. I suspect that these installations are now almost zero especially since IPFire no longer supports 32 bit versions.
The obvious solution to this to make Location Block logging selectable as a firewall option and my quick look at making a patches for this would seem to be fairly simple task to rules.pl and and optionsfw.cgi.
I would be happy to provide the necessary patches should this be acceptable.
I think a couple of things have changed since we introduced this feature and we have also learned a couple of lessons here:
- Yes it is true that this was introduced to keep the logs smaller,
but not necessarily to avoid using storage, mainly to keep them readable by a human. I was always a bit doubtful about this and believe that apart from this, there is no more value in a general location blocking feature. The firewall will drop those packets anyway.
IPFire does a very good job in protecting the internal network from unwanted probes from the WAN, and I agree that for most users, location block does not enhance this protection.
As I run a mail server, I use locationblock as part of my filtering to prevent bad actors from accessing my mail server and I think would be useful for anyone who are running servers on their networks.
I don't like blocking traffic, but I have witnessed over the last few years a substantial rise in abuse from certain countries from where I would never expect to receive emails and locationblock works very well for me in filtering out those unwanted connections.
I currently have 21 countries blocked including XD Hostile networks. All of these blocked countries have been seen to support abusive networks.
It would be possible to generate individual rules for these networks which in some cases I do with Banish, but it is sometimes easier just to drop the whole country.
Easy yes. I am just not sure what we are actually getting out of this.
I don’t like to become political on this list and so I won’t, but I know that some people hold the following beliefs:
- There are “bad” countries out there. In the western world those are
Russia, China, and more that fall into this kind of “category”. Nothing is going to stop them to open up an account on AWS and suddenly they have access to resources anywhere in the world and so actually, if those players were up to anything “bad”, they can come out of any IP space they want. There is also Tor which allows this.
I have been submitting my iptables logs to dshield for over 20 years and in the early days I would be sending a lot less than 1000 a day (this was with IPCop of course).
Over the last few years the number of firewall iptables logs have escalated until it had reached dshield's daily limit of 100,000 /day and I started looking at ways to limit the logging of some of the 'bad actors' firewall probes.
- Looking at actual data, the bullet proof ISPs that annoy us are not
that far away. They are hosted in Europe and very prominently so in the Netherlands - a “good” country.
I use Banish for this as it can block by ASN and can target bullet proof ISPs not only in NL but also in the USA and elsewhere. Using the 'location function' the blocked ASN's are updated when the database is updated.
This is just an example to explain that we all have our idea of the world in our head, but the Internet doesn’t know these borders. Geopolitics is complicated and I don’t think that this is a tool that helps to fight this kind of problem. I hope this is clear what I am trying to say.
Yes my approach is to use multiple tools and Locationblock is just one of the tools I am using. Maybe my approach is over the top though.
I personally trust in Spamhaus which is feeding data into our “Hostile Networks” blocker and they are looking at the actual traffic that they see from those subnets that they are blocking. They have a clear set of rules and they don’t look just at the country and say “ah yes, the bad guys again”.
For emails I also drop packets if listed in a number of DNS blackhole lists using Auto-Dnsbl which I adapted for IPFire. See:
https://people.ipfire.org/~helix/auto-dnsbl/README for info.
I have been using this for about a year and can confirm that it is very effective in removing spammers.
I have stated this recently in the RPZ discussion that I feel that we are lacking good sources for this kind of data. For blocking anything. Loads of people believe “more is more” and to be fair… once you get to blocking 90% of the internet you probably have blocked 90% of the noise. However this won’t do anything for targeted attacks. I personally would rather have more quality here.
I only use a few blocklists with ipblocklist and rely mainly on my locally derived data from failed smtp connections or from data from dshield's daily report.
- People seem to misunderstand this feature: The block as many
countries as they can click on and think that this makes things “more secure”, which of course it doesn’t.
I think this is education and I suspect that very few of these people are running servers on their system and need location block.
I am personally not here to educate people. I am happy to help out and give my opinion, but I suppose considering that people think as I have outlined above, who am I to educate them?
Sorry I didn't express my reply well, I agree that IPFire users should be responsible for their own actions.
It is just very regretful to me when people are ab-using a feature just to “feel more safe”.
- What I personally like as a feature is to apply location filters to
individual rules. That allows to apply rate limits in a few places and easily select some larger groups of potentially less interesting traffic.
I agree and is similar to what I am doing for ports 25 and 587 although with location block.
- Back when this was introduced there was also a lot less use of
botnets as we know them now. Pretty much all recent DoS attacks I remember have been coming from all over the world because systems where compromised wherever they were. This renders the filter absolutely useless.
For this, I am currently testing a dynamic block list on my system which adds an entry to an ipset when an ip address exceeds a threshold in the previous 24 hours. Currently there are about 7000 entries in this set which have been detected over the previous month. Not for everyone because it needs a Whitelist to prevent some 'push' technologies such as Facebook and to protect from bulk inputs such as mailing lists. I also blacklist a large number of ip addresses which are associated with port scanners.
This would work well as a botnet filter but it needs an understanding of the blocking process, so it would not be suitable for everyone.
We have talked about this on the last team call and came to the conclusion that the whole set of blocking features is a total mess. There are several places where things can be blocked, they all heavily overlap. Not to be going on about Spamhaus again, but the DROP list is in the “Hostile Networks” feature, there is a special country code for it, pretty much every IPS ruleset contains it, the IP blocklist feature supports it. Now there are people throwing this all into DNS as well. I believe this is all getting a little bit out of hand.
I don't use the IPS mainly due to bad experiences when trying to use Snort with IPCop. I find it difficult to set up and select the appropriate Rulsets for my requirements. I suspect the IPS would be very effective if I could get it set up correctly.
I personally would prefer *one* place and we really make this work well.
Some people find the IPS has too much overhead - I am beginning to disagree.
Some people add all sorts of blacklists that are clearly for experimental/research use and suddenly complain about things not working anymore. It is hard to figure out what is actually blocking traffic when there are about five places to look for. The result often is to disable the IPS, then something else, and after a while all of this is turned off and the firewall is configured to a bare minimum. We don’t want people to run it like that.
- Infrastructure is becoming more centralised. Some people attempt to
block the US and pretty much nothing works any more. Well…
:)
Therefore I am happy to rethink this feature - personally I would even remove it entirely, but I don’t like to handle the stress about this now.
No I think you should keep it. Locationblock is very useful, but needs to be used in a considered way. I could live without it but its availability makes my life easier. Also being updated on a weekly basis makes keeping track of abusers easier.
We issue a new location database once a day, but the IPFire systems update it only once a week. We deemed that as sufficient as there are not that many changes from one version to another one.
I think this should log now. I don’t think there is any legitimate reason to disable firewall logging at all. This is the paper trail that you want when debugging anything or doing forensics on an attack. Certainly bad flash is not a valid reason for me.
I have started generating some patches to make 'locationblock logging' as a selectable feature and could post them here in the next day or so if of any use.
What would this basically do? Log it as a simple rule for all, or do you include the selected country? I think the latter would make more sense.
OK - I have modified location-block.cgi to add an additional checkbox at the top of the page titled "Log dropped packets", very similar to ipblocklist.cgi. This adds a entry to the /var/ipfire/firewall/ locationblock settings file ON or OFF.
In /usr/lib/firewall/rules.pl I add an additional Iptables LOCATIONBLOCK log rule with the prefix 'LOCBLOCK-$location '. This is selectable from the Log dropped packets entry in the locationblock settings file.
This generates kernel messages such as:
LOCBLOCK-CN IN=ppp0 OUT=ppp0 MAC= SRC=114.139.42.71 DST=81.187.175.103 LEN=60 TOS=0x00 PREC=0x00 TTL=47 ID=34664 DF PROTO=TCP SPT=46837 DPT=23 WINDOW=29040 RES=0x00 SYN URGP=0
As you can see the prefix contains the country code of the dropped packets.
What is the rationale to turn this off? Why don’t we turn this on for everybody all of the time?
I would be happy with that I was suggesting that we use selectable logging to be compatible with earlier IPFire versions.
In fact I think that IPFire's logging environment, which was inherited from IPCop needs a re-visit.
Yes. But this is probably not a thing I would be looking at in IPFire 2.
I am also happy to think a little bit more outside of the box and see what we would fundamentally change in the future, but not implement in IPFire 2.
I was pleased to see that a while back there was a discussion of replacing sylogd with rsyslogd which I think would be a very useful upgrade and enable kernel messages, like on most current linux systems, to be logged to alternative log files instead of everything ending up in /var/log/ messages.
The currentl syslog server could do that. The problem is more that we don’t have good tools to view the logs on the web UI. So there is simply one place where most things go into and we will then filter it out when users use the web UI. This is tremendously slow.
I did look at this a while back and believe that with syslog all kernel messages must go to the same log file and cannot be filtered off as can be done with rsyslog. I use this facility in rsyslog.d on my server to filter off SYN Flood logs to a separate log file with ":msg,contains". I don't believe that syslog has this option.
The replacement with rsyslgd could also inhibit the "last message repeated x times" in syslog and which are ignored by IPFire's Log menu and causes erroneous results in the 'Count' values.
This might be a setting that we can turn off in the current implementation.
I understand that this is hard coded into syslogd and can't be changed. In rsyslog it defaults to "off".
https://www.rsyslog.com/doc/configuration/action/ rsconf1_repeatedmsgreduction.html
We could also think about moving all firewall hits into /var/log/firewall.log or something. That should free up enough in /var/log/messages to make debugging easier.
That would be a sensible option, currently I filter out firewall hits from the messages.log with a perl regex.
I would be happy to hear some more opinions on this from the list.
-Michael
I hope these comments are useful and would be pleased to help where I can.
Of course :)
-Michael
Rob