Logging Locationblock packets

Logging Locationblock packets

[Rob Brewer]

[-- Attachment #1: Type: text/plain, Size: 1230 bytes --]

As  a contributor to the SANS ISC,  I email my IPTABLES logs hourly to 
the 
Dshield database, however the default IPFire IPTABLES logs are incomplete 
since packets dropped by Locationblock are not logged to syslog.

As a workaround I edit the locationblock function of rules.pl to add a 
IPTABLES rule to log the dropped packets. This edit does not of course 
survive any Core-Updates that upgrade rules.pl and the edit needs to be 
re-applied.

In addition, unless the Locationblock packets are logged to syslog the 
data 
displayed in the Firewall logs sections are incomplete as the packets 
dropped by Location Block are not displayed.

I understand that the original intension of the Location Block feature 
was 
to reduce the amount of log messages on installations running on 
extremely 
cheap flash storage. I suspect that these installations are now almost 
zero 
especially since IPFire no longer supports 32 bit versions.

The obvious solution to this to make Location Block logging selectable as 
a 
firewall option and my quick look at making a patches for this would seem 
to 
be fairly simple task to rules.pl and and optionsfw.cgi. 

I would be happy to provide the necessary patches should this be 
acceptable.

Re: Logging Locationblock packets

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 3238 bytes --]

Hello Rob,

Good to hear from you again.

> On 9 Nov 2024, at 14:00, Rob Brewer <ipfire-devel(a)grantura.co.uk> wrote:
> 
> As  a contributor to the SANS ISC,  I email my IPTABLES logs hourly to 
> the 
> Dshield database, however the default IPFire IPTABLES logs are incomplete 
> since packets dropped by Locationblock are not logged to syslog.

This is correct.

> As a workaround I edit the locationblock function of rules.pl to add a 
> IPTABLES rule to log the dropped packets. This edit does not of course 
> survive any Core-Updates that upgrade rules.pl and the edit needs to be 
> re-applied.
> 
> In addition, unless the Locationblock packets are logged to syslog the 
> data 
> displayed in the Firewall logs sections are incomplete as the packets 
> dropped by Location Block are not displayed.
> 
> I understand that the original intension of the Location Block feature 
> was 
> to reduce the amount of log messages on installations running on 
> extremely 
> cheap flash storage. I suspect that these installations are now almost 
> zero 
> especially since IPFire no longer supports 32 bit versions.
> 
> The obvious solution to this to make Location Block logging selectable as 
> a 
> firewall option and my quick look at making a patches for this would seem 
> to 
> be fairly simple task to rules.pl and and optionsfw.cgi. 
> 
> I would be happy to provide the necessary patches should this be 
> acceptable.

I think a couple of things have changed since we introduced this feature and we have also learned a couple of lessons here:

* Yes it is true that this was introduced to keep the logs smaller, but not necessarily to avoid using storage, mainly to keep them readable by a human. I was always a bit doubtful about this and believe that apart from this, there is no more value in a general location blocking feature. The firewall will drop those packets anyway.

* People seem to misunderstand this feature: The block as many countries as they can click on and think that this makes things “more secure”, which of course it doesn’t.

* What I personally like as a feature is to apply location filters to individual rules. That allows to apply rate limits in a few places and easily select some larger groups of potentially less interesting traffic.

* Back when this was introduced there was also a lot less use of botnets as we know them now. Pretty much all recent DoS attacks I remember have been coming from all over the world because systems where compromised wherever they were. This renders the filter absolutely useless.

* Infrastructure is becoming more centralised. Some people attempt to block the US and pretty much nothing works any more. Well…

Therefore I am happy to rethink this feature - personally I would even remove it entirely, but I don’t like to handle the stress about this now.

I think this should log now. I don’t think there is any legitimate reason to disable firewall logging at all. This is the paper trail that you want when debugging anything or doing forensics on an attack. Certainly bad flash is not a valid reason for me.

I would be happy to hear some more opinions on this from the list.

-Michael

Re: Logging Locationblock packets

[Rob Brewer]

[-- Attachment #1: Type: text/plain, Size: 6388 bytes --]

On Tue, 19 Nov 2024 17:10:24 +0000, Michael Tremer wrote:

Hi Michael, thank for your response, I was beginning to think there was 
little interest in this topic.

See below...

> Hello Rob,
>
> Good to hear from you again.
>
>> On 9 Nov 2024, at 14:00, Rob Brewer <ipfire-devel(a)grantura.co.uk>
>> wrote:
>>
>> As  a contributor to the SANS ISC,  I email my IPTABLES logs hourly to
>> the Dshield database, however the default IPFire IPTABLES logs are
>> incomplete since packets dropped by Locationblock are not logged to
>> syslog.
>
> This is correct.
>
>> As a workaround I edit the locationblock function of rules.pl to add a
>> IPTABLES rule to log the dropped packets. This edit does not of course
>> survive any Core-Updates that upgrade rules.pl and the edit needs to be
>> re-applied.
>>
>> In addition, unless the Locationblock packets are logged to syslog the
>> data displayed in the Firewall logs sections are incomplete as the
>> packets dropped by Location Block are not displayed.
>>
>> I understand that the original intension of the Location Block feature
>> was to reduce the amount of log messages on installations running on
>> extremely cheap flash storage. I suspect that these installations are
>> now almost zero especially since IPFire no longer supports 32 bit
>> versions.
>>
>> The obvious solution to this to make Location Block logging selectable
>> as a
>> firewall option and my quick look at making a patches for this would
>> seem to be fairly simple task to rules.pl and and optionsfw.cgi.
>>
>> I would be happy to provide the necessary patches should this be
>> acceptable.
>
> I think a couple of things have changed since we introduced this feature
> and we have also learned a couple of lessons here:
>
> * Yes it is true that this was introduced to keep the logs smaller, but
> not necessarily to avoid using storage, mainly to keep them readable by
> a human. I was always a bit doubtful about this and believe that apart
> from this, there is no more value in a general location blocking
> feature. The firewall will drop those packets anyway.
>

IPFire does a very good job in protecting the internal network from 
unwanted probes from the WAN, and I agree that for most users, location 
block does not enhance this protection.

As I run a mail server, I use locationblock as part of my filtering to 
prevent bad actors from accessing my mail server and I think would be 
useful for anyone who are running servers on their networks.

I don't like blocking traffic, but I have witnessed over the last few 
years a substantial rise in abuse from certain countries from where I 
would never expect to receive emails and locationblock works very well for 
me in filtering out those unwanted connections. 

I currently have 21 countries blocked including XD Hostile networks. All 
of these blocked countries have been seen to support abusive networks.

It would be possible to generate individual rules for these networks which 
in some cases I do with Banish, but it is sometimes easier just to drop 
the whole country.

> * People seem to misunderstand this feature: The block as many countries
> as they can click on and think that this makes things “more secure”,
> which of course it doesn’t.
>

I think this is education and I suspect that very few of these people are 
running servers on their system and need location block.

> * What I personally like as a feature is to apply location filters to
> individual rules. That allows to apply rate limits in a few places and
> easily select some larger groups of potentially less interesting
> traffic.

I agree and is similar to what I am doing for ports 25 and 587 although 
with location block.

>
> * Back when this was introduced there was also a lot less use of botnets
> as we know them now. Pretty much all recent DoS attacks I remember have
> been coming from all over the world because systems where compromised
> wherever they were. This renders the filter absolutely useless.

For this, I am currently testing a dynamic block list on my system which 
adds an entry to an ipset when an ip address exceeds a threshold in the 
previous 24 hours. Currently there are about 7000 entries in this set 
which have been detected over the previous month.  Not for everyone 
because it needs a Whitelist to prevent some 'push' technologies such as 
Facebook and to protect from bulk inputs such as mailing lists. I also 
blacklist a large number of ip addresses which are associated with port 
scanners.

This would work well as a botnet filter but it needs an understanding of 
the blocking process, so it would not be suitable for everyone.

>
> * Infrastructure is becoming more centralised. Some people attempt to
> block the US and pretty much nothing works any more. Well…
>

:)

> Therefore I am happy to rethink this feature - personally I would even
> remove it entirely, but I don’t like to handle the stress about this
> now.

No I think you should keep it. Locationblock is very useful, but needs 
to be used in a considered way. I could live without it but its 
availability makes my life easier. Also being updated on a weekly basis 
makes keeping track of abusers easier.

>
> I think this should log now. I don’t think there is any legitimate
> reason to disable firewall logging at all. This is the paper trail that
> you want when debugging anything or doing forensics on an attack.
> Certainly bad flash is not a valid reason for me.
>

I have started generating some patches to make 'locationblock logging' as 
a selectable feature and could post them here in the next day or so if of 
any use.

In fact I think that IPFire's logging environment, which was inherited 
from IPCop needs a re-visit.
 
I was pleased to see that a while back there was a discussion of replacing 
sylogd with rsyslogd which I think would be a very useful upgrade and 
enable kernel messages, like on most current linux systems, to be logged 
to alternative log files instead of everything ending up in /var/log/
messages.

The replacement with rsyslgd could also inhibit the "last message repeated 
x times" in syslog and which are ignored by IPFire's Log menu and causes 
erroneous results in the 'Count' values.


> I would be happy to hear some more opinions on this from the list.
>
> -Michael

I hope these comments are useful and would be pleased to help where I can.


Rob

Re: Logging Locationblock packets

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 7554 bytes --]

Hi Rob & Michael,

On 20/11/2024 14:36, Rob Brewer wrote:
> On Tue, 19 Nov 2024 17:10:24 +0000, Michael Tremer wrote:
>
> Hi Michael, thank for your response, I was beginning to think there was
> little interest in this topic.
I think people have either been away on vacation or very busy hence the slower response time.
> See below...
>
>> Hello Rob,
>>
>> Good to hear from you again.
>>
>>> On 9 Nov 2024, at 14:00, Rob Brewer <ipfire-devel(a)grantura.co.uk>
>>> wrote:
>>>
>>> As  a contributor to the SANS ISC,  I email my IPTABLES logs hourly to
>>> the Dshield database, however the default IPFire IPTABLES logs are
>>> incomplete since packets dropped by Locationblock are not logged to
>>> syslog.
>> This is correct.
>>
>>> As a workaround I edit the locationblock function of rules.pl to add a
>>> IPTABLES rule to log the dropped packets. This edit does not of course
>>> survive any Core-Updates that upgrade rules.pl and the edit needs to be
>>> re-applied.
>>>
>>> In addition, unless the Locationblock packets are logged to syslog the
>>> data displayed in the Firewall logs sections are incomplete as the
>>> packets dropped by Location Block are not displayed.
>>>
>>> I understand that the original intension of the Location Block feature
>>> was to reduce the amount of log messages on installations running on
>>> extremely cheap flash storage. I suspect that these installations are
>>> now almost zero especially since IPFire no longer supports 32 bit
>>> versions.
>>>
>>> The obvious solution to this to make Location Block logging selectable
>>> as a
>>> firewall option and my quick look at making a patches for this would
>>> seem to be fairly simple task to rules.pl and and optionsfw.cgi.
>>>
>>> I would be happy to provide the necessary patches should this be
>>> acceptable.
>> I think a couple of things have changed since we introduced this feature
>> and we have also learned a couple of lessons here:
>>
>> * Yes it is true that this was introduced to keep the logs smaller, but
>> not necessarily to avoid using storage, mainly to keep them readable by
>> a human. I was always a bit doubtful about this and believe that apart
>> from this, there is no more value in a general location blocking
>> feature. The firewall will drop those packets anyway.
>>
> IPFire does a very good job in protecting the internal network from
> unwanted probes from the WAN, and I agree that for most users, location
> block does not enhance this protection.
>
> As I run a mail server, I use locationblock as part of my filtering to
> prevent bad actors from accessing my mail server and I think would be
> useful for anyone who are running servers on their networks.
>
> I don't like blocking traffic, but I have witnessed over the last few
> years a substantial rise in abuse from certain countries from where I
> would never expect to receive emails and locationblock works very well for
> me in filtering out those unwanted connections.
>
> I currently have 21 countries blocked including XD Hostile networks. All
> of these blocked countries have been seen to support abusive networks.
>
> It would be possible to generate individual rules for these networks which
> in some cases I do with Banish, but it is sometimes easier just to drop
> the whole country.
>
>> * People seem to misunderstand this feature: The block as many countries
>> as they can click on and think that this makes things “more secure”,
>> which of course it doesn’t.
>>
> I think this is education and I suspect that very few of these people are
> running servers on their system and need location block.
>
>> * What I personally like as a feature is to apply location filters to
>> individual rules. That allows to apply rate limits in a few places and
>> easily select some larger groups of potentially less interesting
>> traffic.
> I agree and is similar to what I am doing for ports 25 and 587 although
> with location block.
>
>> * Back when this was introduced there was also a lot less use of botnets
>> as we know them now. Pretty much all recent DoS attacks I remember have
>> been coming from all over the world because systems where compromised
>> wherever they were. This renders the filter absolutely useless.
> For this, I am currently testing a dynamic block list on my system which
> adds an entry to an ipset when an ip address exceeds a threshold in the
> previous 24 hours. Currently there are about 7000 entries in this set
> which have been detected over the previous month.  Not for everyone
> because it needs a Whitelist to prevent some 'push' technologies such as
> Facebook and to protect from bulk inputs such as mailing lists. I also
> blacklist a large number of ip addresses which are associated with port
> scanners.
>
> This would work well as a botnet filter but it needs an understanding of
> the blocking process, so it would not be suitable for everyone.
>
>> * Infrastructure is becoming more centralised. Some people attempt to
>> block the US and pretty much nothing works any more. Well…
>>
> :)
>
>> Therefore I am happy to rethink this feature - personally I would even
>> remove it entirely, but I don’t like to handle the stress about this
>> now.
> No I think you should keep it. Locationblock is very useful, but needs
> to be used in a considered way. I could live without it but its
> availability makes my life easier. Also being updated on a weekly basis
> makes keeping track of abusers easier.
>
>> I think this should log now. I don’t think there is any legitimate
>> reason to disable firewall logging at all. This is the paper trail that
>> you want when debugging anything or doing forensics on an attack.
>> Certainly bad flash is not a valid reason for me.
>>
> I have started generating some patches to make 'locationblock logging' as
> a selectable feature and could post them here in the next day or so if of
> any use.
>
> In fact I think that IPFire's logging environment, which was inherited
> from IPCop needs a re-visit.
>   
> I was pleased to see that a while back there was a discussion of replacing
> sylogd with rsyslogd which I think would be a very useful upgrade and
> enable kernel messages, like on most current linux systems, to be logged
> to alternative log files instead of everything ending up in /var/log/
> messages.
>
> The replacement with rsyslgd could also inhibit the "last message repeated
> x times" in syslog and which are ignored by IPFire's Log menu and causes
> erroneous results in the 'Count' values.
>
>
>> I would be happy to hear some more opinions on this from the list.

I had added this topic onto the agenda for the next video conf call but if it gets resolved on the mailing list that is also fine and I will then, at that point, remove it from the agenda.


I don't use Location Blocking on my production system at all. Virtually everything is blocked as standard. I just have port 80 open for Lets Encrypt and as they use multiple servers from multiple ASN's to validate you have to not use Location Blocking otherwise you can end up blocking the update process. They will not provide a list of the IP's of their servers.

Therefore from my point of view I have no issue with logging being able to be turned on for Location Blocking as I don't use it anyway.

Regards,

Adolf.

>> -Michael
> I hope these comments are useful and would be pleased to help where I can.
>
>
> Rob

Re: Logging Locationblock packets

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 11448 bytes --]

Hello,

> On 20 Nov 2024, at 13:36, Rob Brewer <ipfire-devel(a)grantura.co.uk> wrote:
> 
> On Tue, 19 Nov 2024 17:10:24 +0000, Michael Tremer wrote:
> 
> Hi Michael, thank for your response, I was beginning to think there was 
> little interest in this topic.

I was traveling and so have a little bit of a back log as usual :)

I can’t speak for the others, though.

> 
> See below...
> 
>> Hello Rob,
>> 
>> Good to hear from you again.
>> 
>>> On 9 Nov 2024, at 14:00, Rob Brewer <ipfire-devel(a)grantura.co.uk>
>>> wrote:
>>> 
>>> As  a contributor to the SANS ISC,  I email my IPTABLES logs hourly to
>>> the Dshield database, however the default IPFire IPTABLES logs are
>>> incomplete since packets dropped by Locationblock are not logged to
>>> syslog.
>> 
>> This is correct.
>> 
>>> As a workaround I edit the locationblock function of rules.pl to add a
>>> IPTABLES rule to log the dropped packets. This edit does not of course
>>> survive any Core-Updates that upgrade rules.pl and the edit needs to be
>>> re-applied.
>>> 
>>> In addition, unless the Locationblock packets are logged to syslog the
>>> data displayed in the Firewall logs sections are incomplete as the
>>> packets dropped by Location Block are not displayed.
>>> 
>>> I understand that the original intension of the Location Block feature
>>> was to reduce the amount of log messages on installations running on
>>> extremely cheap flash storage. I suspect that these installations are
>>> now almost zero especially since IPFire no longer supports 32 bit
>>> versions.
>>> 
>>> The obvious solution to this to make Location Block logging selectable
>>> as a
>>> firewall option and my quick look at making a patches for this would
>>> seem to be fairly simple task to rules.pl and and optionsfw.cgi.
>>> 
>>> I would be happy to provide the necessary patches should this be
>>> acceptable.
>> 
>> I think a couple of things have changed since we introduced this feature
>> and we have also learned a couple of lessons here:
>> 
>> * Yes it is true that this was introduced to keep the logs smaller, but
>> not necessarily to avoid using storage, mainly to keep them readable by
>> a human. I was always a bit doubtful about this and believe that apart
>> from this, there is no more value in a general location blocking
>> feature. The firewall will drop those packets anyway.
>> 
> 
> IPFire does a very good job in protecting the internal network from 
> unwanted probes from the WAN, and I agree that for most users, location 
> block does not enhance this protection.
> 
> As I run a mail server, I use locationblock as part of my filtering to 
> prevent bad actors from accessing my mail server and I think would be 
> useful for anyone who are running servers on their networks.
> 
> I don't like blocking traffic, but I have witnessed over the last few 
> years a substantial rise in abuse from certain countries from where I 
> would never expect to receive emails and locationblock works very well for 
> me in filtering out those unwanted connections. 
> 
> I currently have 21 countries blocked including XD Hostile networks. All 
> of these blocked countries have been seen to support abusive networks.
> 
> It would be possible to generate individual rules for these networks which 
> in some cases I do with Banish, but it is sometimes easier just to drop 
> the whole country.

Easy yes. I am just not sure what we are actually getting out of this.

I don’t like to become political on this list and so I won’t, but I know that some people hold the following beliefs:

* There are “bad” countries out there. In the western world those are Russia, China, and more that fall into this kind of “category”. Nothing is going to stop them to open up an account on AWS and suddenly they have access to resources anywhere in the world and so actually, if those players were up to anything “bad”, they can come out of any IP space they want. There is also Tor which allows this.

* Looking at actual data, the bullet proof ISPs that annoy us are not that far away. They are hosted in Europe and very prominently so in the Netherlands - a “good” country.

This is just an example to explain that we all have our idea of the world in our head, but the Internet doesn’t know these borders. Geopolitics is complicated and I don’t think that this is a tool that helps to fight this kind of problem. I hope this is clear what I am trying to say.

I personally trust in Spamhaus which is feeding data into our “Hostile Networks” blocker and they are looking at the actual traffic that they see from those subnets that they are blocking. They have a clear set of rules and they don’t look just at the country and say “ah yes, the bad guys again”.

I have stated this recently in the RPZ discussion that I feel that we are lacking good sources for this kind of data. For blocking anything. Loads of people believe “more is more” and to be fair… once you get to blocking 90% of the internet you probably have blocked 90% of the noise. However this won’t do anything for targeted attacks. I personally would rather have more quality here.

>> * People seem to misunderstand this feature: The block as many countries
>> as they can click on and think that this makes things “more secure”,
>> which of course it doesn’t.
>> 
> 
> I think this is education and I suspect that very few of these people are 
> running servers on their system and need location block.

I am personally not here to educate people. I am happy to help out and give my opinion, but I suppose considering that people think as I have outlined above, who am I to educate them?

It is just very regretful to me when people are ab-using a feature just to “feel more safe”.

>> * What I personally like as a feature is to apply location filters to
>> individual rules. That allows to apply rate limits in a few places and
>> easily select some larger groups of potentially less interesting
>> traffic.
> 
> I agree and is similar to what I am doing for ports 25 and 587 although 
> with location block.
> 
>> 
>> * Back when this was introduced there was also a lot less use of botnets
>> as we know them now. Pretty much all recent DoS attacks I remember have
>> been coming from all over the world because systems where compromised
>> wherever they were. This renders the filter absolutely useless.
> 
> For this, I am currently testing a dynamic block list on my system which 
> adds an entry to an ipset when an ip address exceeds a threshold in the 
> previous 24 hours. Currently there are about 7000 entries in this set 
> which have been detected over the previous month.  Not for everyone 
> because it needs a Whitelist to prevent some 'push' technologies such as 
> Facebook and to protect from bulk inputs such as mailing lists. I also 
> blacklist a large number of ip addresses which are associated with port 
> scanners.
> 
> This would work well as a botnet filter but it needs an understanding of 
> the blocking process, so it would not be suitable for everyone.

We have talked about this on the last team call and came to the conclusion that the whole set of blocking features is a total mess. There are several places where things can be blocked, they all heavily overlap. Not to be going on about Spamhaus again, but the DROP list is in the “Hostile Networks” feature, there is a special country code for it, pretty much every IPS ruleset contains it, the IP blocklist feature supports it. Now there are people throwing this all into DNS as well. I believe this is all getting a little bit out of hand.

I personally would prefer *one* place and we really make this work well.

Some people find the IPS has too much overhead - I am beginning to disagree.

Some people add all sorts of blacklists that are clearly for experimental/research use and suddenly complain about things not working anymore. It is hard to figure out what is actually blocking traffic when there are about five places to look for. The result often is to disable the IPS, then something else, and after a while all of this is turned off and the firewall is configured to a bare minimum. We don’t want people to run it like that.

>> * Infrastructure is becoming more centralised. Some people attempt to
>> block the US and pretty much nothing works any more. Well…
>> 
> 
> :)
> 
>> Therefore I am happy to rethink this feature - personally I would even
>> remove it entirely, but I don’t like to handle the stress about this
>> now.
> 
> No I think you should keep it. Locationblock is very useful, but needs 
> to be used in a considered way. I could live without it but its 
> availability makes my life easier. Also being updated on a weekly basis 
> makes keeping track of abusers easier.

We issue a new location database once a day, but the IPFire systems update it only once a week. We deemed that as sufficient as there are not that many changes from one version to another one.

>> I think this should log now. I don’t think there is any legitimate
>> reason to disable firewall logging at all. This is the paper trail that
>> you want when debugging anything or doing forensics on an attack.
>> Certainly bad flash is not a valid reason for me.
>> 
> 
> I have started generating some patches to make 'locationblock logging' as 
> a selectable feature and could post them here in the next day or so if of 
> any use.

What would this basically do? Log it as a simple rule for all, or do you include the selected country? I think the latter would make more sense.

What is the rationale to turn this off? Why don’t we turn this on for everybody all of the time?

> In fact I think that IPFire's logging environment, which was inherited 
> from IPCop needs a re-visit.

Yes. But this is probably not a thing I would be looking at in IPFire 2.

I am also happy to think a little bit more outside of the box and see what we would fundamentally change in the future, but not implement in IPFire 2.

> I was pleased to see that a while back there was a discussion of replacing 
> sylogd with rsyslogd which I think would be a very useful upgrade and 
> enable kernel messages, like on most current linux systems, to be logged 
> to alternative log files instead of everything ending up in /var/log/
> messages.

The currentl syslog server could do that. The problem is more that we don’t have good tools to view the logs on the web UI. So there is simply one place where most things go into and we will then filter it out when users use the web UI. This is tremendously slow.

> The replacement with rsyslgd could also inhibit the "last message repeated 
> x times" in syslog and which are ignored by IPFire's Log menu and causes 
> erroneous results in the 'Count' values.

This might be a setting that we can turn off in the current implementation.

We could also think about moving all firewall hits into /var/log/firewall.log or something. That should free up enough in /var/log/messages to make debugging easier.

>> I would be happy to hear some more opinions on this from the list.
>> 
>> -Michael
> 
> I hope these comments are useful and would be pleased to help where I can.

Of course :)

-Michael

> 
> 
> Rob

Re: Logging Locationblock packets

[Rob Brewer]

[-- Attachment #1: Type: text/plain, Size: 15006 bytes --]

Hi Michael and Adolf,

I'll combine my reply to both of you.

On Wed, 20 Nov 2024 15:21:28 +0000, Michael Tremer wrote:

> Hello,
> 
>> On 20 Nov 2024, at 13:36, Rob Brewer <ipfire-devel(a)grantura.co.uk>
>> wrote:
>> 
>> On Tue, 19 Nov 2024 17:10:24 +0000, Michael Tremer wrote:
>> 
>> Hi Michael, thank for your response, I was beginning to think there was
>> little interest in this topic.
> 
> I was traveling and so have a little bit of a back log as usual :)
> 
> I can’t speak for the others, though.
> 

Not a problem, I should have said that I previously raised this a topic on 
the forum and Abolf suggested that I raised it on the mailing list as 
well.

> 
>> See below...
>> 
>>> Hello Rob,
>>> 
>>> Good to hear from you again.
>>> 
>>>> On 9 Nov 2024, at 14:00, Rob Brewer <ipfire-devel(a)grantura.co.uk>
>>>> wrote:
>>>> 
>>>> As  a contributor to the SANS ISC,  I email my IPTABLES logs hourly
>>>> to the Dshield database, however the default IPFire IPTABLES logs are
>>>> incomplete since packets dropped by Locationblock are not logged to
>>>> syslog.
>>> 
>>> This is correct.
>>> 
>>>> As a workaround I edit the locationblock function of rules.pl to add
>>>> a IPTABLES rule to log the dropped packets. This edit does not of
>>>> course survive any Core-Updates that upgrade rules.pl and the edit
>>>> needs to be re-applied.
>>>> 
>>>> In addition, unless the Locationblock packets are logged to syslog
>>>> the data displayed in the Firewall logs sections are incomplete as
>>>> the packets dropped by Location Block are not displayed.
>>>> 
>>>> I understand that the original intension of the Location Block
>>>> feature was to reduce the amount of log messages on installations
>>>> running on extremely cheap flash storage. I suspect that these
>>>> installations are now almost zero especially since IPFire no longer
>>>> supports 32 bit versions.
>>>> 
>>>> The obvious solution to this to make Location Block logging
>>>> selectable as a firewall option and my quick look at making a patches
>>>> for this would seem to be fairly simple task to rules.pl and and
>>>> optionsfw.cgi.
>>>> 
>>>> I would be happy to provide the necessary patches should this be
>>>> acceptable.
>>> 
>>> I think a couple of things have changed since we introduced this
>>> feature and we have also learned a couple of lessons here:
>>> 
>>> * Yes it is true that this was introduced to keep the logs smaller,
>>> but not necessarily to avoid using storage, mainly to keep them
>>> readable by a human. I was always a bit doubtful about this and
>>> believe that apart from this, there is no more value in a general
>>> location blocking feature. The firewall will drop those packets
>>> anyway.
>>> 
>>> 
>> IPFire does a very good job in protecting the internal network from
>> unwanted probes from the WAN, and I agree that for most users, location
>> block does not enhance this protection.
>> 
>> As I run a mail server, I use locationblock as part of my filtering to
>> prevent bad actors from accessing my mail server and I think would be
>> useful for anyone who are running servers on their networks.
>> 
>> I don't like blocking traffic, but I have witnessed over the last few
>> years a substantial rise in abuse from certain countries from where I
>> would never expect to receive emails and locationblock works very well
>> for me in filtering out those unwanted connections.
>> 
>> I currently have 21 countries blocked including XD Hostile networks.
>> All of these blocked countries have been seen to support abusive
>> networks.
>> 
>> It would be possible to generate individual rules for these networks
>> which in some cases I do with Banish, but it is sometimes easier just
>> to drop the whole country.
> 
> Easy yes. I am just not sure what we are actually getting out of this.
> 
> I don’t like to become political on this list and so I won’t, but I know
> that some people hold the following beliefs:
> 
> * There are “bad” countries out there. In the western world those are
> Russia, China, and more that fall into this kind of “category”. Nothing
> is going to stop them to open up an account on AWS and suddenly they
> have access to resources anywhere in the world and so actually, if those
> players were up to anything “bad”, they can come out of any IP space
> they want. There is also Tor which allows this.
> 

I have been submitting my iptables logs to dshield for over 20 years and 
in the early days I would be sending a lot less than 1000 a day (this was 
with IPCop of course).

Over the last few years the number of firewall iptables logs have  
escalated until it had reached dshield's daily limit of 100,000 /day  and 
I started looking at ways to limit the logging of some of the 'bad actors' 
firewall probes.


> * Looking at actual data, the bullet proof ISPs that annoy us are not
> that far away. They are hosted in Europe and very prominently so in the
> Netherlands - a “good” country.
> 

I use Banish for this as it can block by ASN and can target bullet proof 
ISPs not only in NL but also in the USA and elsewhere. Using the 'location 
function' the blocked ASN's are updated when the database is updated.

> This is just an example to explain that we all have our idea of the
> world in our head, but the Internet doesn’t know these borders.
> Geopolitics is complicated and I don’t think that this is a tool that
> helps to fight this kind of problem. I hope this is clear what I am
> trying to say.
> 

Yes my approach is to use multiple tools and Locationblock is just one of 
the tools I am using. Maybe my approach is over the top though.

> I personally trust in Spamhaus which is feeding data into our “Hostile
> Networks” blocker and they are looking at the actual traffic that they
> see from those subnets that they are blocking. They have a clear set of
> rules and they don’t look just at the country and say “ah yes, the bad
> guys again”.
>

For emails I also drop packets if listed in a number of DNS blackhole 
lists using Auto-Dnsbl which I adapted for IPFire. See:

 https://people.ipfire.org/~helix/auto-dnsbl/README
 for info.

I have been using this for about a year and can confirm that it is very 
effective in removing spammers.
 
> I have stated this recently in the RPZ discussion that I feel that we
> are lacking good sources for this kind of data. For blocking anything.
> Loads of people believe “more is more” and to be fair… once you get to
> blocking 90% of the internet you probably have blocked 90% of the noise.
> However this won’t do anything for targeted attacks. I personally would
> rather have more quality here.
> 

I only use a few blocklists with ipblocklist and rely mainly on my locally 
derived  data from failed smtp connections or from data from dshield's 
daily report.

>>> * People seem to misunderstand this feature: The block as many
>>> countries as they can click on and think that this makes things “more
>>> secure”,
>>> which of course it doesn’t.
>>> 
>>> 
>> I think this is education and I suspect that very few of these people
>> are running servers on their system and need location block.
> 
> I am personally not here to educate people. I am happy to help out and
> give my opinion, but I suppose considering that people think as I have
> outlined above, who am I to educate them?
> 

Sorry I didn't express my reply well, I agree that IPFire users should be 
responsible for their own actions.

> It is just very regretful to me when people are ab-using a feature just
> to “feel more safe”.
> 
>>> * What I personally like as a feature is to apply location filters to
>>> individual rules. That allows to apply rate limits in a few places and
>>> easily select some larger groups of potentially less interesting
>>> traffic.
>> 
>> I agree and is similar to what I am doing for ports 25 and 587 although
>> with location block.
>> 
>> 
>>> * Back when this was introduced there was also a lot less use of
>>> botnets as we know them now. Pretty much all recent DoS attacks I
>>> remember have been coming from all over the world because systems
>>> where compromised wherever they were. This renders the filter
>>> absolutely useless.
>> 
>> For this, I am currently testing a dynamic block list on my system
>> which adds an entry to an ipset when an ip address exceeds a threshold
>> in the previous 24 hours. Currently there are about 7000 entries in
>> this set which have been detected over the previous month.  Not for
>> everyone because it needs a Whitelist to prevent some 'push'
>> technologies such as Facebook and to protect from bulk inputs such as
>> mailing lists. I also blacklist a large number of ip addresses which
>> are associated with port scanners.
>> 
>> This would work well as a botnet filter but it needs an understanding
>> of the blocking process, so it would not be suitable for everyone.
> 
> We have talked about this on the last team call and came to the
> conclusion that the whole set of blocking features is a total mess.
> There are several places where things can be blocked, they all heavily
> overlap. Not to be going on about Spamhaus again, but the DROP list is
> in the “Hostile Networks” feature, there is a special country code for
> it, pretty much every IPS ruleset contains it, the IP blocklist feature
> supports it. Now there are people throwing this all into DNS as well. I
> believe this is all getting a little bit out of hand.
> 

I don't use the IPS mainly due to bad experiences when trying to use Snort 
with IPCop. I find it difficult to set up and select the appropriate 
Rulsets for my requirements. I suspect the IPS would be very effective if 
I could get it set up correctly.

> I personally would prefer *one* place and we really make this work well.
> 
> Some people find the IPS has too much overhead - I am beginning to
> disagree.
> 
> Some people add all sorts of blacklists that are clearly for
> experimental/research use and suddenly complain about things not working
> anymore. It is hard to figure out what is actually blocking traffic when
> there are about five places to look for. The result often is to disable
> the IPS, then something else, and after a while all of this is turned
> off and the firewall is configured to a bare minimum. We don’t want
> people to run it like that.
> 
>>> * Infrastructure is becoming more centralised. Some people attempt to
>>> block the US and pretty much nothing works any more. Well…
>>> 
>>> 
>> :)
>> 
>>> Therefore I am happy to rethink this feature - personally I would even
>>> remove it entirely, but I don’t like to handle the stress about this
>>> now.
>> 
>> No I think you should keep it. Locationblock is very useful, but needs
>> to be used in a considered way. I could live without it but its
>> availability makes my life easier. Also being updated on a weekly basis
>> makes keeping track of abusers easier.
> 
> We issue a new location database once a day, but the IPFire systems
> update it only once a week. We deemed that as sufficient as there are
> not that many changes from one version to another one.
> 
>>> I think this should log now. I don’t think there is any legitimate
>>> reason to disable firewall logging at all. This is the paper trail
>>> that you want when debugging anything or doing forensics on an attack.
>>> Certainly bad flash is not a valid reason for me.
>>> 
>>> 
>> I have started generating some patches to make 'locationblock logging'
>> as a selectable feature and could post them here in the next day or so
>> if of any use.
> 
> What would this basically do? Log it as a simple rule for all, or do you
> include the selected country? I think the latter would make more sense.
> 

OK - I have modified location-block.cgi to add an additional checkbox at 
the top of the page titled "Log dropped packets", very similar to 
ipblocklist.cgi. This adds a entry to the /var/ipfire/firewall/
locationblock settings file ON or OFF.

In /usr/lib/firewall/rules.pl I add an additional Iptables LOCATIONBLOCK 
log rule with the prefix 'LOCBLOCK-$location '. This is selectable from 
the Log dropped packets entry in the locationblock settings file.

This generates kernel messages such as:

LOCBLOCK-CN IN=ppp0 OUT=ppp0 MAC= SRC=114.139.42.71 DST=81.187.175.103 
LEN=60 TOS=0x00 PREC=0x00 TTL=47 ID=34664 DF PROTO=TCP SPT=46837 DPT=23 
WINDOW=29040 RES=0x00 SYN URGP=0

As you can see the prefix contains the country code of the dropped 
packets.

> What is the rationale to turn this off? Why don’t we turn this on for
> everybody all of the time?
>
 
I would be happy with that I was suggesting that we use selectable logging 
to be compatible with earlier IPFire versions.

>> In fact I think that IPFire's logging environment, which was inherited
>> from IPCop needs a re-visit.
> 
> Yes. But this is probably not a thing I would be looking at in IPFire 2.
> 
> I am also happy to think a little bit more outside of the box and see
> what we would fundamentally change in the future, but not implement in
> IPFire 2.
> 
>> I was pleased to see that a while back there was a discussion of
>> replacing sylogd with rsyslogd which I think would be a very useful
>> upgrade and enable kernel messages, like on most current linux systems,
>> to be logged to alternative log files instead of everything ending up
>> in /var/log/ messages.
> 
> The currentl syslog server could do that. The problem is more that we
> don’t have good tools to view the logs on the web UI. So there is simply
> one place where most things go into and we will then filter it out when
> users use the web UI. This is tremendously slow.
> 

I did look at this a while back and believe that with syslog all kernel 
messages must go to the same log file and cannot be filtered off as can be 
done with rsyslog. I use this facility in rsyslog.d on my server to filter 
off SYN Flood logs to a separate log file with ":msg,contains". I don't 
believe that syslog has this option.

>> The replacement with rsyslgd could also inhibit the "last message
>> repeated x times" in syslog and which are ignored by IPFire's Log menu
>> and causes erroneous results in the 'Count' values.
> 
> This might be a setting that we can turn off in the current
> implementation.
>
 
I understand that this is hard coded into syslogd and can't be changed. In 
rsyslog it defaults to "off".

https://www.rsyslog.com/doc/configuration/action/
rsconf1_repeatedmsgreduction.html

> We could also think about moving all firewall hits into
> /var/log/firewall.log or something. That should free up enough in
> /var/log/messages to make debugging easier.
> 

That would be a sensible option, currently I filter out firewall hits from 
the messages.log with a perl regex. 

>>> I would be happy to hear some more opinions on this from the list.
>>> 
>>> -Michael
>> 
>> I hope these comments are useful and would be pleased to help where I
>> can.
> 
> Of course :)
> 
> -Michael
> 
> 
>> 
>> Rob

Re: Logging Locationblock packets [PATCH]

[Rob Brewer]

[-- Attachment #1: Type: text/plain, Size: 2096 bytes --]

On Wed, 20 Nov 2024 23:08:39 -0000 (UTC), Rob Brewer wrote:

I've tested these patches on Core-Update 189 and can enable and disable 
locationblock logging.
I can provide the patches in a different format if preferred.

Rob

> OK - I have modified location-block.cgi to add an additional checkbox at
> the top of the page titled "Log dropped packets", very similar to
> ipblocklist.cgi. This adds a entry to the /var/ipfire/firewall/
> locationblock settings file ON or OFF.
> 

/srv/web/ipfire/cgi-bin/location-block.cgi
=================================================================
62a63,69
>         # Check if we want to disable logging.
>         if (exists $cgiparams{'LOGGING_ENABLED'}) {
>                 $settings{'LOGGING_ENABLED'} = "on";
>         } else {
>                 $settings{'LOGGING_ENABLED'} = "off";
>       }
> 
93a101
> my $logging;
97a106,111
> if ($settings{'LOGGING_ENABLED'} eq "on") {
>         $logging = "checked='checked'";
> }
> 
> 
> 
108a123,125
>                         <td width='50%' 
class='base'>$Lang::tr{'ipblocklist log'}
>                         <td><input type='checkbox' 
name='LOGGING_ENABLED' $logging></td>
>               </tr>

==========================================================================


> In /usr/lib/firewall/rules.pl I add an additional Iptables LOCATIONBLOCK
> log rule with the prefix 'LOCBLOCK-$location '. This is selectable from
> the Log dropped packets entry in the locationblock settings file.


/usr/lib/firewall/rules.pl
=====================================================================
76c76,77
<       "LOCATIONBLOCK_ENABLED" => "off"
---
>       "LOCATIONBLOCK_ENABLED" => "off",
>       "LOGGING_ENABLED" => "off",
719a721,725
> 
>       # add loggging if enabled from location-block.cgi
>               if ($locationsettings{'LOGGING_ENABLED'} eq "on") {     
>                       run("$IPTABLES -A LOCATIONBLOCK -m set --match-set 
$location src -j LOG --log-prefix 'LOCBLOCK-$location '");
>                       }

=========================================================================

Re: Logging Locationblock packets [PATCH]

[Rob Brewer]

[-- Attachment #1: Type: text/plain, Size: 1896 bytes --]

On Wed, 20 Nov 2024 23:08:39 -0000 (UTC), Rob Brewer wrote:

Sorry I got caught out by my editor's word wrap. I've now turned it off
so hopefully this will be OK.

Rob 

> OK - I have modified location-block.cgi to add an additional checkbox at
> the top of the page titled "Log dropped packets", very similar to
> ipblocklist.cgi. This adds a entry to the /var/ipfire/firewall/
> locationblock settings file ON or OFF.
>

/srv/web/ipfire/cgi-bin/location-block.cgi
=========================================================
62a63,69
> # Check if we want to disable logging.
> if (exists $cgiparams{'LOGGING_ENABLED'}) {
> $settings{'LOGGING_ENABLED'} = "on";
> } else {
> $settings{'LOGGING_ENABLED'} = "off";
> }
>
93a101
> my $logging;
97a106,111
> if ($settings{'LOGGING_ENABLED'} eq "on") {
> $logging = "checked='checked'";
> }
>
>
>
108a123,125
> <td width='50%' class='base'>$Lang::tr{'ipblocklist log'}
> <td><input type='checkbox' name='LOGGING_ENABLED' $logging></td>
> </tr>
====================================================================


> In /usr/lib/firewall/rules.pl I add an additional Iptables LOCATIONBLOCK
> log rule with the prefix 'LOCBLOCK-$location '. This is selectable from
> the Log dropped packets entry in the locationblock settings file.
>


/usr/lib/firewall/rules.pl
=====================================================================
76c76,77
<       "LOCATIONBLOCK_ENABLED" => "off"
---
> "LOCATIONBLOCK_ENABLED" => "off",
> "LOGGING_ENABLED" => "off",
719a721,725
>
> # add loggging if enabled from location-block.cgi
> if ($locationsettings{'LOGGING_ENABLED'} eq "on") {     
> run("$IPTABLES -A LOCATIONBLOCK -m set --match-set $location src -j LOG --log-prefix 'LOCBLOCK-$location '");
> }
=======================================================================
End of encapsulated message

Re: Logging Locationblock packets

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 17644 bytes --]

Hello,

> On 21 Nov 2024, at 00:08, Rob Brewer <ipfire-devel(a)grantura.co.uk> wrote:
> 
> Hi Michael and Adolf,
> 
> I'll combine my reply to both of you.
> 
> On Wed, 20 Nov 2024 15:21:28 +0000, Michael Tremer wrote:
> 
>> Hello,
>> 
>>> On 20 Nov 2024, at 13:36, Rob Brewer <ipfire-devel(a)grantura.co.uk>
>>> wrote:
>>> 
>>> On Tue, 19 Nov 2024 17:10:24 +0000, Michael Tremer wrote:
>>> 
>>> Hi Michael, thank for your response, I was beginning to think there was
>>> little interest in this topic.
>> 
>> I was traveling and so have a little bit of a back log as usual :)
>> 
>> I can’t speak for the others, though.
>> 
> 
> Not a problem, I should have said that I previously raised this a topic on 
> the forum and Abolf suggested that I raised it on the mailing list as 
> well.

Yes, I really prefer this here, because the forum is very overwhelming sometimes. There is too many threads, too many people… Mailing lists are great for development.

>> 
>>> See below...
>>> 
>>>> Hello Rob,
>>>> 
>>>> Good to hear from you again.
>>>> 
>>>>> On 9 Nov 2024, at 14:00, Rob Brewer <ipfire-devel(a)grantura.co.uk>
>>>>> wrote:
>>>>> 
>>>>> As  a contributor to the SANS ISC,  I email my IPTABLES logs hourly
>>>>> to the Dshield database, however the default IPFire IPTABLES logs are
>>>>> incomplete since packets dropped by Locationblock are not logged to
>>>>> syslog.
>>>> 
>>>> This is correct.
>>>> 
>>>>> As a workaround I edit the locationblock function of rules.pl to add
>>>>> a IPTABLES rule to log the dropped packets. This edit does not of
>>>>> course survive any Core-Updates that upgrade rules.pl and the edit
>>>>> needs to be re-applied.
>>>>> 
>>>>> In addition, unless the Locationblock packets are logged to syslog
>>>>> the data displayed in the Firewall logs sections are incomplete as
>>>>> the packets dropped by Location Block are not displayed.
>>>>> 
>>>>> I understand that the original intension of the Location Block
>>>>> feature was to reduce the amount of log messages on installations
>>>>> running on extremely cheap flash storage. I suspect that these
>>>>> installations are now almost zero especially since IPFire no longer
>>>>> supports 32 bit versions.
>>>>> 
>>>>> The obvious solution to this to make Location Block logging
>>>>> selectable as a firewall option and my quick look at making a patches
>>>>> for this would seem to be fairly simple task to rules.pl and and
>>>>> optionsfw.cgi.
>>>>> 
>>>>> I would be happy to provide the necessary patches should this be
>>>>> acceptable.
>>>> 
>>>> I think a couple of things have changed since we introduced this
>>>> feature and we have also learned a couple of lessons here:
>>>> 
>>>> * Yes it is true that this was introduced to keep the logs smaller,
>>>> but not necessarily to avoid using storage, mainly to keep them
>>>> readable by a human. I was always a bit doubtful about this and
>>>> believe that apart from this, there is no more value in a general
>>>> location blocking feature. The firewall will drop those packets
>>>> anyway.
>>>> 
>>>> 
>>> IPFire does a very good job in protecting the internal network from
>>> unwanted probes from the WAN, and I agree that for most users, location
>>> block does not enhance this protection.
>>> 
>>> As I run a mail server, I use locationblock as part of my filtering to
>>> prevent bad actors from accessing my mail server and I think would be
>>> useful for anyone who are running servers on their networks.
>>> 
>>> I don't like blocking traffic, but I have witnessed over the last few
>>> years a substantial rise in abuse from certain countries from where I
>>> would never expect to receive emails and locationblock works very well
>>> for me in filtering out those unwanted connections.
>>> 
>>> I currently have 21 countries blocked including XD Hostile networks.
>>> All of these blocked countries have been seen to support abusive
>>> networks.
>>> 
>>> It would be possible to generate individual rules for these networks
>>> which in some cases I do with Banish, but it is sometimes easier just
>>> to drop the whole country.
>> 
>> Easy yes. I am just not sure what we are actually getting out of this.
>> 
>> I don’t like to become political on this list and so I won’t, but I know
>> that some people hold the following beliefs:
>> 
>> * There are “bad” countries out there. In the western world those are
>> Russia, China, and more that fall into this kind of “category”. Nothing
>> is going to stop them to open up an account on AWS and suddenly they
>> have access to resources anywhere in the world and so actually, if those
>> players were up to anything “bad”, they can come out of any IP space
>> they want. There is also Tor which allows this.
>> 
> 
> I have been submitting my iptables logs to dshield for over 20 years and 
> in the early days I would be sending a lot less than 1000 a day (this was 
> with IPCop of course).
> 
> Over the last few years the number of firewall iptables logs have  
> escalated until it had reached dshield's daily limit of 100,000 /day  and 
> I started looking at ways to limit the logging of some of the 'bad actors' 
> firewall probes.

What do you use to send them the logs? The Python script or one of those alternatives?

>> * Looking at actual data, the bullet proof ISPs that annoy us are not
>> that far away. They are hosted in Europe and very prominently so in the
>> Netherlands - a “good” country.
>> 
> 
> I use Banish for this as it can block by ASN and can target bullet proof 
> ISPs not only in NL but also in the USA and elsewhere. Using the 'location 
> function' the blocked ASN's are updated when the database is updated.
> 
>> This is just an example to explain that we all have our idea of the
>> world in our head, but the Internet doesn’t know these borders.
>> Geopolitics is complicated and I don’t think that this is a tool that
>> helps to fight this kind of problem. I hope this is clear what I am
>> trying to say.
>> 
> 
> Yes my approach is to use multiple tools and Locationblock is just one of 
> the tools I am using. Maybe my approach is over the top though.

Probably not massively, but maybe slightly.

Wouldn’t it be nicer to have everything in one place instead of having multiple tools that work together in concert?

>> I personally trust in Spamhaus which is feeding data into our “Hostile
>> Networks” blocker and they are looking at the actual traffic that they
>> see from those subnets that they are blocking. They have a clear set of
>> rules and they don’t look just at the country and say “ah yes, the bad
>> guys again”.
>> 
> 
> For emails I also drop packets if listed in a number of DNS blackhole 
> lists using Auto-Dnsbl which I adapted for IPFire. See:
> 
> https://people.ipfire.org/~helix/auto-dnsbl/README
> for info.
> 
> I have been using this for about a year and can confirm that it is very 
> effective in removing spammers.
> 
>> I have stated this recently in the RPZ discussion that I feel that we
>> are lacking good sources for this kind of data. For blocking anything.
>> Loads of people believe “more is more” and to be fair… once you get to
>> blocking 90% of the internet you probably have blocked 90% of the noise.
>> However this won’t do anything for targeted attacks. I personally would
>> rather have more quality here.
>> 
> 
> I only use a few blocklists with ipblocklist and rely mainly on my locally 
> derived  data from failed smtp connections or from data from dshield's 
> daily report.
> 
>>>> * People seem to misunderstand this feature: The block as many
>>>> countries as they can click on and think that this makes things “more
>>>> secure”,
>>>> which of course it doesn’t.
>>>> 
>>>> 
>>> I think this is education and I suspect that very few of these people
>>> are running servers on their system and need location block.
>> 
>> I am personally not here to educate people. I am happy to help out and
>> give my opinion, but I suppose considering that people think as I have
>> outlined above, who am I to educate them?
>> 
> 
> Sorry I didn't express my reply well, I agree that IPFire users should be 
> responsible for their own actions.

Naturally. But we should also not give them sharp knives when we don’t expect them to use them properly.

>> It is just very regretful to me when people are ab-using a feature just
>> to “feel more safe”.
>> 
>>>> * What I personally like as a feature is to apply location filters to
>>>> individual rules. That allows to apply rate limits in a few places and
>>>> easily select some larger groups of potentially less interesting
>>>> traffic.
>>> 
>>> I agree and is similar to what I am doing for ports 25 and 587 although
>>> with location block.
>>> 
>>> 
>>>> * Back when this was introduced there was also a lot less use of
>>>> botnets as we know them now. Pretty much all recent DoS attacks I
>>>> remember have been coming from all over the world because systems
>>>> where compromised wherever they were. This renders the filter
>>>> absolutely useless.
>>> 
>>> For this, I am currently testing a dynamic block list on my system
>>> which adds an entry to an ipset when an ip address exceeds a threshold
>>> in the previous 24 hours. Currently there are about 7000 entries in
>>> this set which have been detected over the previous month.  Not for
>>> everyone because it needs a Whitelist to prevent some 'push'
>>> technologies such as Facebook and to protect from bulk inputs such as
>>> mailing lists. I also blacklist a large number of ip addresses which
>>> are associated with port scanners.
>>> 
>>> This would work well as a botnet filter but it needs an understanding
>>> of the blocking process, so it would not be suitable for everyone.
>> 
>> We have talked about this on the last team call and came to the
>> conclusion that the whole set of blocking features is a total mess.
>> There are several places where things can be blocked, they all heavily
>> overlap. Not to be going on about Spamhaus again, but the DROP list is
>> in the “Hostile Networks” feature, there is a special country code for
>> it, pretty much every IPS ruleset contains it, the IP blocklist feature
>> supports it. Now there are people throwing this all into DNS as well. I
>> believe this is all getting a little bit out of hand.
>> 
> 
> I don't use the IPS mainly due to bad experiences when trying to use Snort 
> with IPCop. I find it difficult to set up and select the appropriate 
> Rulsets for my requirements. I suspect the IPS would be very effective if 
> I could get it set up correctly.

This has massively improved in recent times. And it is FAST!

>> I personally would prefer *one* place and we really make this work well.
>> 
>> Some people find the IPS has too much overhead - I am beginning to
>> disagree.
>> 
>> Some people add all sorts of blacklists that are clearly for
>> experimental/research use and suddenly complain about things not working
>> anymore. It is hard to figure out what is actually blocking traffic when
>> there are about five places to look for. The result often is to disable
>> the IPS, then something else, and after a while all of this is turned
>> off and the firewall is configured to a bare minimum. We don’t want
>> people to run it like that.
>> 
>>>> * Infrastructure is becoming more centralised. Some people attempt to
>>>> block the US and pretty much nothing works any more. Well…
>>>> 
>>>> 
>>> :)
>>> 
>>>> Therefore I am happy to rethink this feature - personally I would even
>>>> remove it entirely, but I don’t like to handle the stress about this
>>>> now.
>>> 
>>> No I think you should keep it. Locationblock is very useful, but needs
>>> to be used in a considered way. I could live without it but its
>>> availability makes my life easier. Also being updated on a weekly basis
>>> makes keeping track of abusers easier.
>> 
>> We issue a new location database once a day, but the IPFire systems
>> update it only once a week. We deemed that as sufficient as there are
>> not that many changes from one version to another one.
>> 
>>>> I think this should log now. I don’t think there is any legitimate
>>>> reason to disable firewall logging at all. This is the paper trail
>>>> that you want when debugging anything or doing forensics on an attack.
>>>> Certainly bad flash is not a valid reason for me.
>>>> 
>>>> 
>>> I have started generating some patches to make 'locationblock logging'
>>> as a selectable feature and could post them here in the next day or so
>>> if of any use.
>> 
>> What would this basically do? Log it as a simple rule for all, or do you
>> include the selected country? I think the latter would make more sense.
>> 
> 
> OK - I have modified location-block.cgi to add an additional checkbox at 
> the top of the page titled "Log dropped packets", very similar to 
> ipblocklist.cgi. This adds a entry to the /var/ipfire/firewall/
> locationblock settings file ON or OFF.
> 
> In /usr/lib/firewall/rules.pl I add an additional Iptables LOCATIONBLOCK 
> log rule with the prefix 'LOCBLOCK-$location '. This is selectable from 
> the Log dropped packets entry in the locationblock settings file.
> 
> This generates kernel messages such as:
> 
> LOCBLOCK-CN IN=ppp0 OUT=ppp0 MAC= SRC=114.139.42.71 DST=81.187.175.103 
> LEN=60 TOS=0x00 PREC=0x00 TTL=47 ID=34664 DF PROTO=TCP SPT=46837 DPT=23 
> WINDOW=29040 RES=0x00 SYN URGP=0
> 
> As you can see the prefix contains the country code of the dropped 
> packets.

Cool. So I think coding-wise this is pretty straight forward.

The more crucial question for me is below…

>> What is the rationale to turn this off? Why don’t we turn this on for
>> everybody all of the time?
>> 
> 
> I would be happy with that I was suggesting that we use selectable logging 
> to be compatible with earlier IPFire versions.

What do we have to be compatible with? Just IPFire’s behaviour from last year?

I would be happy to break that if we have a good reason, because the world sometimes moves on. What we have released 10 years ago probably doesn’t have good defaults any more. Sometimes we have to revisit those and sometimes we even have to be pushy with some changes if we want some long-standing users to adopt them. IPFire users usually have no reason to do a reinstall and we don’t want to leave the people behind who have been running IPFire perfectly well for the last 10 years or even longer. So at least I would like to discuss what we push a little harder.

>>> In fact I think that IPFire's logging environment, which was inherited
>>> from IPCop needs a re-visit.
>> 
>> Yes. But this is probably not a thing I would be looking at in IPFire 2.
>> 
>> I am also happy to think a little bit more outside of the box and see
>> what we would fundamentally change in the future, but not implement in
>> IPFire 2.
>> 
>>> I was pleased to see that a while back there was a discussion of
>>> replacing sylogd with rsyslogd which I think would be a very useful
>>> upgrade and enable kernel messages, like on most current linux systems,
>>> to be logged to alternative log files instead of everything ending up
>>> in /var/log/ messages.
>> 
>> The currentl syslog server could do that. The problem is more that we
>> don’t have good tools to view the logs on the web UI. So there is simply
>> one place where most things go into and we will then filter it out when
>> users use the web UI. This is tremendously slow.
>> 
> 
> I did look at this a while back and believe that with syslog all kernel 
> messages must go to the same log file and cannot be filtered off as can be 
> done with rsyslog. I use this facility in rsyslog.d on my server to filter 
> off SYN Flood logs to a separate log file with ":msg,contains". I don't 
> believe that syslog has this option.

Different messages could be sent to different “facilities” and then we can filter them into individual files.

I would not want to split firewall hits into different files though. I just wouldn’t mind if firewall hits were no longer in /var/log/messages.

>>> The replacement with rsyslgd could also inhibit the "last message
>>> repeated x times" in syslog and which are ignored by IPFire's Log menu
>>> and causes erroneous results in the 'Count' values.
>> 
>> This might be a setting that we can turn off in the current
>> implementation.
>> 
> 
> I understand that this is hard coded into syslogd and can't be changed. In 
> rsyslog it defaults to "off".
> 
> https://www.rsyslog.com/doc/configuration/action/
> rsconf1_repeatedmsgreduction.html

What impact does this have for you? I never noticed any problem by summarising the logs.

>> We could also think about moving all firewall hits into
>> /var/log/firewall.log or something. That should free up enough in
>> /var/log/messages to make debugging easier.
>> 
> 
> That would be a sensible option, currently I filter out firewall hits from 
> the messages.log with a perl regex. 
> 
>>>> I would be happy to hear some more opinions on this from the list.
>>>> 
>>>> -Michael
>>> 
>>> I hope these comments are useful and would be pleased to help where I
>>> can.
>> 
>> Of course :)
>> 
>> -Michael
>> 
>> 
>>> 
>>> Rob

-Michael

Re: Logging Locationblock packets

[Rob Brewer]

[-- Attachment #1: Type: text/plain, Size: 20721 bytes --]

On Sat, 23 Nov 2024 10:26:07 +0100, Michael Tremer wrote:

> Hello,
> 
>> On 21 Nov 2024, at 00:08, Rob Brewer <ipfire-devel(a)grantura.co.uk> 
wrote:
>> 
>> Hi Michael and Adolf,
>> 
>> I'll combine my reply to both of you.
>> 
>> On Wed, 20 Nov 2024 15:21:28 +0000, Michael Tremer wrote:
>> 
>>> Hello,
>>> 
>>>> On 20 Nov 2024, at 13:36, Rob Brewer <ipfire-devel(a)grantura.co.uk>
>>>> wrote:
>>>> 
>>>> On Tue, 19 Nov 2024 17:10:24 +0000, Michael Tremer wrote:
>>>> 
>>>> Hi Michael, thank for your response, I was beginning to think there 
was
>>>> little interest in this topic.
>>> 
>>> I was traveling and so have a little bit of a back log as usual :)
>>> 
>>> I can’t speak for the others, though.
>>> 
>> 
>> Not a problem, I should have said that I previously raised this a 
topic 
on 
>> the forum and Adolf suggested that I raised it on the mailing list as 
>> well.
> 
> Yes, I really prefer this here, because the forum is very overwhelming 
sometimes. There is too many threads, too many people… Mailing lists are 
great for development.

That's fine, one query though you don't seem to snip older parts of the 
post, is this your preference?

> 
>>> 
>>>> See below...
>>>> 
>>>>> Hello Rob,
>>>>> 
>>>>> Good to hear from you again.
>>>>> 
>>>>>> On 9 Nov 2024, at 14:00, Rob Brewer <ipfire-devel(a)grantura.co.uk>
>>>>>> wrote:
>>>>>> 
>>>>>> As  a contributor to the SANS ISC,  I email my IPTABLES logs hourly
>>>>>> to the Dshield database, however the default IPFire IPTABLES logs 
are
>>>>>> incomplete since packets dropped by Locationblock are not logged to
>>>>>> syslog.
>>>>> 
>>>>> This is correct.
>>>>> 
>>>>>> As a workaround I edit the locationblock function of rules.pl to 
add
>>>>>> a IPTABLES rule to log the dropped packets. This edit does not of
>>>>>> course survive any Core-Updates that upgrade rules.pl and the edit
>>>>>> needs to be re-applied.
>>>>>> 
>>>>>> In addition, unless the Locationblock packets are logged to syslog
>>>>>> the data displayed in the Firewall logs sections are incomplete as
>>>>>> the packets dropped by Location Block are not displayed.
>>>>>> 
>>>>>> I understand that the original intension of the Location Block
>>>>>> feature was to reduce the amount of log messages on installations
>>>>>> running on extremely cheap flash storage. I suspect that these
>>>>>> installations are now almost zero especially since IPFire no longer
>>>>>> supports 32 bit versions.
>>>>>> 
>>>>>> The obvious solution to this to make Location Block logging
>>>>>> selectable as a firewall option and my quick look at making a 
patches
>>>>>> for this would seem to be fairly simple task to rules.pl and and
>>>>>> optionsfw.cgi.
>>>>>> 
>>>>>> I would be happy to provide the necessary patches should this be
>>>>>> acceptable.
>>>>> 
>>>>> I think a couple of things have changed since we introduced this
>>>>> feature and we have also learned a couple of lessons here:
>>>>> 
>>>>> * Yes it is true that this was introduced to keep the logs smaller,
>>>>> but not necessarily to avoid using storage, mainly to keep them
>>>>> readable by a human. I was always a bit doubtful about this and
>>>>> believe that apart from this, there is no more value in a general
>>>>> location blocking feature. The firewall will drop those packets
>>>>> anyway.
>>>>> 
>>>>> 
>>>> IPFire does a very good job in protecting the internal network from
>>>> unwanted probes from the WAN, and I agree that for most users, 
location
>>>> block does not enhance this protection.
>>>> 
>>>> As I run a mail server, I use locationblock as part of my filtering 
to
>>>> prevent bad actors from accessing my mail server and I think would be
>>>> useful for anyone who are running servers on their networks.
>>>> 
>>>> I don't like blocking traffic, but I have witnessed over the last few
>>>> years a substantial rise in abuse from certain countries from where I
>>>> would never expect to receive emails and locationblock works very 
well
>>>> for me in filtering out those unwanted connections.
>>>> 
>>>> I currently have 21 countries blocked including XD Hostile networks.
>>>> All of these blocked countries have been seen to support abusive
>>>> networks.
>>>> 
>>>> It would be possible to generate individual rules for these networks
>>>> which in some cases I do with Banish, but it is sometimes easier just
>>>> to drop the whole country.
>>> 
>>> Easy yes. I am just not sure what we are actually getting out of this.
>>> 
>>> I don’t like to become political on this list and so I won’t, but I 
know
>>> that some people hold the following beliefs:
>>> 
>>> * There are “bad” countries out there. In the western world those are
>>> Russia, China, and more that fall into this kind of “category”. 
Nothing
>>> is going to stop them to open up an account on AWS and suddenly they
>>> have access to resources anywhere in the world and so actually, if 
those
>>> players were up to anything “bad”, they can come out of any IP space
>>> they want. There is also Tor which allows this.
>>> 
>> 
>> I have been submitting my iptables logs to dshield for over 20 years 
and 
>> in the early days I would be sending a lot less than 1000 a day (this 
was 
>> with IPCop of course).
>> 
>> Over the last few years the number of firewall iptables logs have  
>> escalated until it had reached dshield's daily limit of 100,000 /day  
and 
>> I started looking at ways to limit the logging of some of the 'bad 
actors' 
>> firewall probes.
> 
> What do you use to send them the logs? The Python script or one of 
those 
alternatives?

I use a modified Perl script (iptables.pl) which is about 20 years old was 
originally written for Smoothwall and Ipcop1. This was part of the Logsend 
add-on and I modified for IPCop2 about 15 years ago. 

Now modified for IPFire and using DMA email instead of the original 
sendEmail. I've uploaded my version to people.ipfire.org/~helix/dshield. 

I had intended to write an install/uninstall script but is in the todo 
list. Do you think it may be of interest to anyone if I do?

> 
>>> * Looking at actual data, the bullet proof ISPs that annoy us are not
>>> that far away. They are hosted in Europe and very prominently so in 
the
>>> Netherlands - a “good” country.
>>> 
>> 
>> I use Banish for this as it can block by ASN and can target bullet 
proof 
>> ISPs not only in NL but also in the USA and elsewhere. Using the 
'location 
>> function' the blocked ASN's are updated when the database is updated.
>> 
>>> This is just an example to explain that we all have our idea of the
>>> world in our head, but the Internet doesn’t know these borders.
>>> Geopolitics is complicated and I don’t think that this is a tool that
>>> helps to fight this kind of problem. I hope this is clear what I am
>>> trying to say.
>>> 
>> 
>> Yes my approach is to use multiple tools and Locationblock is just one 
of 
>> the tools I am using. Maybe my approach is over the top though.
> 
> Probably not massively, but maybe slightly.
> 
> Wouldn’t it be nicer to have everything in one place instead of having 
multiple tools that work together in concert?
> 

I think that would be a lot of work, would you consider that for IPFire2 
or 3?

>>> I personally trust in Spamhaus which is feeding data into our “Hostile
>>> Networks” blocker and they are looking at the actual traffic that they
>>> see from those subnets that they are blocking. They have a clear set 
of
>>> rules and they don’t look just at the country and say “ah yes, the bad
>>> guys again”.
>>> 
>> 
>> For emails I also drop packets if listed in a number of DNS blackhole 
>> lists using Auto-Dnsbl which I adapted for IPFire. See:
>> 
>> https://people.ipfire.org/~helix/auto-dnsbl/README
>> for info.
>> 
>> I have been using this for about a year and can confirm that it is 
very 
>> effective in removing spammers.
>> 
>>> I have stated this recently in the RPZ discussion that I feel that we
>>> are lacking good sources for this kind of data. For blocking anything.
>>> Loads of people believe “more is more” and to be fair… once you get to
>>> blocking 90% of the internet you probably have blocked 90% of the 
noise.
>>> However this won’t do anything for targeted attacks. I personally 
would
>>> rather have more quality here.
>>> 
>> 
>> I only use a few blocklists with ipblocklist and rely mainly on my 
locally 
>> derived  data from failed smtp connections or from data from dshield's 
>> daily report.
>> 
>>>>> * People seem to misunderstand this feature: The block as many
>>>>> countries as they can click on and think that this makes things 
“more
>>>>> secure”,
>>>>> which of course it doesn’t.
>>>>> 
>>>>> 
>>>> I think this is education and I suspect that very few of these people
>>>> are running servers on their system and need location block.
>>> 
>>> I am personally not here to educate people. I am happy to help out and
>>> give my opinion, but I suppose considering that people think as I have
>>> outlined above, who am I to educate them?
>>> 
>> 
>> Sorry I didn't express my reply well, I agree that IPFire users should 
be 
>> responsible for their own actions.
> 
> Naturally. But we should also not give them sharp knives when we don’t 
expect them to use them properly.
> 
>>> It is just very regretful to me when people are ab-using a feature 
just
>>> to “feel more safe”.
>>> 
>>>>> * What I personally like as a feature is to apply location filters 
to
>>>>> individual rules. That allows to apply rate limits in a few places 
and
>>>>> easily select some larger groups of potentially less interesting
>>>>> traffic.
>>>> 
>>>> I agree and is similar to what I am doing for ports 25 and 587 
although
>>>> with location block.
>>>> 
>>>> 
>>>>> * Back when this was introduced there was also a lot less use of
>>>>> botnets as we know them now. Pretty much all recent DoS attacks I
>>>>> remember have been coming from all over the world because systems
>>>>> where compromised wherever they were. This renders the filter
>>>>> absolutely useless.
>>>> 
>>>> For this, I am currently testing a dynamic block list on my system
>>>> which adds an entry to an ipset when an ip address exceeds a 
threshold
>>>> in the previous 24 hours. Currently there are about 7000 entries in
>>>> this set which have been detected over the previous month.  Not for
>>>> everyone because it needs a Whitelist to prevent some 'push'
>>>> technologies such as Facebook and to protect from bulk inputs such as
>>>> mailing lists. I also blacklist a large number of ip addresses which
>>>> are associated with port scanners.
>>>> 
>>>> This would work well as a botnet filter but it needs an understanding
>>>> of the blocking process, so it would not be suitable for everyone.
>>> 
>>> We have talked about this on the last team call and came to the
>>> conclusion that the whole set of blocking features is a total mess.
>>> There are several places where things can be blocked, they all heavily
>>> overlap. Not to be going on about Spamhaus again, but the DROP list is
>>> in the “Hostile Networks” feature, there is a special country code for
>>> it, pretty much every IPS ruleset contains it, the IP blocklist 
feature
>>> supports it. Now there are people throwing this all into DNS as well. 
I
>>> believe this is all getting a little bit out of hand.
>>> 
>> 
>> I don't use the IPS mainly due to bad experiences when trying to use 
Snort 
>> with IPCop. I find it difficult to set up and select the appropriate 
>> Rulsets for my requirements. I suspect the IPS would be very effective 
if 
>> I could get it set up correctly.
> 
> This has massively improved in recent times. And it is FAST!
> 

I'll have another look, but my current set-up is working well, so I'm 
reluctant to change.

>>> I personally would prefer *one* place and we really make this work 
well.
>>> 
>>> Some people find the IPS has too much overhead - I am beginning to
>>> disagree.
>>> 
>>> Some people add all sorts of blacklists that are clearly for
>>> experimental/research use and suddenly complain about things not 
working
>>> anymore. It is hard to figure out what is actually blocking traffic 
when
>>> there are about five places to look for. The result often is to 
disable
>>> the IPS, then something else, and after a while all of this is turned
>>> off and the firewall is configured to a bare minimum. We don’t want
>>> people to run it like that.
>>> 
>>>>> * Infrastructure is becoming more centralised. Some people attempt 
to
>>>>> block the US and pretty much nothing works any more. Well…
>>>>> 
>>>>> 
>>>> :)
>>>> 
>>>>> Therefore I am happy to rethink this feature - personally I would 
even
>>>>> remove it entirely, but I don’t like to handle the stress about this
>>>>> now.
>>>> 
>>>> No I think you should keep it. Locationblock is very useful, but 
needs
>>>> to be used in a considered way. I could live without it but its
>>>> availability makes my life easier. Also being updated on a weekly 
basis
>>>> makes keeping track of abusers easier.
>>> 
>>> We issue a new location database once a day, but the IPFire systems
>>> update it only once a week. We deemed that as sufficient as there are
>>> not that many changes from one version to another one.
>>> 
>>>>> I think this should log now. I don’t think there is any legitimate
>>>>> reason to disable firewall logging at all. This is the paper trail
>>>>> that you want when debugging anything or doing forensics on an 
attack.
>>>>> Certainly bad flash is not a valid reason for me.
>>>>> 
>>>>> 
>>>> I have started generating some patches to make 'locationblock 
logging'
>>>> as a selectable feature and could post them here in the next day or 
so
>>>> if of any use.
>>> 
>>> What would this basically do? Log it as a simple rule for all, or do 
you
>>> include the selected country? I think the latter would make more 
sense.
>>> 
>> 
>> OK - I have modified location-block.cgi to add an additional checkbox 
at 
>> the top of the page titled "Log dropped packets", very similar to 
>> ipblocklist.cgi. This adds a entry to the /var/ipfire/firewall/
>> locationblock settings file ON or OFF.
>> 
>> In /usr/lib/firewall/rules.pl I add an additional Iptables 
LOCATIONBLOCK 
>> log rule with the prefix 'LOCBLOCK-$location '. This is selectable 
from 
>> the Log dropped packets entry in the locationblock settings file.
>> 
>> This generates kernel messages such as:
>> 
>> LOCBLOCK-CN IN=ppp0 OUT=ppp0 MAC= SRC=114.139.42.71 DST=81.187.175.103 
>> LEN=60 TOS=0x00 PREC=0x00 TTL=47 ID=34664 DF PROTO=TCP SPT=46837 
DPT=23 
>> WINDOW=29040 RES=0x00 SYN URGP=0
>> 
>> As you can see the prefix contains the country code of the dropped 
>> packets.
> 
> Cool. So I think coding-wise this is pretty straight forward.
> 
> The more crucial question for me is below…
> 

Do have a look at the patches I posted and let me have any comments. If 
it's not needed and we can add locationblock logging without selection 
then that's what I have been doing since LB was first implemented.

>>> What is the rationale to turn this off? Why don’t we turn this on for
>>> everybody all of the time?
>>> 
>> 
>> I would be happy with that I was suggesting that we use selectable 
logging 
>> to be compatible with earlier IPFire versions.
> 
> What do we have to be compatible with? Just IPFire’s behaviour from 
last 
year?
> 
> I would be happy to break that if we have a good reason, because the 
world sometimes moves on. What we have released 10 years ago probably 
doesn’t have good defaults any more. Sometimes we have to revisit those 
and sometimes we even have to be pushy with some changes if we want some 
long-standing users to adopt them. IPFire users usually have no reason to 
do a reinstall and we don’t want to leave the people behind who have been 
running IPFire perfectly well for the last 10 years or even longer. So at 
least I would like to discuss what we push a little harder.
> 

As Adolf pointed out in the Forum,the WIKI stated the rational for not 
logging was to protect flash memory from excessive use. I don't think that 
is appropriate now.

>>>> In fact I think that IPFire's logging environment, which was 
inherited
>>>> from IPCop needs a re-visit.
>>> 
>>> Yes. But this is probably not a thing I would be looking at in IPFire 
2.
>>> 
>>> I am also happy to think a little bit more outside of the box and see
>>> what we would fundamentally change in the future, but not implement in
>>> IPFire 2.
>>> 
>>>> I was pleased to see that a while back there was a discussion of
>>>> replacing sylogd with rsyslogd which I think would be a very useful
>>>> upgrade and enable kernel messages, like on most current linux 
systems,
>>>> to be logged to alternative log files instead of everything ending up
>>>> in /var/log/ messages.
>>> 
>>> The currentl syslog server could do that. The problem is more that we
>>> don’t have good tools to view the logs on the web UI. So there is 
simply
>>> one place where most things go into and we will then filter it out 
when
>>> users use the web UI. This is tremendously slow.
>>> 
>> 
>> I did look at this a while back and believe that with syslog all 
kernel 
>> messages must go to the same log file and cannot be filtered off as 
can 
be 
>> done with rsyslog. I use this facility in rsyslog.d on my server to 
filter 
>> off SYN Flood logs to a separate log file with ":msg,contains". I 
don't 
>> believe that syslog has this option.
> 
> Different messages could be sent to different “facilities” and then we 
can filter them into individual files.
> 
> I would not want to split firewall hits into different files though. I 
just wouldn’t mind if firewall hits were no longer in /var/log/messages.


You need to be careful with the LOG* menus which would need to look in 2 
or more places for historical logs and so the cgi's would need to be 
modified.

I would however welcome kernel message filtering since it would allow my 
use of a separate log directory for my auto-dnsbl program to monitor for 
non-dropped incoming SMTP connections without appearing in 'messages' log 
file. 


> 
>>>> The replacement with rsyslgd could also inhibit the "last message
>>>> repeated x times" in syslog and which are ignored by IPFire's Log 
menu
>>>> and causes erroneous results in the 'Count' values.
>>> 
>>> This might be a setting that we can turn off in the current
>>> implementation.
>>> 
>> 
>> I understand that this is hard coded into syslogd and can't be 
changed. 
In 
>> rsyslog it defaults to "off".
>> 
>> https://www.rsyslog.com/doc/configuration/action/
>> rsconf1_repeatedmsgreduction.html
> 
> What impact does this have for you? I never noticed any problem by 
summarising the logs.

I came across this and it illustrates the point well:

https://dcid.me/notes/syslog-last-message-repeated-x-times-rant

and makes the point that kernel messages can be delayed by several seconds 
and could affect any DOS filtering that may be in use.

The dshield iptables.pl script tries to recover the lost repeated messages 
but looses the time-stamp. 

When comparing the dshield logs with IPFire's logs there is invariably a 
discrepancy with the IPFire's count sometimes considerably less especially 
with multiple port-scans over a minute or so.

Any chance of looking again at rsyslog? IPFire is the only linux system I 
know of still using syslog.

I seem to remember that Adolf had completed all the necessary changes to 
introduce rsyslog a years or so ago but was not implemented,
 

> 
>>> We could also think about moving all firewall hits into
>>> /var/log/firewall.log or something. That should free up enough in
>>> /var/log/messages to make debugging easier.
>>> 
>> 
>> That would be a sensible option, currently I filter out firewall hits 
from 
>> the messages.log with a perl regex. 
>> 
>>>>> I would be happy to hear some more opinions on this from the list.
>>>>> 
>>>>> -Michael
>>>> 
>>>> I hope these comments are useful and would be pleased to help where I
>>>> can.
>>> 
>>> Of course :)
>>> 
>>> -Michael
>>> 
>>> 
>>>> 
>>>> Rob
> 
> -Michael

Rob

Re: Logging Locationblock packets

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 21175 bytes --]

Hi Rob,

On 23/11/2024 15:36, Rob Brewer wrote:
> On Sat, 23 Nov 2024 10:26:07 +0100, Michael Tremer wrote:
>
>> Hello,
>>
>>> On 21 Nov 2024, at 00:08, Rob Brewer <ipfire-devel(a)grantura.co.uk>
> wrote:
>>> Hi Michael and Adolf,
>>>
>>> I'll combine my reply to both of you.
>>>
>>> On Wed, 20 Nov 2024 15:21:28 +0000, Michael Tremer wrote:
>>>
>>>> Hello,
>>>>
>>>>> On 20 Nov 2024, at 13:36, Rob Brewer <ipfire-devel(a)grantura.co.uk>
>>>>> wrote:
>>>>>
>>>>> On Tue, 19 Nov 2024 17:10:24 +0000, Michael Tremer wrote:
>>>>>
>>>>> Hi Michael, thank for your response, I was beginning to think there
> was
>>>>> little interest in this topic.
>>>> I was traveling and so have a little bit of a back log as usual :)
>>>>
>>>> I can’t speak for the others, though.
>>>>
>>> Not a problem, I should have said that I previously raised this a
> topic
> on
>>> the forum and Adolf suggested that I raised it on the mailing list as
>>> well.
>> Yes, I really prefer this here, because the forum is very overwhelming
> sometimes. There is too many threads, too many people… Mailing lists are
> great for development.
>
> That's fine, one query though you don't seem to snip older parts of the
> post, is this your preference?
>
>>>>> See below...
>>>>>
>>>>>> Hello Rob,
>>>>>>
>>>>>> Good to hear from you again.
>>>>>>
>>>>>>> On 9 Nov 2024, at 14:00, Rob Brewer <ipfire-devel(a)grantura.co.uk>
>>>>>>> wrote:
>>>>>>>
>>>>>>> As  a contributor to the SANS ISC,  I email my IPTABLES logs hourly
>>>>>>> to the Dshield database, however the default IPFire IPTABLES logs
> are
>>>>>>> incomplete since packets dropped by Locationblock are not logged to
>>>>>>> syslog.
>>>>>> This is correct.
>>>>>>
>>>>>>> As a workaround I edit the locationblock function of rules.pl to
> add
>>>>>>> a IPTABLES rule to log the dropped packets. This edit does not of
>>>>>>> course survive any Core-Updates that upgrade rules.pl and the edit
>>>>>>> needs to be re-applied.
>>>>>>>
>>>>>>> In addition, unless the Locationblock packets are logged to syslog
>>>>>>> the data displayed in the Firewall logs sections are incomplete as
>>>>>>> the packets dropped by Location Block are not displayed.
>>>>>>>
>>>>>>> I understand that the original intension of the Location Block
>>>>>>> feature was to reduce the amount of log messages on installations
>>>>>>> running on extremely cheap flash storage. I suspect that these
>>>>>>> installations are now almost zero especially since IPFire no longer
>>>>>>> supports 32 bit versions.
>>>>>>>
>>>>>>> The obvious solution to this to make Location Block logging
>>>>>>> selectable as a firewall option and my quick look at making a
> patches
>>>>>>> for this would seem to be fairly simple task to rules.pl and and
>>>>>>> optionsfw.cgi.
>>>>>>>
>>>>>>> I would be happy to provide the necessary patches should this be
>>>>>>> acceptable.
>>>>>> I think a couple of things have changed since we introduced this
>>>>>> feature and we have also learned a couple of lessons here:
>>>>>>
>>>>>> * Yes it is true that this was introduced to keep the logs smaller,
>>>>>> but not necessarily to avoid using storage, mainly to keep them
>>>>>> readable by a human. I was always a bit doubtful about this and
>>>>>> believe that apart from this, there is no more value in a general
>>>>>> location blocking feature. The firewall will drop those packets
>>>>>> anyway.
>>>>>>
>>>>>>
>>>>> IPFire does a very good job in protecting the internal network from
>>>>> unwanted probes from the WAN, and I agree that for most users,
> location
>>>>> block does not enhance this protection.
>>>>>
>>>>> As I run a mail server, I use locationblock as part of my filtering
> to
>>>>> prevent bad actors from accessing my mail server and I think would be
>>>>> useful for anyone who are running servers on their networks.
>>>>>
>>>>> I don't like blocking traffic, but I have witnessed over the last few
>>>>> years a substantial rise in abuse from certain countries from where I
>>>>> would never expect to receive emails and locationblock works very
> well
>>>>> for me in filtering out those unwanted connections.
>>>>>
>>>>> I currently have 21 countries blocked including XD Hostile networks.
>>>>> All of these blocked countries have been seen to support abusive
>>>>> networks.
>>>>>
>>>>> It would be possible to generate individual rules for these networks
>>>>> which in some cases I do with Banish, but it is sometimes easier just
>>>>> to drop the whole country.
>>>> Easy yes. I am just not sure what we are actually getting out of this.
>>>>
>>>> I don’t like to become political on this list and so I won’t, but I
> know
>>>> that some people hold the following beliefs:
>>>>
>>>> * There are “bad” countries out there. In the western world those are
>>>> Russia, China, and more that fall into this kind of “category”.
> Nothing
>>>> is going to stop them to open up an account on AWS and suddenly they
>>>> have access to resources anywhere in the world and so actually, if
> those
>>>> players were up to anything “bad”, they can come out of any IP space
>>>> they want. There is also Tor which allows this.
>>>>
>>> I have been submitting my iptables logs to dshield for over 20 years
> and
>>> in the early days I would be sending a lot less than 1000 a day (this
> was
>>> with IPCop of course).
>>>
>>> Over the last few years the number of firewall iptables logs have
>>> escalated until it had reached dshield's daily limit of 100,000 /day
> and
>>> I started looking at ways to limit the logging of some of the 'bad
> actors'
>>> firewall probes.
>> What do you use to send them the logs? The Python script or one of
> those
> alternatives?
>
> I use a modified Perl script (iptables.pl) which is about 20 years old was
> originally written for Smoothwall and Ipcop1. This was part of the Logsend
> add-on and I modified for IPCop2 about 15 years ago.
>
> Now modified for IPFire and using DMA email instead of the original
> sendEmail. I've uploaded my version to people.ipfire.org/~helix/dshield.
>
> I had intended to write an install/uninstall script but is in the todo
> list. Do you think it may be of interest to anyone if I do?
>
>>>> * Looking at actual data, the bullet proof ISPs that annoy us are not
>>>> that far away. They are hosted in Europe and very prominently so in
> the
>>>> Netherlands - a “good” country.
>>>>
>>> I use Banish for this as it can block by ASN and can target bullet
> proof
>>> ISPs not only in NL but also in the USA and elsewhere. Using the
> 'location
>>> function' the blocked ASN's are updated when the database is updated.
>>>
>>>> This is just an example to explain that we all have our idea of the
>>>> world in our head, but the Internet doesn’t know these borders.
>>>> Geopolitics is complicated and I don’t think that this is a tool that
>>>> helps to fight this kind of problem. I hope this is clear what I am
>>>> trying to say.
>>>>
>>> Yes my approach is to use multiple tools and Locationblock is just one
> of
>>> the tools I am using. Maybe my approach is over the top though.
>> Probably not massively, but maybe slightly.
>>
>> Wouldn’t it be nicer to have everything in one place instead of having
> multiple tools that work together in concert?
> I think that would be a lot of work, would you consider that for IPFire2
> or 3?
>
>>>> I personally trust in Spamhaus which is feeding data into our “Hostile
>>>> Networks” blocker and they are looking at the actual traffic that they
>>>> see from those subnets that they are blocking. They have a clear set
> of
>>>> rules and they don’t look just at the country and say “ah yes, the bad
>>>> guys again”.
>>>>
>>> For emails I also drop packets if listed in a number of DNS blackhole
>>> lists using Auto-Dnsbl which I adapted for IPFire. See:
>>>
>>> https://people.ipfire.org/~helix/auto-dnsbl/README
>>> for info.
>>>
>>> I have been using this for about a year and can confirm that it is
> very
>>> effective in removing spammers.
>>>
>>>> I have stated this recently in the RPZ discussion that I feel that we
>>>> are lacking good sources for this kind of data. For blocking anything.
>>>> Loads of people believe “more is more” and to be fair… once you get to
>>>> blocking 90% of the internet you probably have blocked 90% of the
> noise.
>>>> However this won’t do anything for targeted attacks. I personally
> would
>>>> rather have more quality here.
>>>>
>>> I only use a few blocklists with ipblocklist and rely mainly on my
> locally
>>> derived  data from failed smtp connections or from data from dshield's
>>> daily report.
>>>
>>>>>> * People seem to misunderstand this feature: The block as many
>>>>>> countries as they can click on and think that this makes things
> “more
>>>>>> secure”,
>>>>>> which of course it doesn’t.
>>>>>>
>>>>>>
>>>>> I think this is education and I suspect that very few of these people
>>>>> are running servers on their system and need location block.
>>>> I am personally not here to educate people. I am happy to help out and
>>>> give my opinion, but I suppose considering that people think as I have
>>>> outlined above, who am I to educate them?
>>>>
>>> Sorry I didn't express my reply well, I agree that IPFire users should
> be
>>> responsible for their own actions.
>> Naturally. But we should also not give them sharp knives when we don’t
> expect them to use them properly.
>>>> It is just very regretful to me when people are ab-using a feature
> just
>>>> to “feel more safe”.
>>>>
>>>>>> * What I personally like as a feature is to apply location filters
> to
>>>>>> individual rules. That allows to apply rate limits in a few places
> and
>>>>>> easily select some larger groups of potentially less interesting
>>>>>> traffic.
>>>>> I agree and is similar to what I am doing for ports 25 and 587
> although
>>>>> with location block.
>>>>>
>>>>>
>>>>>> * Back when this was introduced there was also a lot less use of
>>>>>> botnets as we know them now. Pretty much all recent DoS attacks I
>>>>>> remember have been coming from all over the world because systems
>>>>>> where compromised wherever they were. This renders the filter
>>>>>> absolutely useless.
>>>>> For this, I am currently testing a dynamic block list on my system
>>>>> which adds an entry to an ipset when an ip address exceeds a
> threshold
>>>>> in the previous 24 hours. Currently there are about 7000 entries in
>>>>> this set which have been detected over the previous month.  Not for
>>>>> everyone because it needs a Whitelist to prevent some 'push'
>>>>> technologies such as Facebook and to protect from bulk inputs such as
>>>>> mailing lists. I also blacklist a large number of ip addresses which
>>>>> are associated with port scanners.
>>>>>
>>>>> This would work well as a botnet filter but it needs an understanding
>>>>> of the blocking process, so it would not be suitable for everyone.
>>>> We have talked about this on the last team call and came to the
>>>> conclusion that the whole set of blocking features is a total mess.
>>>> There are several places where things can be blocked, they all heavily
>>>> overlap. Not to be going on about Spamhaus again, but the DROP list is
>>>> in the “Hostile Networks” feature, there is a special country code for
>>>> it, pretty much every IPS ruleset contains it, the IP blocklist
> feature
>>>> supports it. Now there are people throwing this all into DNS as well.
> I
>>>> believe this is all getting a little bit out of hand.
>>>>
>>> I don't use the IPS mainly due to bad experiences when trying to use
> Snort
>>> with IPCop. I find it difficult to set up and select the appropriate
>>> Rulsets for my requirements. I suspect the IPS would be very effective
> if
>>> I could get it set up correctly.
>> This has massively improved in recent times. And it is FAST!
>>
> I'll have another look, but my current set-up is working well, so I'm
> reluctant to change.
>
>>>> I personally would prefer *one* place and we really make this work
> well.
>>>> Some people find the IPS has too much overhead - I am beginning to
>>>> disagree.
>>>>
>>>> Some people add all sorts of blacklists that are clearly for
>>>> experimental/research use and suddenly complain about things not
> working
>>>> anymore. It is hard to figure out what is actually blocking traffic
> when
>>>> there are about five places to look for. The result often is to
> disable
>>>> the IPS, then something else, and after a while all of this is turned
>>>> off and the firewall is configured to a bare minimum. We don’t want
>>>> people to run it like that.
>>>>
>>>>>> * Infrastructure is becoming more centralised. Some people attempt
> to
>>>>>> block the US and pretty much nothing works any more. Well…
>>>>>>
>>>>>>
>>>>> :)
>>>>>
>>>>>> Therefore I am happy to rethink this feature - personally I would
> even
>>>>>> remove it entirely, but I don’t like to handle the stress about this
>>>>>> now.
>>>>> No I think you should keep it. Locationblock is very useful, but
> needs
>>>>> to be used in a considered way. I could live without it but its
>>>>> availability makes my life easier. Also being updated on a weekly
> basis
>>>>> makes keeping track of abusers easier.
>>>> We issue a new location database once a day, but the IPFire systems
>>>> update it only once a week. We deemed that as sufficient as there are
>>>> not that many changes from one version to another one.
>>>>
>>>>>> I think this should log now. I don’t think there is any legitimate
>>>>>> reason to disable firewall logging at all. This is the paper trail
>>>>>> that you want when debugging anything or doing forensics on an
> attack.
>>>>>> Certainly bad flash is not a valid reason for me.
>>>>>>
>>>>>>
>>>>> I have started generating some patches to make 'locationblock
> logging'
>>>>> as a selectable feature and could post them here in the next day or
> so
>>>>> if of any use.
>>>> What would this basically do? Log it as a simple rule for all, or do
> you
>>>> include the selected country? I think the latter would make more
> sense.
>>> OK - I have modified location-block.cgi to add an additional checkbox
> at
>>> the top of the page titled "Log dropped packets", very similar to
>>> ipblocklist.cgi. This adds a entry to the /var/ipfire/firewall/
>>> locationblock settings file ON or OFF.
>>>
>>> In /usr/lib/firewall/rules.pl I add an additional Iptables
> LOCATIONBLOCK
>>> log rule with the prefix 'LOCBLOCK-$location '. This is selectable
> from
>>> the Log dropped packets entry in the locationblock settings file.
>>>
>>> This generates kernel messages such as:
>>>
>>> LOCBLOCK-CN IN=ppp0 OUT=ppp0 MAC= SRC=114.139.42.71 DST=81.187.175.103
>>> LEN=60 TOS=0x00 PREC=0x00 TTL=47 ID=34664 DF PROTO=TCP SPT=46837
> DPT=23
>>> WINDOW=29040 RES=0x00 SYN URGP=0
>>>
>>> As you can see the prefix contains the country code of the dropped
>>> packets.
>> Cool. So I think coding-wise this is pretty straight forward.
>>
>> The more crucial question for me is below…
>>
> Do have a look at the patches I posted and let me have any comments. If
> it's not needed and we can add locationblock logging without selection
> then that's what I have been doing since LB was first implemented.
>
>>>> What is the rationale to turn this off? Why don’t we turn this on for
>>>> everybody all of the time?
>>>>
>>> I would be happy with that I was suggesting that we use selectable
> logging
>>> to be compatible with earlier IPFire versions.
>> What do we have to be compatible with? Just IPFire’s behaviour from
> last
> year?
>> I would be happy to break that if we have a good reason, because the
> world sometimes moves on. What we have released 10 years ago probably
> doesn’t have good defaults any more. Sometimes we have to revisit those
> and sometimes we even have to be pushy with some changes if we want some
> long-standing users to adopt them. IPFire users usually have no reason to
> do a reinstall and we don’t want to leave the people behind who have been
> running IPFire perfectly well for the last 10 years or even longer. So at
> least I would like to discuss what we push a little harder.
> As Adolf pointed out in the Forum,the WIKI stated the rational for not
> logging was to protect flash memory from excessive use. I don't think that
> is appropriate now.
>
>>>>> In fact I think that IPFire's logging environment, which was
> inherited
>>>>> from IPCop needs a re-visit.
>>>> Yes. But this is probably not a thing I would be looking at in IPFire
> 2.
>>>> I am also happy to think a little bit more outside of the box and see
>>>> what we would fundamentally change in the future, but not implement in
>>>> IPFire 2.
>>>>
>>>>> I was pleased to see that a while back there was a discussion of
>>>>> replacing sylogd with rsyslogd which I think would be a very useful
>>>>> upgrade and enable kernel messages, like on most current linux
> systems,
>>>>> to be logged to alternative log files instead of everything ending up
>>>>> in /var/log/ messages.
>>>> The currentl syslog server could do that. The problem is more that we
>>>> don’t have good tools to view the logs on the web UI. So there is
> simply
>>>> one place where most things go into and we will then filter it out
> when
>>>> users use the web UI. This is tremendously slow.
>>>>
>>> I did look at this a while back and believe that with syslog all
> kernel
>>> messages must go to the same log file and cannot be filtered off as
> can
> be
>>> done with rsyslog. I use this facility in rsyslog.d on my server to
> filter
>>> off SYN Flood logs to a separate log file with ":msg,contains". I
> don't
>>> believe that syslog has this option.
>> Different messages could be sent to different “facilities” and then we
> can filter them into individual files.
>> I would not want to split firewall hits into different files though. I
> just wouldn’t mind if firewall hits were no longer in /var/log/messages.
>
>
> You need to be careful with the LOG* menus which would need to look in 2
> or more places for historical logs and so the cgi's would need to be
> modified.
>
> I would however welcome kernel message filtering since it would allow my
> use of a separate log directory for my auto-dnsbl program to monitor for
> non-dropped incoming SMTP connections without appearing in 'messages' log
> file.
>
>
>>>>> The replacement with rsyslgd could also inhibit the "last message
>>>>> repeated x times" in syslog and which are ignored by IPFire's Log
> menu
>>>>> and causes erroneous results in the 'Count' values.
>>>> This might be a setting that we can turn off in the current
>>>> implementation.
>>>>
>>> I understand that this is hard coded into syslogd and can't be
> changed.
> In
>>> rsyslog it defaults to "off".
>>>
>>> https://www.rsyslog.com/doc/configuration/action/
>>> rsconf1_repeatedmsgreduction.html
>> What impact does this have for you? I never noticed any problem by
> summarising the logs.
>
> I came across this and it illustrates the point well:
>
> https://dcid.me/notes/syslog-last-message-repeated-x-times-rant
>
> and makes the point that kernel messages can be delayed by several seconds
> and could affect any DOS filtering that may be in use.
>
> The dshield iptables.pl script tries to recover the lost repeated messages
> but looses the time-stamp.
>
> When comparing the dshield logs with IPFire's logs there is invariably a
> discrepancy with the IPFire's count sometimes considerably less especially
> with multiple port-scans over a minute or so.
>
> Any chance of looking again at rsyslog? IPFire is the only linux system I
> know of still using syslog.
>
> I seem to remember that Adolf had completed all the necessary changes to
> introduce rsyslog a years or so ago but was not implemented,

It wasn't me. It was Erik Kapfer. I have looked at it to have a go at it but there have been a lot of other things that are on my plate that I am still working through and of course sometimes things come along that have to be addressed and jump ahead in the queue.

Regards,

Adolf.

>   
>
>>>> We could also think about moving all firewall hits into
>>>> /var/log/firewall.log or something. That should free up enough in
>>>> /var/log/messages to make debugging easier.
>>>>
>>> That would be a sensible option, currently I filter out firewall hits
> from
>>> the messages.log with a perl regex.
>>>
>>>>>> I would be happy to hear some more opinions on this from the list.
>>>>>>
>>>>>> -Michael
>>>>> I hope these comments are useful and would be pleased to help where I
>>>>> can.
>>>> Of course :)
>>>>
>>>> -Michael
>>>>
>>>>
>>>>> Rob
>> -Michael
> Rob

Re: Logging Locationblock packets

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 21892 bytes --]

Hello

> On 23 Nov 2024, at 15:47, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> 
> Hi Rob,
> 
> On 23/11/2024 15:36, Rob Brewer wrote:
>> On Sat, 23 Nov 2024 10:26:07 +0100, Michael Tremer wrote:
>> 
>>> Hello,
>>> 
>>>> On 21 Nov 2024, at 00:08, Rob Brewer <ipfire-devel(a)grantura.co.uk>
>> wrote:
>>>> Hi Michael and Adolf,
>>>> 
>>>> I'll combine my reply to both of you.
>>>> 
>>>> On Wed, 20 Nov 2024 15:21:28 +0000, Michael Tremer wrote:
>>>> 
>>>>> Hello,
>>>>> 
>>>>>> On 20 Nov 2024, at 13:36, Rob Brewer <ipfire-devel(a)grantura.co.uk>
>>>>>> wrote:
>>>>>> 
>>>>>> On Tue, 19 Nov 2024 17:10:24 +0000, Michael Tremer wrote:
>>>>>> 
>>>>>> Hi Michael, thank for your response, I was beginning to think there
>> was
>>>>>> little interest in this topic.
>>>>> I was traveling and so have a little bit of a back log as usual :)
>>>>> 
>>>>> I can’t speak for the others, though.
>>>>> 
>>>> Not a problem, I should have said that I previously raised this a
>> topic
>> on
>>>> the forum and Adolf suggested that I raised it on the mailing list as
>>>> well.
>>> Yes, I really prefer this here, because the forum is very overwhelming
>> sometimes. There is too many threads, too many people… Mailing lists are
>> great for development.
>> 
>> That's fine, one query though you don't seem to snip older parts of the
>> post, is this your preference?
>> 
>>>>>> See below...
>>>>>> 
>>>>>>> Hello Rob,
>>>>>>> 
>>>>>>> Good to hear from you again.
>>>>>>> 
>>>>>>>> On 9 Nov 2024, at 14:00, Rob Brewer <ipfire-devel(a)grantura.co.uk>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> As  a contributor to the SANS ISC,  I email my IPTABLES logs hourly
>>>>>>>> to the Dshield database, however the default IPFire IPTABLES logs
>> are
>>>>>>>> incomplete since packets dropped by Locationblock are not logged to
>>>>>>>> syslog.
>>>>>>> This is correct.
>>>>>>> 
>>>>>>>> As a workaround I edit the locationblock function of rules.pl to
>> add
>>>>>>>> a IPTABLES rule to log the dropped packets. This edit does not of
>>>>>>>> course survive any Core-Updates that upgrade rules.pl and the edit
>>>>>>>> needs to be re-applied.
>>>>>>>> 
>>>>>>>> In addition, unless the Locationblock packets are logged to syslog
>>>>>>>> the data displayed in the Firewall logs sections are incomplete as
>>>>>>>> the packets dropped by Location Block are not displayed.
>>>>>>>> 
>>>>>>>> I understand that the original intension of the Location Block
>>>>>>>> feature was to reduce the amount of log messages on installations
>>>>>>>> running on extremely cheap flash storage. I suspect that these
>>>>>>>> installations are now almost zero especially since IPFire no longer
>>>>>>>> supports 32 bit versions.
>>>>>>>> 
>>>>>>>> The obvious solution to this to make Location Block logging
>>>>>>>> selectable as a firewall option and my quick look at making a
>> patches
>>>>>>>> for this would seem to be fairly simple task to rules.pl and and
>>>>>>>> optionsfw.cgi.
>>>>>>>> 
>>>>>>>> I would be happy to provide the necessary patches should this be
>>>>>>>> acceptable.
>>>>>>> I think a couple of things have changed since we introduced this
>>>>>>> feature and we have also learned a couple of lessons here:
>>>>>>> 
>>>>>>> * Yes it is true that this was introduced to keep the logs smaller,
>>>>>>> but not necessarily to avoid using storage, mainly to keep them
>>>>>>> readable by a human. I was always a bit doubtful about this and
>>>>>>> believe that apart from this, there is no more value in a general
>>>>>>> location blocking feature. The firewall will drop those packets
>>>>>>> anyway.
>>>>>>> 
>>>>>>> 
>>>>>> IPFire does a very good job in protecting the internal network from
>>>>>> unwanted probes from the WAN, and I agree that for most users,
>> location
>>>>>> block does not enhance this protection.
>>>>>> 
>>>>>> As I run a mail server, I use locationblock as part of my filtering
>> to
>>>>>> prevent bad actors from accessing my mail server and I think would be
>>>>>> useful for anyone who are running servers on their networks.
>>>>>> 
>>>>>> I don't like blocking traffic, but I have witnessed over the last few
>>>>>> years a substantial rise in abuse from certain countries from where I
>>>>>> would never expect to receive emails and locationblock works very
>> well
>>>>>> for me in filtering out those unwanted connections.
>>>>>> 
>>>>>> I currently have 21 countries blocked including XD Hostile networks.
>>>>>> All of these blocked countries have been seen to support abusive
>>>>>> networks.
>>>>>> 
>>>>>> It would be possible to generate individual rules for these networks
>>>>>> which in some cases I do with Banish, but it is sometimes easier just
>>>>>> to drop the whole country.
>>>>> Easy yes. I am just not sure what we are actually getting out of this.
>>>>> 
>>>>> I don’t like to become political on this list and so I won’t, but I
>> know
>>>>> that some people hold the following beliefs:
>>>>> 
>>>>> * There are “bad” countries out there. In the western world those are
>>>>> Russia, China, and more that fall into this kind of “category”.
>> Nothing
>>>>> is going to stop them to open up an account on AWS and suddenly they
>>>>> have access to resources anywhere in the world and so actually, if
>> those
>>>>> players were up to anything “bad”, they can come out of any IP space
>>>>> they want. There is also Tor which allows this.
>>>>> 
>>>> I have been submitting my iptables logs to dshield for over 20 years
>> and
>>>> in the early days I would be sending a lot less than 1000 a day (this
>> was
>>>> with IPCop of course).
>>>> 
>>>> Over the last few years the number of firewall iptables logs have
>>>> escalated until it had reached dshield's daily limit of 100,000 /day
>> and
>>>> I started looking at ways to limit the logging of some of the 'bad
>> actors'
>>>> firewall probes.
>>> What do you use to send them the logs? The Python script or one of
>> those
>> alternatives?
>> 
>> I use a modified Perl script (iptables.pl) which is about 20 years old was
>> originally written for Smoothwall and Ipcop1. This was part of the Logsend
>> add-on and I modified for IPCop2 about 15 years ago.
>> 
>> Now modified for IPFire and using DMA email instead of the original
>> sendEmail. I've uploaded my version to people.ipfire.org/~helix/dshield.
>> 
>> I had intended to write an install/uninstall script but is in the todo
>> list. Do you think it may be of interest to anyone if I do?
>> 
>>>>> * Looking at actual data, the bullet proof ISPs that annoy us are not
>>>>> that far away. They are hosted in Europe and very prominently so in
>> the
>>>>> Netherlands - a “good” country.
>>>>> 
>>>> I use Banish for this as it can block by ASN and can target bullet
>> proof
>>>> ISPs not only in NL but also in the USA and elsewhere. Using the
>> 'location
>>>> function' the blocked ASN's are updated when the database is updated.
>>>> 
>>>>> This is just an example to explain that we all have our idea of the
>>>>> world in our head, but the Internet doesn’t know these borders.
>>>>> Geopolitics is complicated and I don’t think that this is a tool that
>>>>> helps to fight this kind of problem. I hope this is clear what I am
>>>>> trying to say.
>>>>> 
>>>> Yes my approach is to use multiple tools and Locationblock is just one
>> of
>>>> the tools I am using. Maybe my approach is over the top though.
>>> Probably not massively, but maybe slightly.
>>> 
>>> Wouldn’t it be nicer to have everything in one place instead of having
>> multiple tools that work together in concert?
>> I think that would be a lot of work, would you consider that for IPFire2
>> or 3?
>> 
>>>>> I personally trust in Spamhaus which is feeding data into our “Hostile
>>>>> Networks” blocker and they are looking at the actual traffic that they
>>>>> see from those subnets that they are blocking. They have a clear set
>> of
>>>>> rules and they don’t look just at the country and say “ah yes, the bad
>>>>> guys again”.
>>>>> 
>>>> For emails I also drop packets if listed in a number of DNS blackhole
>>>> lists using Auto-Dnsbl which I adapted for IPFire. See:
>>>> 
>>>> https://people.ipfire.org/~helix/auto-dnsbl/README
>>>> for info.
>>>> 
>>>> I have been using this for about a year and can confirm that it is
>> very
>>>> effective in removing spammers.
>>>> 
>>>>> I have stated this recently in the RPZ discussion that I feel that we
>>>>> are lacking good sources for this kind of data. For blocking anything.
>>>>> Loads of people believe “more is more” and to be fair… once you get to
>>>>> blocking 90% of the internet you probably have blocked 90% of the
>> noise.
>>>>> However this won’t do anything for targeted attacks. I personally
>> would
>>>>> rather have more quality here.
>>>>> 
>>>> I only use a few blocklists with ipblocklist and rely mainly on my
>> locally
>>>> derived  data from failed smtp connections or from data from dshield's
>>>> daily report.
>>>> 
>>>>>>> * People seem to misunderstand this feature: The block as many
>>>>>>> countries as they can click on and think that this makes things
>> “more
>>>>>>> secure”,
>>>>>>> which of course it doesn’t.
>>>>>>> 
>>>>>>> 
>>>>>> I think this is education and I suspect that very few of these people
>>>>>> are running servers on their system and need location block.
>>>>> I am personally not here to educate people. I am happy to help out and
>>>>> give my opinion, but I suppose considering that people think as I have
>>>>> outlined above, who am I to educate them?
>>>>> 
>>>> Sorry I didn't express my reply well, I agree that IPFire users should
>> be
>>>> responsible for their own actions.
>>> Naturally. But we should also not give them sharp knives when we don’t
>> expect them to use them properly.
>>>>> It is just very regretful to me when people are ab-using a feature
>> just
>>>>> to “feel more safe”.
>>>>> 
>>>>>>> * What I personally like as a feature is to apply location filters
>> to
>>>>>>> individual rules. That allows to apply rate limits in a few places
>> and
>>>>>>> easily select some larger groups of potentially less interesting
>>>>>>> traffic.
>>>>>> I agree and is similar to what I am doing for ports 25 and 587
>> although
>>>>>> with location block.
>>>>>> 
>>>>>> 
>>>>>>> * Back when this was introduced there was also a lot less use of
>>>>>>> botnets as we know them now. Pretty much all recent DoS attacks I
>>>>>>> remember have been coming from all over the world because systems
>>>>>>> where compromised wherever they were. This renders the filter
>>>>>>> absolutely useless.
>>>>>> For this, I am currently testing a dynamic block list on my system
>>>>>> which adds an entry to an ipset when an ip address exceeds a
>> threshold
>>>>>> in the previous 24 hours. Currently there are about 7000 entries in
>>>>>> this set which have been detected over the previous month.  Not for
>>>>>> everyone because it needs a Whitelist to prevent some 'push'
>>>>>> technologies such as Facebook and to protect from bulk inputs such as
>>>>>> mailing lists. I also blacklist a large number of ip addresses which
>>>>>> are associated with port scanners.
>>>>>> 
>>>>>> This would work well as a botnet filter but it needs an understanding
>>>>>> of the blocking process, so it would not be suitable for everyone.
>>>>> We have talked about this on the last team call and came to the
>>>>> conclusion that the whole set of blocking features is a total mess.
>>>>> There are several places where things can be blocked, they all heavily
>>>>> overlap. Not to be going on about Spamhaus again, but the DROP list is
>>>>> in the “Hostile Networks” feature, there is a special country code for
>>>>> it, pretty much every IPS ruleset contains it, the IP blocklist
>> feature
>>>>> supports it. Now there are people throwing this all into DNS as well.
>> I
>>>>> believe this is all getting a little bit out of hand.
>>>>> 
>>>> I don't use the IPS mainly due to bad experiences when trying to use
>> Snort
>>>> with IPCop. I find it difficult to set up and select the appropriate
>>>> Rulsets for my requirements. I suspect the IPS would be very effective
>> if
>>>> I could get it set up correctly.
>>> This has massively improved in recent times. And it is FAST!
>>> 
>> I'll have another look, but my current set-up is working well, so I'm
>> reluctant to change.
>> 
>>>>> I personally would prefer *one* place and we really make this work
>> well.
>>>>> Some people find the IPS has too much overhead - I am beginning to
>>>>> disagree.
>>>>> 
>>>>> Some people add all sorts of blacklists that are clearly for
>>>>> experimental/research use and suddenly complain about things not
>> working
>>>>> anymore. It is hard to figure out what is actually blocking traffic
>> when
>>>>> there are about five places to look for. The result often is to
>> disable
>>>>> the IPS, then something else, and after a while all of this is turned
>>>>> off and the firewall is configured to a bare minimum. We don’t want
>>>>> people to run it like that.
>>>>> 
>>>>>>> * Infrastructure is becoming more centralised. Some people attempt
>> to
>>>>>>> block the US and pretty much nothing works any more. Well…
>>>>>>> 
>>>>>>> 
>>>>>> :)
>>>>>> 
>>>>>>> Therefore I am happy to rethink this feature - personally I would
>> even
>>>>>>> remove it entirely, but I don’t like to handle the stress about this
>>>>>>> now.
>>>>>> No I think you should keep it. Locationblock is very useful, but
>> needs
>>>>>> to be used in a considered way. I could live without it but its
>>>>>> availability makes my life easier. Also being updated on a weekly
>> basis
>>>>>> makes keeping track of abusers easier.
>>>>> We issue a new location database once a day, but the IPFire systems
>>>>> update it only once a week. We deemed that as sufficient as there are
>>>>> not that many changes from one version to another one.
>>>>> 
>>>>>>> I think this should log now. I don’t think there is any legitimate
>>>>>>> reason to disable firewall logging at all. This is the paper trail
>>>>>>> that you want when debugging anything or doing forensics on an
>> attack.
>>>>>>> Certainly bad flash is not a valid reason for me.
>>>>>>> 
>>>>>>> 
>>>>>> I have started generating some patches to make 'locationblock
>> logging'
>>>>>> as a selectable feature and could post them here in the next day or
>> so
>>>>>> if of any use.
>>>>> What would this basically do? Log it as a simple rule for all, or do
>> you
>>>>> include the selected country? I think the latter would make more
>> sense.
>>>> OK - I have modified location-block.cgi to add an additional checkbox
>> at
>>>> the top of the page titled "Log dropped packets", very similar to
>>>> ipblocklist.cgi. This adds a entry to the /var/ipfire/firewall/
>>>> locationblock settings file ON or OFF.
>>>> 
>>>> In /usr/lib/firewall/rules.pl I add an additional Iptables
>> LOCATIONBLOCK
>>>> log rule with the prefix 'LOCBLOCK-$location '. This is selectable
>> from
>>>> the Log dropped packets entry in the locationblock settings file.
>>>> 
>>>> This generates kernel messages such as:
>>>> 
>>>> LOCBLOCK-CN IN=ppp0 OUT=ppp0 MAC= SRC=114.139.42.71 DST=81.187.175.103
>>>> LEN=60 TOS=0x00 PREC=0x00 TTL=47 ID=34664 DF PROTO=TCP SPT=46837
>> DPT=23
>>>> WINDOW=29040 RES=0x00 SYN URGP=0
>>>> 
>>>> As you can see the prefix contains the country code of the dropped
>>>> packets.
>>> Cool. So I think coding-wise this is pretty straight forward.
>>> 
>>> The more crucial question for me is below…
>>> 
>> Do have a look at the patches I posted and let me have any comments. If
>> it's not needed and we can add locationblock logging without selection
>> then that's what I have been doing since LB was first implemented.
>> 
>>>>> What is the rationale to turn this off? Why don’t we turn this on for
>>>>> everybody all of the time?
>>>>> 
>>>> I would be happy with that I was suggesting that we use selectable
>> logging
>>>> to be compatible with earlier IPFire versions.
>>> What do we have to be compatible with? Just IPFire’s behaviour from
>> last
>> year?
>>> I would be happy to break that if we have a good reason, because the
>> world sometimes moves on. What we have released 10 years ago probably
>> doesn’t have good defaults any more. Sometimes we have to revisit those
>> and sometimes we even have to be pushy with some changes if we want some
>> long-standing users to adopt them. IPFire users usually have no reason to
>> do a reinstall and we don’t want to leave the people behind who have been
>> running IPFire perfectly well for the last 10 years or even longer. So at
>> least I would like to discuss what we push a little harder.
>> As Adolf pointed out in the Forum,the WIKI stated the rational for not
>> logging was to protect flash memory from excessive use. I don't think that
>> is appropriate now.
>> 
>>>>>> In fact I think that IPFire's logging environment, which was
>> inherited
>>>>>> from IPCop needs a re-visit.
>>>>> Yes. But this is probably not a thing I would be looking at in IPFire
>> 2.
>>>>> I am also happy to think a little bit more outside of the box and see
>>>>> what we would fundamentally change in the future, but not implement in
>>>>> IPFire 2.
>>>>> 
>>>>>> I was pleased to see that a while back there was a discussion of
>>>>>> replacing sylogd with rsyslogd which I think would be a very useful
>>>>>> upgrade and enable kernel messages, like on most current linux
>> systems,
>>>>>> to be logged to alternative log files instead of everything ending up
>>>>>> in /var/log/ messages.
>>>>> The currentl syslog server could do that. The problem is more that we
>>>>> don’t have good tools to view the logs on the web UI. So there is
>> simply
>>>>> one place where most things go into and we will then filter it out
>> when
>>>>> users use the web UI. This is tremendously slow.
>>>>> 
>>>> I did look at this a while back and believe that with syslog all
>> kernel
>>>> messages must go to the same log file and cannot be filtered off as
>> can
>> be
>>>> done with rsyslog. I use this facility in rsyslog.d on my server to
>> filter
>>>> off SYN Flood logs to a separate log file with ":msg,contains". I
>> don't
>>>> believe that syslog has this option.
>>> Different messages could be sent to different “facilities” and then we
>> can filter them into individual files.
>>> I would not want to split firewall hits into different files though. I
>> just wouldn’t mind if firewall hits were no longer in /var/log/messages.
>> 
>> 
>> You need to be careful with the LOG* menus which would need to look in 2
>> or more places for historical logs and so the cgi's would need to be
>> modified.
>> 
>> I would however welcome kernel message filtering since it would allow my
>> use of a separate log directory for my auto-dnsbl program to monitor for
>> non-dropped incoming SMTP connections without appearing in 'messages' log
>> file.
>> 
>> 
>>>>>> The replacement with rsyslgd could also inhibit the "last message
>>>>>> repeated x times" in syslog and which are ignored by IPFire's Log
>> menu
>>>>>> and causes erroneous results in the 'Count' values.
>>>>> This might be a setting that we can turn off in the current
>>>>> implementation.
>>>>> 
>>>> I understand that this is hard coded into syslogd and can't be
>> changed.
>> In
>>>> rsyslog it defaults to "off".
>>>> 
>>>> https://www.rsyslog.com/doc/configuration/action/
>>>> rsconf1_repeatedmsgreduction.html
>>> What impact does this have for you? I never noticed any problem by
>> summarising the logs.
>> 
>> I came across this and it illustrates the point well:
>> 
>> https://dcid.me/notes/syslog-last-message-repeated-x-times-rant
>> 
>> and makes the point that kernel messages can be delayed by several seconds
>> and could affect any DOS filtering that may be in use.
>> 
>> The dshield iptables.pl script tries to recover the lost repeated messages
>> but looses the time-stamp.
>> 
>> When comparing the dshield logs with IPFire's logs there is invariably a
>> discrepancy with the IPFire's count sometimes considerably less especially
>> with multiple port-scans over a minute or so.
>> 
>> Any chance of looking again at rsyslog? IPFire is the only linux system I
>> know of still using syslog.
>> 
>> I seem to remember that Adolf had completed all the necessary changes to
>> introduce rsyslog a years or so ago but was not implemented,
> 
> It wasn't me. It was Erik Kapfer. I have looked at it to have a go at it but there have been a lot of other things that are on my plate that I am still working through and of course sometimes things come along that have to be addressed and jump ahead in the queue.

Story of my life :)

> Regards,
> 
> Adolf.
> 
>>  
>>>>> We could also think about moving all firewall hits into
>>>>> /var/log/firewall.log or something. That should free up enough in
>>>>> /var/log/messages to make debugging easier.
>>>>> 
>>>> That would be a sensible option, currently I filter out firewall hits
>> from
>>>> the messages.log with a perl regex.
>>>> 
>>>>>>> I would be happy to hear some more opinions on this from the list.
>>>>>>> 
>>>>>>> -Michael
>>>>>> I hope these comments are useful and would be pleased to help where I
>>>>>> can.
>>>>> Of course :)
>>>>> 
>>>>> -Michael
>>>>> 
>>>>> 
>>>>>> Rob
>>> -Michael
>> Rob