Hello *,
currently, the iptables configuration used in IPFire 2.x does not log _every_ packet if logging is enabled for whatever reason, but enforces a rate-limit:
iptables -A LOG_DROP -m limit --limit 10/minute -j LOG
(snip taken from /etc/init.d/firewall)
For several reasons, I consider this a bad idea. (Forgive me for bringing up firewall issues in IPFire 2.x again. :-) )
First, this rate-limit is never mentioned in the firewall WebUI or our documentation, thus being unintentional for most users including me.
Second, it makes debugging very hard - I recently spent several unpleasant days trying to fix a VoIP related network problem, until I got not every packet dropped by IPFire was actually logged. Especially for corner cases or non-deterministic issues, this behaviour makes this more difficult.
Third, it is not compliant. Especially when it comes to post mortem forensics, firewall logs are important. If you cannot trust them since there is no way of telling whether a packet was dropped and not logged, or never seen by the firewall machine, its best to stop logging anything at all.
I therefore propose to drop iptables logging rate-limit in our firewall configurations (which goes for IPFire 3.x as well). Since my systems to not run on problematic hardware (ARM SoCs with SD cards, crappy flash storage, etc.), I have no idea if this will cause issues on some systems/platforms.
@All: Thoughts, please. Is anyone aware of potential trouble?
If not, I will send in a patch within this week.
Thanks, and best regards, Peter Müller
Hi,
On 14 Jul 2019, at 15:56, Peter Müller peter.mueller@ipfire.org wrote:
Hello *,
currently, the iptables configuration used in IPFire 2.x does not log _every_ packet if logging is enabled for whatever reason, but enforces a rate-limit:
iptables -A LOG_DROP -m limit --limit 10/minute -j LOG
(snip taken from /etc/init.d/firewall)
For several reasons, I consider this a bad idea. (Forgive me for bringing up firewall issues in IPFire 2.x again. :-) )
IPFire 2 is still being actively maintained, so feel free to do so.
First, this rate-limit is never mentioned in the firewall WebUI or our documentation, thus being unintentional for most users including me.
Second, it makes debugging very hard - I recently spent several unpleasant days trying to fix a VoIP related network problem, until I got not every packet dropped by IPFire was actually logged. Especially for corner cases or non-deterministic issues, this behaviour makes this more difficult.
Third, it is not compliant. Especially when it comes to post mortem forensics, firewall logs are important. If you cannot trust them since there is no way of telling whether a packet was dropped and not logged, or never seen by the firewall machine, its best to stop logging anything at all.
I therefore propose to drop iptables logging rate-limit in our firewall configurations (which goes for IPFire 3.x as well). Since my systems to not run on problematic hardware (ARM SoCs with SD cards, crappy flash storage, etc.), I have no idea if this will cause issues on some systems/platforms.
@All: Thoughts, please. Is anyone aware of potential trouble?
I generally agree with your points. I am not sure where the number is even coming from. We have carried this over from the beginning of IPFire without thinking about it again.
The motivation why this is there is probably that logging is an expensive operation and that you can use this as a DoS vector and fill up somebody’s hard drive just by sending packets. Logs will grow a lot faster after making this change and maybe we want to have a backstop for this?
-Michael
If not, I will send in a patch within this week.
Thanks, and best regards, Peter Müller -- The road to Hades is easy to travel. -- Bion of Borysthenes
Hi,
On 15/07/2019 11:29, Michael Tremer wrote:
Hi,
On 14 Jul 2019, at 15:56, Peter Müller peter.mueller@ipfire.org wrote:
Hello *,
currently, the iptables configuration used in IPFire 2.x does not log _every_ packet if logging is enabled for whatever reason, but enforces a rate-limit:
iptables -A LOG_DROP -m limit --limit 10/minute -j LOG
(snip taken from /etc/init.d/firewall)
For several reasons, I consider this a bad idea. (Forgive me for bringing up firewall issues in IPFire 2.x again. :-) )
IPFire 2 is still being actively maintained, so feel free to do so.
First, this rate-limit is never mentioned in the firewall WebUI or our documentation, thus being unintentional for most users including me.
Second, it makes debugging very hard - I recently spent several unpleasant days trying to fix a VoIP related network problem, until I got not every packet dropped by IPFire was actually logged. Especially for corner cases or non-deterministic issues, this behaviour makes this more difficult.
Third, it is not compliant. Especially when it comes to post mortem forensics, firewall logs are important. If you cannot trust them since there is no way of telling whether a packet was dropped and not logged, or never seen by the firewall machine, its best to stop logging anything at all.
I therefore propose to drop iptables logging rate-limit in our firewall configurations (which goes for IPFire 3.x as well). Since my systems to not run on problematic hardware (ARM SoCs with SD cards, crappy flash storage, etc.), I have no idea if this will cause issues on some systems/platforms.
@All: Thoughts, please. Is anyone aware of potential trouble?
I generally agree with your points. I am not sure where the number is even coming from. We have carried this over from the beginning of IPFire without thinking about it again.
The motivation why this is there is probably that logging is an expensive operation and that you can use this as a DoS vector and fill up somebody’s hard drive just by sending packets. Logs will grow a lot faster after making this change and maybe we want to have a backstop for this?
-Michael
I think there probably needs to be some sort of limit to prevent the DoS scenario, but 10/minute seems very small for modern hardware. Perhaps something could be done using hashlimit to prevent any one source IP causing a DoS while still allowing dropped packets from other sources to be logged.
Tim
If not, I will send in a patch within this week.
Thanks, and best regards, Peter Müller -- The road to Hades is easy to travel. -- Bion of Borysthenes
Previous setting was to log 10 packets per minute for each event logging is turned on. This made debugging much harder, as the limit was rather strict and chances of dropping a packet without logging it were good.
This patch changes the log rate limit to 10 packets per second per event, to avoid DoS attacks against the log file. I plan to drop log rate limit entirely in future changes, if a better solution for this attack vector is available.
Signed-off-by: Peter Müller peter.mueller@ipfire.org Cc: Tim FitzGeorge ipfr@tfitzgeorge.me.uk Cc: Michael Tremer michael.tremer@ipfire.org --- config/rootfiles/core/135/filelists/files | 1 + src/initscripts/system/firewall | 14 +++++++------- 2 files changed, 8 insertions(+), 7 deletions(-)
diff --git a/config/rootfiles/core/135/filelists/files b/config/rootfiles/core/135/filelists/files index d8df9f65b..e5943ddc0 100644 --- a/config/rootfiles/core/135/filelists/files +++ b/config/rootfiles/core/135/filelists/files @@ -5,6 +5,7 @@ var/ipfire/langs etc/unbound/root.hints etc/rc.d/helper/azure-setup etc/rc.d/init.d/cloud-init +etc/rc.d/init.d/firewall etc/rc.d/init.d/functions etc/rc.d/init.d/networking/red.down/05-remove-dns-forwarders etc/rc.d/init.d/partresize diff --git a/src/initscripts/system/firewall b/src/initscripts/system/firewall index b3483a744..ec396c708 100644 --- a/src/initscripts/system/firewall +++ b/src/initscripts/system/firewall @@ -34,20 +34,20 @@ iptables_init() {
# Empty LOG_DROP and LOG_REJECT chains iptables -N LOG_DROP - iptables -A LOG_DROP -m limit --limit 10/minute -j LOG + iptables -A LOG_DROP -m limit --limit 10/second -j LOG iptables -A LOG_DROP -j DROP iptables -N LOG_REJECT - iptables -A LOG_REJECT -m limit --limit 10/minute -j LOG + iptables -A LOG_REJECT -m limit --limit 10/second -j LOG iptables -A LOG_REJECT -j REJECT
# This chain will log, then DROPs packets with certain bad combinations # of flags might indicate a port-scan attempt (xmas, null, etc) iptables -N PSCAN if [ "$DROPPORTSCAN" == "on" ]; then - iptables -A PSCAN -p tcp -m limit --limit 10/minute -j LOG --log-prefix "DROP_TCP Scan " -m comment --comment "DROP_TCP PScan" - iptables -A PSCAN -p udp -m limit --limit 10/minute -j LOG --log-prefix "DROP_UDP Scan " -m comment --comment "DROP_UDP PScan" - iptables -A PSCAN -p icmp -m limit --limit 10/minute -j LOG --log-prefix "DROP_ICMP Scan " -m comment --comment "DROP_ICMP PScan" - iptables -A PSCAN -f -m limit --limit 10/minute -j LOG --log-prefix "DROP_FRAG Scan " -m comment --comment "DROP_FRAG PScan" + iptables -A PSCAN -p tcp -m limit --limit 10/second -j LOG --log-prefix "DROP_TCP Scan " -m comment --comment "DROP_TCP PScan" + iptables -A PSCAN -p udp -m limit --limit 10/second -j LOG --log-prefix "DROP_UDP Scan " -m comment --comment "DROP_UDP PScan" + iptables -A PSCAN -p icmp -m limit --limit 10/second -j LOG --log-prefix "DROP_ICMP Scan " -m comment --comment "DROP_ICMP PScan" + iptables -A PSCAN -f -m limit --limit 10/second -j LOG --log-prefix "DROP_FRAG Scan " -m comment --comment "DROP_FRAG PScan" fi iptables -A PSCAN -j DROP -m comment --comment "DROP_PScan"
@@ -55,7 +55,7 @@ iptables_init() { # that's not covered above, may just be a broken windows machine iptables -N NEWNOTSYN if [ "$DROPNEWNOTSYN" == "on" ]; then - iptables -A NEWNOTSYN -m limit --limit 10/minute -j LOG --log-prefix "DROP_NEWNOTSYN " + iptables -A NEWNOTSYN -m limit --limit 10/second -j LOG --log-prefix "DROP_NEWNOTSYN " fi iptables -A NEWNOTSYN -j DROP -m comment --comment "DROP_NEWNOTSYN"
Hello, Would it be a good ideea to add root.servers in the CUSTOMINPUT and CUSTOMFORWARD chains, acction ACCEPT just in case that somebody uses the GeoIP feature and blocks the countries where these servers resides?
Ex: I just blocked Japan and Sweden in my Internet facing IPFire and rest IPFire boxes that uses it tries to access all ROOT Servers and can't reach the ones from blocked countries. First line is m.root-servers.net located in Japan and second line is i.root-servers.net located in Sweden:
Jul 29 22:16:27 internet_ipfire kernel: FORWARDFW IN=green0 OUT=ppp0 MAC=*************** SRC=a.b.c.d DST=202.12.27.33 LEN=75 TOS=0x00 PREC=0x00 TTL=63 ID=966 PROTO=UDP SPT=58062 DPT=53 LEN=55 Jul 29 22:16:27 internet_ipfire kernel: FORWARDFW IN=green0 OUT=ppp0 MAC=*************** SRC=a.b.c.d DST=192.36.148.17 LEN=75 TOS=0x00 PREC=0x00 TTL=63 ID=3289 PROTO=UDP SPT=30228 DPT=53 LEN=55
Horace
Sent: Monday, July 29, 2019 at 11:00 PM From: "Peter Müller" peter.mueller@ipfire.org To: "IPFire: Development-List" development@lists.ipfire.org Subject: [PATCH] firewall: raise log rate limit to 10 packets per second
Previous setting was to log 10 packets per minute for each event logging is turned on. This made debugging much harder, as the limit was rather strict and chances of dropping a packet without logging it were good.
This patch changes the log rate limit to 10 packets per second per event, to avoid DoS attacks against the log file. I plan to drop log rate limit entirely in future changes, if a better solution for this attack vector is available.
Signed-off-by: Peter Müller peter.mueller@ipfire.org Cc: Tim FitzGeorge ipfr@tfitzgeorge.me.uk Cc: Michael Tremer michael.tremer@ipfire.org
config/rootfiles/core/135/filelists/files | 1 + src/initscripts/system/firewall | 14 +++++++------- 2 files changed, 8 insertions(+), 7 deletions(-)
diff --git a/config/rootfiles/core/135/filelists/files b/config/rootfiles/core/135/filelists/files index d8df9f65b..e5943ddc0 100644 --- a/config/rootfiles/core/135/filelists/files +++ b/config/rootfiles/core/135/filelists/files @@ -5,6 +5,7 @@ var/ipfire/langs etc/unbound/root.hints etc/rc.d/helper/azure-setup etc/rc.d/init.d/cloud-init +etc/rc.d/init.d/firewall etc/rc.d/init.d/functions etc/rc.d/init.d/networking/red.down/05-remove-dns-forwarders etc/rc.d/init.d/partresize diff --git a/src/initscripts/system/firewall b/src/initscripts/system/firewall index b3483a744..ec396c708 100644 --- a/src/initscripts/system/firewall +++ b/src/initscripts/system/firewall @@ -34,20 +34,20 @@ iptables_init() {
# Empty LOG_DROP and LOG_REJECT chains iptables -N LOG_DROP
- iptables -A LOG_DROP -m limit --limit 10/minute -j LOG
- iptables -A LOG_DROP -m limit --limit 10/second -j LOG iptables -A LOG_DROP -j DROP iptables -N LOG_REJECT
- iptables -A LOG_REJECT -m limit --limit 10/minute -j LOG
iptables -A LOG_REJECT -m limit --limit 10/second -j LOG iptables -A LOG_REJECT -j REJECT
# This chain will log, then DROPs packets with certain bad combinations # of flags might indicate a port-scan attempt (xmas, null, etc) iptables -N PSCAN if [ "$DROPPORTSCAN" == "on" ]; then
iptables -A PSCAN -p tcp -m limit --limit 10/minute -j LOG --log-prefix "DROP_TCP Scan " -m comment --comment "DROP_TCP PScan"iptables -A PSCAN -p udp -m limit --limit 10/minute -j LOG --log-prefix "DROP_UDP Scan " -m comment --comment "DROP_UDP PScan"iptables -A PSCAN -p icmp -m limit --limit 10/minute -j LOG --log-prefix "DROP_ICMP Scan " -m comment --comment "DROP_ICMP PScan"iptables -A PSCAN -f -m limit --limit 10/minute -j LOG --log-prefix "DROP_FRAG Scan " -m comment --comment "DROP_FRAG PScan"
iptables -A PSCAN -p tcp -m limit --limit 10/second -j LOG --log-prefix "DROP_TCP Scan " -m comment --comment "DROP_TCP PScan"iptables -A PSCAN -p udp -m limit --limit 10/second -j LOG --log-prefix "DROP_UDP Scan " -m comment --comment "DROP_UDP PScan"iptables -A PSCAN -p icmp -m limit --limit 10/second -j LOG --log-prefix "DROP_ICMP Scan " -m comment --comment "DROP_ICMP PScan"iptables -A PSCAN -f -m limit --limit 10/second -j LOG --log-prefix "DROP_FRAG Scan " -m comment --comment "DROP_FRAG PScan"@@ -55,7 +55,7 @@ iptables_init() { # that's not covered above, may just be a broken windows machine iptables -N NEWNOTSYN if [ "$DROPNEWNOTSYN" == "on" ]; then
iptables -A NEWNOTSYN -m limit --limit 10/minute -j LOG --log-prefix "DROP_NEWNOTSYN "
iptables -A NEWNOTSYN -m limit --limit 10/second -j LOG --log-prefix "DROP_NEWNOTSYN "-- 2.16.4
-
Horace Michael -
Michael Tremer -
Peter Müller -
Tim FitzGeorge