Re: IPFire 2.27 - Core Update 165 is available for testing

Re: IPFire 2.27 - Core Update 165 is available for testing

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 2091 bytes --]

Hi All,

Running 165 Testing for a day now. Most things working well.

One thing I found is that this morning the daily update of the IPS rulesets had not taken place although the updates were set to occur in 163 and also showed on the WUI page as being set for daily.

After some searching I found that the suricata rules update link was not set in the /etc/fcron.daily directory. Pressing save on the IPS page made no difference.
I then changed the setting to weekly and pressed save and the link was then created in the fcron.weekly directory. Then I changed it to daily and pressed save and now I have the link in my fcron.daily directory but it was not present after the upgrade process.

It is no running with updated rulesets and I will check tomorrow how the update went.

In terms of test results, everything was running after the upgrade (On my testbed I have everything set up and running although I don't yet have a working IPSec client but that should be available for the next testing.

OpenVPN Roadwarrior connections successfully running.
Web Proxy is up and running and working.
Openssh is up and working.
OpenVPN Roadwarrior connections log working.
Domain Name System fully working with five DNS TLS servers.
DHCP server up and fully working.
Backup system working.
All graphs working as before.
All Firewall Log Charts working.

Regards,
Adolf.

On 15/03/2022 12:40, IPFire Project wrote:
> IPFire Logo
> 
> there is a new post from Michael Tremer on the IPFire Blog:
> 
> *IPFire 2.27 - Core Update 165 is available for testing*
> 
>     Another update is ready for testing: IPFire 2.27 - Core Update 165. It comes with various updates for the firewall engine that improve its performance and increase its flexibility, as well as with an updated toolchain, Python 3.10 and various more bug and security fixes.
> 
> Click Here To Read More <https://blog.ipfire.org/post/ipfire-2-27-core-update-165-is-available-for-testing>
> 
> The IPFire Project
> Don't like these emails? Unsubscribe <https://people.ipfire.org/unsubscribe>.
> 

Re: IPFire 2.27 - Core Update 165 is available for testing

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 3354 bytes --]

Hi All,

Further update.

I registered for the Talos VRT Registered rulesets.

I then  added the set to my IPS and after a while the IPS page came back now with Talos VRT added to the providers. I then pressed customise ruleset and got the rules. I selected some rulesets from Talos and then pressed apply. I got the spinning symbol and the browser status bar showed activity ongoing. After some time the browser pointer spinning disk disappeared and the status bar was empty but the IPFire IPS spinning symbol continued. I checked in /tmp and there was no lock file but there was an ids_tmp directory.


I then selected another wui menu and then went back to the IPS menu and got the screen with three providers. Selected customise ruleset again and got the table headed ruleset with the buttons Back and Apply but absolutely no rulesets.


I then deleted the Talos VRT ruleset and after some time got back the main page with only two providers now.

I then pressed customise ruleset again and got all the rulesets for the two remaining providers back again.

So the Talos VRT Registered ruleset  still has some issues.


Regards,

Adolf.

On 17/03/2022 10:29, Adolf Belka wrote:
> Hi All,
>
> Running 165 Testing for a day now. Most things working well.
>
> One thing I found is that this morning the daily update of the IPS rulesets had not taken place although the updates were set to occur in 163 and also showed on the WUI page as being set for daily.
>
> After some searching I found that the suricata rules update link was not set in the /etc/fcron.daily directory. Pressing save on the IPS page made no difference.
> I then changed the setting to weekly and pressed save and the link was then created in the fcron.weekly directory. Then I changed it to daily and pressed save and now I have the link in my fcron.daily directory but it was not present after the upgrade process.
>
> It is no running with updated rulesets and I will check tomorrow how the update went.
>
> In terms of test results, everything was running after the upgrade (On my testbed I have everything set up and running although I don't yet have a working IPSec client but that should be available for the next testing.
>
> OpenVPN Roadwarrior connections successfully running.
> Web Proxy is up and running and working.
> Openssh is up and working.
> OpenVPN Roadwarrior connections log working.
> Domain Name System fully working with five DNS TLS servers.
> DHCP server up and fully working.
> Backup system working.
> All graphs working as before.
> All Firewall Log Charts working.
>
> Regards,
> Adolf.
>
> On 15/03/2022 12:40, IPFire Project wrote:
>> IPFire Logo
>>
>> there is a new post from Michael Tremer on the IPFire Blog:
>>
>> *IPFire 2.27 - Core Update 165 is available for testing*
>>
>>     Another update is ready for testing: IPFire 2.27 - Core Update 165. It comes with various updates for the firewall engine that improve its performance and increase its flexibility, as well as with an updated toolchain, Python 3.10 and various more bug and security fixes.
>>
>> Click Here To Read More <https://blog.ipfire.org/post/ipfire-2-27-core-update-165-is-available-for-testing>
>>
>> The IPFire Project
>> Don't like these emails? Unsubscribe <https://people.ipfire.org/unsubscribe>.
>>

Re: IPFire 2.27 - Core Update 165 is available for testing

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 3767 bytes --]

Hi All,

After changing from daily to weekly and back to daily to get the suricata link in the fcron.daily I can confirm that the updates for the two providers I have selected worked overnight. Also no spinning disk when I went to the IPS page. so everything looks good.

Regards,

Adolf

On 17/03/2022 10:50, Adolf Belka wrote:
> Hi All,
>
> Further update.
>
> I registered for the Talos VRT Registered rulesets.
>
> I then  added the set to my IPS and after a while the IPS page came back now with Talos VRT added to the providers. I then pressed customise ruleset and got the rules. I selected some rulesets from Talos and then pressed apply. I got the spinning symbol and the browser status bar showed activity ongoing. After some time the browser pointer spinning disk disappeared and the status bar was empty but the IPFire IPS spinning symbol continued. I checked in /tmp and there was no lock file but there was an ids_tmp directory.
>
>
> I then selected another wui menu and then went back to the IPS menu and got the screen with three providers. Selected customise ruleset again and got the table headed ruleset with the buttons Back and Apply but absolutely no rulesets.
>
>
> I then deleted the Talos VRT ruleset and after some time got back the main page with only two providers now.
>
> I then pressed customise ruleset again and got all the rulesets for the two remaining providers back again.
>
> So the Talos VRT Registered ruleset  still has some issues.
>
>
> Regards,
>
> Adolf.
>
> On 17/03/2022 10:29, Adolf Belka wrote:
>> Hi All,
>>
>> Running 165 Testing for a day now. Most things working well.
>>
>> One thing I found is that this morning the daily update of the IPS rulesets had not taken place although the updates were set to occur in 163 and also showed on the WUI page as being set for daily.
>>
>> After some searching I found that the suricata rules update link was not set in the /etc/fcron.daily directory. Pressing save on the IPS page made no difference.
>> I then changed the setting to weekly and pressed save and the link was then created in the fcron.weekly directory. Then I changed it to daily and pressed save and now I have the link in my fcron.daily directory but it was not present after the upgrade process.
>>
>> It is no running with updated rulesets and I will check tomorrow how the update went.
>>
>> In terms of test results, everything was running after the upgrade (On my testbed I have everything set up and running although I don't yet have a working IPSec client but that should be available for the next testing.
>>
>> OpenVPN Roadwarrior connections successfully running.
>> Web Proxy is up and running and working.
>> Openssh is up and working.
>> OpenVPN Roadwarrior connections log working.
>> Domain Name System fully working with five DNS TLS servers.
>> DHCP server up and fully working.
>> Backup system working.
>> All graphs working as before.
>> All Firewall Log Charts working.
>>
>> Regards,
>> Adolf.
>>
>> On 15/03/2022 12:40, IPFire Project wrote:
>>> IPFire Logo
>>>
>>> there is a new post from Michael Tremer on the IPFire Blog:
>>>
>>> *IPFire 2.27 - Core Update 165 is available for testing*
>>>
>>>     Another update is ready for testing: IPFire 2.27 - Core Update 165. It comes with various updates for the firewall engine that improve its performance and increase its flexibility, as well as with an updated toolchain, Python 3.10 and various more bug and security fixes.
>>>
>>> Click Here To Read More <https://blog.ipfire.org/post/ipfire-2-27-core-update-165-is-available-for-testing>
>>>
>>> The IPFire Project
>>> Don't like these emails? Unsubscribe <https://people.ipfire.org/unsubscribe>.
>>>

Re: IPFire 2.27 - Core Update 165 is available for testing

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 5057 bytes --]

Hi All,

Ignore my comments in the thread below on the fcron.daily issue. This is nothing to do with the CU164 to CU165. It is a bug but related to the fact that a backup does not restore the suricata link in either the fcron.daily or fcron.weekly or not at all depending on the update frequency setting.


So if the link is there then a CU maintains it where it is.

I have confirmed this by installing a CU163 vm restoring a backup from that time and then discovering that the link was not there. Updated the frequency setting to get the link created and confirmed its presence.

Then I did a CU to 164 and after reboot the suricata link was still there. Then did the same from CU164 to CU165 Testing and also the suricata link stayed there.


What happened was that I had a vm install some time back and did a restore from a backup I had and did not notice that the suricata link was not there. So this vm has not been doing daily backups for a while. Not so easily noticeable as I don't leave the vm's running overnight unless I am doing a CU Testing evaluation.

I will raise a bug for the lack of storing the suricata link.


Regards,

Adolf.

On 18/03/2022 12:37, Adolf Belka wrote:
> Hi All,
>
> After changing from daily to weekly and back to daily to get the suricata link in the fcron.daily I can confirm that the updates for the two providers I have selected worked overnight. Also no spinning disk when I went to the IPS page. so everything looks good.
>
> Regards,
>
> Adolf
>
> On 17/03/2022 10:50, Adolf Belka wrote:
>> Hi All,
>>
>> Further update.
>>
>> I registered for the Talos VRT Registered rulesets.
>>
>> I then  added the set to my IPS and after a while the IPS page came back now with Talos VRT added to the providers. I then pressed customise ruleset and got the rules. I selected some rulesets from Talos and then pressed apply. I got the spinning symbol and the browser status bar showed activity ongoing. After some time the browser pointer spinning disk disappeared and the status bar was empty but the IPFire IPS spinning symbol continued. I checked in /tmp and there was no lock file but there was an ids_tmp directory.
>>
>>
>> I then selected another wui menu and then went back to the IPS menu and got the screen with three providers. Selected customise ruleset again and got the table headed ruleset with the buttons Back and Apply but absolutely no rulesets.
>>
>>
>> I then deleted the Talos VRT ruleset and after some time got back the main page with only two providers now.
>>
>> I then pressed customise ruleset again and got all the rulesets for the two remaining providers back again.
>>
>> So the Talos VRT Registered ruleset  still has some issues.
>>
>>
>> Regards,
>>
>> Adolf.
>>
>> On 17/03/2022 10:29, Adolf Belka wrote:
>>> Hi All,
>>>
>>> Running 165 Testing for a day now. Most things working well.
>>>
>>> One thing I found is that this morning the daily update of the IPS rulesets had not taken place although the updates were set to occur in 163 and also showed on the WUI page as being set for daily.
>>>
>>> After some searching I found that the suricata rules update link was not set in the /etc/fcron.daily directory. Pressing save on the IPS page made no difference.
>>> I then changed the setting to weekly and pressed save and the link was then created in the fcron.weekly directory. Then I changed it to daily and pressed save and now I have the link in my fcron.daily directory but it was not present after the upgrade process.
>>>
>>> It is no running with updated rulesets and I will check tomorrow how the update went.
>>>
>>> In terms of test results, everything was running after the upgrade (On my testbed I have everything set up and running although I don't yet have a working IPSec client but that should be available for the next testing.
>>>
>>> OpenVPN Roadwarrior connections successfully running.
>>> Web Proxy is up and running and working.
>>> Openssh is up and working.
>>> OpenVPN Roadwarrior connections log working.
>>> Domain Name System fully working with five DNS TLS servers.
>>> DHCP server up and fully working.
>>> Backup system working.
>>> All graphs working as before.
>>> All Firewall Log Charts working.
>>>
>>> Regards,
>>> Adolf.
>>>
>>> On 15/03/2022 12:40, IPFire Project wrote:
>>>> IPFire Logo
>>>>
>>>> there is a new post from Michael Tremer on the IPFire Blog:
>>>>
>>>> *IPFire 2.27 - Core Update 165 is available for testing*
>>>>
>>>>     Another update is ready for testing: IPFire 2.27 - Core Update 165. It comes with various updates for the firewall engine that improve its performance and increase its flexibility, as well as with an updated toolchain, Python 3.10 and various more bug and security fixes.
>>>>
>>>> Click Here To Read More <https://blog.ipfire.org/post/ipfire-2-27-core-update-165-is-available-for-testing>
>>>>
>>>> The IPFire Project
>>>> Don't like these emails? Unsubscribe <https://people.ipfire.org/unsubscribe>.
>>>>

Core Update 165 (testing) report

[Peter Müller]

[-- Attachment #1: Type: text/plain, Size: 2619 bytes --]

Hello development folks,

Core Update 165 (testing, see: https://blog.ipfire.org/post/ipfire-2-27-core-update-165-is-available-for-testing)
is running here for roughly a day, with some minor and one major issue(s) spotted so far.

While installing the update went smooth (the improved Pakfire GUI is really nice - thanks
again to Leo!), I bumped into some IPS-related trouble afterwards: Network connections timed
out frequently, no matter whether they were directed to IPFire itself or not, and the GUI
(especially the graphs) took ages to load, if at all.

Stopping the IPS and starting it again via the GUI solved the problem - I believe we should
do this in the update.sh script, but am not sure if there are cross-dependencies to something
else. There is no bug raised for this yet.

Also, I noticed Squid was not restarted during the update, which caused trouble until I
ran "/etc/init.d/squid restart" manually. Bug #12810 has been filed for this, and I just
submitted a patch fixing this a few minutes ago.

Since my IPFire machine is rebooting every night via a scheduled job, I usually do not
conduct a manual reboot immediately after installing an update. This time, I felt it was a
good thing to do.

Afterwards, Tor would not start again unless I disabled the "sandbox" feature manually. I
believe this is not a general problem, but due to the fact that we did not ship libseccomp.
Bug #12807 has been filed for this.

Today (i. e. another reboot later), I noticed charon emitting some iptables warnings while
establishing an IPsec connection (filed as bug #12808), and, more important, some location-
based firewall rules not being properly loaded, which rendered DDNS - and subsequently IPsec
and OpenVPN - useless for me, since I am getting IP addresses assigned dynamically.

Bug #12809 has been filed for the latter, and I believe this is a show-stopper. Oddly enough,
it did not appear after the first reboot, and _some_ location-based firewall rules were loaded
correctly after the second one. Manually reloading the rules via the GUI "solved" the problem
for now, but this is definitely something we need to have a look at before releasing C165.

Apart from these, things look good to me. Tested IPFire functionalities in detail:
- IPsec (N2N connections only)
- Squid (authentication enabled, using an upstream proxy)
- OpenVPN (RW connections only)
- IPS/Suricata (with Emerging Threats community ruleset enabled)
- Guardian
- Quality of Service
- DNS (using DNS over TLS)
- Dynamic DNS
- Tor (relay mode)

Thanks, and best regards,
Peter Müller

Re: Core Update 165 (testing) report

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 3087 bytes --]

Hello,

> On 20 Mar 2022, at 10:35, Peter Müller <peter.mueller(a)ipfire.org> wrote:
> 
> Hello development folks,
> 
> Core Update 165 (testing, see: https://blog.ipfire.org/post/ipfire-2-27-core-update-165-is-available-for-testing)
> is running here for roughly a day, with some minor and one major issue(s) spotted so far.
> 
> While installing the update went smooth (the improved Pakfire GUI is really nice - thanks
> again to Leo!), I bumped into some IPS-related trouble afterwards: Network connections timed
> out frequently, no matter whether they were directed to IPFire itself or not, and the GUI
> (especially the graphs) took ages to load, if at all.

This is a worry to me and should not happen.

> Stopping the IPS and starting it again via the GUI solved the problem - I believe we should
> do this in the update.sh script, but am not sure if there are cross-dependencies to something
> else. There is no bug raised for this yet.

This is even a bigger worry, because the IPS should not do this kind of stuff.

> Also, I noticed Squid was not restarted during the update, which caused trouble until I
> ran "/etc/init.d/squid restart" manually. Bug #12810 has been filed for this, and I just
> submitted a patch fixing this a few minutes ago.
> 
> Since my IPFire machine is rebooting every night via a scheduled job, I usually do not
> conduct a manual reboot immediately after installing an update. This time, I felt it was a
> good thing to do.

Do people still do scheduled reboots?

> Afterwards, Tor would not start again unless I disabled the "sandbox" feature manually. I
> believe this is not a general problem, but due to the fact that we did not ship libseccomp.
> Bug #12807 has been filed for this.

Thank you. Merged.

> Today (i. e. another reboot later), I noticed charon emitting some iptables warnings while
> establishing an IPsec connection (filed as bug #12808), and, more important, some location-
> based firewall rules not being properly loaded, which rendered DDNS - and subsequently IPsec
> and OpenVPN - useless for me, since I am getting IP addresses assigned dynamically.

Do you have more details on this? Do you believe the two problems are related?

> Bug #12809 has been filed for the latter, and I believe this is a show-stopper. Oddly enough,
> it did not appear after the first reboot, and _some_ location-based firewall rules were loaded
> correctly after the second one. Manually reloading the rules via the GUI "solved" the problem
> for now, but this is definitely something we need to have a look at before releasing C165.
> 
> Apart from these, things look good to me. Tested IPFire functionalities in detail:
> - IPsec (N2N connections only)
> - Squid (authentication enabled, using an upstream proxy)
> - OpenVPN (RW connections only)
> - IPS/Suricata (with Emerging Threats community ruleset enabled)
> - Guardian
> - Quality of Service
> - DNS (using DNS over TLS)
> - Dynamic DNS
> - Tor (relay mode)
> 
> Thanks, and best regards,
> Peter Müller

-Michael

[PATCH] rules.pl: Fix creating rules for location based groups.

[Stefan Schantl]

[-- Attachment #1: Type: text/plain, Size: 1679 bytes --]

The former used hash value only contains the country code when
a rule for a single country should be created.

In case a location group is used the hash value refers to the group name,
which does not work here.

The required country code is part of the processed string and can be omitted
from here. This works well for single codes and location groups, because those
are processed in a loop.

Fixes #12809.

Signed-off-by: Stefan Schantl <stefan.schantl(a)ipfire.org>
---
 config/firewall/rules.pl | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/config/firewall/rules.pl b/config/firewall/rules.pl
index d71304986..19dc7a6d1 100644
--- a/config/firewall/rules.pl
+++ b/config/firewall/rules.pl
@@ -411,8 +411,9 @@ sub buildrules {
 					if ($source =~ /mac/) {
 						push(@source_options, $source);
 					} elsif ($source =~ /-m set/) {
-						# Grab location code from hash.
-						my $loc_src = $$hash{$key}[4];
+						# Split given arguments into single chunks to
+						# obtain the set name.
+						my ($a, $b, $c, $loc_src, $e) = split(/ /, $source);
 
 						# Call function to load the networks list for this country.
 						&ipset_restore($loc_src);
@@ -425,8 +426,9 @@ sub buildrules {
 					# Prepare destination options.
 					my @destination_options = ();
 					if ($destination =~ /-m set/) {
-						# Grab location code from hash.
-						my $loc_dst = $$hash{$key}[6];
+						# Split given arguments into single chunks to
+						# obtain the set name.
+						my ($a, $b, $c, $loc_dst, $e) = split(/ /, $destination);
 
 						# Call function to load the networks list for this country.
 						&ipset_restore($loc_dst);
-- 
2.30.2

Re: [PATCH] rules.pl: Fix creating rules for location based groups.

[Peter Müller]

[-- Attachment #1: Type: text/plain, Size: 1858 bytes --]

Acked-by: Peter Müller <peter.mueller(a)ipfire.org>

> The former used hash value only contains the country code when
> a rule for a single country should be created.
> 
> In case a location group is used the hash value refers to the group name,
> which does not work here.
> 
> The required country code is part of the processed string and can be omitted
> from here. This works well for single codes and location groups, because those
> are processed in a loop.
> 
> Fixes #12809.
> 
> Signed-off-by: Stefan Schantl <stefan.schantl(a)ipfire.org>
> ---
>  config/firewall/rules.pl | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/config/firewall/rules.pl b/config/firewall/rules.pl
> index d71304986..19dc7a6d1 100644
> --- a/config/firewall/rules.pl
> +++ b/config/firewall/rules.pl
> @@ -411,8 +411,9 @@ sub buildrules {
>  					if ($source =~ /mac/) {
>  						push(@source_options, $source);
>  					} elsif ($source =~ /-m set/) {
> -						# Grab location code from hash.
> -						my $loc_src = $$hash{$key}[4];
> +						# Split given arguments into single chunks to
> +						# obtain the set name.
> +						my ($a, $b, $c, $loc_src, $e) = split(/ /, $source);
>  
>  						# Call function to load the networks list for this country.
>  						&ipset_restore($loc_src);
> @@ -425,8 +426,9 @@ sub buildrules {
>  					# Prepare destination options.
>  					my @destination_options = ();
>  					if ($destination =~ /-m set/) {
> -						# Grab location code from hash.
> -						my $loc_dst = $$hash{$key}[6];
> +						# Split given arguments into single chunks to
> +						# obtain the set name.
> +						my ($a, $b, $c, $loc_dst, $e) = split(/ /, $destination);
>  
>  						# Call function to load the networks list for this country.
>  						&ipset_restore($loc_dst);