[PATCH] urlfilter: Remove bundled Toulouse blacklist

[PATCH] urlfilter: Remove bundled Toulouse blacklist

[Philippe SCARSELLI]

Related to bug 13969 - CU201: restore URLFilter with the Toulouse list.
To avoid future changes to the list—as happened during CU201—

here is a proposed fix

Remove the bundled Toulouse blacklist archive and create an empty
blacklists directory instead.

During upgrades, detect installations that still contain the bundled
blacklist and remove it while preserving the custom blacklist. Updated
blacklists are preserved.

When restoring a backup, remove previous blacklist entries while keeping
the custom blacklist to prevent clashes between files and symlinks.

Tested upgrades from CU204 with the bundled, Toulouse, and IPFire DBL
blacklists, backup restoration from CU203, and a fresh CU205
installation.

Note: The patch was generated using "git format-patch --no-binary" to avoid including the 329 KB binary blacklist archive in the email.
The patch shows the deletion of config/urlfilter/blacklists.tar.gz, but does not contain the binary data itself.
Therefore, the file must be deleted separately when applying the patch.

Signed-off-by: Philippe SCARSELLI <p27m@orange.fr>
---
 config/backup/backup.pl             |   5 ++-
 config/rootfiles/common/squidguard  |  55 ----------------------------
 config/rootfiles/core/205/update.sh |  11 ++++++
 config/urlfilter/blacklists.tar.gz  | Bin 329408 -> 0 bytes
 lfs/squidguard                      |   2 +-
 5 files changed, 16 insertions(+), 57 deletions(-)
 delete mode 100644 config/urlfilter/blacklists.tar.gz

diff --git a/config/backup/backup.pl b/config/backup/backup.pl
index f948c4d21..ec15b2910 100644
--- a/config/backup/backup.pl
+++ b/config/backup/backup.pl
@@ -85,7 +85,10 @@ restore_backup() {
 
 	# remove all previous blacklist entries from urlfilter
 	# to prevent any clashes between symlinks and files
-	rm -Rf /var/ipfire/urlfilter/blacklists/*
+	find /var/ipfire/urlfilter/blacklists -mindepth 1 \
+		! -path '/var/ipfire/urlfilter/blacklists/custom' \
+		! -path '/var/ipfire/urlfilter/blacklists/custom/*' \
+		-exec rm -rf {} +
 
 	# Extract backup
 	if ! tar xvzpf "${filename}" -C / \
diff --git a/config/rootfiles/common/squidguard b/config/rootfiles/common/squidguard
index 389f03787..8c895062c 100644
--- a/config/rootfiles/common/squidguard
+++ b/config/rootfiles/common/squidguard
@@ -11,21 +11,6 @@ var/ipfire/urlfilter/bin/autoupdate.pl
 var/ipfire/urlfilter/bin/prebuild.pl
 var/ipfire/urlfilter/blacklists
 #var/ipfire/urlfilter/blacklists/README
-#var/ipfire/urlfilter/blacklists/ads
-#var/ipfire/urlfilter/blacklists/ads/domains
-#var/ipfire/urlfilter/blacklists/ads/domains.db
-#var/ipfire/urlfilter/blacklists/ads/urls
-#var/ipfire/urlfilter/blacklists/ads/urls.db
-#var/ipfire/urlfilter/blacklists/aggressive
-#var/ipfire/urlfilter/blacklists/aggressive/domains
-#var/ipfire/urlfilter/blacklists/aggressive/domains.db
-#var/ipfire/urlfilter/blacklists/aggressive/urls
-#var/ipfire/urlfilter/blacklists/aggressive/urls.db
-#var/ipfire/urlfilter/blacklists/audio-video
-#var/ipfire/urlfilter/blacklists/audio-video/domains
-#var/ipfire/urlfilter/blacklists/audio-video/domains.db
-#var/ipfire/urlfilter/blacklists/audio-video/urls
-#var/ipfire/urlfilter/blacklists/audio-video/urls.db
 #var/ipfire/urlfilter/blacklists/custom
 #var/ipfire/urlfilter/blacklists/custom/allowed
 #var/ipfire/urlfilter/blacklists/custom/allowed/domains
@@ -33,46 +18,6 @@ var/ipfire/urlfilter/blacklists
 #var/ipfire/urlfilter/blacklists/custom/blocked
 #var/ipfire/urlfilter/blacklists/custom/blocked/domains
 #var/ipfire/urlfilter/blacklists/custom/blocked/urls
-#var/ipfire/urlfilter/blacklists/drugs
-#var/ipfire/urlfilter/blacklists/drugs/domains
-#var/ipfire/urlfilter/blacklists/drugs/domains.db
-#var/ipfire/urlfilter/blacklists/drugs/urls
-#var/ipfire/urlfilter/blacklists/drugs/urls.db
-#var/ipfire/urlfilter/blacklists/gambling
-#var/ipfire/urlfilter/blacklists/gambling/domains
-#var/ipfire/urlfilter/blacklists/gambling/domains.db
-#var/ipfire/urlfilter/blacklists/gambling/urls
-#var/ipfire/urlfilter/blacklists/gambling/urls.db
-#var/ipfire/urlfilter/blacklists/hacking
-#var/ipfire/urlfilter/blacklists/hacking/domains
-#var/ipfire/urlfilter/blacklists/hacking/domains.db
-#var/ipfire/urlfilter/blacklists/hacking/urls
-#var/ipfire/urlfilter/blacklists/hacking/urls.db
-#var/ipfire/urlfilter/blacklists/mail
-#var/ipfire/urlfilter/blacklists/mail/domains
-#var/ipfire/urlfilter/blacklists/mail/domains.db
-#var/ipfire/urlfilter/blacklists/porn
-#var/ipfire/urlfilter/blacklists/porn/domains
-#var/ipfire/urlfilter/blacklists/porn/domains.db
-#var/ipfire/urlfilter/blacklists/porn/expressions
-#var/ipfire/urlfilter/blacklists/porn/urls
-#var/ipfire/urlfilter/blacklists/porn/urls.db
-#var/ipfire/urlfilter/blacklists/proxy
-#var/ipfire/urlfilter/blacklists/proxy/domains
-#var/ipfire/urlfilter/blacklists/proxy/domains.db
-#var/ipfire/urlfilter/blacklists/proxy/urls
-#var/ipfire/urlfilter/blacklists/proxy/urls.db
-#var/ipfire/urlfilter/blacklists/violence
-#var/ipfire/urlfilter/blacklists/violence/domains
-#var/ipfire/urlfilter/blacklists/violence/domains.db
-#var/ipfire/urlfilter/blacklists/violence/expressions
-#var/ipfire/urlfilter/blacklists/violence/urls
-#var/ipfire/urlfilter/blacklists/violence/urls.db
-#var/ipfire/urlfilter/blacklists/warez
-#var/ipfire/urlfilter/blacklists/warez/domains
-#var/ipfire/urlfilter/blacklists/warez/domains.db
-#var/ipfire/urlfilter/blacklists/warez/urls
-#var/ipfire/urlfilter/blacklists/warez/urls.db
 var/ipfire/urlfilter/settings
 var/ipfire/urlfilter/squidGuard.conf
 var/ipfire/urlfilter/version
diff --git a/config/rootfiles/core/205/update.sh b/config/rootfiles/core/205/update.sh
index 1c4acfbb0..a61f2ff7f 100644
--- a/config/rootfiles/core/205/update.sh
+++ b/config/rootfiles/core/205/update.sh
@@ -37,6 +37,17 @@ done
 rm -rfv \
 	/usr/local/bin/unboundctrl
 
+# remove old installed blacklist entries from urlfilter
+# The blacklist is no longer installed
+if [ ! -e "/var/ipfire/urlfilter/blacklists/.autoupdate.last" ] &&
+	[ -f "/var/ipfire/urlfilter/blacklists/README" ] &&
+	grep -q "(squidGuardRobot)" "/var/ipfire/urlfilter/blacklists/README"; then
+	find /var/ipfire/urlfilter/blacklists -mindepth 1 \
+		! -path "/var/ipfire/urlfilter/blacklists/custom" \
+		! -path "/var/ipfire/urlfilter/blacklists/custom/*" \
+		-exec rm -rf {} +
+fi
+
 # Extract files
 extract_files
 
diff --git a/config/urlfilter/blacklists.tar.gz b/config/urlfilter/blacklists.tar.gz
deleted file mode 100644
index dc21990be..000000000
Binary files a/config/urlfilter/blacklists.tar.gz and /dev/null differ
diff --git a/lfs/squidguard b/lfs/squidguard
index 8ad95c472..63eb39125 100644
--- a/lfs/squidguard
+++ b/lfs/squidguard
@@ -112,7 +112,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
 	cp -f  $(DIR_CONF)/urlfilter/redirect_wrapper /usr/sbin
 	chmod 755 /usr/sbin/redirect_wrapper
 
-	/bin/tar --no-same-owner -xzf $(DIR_CONF)/urlfilter/blacklists.tar.gz -C /var/ipfire/urlfilter
+	-mkdir -p /var/ipfire/urlfilter/blacklists
 	/usr/bin/perl $(DIR_CONF)/urlfilter/makeconf.pl
 	touch /var/ipfire/urlfilter/settings
 	chown -R nobody:nobody /var/ipfire/urlfilter
-- 
2.47.3

Re: [PATCH] urlfilter: Remove bundled Toulouse blacklist

[Michael Tremer]

Hello Phil,

Thank you very much for this patch.

I could not quite figure out what you want to achieve with this change. Is this data being shipped causing some problems? The bug report did not give me the information I was looking for either.

All the best,
-Michael

> On 27 Aug 2026, at 11:16, Philippe SCARSELLI <p27m@orange.fr> wrote:
> 
> Related to bug 13969 - CU201: restore URLFilter with the Toulouse list.
> To avoid future changes to the list—as happened during CU201—
> 
> here is a proposed fix
> 
> Remove the bundled Toulouse blacklist archive and create an empty
> blacklists directory instead.
> 
> During upgrades, detect installations that still contain the bundled
> blacklist and remove it while preserving the custom blacklist. Updated
> blacklists are preserved.
> 
> When restoring a backup, remove previous blacklist entries while keeping
> the custom blacklist to prevent clashes between files and symlinks.
> 
> Tested upgrades from CU204 with the bundled, Toulouse, and IPFire DBL
> blacklists, backup restoration from CU203, and a fresh CU205
> installation.
> 
> Note: The patch was generated using "git format-patch --no-binary" to avoid including the 329 KB binary blacklist archive in the email.
> The patch shows the deletion of config/urlfilter/blacklists.tar.gz, but does not contain the binary data itself.
> Therefore, the file must be deleted separately when applying the patch.
> 
> Signed-off-by: Philippe SCARSELLI <p27m@orange.fr>
> ---
> config/backup/backup.pl             |   5 ++-
> config/rootfiles/common/squidguard  |  55 ----------------------------
> config/rootfiles/core/205/update.sh |  11 ++++++
> config/urlfilter/blacklists.tar.gz  | Bin 329408 -> 0 bytes
> lfs/squidguard                      |   2 +-
> 5 files changed, 16 insertions(+), 57 deletions(-)
> delete mode 100644 config/urlfilter/blacklists.tar.gz
> 
> diff --git a/config/backup/backup.pl b/config/backup/backup.pl
> index f948c4d21..ec15b2910 100644
> --- a/config/backup/backup.pl
> +++ b/config/backup/backup.pl
> @@ -85,7 +85,10 @@ restore_backup() {
> 
> # remove all previous blacklist entries from urlfilter
> # to prevent any clashes between symlinks and files
> - rm -Rf /var/ipfire/urlfilter/blacklists/*
> + find /var/ipfire/urlfilter/blacklists -mindepth 1 \
> + ! -path '/var/ipfire/urlfilter/blacklists/custom' \
> + ! -path '/var/ipfire/urlfilter/blacklists/custom/*' \
> + -exec rm -rf {} +
> 
> # Extract backup
> if ! tar xvzpf "${filename}" -C / \
> diff --git a/config/rootfiles/common/squidguard b/config/rootfiles/common/squidguard
> index 389f03787..8c895062c 100644
> --- a/config/rootfiles/common/squidguard
> +++ b/config/rootfiles/common/squidguard
> @@ -11,21 +11,6 @@ var/ipfire/urlfilter/bin/autoupdate.pl
> var/ipfire/urlfilter/bin/prebuild.pl
> var/ipfire/urlfilter/blacklists
> #var/ipfire/urlfilter/blacklists/README
> -#var/ipfire/urlfilter/blacklists/ads
> -#var/ipfire/urlfilter/blacklists/ads/domains
> -#var/ipfire/urlfilter/blacklists/ads/domains.db
> -#var/ipfire/urlfilter/blacklists/ads/urls
> -#var/ipfire/urlfilter/blacklists/ads/urls.db
> -#var/ipfire/urlfilter/blacklists/aggressive
> -#var/ipfire/urlfilter/blacklists/aggressive/domains
> -#var/ipfire/urlfilter/blacklists/aggressive/domains.db
> -#var/ipfire/urlfilter/blacklists/aggressive/urls
> -#var/ipfire/urlfilter/blacklists/aggressive/urls.db
> -#var/ipfire/urlfilter/blacklists/audio-video
> -#var/ipfire/urlfilter/blacklists/audio-video/domains
> -#var/ipfire/urlfilter/blacklists/audio-video/domains.db
> -#var/ipfire/urlfilter/blacklists/audio-video/urls
> -#var/ipfire/urlfilter/blacklists/audio-video/urls.db
> #var/ipfire/urlfilter/blacklists/custom
> #var/ipfire/urlfilter/blacklists/custom/allowed
> #var/ipfire/urlfilter/blacklists/custom/allowed/domains
> @@ -33,46 +18,6 @@ var/ipfire/urlfilter/blacklists
> #var/ipfire/urlfilter/blacklists/custom/blocked
> #var/ipfire/urlfilter/blacklists/custom/blocked/domains
> #var/ipfire/urlfilter/blacklists/custom/blocked/urls
> -#var/ipfire/urlfilter/blacklists/drugs
> -#var/ipfire/urlfilter/blacklists/drugs/domains
> -#var/ipfire/urlfilter/blacklists/drugs/domains.db
> -#var/ipfire/urlfilter/blacklists/drugs/urls
> -#var/ipfire/urlfilter/blacklists/drugs/urls.db
> -#var/ipfire/urlfilter/blacklists/gambling
> -#var/ipfire/urlfilter/blacklists/gambling/domains
> -#var/ipfire/urlfilter/blacklists/gambling/domains.db
> -#var/ipfire/urlfilter/blacklists/gambling/urls
> -#var/ipfire/urlfilter/blacklists/gambling/urls.db
> -#var/ipfire/urlfilter/blacklists/hacking
> -#var/ipfire/urlfilter/blacklists/hacking/domains
> -#var/ipfire/urlfilter/blacklists/hacking/domains.db
> -#var/ipfire/urlfilter/blacklists/hacking/urls
> -#var/ipfire/urlfilter/blacklists/hacking/urls.db
> -#var/ipfire/urlfilter/blacklists/mail
> -#var/ipfire/urlfilter/blacklists/mail/domains
> -#var/ipfire/urlfilter/blacklists/mail/domains.db
> -#var/ipfire/urlfilter/blacklists/porn
> -#var/ipfire/urlfilter/blacklists/porn/domains
> -#var/ipfire/urlfilter/blacklists/porn/domains.db
> -#var/ipfire/urlfilter/blacklists/porn/expressions
> -#var/ipfire/urlfilter/blacklists/porn/urls
> -#var/ipfire/urlfilter/blacklists/porn/urls.db
> -#var/ipfire/urlfilter/blacklists/proxy
> -#var/ipfire/urlfilter/blacklists/proxy/domains
> -#var/ipfire/urlfilter/blacklists/proxy/domains.db
> -#var/ipfire/urlfilter/blacklists/proxy/urls
> -#var/ipfire/urlfilter/blacklists/proxy/urls.db
> -#var/ipfire/urlfilter/blacklists/violence
> -#var/ipfire/urlfilter/blacklists/violence/domains
> -#var/ipfire/urlfilter/blacklists/violence/domains.db
> -#var/ipfire/urlfilter/blacklists/violence/expressions
> -#var/ipfire/urlfilter/blacklists/violence/urls
> -#var/ipfire/urlfilter/blacklists/violence/urls.db
> -#var/ipfire/urlfilter/blacklists/warez
> -#var/ipfire/urlfilter/blacklists/warez/domains
> -#var/ipfire/urlfilter/blacklists/warez/domains.db
> -#var/ipfire/urlfilter/blacklists/warez/urls
> -#var/ipfire/urlfilter/blacklists/warez/urls.db
> var/ipfire/urlfilter/settings
> var/ipfire/urlfilter/squidGuard.conf
> var/ipfire/urlfilter/version
> diff --git a/config/rootfiles/core/205/update.sh b/config/rootfiles/core/205/update.sh
> index 1c4acfbb0..a61f2ff7f 100644
> --- a/config/rootfiles/core/205/update.sh
> +++ b/config/rootfiles/core/205/update.sh
> @@ -37,6 +37,17 @@ done
> rm -rfv \
> /usr/local/bin/unboundctrl
> 
> +# remove old installed blacklist entries from urlfilter
> +# The blacklist is no longer installed
> +if [ ! -e "/var/ipfire/urlfilter/blacklists/.autoupdate.last" ] &&
> + [ -f "/var/ipfire/urlfilter/blacklists/README" ] &&
> + grep -q "(squidGuardRobot)" "/var/ipfire/urlfilter/blacklists/README"; then
> + find /var/ipfire/urlfilter/blacklists -mindepth 1 \
> + ! -path "/var/ipfire/urlfilter/blacklists/custom" \
> + ! -path "/var/ipfire/urlfilter/blacklists/custom/*" \
> + -exec rm -rf {} +
> +fi
> +
> # Extract files
> extract_files
> 
> diff --git a/config/urlfilter/blacklists.tar.gz b/config/urlfilter/blacklists.tar.gz
> deleted file mode 100644
> index dc21990be..000000000
> Binary files a/config/urlfilter/blacklists.tar.gz and /dev/null differ
> diff --git a/lfs/squidguard b/lfs/squidguard
> index 8ad95c472..63eb39125 100644
> --- a/lfs/squidguard
> +++ b/lfs/squidguard
> @@ -112,7 +112,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
> cp -f  $(DIR_CONF)/urlfilter/redirect_wrapper /usr/sbin
> chmod 755 /usr/sbin/redirect_wrapper
> 
> - /bin/tar --no-same-owner -xzf $(DIR_CONF)/urlfilter/blacklists.tar.gz -C /var/ipfire/urlfilter
> + -mkdir -p /var/ipfire/urlfilter/blacklists
> /usr/bin/perl $(DIR_CONF)/urlfilter/makeconf.pl
> touch /var/ipfire/urlfilter/settings
> chown -R nobody:nobody /var/ipfire/urlfilter
> -- 
> 2.47.3
> 
> 

Re: [PATCH] urlfilter: Remove bundled Toulouse blacklist

[p27m]

[-- Attachment #1: Type: text/plain, Size: 1731 bytes --]

Hi Michael,

Thank you for your reply.

The problem described in the bug report is actually quite simple.

Currently, the blacklist included in the IPFire repository and installed 
with IPFire dates from June 15, 2005, so it is now obsolete.
Since March 2026, the University of Toulouse has changed some 
directories in its blacklist into symbolic links.
Therefore, when restoring a backup containing a blacklist downloaded 
after this change, `tar` can fail because symbolic links cannot replace 
existing directories. As a result, the backup restoration fails.

@adolf previously added a fix to `backup.pl` which removes the existing 
contents of `/var/ipfire/urlfilter/blacklists/` before extracting the 
backup.

However, I recently discovered that the problem could still occur when 
restoring a backup from a backup ISO.

For this reason, I thought that the simplest solution, and the best way 
to avoid similar problems in the future, would be to remove the obsolete 
blacklist archive from the installation.

This patch does not prevent URLFilter from working without an installed 
blacklist. It also ensures that the old blacklist shipped with IPFire 
cannot interfere with restoring a newer blacklist from a backup.

I have tested the patch with upgrades, fresh ISO installations, and 
restoration of backups containing both the Toulouse blacklist and the 
IPFire DBL blacklist.

Best regards,

Philippe

Le 29/08/2026 à 17:31, Michael Tremer a écrit :
> Thank you very much for this patch.
>
> I could not quite figure out what you want to achieve with this change. Is this data being shipped causing some problems? The bug report did not give me the information I was looking for either.
>
> All the best,
> -Michael

[-- Attachment #2: Type: text/html, Size: 2409 bytes --]

Re: [PATCH] urlfilter: Remove bundled Toulouse blacklist

[Michael Tremer]

Hello Phil,

Yes, this makes sense so far. But what actually happens when squidGuard is being started with nothing? I remember that this list has been treated as a dummy. Did you test this case too?

-Michael

> On 29 Aug 2026, at 16:56, p27m <p27m@orange.fr> wrote:
> 
> Hi Michael,
> 
> Thank you for your reply.
> 
> The problem described in the bug report is actually quite simple.
> 
> Currently, the blacklist included in the IPFire repository and installed with IPFire dates from June 15, 2005, so it is now obsolete.
> Since March 2026, the University of Toulouse has changed some directories in its blacklist into symbolic links.
> Therefore, when restoring a backup containing a blacklist downloaded after this change, `tar` can fail because symbolic links cannot replace existing directories. As a result, the backup restoration fails.
> 
> @adolf previously added a fix to `backup.pl` which removes the existing contents of `/var/ipfire/urlfilter/blacklists/` before extracting the backup.
> 
> However, I recently discovered that the problem could still occur when restoring a backup from a backup ISO.
> 
> For this reason, I thought that the simplest solution, and the best way to avoid similar problems in the future, would be to remove the obsolete blacklist archive from the installation.
> 
> This patch does not prevent URLFilter from working without an installed blacklist. It also ensures that the old blacklist shipped with IPFire cannot interfere with restoring a newer blacklist from a backup.
> 
> I have tested the patch with upgrades, fresh ISO installations, and restoration of backups containing both the Toulouse blacklist and the IPFire DBL blacklist.
> 
> Best regards,
> 
> Philippe
> 
> Le 29/08/2026 à 17:31, Michael Tremer a écrit :
>> Thank you very much for this patch.
>> 
>> I could not quite figure out what you want to achieve with this change. Is this data being shipped causing some problems? The bug report did not give me the information I was looking for either.
>> 
>> All the best,
>> -Michael

Re: [PATCH] urlfilter: Remove bundled Toulouse blacklist

[p27m]

Hello Michael,

Yes, I Build it and install master ISO (CU204) on my test virtual machine.

SquidGuard starts without errors when the blacklist directory is empty 
(the patch preserves the custom list).
Naturally, no category-based filtering takes place until a blacklist has 
been downloaded.

I also ran a test using only a custom blacklist.
In this case, SquidGuard starts normally, and the custom domain is 
correctly blocked.

This point was documented in comment #13 of bug 13969:
https://bugzilla.ipfire.org/show_bug.cgi?id=13969#c13

Thus, removing the default blacklist does not prevent 
URLFilter/SquidGuard from starting or functioning.
The user can simply download or restore a blacklist later, or use only 
the custom blacklist.

Best regards,

Philippe

Le 07/09/2026 à 17:22, Michael Tremer a écrit :
> Hello Phil,
>
> Yes, this makes sense so far. But what actually happens when squidGuard is being started with nothing? I remember that this list has been treated as a dummy. Did you test this case too?
>
> -Michael
>
>> On 29 Aug 2026, at 16:56, p27m <p27m@orange.fr> wrote:
>>
>> Hi Michael,
>>
>> Thank you for your reply.
>>
>> The problem described in the bug report is actually quite simple.
>>
>> Currently, the blacklist included in the IPFire repository and installed with IPFire dates from June 15, 2005, so it is now obsolete.
>> Since March 2026, the University of Toulouse has changed some directories in its blacklist into symbolic links.
>> Therefore, when restoring a backup containing a blacklist downloaded after this change, `tar` can fail because symbolic links cannot replace existing directories. As a result, the backup restoration fails.
>>
>> @adolf previously added a fix to `backup.pl` which removes the existing contents of `/var/ipfire/urlfilter/blacklists/` before extracting the backup.
>>
>> However, I recently discovered that the problem could still occur when restoring a backup from a backup ISO.
>>
>> For this reason, I thought that the simplest solution, and the best way to avoid similar problems in the future, would be to remove the obsolete blacklist archive from the installation.
>>
>> This patch does not prevent URLFilter from working without an installed blacklist. It also ensures that the old blacklist shipped with IPFire cannot interfere with restoring a newer blacklist from a backup.
>>
>> I have tested the patch with upgrades, fresh ISO installations, and restoration of backups containing both the Toulouse blacklist and the IPFire DBL blacklist.
>>
>> Best regards,
>>
>> Philippe
>>
>> Le 29/08/2026 à 17:31, Michael Tremer a écrit :
>>> Thank you very much for this patch.
>>>
>>> I could not quite figure out what you want to achieve with this change. Is this data being shipped causing some problems? The bug report did not give me the information I was looking for either.
>>>
>>> All the best,
>>> -Michael
>

Re: [PATCH] urlfilter: Remove bundled Toulouse blacklist

[Adolf Belka]

Hi All,

I am following up on this as it has not had any follow-up for a while..

If there is a concern on the potential impact of not having any bundled Toulouse blocklist in the URL Filter, an alternative would be to have a newer version of the Toulouse Blocklist that includes the symlinks approach that Toulouse started using earlier this year.

Would that be a viable approach? That would then keep the current default status of having a blocklist defined but using one that has the symlinks and therefore does not end up with the problem of trying to create a symlink with the same name as an existing file.

Regards,

Adolf.


On 07/09/2026 18:51, p27m wrote:
> Hello Michael,
> 
> Yes, I Build it and install master ISO (CU204) on my test virtual machine.
> 
> SquidGuard starts without errors when the blacklist directory is empty (the patch preserves the custom list).
> Naturally, no category-based filtering takes place until a blacklist has been downloaded.
> 
> I also ran a test using only a custom blacklist.
> In this case, SquidGuard starts normally, and the custom domain is correctly blocked.
> 
> This point was documented in comment #13 of bug 13969:
> https://bugzilla.ipfire.org/show_bug.cgi?id=13969#c13
> 
> Thus, removing the default blacklist does not prevent URLFilter/SquidGuard from starting or functioning.
> The user can simply download or restore a blacklist later, or use only the custom blacklist.
> 
> Best regards,
> 
> Philippe
> 
> Le 07/09/2026 à 17:22, Michael Tremer a écrit :
>> Hello Phil,
>>
>> Yes, this makes sense so far. But what actually happens when squidGuard is being started with nothing? I remember that this list has been treated as a dummy. Did you test this case too?
>>
>> -Michael
>>
>>> On 29 Aug 2026, at 16:56, p27m <p27m@orange.fr> wrote:
>>>
>>> Hi Michael,
>>>
>>> Thank you for your reply.
>>>
>>> The problem described in the bug report is actually quite simple.
>>>
>>> Currently, the blacklist included in the IPFire repository and installed with IPFire dates from June 15, 2005, so it is now obsolete.
>>> Since March 2026, the University of Toulouse has changed some directories in its blacklist into symbolic links.
>>> Therefore, when restoring a backup containing a blacklist downloaded after this change, `tar` can fail because symbolic links cannot replace existing directories. As a result, the backup restoration fails.
>>>
>>> @adolf previously added a fix to `backup.pl` which removes the existing contents of `/var/ipfire/urlfilter/blacklists/` before extracting the backup.
>>>
>>> However, I recently discovered that the problem could still occur when restoring a backup from a backup ISO.
>>>
>>> For this reason, I thought that the simplest solution, and the best way to avoid similar problems in the future, would be to remove the obsolete blacklist archive from the installation.
>>>
>>> This patch does not prevent URLFilter from working without an installed blacklist. It also ensures that the old blacklist shipped with IPFire cannot interfere with restoring a newer blacklist from a backup.
>>>
>>> I have tested the patch with upgrades, fresh ISO installations, and restoration of backups containing both the Toulouse blacklist and the IPFire DBL blacklist.
>>>
>>> Best regards,
>>>
>>> Philippe
>>>
>>> Le 29/08/2026 à 17:31, Michael Tremer a écrit :
>>>> Thank you very much for this patch.
>>>>
>>>> I could not quite figure out what you want to achieve with this change. Is this data being shipped causing some problems? The bug report did not give me the information I was looking for either.
>>>>
>>>> All the best,
>>>> -Michael
>>
> 

Re: [PATCH] urlfilter: Remove bundled Toulouse blacklist

[p27m]

Hi Adolf,

Thank you for following up on this.

The |blacklists.tar.gz| Toulouse blocklist currently present in the 
IPFire repository dates back to 2015 and is one of the remaining parts 
inherited from the old IPCop URL Filter add-on. It is not part of the 
SquidGuard package itself.

Replacing |blacklists.tar.gz| with a newer version would indeed be a 
viable solution to the current problem with directories being replaced 
by symlinks. It should work without any code changes.

However, this would only be a workaround for the underlying problem. 
There is no guarantee that the structure of the Toulouse blocklist will 
not change again in the future, in which case the same problem could 
reappear.

I also wonder whether it makes sense to continue installing a blocklist 
which may become obsolete again relatively quickly, or potentially 
disappear altogether.

There is already a blocklist maintained by IPFire at:

https://dbl.ipfire.org/lists/squidguard.tar.gz

It might therefore make more sense to use this list as the bundled 
default instead.

However, this would not address the problem when restoring a backup 
containing a Toulouse blocklist that was previously in use by the user. 
The restore process could still encounter the same directory/symlink 
conflict.

The solution I proposed is intended to address both aspects: removing 
the obsolete bundled Toulouse list and making the restore process safe 
in case an older Toulouse list is present in a backup.

That said, you are the experts on the IPFire codebase and its long-term 
maintenance, so I will of course leave the final choice to you.

Regards,

Philippe


Le 27/09/2026 à 12:44, Adolf Belka a écrit :
> Hi All,
>
> I am following up on this as it has not had any follow-up for a while..
>
> If there is a concern on the potential impact of not having any 
> bundled Toulouse blocklist in the URL Filter, an alternative would be 
> to have a newer version of the Toulouse Blocklist that includes the 
> symlinks approach that Toulouse started using earlier this year.
>
> Would that be a viable approach? That would then keep the current 
> default status of having a blocklist defined but using one that has 
> the symlinks and therefore does not end up with the problem of trying 
> to create a symlink with the same name as an existing file.
>
> Regards,
>
> Adolf. 

Re: [PATCH] urlfilter: Remove bundled Toulouse blacklist

[Michael Tremer]

Hello,

I generally do agree with dropping the old data. It is more than obsolete, but that leaves us with some new problems:

* No data is available immediately. Users would have to go down to the section of the page where they can select a list, enable the updates and trigger a manual update. Only then they will be able to start the URL Filter. This is a very convoluted process to enable a feature that should be very easy to configure.

* If we shipped IPFire DBL by default, that would be a very sensible solution actually. But then there is a danger that people will enable the categories they want, enable URL Filter, but never enable the automatic updates. If we enable the automatic updates ourselves, we would run into the problem that a lot of IPFire instances all over the world would be updating their blacklists although they are never being used which will simply waste disk space and bandwidth.

So I think we need a change which is to ship IPFire DBL for new installations, enable the auto-updates by default, but then only make them actually work if URL filter is enabled at the top. That will solve the problem in the latter case.

Does that sound good to you, too?

-Michael

> On 27 Sep 2026, at 13:17, p27m <p27m@orange.fr> wrote:
> 
> Hi Adolf,
> 
> Thank you for following up on this.
> 
> The |blacklists.tar.gz| Toulouse blocklist currently present in the IPFire repository dates back to 2015 and is one of the remaining parts inherited from the old IPCop URL Filter add-on. It is not part of the SquidGuard package itself.
> 
> Replacing |blacklists.tar.gz| with a newer version would indeed be a viable solution to the current problem with directories being replaced by symlinks. It should work without any code changes.
> 
> However, this would only be a workaround for the underlying problem. There is no guarantee that the structure of the Toulouse blocklist will not change again in the future, in which case the same problem could reappear.
> 
> I also wonder whether it makes sense to continue installing a blocklist which may become obsolete again relatively quickly, or potentially disappear altogether.
> 
> There is already a blocklist maintained by IPFire at:
> 
> https://dbl.ipfire.org/lists/squidguard.tar.gz
> 
> It might therefore make more sense to use this list as the bundled default instead.
> 
> However, this would not address the problem when restoring a backup containing a Toulouse blocklist that was previously in use by the user. The restore process could still encounter the same directory/symlink conflict.
> 
> The solution I proposed is intended to address both aspects: removing the obsolete bundled Toulouse list and making the restore process safe in case an older Toulouse list is present in a backup.
> 
> That said, you are the experts on the IPFire codebase and its long-term maintenance, so I will of course leave the final choice to you.
> 
> Regards,
> 
> Philippe
> 
> 
> Le 27/09/2026 à 12:44, Adolf Belka a écrit :
>> Hi All,
>> 
>> I am following up on this as it has not had any follow-up for a while..
>> 
>> If there is a concern on the potential impact of not having any bundled Toulouse blocklist in the URL Filter, an alternative would be to have a newer version of the Toulouse Blocklist that includes the symlinks approach that Toulouse started using earlier this year.
>> 
>> Would that be a viable approach? That would then keep the current default status of having a blocklist defined but using one that has the symlinks and therefore does not end up with the problem of trying to create a symlink with the same name as an existing file.
>> 
>> Regards,
>> 
>> Adolf.
> 

Re: [PATCH] urlfilter: Remove bundled Toulouse blacklist

[p27m]

Hi Michael,

Yes, I think this is a good functional solution, although it would 
require a different implementation.
However, it does not address the original problem that led me to propose 
this patch: the failure of a backup restore on fresh install when the 
backup contains a recent Toulouse blacklist.

In fact, the IPFire DBL list also contains the following directories:
- ads
- porn
- violence

These are symlinks in the current Toulouse `blacklists.tar.gz`:
- ads -> publicite
- porn -> adult
- violence -> agressif

Therefore, if IPFire DBL is installed by default in new IPFire system, 
restoring a backup containing a recent Toulouse blacklist could still 
result in exactly the same directory/symlink conflict during the untar 
restoration.

Regards,
--
Philippe

Le 29/09/2026 à 15:19, Michael Tremer a écrit :
> Hello,
>
> I generally do agree with dropping the old data. It is more than obsolete, but that leaves us with some new problems:
>
> * No data is available immediately. Users would have to go down to the section of the page where they can select a list, enable the updates and trigger a manual update. Only then they will be able to start the URL Filter. This is a very convoluted process to enable a feature that should be very easy to configure.
>
> * If we shipped IPFire DBL by default, that would be a very sensible solution actually. But then there is a danger that people will enable the categories they want, enable URL Filter, but never enable the automatic updates. If we enable the automatic updates ourselves, we would run into the problem that a lot of IPFire instances all over the world would be updating their blacklists although they are never being used which will simply waste disk space and bandwidth.
>
> So I think we need a change which is to ship IPFire DBL for new installations, enable the auto-updates by default, but then only make them actually work if URL filter is enabled at the top. That will solve the problem in the latter case.
>
> Does that sound good to you, too?
>
> -Michael

Re: [PATCH] urlfilter: Remove bundled Toulouse blacklist

[Michael Tremer]

Hello Philippe,

Thanks for getting back so quickly.

I suppose we will just have to clear the directory before the lists are being restored. That should eliminate any problems after extraction.

Where the list should be backed up at all is a completely different story. We have been talking about this many times. Some people expect only the configuration to be included in the backup and others expect a full snapshot of the system. There is also the question of when the backup is going to be restored: A couple of days later you will be okay with a slightly out-of-date list; a couple of months later you will be carrying a lot of data that is basically worthless. But we are doing the same with the IPS rulesets and so on, so we should at least be consistent.

Best,
-Michael

> On 29 Sep 2026, at 16:07, p27m <p27m@orange.fr> wrote:
> 
> Hi Michael,
> 
> Yes, I think this is a good functional solution, although it would require a different implementation.
> However, it does not address the original problem that led me to propose this patch: the failure of a backup restore on fresh install when the backup contains a recent Toulouse blacklist.
> 
> In fact, the IPFire DBL list also contains the following directories:
> - ads
> - porn
> - violence
> 
> These are symlinks in the current Toulouse `blacklists.tar.gz`:
> - ads -> publicite
> - porn -> adult
> - violence -> agressif
> 
> Therefore, if IPFire DBL is installed by default in new IPFire system, restoring a backup containing a recent Toulouse blacklist could still result in exactly the same directory/symlink conflict during the untar restoration.
> 
> Regards,
> --
> Philippe
> 
> Le 29/09/2026 à 15:19, Michael Tremer a écrit :
>> Hello,
>> 
>> I generally do agree with dropping the old data. It is more than obsolete, but that leaves us with some new problems:
>> 
>> * No data is available immediately. Users would have to go down to the section of the page where they can select a list, enable the updates and trigger a manual update. Only then they will be able to start the URL Filter. This is a very convoluted process to enable a feature that should be very easy to configure.
>> 
>> * If we shipped IPFire DBL by default, that would be a very sensible solution actually. But then there is a danger that people will enable the categories they want, enable URL Filter, but never enable the automatic updates. If we enable the automatic updates ourselves, we would run into the problem that a lot of IPFire instances all over the world would be updating their blacklists although they are never being used which will simply waste disk space and bandwidth.
>> 
>> So I think we need a change which is to ship IPFire DBL for new installations, enable the auto-updates by default, but then only make them actually work if URL filter is enabled at the top. That will solve the problem in the latter case.
>> 
>> Does that sound good to you, too?
>> 
>> -Michael
> 

Re: [PATCH] urlfilter: Remove bundled Toulouse blacklist

[p27m]

Hi ,

“Clear the directory before the lists are being restored” is actually 
already what Adolf implemented in |backup.pl| in CU203 to fix this bug.

However, this is not done in |installer/hw.c|, as discussed in Bugzilla:
https://bugzilla.ipfire.org/show_bug.cgi?id=13969#c5

So clearing the directory before restoring the lists there would indeed 
be another approach that should work.

My intention in proposing this patch was simply to help and avoid 
additional work, rather than to argue about which solution is the best one.

I think there are several possible ways to address the issue, so I will 
leave it to you to choose and implement the approach that you consider 
most appropriate for IPFire.

Regards,

Philippe


Le 29/09/2026 à 17:10, Michael Tremer a écrit :
> Hello Philippe,
>
> Thanks for getting back so quickly.
>
> I suppose we will just have to clear the directory before the lists are being restored. That should eliminate any problems after extraction.
>
> Where the list should be backed up at all is a completely different story. We have been talking about this many times. Some people expect only the configuration to be included in the backup and others expect a full snapshot of the system. There is also the question of when the backup is going to be restored: A couple of days later you will be okay with a slightly out-of-date list; a couple of months later you will be carrying a lot of data that is basically worthless. But we are doing the same with the IPS rulesets and so on, so we should at least be consistent.
>
> Best,
> -Michael
>
>> On 29 Sep 2026, at 16:07, p27m <p27m@orange.fr> wrote:
>>
>> Hi Michael,
>>
>> Yes, I think this is a good functional solution, although it would require a different implementation.
>> However, it does not address the original problem that led me to propose this patch: the failure of a backup restore on fresh install when the backup contains a recent Toulouse blacklist.
>>
>> In fact, the IPFire DBL list also contains the following directories:
>> - ads
>> - porn
>> - violence
>>
>> These are symlinks in the current Toulouse `blacklists.tar.gz`:
>> - ads -> publicite
>> - porn -> adult
>> - violence -> agressif
>>
>> Therefore, if IPFire DBL is installed by default in new IPFire system, restoring a backup containing a recent Toulouse blacklist could still result in exactly the same directory/symlink conflict during the untar restoration.
>>
>> Regards,
>> --
>> Philippe
>>
>> Le 29/09/2026 à 15:19, Michael Tremer a écrit :
>>> Hello,
>>>
>>> I generally do agree with dropping the old data. It is more than obsolete, but that leaves us with some new problems:
>>>
>>> * No data is available immediately. Users would have to go down to the section of the page where they can select a list, enable the updates and trigger a manual update. Only then they will be able to start the URL Filter. This is a very convoluted process to enable a feature that should be very easy to configure.
>>>
>>> * If we shipped IPFire DBL by default, that would be a very sensible solution actually. But then there is a danger that people will enable the categories they want, enable URL Filter, but never enable the automatic updates. If we enable the automatic updates ourselves, we would run into the problem that a lot of IPFire instances all over the world would be updating their blacklists although they are never being used which will simply waste disk space and bandwidth.
>>>
>>> So I think we need a change which is to ship IPFire DBL for new installations, enable the auto-updates by default, but then only make them actually work if URL filter is enabled at the top. That will solve the problem in the latter case.
>>>
>>> Does that sound good to you, too?
>>>
>>> -Michael
>