From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bernhard Bitsch To: development@lists.ipfire.org Subject: Re: [PATCH] RPZ: update code to include WEBGUI and additional languages Date: Thu, 06 Feb 2025 20:35:45 +0100 Message-ID: In-Reply-To: <20250206163522.2363178-1-jon.murphy@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0209691955325930092==" List-Id: --===============0209691955325930092== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Tested-by: Bernhard Bitsch I am one of the first testers of this concept and addon. Thx Jon for bringing up this topic and doing the work to make it running. Some comments about this functionality: - From time to time there are discussions about DNS filters like PiHole=20 in the community. It is a bit complicated to define the data path of DNS=20 requests going entirely through the PiHole device. - One solution would be to integrate the filter into IPFire. Don't know=20 about the effort to do this. - The RPZ mechanism is integrated in unbound, our standard DNS 'server'.=20 Unbound is built with RPZ support, yet. - The lists for RPZ filtering are auto-updated using SOA records. - The 'Hagezi lists' ( link in Jon's wiki article ) seem to be well=20 maintained and updated daily. It is necessary to examine these by a=20 greater amount of IPFire users. Especially we should determine the=20 amount of extra memory usage for the lists. - Jon extends the functionality by local allow and block lists. It isn't=20 clear enough, yet, how these can effective integrated in the RPZ=20 mechanism. Is it sufficient to 'reload' the unbound configuration or is=20 it necessary to 'restart' unbound. Am 06.02.2025 um 17:35 schrieb Jon Murphy: > What is it? > Response Policy Zone (RPZ) is a mechanism to define local policies in a > standardized way and load those policies from external sources. > Bottom line: RPZ allows admins to easily block access to websites via DNS l= ookup. >=20 > RPZ can block websites via categories. Examples include: fake websites, an= noying > pop-up ads, newly registered domains, DoH bypass sites, bad "host" services, > maliscious top level domains (e.g., *.zip, *.mov), piracy, gambling, pornog= raphy, > and more. RPZ lists come from various RPZ providers and their available > catagories. >=20 > This RPZ add-on enables the RPZ functionality by adding a couple lines in a > configuration file. This add-on simply adds configuration files and adds > scripts (config, metrics and sleep) to make RPZ easier for the admin to use. >=20 > The RPZ scripts include additional languages: German, Spanish, French, Turk= ish, > and Italian. >=20 > RPZ itself was release in 2010 and has been part of the IPFire build since = ~2015. >=20 > Why is it needed? What is its value? >=20 > - The RPZ concept places this filtering into IPFire, our internet access > gateway, which is (should be) solely used as DNS source of the internal net= work. >=20 > - As most sites use HTTPS it makes it difficult to filter traffic with URL > Filter without also properly configuring conventional (non-transparent) > mode on the proxy. RPZ is a nice replacement for the URL Filter. >=20 > - No need to install and maintain an additional device like PiHole or AdB= lock > browser extensions on multiple user devices. >=20 > - This is an additional layer of protection for users. Less worry someone= will > click on something that gets them into trouble. And, saying this with empha= sis, > the ability to do it in one place! >=20 > - Blocked sites save on unneeded traffic and can lessen the threat of mal= ware > in advertisements >=20 > - Logging allows the admin to see the site blocked and take actions >=20 > - RPZ will be used at the home, home-office (work from home), schools, > ministerial, and at the office. Device counts are small (2-6) to medium (~= 80) > to mediam-large (200+). >=20 > - RPZ can block ads, popups, phishing, scammers, spyware, malware, annoyi= ng > popups, NSFW links, DOH servers, and the usual internet trash. >=20 > ------------------------------ >=20 > Change Log for RPZ add-on >=20 > rpz-1.0.0-18 on 2025-02-05 > - Build for approval & release as IPFire add-on >=20 > --- >=20 > rpz-beta-0.1.18-18.ipfire on 2025-02-01 > rpz.cgi: > - new feature: added a mod key to force a unbound restart >=20 > rpz-config and rpz-make: > - new feature: added action for unbound restart `rpz-config unbound-resta= rt` >=20 > rpz-metrics: > - simple reformatting > - rename far right column from "last update" to "last download" >=20 > --- >=20 > rpz-beta-0.1.17-17.ipfire on 2024-12-09 > rpz-make > - bug fix: corrected validation regex for wildcards like: `*.domain.com` >=20 > --- >=20 > rpz-beta-0.1.16-16.ipfire on 2024-11-18 > rpz-make > - new feature: updated validation regex > - bug fix: moved validation to beginning of process. Now we validate bef= ore > creating config files. >=20 > rpz.cgi: > - new feature: use CSS color variables of the main ipfire theme > - bug fix: empty zonefile remarks were stored as =E2=80=9Cundef=E2=80=9D = and caused a warning > - bug fix: HTML textarea removes the first empty line in a custom list > - thank you Leo! >=20 > --- >=20 > rpz-beta-0.1.15-15.ipfire on 2024-11-04 > rpz.cgi: > - new feature: added new language file for Turkish (thank you Peppe) >=20 > rpz-make > - bug fix: corrected empty allow/block list issue. An empty allow/block = list > will now remove contents of allow/block.rpz files and remove unneeded > allow/block.conf file. (thank you iptom) >=20 > --- >=20 > rpz-beta-0.1.14-14.ipfire on 2024-10-29 > rpz-config: > - bug fix: correct missing rpz extension. `rpz-config list` displayed URL > incorrectly (thank you Bernhard) >=20 > rpz.cgi: > - bug fix: remove extra `"` in language files (thank you Bernhard) > - new feature: slightly dim "apply" button when not enabled >=20 > --- >=20 > rpz-beta-0.1.13-13.ipfire on 2024-10-27 > - skipped >=20 > --- >=20 > rpz-beta-0.1.12-12.ipfire on 2024-10-21 > rpz.cgi: > - new feature: added new language file for French (thank you gw-ipfire) >=20 > --- >=20 > rpz-beta-0.1.11-11.ipfire on 2024-10-18 > rpz.cgi: > - new feature: added new language file for Italian (thank you umberto) > - new feature: added new language file for Spanish (thank you Roberto) >=20 > --- >=20 > rpz-beta-0.1.10-10.ipfire on 2024-10-15 > rpz-make: > - bug fix: corrected validation error for a custom list entry (thank you = siosios) > - e.g., `*.cloudflare-dns.com` >=20 > install.sh: > - bug fix: add chown to correct user created files >=20 > update.sh: > - bug fix: add chown to correct user created files (thank you siosios) >=20 > --- >=20 > rpz-beta-0.1.9-9.ipfire on 2024-10-08 > rpz.cgi: > - new feature: added new language file for German (thank you Leo) > - bug fix: add missing "rpz exitcode 110" > - bug fix: corrected missing RPZ menu item at menu > IPFire >=20 > --- >=20 > rpz-beta-0.1.8-8.ipfire on 2024-10-04 > - skipped >=20 > --- >=20 > rpz-beta-0.1.7-7.ipfire on 2024-10-03 > All: > - new feature: includes beta version numbers for pakfire package, > instead of only `rpz-1.0.0-1.ipfire`, for each release. >=20 > rpz.cgi: > - new feature: added new WebGUI at `rpz.cgi` > - a BIG thank you to Leo Hofmann for all of his work creating the webg= ui!! > - bug fix: corrected missing RPZ menu item at menu > IPFire >=20 > rpz-make: > - new feature: validate entries in allowlist and blocklist > - new feature: add "no-reload" option for WebGUI >=20 > rpz-metrics: > - new feature: info can be sorted by name, by hit count, by line count, by > "enabled" list or all lists >=20 > backups: > - bug fix: include all files in `/var/ipfire/dns/rpz` directory in backup >=20 > update.sh: > - bug fix: corrected ownership for `/var/ipfire/dns/rpz` directory during= an > update >=20 > Build: > - bug fix: `block.rpz.conf` and `block.rpz` from build. Files to be crea= ted > by `rpz-make` >=20 > WebGUI and German language file > Contribution-by: Leo-Andres Hofmann >=20 > Spanish language file > Contribution-by: Roberto Pe=C3=B1a >=20 > Italian language file > Contribution-by: Umberto Parma >=20 > French language file > Contribution-by: gw-ipfire >=20 > Turkish language file > Contribution-by: Peppe Tech >=20 > Contribution-by: Bernhard Bitsch > Contribution-by: Erik Kapfer > Signed-off-by: Jon Murphy --- > config/backup/includes/rpz | 4 + > config/cfgroot/manualpages | 1 + > config/menu/EX-rpz.menu | 6 + > config/rootfiles/common/configroot | 1 + > config/rootfiles/common/web-user-interface | 1 + > config/rootfiles/packages/rpz | 20 + > config/rpz/00-rpz.conf | 10 + > config/rpz/rpz-config | 130 +++ > config/rpz/rpz-functions | 85 ++ > config/rpz/rpz-make | 203 +++++ > config/rpz/rpz-metrics | 170 ++++ > config/rpz/rpz-sleep | 58 ++ > config/rpz/rpz.de.pl | 30 + > config/rpz/rpz.en.pl | 30 + > config/rpz/rpz.es.pl | 30 + > config/rpz/rpz.fr.pl | 30 + > config/rpz/rpz.it.pl | 30 + > config/rpz/rpz.tr.pl | 30 + > html/cgi-bin/rpz.cgi | 923 +++++++++++++++++++++ > lfs/rpz | 96 +++ > make.sh | 3 +- > src/paks/rpz/install.sh | 36 + > src/paks/rpz/uninstall.sh | 38 + > src/paks/rpz/update.sh | 52 ++ > 24 files changed, 2016 insertions(+), 1 deletion(-) > create mode 100644 config/backup/includes/rpz > create mode 100644 config/menu/EX-rpz.menu > create mode 100644 config/rootfiles/packages/rpz > create mode 100644 config/rpz/00-rpz.conf > create mode 100644 config/rpz/rpz-config > create mode 100644 config/rpz/rpz-functions > create mode 100644 config/rpz/rpz-make > create mode 100755 config/rpz/rpz-metrics > create mode 100755 config/rpz/rpz-sleep > create mode 100644 config/rpz/rpz.de.pl > create mode 100644 config/rpz/rpz.en.pl > create mode 100644 config/rpz/rpz.es.pl > create mode 100644 config/rpz/rpz.fr.pl > create mode 100644 config/rpz/rpz.it.pl > create mode 100644 config/rpz/rpz.tr.pl > create mode 100644 html/cgi-bin/rpz.cgi > create mode 100644 lfs/rpz > create mode 100644 src/paks/rpz/install.sh > create mode 100644 src/paks/rpz/uninstall.sh > create mode 100644 src/paks/rpz/update.sh >=20 > diff --git a/config/backup/includes/rpz b/config/backup/includes/rpz > new file mode 100644 > index 000000000..36513e494 > --- /dev/null > +++ b/config/backup/includes/rpz > @@ -0,0 +1,4 @@ > +/var/ipfire/dns/rpz/* > +/etc/unbound/zonefiles/allow.rpz > +/etc/unbound/zonefiles/block.rpz > +/etc/unbound/local.d/*rpz.conf > diff --git a/config/cfgroot/manualpages b/config/cfgroot/manualpages > index 1f7e01efc..d3a48c633 100644 > --- a/config/cfgroot/manualpages > +++ b/config/cfgroot/manualpages > @@ -70,6 +70,7 @@ pakfire.cgi=3Dconfiguration/ipfire/pakfire > wlanap.cgi=3Daddons/wireless > tor.cgi=3Daddons/tor > samba.cgi=3Daddons/samba > +rpz.cgi=3Daddons/rpz > =20 > # Logs menu > logs.cgi/summary.dat=3Dconfiguration/logs/summary > diff --git a/config/menu/EX-rpz.menu b/config/menu/EX-rpz.menu > new file mode 100644 > index 000000000..2f4daf410 > --- /dev/null > +++ b/config/menu/EX-rpz.menu > @@ -0,0 +1,6 @@ > +$subipfire->{'20.rpz'} =3D { > + 'caption' =3D> $Lang::tr{'rpz'}, > + 'uri' =3D> '/cgi-bin/rpz.cgi', > + 'title' =3D> "RPZ", > + 'enabled' =3D> 1, > +}; > diff --git a/config/rootfiles/common/configroot b/config/rootfiles/common/c= onfigroot > index 9839eee45..b30d6aae4 100644 > --- a/config/rootfiles/common/configroot > +++ b/config/rootfiles/common/configroot > @@ -120,6 +120,7 @@ var/ipfire/menu.d/70-log.menu > #var/ipfire/menu.d/EX-apcupsd.menu > #var/ipfire/menu.d/EX-guardian.menu > #var/ipfire/menu.d/EX-mympd.menu > +#var/ipfire/menu.d/EX-rpz.menu > #var/ipfire/menu.d/EX-samba.menu > #var/ipfire/menu.d/EX-tor.menu > #var/ipfire/menu.d/EX-transmission.menu > diff --git a/config/rootfiles/common/web-user-interface b/config/rootfiles/= common/web-user-interface > index 816241dae..e00464076 100644 > --- a/config/rootfiles/common/web-user-interface > +++ b/config/rootfiles/common/web-user-interface > @@ -69,6 +69,7 @@ srv/web/ipfire/cgi-bin/proxy.cgi > srv/web/ipfire/cgi-bin/qos.cgi > srv/web/ipfire/cgi-bin/remote.cgi > srv/web/ipfire/cgi-bin/routing.cgi > +#srv/web/ipfire/cgi-bin/rpz.cgi > #srv/web/ipfire/cgi-bin/samba.cgi > srv/web/ipfire/cgi-bin/services.cgi > srv/web/ipfire/cgi-bin/shutdown.cgi > diff --git a/config/rootfiles/packages/rpz b/config/rootfiles/packages/rpz > new file mode 100644 > index 000000000..1c8663049 > --- /dev/null > +++ b/config/rootfiles/packages/rpz > @@ -0,0 +1,20 @@ > +etc/unbound/local.d/00-rpz.conf > +etc/unbound/zonefiles > +etc/unbound/zonefiles/allow.rpz > +usr/sbin/rpz-config > +usr/sbin/rpz-functions > +usr/sbin/rpz-make > +usr/sbin/rpz-metrics > +usr/sbin/rpz-sleep > +var/ipfire/addon-lang/rpz.de.pl > +var/ipfire/addon-lang/rpz.en.pl > +var/ipfire/addon-lang/rpz.es.pl > +var/ipfire/addon-lang/rpz.fr.pl > +var/ipfire/addon-lang/rpz.it.pl > +var/ipfire/addon-lang/rpz.tr.pl > +var/ipfire/backup/addons/includes/rpz > +var/ipfire/dns/rpz > +var/ipfire/dns/rpz/allowlist > +var/ipfire/dns/rpz/blocklist > +var/ipfire/menu.d/EX-rpz.menu > +srv/web/ipfire/cgi-bin/rpz.cgi > diff --git a/config/rpz/00-rpz.conf b/config/rpz/00-rpz.conf > new file mode 100644 > index 000000000..f005a4f2e > --- /dev/null > +++ b/config/rpz/00-rpz.conf > @@ -0,0 +1,10 @@ > +server: > + module-config: "respip validator iterator" > + > +rpz: > + name: allow.rpz > + zonefile: /etc/unbound/zonefiles/allow.rpz > + rpz-action-override: passthru > + rpz-log: yes > + rpz-log-name: allow > + rpz-signal-nxdomain-ra: yes > diff --git a/config/rpz/rpz-config b/config/rpz/rpz-config > new file mode 100644 > index 000000000..c72d50f9b > --- /dev/null > +++ b/config/rpz/rpz-config > @@ -0,0 +1,130 @@ > +#!/bin/bash > +##########################################################################= ##### > +# = # > +# IPFire.org - A linux based firewall = # > +# Copyright (C) 2024-2025 IPFire Team = # > +# = # > +# This program is free software: you can redistribute it and/or modify = # > +# it under the terms of the GNU General Public License as published by = # > +# the Free Software Foundation, either version 3 of the License, or = # > +# (at your option) any later version. = # > +# = # > +# This program is distributed in the hope that it will be useful, = # > +# but WITHOUT ANY WARRANTY; without even the implied warranty of = # > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the = # > +# GNU General Public License for more details. = # > +# = # > +# You should have received a copy of the GNU General Public License = # > +# along with this program. If not, see . = # > +# = # > +##########################################################################= ##### > + > +version=3D"2025-01-11 - v44" > + > +############### Functions ############### > + > +source /usr/sbin/rpz-functions > + > +############### Main ############### > + > +tagName=3D"unbound" > + > +rpzAction=3D"${1}" # input RPZ action > +rpzName=3D"${2}" # input RPZ name > +rpzURL=3D"${3}" # input RPZ URL > +rpzOption1=3D"${4}" # input RPZ option #1 > +rpzOption2=3D"${5}" # input RPZ option #2 > + > +rpzConfig=3D"/etc/unbound/local.d/${rpzName}.rpz.conf" # output zone c= onf file > +rpzFile=3D"/etc/unbound/zonefiles/${rpzName}.rpz" # output for RP= Z file > + > +rpzLog=3D"yes" # log default is yes > +ucReload=3D"yes" # reload default is yes > + > +while [[ $# -gt 0 ]] ; do > + case "$1" in > + --no-log ) rpzLog=3D"no" ;; > + --no-reload ) ucReload=3D"no" ; checkConf=3D"no" ;; > + esac > + shift # Shift after checking all the cases to get next option > +done > + > +case "${rpzAction}" in > + # add new rpz list > + add ) > + check_name "${rpzName}" # is this a valid name? > + # does this config already exist? If yes, then exit > + if [[ -f "${rpzConfig}" ]] ; then > + msg_log "error: rpz: duplicate - ${rpzConfig} already exists. = exit" > + exit 104 > + fi > + > + # is this a valid URL? > + regex=3D'^https://[-[:alnum:]\+&@#/%?=3D~_|!:,.;]*[-[:alnum:]\+&@#= /%=3D~_|]' > + if ! [[ "${rpzURL}" =3D~ $regex ]] ; then > + msg_log "error: rpz: the URL is not valid: \"${rpzURL}\". exit= ." > + exit 105 > + fi > + > + # create the zone config file > + { > + echo "rpz:" > + echo " name: ${rpzName}.rpz" > + echo " zonefile: ${rpzFile}" > + echo " url: ${rpzURL}" > + echo " rpz-action-override: nxdomain" > + echo " rpz-log: ${rpzLog}" > + echo " rpz-log-name: ${rpzName}" > + echo " rpz-signal-nxdomain-ra: yes" > + } > "${rpzConfig}" > + > + # set-up zonefile > + # create an empty rpz file if it does not exist > + if [[ ! -f "${rpzFile}" ]] ; then > + touch "${rpzFile}" > + # unbound requires these settings for rpz files > + set_permissions "${rpzFile}" "${rpzConfig}" > + fi > + ;; > + > + # trash config file & rpz file > + remove ) > + if ! [[ -f "${rpzConfig}" ]] ; then > + msg_log "error: rpz: cannot remove ${rpzConfig}, does not exis= t. exit" > + exit 106 > + fi > + > + msg_log "info: rpz: remove config file & rpz file \"${rpzName}\"" > + rm "${rpzConfig}" > + rm "${rpzFile}" > + ;; > + > + reload ) > + check_unbound_conf "${checkConf}" > + ;; > + > + list ) > + awk -F':' '/^\s*name:/{ gsub(/[[:blank:]]|\.rpz/, "",$2) ; NAME=3D= $2 } \ > + /^\s*url:/{ gsub(/[[:blank:]]/, "") ; print NAME"=3D"$2":"$3} = ' \ > + /etc/unbound/local.d/*rpz.conf > + exit > + ;; > + > + unbound-restart ) > + check_unbound_conf "${checkConf}" > + unbound_restart > + exit > + ;; > + > + * ) > + msg_log "error: rpz: missing or incorrect parameter" > + printf "Usage: $(basename "$0")