From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer To: development@lists.ipfire.org Subject: Re: [PATCH] RPZ: bug fix and code update Date: Thu, 15 Aug 2024 16:33:36 +0100 Message-ID: <47D3CC0D-F6BB-42BA-A8C7-6CE145CC521F@ipfire.org> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2499773900285721807==" List-Id: --===============2499773900285721807== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello, > On 14 Aug 2024, at 18:14, jon wrote: >=20 > Michael, >=20 > Sorry for putting you on the spot, but what do you want to do with this RPZ= add-on? I am not sure this is a question for only me. I personally just don=E2=80=99t have any capacity to take on another rather l= arge project as I have a huge backlog of so many things and I feel like a lon= ely fighter getting these all over the line. For my own sanity I need to have= a couple of those closed before thinking about the next ones. A couple of months ago we have talked on the monthly call about RPZ and the d= ecision that was made by the people was to not look at this now, just because= there were other more pressing things. We could also not find answers to the= questions that led us into RPZ: * Are there any good lists out there that would allow us to replace the URL F= ilter? That thing is basically on its last leg because of the lack of lists. = We need to be able to block ads and pr0n and that very reliably. * We need to look into privacy when RPZs are being realised over DNS - I don= =E2=80=99t even understand why we are suddenly starting to pull text files ov= er HTTP again. The IPS seems to have most of these lists already. Regarding your code, there are some issues with the coding style, but I heavi= ly appreciate the pioneering to bring this feature to life. What it would nee= d to be finally merged would be a web UI though. So I think we have a long wa= y ahead of us. -Michael > I saw your comments in the Dev Mailing List of "generally being in favor of= trying this path" (bad paraphrasing on my part) >=20 > I saw your comments in bugzilla at https://bugzilla.ipfire.org/show_bug.cgi= ?id=3D13254#c171 >> I am not interested in anything regarding the RPZs right now. They have = not been properly put on the agenda and looking at how much time we have on o= ur hands, this won't make it on the agenda for years. >>=20 >> I don't want to build blockers, but this ticket is about a different pro= blem which I want to solve first. >=20 >=20 > How do you want to go forward? >=20 >=20 > Jon >=20 >=20 >> On Aug 12, 2024, at 2:11=E2=80=AFPM, jon wrote: >>=20 >> More questions! >>=20 >> Currently RPZ config files are at `/etc/unbound/local.d` but this director= y seems like it is for user (admin) customizations. =20 >>=20 >> ``` >> [root(a)ipfire ~] # ls -al /etc/unbound/local.d >> total 68 >> drwxr-xr-x 2 nobody nobody 4096 Aug 12 13:41 . >> drwxr-xr-x 4 root root 4096 Aug 12 00:52 .. >> -rw-r--r-- 1 nobody nobody 436 Jul 12 15:45 00-rpz.conf >> -rw-r--r-- 1 nobody nobody 285 Mar 1 22:12 AmazonTrkrHZ.rpz.conf >> -rw-r--r-- 1 nobody nobody 281 Mar 1 22:02 AppleTrkrHZ.rpz.conf >> -rw-r--r-- 1 nobody nobody 269 Mar 1 21:40 DOHblockHZ.rpz.conf >> ... >> -rw-r--r-- 1 nobody nobody 299 Aug 1 19:42 WinTrkrHZ.rpz.conf >> [root(a)ipfire ~] #=20 >> ``` >>=20 >>=20 >> Each file is a config file per category (or one per RPZ file). This makes= it easy to add or remove a category (or RPZ file). >>=20 >> Should I create a new unbound directory for RPZ config files? Maybe `/etc= /unbound/rpz.d`? Or `/etc/unbound/rpz`? >>=20 >>=20 >> Jon >>=20 >>=20 >>> On Aug 1, 2024, at 1:45=E2=80=AFPM, Jon Murphy = wrote: >>>=20 >>> changed all paths from `/var/ipfire/rpz/` to `/var/ipfire/dns/rpz/` >>> (thank you to Adolf!) >>>=20 >>> rpz-config: >>> - bug: corrected "Type" test from block to allow >>> - removed verbose parameter from various commands >>>=20 >>> rpz-metrics: >>> - bug: corrected grep for rpz name count >>> - bug: fixed divide by zero error (thank you Peppe!) >>>=20 >>> install/uninstall: >>> - bug: corrected scripts (thank you Bernhard!) >>>=20 >>> Signed-off-by: Jon Murphy >>> --- >>> config/backup/includes/rpz | 4 ++-- >>> config/rootfiles/packages/rpz | 6 +++--- >>> config/rpz/rpz-config | 14 +++++++------- >>> config/rpz/rpz-metrics | 9 +++++---- >>> lfs/rpz | 6 +++--- >>> src/paks/rpz/install.sh | 27 +++++++++++++++++++++++++++ >>> src/paks/rpz/uninstall.sh | 31 +++++++++++++++++++++++++++++++ >>> src/paks/rpz/update.sh | 25 +++++++++++++++++++++++++ >>> 8 files changed, 103 insertions(+), 19 deletions(-) >>> create mode 100644 src/paks/rpz/install.sh >>> create mode 100644 src/paks/rpz/uninstall.sh >>> create mode 100644 src/paks/rpz/update.sh >>>=20 >>> diff --git a/config/backup/includes/rpz b/config/backup/includes/rpz >>> index 4d59bb40c..8c7410ebd 100644 >>> --- a/config/backup/includes/rpz >>> +++ b/config/backup/includes/rpz >>> @@ -1,5 +1,5 @@ >>> -/var/ipfire/rpz/allowlist >>> -/var/ipfire/rpz/blocklist >>> +/var/ipfire/dns/rpz/allowlist >>> +/var/ipfire/dns/rpz/blocklist >>> /etc/unbound/zonefiles/allow.rpz >>> /etc/unbound/zonefiles/block.rpz >>> /etc/unbound/local.d/*rpz.conf >>> diff --git a/config/rootfiles/packages/rpz b/config/rootfiles/packages/rpz >>> index 2ffa715dd..183825362 100644 >>> --- a/config/rootfiles/packages/rpz >>> +++ b/config/rootfiles/packages/rpz >>> @@ -6,6 +6,6 @@ usr/sbin/rpz-config >>> usr/sbin/rpz-metrics >>> usr/sbin/rpz-sleep >>> var/ipfire/backup/addons/includes/rpz >>> -var/ipfire/rpz >>> -var/ipfire/rpz/allowlist >>> -var/ipfire/rpz/blocklist >>> +var/ipfire/dns/rpz >>> +var/ipfire/dns/rpz/allowlist >>> +var/ipfire/dns/rpz/blocklist >>> diff --git a/config/rpz/rpz-config b/config/rpz/rpz-config >>> index 98dc0a4ca..a24a5c132 100644 >>> --- a/config/rpz/rpz-config >>> +++ b/config/rpz/rpz-config >>> @@ -19,7 +19,7 @@ >>> # = # >>> #########################################################################= ###### >>>=20 >>> -# v22 - 2024-07-12 >>> +# v23 - 2024-07-30 >>>=20 >>> ############### Functions ############### >>>=20 >>> @@ -54,11 +54,11 @@ check_unbound_conf () { >>> make_rpz_file () { >>> local theType=3D"${1}" # allow or block >>>=20 >>> - theList=3D"/var/ipfire/rpz/${theType}list" # input user list of domains >>> + theList=3D"/var/ipfire/dns/rpz/${theType}list" # input custom list of d= omains >>> theZoneFile=3D"/etc/unbound/zonefiles/${theType}.rpz" # output file for R= PZ >>>=20 >>> theAction=3D'.' >>> - if [[ "${theType}" =3D~ "block" ]] ; then >>> + if [[ "${theType}" =3D~ "allow" ]] ; then >>> theAction=3D'rpz-passthru.' >>> fi >>>=20 >>> @@ -131,8 +131,8 @@ case "${theAction}" in >>> # set-up zone file >>> /usr/bin/touch "${rpzFile}" >>> # unbound requires these settings for rpz files >>> - /bin/chown --verbose nobody:nobody "${rpzFile}" >>> - /bin/chmod --verbose 644 "${rpzFile}" >>> + /bin/chown nobody:nobody "${rpzFile}" >>> + /bin/chmod 644 "${rpzFile}" >>> ;; >>>=20 >>> # trash config file & rpz file >>> @@ -143,8 +143,8 @@ case "${theAction}" in >>> fi >>>=20 >>> msg_log "info: rpz: remove config file & rpz file \"${theName}\"" >>> - /bin/rm --verbose "${rpzConfig}" >>> - /bin/rm --verbose "${rpzFile}" >>> + /bin/rm "${rpzConfig}" >>> + /bin/rm "${rpzFile}" >>>=20 >>> check_unbound_conf >>> ;; >>> diff --git a/config/rpz/rpz-metrics b/config/rpz/rpz-metrics >>> index 0f97c7911..4d932726e 100644 >>> --- a/config/rpz/rpz-metrics >>> +++ b/config/rpz/rpz-metrics >>> @@ -19,7 +19,7 @@ >>> # = # >>> #########################################################################= ###### >>>=20 >>> -# v18 on 2024-07-05 >>> +# v19 on 2024-07-30 >>>=20 >>> ############### Main ############### >>>=20 >>> @@ -33,7 +33,7 @@ messageLogs=3D$( find /var/log/messages* -type f | >>>=20 >>> # get the list of RPZ names & counts from the message log(s) >>> rpzNameCount=3D$( for logf in ${messageLogs} ; do >>> - /usr/bin/zgrep --text --fixed-strings 'info: rpz: applied' "${logf}" | >>> + /usr/bin/zgrep --text --extended-regexp 'info: rpz: applied.* A IN$' "$= {logf}" | >>> /usr/bin/awk '$10 ~ /\[\w*]/ { print $10 }' ; >>> done | /usr/bin/sort | /usr/bin/uniq --count ) >>>=20 >>> @@ -107,8 +107,9 @@ do >>> theLines=3D$( /bin/echo "${output}" | /usr/bin/awk '{ print $1 }' ) >>> totalLines=3D$(( totalLines + theLines )) >>>=20 >>> - #hitsPerLine=3D$( echo "scale=3D0 ; $theHits / $theLines" | bc ) >>> - hitsPerLine=3D$(( 100 * theHits / theLines )) >>> + if [[ "${theLines}" -gt 2 ]] ; then >>> + hitsPerLine=3D$(( 100 * theHits / theLines )) >>> + fi >>> fi >>>=20 >>> # get modification date >>> diff --git a/lfs/rpz b/lfs/rpz >>> index 319c10b7f..73f6f2b1b 100644 >>> --- a/lfs/rpz >>> +++ b/lfs/rpz >>> @@ -67,9 +67,9 @@ $(TARGET) : >>> $(DIR_CONF)/rpz/{rpz-config,rpz-metrics,rpz-sleep} -t /usr/sbin >>>=20 >>> # Install settings folder and two empty files >>> - mkdir -pv /var/ipfire/rpz >>> - touch /var/ipfire/rpz/allowlist >>> - touch /var/ipfire/rpz/blocklist >>> + mkdir -pv /var/ipfire/dns/rpz >>> + touch /var/ipfire/dns/rpz/allowlist >>> + touch /var/ipfire/dns/rpz/blocklist >>>=20 >>> # Add conf file to /etc directory >>> cp -vf $(DIR_CONF)/rpz/00-rpz.conf /etc/unbound/local.d >>> diff --git a/src/paks/rpz/install.sh b/src/paks/rpz/install.sh >>> new file mode 100644 >>> index 000000000..0a797e158 >>> --- /dev/null >>> +++ b/src/paks/rpz/install.sh >>> @@ -0,0 +1,27 @@ >>> +#!/bin/bash >>> +########################################################################= ####### >>> +# = # >>> +# IPFire.org - A linux based firewall = # >>> +# Copyright (C) 2024 IPFire Team = # >>> +# = # >>> +# This program is free software: you can redistribute it and/or modify = # >>> +# it under the terms of the GNU General Public License as published by = # >>> +# the Free Software Foundation, either version 3 of the License, or = # >>> +# (at your option) any later version. = # >>> +# = # >>> +# This program is distributed in the hope that it will be useful, = # >>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of = # >>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the = # >>> +# GNU General Public License for more details. = # >>> +# = # >>> +# You should have received a copy of the GNU General Public License = # >>> +# along with this program. If not, see .= # >>> +# = # >>> +########################################################################= ####### >>> +# >>> +. /opt/pakfire/lib/functions.sh >>> +extract_files >>> +restore_backup ${NAME} >>> + >>> +# restart unbound to load config file >>> +/etc/init.d/unbound restart >>> diff --git a/src/paks/rpz/uninstall.sh b/src/paks/rpz/uninstall.sh >>> new file mode 100644 >>> index 000000000..4fb20e127 >>> --- /dev/null >>> +++ b/src/paks/rpz/uninstall.sh >>> @@ -0,0 +1,31 @@ >>> +#!/bin/bash >>> +########################################################################= ####### >>> +# = # >>> +# IPFire.org - A linux based firewall = # >>> +# Copyright (C) 2024 IPFire Team = # >>> +# = # >>> +# This program is free software: you can redistribute it and/or modify = # >>> +# it under the terms of the GNU General Public License as published by = # >>> +# the Free Software Foundation, either version 3 of the License, or = # >>> +# (at your option) any later version. = # >>> +# = # >>> +# This program is distributed in the hope that it will be useful, = # >>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of = # >>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the = # >>> +# GNU General Public License for more details. = # >>> +# = # >>> +# You should have received a copy of the GNU General Public License = # >>> +# along with this program. If not, see .= # >>> +# = # >>> +########################################################################= ####### >>> +# >>> +. /opt/pakfire/lib/functions.sh >>> + >>> +# stop unbound to delete RPZ conf file >>> +/etc/init.d/unbound stop >>> + >>> +make_backup ${NAME} >>> +remove_files >>> + >>> +# start unbound to load unbound config file >>> +/etc/init.d/unbound start >>> diff --git a/src/paks/rpz/update.sh b/src/paks/rpz/update.sh >>> new file mode 100644 >>> index 000000000..938a93a40 >>> --- /dev/null >>> +++ b/src/paks/rpz/update.sh >>> @@ -0,0 +1,25 @@ >>> +#!/bin/bash >>> +########################################################################= ####### >>> +# = # >>> +# IPFire.org - A linux based firewall = # >>> +# Copyright (C) 2024 IPFire Team = # >>> +# = # >>> +# This program is free software: you can redistribute it and/or modify = # >>> +# it under the terms of the GNU General Public License as published by = # >>> +# the Free Software Foundation, either version 3 of the License, or = # >>> +# (at your option) any later version. = # >>> +# = # >>> +# This program is distributed in the hope that it will be useful, = # >>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of = # >>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the = # >>> +# GNU General Public License for more details. = # >>> +# = # >>> +# You should have received a copy of the GNU General Public License = # >>> +# along with this program. If not, see .= # >>> +# = # >>> +########################################################################= ####### >>> +# >>> +. /opt/pakfire/lib/functions.sh >>> +extract_backup_includes >>> +./uninstall.sh >>> +./install.sh >>> --=20 >>> 2.30.2 >>>=20 >>=20 >=20 --===============2499773900285721807==--