And another question.

I found an error in the `rpz-config` script file. Simple easy one line (actually one word) fix.

Do I resubmit the entire patch? -or- Do I wait until approved and then submit the simple patch for the one line change.

Jon

On Jul 20, 2024, at 1:42 PM, jon jon.murphy@ipfire.org wrote:

I need help with…

1. The custom allow and block lists are currently located at `/var/ipfire/rpz`. Is this correct?

2. The three bash scripts are currently located at `/usr/sbin`. Is this correct?

3. Should this commit be separated into multiple posts? If yes, how do I decide how to split it?

Jon

...

On Jul 20, 2024, at 12:46 PM, Jon Murphy jon.murphy@ipfire.org wrote:

What is it? Response Policy Zone (RPZ) is a mechanism to define local policies in a standardised way and load those policies from external sources. Bottom line: RPZ allows admins to easily block access to websites via DNS lookup.

RPZ can block websites via categories. Examples include: fake websites, annoying pop-up ads, newly registered domains, DoH bypass sites, bad "host" services, maliscious top level domains (e.g., *.zip, *.mov), piracy, gambling, pornography, and more. RPZ lists come from various RPZ providers and their available catagories.

This RPZ add-on enables the RPZ functionality by adding a couple lines in a configuration file. This add-on simply adds a configuration file and adds three scripts (config, metrics and sleep) to make RPZ easier for the admin to use.

RPZ was release in 2010 and has been part of the IPFire build since ~2015.

Why is it needed? Some IPFire admin's utilize pihole to block unwanted websites via DNS lookup. Moving the pihole base functionality (without pretty graphs) to IPFire removes one device from the admin's local network. And hopefully this reduces the pihole questions from the Community.

A list of RPZ providers can be recommended by IPFire and coded into a set list. Or, if prefered, the local admin can choose their own RPZ providers.

This is a:

• simple replacement for pihole base functionality
• RPZ can be a nice replacement for the URL Filter

IPFire Wiki In process at: https://www.ipfire.org/docs/addons/rpz

more info:

• https://en.wikipedia.org/wiki/Response_policy_zone
• https://unbound.docs.nlnetlabs.nl/en/latest/topics/filtering/rpz.html

I need help with...

1. The custom allow and block lists are currently located at `/var/ipfire/rpz`. Is this correct?
2. The three bash scripts are currently located at `/usr/sbin`. Is this correct?

Signed-off-by: Jon Murphy jon.murphy@ipfire.org

config/backup/includes/rpz | 5 + config/rootfiles/packages/rpz | 11 ++ config/rpz/00-rpz.conf | 18 ++++ config/rpz/rpz-config | 194 ++++++++++++++++++++++++++++++++++ config/rpz/rpz-metrics | 143 +++++++++++++++++++++++++ config/rpz/rpz-sleep | 58 ++++++++++ lfs/rpz | 88 +++++++++++++++ make.sh | 2 + 8 files changed, 519 insertions(+) create mode 100644 config/backup/includes/rpz create mode 100644 config/rootfiles/packages/rpz create mode 100644 config/rpz/00-rpz.conf create mode 100644 config/rpz/rpz-config create mode 100644 config/rpz/rpz-metrics create mode 100644 config/rpz/rpz-sleep create mode 100644 lfs/rpz

diff --git a/config/backup/includes/rpz b/config/backup/includes/rpz new file mode 100644 index 000000000..4d59bb40c --- /dev/null +++ b/config/backup/includes/rpz @@ -0,0 +1,5 @@ +/var/ipfire/rpz/allowlist +/var/ipfire/rpz/blocklist +/etc/unbound/zonefiles/allow.rpz +/etc/unbound/zonefiles/block.rpz +/etc/unbound/local.d/*rpz.conf diff --git a/config/rootfiles/packages/rpz b/config/rootfiles/packages/rpz new file mode 100644 index 000000000..2ffa715dd --- /dev/null +++ b/config/rootfiles/packages/rpz @@ -0,0 +1,11 @@ +etc/unbound/local.d/00-rpz.conf +etc/unbound/zonefiles +etc/unbound/zonefiles/allow.rpz +etc/unbound/zonefiles/block.rpz +usr/sbin/rpz-config +usr/sbin/rpz-metrics +usr/sbin/rpz-sleep +var/ipfire/backup/addons/includes/rpz +var/ipfire/rpz +var/ipfire/rpz/allowlist +var/ipfire/rpz/blocklist diff --git a/config/rpz/00-rpz.conf b/config/rpz/00-rpz.conf new file mode 100644 index 000000000..72c1d12e5 --- /dev/null +++ b/config/rpz/00-rpz.conf @@ -0,0 +1,18 @@ +server:

• module-config: "respip validator iterator"

+rpz:

• name: allow.rpz
• zonefile: /etc/unbound/zonefiles/allow.rpz
• rpz-action-override: passthru
• rpz-log: yes
• rpz-log-name: allow
• rpz-signal-nxdomain-ra: yes

+rpz:

• name: block.rpz
• zonefile: /etc/unbound/zonefiles/block.rpz
• rpz-action-override: nxdomain
• rpz-log: yes
• rpz-log-name: block
• rpz-signal-nxdomain-ra: yes

diff --git a/config/rpz/rpz-config b/config/rpz/rpz-config new file mode 100644 index 000000000..98dc0a4ca --- /dev/null +++ b/config/rpz/rpz-config @@ -0,0 +1,194 @@ +#!/bin/bash +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2024 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +###############################################################################

+# v22 - 2024-07-12

+############### Functions ###############

+msg_log () {

• /usr/bin/logger --tag "${tagName}" "$*"
• if tty --silent ; then
• echo "${tagName}:" "$*"
• fi

+}

+check_name () {

• local testName="${1}"
• # check for a valid name
• regex='^[a-zA-Z0-9_]+$'
• if [[ ! "${testName}" =~ $regex ]] ; then
• msg_log "error: rpz: the NAME is not valid: "${testName}". exit."
• exit 1
• fi

+}

+check_unbound_conf () {

• # check the above config files
• msg_log "info: rpz: check for errors with "unbound-checkconf""
• /usr/sbin/unbound-checkconf
• exit_code=$?
• if [[ "${exit_code}" -ne 0 ]] ; then
• msg_log "error: rpz: unbound-checkconf. exit."
• exit "${exit_code}"
• fi

+}

+make_rpz_file () {

• local theType="${1}" # allow or block
• theList="/var/ipfire/rpz/${theType}list" # input user list of domains
• theZoneFile="/etc/unbound/zonefiles/${theType}.rpz" # output file for RPZ
• theAction='.'
• if [[ "${theType}" =~ "block" ]] ; then
• theAction='rpz-passthru.'
• fi
• # does a list exist?
• if [[ -s "${theList}" ]] ; then
• # drop any extra "blanks" and add "CNAME <RPZ action>." to each line
• actionList=$( /usr/bin/awk '{$1=$1};1' "${theList}" |
• /bin/sed "/^[^;].*[[:alnum:]]/ s|$| CNAME ${theAction}|" )
• msg_log "info: rpz: create zonefile for ${theList}"
• /bin/cat <<-EOF > "${theZoneFile}"
• ; Name: ${theType} list
• ; Last modified: $(date "+%Y-%m-%d at %H.%M.%S %Z")
• ;
• ; domains with actions list
• ;
• ${actionList}
• EOF
• # reload the zone that was just updated
• zoneBase=$( basename "${theZoneFile}" )
• /usr/sbin/unbound-control auth_zone_reload -q "${zoneBase}"
• fi

+}

+############### Main ###############

+tagName="unbound"

+theAction="${1}" # input action +theName="${2}" # input RPZ name +theURL="${3}" # input RPZ URL

+check_name "${theName}" # is this a valid name?

+rpzConfig="/etc/unbound/local.d/${theName}.rpz.conf" # output zone conf file +rpzFile="/etc/unbound/zonefiles/${theName}.rpz" # output for RPZ file

+case "${theAction}" in

• # add new rpz list
• add )
• # does this config already exist? If yes, then exit
• if [[ -f "${rpzConfig}" ]] ; then
• msg_log "info: rpz: ${rpzConfig} already exists. exit"
• exit 1
• fi
• # is this a valid URL?
• regex='^https://%5B-%5B:alnum:%5D%5C+&@#/%?=~_%7C!:,.;%5D*%5B-%5B:alnum:%5D%5C+&...]'
• if ! [[ "${theURL}" =~ $regex ]] ; then
• msg_log "error: rpz: the URL is not valid: "${theURL}". exit."
• exit 1
• fi
• # create the zone config file
• msg_log "info: rpz: add config file "${theName}.rpz.conf""
• cat <<-EOF > "${rpzConfig}"
• rpz:
• name: ${theName}.rpz
• zonefile: /etc/unbound/zonefiles/${theName}.rpz
• url: ${theURL}
• rpz-action-override: nxdomain
• rpz-log: yes
• rpz-log-name: ${theName}
• rpz-signal-nxdomain-ra: yes
• EOF
• # set-up zone file
• /usr/bin/touch "${rpzFile}"
• # unbound requires these settings for rpz files
• /bin/chown --verbose nobody:nobody "${rpzFile}"
• /bin/chmod --verbose 644 "${rpzFile}"
• ;;
• # trash config file & rpz file
• remove )
• if ! [[ -f "${rpzConfig}" ]] ; then
• msg_log "info: rpz: ${rpzConfig} does not exist. exit"
• exit 1
• fi
• msg_log "info: rpz: remove config file & rpz file "${theName}""
• /bin/rm --verbose "${rpzConfig}"
• /bin/rm --verbose "${rpzFile}"
• check_unbound_conf
• ;;
• # make a new allow or block rpz file
• make )
• case "${theName}" in
• allow )
• make_rpz_file allow
• ;;
• block )
• make_rpz_file block
• ;;
• allowblock )
• make_rpz_file allow
• make_rpz_file block
• ;;
• • )
• msg_log "error: rpz: the NAME is not valid: "${theName}". exit."
• exit 1
• ;;
• esac
• check_unbound_conf
• ;;
• *)
• msg_log "error: rpz: missing or incorrect parameter"
• /usr/bin/printf "Usage: rpzConfig.sh <ACTION> <NAME> <URL>\n"
• exit 1
• ;;

+esac

+# reload due to the changes +msg_log "rpz: running "unbound-control reload"" +/usr/sbin/unbound-control reload +exit_code=$? +if [[ "${exit_code}" -ne 0 ]] ; then

• msg_log "error: rpz: unbound-control "${theName}". exit."
• exit "${exit_code}"

+fi

+exit diff --git a/config/rpz/rpz-metrics b/config/rpz/rpz-metrics new file mode 100644 index 000000000..0f97c7911 --- /dev/null +++ b/config/rpz/rpz-metrics @@ -0,0 +1,143 @@ +#!/bin/bash +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2024 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +###############################################################################

+# v18 on 2024-07-05

+############### Main ###############

+weeks="${1:-2}" # default to two message logs +sortBy="${2:-name}" # by name or by hits

+# get the list of message logs for N weeks +messageLogs=$( find /var/log/messages* -type f |

• /usr/bin/sort --version-sort |
• head -"${weeks}" )

+# get the list of RPZ names & counts from the message log(s) +rpzNameCount=$( for logf in ${messageLogs} ; do

• /usr/bin/zgrep --text --fixed-strings 'info: rpz: applied' "${logf}" |
• /usr/bin/awk '$10 ~ /[\w*]/ { print $10 }' ;
• done | /usr/bin/sort | /usr/bin/uniq --count )

+# flip results and remove brackets `[` and `]` +rpzNameCount=$( /bin/echo "${rpzNameCount}" |

• /usr/bin/awk '{ print $2, $1 }' |
• /bin/sed --regexp-extended 's|^[(.*)]|\1|' )

+# grab only names +rpzNames=$( /bin/echo "${rpzNameCount}" | /usr/bin/awk '{ print $1 }' )

+# get list of RPZ files +rpzFileList=$( /bin/find /etc/unbound/zonefiles -type f -iname "*.rpz" )

+# get basename of those files +rpzBaseNames=$( /bin/echo "${rpzFileList}" |

• /bin/sed 's|/etc/unbound/zonefiles/||g ; s|.rpz||g ;' )

+# add to rpzNames +rpzNames="${rpzNames}"$'\n'"${rpzBaseNames}"

+# drop duplicate names +rpzNames=$( echo "${rpzNames}" | /usr/bin/sort --unique )

+# get line count for each RPZ +lineCount=$( /bin/echo "${rpzFileList}" | /usr/bin/xargs wc -l )

+# get comment line count and blank line count for each RPZ +commentCount=$( /bin/echo "${rpzFileList}" |

• /usr/bin/xargs /bin/grep --count -e "^$" -e "^;" )

+# get modified date each RPZ +modDateList=$( /bin/echo "${rpzFileList}" | xargs stat -c '%.10y %n' )

+ucListAuthZones=$( /usr/sbin/unbound-control list_auth_zones )

+# get width of RPZ names +pWidth=$( /bin/echo "${rpzNames}" | /usr/bin/awk '{ print $1" " }' | wc -L ) +pFormat="%-${pWidth}s %-8s %-8s %-8s %10s %12s\n"

+# print title line +printf "${pFormat}" "name" "hits" "active" "lines" "hits/line%" "last update" +printf -- "--------------"

+theResults="" +totalLines=0 +totalHits=0 +while read -r theName +do

• printf -- "-" # pretend progress bar
• # get hit count
• theHits="0"
• if output=$( /bin/grep "^${theName}\s" <<< "${rpzNameCount}" ) ; then
• theHits=$( /bin/echo "${output}" |
• /usr/bin/awk '{ print $2 }' )
• totalHits=$(( totalHits + theHits ))
• fi
• # is this RPZ list active?
• theActive="disabled"
• if /bin/grep --quiet "^${theName}.rpz" <<< "${ucListAuthZones}"
• then
• theActive="enabled"
• fi
• # get line count then subtract comment count and blank line count
• # from total line count
• theLines="n/a"
• hitsPerLine="0"
• if output=$( /bin/grep --fixed-strings "/${theName}.rpz" <<< "${lineCount}" ) ; then
• theLines=$( /bin/echo "${output}" | /usr/bin/awk '{ print $1 }' )
• totalLines=$(( totalLines + theLines ))
• #hitsPerLine=$( echo "scale=0 ; $theHits / $theLines" | bc )
• hitsPerLine=$(( 100 * theHits / theLines ))
• fi
• # get modification date
• theModDate="n/a"
• if output=$( /bin/grep --fixed-strings "/${theName}.rpz" <<< "${modDateList}" ) ; then
• theModDate=$( /bin/echo "${output}" | /usr/bin/awk '{ print $1 }' )
• fi
• # add to results list
• theResults+="${theName} ${theHits} ${theActive} ${theLines} ${hitsPerLine} ${theModDate}"$'\n'

+done <<< "${rpzNames}"

+case "${sortBy}" in

• names|name) sortArg=(-k3,3r -k1,1) ;; # sort by "active" then by "name"
• hits|hit) sortArg=(-k3,3r -k2,2nr -k1,1) ;; # sort by "active" then by "hits" then by "name"
• lines|line) sortArg=(-k3,3r -k4,4nr -k1,1) ;; # sort by "active" then by "lines" then by "name"

+esac

+printf -- "--------------\n" +# remove blank lines, sort, print as columns +/bin/echo "${theResults}" |

• /usr/bin/awk '!/^[[:space:]]*$/' |
• /usr/bin/sort "${sortArg[@]}" |
• /usr/bin/awk --assign=width="${pWidth}" \
• '{ printf "%-*s %-8s %-8s %-8s %10s %12s\n", width, $1, $2, $3, $4, $5, $6 }'

+printf "${pFormat}" "" "=======" "" "========" "" "" +printf "${pFormat}" "Totals -->" "${totalHits}" "" "${totalLines}" "" ""

+exit diff --git a/config/rpz/rpz-sleep b/config/rpz/rpz-sleep new file mode 100644 index 000000000..eeef1174a --- /dev/null +++ b/config/rpz/rpz-sleep @@ -0,0 +1,58 @@ +#!/bin/bash +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2024 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +###############################################################################

+# v04 on 2024-07-05

+############### Functions ###############

+# send message to message log +msg_log () {

• /usr/bin/logger --tag "${tagName}" "$*"
• if /usr/bin/tty --silent ; then
• echo "${tagName}:" "$*"
• fi

+}

+############### Main ###############

+tagName="unbound"

+sleepTime="${1:-5m}" # default to sleep for 5m (5 minutes)

+zoneList=$( /usr/sbin/unbound-control list_auth_zones | /usr/bin/awk '{print $1}' )

+for zone in ${zoneList} ; do

• /usr/bin/printf "disable ${zone}\t"
• /usr/sbin/unbound-control rpz_disable "${zone}"

+done

+msg_log "info: rpz: disabled all zones for ${sleepTime}"

+/bin/sleep "${sleepTime}"

+for zone in ${zoneList} ; do

• /usr/bin/printf "enable ${zone}\t"
• /usr/sbin/unbound-control rpz_enable "${zone}"

+done

+msg_log "info: rpz: enabled all zones"

+exit diff --git a/lfs/rpz b/lfs/rpz new file mode 100644 index 000000000..319c10b7f --- /dev/null +++ b/lfs/rpz @@ -0,0 +1,88 @@ +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2024 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +###############################################################################

+############################################################################### +# Definitions +###############################################################################

+include Config

+SUMMARY = response policy zone - RPZ reputation system for unbound DNS

+VER = 1.0.0

+THISAPP = rpz-$(VER) +DIR_APP = $(DIR_SRC)/$(THISAPP) +TARGET = $(DIR_INFO)/$(THISAPP)

+PROG = rpz +PAK_VER = 1

+DEPS =

+SERVICES =

+############################################################################### +# Top-level Rules +###############################################################################

+install : $(TARGET)

+check :

+download :

+b2 :

+dist:

• @$(PAK)

+############################################################################### +# Installation Details +###############################################################################

+$(TARGET) :

• @$(PREBUILD)
• @rm -rf $(DIR_APP)
• # install RPZ scripts
• install -v -m 755 \
• $(DIR_CONF)/rpz/{rpz-config,rpz-metrics,rpz-sleep} -t /usr/sbin
• # Install settings folder and two empty files
• mkdir -pv /var/ipfire/rpz
• touch /var/ipfire/rpz/allowlist
• touch /var/ipfire/rpz/blocklist
• # Add conf file to /etc directory
• cp -vf $(DIR_CONF)/rpz/00-rpz.conf /etc/unbound/local.d
• # create zonefiles directory for the RPZ files and add two empty RPZ
• # files to avoid a unbound config error
• mkdir -pv /etc/unbound/zonefiles
• chown -v nobody:nobody /etc/unbound/zonefiles
• touch /etc/unbound/zonefiles/allow.rpz
• touch /etc/unbound/zonefiles/block.rpz
• # Install backup definition
• cp -vf $(DIR_CONF)/backup/includes/rpz /var/ipfire/backup/addons/includes/rpz
• @rm -rf $(DIR_APP)
• @$(POSTBUILD)

diff --git a/make.sh b/make.sh index 9bbbeb0f1..886d3760a 100755 --- a/make.sh +++ b/make.sh @@ -1721,6 +1721,8 @@ buildipfire() { lfsmake2 btrfs-progs lfsmake2 inotify-tools lfsmake2 grub-btrfs

• lfsmake2 rpz

# Kernelbuild ... current we have no platform that need

# multi kernel builds so KCFG is empty

2.30.2

See comments below...

On Jul 28, 2024, at 6:22 AM, Michael Tremer michael.tremer@ipfire.org wrote:

Hello Jon,

I don’t think it is a good idea to send over the same patch set multiple times.

I think first of all, we need to start at the beginning which is to figure out what this actually is, and why we would want this in IPFire.

I have seen that there have been conversations on community.ipfire.org http://community.ipfire.org/ many months ago, but I did not take many things away from it. It was from my point of view mostly a prove of concept discussion. Correct me if I have missed anything here.

We have talked about RPZ many times on the monthly call since the URL filter feature is falling more and more out of fashion. I think there is also many posts about this on the forum.

Yes, I have seen the posts about the url filter feature falling out of fashion. I did not see notes about RPZ from the monthly calls. I will go back and search telco call notes.

Apart from generally being in favour of trying this path, we decided to postpone any further works into this because we currently have a huge backlog of things to deal with that are moving at snail speed. This is rooted in so many people being simply tied in with other things.

As you can see, the patch set that you posted did not trigger a response - a common thing on this list recently and not because of this particular patch set.

Therefore I would like to avoid opening another can of worms before we have delivered the other things that have already lots of time spent on them and code written. Otherwise we are only getting a longer pipeline and this all is going to be more frustrating for everyone involved. And this is not about the feature per se; this is about getting things done.

This is why I wrote the RPZ patch instead of asking for the Developers to write it. I realize there is a very long pipeline.

So what I suggest doing is the following:

You have a Git repository, push the code there and commit any fixes to it over time so that nothing is getting lost.

This was done a few days ago. But I am not sure if I did it correctly. If not, please let me know.

Explain to me from the beginning, please, how it came to this patch set. What is the motivation?

In the beginning (Fall of 2023) the main motivation was blocking DoH. I had noticed apps using their own method of DNS lookup. Some were friendly DoH lookups (Apple iCloud?) and some not (mozilla.cloudflare-dns.com but not from Firefox). Examples I saw are: mozilla.cloudflare-dns.com http://mozilla.cloudflare-dns.com/ www.switch.ch http://www.switch.ch/ use-application-dns.net http://use-application-dns.net/ (a canary for Firefox?) And a few others...

This worked better than expected and was easier to configure than expected. All of the main code was part of unbound and I just need to activate it with a config file. Updates to the RPZ files are already part of the unbound RPZ code.

As I experimented with RPZ I quickly realized it could easily be used to block other websites. I have been using pihole for a number of years to limit the number of ads and trackers. And RPZ helped with those areas also.

What problems does it solve? What problems remain? Basically: Why would anyone want this in IPFire?

It solved the issue of blocking unwanted DoH from apps. And it solved the issue of unwanted ads and tracking. For me those are huge benefits and the biggest reason for using RPZ.

To test other possibilities, I increased the "type" of lists to include other categories other than ads & trackers. Like blocking bad TLDs, gambling, piracy, bad host sites, etc. This is a nice bonus of using RPZ for me.

Where do you see application for this?

As a replacement for the URL Filter. And as a base functional replacement for pihole (but no pretty graphics).

Why is this realised as an add-on and not part of the core system?

In my opinion it should be part of the core. But I wanted to "test the waters" first to determine interest. If only four people use it, then it isn’t worth adding to core and maybe not as an add-on.

Either way I will support it because it helps me!

Here are my metrics for the past eight days (searching `messages` and `messages.1.gz`):

[root@ipfire ~] # rpz-metrics name hits ---------------------------- allow 5560 This is a custom list AmazonTrkrHZ 5726 AppleTrkrHZ 2956 block 128 This is a custom list DOHblockHZ 25077 dohJPG 3448 HosterHZ 19 MxProPlusHZ 17287 NotSafeSearchHZ 0 ProxyBypassHZ 11 tldHZ 15 WinTrkrHZ 507 ======= Totals --> 60734

For the DoH blocking, the above "hits" including blocking Apple DoH. I’ve been testing this for the past few weeks.

I am particularly interested in what lists are publicly available. You know from URL Filter, that lists have shut down or just been abandoned. There is now very little point running it because the lists are just not as good as they used to be. Not horrible, yet, but over time this will not be getting any better. I am not even aware of any viable paid lists. Is this the same situation for RPZ? A quick Google search did not give me anything that would be free to use over DNS. The situation is somewhat similar with the IPS and IP blocklist feature.

These are free/open lists:

Hagezi - DNS Blocklists • at https://github.com/hagezi/dns-blocklists?tab=readme-ov-file#zap-dns-blocklis... • This list was recommended within the IPFire Community and this is the list I strongly recommend. Gerd (Hagezi) is very responsive to support requests. The Hagezi RPZ list is well supported and the RPZ lists are updated twice per day (if changes are needed). • Example: I came across a domain name that was blocked `f005.backblazeb2.com` and it was fixed (removed from the block list) within a few days.

ThreatFox - DNS Response Policy Zone (RPZ) • at https://threatfox.abuse.ch/export/#rpz

URLHaus - DNS Response Policy Zone (RPZ) • at https://urlhaus.abuse.ch/api/#rpz

jpgpi250 - DNS block list for DoH • at https://github.com/jpgpi250/piholemanual/blob/master/DOH.rpz • this is the original DoH block list that I started with. Peter (jpgpi250) has a nice PDF document explaining RPZ and was very helpful getting me started with RPZ. See https://jpgpi250.github.io/piholemanual/doc/Unbound%20response%20policy%20zo...

I have come across other RPZ lists but these are the best supported lists.

I did not look into paid lists.

I would be interested in hearing your thoughts on this.

Does the above help?

Best, -Michael

...

On 27 Jul 2024, at 21:31, jon jon.murphy@ipfire.org wrote:

And another question.

I found an error in the `rpz-config` script file. Simple easy one line (actually one word) fix.

Do I resubmit the entire patch? -or- Do I wait until approved and then submit the simple patch for the one line change.

Jon

...

On Jul 20, 2024, at 1:42 PM, jon jon.murphy@ipfire.org wrote:

I need help with…

1. The custom allow and block lists are currently located at `/var/ipfire/rpz`. Is this correct?

2. The three bash scripts are currently located at `/usr/sbin`. Is this correct?

3. Should this commit be separated into multiple posts? If yes, how do I decide how to split it?

Jon

...

On Jul 20, 2024, at 12:46 PM, Jon Murphy jon.murphy@ipfire.org wrote:

What is it? Response Policy Zone (RPZ) is a mechanism to define local policies in a standardised way and load those policies from external sources. Bottom line: RPZ allows admins to easily block access to websites via DNS lookup.

RPZ can block websites via categories. Examples include: fake websites, annoying pop-up ads, newly registered domains, DoH bypass sites, bad "host" services, maliscious top level domains (e.g., *.zip, *.mov), piracy, gambling, pornography, and more. RPZ lists come from various RPZ providers and their available catagories.

This RPZ add-on enables the RPZ functionality by adding a couple lines in a configuration file. This add-on simply adds a configuration file and adds three scripts (config, metrics and sleep) to make RPZ easier for the admin to use.

RPZ was release in 2010 and has been part of the IPFire build since ~2015.

Why is it needed? Some IPFire admin's utilize pihole to block unwanted websites via DNS lookup. Moving the pihole base functionality (without pretty graphs) to IPFire removes one device from the admin's local network. And hopefully this reduces the pihole questions from the Community.

A list of RPZ providers can be recommended by IPFire and coded into a set list. Or, if prefered, the local admin can choose their own RPZ providers.

This is a:

• simple replacement for pihole base functionality
• RPZ can be a nice replacement for the URL Filter

IPFire Wiki In process at: https://www.ipfire.org/docs/addons/rpz

more info:

• https://en.wikipedia.org/wiki/Response_policy_zone
• https://unbound.docs.nlnetlabs.nl/en/latest/topics/filtering/rpz.html

I need help with...

1. The custom allow and block lists are currently located at `/var/ipfire/rpz`. Is this correct?
2. The three bash scripts are currently located at `/usr/sbin`. Is this correct?

Signed-off-by: Jon Murphy jon.murphy@ipfire.org

config/backup/includes/rpz | 5 + config/rootfiles/packages/rpz | 11 ++ config/rpz/00-rpz.conf | 18 ++++ config/rpz/rpz-config | 194 ++++++++++++++++++++++++++++++++++ config/rpz/rpz-metrics | 143 +++++++++++++++++++++++++ config/rpz/rpz-sleep | 58 ++++++++++ lfs/rpz | 88 +++++++++++++++ make.sh | 2 + 8 files changed, 519 insertions(+) create mode 100644 config/backup/includes/rpz create mode 100644 config/rootfiles/packages/rpz create mode 100644 config/rpz/00-rpz.conf create mode 100644 config/rpz/rpz-config create mode 100644 config/rpz/rpz-metrics create mode 100644 config/rpz/rpz-sleep create mode 100644 lfs/rpz

diff --git a/config/backup/includes/rpz b/config/backup/includes/rpz new file mode 100644 index 000000000..4d59bb40c --- /dev/null +++ b/config/backup/includes/rpz @@ -0,0 +1,5 @@ +/var/ipfire/rpz/allowlist +/var/ipfire/rpz/blocklist +/etc/unbound/zonefiles/allow.rpz +/etc/unbound/zonefiles/block.rpz +/etc/unbound/local.d/*rpz.conf diff --git a/config/rootfiles/packages/rpz b/config/rootfiles/packages/rpz new file mode 100644 index 000000000..2ffa715dd --- /dev/null +++ b/config/rootfiles/packages/rpz @@ -0,0 +1,11 @@ +etc/unbound/local.d/00-rpz.conf +etc/unbound/zonefiles +etc/unbound/zonefiles/allow.rpz +etc/unbound/zonefiles/block.rpz +usr/sbin/rpz-config +usr/sbin/rpz-metrics +usr/sbin/rpz-sleep +var/ipfire/backup/addons/includes/rpz +var/ipfire/rpz +var/ipfire/rpz/allowlist +var/ipfire/rpz/blocklist diff --git a/config/rpz/00-rpz.conf b/config/rpz/00-rpz.conf new file mode 100644 index 000000000..72c1d12e5 --- /dev/null +++ b/config/rpz/00-rpz.conf @@ -0,0 +1,18 @@ +server:

• module-config: "respip validator iterator"

+rpz:

• name: allow.rpz
• zonefile: /etc/unbound/zonefiles/allow.rpz
• rpz-action-override: passthru
• rpz-log: yes
• rpz-log-name: allow
• rpz-signal-nxdomain-ra: yes

+rpz:

• name: block.rpz
• zonefile: /etc/unbound/zonefiles/block.rpz
• rpz-action-override: nxdomain
• rpz-log: yes
• rpz-log-name: block
• rpz-signal-nxdomain-ra: yes

diff --git a/config/rpz/rpz-config b/config/rpz/rpz-config new file mode 100644 index 000000000..98dc0a4ca --- /dev/null +++ b/config/rpz/rpz-config @@ -0,0 +1,194 @@ +#!/bin/bash +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2024 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +###############################################################################

+# v22 - 2024-07-12

+############### Functions ###############

+msg_log () {

• /usr/bin/logger --tag "${tagName}" "$*"
• if tty --silent ; then
• echo "${tagName}:" "$*"
• fi

+}

+check_name () {

• local testName="${1}"
• # check for a valid name
• regex='^[a-zA-Z0-9_]+$'
• if [[ ! "${testName}" =~ $regex ]] ; then
• msg_log "error: rpz: the NAME is not valid: "${testName}". exit."
• exit 1
• fi

+}

+check_unbound_conf () {

• # check the above config files
• msg_log "info: rpz: check for errors with "unbound-checkconf""
• /usr/sbin/unbound-checkconf
• exit_code=$?
• if [[ "${exit_code}" -ne 0 ]] ; then
• msg_log "error: rpz: unbound-checkconf. exit."
• exit "${exit_code}"
• fi

+}

+make_rpz_file () {

• local theType="${1}" # allow or block
• theList="/var/ipfire/rpz/${theType}list" # input user list of domains
• theZoneFile="/etc/unbound/zonefiles/${theType}.rpz" # output file for RPZ
• theAction='.'
• if [[ "${theType}" =~ "block" ]] ; then
• theAction='rpz-passthru.'
• fi
• # does a list exist?
• if [[ -s "${theList}" ]] ; then
• # drop any extra "blanks" and add "CNAME <RPZ action>." to each line
• actionList=$( /usr/bin/awk '{$1=$1};1' "${theList}" |
• /bin/sed "/^[^;].*[[:alnum:]]/ s|$| CNAME ${theAction}|" )
• msg_log "info: rpz: create zonefile for ${theList}"
• /bin/cat <<-EOF > "${theZoneFile}"
• ; Name: ${theType} list
• ; Last modified: $(date "+%Y-%m-%d at %H.%M.%S %Z")
• ;
• ; domains with actions list
• ;
• ${actionList}
• EOF
• # reload the zone that was just updated
• zoneBase=$( basename "${theZoneFile}" )
• /usr/sbin/unbound-control auth_zone_reload -q "${zoneBase}"
• fi

+}

+############### Main ###############

+tagName="unbound"

+theAction="${1}" # input action +theName="${2}" # input RPZ name +theURL="${3}" # input RPZ URL

+check_name "${theName}" # is this a valid name?

+rpzConfig="/etc/unbound/local.d/${theName}.rpz.conf" # output zone conf file +rpzFile="/etc/unbound/zonefiles/${theName}.rpz" # output for RPZ file

+case "${theAction}" in

• # add new rpz list
• add )
• # does this config already exist? If yes, then exit
• if [[ -f "${rpzConfig}" ]] ; then
• msg_log "info: rpz: ${rpzConfig} already exists. exit"
• exit 1
• fi
• # is this a valid URL?
• regex='^https://%5B-%5B:alnum:%5D%5C+&@#/%?=~_%7C!:,.;%5D*%5B-%5B:alnum:%5D%5C+&...]'
• if ! [[ "${theURL}" =~ $regex ]] ; then
• msg_log "error: rpz: the URL is not valid: "${theURL}". exit."
• exit 1
• fi
• # create the zone config file
• msg_log "info: rpz: add config file "${theName}.rpz.conf""
• cat <<-EOF > "${rpzConfig}"
• rpz:
• name: ${theName}.rpz
• zonefile: /etc/unbound/zonefiles/${theName}.rpz
• url: ${theURL}
• rpz-action-override: nxdomain
• rpz-log: yes
• rpz-log-name: ${theName}
• rpz-signal-nxdomain-ra: yes
• EOF
• # set-up zone file
• /usr/bin/touch "${rpzFile}"
• # unbound requires these settings for rpz files
• /bin/chown --verbose nobody:nobody "${rpzFile}"
• /bin/chmod --verbose 644 "${rpzFile}"
• ;;
• # trash config file & rpz file
• remove )
• if ! [[ -f "${rpzConfig}" ]] ; then
• msg_log "info: rpz: ${rpzConfig} does not exist. exit"
• exit 1
• fi
• msg_log "info: rpz: remove config file & rpz file "${theName}""
• /bin/rm --verbose "${rpzConfig}"
• /bin/rm --verbose "${rpzFile}"
• check_unbound_conf
• ;;
• # make a new allow or block rpz file
• make )
• case "${theName}" in
• allow )
• make_rpz_file allow
• ;;
• block )
• make_rpz_file block
• ;;
• allowblock )
• make_rpz_file allow
• make_rpz_file block
• ;;
• • )
• msg_log "error: rpz: the NAME is not valid: "${theName}". exit."
• exit 1
• ;;
• esac
• check_unbound_conf
• ;;
• *)
• msg_log "error: rpz: missing or incorrect parameter"
• /usr/bin/printf "Usage: rpzConfig.sh <ACTION> <NAME> <URL>\n"
• exit 1
• ;;

+esac

+# reload due to the changes +msg_log "rpz: running "unbound-control reload"" +/usr/sbin/unbound-control reload +exit_code=$? +if [[ "${exit_code}" -ne 0 ]] ; then

• msg_log "error: rpz: unbound-control "${theName}". exit."
• exit "${exit_code}"

+fi

+exit diff --git a/config/rpz/rpz-metrics b/config/rpz/rpz-metrics new file mode 100644 index 000000000..0f97c7911 --- /dev/null +++ b/config/rpz/rpz-metrics @@ -0,0 +1,143 @@ +#!/bin/bash +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2024 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +###############################################################################

+# v18 on 2024-07-05

+############### Main ###############

+weeks="${1:-2}" # default to two message logs +sortBy="${2:-name}" # by name or by hits

+# get the list of message logs for N weeks +messageLogs=$( find /var/log/messages* -type f |

• /usr/bin/sort --version-sort |
• head -"${weeks}" )

+# get the list of RPZ names & counts from the message log(s) +rpzNameCount=$( for logf in ${messageLogs} ; do

• /usr/bin/zgrep --text --fixed-strings 'info: rpz: applied' "${logf}" |
• /usr/bin/awk '$10 ~ /[\w*]/ { print $10 }' ;
• done | /usr/bin/sort | /usr/bin/uniq --count )

+# flip results and remove brackets `[` and `]` +rpzNameCount=$( /bin/echo "${rpzNameCount}" |

• /usr/bin/awk '{ print $2, $1 }' |
• /bin/sed --regexp-extended 's|^[(.*)]|\1|' )

+# grab only names +rpzNames=$( /bin/echo "${rpzNameCount}" | /usr/bin/awk '{ print $1 }' )

+# get list of RPZ files +rpzFileList=$( /bin/find /etc/unbound/zonefiles -type f -iname "*.rpz" )

+# get basename of those files +rpzBaseNames=$( /bin/echo "${rpzFileList}" |

• /bin/sed 's|/etc/unbound/zonefiles/||g ; s|.rpz||g ;' )

+# add to rpzNames +rpzNames="${rpzNames}"$'\n'"${rpzBaseNames}"

+# drop duplicate names +rpzNames=$( echo "${rpzNames}" | /usr/bin/sort --unique )

+# get line count for each RPZ +lineCount=$( /bin/echo "${rpzFileList}" | /usr/bin/xargs wc -l )

+# get comment line count and blank line count for each RPZ +commentCount=$( /bin/echo "${rpzFileList}" |

• /usr/bin/xargs /bin/grep --count -e "^$" -e "^;" )

+# get modified date each RPZ +modDateList=$( /bin/echo "${rpzFileList}" | xargs stat -c '%.10y %n' )

+ucListAuthZones=$( /usr/sbin/unbound-control list_auth_zones )

+# get width of RPZ names +pWidth=$( /bin/echo "${rpzNames}" | /usr/bin/awk '{ print $1" " }' | wc -L ) +pFormat="%-${pWidth}s %-8s %-8s %-8s %10s %12s\n"

+# print title line +printf "${pFormat}" "name" "hits" "active" "lines" "hits/line%" "last update" +printf -- "--------------"

+theResults="" +totalLines=0 +totalHits=0 +while read -r theName +do

• printf -- "-" # pretend progress bar
• # get hit count
• theHits="0"
• if output=$( /bin/grep "^${theName}\s" <<< "${rpzNameCount}" ) ; then
• theHits=$( /bin/echo "${output}" |
• /usr/bin/awk '{ print $2 }' )
• totalHits=$(( totalHits + theHits ))
• fi
• # is this RPZ list active?
• theActive="disabled"
• if /bin/grep --quiet "^${theName}.rpz" <<< "${ucListAuthZones}"
• then
• theActive="enabled"
• fi
• # get line count then subtract comment count and blank line count
• # from total line count
• theLines="n/a"
• hitsPerLine="0"
• if output=$( /bin/grep --fixed-strings "/${theName}.rpz" <<< "${lineCount}" ) ; then
• theLines=$( /bin/echo "${output}" | /usr/bin/awk '{ print $1 }' )
• totalLines=$(( totalLines + theLines ))
• #hitsPerLine=$( echo "scale=0 ; $theHits / $theLines" | bc )
• hitsPerLine=$(( 100 * theHits / theLines ))
• fi
• # get modification date
• theModDate="n/a"
• if output=$( /bin/grep --fixed-strings "/${theName}.rpz" <<< "${modDateList}" ) ; then
• theModDate=$( /bin/echo "${output}" | /usr/bin/awk '{ print $1 }' )
• fi
• # add to results list
• theResults+="${theName} ${theHits} ${theActive} ${theLines} ${hitsPerLine} ${theModDate}"$'\n'

+done <<< "${rpzNames}"

+case "${sortBy}" in

• names|name) sortArg=(-k3,3r -k1,1) ;; # sort by "active" then by "name"
• hits|hit) sortArg=(-k3,3r -k2,2nr -k1,1) ;; # sort by "active" then by "hits" then by "name"
• lines|line) sortArg=(-k3,3r -k4,4nr -k1,1) ;; # sort by "active" then by "lines" then by "name"

+esac

+printf -- "--------------\n" +# remove blank lines, sort, print as columns +/bin/echo "${theResults}" |

• /usr/bin/awk '!/^[[:space:]]*$/' |
• /usr/bin/sort "${sortArg[@]}" |
• /usr/bin/awk --assign=width="${pWidth}" \
• '{ printf "%-*s %-8s %-8s %-8s %10s %12s\n", width, $1, $2, $3, $4, $5, $6 }'

+printf "${pFormat}" "" "=======" "" "========" "" "" +printf "${pFormat}" "Totals -->" "${totalHits}" "" "${totalLines}" "" ""

+exit diff --git a/config/rpz/rpz-sleep b/config/rpz/rpz-sleep new file mode 100644 index 000000000..eeef1174a --- /dev/null +++ b/config/rpz/rpz-sleep @@ -0,0 +1,58 @@ +#!/bin/bash +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2024 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +###############################################################################

+# v04 on 2024-07-05

+############### Functions ###############

+# send message to message log +msg_log () {

• /usr/bin/logger --tag "${tagName}" "$*"
• if /usr/bin/tty --silent ; then
• echo "${tagName}:" "$*"
• fi

+}

+############### Main ###############

+tagName="unbound"

+sleepTime="${1:-5m}" # default to sleep for 5m (5 minutes)

+zoneList=$( /usr/sbin/unbound-control list_auth_zones | /usr/bin/awk '{print $1}' )

+for zone in ${zoneList} ; do

• /usr/bin/printf "disable ${zone}\t"
• /usr/sbin/unbound-control rpz_disable "${zone}"

+done

+msg_log "info: rpz: disabled all zones for ${sleepTime}"

+/bin/sleep "${sleepTime}"

+for zone in ${zoneList} ; do

• /usr/bin/printf "enable ${zone}\t"
• /usr/sbin/unbound-control rpz_enable "${zone}"

+done

+msg_log "info: rpz: enabled all zones"

+exit diff --git a/lfs/rpz b/lfs/rpz new file mode 100644 index 000000000..319c10b7f --- /dev/null +++ b/lfs/rpz @@ -0,0 +1,88 @@ +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2024 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +###############################################################################

+############################################################################### +# Definitions +###############################################################################

+include Config

+SUMMARY = response policy zone - RPZ reputation system for unbound DNS

+VER = 1.0.0

+THISAPP = rpz-$(VER) +DIR_APP = $(DIR_SRC)/$(THISAPP) +TARGET = $(DIR_INFO)/$(THISAPP)

+PROG = rpz +PAK_VER = 1

+DEPS =

+SERVICES =

+############################################################################### +# Top-level Rules +###############################################################################

+install : $(TARGET)

+check :

+download :

+b2 :

+dist:

• @$(PAK)

+############################################################################### +# Installation Details +###############################################################################

+$(TARGET) :

• @$(PREBUILD)
• @rm -rf $(DIR_APP)
• # install RPZ scripts
• install -v -m 755 \
• $(DIR_CONF)/rpz/{rpz-config,rpz-metrics,rpz-sleep} -t /usr/sbin
• # Install settings folder and two empty files
• mkdir -pv /var/ipfire/rpz
• touch /var/ipfire/rpz/allowlist
• touch /var/ipfire/rpz/blocklist
• # Add conf file to /etc directory
• cp -vf $(DIR_CONF)/rpz/00-rpz.conf /etc/unbound/local.d
• # create zonefiles directory for the RPZ files and add two empty RPZ
• # files to avoid a unbound config error
• mkdir -pv /etc/unbound/zonefiles
• chown -v nobody:nobody /etc/unbound/zonefiles
• touch /etc/unbound/zonefiles/allow.rpz
• touch /etc/unbound/zonefiles/block.rpz
• # Install backup definition
• cp -vf $(DIR_CONF)/backup/includes/rpz /var/ipfire/backup/addons/includes/rpz
• @rm -rf $(DIR_APP)
• @$(POSTBUILD)

diff --git a/make.sh b/make.sh index 9bbbeb0f1..886d3760a 100755 --- a/make.sh +++ b/make.sh @@ -1721,6 +1721,8 @@ buildipfire() { lfsmake2 btrfs-progs lfsmake2 inotify-tools lfsmake2 grub-btrfs

• lfsmake2 rpz

# Kernelbuild ... current we have no platform that need

# multi kernel builds so KCFG is empty

2.30.2

Comments below...

On Jul 30, 2024, at 5:05 AM, Adolf Belka adolf.belka@ipfire.org wrote:

Hi Jon, I have given some feedback in your patches below.

On 29/07/2024 04:28, jon wrote:

...

See comments below...

...

On Jul 28, 2024, at 6:22 AM, Michael Tremer michael.tremer@ipfire.org wrote:

Hello Jon,

I don’t think it is a good idea to send over the same patch set multiple times.

I think first of all, we need to start at the beginning which is to figure out what this actually is, and why we would want this in IPFire.

I have seen that there have been conversations on community.ipfire.org http://community.ipfire.org/ many months ago, but I did not take many things away from it. It was from my point of view mostly a prove of concept discussion. Correct me if I have missed anything here.

We have talked about RPZ many times on the monthly call since the URL filter feature is falling more and more out of fashion. I think there is also many posts about this on the forum.

Yes, I have seen the posts about the url filter feature falling out of fashion. I did not see notes about RPZ from the monthly calls. I will go back and search telco call notes.

...

Apart from generally being in favour of trying this path, we decided to postpone any further works into this because we currently have a huge backlog of things to deal with that are moving at snail speed. This is rooted in so many people being simply tied in with other things.

As you can see, the patch set that you posted did not trigger a response - a common thing on this list recently and not because of this particular patch set.

Therefore I would like to avoid opening another can of worms before we have delivered the other things that have already lots of time spent on them and code written. Otherwise we are only getting a longer pipeline and this all is going to be more frustrating for everyone involved. And this is not about the feature per se; this is about getting things done.

This is why I wrote the RPZ patch instead of asking for the Developers to write it. I realize there is a very long pipeline.

...

So what I suggest doing is the following:

You have a Git repository, push the code there and commit any fixes to it over time so that nothing is getting lost.

This was done a few days ago. But I am not sure if I did it correctly. If not, please let me know.

...

Explain to me from the beginning, please, how it came to this patch set. What is the motivation?

In the beginning (Fall of 2023) the main motivation was blocking DoH. I had noticed apps using their own method of DNS lookup. Some were friendly DoH lookups (Apple iCloud?) and some not (mozilla.cloudflare-dns.com but not from Firefox). Examples I saw are: mozilla.cloudflare-dns.com http://mozilla.cloudflare-dns.com/ www.switch.ch http://www.switch.ch/ use-application-dns.net http://use-application-dns.net/ (a canary for Firefox?) And a few others...

This worked better than expected and was easier to configure than expected. All of the main code was part of unbound and I just need to activate it with a config file. Updates to the RPZ files are already part of the unbound RPZ code.

As I experimented with RPZ I quickly realized it could easily be used to block other websites. I have been using pihole for a number of years to limit the number of ads and trackers. And RPZ helped with those areas also.

...

What problems does it solve? What problems remain? Basically: Why would anyone want this in IPFire?

It solved the issue of blocking unwanted DoH from apps. And it solved the issue of unwanted ads and tracking. For me those are huge benefits and the biggest reason for using RPZ.

To test other possibilities, I increased the "type" of lists to include other categories other than ads & trackers. Like blocking bad TLDs, gambling, piracy, bad host sites, etc. This is a nice bonus of using RPZ for me.

...

Where do you see application for this?

As a replacement for the URL Filter. And as a base functional replacement for pihole (but no pretty graphics).

...

Why is this realised as an add-on and not part of the core system?

In my opinion it should be part of the core. But I wanted to "test the waters" first to determine interest. If only four people use it, then it isn’t worth adding to core and maybe not as an add-on.

Either way I will support it because it helps me!

Here are my metrics for the past eight days (searching `messages` and `messages.1.gz`):

[root@ipfire ~] # rpz-metrics name hits

---

allow 5560 This is a custom list AmazonTrkrHZ 5726 AppleTrkrHZ 2956 block 128 This is a custom list DOHblockHZ 25077 dohJPG 3448 HosterHZ 19 MxProPlusHZ 17287 NotSafeSearchHZ 0 ProxyBypassHZ 11 tldHZ 15 WinTrkrHZ 507 ======= Totals --> 60734

For the DoH blocking, the above "hits" including blocking Apple DoH. I’ve been testing this for the past few weeks.

...

I am particularly interested in what lists are publicly available. You know from URL Filter, that lists have shut down or just been abandoned. There is now very little point running it because the lists are just not as good as they used to be. Not horrible, yet, but over time this will not be getting any better. I am not even aware of any viable paid lists. Is this the same situation for RPZ? A quick Google search did not give me anything that would be free to use over DNS. The situation is somewhat similar with the IPS and IP blocklist feature.

These are free/open lists:

Hagezi - DNS Blocklists • at https://github.com/hagezi/dns-blocklists?tab=readme-ov-file#zap-dns-blocklis... • This list was recommended within the IPFire Community and this is the list I strongly recommend. Gerd (Hagezi) is very responsive to support requests. The Hagezi RPZ list is well supported and the RPZ lists are updated twice per day (if changes are needed). • Example: I came across a domain name that was blocked `f005.backblazeb2.com` and it was fixed (removed from the block list) within a few days.

ThreatFox - DNS Response Policy Zone (RPZ) • at https://threatfox.abuse.ch/export/#rpz

URLHaus - DNS Response Policy Zone (RPZ) • at https://urlhaus.abuse.ch/api/#rpz

jpgpi250 - DNS block list for DoH • at https://github.com/jpgpi250/piholemanual/blob/master/DOH.rpz • this is the original DoH block list that I started with. Peter (jpgpi250) has a nice PDF document explaining RPZ and was very helpful getting me started with RPZ. See https://jpgpi250.github.io/piholemanual/doc/Unbound%20response%20policy%20zo...

I have come across other RPZ lists but these are the best supported lists.

I did not look into paid lists.

...

I would be interested in hearing your thoughts on this.

Does the above help?

...

Best, -Michael

...

On 27 Jul 2024, at 21:31, jon jon.murphy@ipfire.org wrote:

And another question.

I found an error in the `rpz-config` script file. Simple easy one line (actually one word) fix.

Do I resubmit the entire patch? -or- Do I wait until approved and then submit the simple patch for the one line change.

Jon

...

On Jul 20, 2024, at 1:42 PM, jon jon.murphy@ipfire.org wrote:

I need help with…

1. The custom allow and block lists are currently located at `/var/ipfire/rpz`. Is this correct?

2. The three bash scripts are currently located at `/usr/sbin`. Is this correct?

3. Should this commit be separated into multiple posts? If yes, how do I decide how to split it?

Jon

...

On Jul 20, 2024, at 12:46 PM, Jon Murphy jon.murphy@ipfire.org wrote:

What is it? Response Policy Zone (RPZ) is a mechanism to define local policies in a standardised way and load those policies from external sources. Bottom line: RPZ allows admins to easily block access to websites via DNS lookup.

RPZ can block websites via categories. Examples include: fake websites, annoying pop-up ads, newly registered domains, DoH bypass sites, bad "host" services, maliscious top level domains (e.g., *.zip, *.mov), piracy, gambling, pornography, and more. RPZ lists come from various RPZ providers and their available catagories.

This RPZ add-on enables the RPZ functionality by adding a couple lines in a configuration file. This add-on simply adds a configuration file and adds three scripts (config, metrics and sleep) to make RPZ easier for the admin to use.

RPZ was release in 2010 and has been part of the IPFire build since ~2015.

Why is it needed? Some IPFire admin's utilize pihole to block unwanted websites via DNS lookup. Moving the pihole base functionality (without pretty graphs) to IPFire removes one device from the admin's local network. And hopefully this reduces the pihole questions from the Community.

A list of RPZ providers can be recommended by IPFire and coded into a set list. Or, if prefered, the local admin can choose their own RPZ providers.

This is a:

• simple replacement for pihole base functionality
• RPZ can be a nice replacement for the URL Filter

IPFire Wiki In process at: https://www.ipfire.org/docs/addons/rpz

more info:

• https://en.wikipedia.org/wiki/Response_policy_zone
• https://unbound.docs.nlnetlabs.nl/en/latest/topics/filtering/rpz.html

I need help with...

1. The custom allow and block lists are currently located at `/var/ipfire/rpz`. Is this correct?

rpz is supposed to be part of the dns so I would either put all the files in /var/ipfire/dns/ or create a directory for rpz under the dns directory.

Yes, I can do this. I’ll go with `/var/ipfire/dns/rpz` if OK.

...

...

...

...

...

2. The three bash scripts are currently located at `/usr/sbin`. Is this correct?

Signed-off-by: Jon Murphy jon.murphy@ipfire.org

config/backup/includes/rpz | 5 + config/rootfiles/packages/rpz | 11 ++ config/rpz/00-rpz.conf | 18 ++++ config/rpz/rpz-config | 194 ++++++++++++++++++++++++++++++++++ config/rpz/rpz-metrics | 143 +++++++++++++++++++++++++ config/rpz/rpz-sleep | 58 ++++++++++ lfs/rpz | 88 +++++++++++++++ make.sh | 2 + 8 files changed, 519 insertions(+) create mode 100644 config/backup/includes/rpz create mode 100644 config/rootfiles/packages/rpz create mode 100644 config/rpz/00-rpz.conf create mode 100644 config/rpz/rpz-config create mode 100644 config/rpz/rpz-metrics create mode 100644 config/rpz/rpz-sleep create mode 100644 lfs/rpz

diff --git a/config/backup/includes/rpz b/config/backup/includes/rpz new file mode 100644 index 000000000..4d59bb40c --- /dev/null +++ b/config/backup/includes/rpz @@ -0,0 +1,5 @@ +/var/ipfire/rpz/allowlist +/var/ipfire/rpz/blocklist +/etc/unbound/zonefiles/allow.rpz +/etc/unbound/zonefiles/block.rpz +/etc/unbound/local.d/*rpz.conf diff --git a/config/rootfiles/packages/rpz b/config/rootfiles/packages/rpz new file mode 100644 index 000000000..2ffa715dd --- /dev/null +++ b/config/rootfiles/packages/rpz @@ -0,0 +1,11 @@ +etc/unbound/local.d/00-rpz.conf +etc/unbound/zonefiles +etc/unbound/zonefiles/allow.rpz +etc/unbound/zonefiles/block.rpz +usr/sbin/rpz-config +usr/sbin/rpz-metrics +usr/sbin/rpz-sleep +var/ipfire/backup/addons/includes/rpz +var/ipfire/rpz +var/ipfire/rpz/allowlist +var/ipfire/rpz/blocklist diff --git a/config/rpz/00-rpz.conf b/config/rpz/00-rpz.conf new file mode 100644 index 000000000..72c1d12e5 --- /dev/null +++ b/config/rpz/00-rpz.conf @@ -0,0 +1,18 @@ +server:

• module-config: "respip validator iterator"

+rpz:

• name: allow.rpz
• zonefile: /etc/unbound/zonefiles/allow.rpz
• rpz-action-override: passthru
• rpz-log: yes
• rpz-log-name: allow
• rpz-signal-nxdomain-ra: yes

+rpz:

• name: block.rpz
• zonefile: /etc/unbound/zonefiles/block.rpz
• rpz-action-override: nxdomain
• rpz-log: yes
• rpz-log-name: block
• rpz-signal-nxdomain-ra: yes

diff --git a/config/rpz/rpz-config b/config/rpz/rpz-config new file mode 100644 index 000000000..98dc0a4ca --- /dev/null +++ b/config/rpz/rpz-config @@ -0,0 +1,194 @@ +#!/bin/bash +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2024 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +###############################################################################

+# v22 - 2024-07-12

+############### Functions ###############

+msg_log () {

• /usr/bin/logger --tag "${tagName}" "$*"
• if tty --silent ; then
• echo "${tagName}:" "$*"
• fi

+}

+check_name () {

• local testName="${1}"
• # check for a valid name
• regex='^[a-zA-Z0-9_]+$'

Is there a specific reason why you have chosen only letters, numbers and underscore and not also the dash. If you allowed that then you could use the validdomainname subroutine in the general-functions.pl file rather than creating another valid name function. Does unbound not accept dash's in the rpz config filename?

A dash did not work with RPZ.

The RPZ "name"s appear in multiple places. I needed something that would work with RPZ and work with a filename and work within the message log.

...

...

...

...

...

• if [[ ! "${testName}" =~ $regex ]] ; then
• msg_log "error: rpz: the NAME is not valid: "${testName}". exit."
• exit 1
• fi

+}

+check_unbound_conf () {

• # check the above config files
• msg_log "info: rpz: check for errors with "unbound-checkconf""
• /usr/sbin/unbound-checkconf

I think checkconf should be added into the unboundctrl.c file https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=src/misc-progs/unboundctrl... so that the command can be ensured to be running in safe mode.

Yes, this can be changed. I do not have a good understanding of the `*ctrl.c` programs so if this is needed, I would need assistance in adding it to c.

...

...

...

...

...

• exit_code=$?
• if [[ "${exit_code}" -ne 0 ]] ; then
• msg_log "error: rpz: unbound-checkconf. exit."
• exit "${exit_code}"
• fi

+}

+make_rpz_file () {

• local theType="${1}" # allow or block
• theList="/var/ipfire/rpz/${theType}list" # input user list of domains
• theZoneFile="/etc/unbound/zonefiles/${theType}.rpz" # output file for RPZ
• theAction='.'
• if [[ "${theType}" =~ "block" ]] ; then
• theAction='rpz-passthru.'
• fi
• # does a list exist?
• if [[ -s "${theList}" ]] ; then
• # drop any extra "blanks" and add "CNAME <RPZ action>." to each line
• actionList=$( /usr/bin/awk '{$1=$1};1' "${theList}" |
• /bin/sed "/^[^;].*[[:alnum:]]/ s|$| CNAME ${theAction}|" )
• msg_log "info: rpz: create zonefile for ${theList}"
• /bin/cat <<-EOF > "${theZoneFile}"
• ; Name: ${theType} list
• ; Last modified: $(date "+%Y-%m-%d at %H.%M.%S %Z")
• ;
• ; domains with actions list
• ;
• ${actionList}
• EOF
• # reload the zone that was just updated
• zoneBase=$( basename "${theZoneFile}" )
• /usr/sbin/unbound-control auth_zone_reload -q "${zoneBase}"

This should also be run via the unboundctrl.c file especially as the unbound-control command is accessing remote systems.

...

...

...

...

...

• fi

+}

+############### Main ###############

+tagName="unbound"

+theAction="${1}" # input action +theName="${2}" # input RPZ name +theURL="${3}" # input RPZ URL

+check_name "${theName}" # is this a valid name?

+rpzConfig="/etc/unbound/local.d/${theName}.rpz.conf" # output zone conf file +rpzFile="/etc/unbound/zonefiles/${theName}.rpz" # output for RPZ file

+case "${theAction}" in

• # add new rpz list
• add )
• # does this config already exist? If yes, then exit
• if [[ -f "${rpzConfig}" ]] ; then
• msg_log "info: rpz: ${rpzConfig} already exists. exit"
• exit 1
• fi
• # is this a valid URL?
• regex='^https://%5B-%5B:alnum:%5D%5C+&@#/%?=~_%7C!:,.;%5D*%5B-%5B:alnum:%5D%5C+&...]'
• if ! [[ "${theURL}" =~ $regex ]] ; then
• msg_log "error: rpz: the URL is not valid: "${theURL}". exit."
• exit 1
• fi
• # create the zone config file
• msg_log "info: rpz: add config file "${theName}.rpz.conf""
• cat <<-EOF > "${rpzConfig}"
• rpz:
• name: ${theName}.rpz
• zonefile: /etc/unbound/zonefiles/${theName}.rpz
• url: ${theURL}
• rpz-action-override: nxdomain
• rpz-log: yes
• rpz-log-name: ${theName}
• rpz-signal-nxdomain-ra: yes
• EOF
• # set-up zone file
• /usr/bin/touch "${rpzFile}"
• # unbound requires these settings for rpz files
• /bin/chown --verbose nobody:nobody "${rpzFile}"
• /bin/chmod --verbose 644 "${rpzFile}"

These should only be using --verbose while doing development. In production they should not be used as the development should have ensured that everything is working as required.

Yes, verbose can be removed.

...

...

...

...

...

• ;;
• # trash config file & rpz file
• remove )
• if ! [[ -f "${rpzConfig}" ]] ; then
• msg_log "info: rpz: ${rpzConfig} does not exist. exit"
• exit 1
• fi
• msg_log "info: rpz: remove config file & rpz file "${theName}""
• /bin/rm --verbose "${rpzConfig}"
• /bin/rm --verbose "${rpzFile}"

Same thing with the --verbose.

...

...

...

...

...

• check_unbound_conf
• ;;
• # make a new allow or block rpz file
• make )
• case "${theName}" in
• allow )
• make_rpz_file allow
• ;;
• block )
• make_rpz_file block
• ;;
• allowblock )
• make_rpz_file allow
• make_rpz_file block
• ;;
• • )
• msg_log "error: rpz: the NAME is not valid: "${theName}". exit."
• exit 1
• ;;
• esac
• check_unbound_conf
• ;;
• *)
• msg_log "error: rpz: missing or incorrect parameter"
• /usr/bin/printf "Usage: rpzConfig.sh <ACTION> <NAME> <URL>\n"
• exit 1
• ;;

+esac

+# reload due to the changes +msg_log "rpz: running "unbound-control reload"" +/usr/sbin/unbound-control reload

Use unboundctrl

...

...

...

...

...

+exit_code=$? +if [[ "${exit_code}" -ne 0 ]] ; then

• msg_log "error: rpz: unbound-control "${theName}". exit."
• exit "${exit_code}"

+fi

+exit

This is a hell of a long bash script.

The above script is only 200 lines.

The below script is only 143 lines

Did you consider doing it in Perl or python instead of bash.

No. The script did not require the speed or complexity of Perl or Python And I don’t have the skills needed to write in those languages.

Maybe it would be better in c even as then it would be compiled and could not be modified on the IPFire system if some malware got in.

If some odd malware got into IPFire I would venture to guess it would not go after just this code...

...

...

...

...

...

diff --git a/config/rpz/rpz-metrics b/config/rpz/rpz-metrics new file mode 100644 index 000000000..0f97c7911 --- /dev/null +++ b/config/rpz/rpz-metrics @@ -0,0 +1,143 @@ +#!/bin/bash +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2024 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +###############################################################################

+# v18 on 2024-07-05

+############### Main ###############

+weeks="${1:-2}" # default to two message logs +sortBy="${2:-name}" # by name or by hits

+# get the list of message logs for N weeks +messageLogs=$( find /var/log/messages* -type f |

• /usr/bin/sort --version-sort |
• head -"${weeks}" )

+# get the list of RPZ names & counts from the message log(s) +rpzNameCount=$( for logf in ${messageLogs} ; do

• /usr/bin/zgrep --text --fixed-strings 'info: rpz: applied' "${logf}" |
• /usr/bin/awk '$10 ~ /[\w*]/ { print $10 }' ;
• done | /usr/bin/sort | /usr/bin/uniq --count )

+# flip results and remove brackets `[` and `]` +rpzNameCount=$( /bin/echo "${rpzNameCount}" |

• /usr/bin/awk '{ print $2, $1 }' |
• /bin/sed --regexp-extended 's|^[(.*)]|\1|' )

+# grab only names +rpzNames=$( /bin/echo "${rpzNameCount}" | /usr/bin/awk '{ print $1 }' )

+# get list of RPZ files +rpzFileList=$( /bin/find /etc/unbound/zonefiles -type f -iname "*.rpz" )

+# get basename of those files +rpzBaseNames=$( /bin/echo "${rpzFileList}" |

• /bin/sed 's|/etc/unbound/zonefiles/||g ; s|.rpz||g ;' )

+# add to rpzNames +rpzNames="${rpzNames}"$'\n'"${rpzBaseNames}"

+# drop duplicate names +rpzNames=$( echo "${rpzNames}" | /usr/bin/sort --unique )

+# get line count for each RPZ +lineCount=$( /bin/echo "${rpzFileList}" | /usr/bin/xargs wc -l )

+# get comment line count and blank line count for each RPZ +commentCount=$( /bin/echo "${rpzFileList}" |

• /usr/bin/xargs /bin/grep --count -e "^$" -e "^;" )

+# get modified date each RPZ +modDateList=$( /bin/echo "${rpzFileList}" | xargs stat -c '%.10y %n' )

+ucListAuthZones=$( /usr/sbin/unbound-control list_auth_zones )

+# get width of RPZ names +pWidth=$( /bin/echo "${rpzNames}" | /usr/bin/awk '{ print $1" " }' | wc -L ) +pFormat="%-${pWidth}s %-8s %-8s %-8s %10s %12s\n"

+# print title line +printf "${pFormat}" "name" "hits" "active" "lines" "hits/line%" "last update" +printf -- "--------------"

+theResults="" +totalLines=0 +totalHits=0 +while read -r theName +do

• printf -- "-" # pretend progress bar
• # get hit count
• theHits="0"
• if output=$( /bin/grep "^${theName}\s" <<< "${rpzNameCount}" ) ; then
• theHits=$( /bin/echo "${output}" |
• /usr/bin/awk '{ print $2 }' )
• totalHits=$(( totalHits + theHits ))
• fi
• # is this RPZ list active?
• theActive="disabled"
• if /bin/grep --quiet "^${theName}.rpz" <<< "${ucListAuthZones}"
• then
• theActive="enabled"
• fi
• # get line count then subtract comment count and blank line count
• # from total line count
• theLines="n/a"
• hitsPerLine="0"
• if output=$( /bin/grep --fixed-strings "/${theName}.rpz" <<< "${lineCount}" ) ; then
• theLines=$( /bin/echo "${output}" | /usr/bin/awk '{ print $1 }' )
• totalLines=$(( totalLines + theLines ))
• #hitsPerLine=$( echo "scale=0 ; $theHits / $theLines" | bc )
• hitsPerLine=$(( 100 * theHits / theLines ))
• fi
• # get modification date
• theModDate="n/a"
• if output=$( /bin/grep --fixed-strings "/${theName}.rpz" <<< "${modDateList}" ) ; then
• theModDate=$( /bin/echo "${output}" | /usr/bin/awk '{ print $1 }' )
• fi
• # add to results list
• theResults+="${theName} ${theHits} ${theActive} ${theLines} ${hitsPerLine} ${theModDate}"$'\n'

+done <<< "${rpzNames}"

+case "${sortBy}" in

• names|name) sortArg=(-k3,3r -k1,1) ;; # sort by "active" then by "name"
• hits|hit) sortArg=(-k3,3r -k2,2nr -k1,1) ;; # sort by "active" then by "hits" then by "name"
• lines|line) sortArg=(-k3,3r -k4,4nr -k1,1) ;; # sort by "active" then by "lines" then by "name"

+esac

+printf -- "--------------\n" +# remove blank lines, sort, print as columns +/bin/echo "${theResults}" |

• /usr/bin/awk '!/^[[:space:]]*$/' |
• /usr/bin/sort "${sortArg[@]}" |
• /usr/bin/awk --assign=width="${pWidth}" \
• '{ printf "%-*s %-8s %-8s %-8s %10s %12s\n", width, $1, $2, $3, $4, $5, $6 }'

+printf "${pFormat}" "" "=======" "" "========" "" "" +printf "${pFormat}" "Totals -->" "${totalHits}" "" "${totalLines}" "" ""

+exit

If I understand right the rpz-metrics info is being shown on a weekly basis. Wouldn't this make more sense to create a log page for unbound-rpz that shows the results in a similar way to the Firewall Logs, IP Address Blocklist Logs etc. So you can see for a whole month or a specific day and you can also export the data into a text format on a page of the browser. Basically that would just be copying one of the existing cgi scripts and modifying the content elements.

Yes, I can do this.

...

...

...

...

...

diff --git a/config/rpz/rpz-sleep b/config/rpz/rpz-sleep new file mode 100644 index 000000000..eeef1174a --- /dev/null +++ b/config/rpz/rpz-sleep @@ -0,0 +1,58 @@ +#!/bin/bash +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2024 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +###############################################################################

+# v04 on 2024-07-05

+############### Functions ###############

+# send message to message log +msg_log () {

• /usr/bin/logger --tag "${tagName}" "$*"
• if /usr/bin/tty --silent ; then
• echo "${tagName}:" "$*"
• fi

+}

+############### Main ###############

+tagName="unbound"

+sleepTime="${1:-5m}" # default to sleep for 5m (5 minutes)

+zoneList=$( /usr/sbin/unbound-control list_auth_zones | /usr/bin/awk '{print $1}' )

+for zone in ${zoneList} ; do

• /usr/bin/printf "disable ${zone}\t"
• /usr/sbin/unbound-control rpz_disable "${zone}"

unboundctrl

...

...

...

...

...

+done

+msg_log "info: rpz: disabled all zones for ${sleepTime}"

+/bin/sleep "${sleepTime}"

+for zone in ${zoneList} ; do

• /usr/bin/printf "enable ${zone}\t"
• /usr/sbin/unbound-control rpz_enable "${zone}"

unboundctrl

...

...

...

...

...

+done

+msg_log "info: rpz: enabled all zones"

+exit diff --git a/lfs/rpz b/lfs/rpz new file mode 100644 index 000000000..319c10b7f --- /dev/null +++ b/lfs/rpz @@ -0,0 +1,88 @@ +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2024 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +###############################################################################

+############################################################################### +# Definitions +###############################################################################

+include Config

+SUMMARY = response policy zone - RPZ reputation system for unbound DNS

+VER = 1.0.0

+THISAPP = rpz-$(VER) +DIR_APP = $(DIR_SRC)/$(THISAPP) +TARGET = $(DIR_INFO)/$(THISAPP)

+PROG = rpz +PAK_VER = 1

+DEPS =

+SERVICES =

+############################################################################### +# Top-level Rules +###############################################################################

+install : $(TARGET)

+check :

+download :

+b2 :

+dist:

• @$(PAK)

+############################################################################### +# Installation Details +###############################################################################

+$(TARGET) :

• @$(PREBUILD)
• @rm -rf $(DIR_APP)
• # install RPZ scripts
• install -v -m 755 \
• $(DIR_CONF)/rpz/{rpz-config,rpz-metrics,rpz-sleep} -t /usr/sbin
• # Install settings folder and two empty files
• mkdir -pv /var/ipfire/rpz
• touch /var/ipfire/rpz/allowlist
• touch /var/ipfire/rpz/blocklist
• # Add conf file to /etc directory
• cp -vf $(DIR_CONF)/rpz/00-rpz.conf /etc/unbound/local.d
• # create zonefiles directory for the RPZ files and add two empty RPZ
• # files to avoid a unbound config error
• mkdir -pv /etc/unbound/zonefiles
• chown -v nobody:nobody /etc/unbound/zonefiles
• touch /etc/unbound/zonefiles/allow.rpz
• touch /etc/unbound/zonefiles/block.rpz
• # Install backup definition
• cp -vf $(DIR_CONF)/backup/includes/rpz /var/ipfire/backup/addons/includes/rpz
• @rm -rf $(DIR_APP)
• @$(POSTBUILD)

diff --git a/make.sh b/make.sh index 9bbbeb0f1..886d3760a 100755 --- a/make.sh +++ b/make.sh @@ -1721,6 +1721,8 @@ buildipfire() { lfsmake2 btrfs-progs lfsmake2 inotify-tools lfsmake2 grub-btrfs

• lfsmake2 rpz

# Kernelbuild ... current we have no platform that need

# multi kernel builds so KCFG is empty

2.30.2

-- Sent from my laptop