What is it? Response Policy Zone (RPZ) is a mechanism to define local policies in a standardised way and load those policies from external sources. Bottom line: RPZ allows admins to easily block access to websites via DNS lookup.
RPZ can block websites via categories. Examples include: fake websites, annoying pop-up ads, newly registered domains, DoH bypass sites, bad "host" services, maliscious top level domains (e.g., *.zip, *.mov), piracy, gambling, pornography, and more. RPZ lists come from various RPZ providers and their available catagories.
This RPZ add-on enables the RPZ functionality by adding a couple lines in a configuration file. This add-on simply adds a configuration file and adds three scripts (config, metrics and sleep) to make RPZ easier for the admin to use.
RPZ was release in 2010 and has been part of the IPFire build since ~2015.
Why is it needed? Some IPFire admin's utilize pihole to block unwanted websites via DNS lookup. Moving the pihole base functionality (without pretty graphs) to IPFire removes one device from the admin's local network. And hopefully this reduces the pihole questions from the Community.
A list of RPZ providers can be recommended by IPFire and coded into a set list. Or, if prefered, the local admin can choose their own RPZ providers.
This is a: * simple replacement for pihole base functionality * RPZ can be a nice replacement for the URL Filter
IPFire Wiki In process at: https://www.ipfire.org/docs/addons/rpz
more info: * https://en.wikipedia.org/wiki/Response_policy_zone * https://unbound.docs.nlnetlabs.nl/en/latest/topics/filtering/rpz.html
I need help with... 1) The custom allow and block lists are currently located at `/var/ipfire/rpz`. Is this correct? 2) The three bash scripts are currently located at `/usr/sbin`. Is this correct?
Signed-off-by: Jon Murphy jon.murphy@ipfire.org --- config/backup/includes/rpz | 5 + config/rootfiles/packages/rpz | 11 ++ config/rpz/00-rpz.conf | 18 ++++ config/rpz/rpz-config | 194 ++++++++++++++++++++++++++++++++++ config/rpz/rpz-metrics | 143 +++++++++++++++++++++++++ config/rpz/rpz-sleep | 58 ++++++++++ lfs/rpz | 88 +++++++++++++++ make.sh | 2 + 8 files changed, 519 insertions(+) create mode 100644 config/backup/includes/rpz create mode 100644 config/rootfiles/packages/rpz create mode 100644 config/rpz/00-rpz.conf create mode 100644 config/rpz/rpz-config create mode 100644 config/rpz/rpz-metrics create mode 100644 config/rpz/rpz-sleep create mode 100644 lfs/rpz
diff --git a/config/backup/includes/rpz b/config/backup/includes/rpz new file mode 100644 index 000000000..4d59bb40c --- /dev/null +++ b/config/backup/includes/rpz @@ -0,0 +1,5 @@ +/var/ipfire/rpz/allowlist +/var/ipfire/rpz/blocklist +/etc/unbound/zonefiles/allow.rpz +/etc/unbound/zonefiles/block.rpz +/etc/unbound/local.d/*rpz.conf diff --git a/config/rootfiles/packages/rpz b/config/rootfiles/packages/rpz new file mode 100644 index 000000000..2ffa715dd --- /dev/null +++ b/config/rootfiles/packages/rpz @@ -0,0 +1,11 @@ +etc/unbound/local.d/00-rpz.conf +etc/unbound/zonefiles +etc/unbound/zonefiles/allow.rpz +etc/unbound/zonefiles/block.rpz +usr/sbin/rpz-config +usr/sbin/rpz-metrics +usr/sbin/rpz-sleep +var/ipfire/backup/addons/includes/rpz +var/ipfire/rpz +var/ipfire/rpz/allowlist +var/ipfire/rpz/blocklist diff --git a/config/rpz/00-rpz.conf b/config/rpz/00-rpz.conf new file mode 100644 index 000000000..72c1d12e5 --- /dev/null +++ b/config/rpz/00-rpz.conf @@ -0,0 +1,18 @@ +server: + module-config: "respip validator iterator" + +rpz: + name: allow.rpz + zonefile: /etc/unbound/zonefiles/allow.rpz + rpz-action-override: passthru + rpz-log: yes + rpz-log-name: allow + rpz-signal-nxdomain-ra: yes + +rpz: + name: block.rpz + zonefile: /etc/unbound/zonefiles/block.rpz + rpz-action-override: nxdomain + rpz-log: yes + rpz-log-name: block + rpz-signal-nxdomain-ra: yes diff --git a/config/rpz/rpz-config b/config/rpz/rpz-config new file mode 100644 index 000000000..98dc0a4ca --- /dev/null +++ b/config/rpz/rpz-config @@ -0,0 +1,194 @@ +#!/bin/bash +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2024 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +############################################################################### + +# v22 - 2024-07-12 + +############### Functions ############### + +msg_log () { + /usr/bin/logger --tag "${tagName}" "$*" + if tty --silent ; then + echo "${tagName}:" "$*" + fi +} + +check_name () { + local testName="${1}" + # check for a valid name + regex='^[a-zA-Z0-9_]+$' + if [[ ! "${testName}" =~ $regex ]] ; then + msg_log "error: rpz: the NAME is not valid: "${testName}". exit." + exit 1 + fi +} + +check_unbound_conf () { + # check the above config files + msg_log "info: rpz: check for errors with "unbound-checkconf"" + /usr/sbin/unbound-checkconf + exit_code=$? + if [[ "${exit_code}" -ne 0 ]] ; then + msg_log "error: rpz: unbound-checkconf. exit." + exit "${exit_code}" + fi +} + +make_rpz_file () { + local theType="${1}" # allow or block + + theList="/var/ipfire/rpz/${theType}list" # input user list of domains + theZoneFile="/etc/unbound/zonefiles/${theType}.rpz" # output file for RPZ + + theAction='.' + if [[ "${theType}" =~ "block" ]] ; then + theAction='rpz-passthru.' + fi + + # does a list exist? + if [[ -s "${theList}" ]] ; then + # drop any extra "blanks" and add "CNAME <RPZ action>." to each line + actionList=$( /usr/bin/awk '{$1=$1};1' "${theList}" | + /bin/sed "/^[^;].*[[:alnum:]]/ s|$| CNAME ${theAction}|" ) + + msg_log "info: rpz: create zonefile for ${theList}" + + /bin/cat <<-EOF > "${theZoneFile}" + ; Name: ${theType} list + ; Last modified: $(date "+%Y-%m-%d at %H.%M.%S %Z") + ; + ; domains with actions list + ; + ${actionList} + EOF + + # reload the zone that was just updated + zoneBase=$( basename "${theZoneFile}" ) + /usr/sbin/unbound-control auth_zone_reload -q "${zoneBase}" + fi +} + +############### Main ############### + +tagName="unbound" + +theAction="${1}" # input action +theName="${2}" # input RPZ name +theURL="${3}" # input RPZ URL + +check_name "${theName}" # is this a valid name? + +rpzConfig="/etc/unbound/local.d/${theName}.rpz.conf" # output zone conf file +rpzFile="/etc/unbound/zonefiles/${theName}.rpz" # output for RPZ file + +case "${theAction}" in + + # add new rpz list + add ) + # does this config already exist? If yes, then exit + if [[ -f "${rpzConfig}" ]] ; then + msg_log "info: rpz: ${rpzConfig} already exists. exit" + exit 1 + fi + + # is this a valid URL? + regex='^https://%5B-%5B:alnum:%5D%5C+&@#/%?=~_%7C!:,.;%5D*%5B-%5B:alnum:%5D%5C+&...]' + if ! [[ "${theURL}" =~ $regex ]] ; then + msg_log "error: rpz: the URL is not valid: "${theURL}". exit." + exit 1 + fi + + # create the zone config file + msg_log "info: rpz: add config file "${theName}.rpz.conf"" + cat <<-EOF > "${rpzConfig}" + rpz: + name: ${theName}.rpz + zonefile: /etc/unbound/zonefiles/${theName}.rpz + url: ${theURL} + rpz-action-override: nxdomain + rpz-log: yes + rpz-log-name: ${theName} + rpz-signal-nxdomain-ra: yes + EOF + + # set-up zone file + /usr/bin/touch "${rpzFile}" + # unbound requires these settings for rpz files + /bin/chown --verbose nobody:nobody "${rpzFile}" + /bin/chmod --verbose 644 "${rpzFile}" + ;; + + # trash config file & rpz file + remove ) + if ! [[ -f "${rpzConfig}" ]] ; then + msg_log "info: rpz: ${rpzConfig} does not exist. exit" + exit 1 + fi + + msg_log "info: rpz: remove config file & rpz file "${theName}"" + /bin/rm --verbose "${rpzConfig}" + /bin/rm --verbose "${rpzFile}" + + check_unbound_conf + ;; + + # make a new allow or block rpz file + make ) + case "${theName}" in + allow ) + make_rpz_file allow + ;; + + block ) + make_rpz_file block + ;; + + allowblock ) + make_rpz_file allow + make_rpz_file block + ;; + + * ) + msg_log "error: rpz: the NAME is not valid: "${theName}". exit." + exit 1 + ;; + esac + + check_unbound_conf + ;; + + *) + msg_log "error: rpz: missing or incorrect parameter" + /usr/bin/printf "Usage: rpzConfig.sh <ACTION> <NAME> <URL>\n" + exit 1 + ;; + +esac + +# reload due to the changes +msg_log "rpz: running "unbound-control reload"" +/usr/sbin/unbound-control reload +exit_code=$? +if [[ "${exit_code}" -ne 0 ]] ; then + msg_log "error: rpz: unbound-control "${theName}". exit." + exit "${exit_code}" +fi + +exit diff --git a/config/rpz/rpz-metrics b/config/rpz/rpz-metrics new file mode 100644 index 000000000..0f97c7911 --- /dev/null +++ b/config/rpz/rpz-metrics @@ -0,0 +1,143 @@ +#!/bin/bash +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2024 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +############################################################################### + +# v18 on 2024-07-05 + +############### Main ############### + +weeks="${1:-2}" # default to two message logs +sortBy="${2:-name}" # by name or by hits + +# get the list of message logs for N weeks +messageLogs=$( find /var/log/messages* -type f | + /usr/bin/sort --version-sort | + head -"${weeks}" ) + +# get the list of RPZ names & counts from the message log(s) +rpzNameCount=$( for logf in ${messageLogs} ; do + /usr/bin/zgrep --text --fixed-strings 'info: rpz: applied' "${logf}" | + /usr/bin/awk '$10 ~ /[\w*]/ { print $10 }' ; + done | /usr/bin/sort | /usr/bin/uniq --count ) + +# flip results and remove brackets `[` and `]` +rpzNameCount=$( /bin/echo "${rpzNameCount}" | + /usr/bin/awk '{ print $2, $1 }' | + /bin/sed --regexp-extended 's|^[(.*)]|\1|' ) + +# grab only names +rpzNames=$( /bin/echo "${rpzNameCount}" | /usr/bin/awk '{ print $1 }' ) + +# get list of RPZ files +rpzFileList=$( /bin/find /etc/unbound/zonefiles -type f -iname "*.rpz" ) + +# get basename of those files +rpzBaseNames=$( /bin/echo "${rpzFileList}" | + /bin/sed 's|/etc/unbound/zonefiles/||g ; s|.rpz||g ;' ) + +# add to rpzNames +rpzNames="${rpzNames}"$'\n'"${rpzBaseNames}" + +# drop duplicate names +rpzNames=$( echo "${rpzNames}" | /usr/bin/sort --unique ) + +# get line count for each RPZ +lineCount=$( /bin/echo "${rpzFileList}" | /usr/bin/xargs wc -l ) + +# get comment line count and blank line count for each RPZ +commentCount=$( /bin/echo "${rpzFileList}" | + /usr/bin/xargs /bin/grep --count -e "^$" -e "^;" ) + +# get modified date each RPZ +modDateList=$( /bin/echo "${rpzFileList}" | xargs stat -c '%.10y %n' ) + +ucListAuthZones=$( /usr/sbin/unbound-control list_auth_zones ) + +# get width of RPZ names +pWidth=$( /bin/echo "${rpzNames}" | /usr/bin/awk '{ print $1" " }' | wc -L ) +pFormat="%-${pWidth}s %-8s %-8s %-8s %10s %12s\n" + +# print title line +printf "${pFormat}" "name" "hits" "active" "lines" "hits/line%" "last update" +printf -- "--------------" + +theResults="" +totalLines=0 +totalHits=0 +while read -r theName +do + printf -- "-" # pretend progress bar + # get hit count + theHits="0" + if output=$( /bin/grep "^${theName}\s" <<< "${rpzNameCount}" ) ; then + theHits=$( /bin/echo "${output}" | + /usr/bin/awk '{ print $2 }' ) + totalHits=$(( totalHits + theHits )) + fi + + # is this RPZ list active? + theActive="disabled" + if /bin/grep --quiet "^${theName}.rpz" <<< "${ucListAuthZones}" + then + theActive="enabled" + fi + + # get line count then subtract comment count and blank line count + # from total line count + theLines="n/a" + hitsPerLine="0" + if output=$( /bin/grep --fixed-strings "/${theName}.rpz" <<< "${lineCount}" ) ; then + theLines=$( /bin/echo "${output}" | /usr/bin/awk '{ print $1 }' ) + totalLines=$(( totalLines + theLines )) + + #hitsPerLine=$( echo "scale=0 ; $theHits / $theLines" | bc ) + hitsPerLine=$(( 100 * theHits / theLines )) + fi + + # get modification date + theModDate="n/a" + if output=$( /bin/grep --fixed-strings "/${theName}.rpz" <<< "${modDateList}" ) ; then + theModDate=$( /bin/echo "${output}" | /usr/bin/awk '{ print $1 }' ) + fi + + # add to results list + theResults+="${theName} ${theHits} ${theActive} ${theLines} ${hitsPerLine} ${theModDate}"$'\n' +done <<< "${rpzNames}" + +case "${sortBy}" in + names|name) sortArg=(-k3,3r -k1,1) ;; # sort by "active" then by "name" + + hits|hit) sortArg=(-k3,3r -k2,2nr -k1,1) ;; # sort by "active" then by "hits" then by "name" + + lines|line) sortArg=(-k3,3r -k4,4nr -k1,1) ;; # sort by "active" then by "lines" then by "name" +esac + +printf -- "--------------\n" +# remove blank lines, sort, print as columns +/bin/echo "${theResults}" | + /usr/bin/awk '!/^[[:space:]]*$/' | + /usr/bin/sort "${sortArg[@]}" | + /usr/bin/awk --assign=width="${pWidth}" \ + '{ printf "%-*s %-8s %-8s %-8s %10s %12s\n", width, $1, $2, $3, $4, $5, $6 }' + +printf "${pFormat}" "" "=======" "" "========" "" "" +printf "${pFormat}" "Totals -->" "${totalHits}" "" "${totalLines}" "" "" + +exit diff --git a/config/rpz/rpz-sleep b/config/rpz/rpz-sleep new file mode 100644 index 000000000..eeef1174a --- /dev/null +++ b/config/rpz/rpz-sleep @@ -0,0 +1,58 @@ +#!/bin/bash +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2024 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +############################################################################### + +# v04 on 2024-07-05 + +############### Functions ############### + +# send message to message log +msg_log () { + /usr/bin/logger --tag "${tagName}" "$*" + if /usr/bin/tty --silent ; then + echo "${tagName}:" "$*" + fi +} + +############### Main ############### + +tagName="unbound" + +sleepTime="${1:-5m}" # default to sleep for 5m (5 minutes) + +zoneList=$( /usr/sbin/unbound-control list_auth_zones | /usr/bin/awk '{print $1}' ) + +for zone in ${zoneList} ; do + /usr/bin/printf "disable ${zone}\t" + /usr/sbin/unbound-control rpz_disable "${zone}" +done + +msg_log "info: rpz: disabled all zones for ${sleepTime}" + +/bin/sleep "${sleepTime}" + +for zone in ${zoneList} ; do + /usr/bin/printf "enable ${zone}\t" + /usr/sbin/unbound-control rpz_enable "${zone}" +done + +msg_log "info: rpz: enabled all zones" + +exit diff --git a/lfs/rpz b/lfs/rpz new file mode 100644 index 000000000..319c10b7f --- /dev/null +++ b/lfs/rpz @@ -0,0 +1,88 @@ +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2024 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +############################################################################### + +############################################################################### +# Definitions +############################################################################### + +include Config + +SUMMARY = response policy zone - RPZ reputation system for unbound DNS + +VER = 1.0.0 + +THISAPP = rpz-$(VER) +DIR_APP = $(DIR_SRC)/$(THISAPP) +TARGET = $(DIR_INFO)/$(THISAPP) + +PROG = rpz +PAK_VER = 1 + +DEPS = + +SERVICES = + +############################################################################### +# Top-level Rules +############################################################################### + +install : $(TARGET) + +check : + +download : + +b2 : + +dist: + @$(PAK) + +############################################################################### +# Installation Details +############################################################################### + +$(TARGET) : + @$(PREBUILD) + @rm -rf $(DIR_APP) + + # install RPZ scripts + install -v -m 755 \ + $(DIR_CONF)/rpz/{rpz-config,rpz-metrics,rpz-sleep} -t /usr/sbin + + # Install settings folder and two empty files + mkdir -pv /var/ipfire/rpz + touch /var/ipfire/rpz/allowlist + touch /var/ipfire/rpz/blocklist + + # Add conf file to /etc directory + cp -vf $(DIR_CONF)/rpz/00-rpz.conf /etc/unbound/local.d + + # create zonefiles directory for the RPZ files and add two empty RPZ + # files to avoid a unbound config error + mkdir -pv /etc/unbound/zonefiles + chown -v nobody:nobody /etc/unbound/zonefiles + touch /etc/unbound/zonefiles/allow.rpz + touch /etc/unbound/zonefiles/block.rpz + + # Install backup definition + cp -vf $(DIR_CONF)/backup/includes/rpz /var/ipfire/backup/addons/includes/rpz + + @rm -rf $(DIR_APP) + @$(POSTBUILD) diff --git a/make.sh b/make.sh index 9bbbeb0f1..886d3760a 100755 --- a/make.sh +++ b/make.sh @@ -1721,6 +1721,8 @@ buildipfire() { lfsmake2 btrfs-progs lfsmake2 inotify-tools lfsmake2 grub-btrfs + lfsmake2 rpz +
# Kernelbuild ... current we have no platform that need # multi kernel builds so KCFG is empty