From mboxrd@z Thu Jan 1 00:00:00 1970 From: jon To: development@lists.ipfire.org Subject: Re: [PATCH] RPZ: install new add-on Date: Sun, 28 Jul 2024 21:28:30 -0500 Message-ID: In-Reply-To: <9934FE55-58F0-403C-AAE9-AF819F23F0CD@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4678702258645145430==" List-Id: --===============4678702258645145430== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable See comments below... > On Jul 28, 2024, at 6:22=E2=80=AFAM, Michael Tremer wrote: >=20 > Hello Jon, >=20 > I don=E2=80=99t think it is a good idea to send over the same patch set mul= tiple times. >=20 > I think first of all, we need to start at the beginning which is to figure = out what this actually is, and why we would want this in IPFire. >=20 > I have seen that there have been conversations on community.ipfire.org many months ago, but I did not take many things aw= ay from it. It was from my point of view mostly a prove of concept discussion= . Correct me if I have missed anything here. >=20 > We have talked about RPZ many times on the monthly call since the URL filte= r feature is falling more and more out of fashion. I think there is also many= posts about this on the forum. Yes, I have seen the posts about the url filter feature falling out of fashio= n. I did not see notes about RPZ from the monthly calls. I will go back and= search telco call notes. >=20 > Apart from generally being in favour of trying this path, we decided to pos= tpone any further works into this because we currently have a huge backlog of= things to deal with that are moving at snail speed. This is rooted in so ma= ny people being simply tied in with other things. >=20 > As you can see, the patch set that you posted did not trigger a response - = a common thing on this list recently and not because of this particular patch= set. >=20 > Therefore I would like to avoid opening another can of worms before we have= delivered the other things that have already lots of time spent on them and = code written. Otherwise we are only getting a longer pipeline and this all is= going to be more frustrating for everyone involved. And this is not about th= e feature per se; this is about getting things done. This is why I wrote the RPZ patch instead of asking for the Developers to wri= te it. I realize there is a very long pipeline. >=20 > So what I suggest doing is the following: >=20 > You have a Git repository, push the code there and commit any fixes to it o= ver time so that nothing is getting lost. This was done a few days ago. But I am not sure if I did it correctly. If n= ot, please let me know. >=20 > Explain to me from the beginning, please, how it came to this patch set. Wh= at is the motivation? In the beginning (Fall of 2023) the main motivation was blocking DoH. I had = noticed apps using their own method of DNS lookup. Some were friendly DoH lo= okups (Apple iCloud?) and some not (mozilla.cloudflare-dns.com but not from F= irefox). Examples I saw are: mozilla.cloudflare-dns.com www.switch.ch use-application-dns.net (a canary for Firef= ox?) And a few others... This worked better than expected and was easier to configure than expected. = All of the main code was part of unbound and I just need to activate it with = a config file. Updates to the RPZ files are already part of the unbound RPZ = code. As I experimented with RPZ I quickly realized it could easily be used to bloc= k other websites. I have been using pihole for a number of years to limit th= e number of ads and trackers. And RPZ helped with those areas also. > What problems does it solve? What problems remain? Basically: Why would any= one want this in IPFire? It solved the issue of blocking unwanted DoH from apps. And it solved the is= sue of unwanted ads and tracking. For me those are huge benefits and the big= gest reason for using RPZ. To test other possibilities, I increased the "type" of lists to include other= categories other than ads & trackers. Like blocking bad TLDs, gambling, pir= acy, bad host sites, etc. This is a nice bonus of using RPZ for me. > Where do you see application for this? As a replacement for the URL Filter. And as a base functional replacement fo= r pihole (but no pretty graphics). > Why is this realised as an add-on and not part of the core system? In my opinion it should be part of the core. But I wanted to "test the water= s" first to determine interest. If only four people use it, then it isn=E2= =80=99t worth adding to core and maybe not as an add-on. Either way I will support it because it helps me! Here are my metrics for the past eight days (searching `messages` and `messag= es.1.gz`): [root(a)ipfire ~] # rpz-metrics name hits =20 ---------------------------- allow 5560 This is a custom list =20 AmazonTrkrHZ 5726 =20 AppleTrkrHZ 2956 =20 block 128 This is a custom list =20 DOHblockHZ 25077 =20 dohJPG 3448 =20 HosterHZ 19 =20 MxProPlusHZ 17287 =20 NotSafeSearchHZ 0 =20 ProxyBypassHZ 11 =20 tldHZ 15 =20 WinTrkrHZ 507 =20 =3D=3D=3D=3D=3D=3D=3D =20 Totals --> 60734 =20 For the DoH blocking, the above "hits" including blocking Apple DoH. I=E2=80= =99ve been testing this for the past few weeks. =20 >=20 > I am particularly interested in what lists are publicly available. You know= from URL Filter, that lists have shut down or just been abandoned. There is = now very little point running it because the lists are just not as good as th= ey used to be. Not horrible, yet, but over time this will not be getting any = better. I am not even aware of any viable paid lists. Is this the same situat= ion for RPZ? A quick Google search did not give me anything that would be fr= ee to use over DNS. The situation is somewhat similar with the IPS and IP blo= cklist feature. These are free/open lists: Hagezi - DNS Blocklists=20 =E2=80=A2 at https://github.com/hagezi/dns-blocklists?tab=3Dreadme-ov-file#za= p-dns-blocklists---for-a-better-internet =E2=80=A2 This list was recommended within the IPFire Community and this is t= he list I strongly recommend. Gerd (Hagezi) is very responsive to support re= quests. The Hagezi RPZ list is well supported and the RPZ lists are updated t= wice per day (if changes are needed). =E2=80=A2 Example: I came across a domain name that was blocked `f005.backbla= zeb2.com` and it was fixed (removed from the block list) within a few days. ThreatFox - DNS Response Policy Zone (RPZ)=20 =E2=80=A2 at https://threatfox.abuse.ch/export/#rpz URLHaus - DNS Response Policy Zone (RPZ) =E2=80=A2 at https://urlhaus.abuse.ch/api/#rpz jpgpi250 - DNS block list for DoH =E2=80=A2 at https://github.com/jpgpi250/piholemanual/blob/master/DOH.rpz =E2=80=A2 this is the original DoH block list that I started with. Peter (jp= gpi250) has a nice PDF document explaining RPZ and was very helpful getting m= e started with RPZ. See https://jpgpi250.github.io/piholemanual/doc/Unbound%= 20response%20policy%20zones.pdf I have come across other RPZ lists but these are the best supported lists. I did not look into paid lists. >=20 >=20 > I would be interested in hearing your thoughts on this. Does the above help? >=20 > Best, > -Michael >=20 >> On 27 Jul 2024, at 21:31, jon wrote: >>=20 >> And another question. >>=20 >> I found an error in the `rpz-config` script file. Simple easy one line (a= ctually one word) fix. =20 >>=20 >> Do I resubmit the entire patch? >> -or- >> Do I wait until approved and then submit the simple patch for the one line= change. >>=20 >>=20 >> Jon >>=20 >>=20 >>> On Jul 20, 2024, at 1:42=E2=80=AFPM, jon wrote: >>>=20 >>> I need help with=E2=80=A6 >>>=20 >>> 1) The custom allow and block lists are currently located at `/var/ipfire= /rpz`. Is this correct? >>>=20 >>> 2) The three bash scripts are currently located at `/usr/sbin`. Is this = correct? >>>=20 >>> 3) Should this commit be separated into multiple posts? If yes, how do I= decide how to split it? >>>=20 >>>=20 >>> Jon=20 >>>=20 >>>=20 >>>> On Jul 20, 2024, at 12:46=E2=80=AFPM, Jon Murphy wrote: >>>>=20 >>>> What is it? >>>> Response Policy Zone (RPZ) is a mechanism to define local policies in a = standardised >>>> way and load those policies from external sources. >>>> Bottom line: RPZ allows admins to easily block access to websites via DN= S lookup. >>>>=20 >>>> RPZ can block websites via categories. Examples include: fake websites,= annoying >>>> pop-up ads, newly registered domains, DoH bypass sites, bad "host" servi= ces, >>>> maliscious top level domains (e.g., *.zip, *.mov), piracy, gambling, por= nography, >>>> and more. RPZ lists come from various RPZ providers and their available >>>> catagories. >>>>=20 >>>> This RPZ add-on enables the RPZ functionality by adding a couple lines i= n a >>>> configuration file. This add-on simply adds a configuration file and ad= ds >>>> three scripts (config, metrics and sleep) to make RPZ easier for the adm= in to use. >>>>=20 >>>> RPZ was release in 2010 and has been part of the IPFire build since ~201= 5. >>>>=20 >>>> Why is it needed? >>>> Some IPFire admin's utilize pihole to block unwanted websites via DNS lo= okup. >>>> Moving the pihole base functionality (without pretty graphs) to IPFire r= emoves >>>> one device from the admin's local network. >>>> And hopefully this reduces the pihole questions from the Community. >>>>=20 >>>> A list of RPZ providers can be recommended by IPFire and coded into a se= t list. >>>> Or, if prefered, the local admin can choose their own RPZ providers. >>>>=20 >>>> This is a: >>>> * simple replacement for pihole base functionality >>>> * RPZ can be a nice replacement for the URL Filter >>>>=20 >>>> IPFire Wiki >>>> In process at: https://www.ipfire.org/docs/addons/rpz >>>>=20 >>>> more info: >>>> * https://en.wikipedia.org/wiki/Response_policy_zone >>>> * https://unbound.docs.nlnetlabs.nl/en/latest/topics/filtering/rpz.html >>>>=20 >>>> I need help with... >>>> 1) The custom allow and block lists are currently located at `/var/ipfir= e/rpz`. Is this correct? >>>> 2) The three bash scripts are currently located at `/usr/sbin`. Is this= correct? >>>>=20 >>>> Signed-off-by: Jon Murphy >>>> --- >>>> config/backup/includes/rpz | 5 + >>>> config/rootfiles/packages/rpz | 11 ++ >>>> config/rpz/00-rpz.conf | 18 ++++ >>>> config/rpz/rpz-config | 194 ++++++++++++++++++++++++++++++++++ >>>> config/rpz/rpz-metrics | 143 +++++++++++++++++++++++++ >>>> config/rpz/rpz-sleep | 58 ++++++++++ >>>> lfs/rpz | 88 +++++++++++++++ >>>> make.sh | 2 + >>>> 8 files changed, 519 insertions(+) >>>> create mode 100644 config/backup/includes/rpz >>>> create mode 100644 config/rootfiles/packages/rpz >>>> create mode 100644 config/rpz/00-rpz.conf >>>> create mode 100644 config/rpz/rpz-config >>>> create mode 100644 config/rpz/rpz-metrics >>>> create mode 100644 config/rpz/rpz-sleep >>>> create mode 100644 lfs/rpz >>>>=20 >>>> diff --git a/config/backup/includes/rpz b/config/backup/includes/rpz >>>> new file mode 100644 >>>> index 000000000..4d59bb40c >>>> --- /dev/null >>>> +++ b/config/backup/includes/rpz >>>> @@ -0,0 +1,5 @@ >>>> +/var/ipfire/rpz/allowlist >>>> +/var/ipfire/rpz/blocklist >>>> +/etc/unbound/zonefiles/allow.rpz >>>> +/etc/unbound/zonefiles/block.rpz >>>> +/etc/unbound/local.d/*rpz.conf >>>> diff --git a/config/rootfiles/packages/rpz b/config/rootfiles/packages/r= pz >>>> new file mode 100644 >>>> index 000000000..2ffa715dd >>>> --- /dev/null >>>> +++ b/config/rootfiles/packages/rpz >>>> @@ -0,0 +1,11 @@ >>>> +etc/unbound/local.d/00-rpz.conf >>>> +etc/unbound/zonefiles >>>> +etc/unbound/zonefiles/allow.rpz >>>> +etc/unbound/zonefiles/block.rpz >>>> +usr/sbin/rpz-config >>>> +usr/sbin/rpz-metrics >>>> +usr/sbin/rpz-sleep >>>> +var/ipfire/backup/addons/includes/rpz >>>> +var/ipfire/rpz >>>> +var/ipfire/rpz/allowlist >>>> +var/ipfire/rpz/blocklist >>>> diff --git a/config/rpz/00-rpz.conf b/config/rpz/00-rpz.conf >>>> new file mode 100644 >>>> index 000000000..72c1d12e5 >>>> --- /dev/null >>>> +++ b/config/rpz/00-rpz.conf >>>> @@ -0,0 +1,18 @@ >>>> +server: >>>> + module-config: "respip validator iterator" >>>> + >>>> +rpz: >>>> + name: allow.rpz >>>> + zonefile: /etc/unbound/zonefiles/allow.rpz >>>> + rpz-action-override: passthru >>>> + rpz-log: yes >>>> + rpz-log-name: allow >>>> + rpz-signal-nxdomain-ra: yes >>>> + >>>> +rpz: >>>> + name: block.rpz >>>> + zonefile: /etc/unbound/zonefiles/block.rpz >>>> + rpz-action-override: nxdomain >>>> + rpz-log: yes >>>> + rpz-log-name: block >>>> + rpz-signal-nxdomain-ra: yes >>>> diff --git a/config/rpz/rpz-config b/config/rpz/rpz-config >>>> new file mode 100644 >>>> index 000000000..98dc0a4ca >>>> --- /dev/null >>>> +++ b/config/rpz/rpz-config >>>> @@ -0,0 +1,194 @@ >>>> +#!/bin/bash >>>> +#######################################################################= ######## >>>> +# = # >>>> +# IPFire.org - A linux based firewall = # >>>> +# Copyright (C) 2024 IPFire Team = # >>>> +# = # >>>> +# This program is free software: you can redistribute it and/or modify= # >>>> +# it under the terms of the GNU General Public License as published by= # >>>> +# the Free Software Foundation, either version 3 of the License, or = # >>>> +# (at your option) any later version. = # >>>> +# = # >>>> +# This program is distributed in the hope that it will be useful, = # >>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of = # >>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the = # >>>> +# GNU General Public License for more details. = # >>>> +# = # >>>> +# You should have received a copy of the GNU General Public License = # >>>> +# along with this program. If not, see = . # >>>> +# = # >>>> +#######################################################################= ######## >>>> + >>>> +# v22 - 2024-07-12 >>>> + >>>> +############### Functions ############### >>>> + >>>> +msg_log () { >>>> + /usr/bin/logger --tag "${tagName}" "$*" >>>> + if tty --silent ; then >>>> + echo "${tagName}:" "$*" >>>> + fi >>>> +} >>>> + >>>> +check_name () { >>>> + local testName=3D"${1}" >>>> + # check for a valid name >>>> + regex=3D'^[a-zA-Z0-9_]+$' >>>> + if [[ ! "${testName}" =3D~ $regex ]] ; then >>>> + msg_log "error: rpz: the NAME is not valid: \"${testName}\". exit." >>>> + exit 1 >>>> + fi >>>> +} >>>> + >>>> +check_unbound_conf () { >>>> + # check the above config files >>>> + msg_log "info: rpz: check for errors with \"unbound-checkconf\"" >>>> + /usr/sbin/unbound-checkconf >>>> + exit_code=3D$? >>>> + if [[ "${exit_code}" -ne 0 ]] ; then >>>> + msg_log "error: rpz: unbound-checkconf. exit." >>>> + exit "${exit_code}" >>>> + fi >>>> +} >>>> + >>>> +make_rpz_file () { >>>> + local theType=3D"${1}" # allow or block >>>> + >>>> + theList=3D"/var/ipfire/rpz/${theType}list" # input user list of domains >>>> + theZoneFile=3D"/etc/unbound/zonefiles/${theType}.rpz" # output file fo= r RPZ >>>> + >>>> + theAction=3D'.' >>>> + if [[ "${theType}" =3D~ "block" ]] ; then >>>> + theAction=3D'rpz-passthru.' >>>> + fi >>>> + >>>> + # does a list exist? >>>> + if [[ -s "${theList}" ]] ; then >>>> + # drop any extra "blanks" and add "CNAME ." to each line >>>> + actionList=3D$( /usr/bin/awk '{$1=3D$1};1' "${theList}" | >>>> + /bin/sed "/^[^;].*[[:alnum:]]/ s|$| CNAME ${theAction}|" ) >>>> + >>>> + msg_log "info: rpz: create zonefile for ${theList}" >>>> + >>>> + /bin/cat <<-EOF > "${theZoneFile}" >>>> + ; Name: ${theType} list >>>> + ; Last modified: $(date "+%Y-%m-%d at %H.%M.%S %Z") >>>> + ; >>>> + ; domains with actions list >>>> + ; >>>> + ${actionList} >>>> + EOF >>>> + >>>> + # reload the zone that was just updated >>>> + zoneBase=3D$( basename "${theZoneFile}" ) >>>> + /usr/sbin/unbound-control auth_zone_reload -q "${zoneBase}" >>>> + fi >>>> +} >>>> + >>>> +############### Main ############### >>>> + >>>> +tagName=3D"unbound" >>>> + >>>> +theAction=3D"${1}" # input action >>>> +theName=3D"${2}" # input RPZ name >>>> +theURL=3D"${3}" # input RPZ URL >>>> + >>>> +check_name "${theName}" # is this a valid name? >>>> + >>>> +rpzConfig=3D"/etc/unbound/local.d/${theName}.rpz.conf" # output zone co= nf file >>>> +rpzFile=3D"/etc/unbound/zonefiles/${theName}.rpz" # output for RPZ file >>>> + >>>> +case "${theAction}" in >>>> + >>>> + # add new rpz list >>>> + add ) >>>> + # does this config already exist? If yes, then exit >>>> + if [[ -f "${rpzConfig}" ]] ; then >>>> + msg_log "info: rpz: ${rpzConfig} already exists. exit" >>>> + exit 1 >>>> + fi >>>> + >>>> + # is this a valid URL? >>>> + regex=3D'^https://[-[:alnum:]\+&@#/%?=3D~_|!:,.;]*[-[:alnum:]\+&@#/%= =3D~_|]' >>>> + if ! [[ "${theURL}" =3D~ $regex ]] ; then >>>> + msg_log "error: rpz: the URL is not valid: \"${theURL}\". exit." >>>> + exit 1 >>>> + fi >>>> + >>>> + # create the zone config file >>>> + msg_log "info: rpz: add config file \"${theName}.rpz.conf\"" >>>> + cat <<-EOF > "${rpzConfig}" >>>> + rpz: >>>> + name: ${theName}.rpz >>>> + zonefile: /etc/unbound/zonefiles/${theName}.rpz >>>> + url: ${theURL} >>>> + rpz-action-override: nxdomain >>>> + rpz-log: yes >>>> + rpz-log-name: ${theName} >>>> + rpz-signal-nxdomain-ra: yes >>>> + EOF >>>> + >>>> + # set-up zone file >>>> + /usr/bin/touch "${rpzFile}" >>>> + # unbound requires these settings for rpz files >>>> + /bin/chown --verbose nobody:nobody "${rpzFile}" >>>> + /bin/chmod --verbose 644 "${rpzFile}" >>>> + ;; >>>> + >>>> + # trash config file & rpz file >>>> + remove ) >>>> + if ! [[ -f "${rpzConfig}" ]] ; then >>>> + msg_log "info: rpz: ${rpzConfig} does not exist. exit" >>>> + exit 1 >>>> + fi >>>> + >>>> + msg_log "info: rpz: remove config file & rpz file \"${theName}\"" >>>> + /bin/rm --verbose "${rpzConfig}" >>>> + /bin/rm --verbose "${rpzFile}" >>>> + >>>> + check_unbound_conf >>>> + ;; >>>> + >>>> + # make a new allow or block rpz file >>>> + make ) >>>> + case "${theName}" in >>>> + allow ) >>>> + make_rpz_file allow >>>> + ;; >>>> + >>>> + block ) >>>> + make_rpz_file block >>>> + ;; >>>> + >>>> + allowblock ) >>>> + make_rpz_file allow >>>> + make_rpz_file block >>>> + ;; >>>> + >>>> + * ) >>>> + msg_log "error: rpz: the NAME is not valid: \"${theName}\". exit." >>>> + exit 1 >>>> + ;; >>>> + esac >>>> + >>>> + check_unbound_conf >>>> + ;; >>>> + >>>> + *) >>>> + msg_log "error: rpz: missing or incorrect parameter" >>>> + /usr/bin/printf "Usage: rpzConfig.sh \n" >>>> + exit 1 >>>> + ;; >>>> + >>>> +esac >>>> + >>>> +# reload due to the changes >>>> +msg_log "rpz: running \"unbound-control reload\"" >>>> +/usr/sbin/unbound-control reload >>>> +exit_code=3D$? >>>> +if [[ "${exit_code}" -ne 0 ]] ; then >>>> + msg_log "error: rpz: unbound-control \"${theName}\". exit." >>>> + exit "${exit_code}" >>>> +fi >>>> + >>>> +exit >>>> diff --git a/config/rpz/rpz-metrics b/config/rpz/rpz-metrics >>>> new file mode 100644 >>>> index 000000000..0f97c7911 >>>> --- /dev/null >>>> +++ b/config/rpz/rpz-metrics >>>> @@ -0,0 +1,143 @@ >>>> +#!/bin/bash >>>> +#######################################################################= ######## >>>> +# = # >>>> +# IPFire.org - A linux based firewall = # >>>> +# Copyright (C) 2024 IPFire Team = # >>>> +# = # >>>> +# This program is free software: you can redistribute it and/or modify= # >>>> +# it under the terms of the GNU General Public License as published by= # >>>> +# the Free Software Foundation, either version 3 of the License, or = # >>>> +# (at your option) any later version. = # >>>> +# = # >>>> +# This program is distributed in the hope that it will be useful, = # >>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of = # >>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the = # >>>> +# GNU General Public License for more details. = # >>>> +# = # >>>> +# You should have received a copy of the GNU General Public License = # >>>> +# along with this program. If not, see = . # >>>> +# = # >>>> +#######################################################################= ######## >>>> + >>>> +# v18 on 2024-07-05 >>>> + >>>> +############### Main ############### >>>> + >>>> +weeks=3D"${1:-2}" # default to two message logs >>>> +sortBy=3D"${2:-name}" # by name or by hits >>>> + >>>> +# get the list of message logs for N weeks >>>> +messageLogs=3D$( find /var/log/messages* -type f | >>>> + /usr/bin/sort --version-sort | >>>> + head -"${weeks}" ) >>>> + >>>> +# get the list of RPZ names & counts from the message log(s) >>>> +rpzNameCount=3D$( for logf in ${messageLogs} ; do >>>> + /usr/bin/zgrep --text --fixed-strings 'info: rpz: applied' "${logf}" | >>>> + /usr/bin/awk '$10 ~ /\[\w*]/ { print $10 }' ; >>>> + done | /usr/bin/sort | /usr/bin/uniq --count ) >>>> + >>>> +# flip results and remove brackets `[` and `]` >>>> +rpzNameCount=3D$( /bin/echo "${rpzNameCount}" | >>>> + /usr/bin/awk '{ print $2, $1 }' | >>>> + /bin/sed --regexp-extended 's|^\[(.*)\]|\1|' ) >>>> + >>>> +# grab only names >>>> +rpzNames=3D$( /bin/echo "${rpzNameCount}" | /usr/bin/awk '{ print $1 }'= ) >>>> + >>>> +# get list of RPZ files >>>> +rpzFileList=3D$( /bin/find /etc/unbound/zonefiles -type f -iname "*.rpz= " ) >>>> + >>>> +# get basename of those files >>>> +rpzBaseNames=3D$( /bin/echo "${rpzFileList}" | >>>> + /bin/sed 's|/etc/unbound/zonefiles/||g ; s|\.rpz||g ;' ) >>>> + >>>> +# add to rpzNames >>>> +rpzNames=3D"${rpzNames}"$'\n'"${rpzBaseNames}" >>>> + >>>> +# drop duplicate names >>>> +rpzNames=3D$( echo "${rpzNames}" | /usr/bin/sort --unique ) >>>> + >>>> +# get line count for each RPZ >>>> +lineCount=3D$( /bin/echo "${rpzFileList}" | /usr/bin/xargs wc -l ) >>>> + >>>> +# get comment line count and blank line count for each RPZ >>>> +commentCount=3D$( /bin/echo "${rpzFileList}" | >>>> + /usr/bin/xargs /bin/grep --count -e "^$" -e "^;" ) >>>> + >>>> +# get modified date each RPZ >>>> +modDateList=3D$( /bin/echo "${rpzFileList}" | xargs stat -c '%.10y %n'= ) >>>> + >>>> +ucListAuthZones=3D$( /usr/sbin/unbound-control list_auth_zones ) >>>> + >>>> +# get width of RPZ names >>>> +pWidth=3D$( /bin/echo "${rpzNames}" | /usr/bin/awk '{ print $1" " }' = | wc -L ) >>>> +pFormat=3D"%-${pWidth}s %-8s %-8s %-8s %10s %12s\n" >>>> + >>>> +# print title line >>>> +printf "${pFormat}" "name" "hits" "active" "lines" "hits/line%" "last u= pdate" >>>> +printf -- "--------------" >>>> + >>>> +theResults=3D"" >>>> +totalLines=3D0 >>>> +totalHits=3D0 >>>> +while read -r theName >>>> +do >>>> + printf -- "-" # pretend progress bar >>>> + # get hit count >>>> + theHits=3D"0" >>>> + if output=3D$( /bin/grep "^${theName}\s" <<< "${rpzNameCount}" ) ; then >>>> + theHits=3D$( /bin/echo "${output}" | >>>> + /usr/bin/awk '{ print $2 }' ) >>>> + totalHits=3D$(( totalHits + theHits )) >>>> + fi >>>> + >>>> + # is this RPZ list active? >>>> + theActive=3D"disabled" >>>> + if /bin/grep --quiet "^${theName}\.rpz" <<< "${ucListAuthZones}" >>>> + then >>>> + theActive=3D"enabled" >>>> + fi >>>> + >>>> + # get line count then subtract comment count and blank line count >>>> + # from total line count >>>> + theLines=3D"n/a" >>>> + hitsPerLine=3D"0" >>>> + if output=3D$( /bin/grep --fixed-strings "/${theName}.rpz" <<< "${line= Count}" ) ; then >>>> + theLines=3D$( /bin/echo "${output}" | /usr/bin/awk '{ print $1 }' ) >>>> + totalLines=3D$(( totalLines + theLines )) >>>> + >>>> + #hitsPerLine=3D$( echo "scale=3D0 ; $theHits / $theLines" | bc ) >>>> + hitsPerLine=3D$(( 100 * theHits / theLines )) >>>> + fi >>>> + >>>> + # get modification date >>>> + theModDate=3D"n/a" >>>> + if output=3D$( /bin/grep --fixed-strings "/${theName}.rpz" <<< "${modD= ateList}" ) ; then >>>> + theModDate=3D$( /bin/echo "${output}" | /usr/bin/awk '{ print $1 }' ) >>>> + fi >>>> + >>>> + # add to results list >>>> + theResults+=3D"${theName} ${theHits} ${theActive} ${theLines} ${hitsPe= rLine} ${theModDate}"$'\n' >>>> +done <<< "${rpzNames}" >>>> + >>>> +case "${sortBy}" in >>>> + names|name) sortArg=3D(-k3,3r -k1,1) ;; # sort by "active" then by "na= me" >>>> + >>>> + hits|hit) sortArg=3D(-k3,3r -k2,2nr -k1,1) ;; # sort by "active" then = by "hits" then by "name" >>>> + >>>> + lines|line) sortArg=3D(-k3,3r -k4,4nr -k1,1) ;; # sort by "active" the= n by "lines" then by "name" >>>> +esac >>>> + >>>> +printf -- "--------------\n" >>>> +# remove blank lines, sort, print as columns >>>> +/bin/echo "${theResults}" | >>>> + /usr/bin/awk '!/^[[:space:]]*$/' | >>>> + /usr/bin/sort "${sortArg[@]}" | >>>> + /usr/bin/awk --assign=3Dwidth=3D"${pWidth}" \ >>>> + '{ printf "%-*s %-8s %-8s %-8s %10s %12s\n", width, $1, $2, $3, $4, $5= , $6 }' >>>> + >>>> +printf "${pFormat}" "" "=3D=3D=3D=3D=3D=3D=3D" "" "=3D=3D=3D=3D=3D=3D= =3D=3D" "" "" >>>> +printf "${pFormat}" "Totals -->" "${totalHits}" "" "${totalLines}" "" "" >>>> + >>>> +exit >>>> diff --git a/config/rpz/rpz-sleep b/config/rpz/rpz-sleep >>>> new file mode 100644 >>>> index 000000000..eeef1174a >>>> --- /dev/null >>>> +++ b/config/rpz/rpz-sleep >>>> @@ -0,0 +1,58 @@ >>>> +#!/bin/bash >>>> +#######################################################################= ######## >>>> +# = # >>>> +# IPFire.org - A linux based firewall = # >>>> +# Copyright (C) 2024 IPFire Team = # >>>> +# = # >>>> +# This program is free software: you can redistribute it and/or modify= # >>>> +# it under the terms of the GNU General Public License as published by= # >>>> +# the Free Software Foundation, either version 3 of the License, or = # >>>> +# (at your option) any later version. = # >>>> +# = # >>>> +# This program is distributed in the hope that it will be useful, = # >>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of = # >>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the = # >>>> +# GNU General Public License for more details. = # >>>> +# = # >>>> +# You should have received a copy of the GNU General Public License = # >>>> +# along with this program. If not, see = . # >>>> +# = # >>>> +#######################################################################= ######## >>>> + >>>> +# v04 on 2024-07-05 >>>> + >>>> +############### Functions ############### >>>> + >>>> +# send message to message log >>>> +msg_log () { >>>> + /usr/bin/logger --tag "${tagName}" "$*" >>>> + if /usr/bin/tty --silent ; then >>>> + echo "${tagName}:" "$*" >>>> + fi >>>> +} >>>> + >>>> +############### Main ############### >>>> + >>>> +tagName=3D"unbound" >>>> + >>>> +sleepTime=3D"${1:-5m}" # default to sleep for 5m (5 minutes) >>>> + >>>> +zoneList=3D$( /usr/sbin/unbound-control list_auth_zones | /usr/bin/awk = '{print $1}' ) >>>> + >>>> +for zone in ${zoneList} ; do >>>> + /usr/bin/printf "disable ${zone}\t" >>>> + /usr/sbin/unbound-control rpz_disable "${zone}" >>>> +done >>>> + >>>> +msg_log "info: rpz: disabled all zones for ${sleepTime}" >>>> + >>>> +/bin/sleep "${sleepTime}" >>>> + >>>> +for zone in ${zoneList} ; do >>>> + /usr/bin/printf "enable ${zone}\t" >>>> + /usr/sbin/unbound-control rpz_enable "${zone}" >>>> +done >>>> + >>>> +msg_log "info: rpz: enabled all zones" >>>> + >>>> +exit >>>> diff --git a/lfs/rpz b/lfs/rpz >>>> new file mode 100644 >>>> index 000000000..319c10b7f >>>> --- /dev/null >>>> +++ b/lfs/rpz >>>> @@ -0,0 +1,88 @@ >>>> +#######################################################################= ######## >>>> +# = # >>>> +# IPFire.org - A linux based firewall = # >>>> +# Copyright (C) 2024 IPFire Team = # >>>> +# = # >>>> +# This program is free software: you can redistribute it and/or modify = # >>>> +# it under the terms of the GNU General Public License as published by = # >>>> +# the Free Software Foundation, either version 3 of the License, or = # >>>> +# (at your option) any later version. = # >>>> +# = # >>>> +# This program is distributed in the hope that it will be useful, = # >>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of = # >>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the = # >>>> +# GNU General Public License for more details. = # >>>> +# = # >>>> +# You should have received a copy of the GNU General Public License = # >>>> +# along with this program. If not, see .= # >>>> +# = # >>>> +#######################################################################= ######## >>>> + >>>> +#######################################################################= ######## >>>> +# Definitions >>>> +#######################################################################= ######## >>>> + >>>> +include Config >>>> + >>>> +SUMMARY =3D response policy zone - RPZ reputation system for unbound= DNS >>>> + >>>> +VER =3D 1.0.0 >>>> + >>>> +THISAPP =3D rpz-$(VER) >>>> +DIR_APP =3D $(DIR_SRC)/$(THISAPP) >>>> +TARGET =3D $(DIR_INFO)/$(THISAPP) >>>> + >>>> +PROG =3D rpz >>>> +PAK_VER =3D 1 >>>> + >>>> +DEPS =3D >>>> + >>>> +SERVICES =3D >>>> + >>>> +#######################################################################= ######## >>>> +# Top-level Rules >>>> +#######################################################################= ######## >>>> + >>>> +install : $(TARGET) >>>> + >>>> +check : >>>> + >>>> +download : >>>> + >>>> +b2 : >>>> + >>>> +dist: >>>> + @$(PAK) >>>> + >>>> +#######################################################################= ######## >>>> +# Installation Details >>>> +#######################################################################= ######## >>>> + >>>> +$(TARGET) : >>>> + @$(PREBUILD) >>>> + @rm -rf $(DIR_APP) >>>> + >>>> + # install RPZ scripts >>>> + install -v -m 755 \ >>>> + $(DIR_CONF)/rpz/{rpz-config,rpz-metrics,rpz-sleep} -t /usr/sbin >>>> + >>>> + # Install settings folder and two empty files >>>> + mkdir -pv /var/ipfire/rpz >>>> + touch /var/ipfire/rpz/allowlist >>>> + touch /var/ipfire/rpz/blocklist >>>> + >>>> + # Add conf file to /etc directory >>>> + cp -vf $(DIR_CONF)/rpz/00-rpz.conf /etc/unbound/local.d >>>> + >>>> + # create zonefiles directory for the RPZ files and add two empty RPZ >>>> + # files to avoid a unbound config error >>>> + mkdir -pv /etc/unbound/zonefiles >>>> + chown -v nobody:nobody /etc/unbound/zonefiles >>>> + touch /etc/unbound/zonefiles/allow.rpz >>>> + touch /etc/unbound/zonefiles/block.rpz >>>> + >>>> + # Install backup definition >>>> + cp -vf $(DIR_CONF)/backup/includes/rpz /var/ipfire/backup/addons/inclu= des/rpz >>>> + >>>> + @rm -rf $(DIR_APP) >>>> + @$(POSTBUILD) >>>> diff --git a/make.sh b/make.sh >>>> index 9bbbeb0f1..886d3760a 100755 >>>> --- a/make.sh >>>> +++ b/make.sh >>>> @@ -1721,6 +1721,8 @@ buildipfire() { >>>> lfsmake2 btrfs-progs >>>> lfsmake2 inotify-tools >>>> lfsmake2 grub-btrfs >>>> + lfsmake2 rpz >>>> + >>>>=20 >>>> # Kernelbuild ... current we have no platform that need >>>> # multi kernel builds so KCFG is empty >>>> --=20 >>>> 2.30.2 >>>>=20 >>>=20 >>=20 >=20 --===============4678702258645145430==--