From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4ZHxrc2Ty3z32pC for ; Wed, 19 Mar 2025 18:22:24 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail01.haj.ipfire.org", Issuer "R10" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4ZHxrX1tMTz32p7 for ; Wed, 19 Mar 2025 18:22:20 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail01.ipfire.org (Postfix) with ESMTPSA id 4ZHxrV4dZ1z5cJ for ; Wed, 19 Mar 2025 18:22:18 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1742408539; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z90tXjimmMUgGa3cRDL2YHe+7MulYItISukGS93+hcM=; b=twviMW8WiK51Nc4nFBe86nydqeHj1hnc6UT6ka8MkbhzVIvCtQpGfyst1VL3WPnPuwyQy4 oX1g8SaDb27BP+Dw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1742408539; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z90tXjimmMUgGa3cRDL2YHe+7MulYItISukGS93+hcM=; b=VPYhYa2KB2vNuThZMsPnoqFCUUxztII2RHc9PaQtPjQnNk5VzrWbkztXIeiTmD+hZmwCZx uDjONsZzioutD0m5BlZH5JioO4OWGT7we3n9NBAZQMnuUzxYy2uXjFN+GKSKrnvz4I5a5Y EQlsHyMH6+FxJ+gedgC/L5ilcx02Vxg6jX74J7E9+7ko5bXKLgbcxqvI0ejprsLNeoBQGu XkWb2BCvof7HjAIJzot6biIuFXdWqutSktqqe9x9/5yzZQcedt8WcyYwkb5ni7sRyH98VK d/A9KJEEPaYOjNwzIHjVuu9F+1LgTPeTdiIgsjgu0PXNazd6ujwVunQjw6Da+A== Message-ID: <666c6e88-02ae-4e69-b5b8-393e78a8fb8a@ipfire.org> Date: Wed, 19 Mar 2025 13:22:14 -0500 Precedence: list List-Id: List-Subscribe: , List-Unsubscribe: , List-Post: List-Help: Sender: Mail-Followup-To: MIME-Version: 1.0 Subject: Re: [PATCH] RPZ: update code to include WEBGUI and additional languages To: development@lists.ipfire.org References: <20250206163522.2363178-1-jon.murphy@ipfire.org> <8b594873-86ca-46b9-bb4b-94fd6b0239b1@ipfire.org> <9A0DBDA4-75B0-40D2-AE06-78D9BA5EE7D3@ipfire.org> Content-Language: en-US From: Jon Murphy In-Reply-To: <9A0DBDA4-75B0-40D2-AE06-78D9BA5EE7D3@ipfire.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Michael, > Where in the code is this implemented? I cannot find anything like this: Keep in mind I am not a "C" person.  Maybe in this section?: https://git.ipfire.org/?p=thirdparty/unbound.git;a=blob;f=services/authzone.c;hb=30b9cb5f813003d0a2b1c2e678652396615b1b7d#l5875 — When I was just learning about RPZ I created a separate RPZ files for testing.  When I changed the SOA line with a new serial number, the RPZ file download would happen in about 5 minutes. https://people.ipfire.org/~jon/sblack-adhoc.rpz That is how I found out the SOA line is watched for a serial number change. I’ll reconfirm my finding. >>> The second reason is that we have a lot of firewalls out there. Not >>> all of them will enable this feature and all of the lists, but even >>> if it is a good chunk, we will generate terabytes of traffic which >>> put load on the infrastructure and will cost money. It simply is not >>> what we want to do, regardless of self-hosting those lists and >>> pulling them from somewhere else. So I understand, are you thinking of hosting RPZ AXFR (DNS zone transfer) on IPFire infrastructure? Jon On 3/19/25 5:35 AM, Michael Tremer wrote: > Hello Jon, > > Where in the code is this implemented? I cannot find anything like this: > > Unbound loads the entire file into memory and then starts parsing it. The only special treatment there is is to check whether the first line is a valid zone entry. It does not even have to be a SOA record. > > https://git.ipfire.org/?p=thirdparty/unbound.git;a=blob;f=services/authzone.c;hb=30b9cb5f813003d0a2b1c2e678652396615b1b7d#l1188 > > I am also concerned that Unbound will not be able to support an upstream proxy for any downloads. The caching situation is also unclear for me, so I believe that we will be looking at writing a custom downloader that implements all these things. > > -Michael > >> On 19 Mar 2025, at 02:58, Jon Murphy wrote: >> >> Michael, >> >>> The emphasis is on the repeated downloads of the same list. That is >> ​> what cannot happen. >> >> The Unbound RPZ code, as installed within IPFire, watches for a change >> ​in the SOA line of each RPZ file. This is an example of the first few >> ​lines for every RPZ file. >> >> $TTL 300 >> @ SOA localhost. root.localhost. 1742298960 43200 3600 86400 300 >> NS localhost. >> ; >> ; Title: HaGeZi's Pop-Up Ads DNS Blocklist >> ; Description: Blocks annoying and malicious pop-up ads. >> >> If the SOA serial number changes (e.g. the 1742298960), then Unbound RPZ >> ​code does its thing and downloads. Otherwise there is no download. >> >>> So there has to be a way to ensure that we won’t download a list again >> ​> unless it has actually changed. >> >> This should do what you want but I may be missing your point. >> >>> DNS has a builtin functionality called AXFR. It simply does the job >> ​> for you. I was just wondering whether that was not being used. >> >> I need to read about AXFR/IXFR and learn a little more. >> >> Jon >> >> On 3/17/25 5:35 AM, Michael Tremer wrote: >>> Good Morning Jon, >>> >>>> On 16 Mar 2025, at 17:00, Jon Murphy wrote: >>>> >>>> Michael, >>>> >>>> I was reading through you response again an I want to understand this post: >>>> >>>>> I have also stated that we cannot download any lists over HTTPS again and again and again. The implementation that we have here seems to exactly do that and therefore I think that my feedback has been dismissed entirely. >>>> So if RPZ doesn't use HTTPS, what is it using? I am missing a key point here. >>> The emphasis is on the repeated downloads of the same list. That is what cannot happen. >>> >>> Although it might not affect a lot of people in our general user-base, there are some that have a metered connection and will pay for data by volume. Some of the lists I looked at are just under 20 MiB. Therefore we need to keep any traffic down to a minimum. The second reason is that we have a lot of firewalls out there. Not all of them will enable this feature and all of the lists, but even if it is a good chunk, we will generate terabytes of traffic which put load on the infrastructure and will cost money. It simply is not what we want to do, regardless of self-hosting those lists and pulling them from somewhere else. >>> >>> So there has to be a way to ensure that we won’t download a list again unless it has actually changed. >>> >>> DNS has a builtin functionality called AXFR. It simply does the job for you. I was just wondering whether that was not being used. >>> >>> HTTPS is an option because that is simply what we use elsewhere, but extra functionality will have to be built for it. >>> >>> -Michael >>> >>>> Jon >>>> >>>> >>>> On 2/13/25 3:34 PM, jon wrote: >>>>> Michael, >>>>> >>>>> I’ve read through your comments a few times and I ended up with many more questions. >>>>> >>>>> >>>>>> What I rather mean is that it has never been added as a topic on the agenda and it has not been pitched by yourself. >>>>> To me the efforts to get new code accepted seem to have changed and it seemed easier in the past. In the past I made the Core Team aware via the Dev Mailing List and wrote a simple two or three paragraphs of "What is it? / What is the value? / Here is the code" >>>>> >>>>> >>>>> So in an effort to move forward: How exactly is something presented to the Core Team? >>>>> >>>>> Is there an example of a recent effort that was presented that I can see as a sample? (This type of info can also be added to the Wiki) >>>>> >>>>> I understand you want it this way, but I don’t know what exactly is needed. Please be specific. >>>>> >>>>> >>>>> Jon >>>>> >>>>> PS - I am not ignoring your other comments, I am just trying to move forward and keep things simple. >>>>> >>>>> >>>>> >>>>>> On Feb 8, 2025, at 1:27 PM, Michael Tremer wrote: >>>>>> >>>>>> Hello Jon, >>>>>> >>>>>> Thanks for your reply. And good that you are copying everyone into this conversation. >>>>>> >>>>>>> On 8 Feb 2025, at 18:41, jon wrote: >>>>>>> >>>>>>> Michael, >>>>>>> >>>>>>>> I think I have covered this all at lengths before that this project has been started as a separate effort >>>>>>> Yes, this has been a separate effort (a very public separate effort). Yes, as you pointed this out early on with the "proof-of-concept" and then my request for people to help test RPZ. Nothing was hidden. >>>>>>> >>>>>>> This was done because you (and maybe others) did not have the time and I wanted to help and because I needed assistance with RPZ. I tried my best to do this without bothering you. >>>>>> I don’t that it is accurate that nobody wanted to help on this. The list was always open - although not every email has been replied to swiftly it is also your responsibility to raise a question again if it was missed. People here have open ears. >>>>>> >>>>>> It was also stated on this very list on in our documentation that working on something without involving the core team is a risky undertaking. Of course IPFire is free software and so everyone is free to fork if they wish to do so. >>>>>> >>>>>>>> and as far as I am aware none of the other team members has been involved. This has not been discussed either on this list, on our calls. >>>>>>> You were aware many steps along the way. See your email on July 28, 2024, August 15, 2024, September 30, 2024, December 23, 2024, and January 16. My attempts to get the team involved were met with "things are busy" and sometimes silence. (Yes, I get it, people are busy.) >>>>>>> >>>>>>> You and Adolf, Leo, Erik and Bernhard have been aware since the beginning. You mention you were aware of the "proof-of-concept". If you include those beginning posts, since Sep 2023. >>>>>> Yes, I am aware of a proof-of-concept that I have been running myself for a long time. I am also aware of the efforts that you have been taking. >>>>>> >>>>>> Yet I don’t think there has ever been any joint effort, or am I seeing that wrong? >>>>>> >>>>>>>> This has not been discussed . . . on our calls. >>>>>>> On the July 28th you stated: >>>>>>> "We have talked about RPZ many times on the monthly call since the URL filter feature is falling more and more out of fashion. I think there is also many posts about this on the forum." >>>>>>> >>>>>>> Please don’t insult me again by stating "you know what I mean". >>>>>>> >>>>>>> And it has been discussed but not documented in the Monthly Meeting notes. >>>>>> I am not at all insulting you. I don’t want to take this down to a personal level at all. This is a public mailing list and people who read this don’t need to listen to an argument we are having. They are here for the tech inside IPFire. >>>>>> >>>>>> When I wrote that it has not been discussed that does not mean that we have not been touching on the topic. We have been talking about lots of things on the calls, the weather, politics, how our pets are. None of that makes it to the logs. What I rather mean is that it has never been added as a topic on the agenda and it has not been pitched by yourself. >>>>>> >>>>>>>> Instead there has been a separate conversation on the forum with the occasional dip here to the list. But that was not a regular two-way conversation. >>>>>>> Regular conversation on the Dev Mailing list is many times met with silence. I get it, people are busy. >>>>>>> >>>>>>> And regular two-way conversation doesn’t happen on the list. At least not with me. I’d be happy to point out the posts that were met with silence. >>>>>>> Again, I get it, people are busy. >>>>>> And you think my emails are not being met with silence? This has nothing to do with this specific topic. This has something to do with how occupied people are and how engaged they are on certain topics. Not everyone is involved in all the things and simply will ignore emails simply based on their subject line. >>>>>> >>>>>>> But the "dip here to the list" were my attempts to get a conversation started. As I said, many time met with silence. >>>>>>> >>>>>>> The only place I was not met with silence was on the Community. You have a great group of people in the Community. It is a shame you don’t want to have others help. It would reduce your workload. >>>>>> You should stop making statements that are not true. Who doesn’t want anyone to help? >>>>>> >>>>>> Not having this conversation on a Saturday evening would reduce my workload. At least it would free up time for something else. Helping with the things that are already on the go would reduce the workload of the entire team. Starting one thing at a time and finishing it is a lot better to manage than starting a hundred things and not even finish one. I can tell you that I already have a hundred things on the go. >>>>>> >>>>>>>> Therefore, what am I supposed to do with this email? >>>>>>> To me it is beyond obvious… >>>>>>> >>>>>>> If it isn’t what you want, then guide me with how to do this the correct way. And be specific. I am trying to help. I am trying to make things better. I am trying to do things the right way. >>>>>> To me it isn’t. This is yet another project that has been dumped to the list like so many before and later on everyone has left to have the team deal with the rest. >>>>>> >>>>>> It is a huge patch set. You explained what the vision is, but that is about it. There is no chance this will continue if this disagreement isn’t solved first. I didn’t even look at the code. >>>>>> >>>>>>>> I don’t want to merge code that I don’t agree with. >>>>>>> I asked multiple times if you "agreed with the concept" and again, met with silence. Yes I get it, people are busy. >>>>>> Having support for RPZ? Yes, it was definitely on the roadmap. That I agree with. >>>>>> >>>>>>>> So many fundamental things that I have been raising have either not been discussed or outright dismissed. >>>>>>> You mentioned this a in the past, but for some reason you do not disclose what I dismissed. Why do you continue to make this harder, wouldn’t it not be easier to tell me what I have dismissed? >>>>>>> >>>>>>> I have sent multiple emails trying to answer your concerns and comments. On July 28, Aug 14, Aug 22, Aug 23, Sep 30, etc. >>>>>>> >>>>>>> I’ve gone through all of the questions you asked and I cannot find a "dismissed" item. >>>>>> Maybe I need to be *more clear*. I feel humoured by this. >>>>>> >>>>>> It is late on a Saturday and I want my dinner soon, but certainly I have stated that this should never be an add-on considering it is supposed to replace URL Filter. We should never allow people to add their own sources. I have also stated that we cannot download any lists over HTTPS again and again and again. The implementation that we have here seems to exactly do that and therefore I think that my feedback has been dismissed entirely. >>>>>> >>>>>>>> I don’t want to merge code that has no future inside IPFire as there is no constructive conversation with the maintainers of it. >>>>>>> The maintainers of Unbound and/or RPZ? >>>>>>> >>>>>>> The maintainers of Hagezi list, the threatfox list, the urlhaus list, etc.? >>>>>>> >>>>>>> What else? The maintainers or the RPZ scripts? That is me. Let’s talk! >>>>>> You. I don’t care much about the providers of the lists. >>>>>> >>>>>>> See, this is where it gets confusing. There are hundreds of open source packages as part of IPFire. Pick the last five years of items added to the IPFire build. You're telling me you have "constructive conversation with the maintainers" of all of the added packages? >>>>>> They publish their software and they don’t care whether I am pulling it or not. They publish it with the commitment to maintain it - sometimes for better and sometimes for worse. >>>>>> >>>>>> You care about me pulling your code and I don’t know whether you would commit to maintain this. >>>>>> >>>>>> These two are very different cases. >>>>>> >>>>>>> Pick the IP Blocklists list (i.e., 3CORESEC, ABUSECH, DSHIELD, SPAMHAUS, etc.) or the Suricata lists (i.e.,Emergingthreats.net ,Abuse.ch , etc.). So you’ve have "constructive conversation with the maintainers"? >>>>>> Yes, occasionally I have phone calls with a few of these providers. >>>>>> >>>>>>>> Having been trying for a long time to make you aware of this, nothing of this should come as a surprise. >>>>>>> Ha! Yes a surprise. In the beginning you seemed interested as IPFire needed a replacement for URL Filter. You asked good questions about the lists picked, asked for the value to the users, etc. And I answered the best I could. >>>>>>> >>>>>>> You even asked: “Why is this realised as an add-on and not part of the core system?” from your Jul 28, 2024 email. >>>>>> Ah, so, why is the patch creating an add-on? Not that I am saying that what I say is law, but it has not been challenged either. If my input is being ignored, why should I put this to the top of my list of priorities? I am not disappointed about this, just trying to be very good with my time. >>>>>> >>>>>>> And on January 16, 2025 I wrote a message looking for help. And you were kind to respond quickly. So in three weeks time, since the kind response, something has changed. You went from supportive to "this". >>>>>>> >>>>>>> So yes, I am surprised. >>>>>> Well, maybe I should not have replied to that email. It was clear that you were on some path that was not right, but you were not interested before in finding the right path from the beginning. >>>>>> >>>>>>>> Please consider if that can be changed and if there is a path forward with this. >>>>>>> Be more specific, what has to change? What exactly did I dismiss? >>>>>> Dismissal is just my assumption. I don’t know what you actually did with my feedback. I can only see the end product that does not seem contain much of it. Repeatedly I have been pointing out that we should think before we build. I am sure a lot of hours have now gone into some code that simply does not satisfy me. And I am not not talking about the code itself, what it does is what I don’t think is right for us. >>>>>> >>>>>> The process is very clear for me that we should first of all think whether we want a certain feature now. Then there should be a clear roadmap for everyone to follow; tasks can be split-up as we go and hopefully then have something that is maintainable, interesting for our users and even would do us proud. This is how this should work. >>>>>> >>>>>> So, what has to change? I don’t think with shouting at each other, throwing patches around and making me generally unhappy is a good start. >>>>>> >>>>>> -Michael >>>>>> >>>>>>> Jon >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On Feb 6, 2025, at 2:13 PM, Michael Tremer wrote: >>>>>>>> >>>>>>>> Hello Jon, >>>>>>>> >>>>>>>> Well, here we are again with another patch regarding this feature. >>>>>>>> >>>>>>>> I cannot quite see from your email what the question is, but if this is a request to have this merged into IPFire, I am once again sorry to disappoint you. >>>>>>>> >>>>>>>> I think I have covered this all at lengths before that this project has been started as a separate effort and as far as I am aware none of the other team members has been involved. This has not been discussed either on this list, on our calls. Instead there has been a separate conversation on the forum with the occasional dip here to the list. But that was not a regular two-way conversation. Therefore, what am I supposed to do with this email? >>>>>>>> >>>>>>>> I don’t want to merge code that I don’t agree with. So many fundamental things that I have been raising have either not been discussed or outright dismissed. >>>>>>>> >>>>>>>> I don’t want to merge code that has no future inside IPFire as there is no constructive conversation with the maintainers of it. >>>>>>>> >>>>>>>> Having been trying for a long time to make you aware of this, nothing of this should come as a surprise. >>>>>>>> >>>>>>>> Please consider if that can be changed and if there is a path forward with this. >>>>>>>> >>>>>>>> All the best, >>>>>>>> -Michael >>>>>>>> >>>>>>>>> On 6 Feb 2025, at 16:35, Jon Murphy wrote: >>>>>>>>> >>>>>>>>> What is it? >>>>>>>>> Response Policy Zone (RPZ) is a mechanism to define local policies in a >>>>>>>>> standardized way and load those policies from external sources. >>>>>>>>> Bottom line: RPZ allows admins to easily block access to websites via DNS lookup. >>>>>>>>> >>>>>>>>> RPZ can block websites via categories. Examples include: fake websites, annoying >>>>>>>>> pop-up ads, newly registered domains, DoH bypass sites, bad "host" services, >>>>>>>>> maliscious top level domains (e.g., *.zip, *.mov), piracy, gambling, pornography, >>>>>>>>> and more. RPZ lists come from various RPZ providers and their available >>>>>>>>> catagories. >>>>>>>>> >>>>>>>>> This RPZ add-on enables the RPZ functionality by adding a couple lines in a >>>>>>>>> configuration file. This add-on simply adds configuration files and adds >>>>>>>>> scripts (config, metrics and sleep) to make RPZ easier for the admin to use. >>>>>>>>> >>>>>>>>> The RPZ scripts include additional languages: German, Spanish, French, Turkish, >>>>>>>>> and Italian. >>>>>>>>> >>>>>>>>> RPZ itself was release in 2010 and has been part of the IPFire build since ~2015. >>>>>>>>> >>>>>>>>> Why is it needed? What is its value? >>>>>>>>> >>>>>>>>> - The RPZ concept places this filtering into IPFire, our internet access >>>>>>>>> gateway, which is (should be) solely used as DNS source of the internal network. >>>>>>>>> >>>>>>>>> - As most sites use HTTPS it makes it difficult to filter traffic with URL >>>>>>>>> Filter without also properly configuring conventional (non-transparent) >>>>>>>>> mode on the proxy. RPZ is a nice replacement for the URL Filter. >>>>>>>>> >>>>>>>>> - No need to install and maintain an additional device like PiHole or AdBlock >>>>>>>>> browser extensions on multiple user devices. >>>>>>>>> >>>>>>>>> - This is an additional layer of protection for users. Less worry someone will >>>>>>>>> click on something that gets them into trouble. And, saying this with emphasis, >>>>>>>>> the ability to do it in one place! >>>>>>>>> >>>>>>>>> - Blocked sites save on unneeded traffic and can lessen the threat of malware >>>>>>>>> in advertisements >>>>>>>>> >>>>>>>>> - Logging allows the admin to see the site blocked and take actions >>>>>>>>> >>>>>>>>> - RPZ will be used at the home, home-office (work from home), schools, >>>>>>>>> ministerial, and at the office. Device counts are small (2-6) to medium (~80) >>>>>>>>> to mediam-large (200+). >>>>>>>>> >>>>>>>>> - RPZ can block ads, popups, phishing, scammers, spyware, malware, annoying >>>>>>>>> popups, NSFW links, DOH servers, and the usual internet trash. >>>>>>>>> >>>>>>>>> ------------------------------ >>>>>>>>> >>>>>>>>> Change Log for RPZ add-on >>>>>>>>> >>>>>>>>> rpz-1.0.0-18 on 2025-02-05 >>>>>>>>> - Build for approval & release as IPFire add-on >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> rpz-beta-0.1.18-18.ipfire on 2025-02-01 >>>>>>>>> rpz.cgi: >>>>>>>>> - new feature: added a mod key to force a unbound restart >>>>>>>>> >>>>>>>>> rpz-config and rpz-make: >>>>>>>>> - new feature: added action for unbound restart `rpz-config unbound-restart` >>>>>>>>> >>>>>>>>> rpz-metrics: >>>>>>>>> - simple reformatting >>>>>>>>> - rename far right column from "last update" to "last download" >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> rpz-beta-0.1.17-17.ipfire on 2024-12-09 >>>>>>>>> rpz-make >>>>>>>>> - bug fix: corrected validation regex for wildcards like: `*.domain.com` >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> rpz-beta-0.1.16-16.ipfire on 2024-11-18 >>>>>>>>> rpz-make >>>>>>>>> - new feature: updated validation regex >>>>>>>>> - bug fix: moved validation to beginning of process. Now we validate before >>>>>>>>> creating config files. >>>>>>>>> >>>>>>>>> rpz.cgi: >>>>>>>>> - new feature: use CSS color variables of the main ipfire theme >>>>>>>>> - bug fix: empty zonefile remarks were stored as “undef” and caused a warning >>>>>>>>> - bug fix: HTML textarea removes the first empty line in a custom list >>>>>>>>> - thank you Leo! >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> rpz-beta-0.1.15-15.ipfire on 2024-11-04 >>>>>>>>> rpz.cgi: >>>>>>>>> - new feature: added new language file for Turkish (thank you Peppe) >>>>>>>>> >>>>>>>>> rpz-make >>>>>>>>> - bug fix: corrected empty allow/block list issue. An empty allow/block list >>>>>>>>> will now remove contents of allow/block.rpz files and remove unneeded >>>>>>>>> allow/block.conf file. (thank you iptom) >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> rpz-beta-0.1.14-14.ipfire on 2024-10-29 >>>>>>>>> rpz-config: >>>>>>>>> - bug fix: correct missing rpz extension. `rpz-config list` displayed URL >>>>>>>>> incorrectly (thank you Bernhard) >>>>>>>>> >>>>>>>>> rpz.cgi: >>>>>>>>> - bug fix: remove extra `"` in language files (thank you Bernhard) >>>>>>>>> - new feature: slightly dim "apply" button when not enabled >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> rpz-beta-0.1.13-13.ipfire on 2024-10-27 >>>>>>>>> - skipped >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> rpz-beta-0.1.12-12.ipfire on 2024-10-21 >>>>>>>>> rpz.cgi: >>>>>>>>> - new feature: added new language file for French (thank you gw-ipfire) >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> rpz-beta-0.1.11-11.ipfire on 2024-10-18 >>>>>>>>> rpz.cgi: >>>>>>>>> - new feature: added new language file for Italian (thank you umberto) >>>>>>>>> - new feature: added new language file for Spanish (thank you Roberto) >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> rpz-beta-0.1.10-10.ipfire on 2024-10-15 >>>>>>>>> rpz-make: >>>>>>>>> - bug fix: corrected validation error for a custom list entry (thank you siosios) >>>>>>>>> - e.g., `*.cloudflare-dns.com` >>>>>>>>> >>>>>>>>> install.sh: >>>>>>>>> - bug fix: add chown to correct user created files >>>>>>>>> >>>>>>>>> update.sh: >>>>>>>>> - bug fix: add chown to correct user created files (thank you siosios) >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> rpz-beta-0.1.9-9.ipfire on 2024-10-08 >>>>>>>>> rpz.cgi: >>>>>>>>> - new feature: added new language file for German (thank you Leo) >>>>>>>>> - bug fix: add missing "rpz exitcode 110" >>>>>>>>> - bug fix: corrected missing RPZ menu item at menu > IPFire >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> rpz-beta-0.1.8-8.ipfire on 2024-10-04 >>>>>>>>> - skipped >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> rpz-beta-0.1.7-7.ipfire on 2024-10-03 >>>>>>>>> All: >>>>>>>>> - new feature: includes beta version numbers for pakfire package, >>>>>>>>> instead of only `rpz-1.0.0-1.ipfire`, for each release. >>>>>>>>> >>>>>>>>> rpz.cgi: >>>>>>>>> - new feature: added new WebGUI at `rpz.cgi` >>>>>>>>> - a BIG thank you to Leo Hofmann for all of his work creating the webgui!! >>>>>>>>> - bug fix: corrected missing RPZ menu item at menu > IPFire >>>>>>>>> >>>>>>>>> rpz-make: >>>>>>>>> - new feature: validate entries in allowlist and blocklist >>>>>>>>> - new feature: add "no-reload" option for WebGUI >>>>>>>>> >>>>>>>>> rpz-metrics: >>>>>>>>> - new feature: info can be sorted by name, by hit count, by line count, by >>>>>>>>> "enabled" list or all lists >>>>>>>>> >>>>>>>>> backups: >>>>>>>>> - bug fix: include all files in `/var/ipfire/dns/rpz` directory in backup >>>>>>>>> >>>>>>>>> update.sh: >>>>>>>>> - bug fix: corrected ownership for `/var/ipfire/dns/rpz` directory during an >>>>>>>>> update >>>>>>>>> >>>>>>>>> Build: >>>>>>>>> - bug fix: `block.rpz.conf` and `block.rpz` from build. Files to be created >>>>>>>>> by `rpz-make` >>>>>>>>> >>>>>>>>> WebGUI and German language file >>>>>>>>> Contribution-by: Leo-Andres Hofmann >>>>>>>>> >>>>>>>>> Spanish language file >>>>>>>>> Contribution-by: Roberto Peña >>>>>>>>> >>>>>>>>> Italian language file >>>>>>>>> Contribution-by: Umberto Parma >>>>>>>>> >>>>>>>>> French language file >>>>>>>>> Contribution-by: gw-ipfire >>>>>>>>> >>>>>>>>> Turkish language file >>>>>>>>> Contribution-by: Peppe Tech >>>>>>>>> >>>>>>>>> Contribution-by: Bernhard Bitsch >>>>>>>>> Contribution-by: Erik Kapfer >>>>>>>>> Signed-off-by: Jon Murphy >>>>>>>> --- >>>>>>>>> config/backup/includes/rpz | 4 + >>>>>>>>> config/cfgroot/manualpages | 1 + >>>>>>>>> config/menu/EX-rpz.menu | 6 + >>>>>>>>> config/rootfiles/common/configroot | 1 + >>>>>>>>> config/rootfiles/common/web-user-interface | 1 + >>>>>>>>> config/rootfiles/packages/rpz | 20 + >>>>>>>>> config/rpz/00-rpz.conf | 10 + >>>>>>>>> config/rpz/rpz-config | 130 +++ >>>>>>>>> config/rpz/rpz-functions | 85 ++ >>>>>>>>> config/rpz/rpz-make | 203 +++++ >>>>>>>>> config/rpz/rpz-metrics | 170 ++++ >>>>>>>>> config/rpz/rpz-sleep | 58 ++ >>>>>>>>> config/rpz/rpz.de.pl | 30 + >>>>>>>>> config/rpz/rpz.en.pl | 30 + >>>>>>>>> config/rpz/rpz.es.pl | 30 + >>>>>>>>> config/rpz/rpz.fr.pl | 30 + >>>>>>>>> config/rpz/rpz.it.pl | 30 + >>>>>>>>> config/rpz/rpz.tr.pl | 30 + >>>>>>>>> html/cgi-bin/rpz.cgi | 923 +++++++++++++++++++++ >>>>>>>>> lfs/rpz | 96 +++ >>>>>>>>> make.sh | 3 +- >>>>>>>>> src/paks/rpz/install.sh | 36 + >>>>>>>>> src/paks/rpz/uninstall.sh | 38 + >>>>>>>>> src/paks/rpz/update.sh | 52 ++ >>>>>>>>> 24 files changed, 2016 insertions(+), 1 deletion(-) >>>>>>>>> create mode 100644 config/backup/includes/rpz >>>>>>>>> create mode 100644 config/menu/EX-rpz.menu >>>>>>>>> create mode 100644 config/rootfiles/packages/rpz >>>>>>>>> create mode 100644 config/rpz/00-rpz.conf >>>>>>>>> create mode 100644 config/rpz/rpz-config >>>>>>>>> create mode 100644 config/rpz/rpz-functions >>>>>>>>> create mode 100644 config/rpz/rpz-make >>>>>>>>> create mode 100755 config/rpz/rpz-metrics >>>>>>>>> create mode 100755 config/rpz/rpz-sleep >>>>>>>>> create mode 100644 config/rpz/rpz.de.pl >>>>>>>>> create mode 100644 config/rpz/rpz.en.pl >>>>>>>>> create mode 100644 config/rpz/rpz.es.pl >>>>>>>>> create mode 100644 config/rpz/rpz.fr.pl >>>>>>>>> create mode 100644 config/rpz/rpz.it.pl >>>>>>>>> create mode 100644 config/rpz/rpz.tr.pl >>>>>>>>> create mode 100644 html/cgi-bin/rpz.cgi >>>>>>>>> create mode 100644 lfs/rpz >>>>>>>>> create mode 100644 src/paks/rpz/install.sh >>>>>>>>> create mode 100644 src/paks/rpz/uninstall.sh >>>>>>>>> create mode 100644 src/paks/rpz/update.sh >>>>>>>>> >>>>>>>>> diff --git a/config/backup/includes/rpz b/config/backup/includes/rpz >>>>>>>>> new file mode 100644 >>>>>>>>> index 000000000..36513e494 >>>>>>>>> --- /dev/null >>>>>>>>> +++ b/config/backup/includes/rpz >>>>>>>>> @@ -0,0 +1,4 @@ >>>>>>>>> +/var/ipfire/dns/rpz/* >>>>>>>>> +/etc/unbound/zonefiles/allow.rpz >>>>>>>>> +/etc/unbound/zonefiles/block.rpz >>>>>>>>> +/etc/unbound/local.d/*rpz.conf >>>>>>>>> diff --git a/config/cfgroot/manualpages b/config/cfgroot/manualpages >>>>>>>>> index 1f7e01efc..d3a48c633 100644 >>>>>>>>> --- a/config/cfgroot/manualpages >>>>>>>>> +++ b/config/cfgroot/manualpages >>>>>>>>> @@ -70,6 +70,7 @@ pakfire.cgi=configuration/ipfire/pakfire >>>>>>>>> wlanap.cgi=addons/wireless >>>>>>>>> tor.cgi=addons/tor >>>>>>>>> samba.cgi=addons/samba >>>>>>>>> +rpz.cgi=addons/rpz >>>>>>>>> >>>>>>>>> # Logs menu >>>>>>>>> logs.cgi/summary.dat=configuration/logs/summary >>>>>>>>> diff --git a/config/menu/EX-rpz.menu b/config/menu/EX-rpz.menu >>>>>>>>> new file mode 100644 >>>>>>>>> index 000000000..2f4daf410 >>>>>>>>> --- /dev/null >>>>>>>>> +++ b/config/menu/EX-rpz.menu >>>>>>>>> @@ -0,0 +1,6 @@ >>>>>>>>> +$subipfire->{'20.rpz'} = { >>>>>>>>> + 'caption' => $Lang::tr{'rpz'}, >>>>>>>>> + 'uri' => '/cgi-bin/rpz.cgi', >>>>>>>>> + 'title' => "RPZ", >>>>>>>>> + 'enabled' => 1, >>>>>>>>> +}; >>>>>>>>> diff --git a/config/rootfiles/common/configroot b/config/rootfiles/common/configroot >>>>>>>>> index 9839eee45..b30d6aae4 100644 >>>>>>>>> --- a/config/rootfiles/common/configroot >>>>>>>>> +++ b/config/rootfiles/common/configroot >>>>>>>>> @@ -120,6 +120,7 @@ var/ipfire/menu.d/70-log.menu >>>>>>>>> #var/ipfire/menu.d/EX-apcupsd.menu >>>>>>>>> #var/ipfire/menu.d/EX-guardian.menu >>>>>>>>> #var/ipfire/menu.d/EX-mympd.menu >>>>>>>>> +#var/ipfire/menu.d/EX-rpz.menu >>>>>>>>> #var/ipfire/menu.d/EX-samba.menu >>>>>>>>> #var/ipfire/menu.d/EX-tor.menu >>>>>>>>> #var/ipfire/menu.d/EX-transmission.menu >>>>>>>>> diff --git a/config/rootfiles/common/web-user-interface b/config/rootfiles/common/web-user-interface >>>>>>>>> index 816241dae..e00464076 100644 >>>>>>>>> --- a/config/rootfiles/common/web-user-interface >>>>>>>>> +++ b/config/rootfiles/common/web-user-interface >>>>>>>>> @@ -69,6 +69,7 @@ srv/web/ipfire/cgi-bin/proxy.cgi >>>>>>>>> srv/web/ipfire/cgi-bin/qos.cgi >>>>>>>>> srv/web/ipfire/cgi-bin/remote.cgi >>>>>>>>> srv/web/ipfire/cgi-bin/routing.cgi >>>>>>>>> +#srv/web/ipfire/cgi-bin/rpz.cgi >>>>>>>>> #srv/web/ipfire/cgi-bin/samba.cgi >>>>>>>>> srv/web/ipfire/cgi-bin/services.cgi >>>>>>>>> srv/web/ipfire/cgi-bin/shutdown.cgi >>>>>>>>> diff --git a/config/rootfiles/packages/rpz b/config/rootfiles/packages/rpz >>>>>>>>> new file mode 100644 >>>>>>>>> index 000000000..1c8663049 >>>>>>>>> --- /dev/null >>>>>>>>> +++ b/config/rootfiles/packages/rpz >>>>>>>>> @@ -0,0 +1,20 @@ >>>>>>>>> +etc/unbound/local.d/00-rpz.conf >>>>>>>>> +etc/unbound/zonefiles >>>>>>>>> +etc/unbound/zonefiles/allow.rpz >>>>>>>>> +usr/sbin/rpz-config >>>>>>>>> +usr/sbin/rpz-functions >>>>>>>>> +usr/sbin/rpz-make >>>>>>>>> +usr/sbin/rpz-metrics >>>>>>>>> +usr/sbin/rpz-sleep >>>>>>>>> +var/ipfire/addon-lang/rpz.de.pl >>>>>>>>> +var/ipfire/addon-lang/rpz.en.pl >>>>>>>>> +var/ipfire/addon-lang/rpz.es.pl >>>>>>>>> +var/ipfire/addon-lang/rpz.fr.pl >>>>>>>>> +var/ipfire/addon-lang/rpz.it.pl >>>>>>>>> +var/ipfire/addon-lang/rpz.tr.pl >>>>>>>>> +var/ipfire/backup/addons/includes/rpz >>>>>>>>> +var/ipfire/dns/rpz >>>>>>>>> +var/ipfire/dns/rpz/allowlist >>>>>>>>> +var/ipfire/dns/rpz/blocklist >>>>>>>>> +var/ipfire/menu.d/EX-rpz.menu >>>>>>>>> +srv/web/ipfire/cgi-bin/rpz.cgi >>>>>>>>> diff --git a/config/rpz/00-rpz.conf b/config/rpz/00-rpz.conf >>>>>>>>> new file mode 100644 >>>>>>>>> index 000000000..f005a4f2e >>>>>>>>> --- /dev/null >>>>>>>>> +++ b/config/rpz/00-rpz.conf >>>>>>>>> @@ -0,0 +1,10 @@ >>>>>>>>> +server: >>>>>>>>> + module-config: "respip validator iterator" >>>>>>>>> + >>>>>>>>> +rpz: >>>>>>>>> + name: allow.rpz >>>>>>>>> + zonefile: /etc/unbound/zonefiles/allow.rpz >>>>>>>>> + rpz-action-override: passthru >>>>>>>>> + rpz-log: yes >>>>>>>>> + rpz-log-name: allow >>>>>>>>> + rpz-signal-nxdomain-ra: yes >>>>>>>>> diff --git a/config/rpz/rpz-config b/config/rpz/rpz-config >>>>>>>>> new file mode 100644 >>>>>>>>> index 000000000..c72d50f9b >>>>>>>>> --- /dev/null >>>>>>>>> +++ b/config/rpz/rpz-config >>>>>>>>> @@ -0,0 +1,130 @@ >>>>>>>>> +#!/bin/bash >>>>>>>>> +############################################################################### >>>>>>>>> +# # >>>>>>>>> +# IPFire.org - A linux based firewall # >>>>>>>>> +# Copyright (C) 2024-2025 IPFire Team # >>>>>>>>> +# # >>>>>>>>> +# This program is free software: you can redistribute it and/or modify # >>>>>>>>> +# it under the terms of the GNU General Public License as published by # >>>>>>>>> +# the Free Software Foundation, either version 3 of the License, or # >>>>>>>>> +# (at your option) any later version. # >>>>>>>>> +# # >>>>>>>>> +# This program is distributed in the hope that it will be useful, # >>>>>>>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of # >>>>>>>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # >>>>>>>>> +# GNU General Public License for more details. # >>>>>>>>> +# # >>>>>>>>> +# You should have received a copy of the GNU General Public License # >>>>>>>>> +# along with this program. If not, see . # >>>>>>>>> +# # >>>>>>>>> +############################################################################### >>>>>>>>> + >>>>>>>>> +version="2025-01-11 - v44" >>>>>>>>> + >>>>>>>>> +############### Functions ############### >>>>>>>>> + >>>>>>>>> +source /usr/sbin/rpz-functions >>>>>>>>> + >>>>>>>>> +############### Main ############### >>>>>>>>> + >>>>>>>>> +tagName="unbound" >>>>>>>>> + >>>>>>>>> +rpzAction="${1}" # input RPZ action >>>>>>>>> +rpzName="${2}" # input RPZ name >>>>>>>>> +rpzURL="${3}" # input RPZ URL >>>>>>>>> +rpzOption1="${4}" # input RPZ option #1 >>>>>>>>> +rpzOption2="${5}" # input RPZ option #2 >>>>>>>>> + >>>>>>>>> +rpzConfig="/etc/unbound/local.d/${rpzName}.rpz.conf" # output zone conf file >>>>>>>>> +rpzFile="/etc/unbound/zonefiles/${rpzName}.rpz" # output for RPZ file >>>>>>>>> + >>>>>>>>> +rpzLog="yes" # log default is yes >>>>>>>>> +ucReload="yes" # reload default is yes >>>>>>>>> + >>>>>>>>> +while [[ $# -gt 0 ]] ; do >>>>>>>>> + case "$1" in >>>>>>>>> + --no-log ) rpzLog="no" ;; >>>>>>>>> + --no-reload ) ucReload="no" ; checkConf="no" ;; >>>>>>>>> + esac >>>>>>>>> + shift # Shift after checking all the cases to get next option >>>>>>>>> +done >>>>>>>>> + >>>>>>>>> +case "${rpzAction}" in >>>>>>>>> + # add new rpz list >>>>>>>>> + add ) >>>>>>>>> + check_name "${rpzName}" # is this a valid name? >>>>>>>>> + # does this config already exist? If yes, then exit >>>>>>>>> + if [[ -f "${rpzConfig}" ]] ; then >>>>>>>>> + msg_log "error: rpz: duplicate - ${rpzConfig} already exists. exit" >>>>>>>>> + exit 104 >>>>>>>>> + fi >>>>>>>>> + >>>>>>>>> + # is this a valid URL? >>>>>>>>> + regex='^https://[-[:alnum:]\+&@#/%?=~_|!:,.;]*[-[:alnum:]\+&@#/%=~_|]' >>>>>>>>> + if ! [[ "${rpzURL}" =~ $regex ]] ; then >>>>>>>>> + msg_log "error: rpz: the URL is not valid: \"${rpzURL}\". exit." >>>>>>>>> + exit 105 >>>>>>>>> + fi >>>>>>>>> + >>>>>>>>> + # create the zone config file >>>>>>>>> + { >>>>>>>>> + echo "rpz:" >>>>>>>>> + echo " name: ${rpzName}.rpz" >>>>>>>>> + echo " zonefile: ${rpzFile}" >>>>>>>>> + echo " url: ${rpzURL}" >>>>>>>>> + echo " rpz-action-override: nxdomain" >>>>>>>>> + echo " rpz-log: ${rpzLog}" >>>>>>>>> + echo " rpz-log-name: ${rpzName}" >>>>>>>>> + echo " rpz-signal-nxdomain-ra: yes" >>>>>>>>> + } > "${rpzConfig}" >>>>>>>>> + >>>>>>>>> + # set-up zonefile >>>>>>>>> + # create an empty rpz file if it does not exist >>>>>>>>> + if [[ ! -f "${rpzFile}" ]] ; then >>>>>>>>> + touch "${rpzFile}" >>>>>>>>> + # unbound requires these settings for rpz files >>>>>>>>> + set_permissions "${rpzFile}" "${rpzConfig}" >>>>>>>>> + fi >>>>>>>>> + ;; >>>>>>>>> + >>>>>>>>> + # trash config file & rpz file >>>>>>>>> + remove ) >>>>>>>>> + if ! [[ -f "${rpzConfig}" ]] ; then >>>>>>>>> + msg_log "error: rpz: cannot remove ${rpzConfig}, does not exist. exit" >>>>>>>>> + exit 106 >>>>>>>>> + fi >>>>>>>>> + >>>>>>>>> + msg_log "info: rpz: remove config file & rpz file \"${rpzName}\"" >>>>>>>>> + rm "${rpzConfig}" >>>>>>>>> + rm "${rpzFile}" >>>>>>>>> + ;; >>>>>>>>> + >>>>>>>>> + reload ) >>>>>>>>> + check_unbound_conf "${checkConf}" >>>>>>>>> + ;; >>>>>>>>> + >>>>>>>>> + list ) >>>>>>>>> + awk -F':' '/^\s*name:/{ gsub(/[[:blank:]]|\.rpz/, "",$2) ; NAME=$2 } \ >>>>>>>>> + /^\s*url:/{ gsub(/[[:blank:]]/, "") ; print NAME"="$2":"$3} ' \ >>>>>>>>> + /etc/unbound/local.d/*rpz.conf >>>>>>>>> + exit >>>>>>>>> + ;; >>>>>>>>> + >>>>>>>>> + unbound-restart ) >>>>>>>>> + check_unbound_conf "${checkConf}" >>>>>>>>> + unbound_restart >>>>>>>>> + exit >>>>>>>>> + ;; >>>>>>>>> + >>>>>>>>> + * ) >>>>>>>>> + msg_log "error: rpz: missing or incorrect parameter" >>>>>>>>> + printf "Usage: $(basename "$0")