From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bernhard Bitsch To: development@lists.ipfire.org Subject: Re: [PATCH] RPZ: update code to include WEBGUI and additional languages Date: Fri, 14 Feb 2025 15:16:49 +0100 Message-ID: <7918fcbf-9787-4b55-aa63-09a215c9d294@ipfire.org> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1659952175765241016==" List-Id: --===============1659952175765241016== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello Michael, I use the hagezi lists (=20 https://github.com/hagezi/dns-blocklists?tab=3Dreadme-ov-file#zap-dns-blockli= sts---for-a-better-internet=20 ) which allow more selections. Regards, Bernhard Am 14.02.2025 um 14:52 schrieb Michael Tremer: > Hello Bernhard, >=20 >> On 14 Feb 2025, at 12:58, Bernhard Bitsch wrote: >> >> Hello Michael, >> >> thanks for your clear words. But nevertheless let be comment at some topic= s. >> >> Am 14.02.2025 um 13:07 schrieb Michael Tremer: >>> Hello Jon, >>> It very much depends on the kind of contribution. A one-line patch obviou= sly has fewer strings attached to it than a larger patch set like this. >>> However, we have outlined the process in the wiki already, starting from = here: >>> https://www.ipfire.org/docs/devel/submit-patches >>> This contains some useful pointers about the how (how do I actually make = my changes happen, how do I build IPFire, etc), and at the bottom it contains= a lot of information about the format the changes should be submitted; split= into smaller chunks that are ideally as independent from each other so that = they can be individually reviewed and merged. Usually a development process t= akes long time and we have already shipped parts of code that we will need fo= r certain features that are not ready yet. This is a good practice to let cod= e mature, especially when it is touching rather critical bits like the firewa= ll and networking stacks. >>> There are also some guidelines on how to write a good commit message and = how to use Git tags: >>> https://www.ipfire.org/docs/devel/git/commit-messages >>> https://www.ipfire.org/docs/devel/git/tags >>> Then there is something about how to get in touch with the right person a= nd legal stuff: >>> https://www.ipfire.org/docs/devel/contact >>> https://www.ipfire.org/docs/legal/ipca >> >> IMO, this facts are known by Jon. He just did a big part besides this gene= ral process. Including publishing his efforts to the community for review and= test. >> >>> Finally, we have a bit of an underused roadmap section on the wiki. It wo= uld be nice if we could use that a little bit more because then it would be e= asier for everyone to keep track of progress on certain features; people coul= d see what is being worked on and see if they can help development and testin= g and so on: >>> https://www.ipfire.org/docs/roadmap >>> There is a template on how to create new pages: >>> https://www.ipfire.org/docs/roadmap/template >>> And this is a good example of what this could look like: >>> https://www.ipfire.org/docs/roadmap/openvpn-26 >>> All of these steps are coming *after* there has been some initial discuss= ion about what actually has a chance to become part of the distribution. For = that, we do not have any specific guidelines because it is not very trivial t= o write these things. There are just too many possibilities. In the past, the= re has also been very little need for this, but that does not mean that there= have not been problems before. >>> The reason why I am raising the bar this high here is simply that we have= made mistakes in the past that we don=E2=80=99t want to repeat. We have lear= ned a couple of lessons in a not very pleasant way and I under no circumstanc= es would want to do this again. The objective is that we want to provide an e= xcellent distribution. Although IPFire of course has its shortcomings here an= d there, it is a very stable distribution and we have a very good track recor= d that I want to keep. This is what our users deserve. >>> In the past, people have =E2=80=9Cdropped=E2=80=9D their patches on this = list (or sometimes elsewhere) and we were left with dealing with the entire i= ntegration only to find out later what problems there were hidden in the code= . The original author(s) had no interest in fixing any of that because it wor= ked just fine for them, and so why spend any time on the problems of somebody= else? Usually I am the fallback for this and I simply don=E2=80=99t want to = be that. I have lots of my own projects inside IPFire that are moving at snai= l speed because fixing existing code usually takes priority over writing new = code. >>> Therefore we need a commitment to sort out these problems in the first pl= ace. It has to be proven that people actually *care* about the patches that t= hey post here. I am not sure this needs writing down as this should be the sa= me policy with almost any open source project. If you contribute a line, ther= e is probably less maintenance required in the future, but if you contribute = a large code base, then you will need to look after it for the foreseeable fu= ture. It is your feature and not mine after all. >> >> Being one of the first testers and revisors, I commit me to support the de= velopment. Point. >> >>> Then, what actually has a chance to make it into the distribution? Probab= ly not a lot. IPFire has a very clear use case. There will not be any space f= or a desktop environment and running Chrome on it, we also don=E2=80=99t it t= o make coffee and cook me a dinner. We would currently only accept things tha= t were actually maintainable by the current team in case a contributor moves = on (see above), because we simply only have so much man power. We already hav= e a large zoo of features that are very abandoned and we are potentially look= ing at getting rid of more things simply because we cannot support them prope= rly. Time just doesn=E2=80=99t permit. Adding something large is therefore ve= ry difficult at the moment. >>> I understand that in this specific case you have been trying to not invol= ve the development team and I understand your motivation. But you cannot forg= et about how much time and effort a review process can take. Therefore we wan= t to plan things well; we want to even split it; and we want to have a conver= sation in advance so that the roadmap is clear and the actual code review ide= ally only becomes a formality. >>> All of this above has been for a general case. Please read through this a= nd feel free to ask any questions if something isn=E2=80=99t clear. >>> To move forward with this feature, we should start by planning a roadmap.= We need to discuss what this project should cover and what it should not cov= er. I believe we don=E2=80=99t need to talk much about implementation details= because you have figured out a lot of them; we need to find what feature we = want to provide to our users. Are you up for that? >> >> As a first input to the 'official' discussion, some thoughts. >> - Usage of encrypted data traffic is increasing. Especially in communicati= ons not aware to the user. These unwanted traffic is more and more difficult = to block with IP orientated tools. Even the proxy based solution do not reall= y function efficient. >> - To circumvent these problems a widely used approach is DNS filtering. Th= is comes in two flavours: 1. online providers, 2. local filtering DNS servers= like PiHole. >> - Solution 1 isn't wantable. Many providers store the access data, outside= the local network. >> - Solution 2 installs another DNS server inside the local networks. It isn= 't trivial to ensure that all DNS traffic is handled by this server. >> - Therefore it is desirable that this filtering is done in the only DNS se= rver in a IPFire governed network, unbound on the IPFire system. The RPZ func= tionality allows exactly this. >> - There are, IMO, good filter lists outside. The updating of these lists i= s part of the RPZ module. There is some effort left to investigate these and = to develop recommendations for a good mix of appliance, we do this for the ex= ternal DNS servers already. >=20 > You are thinking too much about the *how* here, but we should think of the = *what* first. What features do we want to offer and what do we not want to of= fer. >=20 > Regarding the lists: I am running a test with one client which has about 20= 00 users on their network on a daily basis. There have been *huge* amounts of= false positives from the OISD list and it has been occasionally painful to w= hitelist those. The quality of this feature stands and falls with the quality= of the lists that we can feed into this. >=20 > I have also raised before that those lists collect random stuff together an= d in the end you only have one list that you can either take or leave. Coming= from URL filter, we have some categorisation there like =E2=80=9Cadult=E2=80= =9D, =E2=80=9Cgambling=E2=80=9D, =E2=80=9Csocial networks=E2=80=9D, and so on= which we don=E2=80=99t have here. Would that not be nice to have, too? >=20 >> Regards, >> Bernhard >> >>> Best, >>> -Michael >>>> On 13 Feb 2025, at 21:34, jon wrote: >>>> >>>> Michael, >>>> >>>> I=E2=80=99ve read through your comments a few times and I ended up with = many more questions. >>>> >>>> >>>>> What I rather mean is that it has never been added as a topic on the ag= enda and it has not been pitched by yourself. >>>> >>>> >>>> To me the efforts to get new code accepted seem to have changed and it s= eemed easier in the past. In the past I made the Core Team aware via the Dev= Mailing List and wrote a simple two or three paragraphs of "What is it? / Wh= at is the value? / Here is the code" >>>> >>>> >>>> So in an effort to move forward: How exactly is something presented to = the Core Team? >>>> >>>> Is there an example of a recent effort that was presented that I can see= as a sample? (This type of info can also be added to the Wiki) >>>> >>>> I understand you want it this way, but I don=E2=80=99t know what exactly= is needed. Please be specific. >>>> >>>> >>>> Jon >>>> >>>> PS - I am not ignoring your other comments, I am just trying to move for= ward and keep things simple. >>>> >>>> >>>> >>>>> On Feb 8, 2025, at 1:27=E2=80=AFPM, Michael Tremer wrote: >>>>> >>>>> Hello Jon, >>>>> >>>>> Thanks for your reply. And good that you are copying everyone into this= conversation. >>>>> >>>>>> On 8 Feb 2025, at 18:41, jon wrote: >>>>>> >>>>>> Michael, >>>>>> >>>>>>> I think I have covered this all at lengths before that this project h= as been started as a separate effort >>>>>> >>>>>> Yes, this has been a separate effort (a very public separate effort). = Yes, as you pointed this out early on with the "proof-of-concept" and then m= y request for people to help test RPZ. Nothing was hidden. >>>>>> >>>>>> This was done because you (and maybe others) did not have the time and= I wanted to help and because I needed assistance with RPZ. I tried my best = to do this without bothering you. >>>>> >>>>> I don=E2=80=99t that it is accurate that nobody wanted to help on this.= The list was always open - although not every email has been replied to swif= tly it is also your responsibility to raise a question again if it was missed= . People here have open ears. >>>>> >>>>> It was also stated on this very list on in our documentation that worki= ng on something without involving the core team is a risky undertaking. Of co= urse IPFire is free software and so everyone is free to fork if they wish to = do so. >>>>> >>>>>>> and as far as I am aware none of the other team members has been invo= lved. This has not been discussed either on this list, on our calls. >>>>>> >>>>>> You were aware many steps along the way. See your email on July 28, 2= 024, August 15, 2024, September 30, 2024, December 23, 2024, and January 16. = My attempts to get the team involved were met with "things are busy" and som= etimes silence. (Yes, I get it, people are busy.) >>>>>> >>>>>> You and Adolf, Leo, Erik and Bernhard have been aware since the beginn= ing. You mention you were aware of the "proof-of-concept". If you include t= hose beginning posts, since Sep 2023. >>>>> >>>>> Yes, I am aware of a proof-of-concept that I have been running myself f= or a long time. I am also aware of the efforts that you have been taking. >>>>> >>>>> Yet I don=E2=80=99t think there has ever been any joint effort, or am I= seeing that wrong? >>>>> >>>>>>> This has not been discussed . . . on our calls. >>>>>> >>>>>> On the July 28th you stated: >>>>>> "We have talked about RPZ many times on the monthly call since the URL= filter feature is falling more and more out of fashion. I think there is als= o many posts about this on the forum." >>>>>> >>>>>> Please don=E2=80=99t insult me again by stating "you know what I mean". >>>>>> >>>>>> And it has been discussed but not documented in the Monthly Meeting no= tes. >>>>> >>>>> I am not at all insulting you. I don=E2=80=99t want to take this down t= o a personal level at all. This is a public mailing list and people who read = this don=E2=80=99t need to listen to an argument we are having. They are here= for the tech inside IPFire. >>>>> >>>>> When I wrote that it has not been discussed that does not mean that we = have not been touching on the topic. We have been talking about lots of thing= s on the calls, the weather, politics, how our pets are. None of that makes i= t to the logs. What I rather mean is that it has never been added as a topic = on the agenda and it has not been pitched by yourself. >>>>> >>>>>>> Instead there has been a separate conversation on the forum with the = occasional dip here to the list. But that was not a regular two-way conversat= ion. >>>>>> >>>>>> Regular conversation on the Dev Mailing list is many times met with si= lence. I get it, people are busy. >>>>>> >>>>>> And regular two-way conversation doesn=E2=80=99t happen on the list. = At least not with me. I=E2=80=99d be happy to point out the posts that were = met with silence. >>>>>> Again, I get it, people are busy. >>>>> >>>>> And you think my emails are not being met with silence? This has nothin= g to do with this specific topic. This has something to do with how occupied = people are and how engaged they are on certain topics. Not everyone is involv= ed in all the things and simply will ignore emails simply based on their subj= ect line. >>>>> >>>>>> But the "dip here to the list" were my attempts to get a conversation = started. As I said, many time met with silence. >>>>>> >>>>>> The only place I was not met with silence was on the Community. You h= ave a great group of people in the Community. It is a shame you don=E2=80=99= t want to have others help. It would reduce your workload. >>>>> >>>>> You should stop making statements that are not true. Who doesn=E2=80=99= t want anyone to help? >>>>> >>>>> Not having this conversation on a Saturday evening would reduce my work= load. At least it would free up time for something else. Helping with the thi= ngs that are already on the go would reduce the workload of the entire team. = Starting one thing at a time and finishing it is a lot better to manage than = starting a hundred things and not even finish one. I can tell you that I alre= ady have a hundred things on the go. >>>>> >>>>>>> Therefore, what am I supposed to do with this email? >>>>>> >>>>>> To me it is beyond obvious=E2=80=A6 >>>>>> >>>>>> If it isn=E2=80=99t what you want, then guide me with how to do this t= he correct way. And be specific. I am trying to help. I am trying to make t= hings better. I am trying to do things the right way. >>>>> >>>>> To me it isn=E2=80=99t. This is yet another project that has been dumpe= d to the list like so many before and later on everyone has left to have the = team deal with the rest. >>>>> >>>>> It is a huge patch set. You explained what the vision is, but that is a= bout it. There is no chance this will continue if this disagreement isn=E2=80= =99t solved first. I didn=E2=80=99t even look at the code. >>>>> >>>>>>> I don=E2=80=99t want to merge code that I don=E2=80=99t agree with. >>>>>> >>>>>> I asked multiple times if you "agreed with the concept" and again, met= with silence. Yes I get it, people are busy. >>>>> >>>>> Having support for RPZ? Yes, it was definitely on the roadmap. That I a= gree with. >>>>> >>>>>>> So many fundamental things that I have been raising have either not b= een discussed or outright dismissed. >>>>>> >>>>>> You mentioned this a in the past, but for some reason you do not discl= ose what I dismissed. Why do you continue to make this harder, wouldn=E2=80= =99t it not be easier to tell me what I have dismissed? >>>>>> >>>>>> I have sent multiple emails trying to answer your concerns and comment= s. On July 28, Aug 14, Aug 22, Aug 23, Sep 30, etc. >>>>>> >>>>>> I=E2=80=99ve gone through all of the questions you asked and I cannot = find a "dismissed" item. >>>>> >>>>> Maybe I need to be *more clear*. I feel humoured by this. >>>>> >>>>> It is late on a Saturday and I want my dinner soon, but certainly I hav= e stated that this should never be an add-on considering it is supposed to re= place URL Filter. We should never allow people to add their own sources. I ha= ve also stated that we cannot download any lists over HTTPS again and again a= nd again. The implementation that we have here seems to exactly do that and t= herefore I think that my feedback has been dismissed entirely. >>>>> >>>>>>> I don=E2=80=99t want to merge code that has no future inside IPFire a= s there is no constructive conversation with the maintainers of it. >>>>>> >>>>>> The maintainers of Unbound and/or RPZ? >>>>>> >>>>>> The maintainers of Hagezi list, the threatfox list, the urlhaus list, = etc.? >>>>>> >>>>>> What else? The maintainers or the RPZ scripts? That is me. Let=E2= =80=99s talk! >>>>> >>>>> You. I don=E2=80=99t care much about the providers of the lists. >>>>> >>>>>> See, this is where it gets confusing. There are hundreds of open sour= ce packages as part of IPFire. Pick the last five years of items added to th= e IPFire build. You're telling me you have "constructive conversation with t= he maintainers" of all of the added packages? >>>>> >>>>> They publish their software and they don=E2=80=99t care whether I am pu= lling it or not. They publish it with the commitment to maintain it - sometim= es for better and sometimes for worse. >>>>> >>>>> You care about me pulling your code and I don=E2=80=99t know whether yo= u would commit to maintain this. >>>>> >>>>> These two are very different cases. >>>>> >>>>>> Pick the IP Blocklists list (i.e., 3CORESEC, ABUSECH, DSHIELD, SPAMHAU= S, etc.) or the Suricata lists (i.e., Emergingthreats.net, Abuse.ch, etc.). = So you=E2=80=99ve have "constructive conversation with the maintainers"? >>>>> >>>>> Yes, occasionally I have phone calls with a few of these providers. >>>>> >>>>>>> Having been trying for a long time to make you aware of this, nothing= of this should come as a surprise. >>>>>> >>>>>> Ha! Yes a surprise. In the beginning you seemed interested as IPFire= needed a replacement for URL Filter. You asked good questions about the lis= ts picked, asked for the value to the users, etc. And I answered the best I = could. >>>>>> >>>>>> You even asked: =E2=80=9CWhy is this realised as an add-on and not par= t of the core system?=E2=80=9D from your Jul 28, 2024 email. >>>>> >>>>> Ah, so, why is the patch creating an add-on? Not that I am saying that = what I say is law, but it has not been challenged either. If my input is bein= g ignored, why should I put this to the top of my list of priorities? I am no= t disappointed about this, just trying to be very good with my time. >>>>> >>>>>> And on January 16, 2025 I wrote a message looking for help. And you w= ere kind to respond quickly. So in three weeks time, since the kind response= , something has changed. You went from supportive to "this". >>>>>> >>>>>> So yes, I am surprised. >>>>> >>>>> Well, maybe I should not have replied to that email. It was clear that = you were on some path that was not right, but you were not interested before = in finding the right path from the beginning. >>>>> >>>>>>> Please consider if that can be changed and if there is a path forward= with this. >>>>>> >>>>>> Be more specific, what has to change? What exactly did I dismiss? >>>>> >>>>> Dismissal is just my assumption. I don=E2=80=99t know what you actually= did with my feedback. I can only see the end product that does not seem cont= ain much of it. Repeatedly I have been pointing out that we should think befo= re we build. I am sure a lot of hours have now gone into some code that simpl= y does not satisfy me. And I am not not talking about the code itself, what i= t does is what I don=E2=80=99t think is right for us. >>>>> >>>>> The process is very clear for me that we should first of all think whet= her we want a certain feature now. Then there should be a clear roadmap for e= veryone to follow; tasks can be split-up as we go and hopefully then have som= ething that is maintainable, interesting for our users and even would do us p= roud. This is how this should work. >>>>> >>>>> So, what has to change? I don=E2=80=99t think with shouting at each oth= er, throwing patches around and making me generally unhappy is a good start. >>>>> >>>>> -Michael >>>>> >>>>>> Jon >>>>>> >>>>>> >>>>>> >>>>>>> On Feb 6, 2025, at 2:13=E2=80=AFPM, Michael Tremer wrote: >>>>>>> >>>>>>> Hello Jon, >>>>>>> >>>>>>> Well, here we are again with another patch regarding this feature. >>>>>>> >>>>>>> I cannot quite see from your email what the question is, but if this = is a request to have this merged into IPFire, I am once again sorry to disapp= oint you. >>>>>>> >>>>>>> I think I have covered this all at lengths before that this project h= as been started as a separate effort and as far as I am aware none of the oth= er team members has been involved. This has not been discussed either on this= list, on our calls. Instead there has been a separate conversation on the fo= rum with the occasional dip here to the list. But that was not a regular two-= way conversation. Therefore, what am I supposed to do with this email? >>>>>>> >>>>>>> I don=E2=80=99t want to merge code that I don=E2=80=99t agree with. S= o many fundamental things that I have been raising have either not been discu= ssed or outright dismissed. >>>>>>> >>>>>>> I don=E2=80=99t want to merge code that has no future inside IPFire a= s there is no constructive conversation with the maintainers of it. >>>>>>> >>>>>>> Having been trying for a long time to make you aware of this, nothing= of this should come as a surprise. >>>>>>> >>>>>>> Please consider if that can be changed and if there is a path forward= with this. >>>>>>> >>>>>>> All the best, >>>>>>> -Michael >>>>>>> >>>>>>>> On 6 Feb 2025, at 16:35, Jon Murphy wrote: >>>>>>>> >>>>>>>> What is it? >>>>>>>> Response Policy Zone (RPZ) is a mechanism to define local policies i= n a >>>>>>>> standardized way and load those policies from external sources. >>>>>>>> Bottom line: RPZ allows admins to easily block access to websites vi= a DNS lookup. >>>>>>>> >>>>>>>> RPZ can block websites via categories. Examples include: fake websi= tes, annoying >>>>>>>> pop-up ads, newly registered domains, DoH bypass sites, bad "host" s= ervices, >>>>>>>> maliscious top level domains (e.g., *.zip, *.mov), piracy, gambling,= pornography, >>>>>>>> and more. RPZ lists come from various RPZ providers and their avail= able >>>>>>>> catagories. >>>>>>>> >>>>>>>> This RPZ add-on enables the RPZ functionality by adding a couple lin= es in a >>>>>>>> configuration file. This add-on simply adds configuration files and= adds >>>>>>>> scripts (config, metrics and sleep) to make RPZ easier for the admin= to use. >>>>>>>> >>>>>>>> The RPZ scripts include additional languages: German, Spanish, Frenc= h, Turkish, >>>>>>>> and Italian. >>>>>>>> >>>>>>>> RPZ itself was release in 2010 and has been part of the IPFire build= since ~2015. >>>>>>>> >>>>>>>> Why is it needed? What is its value? >>>>>>>> >>>>>>>> - The RPZ concept places this filtering into IPFire, our internet ac= cess >>>>>>>> gateway, which is (should be) solely used as DNS source of the inter= nal network. >>>>>>>> >>>>>>>> - As most sites use HTTPS it makes it difficult to filter traffic wi= th URL >>>>>>>> Filter without also properly configuring conventional (non-transpare= nt) >>>>>>>> mode on the proxy. RPZ is a nice replacement for the URL Filter. >>>>>>>> >>>>>>>> - No need to install and maintain an additional device like PiHole o= r AdBlock >>>>>>>> browser extensions on multiple user devices. >>>>>>>> >>>>>>>> - This is an additional layer of protection for users. Less worry so= meone will >>>>>>>> click on something that gets them into trouble. And, saying this wit= h emphasis, >>>>>>>> the ability to do it in one place! >>>>>>>> >>>>>>>> - Blocked sites save on unneeded traffic and can lessen the threat o= f malware >>>>>>>> in advertisements >>>>>>>> >>>>>>>> - Logging allows the admin to see the site blocked and take actions >>>>>>>> >>>>>>>> - RPZ will be used at the home, home-office (work from home), school= s, >>>>>>>> ministerial, and at the office. Device counts are small (2-6) to me= dium (~80) >>>>>>>> to mediam-large (200+). >>>>>>>> >>>>>>>> - RPZ can block ads, popups, phishing, scammers, spyware, malware, a= nnoying >>>>>>>> popups, NSFW links, DOH servers, and the usual internet trash. >>>>>>>> >>>>>>>> ------------------------------ >>>>>>>> >>>>>>>> Change Log for RPZ add-on >>>>>>>> >>>>>>>> rpz-1.0.0-18 on 2025-02-05 >>>>>>>> - Build for approval & release as IPFire add-on >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> rpz-beta-0.1.18-18.ipfire on 2025-02-01 >>>>>>>> rpz.cgi: >>>>>>>> - new feature: added a mod key to force a unbound restart >>>>>>>> >>>>>>>> rpz-config and rpz-make: >>>>>>>> - new feature: added action for unbound restart `rpz-config unbound-= restart` >>>>>>>> >>>>>>>> rpz-metrics: >>>>>>>> - simple reformatting >>>>>>>> - rename far right column from "last update" to "last download" >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> rpz-beta-0.1.17-17.ipfire on 2024-12-09 >>>>>>>> rpz-make >>>>>>>> - bug fix: corrected validation regex for wildcards like: `*.domain.= com` >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> rpz-beta-0.1.16-16.ipfire on 2024-11-18 >>>>>>>> rpz-make >>>>>>>> - new feature: updated validation regex >>>>>>>> - bug fix: moved validation to beginning of process. Now we validat= e before >>>>>>>> creating config files. >>>>>>>> >>>>>>>> rpz.cgi: >>>>>>>> - new feature: use CSS color variables of the main ipfire theme >>>>>>>> - bug fix: empty zonefile remarks were stored as =E2=80=9Cundef=E2= =80=9D and caused a warning >>>>>>>> - bug fix: HTML textarea removes the first empty line in a custom li= st >>>>>>>> - thank you Leo! >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> rpz-beta-0.1.15-15.ipfire on 2024-11-04 >>>>>>>> rpz.cgi: >>>>>>>> - new feature: added new language file for Turkish (thank you Peppe) >>>>>>>> >>>>>>>> rpz-make >>>>>>>> - bug fix: corrected empty allow/block list issue. An empty allow/b= lock list >>>>>>>> will now remove contents of allow/block.rpz files and remove unneeded >>>>>>>> allow/block.conf file. (thank you iptom) >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> rpz-beta-0.1.14-14.ipfire on 2024-10-29 >>>>>>>> rpz-config: >>>>>>>> - bug fix: correct missing rpz extension. `rpz-config list` displaye= d URL >>>>>>>> incorrectly (thank you Bernhard) >>>>>>>> >>>>>>>> rpz.cgi: >>>>>>>> - bug fix: remove extra `"` in language files (thank you Bernhard) >>>>>>>> - new feature: slightly dim "apply" button when not enabled >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> rpz-beta-0.1.13-13.ipfire on 2024-10-27 >>>>>>>> - skipped >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> rpz-beta-0.1.12-12.ipfire on 2024-10-21 >>>>>>>> rpz.cgi: >>>>>>>> - new feature: added new language file for French (thank you gw-ipf= ire) >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> rpz-beta-0.1.11-11.ipfire on 2024-10-18 >>>>>>>> rpz.cgi: >>>>>>>> - new feature: added new language file for Italian (thank you umbert= o) >>>>>>>> - new feature: added new language file for Spanish (thank you Robert= o) >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> rpz-beta-0.1.10-10.ipfire on 2024-10-15 >>>>>>>> rpz-make: >>>>>>>> - bug fix: corrected validation error for a custom list entry (thank= you siosios) >>>>>>>> - e.g., `*.cloudflare-dns.com` >>>>>>>> >>>>>>>> install.sh: >>>>>>>> - bug fix: add chown to correct user created files >>>>>>>> >>>>>>>> update.sh: >>>>>>>> - bug fix: add chown to correct user created files (thank you siosio= s) >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> rpz-beta-0.1.9-9.ipfire on 2024-10-08 >>>>>>>> rpz.cgi: >>>>>>>> - new feature: added new language file for German (thank you Leo) >>>>>>>> - bug fix: add missing "rpz exitcode 110" >>>>>>>> - bug fix: corrected missing RPZ menu item at menu > IPFire >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> rpz-beta-0.1.8-8.ipfire on 2024-10-04 >>>>>>>> - skipped >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> rpz-beta-0.1.7-7.ipfire on 2024-10-03 >>>>>>>> All: >>>>>>>> - new feature: includes beta version numbers for pakfire package, >>>>>>>> instead of only `rpz-1.0.0-1.ipfire`, for each release. >>>>>>>> >>>>>>>> rpz.cgi: >>>>>>>> - new feature: added new WebGUI at `rpz.cgi` >>>>>>>> - a BIG thank you to Leo Hofmann for all of his work creating the we= bgui!! >>>>>>>> - bug fix: corrected missing RPZ menu item at menu > IPFire >>>>>>>> >>>>>>>> rpz-make: >>>>>>>> - new feature: validate entries in allowlist and blocklist >>>>>>>> - new feature: add "no-reload" option for WebGUI >>>>>>>> >>>>>>>> rpz-metrics: >>>>>>>> - new feature: info can be sorted by name, by hit count, by line cou= nt, by >>>>>>>> "enabled" list or all lists >>>>>>>> >>>>>>>> backups: >>>>>>>> - bug fix: include all files in `/var/ipfire/dns/rpz` directory in b= ackup >>>>>>>> >>>>>>>> update.sh: >>>>>>>> - bug fix: corrected ownership for `/var/ipfire/dns/rpz` directory d= uring an >>>>>>>> update >>>>>>>> >>>>>>>> Build: >>>>>>>> - bug fix: `block.rpz.conf` and `block.rpz` from build. Files to be= created >>>>>>>> by `rpz-make` >>>>>>>> >>>>>>>> WebGUI and German language file >>>>>>>> Contribution-by: Leo-Andres Hofmann >>>>>>>> >>>>>>>> Spanish language file >>>>>>>> Contribution-by: Roberto Pe=C3=B1a >>>>>>>> >>>>>>>> Italian language file >>>>>>>> Contribution-by: Umberto Parma >>>>>>>> >>>>>>>> French language file >>>>>>>> Contribution-by: gw-ipfire >>>>>>>> >>>>>>>> Turkish language file >>>>>>>> Contribution-by: Peppe Tech >>>>>>>> >>>>>>>> Contribution-by: Bernhard Bitsch >>>>>>>> Contribution-by: Erik Kapfer >>>>>>>> Signed-off-by: Jon Murphy >>>>>>> --- >>>>>>>> config/backup/includes/rpz | 4 + >>>>>>>> config/cfgroot/manualpages | 1 + >>>>>>>> config/menu/EX-rpz.menu | 6 + >>>>>>>> config/rootfiles/common/configroot | 1 + >>>>>>>> config/rootfiles/common/web-user-interface | 1 + >>>>>>>> config/rootfiles/packages/rpz | 20 + >>>>>>>> config/rpz/00-rpz.conf | 10 + >>>>>>>> config/rpz/rpz-config | 130 +++ >>>>>>>> config/rpz/rpz-functions | 85 ++ >>>>>>>> config/rpz/rpz-make | 203 +++++ >>>>>>>> config/rpz/rpz-metrics | 170 ++++ >>>>>>>> config/rpz/rpz-sleep | 58 ++ >>>>>>>> config/rpz/rpz.de.pl | 30 + >>>>>>>> config/rpz/rpz.en.pl | 30 + >>>>>>>> config/rpz/rpz.es.pl | 30 + >>>>>>>> config/rpz/rpz.fr.pl | 30 + >>>>>>>> config/rpz/rpz.it.pl | 30 + >>>>>>>> config/rpz/rpz.tr.pl | 30 + >>>>>>>> html/cgi-bin/rpz.cgi | 923 +++++++++++++++++++= ++ >>>>>>>> lfs/rpz | 96 +++ >>>>>>>> make.sh | 3 +- >>>>>>>> src/paks/rpz/install.sh | 36 + >>>>>>>> src/paks/rpz/uninstall.sh | 38 + >>>>>>>> src/paks/rpz/update.sh | 52 ++ >>>>>>>> 24 files changed, 2016 insertions(+), 1 deletion(-) >>>>>>>> create mode 100644 config/backup/includes/rpz >>>>>>>> create mode 100644 config/menu/EX-rpz.menu >>>>>>>> create mode 100644 config/rootfiles/packages/rpz >>>>>>>> create mode 100644 config/rpz/00-rpz.conf >>>>>>>> create mode 100644 config/rpz/rpz-config >>>>>>>> create mode 100644 config/rpz/rpz-functions >>>>>>>> create mode 100644 config/rpz/rpz-make >>>>>>>> create mode 100755 config/rpz/rpz-metrics >>>>>>>> create mode 100755 config/rpz/rpz-sleep >>>>>>>> create mode 100644 config/rpz/rpz.de.pl >>>>>>>> create mode 100644 config/rpz/rpz.en.pl >>>>>>>> create mode 100644 config/rpz/rpz.es.pl >>>>>>>> create mode 100644 config/rpz/rpz.fr.pl >>>>>>>> create mode 100644 config/rpz/rpz.it.pl >>>>>>>> create mode 100644 config/rpz/rpz.tr.pl >>>>>>>> create mode 100644 html/cgi-bin/rpz.cgi >>>>>>>> create mode 100644 lfs/rpz >>>>>>>> create mode 100644 src/paks/rpz/install.sh >>>>>>>> create mode 100644 src/paks/rpz/uninstall.sh >>>>>>>> create mode 100644 src/paks/rpz/update.sh >>>>>>>> >>>>>>>> diff --git a/config/backup/includes/rpz b/config/backup/includes/rpz >>>>>>>> new file mode 100644 >>>>>>>> index 000000000..36513e494 >>>>>>>> --- /dev/null >>>>>>>> +++ b/config/backup/includes/rpz >>>>>>>> @@ -0,0 +1,4 @@ >>>>>>>> +/var/ipfire/dns/rpz/* >>>>>>>> +/etc/unbound/zonefiles/allow.rpz >>>>>>>> +/etc/unbound/zonefiles/block.rpz >>>>>>>> +/etc/unbound/local.d/*rpz.conf >>>>>>>> diff --git a/config/cfgroot/manualpages b/config/cfgroot/manualpages >>>>>>>> index 1f7e01efc..d3a48c633 100644 >>>>>>>> --- a/config/cfgroot/manualpages >>>>>>>> +++ b/config/cfgroot/manualpages >>>>>>>> @@ -70,6 +70,7 @@ pakfire.cgi=3Dconfiguration/ipfire/pakfire >>>>>>>> wlanap.cgi=3Daddons/wireless >>>>>>>> tor.cgi=3Daddons/tor >>>>>>>> samba.cgi=3Daddons/samba >>>>>>>> +rpz.cgi=3Daddons/rpz >>>>>>>> >>>>>>>> # Logs menu >>>>>>>> logs.cgi/summary.dat=3Dconfiguration/logs/summary >>>>>>>> diff --git a/config/menu/EX-rpz.menu b/config/menu/EX-rpz.menu >>>>>>>> new file mode 100644 >>>>>>>> index 000000000..2f4daf410 >>>>>>>> --- /dev/null >>>>>>>> +++ b/config/menu/EX-rpz.menu >>>>>>>> @@ -0,0 +1,6 @@ >>>>>>>> +$subipfire->{'20.rpz'} =3D { >>>>>>>> + 'caption' =3D> $Lang::tr{'rpz'}, >>>>>>>> + 'uri' =3D> '/cgi-bin/rpz.cgi', >>>>>>>> + 'title' =3D> "RPZ", >>>>>>>> + 'enabled' =3D> 1, >>>>>>>> +}; >>>>>>>> diff --git a/config/rootfiles/common/configroot b/config/rootfiles/c= ommon/configroot >>>>>>>> index 9839eee45..b30d6aae4 100644 >>>>>>>> --- a/config/rootfiles/common/configroot >>>>>>>> +++ b/config/rootfiles/common/configroot >>>>>>>> @@ -120,6 +120,7 @@ var/ipfire/menu.d/70-log.menu >>>>>>>> #var/ipfire/menu.d/EX-apcupsd.menu >>>>>>>> #var/ipfire/menu.d/EX-guardian.menu >>>>>>>> #var/ipfire/menu.d/EX-mympd.menu >>>>>>>> +#var/ipfire/menu.d/EX-rpz.menu >>>>>>>> #var/ipfire/menu.d/EX-samba.menu >>>>>>>> #var/ipfire/menu.d/EX-tor.menu >>>>>>>> #var/ipfire/menu.d/EX-transmission.menu >>>>>>>> diff --git a/config/rootfiles/common/web-user-interface b/config/roo= tfiles/common/web-user-interface >>>>>>>> index 816241dae..e00464076 100644 >>>>>>>> --- a/config/rootfiles/common/web-user-interface >>>>>>>> +++ b/config/rootfiles/common/web-user-interface >>>>>>>> @@ -69,6 +69,7 @@ srv/web/ipfire/cgi-bin/proxy.cgi >>>>>>>> srv/web/ipfire/cgi-bin/qos.cgi >>>>>>>> srv/web/ipfire/cgi-bin/remote.cgi >>>>>>>> srv/web/ipfire/cgi-bin/routing.cgi >>>>>>>> +#srv/web/ipfire/cgi-bin/rpz.cgi >>>>>>>> #srv/web/ipfire/cgi-bin/samba.cgi >>>>>>>> srv/web/ipfire/cgi-bin/services.cgi >>>>>>>> srv/web/ipfire/cgi-bin/shutdown.cgi >>>>>>>> diff --git a/config/rootfiles/packages/rpz b/config/rootfiles/packag= es/rpz >>>>>>>> new file mode 100644 >>>>>>>> index 000000000..1c8663049 >>>>>>>> --- /dev/null >>>>>>>> +++ b/config/rootfiles/packages/rpz >>>>>>>> @@ -0,0 +1,20 @@ >>>>>>>> +etc/unbound/local.d/00-rpz.conf >>>>>>>> +etc/unbound/zonefiles >>>>>>>> +etc/unbound/zonefiles/allow.rpz >>>>>>>> +usr/sbin/rpz-config >>>>>>>> +usr/sbin/rpz-functions >>>>>>>> +usr/sbin/rpz-make >>>>>>>> +usr/sbin/rpz-metrics >>>>>>>> +usr/sbin/rpz-sleep >>>>>>>> +var/ipfire/addon-lang/rpz.de.pl >>>>>>>> +var/ipfire/addon-lang/rpz.en.pl >>>>>>>> +var/ipfire/addon-lang/rpz.es.pl >>>>>>>> +var/ipfire/addon-lang/rpz.fr.pl >>>>>>>> +var/ipfire/addon-lang/rpz.it.pl >>>>>>>> +var/ipfire/addon-lang/rpz.tr.pl >>>>>>>> +var/ipfire/backup/addons/includes/rpz >>>>>>>> +var/ipfire/dns/rpz >>>>>>>> +var/ipfire/dns/rpz/allowlist >>>>>>>> +var/ipfire/dns/rpz/blocklist >>>>>>>> +var/ipfire/menu.d/EX-rpz.menu >>>>>>>> +srv/web/ipfire/cgi-bin/rpz.cgi >>>>>>>> diff --git a/config/rpz/00-rpz.conf b/config/rpz/00-rpz.conf >>>>>>>> new file mode 100644 >>>>>>>> index 000000000..f005a4f2e >>>>>>>> --- /dev/null >>>>>>>> +++ b/config/rpz/00-rpz.conf >>>>>>>> @@ -0,0 +1,10 @@ >>>>>>>> +server: >>>>>>>> + module-config: "respip validator iterator" >>>>>>>> + >>>>>>>> +rpz: >>>>>>>> + name: allow.rpz >>>>>>>> + zonefile: /etc/unbound/zonefiles/allow.rpz >>>>>>>> + rpz-action-override: passthru >>>>>>>> + rpz-log: yes >>>>>>>> + rpz-log-name: allow >>>>>>>> + rpz-signal-nxdomain-ra: yes >>>>>>>> diff --git a/config/rpz/rpz-config b/config/rpz/rpz-config >>>>>>>> new file mode 100644 >>>>>>>> index 000000000..c72d50f9b >>>>>>>> --- /dev/null >>>>>>>> +++ b/config/rpz/rpz-config >>>>>>>> @@ -0,0 +1,130 @@ >>>>>>>> +#!/bin/bash >>>>>>>> +###################################################################= ############ >>>>>>>> +# = # >>>>>>>> +# IPFire.org - A linux based firewall = # >>>>>>>> +# Copyright (C) 2024-2025 IPFire Team = # >>>>>>>> +# = # >>>>>>>> +# This program is free software: you can redistribute it and/or mo= dify # >>>>>>>> +# it under the terms of the GNU General Public License as publishe= d by # >>>>>>>> +# the Free Software Foundation, either version 3 of the License, o= r # >>>>>>>> +# (at your option) any later version. = # >>>>>>>> +# = # >>>>>>>> +# This program is distributed in the hope that it will be useful, = # >>>>>>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of = # >>>>>>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the = # >>>>>>>> +# GNU General Public License for more details. = # >>>>>>>> +# = # >>>>>>>> +# You should have received a copy of the GNU General Public Licens= e # >>>>>>>> +# along with this program. If not, see . # >>>>>>>> +# = # >>>>>>>> +###################################################################= ############ >>>>>>>> + >>>>>>>> +version=3D"2025-01-11 - v44" >>>>>>>> + >>>>>>>> +############### Functions ############### >>>>>>>> + >>>>>>>> +source /usr/sbin/rpz-functions >>>>>>>> + >>>>>>>> +############### Main ############### >>>>>>>> + >>>>>>>> +tagName=3D"unbound" >>>>>>>> + >>>>>>>> +rpzAction=3D"${1}" # input RPZ action >>>>>>>> +rpzName=3D"${2}" # input RPZ name >>>>>>>> +rpzURL=3D"${3}" # input RPZ URL >>>>>>>> +rpzOption1=3D"${4}" # input RPZ option #1 >>>>>>>> +rpzOption2=3D"${5}" # input RPZ option #2 >>>>>>>> + >>>>>>>> +rpzConfig=3D"/etc/unbound/local.d/${rpzName}.rpz.conf" # output= zone conf file >>>>>>>> +rpzFile=3D"/etc/unbound/zonefiles/${rpzName}.rpz" # output= for RPZ file >>>>>>>> + >>>>>>>> +rpzLog=3D"yes" # log default is yes >>>>>>>> +ucReload=3D"yes" # reload default is yes >>>>>>>> + >>>>>>>> +while [[ $# -gt 0 ]] ; do >>>>>>>> + case "$1" in >>>>>>>> + --no-log ) rpzLog=3D"no" ;; >>>>>>>> + --no-reload ) ucReload=3D"no" ; checkConf=3D"no" ;; >>>>>>>> + esac >>>>>>>> + shift # Shift after checking all the cases to get next op= tion >>>>>>>> +done >>>>>>>> + >>>>>>>> +case "${rpzAction}" in >>>>>>>> + # add new rpz list >>>>>>>> + add ) >>>>>>>> + check_name "${rpzName}" # is this a valid name? >>>>>>>> + # does this config already exist? If yes, then exit >>>>>>>> + if [[ -f "${rpzConfig}" ]] ; then >>>>>>>> + msg_log "error: rpz: duplicate - ${rpzConfig} already e= xists. exit" >>>>>>>> + exit 104 >>>>>>>> + fi >>>>>>>> + >>>>>>>> + # is this a valid URL? >>>>>>>> + regex=3D'^https://[-[:alnum:]\+&@#/%?=3D~_|!:,.;]*[-[:alnum= :]\+&@#/%=3D~_|]' >>>>>>>> + if ! [[ "${rpzURL}" =3D~ $regex ]] ; then >>>>>>>> + msg_log "error: rpz: the URL is not valid: \"${rpzURL}\= ". exit." >>>>>>>> + exit 105 >>>>>>>> + fi >>>>>>>> + >>>>>>>> + # create the zone config file >>>>>>>> + { >>>>>>>> + echo "rpz:" >>>>>>>> + echo " name: ${rpzName}.rpz" >>>>>>>> + echo " zonefile: ${rpzFile}" >>>>>>>> + echo " url: ${rpzURL}" >>>>>>>> + echo " rpz-action-override: nxdomain" >>>>>>>> + echo " rpz-log: ${rpzLog}" >>>>>>>> + echo " rpz-log-name: ${rpzName}" >>>>>>>> + echo " rpz-signal-nxdomain-ra: yes" >>>>>>>> + } > "${rpzConfig}" >>>>>>>> + >>>>>>>> + # set-up zonefile >>>>>>>> + # create an empty rpz file if it does not exist >>>>>>>> + if [[ ! -f "${rpzFile}" ]] ; then >>>>>>>> + touch "${rpzFile}" >>>>>>>> + # unbound requires these settings for rpz files >>>>>>>> + set_permissions "${rpzFile}" "${rpzConfig}" >>>>>>>> + fi >>>>>>>> + ;; >>>>>>>> + >>>>>>>> + # trash config file & rpz file >>>>>>>> + remove ) >>>>>>>> + if ! [[ -f "${rpzConfig}" ]] ; then >>>>>>>> + msg_log "error: rpz: cannot remove ${rpzConfig}, does n= ot exist. exit" >>>>>>>> + exit 106 >>>>>>>> + fi >>>>>>>> + >>>>>>>> + msg_log "info: rpz: remove config file & rpz file \"${rpzNa= me}\"" >>>>>>>> + rm "${rpzConfig}" >>>>>>>> + rm "${rpzFile}" >>>>>>>> + ;; >>>>>>>> + >>>>>>>> + reload ) >>>>>>>> + check_unbound_conf "${checkConf}" >>>>>>>> + ;; >>>>>>>> + >>>>>>>> + list ) >>>>>>>> + awk -F':' '/^\s*name:/{ gsub(/[[:blank:]]|\.rpz/, "",$2) ; = NAME=3D$2 } \ >>>>>>>> + /^\s*url:/{ gsub(/[[:blank:]]/, "") ; print NAME"=3D"$2= ":"$3} ' \ >>>>>>>> + /etc/unbound/local.d/*rpz.conf >>>>>>>> + exit >>>>>>>> + ;; >>>>>>>> + >>>>>>>> + unbound-restart ) >>>>>>>> + check_unbound_conf "${checkConf}" >>>>>>>> + unbound_restart >>>>>>>> + exit >>>>>>>> + ;; >>>>>>>> + >>>>>>>> + * ) >>>>>>>> + msg_log "error: rpz: missing or incorrect parameter" >>>>>>>> + printf "Usage: $(basename "$0")