From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer To: development@lists.ipfire.org Subject: Re: [PATCH] RPZ: update code to include WEBGUI and additional languages Date: Fri, 14 Feb 2025 13:52:12 +0000 Message-ID: In-Reply-To: <2ec810a4-7bca-4e98-9c21-2720ae344fb4@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1660557162842222443==" List-Id: --===============1660557162842222443== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello Bernhard, > On 14 Feb 2025, at 12:58, Bernhard Bitsch wrote: >=20 > Hello Michael, >=20 > thanks for your clear words. But nevertheless let be comment at some topics. >=20 > Am 14.02.2025 um 13:07 schrieb Michael Tremer: >> Hello Jon, >> It very much depends on the kind of contribution. A one-line patch obvious= ly has fewer strings attached to it than a larger patch set like this. >> However, we have outlined the process in the wiki already, starting from h= ere: >> https://www.ipfire.org/docs/devel/submit-patches >> This contains some useful pointers about the how (how do I actually make m= y changes happen, how do I build IPFire, etc), and at the bottom it contains = a lot of information about the format the changes should be submitted; split = into smaller chunks that are ideally as independent from each other so that t= hey can be individually reviewed and merged. Usually a development process ta= kes long time and we have already shipped parts of code that we will need for= certain features that are not ready yet. This is a good practice to let code= mature, especially when it is touching rather critical bits like the firewal= l and networking stacks. >> There are also some guidelines on how to write a good commit message and h= ow to use Git tags: >> https://www.ipfire.org/docs/devel/git/commit-messages >> https://www.ipfire.org/docs/devel/git/tags >> Then there is something about how to get in touch with the right person an= d legal stuff: >> https://www.ipfire.org/docs/devel/contact >> https://www.ipfire.org/docs/legal/ipca >=20 > IMO, this facts are known by Jon. He just did a big part besides this gener= al process. Including publishing his efforts to the community for review and = test. >=20 >> Finally, we have a bit of an underused roadmap section on the wiki. It wou= ld be nice if we could use that a little bit more because then it would be ea= sier for everyone to keep track of progress on certain features; people could= see what is being worked on and see if they can help development and testing= and so on: >> https://www.ipfire.org/docs/roadmap >> There is a template on how to create new pages: >> https://www.ipfire.org/docs/roadmap/template >> And this is a good example of what this could look like: >> https://www.ipfire.org/docs/roadmap/openvpn-26 >> All of these steps are coming *after* there has been some initial discussi= on about what actually has a chance to become part of the distribution. For t= hat, we do not have any specific guidelines because it is not very trivial to= write these things. There are just too many possibilities. In the past, ther= e has also been very little need for this, but that does not mean that there = have not been problems before. >> The reason why I am raising the bar this high here is simply that we have = made mistakes in the past that we don=E2=80=99t want to repeat. We have learn= ed a couple of lessons in a not very pleasant way and I under no circumstance= s would want to do this again. The objective is that we want to provide an ex= cellent distribution. Although IPFire of course has its shortcomings here and= there, it is a very stable distribution and we have a very good track record= that I want to keep. This is what our users deserve. >> In the past, people have =E2=80=9Cdropped=E2=80=9D their patches on this l= ist (or sometimes elsewhere) and we were left with dealing with the entire in= tegration only to find out later what problems there were hidden in the code.= The original author(s) had no interest in fixing any of that because it work= ed just fine for them, and so why spend any time on the problems of somebody = else? Usually I am the fallback for this and I simply don=E2=80=99t want to b= e that. I have lots of my own projects inside IPFire that are moving at snail= speed because fixing existing code usually takes priority over writing new c= ode. >> Therefore we need a commitment to sort out these problems in the first pla= ce. It has to be proven that people actually *care* about the patches that th= ey post here. I am not sure this needs writing down as this should be the sam= e policy with almost any open source project. If you contribute a line, there= is probably less maintenance required in the future, but if you contribute a= large code base, then you will need to look after it for the foreseeable fut= ure. It is your feature and not mine after all. >=20 > Being one of the first testers and revisors, I commit me to support the dev= elopment. Point. >=20 >> Then, what actually has a chance to make it into the distribution? Probabl= y not a lot. IPFire has a very clear use case. There will not be any space fo= r a desktop environment and running Chrome on it, we also don=E2=80=99t it to= make coffee and cook me a dinner. We would currently only accept things that= were actually maintainable by the current team in case a contributor moves o= n (see above), because we simply only have so much man power. We already have= a large zoo of features that are very abandoned and we are potentially looki= ng at getting rid of more things simply because we cannot support them proper= ly. Time just doesn=E2=80=99t permit. Adding something large is therefore ver= y difficult at the moment. >> I understand that in this specific case you have been trying to not involv= e the development team and I understand your motivation. But you cannot forge= t about how much time and effort a review process can take. Therefore we want= to plan things well; we want to even split it; and we want to have a convers= ation in advance so that the roadmap is clear and the actual code review idea= lly only becomes a formality. >> All of this above has been for a general case. Please read through this an= d feel free to ask any questions if something isn=E2=80=99t clear. >> To move forward with this feature, we should start by planning a roadmap. = We need to discuss what this project should cover and what it should not cove= r. I believe we don=E2=80=99t need to talk much about implementation details = because you have figured out a lot of them; we need to find what feature we w= ant to provide to our users. Are you up for that? >=20 > As a first input to the 'official' discussion, some thoughts. > - Usage of encrypted data traffic is increasing. Especially in communicatio= ns not aware to the user. These unwanted traffic is more and more difficult t= o block with IP orientated tools. Even the proxy based solution do not really= function efficient. > - To circumvent these problems a widely used approach is DNS filtering. Thi= s comes in two flavours: 1. online providers, 2. local filtering DNS servers = like PiHole. > - Solution 1 isn't wantable. Many providers store the access data, outside = the local network. > - Solution 2 installs another DNS server inside the local networks. It isn'= t trivial to ensure that all DNS traffic is handled by this server. > - Therefore it is desirable that this filtering is done in the only DNS ser= ver in a IPFire governed network, unbound on the IPFire system. The RPZ funct= ionality allows exactly this. > - There are, IMO, good filter lists outside. The updating of these lists is= part of the RPZ module. There is some effort left to investigate these and t= o develop recommendations for a good mix of appliance, we do this for the ext= ernal DNS servers already. You are thinking too much about the *how* here, but we should think of the *w= hat* first. What features do we want to offer and what do we not want to offe= r. Regarding the lists: I am running a test with one client which has about 2000= users on their network on a daily basis. There have been *huge* amounts of f= alse positives from the OISD list and it has been occasionally painful to whi= telist those. The quality of this feature stands and falls with the quality o= f the lists that we can feed into this. I have also raised before that those lists collect random stuff together and = in the end you only have one list that you can either take or leave. Coming f= rom URL filter, we have some categorisation there like =E2=80=9Cadult=E2=80= =9D, =E2=80=9Cgambling=E2=80=9D, =E2=80=9Csocial networks=E2=80=9D, and so on= which we don=E2=80=99t have here. Would that not be nice to have, too? > Regards, > Bernhard >=20 >> Best, >> -Michael >>> On 13 Feb 2025, at 21:34, jon wrote: >>>=20 >>> Michael, >>>=20 >>> I=E2=80=99ve read through your comments a few times and I ended up with m= any more questions. >>>=20 >>>=20 >>>> What I rather mean is that it has never been added as a topic on the age= nda and it has not been pitched by yourself. >>>=20 >>>=20 >>> To me the efforts to get new code accepted seem to have changed and it se= emed easier in the past. In the past I made the Core Team aware via the Dev = Mailing List and wrote a simple two or three paragraphs of "What is it? / Wha= t is the value? / Here is the code" >>>=20 >>>=20 >>> So in an effort to move forward: How exactly is something presented to t= he Core Team? >>>=20 >>> Is there an example of a recent effort that was presented that I can see = as a sample? (This type of info can also be added to the Wiki) >>>=20 >>> I understand you want it this way, but I don=E2=80=99t know what exactly = is needed. Please be specific. >>>=20 >>>=20 >>> Jon >>>=20 >>> PS - I am not ignoring your other comments, I am just trying to move forw= ard and keep things simple. >>>=20 >>>=20 >>>=20 >>>> On Feb 8, 2025, at 1:27=E2=80=AFPM, Michael Tremer wrote: >>>>=20 >>>> Hello Jon, >>>>=20 >>>> Thanks for your reply. And good that you are copying everyone into this = conversation. >>>>=20 >>>>> On 8 Feb 2025, at 18:41, jon wrote: >>>>>=20 >>>>> Michael, >>>>>=20 >>>>>> I think I have covered this all at lengths before that this project ha= s been started as a separate effort >>>>>=20 >>>>> Yes, this has been a separate effort (a very public separate effort). = Yes, as you pointed this out early on with the "proof-of-concept" and then my= request for people to help test RPZ. Nothing was hidden. >>>>>=20 >>>>> This was done because you (and maybe others) did not have the time and = I wanted to help and because I needed assistance with RPZ. I tried my best t= o do this without bothering you. >>>>=20 >>>> I don=E2=80=99t that it is accurate that nobody wanted to help on this. = The list was always open - although not every email has been replied to swift= ly it is also your responsibility to raise a question again if it was missed.= People here have open ears. >>>>=20 >>>> It was also stated on this very list on in our documentation that workin= g on something without involving the core team is a risky undertaking. Of cou= rse IPFire is free software and so everyone is free to fork if they wish to d= o so. >>>>=20 >>>>>> and as far as I am aware none of the other team members has been invol= ved. This has not been discussed either on this list, on our calls. >>>>>=20 >>>>> You were aware many steps along the way. See your email on July 28, 20= 24, August 15, 2024, September 30, 2024, December 23, 2024, and January 16. = My attempts to get the team involved were met with "things are busy" and some= times silence. (Yes, I get it, people are busy.) >>>>>=20 >>>>> You and Adolf, Leo, Erik and Bernhard have been aware since the beginni= ng. You mention you were aware of the "proof-of-concept". If you include th= ose beginning posts, since Sep 2023. >>>>=20 >>>> Yes, I am aware of a proof-of-concept that I have been running myself fo= r a long time. I am also aware of the efforts that you have been taking. >>>>=20 >>>> Yet I don=E2=80=99t think there has ever been any joint effort, or am I = seeing that wrong? >>>>=20 >>>>>> This has not been discussed . . . on our calls. >>>>>=20 >>>>> On the July 28th you stated: >>>>> "We have talked about RPZ many times on the monthly call since the URL = filter feature is falling more and more out of fashion. I think there is also= many posts about this on the forum." >>>>>=20 >>>>> Please don=E2=80=99t insult me again by stating "you know what I mean". >>>>>=20 >>>>> And it has been discussed but not documented in the Monthly Meeting not= es. >>>>=20 >>>> I am not at all insulting you. I don=E2=80=99t want to take this down to= a personal level at all. This is a public mailing list and people who read t= his don=E2=80=99t need to listen to an argument we are having. They are here = for the tech inside IPFire. >>>>=20 >>>> When I wrote that it has not been discussed that does not mean that we h= ave not been touching on the topic. We have been talking about lots of things= on the calls, the weather, politics, how our pets are. None of that makes it= to the logs. What I rather mean is that it has never been added as a topic o= n the agenda and it has not been pitched by yourself. >>>>=20 >>>>>> Instead there has been a separate conversation on the forum with the o= ccasional dip here to the list. But that was not a regular two-way conversati= on. >>>>>=20 >>>>> Regular conversation on the Dev Mailing list is many times met with sil= ence. I get it, people are busy. >>>>>=20 >>>>> And regular two-way conversation doesn=E2=80=99t happen on the list. A= t least not with me. I=E2=80=99d be happy to point out the posts that were m= et with silence. >>>>> Again, I get it, people are busy. >>>>=20 >>>> And you think my emails are not being met with silence? This has nothing= to do with this specific topic. This has something to do with how occupied p= eople are and how engaged they are on certain topics. Not everyone is involve= d in all the things and simply will ignore emails simply based on their subje= ct line. >>>>=20 >>>>> But the "dip here to the list" were my attempts to get a conversation s= tarted. As I said, many time met with silence. >>>>>=20 >>>>> The only place I was not met with silence was on the Community. You ha= ve a great group of people in the Community. It is a shame you don=E2=80=99t= want to have others help. It would reduce your workload. >>>>=20 >>>> You should stop making statements that are not true. Who doesn=E2=80=99t= want anyone to help? >>>>=20 >>>> Not having this conversation on a Saturday evening would reduce my workl= oad. At least it would free up time for something else. Helping with the thin= gs that are already on the go would reduce the workload of the entire team. S= tarting one thing at a time and finishing it is a lot better to manage than s= tarting a hundred things and not even finish one. I can tell you that I alrea= dy have a hundred things on the go. >>>>=20 >>>>>> Therefore, what am I supposed to do with this email? >>>>>=20 >>>>> To me it is beyond obvious=E2=80=A6 >>>>>=20 >>>>> If it isn=E2=80=99t what you want, then guide me with how to do this th= e correct way. And be specific. I am trying to help. I am trying to make th= ings better. I am trying to do things the right way. >>>>=20 >>>> To me it isn=E2=80=99t. This is yet another project that has been dumped= to the list like so many before and later on everyone has left to have the t= eam deal with the rest. >>>>=20 >>>> It is a huge patch set. You explained what the vision is, but that is ab= out it. There is no chance this will continue if this disagreement isn=E2=80= =99t solved first. I didn=E2=80=99t even look at the code. >>>>=20 >>>>>> I don=E2=80=99t want to merge code that I don=E2=80=99t agree with. >>>>>=20 >>>>> I asked multiple times if you "agreed with the concept" and again, met = with silence. Yes I get it, people are busy. >>>>=20 >>>> Having support for RPZ? Yes, it was definitely on the roadmap. That I ag= ree with. >>>>=20 >>>>>> So many fundamental things that I have been raising have either not be= en discussed or outright dismissed. >>>>>=20 >>>>> You mentioned this a in the past, but for some reason you do not disclo= se what I dismissed. Why do you continue to make this harder, wouldn=E2=80= =99t it not be easier to tell me what I have dismissed? >>>>>=20 >>>>> I have sent multiple emails trying to answer your concerns and comments= . On July 28, Aug 14, Aug 22, Aug 23, Sep 30, etc. >>>>>=20 >>>>> I=E2=80=99ve gone through all of the questions you asked and I cannot f= ind a "dismissed" item. >>>>=20 >>>> Maybe I need to be *more clear*. I feel humoured by this. >>>>=20 >>>> It is late on a Saturday and I want my dinner soon, but certainly I have= stated that this should never be an add-on considering it is supposed to rep= lace URL Filter. We should never allow people to add their own sources. I hav= e also stated that we cannot download any lists over HTTPS again and again an= d again. The implementation that we have here seems to exactly do that and th= erefore I think that my feedback has been dismissed entirely. >>>>=20 >>>>>> I don=E2=80=99t want to merge code that has no future inside IPFire as= there is no constructive conversation with the maintainers of it. >>>>>=20 >>>>> The maintainers of Unbound and/or RPZ? >>>>>=20 >>>>> The maintainers of Hagezi list, the threatfox list, the urlhaus list, e= tc.? >>>>>=20 >>>>> What else? The maintainers or the RPZ scripts? That is me. Let=E2=80= =99s talk! >>>>=20 >>>> You. I don=E2=80=99t care much about the providers of the lists. >>>>=20 >>>>> See, this is where it gets confusing. There are hundreds of open sourc= e packages as part of IPFire. Pick the last five years of items added to the= IPFire build. You're telling me you have "constructive conversation with th= e maintainers" of all of the added packages? >>>>=20 >>>> They publish their software and they don=E2=80=99t care whether I am pul= ling it or not. They publish it with the commitment to maintain it - sometime= s for better and sometimes for worse. >>>>=20 >>>> You care about me pulling your code and I don=E2=80=99t know whether you= would commit to maintain this. >>>>=20 >>>> These two are very different cases. >>>>=20 >>>>> Pick the IP Blocklists list (i.e., 3CORESEC, ABUSECH, DSHIELD, SPAMHAUS= , etc.) or the Suricata lists (i.e., Emergingthreats.net, Abuse.ch, etc.). = So you=E2=80=99ve have "constructive conversation with the maintainers"? >>>>=20 >>>> Yes, occasionally I have phone calls with a few of these providers. >>>>=20 >>>>>> Having been trying for a long time to make you aware of this, nothing = of this should come as a surprise. >>>>>=20 >>>>> Ha! Yes a surprise. In the beginning you seemed interested as IPFire = needed a replacement for URL Filter. You asked good questions about the list= s picked, asked for the value to the users, etc. And I answered the best I c= ould. >>>>>=20 >>>>> You even asked: =E2=80=9CWhy is this realised as an add-on and not part= of the core system?=E2=80=9D from your Jul 28, 2024 email. >>>>=20 >>>> Ah, so, why is the patch creating an add-on? Not that I am saying that w= hat I say is law, but it has not been challenged either. If my input is being= ignored, why should I put this to the top of my list of priorities? I am not= disappointed about this, just trying to be very good with my time. >>>>=20 >>>>> And on January 16, 2025 I wrote a message looking for help. And you we= re kind to respond quickly. So in three weeks time, since the kind response,= something has changed. You went from supportive to "this". >>>>>=20 >>>>> So yes, I am surprised. >>>>=20 >>>> Well, maybe I should not have replied to that email. It was clear that y= ou were on some path that was not right, but you were not interested before i= n finding the right path from the beginning. >>>>=20 >>>>>> Please consider if that can be changed and if there is a path forward = with this. >>>>>=20 >>>>> Be more specific, what has to change? What exactly did I dismiss? >>>>=20 >>>> Dismissal is just my assumption. I don=E2=80=99t know what you actually = did with my feedback. I can only see the end product that does not seem conta= in much of it. Repeatedly I have been pointing out that we should think befor= e we build. I am sure a lot of hours have now gone into some code that simply= does not satisfy me. And I am not not talking about the code itself, what it= does is what I don=E2=80=99t think is right for us. >>>>=20 >>>> The process is very clear for me that we should first of all think wheth= er we want a certain feature now. Then there should be a clear roadmap for ev= eryone to follow; tasks can be split-up as we go and hopefully then have some= thing that is maintainable, interesting for our users and even would do us pr= oud. This is how this should work. >>>>=20 >>>> So, what has to change? I don=E2=80=99t think with shouting at each othe= r, throwing patches around and making me generally unhappy is a good start. >>>>=20 >>>> -Michael >>>>=20 >>>>> Jon >>>>>=20 >>>>>=20 >>>>>=20 >>>>>> On Feb 6, 2025, at 2:13=E2=80=AFPM, Michael Tremer wrote: >>>>>>=20 >>>>>> Hello Jon, >>>>>>=20 >>>>>> Well, here we are again with another patch regarding this feature. >>>>>>=20 >>>>>> I cannot quite see from your email what the question is, but if this i= s a request to have this merged into IPFire, I am once again sorry to disappo= int you. >>>>>>=20 >>>>>> I think I have covered this all at lengths before that this project ha= s been started as a separate effort and as far as I am aware none of the othe= r team members has been involved. This has not been discussed either on this = list, on our calls. Instead there has been a separate conversation on the for= um with the occasional dip here to the list. But that was not a regular two-w= ay conversation. Therefore, what am I supposed to do with this email? >>>>>>=20 >>>>>> I don=E2=80=99t want to merge code that I don=E2=80=99t agree with. So= many fundamental things that I have been raising have either not been discus= sed or outright dismissed. >>>>>>=20 >>>>>> I don=E2=80=99t want to merge code that has no future inside IPFire as= there is no constructive conversation with the maintainers of it. >>>>>>=20 >>>>>> Having been trying for a long time to make you aware of this, nothing = of this should come as a surprise. >>>>>>=20 >>>>>> Please consider if that can be changed and if there is a path forward = with this. >>>>>>=20 >>>>>> All the best, >>>>>> -Michael >>>>>>=20 >>>>>>> On 6 Feb 2025, at 16:35, Jon Murphy wrote: >>>>>>>=20 >>>>>>> What is it? >>>>>>> Response Policy Zone (RPZ) is a mechanism to define local policies in= a >>>>>>> standardized way and load those policies from external sources. >>>>>>> Bottom line: RPZ allows admins to easily block access to websites via= DNS lookup. >>>>>>>=20 >>>>>>> RPZ can block websites via categories. Examples include: fake websit= es, annoying >>>>>>> pop-up ads, newly registered domains, DoH bypass sites, bad "host" se= rvices, >>>>>>> maliscious top level domains (e.g., *.zip, *.mov), piracy, gambling, = pornography, >>>>>>> and more. RPZ lists come from various RPZ providers and their availa= ble >>>>>>> catagories. >>>>>>>=20 >>>>>>> This RPZ add-on enables the RPZ functionality by adding a couple line= s in a >>>>>>> configuration file. This add-on simply adds configuration files and = adds >>>>>>> scripts (config, metrics and sleep) to make RPZ easier for the admin = to use. >>>>>>>=20 >>>>>>> The RPZ scripts include additional languages: German, Spanish, French= , Turkish, >>>>>>> and Italian. >>>>>>>=20 >>>>>>> RPZ itself was release in 2010 and has been part of the IPFire build = since ~2015. >>>>>>>=20 >>>>>>> Why is it needed? What is its value? >>>>>>>=20 >>>>>>> - The RPZ concept places this filtering into IPFire, our internet acc= ess >>>>>>> gateway, which is (should be) solely used as DNS source of the intern= al network. >>>>>>>=20 >>>>>>> - As most sites use HTTPS it makes it difficult to filter traffic wit= h URL >>>>>>> Filter without also properly configuring conventional (non-transparen= t) >>>>>>> mode on the proxy. RPZ is a nice replacement for the URL Filter. >>>>>>>=20 >>>>>>> - No need to install and maintain an additional device like PiHole or= AdBlock >>>>>>> browser extensions on multiple user devices. >>>>>>>=20 >>>>>>> - This is an additional layer of protection for users. Less worry som= eone will >>>>>>> click on something that gets them into trouble. And, saying this with= emphasis, >>>>>>> the ability to do it in one place! >>>>>>>=20 >>>>>>> - Blocked sites save on unneeded traffic and can lessen the threat of= malware >>>>>>> in advertisements >>>>>>>=20 >>>>>>> - Logging allows the admin to see the site blocked and take actions >>>>>>>=20 >>>>>>> - RPZ will be used at the home, home-office (work from home), schools, >>>>>>> ministerial, and at the office. Device counts are small (2-6) to med= ium (~80) >>>>>>> to mediam-large (200+). >>>>>>>=20 >>>>>>> - RPZ can block ads, popups, phishing, scammers, spyware, malware, an= noying >>>>>>> popups, NSFW links, DOH servers, and the usual internet trash. >>>>>>>=20 >>>>>>> ------------------------------ >>>>>>>=20 >>>>>>> Change Log for RPZ add-on >>>>>>>=20 >>>>>>> rpz-1.0.0-18 on 2025-02-05 >>>>>>> - Build for approval & release as IPFire add-on >>>>>>>=20 >>>>>>> --- >>>>>>>=20 >>>>>>> rpz-beta-0.1.18-18.ipfire on 2025-02-01 >>>>>>> rpz.cgi: >>>>>>> - new feature: added a mod key to force a unbound restart >>>>>>>=20 >>>>>>> rpz-config and rpz-make: >>>>>>> - new feature: added action for unbound restart `rpz-config unbound-r= estart` >>>>>>>=20 >>>>>>> rpz-metrics: >>>>>>> - simple reformatting >>>>>>> - rename far right column from "last update" to "last download" >>>>>>>=20 >>>>>>> --- >>>>>>>=20 >>>>>>> rpz-beta-0.1.17-17.ipfire on 2024-12-09 >>>>>>> rpz-make >>>>>>> - bug fix: corrected validation regex for wildcards like: `*.domain.c= om` >>>>>>>=20 >>>>>>> --- >>>>>>>=20 >>>>>>> rpz-beta-0.1.16-16.ipfire on 2024-11-18 >>>>>>> rpz-make >>>>>>> - new feature: updated validation regex >>>>>>> - bug fix: moved validation to beginning of process. Now we validate= before >>>>>>> creating config files. >>>>>>>=20 >>>>>>> rpz.cgi: >>>>>>> - new feature: use CSS color variables of the main ipfire theme >>>>>>> - bug fix: empty zonefile remarks were stored as =E2=80=9Cundef=E2=80= =9D and caused a warning >>>>>>> - bug fix: HTML textarea removes the first empty line in a custom list >>>>>>> - thank you Leo! >>>>>>>=20 >>>>>>> --- >>>>>>>=20 >>>>>>> rpz-beta-0.1.15-15.ipfire on 2024-11-04 >>>>>>> rpz.cgi: >>>>>>> - new feature: added new language file for Turkish (thank you Peppe) >>>>>>>=20 >>>>>>> rpz-make >>>>>>> - bug fix: corrected empty allow/block list issue. An empty allow/bl= ock list >>>>>>> will now remove contents of allow/block.rpz files and remove unneeded >>>>>>> allow/block.conf file. (thank you iptom) >>>>>>>=20 >>>>>>> --- >>>>>>>=20 >>>>>>> rpz-beta-0.1.14-14.ipfire on 2024-10-29 >>>>>>> rpz-config: >>>>>>> - bug fix: correct missing rpz extension. `rpz-config list` displayed= URL >>>>>>> incorrectly (thank you Bernhard) >>>>>>>=20 >>>>>>> rpz.cgi: >>>>>>> - bug fix: remove extra `"` in language files (thank you Bernhard) >>>>>>> - new feature: slightly dim "apply" button when not enabled >>>>>>>=20 >>>>>>> --- >>>>>>>=20 >>>>>>> rpz-beta-0.1.13-13.ipfire on 2024-10-27 >>>>>>> - skipped >>>>>>>=20 >>>>>>> --- >>>>>>>=20 >>>>>>> rpz-beta-0.1.12-12.ipfire on 2024-10-21 >>>>>>> rpz.cgi: >>>>>>> - new feature: added new language file for French (thank you gw-ipfi= re) >>>>>>>=20 >>>>>>> --- >>>>>>>=20 >>>>>>> rpz-beta-0.1.11-11.ipfire on 2024-10-18 >>>>>>> rpz.cgi: >>>>>>> - new feature: added new language file for Italian (thank you umberto) >>>>>>> - new feature: added new language file for Spanish (thank you Roberto) >>>>>>>=20 >>>>>>> --- >>>>>>>=20 >>>>>>> rpz-beta-0.1.10-10.ipfire on 2024-10-15 >>>>>>> rpz-make: >>>>>>> - bug fix: corrected validation error for a custom list entry (thank = you siosios) >>>>>>> - e.g., `*.cloudflare-dns.com` >>>>>>>=20 >>>>>>> install.sh: >>>>>>> - bug fix: add chown to correct user created files >>>>>>>=20 >>>>>>> update.sh: >>>>>>> - bug fix: add chown to correct user created files (thank you siosios) >>>>>>>=20 >>>>>>> --- >>>>>>>=20 >>>>>>> rpz-beta-0.1.9-9.ipfire on 2024-10-08 >>>>>>> rpz.cgi: >>>>>>> - new feature: added new language file for German (thank you Leo) >>>>>>> - bug fix: add missing "rpz exitcode 110" >>>>>>> - bug fix: corrected missing RPZ menu item at menu > IPFire >>>>>>>=20 >>>>>>> --- >>>>>>>=20 >>>>>>> rpz-beta-0.1.8-8.ipfire on 2024-10-04 >>>>>>> - skipped >>>>>>>=20 >>>>>>> --- >>>>>>>=20 >>>>>>> rpz-beta-0.1.7-7.ipfire on 2024-10-03 >>>>>>> All: >>>>>>> - new feature: includes beta version numbers for pakfire package, >>>>>>> instead of only `rpz-1.0.0-1.ipfire`, for each release. >>>>>>>=20 >>>>>>> rpz.cgi: >>>>>>> - new feature: added new WebGUI at `rpz.cgi` >>>>>>> - a BIG thank you to Leo Hofmann for all of his work creating the web= gui!! >>>>>>> - bug fix: corrected missing RPZ menu item at menu > IPFire >>>>>>>=20 >>>>>>> rpz-make: >>>>>>> - new feature: validate entries in allowlist and blocklist >>>>>>> - new feature: add "no-reload" option for WebGUI >>>>>>>=20 >>>>>>> rpz-metrics: >>>>>>> - new feature: info can be sorted by name, by hit count, by line coun= t, by >>>>>>> "enabled" list or all lists >>>>>>>=20 >>>>>>> backups: >>>>>>> - bug fix: include all files in `/var/ipfire/dns/rpz` directory in ba= ckup >>>>>>>=20 >>>>>>> update.sh: >>>>>>> - bug fix: corrected ownership for `/var/ipfire/dns/rpz` directory du= ring an >>>>>>> update >>>>>>>=20 >>>>>>> Build: >>>>>>> - bug fix: `block.rpz.conf` and `block.rpz` from build. Files to be = created >>>>>>> by `rpz-make` >>>>>>>=20 >>>>>>> WebGUI and German language file >>>>>>> Contribution-by: Leo-Andres Hofmann >>>>>>>=20 >>>>>>> Spanish language file >>>>>>> Contribution-by: Roberto Pe=C3=B1a >>>>>>>=20 >>>>>>> Italian language file >>>>>>> Contribution-by: Umberto Parma >>>>>>>=20 >>>>>>> French language file >>>>>>> Contribution-by: gw-ipfire >>>>>>>=20 >>>>>>> Turkish language file >>>>>>> Contribution-by: Peppe Tech >>>>>>>=20 >>>>>>> Contribution-by: Bernhard Bitsch >>>>>>> Contribution-by: Erik Kapfer >>>>>>> Signed-off-by: Jon Murphy >>>>>> --- >>>>>>> config/backup/includes/rpz | 4 + >>>>>>> config/cfgroot/manualpages | 1 + >>>>>>> config/menu/EX-rpz.menu | 6 + >>>>>>> config/rootfiles/common/configroot | 1 + >>>>>>> config/rootfiles/common/web-user-interface | 1 + >>>>>>> config/rootfiles/packages/rpz | 20 + >>>>>>> config/rpz/00-rpz.conf | 10 + >>>>>>> config/rpz/rpz-config | 130 +++ >>>>>>> config/rpz/rpz-functions | 85 ++ >>>>>>> config/rpz/rpz-make | 203 +++++ >>>>>>> config/rpz/rpz-metrics | 170 ++++ >>>>>>> config/rpz/rpz-sleep | 58 ++ >>>>>>> config/rpz/rpz.de.pl | 30 + >>>>>>> config/rpz/rpz.en.pl | 30 + >>>>>>> config/rpz/rpz.es.pl | 30 + >>>>>>> config/rpz/rpz.fr.pl | 30 + >>>>>>> config/rpz/rpz.it.pl | 30 + >>>>>>> config/rpz/rpz.tr.pl | 30 + >>>>>>> html/cgi-bin/rpz.cgi | 923 +++++++++++++++++++++ >>>>>>> lfs/rpz | 96 +++ >>>>>>> make.sh | 3 +- >>>>>>> src/paks/rpz/install.sh | 36 + >>>>>>> src/paks/rpz/uninstall.sh | 38 + >>>>>>> src/paks/rpz/update.sh | 52 ++ >>>>>>> 24 files changed, 2016 insertions(+), 1 deletion(-) >>>>>>> create mode 100644 config/backup/includes/rpz >>>>>>> create mode 100644 config/menu/EX-rpz.menu >>>>>>> create mode 100644 config/rootfiles/packages/rpz >>>>>>> create mode 100644 config/rpz/00-rpz.conf >>>>>>> create mode 100644 config/rpz/rpz-config >>>>>>> create mode 100644 config/rpz/rpz-functions >>>>>>> create mode 100644 config/rpz/rpz-make >>>>>>> create mode 100755 config/rpz/rpz-metrics >>>>>>> create mode 100755 config/rpz/rpz-sleep >>>>>>> create mode 100644 config/rpz/rpz.de.pl >>>>>>> create mode 100644 config/rpz/rpz.en.pl >>>>>>> create mode 100644 config/rpz/rpz.es.pl >>>>>>> create mode 100644 config/rpz/rpz.fr.pl >>>>>>> create mode 100644 config/rpz/rpz.it.pl >>>>>>> create mode 100644 config/rpz/rpz.tr.pl >>>>>>> create mode 100644 html/cgi-bin/rpz.cgi >>>>>>> create mode 100644 lfs/rpz >>>>>>> create mode 100644 src/paks/rpz/install.sh >>>>>>> create mode 100644 src/paks/rpz/uninstall.sh >>>>>>> create mode 100644 src/paks/rpz/update.sh >>>>>>>=20 >>>>>>> diff --git a/config/backup/includes/rpz b/config/backup/includes/rpz >>>>>>> new file mode 100644 >>>>>>> index 000000000..36513e494 >>>>>>> --- /dev/null >>>>>>> +++ b/config/backup/includes/rpz >>>>>>> @@ -0,0 +1,4 @@ >>>>>>> +/var/ipfire/dns/rpz/* >>>>>>> +/etc/unbound/zonefiles/allow.rpz >>>>>>> +/etc/unbound/zonefiles/block.rpz >>>>>>> +/etc/unbound/local.d/*rpz.conf >>>>>>> diff --git a/config/cfgroot/manualpages b/config/cfgroot/manualpages >>>>>>> index 1f7e01efc..d3a48c633 100644 >>>>>>> --- a/config/cfgroot/manualpages >>>>>>> +++ b/config/cfgroot/manualpages >>>>>>> @@ -70,6 +70,7 @@ pakfire.cgi=3Dconfiguration/ipfire/pakfire >>>>>>> wlanap.cgi=3Daddons/wireless >>>>>>> tor.cgi=3Daddons/tor >>>>>>> samba.cgi=3Daddons/samba >>>>>>> +rpz.cgi=3Daddons/rpz >>>>>>>=20 >>>>>>> # Logs menu >>>>>>> logs.cgi/summary.dat=3Dconfiguration/logs/summary >>>>>>> diff --git a/config/menu/EX-rpz.menu b/config/menu/EX-rpz.menu >>>>>>> new file mode 100644 >>>>>>> index 000000000..2f4daf410 >>>>>>> --- /dev/null >>>>>>> +++ b/config/menu/EX-rpz.menu >>>>>>> @@ -0,0 +1,6 @@ >>>>>>> +$subipfire->{'20.rpz'} =3D { >>>>>>> + 'caption' =3D> $Lang::tr{'rpz'}, >>>>>>> + 'uri' =3D> '/cgi-bin/rpz.cgi', >>>>>>> + 'title' =3D> "RPZ", >>>>>>> + 'enabled' =3D> 1, >>>>>>> +}; >>>>>>> diff --git a/config/rootfiles/common/configroot b/config/rootfiles/co= mmon/configroot >>>>>>> index 9839eee45..b30d6aae4 100644 >>>>>>> --- a/config/rootfiles/common/configroot >>>>>>> +++ b/config/rootfiles/common/configroot >>>>>>> @@ -120,6 +120,7 @@ var/ipfire/menu.d/70-log.menu >>>>>>> #var/ipfire/menu.d/EX-apcupsd.menu >>>>>>> #var/ipfire/menu.d/EX-guardian.menu >>>>>>> #var/ipfire/menu.d/EX-mympd.menu >>>>>>> +#var/ipfire/menu.d/EX-rpz.menu >>>>>>> #var/ipfire/menu.d/EX-samba.menu >>>>>>> #var/ipfire/menu.d/EX-tor.menu >>>>>>> #var/ipfire/menu.d/EX-transmission.menu >>>>>>> diff --git a/config/rootfiles/common/web-user-interface b/config/root= files/common/web-user-interface >>>>>>> index 816241dae..e00464076 100644 >>>>>>> --- a/config/rootfiles/common/web-user-interface >>>>>>> +++ b/config/rootfiles/common/web-user-interface >>>>>>> @@ -69,6 +69,7 @@ srv/web/ipfire/cgi-bin/proxy.cgi >>>>>>> srv/web/ipfire/cgi-bin/qos.cgi >>>>>>> srv/web/ipfire/cgi-bin/remote.cgi >>>>>>> srv/web/ipfire/cgi-bin/routing.cgi >>>>>>> +#srv/web/ipfire/cgi-bin/rpz.cgi >>>>>>> #srv/web/ipfire/cgi-bin/samba.cgi >>>>>>> srv/web/ipfire/cgi-bin/services.cgi >>>>>>> srv/web/ipfire/cgi-bin/shutdown.cgi >>>>>>> diff --git a/config/rootfiles/packages/rpz b/config/rootfiles/package= s/rpz >>>>>>> new file mode 100644 >>>>>>> index 000000000..1c8663049 >>>>>>> --- /dev/null >>>>>>> +++ b/config/rootfiles/packages/rpz >>>>>>> @@ -0,0 +1,20 @@ >>>>>>> +etc/unbound/local.d/00-rpz.conf >>>>>>> +etc/unbound/zonefiles >>>>>>> +etc/unbound/zonefiles/allow.rpz >>>>>>> +usr/sbin/rpz-config >>>>>>> +usr/sbin/rpz-functions >>>>>>> +usr/sbin/rpz-make >>>>>>> +usr/sbin/rpz-metrics >>>>>>> +usr/sbin/rpz-sleep >>>>>>> +var/ipfire/addon-lang/rpz.de.pl >>>>>>> +var/ipfire/addon-lang/rpz.en.pl >>>>>>> +var/ipfire/addon-lang/rpz.es.pl >>>>>>> +var/ipfire/addon-lang/rpz.fr.pl >>>>>>> +var/ipfire/addon-lang/rpz.it.pl >>>>>>> +var/ipfire/addon-lang/rpz.tr.pl >>>>>>> +var/ipfire/backup/addons/includes/rpz >>>>>>> +var/ipfire/dns/rpz >>>>>>> +var/ipfire/dns/rpz/allowlist >>>>>>> +var/ipfire/dns/rpz/blocklist >>>>>>> +var/ipfire/menu.d/EX-rpz.menu >>>>>>> +srv/web/ipfire/cgi-bin/rpz.cgi >>>>>>> diff --git a/config/rpz/00-rpz.conf b/config/rpz/00-rpz.conf >>>>>>> new file mode 100644 >>>>>>> index 000000000..f005a4f2e >>>>>>> --- /dev/null >>>>>>> +++ b/config/rpz/00-rpz.conf >>>>>>> @@ -0,0 +1,10 @@ >>>>>>> +server: >>>>>>> + module-config: "respip validator iterator" >>>>>>> + >>>>>>> +rpz: >>>>>>> + name: allow.rpz >>>>>>> + zonefile: /etc/unbound/zonefiles/allow.rpz >>>>>>> + rpz-action-override: passthru >>>>>>> + rpz-log: yes >>>>>>> + rpz-log-name: allow >>>>>>> + rpz-signal-nxdomain-ra: yes >>>>>>> diff --git a/config/rpz/rpz-config b/config/rpz/rpz-config >>>>>>> new file mode 100644 >>>>>>> index 000000000..c72d50f9b >>>>>>> --- /dev/null >>>>>>> +++ b/config/rpz/rpz-config >>>>>>> @@ -0,0 +1,130 @@ >>>>>>> +#!/bin/bash >>>>>>> +####################################################################= ########### >>>>>>> +# = # >>>>>>> +# IPFire.org - A linux based firewall = # >>>>>>> +# Copyright (C) 2024-2025 IPFire Team = # >>>>>>> +# = # >>>>>>> +# This program is free software: you can redistribute it and/or mod= ify # >>>>>>> +# it under the terms of the GNU General Public License as published= by # >>>>>>> +# the Free Software Foundation, either version 3 of the License, or= # >>>>>>> +# (at your option) any later version. = # >>>>>>> +# = # >>>>>>> +# This program is distributed in the hope that it will be useful, = # >>>>>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of = # >>>>>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the = # >>>>>>> +# GNU General Public License for more details. = # >>>>>>> +# = # >>>>>>> +# You should have received a copy of the GNU General Public License= # >>>>>>> +# along with this program. If not, see . # >>>>>>> +# = # >>>>>>> +####################################################################= ########### >>>>>>> + >>>>>>> +version=3D"2025-01-11 - v44" >>>>>>> + >>>>>>> +############### Functions ############### >>>>>>> + >>>>>>> +source /usr/sbin/rpz-functions >>>>>>> + >>>>>>> +############### Main ############### >>>>>>> + >>>>>>> +tagName=3D"unbound" >>>>>>> + >>>>>>> +rpzAction=3D"${1}" # input RPZ action >>>>>>> +rpzName=3D"${2}" # input RPZ name >>>>>>> +rpzURL=3D"${3}" # input RPZ URL >>>>>>> +rpzOption1=3D"${4}" # input RPZ option #1 >>>>>>> +rpzOption2=3D"${5}" # input RPZ option #2 >>>>>>> + >>>>>>> +rpzConfig=3D"/etc/unbound/local.d/${rpzName}.rpz.conf" # output = zone conf file >>>>>>> +rpzFile=3D"/etc/unbound/zonefiles/${rpzName}.rpz" # output = for RPZ file >>>>>>> + >>>>>>> +rpzLog=3D"yes" # log default is yes >>>>>>> +ucReload=3D"yes" # reload default is yes >>>>>>> + >>>>>>> +while [[ $# -gt 0 ]] ; do >>>>>>> + case "$1" in >>>>>>> + --no-log ) rpzLog=3D"no" ;; >>>>>>> + --no-reload ) ucReload=3D"no" ; checkConf=3D"no" ;; >>>>>>> + esac >>>>>>> + shift # Shift after checking all the cases to get next opt= ion >>>>>>> +done >>>>>>> + >>>>>>> +case "${rpzAction}" in >>>>>>> + # add new rpz list >>>>>>> + add ) >>>>>>> + check_name "${rpzName}" # is this a valid name? >>>>>>> + # does this config already exist? If yes, then exit >>>>>>> + if [[ -f "${rpzConfig}" ]] ; then >>>>>>> + msg_log "error: rpz: duplicate - ${rpzConfig} already ex= ists. exit" >>>>>>> + exit 104 >>>>>>> + fi >>>>>>> + >>>>>>> + # is this a valid URL? >>>>>>> + regex=3D'^https://[-[:alnum:]\+&@#/%?=3D~_|!:,.;]*[-[:alnum:= ]\+&@#/%=3D~_|]' >>>>>>> + if ! [[ "${rpzURL}" =3D~ $regex ]] ; then >>>>>>> + msg_log "error: rpz: the URL is not valid: \"${rpzURL}\"= . exit." >>>>>>> + exit 105 >>>>>>> + fi >>>>>>> + >>>>>>> + # create the zone config file >>>>>>> + { >>>>>>> + echo "rpz:" >>>>>>> + echo " name: ${rpzName}.rpz" >>>>>>> + echo " zonefile: ${rpzFile}" >>>>>>> + echo " url: ${rpzURL}" >>>>>>> + echo " rpz-action-override: nxdomain" >>>>>>> + echo " rpz-log: ${rpzLog}" >>>>>>> + echo " rpz-log-name: ${rpzName}" >>>>>>> + echo " rpz-signal-nxdomain-ra: yes" >>>>>>> + } > "${rpzConfig}" >>>>>>> + >>>>>>> + # set-up zonefile >>>>>>> + # create an empty rpz file if it does not exist >>>>>>> + if [[ ! -f "${rpzFile}" ]] ; then >>>>>>> + touch "${rpzFile}" >>>>>>> + # unbound requires these settings for rpz files >>>>>>> + set_permissions "${rpzFile}" "${rpzConfig}" >>>>>>> + fi >>>>>>> + ;; >>>>>>> + >>>>>>> + # trash config file & rpz file >>>>>>> + remove ) >>>>>>> + if ! [[ -f "${rpzConfig}" ]] ; then >>>>>>> + msg_log "error: rpz: cannot remove ${rpzConfig}, does no= t exist. exit" >>>>>>> + exit 106 >>>>>>> + fi >>>>>>> + >>>>>>> + msg_log "info: rpz: remove config file & rpz file \"${rpzNam= e}\"" >>>>>>> + rm "${rpzConfig}" >>>>>>> + rm "${rpzFile}" >>>>>>> + ;; >>>>>>> + >>>>>>> + reload ) >>>>>>> + check_unbound_conf "${checkConf}" >>>>>>> + ;; >>>>>>> + >>>>>>> + list ) >>>>>>> + awk -F':' '/^\s*name:/{ gsub(/[[:blank:]]|\.rpz/, "",$2) ; N= AME=3D$2 } \ >>>>>>> + /^\s*url:/{ gsub(/[[:blank:]]/, "") ; print NAME"=3D"$2"= :"$3} ' \ >>>>>>> + /etc/unbound/local.d/*rpz.conf >>>>>>> + exit >>>>>>> + ;; >>>>>>> + >>>>>>> + unbound-restart ) >>>>>>> + check_unbound_conf "${checkConf}" >>>>>>> + unbound_restart >>>>>>> + exit >>>>>>> + ;; >>>>>>> + >>>>>>> + * ) >>>>>>> + msg_log "error: rpz: missing or incorrect parameter" >>>>>>> + printf "Usage: $(basename "$0")