From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.haj.ipfire.org (localhost [IPv6:::1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4dfw0b0Dzlz2y1D for ; Mon, 29 Dec 2025 12:05:51 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature ECDSA (secp384r1 raw public key) server-digest SHA384 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail01.haj.ipfire.org", Issuer "R12" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4dfw0W3qWJz2xQW for ; Mon, 29 Dec 2025 12:05:47 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail01.ipfire.org (Postfix) with ESMTPSA id 4dfw0V0v0szlQ for ; Mon, 29 Dec 2025 12:05:46 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1767009946; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=qYCuz70H34GYuUr+xKaouH5uXtr1ambk6WsOySwmC34=; b=dtWRMdk39CPoUgEnby4PDTR8ibNP8YfdSz4GZK9G5Ff9uOYSN5D0BaOB+hrhiYnrvswBb1 IN/6BGTbV+fA2gCg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1767009946; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=qYCuz70H34GYuUr+xKaouH5uXtr1ambk6WsOySwmC34=; b=nTg35/Y4iJpslJ4700S2wNTB5WPoZIjM1kswcTGfJeJNpcuGtFJvOJYO1OuPXBUu1HpyNC n/E+KURjV34luiRCVorjC9DBowt2Q/69dkKg2lbBNKlENBd+jFVqcY2wAhdHB8oyyN+Uwb U0mxKtNtILtLS3l9DOnj85MoFFFHdR6AfYMjJGi5wxP4TO+zhiIeu5BTRCY8QAMISHIViB iNv1XQq2/McQ1HiRfhjAsgy/diTKZtBkKrLYZhvOB7Btp78u2/A4p4wcMQpR/gaz1lnhg6 Ox7vJbBEGH7LvurfHBrPlMjXpLuQj+c+oT94eszjCcf/ssFkWFgTHxyvFGsUGw== From: Michael Tremer Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Precedence: list List-Id: List-Subscribe: , List-Unsubscribe: , List-Post: List-Help: Sender: Mail-Followup-To: Mime-Version: 1.0 Subject: Let's launch our own blocklists... Message-Id: <7EF00B55-81C0-493F-A70F-B1DDD45363E2@ipfire.org> Date: Mon, 29 Dec 2025 12:05:45 +0000 To: Development Hello everyone, I hope everyone had a great Christmas and a couple of quiet days to = relax from all the stress that was the year 2025. Having a couple of quieter days, I have been working on a new, little = (hopefully) side project that has probably been high up on our radar = since the Shalla list has shut down in 2020, or maybe even earlier. The = goal of the project is to provide good lists with categories of domain = names which are usually used to block access to these domains. I simply call this IPFire DNSBL which is short for IPFire DNS = Blocklists. How did we get here? As stated before, the URL filter feature in IPFire has the problem that = there are not many good blocklists available any more. There used to be = a couple more - most famously the Shalla list - but we are now down to a = single list from the University of Toulouse. It is a great list, but it = is not always the best fit for all users. Then there has been talk about whether we could implement more blocking = features into IPFire that don=E2=80=99t involve the proxy. Most famously = blocking over DNS. The problem here remains a the blocking feature is = only as good as the data that is fed into it. Some people have been = putting forward a number of lists that were suitable for them, but they = would not have replaced the blocking functionality as we know it. Their = aim is to provide =E2=80=9Cone list for everything=E2=80=9D but that is = not what people usually want. It is targeted at a classic home user and = the only separation that is being made is any adult/porn/NSFW content = which usually is put into a separate list. It would have been technically possible to include these lists and let = the users decide, but that is not the aim of IPFire. We want to do the = job for the user so that their job is getting easier. Including obscure = lists that don=E2=80=99t have a clear outline of what they actually want = to block (=E2=80=9Cbad content=E2=80=9D is not a category) and passing = the burden of figuring out whether they need the =E2=80=9CLight=E2=80=9D, = =E2=80=9CNormal=E2=80=9D, =E2=80=9CPro=E2=80=9D, =E2=80=9CPro++=E2=80=9D, = =E2=80=9CUltimate=E2=80=9D or even a =E2=80=9CVenti=E2=80=9D list with = cream on top is really not going to work. It is all confusing and will = lead to a bad user experience. An even bigger problem that is however completely impossible to solve is = bad licensing of these lists. A user has asked the publisher of the = HaGeZi list whether they could be included in IPFire and under what = terms. The response was that the list is available under the terms of = the GNU General Public License v3, but that does not seem to be true. = The list contains data from various sources. Many of them are licensed = under incompatible licenses (CC BY-SA 4.0, MPL, Apache2, =E2=80=A6) and = unless there is a non-public agreement that this data may be = redistributed, there is a huge legal issue here. We would expose our = users to potential copyright infringement which we cannot do under any = circumstances. Furthermore many lists are available under a = non-commercial license which excludes them from being used in any kind = of business. Plenty of IPFire systems are running in businesses, if not = even the vast majority. In short, these lists are completely unusable for us. Apart from HaGeZi, = I consider OISD to have the same problem. Enough about all the things that are bad. Let=E2=80=99s talk about the = new, good things: Many blacklists on the internet are an amalgamation of other lists. = These lists vary in quality with some of them being not that good and = without a clear focus and others being excellent data. Since we don=E2=80=99= t have the man power to start from scratch, I felt that we can copy the = concept that HaGeZi and OISD have started and simply create a new list = that is based on other lists at the beginning to have a good starting = point. That way, we have much better control over what is going on these = lists and we can shape and mould them as we need them. Most importantly, = we don=E2=80=99t create a single lists, but many lists that have a clear = focus and allow users to choose what they want to block and what not. So the current experimental stage that I am in has these lists: * Ads * Dating * DoH * Gambling * Malware * Porn * Social * Violence The categories have been determined by what source lists we have = available with good data and are compatible with our chosen license CC = BY-SA 4.0. This is the same license that we are using for the IPFire = Location database, too. The main use-cases for any kind of blocking are to comply with legal = requirements in networks with children (i.e. schools) to remove any kind = of pornographic content, sometimes block social media as well. Gambling = and violence are commonly blocked, too. Even more common would be = filtering advertising and any malicious content. The latter is especially difficult because so many source lists throw = phishing, spyware, malvertising, tracking and other things into the same = bucket. Here this is currently all in the malware list which has = therefore become quite large. I am not sure whether this will stay like = this in the future or if we will have to make some adjustments, but that = is exactly why this is now entering some larger testing. What has been built so far? In order to put these lists together = properly, track any data about where it is coming from, I have built a = tool in Python available here: https://git.ipfire.org/?p=3Ddnsbl.git;a=3Dsummary This tool will automatically update all lists once an hour if there have = been any changes and export them in various formats. The exported lists = are available for download here: https://dnsbl.ipfire.org/lists/ If you download and open any of the files, you will see a large header = that includes copyright information and lists all sources that have been = used to create the individual lists. This way we ensure maximum = transparency, comply with the terms of the individual licenses of the = source lists and give credit to the people who help us to put together = the most perfect list for our users. I would like this to become a project that is not only being used in = IPFire. We can and will be compatible with other solutions like AdGuard, = PiHole so that people can use our lists if they would like to even = though they are not using IPFire. Hopefully, these users will also feed = back to us so that we can improve our lists over time and make them one = of the best options out there. All lists are available as a simple text file that lists the domains. = Then there is a hosts file available as well as a DNS zone file and an = RPZ file. Each list is individually available to be used in squidGuard = and there is a larger tarball available with all lists that can be used = in IPFire=E2=80=99s URL Filter. I am planning to add Suricata/Snort = signatures whenever I have time to do so. Even though it is not a good = idea to filter pornographic content this way, I suppose that catching = malware and blocking DoH are good use-cases for an IPS. Time will = tell=E2=80=A6 As a start, we will make these lists available in IPFire=E2=80=99s URL = Filter and collect some feedback about how we are doing. Afterwards, we = can see where else we can take this project. If you want to enable this on your system, simply add the URL to your = autoupdate.urls file like here: = https://git.ipfire.org/?p=3Dpeople/ms/ipfire-2.x.git;a=3Dcommitdiff;h=3Dbf= 675bb937faa7617474b3cc84435af3b1f7f45f This email is just a brain dump from me to this list. I would be happy = to answer any questions about implementation details, etc. if people are = interested. Right now, this email is long enough already=E2=80=A6 All the best, -Michael=