public inbox for development@lists.ipfire.org
 help / color / mirror / Atom feed
* Feedback about the DNS FW
@ 2026-04-29 17:43 Bernhard Bitsch
  2026-04-29 20:09 ` Michael Tremer
  0 siblings, 1 reply; 6+ messages in thread
From: Bernhard Bitsch @ 2026-04-29 17:43 UTC (permalink / raw)
  To: IPFire Development

Hi,

after using the new DNS FW ( congrats to this nice feature! ), I found 
some issues.
- Each 'save' in WUI page increases the memory consumption. Even if 
nothing changed. A restart of unbound frees this huge allocation.
- Knowing from using Jon's RPZ prototype, I checked whether a single 
reload ( used in DNS FW? ) propagates the changes, new list and/or 
allow/deny entries, really. I found cases where this isn't true. A 
unbound restart yielded the right behaviour.


I must apologize not to have tested the release. But I haven't the 
equipment, yet ( only one production system ).

Regards,
Bernhard


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Feedback about the DNS FW
  2026-04-29 17:43 Feedback about the DNS FW Bernhard Bitsch
@ 2026-04-29 20:09 ` Michael Tremer
  2026-04-29 22:19   ` Bernhard Bitsch
  0 siblings, 1 reply; 6+ messages in thread
From: Michael Tremer @ 2026-04-29 20:09 UTC (permalink / raw)
  To: Bernhard Bitsch; +Cc: IPFire Development

Hello Bernhard,

> On 29 Apr 2026, at 18:43, Bernhard Bitsch <bbitsch@ipfire.org> wrote:
> 
> Hi,
> 
> after using the new DNS FW ( congrats to this nice feature! ), I found some issues.

Thanks. I believe that the entire feature has received very poor testing. Considering how many people have stated how important it is to them, really critical issues have been reported very late in the release process which indicates that the feature has not been tested, or if people found those bugs, they have not been reported.

I have to say that I am very disappointed about this. But it has nothing to do with your question.

> - Each 'save' in WUI page increases the memory consumption. Even if nothing changed. A restart of unbound frees this huge allocation.

Yes, this is known. It is a problem inside Unbound and there is nothing we can do about it. I did not report it to Unbound, but I am sure they should be made aware.

Unbound in general is using a lot of memory when it is downloading the lists. I have imported the lists into PowerDNS Recursor and it raises its memory consumption by about ~300 MiB when Unbound is going into 1.6-1.7 GiB.

For now we are stuck with Unbound, but it has always been giving us a lot of trouble.

> - Knowing from using Jon's RPZ prototype, I checked whether a single reload ( used in DNS FW? ) propagates the changes, new list and/or allow/deny entries, really. I found cases where this isn't true. A unbound restart yielded the right behaviour.
> 
> 
> I must apologize not to have tested the release. But I haven't the equipment, yet ( only one production system ).
> 
> Regards,
> Bernhard
> 



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Feedback about the DNS FW
  2026-04-29 20:09 ` Michael Tremer
@ 2026-04-29 22:19   ` Bernhard Bitsch
  2026-04-30 10:06     ` Michael Tremer
  0 siblings, 1 reply; 6+ messages in thread
From: Bernhard Bitsch @ 2026-04-29 22:19 UTC (permalink / raw)
  To: Michael Tremer; +Cc: IPFire Development

Hello Michael,

Am 29.04.2026 um 22:09 schrieb Michael Tremer:
> Hello Bernhard,
> 
>> On 29 Apr 2026, at 18:43, Bernhard Bitsch <bbitsch@ipfire.org> wrote:
>>
>> Hi,
>>
>> after using the new DNS FW ( congrats to this nice feature! ), I found some issues.
> 
> Thanks. I believe that the entire feature has received very poor testing. Considering how many people have stated how important it is to them, really critical issues have been reported very late in the release process which indicates that the feature has not been tested, or if people found those bugs, they have not been reported.
> 
> I have to say that I am very disappointed about this. But it has nothing to do with your question.
> 
>> - Each 'save' in WUI page increases the memory consumption. Even if nothing changed. A restart of unbound frees this huge allocation.
> 
> Yes, this is known. It is a problem inside Unbound and there is nothing we can do about it. I did not report it to Unbound, but I am sure they should be made aware.
> 
> Unbound in general is using a lot of memory when it is downloading the lists. I have imported the lists into PowerDNS Recursor and it raises its memory consumption by about ~300 MiB when Unbound is going into 1.6-1.7 GiB.
> 

Some more investigation in unbound docs about the operation fast_reload 
("This command is experimental at this time.") and some experiments I 
can state, that the fast_reload doesn't free the copy of the state. This 
gives the increase in memory consumption.
The runtime for the 'reload_keep_cache' operation is about the same as 
for 'fast_reload' ( just my feeling ). But the memory load doesn't 
increase ( measured just with the WUI memory stats ).

> For now we are stuck with Unbound, but it has always been giving us a lot of trouble.
> 

Does this mean we are switching to PowerDNs? But we should have a stable 
system meantime. What about going back to the 'reload_keep_cache' operation?

Regards,
Bernhard
>> - Knowing from using Jon's RPZ prototype, I checked whether a single reload ( used in DNS FW? ) propagates the changes, new list and/or allow/deny entries, really. I found cases where this isn't true. A unbound restart yielded the right behaviour.
>>
>>
>> I must apologize not to have tested the release. But I haven't the equipment, yet ( only one production system ).
>>
>> Regards,
>> Bernhard
>>
> 



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Feedback about the DNS FW
  2026-04-29 22:19   ` Bernhard Bitsch
@ 2026-04-30 10:06     ` Michael Tremer
       [not found]       ` <210f08e2-c1ed-46c9-9e51-65ec200fe487@Canary>
  0 siblings, 1 reply; 6+ messages in thread
From: Michael Tremer @ 2026-04-30 10:06 UTC (permalink / raw)
  To: Bernhard Bitsch; +Cc: IPFire Development

Hello Bernhard,

> On 29 Apr 2026, at 23:19, Bernhard Bitsch <bbitsch@ipfire.org> wrote:
> 
> Hello Michael,
> 
> Am 29.04.2026 um 22:09 schrieb Michael Tremer:
>> Hello Bernhard,
>>> On 29 Apr 2026, at 18:43, Bernhard Bitsch <bbitsch@ipfire.org> wrote:
>>> 
>>> Hi,
>>> 
>>> after using the new DNS FW ( congrats to this nice feature! ), I found some issues.
>> Thanks. I believe that the entire feature has received very poor testing. Considering how many people have stated how important it is to them, really critical issues have been reported very late in the release process which indicates that the feature has not been tested, or if people found those bugs, they have not been reported.
>> I have to say that I am very disappointed about this. But it has nothing to do with your question.
>>> - Each 'save' in WUI page increases the memory consumption. Even if nothing changed. A restart of unbound frees this huge allocation.
>> Yes, this is known. It is a problem inside Unbound and there is nothing we can do about it. I did not report it to Unbound, but I am sure they should be made aware.
>> Unbound in general is using a lot of memory when it is downloading the lists. I have imported the lists into PowerDNS Recursor and it raises its memory consumption by about ~300 MiB when Unbound is going into 1.6-1.7 GiB.
> 
> Some more investigation in unbound docs about the operation fast_reload ("This command is experimental at this time.") and some experiments I can state, that the fast_reload doesn't free the copy of the state. This gives the increase in memory consumption.
> The runtime for the 'reload_keep_cache' operation is about the same as for 'fast_reload' ( just my feeling ). But the memory load doesn't increase ( measured just with the WUI memory stats ).

Well, that is not experimental then, it is utterly broken.

We are however between a rock and a hard place, because the alternative would be to run a regular reload which will result in Unbound stopping to process any queries, reload the zones and then resuming. On some hardware, we are in the area of minutes to load the zones which will cause absolute chaos if there is no DNS resolution for that time.

>> For now we are stuck with Unbound, but it has always been giving us a lot of trouble.
> 
> Does this mean we are switching to PowerDNs? But we should have a stable system meantime. What about going back to the 'reload_keep_cache' operation?

No, we are not switching to anything at the moment because I simply don’t have the time. We will however do it at some point in the future. PowerDNS Recursor is one of the candidates because it is very scriptable with Lua. Knot Resolver would also be a good option.

I tested running them on IPFire and they both work great, but we have a lot of custom tooling which will be quite time-consuming to migrate to any of the other solutions.

-Michael

> Regards,
> Bernhard
>>> - Knowing from using Jon's RPZ prototype, I checked whether a single reload ( used in DNS FW? ) propagates the changes, new list and/or allow/deny entries, really. I found cases where this isn't true. A unbound restart yielded the right behaviour.
>>> 
>>> 
>>> I must apologize not to have tested the release. But I haven't the equipment, yet ( only one production system ).
>>> 
>>> Regards,
>>> Bernhard
>>> 
> 
> 



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Feedback about the DNS FW
       [not found]       ` <210f08e2-c1ed-46c9-9e51-65ec200fe487@Canary>
@ 2026-05-02 20:23         ` Bernhard Bitsch
  2026-05-06  8:29           ` Michael Tremer
  0 siblings, 1 reply; 6+ messages in thread
From: Bernhard Bitsch @ 2026-05-02 20:23 UTC (permalink / raw)
  To: development

The runtime of 'fast_reload' is very long. Reason is the minimization of 
response time. This is a known effect in real time systems. If you 
minimize the function runtime, you pay it with worse responsiveness.
In case of unbound reload, you can either restart the process ( 
resulting in no response between stop and end of startup ), or you 
update the config while the functionality is running ( with full 
response based on the intermediate states ). Latter means that whilst 
the reload two processes are running, which must synchronise.

There are three possiblities:
- fast_reload, unbound is full responsive but the operation lasts some time.
- reload or reload_keep_cache, unbound may inresponsive some time, in 
some cases this may result in a restart
- restart ( stop, start ), unbound is not responsive until it is fully 
initialised again.

Which option to choose may depend on runtime on some devices ( as 
Michael mentioned ).
But, as investigated, the fast_reload operation needs a temporary cache. 
This isn't freed after termination of the reload function. This yields 
an increase in memory consumption over time. Each change in 
configuration adds the cache size.

Remains the problem what to do. Whether the newest version of unbound 
handles this problem, I couldn't find out.

Regards,
Bernhard



Am 02.05.2026 um 19:03 schrieb Jon Murphy:
> I am seeing the same as Bernhard in his first post. Last night I disabled phishing (and clicked save) and this morning I saw a big memory jump (from last night).
> 
> This is on an LWL Mini (APU4D4) with 4 GB RAM
> 
> FYI - after clicking Save, it takes about 2 minutes for the DNS Firewall WebGUI page to reload, but unbound is still processing RPZ requests.
> 
> —
> Jon
> 
>> On Thursday, Apr 30, 2026 at 5:06 AM, Michael Tremer <michael.tremer@ipfire.org (mailto:michael.tremer@ipfire.org)> wrote:
>> Hello Bernhard,
>>
>>> On 29 Apr 2026, at 23:19, Bernhard Bitsch <bbitsch@ipfire.org> wrote:
>>>
>>> Hello Michael,
>>>
>>> Am 29.04.2026 um 22:09 schrieb Michael Tremer:
>>>> Hello Bernhard,
>>>>> On 29 Apr 2026, at 18:43, Bernhard Bitsch <bbitsch@ipfire.org> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> after using the new DNS FW ( congrats to this nice feature! ), I found some issues.
>>>> Thanks. I believe that the entire feature has received very poor testing. Considering how many people have stated how important it is to them, really critical issues have been reported very late in the release process which indicates that the feature has not been tested, or if people found those bugs, they have not been reported.
>>>> I have to say that I am very disappointed about this. But it has nothing to do with your question.
>>>>> - Each 'save' in WUI page increases the memory consumption. Even if nothing changed. A restart of unbound frees this huge allocation.
>>>> Yes, this is known. It is a problem inside Unbound and there is nothing we can do about it. I did not report it to Unbound, but I am sure they should be made aware.
>>>> Unbound in general is using a lot of memory when it is downloading the lists. I have imported the lists into PowerDNS Recursor and it raises its memory consumption by about ~300 MiB when Unbound is going into 1.6-1.7 GiB.
>>>
>>> Some more investigation in unbound docs about the operation fast_reload ("This command is experimental at this time.") and some experiments I can state, that the fast_reload doesn't free the copy of the state. This gives the increase in memory consumption.
>>> The runtime for the 'reload_keep_cache' operation is about the same as for 'fast_reload' ( just my feeling ). But the memory load doesn't increase ( measured just with the WUI memory stats ).
>>
>> Well, that is not experimental then, it is utterly broken.
>>
>> We are however between a rock and a hard place, because the alternative would be to run a regular reload which will result in Unbound stopping to process any queries, reload the zones and then resuming. On some hardware, we are in the area of minutes to load the zones which will cause absolute chaos if there is no DNS resolution for that time.
>>
>>>> For now we are stuck with Unbound, but it has always been giving us a lot of trouble.
>>>
>>> Does this mean we are switching to PowerDNs? But we should have a stable system meantime. What about going back to the 'reload_keep_cache' operation?
>>
>> No, we are not switching to anything at the moment because I simply don’t have the time. We will however do it at some point in the future. PowerDNS Recursor is one of the candidates because it is very scriptable with Lua. Knot Resolver would also be a good option.
>>
>> I tested running them on IPFire and they both work great, but we have a lot of custom tooling which will be quite time-consuming to migrate to any of the other solutions.
>>
>> -Michael
>>
>>> Regards,
>>> Bernhard
>>>>> - Knowing from using Jon's RPZ prototype, I checked whether a single reload ( used in DNS FW? ) propagates the changes, new list and/or allow/deny entries, really. I found cases where this isn't true. A unbound restart yielded the right behaviour.
>>>>>
>>>>>
>>>>> I must apologize not to have tested the release. But I haven't the equipment, yet ( only one production system ).
>>>>>
>>>>> Regards,
>>>>> Bernhard
>>>>>
>>>
>>>
>>
>>
> 



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Feedback about the DNS FW
  2026-05-02 20:23         ` Bernhard Bitsch
@ 2026-05-06  8:29           ` Michael Tremer
  0 siblings, 0 replies; 6+ messages in thread
From: Michael Tremer @ 2026-05-06  8:29 UTC (permalink / raw)
  To: Bernhard Bitsch; +Cc: development

Hello Bernhard,

> On 2 May 2026, at 21:23, Bernhard Bitsch <bbitsch@ipfire.org> wrote:
> 
> The runtime of 'fast_reload' is very long. Reason is the minimization of response time. This is a known effect in real time systems. If you minimize the function runtime, you pay it with worse responsiveness.
> In case of unbound reload, you can either restart the process ( resulting in no response between stop and end of startup ), or you update the config while the functionality is running ( with full response based on the intermediate states ). Latter means that whilst the reload two processes are running, which must synchronise.
> 
> There are three possiblities:
> - fast_reload, unbound is full responsive but the operation lasts some time.
> - reload or reload_keep_cache, unbound may inresponsive some time, in some cases this may result in a restart
> - restart ( stop, start ), unbound is not responsive until it is fully initialised again.
> 
> Which option to choose may depend on runtime on some devices ( as Michael mentioned ).
> But, as investigated, the fast_reload operation needs a temporary cache. This isn't freed after termination of the reload function. This yields an increase in memory consumption over time. Each change in configuration adds the cache size.
> 
> Remains the problem what to do. Whether the newest version of unbound handles this problem, I couldn't find out.

I don’t think that this is a very complicated question. It simply isn’t an option to restart or classic reload unbound, because reloading the zones for a minute without any DNS resolution for the network is an outage.

Therefore fast-reload is the only option we have. That it does not free its memory is a bug in my opinion. So that has to be fixed.

-Michael

> Regards,
> Bernhard
> 
> 
> 
> Am 02.05.2026 um 19:03 schrieb Jon Murphy:
>> I am seeing the same as Bernhard in his first post. Last night I disabled phishing (and clicked save) and this morning I saw a big memory jump (from last night).
>> This is on an LWL Mini (APU4D4) with 4 GB RAM
>> FYI - after clicking Save, it takes about 2 minutes for the DNS Firewall WebGUI page to reload, but unbound is still processing RPZ requests.
>> —
>> Jon
>>> On Thursday, Apr 30, 2026 at 5:06 AM, Michael Tremer <michael.tremer@ipfire.org (mailto:michael.tremer@ipfire.org)> wrote:
>>> Hello Bernhard,
>>> 
>>>> On 29 Apr 2026, at 23:19, Bernhard Bitsch <bbitsch@ipfire.org> wrote:
>>>> 
>>>> Hello Michael,
>>>> 
>>>> Am 29.04.2026 um 22:09 schrieb Michael Tremer:
>>>>> Hello Bernhard,
>>>>>> On 29 Apr 2026, at 18:43, Bernhard Bitsch <bbitsch@ipfire.org> wrote:
>>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> after using the new DNS FW ( congrats to this nice feature! ), I found some issues.
>>>>> Thanks. I believe that the entire feature has received very poor testing. Considering how many people have stated how important it is to them, really critical issues have been reported very late in the release process which indicates that the feature has not been tested, or if people found those bugs, they have not been reported.
>>>>> I have to say that I am very disappointed about this. But it has nothing to do with your question.
>>>>>> - Each 'save' in WUI page increases the memory consumption. Even if nothing changed. A restart of unbound frees this huge allocation.
>>>>> Yes, this is known. It is a problem inside Unbound and there is nothing we can do about it. I did not report it to Unbound, but I am sure they should be made aware.
>>>>> Unbound in general is using a lot of memory when it is downloading the lists. I have imported the lists into PowerDNS Recursor and it raises its memory consumption by about ~300 MiB when Unbound is going into 1.6-1.7 GiB.
>>>> 
>>>> Some more investigation in unbound docs about the operation fast_reload ("This command is experimental at this time.") and some experiments I can state, that the fast_reload doesn't free the copy of the state. This gives the increase in memory consumption.
>>>> The runtime for the 'reload_keep_cache' operation is about the same as for 'fast_reload' ( just my feeling ). But the memory load doesn't increase ( measured just with the WUI memory stats ).
>>> 
>>> Well, that is not experimental then, it is utterly broken.
>>> 
>>> We are however between a rock and a hard place, because the alternative would be to run a regular reload which will result in Unbound stopping to process any queries, reload the zones and then resuming. On some hardware, we are in the area of minutes to load the zones which will cause absolute chaos if there is no DNS resolution for that time.
>>> 
>>>>> For now we are stuck with Unbound, but it has always been giving us a lot of trouble.
>>>> 
>>>> Does this mean we are switching to PowerDNs? But we should have a stable system meantime. What about going back to the 'reload_keep_cache' operation?
>>> 
>>> No, we are not switching to anything at the moment because I simply don’t have the time. We will however do it at some point in the future. PowerDNS Recursor is one of the candidates because it is very scriptable with Lua. Knot Resolver would also be a good option.
>>> 
>>> I tested running them on IPFire and they both work great, but we have a lot of custom tooling which will be quite time-consuming to migrate to any of the other solutions.
>>> 
>>> -Michael
>>> 
>>>> Regards,
>>>> Bernhard
>>>>>> - Knowing from using Jon's RPZ prototype, I checked whether a single reload ( used in DNS FW? ) propagates the changes, new list and/or allow/deny entries, really. I found cases where this isn't true. A unbound restart yielded the right behaviour.
>>>>>> 
>>>>>> 
>>>>>> I must apologize not to have tested the release. But I haven't the equipment, yet ( only one production system ).
>>>>>> 
>>>>>> Regards,
>>>>>> Bernhard
>>>>>> 
>>>> 
>>>> 
>>> 
>>> 
> 
> 



^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2026-05-06  8:29 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-04-29 17:43 Feedback about the DNS FW Bernhard Bitsch
2026-04-29 20:09 ` Michael Tremer
2026-04-29 22:19   ` Bernhard Bitsch
2026-04-30 10:06     ` Michael Tremer
     [not found]       ` <210f08e2-c1ed-46c9-9e51-65ec200fe487@Canary>
2026-05-02 20:23         ` Bernhard Bitsch
2026-05-06  8:29           ` Michael Tremer

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox