Re: Feedback on drop-hostile graph changes for bug#12981

Re: Feedback on drop-hostile graph changes for bug#12981

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 2926 bytes --]

Hi Jon and Michael,

On 10/02/2024 15:09, jon wrote:
> Keep in mind there is a fcron for old RRDs (over 1 year old):
> 
> # Cleanup the collectd RRD (graphs)
> %weekly * * /bin/find /var/log/rrd -mtime +365 -type f -name '*.rrd' -delete -o -type d -empty -delete
The problem here is that we are changing the Hostile data from DROP_HOSTILE into DROP_HOSTILE_IN & DROP_HOSTILE_OUT but we want to keep the history of DROP_HOSTILE and in a fresh install it is only creating the DROP_HOSTILE_IN and DROP_HOSTILE_OUT directories and the graph is also looking if there is any data historically from the DROP_HOSTILE data.
> 
> 
>> On Feb 10, 2024, at 7:14 AM, Adolf Belka <adolf.belka(a)ipfire.org <mailto:adolf.belka(a)ipfire.org>> wrote:
>>
>> Hi Michael,
>>
>> Sorry for delay in feedback.
>>
>> I tried out the drop-hostile changes with both an update from CU182 to CU184 and a fresh install of CU184 and had an error message showing with the graph in both cases.
>>
>> When I did the update from CU182 to CU184 the error message
>>
>> /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE_DROP_IN/ipt_bytes-DROP_HOSTILE.rrd
>>
>> was not present.
>>
>> See the screenshot attachment.
>>
>> Checking the directories there was only the iptables-filter-HOSTILE_DROP directory and not the iptables-filter-HOSTILE_DROP_IN or iptables-filter-HOSTILE_DROP_OUT directories.
>>
>> Maybe something needs to be done in the update.sh script to create the new directories. I am not sure what though.
>>
>>
>> When I did a fresh install from CU184 it was the other way round.
>>
>> /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE_DROP/ipt_bytes-DROP_HOSTILE.rrd
>>
>> was not present.
>>
>> Checking the directories there were the iptables-filter-HOSTILE_DROP_IN and iptables-filter-HOSTILE_DROP_OUT directories but not the iptables-filter-HOSTILE_DROP directory.
>>
>> For a fresh install then there will be no history with the old naming so here I would think we need to create the old directory name as standard for everyone but it will just not have any data. If the user does a restore of an old backup then that HOSTILE_DROP data would become available.

I think I might have found out what has caused this. In my original patch set I changed collectd.conf to only chain the filters for the IN and OUT and I removed the Chain filter HOSTILE_DROP DROP_HOSTILE entry in the plugin iptables section.

As we are continuing to use the HOSTILE_DROP for the history I have added that old line back into collectd.conf and am running a build and will test out that fresh install to see if it solves the problem for that particular issue.

Regards,
Adolf.
>>
>>
>> On the fresh install of CU148 I did a restore of a backup from CU182 and then the graph worked as all three directories then were present.
>>
>>
>> Regards,
>>
>> Adolf.
>> <firewall hits graph core update.png>
> 

Re: Feedback on drop-hostile graph changes for bug#12981

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 3389 bytes --]

Hi Michael,

On 10/02/2024 15:22, Adolf Belka wrote:
> Hi Jon and Michael,
>
> On 10/02/2024 15:09, jon wrote:
>> Keep in mind there is a fcron for old RRDs (over 1 year old):
>>
>> # Cleanup the collectd RRD (graphs)
>> %weekly * * /bin/find /var/log/rrd -mtime +365 -type f -name '*.rrd' -delete -o -type d -empty -delete
> The problem here is that we are changing the Hostile data from DROP_HOSTILE into DROP_HOSTILE_IN & DROP_HOSTILE_OUT but we want to keep the history of DROP_HOSTILE and in a fresh install it is only creating the DROP_HOSTILE_IN and DROP_HOSTILE_OUT directories and the graph is also looking if there is any data historically from the DROP_HOSTILE data.
>>
>>
>>> On Feb 10, 2024, at 7:14 AM, Adolf Belka <adolf.belka(a)ipfire.org <mailto:adolf.belka(a)ipfire.org>> wrote:
>>>
>>> Hi Michael,
>>>
>>> Sorry for delay in feedback.
>>>
>>> I tried out the drop-hostile changes with both an update from CU182 to CU184 and a fresh install of CU184 and had an error message showing with the graph in both cases.
>>>
>>> When I did the update from CU182 to CU184 the error message
>>>
>>> /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE_DROP_IN/ipt_bytes-DROP_HOSTILE.rrd
>>>
>>> was not present.
>>>
>>> See the screenshot attachment.
>>>
>>> Checking the directories there was only the iptables-filter-HOSTILE_DROP directory and not the iptables-filter-HOSTILE_DROP_IN or iptables-filter-HOSTILE_DROP_OUT directories.
>>>
>>> Maybe something needs to be done in the update.sh script to create the new directories. I am not sure what though.
>>>
>>>
>>> When I did a fresh install from CU184 it was the other way round.
>>>
>>> /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE_DROP/ipt_bytes-DROP_HOSTILE.rrd
>>>
>>> was not present.
>>>
>>> Checking the directories there were the iptables-filter-HOSTILE_DROP_IN and iptables-filter-HOSTILE_DROP_OUT directories but not the iptables-filter-HOSTILE_DROP directory.
>>>
>>> For a fresh install then there will be no history with the old naming so here I would think we need to create the old directory name as standard for everyone but it will just not have any data. If the user does a restore of an old backup then that HOSTILE_DROP data would become available.
>
> I think I might have found out what has caused this. In my original patch set I changed collectd.conf to only chain the filters for the IN and OUT and I removed the Chain filter HOSTILE_DROP DROP_HOSTILE entry in the plugin iptables section.
>
> As we are continuing to use the HOSTILE_DROP for the history I have added that old line back into collectd.conf and am running a build and will test out that fresh install to see if it solves the problem for that particular issue.
>
Looks like I don't understand the whole collectd/rrd/graphs thing enough.

Adding that line back into the collectd.conf file did not cause the iptables-filter-HOSTILE_DROP directory to be created.

I have looked through all the code changes done and I can't understand why this is occurring let alone how to fix it.


Regards,

Adolf.

> Regards,
> Adolf.
>>>
>>>
>>> On the fresh install of CU148 I did a restore of a backup from CU182 and then the graph worked as all three directories then were present.
>>>
>>>
>>> Regards,
>>>
>>> Adolf.
>>> <firewall hits graph core update.png>
>>

Re: Feedback on drop-hostile graph changes for bug#12981

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 4578 bytes --]

Hi Michael,

On 10/02/2024 18:08, Adolf Belka wrote:
> Hi Michael,
>
> On 10/02/2024 15:22, Adolf Belka wrote:
>> Hi Jon and Michael,
>>
>> On 10/02/2024 15:09, jon wrote:
>>> Keep in mind there is a fcron for old RRDs (over 1 year old):
>>>
>>> # Cleanup the collectd RRD (graphs)
>>> %weekly * * /bin/find /var/log/rrd -mtime +365 -type f -name '*.rrd' -delete -o -type d -empty -delete
>> The problem here is that we are changing the Hostile data from DROP_HOSTILE into DROP_HOSTILE_IN & DROP_HOSTILE_OUT but we want to keep the history of DROP_HOSTILE and in a fresh install it is only creating the DROP_HOSTILE_IN and DROP_HOSTILE_OUT directories and the graph is also looking if there is any data historically from the DROP_HOSTILE data.
>>>
>>>
>>>> On Feb 10, 2024, at 7:14 AM, Adolf Belka <adolf.belka(a)ipfire.org <mailto:adolf.belka(a)ipfire.org>> wrote:
>>>>
>>>> Hi Michael,
>>>>
>>>> Sorry for delay in feedback.
>>>>
>>>> I tried out the drop-hostile changes with both an update from CU182 to CU184 and a fresh install of CU184 and had an error message showing with the graph in both cases.
>>>>
>>>> When I did the update from CU182 to CU184 the error message
>>>>
>>>> /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE_DROP_IN/ipt_bytes-DROP_HOSTILE.rrd
>>>>
>>>> was not present.
>>>>
>>>> See the screenshot attachment.
>>>>
>>>> Checking the directories there was only the iptables-filter-HOSTILE_DROP directory and not the iptables-filter-HOSTILE_DROP_IN or iptables-filter-HOSTILE_DROP_OUT directories.
>>>>
>>>> Maybe something needs to be done in the update.sh script to create the new directories. I am not sure what though.
>>>>
>>>>
>>>> When I did a fresh install from CU184 it was the other way round.
>>>>
>>>> /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE_DROP/ipt_bytes-DROP_HOSTILE.rrd
>>>>
>>>> was not present.
>>>>
>>>> Checking the directories there were the iptables-filter-HOSTILE_DROP_IN and iptables-filter-HOSTILE_DROP_OUT directories but not the iptables-filter-HOSTILE_DROP directory.
>>>>
>>>> For a fresh install then there will be no history with the old naming so here I would think we need to create the old directory name as standard for everyone but it will just not have any data. If the user does a restore of an old backup then that HOSTILE_DROP data would become available.
>>
>> I think I might have found out what has caused this. In my original patch set I changed collectd.conf to only chain the filters for the IN and OUT and I removed the Chain filter HOSTILE_DROP DROP_HOSTILE entry in the plugin iptables section.
>>
>> As we are continuing to use the HOSTILE_DROP for the history I have added that old line back into collectd.conf and am running a build and will test out that fresh install to see if it solves the problem for that particular issue.
>>
> Looks like I don't understand the whole collectd/rrd/graphs thing enough.
>
> Adding that line back into the collectd.conf file did not cause the iptables-filter-HOSTILE_DROP directory to be created.
>
> I have looked through all the code changes done and I can't understand why this is occurring let alone how to fix it.
>
>
I still don't understand why it does not work but the code change you did with

https://git.ipfire.org/?p=people/ms/ipfire-2.x.git;a=blobdiff;f=config/cfgroot/graphs.pl;h=a23e49c98093ff435b7b929df93f0a4abfe86ac8;hp=f527447b5310bf6d7309184dda46842b06e713d1;hb=a6e4c650a7303dfc7612ee806122a7cfb6cc2632;hpb=0ad5ffaff8443ff32a996c8f3854fcf90d6dc26c

should deal with the HOSTILE_DROP directory not being present for a fresh install.

The CDEF command you created

"CDEF:hostile=hostilelegacy,UN,hostilein,hostileout,+,hostilelegacy,IF",

should end up replacing hostilelegacy,UN with 1 because hostilelegacy should be Unknown due to the directory not existing. Then the IF statement takes hostilein + hostileout if hostilelegacy is Unknown or takes hostilelegacy if hostilelegacy is valid data.

So I have been able to follow the RPN CDEF command (I struggle with RPN) after a lot of reading and the command should be doing what you intended it to do but it doesn't seem to be and I haven't been able to figure out why.

Regards,

Adolf.


> Regards,
>
> Adolf.
>
>> Regards,
>> Adolf.
>>>>
>>>>
>>>> On the fresh install of CU148 I did a restore of a backup from CU182 and then the graph worked as all three directories then were present.
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Adolf.
>>>> <firewall hits graph core update.png>
>>>

Re: Feedback on drop-hostile graph changes for bug#12981

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 5070 bytes --]

Hi Michael,

On 10/02/2024 18:42, Adolf Belka wrote:
> Hi Michael,
>
> On 10/02/2024 18:08, Adolf Belka wrote:
>> Hi Michael,
>>
>> On 10/02/2024 15:22, Adolf Belka wrote:
>>> Hi Jon and Michael,
>>>
>>> On 10/02/2024 15:09, jon wrote:
>>>> Keep in mind there is a fcron for old RRDs (over 1 year old):
>>>>
>>>> # Cleanup the collectd RRD (graphs)
>>>> %weekly * * /bin/find /var/log/rrd -mtime +365 -type f -name '*.rrd' -delete -o -type d -empty -delete
>>> The problem here is that we are changing the Hostile data from DROP_HOSTILE into DROP_HOSTILE_IN & DROP_HOSTILE_OUT but we want to keep the history of DROP_HOSTILE and in a fresh install it is only creating the DROP_HOSTILE_IN and DROP_HOSTILE_OUT directories and the graph is also looking if there is any data historically from the DROP_HOSTILE data.
>>>>
>>>>
>>>>> On Feb 10, 2024, at 7:14 AM, Adolf Belka <adolf.belka(a)ipfire.org <mailto:adolf.belka(a)ipfire.org>> wrote:
>>>>>
>>>>> Hi Michael,
>>>>>
>>>>> Sorry for delay in feedback.
>>>>>
>>>>> I tried out the drop-hostile changes with both an update from CU182 to CU184 and a fresh install of CU184 and had an error message showing with the graph in both cases.
>>>>>
>>>>> When I did the update from CU182 to CU184 the error message
>>>>>
>>>>> /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE_DROP_IN/ipt_bytes-DROP_HOSTILE.rrd
>>>>>
>>>>> was not present.
>>>>>
>>>>> See the screenshot attachment.
>>>>>
>>>>> Checking the directories there was only the iptables-filter-HOSTILE_DROP directory and not the iptables-filter-HOSTILE_DROP_IN or iptables-filter-HOSTILE_DROP_OUT directories.
>>>>>
>>>>> Maybe something needs to be done in the update.sh script to create the new directories. I am not sure what though.
>>>>>
>>>>>
>>>>> When I did a fresh install from CU184 it was the other way round.
>>>>>
>>>>> /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE_DROP/ipt_bytes-DROP_HOSTILE.rrd
>>>>>
>>>>> was not present.
>>>>>
>>>>> Checking the directories there were the iptables-filter-HOSTILE_DROP_IN and iptables-filter-HOSTILE_DROP_OUT directories but not the iptables-filter-HOSTILE_DROP directory.
>>>>>
>>>>> For a fresh install then there will be no history with the old naming so here I would think we need to create the old directory name as standard for everyone but it will just not have any data. If the user does a restore of an old backup then that HOSTILE_DROP data would become available.
>>>
>>> I think I might have found out what has caused this. In my original patch set I changed collectd.conf to only chain the filters for the IN and OUT and I removed the Chain filter HOSTILE_DROP DROP_HOSTILE entry in the plugin iptables section.
>>>
>>> As we are continuing to use the HOSTILE_DROP for the history I have added that old line back into collectd.conf and am running a build and will test out that fresh install to see if it solves the problem for that particular issue.
>>>
>> Looks like I don't understand the whole collectd/rrd/graphs thing enough.
>>
>> Adding that line back into the collectd.conf file did not cause the iptables-filter-HOSTILE_DROP directory to be created.
>>
>> I have looked through all the code changes done and I can't understand why this is occurring let alone how to fix it.
>>
>>
> I still don't understand why it does not work but the code change you did with
>
> https://git.ipfire.org/?p=people/ms/ipfire-2.x.git;a=blobdiff;f=config/cfgroot/graphs.pl;h=a23e49c98093ff435b7b929df93f0a4abfe86ac8;hp=f527447b5310bf6d7309184dda46842b06e713d1;hb=a6e4c650a7303dfc7612ee806122a7cfb6cc2632;hpb=0ad5ffaff8443ff32a996c8f3854fcf90d6dc26c
>
> should deal with the HOSTILE_DROP directory not being present for a fresh install.
>
> The CDEF command you created
>
> "CDEF:hostile=hostilelegacy,UN,hostilein,hostileout,+,hostilelegacy,IF",
>
> should end up replacing hostilelegacy,UN with 1 because hostilelegacy should be Unknown due to the directory not existing. Then the IF statement takes hostilein + hostileout if hostilelegacy is Unknown or takes hostilelegacy if hostilelegacy is valid data.
>
> So I have been able to follow the RPN CDEF command (I struggle with RPN) after a lot of reading and the command should be doing what you intended it to do but it doesn't seem to be and I haven't been able to figure out why.
>
I think that the problem is due to the fact that an Unknown value for CDEF means an unkbnown value out of the .rrd database. The database not being present is dealt with as an "I can't find or open the file" error and the CDEF RPN sequence doesn't even get used. The code goes to the RRDs::error command.

Regards,

Adolf.


> Regards,
>
> Adolf.
>
>
>> Regards,
>>
>> Adolf.
>>
>>> Regards,
>>> Adolf.
>>>>>
>>>>>
>>>>> On the fresh install of CU148 I did a restore of a backup from CU182 and then the graph worked as all three directories then were present.
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Adolf.
>>>>> <firewall hits graph core update.png>
>>>>

Re: Feedback on drop-hostile graph changes for bug#12981

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 5791 bytes --]

Hi Michael,

I have figured out why the Core Update from 182 to 184 did not end up with the DROP_HOSTILE_IN & DROP_HOSTILE_OUT directories available.

The file collectd.conf is not in the list of files to be shipped so it did not get updated with the update.sh script.

I manually edited the collectd.conf file on my core updated vm version and restarted collectd and the two new directories are nowe present and the graph is correctly made because with the core update version the DROP_HOSTILE directory was present in the CU182 version.

Regards,

Adolf.


On 10/02/2024 19:11, Adolf Belka wrote:
> Hi Michael,
>
> On 10/02/2024 18:42, Adolf Belka wrote:
>> Hi Michael,
>>
>> On 10/02/2024 18:08, Adolf Belka wrote:
>>> Hi Michael,
>>>
>>> On 10/02/2024 15:22, Adolf Belka wrote:
>>>> Hi Jon and Michael,
>>>>
>>>> On 10/02/2024 15:09, jon wrote:
>>>>> Keep in mind there is a fcron for old RRDs (over 1 year old):
>>>>>
>>>>> # Cleanup the collectd RRD (graphs)
>>>>> %weekly * * /bin/find /var/log/rrd -mtime +365 -type f -name '*.rrd' -delete -o -type d -empty -delete
>>>> The problem here is that we are changing the Hostile data from DROP_HOSTILE into DROP_HOSTILE_IN & DROP_HOSTILE_OUT but we want to keep the history of DROP_HOSTILE and in a fresh install it is only creating the DROP_HOSTILE_IN and DROP_HOSTILE_OUT directories and the graph is also looking if there is any data historically from the DROP_HOSTILE data.
>>>>>
>>>>>
>>>>>> On Feb 10, 2024, at 7:14 AM, Adolf Belka <adolf.belka(a)ipfire.org <mailto:adolf.belka(a)ipfire.org>> wrote:
>>>>>>
>>>>>> Hi Michael,
>>>>>>
>>>>>> Sorry for delay in feedback.
>>>>>>
>>>>>> I tried out the drop-hostile changes with both an update from CU182 to CU184 and a fresh install of CU184 and had an error message showing with the graph in both cases.
>>>>>>
>>>>>> When I did the update from CU182 to CU184 the error message
>>>>>>
>>>>>> /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE_DROP_IN/ipt_bytes-DROP_HOSTILE.rrd
>>>>>>
>>>>>> was not present.
>>>>>>
>>>>>> See the screenshot attachment.
>>>>>>
>>>>>> Checking the directories there was only the iptables-filter-HOSTILE_DROP directory and not the iptables-filter-HOSTILE_DROP_IN or iptables-filter-HOSTILE_DROP_OUT directories.
>>>>>>
>>>>>> Maybe something needs to be done in the update.sh script to create the new directories. I am not sure what though.
>>>>>>
>>>>>>
>>>>>> When I did a fresh install from CU184 it was the other way round.
>>>>>>
>>>>>> /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE_DROP/ipt_bytes-DROP_HOSTILE.rrd
>>>>>>
>>>>>> was not present.
>>>>>>
>>>>>> Checking the directories there were the iptables-filter-HOSTILE_DROP_IN and iptables-filter-HOSTILE_DROP_OUT directories but not the iptables-filter-HOSTILE_DROP directory.
>>>>>>
>>>>>> For a fresh install then there will be no history with the old naming so here I would think we need to create the old directory name as standard for everyone but it will just not have any data. If the user does a restore of an old backup then that HOSTILE_DROP data would become available.
>>>>
>>>> I think I might have found out what has caused this. In my original patch set I changed collectd.conf to only chain the filters for the IN and OUT and I removed the Chain filter HOSTILE_DROP DROP_HOSTILE entry in the plugin iptables section.
>>>>
>>>> As we are continuing to use the HOSTILE_DROP for the history I have added that old line back into collectd.conf and am running a build and will test out that fresh install to see if it solves the problem for that particular issue.
>>>>
>>> Looks like I don't understand the whole collectd/rrd/graphs thing enough.
>>>
>>> Adding that line back into the collectd.conf file did not cause the iptables-filter-HOSTILE_DROP directory to be created.
>>>
>>> I have looked through all the code changes done and I can't understand why this is occurring let alone how to fix it.
>>>
>>>
>> I still don't understand why it does not work but the code change you did with
>>
>> https://git.ipfire.org/?p=people/ms/ipfire-2.x.git;a=blobdiff;f=config/cfgroot/graphs.pl;h=a23e49c98093ff435b7b929df93f0a4abfe86ac8;hp=f527447b5310bf6d7309184dda46842b06e713d1;hb=a6e4c650a7303dfc7612ee806122a7cfb6cc2632;hpb=0ad5ffaff8443ff32a996c8f3854fcf90d6dc26c
>>
>> should deal with the HOSTILE_DROP directory not being present for a fresh install.
>>
>> The CDEF command you created
>>
>> "CDEF:hostile=hostilelegacy,UN,hostilein,hostileout,+,hostilelegacy,IF",
>>
>> should end up replacing hostilelegacy,UN with 1 because hostilelegacy should be Unknown due to the directory not existing. Then the IF statement takes hostilein + hostileout if hostilelegacy is Unknown or takes hostilelegacy if hostilelegacy is valid data.
>>
>> So I have been able to follow the RPN CDEF command (I struggle with RPN) after a lot of reading and the command should be doing what you intended it to do but it doesn't seem to be and I haven't been able to figure out why.
>>
> I think that the problem is due to the fact that an Unknown value for CDEF means an unkbnown value out of the .rrd database. The database not being present is dealt with as an "I can't find or open the file" error and the CDEF RPN sequence doesn't even get used. The code goes to the RRDs::error command.
>
> Regards,
>
> Adolf.
>
>
>> Regards,
>>
>> Adolf.
>>
>>
>>> Regards,
>>>
>>> Adolf.
>>>
>>>> Regards,
>>>> Adolf.
>>>>>>
>>>>>>
>>>>>> On the fresh install of CU148 I did a restore of a backup from CU182 and then the graph worked as all three directories then were present.
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Adolf.
>>>>>> <firewall hits graph core update.png>
>>>>>

Re: Feedback on drop-hostile graph changes for bug#12981

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 6426 bytes --]

Hi Michael,

I have also figured out a fix for the fresh install approach where the old DROP_HOSTILE directory is not present. I have tested it out and the graph is shown and also when I then added in the DROP_HOSTILE directory from a backup when the line for Total Hostile Networks was shown.

I will submit a patch with the update I made to graphs.pl and you can judge if what  have done is reasonable or if it can be simplified further.

Regards,

Adolf.

On 11/02/2024 14:12, Adolf Belka wrote:
> Hi Michael,
>
> I have figured out why the Core Update from 182 to 184 did not end up with the DROP_HOSTILE_IN & DROP_HOSTILE_OUT directories available.
>
> The file collectd.conf is not in the list of files to be shipped so it did not get updated with the update.sh script.
>
> I manually edited the collectd.conf file on my core updated vm version and restarted collectd and the two new directories are nowe present and the graph is correctly made because with the core update version the DROP_HOSTILE directory was present in the CU182 version.
>
> Regards,
>
> Adolf.
>
>
> On 10/02/2024 19:11, Adolf Belka wrote:
>> Hi Michael,
>>
>> On 10/02/2024 18:42, Adolf Belka wrote:
>>> Hi Michael,
>>>
>>> On 10/02/2024 18:08, Adolf Belka wrote:
>>>> Hi Michael,
>>>>
>>>> On 10/02/2024 15:22, Adolf Belka wrote:
>>>>> Hi Jon and Michael,
>>>>>
>>>>> On 10/02/2024 15:09, jon wrote:
>>>>>> Keep in mind there is a fcron for old RRDs (over 1 year old):
>>>>>>
>>>>>> # Cleanup the collectd RRD (graphs)
>>>>>> %weekly * * /bin/find /var/log/rrd -mtime +365 -type f -name '*.rrd' -delete -o -type d -empty -delete
>>>>> The problem here is that we are changing the Hostile data from DROP_HOSTILE into DROP_HOSTILE_IN & DROP_HOSTILE_OUT but we want to keep the history of DROP_HOSTILE and in a fresh install it is only creating the DROP_HOSTILE_IN and DROP_HOSTILE_OUT directories and the graph is also looking if there is any data historically from the DROP_HOSTILE data.
>>>>>>
>>>>>>
>>>>>>> On Feb 10, 2024, at 7:14 AM, Adolf Belka <adolf.belka(a)ipfire.org <mailto:adolf.belka(a)ipfire.org>> wrote:
>>>>>>>
>>>>>>> Hi Michael,
>>>>>>>
>>>>>>> Sorry for delay in feedback.
>>>>>>>
>>>>>>> I tried out the drop-hostile changes with both an update from CU182 to CU184 and a fresh install of CU184 and had an error message showing with the graph in both cases.
>>>>>>>
>>>>>>> When I did the update from CU182 to CU184 the error message
>>>>>>>
>>>>>>> /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE_DROP_IN/ipt_bytes-DROP_HOSTILE.rrd
>>>>>>>
>>>>>>> was not present.
>>>>>>>
>>>>>>> See the screenshot attachment.
>>>>>>>
>>>>>>> Checking the directories there was only the iptables-filter-HOSTILE_DROP directory and not the iptables-filter-HOSTILE_DROP_IN or iptables-filter-HOSTILE_DROP_OUT directories.
>>>>>>>
>>>>>>> Maybe something needs to be done in the update.sh script to create the new directories. I am not sure what though.
>>>>>>>
>>>>>>>
>>>>>>> When I did a fresh install from CU184 it was the other way round.
>>>>>>>
>>>>>>> /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE_DROP/ipt_bytes-DROP_HOSTILE.rrd
>>>>>>>
>>>>>>> was not present.
>>>>>>>
>>>>>>> Checking the directories there were the iptables-filter-HOSTILE_DROP_IN and iptables-filter-HOSTILE_DROP_OUT directories but not the iptables-filter-HOSTILE_DROP directory.
>>>>>>>
>>>>>>> For a fresh install then there will be no history with the old naming so here I would think we need to create the old directory name as standard for everyone but it will just not have any data. If the user does a restore of an old backup then that HOSTILE_DROP data would become available.
>>>>>
>>>>> I think I might have found out what has caused this. In my original patch set I changed collectd.conf to only chain the filters for the IN and OUT and I removed the Chain filter HOSTILE_DROP DROP_HOSTILE entry in the plugin iptables section.
>>>>>
>>>>> As we are continuing to use the HOSTILE_DROP for the history I have added that old line back into collectd.conf and am running a build and will test out that fresh install to see if it solves the problem for that particular issue.
>>>>>
>>>> Looks like I don't understand the whole collectd/rrd/graphs thing enough.
>>>>
>>>> Adding that line back into the collectd.conf file did not cause the iptables-filter-HOSTILE_DROP directory to be created.
>>>>
>>>> I have looked through all the code changes done and I can't understand why this is occurring let alone how to fix it.
>>>>
>>>>
>>> I still don't understand why it does not work but the code change you did with
>>>
>>> https://git.ipfire.org/?p=people/ms/ipfire-2.x.git;a=blobdiff;f=config/cfgroot/graphs.pl;h=a23e49c98093ff435b7b929df93f0a4abfe86ac8;hp=f527447b5310bf6d7309184dda46842b06e713d1;hb=a6e4c650a7303dfc7612ee806122a7cfb6cc2632;hpb=0ad5ffaff8443ff32a996c8f3854fcf90d6dc26c
>>>
>>> should deal with the HOSTILE_DROP directory not being present for a fresh install.
>>>
>>> The CDEF command you created
>>>
>>> "CDEF:hostile=hostilelegacy,UN,hostilein,hostileout,+,hostilelegacy,IF",
>>>
>>> should end up replacing hostilelegacy,UN with 1 because hostilelegacy should be Unknown due to the directory not existing. Then the IF statement takes hostilein + hostileout if hostilelegacy is Unknown or takes hostilelegacy if hostilelegacy is valid data.
>>>
>>> So I have been able to follow the RPN CDEF command (I struggle with RPN) after a lot of reading and the command should be doing what you intended it to do but it doesn't seem to be and I haven't been able to figure out why.
>>>
>> I think that the problem is due to the fact that an Unknown value for CDEF means an unkbnown value out of the .rrd database. The database not being present is dealt with as an "I can't find or open the file" error and the CDEF RPN sequence doesn't even get used. The code goes to the RRDs::error command.
>>
>> Regards,
>>
>> Adolf.
>>
>>
>>> Regards,
>>>
>>> Adolf.
>>>
>>>
>>>> Regards,
>>>>
>>>> Adolf.
>>>>
>>>>> Regards,
>>>>> Adolf.
>>>>>>>
>>>>>>>
>>>>>>> On the fresh install of CU148 I did a restore of a backup from CU182 and then the graph worked as all three directories then were present.
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Adolf.
>>>>>>> <firewall hits graph core update.png>
>>>>>>