From mboxrd@z Thu Jan 1 00:00:00 1970 From: Adolf Belka To: development@lists.ipfire.org Subject: Re: Feedback on drop-hostile graph changes for bug#12981 Date: Tue, 13 Feb 2024 18:45:48 +0100 Message-ID: <8833db8f-6564-47d4-a710-3541cb564078@ipfire.org> In-Reply-To: <9c162bbf-33a6-4260-8abd-87dbbb8a51be@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6498769509637820418==" List-Id: --===============6498769509637820418== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Michael, I have also figured out a fix for the fresh install approach where the old DR= OP_HOSTILE directory is not present. I have tested it out and the graph is sh= own and also when I then added in the DROP_HOSTILE directory from a backup wh= en the line for Total Hostile Networks was shown. I will submit a patch with the update I made to graphs.pl and you can judge i= f what=C2=A0 have done is reasonable or if it can be simplified further. Regards, Adolf. On 11/02/2024 14:12, Adolf Belka wrote: > Hi Michael, > > I have figured out why the Core Update from 182 to 184 did not end up with = the DROP_HOSTILE_IN & DROP_HOSTILE_OUT directories available. > > The file collectd.conf is not in the list of files to be shipped so it did = not get updated with the update.sh script. > > I manually edited the collectd.conf file on my core updated vm version and = restarted collectd and the two new directories are nowe present and the graph= is correctly made because with the core update version the DROP_HOSTILE dire= ctory was present in the CU182 version. > > Regards, > > Adolf. > > > On 10/02/2024 19:11, Adolf Belka wrote: >> Hi Michael, >> >> On 10/02/2024 18:42, Adolf Belka wrote: >>> Hi Michael, >>> >>> On 10/02/2024 18:08, Adolf Belka wrote: >>>> Hi Michael, >>>> >>>> On 10/02/2024 15:22, Adolf Belka wrote: >>>>> Hi Jon and Michael, >>>>> >>>>> On 10/02/2024 15:09, jon wrote: >>>>>> Keep in mind there is a fcron for old RRDs (over 1 year old): >>>>>> >>>>>> # Cleanup the collectd RRD (graphs) >>>>>> %weekly * * /bin/find /var/log/rrd -mtime +365 -type f -name '*.rrd' -= delete -o -type d -empty -delete >>>>> The problem here is that we are changing the Hostile data from DROP_HOS= TILE into DROP_HOSTILE_IN & DROP_HOSTILE_OUT but we want to keep the history = of DROP_HOSTILE and in a fresh install it is only creating the DROP_HOSTILE_I= N and DROP_HOSTILE_OUT directories and the graph is also looking if there is = any data historically from the DROP_HOSTILE data. >>>>>> >>>>>> >>>>>>> On Feb 10, 2024, at 7:14 AM, Adolf Belka > wrote: >>>>>>> >>>>>>> Hi Michael, >>>>>>> >>>>>>> Sorry for delay in feedback. >>>>>>> >>>>>>> I tried out the drop-hostile changes with both an update from CU182 t= o CU184 and a fresh install of CU184 and had an error message showing with th= e graph in both cases. >>>>>>> >>>>>>> When I did the update from CU182 to CU184 the error message >>>>>>> >>>>>>> /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE_DROP_IN/ipt_b= ytes-DROP_HOSTILE.rrd >>>>>>> >>>>>>> was not present. >>>>>>> >>>>>>> See the screenshot attachment. >>>>>>> >>>>>>> Checking the directories there was only the iptables-filter-HOSTILE_D= ROP directory and not the iptables-filter-HOSTILE_DROP_IN or iptables-filter-= HOSTILE_DROP_OUT directories. >>>>>>> >>>>>>> Maybe something needs to be done in the update.sh script to create th= e new directories. I am not sure what though. >>>>>>> >>>>>>> >>>>>>> When I did a fresh install from CU184 it was the other way round. >>>>>>> >>>>>>> /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE_DROP/ipt_byte= s-DROP_HOSTILE.rrd >>>>>>> >>>>>>> was not present. >>>>>>> >>>>>>> Checking the directories there were the iptables-filter-HOSTILE_DROP_= IN and iptables-filter-HOSTILE_DROP_OUT directories but not the iptables-filt= er-HOSTILE_DROP directory. >>>>>>> >>>>>>> For a fresh install then there will be no history with the old naming= so here I would think we need to create the old directory name as standard f= or everyone but it will just not have any data. If the user does a restore of= an old backup then that HOSTILE_DROP data would become available. >>>>> >>>>> I think I might have found out what has caused this. In my original pat= ch set I changed collectd.conf to only chain the filters for the IN and OUT a= nd I removed the Chain filter HOSTILE_DROP DROP_HOSTILE entry in the plugin i= ptables section. >>>>> >>>>> As we are continuing to use the HOSTILE_DROP for the history I have add= ed that old line back into collectd.conf and am running a build and will test= out that fresh install to see if it solves the problem for that particular i= ssue. >>>>> >>>> Looks like I don't understand the whole collectd/rrd/graphs thing enough. >>>> >>>> Adding that line back into the collectd.conf file did not cause the ipta= bles-filter-HOSTILE_DROP directory to be created. >>>> >>>> I have looked through all the code changes done and I can't understand w= hy this is occurring let alone how to fix it. >>>> >>>> >>> I still don't understand why it does not work but the code change you did= with >>> >>> https://git.ipfire.org/?p=3Dpeople/ms/ipfire-2.x.git;a=3Dblobdiff;f=3Dcon= fig/cfgroot/graphs.pl;h=3Da23e49c98093ff435b7b929df93f0a4abfe86ac8;hp=3Df5274= 47b5310bf6d7309184dda46842b06e713d1;hb=3Da6e4c650a7303dfc7612ee806122a7cfb6cc= 2632;hpb=3D0ad5ffaff8443ff32a996c8f3854fcf90d6dc26c >>> >>> should deal with the HOSTILE_DROP directory not being present for a fresh= install. >>> >>> The CDEF command you created >>> >>> "CDEF:hostile=3Dhostilelegacy,UN,hostilein,hostileout,+,hostilelegacy,IF", >>> >>> should end up replacing hostilelegacy,UN with 1 because hostilelegacy sho= uld be Unknown due to the directory not existing. Then the IF statement takes= hostilein + hostileout if hostilelegacy is Unknown or takes hostilelegacy if= hostilelegacy is valid data. >>> >>> So I have been able to follow the RPN CDEF command (I struggle with RPN) = after a lot of reading and the command should be doing what you intended it t= o do but it doesn't seem to be and I haven't been able to figure out why. >>> >> I think that the problem is due to the fact that an Unknown value for CDEF= means an unkbnown value out of the .rrd database. The database not being pre= sent is dealt with as an "I can't find or open the file" error and the CDEF R= PN sequence doesn't even get used. The code goes to the RRDs::error command. >> >> Regards, >> >> Adolf. >> >> >>> Regards, >>> >>> Adolf. >>> >>> >>>> Regards, >>>> >>>> Adolf. >>>> >>>>> Regards, >>>>> Adolf. >>>>>>> >>>>>>> >>>>>>> On the fresh install of CU148 I did a restore of a backup from CU182 = and then the graph worked as all three directories then were present. >>>>>>> >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Adolf. >>>>>>> >>>>>> --===============6498769509637820418==--