Jon

On Aug 25, 2022, at 11:47 AM, Jon Murphy <jcmurphy26@gmail.com> wrote:

Michael,

On Aug 25, 2022, at 11:32 AM, Michael Tremer <michael.tremer@ipfire.org> wrote:

Hello,

On 25 Aug 2022, at 17:17, Jon Murphy <jcmurphy26@gmail.com> wrote:

Michael,

On Aug 25, 2022, at 10:44 AM, Michael Tremer <michael.tremer@ipfire.org> wrote:

Hello Jon,

On 25 Aug 2022, at 15:40, Jon Murphy <jcmurphy26@gmail.com> wrote:

Hey Michael,

Over the years I’ve used four different IPFire boxes. And during a "late-summer cleaning" I found lots of old RRD files. I ran this command:
`find /var/log/rrd -mtime +365 -type f -name '*.rrd' -ls`

Was this just because you had a look around in the file system, or did you suffer from low disk space?

Just looking…

:)

And among those were lots of old unused openvpn graphs. I think I had 8-10 old openvpn graphs that were more than 1 year old.

I had started to write a wiki page for users about finding and deleting old `/var/log/rrd/collectd/localhost/openvpn-*` RRDs. And then the wiki grew to ALL old graphs.

Below is a snipped version of my original `find`.   I think there were ~40 total old obsolete RRD files.

```
[root@ipfire ~] # find /var/log/rrd -mtime +365 -type f -name '*.rrd' -ls
   1838    304 -rw-r--r--   1 root     root       307712 Nov 17 2019 /var/log/rrd/collectd/localhost/interface/if_octets-dummy0.rrd
   1794    152 -rw-r--r--   1 root     root       154888 Jul 18 2021 /var/log/rrd/collectd/localhost/sensors-soc_dts0-virtual-0/temperature-temp1.rrd
   1821    152 -rw-r--r--   1 root     root       154888 Jul 18 2021 /var/log/rrd/collectd/localhost/sensors-it8721-isa-0a30/temperature-temp2.rrd
. . .
   1831    152 -rw-r--r--   1 root     root       154888 Jul 18 2021 /var/log/rrd/collectd/localhost/sensors-it8721-isa-0a30/fanspeed-fan3.rrd
   1888    152 -rw-r--r--   1 root     root       154888 Jul 18 2021 /var/log/rrd/collectd/localhost/sensors-soc_dts1-virtual-0/temperature-temp1.rrd
   1785    152 -rw-r--r--   1 root     root       154888 Aug 12 2017 /var/log/rrd/collectd/localhost/sensors-smsc47b397-isa-0480/temperature-temp2.rrd
. . .
   1791    152 -rw-r--r--   1 root     root       154888 Aug 12 2017 /var/log/rrd/collectd/localhost/sensors-smsc47b397-isa-0480/fanspeed-fan3.rrd
   1880    152 -rw-r--r--   1 root     root       154888 Jul 18 2021 /var/log/rrd/collectd/localhost/thermal-thermal_zone1/temperature-temperature.rrd
   1875    152 -rw-r--r--   1 root     root       154888 Jul 18 2021 /var/log/rrd/collectd/localhost/sensors-coretemp-isa-0000/temperature-temp4.rrd
   1874    152 -rw-r--r--   1 root     root       154888 Jul 18 2021 /var/log/rrd/collectd/localhost/sensors-coretemp-isa-0000/temperature-temp5.rrd
   1781    152 -rw-r--r--   1 root     root       154888 Jan 13 2020 /var/log/rrd/collectd/localhost/thermal-thermal_zone2/temperature-temperature.rrd
   1937     56 -rw-r--r--   1 nobody   nobody      56800 Jul 18 2021 /var/log/rrd/iptraffic/192.168.60.105.rrd
   1938     56 -rw-r--r--   1 nobody   nobody      56800 Jul 18 2021 /var/log/rrd/iptraffic/192.168.60.147.rrd
   1935     56 -rw-r--r--   1 nobody   nobody      56800 Jul 18 2021 /var/log/rrd/iptraffic/192.168.60.194.rrd
[root@ipfire ~] #
```

As you can see some of the files were left over from 2017 and they did not get cleaned out during hardware changes. I obviously did not follow the migrations wiki!

Well, I suppose if it doesn’t happen automatically, we cannot rely on this :) I never follow this either.

The wiki page was to be instructions for users to `find` the files and then to delete the files via the command line. While is was writing the wiki I remembered a Dev Mailing list message about sending users to the shell partially defeats the purpose of IPFire (bad paraphrase!). So instead of writing a wiki I wrote it a weekly fcronjob to the help everyone.

More than you ever wanted to know! Hope this helps!

Yes it does, but in the end my concern remains:

Is it not possible that an OpenVPN connection is only being used once a year or something like that? Would we then not wipe any data? Wouldn’t those databases come back again?

I will test by wiping the current data and then connecting via OpenVPN to see if the databases and graphs recover.

I am quite confident that should work just fine.

I did a quick test and the folder and RRD file are both recreated.

Cool!

I suppose a database that has not been touched in over a year would not be carrying any information because that would have been rotated out of it (because that is how RRD databases work). So maybe that can go. But I would argue only then. So we would have to check what the longest time is those databases retain any information.

Who is the RRD guru for IPFire?

I think there is an RRD dump command. I will research.

rrdtool dump /var/log/rrd/collectd/localhost/entropy/entropy.rrd

That will show you a looooong list with all the data points.

Looks like it lasts a year!

Beginning of a section of round robin:

```
</database>
</rra>
<rra>
<cf>AVERAGE</cf>
<pdp_per_row>878</pdp_per_row> 

<params>
<xff>1.0000000000e-01</xff>
</params>
<cdp_prep>
<ds>
<primary_value>2.5600000000e+02</primary_value>
<secondary_value>2.5600000000e+02</secondary_value>
<value>2.0326400000e+05</value>
<unknown_datapoints>0</unknown_datapoints>
</ds>
</cdp_prep>
<database>
 <row><v>3.3458068337e+03</v></row>
 <row><v>3.3946542141e+03</v></row>
 <row><v>3.3813018223e+03</v></row>
 <row><v>3.3501125285e+03</v></row>

```

End of the round robin:

```
 <row><v>2.5600000000e+02</v></row>
 <row><v>2.5600000000e+02</v></row>
 <row><v>2.5600000000e+02</v></row>
 <row><v>2.5600000000e+02</v></row>
 <row><v>2.5600000000e+02</v></row>
 <row><v>2.5600000000e+02</v></row>
 <row><v>2.5600000000e+02</v></row>
</database>
</rra>
<rra>
<cf>MIN</cf>
<pdp_per_row>878</pdp_per_row> 

```

Personally, I would redesign this whole thing a little. I would make them store a lot more data and I would even look for more metrics to collect. So, the extra space for a RRD database wouldn’t really bother me, even if the whole directory is going to use up gigabytes. If that is a problem for other users, I would like to know.

OK on my side! I am at 48% (Use%) for a 16 GB drive.

Yes, 16GB is perfectly fine.

I’d like to see a periodic bandwidth test (speedtest). Once in awhile I drop to near 0 Mbps bandwidth. Then it is time to restart the cable gateway and maybe the local network.

Yeah, that would be really nice to have :) I am just not sure how much traffic all IPFire devices like that would create.

Best,
-Michael

Jon

Best,
-Michael

Jon

On Aug 25, 2022, at 5:48 AM, Michael Tremer <michael.tremer@ipfire.org> wrote:

Hello Jon,

Thanks for the patch.

Sorry if this sounds like a stupid question, but what is the rationale for this?

-Michael

On 25 Aug 2022, at 01:31, Jon Murphy <jon.murphy@ipfire.org> wrote:

- Created (mostly) for old openvpn graphs
- RRD removed when no graph modification for +365 days
- chosen since graph max out is 365 days
- fcron job runs once per week
- chosen since this is just a cleanup and it doesnt need to run everyday
Note: logging can be added if needed.

Signed-off-by: Jon Murphy <jon.murphy@ipfire.org>
---
config/cron/crontab | 3 +++
1 file changed, 3 insertions(+)

diff --git a/config/cron/crontab b/config/cron/crontab
index b727531fc..7088e0b75 100644
--- a/config/cron/crontab
+++ b/config/cron/crontab
@@ -80,5 +80,8 @@ HOME=/
# Cleanup the mail spool directory
%weekly * * /usr/sbin/dma-cleanup-spool

+# Cleanup the collectd RRD (graphs)
+%weekly * * /bin/find /var/log/rrd -mtime +365 -type f -name '*.rrd' -delete -o -type d -empty -delete
+
# Update DNS trust anchor
%daily,random * * @runas(nobody) /usr/sbin/unbound-anchor -a /var/lib/unbound/root.key -c /etc/unbound/icannbundle.pem
--
2.30.2