Change to collectd.conf and collect.custom

Change to collectd.conf and collect.custom

[jon]

[-- Attachment #1: Type: text/plain, Size: 1591 bytes --]

All,

I’d like to make a change to the `/etc/collectd.conf` to remove a few monitored processes.  The items appear in the <Plugin processes> section of the config file.  And the processes appear at the bottom of the https://ipfire.localdomain:444/cgi-bin/services.cgi WebGUI page.

Why you ask?  Mostly because smbd, nmbd, squidguard, qemu, or mpd are not used (at least by me!).  And so the users of those items are not left out, I’d like to move those items to a `collect.custom`.

So overall idea is the core processes would be in `collectd.conf` file.  And the add-on processes will be in `collect.custom`.

This is a snip of the plugin process section of the `collectd.conf` file:

```
<Plugin processes>				<Plugin processes>
	Process “sshd"		to			Process "sshd"
	Process “smbd"		this			Process "squid"
	Process “nmbd”		--->			Process "charon"
	Process "squid"					Process "openvpn"
	Process “squidguard"			</Plugin>
	Process “charon"					
	Process “openvpn"					
	Process "qemu"
	Process "mpd"
</Plugin>
```


The items removed from the `collectd.conf` file would move to `collect.custom`.

<Plugin processes>
	Process "smbd"
	Process "nmbd"
	Process "squidguard"
	Process "qemu"
	Process "mpd"
</Plugin>


But the problem is I don’t want to overwrite the `/etc/collectd.custom` file incase an admin has modified it for their own use.  


The big question
================

So how do I change the `collectd.custom` file in `ipfire-2.x/config/collectd` AND not overwrite the admin’s `/etc/collectd.custom` file?

Re: Change to collectd.conf and collect.custom

[Bernhard Bitsch]

[-- Attachment #1: Type: text/plain, Size: 1984 bytes --]

Hi,

good idea.
Why not put these 'addon' definitions into a file /etc/collectd.addons ?
One problem rermains. services.cgi displays all existent collectd RRDs. 
Therefore these should be deleted in case of uninstallation of a addon.

Regards,
Bernhard

Am 17.01.2023 um 05:10 schrieb jon:
> All,
> 
> I’d like to make a change to the `/etc/collectd.conf` to remove a few monitored processes.  The items appear in the <Plugin processes> section of the config file.  And the processes appear at the bottom of the https://ipfire.localdomain:444/cgi-bin/services.cgi WebGUI page.
> 
> Why you ask?  Mostly because smbd, nmbd, squidguard, qemu, or mpd are not used (at least by me!).  And so the users of those items are not left out, I’d like to move those items to a `collect.custom`.
> 
> So overall idea is the core processes would be in `collectd.conf` file.  And the add-on processes will be in `collect.custom`.
> 
> This is a snip of the plugin process section of the `collectd.conf` file:
> 
> ```
> <Plugin processes>				<Plugin processes>
> 	Process “sshd"		to			Process "sshd"
> 	Process “smbd"		this			Process "squid"
> 	Process “nmbd”		--->			Process "charon"
> 	Process "squid"					Process "openvpn"
> 	Process “squidguard"			</Plugin>
> 	Process “charon"					
> 	Process “openvpn"					
> 	Process "qemu"
> 	Process "mpd"
> </Plugin>
> ```
> 
> 
> The items removed from the `collectd.conf` file would move to `collect.custom`.
> 
> <Plugin processes>
> 	Process "smbd"
> 	Process "nmbd"
> 	Process "squidguard"
> 	Process "qemu"
> 	Process "mpd"
> </Plugin>
> 
> 
> But the problem is I don’t want to overwrite the `/etc/collectd.custom` file incase an admin has modified it for their own use.
> 
> 
> The big question
> ================
> 
> So how do I change the `collectd.custom` file in `ipfire-2.x/config/collectd` AND not overwrite the admin’s `/etc/collectd.custom` file?
> 
> 
> 
> 

Re: Change to collectd.conf and collect.custom

[Bernhard Bitsch]

[-- Attachment #1: Type: text/plain, Size: 4638 bytes --]

Hi,


Am 26.01.2023 um 15:34 schrieb Michael Tremer:
> Hello Jon,
> 
> Hmm, but do you think they are essential for troubleshooting?
> 
> Cannot “htop” tell you exactly the same?
> 

I don't think "htop" measures the same. "htop" gives the actual values, 
the "collectd" RRDs contain the history.

Regards,
Bernhard

> Best,
> -Michael
> 
>> On 21 Jan 2023, at 17:26, jon <jon.murphy(a)ipfire.org> wrote:
>>
>> Hello!
>>
>>
>>> On Jan 20, 2023, at 10:19 AM, Michael Tremer <michael.tremer(a)ipfire.org> wrote:
>>>
>>> Hello,
>>>
>>> Can I jump in and ask what you think the purpose of this graph is?
>>
>> For me it is mostly to troubleshoot.  When I was experimenting with pmacct I would watch the Graphs.
>>
>> FYI - My Graphs are different since I was experiment with the RRD colors.
>>
>> <Screen Shot 2023-01-21 at 11.20.47 AM.png>
>>
>>>
>>> I have no idea what they should tell me. Can we not get rid of them entirely?
>>
>> I’d like to keep them but it honestly does not matter…
>>
>>>
>>> I believe the sole reason why those graphs exist is because collectd *can* collect this kind of data. But I have never looked at them and realised like “hey, this does not look right”…
>>>
>>> -Michael
>>>
>>>> On 17 Jan 2023, at 18:34, jon <jon.murphy(a)ipfire.org> wrote:
>>>>
>>>> Hello!
>>>>
>>>>> On Jan 17, 2023, at 7:50 AM, Bernhard Bitsch <bbitsch(a)ipfire.org> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> good idea.
>>>>> Why not put these 'addon' definitions into a file /etc/collectd.addons ?
>>>>
>>>> Ha! That was my first experiment!
>>>>
>>>> jon(a)deb11HPZ:~/dev/ipfire-2.x$ cat config/collectd/collectd.addon
>>>> # Use this file to add add-on processes collectd
>>>>
>>>> <Plugin processes>
>>>> Process "smbd"
>>>> Process "nmbd"
>>>> Process "squidguard"
>>>> Process "qemu"
>>>> Process "mpd"
>>>> </Plugin>
>>>> jon(a)deb11HPZ:~/dev/ipfire-2.x$
>>>>
>>>> I was worried it might cause confusion by adding one more include file.
>>>>
>>>>> One problem rermains. services.cgi displays all existent collectd RRDs. Therefore these should be deleted in case of uninstallation of a addon.
>>>>
>>>> I did see the delete issue but I am not sure how to clean-up the old RRDs.
>>>>
>>>> There is a cron to clean-up year old RRDs:
>>>>
>>>> [root(a)ipfire ~] # fcrontab -l | grep -i rrd
>>>> 2023-01-17 11:58:14  INFO listing root's fcrontab
>>>> # Cleanup the collectd RRD (graphs)
>>>> %weekly * * /bin/find /var/log/rrd -mtime +365 -type f -name '*.rrd' -delete -o -type d -empty -delete
>>>> [root(a)ipfire ~] #
>>>>
>>>> I can easily create
>>>> /bin/find /var/log/rrd/collectd/localhost/processes* -mtime 1 -type f -name '*.rrd'
>>>>
>>>> But this type of clean-up seems too hacked...
>>>>
>>>>
>>>>>
>>>>> Regards,
>>>>> Bernhard
>>>>>
>>>>> Am 17.01.2023 um 05:10 schrieb jon:
>>>>>> All,
>>>>>> I’d like to make a change to the `/etc/collectd.conf` to remove a few monitored processes.  The items appear in the <Plugin processes> section of the config file.  And the processes appear at the bottom of the https://ipfire.localdomain:444/cgi-bin/services.cgi WebGUI page.
>>>>>> Why you ask?  Mostly because smbd, nmbd, squidguard, qemu, or mpd are not used (at least by me!).  And so the users of those items are not left out, I’d like to move those items to a `collect.custom`.
>>>>>> So overall idea is the core processes would be in `collectd.conf` file.  And the add-on processes will be in `collect.custom`.
>>>>>> This is a snip of the plugin process section of the `collectd.conf` file:
>>>>>> ```
>>>>>> <Plugin processes> <Plugin processes>
>>>>>> Process “sshd" to Process "sshd"
>>>>>> Process “smbd" this Process "squid"
>>>>>> Process “nmbd” ---> Process "charon"
>>>>>> Process "squid" Process "openvpn"
>>>>>> Process “squidguard" </Plugin>
>>>>>> Process “charon"
>>>>>> Process “openvpn"
>>>>>> Process "qemu"
>>>>>> Process "mpd"
>>>>>> </Plugin>
>>>>>> ```
>>>>>> The items removed from the `collectd.conf` file would move to `collect.custom`.
>>>>>> <Plugin processes>
>>>>>> Process "smbd"
>>>>>> Process "nmbd"
>>>>>> Process "squidguard"
>>>>>> Process "qemu"
>>>>>> Process "mpd"
>>>>>> </Plugin>
>>>>>> But the problem is I don’t want to overwrite the `/etc/collectd.custom` file incase an admin has modified it for their own use.
>>>>>> The big question
>>>>>> ================
>>>>>> So how do I change the `collectd.custom` file in `ipfire-2.x/config/collectd` AND not overwrite the admin’s `/etc/collectd.custom` file?
>>>>
>>>
>>
> 

Re: Change to collectd.conf and collect.custom

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 4350 bytes --]

Hello Jon,

Hmm, but do you think they are essential for troubleshooting?

Cannot “htop” tell you exactly the same?

Best,
-Michael

> On 21 Jan 2023, at 17:26, jon <jon.murphy(a)ipfire.org> wrote:
> 
> Hello!
> 
> 
>> On Jan 20, 2023, at 10:19 AM, Michael Tremer <michael.tremer(a)ipfire.org> wrote:
>> 
>> Hello,
>> 
>> Can I jump in and ask what you think the purpose of this graph is?
> 
> For me it is mostly to troubleshoot.  When I was experimenting with pmacct I would watch the Graphs.
> 
> FYI - My Graphs are different since I was experiment with the RRD colors.
> 
> <Screen Shot 2023-01-21 at 11.20.47 AM.png>
> 
>> 
>> I have no idea what they should tell me. Can we not get rid of them entirely?
> 
> I’d like to keep them but it honestly does not matter…
> 
>> 
>> I believe the sole reason why those graphs exist is because collectd *can* collect this kind of data. But I have never looked at them and realised like “hey, this does not look right”…
>> 
>> -Michael
>> 
>>> On 17 Jan 2023, at 18:34, jon <jon.murphy(a)ipfire.org> wrote:
>>> 
>>> Hello!
>>> 
>>>> On Jan 17, 2023, at 7:50 AM, Bernhard Bitsch <bbitsch(a)ipfire.org> wrote:
>>>> 
>>>> Hi,
>>>> 
>>>> good idea.
>>>> Why not put these 'addon' definitions into a file /etc/collectd.addons ?
>>> 
>>> Ha! That was my first experiment!
>>> 
>>> jon(a)deb11HPZ:~/dev/ipfire-2.x$ cat config/collectd/collectd.addon
>>> # Use this file to add add-on processes collectd
>>> 
>>> <Plugin processes>
>>> Process "smbd"
>>> Process "nmbd"
>>> Process "squidguard"
>>> Process "qemu"
>>> Process "mpd"
>>> </Plugin>
>>> jon(a)deb11HPZ:~/dev/ipfire-2.x$ 
>>> 
>>> I was worried it might cause confusion by adding one more include file.
>>> 
>>>> One problem rermains. services.cgi displays all existent collectd RRDs. Therefore these should be deleted in case of uninstallation of a addon.
>>> 
>>> I did see the delete issue but I am not sure how to clean-up the old RRDs.
>>> 
>>> There is a cron to clean-up year old RRDs:
>>> 
>>> [root(a)ipfire ~] # fcrontab -l | grep -i rrd
>>> 2023-01-17 11:58:14  INFO listing root's fcrontab
>>> # Cleanup the collectd RRD (graphs)
>>> %weekly * * /bin/find /var/log/rrd -mtime +365 -type f -name '*.rrd' -delete -o -type d -empty -delete
>>> [root(a)ipfire ~] # 
>>> 
>>> I can easily create 
>>> /bin/find /var/log/rrd/collectd/localhost/processes* -mtime 1 -type f -name '*.rrd'
>>> 
>>> But this type of clean-up seems too hacked...
>>> 
>>> 
>>>> 
>>>> Regards,
>>>> Bernhard
>>>> 
>>>> Am 17.01.2023 um 05:10 schrieb jon:
>>>>> All,
>>>>> I’d like to make a change to the `/etc/collectd.conf` to remove a few monitored processes.  The items appear in the <Plugin processes> section of the config file.  And the processes appear at the bottom of the https://ipfire.localdomain:444/cgi-bin/services.cgi WebGUI page.
>>>>> Why you ask?  Mostly because smbd, nmbd, squidguard, qemu, or mpd are not used (at least by me!).  And so the users of those items are not left out, I’d like to move those items to a `collect.custom`.
>>>>> So overall idea is the core processes would be in `collectd.conf` file.  And the add-on processes will be in `collect.custom`.
>>>>> This is a snip of the plugin process section of the `collectd.conf` file:
>>>>> ```
>>>>> <Plugin processes> <Plugin processes>
>>>>> Process “sshd" to Process "sshd"
>>>>> Process “smbd" this Process "squid"
>>>>> Process “nmbd” ---> Process "charon"
>>>>> Process "squid" Process "openvpn"
>>>>> Process “squidguard" </Plugin>
>>>>> Process “charon" 
>>>>> Process “openvpn" 
>>>>> Process "qemu"
>>>>> Process "mpd"
>>>>> </Plugin>
>>>>> ```
>>>>> The items removed from the `collectd.conf` file would move to `collect.custom`.
>>>>> <Plugin processes>
>>>>> Process "smbd"
>>>>> Process "nmbd"
>>>>> Process "squidguard"
>>>>> Process "qemu"
>>>>> Process "mpd"
>>>>> </Plugin>
>>>>> But the problem is I don’t want to overwrite the `/etc/collectd.custom` file incase an admin has modified it for their own use.
>>>>> The big question
>>>>> ================
>>>>> So how do I change the `collectd.custom` file in `ipfire-2.x/config/collectd` AND not overwrite the admin’s `/etc/collectd.custom` file?
>>> 
>> 
> 

Re: Change to collectd.conf and collect.custom

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 3510 bytes --]

Hello,

Can I jump in and ask what you think the purpose of this graph is?

I have no idea what they should tell me. Can we not get rid of them entirely?

I believe the sole reason why those graphs exist is because collectd *can* collect this kind of data. But I have never looked at them and realised like “hey, this does not look right”…

-Michael

> On 17 Jan 2023, at 18:34, jon <jon.murphy(a)ipfire.org> wrote:
> 
> Hello!
> 
>> On Jan 17, 2023, at 7:50 AM, Bernhard Bitsch <bbitsch(a)ipfire.org> wrote:
>> 
>> Hi,
>> 
>> good idea.
>> Why not put these 'addon' definitions into a file /etc/collectd.addons ?
> 
> Ha! That was my first experiment!
> 
> jon(a)deb11HPZ:~/dev/ipfire-2.x$ cat config/collectd/collectd.addon
> # Use this file to add add-on processes collectd
> 
> <Plugin processes>
> Process "smbd"
> Process "nmbd"
> Process "squidguard"
> Process "qemu"
> Process "mpd"
> </Plugin>
> jon(a)deb11HPZ:~/dev/ipfire-2.x$ 
> 
> I was worried it might cause confusion by adding one more include file.
> 
>> One problem rermains. services.cgi displays all existent collectd RRDs. Therefore these should be deleted in case of uninstallation of a addon.
> 
> I did see the delete issue but I am not sure how to clean-up the old RRDs.
> 
> There is a cron to clean-up year old RRDs:
> 
> [root(a)ipfire ~] # fcrontab -l | grep -i rrd
> 2023-01-17 11:58:14  INFO listing root's fcrontab
> # Cleanup the collectd RRD (graphs)
> %weekly * * /bin/find /var/log/rrd -mtime +365 -type f -name '*.rrd' -delete -o -type d -empty -delete
> [root(a)ipfire ~] # 
> 
> I can easily create 
> /bin/find /var/log/rrd/collectd/localhost/processes* -mtime 1 -type f -name '*.rrd'
> 
> But this type of clean-up seems too hacked...
> 
> 
>> 
>> Regards,
>> Bernhard
>> 
>> Am 17.01.2023 um 05:10 schrieb jon:
>>> All,
>>> I’d like to make a change to the `/etc/collectd.conf` to remove a few monitored processes.  The items appear in the <Plugin processes> section of the config file.  And the processes appear at the bottom of the https://ipfire.localdomain:444/cgi-bin/services.cgi WebGUI page.
>>> Why you ask?  Mostly because smbd, nmbd, squidguard, qemu, or mpd are not used (at least by me!).  And so the users of those items are not left out, I’d like to move those items to a `collect.custom`.
>>> So overall idea is the core processes would be in `collectd.conf` file.  And the add-on processes will be in `collect.custom`.
>>> This is a snip of the plugin process section of the `collectd.conf` file:
>>> ```
>>> <Plugin processes> <Plugin processes>
>>> Process “sshd" to Process "sshd"
>>> Process “smbd" this Process "squid"
>>> Process “nmbd” ---> Process "charon"
>>> Process "squid" Process "openvpn"
>>> Process “squidguard" </Plugin>
>>> Process “charon" 
>>> Process “openvpn" 
>>> Process "qemu"
>>> Process "mpd"
>>> </Plugin>
>>> ```
>>> The items removed from the `collectd.conf` file would move to `collect.custom`.
>>> <Plugin processes>
>>> Process "smbd"
>>> Process "nmbd"
>>> Process "squidguard"
>>> Process "qemu"
>>> Process "mpd"
>>> </Plugin>
>>> But the problem is I don’t want to overwrite the `/etc/collectd.custom` file incase an admin has modified it for their own use.
>>> The big question
>>> ================
>>> So how do I change the `collectd.custom` file in `ipfire-2.x/config/collectd` AND not overwrite the admin’s `/etc/collectd.custom` file?
>