Re: Updating 'vnstat' to 2.6 - need some help with necessary changes

[Michael Tremer <michael.tremer@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 5595 bytes --]

Hi,

Thanks for looking into this.

> On 22 Mar 2020, at 12:08, Matthias Fischer <matthias.fischer(a)ipfire.org> wrote:
> 
> Hi,
> 
> I successfully built 'vnstat 2.6' under Core 142/32bit.
> 
> Right now a test version is running on my testmachine under Core 138 -
> no seen problems.
> 
> Upgrading lfs and rootfile was enough for building, but I'm struggling
> (a bit) with the necessary changes in some IPFire scripts. So I'm asking
> here if there are ideas how to do this the best way.
> 
> Be warned, this is long... ;-)
> 
> Situation:
> 1. Building is fine, but since 'vnstat 2.0' "C struct dump database has
> been replaced with sqlite database".
> See:
> [humdi-URL deleted ... Thanks Peter!]
> 
> This is one of the most important changes! The old interface files
> '.green0', 'red0' etc. are only needed *once* after the update. After a
> successful update and import (this is done automatically at the first
> start of 'vnstatd -d --alwaysadd', see below), these files are obsolete
> and can be deleted.
> 
> What I changed by hand to get this running:
> 1. GIT: Added '/usr/sbin/vnstatd' to rootfile. The new version needs it.
> 
> 2. GIT: Deleted 'touch /var/log/vnstat/tag' from lfs and
> '/var/log/vnstat/tag' from rootfile.
> 
> *Question*:
> Perhaps the 'sed'-line in 'lfs' replacing 'eth0' with 'green0' can be
> deleted, too ("leave empty for automatic selection")!?
> 
> 3. Deleted the following lines from '/usr/local/bin/makegraphs':
> ...
> ## Update vnstat
> system ('/usr/bin/vnstat -u');
> ...
> 
> 4. Stopped 'vnstat' and 'collectd' (to be sure, I stopped both, perhaps
> only stopping 'vnstat' is needed here).
> 
> 5. Installed the new files on my testmachine:
> /etc/vnstat.conf
> /usr/bin/vnstat
> /usr/bin/vnstati
> /usr/sbin/vnstatd
> 
> 6. Started 'vnstatd -d --alwaysadd'.
> '-d' => fork process to background
> '--alwaysadd' => always add new interfaces even when some dbs exist
> 
> *Result*:
> ...
> [root(a)ipfiretest vnstat]# vnstatd -d --alwaysadd
> Importing data from legacy database "green0".
> Importing data from legacy database "blue0".
> Importing data from legacy database "red0".
> No interfaces found in database, adding available interfaces...
> ...
> 
> This created a new file: '/var/log/vnstat/vnstat.db' (SQL - see above),
> plus 'vnstatd' running in background with a PID.

I suppose it makes a lot of sense to migrate to vnstatd instead of running “vnstat -u” from cron.

> *Question*:
> Where and how should the start/stop/reload of 'vnstatd' be added?
> I think of adding appopriate commands to '/etc/initd/vnstat' but can't
> get a grip on this. Would need some help with this.
> 
> Possibilities (from 'man vnstatd'):
> "SIGNALS
> The daemon is listening to signals SIGHUP, SIGINT and SIGTERM. Sending
> the SIGHUP signal to the daemon will cause cached data to be written to
> disk, a rescan of the database directory and a reload of settings from
> the configuration file. However, the pid file location will not be
> changed even if it's configuration setting has been modified.
> 
> SIGTERM and SIGINT signals will cause the daemon to write all cached
> data to disk and then exit."
> 
> start section would need:
> 'vnstatd -d --alwaysadd'

Yes, you can simply use the standard functions that our initscripts have here.

https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=src/initscripts/system/template;h=69dc2a6a2202fb30da61a7fff6625aea531802e9;hb=HEAD

loadproc will start the daemon and print “OK”.

> stop section:
> SIGTERM or SIGINT (tested with 'htop' sending SIGTERM: works)

killproc will send SIGTERM.

> reload section (if config was changed):
> SIGHUP

reloadproc will handle this for you :)

> How to do this the best way? I'm stuck with this.
> 
> 7. Deleted all old interface files in '/var/log/vnstat (.green0, .red0,
> etc.)
> 
> *Question*:
> In my opinion, this could be done after upgrading *plus* a successful
> import of the old interface files *plus* a successful start of 'vnstatd
> -d --alwaysadd'. If 'vnstatd.db' was successfully created, these files
> are not needed anymore.
> 
> How to do this the best way?

Should we add a function to the initscript that checks if any files like that exist and then run a conversion command?

That would be safe, even if people restore an older backup.

> 8. Since the graphs are no longer updated through 'makegraphs', I
> created a symlink in '/etc/fcron.minutely' to '/var/usr/sbin/vnstat'
> (*No* parameters, '-u' doesn't exist anymore).

Makegraphs does some other things, too. I am not sure if we can remove the symlink, yet.

> Now every minute 'vnstat' now grabs the data collected by'vnstatd'.

The vnstat call can of course be removed.

> 
> 9. Edited 'traffic.cgi', changed:
> 
> system("/usr/bin/vnstati -c 5
> 
> To:
> 
> system("/usr/bin/vnstati -c 2
> 
> This keeps the graphs a bit more "updated".
> From 'man vnstat':
> '-c (--cache-time) => Update output file only if at least *time* minutes
> have passed since the previous file update.

I think once a minute is cool too. It does not take a lot to generate those and I think it is valuable that they are recent.

> Attachments:
> Screenshot fomr my testmachine after importing the old interface files
> from my production machine.
> I made some minor adjustments to 'vnstat.conf', attached.

What are those?

-Michael

> 
> Thanks for any help!
> 
> Best,
> Matthias
> 
> 
> <vnstat.conf><vnstat_2_6_test.png>