For details see: https://humdi.net/vnstat/CHANGES
Changed "SaveInterval 5" to "SaveInterval 1" in '/etc/vnstat.conf', triggered by https://forum.ipfire.org/viewtopic.php?f=22&t=20448 to avoid data loss with 1Gbit connections and high traffic.
Thanks to BeBiMa for the man page link (https://humdi.net/vnstat/man/vnstatd.html). ;-)
Best, Matthias
Signed-off-by: Matthias Fischer matthias.fischer@ipfire.org --- lfs/vnstat | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/lfs/vnstat b/lfs/vnstat index 376a1e999..a61bc5ed2 100644 --- a/lfs/vnstat +++ b/lfs/vnstat @@ -1,7 +1,7 @@ ############################################################################### # # # IPFire.org - A linux based firewall # -# Copyright (C) 2007-2017 IPFire Team info@ipfire.org # +# Copyright (C) 2007-2018 IPFire Team info@ipfire.org # # # # This program is free software: you can redistribute it and/or modify # # it under the terms of the GNU General Public License as published by # @@ -24,7 +24,7 @@
include Config
-VER = 1.17 +VER = 1.18
THISAPP = vnstat-$(VER) DL_FILE = $(THISAPP).tar.gz @@ -40,7 +40,7 @@ objects = $(DL_FILE)
$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
-$(DL_FILE)_MD5 = 8de1c7e40806509943804bb4b26f5409 +$(DL_FILE)_MD5 = c9abaeb0ce758c16f6cdfa247bd8606c
install : $(TARGET)
@@ -81,6 +81,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) cd $(DIR_APP) && make all $(MAKETUNING) LOCAL_CONFIGURE_OPTIONS="--enable-readline=yes" cd $(DIR_APP) && make install sed -i 's|eth0|green0|g' /etc/vnstat.conf + sed -i 's|SaveInterval 5|SaveInterval 1|g' /etc/vnstat.conf sed -i 's|/var/lib/vnstat|/var/log/vnstat|g' /etc/vnstat.conf sed -i 's|/var/log/vnstat/vnstat.log|/var/log/vnstat.log|g' /etc/vnstat.conf sed -i 's|/var/run/vnstat/vnstat.pid|/var/run/vnstat.pid|g' /etc/vnstat.conf
Hi,
this would have been better split into two patches so that one thing can be merged without the other.
I will take it both together for now.
Best, -Michael
P.S. I did not appreciate that comment of that forum user about this bug. Bugs happen. If he doesn't like it, he can take lots of €€€€ into his hand and pay other people to get their bugs.
On Tue, 2018-03-20 at 20:46 +0100, Matthias Fischer wrote:
For details see: https://humdi.net/vnstat/CHANGES
Changed "SaveInterval 5" to "SaveInterval 1" in '/etc/vnstat.conf', triggered by https://forum.ipfire.org/viewtopic.php?f=22&t=20448 to avoid data loss with 1Gbit connections and high traffic.
Thanks to BeBiMa for the man page link (https://humdi.net/vnstat/man/vnstatd.h tml). ;-)
Best, Matthias
Signed-off-by: Matthias Fischer matthias.fischer@ipfire.org
lfs/vnstat | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/lfs/vnstat b/lfs/vnstat index 376a1e999..a61bc5ed2 100644 --- a/lfs/vnstat +++ b/lfs/vnstat @@ -1,7 +1,7 @@ ############################################################################# ## # # # IPFire.org - A linux based firewall # -# Copyright (C) 2007-2017 IPFire Team info@ipfire.org # +# Copyright (C) 2007-2018 IPFire Team info@ipfire.org # # # # This program is free software: you can redistribute it and/or modify # # it under the terms of the GNU General Public License as published by # @@ -24,7 +24,7 @@
include Config
-VER = 1.17 +VER = 1.18
THISAPP = vnstat-$(VER) DL_FILE = $(THISAPP).tar.gz @@ -40,7 +40,7 @@ objects = $(DL_FILE)
$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
-$(DL_FILE)_MD5 = 8de1c7e40806509943804bb4b26f5409 +$(DL_FILE)_MD5 = c9abaeb0ce758c16f6cdfa247bd8606c
install : $(TARGET)
@@ -81,6 +81,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) cd $(DIR_APP) && make all $(MAKETUNING) LOCAL_CONFIGUR E_OPTIONS="--enable-readline=yes" cd $(DIR_APP) && make install sed -i 's|eth0|green0|g' /etc/vnstat.conf
- sed -i 's|SaveInterval 5|SaveInterval 1|g' /etc/vnstat.conf sed -i 's|/var/lib/vnstat|/var/log/vnstat|g' /etc/vnstat.conf sed -i 's|/var/log/vnstat/vnstat.log|/var/log/vnstat.log|g'
/etc/vnstat.conf sed -i 's|/var/run/vnstat/vnstat.pid|/var/run/vnstat.pid|g' /etc/vnstat.conf
Hi Michael,
On 20.03.2018 21:34, Michael Tremer wrote:
this would have been better split into two patches so that one thing can be merged without the other.
*Sigh* Yes, I mentioned that. The next morning. "It was late, and I needed the money...!" :-)
I will take it both together for now.
:-)
Best, -Michael
P.S. I did not appreciate that comment of that forum user about this bug. Bugs happen. If he doesn't like it, he can take lots of €€€€ into his hand and pay other people to get their bugs.
Me too. But I tried to be patient, took a deep breath and tried to keep calm and ~professional. Wasn't easy. Besides, I don't think this is really a "bug". 'vnstat' can handle his speed and download rates with no problems. It just happened that they were a bit too much for the standard config - in *his* case.
Best, Matthias
Gesendet: Mittwoch, 21. März 2018 um 18:38 Uhr Von: "Matthias Fischer" matthias.fischer@ipfire.org An: "Michael Tremer" michael.tremer@ipfire.org, "IPFire: Development-List" development@lists.ipfire.org Betreff: Re: [PATCH] vnstat: Update to 1.18
P.S. I did not appreciate that comment of that forum user about this bug. Bugs happen. If he doesn't like it, he can take lots of €€€€ into his hand and pay other people to get their bugs.
Me too. But I tried to be patient, took a deep breath and tried to keep calm and ~professional. Wasn't easy. Besides, I don't think this is really a "bug". 'vnstat' can handle his speed and download rates with no problems. It just happened that they were a bit too much for the standard config - in *his* case.
Best, Matthias
Hi,
just my opinion ( I've investigated now a bit about this "problem" ;) ) The "solution" isn't really that. The changed parameter is only used by vnstatd, which isn't used in IPFire. I can't imagine what did the remedy. We don't know exactly his configuration. Therefore I gave him some hints to help clarifying the case. If he doesn't answer, just forget his unqualified comment.
Best, Bernhard
Gesendet: Mittwoch, 21. März 2018 um 20:30 Uhr Von: "Bernhard Bitsch" Bernhard.Bitsch@gmx.de An: "Matthias Fischer" matthias.fischer@ipfire.org Cc: "IPFire: Development-List" development@lists.ipfire.org Betreff: Aw: Re: [PATCH] vnstat: Update to 1.18
Gesendet: Mittwoch, 21. März 2018 um 18:38 Uhr Von: "Matthias Fischer" matthias.fischer@ipfire.org An: "Michael Tremer" michael.tremer@ipfire.org, "IPFire: Development-List" development@lists.ipfire.org Betreff: Re: [PATCH] vnstat: Update to 1.18
P.S. I did not appreciate that comment of that forum user about this bug. Bugs happen. If he doesn't like it, he can take lots of €€€€ into his hand and pay other people to get their bugs.
Me too. But I tried to be patient, took a deep breath and tried to keep calm and ~professional. Wasn't easy. Besides, I don't think this is really a "bug". 'vnstat' can handle his speed and download rates with no problems. It just happened that they were a bit too much for the standard config - in *his* case.
Best, Matthias
Hi,
just my opinion ( I've investigated now a bit about this "problem" ;) ) The "solution" isn't really that. The changed parameter is only used by vnstatd, which isn't used in IPFire. I can't imagine what did the remedy. We don't know exactly his configuration. Therefore I gave him some hints to help clarifying the case. If he doesn't answer, just forget his unqualified comment.
Best, Bernhard
News about the vnstat problem. Intensive PMs with Zonediver showed that the effect is caused by 32 bit byte counters of the NIC. ( vnstat reads /proc/net/dev ) He changed the fcrontab period for makegraphs to 3 minutes now. Amount of bytes received in this period is less than 2^32.
Remain two questions: - Is this a normal behaviour for the combination of i210-T1 NIC and associated driver or is this specific to his system? - Can we use the vnstatd daemon? This would allow to increase the vnstat update frequency independent from makegraphs.
I've found a thumb rule for the maximal update period for a given bandwidth. I've documented this in the forum thread.
Best, Bernhard
On Fri, 2018-03-23 at 13:39 +0100, Bernhard Bitsch wrote:
Gesendet: Mittwoch, 21. März 2018 um 20:30 Uhr Von: "Bernhard Bitsch" Bernhard.Bitsch@gmx.de An: "Matthias Fischer" matthias.fischer@ipfire.org Cc: "IPFire: Development-List" development@lists.ipfire.org Betreff: Aw: Re: [PATCH] vnstat: Update to 1.18
Gesendet: Mittwoch, 21. März 2018 um 18:38 Uhr Von: "Matthias Fischer" matthias.fischer@ipfire.org An: "Michael Tremer" michael.tremer@ipfire.org, "IPFire: Development- List" development@lists.ipfire.org Betreff: Re: [PATCH] vnstat: Update to 1.18
P.S. I did not appreciate that comment of that forum user about this bug. Bugs happen. If he doesn't like it, he can take lots of €€€€ into his hand and pay other people to get their bugs.
Me too. But I tried to be patient, took a deep breath and tried to keep calm and ~professional. Wasn't easy. Besides, I don't think this is really a "bug". 'vnstat' can handle his speed and download rates with no problems. It just happened that they were a bit too much for the standard config - in *his* case.
Best, Matthias
Hi,
just my opinion ( I've investigated now a bit about this "problem" ;) ) The "solution" isn't really that. The changed parameter is only used by vnstatd, which isn't used in IPFire. I can't imagine what did the remedy. We don't know exactly his configuration. Therefore I gave him some hints to help clarifying the case. If he doesn't answer, just forget his unqualified comment.
Best, Bernhard
News about the vnstat problem. Intensive PMs with Zonediver showed that the effect is caused by 32 bit byte counters of the NIC. ( vnstat reads /proc/net/dev ) He changed the fcrontab period for makegraphs to 3 minutes now. Amount of bytes received in this period is less than 2^32.
Remain two questions:
- Is this a normal behaviour for the combination of i210-T1 NIC and associated
driver or is this specific to his system?
I cannot speak for that NIC because I do not use that anywhere. Surprised to hear this the first time since the i210 is also on the APU boards and it actually is a stripped down version of a bigger Intel NIC.
- Can we use the vnstatd daemon? This would allow to increase the vnstat
update frequency independent from makegraphs.
I suppose we could. Arne has built this into the distribution and could probably comment more. If I remember correctly, the daemon wasn't available when we adopted vnstat.
But even collecting once a minute wouldn't be enough because with max. theoretical throughput of a Gigabit NIC, 2^32 bytes are transferred in 32 seconds. And if this is only a problem with that specific NIC, I actually wouldn't want to collect every 30 second on all systems. Wastes a lot of resources and costs a lot of write cycles on flash.
I've found a thumb rule for the maximal update period for a given bandwidth. I've documented this in the forum thread.
Best, Bernhard
-Michael
Gesendet: Freitag, 23. März 2018 um 16:54 Uhr Von: "Michael Tremer" michael.tremer@ipfire.org An: "Bernhard Bitsch" Bernhard.Bitsch@gmx.de, "IPFire Development" development@lists.ipfire.org Betreff: Re: Aw: Re: [PATCH] vnstat: Update to 1.18
On Fri, 2018-03-23 at 13:39 +0100, Bernhard Bitsch wrote:
Gesendet: Mittwoch, 21. März 2018 um 20:30 Uhr Von: "Bernhard Bitsch" Bernhard.Bitsch@gmx.de An: "Matthias Fischer" matthias.fischer@ipfire.org Cc: "IPFire: Development-List" development@lists.ipfire.org Betreff: Aw: Re: [PATCH] vnstat: Update to 1.18
Gesendet: Mittwoch, 21. März 2018 um 18:38 Uhr Von: "Matthias Fischer" matthias.fischer@ipfire.org An: "Michael Tremer" michael.tremer@ipfire.org, "IPFire: Development- List" development@lists.ipfire.org Betreff: Re: [PATCH] vnstat: Update to 1.18
P.S. I did not appreciate that comment of that forum user about this bug. Bugs happen. If he doesn't like it, he can take lots of €€€€ into his hand and pay other people to get their bugs.
Me too. But I tried to be patient, took a deep breath and tried to keep calm and ~professional. Wasn't easy. Besides, I don't think this is really a "bug". 'vnstat' can handle his speed and download rates with no problems. It just happened that they were a bit too much for the standard config - in *his* case.
Best, Matthias
Hi,
just my opinion ( I've investigated now a bit about this "problem" ;) ) The "solution" isn't really that. The changed parameter is only used by vnstatd, which isn't used in IPFire. I can't imagine what did the remedy. We don't know exactly his configuration. Therefore I gave him some hints to help clarifying the case. If he doesn't answer, just forget his unqualified comment.
Best, Bernhard
News about the vnstat problem. Intensive PMs with Zonediver showed that the effect is caused by 32 bit byte counters of the NIC. ( vnstat reads /proc/net/dev ) He changed the fcrontab period for makegraphs to 3 minutes now. Amount of bytes received in this period is less than 2^32.
Remain two questions:
- Is this a normal behaviour for the combination of i210-T1 NIC and associated
driver or is this specific to his system?
I cannot speak for that NIC because I do not use that anywhere. Surprised to hear this the first time since the i210 is also on the APU boards and it actually is a stripped down version of a bigger Intel NIC.
- Can we use the vnstatd daemon? This would allow to increase the vnstat
update frequency independent from makegraphs.
I suppose we could. Arne has built this into the distribution and could probably comment more. If I remember correctly, the daemon wasn't available when we adopted vnstat.
That's right.
But even collecting once a minute wouldn't be enough because with max. theoretical throughput of a Gigabit NIC, 2^32 bytes are transferred in 32 seconds. And if this is only a problem with that specific NIC, I actually wouldn't want to collect every 30 second on all systems. Wastes a lot of resources and costs a lot of write cycles on flash.
I've found a thumb rule for the maximal update period for a given bandwidth. I've documented this in the forum thread.
Best, Bernhard
-Michael
The default value of vnstatd are 30 seconds for UpdateInterval (short enough for 2^32 bytes on 1 GBit/s ), writes to disk is ruled by parameter SaveInterval ( default 5 minutes, which is our period from cron ). Thus vnstatd should give a better resolution ( for some systems without information leaks! ) with similiar load as the cron based solution implemented at the moment.
-Bernhard
The daemon has the disadvantage of static configuration of all interfaces.
This isn’t an information leak. It is a bug in the device driver of that NIC and therefore I don’t think that the solution is to play around with vnstat. The solution is to fix that driver.
-Michael
On 28 Mar 2018, at 12:55, Bernhard Bitsch Bernhard.Bitsch@gmx.de wrote:
Gesendet: Freitag, 23. März 2018 um 16:54 Uhr Von: "Michael Tremer" michael.tremer@ipfire.org An: "Bernhard Bitsch" Bernhard.Bitsch@gmx.de, "IPFire Development" development@lists.ipfire.org Betreff: Re: Aw: Re: [PATCH] vnstat: Update to 1.18
On Fri, 2018-03-23 at 13:39 +0100, Bernhard Bitsch wrote:
Gesendet: Mittwoch, 21. März 2018 um 20:30 Uhr Von: "Bernhard Bitsch" Bernhard.Bitsch@gmx.de An: "Matthias Fischer" matthias.fischer@ipfire.org Cc: "IPFire: Development-List" development@lists.ipfire.org Betreff: Aw: Re: [PATCH] vnstat: Update to 1.18
Gesendet: Mittwoch, 21. März 2018 um 18:38 Uhr Von: "Matthias Fischer" matthias.fischer@ipfire.org An: "Michael Tremer" michael.tremer@ipfire.org, "IPFire: Development- List" development@lists.ipfire.org Betreff: Re: [PATCH] vnstat: Update to 1.18
P.S. I did not appreciate that comment of that forum user about this bug. Bugs happen. If he doesn't like it, he can take lots of €€€€ into his hand and pay other people to get their bugs.
Me too. But I tried to be patient, took a deep breath and tried to keep calm and ~professional. Wasn't easy. Besides, I don't think this is really a "bug". 'vnstat' can handle his speed and download rates with no problems. It just happened that they were a bit too much for the standard config - in *his* case.
Best, Matthias
Hi,
just my opinion ( I've investigated now a bit about this "problem" ;) ) The "solution" isn't really that. The changed parameter is only used by vnstatd, which isn't used in IPFire. I can't imagine what did the remedy. We don't know exactly his configuration. Therefore I gave him some hints to help clarifying the case. If he doesn't answer, just forget his unqualified comment.
Best, Bernhard
News about the vnstat problem. Intensive PMs with Zonediver showed that the effect is caused by 32 bit byte counters of the NIC. ( vnstat reads /proc/net/dev ) He changed the fcrontab period for makegraphs to 3 minutes now. Amount of bytes received in this period is less than 2^32.
Remain two questions:
- Is this a normal behaviour for the combination of i210-T1 NIC and associated
driver or is this specific to his system?
I cannot speak for that NIC because I do not use that anywhere. Surprised to hear this the first time since the i210 is also on the APU boards and it actually is a stripped down version of a bigger Intel NIC.
- Can we use the vnstatd daemon? This would allow to increase the vnstat
update frequency independent from makegraphs.
I suppose we could. Arne has built this into the distribution and could probably comment more. If I remember correctly, the daemon wasn't available when we adopted vnstat.
That's right.
But even collecting once a minute wouldn't be enough because with max. theoretical throughput of a Gigabit NIC, 2^32 bytes are transferred in 32 seconds. And if this is only a problem with that specific NIC, I actually wouldn't want to collect every 30 second on all systems. Wastes a lot of resources and costs a lot of write cycles on flash.
I've found a thumb rule for the maximal update period for a given bandwidth. I've documented this in the forum thread.
Best, Bernhard
-Michael
The default value of vnstatd are 30 seconds for UpdateInterval (short enough for 2^32 bytes on 1 GBit/s ), writes to disk is ruled by parameter SaveInterval ( default 5 minutes, which is our period from cron ). Thus vnstatd should give a better resolution ( for some systems without information leaks! ) with similiar load as the cron based solution implemented at the moment.
-Bernhard
What do you mean with static configuration? As far as I've seen in the sources, the functionality of vnstatd and 'vnstat -u' should be identical.
You're right about the bug in the device driver ( which causes an information leak ), but vnstatd can help as work-around.
-Bernhard
Gesendet: Mittwoch, 28. März 2018 um 20:04 Uhr Von: "Michael Tremer" michael.tremer@ipfire.org An: "Bernhard Bitsch" Bernhard.Bitsch@gmx.de Cc: "IPFire Development" development@lists.ipfire.org Betreff: Re: Aw: Re: Re: [PATCH] vnstat: Update to 1.18
The daemon has the disadvantage of static configuration of all interfaces.
This isn’t an information leak. It is a bug in the device driver of that NIC and therefore I don’t think that the solution is to play around with vnstat. The solution is to fix that driver.
-Michael
On 28 Mar 2018, at 12:55, Bernhard Bitsch Bernhard.Bitsch@gmx.de wrote:
Gesendet: Freitag, 23. März 2018 um 16:54 Uhr Von: "Michael Tremer" michael.tremer@ipfire.org An: "Bernhard Bitsch" Bernhard.Bitsch@gmx.de, "IPFire Development" development@lists.ipfire.org Betreff: Re: Aw: Re: [PATCH] vnstat: Update to 1.18
On Fri, 2018-03-23 at 13:39 +0100, Bernhard Bitsch wrote:
Gesendet: Mittwoch, 21. März 2018 um 20:30 Uhr Von: "Bernhard Bitsch" Bernhard.Bitsch@gmx.de An: "Matthias Fischer" matthias.fischer@ipfire.org Cc: "IPFire: Development-List" development@lists.ipfire.org Betreff: Aw: Re: [PATCH] vnstat: Update to 1.18
Gesendet: Mittwoch, 21. März 2018 um 18:38 Uhr Von: "Matthias Fischer" matthias.fischer@ipfire.org An: "Michael Tremer" michael.tremer@ipfire.org, "IPFire: Development- List" development@lists.ipfire.org Betreff: Re: [PATCH] vnstat: Update to 1.18
> > P.S. I did not appreciate that comment of that forum user about this > bug. Bugs > happen. If he doesn't like it, he can take lots of €€€€ into his hand > and pay > other people to get their bugs.
Me too. But I tried to be patient, took a deep breath and tried to keep calm and ~professional. Wasn't easy. Besides, I don't think this is really a "bug". 'vnstat' can handle his speed and download rates with no problems. It just happened that they were a bit too much for the standard config - in *his* case.
Best, Matthias
Hi,
just my opinion ( I've investigated now a bit about this "problem" ;) ) The "solution" isn't really that. The changed parameter is only used by vnstatd, which isn't used in IPFire. I can't imagine what did the remedy. We don't know exactly his configuration. Therefore I gave him some hints to help clarifying the case. If he doesn't answer, just forget his unqualified comment.
Best, Bernhard
News about the vnstat problem. Intensive PMs with Zonediver showed that the effect is caused by 32 bit byte counters of the NIC. ( vnstat reads /proc/net/dev ) He changed the fcrontab period for makegraphs to 3 minutes now. Amount of bytes received in this period is less than 2^32.
Remain two questions:
- Is this a normal behaviour for the combination of i210-T1 NIC and associated
driver or is this specific to his system?
I cannot speak for that NIC because I do not use that anywhere. Surprised to hear this the first time since the i210 is also on the APU boards and it actually is a stripped down version of a bigger Intel NIC.
- Can we use the vnstatd daemon? This would allow to increase the vnstat
update frequency independent from makegraphs.
I suppose we could. Arne has built this into the distribution and could probably comment more. If I remember correctly, the daemon wasn't available when we adopted vnstat.
That's right.
But even collecting once a minute wouldn't be enough because with max. theoretical throughput of a Gigabit NIC, 2^32 bytes are transferred in 32 seconds. And if this is only a problem with that specific NIC, I actually wouldn't want to collect every 30 second on all systems. Wastes a lot of resources and costs a lot of write cycles on flash.
I've found a thumb rule for the maximal update period for a given bandwidth. I've documented this in the forum thread.
Best, Bernhard
-Michael
The default value of vnstatd are 30 seconds for UpdateInterval (short enough for 2^32 bytes on 1 GBit/s ), writes to disk is ruled by parameter SaveInterval ( default 5 minutes, which is our period from cron ). Thus vnstatd should give a better resolution ( for some systems without information leaks! ) with similiar load as the cron based solution implemented at the moment.
-Bernhard
-
Bernhard Bitsch -
Matthias Fischer -
Michael Tremer