* [PATCH 1/2] pmacct: New addon
@ 2021-03-08 11:57 Adolf Belka (ipfire)
2021-03-18 13:07 ` Michael Tremer
0 siblings, 1 reply; 7+ messages in thread
From: Adolf Belka (ipfire) @ 2021-03-08 11:57 UTC (permalink / raw)
To: development
[-- Attachment #1: Type: text/plain, Size: 21905 bytes --]
From: Adolf Belka <adolf.belka(a)ipfire.org>
- What is it?
pmacct is a monitoring tool for network management tasks. Data collected
can be used for analysis and troubleshooting purposes to maintain the
health of the network. pmacct can collect, replicate and export network
information. It can cache in memory tables, store persistently to SQLite3
and output to flat-files like CSV, formatted, and JSON.
- Why is it needed?
To monitor data usage (IP-based or MAC-based data accounting) down to the
client level. Net-Traffic will monitor traffic for the entire RED, GREEN,
etc. networks, but it cannot pinpoint which client is using lots of data.
Connections will take a snapshot but not show day by day sums. pmacct can
help admins keep tabs on users that use too much data.
- What are the use cases?
An ISP may implement data caps and if the limit is over-run then you have
to pay for every additional xxGB of data used. Typical charges can be
around $10 per 50GB. With pmacct you can identify the high users and take
action, hopefully before the limit is breached.
- This is being introduced as a command line only tool. However, at a later
date, if it is useful to enough additional users a WUI page could be
developed as discussed in the development mailing list
https://lists.ipfire.org/pipermail/development/2021-January/009174.html
Co-authored-by: Jon Murphy
Co-authored-by: Erik Kapfer <ummeegge(a)ipfire.org>
Signed-off-by: Adolf Belka <adolf.belka(a)ipfire.org>
---
config/backup/includes/pmacct | 2 +
config/pmacct/pmacct.conf | 24 +++++
config/pmacct/pmacct.init | 57 +++++++++++
config/pmacct/pmacct_memory_example.conf | 24 +++++
config/pmacct/pmacct_sqlite3_example.conf | 25 +++++
config/rootfiles/packages/pmacct | 111 ++++++++++++++++++++++
lfs/pmacct | 107 +++++++++++++++++++++
make.sh | 2 +
src/paks/pmacct/install.sh | 36 +++++++
src/paks/pmacct/uninstall.sh | 33 +++++++
src/paks/pmacct/update.sh | 26 +++++
11 files changed, 447 insertions(+)
create mode 100644 config/backup/includes/pmacct
create mode 100644 config/pmacct/pmacct.conf
create mode 100644 config/pmacct/pmacct.init
create mode 100644 config/pmacct/pmacct_memory_example.conf
create mode 100644 config/pmacct/pmacct_sqlite3_example.conf
create mode 100644 config/rootfiles/packages/pmacct
create mode 100644 lfs/pmacct
create mode 100644 src/paks/pmacct/install.sh
create mode 100644 src/paks/pmacct/uninstall.sh
create mode 100644 src/paks/pmacct/update.sh
diff --git a/config/backup/includes/pmacct b/config/backup/includes/pmacct
new file mode 100644
index 000000000..4359067ad
--- /dev/null
+++ b/config/backup/includes/pmacct
@@ -0,0 +1,2 @@
+/etc/pmacct/
+/var/spool/pmacct/
diff --git a/config/pmacct/pmacct.conf b/config/pmacct/pmacct.conf
new file mode 100644
index 000000000..3c1d47efa
--- /dev/null
+++ b/config/pmacct/pmacct.conf
@@ -0,0 +1,24 @@
+!
+! Pmacctd configuration file for IPFire environment
+!
+
+syslog: daemon
+daemonize: true
+debug: false
+promisc: true
+interface: green0
+
+
+!
+! "plugin1" plugin configuration
+!
+plugins: memory[plugin1]
+
+plugin_buffer_size[plugin1]: 102400
+plugin_pipe_size[plugin1]: 10240000
+
+imt_mem_pools_number: 256
+imt_path[plugin1]: /var/spool/pmacct/plugin1.pipe
+
+aggregate[plugin1]: src_host, src_port, src_mac, dst_host, dst_port, dst_mac, proto
+aggregate_filter[plugin1]: ip
\ No newline at end of file
diff --git a/config/pmacct/pmacct.init b/config/pmacct/pmacct.init
new file mode 100644
index 000000000..31f9ce49b
--- /dev/null
+++ b/config/pmacct/pmacct.init
@@ -0,0 +1,57 @@
+#!/bin/bash
+
+# Begin $rc_base/init.d/pmacct
+
+# Starts and stops pmacct daemon
+# Date: 2021-02-25 12:25:30 (Thu, 25 Feb 2021)
+
+# Locations
+CONF="/etc/pmacct/pmacct.conf"
+CONTENT="/var/pmacct"
+# Pmacctd binary
+PMACCT="/usr/sbin/pmacctd"
+# Pmacctd start options '-f' for the configuration file
+OPT="-f ${CONF}"
+DESC="pmacct daemon"
+# Pmacctd syslog facilitiy
+#SYSLOG="local1"
+
+
+. /etc/sysconfig/rc
+. $rc_functions
+
+case "$1" in
+ start)
+ boot_mesg "Starting the ${DESC}... "
+ loadproc ${PMACCT} ${OPT}
+ sleep 2
+ if P=$(pgrep -fl pmacctd); then
+ echo -e "\n${P}"
+ else
+ echo -e "\n${PMACCT} is not running... "
+ fi
+ ;;
+
+ stop)
+ boot_mesg "Stopping the ${DESC}... "
+ killproc ${PMACCT}
+ ;;
+
+ restart)
+ $0 stop
+ sleep 1
+ $0 start
+ ;;
+
+ status)
+ statusproc ${PMACCT}
+ ;;
+
+ *)
+ echo "Usage: $0 {start|stop|restart|status}"
+ exit 1
+ ;;
+esac
+
+# End $rc_base/init.d/pmacct
+
diff --git a/config/pmacct/pmacct_memory_example.conf b/config/pmacct/pmacct_memory_example.conf
new file mode 100644
index 000000000..0babdaaf7
--- /dev/null
+++ b/config/pmacct/pmacct_memory_example.conf
@@ -0,0 +1,24 @@
+!
+! Pmacctd configuration file for IPFire environment
+!
+
+syslog: daemon
+daemonize: true
+debug: false
+promisc: true
+interface: green0
+
+
+!
+! "plugin1" plugin configuration
+!
+plugins: memory[plugin1]
+
+plugin_buffer_size[plugin1]: 102400
+plugin_pipe_size[plugin1]: 10240000
+
+imt_mem_pools_number: 256
+imt_path[plugin1]: /var/spool/pmacct/plugin1.pipe
+
+aggregate[plugin1]: src_host, src_port, src_mac, dst_host, dst_port, dst_mac, proto
+aggregate_filter[plugin1]: ip
diff --git a/config/pmacct/pmacct_sqlite3_example.conf b/config/pmacct/pmacct_sqlite3_example.conf
new file mode 100644
index 000000000..7fb996408
--- /dev/null
+++ b/config/pmacct/pmacct_sqlite3_example.conf
@@ -0,0 +1,25 @@
+!
+! Pmacctd configuration file for IPFire environment
+!
+
+syslog: daemon
+daemonize: true
+debug: false
+promisc: true
+interface: green0
+
+
+!
+! "plugin2" plugin configuration
+!
+plugins: sqlite3[plugin2]
+
+sql_db[plugin2]: /var/spool/pmacct/pmacct_sqlitev1.db
+sql_table_version[plugin2]: 1
+sql_history[plugin2]: 5m
+sql_refresh_time[plugin2]: 300
+sql_history_roundoff[plugin2]: m
+sql_dont_try_update: true
+
+aggregate[plugin2]: src_host, src_port, src_mac, dst_host, dst_port, dst_mac, proto
+aggregate_filter[plugin2]: ip
diff --git a/config/rootfiles/packages/pmacct b/config/rootfiles/packages/pmacct
new file mode 100644
index 000000000..fed83ee67
--- /dev/null
+++ b/config/rootfiles/packages/pmacct
@@ -0,0 +1,111 @@
+etc/pmacct
+etc/pmacct/pmacct.conf
+etc/pmacct/pmacct_memory_example.conf
+etc/pmacct/pmacct_sqlite3_example.conf
+etc/rc.d/init.d/pmacct
+usr/bin/pmacct
+#usr/lib/pmacct
+#usr/lib/pmacct/examples
+#usr/lib/pmacct/examples/custom
+#usr/lib/pmacct/examples/custom/libcustom.la
+#usr/lib/pmacct/examples/lg
+#usr/lib/pmacct/examples/lg/pmbgp
+usr/sbin/nfacctd
+usr/sbin/pmacctd
+usr/sbin/pmbgpd
+usr/sbin/pmbmpd
+usr/sbin/pmtelemetryd
+usr/sbin/sfacctd
+usr/share/pmacct
+#usr/share/pmacct/CONFIG-KEYS
+#usr/share/pmacct/FAQS
+#usr/share/pmacct/QUICKSTART
+#usr/share/pmacct/UPGRADE
+#usr/share/pmacct/docs
+#usr/share/pmacct/docs/IPFIX
+#usr/share/pmacct/docs/LOOKING_GLASS_FORMAT
+#usr/share/pmacct/docs/MSGLOG_DUMP_FORMATS
+#usr/share/pmacct/docs/SIGNALS
+#usr/share/pmacct/docs/TRIGGER_VARS
+#usr/share/pmacct/examples
+usr/share/pmacct/examples/allow.lst.example
+#usr/share/pmacct/examples/amqp
+usr/share/pmacct/examples/amqp/amqp_receiver.py
+#usr/share/pmacct/examples/avro
+usr/share/pmacct/examples/avro/avro_file_decoder.py
+usr/share/pmacct/examples/bgp_agent.map.example
+usr/share/pmacct/examples/bgp_md5.lst.example
+usr/share/pmacct/examples/bgp_xconnects.map.example
+usr/share/pmacct/examples/flow_to_rd.map.example
+#usr/share/pmacct/examples/kafka
+usr/share/pmacct/examples/kafka/kafka_consumer.py
+#usr/share/pmacct/examples/lg
+usr/share/pmacct/examples/lg/pmbgp.py
+usr/share/pmacct/examples/networks.lst.example
+usr/share/pmacct/examples/nfacctd-sql.conf.example
+usr/share/pmacct/examples/pcap_interfaces.map.example
+usr/share/pmacct/examples/peers.map.example
+usr/share/pmacct/examples/pmacctd-imt.conf.example
+usr/share/pmacct/examples/pmacctd-multiple-plugins.conf.example
+usr/share/pmacct/examples/pmacctd-sql.conf.example
+usr/share/pmacct/examples/pmacctd-sqlite3.conf.example
+usr/share/pmacct/examples/ports.lst.example
+usr/share/pmacct/examples/pretag.map.example
+usr/share/pmacct/examples/primitives.lst.example
+usr/share/pmacct/examples/probe_netflow.conf.example
+usr/share/pmacct/examples/probe_sflow.conf.example
+usr/share/pmacct/examples/sampling.map.example
+usr/share/pmacct/examples/tee_receivers.lst.example
+#usr/share/pmacct/sql
+#usr/share/pmacct/sql/README.64bit
+#usr/share/pmacct/sql/README.GeoIP
+#usr/share/pmacct/sql/README.IPv6
+#usr/share/pmacct/sql/README.cos
+#usr/share/pmacct/sql/README.custom_primitives
+#usr/share/pmacct/sql/README.etype
+#usr/share/pmacct/sql/README.export_proto
+#usr/share/pmacct/sql/README.iface
+#usr/share/pmacct/sql/README.label
+#usr/share/pmacct/sql/README.mask
+#usr/share/pmacct/sql/README.mpls
+#usr/share/pmacct/sql/README.mysql
+#usr/share/pmacct/sql/README.nat
+#usr/share/pmacct/sql/README.pgsql
+#usr/share/pmacct/sql/README.sampling
+usr/share/pmacct/sql/README.sqlite3
+#usr/share/pmacct/sql/README.tag2
+#usr/share/pmacct/sql/README.timestamp
+#usr/share/pmacct/sql/README.tunnel
+#usr/share/pmacct/sql/pmacct-create-db.pgsql
+#usr/share/pmacct/sql/pmacct-create-db_bgp_v1.mysql
+#usr/share/pmacct/sql/pmacct-create-db_v1.mysql
+#usr/share/pmacct/sql/pmacct-create-db_v2.mysql
+#usr/share/pmacct/sql/pmacct-create-db_v3.mysql
+#usr/share/pmacct/sql/pmacct-create-db_v4.mysql
+#usr/share/pmacct/sql/pmacct-create-db_v5.mysql
+#usr/share/pmacct/sql/pmacct-create-db_v6.mysql
+#usr/share/pmacct/sql/pmacct-create-db_v7.mysql
+#usr/share/pmacct/sql/pmacct-create-db_v8.mysql
+#usr/share/pmacct/sql/pmacct-create-db_v9.mysql
+#usr/share/pmacct/sql/pmacct-create-table_bgp_v1.pgsql
+usr/share/pmacct/sql/pmacct-create-table_bgp_v1.sqlite3
+#usr/share/pmacct/sql/pmacct-create-table_v1.pgsql
+usr/share/pmacct/sql/pmacct-create-table_v1.sqlite3
+#usr/share/pmacct/sql/pmacct-create-table_v2.pgsql
+usr/share/pmacct/sql/pmacct-create-table_v2.sqlite3
+#usr/share/pmacct/sql/pmacct-create-table_v3.pgsql
+usr/share/pmacct/sql/pmacct-create-table_v3.sqlite3
+#usr/share/pmacct/sql/pmacct-create-table_v4.pgsql
+usr/share/pmacct/sql/pmacct-create-table_v4.sqlite3
+#usr/share/pmacct/sql/pmacct-create-table_v5.pgsql
+usr/share/pmacct/sql/pmacct-create-table_v5.sqlite3
+#usr/share/pmacct/sql/pmacct-create-table_v6.pgsql
+usr/share/pmacct/sql/pmacct-create-table_v6.sqlite3
+usr/share/pmacct/sql/pmacct-create-table_v7.sqlite3
+#usr/share/pmacct/sql/pmacct-create-table_v7_v8.pgsql
+usr/share/pmacct/sql/pmacct-create-table_v8.sqlite3
+#usr/share/pmacct/sql/pmacct-create-table_v9.pgsql
+usr/share/pmacct/sql/pmacct-create-table_v9.sqlite3
+#usr/share/pmacct/sql/pmacct-grant-db.mysql
+var/ipfire/backup/addons/includes/pmacct
+var/spool/pmacct
diff --git a/lfs/pmacct b/lfs/pmacct
new file mode 100644
index 000000000..c6cc5270e
--- /dev/null
+++ b/lfs/pmacct
@@ -0,0 +1,107 @@
+###############################################################################
+# #
+# IPFire.org - A linux based firewall #
+# Copyright (C) 2019 IPFire Team <info(a)ipfire.org> #
+# #
+# This program is free software: you can redistribute it and/or modify #
+# it under the terms of the GNU General Public License as published by #
+# the Free Software Foundation, either version 3 of the License, or #
+# (at your option) any later version. #
+# #
+# This program is distributed in the hope that it will be useful, #
+# but WITHOUT ANY WARRANTY; without even the implied warranty of #
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
+# GNU General Public License for more details. #
+# #
+# You should have received a copy of the GNU General Public License #
+# along with this program. If not, see <http://www.gnu.org/licenses/>. #
+# #
+###############################################################################
+
+###############################################################################
+# Definitions
+###############################################################################
+
+include Config
+
+VER = 1.7.6
+
+THISAPP = pmacct-$(VER)
+DL_FILE = $(THISAPP).tar.gz
+DL_FROM = $(URL_IPFIRE)
+DIR_APP = $(DIR_SRC)/$(THISAPP)
+TARGET = $(DIR_INFO)/$(THISAPP)
+PROG = pmacct
+PAK_VER = 1
+
+DEPS = libcdada
+
+###############################################################################
+# Top-level Rules
+###############################################################################
+
+objects = $(DL_FILE)
+
+$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
+
+$(DL_FILE)_MD5 = b829cfdc394d5acd6d32fb23de181a6b
+
+install : $(TARGET)
+
+check : $(patsubst %,$(DIR_CHK)/%,$(objects))
+
+download :$(patsubst %,$(DIR_DL)/%,$(objects))
+
+md5 : $(subst %,%_MD5,$(objects))
+
+dist:
+ @$(PAK)
+
+###############################################################################
+# Downloading, checking, md5sum
+###############################################################################
+
+$(patsubst %,$(DIR_CHK)/%,$(objects)) :
+ @$(CHECK)
+
+$(patsubst %,$(DIR_DL)/%,$(objects)) :
+ @$(LOAD)
+
+$(subst %,%_MD5,$(objects)) :
+ @$(MD5)
+
+###############################################################################
+# Installation Details
+###############################################################################
+
+$(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
+ @$(PREBUILD)
+ @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar zxf $(DIR_DL)/$(DL_FILE)
+ cd $(DIR_APP) && ./autogen.sh
+ cd $(DIR_APP) && ./configure \
+ --prefix=/usr \
+ --sysconfdir=/etc/pmacct \
+ --enable-sqlite3 \
+ --enable-l2 \
+ --enable-jansson
+
+ # Add directory for plugins and pipes
+ -mkdir -v /var/spool/pmacct
+
+ # Add configuration folder and copy conf files to it
+ -mkdir -v /etc/pmacct
+ cp -vf $(DIR_CONF)/pmacct/*.conf /etc/pmacct
+
+ # Copy initscript to dir
+ cp -vf $(DIR_CONF)/pmacct/pmacct.init /etc/rc.d/init.d/pmacct
+ chmod 754 /etc/rc.d/init.d/pmacct
+ chown root:root /etc/rc.d/init.d/pmacct
+
+ cd $(DIR_APP) && make $(MAKETUNING)
+ cd $(DIR_APP) && make install
+
+ # Install backup definition
+ install -v -m 644 ${DIR_SRC}/config/backup/includes/pmacct \
+ /var/ipfire/backup/addons/includes/pmacct
+ @rm -rf $(DIR_APP)
+ @$(POSTBUILD)
diff --git a/make.sh b/make.sh
index 43cccaca0..d42c9b1e2 100755
--- a/make.sh
+++ b/make.sh
@@ -1646,6 +1646,8 @@ buildipfire() {
lfsmake2 ncdu
lfsmake2 lshw
lfsmake2 socat
+ lfsmake2 libcdada
+ lfsmake2 pmacct
}
buildinstaller() {
diff --git a/src/paks/pmacct/install.sh b/src/paks/pmacct/install.sh
new file mode 100644
index 000000000..8178cc7bc
--- /dev/null
+++ b/src/paks/pmacct/install.sh
@@ -0,0 +1,36 @@
+#!/bin/bash
+############################################################################
+# #
+# This file is part of the IPFire Firewall. #
+# #
+# IPFire is free software; you can redistribute it and/or modify #
+# it under the terms of the GNU General Public License as published by #
+# the Free Software Foundation; either version 2 of the License, or #
+# (at your option) any later version. #
+# #
+# IPFire is distributed in the hope that it will be useful, #
+# but WITHOUT ANY WARRANTY; without even the implied warranty of #
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
+# GNU General Public License for more details. #
+# #
+# You should have received a copy of the GNU General Public License #
+# along with IPFire; if not, write to the Free Software #
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
+# #
+# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>. #
+# #
+############################################################################
+#
+. /opt/pakfire/lib/functions.sh
+
+extract_files
+restore_backup ${NAME}
+
+# Add symlinks for runlevels
+ln -s ../init.d/${NAME} /etc/rc.d/rc0.d/K85${NAME}
+ln -s ../init.d/${NAME} /etc/rc.d/rc3.d/S50${NAME}
+ln -s ../init.d/${NAME} /etc/rc.d/rc6.d/K85${NAME}
+start_service --background ${NAME}
+
+
+# EOF
diff --git a/src/paks/pmacct/uninstall.sh b/src/paks/pmacct/uninstall.sh
new file mode 100644
index 000000000..51631e8c1
--- /dev/null
+++ b/src/paks/pmacct/uninstall.sh
@@ -0,0 +1,33 @@
+#!/bin/bash
+############################################################################
+# #
+# This file is part of the IPFire Firewall. #
+# #
+# IPFire is free software; you can redistribute it and/or modify #
+# it under the terms of the GNU General Public License as published by #
+# the Free Software Foundation; either version 2 of the License, or #
+# (at your option) any later version. #
+# #
+# IPFire is distributed in the hope that it will be useful, #
+# but WITHOUT ANY WARRANTY; without even the implied warranty of #
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
+# GNU General Public License for more details. #
+# #
+# You should have received a copy of the GNU General Public License #
+# along with IPFire; if not, write to the Free Software #
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
+# #
+# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>. #
+# #
+############################################################################
+#
+. /opt/pakfire/lib/functions.sh
+
+stop_service ${NAME}
+make_backup ${NAME}
+remove_files
+
+# Delete symlinks
+rm -rfv /etc/rc.d/rc?.d/???${NAME}
+
+# EOF
diff --git a/src/paks/pmacct/update.sh b/src/paks/pmacct/update.sh
new file mode 100644
index 000000000..89c40d0d7
--- /dev/null
+++ b/src/paks/pmacct/update.sh
@@ -0,0 +1,26 @@
+#!/bin/bash
+############################################################################
+# #
+# This file is part of the IPFire Firewall. #
+# #
+# IPFire is free software; you can redistribute it and/or modify #
+# it under the terms of the GNU General Public License as published by #
+# the Free Software Foundation; either version 2 of the License, or #
+# (at your option) any later version. #
+# #
+# IPFire is distributed in the hope that it will be useful, #
+# but WITHOUT ANY WARRANTY; without even the implied warranty of #
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
+# GNU General Public License for more details. #
+# #
+# You should have received a copy of the GNU General Public License #
+# along with IPFire; if not, write to the Free Software #
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
+# #
+# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>. #
+# #
+############################################################################
+#
+. /opt/pakfire/lib/functions.sh
+./uninstall.sh
+./install.sh
--
2.30.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/2] pmacct: New addon
2021-03-08 11:57 [PATCH 1/2] pmacct: New addon Adolf Belka (ipfire)
@ 2021-03-18 13:07 ` Michael Tremer
2021-03-18 13:34 ` Adolf Belka
0 siblings, 1 reply; 7+ messages in thread
From: Michael Tremer @ 2021-03-18 13:07 UTC (permalink / raw)
To: development
[-- Attachment #1: Type: text/plain, Size: 24826 bytes --]
Hello,
I will split this email into two parts:
The technical part:
* The initscript isn’t based on our template and uses sleep to check if starting a daemon has been successful. That should not be necessary and will potentially slow down the boot process.
* I am not very sure if the example configuration files are helpful. Maybe someone can clear this up a little bit for me what is the rationale is behind them.
* There are loads of daemons being shipped and only one is being used.
* How much load does this generate? How large is the captured data?
Then there is the other part:
> On 8 Mar 2021, at 11:57, Adolf Belka (ipfire) <adolf.belka(a)ipfire.org> wrote:
>
> From: Adolf Belka <adolf.belka(a)ipfire.org>
>
> - What is it?
> pmacct is a monitoring tool for network management tasks. Data collected
> can be used for analysis and troubleshooting purposes to maintain the
> health of the network. pmacct can collect, replicate and export network
> information. It can cache in memory tables, store persistently to SQLite3
> and output to flat-files like CSV, formatted, and JSON.
Okay.
> - Why is it needed?
> To monitor data usage (IP-based or MAC-based data accounting) down to the
> client level. Net-Traffic will monitor traffic for the entire RED, GREEN,
> etc. networks, but it cannot pinpoint which client is using lots of data.
> Connections will take a snapshot but not show day by day sums. pmacct can
> help admins keep tabs on users that use too much data.
I agree with the goal of it. But how does this add-on achieve that?
The data is being written to a (usually binary) file which cannot be visualised in the web UI. Does this not want a CGI script that allows people to see what is happening?
Sending manual SQL queries probably isn’t much fun :)
> - What are the use cases?
> An ISP may implement data caps and if the limit is over-run then you have
> to pay for every additional xxGB of data used. Typical charges can be
> around $10 per 50GB. With pmacct you can identify the high users and take
> action, hopefully before the limit is breached.
> - This is being introduced as a command line only tool. However, at a later
> date, if it is useful to enough additional users a WUI page could be
> developed as discussed in the development mailing list
> https://lists.ipfire.org/pipermail/development/2021-January/009174.html
I have been thinking about this for a while now and would say that I can approve merging this as a CLI tool. But it would not really enhance IPFire’s functionality because it is only usable for people who spend a lot of time writing their own scripts or SQL queries to get information that can be used to make decisions.
Adding another alternative to vnstat will result in us having to maintain two things that roughly do the same - or where one’s features are a subset of the other one.
I would like to avoid merging this and then see it abandoned because it is too complicated to use. We already have a lot of virtually unmaintained stuff around and I would really like to drop a lot of it, although I know that there are a few people actually using it.
Thoughts?
Best,
-Michael
> Co-authored-by: Jon Murphy
> Co-authored-by: Erik Kapfer <ummeegge(a)ipfire.org>
>
> Signed-off-by: Adolf Belka <adolf.belka(a)ipfire.org>
> ---
> config/backup/includes/pmacct | 2 +
> config/pmacct/pmacct.conf | 24 +++++
> config/pmacct/pmacct.init | 57 +++++++++++
> config/pmacct/pmacct_memory_example.conf | 24 +++++
> config/pmacct/pmacct_sqlite3_example.conf | 25 +++++
> config/rootfiles/packages/pmacct | 111 ++++++++++++++++++++++
> lfs/pmacct | 107 +++++++++++++++++++++
> make.sh | 2 +
> src/paks/pmacct/install.sh | 36 +++++++
> src/paks/pmacct/uninstall.sh | 33 +++++++
> src/paks/pmacct/update.sh | 26 +++++
> 11 files changed, 447 insertions(+)
> create mode 100644 config/backup/includes/pmacct
> create mode 100644 config/pmacct/pmacct.conf
> create mode 100644 config/pmacct/pmacct.init
> create mode 100644 config/pmacct/pmacct_memory_example.conf
> create mode 100644 config/pmacct/pmacct_sqlite3_example.conf
> create mode 100644 config/rootfiles/packages/pmacct
> create mode 100644 lfs/pmacct
> create mode 100644 src/paks/pmacct/install.sh
> create mode 100644 src/paks/pmacct/uninstall.sh
> create mode 100644 src/paks/pmacct/update.sh
>
> diff --git a/config/backup/includes/pmacct b/config/backup/includes/pmacct
> new file mode 100644
> index 000000000..4359067ad
> --- /dev/null
> +++ b/config/backup/includes/pmacct
> @@ -0,0 +1,2 @@
> +/etc/pmacct/
> +/var/spool/pmacct/
> diff --git a/config/pmacct/pmacct.conf b/config/pmacct/pmacct.conf
> new file mode 100644
> index 000000000..3c1d47efa
> --- /dev/null
> +++ b/config/pmacct/pmacct.conf
> @@ -0,0 +1,24 @@
> +!
> +! Pmacctd configuration file for IPFire environment
> +!
> +
> +syslog: daemon
> +daemonize: true
> +debug: false
> +promisc: true
> +interface: green0
> +
> +
> +!
> +! "plugin1" plugin configuration
> +!
> +plugins: memory[plugin1]
> +
> +plugin_buffer_size[plugin1]: 102400
> +plugin_pipe_size[plugin1]: 10240000
> +
> +imt_mem_pools_number: 256
> +imt_path[plugin1]: /var/spool/pmacct/plugin1.pipe
> +
> +aggregate[plugin1]: src_host, src_port, src_mac, dst_host, dst_port, dst_mac, proto
> +aggregate_filter[plugin1]: ip
> \ No newline at end of file
> diff --git a/config/pmacct/pmacct.init b/config/pmacct/pmacct.init
> new file mode 100644
> index 000000000..31f9ce49b
> --- /dev/null
> +++ b/config/pmacct/pmacct.init
> @@ -0,0 +1,57 @@
> +#!/bin/bash
> +
> +# Begin $rc_base/init.d/pmacct
> +
> +# Starts and stops pmacct daemon
> +# Date: 2021-02-25 12:25:30 (Thu, 25 Feb 2021)
> +
> +# Locations
> +CONF="/etc/pmacct/pmacct.conf"
> +CONTENT="/var/pmacct"
> +# Pmacctd binary
> +PMACCT="/usr/sbin/pmacctd"
> +# Pmacctd start options '-f' for the configuration file
> +OPT="-f ${CONF}"
> +DESC="pmacct daemon"
> +# Pmacctd syslog facilitiy
> +#SYSLOG="local1"
> +
> +
> +. /etc/sysconfig/rc
> +. $rc_functions
> +
> +case "$1" in
> + start)
> + boot_mesg "Starting the ${DESC}... "
> + loadproc ${PMACCT} ${OPT}
> + sleep 2
> + if P=$(pgrep -fl pmacctd); then
> + echo -e "\n${P}"
> + else
> + echo -e "\n${PMACCT} is not running... "
> + fi
> + ;;
> +
> + stop)
> + boot_mesg "Stopping the ${DESC}... "
> + killproc ${PMACCT}
> + ;;
> +
> + restart)
> + $0 stop
> + sleep 1
> + $0 start
> + ;;
> +
> + status)
> + statusproc ${PMACCT}
> + ;;
> +
> + *)
> + echo "Usage: $0 {start|stop|restart|status}"
> + exit 1
> + ;;
> +esac
> +
> +# End $rc_base/init.d/pmacct
> +
> diff --git a/config/pmacct/pmacct_memory_example.conf b/config/pmacct/pmacct_memory_example.conf
> new file mode 100644
> index 000000000..0babdaaf7
> --- /dev/null
> +++ b/config/pmacct/pmacct_memory_example.conf
> @@ -0,0 +1,24 @@
> +!
> +! Pmacctd configuration file for IPFire environment
> +!
> +
> +syslog: daemon
> +daemonize: true
> +debug: false
> +promisc: true
> +interface: green0
> +
> +
> +!
> +! "plugin1" plugin configuration
> +!
> +plugins: memory[plugin1]
> +
> +plugin_buffer_size[plugin1]: 102400
> +plugin_pipe_size[plugin1]: 10240000
> +
> +imt_mem_pools_number: 256
> +imt_path[plugin1]: /var/spool/pmacct/plugin1.pipe
> +
> +aggregate[plugin1]: src_host, src_port, src_mac, dst_host, dst_port, dst_mac, proto
> +aggregate_filter[plugin1]: ip
> diff --git a/config/pmacct/pmacct_sqlite3_example.conf b/config/pmacct/pmacct_sqlite3_example.conf
> new file mode 100644
> index 000000000..7fb996408
> --- /dev/null
> +++ b/config/pmacct/pmacct_sqlite3_example.conf
> @@ -0,0 +1,25 @@
> +!
> +! Pmacctd configuration file for IPFire environment
> +!
> +
> +syslog: daemon
> +daemonize: true
> +debug: false
> +promisc: true
> +interface: green0
> +
> +
> +!
> +! "plugin2" plugin configuration
> +!
> +plugins: sqlite3[plugin2]
> +
> +sql_db[plugin2]: /var/spool/pmacct/pmacct_sqlitev1.db
> +sql_table_version[plugin2]: 1
> +sql_history[plugin2]: 5m
> +sql_refresh_time[plugin2]: 300
> +sql_history_roundoff[plugin2]: m
> +sql_dont_try_update: true
> +
> +aggregate[plugin2]: src_host, src_port, src_mac, dst_host, dst_port, dst_mac, proto
> +aggregate_filter[plugin2]: ip
> diff --git a/config/rootfiles/packages/pmacct b/config/rootfiles/packages/pmacct
> new file mode 100644
> index 000000000..fed83ee67
> --- /dev/null
> +++ b/config/rootfiles/packages/pmacct
> @@ -0,0 +1,111 @@
> +etc/pmacct
> +etc/pmacct/pmacct.conf
> +etc/pmacct/pmacct_memory_example.conf
> +etc/pmacct/pmacct_sqlite3_example.conf
> +etc/rc.d/init.d/pmacct
> +usr/bin/pmacct
> +#usr/lib/pmacct
> +#usr/lib/pmacct/examples
> +#usr/lib/pmacct/examples/custom
> +#usr/lib/pmacct/examples/custom/libcustom.la
> +#usr/lib/pmacct/examples/lg
> +#usr/lib/pmacct/examples/lg/pmbgp
> +usr/sbin/nfacctd
> +usr/sbin/pmacctd
> +usr/sbin/pmbgpd
> +usr/sbin/pmbmpd
> +usr/sbin/pmtelemetryd
> +usr/sbin/sfacctd
> +usr/share/pmacct
> +#usr/share/pmacct/CONFIG-KEYS
> +#usr/share/pmacct/FAQS
> +#usr/share/pmacct/QUICKSTART
> +#usr/share/pmacct/UPGRADE
> +#usr/share/pmacct/docs
> +#usr/share/pmacct/docs/IPFIX
> +#usr/share/pmacct/docs/LOOKING_GLASS_FORMAT
> +#usr/share/pmacct/docs/MSGLOG_DUMP_FORMATS
> +#usr/share/pmacct/docs/SIGNALS
> +#usr/share/pmacct/docs/TRIGGER_VARS
> +#usr/share/pmacct/examples
> +usr/share/pmacct/examples/allow.lst.example
> +#usr/share/pmacct/examples/amqp
> +usr/share/pmacct/examples/amqp/amqp_receiver.py
> +#usr/share/pmacct/examples/avro
> +usr/share/pmacct/examples/avro/avro_file_decoder.py
> +usr/share/pmacct/examples/bgp_agent.map.example
> +usr/share/pmacct/examples/bgp_md5.lst.example
> +usr/share/pmacct/examples/bgp_xconnects.map.example
> +usr/share/pmacct/examples/flow_to_rd.map.example
> +#usr/share/pmacct/examples/kafka
> +usr/share/pmacct/examples/kafka/kafka_consumer.py
> +#usr/share/pmacct/examples/lg
> +usr/share/pmacct/examples/lg/pmbgp.py
> +usr/share/pmacct/examples/networks.lst.example
> +usr/share/pmacct/examples/nfacctd-sql.conf.example
> +usr/share/pmacct/examples/pcap_interfaces.map.example
> +usr/share/pmacct/examples/peers.map.example
> +usr/share/pmacct/examples/pmacctd-imt.conf.example
> +usr/share/pmacct/examples/pmacctd-multiple-plugins.conf.example
> +usr/share/pmacct/examples/pmacctd-sql.conf.example
> +usr/share/pmacct/examples/pmacctd-sqlite3.conf.example
> +usr/share/pmacct/examples/ports.lst.example
> +usr/share/pmacct/examples/pretag.map.example
> +usr/share/pmacct/examples/primitives.lst.example
> +usr/share/pmacct/examples/probe_netflow.conf.example
> +usr/share/pmacct/examples/probe_sflow.conf.example
> +usr/share/pmacct/examples/sampling.map.example
> +usr/share/pmacct/examples/tee_receivers.lst.example
> +#usr/share/pmacct/sql
> +#usr/share/pmacct/sql/README.64bit
> +#usr/share/pmacct/sql/README.GeoIP
> +#usr/share/pmacct/sql/README.IPv6
> +#usr/share/pmacct/sql/README.cos
> +#usr/share/pmacct/sql/README.custom_primitives
> +#usr/share/pmacct/sql/README.etype
> +#usr/share/pmacct/sql/README.export_proto
> +#usr/share/pmacct/sql/README.iface
> +#usr/share/pmacct/sql/README.label
> +#usr/share/pmacct/sql/README.mask
> +#usr/share/pmacct/sql/README.mpls
> +#usr/share/pmacct/sql/README.mysql
> +#usr/share/pmacct/sql/README.nat
> +#usr/share/pmacct/sql/README.pgsql
> +#usr/share/pmacct/sql/README.sampling
> +usr/share/pmacct/sql/README.sqlite3
> +#usr/share/pmacct/sql/README.tag2
> +#usr/share/pmacct/sql/README.timestamp
> +#usr/share/pmacct/sql/README.tunnel
> +#usr/share/pmacct/sql/pmacct-create-db.pgsql
> +#usr/share/pmacct/sql/pmacct-create-db_bgp_v1.mysql
> +#usr/share/pmacct/sql/pmacct-create-db_v1.mysql
> +#usr/share/pmacct/sql/pmacct-create-db_v2.mysql
> +#usr/share/pmacct/sql/pmacct-create-db_v3.mysql
> +#usr/share/pmacct/sql/pmacct-create-db_v4.mysql
> +#usr/share/pmacct/sql/pmacct-create-db_v5.mysql
> +#usr/share/pmacct/sql/pmacct-create-db_v6.mysql
> +#usr/share/pmacct/sql/pmacct-create-db_v7.mysql
> +#usr/share/pmacct/sql/pmacct-create-db_v8.mysql
> +#usr/share/pmacct/sql/pmacct-create-db_v9.mysql
> +#usr/share/pmacct/sql/pmacct-create-table_bgp_v1.pgsql
> +usr/share/pmacct/sql/pmacct-create-table_bgp_v1.sqlite3
> +#usr/share/pmacct/sql/pmacct-create-table_v1.pgsql
> +usr/share/pmacct/sql/pmacct-create-table_v1.sqlite3
> +#usr/share/pmacct/sql/pmacct-create-table_v2.pgsql
> +usr/share/pmacct/sql/pmacct-create-table_v2.sqlite3
> +#usr/share/pmacct/sql/pmacct-create-table_v3.pgsql
> +usr/share/pmacct/sql/pmacct-create-table_v3.sqlite3
> +#usr/share/pmacct/sql/pmacct-create-table_v4.pgsql
> +usr/share/pmacct/sql/pmacct-create-table_v4.sqlite3
> +#usr/share/pmacct/sql/pmacct-create-table_v5.pgsql
> +usr/share/pmacct/sql/pmacct-create-table_v5.sqlite3
> +#usr/share/pmacct/sql/pmacct-create-table_v6.pgsql
> +usr/share/pmacct/sql/pmacct-create-table_v6.sqlite3
> +usr/share/pmacct/sql/pmacct-create-table_v7.sqlite3
> +#usr/share/pmacct/sql/pmacct-create-table_v7_v8.pgsql
> +usr/share/pmacct/sql/pmacct-create-table_v8.sqlite3
> +#usr/share/pmacct/sql/pmacct-create-table_v9.pgsql
> +usr/share/pmacct/sql/pmacct-create-table_v9.sqlite3
> +#usr/share/pmacct/sql/pmacct-grant-db.mysql
> +var/ipfire/backup/addons/includes/pmacct
> +var/spool/pmacct
> diff --git a/lfs/pmacct b/lfs/pmacct
> new file mode 100644
> index 000000000..c6cc5270e
> --- /dev/null
> +++ b/lfs/pmacct
> @@ -0,0 +1,107 @@
> +###############################################################################
> +# #
> +# IPFire.org - A linux based firewall #
> +# Copyright (C) 2019 IPFire Team <info(a)ipfire.org> #
> +# #
> +# This program is free software: you can redistribute it and/or modify #
> +# it under the terms of the GNU General Public License as published by #
> +# the Free Software Foundation, either version 3 of the License, or #
> +# (at your option) any later version. #
> +# #
> +# This program is distributed in the hope that it will be useful, #
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of #
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
> +# GNU General Public License for more details. #
> +# #
> +# You should have received a copy of the GNU General Public License #
> +# along with this program. If not, see <http://www.gnu.org/licenses/>. #
> +# #
> +###############################################################################
> +
> +###############################################################################
> +# Definitions
> +###############################################################################
> +
> +include Config
> +
> +VER = 1.7.6
> +
> +THISAPP = pmacct-$(VER)
> +DL_FILE = $(THISAPP).tar.gz
> +DL_FROM = $(URL_IPFIRE)
> +DIR_APP = $(DIR_SRC)/$(THISAPP)
> +TARGET = $(DIR_INFO)/$(THISAPP)
> +PROG = pmacct
> +PAK_VER = 1
> +
> +DEPS = libcdada
> +
> +###############################################################################
> +# Top-level Rules
> +###############################################################################
> +
> +objects = $(DL_FILE)
> +
> +$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
> +
> +$(DL_FILE)_MD5 = b829cfdc394d5acd6d32fb23de181a6b
> +
> +install : $(TARGET)
> +
> +check : $(patsubst %,$(DIR_CHK)/%,$(objects))
> +
> +download :$(patsubst %,$(DIR_DL)/%,$(objects))
> +
> +md5 : $(subst %,%_MD5,$(objects))
> +
> +dist:
> + @$(PAK)
> +
> +###############################################################################
> +# Downloading, checking, md5sum
> +###############################################################################
> +
> +$(patsubst %,$(DIR_CHK)/%,$(objects)) :
> + @$(CHECK)
> +
> +$(patsubst %,$(DIR_DL)/%,$(objects)) :
> + @$(LOAD)
> +
> +$(subst %,%_MD5,$(objects)) :
> + @$(MD5)
> +
> +###############################################################################
> +# Installation Details
> +###############################################################################
> +
> +$(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
> + @$(PREBUILD)
> + @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar zxf $(DIR_DL)/$(DL_FILE)
> + cd $(DIR_APP) && ./autogen.sh
> + cd $(DIR_APP) && ./configure \
> + --prefix=/usr \
> + --sysconfdir=/etc/pmacct \
> + --enable-sqlite3 \
> + --enable-l2 \
> + --enable-jansson
> +
> + # Add directory for plugins and pipes
> + -mkdir -v /var/spool/pmacct
> +
> + # Add configuration folder and copy conf files to it
> + -mkdir -v /etc/pmacct
> + cp -vf $(DIR_CONF)/pmacct/*.conf /etc/pmacct
> +
> + # Copy initscript to dir
> + cp -vf $(DIR_CONF)/pmacct/pmacct.init /etc/rc.d/init.d/pmacct
> + chmod 754 /etc/rc.d/init.d/pmacct
> + chown root:root /etc/rc.d/init.d/pmacct
> +
> + cd $(DIR_APP) && make $(MAKETUNING)
> + cd $(DIR_APP) && make install
> +
> + # Install backup definition
> + install -v -m 644 ${DIR_SRC}/config/backup/includes/pmacct \
> + /var/ipfire/backup/addons/includes/pmacct
> + @rm -rf $(DIR_APP)
> + @$(POSTBUILD)
> diff --git a/make.sh b/make.sh
> index 43cccaca0..d42c9b1e2 100755
> --- a/make.sh
> +++ b/make.sh
> @@ -1646,6 +1646,8 @@ buildipfire() {
> lfsmake2 ncdu
> lfsmake2 lshw
> lfsmake2 socat
> + lfsmake2 libcdada
> + lfsmake2 pmacct
> }
>
> buildinstaller() {
> diff --git a/src/paks/pmacct/install.sh b/src/paks/pmacct/install.sh
> new file mode 100644
> index 000000000..8178cc7bc
> --- /dev/null
> +++ b/src/paks/pmacct/install.sh
> @@ -0,0 +1,36 @@
> +#!/bin/bash
> +############################################################################
> +# #
> +# This file is part of the IPFire Firewall. #
> +# #
> +# IPFire is free software; you can redistribute it and/or modify #
> +# it under the terms of the GNU General Public License as published by #
> +# the Free Software Foundation; either version 2 of the License, or #
> +# (at your option) any later version. #
> +# #
> +# IPFire is distributed in the hope that it will be useful, #
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of #
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
> +# GNU General Public License for more details. #
> +# #
> +# You should have received a copy of the GNU General Public License #
> +# along with IPFire; if not, write to the Free Software #
> +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
> +# #
> +# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>. #
> +# #
> +############################################################################
> +#
> +. /opt/pakfire/lib/functions.sh
> +
> +extract_files
> +restore_backup ${NAME}
> +
> +# Add symlinks for runlevels
> +ln -s ../init.d/${NAME} /etc/rc.d/rc0.d/K85${NAME}
> +ln -s ../init.d/${NAME} /etc/rc.d/rc3.d/S50${NAME}
> +ln -s ../init.d/${NAME} /etc/rc.d/rc6.d/K85${NAME}
> +start_service --background ${NAME}
> +
> +
> +# EOF
> diff --git a/src/paks/pmacct/uninstall.sh b/src/paks/pmacct/uninstall.sh
> new file mode 100644
> index 000000000..51631e8c1
> --- /dev/null
> +++ b/src/paks/pmacct/uninstall.sh
> @@ -0,0 +1,33 @@
> +#!/bin/bash
> +############################################################################
> +# #
> +# This file is part of the IPFire Firewall. #
> +# #
> +# IPFire is free software; you can redistribute it and/or modify #
> +# it under the terms of the GNU General Public License as published by #
> +# the Free Software Foundation; either version 2 of the License, or #
> +# (at your option) any later version. #
> +# #
> +# IPFire is distributed in the hope that it will be useful, #
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of #
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
> +# GNU General Public License for more details. #
> +# #
> +# You should have received a copy of the GNU General Public License #
> +# along with IPFire; if not, write to the Free Software #
> +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
> +# #
> +# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>. #
> +# #
> +############################################################################
> +#
> +. /opt/pakfire/lib/functions.sh
> +
> +stop_service ${NAME}
> +make_backup ${NAME}
> +remove_files
> +
> +# Delete symlinks
> +rm -rfv /etc/rc.d/rc?.d/???${NAME}
> +
> +# EOF
> diff --git a/src/paks/pmacct/update.sh b/src/paks/pmacct/update.sh
> new file mode 100644
> index 000000000..89c40d0d7
> --- /dev/null
> +++ b/src/paks/pmacct/update.sh
> @@ -0,0 +1,26 @@
> +#!/bin/bash
> +############################################################################
> +# #
> +# This file is part of the IPFire Firewall. #
> +# #
> +# IPFire is free software; you can redistribute it and/or modify #
> +# it under the terms of the GNU General Public License as published by #
> +# the Free Software Foundation; either version 2 of the License, or #
> +# (at your option) any later version. #
> +# #
> +# IPFire is distributed in the hope that it will be useful, #
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of #
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
> +# GNU General Public License for more details. #
> +# #
> +# You should have received a copy of the GNU General Public License #
> +# along with IPFire; if not, write to the Free Software #
> +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
> +# #
> +# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>. #
> +# #
> +############################################################################
> +#
> +. /opt/pakfire/lib/functions.sh
> +./uninstall.sh
> +./install.sh
> --
> 2.30.1
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/2] pmacct: New addon
2021-03-18 13:07 ` Michael Tremer
@ 2021-03-18 13:34 ` Adolf Belka
2021-03-18 18:25 ` Michael Tremer
0 siblings, 1 reply; 7+ messages in thread
From: Adolf Belka @ 2021-03-18 13:34 UTC (permalink / raw)
To: development
[-- Attachment #1: Type: text/plain, Size: 27215 bytes --]
Hi Michael,
On 18/03/2021 14:07, Michael Tremer wrote:
> Hello,
>
> I will split this email into two parts:
>
> The technical part:
>
> * The initscript isn’t based on our template and uses sleep to check if starting a daemon has been successful. That should not be necessary and will potentially slow down the boot process.
I will have another look at the initscript and see if it can use the standard IPFire template.
>
> * I am not very sure if the example configuration files are helpful. Maybe someone can clear this up a little bit for me what is the rationale is behind them.
The idea was that the two simplest config example scripts were provided that give a user the ability to get data into memory or into a sqlite file. The actual conf file provided is a copy of the memory version. These give a basic starting point that people can then add additional info to be recorded etc.
>
> * There are loads of daemons being shipped and only one is being used.
I think Jon or Erik need to comment on this question.
>
> * How much load does this generate? How large is the captured data?
Jon needs to provide this feedback.
>
> Then there is the other part:
>
>> On 8 Mar 2021, at 11:57, Adolf Belka (ipfire) <adolf.belka(a)ipfire.org> wrote:
>>
>> From: Adolf Belka <adolf.belka(a)ipfire.org>
>>
>> - What is it?
>> pmacct is a monitoring tool for network management tasks. Data collected
>> can be used for analysis and troubleshooting purposes to maintain the
>> health of the network. pmacct can collect, replicate and export network
>> information. It can cache in memory tables, store persistently to SQLite3
>> and output to flat-files like CSV, formatted, and JSON.
>
> Okay.
>
>> - Why is it needed?
>> To monitor data usage (IP-based or MAC-based data accounting) down to the
>> client level. Net-Traffic will monitor traffic for the entire RED, GREEN,
>> etc. networks, but it cannot pinpoint which client is using lots of data.
>> Connections will take a snapshot but not show day by day sums. pmacct can
>> help admins keep tabs on users that use too much data.
>
> I agree with the goal of it. But how does this add-on achieve that?
Jon should comment on this as he had this use case and can give feedback how the data has helped him in his investigations.
>
> The data is being written to a (usually binary) file which cannot be visualised in the web UI. Does this not want a CGI script that allows people to see what is happening?
We all thought that this would ultimately need a CGI script to set up and visualise the data. We believed it was worth getting this running in actual use as a CLI tool to start with and then get more feedback on the best items that would need to be captured in the CGI script. The tool has a lot of capabilities and I think it will not be possible to do everything that it can do in a WUI page. So some form of file or db offload will need to be available anyway but the items identified as being commonly required would be the ones to be developed into a CGI WUI page.
>
> Sending manual SQL queries probably isn’t much fun :)
>
>> - What are the use cases?
>> An ISP may implement data caps and if the limit is over-run then you have
>> to pay for every additional xxGB of data used. Typical charges can be
>> around $10 per 50GB. With pmacct you can identify the high users and take
>> action, hopefully before the limit is breached.
>> - This is being introduced as a command line only tool. However, at a later
>> date, if it is useful to enough additional users a WUI page could be
>> developed as discussed in the development mailing list
>> https://lists.ipfire.org/pipermail/development/2021-January/009174.html
>
> I have been thinking about this for a while now and would say that I can approve merging this as a CLI tool. But it would not really enhance IPFire’s functionality because it is only usable for people who spend a lot of time writing their own scripts or SQL queries to get information that can be used to make decisions.
We recognise that and had discussed the CGI WUI page as mentioned above.
>
> Adding another alternative to vnstat will result in us having to maintain two things that roughly do the same - or where one’s features are a subset of the other one.
I think Jon or Erik need to comment on any comparison with vnstat.
>
> I would like to avoid merging this and then see it abandoned because it is too complicated to use. We already have a lot of virtually unmaintained stuff around and I would really like to drop a lot of it, although I know that there are a few people actually using it.
Jon and Erik should comment on the complication of use or not. Jon especially has done quite a bit of work with it, installing the .ipfire package versions that I have created.
I intend to also have a play around with it but for me it is more some playful fun as I just have a home network with only myself as the user and admin.
>
> Thoughts?
>
> Best,
> -Michael
>
>> Co-authored-by: Jon Murphy
>> Co-authored-by: Erik Kapfer <ummeegge(a)ipfire.org>
>>
>> Signed-off-by: Adolf Belka <adolf.belka(a)ipfire.org>
>> ---
>> config/backup/includes/pmacct | 2 +
>> config/pmacct/pmacct.conf | 24 +++++
>> config/pmacct/pmacct.init | 57 +++++++++++
>> config/pmacct/pmacct_memory_example.conf | 24 +++++
>> config/pmacct/pmacct_sqlite3_example.conf | 25 +++++
>> config/rootfiles/packages/pmacct | 111 ++++++++++++++++++++++
>> lfs/pmacct | 107 +++++++++++++++++++++
>> make.sh | 2 +
>> src/paks/pmacct/install.sh | 36 +++++++
>> src/paks/pmacct/uninstall.sh | 33 +++++++
>> src/paks/pmacct/update.sh | 26 +++++
>> 11 files changed, 447 insertions(+)
>> create mode 100644 config/backup/includes/pmacct
>> create mode 100644 config/pmacct/pmacct.conf
>> create mode 100644 config/pmacct/pmacct.init
>> create mode 100644 config/pmacct/pmacct_memory_example.conf
>> create mode 100644 config/pmacct/pmacct_sqlite3_example.conf
>> create mode 100644 config/rootfiles/packages/pmacct
>> create mode 100644 lfs/pmacct
>> create mode 100644 src/paks/pmacct/install.sh
>> create mode 100644 src/paks/pmacct/uninstall.sh
>> create mode 100644 src/paks/pmacct/update.sh
>>
>> diff --git a/config/backup/includes/pmacct b/config/backup/includes/pmacct
>> new file mode 100644
>> index 000000000..4359067ad
>> --- /dev/null
>> +++ b/config/backup/includes/pmacct
>> @@ -0,0 +1,2 @@
>> +/etc/pmacct/
>> +/var/spool/pmacct/
>> diff --git a/config/pmacct/pmacct.conf b/config/pmacct/pmacct.conf
>> new file mode 100644
>> index 000000000..3c1d47efa
>> --- /dev/null
>> +++ b/config/pmacct/pmacct.conf
>> @@ -0,0 +1,24 @@
>> +!
>> +! Pmacctd configuration file for IPFire environment
>> +!
>> +
>> +syslog: daemon
>> +daemonize: true
>> +debug: false
>> +promisc: true
>> +interface: green0
>> +
>> +
>> +!
>> +! "plugin1" plugin configuration
>> +!
>> +plugins: memory[plugin1]
>> +
>> +plugin_buffer_size[plugin1]: 102400
>> +plugin_pipe_size[plugin1]: 10240000
>> +
>> +imt_mem_pools_number: 256
>> +imt_path[plugin1]: /var/spool/pmacct/plugin1.pipe
>> +
>> +aggregate[plugin1]: src_host, src_port, src_mac, dst_host, dst_port, dst_mac, proto
>> +aggregate_filter[plugin1]: ip
>> \ No newline at end of file
>> diff --git a/config/pmacct/pmacct.init b/config/pmacct/pmacct.init
>> new file mode 100644
>> index 000000000..31f9ce49b
>> --- /dev/null
>> +++ b/config/pmacct/pmacct.init
>> @@ -0,0 +1,57 @@
>> +#!/bin/bash
>> +
>> +# Begin $rc_base/init.d/pmacct
>> +
>> +# Starts and stops pmacct daemon
>> +# Date: 2021-02-25 12:25:30 (Thu, 25 Feb 2021)
>> +
>> +# Locations
>> +CONF="/etc/pmacct/pmacct.conf"
>> +CONTENT="/var/pmacct"
>> +# Pmacctd binary
>> +PMACCT="/usr/sbin/pmacctd"
>> +# Pmacctd start options '-f' for the configuration file
>> +OPT="-f ${CONF}"
>> +DESC="pmacct daemon"
>> +# Pmacctd syslog facilitiy
>> +#SYSLOG="local1"
>> +
>> +
>> +. /etc/sysconfig/rc
>> +. $rc_functions
>> +
>> +case "$1" in
>> + start)
>> + boot_mesg "Starting the ${DESC}... "
>> + loadproc ${PMACCT} ${OPT}
>> + sleep 2
>> + if P=$(pgrep -fl pmacctd); then
>> + echo -e "\n${P}"
>> + else
>> + echo -e "\n${PMACCT} is not running... "
>> + fi
>> + ;;
>> +
>> + stop)
>> + boot_mesg "Stopping the ${DESC}... "
>> + killproc ${PMACCT}
>> + ;;
>> +
>> + restart)
>> + $0 stop
>> + sleep 1
>> + $0 start
>> + ;;
>> +
>> + status)
>> + statusproc ${PMACCT}
>> + ;;
>> +
>> + *)
>> + echo "Usage: $0 {start|stop|restart|status}"
>> + exit 1
>> + ;;
>> +esac
>> +
>> +# End $rc_base/init.d/pmacct
>> +
>> diff --git a/config/pmacct/pmacct_memory_example.conf b/config/pmacct/pmacct_memory_example.conf
>> new file mode 100644
>> index 000000000..0babdaaf7
>> --- /dev/null
>> +++ b/config/pmacct/pmacct_memory_example.conf
>> @@ -0,0 +1,24 @@
>> +!
>> +! Pmacctd configuration file for IPFire environment
>> +!
>> +
>> +syslog: daemon
>> +daemonize: true
>> +debug: false
>> +promisc: true
>> +interface: green0
>> +
>> +
>> +!
>> +! "plugin1" plugin configuration
>> +!
>> +plugins: memory[plugin1]
>> +
>> +plugin_buffer_size[plugin1]: 102400
>> +plugin_pipe_size[plugin1]: 10240000
>> +
>> +imt_mem_pools_number: 256
>> +imt_path[plugin1]: /var/spool/pmacct/plugin1.pipe
>> +
>> +aggregate[plugin1]: src_host, src_port, src_mac, dst_host, dst_port, dst_mac, proto
>> +aggregate_filter[plugin1]: ip
>> diff --git a/config/pmacct/pmacct_sqlite3_example.conf b/config/pmacct/pmacct_sqlite3_example.conf
>> new file mode 100644
>> index 000000000..7fb996408
>> --- /dev/null
>> +++ b/config/pmacct/pmacct_sqlite3_example.conf
>> @@ -0,0 +1,25 @@
>> +!
>> +! Pmacctd configuration file for IPFire environment
>> +!
>> +
>> +syslog: daemon
>> +daemonize: true
>> +debug: false
>> +promisc: true
>> +interface: green0
>> +
>> +
>> +!
>> +! "plugin2" plugin configuration
>> +!
>> +plugins: sqlite3[plugin2]
>> +
>> +sql_db[plugin2]: /var/spool/pmacct/pmacct_sqlitev1.db
>> +sql_table_version[plugin2]: 1
>> +sql_history[plugin2]: 5m
>> +sql_refresh_time[plugin2]: 300
>> +sql_history_roundoff[plugin2]: m
>> +sql_dont_try_update: true
>> +
>> +aggregate[plugin2]: src_host, src_port, src_mac, dst_host, dst_port, dst_mac, proto
>> +aggregate_filter[plugin2]: ip
>> diff --git a/config/rootfiles/packages/pmacct b/config/rootfiles/packages/pmacct
>> new file mode 100644
>> index 000000000..fed83ee67
>> --- /dev/null
>> +++ b/config/rootfiles/packages/pmacct
>> @@ -0,0 +1,111 @@
>> +etc/pmacct
>> +etc/pmacct/pmacct.conf
>> +etc/pmacct/pmacct_memory_example.conf
>> +etc/pmacct/pmacct_sqlite3_example.conf
>> +etc/rc.d/init.d/pmacct
>> +usr/bin/pmacct
>> +#usr/lib/pmacct
>> +#usr/lib/pmacct/examples
>> +#usr/lib/pmacct/examples/custom
>> +#usr/lib/pmacct/examples/custom/libcustom.la
>> +#usr/lib/pmacct/examples/lg
>> +#usr/lib/pmacct/examples/lg/pmbgp
>> +usr/sbin/nfacctd
>> +usr/sbin/pmacctd
>> +usr/sbin/pmbgpd
>> +usr/sbin/pmbmpd
>> +usr/sbin/pmtelemetryd
>> +usr/sbin/sfacctd
>> +usr/share/pmacct
>> +#usr/share/pmacct/CONFIG-KEYS
>> +#usr/share/pmacct/FAQS
>> +#usr/share/pmacct/QUICKSTART
>> +#usr/share/pmacct/UPGRADE
>> +#usr/share/pmacct/docs
>> +#usr/share/pmacct/docs/IPFIX
>> +#usr/share/pmacct/docs/LOOKING_GLASS_FORMAT
>> +#usr/share/pmacct/docs/MSGLOG_DUMP_FORMATS
>> +#usr/share/pmacct/docs/SIGNALS
>> +#usr/share/pmacct/docs/TRIGGER_VARS
>> +#usr/share/pmacct/examples
>> +usr/share/pmacct/examples/allow.lst.example
>> +#usr/share/pmacct/examples/amqp
>> +usr/share/pmacct/examples/amqp/amqp_receiver.py
>> +#usr/share/pmacct/examples/avro
>> +usr/share/pmacct/examples/avro/avro_file_decoder.py
>> +usr/share/pmacct/examples/bgp_agent.map.example
>> +usr/share/pmacct/examples/bgp_md5.lst.example
>> +usr/share/pmacct/examples/bgp_xconnects.map.example
>> +usr/share/pmacct/examples/flow_to_rd.map.example
>> +#usr/share/pmacct/examples/kafka
>> +usr/share/pmacct/examples/kafka/kafka_consumer.py
>> +#usr/share/pmacct/examples/lg
>> +usr/share/pmacct/examples/lg/pmbgp.py
>> +usr/share/pmacct/examples/networks.lst.example
>> +usr/share/pmacct/examples/nfacctd-sql.conf.example
>> +usr/share/pmacct/examples/pcap_interfaces.map.example
>> +usr/share/pmacct/examples/peers.map.example
>> +usr/share/pmacct/examples/pmacctd-imt.conf.example
>> +usr/share/pmacct/examples/pmacctd-multiple-plugins.conf.example
>> +usr/share/pmacct/examples/pmacctd-sql.conf.example
>> +usr/share/pmacct/examples/pmacctd-sqlite3.conf.example
>> +usr/share/pmacct/examples/ports.lst.example
>> +usr/share/pmacct/examples/pretag.map.example
>> +usr/share/pmacct/examples/primitives.lst.example
>> +usr/share/pmacct/examples/probe_netflow.conf.example
>> +usr/share/pmacct/examples/probe_sflow.conf.example
>> +usr/share/pmacct/examples/sampling.map.example
>> +usr/share/pmacct/examples/tee_receivers.lst.example
>> +#usr/share/pmacct/sql
>> +#usr/share/pmacct/sql/README.64bit
>> +#usr/share/pmacct/sql/README.GeoIP
>> +#usr/share/pmacct/sql/README.IPv6
>> +#usr/share/pmacct/sql/README.cos
>> +#usr/share/pmacct/sql/README.custom_primitives
>> +#usr/share/pmacct/sql/README.etype
>> +#usr/share/pmacct/sql/README.export_proto
>> +#usr/share/pmacct/sql/README.iface
>> +#usr/share/pmacct/sql/README.label
>> +#usr/share/pmacct/sql/README.mask
>> +#usr/share/pmacct/sql/README.mpls
>> +#usr/share/pmacct/sql/README.mysql
>> +#usr/share/pmacct/sql/README.nat
>> +#usr/share/pmacct/sql/README.pgsql
>> +#usr/share/pmacct/sql/README.sampling
>> +usr/share/pmacct/sql/README.sqlite3
>> +#usr/share/pmacct/sql/README.tag2
>> +#usr/share/pmacct/sql/README.timestamp
>> +#usr/share/pmacct/sql/README.tunnel
>> +#usr/share/pmacct/sql/pmacct-create-db.pgsql
>> +#usr/share/pmacct/sql/pmacct-create-db_bgp_v1.mysql
>> +#usr/share/pmacct/sql/pmacct-create-db_v1.mysql
>> +#usr/share/pmacct/sql/pmacct-create-db_v2.mysql
>> +#usr/share/pmacct/sql/pmacct-create-db_v3.mysql
>> +#usr/share/pmacct/sql/pmacct-create-db_v4.mysql
>> +#usr/share/pmacct/sql/pmacct-create-db_v5.mysql
>> +#usr/share/pmacct/sql/pmacct-create-db_v6.mysql
>> +#usr/share/pmacct/sql/pmacct-create-db_v7.mysql
>> +#usr/share/pmacct/sql/pmacct-create-db_v8.mysql
>> +#usr/share/pmacct/sql/pmacct-create-db_v9.mysql
>> +#usr/share/pmacct/sql/pmacct-create-table_bgp_v1.pgsql
>> +usr/share/pmacct/sql/pmacct-create-table_bgp_v1.sqlite3
>> +#usr/share/pmacct/sql/pmacct-create-table_v1.pgsql
>> +usr/share/pmacct/sql/pmacct-create-table_v1.sqlite3
>> +#usr/share/pmacct/sql/pmacct-create-table_v2.pgsql
>> +usr/share/pmacct/sql/pmacct-create-table_v2.sqlite3
>> +#usr/share/pmacct/sql/pmacct-create-table_v3.pgsql
>> +usr/share/pmacct/sql/pmacct-create-table_v3.sqlite3
>> +#usr/share/pmacct/sql/pmacct-create-table_v4.pgsql
>> +usr/share/pmacct/sql/pmacct-create-table_v4.sqlite3
>> +#usr/share/pmacct/sql/pmacct-create-table_v5.pgsql
>> +usr/share/pmacct/sql/pmacct-create-table_v5.sqlite3
>> +#usr/share/pmacct/sql/pmacct-create-table_v6.pgsql
>> +usr/share/pmacct/sql/pmacct-create-table_v6.sqlite3
>> +usr/share/pmacct/sql/pmacct-create-table_v7.sqlite3
>> +#usr/share/pmacct/sql/pmacct-create-table_v7_v8.pgsql
>> +usr/share/pmacct/sql/pmacct-create-table_v8.sqlite3
>> +#usr/share/pmacct/sql/pmacct-create-table_v9.pgsql
>> +usr/share/pmacct/sql/pmacct-create-table_v9.sqlite3
>> +#usr/share/pmacct/sql/pmacct-grant-db.mysql
>> +var/ipfire/backup/addons/includes/pmacct
>> +var/spool/pmacct
>> diff --git a/lfs/pmacct b/lfs/pmacct
>> new file mode 100644
>> index 000000000..c6cc5270e
>> --- /dev/null
>> +++ b/lfs/pmacct
>> @@ -0,0 +1,107 @@
>> +###############################################################################
>> +# #
>> +# IPFire.org - A linux based firewall #
>> +# Copyright (C) 2019 IPFire Team <info(a)ipfire.org> #
>> +# #
>> +# This program is free software: you can redistribute it and/or modify #
>> +# it under the terms of the GNU General Public License as published by #
>> +# the Free Software Foundation, either version 3 of the License, or #
>> +# (at your option) any later version. #
>> +# #
>> +# This program is distributed in the hope that it will be useful, #
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of #
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
>> +# GNU General Public License for more details. #
>> +# #
>> +# You should have received a copy of the GNU General Public License #
>> +# along with this program. If not, see <http://www.gnu.org/licenses/>. #
>> +# #
>> +###############################################################################
>> +
>> +###############################################################################
>> +# Definitions
>> +###############################################################################
>> +
>> +include Config
>> +
>> +VER = 1.7.6
>> +
>> +THISAPP = pmacct-$(VER)
>> +DL_FILE = $(THISAPP).tar.gz
>> +DL_FROM = $(URL_IPFIRE)
>> +DIR_APP = $(DIR_SRC)/$(THISAPP)
>> +TARGET = $(DIR_INFO)/$(THISAPP)
>> +PROG = pmacct
>> +PAK_VER = 1
>> +
>> +DEPS = libcdada
>> +
>> +###############################################################################
>> +# Top-level Rules
>> +###############################################################################
>> +
>> +objects = $(DL_FILE)
>> +
>> +$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
>> +
>> +$(DL_FILE)_MD5 = b829cfdc394d5acd6d32fb23de181a6b
>> +
>> +install : $(TARGET)
>> +
>> +check : $(patsubst %,$(DIR_CHK)/%,$(objects))
>> +
>> +download :$(patsubst %,$(DIR_DL)/%,$(objects))
>> +
>> +md5 : $(subst %,%_MD5,$(objects))
>> +
>> +dist:
>> + @$(PAK)
>> +
>> +###############################################################################
>> +# Downloading, checking, md5sum
>> +###############################################################################
>> +
>> +$(patsubst %,$(DIR_CHK)/%,$(objects)) :
>> + @$(CHECK)
>> +
>> +$(patsubst %,$(DIR_DL)/%,$(objects)) :
>> + @$(LOAD)
>> +
>> +$(subst %,%_MD5,$(objects)) :
>> + @$(MD5)
>> +
>> +###############################################################################
>> +# Installation Details
>> +###############################################################################
>> +
>> +$(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
>> + @$(PREBUILD)
>> + @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar zxf $(DIR_DL)/$(DL_FILE)
>> + cd $(DIR_APP) && ./autogen.sh
>> + cd $(DIR_APP) && ./configure \
>> + --prefix=/usr \
>> + --sysconfdir=/etc/pmacct \
>> + --enable-sqlite3 \
>> + --enable-l2 \
>> + --enable-jansson
>> +
>> + # Add directory for plugins and pipes
>> + -mkdir -v /var/spool/pmacct
>> +
>> + # Add configuration folder and copy conf files to it
>> + -mkdir -v /etc/pmacct
>> + cp -vf $(DIR_CONF)/pmacct/*.conf /etc/pmacct
>> +
>> + # Copy initscript to dir
>> + cp -vf $(DIR_CONF)/pmacct/pmacct.init /etc/rc.d/init.d/pmacct
>> + chmod 754 /etc/rc.d/init.d/pmacct
>> + chown root:root /etc/rc.d/init.d/pmacct
>> +
>> + cd $(DIR_APP) && make $(MAKETUNING)
>> + cd $(DIR_APP) && make install
>> +
>> + # Install backup definition
>> + install -v -m 644 ${DIR_SRC}/config/backup/includes/pmacct \
>> + /var/ipfire/backup/addons/includes/pmacct
>> + @rm -rf $(DIR_APP)
>> + @$(POSTBUILD)
>> diff --git a/make.sh b/make.sh
>> index 43cccaca0..d42c9b1e2 100755
>> --- a/make.sh
>> +++ b/make.sh
>> @@ -1646,6 +1646,8 @@ buildipfire() {
>> lfsmake2 ncdu
>> lfsmake2 lshw
>> lfsmake2 socat
>> + lfsmake2 libcdada
>> + lfsmake2 pmacct
>> }
>>
>> buildinstaller() {
>> diff --git a/src/paks/pmacct/install.sh b/src/paks/pmacct/install.sh
>> new file mode 100644
>> index 000000000..8178cc7bc
>> --- /dev/null
>> +++ b/src/paks/pmacct/install.sh
>> @@ -0,0 +1,36 @@
>> +#!/bin/bash
>> +############################################################################
>> +# #
>> +# This file is part of the IPFire Firewall. #
>> +# #
>> +# IPFire is free software; you can redistribute it and/or modify #
>> +# it under the terms of the GNU General Public License as published by #
>> +# the Free Software Foundation; either version 2 of the License, or #
>> +# (at your option) any later version. #
>> +# #
>> +# IPFire is distributed in the hope that it will be useful, #
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of #
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
>> +# GNU General Public License for more details. #
>> +# #
>> +# You should have received a copy of the GNU General Public License #
>> +# along with IPFire; if not, write to the Free Software #
>> +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
>> +# #
>> +# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>. #
>> +# #
>> +############################################################################
>> +#
>> +. /opt/pakfire/lib/functions.sh
>> +
>> +extract_files
>> +restore_backup ${NAME}
>> +
>> +# Add symlinks for runlevels
>> +ln -s ../init.d/${NAME} /etc/rc.d/rc0.d/K85${NAME}
>> +ln -s ../init.d/${NAME} /etc/rc.d/rc3.d/S50${NAME}
>> +ln -s ../init.d/${NAME} /etc/rc.d/rc6.d/K85${NAME}
>> +start_service --background ${NAME}
>> +
>> +
>> +# EOF
>> diff --git a/src/paks/pmacct/uninstall.sh b/src/paks/pmacct/uninstall.sh
>> new file mode 100644
>> index 000000000..51631e8c1
>> --- /dev/null
>> +++ b/src/paks/pmacct/uninstall.sh
>> @@ -0,0 +1,33 @@
>> +#!/bin/bash
>> +############################################################################
>> +# #
>> +# This file is part of the IPFire Firewall. #
>> +# #
>> +# IPFire is free software; you can redistribute it and/or modify #
>> +# it under the terms of the GNU General Public License as published by #
>> +# the Free Software Foundation; either version 2 of the License, or #
>> +# (at your option) any later version. #
>> +# #
>> +# IPFire is distributed in the hope that it will be useful, #
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of #
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
>> +# GNU General Public License for more details. #
>> +# #
>> +# You should have received a copy of the GNU General Public License #
>> +# along with IPFire; if not, write to the Free Software #
>> +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
>> +# #
>> +# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>. #
>> +# #
>> +############################################################################
>> +#
>> +. /opt/pakfire/lib/functions.sh
>> +
>> +stop_service ${NAME}
>> +make_backup ${NAME}
>> +remove_files
>> +
>> +# Delete symlinks
>> +rm -rfv /etc/rc.d/rc?.d/???${NAME}
>> +
>> +# EOF
>> diff --git a/src/paks/pmacct/update.sh b/src/paks/pmacct/update.sh
>> new file mode 100644
>> index 000000000..89c40d0d7
>> --- /dev/null
>> +++ b/src/paks/pmacct/update.sh
>> @@ -0,0 +1,26 @@
>> +#!/bin/bash
>> +############################################################################
>> +# #
>> +# This file is part of the IPFire Firewall. #
>> +# #
>> +# IPFire is free software; you can redistribute it and/or modify #
>> +# it under the terms of the GNU General Public License as published by #
>> +# the Free Software Foundation; either version 2 of the License, or #
>> +# (at your option) any later version. #
>> +# #
>> +# IPFire is distributed in the hope that it will be useful, #
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of #
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
>> +# GNU General Public License for more details. #
>> +# #
>> +# You should have received a copy of the GNU General Public License #
>> +# along with IPFire; if not, write to the Free Software #
>> +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
>> +# #
>> +# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>. #
>> +# #
>> +############################################################################
>> +#
>> +. /opt/pakfire/lib/functions.sh
>> +./uninstall.sh
>> +./install.sh
>> --
>> 2.30.1
>>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/2] pmacct: New addon
2021-03-18 13:34 ` Adolf Belka
@ 2021-03-18 18:25 ` Michael Tremer
2021-03-18 18:43 ` Adolf Belka
0 siblings, 1 reply; 7+ messages in thread
From: Michael Tremer @ 2021-03-18 18:25 UTC (permalink / raw)
To: development
[-- Attachment #1: Type: text/plain, Size: 28299 bytes --]
Hey,
> On 18 Mar 2021, at 13:34, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>
> Hi Michael,
>
> On 18/03/2021 14:07, Michael Tremer wrote:
>> Hello,
>> I will split this email into two parts:
>> The technical part:
>> * The initscript isn’t based on our template and uses sleep to check if starting a daemon has been successful. That should not be necessary and will potentially slow down the boot process.
> I will have another look at the initscript and see if it can use the standard IPFire template.
Yes, it is not too different from your version:
https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=src/initscripts/system/template;h=69dc2a6a2202fb30da61a7fff6625aea531802e9;hb=HEAD
There isn’t much documentation around it but there are loads of other scripts that will be helpful to figure out any convention or boilerplate code to copy.
>> * I am not very sure if the example configuration files are helpful. Maybe someone can clear this up a little bit for me what is the rationale is behind them.
> The idea was that the two simplest config example scripts were provided that give a user the ability to get data into memory or into a sqlite file. The actual conf file provided is a copy of the memory version. These give a basic starting point that people can then add additional info to be recorded etc.
Does anybody want to use anything else than SQLite in the context of IPFire?
>> * There are loads of daemons being shipped and only one is being used.
> I think Jon or Erik need to comment on this question.
>> * How much load does this generate? How large is the captured data?
> Jon needs to provide this feedback.
>> Then there is the other part:
>>> On 8 Mar 2021, at 11:57, Adolf Belka (ipfire) <adolf.belka(a)ipfire.org> wrote:
>>>
>>> From: Adolf Belka <adolf.belka(a)ipfire.org>
>>>
>>> - What is it?
>>> pmacct is a monitoring tool for network management tasks. Data collected
>>> can be used for analysis and troubleshooting purposes to maintain the
>>> health of the network. pmacct can collect, replicate and export network
>>> information. It can cache in memory tables, store persistently to SQLite3
>>> and output to flat-files like CSV, formatted, and JSON.
>> Okay.
>>> - Why is it needed?
>>> To monitor data usage (IP-based or MAC-based data accounting) down to the
>>> client level. Net-Traffic will monitor traffic for the entire RED, GREEN,
>>> etc. networks, but it cannot pinpoint which client is using lots of data.
>>> Connections will take a snapshot but not show day by day sums. pmacct can
>>> help admins keep tabs on users that use too much data.
>> I agree with the goal of it. But how does this add-on achieve that?
> Jon should comment on this as he had this use case and can give feedback how the data has helped him in his investigations.
>> The data is being written to a (usually binary) file which cannot be visualised in the web UI. Does this not want a CGI script that allows people to see what is happening?
> We all thought that this would ultimately need a CGI script to set up and visualise the data. We believed it was worth getting this running in actual use as a CLI tool to start with and then get more feedback on the best items that would need to be captured in the CGI script. The tool has a lot of capabilities and I think it will not be possible to do everything that it can do in a WUI page. So some form of file or db offload will need to be available anyway but the items identified as being commonly required would be the ones to be developed into a CGI WUI page.
>> Sending manual SQL queries probably isn’t much fun :)
>>> - What are the use cases?
>>> An ISP may implement data caps and if the limit is over-run then you have
>>> to pay for every additional xxGB of data used. Typical charges can be
>>> around $10 per 50GB. With pmacct you can identify the high users and take
>>> action, hopefully before the limit is breached.
>>> - This is being introduced as a command line only tool. However, at a later
>>> date, if it is useful to enough additional users a WUI page could be
>>> developed as discussed in the development mailing list
>>> https://lists.ipfire.org/pipermail/development/2021-January/009174.html
>> I have been thinking about this for a while now and would say that I can approve merging this as a CLI tool. But it would not really enhance IPFire’s functionality because it is only usable for people who spend a lot of time writing their own scripts or SQL queries to get information that can be used to make decisions.
> We recognise that and had discussed the CGI WUI page as mentioned above.
Was this here on the list or elsewhere?
>> Adding another alternative to vnstat will result in us having to maintain two things that roughly do the same - or where one’s features are a subset of the other one.
> I think Jon or Erik need to comment on any comparison with vnstat.
>> I would like to avoid merging this and then see it abandoned because it is too complicated to use. We already have a lot of virtually unmaintained stuff around and I would really like to drop a lot of it, although I know that there are a few people actually using it.
> Jon and Erik should comment on the complication of use or not. Jon especially has done quite a bit of work with it, installing the .ipfire package versions that I have created.
> I intend to also have a play around with it but for me it is more some playful fun as I just have a home network with only myself as the user and admin.
>> Thoughts?
>> Best,
>> -Michael
>>> Co-authored-by: Jon Murphy
>>> Co-authored-by: Erik Kapfer <ummeegge(a)ipfire.org>
>>>
>>> Signed-off-by: Adolf Belka <adolf.belka(a)ipfire.org>
>>> ---
>>> config/backup/includes/pmacct | 2 +
>>> config/pmacct/pmacct.conf | 24 +++++
>>> config/pmacct/pmacct.init | 57 +++++++++++
>>> config/pmacct/pmacct_memory_example.conf | 24 +++++
>>> config/pmacct/pmacct_sqlite3_example.conf | 25 +++++
>>> config/rootfiles/packages/pmacct | 111 ++++++++++++++++++++++
>>> lfs/pmacct | 107 +++++++++++++++++++++
>>> make.sh | 2 +
>>> src/paks/pmacct/install.sh | 36 +++++++
>>> src/paks/pmacct/uninstall.sh | 33 +++++++
>>> src/paks/pmacct/update.sh | 26 +++++
>>> 11 files changed, 447 insertions(+)
>>> create mode 100644 config/backup/includes/pmacct
>>> create mode 100644 config/pmacct/pmacct.conf
>>> create mode 100644 config/pmacct/pmacct.init
>>> create mode 100644 config/pmacct/pmacct_memory_example.conf
>>> create mode 100644 config/pmacct/pmacct_sqlite3_example.conf
>>> create mode 100644 config/rootfiles/packages/pmacct
>>> create mode 100644 lfs/pmacct
>>> create mode 100644 src/paks/pmacct/install.sh
>>> create mode 100644 src/paks/pmacct/uninstall.sh
>>> create mode 100644 src/paks/pmacct/update.sh
>>>
>>> diff --git a/config/backup/includes/pmacct b/config/backup/includes/pmacct
>>> new file mode 100644
>>> index 000000000..4359067ad
>>> --- /dev/null
>>> +++ b/config/backup/includes/pmacct
>>> @@ -0,0 +1,2 @@
>>> +/etc/pmacct/
>>> +/var/spool/pmacct/
>>> diff --git a/config/pmacct/pmacct.conf b/config/pmacct/pmacct.conf
>>> new file mode 100644
>>> index 000000000..3c1d47efa
>>> --- /dev/null
>>> +++ b/config/pmacct/pmacct.conf
>>> @@ -0,0 +1,24 @@
>>> +!
>>> +! Pmacctd configuration file for IPFire environment
>>> +!
>>> +
>>> +syslog: daemon
>>> +daemonize: true
>>> +debug: false
>>> +promisc: true
>>> +interface: green0
>>> +
>>> +
>>> +!
>>> +! "plugin1" plugin configuration
>>> +!
>>> +plugins: memory[plugin1]
>>> +
>>> +plugin_buffer_size[plugin1]: 102400
>>> +plugin_pipe_size[plugin1]: 10240000
>>> +
>>> +imt_mem_pools_number: 256
>>> +imt_path[plugin1]: /var/spool/pmacct/plugin1.pipe
>>> +
>>> +aggregate[plugin1]: src_host, src_port, src_mac, dst_host, dst_port, dst_mac, proto
>>> +aggregate_filter[plugin1]: ip
>>> \ No newline at end of file
>>> diff --git a/config/pmacct/pmacct.init b/config/pmacct/pmacct.init
>>> new file mode 100644
>>> index 000000000..31f9ce49b
>>> --- /dev/null
>>> +++ b/config/pmacct/pmacct.init
>>> @@ -0,0 +1,57 @@
>>> +#!/bin/bash
>>> +
>>> +# Begin $rc_base/init.d/pmacct
>>> +
>>> +# Starts and stops pmacct daemon
>>> +# Date: 2021-02-25 12:25:30 (Thu, 25 Feb 2021)
>>> +
>>> +# Locations
>>> +CONF="/etc/pmacct/pmacct.conf"
>>> +CONTENT="/var/pmacct"
>>> +# Pmacctd binary
>>> +PMACCT="/usr/sbin/pmacctd"
>>> +# Pmacctd start options '-f' for the configuration file
>>> +OPT="-f ${CONF}"
>>> +DESC="pmacct daemon"
>>> +# Pmacctd syslog facilitiy
>>> +#SYSLOG="local1"
>>> +
>>> +
>>> +. /etc/sysconfig/rc
>>> +. $rc_functions
>>> +
>>> +case "$1" in
>>> + start)
>>> + boot_mesg "Starting the ${DESC}... "
>>> + loadproc ${PMACCT} ${OPT}
>>> + sleep 2
>>> + if P=$(pgrep -fl pmacctd); then
>>> + echo -e "\n${P}"
>>> + else
>>> + echo -e "\n${PMACCT} is not running... "
>>> + fi
>>> + ;;
>>> +
>>> + stop)
>>> + boot_mesg "Stopping the ${DESC}... "
>>> + killproc ${PMACCT}
>>> + ;;
>>> +
>>> + restart)
>>> + $0 stop
>>> + sleep 1
>>> + $0 start
>>> + ;;
>>> +
>>> + status)
>>> + statusproc ${PMACCT}
>>> + ;;
>>> +
>>> + *)
>>> + echo "Usage: $0 {start|stop|restart|status}"
>>> + exit 1
>>> + ;;
>>> +esac
>>> +
>>> +# End $rc_base/init.d/pmacct
>>> +
>>> diff --git a/config/pmacct/pmacct_memory_example.conf b/config/pmacct/pmacct_memory_example.conf
>>> new file mode 100644
>>> index 000000000..0babdaaf7
>>> --- /dev/null
>>> +++ b/config/pmacct/pmacct_memory_example.conf
>>> @@ -0,0 +1,24 @@
>>> +!
>>> +! Pmacctd configuration file for IPFire environment
>>> +!
>>> +
>>> +syslog: daemon
>>> +daemonize: true
>>> +debug: false
>>> +promisc: true
>>> +interface: green0
>>> +
>>> +
>>> +!
>>> +! "plugin1" plugin configuration
>>> +!
>>> +plugins: memory[plugin1]
>>> +
>>> +plugin_buffer_size[plugin1]: 102400
>>> +plugin_pipe_size[plugin1]: 10240000
>>> +
>>> +imt_mem_pools_number: 256
>>> +imt_path[plugin1]: /var/spool/pmacct/plugin1.pipe
>>> +
>>> +aggregate[plugin1]: src_host, src_port, src_mac, dst_host, dst_port, dst_mac, proto
>>> +aggregate_filter[plugin1]: ip
>>> diff --git a/config/pmacct/pmacct_sqlite3_example.conf b/config/pmacct/pmacct_sqlite3_example.conf
>>> new file mode 100644
>>> index 000000000..7fb996408
>>> --- /dev/null
>>> +++ b/config/pmacct/pmacct_sqlite3_example.conf
>>> @@ -0,0 +1,25 @@
>>> +!
>>> +! Pmacctd configuration file for IPFire environment
>>> +!
>>> +
>>> +syslog: daemon
>>> +daemonize: true
>>> +debug: false
>>> +promisc: true
>>> +interface: green0
>>> +
>>> +
>>> +!
>>> +! "plugin2" plugin configuration
>>> +!
>>> +plugins: sqlite3[plugin2]
>>> +
>>> +sql_db[plugin2]: /var/spool/pmacct/pmacct_sqlitev1.db
>>> +sql_table_version[plugin2]: 1
>>> +sql_history[plugin2]: 5m
>>> +sql_refresh_time[plugin2]: 300
>>> +sql_history_roundoff[plugin2]: m
>>> +sql_dont_try_update: true
>>> +
>>> +aggregate[plugin2]: src_host, src_port, src_mac, dst_host, dst_port, dst_mac, proto
>>> +aggregate_filter[plugin2]: ip
>>> diff --git a/config/rootfiles/packages/pmacct b/config/rootfiles/packages/pmacct
>>> new file mode 100644
>>> index 000000000..fed83ee67
>>> --- /dev/null
>>> +++ b/config/rootfiles/packages/pmacct
>>> @@ -0,0 +1,111 @@
>>> +etc/pmacct
>>> +etc/pmacct/pmacct.conf
>>> +etc/pmacct/pmacct_memory_example.conf
>>> +etc/pmacct/pmacct_sqlite3_example.conf
>>> +etc/rc.d/init.d/pmacct
>>> +usr/bin/pmacct
>>> +#usr/lib/pmacct
>>> +#usr/lib/pmacct/examples
>>> +#usr/lib/pmacct/examples/custom
>>> +#usr/lib/pmacct/examples/custom/libcustom.la
>>> +#usr/lib/pmacct/examples/lg
>>> +#usr/lib/pmacct/examples/lg/pmbgp
>>> +usr/sbin/nfacctd
>>> +usr/sbin/pmacctd
>>> +usr/sbin/pmbgpd
>>> +usr/sbin/pmbmpd
>>> +usr/sbin/pmtelemetryd
>>> +usr/sbin/sfacctd
>>> +usr/share/pmacct
>>> +#usr/share/pmacct/CONFIG-KEYS
>>> +#usr/share/pmacct/FAQS
>>> +#usr/share/pmacct/QUICKSTART
>>> +#usr/share/pmacct/UPGRADE
>>> +#usr/share/pmacct/docs
>>> +#usr/share/pmacct/docs/IPFIX
>>> +#usr/share/pmacct/docs/LOOKING_GLASS_FORMAT
>>> +#usr/share/pmacct/docs/MSGLOG_DUMP_FORMATS
>>> +#usr/share/pmacct/docs/SIGNALS
>>> +#usr/share/pmacct/docs/TRIGGER_VARS
>>> +#usr/share/pmacct/examples
>>> +usr/share/pmacct/examples/allow.lst.example
>>> +#usr/share/pmacct/examples/amqp
>>> +usr/share/pmacct/examples/amqp/amqp_receiver.py
>>> +#usr/share/pmacct/examples/avro
>>> +usr/share/pmacct/examples/avro/avro_file_decoder.py
>>> +usr/share/pmacct/examples/bgp_agent.map.example
>>> +usr/share/pmacct/examples/bgp_md5.lst.example
>>> +usr/share/pmacct/examples/bgp_xconnects.map.example
>>> +usr/share/pmacct/examples/flow_to_rd.map.example
>>> +#usr/share/pmacct/examples/kafka
>>> +usr/share/pmacct/examples/kafka/kafka_consumer.py
>>> +#usr/share/pmacct/examples/lg
>>> +usr/share/pmacct/examples/lg/pmbgp.py
>>> +usr/share/pmacct/examples/networks.lst.example
>>> +usr/share/pmacct/examples/nfacctd-sql.conf.example
>>> +usr/share/pmacct/examples/pcap_interfaces.map.example
>>> +usr/share/pmacct/examples/peers.map.example
>>> +usr/share/pmacct/examples/pmacctd-imt.conf.example
>>> +usr/share/pmacct/examples/pmacctd-multiple-plugins.conf.example
>>> +usr/share/pmacct/examples/pmacctd-sql.conf.example
>>> +usr/share/pmacct/examples/pmacctd-sqlite3.conf.example
>>> +usr/share/pmacct/examples/ports.lst.example
>>> +usr/share/pmacct/examples/pretag.map.example
>>> +usr/share/pmacct/examples/primitives.lst.example
>>> +usr/share/pmacct/examples/probe_netflow.conf.example
>>> +usr/share/pmacct/examples/probe_sflow.conf.example
>>> +usr/share/pmacct/examples/sampling.map.example
>>> +usr/share/pmacct/examples/tee_receivers.lst.example
>>> +#usr/share/pmacct/sql
>>> +#usr/share/pmacct/sql/README.64bit
>>> +#usr/share/pmacct/sql/README.GeoIP
>>> +#usr/share/pmacct/sql/README.IPv6
>>> +#usr/share/pmacct/sql/README.cos
>>> +#usr/share/pmacct/sql/README.custom_primitives
>>> +#usr/share/pmacct/sql/README.etype
>>> +#usr/share/pmacct/sql/README.export_proto
>>> +#usr/share/pmacct/sql/README.iface
>>> +#usr/share/pmacct/sql/README.label
>>> +#usr/share/pmacct/sql/README.mask
>>> +#usr/share/pmacct/sql/README.mpls
>>> +#usr/share/pmacct/sql/README.mysql
>>> +#usr/share/pmacct/sql/README.nat
>>> +#usr/share/pmacct/sql/README.pgsql
>>> +#usr/share/pmacct/sql/README.sampling
>>> +usr/share/pmacct/sql/README.sqlite3
>>> +#usr/share/pmacct/sql/README.tag2
>>> +#usr/share/pmacct/sql/README.timestamp
>>> +#usr/share/pmacct/sql/README.tunnel
>>> +#usr/share/pmacct/sql/pmacct-create-db.pgsql
>>> +#usr/share/pmacct/sql/pmacct-create-db_bgp_v1.mysql
>>> +#usr/share/pmacct/sql/pmacct-create-db_v1.mysql
>>> +#usr/share/pmacct/sql/pmacct-create-db_v2.mysql
>>> +#usr/share/pmacct/sql/pmacct-create-db_v3.mysql
>>> +#usr/share/pmacct/sql/pmacct-create-db_v4.mysql
>>> +#usr/share/pmacct/sql/pmacct-create-db_v5.mysql
>>> +#usr/share/pmacct/sql/pmacct-create-db_v6.mysql
>>> +#usr/share/pmacct/sql/pmacct-create-db_v7.mysql
>>> +#usr/share/pmacct/sql/pmacct-create-db_v8.mysql
>>> +#usr/share/pmacct/sql/pmacct-create-db_v9.mysql
>>> +#usr/share/pmacct/sql/pmacct-create-table_bgp_v1.pgsql
>>> +usr/share/pmacct/sql/pmacct-create-table_bgp_v1.sqlite3
>>> +#usr/share/pmacct/sql/pmacct-create-table_v1.pgsql
>>> +usr/share/pmacct/sql/pmacct-create-table_v1.sqlite3
>>> +#usr/share/pmacct/sql/pmacct-create-table_v2.pgsql
>>> +usr/share/pmacct/sql/pmacct-create-table_v2.sqlite3
>>> +#usr/share/pmacct/sql/pmacct-create-table_v3.pgsql
>>> +usr/share/pmacct/sql/pmacct-create-table_v3.sqlite3
>>> +#usr/share/pmacct/sql/pmacct-create-table_v4.pgsql
>>> +usr/share/pmacct/sql/pmacct-create-table_v4.sqlite3
>>> +#usr/share/pmacct/sql/pmacct-create-table_v5.pgsql
>>> +usr/share/pmacct/sql/pmacct-create-table_v5.sqlite3
>>> +#usr/share/pmacct/sql/pmacct-create-table_v6.pgsql
>>> +usr/share/pmacct/sql/pmacct-create-table_v6.sqlite3
>>> +usr/share/pmacct/sql/pmacct-create-table_v7.sqlite3
>>> +#usr/share/pmacct/sql/pmacct-create-table_v7_v8.pgsql
>>> +usr/share/pmacct/sql/pmacct-create-table_v8.sqlite3
>>> +#usr/share/pmacct/sql/pmacct-create-table_v9.pgsql
>>> +usr/share/pmacct/sql/pmacct-create-table_v9.sqlite3
>>> +#usr/share/pmacct/sql/pmacct-grant-db.mysql
>>> +var/ipfire/backup/addons/includes/pmacct
>>> +var/spool/pmacct
>>> diff --git a/lfs/pmacct b/lfs/pmacct
>>> new file mode 100644
>>> index 000000000..c6cc5270e
>>> --- /dev/null
>>> +++ b/lfs/pmacct
>>> @@ -0,0 +1,107 @@
>>> +###############################################################################
>>> +# #
>>> +# IPFire.org - A linux based firewall #
>>> +# Copyright (C) 2019 IPFire Team <info(a)ipfire.org> #
>>> +# #
>>> +# This program is free software: you can redistribute it and/or modify #
>>> +# it under the terms of the GNU General Public License as published by #
>>> +# the Free Software Foundation, either version 3 of the License, or #
>>> +# (at your option) any later version. #
>>> +# #
>>> +# This program is distributed in the hope that it will be useful, #
>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of #
>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
>>> +# GNU General Public License for more details. #
>>> +# #
>>> +# You should have received a copy of the GNU General Public License #
>>> +# along with this program. If not, see <http://www.gnu.org/licenses/>. #
>>> +# #
>>> +###############################################################################
>>> +
>>> +###############################################################################
>>> +# Definitions
>>> +###############################################################################
>>> +
>>> +include Config
>>> +
>>> +VER = 1.7.6
>>> +
>>> +THISAPP = pmacct-$(VER)
>>> +DL_FILE = $(THISAPP).tar.gz
>>> +DL_FROM = $(URL_IPFIRE)
>>> +DIR_APP = $(DIR_SRC)/$(THISAPP)
>>> +TARGET = $(DIR_INFO)/$(THISAPP)
>>> +PROG = pmacct
>>> +PAK_VER = 1
>>> +
>>> +DEPS = libcdada
>>> +
>>> +###############################################################################
>>> +# Top-level Rules
>>> +###############################################################################
>>> +
>>> +objects = $(DL_FILE)
>>> +
>>> +$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
>>> +
>>> +$(DL_FILE)_MD5 = b829cfdc394d5acd6d32fb23de181a6b
>>> +
>>> +install : $(TARGET)
>>> +
>>> +check : $(patsubst %,$(DIR_CHK)/%,$(objects))
>>> +
>>> +download :$(patsubst %,$(DIR_DL)/%,$(objects))
>>> +
>>> +md5 : $(subst %,%_MD5,$(objects))
>>> +
>>> +dist:
>>> + @$(PAK)
>>> +
>>> +###############################################################################
>>> +# Downloading, checking, md5sum
>>> +###############################################################################
>>> +
>>> +$(patsubst %,$(DIR_CHK)/%,$(objects)) :
>>> + @$(CHECK)
>>> +
>>> +$(patsubst %,$(DIR_DL)/%,$(objects)) :
>>> + @$(LOAD)
>>> +
>>> +$(subst %,%_MD5,$(objects)) :
>>> + @$(MD5)
>>> +
>>> +###############################################################################
>>> +# Installation Details
>>> +###############################################################################
>>> +
>>> +$(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
>>> + @$(PREBUILD)
>>> + @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar zxf $(DIR_DL)/$(DL_FILE)
>>> + cd $(DIR_APP) && ./autogen.sh
>>> + cd $(DIR_APP) && ./configure \
>>> + --prefix=/usr \
>>> + --sysconfdir=/etc/pmacct \
>>> + --enable-sqlite3 \
>>> + --enable-l2 \
>>> + --enable-jansson
>>> +
>>> + # Add directory for plugins and pipes
>>> + -mkdir -v /var/spool/pmacct
>>> +
>>> + # Add configuration folder and copy conf files to it
>>> + -mkdir -v /etc/pmacct
>>> + cp -vf $(DIR_CONF)/pmacct/*.conf /etc/pmacct
>>> +
>>> + # Copy initscript to dir
>>> + cp -vf $(DIR_CONF)/pmacct/pmacct.init /etc/rc.d/init.d/pmacct
>>> + chmod 754 /etc/rc.d/init.d/pmacct
>>> + chown root:root /etc/rc.d/init.d/pmacct
>>> +
>>> + cd $(DIR_APP) && make $(MAKETUNING)
>>> + cd $(DIR_APP) && make install
>>> +
>>> + # Install backup definition
>>> + install -v -m 644 ${DIR_SRC}/config/backup/includes/pmacct \
>>> + /var/ipfire/backup/addons/includes/pmacct
>>> + @rm -rf $(DIR_APP)
>>> + @$(POSTBUILD)
>>> diff --git a/make.sh b/make.sh
>>> index 43cccaca0..d42c9b1e2 100755
>>> --- a/make.sh
>>> +++ b/make.sh
>>> @@ -1646,6 +1646,8 @@ buildipfire() {
>>> lfsmake2 ncdu
>>> lfsmake2 lshw
>>> lfsmake2 socat
>>> + lfsmake2 libcdada
>>> + lfsmake2 pmacct
>>> }
>>>
>>> buildinstaller() {
>>> diff --git a/src/paks/pmacct/install.sh b/src/paks/pmacct/install.sh
>>> new file mode 100644
>>> index 000000000..8178cc7bc
>>> --- /dev/null
>>> +++ b/src/paks/pmacct/install.sh
>>> @@ -0,0 +1,36 @@
>>> +#!/bin/bash
>>> +############################################################################
>>> +# #
>>> +# This file is part of the IPFire Firewall. #
>>> +# #
>>> +# IPFire is free software; you can redistribute it and/or modify #
>>> +# it under the terms of the GNU General Public License as published by #
>>> +# the Free Software Foundation; either version 2 of the License, or #
>>> +# (at your option) any later version. #
>>> +# #
>>> +# IPFire is distributed in the hope that it will be useful, #
>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of #
>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
>>> +# GNU General Public License for more details. #
>>> +# #
>>> +# You should have received a copy of the GNU General Public License #
>>> +# along with IPFire; if not, write to the Free Software #
>>> +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
>>> +# #
>>> +# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>. #
>>> +# #
>>> +############################################################################
>>> +#
>>> +. /opt/pakfire/lib/functions.sh
>>> +
>>> +extract_files
>>> +restore_backup ${NAME}
>>> +
>>> +# Add symlinks for runlevels
>>> +ln -s ../init.d/${NAME} /etc/rc.d/rc0.d/K85${NAME}
>>> +ln -s ../init.d/${NAME} /etc/rc.d/rc3.d/S50${NAME}
>>> +ln -s ../init.d/${NAME} /etc/rc.d/rc6.d/K85${NAME}
>>> +start_service --background ${NAME}
>>> +
>>> +
>>> +# EOF
>>> diff --git a/src/paks/pmacct/uninstall.sh b/src/paks/pmacct/uninstall.sh
>>> new file mode 100644
>>> index 000000000..51631e8c1
>>> --- /dev/null
>>> +++ b/src/paks/pmacct/uninstall.sh
>>> @@ -0,0 +1,33 @@
>>> +#!/bin/bash
>>> +############################################################################
>>> +# #
>>> +# This file is part of the IPFire Firewall. #
>>> +# #
>>> +# IPFire is free software; you can redistribute it and/or modify #
>>> +# it under the terms of the GNU General Public License as published by #
>>> +# the Free Software Foundation; either version 2 of the License, or #
>>> +# (at your option) any later version. #
>>> +# #
>>> +# IPFire is distributed in the hope that it will be useful, #
>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of #
>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
>>> +# GNU General Public License for more details. #
>>> +# #
>>> +# You should have received a copy of the GNU General Public License #
>>> +# along with IPFire; if not, write to the Free Software #
>>> +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
>>> +# #
>>> +# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>. #
>>> +# #
>>> +############################################################################
>>> +#
>>> +. /opt/pakfire/lib/functions.sh
>>> +
>>> +stop_service ${NAME}
>>> +make_backup ${NAME}
>>> +remove_files
>>> +
>>> +# Delete symlinks
>>> +rm -rfv /etc/rc.d/rc?.d/???${NAME}
>>> +
>>> +# EOF
>>> diff --git a/src/paks/pmacct/update.sh b/src/paks/pmacct/update.sh
>>> new file mode 100644
>>> index 000000000..89c40d0d7
>>> --- /dev/null
>>> +++ b/src/paks/pmacct/update.sh
>>> @@ -0,0 +1,26 @@
>>> +#!/bin/bash
>>> +############################################################################
>>> +# #
>>> +# This file is part of the IPFire Firewall. #
>>> +# #
>>> +# IPFire is free software; you can redistribute it and/or modify #
>>> +# it under the terms of the GNU General Public License as published by #
>>> +# the Free Software Foundation; either version 2 of the License, or #
>>> +# (at your option) any later version. #
>>> +# #
>>> +# IPFire is distributed in the hope that it will be useful, #
>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of #
>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
>>> +# GNU General Public License for more details. #
>>> +# #
>>> +# You should have received a copy of the GNU General Public License #
>>> +# along with IPFire; if not, write to the Free Software #
>>> +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
>>> +# #
>>> +# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>. #
>>> +# #
>>> +############################################################################
>>> +#
>>> +. /opt/pakfire/lib/functions.sh
>>> +./uninstall.sh
>>> +./install.sh
>>> --
>>> 2.30.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/2] pmacct: New addon
2021-03-18 18:25 ` Michael Tremer
@ 2021-03-18 18:43 ` Adolf Belka
2021-03-19 9:20 ` Adolf Belka
0 siblings, 1 reply; 7+ messages in thread
From: Adolf Belka @ 2021-03-18 18:43 UTC (permalink / raw)
To: development
[-- Attachment #1: Type: text/plain, Size: 29753 bytes --]
Hi Michael,
On 18/03/2021 19:25, Michael Tremer wrote:
> Hey,
>
>> On 18 Mar 2021, at 13:34, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>
>> Hi Michael,
>>
>> On 18/03/2021 14:07, Michael Tremer wrote:
>>> Hello,
>>> I will split this email into two parts:
>>> The technical part:
>>> * The initscript isn’t based on our template and uses sleep to check if starting a daemon has been successful. That should not be necessary and will potentially slow down the boot process.
>> I will have another look at the initscript and see if it can use the standard IPFire template.
>
> Yes, it is not too different from your version:
>
> https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=src/initscripts/system/template;h=69dc2a6a2202fb30da61a7fff6625aea531802e9;hb=HEAD
>
> There isn’t much documentation around it but there are loads of other scripts that will be helpful to figure out any convention or boilerplate code to copy.
>
>>> * I am not very sure if the example configuration files are helpful. Maybe someone can clear this up a little bit for me what is the rationale is behind them.
>> The idea was that the two simplest config example scripts were provided that give a user the ability to get data into memory or into a sqlite file. The actual conf file provided is a copy of the memory version. These give a basic starting point that people can then add additional info to be recorded etc.
>
> Does anybody want to use anything else than SQLite in the context of IPFire?
I think we weren't sure and the memory and sqlite options were the easiest to get going and give real data. There are also mysql and postgresql options but I think we need to see what the interest is.
>
>>> * There are loads of daemons being shipped and only one is being used.
>> I think Jon or Erik need to comment on this question.
>>> * How much load does this generate? How large is the captured data?
>> Jon needs to provide this feedback.
>>> Then there is the other part:
>>>> On 8 Mar 2021, at 11:57, Adolf Belka (ipfire) <adolf.belka(a)ipfire.org> wrote:
>>>>
>>>> From: Adolf Belka <adolf.belka(a)ipfire.org>
>>>>
>>>> - What is it?
>>>> pmacct is a monitoring tool for network management tasks. Data collected
>>>> can be used for analysis and troubleshooting purposes to maintain the
>>>> health of the network. pmacct can collect, replicate and export network
>>>> information. It can cache in memory tables, store persistently to SQLite3
>>>> and output to flat-files like CSV, formatted, and JSON.
>>> Okay.
>>>> - Why is it needed?
>>>> To monitor data usage (IP-based or MAC-based data accounting) down to the
>>>> client level. Net-Traffic will monitor traffic for the entire RED, GREEN,
>>>> etc. networks, but it cannot pinpoint which client is using lots of data.
>>>> Connections will take a snapshot but not show day by day sums. pmacct can
>>>> help admins keep tabs on users that use too much data.
>>> I agree with the goal of it. But how does this add-on achieve that?
>> Jon should comment on this as he had this use case and can give feedback how the data has helped him in his investigations.
>>> The data is being written to a (usually binary) file which cannot be visualised in the web UI. Does this not want a CGI script that allows people to see what is happening?
>> We all thought that this would ultimately need a CGI script to set up and visualise the data. We believed it was worth getting this running in actual use as a CLI tool to start with and then get more feedback on the best items that would need to be captured in the CGI script. The tool has a lot of capabilities and I think it will not be possible to do everything that it can do in a WUI page. So some form of file or db offload will need to be available anyway but the items identified as being commonly required would be the ones to be developed into a CGI WUI page.
>>> Sending manual SQL queries probably isn’t much fun :)
>>>> - What are the use cases?
>>>> An ISP may implement data caps and if the limit is over-run then you have
>>>> to pay for every additional xxGB of data used. Typical charges can be
>>>> around $10 per 50GB. With pmacct you can identify the high users and take
>>>> action, hopefully before the limit is breached.
>>>> - This is being introduced as a command line only tool. However, at a later
>>>> date, if it is useful to enough additional users a WUI page could be
>>>> developed as discussed in the development mailing list
>>>> https://lists.ipfire.org/pipermail/development/2021-January/009174.html
>>> I have been thinking about this for a while now and would say that I can approve merging this as a CLI tool. But it would not really enhance IPFire’s functionality because it is only usable for people who spend a lot of time writing their own scripts or SQL queries to get information that can be used to make decisions.
>> We recognise that and had discussed the CGI WUI page as mentioned above.
>
> Was this here on the list or elsewhere?
This was in a private conversation on the IPFire Community forum. We thought it best to have all our back and forth discussions between ourselves to start with. We were all finding our feet with it as well and we thought it best not to swamp the development list with our musings and questions to each other. If you want to see all the discussion history, then we can add you to the members of that conversation.
If the addon gets accepted then any future discussions would be done on the Development mailing list to get everyones input.
>
>>> Adding another alternative to vnstat will result in us having to maintain two things that roughly do the same - or where one’s features are a subset of the other one.
>> I think Jon or Erik need to comment on any comparison with vnstat.
>>> I would like to avoid merging this and then see it abandoned because it is too complicated to use. We already have a lot of virtually unmaintained stuff around and I would really like to drop a lot of it, although I know that there are a few people actually using it.
>> Jon and Erik should comment on the complication of use or not. Jon especially has done quite a bit of work with it, installing the .ipfire package versions that I have created.
>> I intend to also have a play around with it but for me it is more some playful fun as I just have a home network with only myself as the user and admin.
>>> Thoughts?
>>> Best,
>>> -Michael
>>>> Co-authored-by: Jon Murphy
>>>> Co-authored-by: Erik Kapfer <ummeegge(a)ipfire.org>
>>>>
>>>> Signed-off-by: Adolf Belka <adolf.belka(a)ipfire.org>
>>>> ---
>>>> config/backup/includes/pmacct | 2 +
>>>> config/pmacct/pmacct.conf | 24 +++++
>>>> config/pmacct/pmacct.init | 57 +++++++++++
>>>> config/pmacct/pmacct_memory_example.conf | 24 +++++
>>>> config/pmacct/pmacct_sqlite3_example.conf | 25 +++++
>>>> config/rootfiles/packages/pmacct | 111 ++++++++++++++++++++++
>>>> lfs/pmacct | 107 +++++++++++++++++++++
>>>> make.sh | 2 +
>>>> src/paks/pmacct/install.sh | 36 +++++++
>>>> src/paks/pmacct/uninstall.sh | 33 +++++++
>>>> src/paks/pmacct/update.sh | 26 +++++
>>>> 11 files changed, 447 insertions(+)
>>>> create mode 100644 config/backup/includes/pmacct
>>>> create mode 100644 config/pmacct/pmacct.conf
>>>> create mode 100644 config/pmacct/pmacct.init
>>>> create mode 100644 config/pmacct/pmacct_memory_example.conf
>>>> create mode 100644 config/pmacct/pmacct_sqlite3_example.conf
>>>> create mode 100644 config/rootfiles/packages/pmacct
>>>> create mode 100644 lfs/pmacct
>>>> create mode 100644 src/paks/pmacct/install.sh
>>>> create mode 100644 src/paks/pmacct/uninstall.sh
>>>> create mode 100644 src/paks/pmacct/update.sh
>>>>
>>>> diff --git a/config/backup/includes/pmacct b/config/backup/includes/pmacct
>>>> new file mode 100644
>>>> index 000000000..4359067ad
>>>> --- /dev/null
>>>> +++ b/config/backup/includes/pmacct
>>>> @@ -0,0 +1,2 @@
>>>> +/etc/pmacct/
>>>> +/var/spool/pmacct/
>>>> diff --git a/config/pmacct/pmacct.conf b/config/pmacct/pmacct.conf
>>>> new file mode 100644
>>>> index 000000000..3c1d47efa
>>>> --- /dev/null
>>>> +++ b/config/pmacct/pmacct.conf
>>>> @@ -0,0 +1,24 @@
>>>> +!
>>>> +! Pmacctd configuration file for IPFire environment
>>>> +!
>>>> +
>>>> +syslog: daemon
>>>> +daemonize: true
>>>> +debug: false
>>>> +promisc: true
>>>> +interface: green0
>>>> +
>>>> +
>>>> +!
>>>> +! "plugin1" plugin configuration
>>>> +!
>>>> +plugins: memory[plugin1]
>>>> +
>>>> +plugin_buffer_size[plugin1]: 102400
>>>> +plugin_pipe_size[plugin1]: 10240000
>>>> +
>>>> +imt_mem_pools_number: 256
>>>> +imt_path[plugin1]: /var/spool/pmacct/plugin1.pipe
>>>> +
>>>> +aggregate[plugin1]: src_host, src_port, src_mac, dst_host, dst_port, dst_mac, proto
>>>> +aggregate_filter[plugin1]: ip
>>>> \ No newline at end of file
>>>> diff --git a/config/pmacct/pmacct.init b/config/pmacct/pmacct.init
>>>> new file mode 100644
>>>> index 000000000..31f9ce49b
>>>> --- /dev/null
>>>> +++ b/config/pmacct/pmacct.init
>>>> @@ -0,0 +1,57 @@
>>>> +#!/bin/bash
>>>> +
>>>> +# Begin $rc_base/init.d/pmacct
>>>> +
>>>> +# Starts and stops pmacct daemon
>>>> +# Date: 2021-02-25 12:25:30 (Thu, 25 Feb 2021)
>>>> +
>>>> +# Locations
>>>> +CONF="/etc/pmacct/pmacct.conf"
>>>> +CONTENT="/var/pmacct"
>>>> +# Pmacctd binary
>>>> +PMACCT="/usr/sbin/pmacctd"
>>>> +# Pmacctd start options '-f' for the configuration file
>>>> +OPT="-f ${CONF}"
>>>> +DESC="pmacct daemon"
>>>> +# Pmacctd syslog facilitiy
>>>> +#SYSLOG="local1"
>>>> +
>>>> +
>>>> +. /etc/sysconfig/rc
>>>> +. $rc_functions
>>>> +
>>>> +case "$1" in
>>>> + start)
>>>> + boot_mesg "Starting the ${DESC}... "
>>>> + loadproc ${PMACCT} ${OPT}
>>>> + sleep 2
>>>> + if P=$(pgrep -fl pmacctd); then
>>>> + echo -e "\n${P}"
>>>> + else
>>>> + echo -e "\n${PMACCT} is not running... "
>>>> + fi
>>>> + ;;
>>>> +
>>>> + stop)
>>>> + boot_mesg "Stopping the ${DESC}... "
>>>> + killproc ${PMACCT}
>>>> + ;;
>>>> +
>>>> + restart)
>>>> + $0 stop
>>>> + sleep 1
>>>> + $0 start
>>>> + ;;
>>>> +
>>>> + status)
>>>> + statusproc ${PMACCT}
>>>> + ;;
>>>> +
>>>> + *)
>>>> + echo "Usage: $0 {start|stop|restart|status}"
>>>> + exit 1
>>>> + ;;
>>>> +esac
>>>> +
>>>> +# End $rc_base/init.d/pmacct
>>>> +
>>>> diff --git a/config/pmacct/pmacct_memory_example.conf b/config/pmacct/pmacct_memory_example.conf
>>>> new file mode 100644
>>>> index 000000000..0babdaaf7
>>>> --- /dev/null
>>>> +++ b/config/pmacct/pmacct_memory_example.conf
>>>> @@ -0,0 +1,24 @@
>>>> +!
>>>> +! Pmacctd configuration file for IPFire environment
>>>> +!
>>>> +
>>>> +syslog: daemon
>>>> +daemonize: true
>>>> +debug: false
>>>> +promisc: true
>>>> +interface: green0
>>>> +
>>>> +
>>>> +!
>>>> +! "plugin1" plugin configuration
>>>> +!
>>>> +plugins: memory[plugin1]
>>>> +
>>>> +plugin_buffer_size[plugin1]: 102400
>>>> +plugin_pipe_size[plugin1]: 10240000
>>>> +
>>>> +imt_mem_pools_number: 256
>>>> +imt_path[plugin1]: /var/spool/pmacct/plugin1.pipe
>>>> +
>>>> +aggregate[plugin1]: src_host, src_port, src_mac, dst_host, dst_port, dst_mac, proto
>>>> +aggregate_filter[plugin1]: ip
>>>> diff --git a/config/pmacct/pmacct_sqlite3_example.conf b/config/pmacct/pmacct_sqlite3_example.conf
>>>> new file mode 100644
>>>> index 000000000..7fb996408
>>>> --- /dev/null
>>>> +++ b/config/pmacct/pmacct_sqlite3_example.conf
>>>> @@ -0,0 +1,25 @@
>>>> +!
>>>> +! Pmacctd configuration file for IPFire environment
>>>> +!
>>>> +
>>>> +syslog: daemon
>>>> +daemonize: true
>>>> +debug: false
>>>> +promisc: true
>>>> +interface: green0
>>>> +
>>>> +
>>>> +!
>>>> +! "plugin2" plugin configuration
>>>> +!
>>>> +plugins: sqlite3[plugin2]
>>>> +
>>>> +sql_db[plugin2]: /var/spool/pmacct/pmacct_sqlitev1.db
>>>> +sql_table_version[plugin2]: 1
>>>> +sql_history[plugin2]: 5m
>>>> +sql_refresh_time[plugin2]: 300
>>>> +sql_history_roundoff[plugin2]: m
>>>> +sql_dont_try_update: true
>>>> +
>>>> +aggregate[plugin2]: src_host, src_port, src_mac, dst_host, dst_port, dst_mac, proto
>>>> +aggregate_filter[plugin2]: ip
>>>> diff --git a/config/rootfiles/packages/pmacct b/config/rootfiles/packages/pmacct
>>>> new file mode 100644
>>>> index 000000000..fed83ee67
>>>> --- /dev/null
>>>> +++ b/config/rootfiles/packages/pmacct
>>>> @@ -0,0 +1,111 @@
>>>> +etc/pmacct
>>>> +etc/pmacct/pmacct.conf
>>>> +etc/pmacct/pmacct_memory_example.conf
>>>> +etc/pmacct/pmacct_sqlite3_example.conf
>>>> +etc/rc.d/init.d/pmacct
>>>> +usr/bin/pmacct
>>>> +#usr/lib/pmacct
>>>> +#usr/lib/pmacct/examples
>>>> +#usr/lib/pmacct/examples/custom
>>>> +#usr/lib/pmacct/examples/custom/libcustom.la
>>>> +#usr/lib/pmacct/examples/lg
>>>> +#usr/lib/pmacct/examples/lg/pmbgp
>>>> +usr/sbin/nfacctd
>>>> +usr/sbin/pmacctd
>>>> +usr/sbin/pmbgpd
>>>> +usr/sbin/pmbmpd
>>>> +usr/sbin/pmtelemetryd
>>>> +usr/sbin/sfacctd
>>>> +usr/share/pmacct
>>>> +#usr/share/pmacct/CONFIG-KEYS
>>>> +#usr/share/pmacct/FAQS
>>>> +#usr/share/pmacct/QUICKSTART
>>>> +#usr/share/pmacct/UPGRADE
>>>> +#usr/share/pmacct/docs
>>>> +#usr/share/pmacct/docs/IPFIX
>>>> +#usr/share/pmacct/docs/LOOKING_GLASS_FORMAT
>>>> +#usr/share/pmacct/docs/MSGLOG_DUMP_FORMATS
>>>> +#usr/share/pmacct/docs/SIGNALS
>>>> +#usr/share/pmacct/docs/TRIGGER_VARS
>>>> +#usr/share/pmacct/examples
>>>> +usr/share/pmacct/examples/allow.lst.example
>>>> +#usr/share/pmacct/examples/amqp
>>>> +usr/share/pmacct/examples/amqp/amqp_receiver.py
>>>> +#usr/share/pmacct/examples/avro
>>>> +usr/share/pmacct/examples/avro/avro_file_decoder.py
>>>> +usr/share/pmacct/examples/bgp_agent.map.example
>>>> +usr/share/pmacct/examples/bgp_md5.lst.example
>>>> +usr/share/pmacct/examples/bgp_xconnects.map.example
>>>> +usr/share/pmacct/examples/flow_to_rd.map.example
>>>> +#usr/share/pmacct/examples/kafka
>>>> +usr/share/pmacct/examples/kafka/kafka_consumer.py
>>>> +#usr/share/pmacct/examples/lg
>>>> +usr/share/pmacct/examples/lg/pmbgp.py
>>>> +usr/share/pmacct/examples/networks.lst.example
>>>> +usr/share/pmacct/examples/nfacctd-sql.conf.example
>>>> +usr/share/pmacct/examples/pcap_interfaces.map.example
>>>> +usr/share/pmacct/examples/peers.map.example
>>>> +usr/share/pmacct/examples/pmacctd-imt.conf.example
>>>> +usr/share/pmacct/examples/pmacctd-multiple-plugins.conf.example
>>>> +usr/share/pmacct/examples/pmacctd-sql.conf.example
>>>> +usr/share/pmacct/examples/pmacctd-sqlite3.conf.example
>>>> +usr/share/pmacct/examples/ports.lst.example
>>>> +usr/share/pmacct/examples/pretag.map.example
>>>> +usr/share/pmacct/examples/primitives.lst.example
>>>> +usr/share/pmacct/examples/probe_netflow.conf.example
>>>> +usr/share/pmacct/examples/probe_sflow.conf.example
>>>> +usr/share/pmacct/examples/sampling.map.example
>>>> +usr/share/pmacct/examples/tee_receivers.lst.example
>>>> +#usr/share/pmacct/sql
>>>> +#usr/share/pmacct/sql/README.64bit
>>>> +#usr/share/pmacct/sql/README.GeoIP
>>>> +#usr/share/pmacct/sql/README.IPv6
>>>> +#usr/share/pmacct/sql/README.cos
>>>> +#usr/share/pmacct/sql/README.custom_primitives
>>>> +#usr/share/pmacct/sql/README.etype
>>>> +#usr/share/pmacct/sql/README.export_proto
>>>> +#usr/share/pmacct/sql/README.iface
>>>> +#usr/share/pmacct/sql/README.label
>>>> +#usr/share/pmacct/sql/README.mask
>>>> +#usr/share/pmacct/sql/README.mpls
>>>> +#usr/share/pmacct/sql/README.mysql
>>>> +#usr/share/pmacct/sql/README.nat
>>>> +#usr/share/pmacct/sql/README.pgsql
>>>> +#usr/share/pmacct/sql/README.sampling
>>>> +usr/share/pmacct/sql/README.sqlite3
>>>> +#usr/share/pmacct/sql/README.tag2
>>>> +#usr/share/pmacct/sql/README.timestamp
>>>> +#usr/share/pmacct/sql/README.tunnel
>>>> +#usr/share/pmacct/sql/pmacct-create-db.pgsql
>>>> +#usr/share/pmacct/sql/pmacct-create-db_bgp_v1.mysql
>>>> +#usr/share/pmacct/sql/pmacct-create-db_v1.mysql
>>>> +#usr/share/pmacct/sql/pmacct-create-db_v2.mysql
>>>> +#usr/share/pmacct/sql/pmacct-create-db_v3.mysql
>>>> +#usr/share/pmacct/sql/pmacct-create-db_v4.mysql
>>>> +#usr/share/pmacct/sql/pmacct-create-db_v5.mysql
>>>> +#usr/share/pmacct/sql/pmacct-create-db_v6.mysql
>>>> +#usr/share/pmacct/sql/pmacct-create-db_v7.mysql
>>>> +#usr/share/pmacct/sql/pmacct-create-db_v8.mysql
>>>> +#usr/share/pmacct/sql/pmacct-create-db_v9.mysql
>>>> +#usr/share/pmacct/sql/pmacct-create-table_bgp_v1.pgsql
>>>> +usr/share/pmacct/sql/pmacct-create-table_bgp_v1.sqlite3
>>>> +#usr/share/pmacct/sql/pmacct-create-table_v1.pgsql
>>>> +usr/share/pmacct/sql/pmacct-create-table_v1.sqlite3
>>>> +#usr/share/pmacct/sql/pmacct-create-table_v2.pgsql
>>>> +usr/share/pmacct/sql/pmacct-create-table_v2.sqlite3
>>>> +#usr/share/pmacct/sql/pmacct-create-table_v3.pgsql
>>>> +usr/share/pmacct/sql/pmacct-create-table_v3.sqlite3
>>>> +#usr/share/pmacct/sql/pmacct-create-table_v4.pgsql
>>>> +usr/share/pmacct/sql/pmacct-create-table_v4.sqlite3
>>>> +#usr/share/pmacct/sql/pmacct-create-table_v5.pgsql
>>>> +usr/share/pmacct/sql/pmacct-create-table_v5.sqlite3
>>>> +#usr/share/pmacct/sql/pmacct-create-table_v6.pgsql
>>>> +usr/share/pmacct/sql/pmacct-create-table_v6.sqlite3
>>>> +usr/share/pmacct/sql/pmacct-create-table_v7.sqlite3
>>>> +#usr/share/pmacct/sql/pmacct-create-table_v7_v8.pgsql
>>>> +usr/share/pmacct/sql/pmacct-create-table_v8.sqlite3
>>>> +#usr/share/pmacct/sql/pmacct-create-table_v9.pgsql
>>>> +usr/share/pmacct/sql/pmacct-create-table_v9.sqlite3
>>>> +#usr/share/pmacct/sql/pmacct-grant-db.mysql
>>>> +var/ipfire/backup/addons/includes/pmacct
>>>> +var/spool/pmacct
>>>> diff --git a/lfs/pmacct b/lfs/pmacct
>>>> new file mode 100644
>>>> index 000000000..c6cc5270e
>>>> --- /dev/null
>>>> +++ b/lfs/pmacct
>>>> @@ -0,0 +1,107 @@
>>>> +###############################################################################
>>>> +# #
>>>> +# IPFire.org - A linux based firewall #
>>>> +# Copyright (C) 2019 IPFire Team <info(a)ipfire.org> #
>>>> +# #
>>>> +# This program is free software: you can redistribute it and/or modify #
>>>> +# it under the terms of the GNU General Public License as published by #
>>>> +# the Free Software Foundation, either version 3 of the License, or #
>>>> +# (at your option) any later version. #
>>>> +# #
>>>> +# This program is distributed in the hope that it will be useful, #
>>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of #
>>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
>>>> +# GNU General Public License for more details. #
>>>> +# #
>>>> +# You should have received a copy of the GNU General Public License #
>>>> +# along with this program. If not, see <http://www.gnu.org/licenses/>. #
>>>> +# #
>>>> +###############################################################################
>>>> +
>>>> +###############################################################################
>>>> +# Definitions
>>>> +###############################################################################
>>>> +
>>>> +include Config
>>>> +
>>>> +VER = 1.7.6
>>>> +
>>>> +THISAPP = pmacct-$(VER)
>>>> +DL_FILE = $(THISAPP).tar.gz
>>>> +DL_FROM = $(URL_IPFIRE)
>>>> +DIR_APP = $(DIR_SRC)/$(THISAPP)
>>>> +TARGET = $(DIR_INFO)/$(THISAPP)
>>>> +PROG = pmacct
>>>> +PAK_VER = 1
>>>> +
>>>> +DEPS = libcdada
>>>> +
>>>> +###############################################################################
>>>> +# Top-level Rules
>>>> +###############################################################################
>>>> +
>>>> +objects = $(DL_FILE)
>>>> +
>>>> +$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
>>>> +
>>>> +$(DL_FILE)_MD5 = b829cfdc394d5acd6d32fb23de181a6b
>>>> +
>>>> +install : $(TARGET)
>>>> +
>>>> +check : $(patsubst %,$(DIR_CHK)/%,$(objects))
>>>> +
>>>> +download :$(patsubst %,$(DIR_DL)/%,$(objects))
>>>> +
>>>> +md5 : $(subst %,%_MD5,$(objects))
>>>> +
>>>> +dist:
>>>> + @$(PAK)
>>>> +
>>>> +###############################################################################
>>>> +# Downloading, checking, md5sum
>>>> +###############################################################################
>>>> +
>>>> +$(patsubst %,$(DIR_CHK)/%,$(objects)) :
>>>> + @$(CHECK)
>>>> +
>>>> +$(patsubst %,$(DIR_DL)/%,$(objects)) :
>>>> + @$(LOAD)
>>>> +
>>>> +$(subst %,%_MD5,$(objects)) :
>>>> + @$(MD5)
>>>> +
>>>> +###############################################################################
>>>> +# Installation Details
>>>> +###############################################################################
>>>> +
>>>> +$(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
>>>> + @$(PREBUILD)
>>>> + @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar zxf $(DIR_DL)/$(DL_FILE)
>>>> + cd $(DIR_APP) && ./autogen.sh
>>>> + cd $(DIR_APP) && ./configure \
>>>> + --prefix=/usr \
>>>> + --sysconfdir=/etc/pmacct \
>>>> + --enable-sqlite3 \
>>>> + --enable-l2 \
>>>> + --enable-jansson
>>>> +
>>>> + # Add directory for plugins and pipes
>>>> + -mkdir -v /var/spool/pmacct
>>>> +
>>>> + # Add configuration folder and copy conf files to it
>>>> + -mkdir -v /etc/pmacct
>>>> + cp -vf $(DIR_CONF)/pmacct/*.conf /etc/pmacct
>>>> +
>>>> + # Copy initscript to dir
>>>> + cp -vf $(DIR_CONF)/pmacct/pmacct.init /etc/rc.d/init.d/pmacct
>>>> + chmod 754 /etc/rc.d/init.d/pmacct
>>>> + chown root:root /etc/rc.d/init.d/pmacct
>>>> +
>>>> + cd $(DIR_APP) && make $(MAKETUNING)
>>>> + cd $(DIR_APP) && make install
>>>> +
>>>> + # Install backup definition
>>>> + install -v -m 644 ${DIR_SRC}/config/backup/includes/pmacct \
>>>> + /var/ipfire/backup/addons/includes/pmacct
>>>> + @rm -rf $(DIR_APP)
>>>> + @$(POSTBUILD)
>>>> diff --git a/make.sh b/make.sh
>>>> index 43cccaca0..d42c9b1e2 100755
>>>> --- a/make.sh
>>>> +++ b/make.sh
>>>> @@ -1646,6 +1646,8 @@ buildipfire() {
>>>> lfsmake2 ncdu
>>>> lfsmake2 lshw
>>>> lfsmake2 socat
>>>> + lfsmake2 libcdada
>>>> + lfsmake2 pmacct
>>>> }
>>>>
>>>> buildinstaller() {
>>>> diff --git a/src/paks/pmacct/install.sh b/src/paks/pmacct/install.sh
>>>> new file mode 100644
>>>> index 000000000..8178cc7bc
>>>> --- /dev/null
>>>> +++ b/src/paks/pmacct/install.sh
>>>> @@ -0,0 +1,36 @@
>>>> +#!/bin/bash
>>>> +############################################################################
>>>> +# #
>>>> +# This file is part of the IPFire Firewall. #
>>>> +# #
>>>> +# IPFire is free software; you can redistribute it and/or modify #
>>>> +# it under the terms of the GNU General Public License as published by #
>>>> +# the Free Software Foundation; either version 2 of the License, or #
>>>> +# (at your option) any later version. #
>>>> +# #
>>>> +# IPFire is distributed in the hope that it will be useful, #
>>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of #
>>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
>>>> +# GNU General Public License for more details. #
>>>> +# #
>>>> +# You should have received a copy of the GNU General Public License #
>>>> +# along with IPFire; if not, write to the Free Software #
>>>> +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
>>>> +# #
>>>> +# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>. #
>>>> +# #
>>>> +############################################################################
>>>> +#
>>>> +. /opt/pakfire/lib/functions.sh
>>>> +
>>>> +extract_files
>>>> +restore_backup ${NAME}
>>>> +
>>>> +# Add symlinks for runlevels
>>>> +ln -s ../init.d/${NAME} /etc/rc.d/rc0.d/K85${NAME}
>>>> +ln -s ../init.d/${NAME} /etc/rc.d/rc3.d/S50${NAME}
>>>> +ln -s ../init.d/${NAME} /etc/rc.d/rc6.d/K85${NAME}
>>>> +start_service --background ${NAME}
>>>> +
>>>> +
>>>> +# EOF
>>>> diff --git a/src/paks/pmacct/uninstall.sh b/src/paks/pmacct/uninstall.sh
>>>> new file mode 100644
>>>> index 000000000..51631e8c1
>>>> --- /dev/null
>>>> +++ b/src/paks/pmacct/uninstall.sh
>>>> @@ -0,0 +1,33 @@
>>>> +#!/bin/bash
>>>> +############################################################################
>>>> +# #
>>>> +# This file is part of the IPFire Firewall. #
>>>> +# #
>>>> +# IPFire is free software; you can redistribute it and/or modify #
>>>> +# it under the terms of the GNU General Public License as published by #
>>>> +# the Free Software Foundation; either version 2 of the License, or #
>>>> +# (at your option) any later version. #
>>>> +# #
>>>> +# IPFire is distributed in the hope that it will be useful, #
>>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of #
>>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
>>>> +# GNU General Public License for more details. #
>>>> +# #
>>>> +# You should have received a copy of the GNU General Public License #
>>>> +# along with IPFire; if not, write to the Free Software #
>>>> +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
>>>> +# #
>>>> +# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>. #
>>>> +# #
>>>> +############################################################################
>>>> +#
>>>> +. /opt/pakfire/lib/functions.sh
>>>> +
>>>> +stop_service ${NAME}
>>>> +make_backup ${NAME}
>>>> +remove_files
>>>> +
>>>> +# Delete symlinks
>>>> +rm -rfv /etc/rc.d/rc?.d/???${NAME}
>>>> +
>>>> +# EOF
>>>> diff --git a/src/paks/pmacct/update.sh b/src/paks/pmacct/update.sh
>>>> new file mode 100644
>>>> index 000000000..89c40d0d7
>>>> --- /dev/null
>>>> +++ b/src/paks/pmacct/update.sh
>>>> @@ -0,0 +1,26 @@
>>>> +#!/bin/bash
>>>> +############################################################################
>>>> +# #
>>>> +# This file is part of the IPFire Firewall. #
>>>> +# #
>>>> +# IPFire is free software; you can redistribute it and/or modify #
>>>> +# it under the terms of the GNU General Public License as published by #
>>>> +# the Free Software Foundation; either version 2 of the License, or #
>>>> +# (at your option) any later version. #
>>>> +# #
>>>> +# IPFire is distributed in the hope that it will be useful, #
>>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of #
>>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
>>>> +# GNU General Public License for more details. #
>>>> +# #
>>>> +# You should have received a copy of the GNU General Public License #
>>>> +# along with IPFire; if not, write to the Free Software #
>>>> +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
>>>> +# #
>>>> +# Copyright (C) 2007 IPFire-Team <info(a)ipfire.org>. #
>>>> +# #
>>>> +############################################################################
>>>> +#
>>>> +. /opt/pakfire/lib/functions.sh
>>>> +./uninstall.sh
>>>> +./install.sh
>>>> --
>>>> 2.30.1
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/2] pmacct: New addon
2021-03-18 18:43 ` Adolf Belka
@ 2021-03-19 9:20 ` Adolf Belka
2021-03-19 17:19 ` ummeegge
0 siblings, 1 reply; 7+ messages in thread
From: Adolf Belka @ 2021-03-19 9:20 UTC (permalink / raw)
To: development
[-- Attachment #1: Type: text/plain, Size: 33348 bytes --]
Hi Michael,
On 18/03/2021 19:43, Adolf Belka wrote:
> Hi Michael,
>
> On 18/03/2021 19:25, Michael Tremer wrote:
>> Hey,
>>
>>> On 18 Mar 2021, at 13:34, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>
>>> Hi Michael,
>>>
>>> On 18/03/2021 14:07, Michael Tremer wrote:
>>>> Hello,
>>>> I will split this email into two parts:
>>>> The technical part:
>>>> * The initscript isn’t based on our template and uses sleep to
>>>> check if starting a daemon has been successful. That should not be
>>>> necessary and will potentially slow down the boot process.
>>> I will have another look at the initscript and see if it can use the
>>> standard IPFire template.
>>
>> Yes, it is not too different from your version:
>>
>> https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=src/initscripts/system/template;h=69dc2a6a2202fb30da61a7fff6625aea531802e9;hb=HEAD
>>
>> There isn’t much documentation around it but there are loads of other
>> scripts that will be helpful to figure out any convention or
>> boilerplate code to copy.
>>
>>>> * I am not very sure if the example configuration files are
>>>> helpful. Maybe someone can clear this up a little bit for me what
>>>> is the rationale is behind them.
>>> The idea was that the two simplest config example scripts were
>>> provided that give a user the ability to get data into memory or
>>> into a sqlite file. The actual conf file provided is a copy of the
>>> memory version. These give a basic starting point that people can
>>> then add additional info to be recorded etc.
>>
>> Does anybody want to use anything else than SQLite in the context of
>> IPFire?
> I think we weren't sure and the memory and sqlite options were the
> easiest to get going and give real data. There are also mysql and
> postgresql options but I think we need to see what the interest is.
>>
>>>> * There are loads of daemons being shipped and only one is being used.
>>>>
>>>> I looked through our conversations and it looks like these were
>>>> left there because we weren't sure if they would be used or not.
>>>>
>>>> pmacctd is the daemon being used by Jon. pmacct is the frontend to
>>>> pmacctd when being used in memory mode.
>>>>
>>>> For the others I looked in the pmacct website and found the following:-
>>>>
I looked through our conversations and it looks like these were left
there because we weren't sure if they would be used or not.
pmacctd is the daemon being used by Jon. pmacct is the frontend to
pmacctd when being used in memory mode.
For the others I looked in the pmacct website and found the following:-
'pmacctd' is the libpcap-based IPv4/IPv6 accounting daemon
'nfacctd' is the NetFlow (versions supported NetFlow v1, v5 and v9) and
IPFIX accounting daemon
'sfacctd' is the sFlow v2/v4 and v5 accounting daemon
'uacctd' is the Linux NFLOG-based accounting daemon (historically, it
was using ULOG, hence its name)
'pmtelemetryd' is the Streaming Telemetry collector, where "Streaming
telemetry [ .. ] data can be used for analysis and troubleshooting
purposes to maintain the health of the network. This is achieved by
leveraging the capabilities of machine-to-machine communication. [ .. ]"
(quoting Cisco IOS-XR Telemetry Configuration Guide at the time of this
writing)
'pmbgpd' is the pmacct BGP collector and 'pmbmpd'is the pmacct BMP
collector.
I don't know what most of that means but it looks to be quite different
to what the addon was created for, so I would propose that I re-do the
rootfiles and remove those daemons from the install. If in future there
is demand for others we can add them back in with an update patch. I
will do that together with the other changes already discussed.
Regards,
Adolf.
>>> I think Jon or Erik need to comment on this question.
>>>> * How much load does this generate? How large is the captured data?
>>> Jon needs to provide this feedback.
>>>> Then there is the other part:
>>>>> On 8 Mar 2021, at 11:57, Adolf Belka (ipfire)
>>>>> <adolf.belka(a)ipfire.org> wrote:
>>>>>
>>>>> From: Adolf Belka <adolf.belka(a)ipfire.org>
>>>>>
>>>>> - What is it?
>>>>> pmacct is a monitoring tool for network management tasks. Data
>>>>> collected
>>>>> can be used for analysis and troubleshooting purposes to
>>>>> maintain the
>>>>> health of the network. pmacct can collect, replicate and export
>>>>> network
>>>>> information. It can cache in memory tables, store persistently
>>>>> to SQLite3
>>>>> and output to flat-files like CSV, formatted, and JSON.
>>>> Okay.
>>>>> - Why is it needed?
>>>>> To monitor data usage (IP-based or MAC-based data accounting)
>>>>> down to the
>>>>> client level. Net-Traffic will monitor traffic for the entire
>>>>> RED, GREEN,
>>>>> etc. networks, but it cannot pinpoint which client is using
>>>>> lots of data.
>>>>> Connections will take a snapshot but not show day by day sums.
>>>>> pmacct can
>>>>> help admins keep tabs on users that use too much data.
>>>> I agree with the goal of it. But how does this add-on achieve that?
>>> Jon should comment on this as he had this use case and can give
>>> feedback how the data has helped him in his investigations.
>>>> The data is being written to a (usually binary) file which cannot
>>>> be visualised in the web UI. Does this not want a CGI script that
>>>> allows people to see what is happening?
>>> We all thought that this would ultimately need a CGI script to set
>>> up and visualise the data. We believed it was worth getting this
>>> running in actual use as a CLI tool to start with and then get more
>>> feedback on the best items that would need to be captured in the CGI
>>> script. The tool has a lot of capabilities and I think it will not
>>> be possible to do everything that it can do in a WUI page. So some
>>> form of file or db offload will need to be available anyway but the
>>> items identified as being commonly required would be the ones to be
>>> developed into a CGI WUI page.
>>>> Sending manual SQL queries probably isn’t much fun :)
>>>>> - What are the use cases?
>>>>> An ISP may implement data caps and if the limit is over-run
>>>>> then you have
>>>>> to pay for every additional xxGB of data used. Typical charges
>>>>> can be
>>>>> around $10 per 50GB. With pmacct you can identify the high
>>>>> users and take
>>>>> action, hopefully before the limit is breached.
>>>>> - This is being introduced as a command line only tool. However,
>>>>> at a later
>>>>> date, if it is useful to enough additional users a WUI page
>>>>> could be
>>>>> developed as discussed in the development mailing list
>>>>> https://lists.ipfire.org/pipermail/development/2021-January/009174.html
>>>>>
>>>> I have been thinking about this for a while now and would say that
>>>> I can approve merging this as a CLI tool. But it would not really
>>>> enhance IPFire’s functionality because it is only usable for people
>>>> who spend a lot of time writing their own scripts or SQL queries to
>>>> get information that can be used to make decisions.
>>> We recognise that and had discussed the CGI WUI page as mentioned
>>> above.
>>
>> Was this here on the list or elsewhere?
> This was in a private conversation on the IPFire Community forum. We
> thought it best to have all our back and forth discussions between
> ourselves to start with. We were all finding our feet with it as well
> and we thought it best not to swamp the development list with our
> musings and questions to each other. If you want to see all the
> discussion history, then we can add you to the members of that
> conversation.
> If the addon gets accepted then any future discussions would be done
> on the Development mailing list to get everyones input.
>>
>>>> Adding another alternative to vnstat will result in us having to
>>>> maintain two things that roughly do the same - or where one’s
>>>> features are a subset of the other one.
>>> I think Jon or Erik need to comment on any comparison with vnstat.
>>>> I would like to avoid merging this and then see it abandoned
>>>> because it is too complicated to use. We already have a lot of
>>>> virtually unmaintained stuff around and I would really like to drop
>>>> a lot of it, although I know that there are a few people actually
>>>> using it.
>>> Jon and Erik should comment on the complication of use or not. Jon
>>> especially has done quite a bit of work with it, installing the
>>> .ipfire package versions that I have created.
>>> I intend to also have a play around with it but for me it is more
>>> some playful fun as I just have a home network with only myself as
>>> the user and admin.
>>>> Thoughts?
>>>> Best,
>>>> -Michael
>>>>> Co-authored-by: Jon Murphy
>>>>> Co-authored-by: Erik Kapfer <ummeegge(a)ipfire.org>
>>>>>
>>>>> Signed-off-by: Adolf Belka <adolf.belka(a)ipfire.org>
>>>>> ---
>>>>> config/backup/includes/pmacct | 2 +
>>>>> config/pmacct/pmacct.conf | 24 +++++
>>>>> config/pmacct/pmacct.init | 57 +++++++++++
>>>>> config/pmacct/pmacct_memory_example.conf | 24 +++++
>>>>> config/pmacct/pmacct_sqlite3_example.conf | 25 +++++
>>>>> config/rootfiles/packages/pmacct | 111
>>>>> ++++++++++++++++++++++
>>>>> lfs/pmacct | 107 +++++++++++++++++++++
>>>>> make.sh | 2 +
>>>>> src/paks/pmacct/install.sh | 36 +++++++
>>>>> src/paks/pmacct/uninstall.sh | 33 +++++++
>>>>> src/paks/pmacct/update.sh | 26 +++++
>>>>> 11 files changed, 447 insertions(+)
>>>>> create mode 100644 config/backup/includes/pmacct
>>>>> create mode 100644 config/pmacct/pmacct.conf
>>>>> create mode 100644 config/pmacct/pmacct.init
>>>>> create mode 100644 config/pmacct/pmacct_memory_example.conf
>>>>> create mode 100644 config/pmacct/pmacct_sqlite3_example.conf
>>>>> create mode 100644 config/rootfiles/packages/pmacct
>>>>> create mode 100644 lfs/pmacct
>>>>> create mode 100644 src/paks/pmacct/install.sh
>>>>> create mode 100644 src/paks/pmacct/uninstall.sh
>>>>> create mode 100644 src/paks/pmacct/update.sh
>>>>>
>>>>> diff --git a/config/backup/includes/pmacct
>>>>> b/config/backup/includes/pmacct
>>>>> new file mode 100644
>>>>> index 000000000..4359067ad
>>>>> --- /dev/null
>>>>> +++ b/config/backup/includes/pmacct
>>>>> @@ -0,0 +1,2 @@
>>>>> +/etc/pmacct/
>>>>> +/var/spool/pmacct/
>>>>> diff --git a/config/pmacct/pmacct.conf b/config/pmacct/pmacct.conf
>>>>> new file mode 100644
>>>>> index 000000000..3c1d47efa
>>>>> --- /dev/null
>>>>> +++ b/config/pmacct/pmacct.conf
>>>>> @@ -0,0 +1,24 @@
>>>>> +!
>>>>> +! Pmacctd configuration file for IPFire environment
>>>>> +!
>>>>> +
>>>>> +syslog: daemon
>>>>> +daemonize: true
>>>>> +debug: false
>>>>> +promisc: true
>>>>> +interface: green0
>>>>> +
>>>>> +
>>>>> +!
>>>>> +! "plugin1" plugin configuration
>>>>> +!
>>>>> +plugins: memory[plugin1]
>>>>> +
>>>>> +plugin_buffer_size[plugin1]: 102400
>>>>> +plugin_pipe_size[plugin1]: 10240000
>>>>> +
>>>>> +imt_mem_pools_number: 256
>>>>> +imt_path[plugin1]: /var/spool/pmacct/plugin1.pipe
>>>>> +
>>>>> +aggregate[plugin1]: src_host, src_port, src_mac, dst_host,
>>>>> dst_port, dst_mac, proto
>>>>> +aggregate_filter[plugin1]: ip
>>>>> \ No newline at end of file
>>>>> diff --git a/config/pmacct/pmacct.init b/config/pmacct/pmacct.init
>>>>> new file mode 100644
>>>>> index 000000000..31f9ce49b
>>>>> --- /dev/null
>>>>> +++ b/config/pmacct/pmacct.init
>>>>> @@ -0,0 +1,57 @@
>>>>> +#!/bin/bash
>>>>> +
>>>>> +# Begin $rc_base/init.d/pmacct
>>>>> +
>>>>> +# Starts and stops pmacct daemon
>>>>> +# Date: 2021-02-25 12:25:30 (Thu, 25 Feb 2021)
>>>>> +
>>>>> +# Locations
>>>>> +CONF="/etc/pmacct/pmacct.conf"
>>>>> +CONTENT="/var/pmacct"
>>>>> +# Pmacctd binary
>>>>> +PMACCT="/usr/sbin/pmacctd"
>>>>> +# Pmacctd start options '-f' for the configuration file
>>>>> +OPT="-f ${CONF}"
>>>>> +DESC="pmacct daemon"
>>>>> +# Pmacctd syslog facilitiy
>>>>> +#SYSLOG="local1"
>>>>> +
>>>>> +
>>>>> +. /etc/sysconfig/rc
>>>>> +. $rc_functions
>>>>> +
>>>>> +case "$1" in
>>>>> + start)
>>>>> + boot_mesg "Starting the ${DESC}... "
>>>>> + loadproc ${PMACCT} ${OPT}
>>>>> + sleep 2
>>>>> + if P=$(pgrep -fl pmacctd); then
>>>>> + echo -e "\n${P}"
>>>>> + else
>>>>> + echo -e "\n${PMACCT} is not running... "
>>>>> + fi
>>>>> + ;;
>>>>> +
>>>>> + stop)
>>>>> + boot_mesg "Stopping the ${DESC}... "
>>>>> + killproc ${PMACCT}
>>>>> + ;;
>>>>> +
>>>>> + restart)
>>>>> + $0 stop
>>>>> + sleep 1
>>>>> + $0 start
>>>>> + ;;
>>>>> +
>>>>> + status)
>>>>> + statusproc ${PMACCT}
>>>>> + ;;
>>>>> +
>>>>> + *)
>>>>> + echo "Usage: $0 {start|stop|restart|status}"
>>>>> + exit 1
>>>>> + ;;
>>>>> +esac
>>>>> +
>>>>> +# End $rc_base/init.d/pmacct
>>>>> +
>>>>> diff --git a/config/pmacct/pmacct_memory_example.conf
>>>>> b/config/pmacct/pmacct_memory_example.conf
>>>>> new file mode 100644
>>>>> index 000000000..0babdaaf7
>>>>> --- /dev/null
>>>>> +++ b/config/pmacct/pmacct_memory_example.conf
>>>>> @@ -0,0 +1,24 @@
>>>>> +!
>>>>> +! Pmacctd configuration file for IPFire environment
>>>>> +!
>>>>> +
>>>>> +syslog: daemon
>>>>> +daemonize: true
>>>>> +debug: false
>>>>> +promisc: true
>>>>> +interface: green0
>>>>> +
>>>>> +
>>>>> +!
>>>>> +! "plugin1" plugin configuration
>>>>> +!
>>>>> +plugins: memory[plugin1]
>>>>> +
>>>>> +plugin_buffer_size[plugin1]: 102400
>>>>> +plugin_pipe_size[plugin1]: 10240000
>>>>> +
>>>>> +imt_mem_pools_number: 256
>>>>> +imt_path[plugin1]: /var/spool/pmacct/plugin1.pipe
>>>>> +
>>>>> +aggregate[plugin1]: src_host, src_port, src_mac, dst_host,
>>>>> dst_port, dst_mac, proto
>>>>> +aggregate_filter[plugin1]: ip
>>>>> diff --git a/config/pmacct/pmacct_sqlite3_example.conf
>>>>> b/config/pmacct/pmacct_sqlite3_example.conf
>>>>> new file mode 100644
>>>>> index 000000000..7fb996408
>>>>> --- /dev/null
>>>>> +++ b/config/pmacct/pmacct_sqlite3_example.conf
>>>>> @@ -0,0 +1,25 @@
>>>>> +!
>>>>> +! Pmacctd configuration file for IPFire environment
>>>>> +!
>>>>> +
>>>>> +syslog: daemon
>>>>> +daemonize: true
>>>>> +debug: false
>>>>> +promisc: true
>>>>> +interface: green0
>>>>> +
>>>>> +
>>>>> +!
>>>>> +! "plugin2" plugin configuration
>>>>> +!
>>>>> +plugins: sqlite3[plugin2]
>>>>> +
>>>>> +sql_db[plugin2]: /var/spool/pmacct/pmacct_sqlitev1.db
>>>>> +sql_table_version[plugin2]: 1
>>>>> +sql_history[plugin2]: 5m
>>>>> +sql_refresh_time[plugin2]: 300
>>>>> +sql_history_roundoff[plugin2]: m
>>>>> +sql_dont_try_update: true
>>>>> +
>>>>> +aggregate[plugin2]: src_host, src_port, src_mac, dst_host,
>>>>> dst_port, dst_mac, proto
>>>>> +aggregate_filter[plugin2]: ip
>>>>> diff --git a/config/rootfiles/packages/pmacct
>>>>> b/config/rootfiles/packages/pmacct
>>>>> new file mode 100644
>>>>> index 000000000..fed83ee67
>>>>> --- /dev/null
>>>>> +++ b/config/rootfiles/packages/pmacct
>>>>> @@ -0,0 +1,111 @@
>>>>> +etc/pmacct
>>>>> +etc/pmacct/pmacct.conf
>>>>> +etc/pmacct/pmacct_memory_example.conf
>>>>> +etc/pmacct/pmacct_sqlite3_example.conf
>>>>> +etc/rc.d/init.d/pmacct
>>>>> +usr/bin/pmacct
>>>>> +#usr/lib/pmacct
>>>>> +#usr/lib/pmacct/examples
>>>>> +#usr/lib/pmacct/examples/custom
>>>>> +#usr/lib/pmacct/examples/custom/libcustom.la
>>>>> +#usr/lib/pmacct/examples/lg
>>>>> +#usr/lib/pmacct/examples/lg/pmbgp
>>>>> +usr/sbin/nfacctd
>>>>> +usr/sbin/pmacctd
>>>>> +usr/sbin/pmbgpd
>>>>> +usr/sbin/pmbmpd
>>>>> +usr/sbin/pmtelemetryd
>>>>> +usr/sbin/sfacctd
>>>>> +usr/share/pmacct
>>>>> +#usr/share/pmacct/CONFIG-KEYS
>>>>> +#usr/share/pmacct/FAQS
>>>>> +#usr/share/pmacct/QUICKSTART
>>>>> +#usr/share/pmacct/UPGRADE
>>>>> +#usr/share/pmacct/docs
>>>>> +#usr/share/pmacct/docs/IPFIX
>>>>> +#usr/share/pmacct/docs/LOOKING_GLASS_FORMAT
>>>>> +#usr/share/pmacct/docs/MSGLOG_DUMP_FORMATS
>>>>> +#usr/share/pmacct/docs/SIGNALS
>>>>> +#usr/share/pmacct/docs/TRIGGER_VARS
>>>>> +#usr/share/pmacct/examples
>>>>> +usr/share/pmacct/examples/allow.lst.example
>>>>> +#usr/share/pmacct/examples/amqp
>>>>> +usr/share/pmacct/examples/amqp/amqp_receiver.py
>>>>> +#usr/share/pmacct/examples/avro
>>>>> +usr/share/pmacct/examples/avro/avro_file_decoder.py
>>>>> +usr/share/pmacct/examples/bgp_agent.map.example
>>>>> +usr/share/pmacct/examples/bgp_md5.lst.example
>>>>> +usr/share/pmacct/examples/bgp_xconnects.map.example
>>>>> +usr/share/pmacct/examples/flow_to_rd.map.example
>>>>> +#usr/share/pmacct/examples/kafka
>>>>> +usr/share/pmacct/examples/kafka/kafka_consumer.py
>>>>> +#usr/share/pmacct/examples/lg
>>>>> +usr/share/pmacct/examples/lg/pmbgp.py
>>>>> +usr/share/pmacct/examples/networks.lst.example
>>>>> +usr/share/pmacct/examples/nfacctd-sql.conf.example
>>>>> +usr/share/pmacct/examples/pcap_interfaces.map.example
>>>>> +usr/share/pmacct/examples/peers.map.example
>>>>> +usr/share/pmacct/examples/pmacctd-imt.conf.example
>>>>> +usr/share/pmacct/examples/pmacctd-multiple-plugins.conf.example
>>>>> +usr/share/pmacct/examples/pmacctd-sql.conf.example
>>>>> +usr/share/pmacct/examples/pmacctd-sqlite3.conf.example
>>>>> +usr/share/pmacct/examples/ports.lst.example
>>>>> +usr/share/pmacct/examples/pretag.map.example
>>>>> +usr/share/pmacct/examples/primitives.lst.example
>>>>> +usr/share/pmacct/examples/probe_netflow.conf.example
>>>>> +usr/share/pmacct/examples/probe_sflow.conf.example
>>>>> +usr/share/pmacct/examples/sampling.map.example
>>>>> +usr/share/pmacct/examples/tee_receivers.lst.example
>>>>> +#usr/share/pmacct/sql
>>>>> +#usr/share/pmacct/sql/README.64bit
>>>>> +#usr/share/pmacct/sql/README.GeoIP
>>>>> +#usr/share/pmacct/sql/README.IPv6
>>>>> +#usr/share/pmacct/sql/README.cos
>>>>> +#usr/share/pmacct/sql/README.custom_primitives
>>>>> +#usr/share/pmacct/sql/README.etype
>>>>> +#usr/share/pmacct/sql/README.export_proto
>>>>> +#usr/share/pmacct/sql/README.iface
>>>>> +#usr/share/pmacct/sql/README.label
>>>>> +#usr/share/pmacct/sql/README.mask
>>>>> +#usr/share/pmacct/sql/README.mpls
>>>>> +#usr/share/pmacct/sql/README.mysql
>>>>> +#usr/share/pmacct/sql/README.nat
>>>>> +#usr/share/pmacct/sql/README.pgsql
>>>>> +#usr/share/pmacct/sql/README.sampling
>>>>> +usr/share/pmacct/sql/README.sqlite3
>>>>> +#usr/share/pmacct/sql/README.tag2
>>>>> +#usr/share/pmacct/sql/README.timestamp
>>>>> +#usr/share/pmacct/sql/README.tunnel
>>>>> +#usr/share/pmacct/sql/pmacct-create-db.pgsql
>>>>> +#usr/share/pmacct/sql/pmacct-create-db_bgp_v1.mysql
>>>>> +#usr/share/pmacct/sql/pmacct-create-db_v1.mysql
>>>>> +#usr/share/pmacct/sql/pmacct-create-db_v2.mysql
>>>>> +#usr/share/pmacct/sql/pmacct-create-db_v3.mysql
>>>>> +#usr/share/pmacct/sql/pmacct-create-db_v4.mysql
>>>>> +#usr/share/pmacct/sql/pmacct-create-db_v5.mysql
>>>>> +#usr/share/pmacct/sql/pmacct-create-db_v6.mysql
>>>>> +#usr/share/pmacct/sql/pmacct-create-db_v7.mysql
>>>>> +#usr/share/pmacct/sql/pmacct-create-db_v8.mysql
>>>>> +#usr/share/pmacct/sql/pmacct-create-db_v9.mysql
>>>>> +#usr/share/pmacct/sql/pmacct-create-table_bgp_v1.pgsql
>>>>> +usr/share/pmacct/sql/pmacct-create-table_bgp_v1.sqlite3
>>>>> +#usr/share/pmacct/sql/pmacct-create-table_v1.pgsql
>>>>> +usr/share/pmacct/sql/pmacct-create-table_v1.sqlite3
>>>>> +#usr/share/pmacct/sql/pmacct-create-table_v2.pgsql
>>>>> +usr/share/pmacct/sql/pmacct-create-table_v2.sqlite3
>>>>> +#usr/share/pmacct/sql/pmacct-create-table_v3.pgsql
>>>>> +usr/share/pmacct/sql/pmacct-create-table_v3.sqlite3
>>>>> +#usr/share/pmacct/sql/pmacct-create-table_v4.pgsql
>>>>> +usr/share/pmacct/sql/pmacct-create-table_v4.sqlite3
>>>>> +#usr/share/pmacct/sql/pmacct-create-table_v5.pgsql
>>>>> +usr/share/pmacct/sql/pmacct-create-table_v5.sqlite3
>>>>> +#usr/share/pmacct/sql/pmacct-create-table_v6.pgsql
>>>>> +usr/share/pmacct/sql/pmacct-create-table_v6.sqlite3
>>>>> +usr/share/pmacct/sql/pmacct-create-table_v7.sqlite3
>>>>> +#usr/share/pmacct/sql/pmacct-create-table_v7_v8.pgsql
>>>>> +usr/share/pmacct/sql/pmacct-create-table_v8.sqlite3
>>>>> +#usr/share/pmacct/sql/pmacct-create-table_v9.pgsql
>>>>> +usr/share/pmacct/sql/pmacct-create-table_v9.sqlite3
>>>>> +#usr/share/pmacct/sql/pmacct-grant-db.mysql
>>>>> +var/ipfire/backup/addons/includes/pmacct
>>>>> +var/spool/pmacct
>>>>> diff --git a/lfs/pmacct b/lfs/pmacct
>>>>> new file mode 100644
>>>>> index 000000000..c6cc5270e
>>>>> --- /dev/null
>>>>> +++ b/lfs/pmacct
>>>>> @@ -0,0 +1,107 @@
>>>>> +###############################################################################
>>>>>
>>>>> +# #
>>>>> +# IPFire.org - A linux based
>>>>> firewall #
>>>>> +# Copyright (C) 2019 IPFire Team
>>>>> <info(a)ipfire.org> #
>>>>> +# #
>>>>> +# This program is free software: you can redistribute it and/or
>>>>> modify #
>>>>> +# it under the terms of the GNU General Public License as
>>>>> published by #
>>>>> +# the Free Software Foundation, either version 3 of the License,
>>>>> or #
>>>>> +# (at your option) any later
>>>>> version. #
>>>>> +# #
>>>>> +# This program is distributed in the hope that it will be
>>>>> useful, #
>>>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty
>>>>> of #
>>>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
>>>>> the #
>>>>> +# GNU General Public License for more
>>>>> details. #
>>>>> +# #
>>>>> +# You should have received a copy of the GNU General Public
>>>>> License #
>>>>> +# along with this program. If not, see
>>>>> <http://www.gnu.org/licenses/>. #
>>>>> +# #
>>>>> +###############################################################################
>>>>>
>>>>> +
>>>>> +###############################################################################
>>>>>
>>>>> +# Definitions
>>>>> +###############################################################################
>>>>>
>>>>> +
>>>>> +include Config
>>>>> +
>>>>> +VER = 1.7.6
>>>>> +
>>>>> +THISAPP = pmacct-$(VER)
>>>>> +DL_FILE = $(THISAPP).tar.gz
>>>>> +DL_FROM = $(URL_IPFIRE)
>>>>> +DIR_APP = $(DIR_SRC)/$(THISAPP)
>>>>> +TARGET = $(DIR_INFO)/$(THISAPP)
>>>>> +PROG = pmacct
>>>>> +PAK_VER = 1
>>>>> +
>>>>> +DEPS = libcdada
>>>>> +
>>>>> +###############################################################################
>>>>>
>>>>> +# Top-level Rules
>>>>> +###############################################################################
>>>>>
>>>>> +
>>>>> +objects = $(DL_FILE)
>>>>> +
>>>>> +$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
>>>>> +
>>>>> +$(DL_FILE)_MD5 = b829cfdc394d5acd6d32fb23de181a6b
>>>>> +
>>>>> +install : $(TARGET)
>>>>> +
>>>>> +check : $(patsubst %,$(DIR_CHK)/%,$(objects))
>>>>> +
>>>>> +download :$(patsubst %,$(DIR_DL)/%,$(objects))
>>>>> +
>>>>> +md5 : $(subst %,%_MD5,$(objects))
>>>>> +
>>>>> +dist:
>>>>> + @$(PAK)
>>>>> +
>>>>> +###############################################################################
>>>>>
>>>>> +# Downloading, checking, md5sum
>>>>> +###############################################################################
>>>>>
>>>>> +
>>>>> +$(patsubst %,$(DIR_CHK)/%,$(objects)) :
>>>>> + @$(CHECK)
>>>>> +
>>>>> +$(patsubst %,$(DIR_DL)/%,$(objects)) :
>>>>> + @$(LOAD)
>>>>> +
>>>>> +$(subst %,%_MD5,$(objects)) :
>>>>> + @$(MD5)
>>>>> +
>>>>> +###############################################################################
>>>>>
>>>>> +# Installation Details
>>>>> +###############################################################################
>>>>>
>>>>> +
>>>>> +$(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
>>>>> + @$(PREBUILD)
>>>>> + @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar zxf
>>>>> $(DIR_DL)/$(DL_FILE)
>>>>> + cd $(DIR_APP) && ./autogen.sh
>>>>> + cd $(DIR_APP) && ./configure \
>>>>> + --prefix=/usr \
>>>>> + --sysconfdir=/etc/pmacct \
>>>>> + --enable-sqlite3 \
>>>>> + --enable-l2 \
>>>>> + --enable-jansson
>>>>> +
>>>>> + # Add directory for plugins and pipes
>>>>> + -mkdir -v /var/spool/pmacct
>>>>> +
>>>>> + # Add configuration folder and copy conf files to it
>>>>> + -mkdir -v /etc/pmacct
>>>>> + cp -vf $(DIR_CONF)/pmacct/*.conf /etc/pmacct
>>>>> +
>>>>> + # Copy initscript to dir
>>>>> + cp -vf $(DIR_CONF)/pmacct/pmacct.init /etc/rc.d/init.d/pmacct
>>>>> + chmod 754 /etc/rc.d/init.d/pmacct
>>>>> + chown root:root /etc/rc.d/init.d/pmacct
>>>>> +
>>>>> + cd $(DIR_APP) && make $(MAKETUNING)
>>>>> + cd $(DIR_APP) && make install
>>>>> +
>>>>> + # Install backup definition
>>>>> + install -v -m 644 ${DIR_SRC}/config/backup/includes/pmacct \
>>>>> + /var/ipfire/backup/addons/includes/pmacct
>>>>> + @rm -rf $(DIR_APP)
>>>>> + @$(POSTBUILD)
>>>>> diff --git a/make.sh b/make.sh
>>>>> index 43cccaca0..d42c9b1e2 100755
>>>>> --- a/make.sh
>>>>> +++ b/make.sh
>>>>> @@ -1646,6 +1646,8 @@ buildipfire() {
>>>>> lfsmake2 ncdu
>>>>> lfsmake2 lshw
>>>>> lfsmake2 socat
>>>>> + lfsmake2 libcdada
>>>>> + lfsmake2 pmacct
>>>>> }
>>>>>
>>>>> buildinstaller() {
>>>>> diff --git a/src/paks/pmacct/install.sh b/src/paks/pmacct/install.sh
>>>>> new file mode 100644
>>>>> index 000000000..8178cc7bc
>>>>> --- /dev/null
>>>>> +++ b/src/paks/pmacct/install.sh
>>>>> @@ -0,0 +1,36 @@
>>>>> +#!/bin/bash
>>>>> +############################################################################
>>>>>
>>>>> +# #
>>>>> +# This file is part of the IPFire
>>>>> Firewall. #
>>>>> +# #
>>>>> +# IPFire is free software; you can redistribute it and/or
>>>>> modify #
>>>>> +# it under the terms of the GNU General Public License as
>>>>> published by #
>>>>> +# the Free Software Foundation; either version 2 of the License,
>>>>> or #
>>>>> +# (at your option) any later
>>>>> version. #
>>>>> +# #
>>>>> +# IPFire is distributed in the hope that it will be
>>>>> useful, #
>>>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty
>>>>> of #
>>>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
>>>>> the #
>>>>> +# GNU General Public License for more
>>>>> details. #
>>>>> +# #
>>>>> +# You should have received a copy of the GNU General Public
>>>>> License #
>>>>> +# along with IPFire; if not, write to the Free
>>>>> Software #
>>>>> +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
>>>>> 02111-1307 USA #
>>>>> +# #
>>>>> +# Copyright (C) 2007 IPFire-Team
>>>>> <info(a)ipfire.org>. #
>>>>> +# #
>>>>> +############################################################################
>>>>>
>>>>> +#
>>>>> +. /opt/pakfire/lib/functions.sh
>>>>> +
>>>>> +extract_files
>>>>> +restore_backup ${NAME}
>>>>> +
>>>>> +# Add symlinks for runlevels
>>>>> +ln -s ../init.d/${NAME} /etc/rc.d/rc0.d/K85${NAME}
>>>>> +ln -s ../init.d/${NAME} /etc/rc.d/rc3.d/S50${NAME}
>>>>> +ln -s ../init.d/${NAME} /etc/rc.d/rc6.d/K85${NAME}
>>>>> +start_service --background ${NAME}
>>>>> +
>>>>> +
>>>>> +# EOF
>>>>> diff --git a/src/paks/pmacct/uninstall.sh
>>>>> b/src/paks/pmacct/uninstall.sh
>>>>> new file mode 100644
>>>>> index 000000000..51631e8c1
>>>>> --- /dev/null
>>>>> +++ b/src/paks/pmacct/uninstall.sh
>>>>> @@ -0,0 +1,33 @@
>>>>> +#!/bin/bash
>>>>> +############################################################################
>>>>>
>>>>> +# #
>>>>> +# This file is part of the IPFire
>>>>> Firewall. #
>>>>> +# #
>>>>> +# IPFire is free software; you can redistribute it and/or
>>>>> modify #
>>>>> +# it under the terms of the GNU General Public License as
>>>>> published by #
>>>>> +# the Free Software Foundation; either version 2 of the License,
>>>>> or #
>>>>> +# (at your option) any later
>>>>> version. #
>>>>> +# #
>>>>> +# IPFire is distributed in the hope that it will be
>>>>> useful, #
>>>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty
>>>>> of #
>>>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
>>>>> the #
>>>>> +# GNU General Public License for more
>>>>> details. #
>>>>> +# #
>>>>> +# You should have received a copy of the GNU General Public
>>>>> License #
>>>>> +# along with IPFire; if not, write to the Free
>>>>> Software #
>>>>> +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
>>>>> 02111-1307 USA #
>>>>> +# #
>>>>> +# Copyright (C) 2007 IPFire-Team
>>>>> <info(a)ipfire.org>. #
>>>>> +# #
>>>>> +############################################################################
>>>>>
>>>>> +#
>>>>> +. /opt/pakfire/lib/functions.sh
>>>>> +
>>>>> +stop_service ${NAME}
>>>>> +make_backup ${NAME}
>>>>> +remove_files
>>>>> +
>>>>> +# Delete symlinks
>>>>> +rm -rfv /etc/rc.d/rc?.d/???${NAME}
>>>>> +
>>>>> +# EOF
>>>>> diff --git a/src/paks/pmacct/update.sh b/src/paks/pmacct/update.sh
>>>>> new file mode 100644
>>>>> index 000000000..89c40d0d7
>>>>> --- /dev/null
>>>>> +++ b/src/paks/pmacct/update.sh
>>>>> @@ -0,0 +1,26 @@
>>>>> +#!/bin/bash
>>>>> +############################################################################
>>>>>
>>>>> +# #
>>>>> +# This file is part of the IPFire
>>>>> Firewall. #
>>>>> +# #
>>>>> +# IPFire is free software; you can redistribute it and/or
>>>>> modify #
>>>>> +# it under the terms of the GNU General Public License as
>>>>> published by #
>>>>> +# the Free Software Foundation; either version 2 of the License,
>>>>> or #
>>>>> +# (at your option) any later
>>>>> version. #
>>>>> +# #
>>>>> +# IPFire is distributed in the hope that it will be
>>>>> useful, #
>>>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty
>>>>> of #
>>>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
>>>>> the #
>>>>> +# GNU General Public License for more
>>>>> details. #
>>>>> +# #
>>>>> +# You should have received a copy of the GNU General Public
>>>>> License #
>>>>> +# along with IPFire; if not, write to the Free
>>>>> Software #
>>>>> +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
>>>>> 02111-1307 USA #
>>>>> +# #
>>>>> +# Copyright (C) 2007 IPFire-Team
>>>>> <info(a)ipfire.org>. #
>>>>> +# #
>>>>> +############################################################################
>>>>>
>>>>> +#
>>>>> +. /opt/pakfire/lib/functions.sh
>>>>> +./uninstall.sh
>>>>> +./install.sh
>>>>> --
>>>>> 2.30.1
>>
--
Sent from my laptop
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/2] pmacct: New addon
2021-03-19 9:20 ` Adolf Belka
@ 2021-03-19 17:19 ` ummeegge
0 siblings, 0 replies; 7+ messages in thread
From: ummeegge @ 2021-03-19 17:19 UTC (permalink / raw)
To: development
[-- Attachment #1: Type: text/plain, Size: 39454 bytes --]
Hi all,
Am Freitag, dem 19.03.2021 um 10:20 +0100 schrieb Adolf Belka:
> Hi Michael,
>
> On 18/03/2021 19:43, Adolf Belka wrote:
> > Hi Michael,
> >
> > On 18/03/2021 19:25, Michael Tremer wrote:
> > > Hey,
> > >
> > > > On 18 Mar 2021, at 13:34, Adolf Belka <adolf.belka(a)ipfire.org>
> > > > wrote:
> > > >
> > > > Hi Michael,
> > > >
> > > > On 18/03/2021 14:07, Michael Tremer wrote:
> > > > > Hello,
> > > > > I will split this email into two parts:
> > > > > The technical part:
> > > > > * The initscript isn’t based on our template and uses sleep
> > > > > to
> > > > > check if starting a daemon has been successful. That should
> > > > > not be
> > > > > necessary and will potentially slow down the boot process.
> > > > I will have another look at the initscript and see if it can
> > > > use the
> > > > standard IPFire template.
> > >
> > > Yes, it is not too different from your version:
> > >
> > > https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=src/initscripts/system/template;h=69dc2a6a2202fb30da61a7fff6625aea531802e9;hb=HEAD
> > >
> > > There isn’t much documentation around it but there are loads of
> > > other
> > > scripts that will be helpful to figure out any convention or
> > > boilerplate code to copy.
> > >
> > > > > * I am not very sure if the example configuration files are
> > > > > helpful. Maybe someone can clear this up a little bit for me
> > > > > what
> > > > > is the rationale is behind them.
> > > > The idea was that the two simplest config example scripts were
> > > > provided that give a user the ability to get data into memory
> > > > or
> > > > into a sqlite file. The actual conf file provided is a copy of
> > > > the
> > > > memory version. These give a basic starting point that people
> > > > can
> > > > then add additional info to be recorded etc.
> > >
> > > Does anybody want to use anything else than SQLite in the context
> > > of
> > > IPFire?
> > I think we weren't sure and the memory and sqlite options were the
> > easiest to get going and give real data. There are also mysql and
> > postgresql options but I think we need to see what the interest is.
there would be needed the enable switch in configure to use pgsql or
mysql which is not enabled in this LFS.
The memory table is cheaper according to system resources and is able
to feed also 3rd party solutions like e.g. RRD.
> > >
> > > > > * There are loads of daemons being shipped and only one is
> > > > > being used.
> > > > >
> > > > > I looked through our conversations and it looks like these
> > > > > were
> > > > > left there because we weren't sure if they would be used or
> > > > > not.
> > > > >
> > > > > pmacctd is the daemon being used by Jon. pmacct is the
> > > > > frontend to
> > > > > pmacctd when being used in memory mode.
> > > > >
> > > > > For the others I looked in the pmacct website and found the
> > > > > following:-
> > > > >
> I looked through our conversations and it looks like these were left
> there because we weren't sure if they would be used or not.
>
> pmacctd is the daemon being used by Jon. pmacct is the frontend to
> pmacctd when being used in memory mode.
Exactly
>
> For the others I looked in the pmacct website and found the
> following:-
>
> 'pmacctd' is the libpcap-based IPv4/IPv6 accounting daemon
>
> 'nfacctd' is the NetFlow (versions supported NetFlow v1, v5 and v9)
> and
> IPFIX accounting daemon
>
> 'sfacctd' is the sFlow v2/v4 and v5 accounting daemon
>
> 'uacctd' is the Linux NFLOG-based accounting daemon (historically, it
> was using ULOG, hence its name)
>
> 'pmtelemetryd' is the Streaming Telemetry collector, where "Streaming
> telemetry [ .. ] data can be used for analysis and troubleshooting
> purposes to maintain the health of the network. This is achieved by
> leveraging the capabilities of machine-to-machine communication. [ ..
> ]"
> (quoting Cisco IOS-XR Telemetry Configuration Guide at the time of
> this
> writing)
>
> 'pmbgpd' is the pmacct BGP collector and 'pmbmpd'is the pmacct BMP
> collector.
Pmacct can also handle BGP data via nfacct and the *bgp* binaries.
>
>
> I don't know what most of that means but it looks to be quite
> different
> to what the addon was created for, so I would propose that I re-do
> the
> rootfiles and remove those daemons from the install. If in future
> there
> is demand for others we can add them back in with an update patch. I
> will do that together with the other changes already discussed.
>
As a beside info.
Best,
Erik
> Regards,
>
> Adolf.
>
> > > > I think Jon or Erik need to comment on this question.
> > > > > * How much load does this generate? How large is the captured
> > > > > data?
> > > > Jon needs to provide this feedback.
> > > > > Then there is the other part:
> > > > > > On 8 Mar 2021, at 11:57, Adolf Belka (ipfire)
> > > > > > <adolf.belka(a)ipfire.org> wrote:
> > > > > >
> > > > > > From: Adolf Belka <adolf.belka(a)ipfire.org>
> > > > > >
> > > > > > - What is it?
> > > > > > pmacct is a monitoring tool for network management
> > > > > > tasks. Data
> > > > > > collected
> > > > > > can be used for analysis and troubleshooting purposes to
> > > > > > maintain the
> > > > > > health of the network. pmacct can collect, replicate and
> > > > > > export
> > > > > > network
> > > > > > information. It can cache in memory tables, store
> > > > > > persistently
> > > > > > to SQLite3
> > > > > > and output to flat-files like CSV, formatted, and JSON.
> > > > > Okay.
> > > > > > - Why is it needed?
> > > > > > To monitor data usage (IP-based or MAC-based data
> > > > > > accounting)
> > > > > > down to the
> > > > > > client level. Net-Traffic will monitor traffic for the
> > > > > > entire
> > > > > > RED, GREEN,
> > > > > > etc. networks, but it cannot pinpoint which client is
> > > > > > using
> > > > > > lots of data.
> > > > > > Connections will take a snapshot but not show day by day
> > > > > > sums.
> > > > > > pmacct can
> > > > > > help admins keep tabs on users that use too much data.
> > > > > I agree with the goal of it. But how does this add-on achieve
> > > > > that?
> > > > Jon should comment on this as he had this use case and can give
> > > > feedback how the data has helped him in his investigations.
> > > > > The data is being written to a (usually binary) file which
> > > > > cannot
> > > > > be visualised in the web UI. Does this not want a CGI script
> > > > > that
> > > > > allows people to see what is happening?
> > > > We all thought that this would ultimately need a CGI script to
> > > > set
> > > > up and visualise the data. We believed it was worth getting
> > > > this
> > > > running in actual use as a CLI tool to start with and then get
> > > > more
> > > > feedback on the best items that would need to be captured in
> > > > the CGI
> > > > script. The tool has a lot of capabilities and I think it will
> > > > not
> > > > be possible to do everything that it can do in a WUI page. So
> > > > some
> > > > form of file or db offload will need to be available anyway but
> > > > the
> > > > items identified as being commonly required would be the ones
> > > > to be
> > > > developed into a CGI WUI page.
> > > > > Sending manual SQL queries probably isn’t much fun :)
> > > > > > - What are the use cases?
> > > > > > An ISP may implement data caps and if the limit is over-
> > > > > > run
> > > > > > then you have
> > > > > > to pay for every additional xxGB of data used. Typical
> > > > > > charges
> > > > > > can be
> > > > > > around $10 per 50GB. With pmacct you can identify the
> > > > > > high
> > > > > > users and take
> > > > > > action, hopefully before the limit is breached.
> > > > > > - This is being introduced as a command line only tool.
> > > > > > However,
> > > > > > at a later
> > > > > > date, if it is useful to enough additional users a WUI
> > > > > > page
> > > > > > could be
> > > > > > developed as discussed in the development mailing list
> > > > > >
> > > > > > https://lists.ipfire.org/pipermail/development/2021-January/009174.html
> > > > > >
> > > > > >
> > > > > I have been thinking about this for a while now and would say
> > > > > that
> > > > > I can approve merging this as a CLI tool. But it would not
> > > > > really
> > > > > enhance IPFire’s functionality because it is only usable for
> > > > > people
> > > > > who spend a lot of time writing their own scripts or SQL
> > > > > queries to
> > > > > get information that can be used to make decisions.
> > > > We recognise that and had discussed the CGI WUI page as
> > > > mentioned
> > > > above.
> > >
> > > Was this here on the list or elsewhere?
> > This was in a private conversation on the IPFire Community forum.
> > We
> > thought it best to have all our back and forth discussions between
> > ourselves to start with. We were all finding our feet with it as
> > well
> > and we thought it best not to swamp the development list with our
> > musings and questions to each other. If you want to see all the
> > discussion history, then we can add you to the members of that
> > conversation.
> > If the addon gets accepted then any future discussions would be
> > done
> > on the Development mailing list to get everyones input.
> > >
> > > > > Adding another alternative to vnstat will result in us having
> > > > > to
> > > > > maintain two things that roughly do the same - or where one’s
> > > > > features are a subset of the other one.
> > > > I think Jon or Erik need to comment on any comparison with
> > > > vnstat.
> > > > > I would like to avoid merging this and then see it abandoned
> > > > > because it is too complicated to use. We already have a lot
> > > > > of
> > > > > virtually unmaintained stuff around and I would really like
> > > > > to drop
> > > > > a lot of it, although I know that there are a few people
> > > > > actually
> > > > > using it.
> > > > Jon and Erik should comment on the complication of use or not.
> > > > Jon
> > > > especially has done quite a bit of work with it, installing the
> > > > .ipfire package versions that I have created.
> > > > I intend to also have a play around with it but for me it is
> > > > more
> > > > some playful fun as I just have a home network with only myself
> > > > as
> > > > the user and admin.
> > > > > Thoughts?
> > > > > Best,
> > > > > -Michael
> > > > > > Co-authored-by: Jon Murphy
> > > > > > Co-authored-by: Erik Kapfer <ummeegge(a)ipfire.org>
> > > > > >
> > > > > > Signed-off-by: Adolf Belka <adolf.belka(a)ipfire.org>
> > > > > > ---
> > > > > > config/backup/includes/pmacct | 2 +
> > > > > > config/pmacct/pmacct.conf | 24 +++++
> > > > > > config/pmacct/pmacct.init | 57 +++++++++++
> > > > > > config/pmacct/pmacct_memory_example.conf | 24 +++++
> > > > > > config/pmacct/pmacct_sqlite3_example.conf | 25 +++++
> > > > > > config/rootfiles/packages/pmacct | 111
> > > > > > ++++++++++++++++++++++
> > > > > > lfs/pmacct | 107
> > > > > > +++++++++++++++++++++
> > > > > > make.sh | 2 +
> > > > > > src/paks/pmacct/install.sh | 36 +++++++
> > > > > > src/paks/pmacct/uninstall.sh | 33 +++++++
> > > > > > src/paks/pmacct/update.sh | 26 +++++
> > > > > > 11 files changed, 447 insertions(+)
> > > > > > create mode 100644 config/backup/includes/pmacct
> > > > > > create mode 100644 config/pmacct/pmacct.conf
> > > > > > create mode 100644 config/pmacct/pmacct.init
> > > > > > create mode 100644 config/pmacct/pmacct_memory_example.conf
> > > > > > create mode 100644
> > > > > > config/pmacct/pmacct_sqlite3_example.conf
> > > > > > create mode 100644 config/rootfiles/packages/pmacct
> > > > > > create mode 100644 lfs/pmacct
> > > > > > create mode 100644 src/paks/pmacct/install.sh
> > > > > > create mode 100644 src/paks/pmacct/uninstall.sh
> > > > > > create mode 100644 src/paks/pmacct/update.sh
> > > > > >
> > > > > > diff --git a/config/backup/includes/pmacct
> > > > > > b/config/backup/includes/pmacct
> > > > > > new file mode 100644
> > > > > > index 000000000..4359067ad
> > > > > > --- /dev/null
> > > > > > +++ b/config/backup/includes/pmacct
> > > > > > @@ -0,0 +1,2 @@
> > > > > > +/etc/pmacct/
> > > > > > +/var/spool/pmacct/
> > > > > > diff --git a/config/pmacct/pmacct.conf
> > > > > > b/config/pmacct/pmacct.conf
> > > > > > new file mode 100644
> > > > > > index 000000000..3c1d47efa
> > > > > > --- /dev/null
> > > > > > +++ b/config/pmacct/pmacct.conf
> > > > > > @@ -0,0 +1,24 @@
> > > > > > +!
> > > > > > +! Pmacctd configuration file for IPFire environment
> > > > > > +!
> > > > > > +
> > > > > > +syslog: daemon
> > > > > > +daemonize: true
> > > > > > +debug: false
> > > > > > +promisc: true
> > > > > > +interface: green0
> > > > > > +
> > > > > > +
> > > > > > +!
> > > > > > +! "plugin1" plugin configuration
> > > > > > +!
> > > > > > +plugins: memory[plugin1]
> > > > > > +
> > > > > > +plugin_buffer_size[plugin1]: 102400
> > > > > > +plugin_pipe_size[plugin1]: 10240000
> > > > > > +
> > > > > > +imt_mem_pools_number: 256
> > > > > > +imt_path[plugin1]: /var/spool/pmacct/plugin1.pipe
> > > > > > +
> > > > > > +aggregate[plugin1]: src_host, src_port, src_mac, dst_host,
> > > > > > dst_port, dst_mac, proto
> > > > > > +aggregate_filter[plugin1]: ip
> > > > > > \ No newline at end of file
> > > > > > diff --git a/config/pmacct/pmacct.init
> > > > > > b/config/pmacct/pmacct.init
> > > > > > new file mode 100644
> > > > > > index 000000000..31f9ce49b
> > > > > > --- /dev/null
> > > > > > +++ b/config/pmacct/pmacct.init
> > > > > > @@ -0,0 +1,57 @@
> > > > > > +#!/bin/bash
> > > > > > +
> > > > > > +# Begin $rc_base/init.d/pmacct
> > > > > > +
> > > > > > +# Starts and stops pmacct daemon
> > > > > > +# Date: 2021-02-25 12:25:30 (Thu, 25 Feb 2021)
> > > > > > +
> > > > > > +# Locations
> > > > > > +CONF="/etc/pmacct/pmacct.conf"
> > > > > > +CONTENT="/var/pmacct"
> > > > > > +# Pmacctd binary
> > > > > > +PMACCT="/usr/sbin/pmacctd"
> > > > > > +# Pmacctd start options '-f' for the configuration file
> > > > > > +OPT="-f ${CONF}"
> > > > > > +DESC="pmacct daemon"
> > > > > > +# Pmacctd syslog facilitiy
> > > > > > +#SYSLOG="local1"
> > > > > > +
> > > > > > +
> > > > > > +. /etc/sysconfig/rc
> > > > > > +. $rc_functions
> > > > > > +
> > > > > > +case "$1" in
> > > > > > + start)
> > > > > > + boot_mesg "Starting the ${DESC}... "
> > > > > > + loadproc ${PMACCT} ${OPT}
> > > > > > + sleep 2
> > > > > > + if P=$(pgrep -fl pmacctd); then
> > > > > > + echo -e "\n${P}"
> > > > > > + else
> > > > > > + echo -e "\n${PMACCT} is not running... "
> > > > > > + fi
> > > > > > + ;;
> > > > > > +
> > > > > > + stop)
> > > > > > + boot_mesg "Stopping the ${DESC}... "
> > > > > > + killproc ${PMACCT}
> > > > > > + ;;
> > > > > > +
> > > > > > + restart)
> > > > > > + $0 stop
> > > > > > + sleep 1
> > > > > > + $0 start
> > > > > > + ;;
> > > > > > +
> > > > > > + status)
> > > > > > + statusproc ${PMACCT}
> > > > > > + ;;
> > > > > > +
> > > > > > + *)
> > > > > > + echo "Usage: $0 {start|stop|restart|status}"
> > > > > > + exit 1
> > > > > > + ;;
> > > > > > +esac
> > > > > > +
> > > > > > +# End $rc_base/init.d/pmacct
> > > > > > +
> > > > > > diff --git a/config/pmacct/pmacct_memory_example.conf
> > > > > > b/config/pmacct/pmacct_memory_example.conf
> > > > > > new file mode 100644
> > > > > > index 000000000..0babdaaf7
> > > > > > --- /dev/null
> > > > > > +++ b/config/pmacct/pmacct_memory_example.conf
> > > > > > @@ -0,0 +1,24 @@
> > > > > > +!
> > > > > > +! Pmacctd configuration file for IPFire environment
> > > > > > +!
> > > > > > +
> > > > > > +syslog: daemon
> > > > > > +daemonize: true
> > > > > > +debug: false
> > > > > > +promisc: true
> > > > > > +interface: green0
> > > > > > +
> > > > > > +
> > > > > > +!
> > > > > > +! "plugin1" plugin configuration
> > > > > > +!
> > > > > > +plugins: memory[plugin1]
> > > > > > +
> > > > > > +plugin_buffer_size[plugin1]: 102400
> > > > > > +plugin_pipe_size[plugin1]: 10240000
> > > > > > +
> > > > > > +imt_mem_pools_number: 256
> > > > > > +imt_path[plugin1]: /var/spool/pmacct/plugin1.pipe
> > > > > > +
> > > > > > +aggregate[plugin1]: src_host, src_port, src_mac, dst_host,
> > > > > > dst_port, dst_mac, proto
> > > > > > +aggregate_filter[plugin1]: ip
> > > > > > diff --git a/config/pmacct/pmacct_sqlite3_example.conf
> > > > > > b/config/pmacct/pmacct_sqlite3_example.conf
> > > > > > new file mode 100644
> > > > > > index 000000000..7fb996408
> > > > > > --- /dev/null
> > > > > > +++ b/config/pmacct/pmacct_sqlite3_example.conf
> > > > > > @@ -0,0 +1,25 @@
> > > > > > +!
> > > > > > +! Pmacctd configuration file for IPFire environment
> > > > > > +!
> > > > > > +
> > > > > > +syslog: daemon
> > > > > > +daemonize: true
> > > > > > +debug: false
> > > > > > +promisc: true
> > > > > > +interface: green0
> > > > > > +
> > > > > > +
> > > > > > +!
> > > > > > +! "plugin2" plugin configuration
> > > > > > +!
> > > > > > +plugins: sqlite3[plugin2]
> > > > > > +
> > > > > > +sql_db[plugin2]: /var/spool/pmacct/pmacct_sqlitev1.db
> > > > > > +sql_table_version[plugin2]: 1
> > > > > > +sql_history[plugin2]: 5m
> > > > > > +sql_refresh_time[plugin2]: 300
> > > > > > +sql_history_roundoff[plugin2]: m
> > > > > > +sql_dont_try_update: true
> > > > > > +
> > > > > > +aggregate[plugin2]: src_host, src_port, src_mac, dst_host,
> > > > > > dst_port, dst_mac, proto
> > > > > > +aggregate_filter[plugin2]: ip
> > > > > > diff --git a/config/rootfiles/packages/pmacct
> > > > > > b/config/rootfiles/packages/pmacct
> > > > > > new file mode 100644
> > > > > > index 000000000..fed83ee67
> > > > > > --- /dev/null
> > > > > > +++ b/config/rootfiles/packages/pmacct
> > > > > > @@ -0,0 +1,111 @@
> > > > > > +etc/pmacct
> > > > > > +etc/pmacct/pmacct.conf
> > > > > > +etc/pmacct/pmacct_memory_example.conf
> > > > > > +etc/pmacct/pmacct_sqlite3_example.conf
> > > > > > +etc/rc.d/init.d/pmacct
> > > > > > +usr/bin/pmacct
> > > > > > +#usr/lib/pmacct
> > > > > > +#usr/lib/pmacct/examples
> > > > > > +#usr/lib/pmacct/examples/custom
> > > > > > +#usr/lib/pmacct/examples/custom/libcustom.la
> > > > > > +#usr/lib/pmacct/examples/lg
> > > > > > +#usr/lib/pmacct/examples/lg/pmbgp
> > > > > > +usr/sbin/nfacctd
> > > > > > +usr/sbin/pmacctd
> > > > > > +usr/sbin/pmbgpd
> > > > > > +usr/sbin/pmbmpd
> > > > > > +usr/sbin/pmtelemetryd
> > > > > > +usr/sbin/sfacctd
> > > > > > +usr/share/pmacct
> > > > > > +#usr/share/pmacct/CONFIG-KEYS
> > > > > > +#usr/share/pmacct/FAQS
> > > > > > +#usr/share/pmacct/QUICKSTART
> > > > > > +#usr/share/pmacct/UPGRADE
> > > > > > +#usr/share/pmacct/docs
> > > > > > +#usr/share/pmacct/docs/IPFIX
> > > > > > +#usr/share/pmacct/docs/LOOKING_GLASS_FORMAT
> > > > > > +#usr/share/pmacct/docs/MSGLOG_DUMP_FORMATS
> > > > > > +#usr/share/pmacct/docs/SIGNALS
> > > > > > +#usr/share/pmacct/docs/TRIGGER_VARS
> > > > > > +#usr/share/pmacct/examples
> > > > > > +usr/share/pmacct/examples/allow.lst.example
> > > > > > +#usr/share/pmacct/examples/amqp
> > > > > > +usr/share/pmacct/examples/amqp/amqp_receiver.py
> > > > > > +#usr/share/pmacct/examples/avro
> > > > > > +usr/share/pmacct/examples/avro/avro_file_decoder.py
> > > > > > +usr/share/pmacct/examples/bgp_agent.map.example
> > > > > > +usr/share/pmacct/examples/bgp_md5.lst.example
> > > > > > +usr/share/pmacct/examples/bgp_xconnects.map.example
> > > > > > +usr/share/pmacct/examples/flow_to_rd.map.example
> > > > > > +#usr/share/pmacct/examples/kafka
> > > > > > +usr/share/pmacct/examples/kafka/kafka_consumer.py
> > > > > > +#usr/share/pmacct/examples/lg
> > > > > > +usr/share/pmacct/examples/lg/pmbgp.py
> > > > > > +usr/share/pmacct/examples/networks.lst.example
> > > > > > +usr/share/pmacct/examples/nfacctd-sql.conf.example
> > > > > > +usr/share/pmacct/examples/pcap_interfaces.map.example
> > > > > > +usr/share/pmacct/examples/peers.map.example
> > > > > > +usr/share/pmacct/examples/pmacctd-imt.conf.example
> > > > > > +usr/share/pmacct/examples/pmacctd-multiple-
> > > > > > plugins.conf.example
> > > > > > +usr/share/pmacct/examples/pmacctd-sql.conf.example
> > > > > > +usr/share/pmacct/examples/pmacctd-sqlite3.conf.example
> > > > > > +usr/share/pmacct/examples/ports.lst.example
> > > > > > +usr/share/pmacct/examples/pretag.map.example
> > > > > > +usr/share/pmacct/examples/primitives.lst.example
> > > > > > +usr/share/pmacct/examples/probe_netflow.conf.example
> > > > > > +usr/share/pmacct/examples/probe_sflow.conf.example
> > > > > > +usr/share/pmacct/examples/sampling.map.example
> > > > > > +usr/share/pmacct/examples/tee_receivers.lst.example
> > > > > > +#usr/share/pmacct/sql
> > > > > > +#usr/share/pmacct/sql/README.64bit
> > > > > > +#usr/share/pmacct/sql/README.GeoIP
> > > > > > +#usr/share/pmacct/sql/README.IPv6
> > > > > > +#usr/share/pmacct/sql/README.cos
> > > > > > +#usr/share/pmacct/sql/README.custom_primitives
> > > > > > +#usr/share/pmacct/sql/README.etype
> > > > > > +#usr/share/pmacct/sql/README.export_proto
> > > > > > +#usr/share/pmacct/sql/README.iface
> > > > > > +#usr/share/pmacct/sql/README.label
> > > > > > +#usr/share/pmacct/sql/README.mask
> > > > > > +#usr/share/pmacct/sql/README.mpls
> > > > > > +#usr/share/pmacct/sql/README.mysql
> > > > > > +#usr/share/pmacct/sql/README.nat
> > > > > > +#usr/share/pmacct/sql/README.pgsql
> > > > > > +#usr/share/pmacct/sql/README.sampling
> > > > > > +usr/share/pmacct/sql/README.sqlite3
> > > > > > +#usr/share/pmacct/sql/README.tag2
> > > > > > +#usr/share/pmacct/sql/README.timestamp
> > > > > > +#usr/share/pmacct/sql/README.tunnel
> > > > > > +#usr/share/pmacct/sql/pmacct-create-db.pgsql
> > > > > > +#usr/share/pmacct/sql/pmacct-create-db_bgp_v1.mysql
> > > > > > +#usr/share/pmacct/sql/pmacct-create-db_v1.mysql
> > > > > > +#usr/share/pmacct/sql/pmacct-create-db_v2.mysql
> > > > > > +#usr/share/pmacct/sql/pmacct-create-db_v3.mysql
> > > > > > +#usr/share/pmacct/sql/pmacct-create-db_v4.mysql
> > > > > > +#usr/share/pmacct/sql/pmacct-create-db_v5.mysql
> > > > > > +#usr/share/pmacct/sql/pmacct-create-db_v6.mysql
> > > > > > +#usr/share/pmacct/sql/pmacct-create-db_v7.mysql
> > > > > > +#usr/share/pmacct/sql/pmacct-create-db_v8.mysql
> > > > > > +#usr/share/pmacct/sql/pmacct-create-db_v9.mysql
> > > > > > +#usr/share/pmacct/sql/pmacct-create-table_bgp_v1.pgsql
> > > > > > +usr/share/pmacct/sql/pmacct-create-table_bgp_v1.sqlite3
> > > > > > +#usr/share/pmacct/sql/pmacct-create-table_v1.pgsql
> > > > > > +usr/share/pmacct/sql/pmacct-create-table_v1.sqlite3
> > > > > > +#usr/share/pmacct/sql/pmacct-create-table_v2.pgsql
> > > > > > +usr/share/pmacct/sql/pmacct-create-table_v2.sqlite3
> > > > > > +#usr/share/pmacct/sql/pmacct-create-table_v3.pgsql
> > > > > > +usr/share/pmacct/sql/pmacct-create-table_v3.sqlite3
> > > > > > +#usr/share/pmacct/sql/pmacct-create-table_v4.pgsql
> > > > > > +usr/share/pmacct/sql/pmacct-create-table_v4.sqlite3
> > > > > > +#usr/share/pmacct/sql/pmacct-create-table_v5.pgsql
> > > > > > +usr/share/pmacct/sql/pmacct-create-table_v5.sqlite3
> > > > > > +#usr/share/pmacct/sql/pmacct-create-table_v6.pgsql
> > > > > > +usr/share/pmacct/sql/pmacct-create-table_v6.sqlite3
> > > > > > +usr/share/pmacct/sql/pmacct-create-table_v7.sqlite3
> > > > > > +#usr/share/pmacct/sql/pmacct-create-table_v7_v8.pgsql
> > > > > > +usr/share/pmacct/sql/pmacct-create-table_v8.sqlite3
> > > > > > +#usr/share/pmacct/sql/pmacct-create-table_v9.pgsql
> > > > > > +usr/share/pmacct/sql/pmacct-create-table_v9.sqlite3
> > > > > > +#usr/share/pmacct/sql/pmacct-grant-db.mysql
> > > > > > +var/ipfire/backup/addons/includes/pmacct
> > > > > > +var/spool/pmacct
> > > > > > diff --git a/lfs/pmacct b/lfs/pmacct
> > > > > > new file mode 100644
> > > > > > index 000000000..c6cc5270e
> > > > > > --- /dev/null
> > > > > > +++ b/lfs/pmacct
> > > > > > @@ -0,0 +1,107 @@
> > > > > > +##########################################################
> > > > > > #####################
> > > > > >
> > > > > > +# #
> > > > > > +# IPFire.org - A linux based
> > > > > > firewall #
> > > > > > +# Copyright (C) 2019 IPFire Team
> > > > > > <info(a)ipfire.org> #
> > > > > > +# #
> > > > > > +# This program is free software: you can redistribute it
> > > > > > and/or
> > > > > > modify #
> > > > > > +# it under the terms of the GNU General Public License as
> > > > > > published by #
> > > > > > +# the Free Software Foundation, either version 3 of the
> > > > > > License,
> > > > > > or #
> > > > > > +# (at your option) any later
> > > > > > version. #
> > > > > > +# #
> > > > > > +# This program is distributed in the hope that it will be
> > > > > > useful, #
> > > > > > +# but WITHOUT ANY WARRANTY; without even the implied
> > > > > > warranty
> > > > > > of #
> > > > > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
> > > > > > the #
> > > > > > +# GNU General Public License for more
> > > > > > details. #
> > > > > > +# #
> > > > > > +# You should have received a copy of the GNU General
> > > > > > Public
> > > > > > License #
> > > > > > +# along with this program. If not, see
> > > > > > <http://www.gnu.org/licenses/>. #
> > > > > > +# #
> > > > > > +##########################################################
> > > > > > #####################
> > > > > >
> > > > > > +
> > > > > > +##########################################################
> > > > > > #####################
> > > > > >
> > > > > > +# Definitions
> > > > > > +##########################################################
> > > > > > #####################
> > > > > >
> > > > > > +
> > > > > > +include Config
> > > > > > +
> > > > > > +VER = 1.7.6
> > > > > > +
> > > > > > +THISAPP = pmacct-$(VER)
> > > > > > +DL_FILE = $(THISAPP).tar.gz
> > > > > > +DL_FROM = $(URL_IPFIRE)
> > > > > > +DIR_APP = $(DIR_SRC)/$(THISAPP)
> > > > > > +TARGET = $(DIR_INFO)/$(THISAPP)
> > > > > > +PROG = pmacct
> > > > > > +PAK_VER = 1
> > > > > > +
> > > > > > +DEPS = libcdada
> > > > > > +
> > > > > > +##########################################################
> > > > > > #####################
> > > > > >
> > > > > > +# Top-level Rules
> > > > > > +##########################################################
> > > > > > #####################
> > > > > >
> > > > > > +
> > > > > > +objects = $(DL_FILE)
> > > > > > +
> > > > > > +$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
> > > > > > +
> > > > > > +$(DL_FILE)_MD5 = b829cfdc394d5acd6d32fb23de181a6b
> > > > > > +
> > > > > > +install : $(TARGET)
> > > > > > +
> > > > > > +check : $(patsubst %,$(DIR_CHK)/%,$(objects))
> > > > > > +
> > > > > > +download :$(patsubst %,$(DIR_DL)/%,$(objects))
> > > > > > +
> > > > > > +md5 : $(subst %,%_MD5,$(objects))
> > > > > > +
> > > > > > +dist:
> > > > > > + @$(PAK)
> > > > > > +
> > > > > > +##########################################################
> > > > > > #####################
> > > > > >
> > > > > > +# Downloading, checking, md5sum
> > > > > > +##########################################################
> > > > > > #####################
> > > > > >
> > > > > > +
> > > > > > +$(patsubst %,$(DIR_CHK)/%,$(objects)) :
> > > > > > + @$(CHECK)
> > > > > > +
> > > > > > +$(patsubst %,$(DIR_DL)/%,$(objects)) :
> > > > > > + @$(LOAD)
> > > > > > +
> > > > > > +$(subst %,%_MD5,$(objects)) :
> > > > > > + @$(MD5)
> > > > > > +
> > > > > > +##########################################################
> > > > > > #####################
> > > > > >
> > > > > > +# Installation Details
> > > > > > +##########################################################
> > > > > > #####################
> > > > > >
> > > > > > +
> > > > > > +$(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
> > > > > > + @$(PREBUILD)
> > > > > > + @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar zxf
> > > > > > $(DIR_DL)/$(DL_FILE)
> > > > > > + cd $(DIR_APP) && ./autogen.sh
> > > > > > + cd $(DIR_APP) && ./configure \
> > > > > > + --prefix=/usr \
> > > > > > + --sysconfdir=/etc/pmacct \
> > > > > > + --enable-sqlite3 \
> > > > > > + --enable-l2 \
> > > > > > + --enable-jansson
> > > > > > +
> > > > > > + # Add directory for plugins and pipes
> > > > > > + -mkdir -v /var/spool/pmacct
> > > > > > +
> > > > > > + # Add configuration folder and copy conf files to it
> > > > > > + -mkdir -v /etc/pmacct
> > > > > > + cp -vf $(DIR_CONF)/pmacct/*.conf /etc/pmacct
> > > > > > +
> > > > > > + # Copy initscript to dir
> > > > > > + cp -vf $(DIR_CONF)/pmacct/pmacct.init
> > > > > > /etc/rc.d/init.d/pmacct
> > > > > > + chmod 754 /etc/rc.d/init.d/pmacct
> > > > > > + chown root:root /etc/rc.d/init.d/pmacct
> > > > > > +
> > > > > > + cd $(DIR_APP) && make $(MAKETUNING)
> > > > > > + cd $(DIR_APP) && make install
> > > > > > +
> > > > > > + # Install backup definition
> > > > > > + install -v -m 644
> > > > > > ${DIR_SRC}/config/backup/includes/pmacct \
> > > > > > + /var/ipfire/backup/addons/includes/pmacct
> > > > > > + @rm -rf $(DIR_APP)
> > > > > > + @$(POSTBUILD)
> > > > > > diff --git a/make.sh b/make.sh
> > > > > > index 43cccaca0..d42c9b1e2 100755
> > > > > > --- a/make.sh
> > > > > > +++ b/make.sh
> > > > > > @@ -1646,6 +1646,8 @@ buildipfire() {
> > > > > > lfsmake2 ncdu
> > > > > > lfsmake2 lshw
> > > > > > lfsmake2 socat
> > > > > > + lfsmake2 libcdada
> > > > > > + lfsmake2 pmacct
> > > > > > }
> > > > > >
> > > > > > buildinstaller() {
> > > > > > diff --git a/src/paks/pmacct/install.sh
> > > > > > b/src/paks/pmacct/install.sh
> > > > > > new file mode 100644
> > > > > > index 000000000..8178cc7bc
> > > > > > --- /dev/null
> > > > > > +++ b/src/paks/pmacct/install.sh
> > > > > > @@ -0,0 +1,36 @@
> > > > > > +#!/bin/bash
> > > > > > +##########################################################
> > > > > > ##################
> > > > > >
> > > > > > +# #
> > > > > > +# This file is part of the IPFire
> > > > > > Firewall. #
> > > > > > +# #
> > > > > > +# IPFire is free software; you can redistribute it and/or
> > > > > > modify #
> > > > > > +# it under the terms of the GNU General Public License as
> > > > > > published by #
> > > > > > +# the Free Software Foundation; either version 2 of the
> > > > > > License,
> > > > > > or #
> > > > > > +# (at your option) any later
> > > > > > version. #
> > > > > > +# #
> > > > > > +# IPFire is distributed in the hope that it will be
> > > > > > useful, #
> > > > > > +# but WITHOUT ANY WARRANTY; without even the implied
> > > > > > warranty
> > > > > > of #
> > > > > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
> > > > > > the #
> > > > > > +# GNU General Public License for more
> > > > > > details. #
> > > > > > +# #
> > > > > > +# You should have received a copy of the GNU General
> > > > > > Public
> > > > > > License #
> > > > > > +# along with IPFire; if not, write to the Free
> > > > > > Software #
> > > > > > +# Foundation, Inc., 59 Temple Place, Suite 330, Boston,
> > > > > > MA
> > > > > > 02111-1307 USA #
> > > > > > +# #
> > > > > > +# Copyright (C) 2007 IPFire-Team
> > > > > > <info(a)ipfire.org>. #
> > > > > > +# #
> > > > > > +##########################################################
> > > > > > ##################
> > > > > >
> > > > > > +#
> > > > > > +. /opt/pakfire/lib/functions.sh
> > > > > > +
> > > > > > +extract_files
> > > > > > +restore_backup ${NAME}
> > > > > > +
> > > > > > +# Add symlinks for runlevels
> > > > > > +ln -s ../init.d/${NAME} /etc/rc.d/rc0.d/K85${NAME}
> > > > > > +ln -s ../init.d/${NAME} /etc/rc.d/rc3.d/S50${NAME}
> > > > > > +ln -s ../init.d/${NAME} /etc/rc.d/rc6.d/K85${NAME}
> > > > > > +start_service --background ${NAME}
> > > > > > +
> > > > > > +
> > > > > > +# EOF
> > > > > > diff --git a/src/paks/pmacct/uninstall.sh
> > > > > > b/src/paks/pmacct/uninstall.sh
> > > > > > new file mode 100644
> > > > > > index 000000000..51631e8c1
> > > > > > --- /dev/null
> > > > > > +++ b/src/paks/pmacct/uninstall.sh
> > > > > > @@ -0,0 +1,33 @@
> > > > > > +#!/bin/bash
> > > > > > +##########################################################
> > > > > > ##################
> > > > > >
> > > > > > +# #
> > > > > > +# This file is part of the IPFire
> > > > > > Firewall. #
> > > > > > +# #
> > > > > > +# IPFire is free software; you can redistribute it and/or
> > > > > > modify #
> > > > > > +# it under the terms of the GNU General Public License as
> > > > > > published by #
> > > > > > +# the Free Software Foundation; either version 2 of the
> > > > > > License,
> > > > > > or #
> > > > > > +# (at your option) any later
> > > > > > version. #
> > > > > > +# #
> > > > > > +# IPFire is distributed in the hope that it will be
> > > > > > useful, #
> > > > > > +# but WITHOUT ANY WARRANTY; without even the implied
> > > > > > warranty
> > > > > > of #
> > > > > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
> > > > > > the #
> > > > > > +# GNU General Public License for more
> > > > > > details. #
> > > > > > +# #
> > > > > > +# You should have received a copy of the GNU General
> > > > > > Public
> > > > > > License #
> > > > > > +# along with IPFire; if not, write to the Free
> > > > > > Software #
> > > > > > +# Foundation, Inc., 59 Temple Place, Suite 330, Boston,
> > > > > > MA
> > > > > > 02111-1307 USA #
> > > > > > +# #
> > > > > > +# Copyright (C) 2007 IPFire-Team
> > > > > > <info(a)ipfire.org>. #
> > > > > > +# #
> > > > > > +##########################################################
> > > > > > ##################
> > > > > >
> > > > > > +#
> > > > > > +. /opt/pakfire/lib/functions.sh
> > > > > > +
> > > > > > +stop_service ${NAME}
> > > > > > +make_backup ${NAME}
> > > > > > +remove_files
> > > > > > +
> > > > > > +# Delete symlinks
> > > > > > +rm -rfv /etc/rc.d/rc?.d/???${NAME}
> > > > > > +
> > > > > > +# EOF
> > > > > > diff --git a/src/paks/pmacct/update.sh
> > > > > > b/src/paks/pmacct/update.sh
> > > > > > new file mode 100644
> > > > > > index 000000000..89c40d0d7
> > > > > > --- /dev/null
> > > > > > +++ b/src/paks/pmacct/update.sh
> > > > > > @@ -0,0 +1,26 @@
> > > > > > +#!/bin/bash
> > > > > > +##########################################################
> > > > > > ##################
> > > > > >
> > > > > > +# #
> > > > > > +# This file is part of the IPFire
> > > > > > Firewall. #
> > > > > > +# #
> > > > > > +# IPFire is free software; you can redistribute it and/or
> > > > > > modify #
> > > > > > +# it under the terms of the GNU General Public License as
> > > > > > published by #
> > > > > > +# the Free Software Foundation; either version 2 of the
> > > > > > License,
> > > > > > or #
> > > > > > +# (at your option) any later
> > > > > > version. #
> > > > > > +# #
> > > > > > +# IPFire is distributed in the hope that it will be
> > > > > > useful, #
> > > > > > +# but WITHOUT ANY WARRANTY; without even the implied
> > > > > > warranty
> > > > > > of #
> > > > > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
> > > > > > the #
> > > > > > +# GNU General Public License for more
> > > > > > details. #
> > > > > > +# #
> > > > > > +# You should have received a copy of the GNU General
> > > > > > Public
> > > > > > License #
> > > > > > +# along with IPFire; if not, write to the Free
> > > > > > Software #
> > > > > > +# Foundation, Inc., 59 Temple Place, Suite 330, Boston,
> > > > > > MA
> > > > > > 02111-1307 USA #
> > > > > > +# #
> > > > > > +# Copyright (C) 2007 IPFire-Team
> > > > > > <info(a)ipfire.org>. #
> > > > > > +# #
> > > > > > +##########################################################
> > > > > > ##################
> > > > > >
> > > > > > +#
> > > > > > +. /opt/pakfire/lib/functions.sh
> > > > > > +./uninstall.sh
> > > > > > +./install.sh
> > > > > > --
> > > > > > 2.30.1
> > >
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-03-19 17:19 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-08 11:57 [PATCH 1/2] pmacct: New addon Adolf Belka (ipfire)
2021-03-18 13:07 ` Michael Tremer
2021-03-18 13:34 ` Adolf Belka
2021-03-18 18:25 ` Michael Tremer
2021-03-18 18:43 ` Adolf Belka
2021-03-19 9:20 ` Adolf Belka
2021-03-19 17:19 ` ummeegge
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox