Hi Kay,
could you please send this patch attached to the email? My email client shows it as embedded text and all tabs or leading spaces there might have been have been stripped away.
A git branch would be even better. Do you have got access to our git repositories? If not, you can also use github.
-Michael
On Sun, 2013-04-07 at 10:23 +0200, Kay-Michael Köhler wrote:
I finished progress on sorting with connections.cgi and i want to share with you guys.
Green arrows for sorting on source ip, source port, destination ip and destination port, protocol, connections status, upload/download and ttl will now appear on iptables connections tracking WUI.
It would be a pleasure to hear from you guys if you have further suggestions.
To keep the sorting less time consuming and with a minimum memory footprint, i added a bash script what is doing all the sorting and removed the sort command from the piped command line.
The bash script "consort.sh" goes to /usr/local/bin.
The two diffs diff against the "next" git repository:
diff --git a/src/scripts/consort.sh b/src/scripts/consort.sh new file mode 100755 index 0000000..1682f7a --- /dev/null +++ b/src/scripts/consort.sh @@ -0,0 +1,158 @@ +#/bin/bash +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2007-2013 IPFire Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +###############################################################################
+# sort conntrack table entries based on ip addresses +# @parm sort field +do_ip_sort() {
- sed \
- -r \
- 's/.*src=([0-9.]+).*dst=([0-9.]+).*src=.*/'$1'#\0/' $FILE_NAME \
- | sort \
- -t. \
- -k 1,1n$SORT_ORDER -k 2,2n$SORT_ORDER -k 3,3n$SORT_ORDER -k 4,4n
$SORT_ORDER \
- | sed \
- -r \
- 's/.*#(.*)/\1/'
+}
+# sort conntrack table entries based on port addresses +# @parm sort field +do_port_sort() {
- sed \
- -r \
- 's/.*sport=([0-9]+).*dport=([0-9]+).*src=.*/'$1'#\0/' $FILE_NAME \
- | sort \
- -t# \
- -k 1,1n$SORT_ORDER \
- | sed \
- -r \
- 's/.*#(.*)/\1/'
+}
+# sort conntrack table entries based on protocol +do_protocol_sort() {
- sed \
- -r \
- 's/^[0-9a-zA-Z]+[ ]+[0-9]+[ ]+([a-zA-Z0-9]+)/\1#\0/' $FILE_NAME \
- | sort \
- -t# \
- -k 1,1$SORT_ORDER \
- | sed \
- -r \
- 's/.*#(.*)/\1/'
+}
+# sort conntrack table entries based on connection status +do_status_sort() {
- sed \
- -r \
's/^[0-9a-zA-Z]+[ ]+[0-9]+[ ]+[a-zA-Z0-9]+[ ]+[0-9]+[ ]+[0-9]+[ ]+([a-zA-Z_0-9]+)[ ]+|^[0-9a-zA-Z]+[ ]+[0-9]+[ ]+[a-zA-Z0-9]+[ ]+[0-9]+[ ]+[0-9]+([ ]+)/\1#\0/' $FILE_NAME \
- | sort \
- -t# \
- -k 1,1$SORT_ORDER \
- | sed \
- -r \
- 's/.*#(.*)/\1/'
+}
+# sort conntrack table entries based on connection time to life +do_ttl_sort() {
- sed \
- -r \
's/^[0-9a-zA-Z]+[ ]+[0-9]+[ ]+[a-zA-Z0-9]+[ ]+[0-9]+[ ]+([0-9]+)[ ]+/\1#\0/' $FILE_NAME \
- | sort \
- -t# \
- -k 1,1n$SORT_ORDER \
- | sed \
- -r \
- 's/.*#(.*)/\1/'
+}
+# sort conntrack table entries based on downloaded bytes +do_downloaded_bytes_sort() {
- sed \
- -r \
- 's/.*src=.*bytes=([0-9]+).*src=/\1#\0/' $FILE_NAME \
- | sort \
- -t# \
- -k 1,1n$SORT_ORDER \
- | sed \
- -r \
- 's/.*#(.*)/\1/'
+}
+# sort conntrack table entries based on uploaded bytes +do_uploaded_bytes_sort() {
- sed \
- -r \
- 's/.*src=.*bytes=([0-9]+).*/\1#\0/' $FILE_NAME \
- | sort \
- -t# \
- -k 1,1n$SORT_ORDER \
- | sed \
- -r \
- 's/.*#(.*)/\1/'
+}
+SORT_ORDER= +FILE_NAME=
+if [ $# -lt 2 ]; then
- echo "Usage: consort <sort criteria
1=srcIp,2=dstIp,3=srcPort,4=dstPort,5=protocol,6=connection status> <a=ascending,d=descending> [input file]"
- echo " consort.sh 1 a a.txt"
- echo " cat a.txt | consort 1 d"
- exit;
+fi
+if [[ 'a d A D' =~ $2 ]]; then
- if [[ 'd D' =~ $2 ]]; then
- SORT_ORDER=r
- fi
+else
- echo "Unknown sort order "$2""
- exit;
+fi
+if [ $# == 3 ]; then
- if [ ! -f $3 ]; then
- echo "File not found."
- exit;
- fi
- FILE_NAME=$3
+fi
+if [[ '1 2' =~ $1 ]]; then
- do_ip_sort $1
+elif [[ '3 4' =~ $1 ]]; then
- do_port_sort $(($1-2))
+elif [[ '5' =~ $1 ]]; then
- do_protocol_sort
+elif [[ '6' =~ $1 ]]; then
- do_status_sort
+elif [[ '7' =~ $1 ]]; then
- do_ttl_sort
+elif [[ '8' =~ $1 ]]; then
- do_downloaded_bytes_sort
+elif [[ '9' =~ $1 ]]; then
- do_uploaded_bytes_sort
+else
- echo "Unknown sort criteria "$1""
+fi
diff --git a/html/cgi-bin/connections.cgi b/html/cgi-bin/connections.cgi index 1edf3e5..d566cf7 100644 --- a/html/cgi-bin/connections.cgi +++ b/html/cgi-bin/connections.cgi @@ -34,6 +34,31 @@
my $colour_multicast = "#A0A0A0";
+# sort arguments for connection tracking table +# the sort field. eg. 1=src IP, 2=dst IP, 3=src port, 4=dst port +my $SORT_FIELD = 0; +# the sort order. (a)scending orr (d)escending +my $SORT_ORDER = 0; +# cgi query arguments +my %cgiin; +# debug mode +my $debug = 0;
+# retrieve query arguments +# note: let a-z A-Z and 0-9 pass as value only +if (length ($ENV{'QUERY_STRING'}) > 0){
- my $name;
- my $value;
- my $buffer = $ENV{'QUERY_STRING'};
- my @pairs = split(/&/, $buffer);
- foreach my $pair (@pairs){
- ($name, $value) = split(/=/, $pair);
- $value =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg; # e.g.
"%20" => " "
- $value =~ s/[^a-zA-Z0-9]*//g; # a-Z 0-9 will pass
- $cgiin{$name} = $value;
- }
+}
&Header::showhttpheaders();
my @network=(); @@ -43,12 +68,40 @@ my %netsettings=(); &General::readhash("${General::swroot}/ethernet/settings", % netsettings);
+# output cgi query arrguments to browser on debug +if ( $debug ){
- &Header::openbox('100%', 'center', 'DEBUG');
- my $debugCount = 0;
- foreach my $line (sort keys %cgiin) {
- print "$line = '$cgiin{$line}'<br />\n";
- $debugCount++;
- }
- print " Count: $debugCount\n";
- &Header::closebox();
+}
#workaround to suppress a warning when a variable is used only once my @dummy = ( ${Header::table1colour} ); undef (@dummy);
-# Read the connection tracking table. -open(CONNTRACK, "/usr/local/bin/getconntracktable | sort -k 5,5 --numeric-sort --reverse |") or die "Unable to read conntrack table"; +# check sorting arguments +if ( $cgiin{'sort_field'} ~~ [ '1','2','3','4','5','6','7','8','9' ] ) {
- $SORT_FIELD = $cgiin{'sort_field'};
- if ( $cgiin{'sort_order'} ~~ [ 'a','d','A','D' ] ) {
- $SORT_ORDER = lc($cgiin{'sort_order'});
- }
+}
+# Read and sort the connection tracking table +# do sorting +if ($SORT_FIELD and $SORT_ORDER) {
- # field sorting when sorting arguments are sane
- open(CONNTRACK, "/usr/local/bin/getconntracktable
| /usr/local/bin/consort.sh $SORT_FIELD $SORT_ORDER |") or die "Unable to read conntrack table"; +} else {
- # default sorting with no query arguments
- open(CONNTRACK, "/usr/local/bin/getconntracktable | sort -k 5,5
--numeric-sort --reverse |") or die "Unable to read conntrack table"; +} my @conntrack = <CONNTRACK>; close(CONNTRACK);
@@ -263,21 +316,81 @@
<br> END
+if ($SORT_FIELD and $SORT_ORDER) {
- my @sort_field_name = (
- $Lang::tr{'source ip'},
- $Lang::tr{'destination ip'},
- $Lang::tr{'source port'},
- $Lang::tr{'destination port'},
- $Lang::tr{'protocol'},
- $Lang::tr{'connection'}.' '.$Lang::tr{'status'},
- $Lang::tr{'expires'}.' ('.$Lang::tr{'seconds'}.')',
- $Lang::tr{'download'},
- $Lang::tr{'upload'}
- );
- my $sort_order_name;
- if (lc($SORT_ORDER) eq "a") {
- $sort_order_name = $Lang::tr{'sort ascending'};
- } else {
- $sort_order_name = $Lang::tr{'sort descending'};
- }
+print <<END
<div style="font-weight:bold;margin:10px;font-size: 70%">
- $sort_order_name: $sort_field_name[$SORT_FIELD-1]
</div>
+END +; +}
# Print table header. print <<END;
<table width='100%'> - <tr> + <tr valign="top""> + <th align='center'> + <a href="?sort_field=5&sort_order=d"><img style="width:10px" src="/images/up.gif"></a> + <a href="?sort_field=5&sort_order=a"><img style="width:10px" src="/images/down.gif"></a> + </th> + <th align='center' colspan="2"> + <a href="?sort_field=1&sort_order=d"><img style="width:10px" src="/images/up.gif"></a> + <a href="?sort_field=1&sort_order=a"><img style="width:10px" src="/images/down.gif"></a> + + <a href="?sort_field=3&sort_order=d"><img style="width:10px" src="/images/up.gif"></a> + <a href="?sort_field=3&sort_order=a"><img style="width:10px" src="/images/down.gif"></a> + </th> + <th align='center' colspan="2"> + <a href="?sort_field=2&sort_order=d"><img style="width:10px" src="/images/up.gif"></a> + <a href="?sort_field=2&sort_order=a"><img style="width:10px" src="/images/down.gif"></a> + + <a href="?sort_field=4&sort_order=d"><img style="width:10px" src="/images/up.gif"></a> + <a href="?sort_field=4&sort_order=a"><img style="width:10px" src="/images/down.gif"></a> + </th> + <th align='center'> + <a href="?sort_field=8&sort_order=d"><img style="width:10px" src="/images/up.gif"></a> + <a href="?sort_field=8&sort_order=a"><img style="width:10px" src="/images/down.gif"></a> + + <a href="?sort_field=9&sort_order=d"><img style="width:10px" src="/images/up.gif"></a> + <a href="?sort_field=9&sort_order=a"><img style="width:10px" src="/images/down.gif"></a> + </th> + <th align='center'> + <a href="?sort_field=6&sort_order=d"><img style="width:10px" src="/images/up.gif"></a> + <a href="?sort_field=6&sort_order=a"><img style="width:10px" src="/images/down.gif"></a> + </th> + <th align='center'> + <a href="?sort_field=7&sort_order=d"><img style="width:10px" src="/images/up.gif"></a> + <a href="?sort_field=7&sort_order=a"><img style="width:10px" src="/images/down.gif"></a> + </th> + </tr> + <tr valign="top""> <th align='center'> $Lang::tr{'protocol'} </th> - <th align='center'> + <th align='center' colspan="2"> $Lang::tr{'source ip and port'} </th> - <th> </th> - <th align='center'> + <th align='center' colspan="2"> $Lang::tr{'dest ip and port'} </th> - <th> </th> <th align='center'> $Lang::tr{'download'} / <br>$Lang::tr{'upload'}
2013/3/25 Michael Tremer michael.tremer@ipfire.org Hey,
a shell script is not the best way how to sort things, but I think we can accept this because before the current version of the CGI file, there was a lot of sorting done with shell commands as well. I did not try to run your code, because I am waiting for the other "two more sorting options". Michael On Sat, 2013-03-23 at 09:38 +0100, Kay-Michael Köhler wrote: > I did some progress on sorting with connections.cgi and i want to > share with you guys. > > > To keep the sorting less time consuming and with a minimum memory > footprint, i added a bash script what is doing all the sorting and > removed the sort command from the piped command line. > > > Eight green arrows for sorting on source ip, source port, destination > ip and destination port will now appear on iptables connections > tracking WUI. > > > I will add two more sorting option later after finished some other > work for ipfire. > > > The bash script "consort.sh" goes to /usr/local/bin. > > > Ok, now the two diffs to the actual git > > > diff --git a/src/scripts/consort.sh b/src/scripts/consort.sh > new file mode 100755 > index 0000000..1633beb > --- /dev/null > +++ b/src/scripts/consort.sh > @@ -0,0 +1,83 @@ > +#/bin/bash > +############################################################################### > +# > # > +# IPFire.org - A linux based firewall > # > +# Copyright (C) 2007-2013 IPFire Team <info@ipfire.org> > # > +# > # > +# This program is free software: you can redistribute it and/or > modify # > +# it under the terms of the GNU General Public License as published > by # > +# the Free Software Foundation, either version 3 of the License, or > # > +# (at your option) any later version. > # > +# > # > +# This program is distributed in the hope that it will be useful, > # > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > # > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > # > +# GNU General Public License for more details. > # > +# > # > +# You should have received a copy of the GNU General Public License > # > +# along with this program. If not, see > <http://www.gnu.org/licenses/>. # > +# > # > +############################################################################### > + > +# sort conntrack table entries based on ip addresses > +# @parm sort field > +do_ip_sort() { > + sed \ > + -r \ > + 's/.*src=([0-9\.]+).*dst=([0-9\.]+).*src=.*/\'$1'#\0/' $FILE_NAME \ > + | sort \ > + -t. \ > + -k 1,1n$SORT_ORDER -k 2,2n$SORT_ORDER -k 3,3n$SORT_ORDER -k 4,4n > $SORT_ORDER \ > + | sed \ > + -r \ > + 's/.*#(.*)/\1/' > +} > + > +# sort conntrack table entries based on port addresses > +# @parm sort field > +do_port_sort() { > + sed \ > + -r \ > + 's/.*sport=([0-9]+).*dport=([0-9]+).*src=.*/\'$1'#\0/' $FILE_NAME \ > + | sort \ > + -t# \ > + -k 1,1n$SORT_ORDER \ > + | sed \ > + -r \ > + 's/.*#(.*)/\1/' > +} > + > +SORT_ORDER= > +FILE_NAME= > + > +if [ $# -lt 2 ]; then > + echo "Usage: consort <sort criteria > 1=srcIp,2=dstIp,3=srcPort,4=dstPort> <a=ascending,d=descending> [input > file]" > + echo " consort.sh 1 a.txt" > + echo " cat a.txt | consort 1" > + exit; > +fi > + > +if [[ 'a d A D' =~ $2 ]]; then > + if [[ 'd D' =~ $2 ]]; then > + SORT_ORDER=r > + fi > +else > + echo "Unknown sort order \"$2\"" > + exit; > +fi > + > +if [ $# == 3 ]; then > + if [ ! -f $3 ]; then > + echo "File not found." > + exit; > + fi > + FILE_NAME=$3 > +fi > + > +if [[ '1 2' =~ $1 ]]; then > + do_ip_sort $1 > +elif [[ '3 4' =~ $1 ]]; then > + do_port_sort $(($1-2)) > +else > + echo "Unknown sort criteria \"$1\"" > +fi > > > > > diff --git a/html/cgi-bin/connections.cgi > b/html/cgi-bin/connections.cgi > index 1edf3e5..0c20957 100644 > --- a/html/cgi-bin/connections.cgi > +++ b/html/cgi-bin/connections.cgi > @@ -34,6 +34,31 @@ > > my $colour_multicast = "#A0A0A0"; > > +# sort arguments for connection tracking table > +# the sort field. eg. 1=src IP, 2=dst IP, 3=src port, 4=dst port > +my $SORT_FIELD = 0; > +# the sort order. (a)scending orr (d)escending > +my $SORT_ORDER = 0; > +# cgi query arguments > +my %cgiin; > +# debug mode > +my $debug = 0; > + > +# retrieve query arguments > +# note: let a-z A-Z and 0-9 pass as value only > +if (length ($ENV{'QUERY_STRING'}) > 0){ > + my $name; > + my $value; > + my $buffer = $ENV{'QUERY_STRING'}; > + my @pairs = split(/&/, $buffer); > + foreach my $pair (@pairs){ > + ($name, $value) = split(/=/, $pair); > + $value =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg; # e.g. > "%20" => " " > + $value =~ s/[^a-zA-Z0-9]*//g; # a-Z 0-9 will pass > + $cgiin{$name} = $value; > + } > +} > + > &Header::showhttpheaders(); > > my @network=(); > @@ -43,12 +68,43 @@ > my %netsettings=(); > &General::readhash("${General::swroot}/ethernet/settings", \% > netsettings); > > +# output cgi query arrguments to browser on debug > +if ( $debug ){ > + &Header::openbox('100%', 'center', 'DEBUG'); > + my $debugCount = 0; > + foreach my $line (sort keys %cgiin) { > + print "$line = '$cgiin{$line}'<br />\n"; > + $debugCount++; > + } > + print " Count: $debugCount\n"; > + &Header::closebox(); > +} > + > #workaround to suppress a warning when a variable is used only once > my @dummy = ( ${Header::table1colour} ); > undef (@dummy); > > -# Read the connection tracking table. > -open(CONNTRACK, "/usr/local/bin/getconntracktable | sort -k 5,5 > --numeric-sort --reverse |") or die "Unable to read conntrack table"; > + > + > +# check sorting arguments > +if ( $cgiin{'sort_field'} ~~ [ '1','2','3','4' ] ) { > + $SORT_FIELD = $cgiin{'sort_field'}; > + > + if ( $cgiin{'sort_order'} ~~ [ 'a','d','A','D' ] ) { > + $SORT_ORDER = lc($cgiin{'sort_order'}); > + } > +} > + > +# Read and sort the connection tracking table > +# do sorting > +if ($SORT_FIELD and $SORT_ORDER) { > + # field sorting when sorting arguments are sane > + open(CONNTRACK, "/usr/local/bin/getconntracktable > | /usr/local/bin/consort.sh $SORT_FIELD $SORT_ORDER |") or die "Unable > to read conntrack table"; > +} else { > + # default sorting with no query arguments > + open(CONNTRACK, "/usr/local/bin/getconntracktable | sort -k 5,5 > --numeric-sort --reverse |") or die "Unable to read conntrack table"; > +} > + > my @conntrack = <CONNTRACK>; > close(CONNTRACK); > > @@ -263,21 +319,49 @@ > <br> > END > > +if ($SORT_FIELD and $SORT_ORDER) { > + my @sort_field_name = ( > + $Lang::tr{'source ip'}, > + $Lang::tr{'destination ip'}, > + $Lang::tr{'source port'}, > + $Lang::tr{'destination port'} > + ); > + my $sort_order_name; > + if (lc($SORT_ORDER) eq "a") { > + $sort_order_name = $Lang::tr{'sort ascending'}; > + } else { > + $sort_order_name = $Lang::tr{'sort descending'}; > + } > + > +print <<END > + <div style="font-weight:bold;margin:10px;font-size: 80%"> > + $sort_order_name: $sort_field_name[$SORT_FIELD-1] > + </div> > +END > +; > +} > + > # Print table header. > print <<END; > <table width='100%'> > - <tr> > + <tr valign="top""> > <th align='center'> > $Lang::tr{'protocol'} > </th> > - <th align='center'> > + <th align='center' colspan="2"> > + <a href="?sort_field=1&sort_order=d"><img style="width:10px" > src="/images/up.gif"></a> > + <a href="?sort_field=1&sort_order=a"><img style="width:10px" > src="/images/down.gif"></a> > $Lang::tr{'source ip and port'} > + <a href="?sort_field=3&sort_order=d"><img style="width:10px" > src="/images/up.gif"></a> > + <a href="?sort_field=3&sort_order=a"><img style="width:10px" > src="/images/down.gif"></a> > </th> > - <th> </th> > - <th align='center'> > + <th align='center' colspan="2"> > + <a href="?sort_field=2&sort_order=d"><img style="width:10px" > src="/images/up.gif"></a> > + <a href="?sort_field=2&sort_order=a"><img style="width:10px" > src="/images/down.gif"></a> > $Lang::tr{'dest ip and port'} > + <a href="?sort_field=4&sort_order=d"><img style="width:10px" > src="/images/up.gif"></a> > + <a href="?sort_field=4&sort_order=a"><img style="width:10px" > src="/images/down.gif"></a> > </th> > - <th> </th> > <th align='center'> > $Lang::tr{'download'} / > <br>$Lang::tr{'upload'} > > > > > 2013/3/10 Michael Tremer <michael.tremer@ipfire.org> > Hi, > > sure, this is fine with me. Just try to make the sorting > process > efficient so that even ten thousands of connections are > properly > displayed. > > -Michael > > On Sun, 2013-03-10 at 17:01 +0100, Kay-Michael Köhler wrote: > > Hello everyone > > > > > > i'm going to start development on connections.cgi to have > some kind of > > sorting at "iptables connection > tracking" (status->connections) > > > > > > I think it is a good idea to have the following (asc/desc) > sort > > options: > > > > > > "Protocol" > > "Source IP:Port" > > "Destination IP Port" > > "Connection status" > > > > > > If you guys agree it would be a please for me to share and > post the > > patch here when i'm done. > > > > > > Regards > > > > > > Kay-Michael > > > _______________________________________________ > > Development mailing list > > Development@lists.ipfire.org > > http://lists.ipfire.org/mailman/listinfo/development > > _______________________________________________ > Development mailing list > Development@lists.ipfire.org > http://lists.ipfire.org/mailman/listinfo/development > >