Re: connections.cgi: iptables connection tracking.

* Re: connections.cgi: iptables connection tracking.
       [not found] <CALL-FuUoiKzat4wExtUD6YCOPug=gzjGjJAmJT4h_aNEA73ZKg@mail.gmail.com>
@ 2013-03-26 18:18 ` Michael Tremer
  0 siblings, 0 replies; 5+ messages in thread
From: Michael Tremer @ 2013-03-26 18:18 UTC (permalink / raw)
  To: development

[-- Attachment #1: Type: text/plain, Size: 14992 bytes --]

No, I think it is not worth it, because this is not a high performance
code path...

On Mon, 2013-03-25 at 21:50 +0100, Kay-Michael Köhler wrote:
> Y, the best way is to rewrite that part of the cgi from scratch,
> right, migrating sed and sort to perl map/sort statements (where
> memory and performace could be an issue on small appliances then), but
> the question is would it worth the energy when ipfire 3 is ahead. Tell
> me and i do that job for ipf2 with pure perl.
> 
> In case someone is working with my patch at the moment, it would be a
> pleasure to receive any feedback.

I thought there was more coming?

> 
> 
> 2013/3/25 Michael Tremer <michael.tremer(a)ipfire.org>
>         Hey,
>         
>         a shell script is not the best way how to sort things, but I
>         think we
>         can accept this because before the current version of the CGI
>         file,
>         there was a lot of sorting done with shell commands as well.
>         
>         I did not try to run your code, because I am waiting for the
>         other "two
>         more sorting options".
>         
>         Michael
>         
>         On Sat, 2013-03-23 at 09:38 +0100, Kay-Michael Köhler wrote:
>         > I did some progress on sorting with connections.cgi and i
>         want to
>         > share with you guys.
>         >
>         >
>         > To keep the sorting less time consuming and with a minimum
>         memory
>         > footprint, i added a bash script what is doing all the
>         sorting and
>         > removed the sort command from the piped command line.
>         >
>         >
>         > Eight green arrows for sorting on source ip, source port,
>         destination
>         > ip and destination port will now appear on iptables
>         connections
>         > tracking WUI.
>         >
>         >
>         > I will add two more sorting option later after finished some
>         other
>         > work for ipfire.
>         >
>         >
>         > The bash script "consort.sh" goes to /usr/local/bin.
>         >
>         >
>         > Ok, now the two diffs to the actual git
>         >
>         >
>         > diff --git a/src/scripts/consort.sh b/src/scripts/consort.sh
>         > new file mode 100755
>         > index 0000000..1633beb
>         > --- /dev/null
>         > +++ b/src/scripts/consort.sh
>         > @@ -0,0 +1,83 @@
>         > +#/bin/bash
>         >
>         +###############################################################################
>         > +#
>         > #
>         > +# IPFire.org - A linux based firewall
>         > #
>         > +# Copyright (C) 2007-2013  IPFire Team  <info(a)ipfire.org>
>         > #
>         > +#
>         > #
>         > +# This program is free software: you can redistribute it
>         and/or
>         > modify        #
>         > +# it under the terms of the GNU General Public License as
>         published
>         > by        #
>         > +# the Free Software Foundation, either version 3 of the
>         License, or
>         > #
>         > +# (at your option) any later version.
>         > #
>         > +#
>         > #
>         > +# This program is distributed in the hope that it will be
>         useful,
>         > #
>         > +# but WITHOUT ANY WARRANTY; without even the implied
>         warranty of
>         >          #
>         > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
>         the
>         > #
>         > +# GNU General Public License for more details.
>         >          #
>         > +#
>         > #
>         > +# You should have received a copy of the GNU General Public
>         License
>         > #
>         > +# along with this program.  If not, see
>         > <http://www.gnu.org/licenses/>.       #
>         > +#
>         > #
>         >
>         +###############################################################################
>         > +
>         > +# sort conntrack table entries based on ip addresses
>         > +# @parm sort field
>         > +do_ip_sort() {
>         > + sed \
>         > + -r \
>         > + 's/.*src=([0-9\.]+).*dst=([0-9\.]+).*src=.*/\'$1'#\0/'
>         $FILE_NAME \
>         > + | sort \
>         > + -t. \
>         > + -k 1,1n$SORT_ORDER -k 2,2n$SORT_ORDER -k 3,3n$SORT_ORDER
>         -k 4,4n
>         > $SORT_ORDER \
>         > + | sed \
>         > + -r \
>         > + 's/.*#(.*)/\1/'
>         > +}
>         > +
>         > +# sort conntrack table entries based on port addresses
>         > +# @parm sort field
>         > +do_port_sort() {
>         > + sed \
>         > + -r \
>         > + 's/.*sport=([0-9]+).*dport=([0-9]+).*src=.*/\'$1'#\0/'
>         $FILE_NAME \
>         > + | sort \
>         > + -t# \
>         > + -k 1,1n$SORT_ORDER \
>         > + | sed \
>         > + -r \
>         > + 's/.*#(.*)/\1/'
>         > +}
>         > +
>         > +SORT_ORDER=
>         > +FILE_NAME=
>         > +
>         > +if [ $# -lt 2 ]; then
>         > + echo "Usage: consort <sort criteria
>         > 1=srcIp,2=dstIp,3=srcPort,4=dstPort>
>         <a=ascending,d=descending> [input
>         > file]"
>         > + echo " consort.sh 1 a.txt"
>         > + echo " cat a.txt | consort 1"
>         > + exit;
>         > +fi
>         > +
>         > +if [[ 'a d A D' =~ $2 ]]; then
>         > + if [[ 'd D' =~ $2 ]]; then
>         > + SORT_ORDER=r
>         > + fi
>         > +else
>         > + echo "Unknown sort order \"$2\""
>         > + exit;
>         > +fi
>         > +
>         > +if [ $# == 3 ]; then
>         > + if [ ! -f $3 ]; then
>         > + echo "File not found."
>         > + exit;
>         > + fi
>         > + FILE_NAME=$3
>         > +fi
>         > +
>         > +if [[ '1 2' =~ $1 ]]; then
>         > + do_ip_sort $1
>         > +elif [[ '3 4' =~ $1 ]]; then
>         > + do_port_sort $(($1-2))
>         > +else
>         > + echo "Unknown sort criteria \"$1\""
>         > +fi
>         >
>         >
>         >
>         >
>         > diff --git a/html/cgi-bin/connections.cgi
>         > b/html/cgi-bin/connections.cgi
>         > index 1edf3e5..0c20957 100644
>         > --- a/html/cgi-bin/connections.cgi
>         > +++ b/html/cgi-bin/connections.cgi
>         > @@ -34,6 +34,31 @@
>         >
>         >  my $colour_multicast = "#A0A0A0";
>         >
>         > +# sort arguments for connection tracking table
>         > +# the sort field. eg. 1=src IP, 2=dst IP, 3=src port, 4=dst
>         port
>         > +my $SORT_FIELD = 0;
>         > +# the sort order. (a)scending orr (d)escending
>         > +my $SORT_ORDER = 0;
>         > +# cgi query arguments
>         > +my %cgiin;
>         > +# debug mode
>         > +my $debug = 0;
>         > +
>         > +# retrieve query arguments
>         > +# note: let a-z A-Z and 0-9 pass as value only
>         > +if (length ($ENV{'QUERY_STRING'}) > 0){
>         > + my $name;
>         > + my $value;
>         > + my $buffer = $ENV{'QUERY_STRING'};
>         > + my @pairs = split(/&/, $buffer);
>         > + foreach my $pair (@pairs){
>         > + ($name, $value) = split(/=/, $pair);
>         > + $value =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C",
>         hex($1))/eg; # e.g.
>         > "%20" => " "
>         > + $value =~ s/[^a-zA-Z0-9]*//g; # a-Z 0-9 will pass
>         > + $cgiin{$name} = $value;
>         > + }
>         > +}
>         > +
>         >  &Header::showhttpheaders();
>         >
>         >  my @network=();
>         > @@ -43,12 +68,43 @@
>         >  my %netsettings=();
>         >  &General::readhash("${General::swroot}/ethernet/settings",
>         \%
>         > netsettings);
>         >
>         > +# output cgi query arrguments to browser on debug
>         > +if ( $debug ){
>         > + &Header::openbox('100%', 'center', 'DEBUG');
>         > + my $debugCount = 0;
>         > + foreach my $line (sort keys %cgiin) {
>         > + print "$line = '$cgiin{$line}'<br />\n";
>         > + $debugCount++;
>         > + }
>         > + print "&nbsp;Count: $debugCount\n";
>         > + &Header::closebox();
>         > +}
>         > +
>         >  #workaround to suppress a warning when a variable is used
>         only once
>         >  my @dummy = ( ${Header::table1colour} );
>         >  undef (@dummy);
>         >
>         > -# Read the connection tracking table.
>         > -open(CONNTRACK, "/usr/local/bin/getconntracktable | sort -k
>         5,5
>         > --numeric-sort --reverse |") or die "Unable to read
>         conntrack table";
>         > +
>         > +
>         > +# check sorting arguments
>         > +if ( $cgiin{'sort_field'} ~~ [ '1','2','3','4' ] ) {
>         > + $SORT_FIELD = $cgiin{'sort_field'};
>         > +
>         > + if ( $cgiin{'sort_order'} ~~ [ 'a','d','A','D' ] ) {
>         > + $SORT_ORDER = lc($cgiin{'sort_order'});
>         > + }
>         > +}
>         > +
>         > +# Read and sort the connection tracking table
>         > +# do sorting
>         > +if ($SORT_FIELD and $SORT_ORDER) {
>         > + # field sorting when sorting arguments are sane
>         > + open(CONNTRACK, "/usr/local/bin/getconntracktable
>         > | /usr/local/bin/consort.sh $SORT_FIELD $SORT_ORDER |") or
>         die "Unable
>         > to read conntrack table";
>         > +} else {
>         > + # default sorting with no query arguments
>         > + open(CONNTRACK, "/usr/local/bin/getconntracktable | sort
>         -k 5,5
>         > --numeric-sort --reverse |") or die "Unable to read
>         conntrack table";
>         > +}
>         > +
>         >  my @conntrack = <CONNTRACK>;
>         >  close(CONNTRACK);
>         >
>         > @@ -263,21 +319,49 @@
>         >   <br>
>         >  END
>         >
>         > +if ($SORT_FIELD and $SORT_ORDER) {
>         > + my @sort_field_name = (
>         > + $Lang::tr{'source ip'},
>         > + $Lang::tr{'destination ip'},
>         > + $Lang::tr{'source port'},
>         > + $Lang::tr{'destination port'}
>         > + );
>         > + my $sort_order_name;
>         > + if (lc($SORT_ORDER) eq "a") {
>         > + $sort_order_name = $Lang::tr{'sort ascending'};
>         > + } else {
>         > + $sort_order_name = $Lang::tr{'sort descending'};
>         > + }
>         > +
>         > +print <<END
>         > + <div style="font-weight:bold;margin:10px;font-size: 80%">
>         > + $sort_order_name: $sort_field_name[$SORT_FIELD-1]
>         > + </div>
>         > +END
>         > +;
>         > +}
>         > +
>         >  # Print table header.
>         >  print <<END;
>         >   <table width='100%'>
>         > - <tr>
>         > + <tr valign="top"">
>         >   <th align='center'>
>         >   $Lang::tr{'protocol'}
>         >   </th>
>         > - <th align='center'>
>         > + <th align='center' colspan="2">
>         > + <a href="?sort_field=1&sort_order=d"><img
>         style="width:10px"
>         > src="/images/up.gif"></a>
>         > + <a href="?sort_field=1&sort_order=a"><img
>         style="width:10px"
>         > src="/images/down.gif"></a>
>         >   $Lang::tr{'source ip and port'}
>         > + <a href="?sort_field=3&sort_order=d"><img
>         style="width:10px"
>         > src="/images/up.gif"></a>
>         > + <a href="?sort_field=3&sort_order=a"><img
>         style="width:10px"
>         > src="/images/down.gif"></a>
>         >   </th>
>         > - <th>&nbsp;</th>
>         > - <th align='center'>
>         > + <th align='center' colspan="2">
>         > + <a href="?sort_field=2&sort_order=d"><img
>         style="width:10px"
>         > src="/images/up.gif"></a>
>         > + <a href="?sort_field=2&sort_order=a"><img
>         style="width:10px"
>         > src="/images/down.gif"></a>
>         >   $Lang::tr{'dest ip and port'}
>         > + <a href="?sort_field=4&sort_order=d"><img
>         style="width:10px"
>         > src="/images/up.gif"></a>
>         > + <a href="?sort_field=4&sort_order=a"><img
>         style="width:10px"
>         > src="/images/down.gif"></a>
>         >   </th>
>         > - <th>&nbsp;</th>
>         >   <th align='center'>
>         >   $Lang::tr{'download'} /
>         >   <br>$Lang::tr{'upload'}
>         >
>         >
>         >
>         >
>         > 2013/3/10 Michael Tremer <michael.tremer(a)ipfire.org>
>         >         Hi,
>         >
>         >         sure, this is fine with me. Just try to make the
>         sorting
>         >         process
>         >         efficient so that even ten thousands of connections
>         are
>         >         properly
>         >         displayed.
>         >
>         >         -Michael
>         >
>         >         On Sun, 2013-03-10 at 17:01 +0100, Kay-Michael
>         Köhler wrote:
>         >         > Hello everyone
>         >         >
>         >         >
>         >         > i'm going to start development on connections.cgi
>         to have
>         >         some kind of
>         >         > sorting at "iptables connection
>         >         tracking" (status->connections)
>         >         >
>         >         >
>         >         > I think it is a good idea to have the following
>         (asc/desc)
>         >         sort
>         >         > options:
>         >         >
>         >         >
>         >         > "Protocol"
>         >         > "Source IP:Port"
>         >         > "Destination IP Port"
>         >         > "Connection status"
>         >         >
>         >         >
>         >         > If you guys agree it would be a please for me to
>         share and
>         >         post the
>         >         > patch here when i'm done.
>         >         >
>         >         >
>         >         > Regards
>         >         >
>         >         >
>         >         > Kay-Michael
>         >
>         >         > _______________________________________________
>         >         > Development mailing list
>         >         > Development(a)lists.ipfire.org
>         >         >
>         http://lists.ipfire.org/mailman/listinfo/development
>         >
>         >         _______________________________________________
>         >         Development mailing list
>         >         Development(a)lists.ipfire.org
>         >         http://lists.ipfire.org/mailman/listinfo/development
>         >
>         >
>         
>         
> 
> 
> _______________________________________________
> Development mailing list
> Development(a)lists.ipfire.org
> http://lists.ipfire.org/mailman/listinfo/development

^ permalink raw reply	[flat|nested] 5+ messages in thread