This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "IPFire 2.x development tree".

The branch, next has been updated via e698090e7f696923ff146b272b587a3eeca34c6c (commit) via 6084e66e70bc5a8f598029b075eeda1fc842fa00 (commit) via 1622e5c1f3781f1b6e370cb540ecabe17383acad (commit) from b563d5bd69c862c9ae1dd53fb1acff3d76af8761 (commit)

Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below.

- Log ----------------------------------------------------------------- commit e698090e7f696923ff146b272b587a3eeca34c6c Author: Stefan Schantl stefan.schantl@ipfire.org Date: Fri Apr 3 16:25:01 2020 +0200

IDS: Dynamically generate and import the HTTP ports.

With this commit suricata reads the HTTP port declarations from a newly introduced external file (/var/ipfire/suricata/suricata-http-ports.yaml).

This file dynamically will be generated. HTTP ports always are the default port "80" and "81" for update Accelerator and HTTP access to the WUI. In case the Web-proxy is used, the configured proxy port and/or Transparent Proxy port also will be declared as a HTTP port and written to that file.

In case one of the proxy ports will be changed, the HTTP port file will be re-generated and suricate restarted if launched. Also if an old backup with snort will be restored the convert script handles the generation of the HTTP ports file.

Finally the suricata-generate-http-ports-file as a tiny script which simply generates the http ports file and needs to be launched during the installation of a core update. (The script will no be required anymore, so it could be deleted afterwards.)

Fixes #12308.

Signed-off-by: Stefan Schantl stefan.schantl@ipfire.org Signed-off-by: Arne Fitzenreiter arne_f@ipfire.org

commit 6084e66e70bc5a8f598029b075eeda1fc842fa00 Author: Stefan Schantl stefan.schantl@ipfire.org Date: Sun Apr 5 13:03:24 2020 +0200

suricata.yaml: Re-add EVE log section.

Hopefully the EVE log will display some more content when trying to debug suricata events and rules.

Fixes #12315.

Signed-off-by: Stefan Schantl stefan.schantl@ipfire.org Acked-by: Michael Tremer michael.tremer@ipfire.org Signed-off-by: Arne Fitzenreiter arne_f@ipfire.org

commit 1622e5c1f3781f1b6e370cb540ecabe17383acad Author: Stefan Schantl stefan.schantl@ipfire.org Date: Mon Apr 6 14:34:21 2020 +0200

ids.cgi: Fix logic if suricata needs to be restarted.

Signed-off-by: Stefan Schantl stefan.schantl@ipfire.org Signed-off-by: Arne Fitzenreiter arne_f@ipfire.org

-----------------------------------------------------------------------

Summary of changes: config/cfgroot/ids-functions.pl | 51 +++++ config/suricata/convert-snort | 18 +- ...sids-file => suricata-generate-http-ports-file} | 27 +-- config/suricata/suricata.yaml | 213 ++++++++++++++++++++- html/cgi-bin/ids.cgi | 17 +- html/cgi-bin/proxy.cgi | 36 +++- 6 files changed, 329 insertions(+), 33 deletions(-) copy config/suricata/{convert-ids-modifysids-file => suricata-generate-http-ports-file} (73%)

Difference in files: diff --git a/config/cfgroot/ids-functions.pl b/config/cfgroot/ids-functions.pl index 7a2d98f03..f124b12be 100644 --- a/config/cfgroot/ids-functions.pl +++ b/config/cfgroot/ids-functions.pl @@ -37,6 +37,9 @@ our $homenet_file = "$settingsdir/suricata-homenet.yaml"; # File where the addresses of the used DNS servers are stored. our $dns_servers_file = "$settingsdir/suricata-dns-servers.yaml";

+# File where the HTTP ports definition is stored. +our $http_ports_file = "$settingsdir/suricata-http-ports.yaml"; + # File which contains the enabled sids. our $enabled_sids_file = "$settingsdir/oinkmaster-enabled-sids.conf";

@@ -89,6 +92,10 @@ my @suricatactrl_cmds = ( 'start', 'stop', 'restart', 'reload', 'fix-rules-dir', # Array with supported cron intervals. my @cron_intervals = ('off', 'daily', 'weekly' );

+# Array which contains the HTTP ports, which statically will be declared as HTTP_PORTS in the +# http_ports_file. +my @http_ports = ('80', '81'); + # ## Function to check and create all IDS related files, if the does not exist. # @@ -753,6 +760,50 @@ sub generate_dns_servers_file() { close(FILE); }

+# +# Function to generate and write the file which contains the HTTP_PORTS definition. +# +sub generate_http_ports_file() { + my %proxysettings; + + # Read-in proxy settings + &General::readhash("${General::swroot}/proxy/advanced/settings", %proxysettings); + + # Check if the proxy is enabled. + if (( -e "${General::swroot}/proxy/enable") || (-e "${General::swroot}/proxy/enable_blue")) { + # Add the proxy port to the array of HTTP ports. + push(@http_ports, $proxysettings{'PROXY_PORT'}); + } + + # Check if the transparent mode of the proxy is enabled. + if ((-e "${General::swroot}/proxy/transparent") || (-e "${General::swroot}/proxy/transparent_blue")) { + # Add the transparent proxy port to the array of HTTP ports. + push(@http_ports, $proxysettings{'TRANSPARENT_PORT'}); + } + + # Format HTTP_PORTS declaration. + my $line = ""; + + # Generate line which will be written to the http ports file. + $line = join(",", @http_ports); + + # Open file to store the HTTP_PORTS. + open(FILE, ">$http_ports_file") or die "Could not open $http_ports_file. $!\n"; + + # Print yaml header. + print FILE "%YAML 1.1\n"; + print FILE "---\n\n"; + + # Print notice about autogenerated file. + print FILE "#Autogenerated file. Any custom changes will be overwritten!\n"; + + # Print the generated HTTP_PORTS declaration to the file. + print FILE "HTTP_PORTS:\t"[$line]"\n"; + + # Close file handle. + close(FILE); +} + # ## Function to generate and write the file for used rulefiles. # diff --git a/config/suricata/convert-snort b/config/suricata/convert-snort index ee52548e9..3e938137e 100644 --- a/config/suricata/convert-snort +++ b/config/suricata/convert-snort @@ -2,7 +2,7 @@ ############################################################################### # # # IPFire.org - A linux based firewall # -# Copyright (C) 2019 IPFire Development Team info@ipfire.org # +# Copyright (C) 2020 IPFire Development Team info@ipfire.org # # # # This program is free software: you can redistribute it and/or modify # # it under the terms of the GNU General Public License as published by # @@ -298,7 +298,17 @@ if (-f $IDS::rulestarball) { &IDS::set_ownership("$IDS::dns_servers_file");

# -## Step 11: Setup automatic ruleset updates. +## Step 11: Generate file which contains the HTTP ports. +# + +# Call subfunction to generate the file. +&IDS::generate_http_ports_file(); + +# Set correct ownership for the http_ports_file. +&IDS::set_ownership("$IDS::http_ports_file"); + +# +## Step 12: Setup automatic ruleset updates. #

# Check if a ruleset is configured. @@ -308,7 +318,7 @@ if($rulessettings{"RULES"}) { }

# -## Step 12: Grab used ruleset files from snort config file and convert +## Step 13: Grab used ruleset files from snort config file and convert ## them into the new format. #

@@ -354,7 +364,7 @@ close(SNORTCONF); &IDS::write_used_rulefiles_file(@enabled_rule_files);

# -## Step 13: Start the IDS if enabled. +## Step 14: Start the IDS if enabled. #

# Check if the IDS should be started. diff --git a/config/suricata/suricata-generate-http-ports-file b/config/suricata/suricata-generate-http-ports-file new file mode 100644 index 000000000..f0d6bb823 --- /dev/null +++ b/config/suricata/suricata-generate-http-ports-file @@ -0,0 +1,47 @@ +#!/usr/bin/perl +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2012 IPFire Development Team info@ipfire.org # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see http://www.gnu.org/licenses/. # +# # +############################################################################### + +use strict; + +require '/var/ipfire/general-functions.pl'; +require "${General::swroot}/ids-functions.pl"; + +exit unless(-f $IDS::ids_settings_file and -f $IDS::rules_settings_file); + +# +## Step 1: Generate and write the HTTP ports file. +# + +# Call subfunction to generate the HTTP ports file. +&IDS::generate_http_ports_file(); + +# Set correct ownership. +&IDS::set_ownership("$IDS::http_ports_file"); + +# +## Step 2: Restart suricata if necessary. +# + +# Check if the IDS should be started. +if(&IDS::ids_is_running()) { + # Call suricatactrl and reload the rules. + &IDS::call_suricatactrl("restart"); +} diff --git a/config/suricata/suricata.yaml b/config/suricata/suricata.yaml index 54016a887..1f33ea0f3 100644 --- a/config/suricata/suricata.yaml +++ b/config/suricata/suricata.yaml @@ -30,7 +30,9 @@ vars: ENIP_SERVER: "$HOME_NET"

port-groups: - HTTP_PORTS: "[80,81]" + # Incluse HTTP_PORTS declaration from external file. + include: /var/ipfire/suricata/suricata-http-ports.yaml + SHELLCODE_PORTS: "!80" ORACLE_PORTS: 1521 SSH_PORTS: "[22,222]" @@ -90,6 +92,215 @@ outputs: threads: no # per thread stats #null-values: yes # print counters that have value 0

+ # Extensible Event Format (nicknamed EVE) event log in JSON format + - eve-log: + enabled: no + filetype: regular #regular|syslog|unix_dgram|unix_stream|redis + filename: eve.json + #prefix: "@cee: " # prefix to prepend to each log entry + # the following are valid when type: syslog above + #identity: "suricata" + #facility: local5 + #level: Info ## possible levels: Emergency, Alert, Critical, + ## Error, Warning, Notice, Info, Debug + #redis: + # server: 127.0.0.1 + # port: 6379 + # async: true ## if redis replies are read asynchronously + # mode: list ## possible values: list|lpush (default), rpush, channel|publish + # ## lpush and rpush are using a Redis list. "list" is an alias for lpush + # ## publish is using a Redis channel. "channel" is an alias for publish + # key: suricata ## key or channel to use (default to suricata) + # Redis pipelining set up. This will enable to only do a query every + # 'batch-size' events. This should lower the latency induced by network + # connection at the cost of some memory. There is no flushing implemented + # so this setting as to be reserved to high traffic suricata. + # pipelining: + # enabled: yes ## set enable to yes to enable query pipelining + # batch-size: 10 ## number of entry to keep in buffer + + # Include top level metadata. Default yes. + #metadata: no + + # include the name of the input pcap file in pcap file processing mode + pcap-file: false + + # Community Flow ID + # Adds a 'community_id' field to EVE records. These are meant to give + # a records a predictable flow id that can be used to match records to + # output of other tools such as Bro. + # + # Takes a 'seed' that needs to be same across sensors and tools + # to make the id less predictable. + + # enable/disable the community id feature. + community-id: false + # Seed value for the ID output. Valid values are 0-65535. + community-id-seed: 0 + + # HTTP X-Forwarded-For support by adding an extra field or overwriting + # the source or destination IP address (depending on flow direction) + # with the one reported in the X-Forwarded-For HTTP header. This is + # helpful when reviewing alerts for traffic that is being reverse + # or forward proxied. + xff: + enabled: no + # Two operation modes are available, "extra-data" and "overwrite". + mode: extra-data + # Two proxy deployments are supported, "reverse" and "forward". In + # a "reverse" deployment the IP address used is the last one, in a + # "forward" deployment the first IP address is used. + deployment: reverse + # Header name where the actual IP address will be reported, if more + # than one IP address is present, the last IP address will be the + # one taken into consideration. + header: X-Forwarded-For + + types: + - alert: + # payload: yes # enable dumping payload in Base64 + # payload-buffer-size: 4kb # max size of payload buffer to output in eve-log + # payload-printable: yes # enable dumping payload in printable (lossy) format + # packet: yes # enable dumping of packet (without stream segments) + # metadata: no # enable inclusion of app layer metadata with alert. Default yes + # http-body: yes # Requires metadata; enable dumping of http body in Base64 + # http-body-printable: yes # Requires metadata; enable dumping of http body in printable format + + # Enable the logging of tagged packets for rules using the + # "tag" keyword. + tagged-packets: yes + - anomaly: + # Anomaly log records describe unexpected conditions such + # as truncated packets, packets with invalid IP/UDP/TCP + # length values, and other events that render the packet + # invalid for further processing or describe unexpected + # behavior on an established stream. Networks which + # experience high occurrences of anomalies may experience + # packet processing degradation. + # + # Anomalies are reported for the following: + # 1. Decode: Values and conditions that are detected while + # decoding individual packets. This includes invalid or + # unexpected values for low-level protocol lengths as well + # as stream related events (TCP 3-way handshake issues, + # unexpected sequence number, etc). + # 2. Stream: This includes stream related events (TCP + # 3-way handshake issues, unexpected sequence number, + # etc). + # 3. Application layer: These denote application layer + # specific conditions that are unexpected, invalid or are + # unexpected given the application monitoring state. + # + # By default, anomaly logging is disabled. When anomaly + # logging is enabled, applayer anomaly reporting is + # enabled. + enabled: yes + # + # Choose one or more types of anomaly logging and whether to enable + # logging of the packet header for packet anomalies. + types: + # decode: no + # stream: no + # applayer: yes + #packethdr: no + - http: + extended: yes # enable this for extended logging information + # custom allows additional http fields to be included in eve-log + # the example below adds three additional fields when uncommented + #custom: [Accept-Encoding, Accept-Language, Authorization] + # set this value to one and only one among {both, request, response} + # to dump all http headers for every http request and/or response + # dump-all-headers: none + - dns: + # This configuration uses the new DNS logging format, + # the old configuration is still available: + # https://suricata.readthedocs.io/en/latest/output/eve/eve-json-output.html#dn... + + # As of Suricata 5.0, version 2 of the eve dns output + # format is the default. + #version: 2 + + # Enable/disable this logger. Default: enabled. + #enabled: yes + + # Control logging of requests and responses: + # - requests: enable logging of DNS queries + # - responses: enable logging of DNS answers + # By default both requests and responses are logged. + #requests: no + #responses: no + + # Format of answer logging: + # - detailed: array item per answer + # - grouped: answers aggregated by type + # Default: all + #formats: [detailed, grouped] + + # Types to log, based on the query type. + # Default: all. + #types: [a, aaaa, cname, mx, ns, ptr, txt] + - tls: + extended: yes # enable this for extended logging information + # output TLS transaction where the session is resumed using a + # session id + #session-resumption: no + # custom allows to control which tls fields that are included + # in eve-log + #custom: [subject, issuer, session_resumed, serial, fingerprint, sni, version, not_before, not_after, certificate, chain, ja3, ja3s] + - files: + force-magic: no # force logging magic on all logged files + # force logging of checksums, available hash functions are md5, + # sha1 and sha256 + #force-hash: [md5] + #- drop: + # alerts: yes # log alerts that caused drops + # flows: all # start or all: 'start' logs only a single drop + # # per flow direction. All logs each dropped pkt. + - smtp: + #extended: yes # enable this for extended logging information + # this includes: bcc, message-id, subject, x_mailer, user-agent + # custom fields logging from the list: + # reply-to, bcc, message-id, subject, x-mailer, user-agent, received, + # x-originating-ip, in-reply-to, references, importance, priority, + # sensitivity, organization, content-md5, date + #custom: [received, x-mailer, x-originating-ip, relays, reply-to, bcc] + # output md5 of fields: body, subject + # for the body you need to set app-layer.protocols.smtp.mime.body-md5 + # to yes + #md5: [body, subject] + + #- dnp3 + - ftp + #- rdp + - nfs + - smb + - tftp + - ikev2 + - krb5 + - snmp + #- sip + - dhcp: + enabled: yes + # When extended mode is on, all DHCP messages are logged + # with full detail. When extended mode is off (the + # default), just enough information to map a MAC address + # to an IP address is logged. + extended: no + - ssh + - stats: + totals: yes # stats for all threads merged together + threads: no # per thread stats + deltas: no # include delta values + # bi-directional flows + - flow + # uni-directional flows + #- netflow + + # Metadata event type. Triggered whenever a pktvar is saved + # and will include the pktvars, flowvars, flowbits and + # flowints. + #- metadata + logging: # The default log level, can be overridden in an output section. # Note that debug level logging will only be emitted if Suricata was diff --git a/html/cgi-bin/ids.cgi b/html/cgi-bin/ids.cgi index c3e5eefdb..bc31a341f 100644 --- a/html/cgi-bin/ids.cgi +++ b/html/cgi-bin/ids.cgi @@ -2,7 +2,7 @@ ############################################################################### # # # IPFire.org - A linux based firewall # -# Copyright (C) 2007-2018 IPFire Team info@ipfire.org # +# Copyright (C) 2007-2020 IPFire Team info@ipfire.org # # # # This program is free software: you can redistribute it and/or modify # # it under the terms of the GNU General Public License as published by # @@ -417,17 +417,17 @@ if ($cgiparams{'RULESET'} eq $Lang::tr{'save'}) {

# Loop through the hash of idsrules. foreach my $rulefile(keys %idsrules) { + # Check if the state of the rulefile has been changed. + unless ($cgiparams{$rulefile} eq $idsrules{$rulefile}{'Rulefile'}{'State'}) { + # A restart of suricata is required to apply the changes of the used rulefiles. + $suricata_restart_required = 1; + } + # Check if the rulefile is enabled. if ($cgiparams{$rulefile} eq "on") { # Add rulefile to the array of enabled rulefiles. push(@enabled_rulefiles, $rulefile);

- # Check if the state of the rulefile has been changed. - unless ($cgiparams{$rulefile} eq $idsrules{$rulefile}{'Rulefile'}{'State'}) { - # A restart of suricata is required to apply the changes of the used rulefiles. - $suricata_restart_required = 1; - } - # Drop item from cgiparams hash. delete $cgiparams{$rulefile}; } @@ -625,6 +625,9 @@ if ($cgiparams{'RULESET'} eq $Lang::tr{'save'}) { # Generate file to the store the DNS servers. &IDS::generate_dns_servers_file();

+ # Generate file to store the HTTP ports. + &IDS::generate_http_ports_file(); + # Write the modify sid's file and pass the taken ruleaction. &IDS::write_modify_sids_file();

diff --git a/html/cgi-bin/proxy.cgi b/html/cgi-bin/proxy.cgi index 06aca579b..73646a5ae 100644 --- a/html/cgi-bin/proxy.cgi +++ b/html/cgi-bin/proxy.cgi @@ -2,7 +2,7 @@ ############################################################################### # # # IPFire.org - A linux based firewall # -# Copyright (C) 2007-2013 IPFire Team info@ipfire.org # +# Copyright (C) 2007-2020 IPFire Team info@ipfire.org # # # # This program is free software: you can redistribute it and/or modify # # it under the terms of the GNU General Public License as published by # @@ -37,6 +37,8 @@ require '/var/ipfire/general-functions.pl'; require "${General::swroot}/lang.pl"; require "${General::swroot}/header.pl";

+require "${General::swroot}/ids-functions.pl"; + my @squidversion = `/usr/sbin/squid -v`; my $http_port='81'; my $https_port='444'; @@ -550,6 +552,29 @@ ERROR:

if ($proxysettings{'VALID'} eq 'yes') { + # Determine if suricata may needs to be restarted. + my $suricata_proxy_ports_changed; + + # Check if the IDS is running + if(&IDS::ids_is_running()) { + my %oldproxysettings; + + # Read-in current proxy settings and store them as oldsettings hash. + &General::readhash("${General::swroot}/proxy/advanced/settings", %oldproxysettings); + + # Check if the proxy port has been changed. + unless ($proxysettings{'PROXY_PORT'} eq $oldproxysettings{'PROXY_PORT'}) { + # Port has changed, suricata needs to be adjusted. + $suricata_proxy_ports_changed = 1; + } + + # Check if the transparent port has been changed. + unless ($proxysettings{'TRANSPARENT_PORT'} eq $oldproxysettings{'TRANSPARENT_PORT'}) { + # Transparent port has changed, suricata needs to be adjusted. + $suricata_proxy_ports_changed = 1; + } + } + &write_acls;

delete $proxysettings{'SRC_SUBNETS'}; @@ -627,6 +652,15 @@ ERROR:

if ($proxysettings{'ACTION'} eq $Lang::tr{'advproxy save and restart'}) { system('/usr/local/bin/squidctrl restart >/dev/null 2>&1'); } if ($proxysettings{'ACTION'} eq $Lang::tr{'proxy reconfigure'}) { system('/usr/local/bin/squidctrl reconfigure >/dev/null 2>&1'); } + + # Check if the suricata_proxy_ports_changed flag has been set. + if ($suricata_proxy_ports_changed) { + # Re-generate HTTP ports file. + &IDS::generate_http_ports_file(); + + # Restart suricata. + &IDS::call_suricatactrl("restart"); + } } }

hooks/post-receive -- IPFire 2.x development tree