Re: Betatest Guardian 2.0

[Stefan Schantl <stefan.schantl@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 15578 bytes --]

Hello Matthias,

thanks for the hint - changed.

Best regards,

-Stefan
> Hi,
> 
> On 26.07.2016 17:10, Michael Tremer wrote:
> > 
> > Did anyone try to monitor the size of the log files that guardian
> > is parsing as
> > well? Could it be that every line that is read remains in memory?
> > 
> > This is just an idea...
> Could be, but I'm not so firm with such behaviour. I'm using 'syslog'
> and memory raises, see below.
> 
> Some things I found in the meantime while playing around:
> 
> '/etc/init.d/guardian' needs a 'sleep'-command for restart-option.
> Otherwise we get a warning that '/usr/sbin/guardian' is still
> running:
> 
> ...
> root(a)ipfire: ~ # /etc/init.d/guardian restart
> Stopping Guardian...
> Starting Guardian...
> Unable to continue: /usr/sbin/guardian is running
>                   [ WARN ]
> ...
> 
> After adding 'sleep 2' between '$0 stop' and '$0 start' in
> '/etc/init.d/guardian', warning is gone:
> 
> ...
>         restart)
>                 $0 stop
>                 sleep 2
>                 $0 start
> ...
> 
> Output:
> 
> root(a)ipfire: /etc/init.d # /etc/init.d/guardian restart
> Stopping Guardian...
> Starting Guardian...
>                   [  OK  ]
> 
> ##########
> 
> Each saving through GUI alters memory usage of 'guardian' process.
> 
> Example (logging to 'syslog'!).
> While switching (e.g.) 'Loglevel' from '2' to '3' and back again,
> each
> saving alters memory usage for about 9 MB (see my former message
> above,
> 21.7.2016/11:07pm). I stopped at ~56289 MB.
> 
> After stopping and starting 'guardian' process is at ~14334 MB again.
> 
> If you do nothing, it stays there.
> 
> ##########
> 
> Saving firewall rules changes sometimes 'pstree'-output for
> 'guardian':
> 
> Before:
> 
> root(a)ipfire: /etc/init.d # pstree
> init-+-acpid
>      |-6*[agetty]
>      |-clamd---{clamd}
>      |-collectd---3*[{collectd}]
>      |-dhcpd
>      |-dnsmasq
>      |-fcron
>      |-freshclam
>      |-guardian---4*[{guardian}]
>      |-httpd---10*[httpd]
>      |-klogd
>      |-privoxy---11*[{privoxy}]
>      |-saslauthd---saslauthd
>      |-snort---{snort}
>      |-squid---squid-+-16*[redirect_wrappe-+-squidGuard]
>      |               |                     `-squidclamav]
>      |               `-16*[{squid}]
>      |-sshd---bash---pstree
>      |-syslogd
>      `-udevd
> 
> As you see, output for'guardian' is:
> 
> ...
>      |-guardian---4*[{guardian}]
> 
> ...
> 
> Today, after activating/deactivating one firewall rule and clicking
> 'Apply changes':
> 
> root(a)ipfire: ~ # pstree
> init-+-acpid
>      |-6*[agetty]
>      |-clamd---2*[{clamd}]
>      |-collectd---3*[{collectd}]
>      |-dhcpd
>      |-dnsmasq
>      |-fcron
>      |-freshclam
>      |-guardian-+-iptables
>      |          `-4*[{guardian}]
>      |-httpd---10*[httpd]
>      |-klogd
>      |-privoxy
>      |-saslauthd---saslauthd
>      |-snort---{snort}
>      |-squid---squid-+-redirect_wrappe-+-squidGuard
>      |               |                 `-squidclamav
>      |               `-16*[{squid}]
>      |-sshd---bash---pstree
>      |-syslogd
>      `-udevd
> 
> Suddenly its says:
> 
> ...
>      |-guardian-+-iptables
>      |          `-4*[{guardian}]
> ...
> 
> I don't know why, perhaps someone has an idea what happened here?
> 
> Best,
> Matthias
> 
> > 
> > Best,
> > -Michael
> > 
> > On Sat, 2016-07-23 at 00:23 +0200, Matthias Fischer wrote:
> > > 
> > > Correction: in the meanwhile it jumped to 47890 KB, I don't know
> > > why.
> > > Logrotation?.
> > > 
> > > On 22.07.2016 22:28, Matthias Fischer wrote:
> > > > 
> > > > Hi,
> > > > 
> > > > ...for the records...:
> > > > 
> > > > Since I switched "Loglevel" to OFF, memory usage stays at
> > > > "14333 KB" and
> > > > didn't change/rise since then.
> > > > 
> > > > HTH,
> > > > Matthias
> > > > 
> > > > On 21.07.2016 23:07, Matthias Fischer wrote:
> > > > > 
> > > > > Hi,
> > > > > 
> > > > > Sounds interesting.
> > > > > 
> > > > > So I thought I take a little test...
> > > > > 
> > > > > Initial RAM-Usage: 14334 KB
> > > > > 
> > > > > First I just switched logging, did nothing else:
> > > > > 
> > > > > syslog => file => 22726 KB
> > > > > file => syslog => 31117 KB
> > > > > syslog => file => 39507/47898 KB (RAM suddenly altered. Why?
> > > > > No idea.)
> > > > > file => syslog => 56289 KB
> > > > > 
> > > > > Restarted through console:
> > > > > 
> > > > > root(a)ipfire: /var/log/guardian # guardianctrl restart
> > > > > Stopping Guardian...
> > > > > Starting Guardian...
> > > > > Unable to continue: /usr/sbin/guardian is running
> > > > >                   [ WARN ]
> > > > > 
> > > > > Hm?
> > > > > 
> > > > > Stopped through console, no output, 'guardian' not found
> > > > > anymore,
> > > > > neither in GUI nor through console:
> > > > > 
> > > > > root(a)ipfire: /var/log/guardian # ps ax | grep guardian
> > > > >  6962 pts/1    S+     0:00 grep guardian
> > > > > 
> > > > > Started through console and we're exactly where we started
> > > > > (14334 KB).
> > > > > 
> > > > > The same happens if I switch the 'Priority-level' or the
> > > > > 'Firewall-
> > > > > Action'.
> > > > > 
> > > > > Initial: 2
> > > > > 2 => 3 => 22723 KB
> > > > > 3 => 2 => 31112 KB
> > > > > 
> > > > > Firewall-Action:
> > > > > Reject => Drop => 39501 KB
> > > > > 
> > > > > Stop => Start => 14334 KB
> > > > > 
> > > > > Interestingly, during MY (log-)switching, 'guardian' never
> > > > > stopped.
> > > > > 
> > > > > HTH,
> > > > > Matthias
> > > > > 
> > > > > On 21.07.2016 21:52, Flying Trashcan wrote:
> > > > > > 
> > > > > > I am now noticing that when I switch from Log facility
> > > > > > “file” to
> > > > > > “syslog”, Guardian Daemon stops and doesn’t
> > > > > > restart.  Switching from
> > > > > > syslog to file didn’t stop the service, only switching back
> > > > > > to syslog
> > > > > > from file.  I can manually start the service and be back to
> > > > > > normal.  Not
> > > > > > a big deal, but if someone made the switch and didn’t think
> > > > > > to manually
> > > > > > start the service, it could be left without running
> > > > > > Guardian.
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > On Jul 21, 2016, at 4:25 AM, Matthias Fischer <matthias.f
> > > > > > > ischer(a)ipfire
> > > > > > > .org> wrote:
> > > > > > > 
> > > > > > > Hi,
> > > > > > > 
> > > > > > > I mentioned this earlier, but it seems that 'guardian'
> > > > > > > has some kind
> > > > > > > of
> > > > > > > memory leak?
> > > > > > > 
> > > > > > > It started about two days ago with ~14 MB RAM. Then it
> > > > > > > jumped to ~34
> > > > > > > MB,
> > > > > > > then to ~48 MB - today it suddenly uses 71 MB.
> > > > > > > 
> > > > > > > And if I start it on my testmachine (offline!) it uses
> > > > > > > ~90 MB.
> > > > > > > 
> > > > > > > Can someone confirm?
> > > > > > > 
> > > > > > > Besides this, its working without seen problems.
> > > > > > > 
> > > > > > > Best,
> > > > > > > Matthias
> > > > > > > 
> > > > > > > On 20.07.2016 15:33, Stefan Schantl wrote:
> > > > > > > > 
> > > > > > > > Hello testers,
> > > > > > > > 
> > > > > > > > I've uploaded  a new test version (003).
> > > > > > > > 
> > > > > > > > Update or fresh install works like described in the
> > > > > > > > announcement
> > > > > > > > mail.
> > > > > > > > 
> > > > > > > > The Changelog can be found here:
> > > > > > > > 
> > > > > > > > http://people.ipfire.org/~stevee/guardian-2.0/Changelog
> > > > > > > > .txt
> > > > > > > > 
> > > > > > > > At the moment I'm missing feedback for the following
> > > > > > > > functions:
> > > > > > > > 
> > > > > > > > * Manually blocking / unblocking addresses.
> > > > > > > > * Dealing with the ignore list.
> > > > > > > > * Owncloud message parser.
> > > > > > > > * Logrotate, there should be an corresponding log entry
> > > > > > > > in the
> > > > > > > > guardian
> > > > > > > > logfile after rotation of the logfiles have been done.
> > > > > > > > * Reload of the ignore list after "Red" has been
> > > > > > > > reconnected. There
> > > > > > > > also a corresponding log entry should be logged to the
> > > > > > > > logfile and
> > > > > > > > the
> > > > > > > > new "Red-address" should also be logged as part of the
> > > > > > > > ignore list
> > > > > > > > (If
> > > > > > > > you own an dynamic assigned one).
> > > > > > > > 
> > > > > > > > As always please report your bugs or experience with
> > > > > > > > the new version
> > > > > > > > to
> > > > > > > > this list.
> > > > > > > > 
> > > > > > > > Best regards,
> > > > > > > > 
> > > > > > > > -Stefan
> > > > > > > > 
> > > > > > > > > 
> > > > > > > > > Hello mailing list followers,
> > > > > > > > > 
> > > > > > > > > this is the official release announcement for the
> > > > > > > > > first beta
> > > > > > > > > release
> > > > > > > > > of
> > > > > > > > > the new Guardian 2.0 approach.
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > - What are the differences to the current version of
> > > > > > > > > guardian
> > > > > > > > > (legacy)
> > > > > > > > > and the first approach of guardian 2.0?
> > > > > > > > > 
> > > > > > > > > The most important difference is, that the new
> > > > > > > > > version of Guardian
> > > > > > > > > 2.0
> > > > > > > > > completely has been re-written from scratch and
> > > > > > > > > released under the
> > > > > > > > > terms of the GPLv3. The legacy version of guardian is
> > > > > > > > > not
> > > > > > > > > maintained
> > > > > > > > > anymore by it's developer and the software has been
> > > > > > > > > released
> > > > > > > > > without
> > > > > > > > > any license details at all.
> > > > > > > > > 
> > > > > > > > > Guardian 2.0 has a very modular code base and has
> > > > > > > > > been designed as
> > > > > > > > > a
> > > > > > > > > multi-threaded application. This allows a parallel
> > > > > > > > > parsing of all
> > > > > > > > > monitored logfiles and faster actions, if one of the
> > > > > > > > > used modules
> > > > > > > > > detects an attack.
> > > > > > > > > 
> > > > > > > > > A very important difference to the legacy version is
> > > > > > > > > the support
> > > > > > > > > of
> > > > > > > > > configuring and managing the entire service through
> > > > > > > > > the IPFire
> > > > > > > > > webinterface. The entire configuration, managing of
> > > > > > > > > current
> > > > > > > > > blocked
> > > > > > > > > hosts, unblocking them or editing the ignored hosts
> > > > > > > > > list now can
> > > > > > > > > be
> > > > > > > > > done in a graphical way. 
> > > > > > > > > 
> > > > > > > > > The legacy version of guardian only supported parsing
> > > > > > > > > snort
> > > > > > > > > alerts.
> > > > > > > > > HTTPD and SSH support has been patched by the IPFire
> > > > > > > > > development
> > > > > > > > > team
> > > > > > > > > some time ago. Guardian 2.0 supports all of them out
> > > > > > > > > of the box
> > > > > > > > > and
> > > > > > > > > includes a filter to detect owncloud login brute-
> > > > > > > > > force attempts.
> > > > > > > > > As a
> > > > > > > > > benefit of the new modular design, additional filters
> > > > > > > > > easily can
> > > > > > > > > be
> > > > > > > > > added.
> > > > > > > > > 
> > > > > > > > > Guardian 2.0 is able to reload it's configuration,
> > > > > > > > > reloading
> > > > > > > > > the ignore list during runtime and handle, if the
> > > > > > > > > logfiles will
> > > > > > > > > get
> > > > > > > > > rotated by logrotate. This actions can be called by
> > > > > > > > > using the
> > > > > > > > > webinterface or from the command line interface by
> > > > > > > > > using
> > > > > > > > > "guardianctrl".
> > > > > > > > > 
> > > > > > > > > These are just a handful of the changes and benefits
> > > > > > > > > which comes
> > > > > > > > > with
> > > > > > > > > Guardian 2.0, a complete list would be to long for
> > > > > > > > > this mailing
> > > > > > > > > list.
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > - How to join testing?
> > > > > > > > > 
> > > > > > > > > To get part of the testing team, simple navigate to h
> > > > > > > > > ttp://people.
> > > > > > > > > ipf
> > > > > > > > > ir
> > > > > > > > > e.org/~stevee/guardian-2.0/ and download the latest
> > > > > > > > > tarball
> > > > > > > > > (currently
> > > > > > > > > 002). Please take care to download the correct one,
> > > > > > > > > based on your
> > > > > > > > > used
> > > > > > > > > architecture. The i585 packages are for 32Bit
> > > > > > > > > installations of
> > > > > > > > > IPFire,
> > > > > > > > > the x86_64 packages only can be used on 64Bit
> > > > > > > > > installations.
> > > > > > > > > 
> > > > > > > > > Put the downloaded file on your IPFire test system
> > > > > > > > > and extract the
> > > > > > > > > package by using "tar -xvf guardian-2.0-
> > > > > > > > > 002.<arch>.tar.gz -C /".
> > > > > > > > > 
> > > > > > > > > The final installation step would be to regenerate
> > > > > > > > > the language
> > > > > > > > > cache
> > > > > > > > > by executing "update-lang-cache" on the console.
> > > > > > > > > 
> > > > > > > > > From now you can find a new menu item called
> > > > > > > > > "Guardian" in your
> > > > > > > > > "Service" menu after you have logged-in into your
> > > > > > > > > IPFire's
> > > > > > > > > webinterface.
> > > > > > > > > 
> > > > > > > > > Documentation can be found on the IPFire wiki: http:/
> > > > > > > > > /wiki.ipfire.
> > > > > > > > > org
> > > > > > > > > /e
> > > > > > > > > n/addons/guardian/start#the_guardian_20_addon
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > - Where to post bugs reports or provide feedback?
> > > > > > > > > 
> > > > > > > > > If you find any bugs, please report them as usual on
> > > > > > > > > the IPFire
> > > > > > > > > bugtracker, which can be found at https://bugzilla.ip
> > > > > > > > > fire.org.
> > > > > > > > > 
> > > > > > > > > To provide feedback or to join a discussion, please
> > > > > > > > > send your
> > > > > > > > > mails
> > > > > > > > > to
> > > > > > > > > "development(a)lists.ipfire.org" (Please register first
> > > > > > > > > at http://li
> > > > > > > > > sts
> > > > > > > > > .i
> > > > > > > > > pfire.org if not yet done).
> > > > > > > > > 
> > > > > > > > > The source code can be found at http://git.ipfire.org
> > > > > > > > > /?p=people/st
> > > > > > > > > eve
> > > > > > > > > e/
> > > > > > > > > guardian.git;a=summary
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > Happy testing,
> > > > > > > > > 
> > > > > > > > > -Stefan
> > > > > > > > > 
> > > > > > 
> > > > > 
> > > > 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]