From mboxrd@z Thu Jan 1 00:00:00 1970 From: Matthias Fischer To: development@lists.ipfire.org Subject: Re: Betatest Guardian 2.0 Date: Tue, 26 Jul 2016 20:31:03 +0200 Message-ID: <575c8e84-ed8e-c139-e82b-02bfe919b298@ipfire.org> In-Reply-To: <1469545859.2710.303.camel@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0151747868047348539==" List-Id: --===============0151747868047348539== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi, On 26.07.2016 17:10, Michael Tremer wrote: > Did anyone try to monitor the size of the log files that guardian is parsin= g as > well? Could it be that every line that is read remains in memory? >=20 > This is just an idea... Could be, but I'm not so firm with such behaviour. I'm using 'syslog' and memory raises, see below. Some things I found in the meantime while playing around: '/etc/init.d/guardian' needs a 'sleep'-command for restart-option. Otherwise we get a warning that '/usr/sbin/guardian' is still running: ... root(a)ipfire: ~ # /etc/init.d/guardian restart Stopping Guardian... Starting Guardian... Unable to continue: /usr/sbin/guardian is running [ WARN ] ... After adding 'sleep 2' between '$0 stop' and '$0 start' in '/etc/init.d/guardian', warning is gone: ... restart) $0 stop sleep 2 $0 start ... Output: root(a)ipfire: /etc/init.d # /etc/init.d/guardian restart Stopping Guardian... Starting Guardian... [ OK ] ########## Each saving through GUI alters memory usage of 'guardian' process. Example (logging to 'syslog'!). While switching (e.g.) 'Loglevel' from '2' to '3' and back again, each saving alters memory usage for about 9 MB (see my former message above, 21.7.2016/11:07pm). I stopped at ~56289 MB. After stopping and starting 'guardian' process is at ~14334 MB again. If you do nothing, it stays there. ########## Saving firewall rules changes sometimes 'pstree'-output for 'guardian': Before: root(a)ipfire: /etc/init.d # pstree init-+-acpid |-6*[agetty] |-clamd---{clamd} |-collectd---3*[{collectd}] |-dhcpd |-dnsmasq |-fcron |-freshclam |-guardian---4*[{guardian}] |-httpd---10*[httpd] |-klogd |-privoxy---11*[{privoxy}] |-saslauthd---saslauthd |-snort---{snort} |-squid---squid-+-16*[redirect_wrappe-+-squidGuard] | | `-squidclamav] | `-16*[{squid}] |-sshd---bash---pstree |-syslogd `-udevd As you see, output for'guardian' is: ... |-guardian---4*[{guardian}] ... Today, after activating/deactivating one firewall rule and clicking 'Apply changes': root(a)ipfire: ~ # pstree init-+-acpid |-6*[agetty] |-clamd---2*[{clamd}] |-collectd---3*[{collectd}] |-dhcpd |-dnsmasq |-fcron |-freshclam |-guardian-+-iptables | `-4*[{guardian}] |-httpd---10*[httpd] |-klogd |-privoxy |-saslauthd---saslauthd |-snort---{snort} |-squid---squid-+-redirect_wrappe-+-squidGuard | | `-squidclamav | `-16*[{squid}] |-sshd---bash---pstree |-syslogd `-udevd Suddenly its says: ... |-guardian-+-iptables | `-4*[{guardian}] ... I don't know why, perhaps someone has an idea what happened here? Best, Matthias > Best, > -Michael >=20 > On Sat, 2016-07-23 at 00:23 +0200, Matthias Fischer wrote: >> Correction: in the meanwhile it jumped to 47890 KB, I don't know why. >> Logrotation?. >>=20 >> On 22.07.2016 22:28, Matthias Fischer wrote: >> > Hi, >> >=20 >> > ...for the records...: >> >=20 >> > Since I switched "Loglevel" to OFF, memory usage stays at "14333 KB" and >> > didn't change/rise since then. >> >=20 >> > HTH, >> > Matthias >> >=20 >> > On 21.07.2016 23:07, Matthias Fischer wrote: >> > > Hi, >> > >=20 >> > > Sounds interesting. >> > >=20 >> > > So I thought I take a little test... >> > >=20 >> > > Initial RAM-Usage: 14334 KB >> > >=20 >> > > First I just switched logging, did nothing else: >> > >=20 >> > > syslog =3D> file =3D> 22726 KB >> > > file =3D> syslog =3D> 31117 KB >> > > syslog =3D> file =3D> 39507/47898 KB (RAM suddenly altered. Why? No id= ea.) >> > > file =3D> syslog =3D> 56289 KB >> > >=20 >> > > Restarted through console: >> > >=20 >> > > root(a)ipfire: /var/log/guardian # guardianctrl restart >> > > Stopping Guardian... >> > > Starting Guardian... >> > > Unable to continue: /usr/sbin/guardian is running >> > > [ WARN ] >> > >=20 >> > > Hm? >> > >=20 >> > > Stopped through console, no output, 'guardian' not found anymore, >> > > neither in GUI nor through console: >> > >=20 >> > > root(a)ipfire: /var/log/guardian # ps ax | grep guardian >> > > 6962 pts/1 S+ 0:00 grep guardian >> > >=20 >> > > Started through console and we're exactly where we started (14334 KB). >> > >=20 >> > > The same happens if I switch the 'Priority-level' or the 'Firewall- >> > > Action'. >> > >=20 >> > > Initial: 2 >> > > 2 =3D> 3 =3D> 22723 KB >> > > 3 =3D> 2 =3D> 31112 KB >> > >=20 >> > > Firewall-Action: >> > > Reject =3D> Drop =3D> 39501 KB >> > >=20 >> > > Stop =3D> Start =3D> 14334 KB >> > >=20 >> > > Interestingly, during MY (log-)switching, 'guardian' never stopped. >> > >=20 >> > > HTH, >> > > Matthias >> > >=20 >> > > On 21.07.2016 21:52, Flying Trashcan wrote: >> > > > I am now noticing that when I switch from Log facility =E2=80=9Cfile= =E2=80=9D to >> > > > =E2=80=9Csyslog=E2=80=9D, Guardian Daemon stops and doesn=E2=80=99t = restart. Switching from >> > > > syslog to file didn=E2=80=99t stop the service, only switching back = to syslog >> > > > from file. I can manually start the service and be back to normal. = Not >> > > > a big deal, but if someone made the switch and didn=E2=80=99t think = to manually >> > > > start the service, it could be left without running Guardian. >> > > >=20 >> > > >=20 >> > > >=20 >> > > >=20 >> > > > > On Jul 21, 2016, at 4:25 AM, Matthias Fischer > > > > > .org> wrote: >> > > > >=20 >> > > > > Hi, >> > > > >=20 >> > > > > I mentioned this earlier, but it seems that 'guardian' has some ki= nd >> > > > > of >> > > > > memory leak? >> > > > >=20 >> > > > > It started about two days ago with ~14 MB RAM. Then it jumped to ~= 34 >> > > > > MB, >> > > > > then to ~48 MB - today it suddenly uses 71 MB. >> > > > >=20 >> > > > > And if I start it on my testmachine (offline!) it uses ~90 MB. >> > > > >=20 >> > > > > Can someone confirm? >> > > > >=20 >> > > > > Besides this, its working without seen problems. >> > > > >=20 >> > > > > Best, >> > > > > Matthias >> > > > >=20 >> > > > > On 20.07.2016 15:33, Stefan Schantl wrote: >> > > > > > Hello testers, >> > > > > >=20 >> > > > > > I've uploaded a new test version (003). >> > > > > >=20 >> > > > > > Update or fresh install works like described in the announcement >> > > > > > mail. >> > > > > >=20 >> > > > > > The Changelog can be found here: >> > > > > >=20 >> > > > > > http://people.ipfire.org/~stevee/guardian-2.0/Changelog.txt >> > > > > >=20 >> > > > > > At the moment I'm missing feedback for the following functions: >> > > > > >=20 >> > > > > > * Manually blocking / unblocking addresses. >> > > > > > * Dealing with the ignore list. >> > > > > > * Owncloud message parser. >> > > > > > * Logrotate, there should be an corresponding log entry in the >> > > > > > guardian >> > > > > > logfile after rotation of the logfiles have been done. >> > > > > > * Reload of the ignore list after "Red" has been reconnected. Th= ere >> > > > > > also a corresponding log entry should be logged to the logfile a= nd >> > > > > > the >> > > > > > new "Red-address" should also be logged as part of the ignore li= st >> > > > > > (If >> > > > > > you own an dynamic assigned one). >> > > > > >=20 >> > > > > > As always please report your bugs or experience with the new ver= sion >> > > > > > to >> > > > > > this list. >> > > > > >=20 >> > > > > > Best regards, >> > > > > >=20 >> > > > > > -Stefan >> > > > > >=20 >> > > > > > > Hello mailing list followers, >> > > > > > >=20 >> > > > > > > this is the official release announcement for the first beta >> > > > > > > release >> > > > > > > of >> > > > > > > the new Guardian 2.0 approach. >> > > > > > >=20 >> > > > > > >=20 >> > > > > > > - What are the differences to the current version of guardian >> > > > > > > (legacy) >> > > > > > > and the first approach of guardian 2.0? >> > > > > > >=20 >> > > > > > > The most important difference is, that the new version of Guar= dian >> > > > > > > 2.0 >> > > > > > > completely has been re-written from scratch and released under= the >> > > > > > > terms of the GPLv3. The legacy version of guardian is not >> > > > > > > maintained >> > > > > > > anymore by it's developer and the software has been released >> > > > > > > without >> > > > > > > any license details at all. >> > > > > > >=20 >> > > > > > > Guardian 2.0 has a very modular code base and has been designe= d as >> > > > > > > a >> > > > > > > multi-threaded application. This allows a parallel parsing of = all >> > > > > > > monitored logfiles and faster actions, if one of the used modu= les >> > > > > > > detects an attack. >> > > > > > >=20 >> > > > > > > A very important difference to the legacy version is the suppo= rt >> > > > > > > of >> > > > > > > configuring and managing the entire service through the IPFire >> > > > > > > webinterface. The entire configuration, managing of current >> > > > > > > blocked >> > > > > > > hosts, unblocking them or editing the ignored hosts list now c= an >> > > > > > > be >> > > > > > > done in a graphical way.=20 >> > > > > > >=20 >> > > > > > > The legacy version of guardian only supported parsing snort >> > > > > > > alerts. >> > > > > > > HTTPD and SSH support has been patched by the IPFire developme= nt >> > > > > > > team >> > > > > > > some time ago. Guardian 2.0 supports all of them out of the box >> > > > > > > and >> > > > > > > includes a filter to detect owncloud login brute-force attempt= s. >> > > > > > > As a >> > > > > > > benefit of the new modular design, additional filters easily c= an >> > > > > > > be >> > > > > > > added. >> > > > > > >=20 >> > > > > > > Guardian 2.0 is able to reload it's configuration, reloading >> > > > > > > the ignore list during runtime and handle, if the logfiles will >> > > > > > > get >> > > > > > > rotated by logrotate. This actions can be called by using the >> > > > > > > webinterface or from the command line interface by using >> > > > > > > "guardianctrl". >> > > > > > >=20 >> > > > > > > These are just a handful of the changes and benefits which com= es >> > > > > > > with >> > > > > > > Guardian 2.0, a complete list would be to long for this mailing >> > > > > > > list. >> > > > > > >=20 >> > > > > > >=20 >> > > > > > > - How to join testing? >> > > > > > >=20 >> > > > > > > To get part of the testing team, simple navigate to http://peo= ple. >> > > > > > > ipf >> > > > > > > ir >> > > > > > > e.org/~stevee/guardian-2.0/ and download the latest tarball >> > > > > > > (currently >> > > > > > > 002). Please take care to download the correct one, based on y= our >> > > > > > > used >> > > > > > > architecture. The i585 packages are for 32Bit installations of >> > > > > > > IPFire, >> > > > > > > the x86_64 packages only can be used on 64Bit installations. >> > > > > > >=20 >> > > > > > > Put the downloaded file on your IPFire test system and extract= the >> > > > > > > package by using "tar -xvf guardian-2.0-002..tar.gz -C /= ". >> > > > > > >=20 >> > > > > > > The final installation step would be to regenerate the language >> > > > > > > cache >> > > > > > > by executing "update-lang-cache" on the console. >> > > > > > >=20 >> > > > > > > From now you can find a new menu item called "Guardian" in your >> > > > > > > "Service" menu after you have logged-in into your IPFire's >> > > > > > > webinterface. >> > > > > > >=20 >> > > > > > > Documentation can be found on the IPFire wiki: http://wiki.ipf= ire. >> > > > > > > org >> > > > > > > /e >> > > > > > > n/addons/guardian/start#the_guardian_20_addon >> > > > > > >=20 >> > > > > > >=20 >> > > > > > > - Where to post bugs reports or provide feedback? >> > > > > > >=20 >> > > > > > > If you find any bugs, please report them as usual on the IPFire >> > > > > > > bugtracker, which can be found at https://bugzilla.ipfire.org. >> > > > > > >=20 >> > > > > > > To provide feedback or to join a discussion, please send your >> > > > > > > mails >> > > > > > > to >> > > > > > > "development(a)lists.ipfire.org" (Please register first at htt= p://li >> > > > > > > sts >> > > > > > > .i >> > > > > > > pfire.org if not yet done). >> > > > > > >=20 >> > > > > > > The source code can be found at http://git.ipfire.org/?p=3Dpeo= ple/st >> > > > > > > eve >> > > > > > > e/ >> > > > > > > guardian.git;a=3Dsummary >> > > > > > >=20 >> > > > > > >=20 >> > > > > > > Happy testing, >> > > > > > >=20 >> > > > > > > -Stefan >> > > > > > >=20 >> > > > > >=20 >> > > > >=20 >> > > >=20 >> > > >=20 >> > >=20 >> > >=20 >> >=20 >> >=20 >>=20 >=20 --===============0151747868047348539==--