Testing Core Update 118

Testing Core Update 118

[Peter Müller]

[-- Attachment #1: Type: text/plain, Size: 814 bytes --]

Hello,

I installed the Core Update 118 (beta) a few minutes ago:

IPsec N2N		WORKS
NRPE			WORKS
Squid			WORKS
WebUI			WORKS
Snort (IDS)		WORKS

Currently, there are these two (minor) issues:
(a) A patch of mine (marking 3DES and 1024 bit DH keys
as weak, see: https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=6fc0f5eb92c51067f79cf195d9c0e584556b29a7)
seems not to be applied. Maybe forgot to ship?

(b) Changing the remote syslogging settings does
not cause a crash anymore (this is fixed). However,
changing protocol to "TCP" does not change anything
in the configuration @ /etc/syslog.conf . Mea culpa,
I've got to have a look at this *again*, but at least
we do not have the segfault anymore.

Apart from that, everything changed in 118 seems to work.

Best regards,
Peter Müller

Re: Testing Core Update 118

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 1294 bytes --]

Hi,

On Tue, 2018-01-30 at 18:44 +0100, Peter Müller wrote:
> Hello,
> 
> I installed the Core Update 118 (beta) a few minutes ago:
> 
> IPsec N2N		WORKS
> NRPE			WORKS
> Squid			WORKS
> WebUI			WORKS
> Snort (IDS)		WORKS
> 
> Currently, there are these two (minor) issues:
> (a) A patch of mine (marking 3DES and 1024 bit DH keys
> as weak, see: https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=6fc0f5eb92c5
> 1067f79cf195d9c0e584556b29a7)
> seems not to be applied. Maybe forgot to ship?

Yes, ovpnmain.cgi was not in the updater. I added it.

Not sure if Arne will re-package it.

> (b) Changing the remote syslogging settings does
> not cause a crash anymore (this is fixed). However,
> changing protocol to "TCP" does not change anything
> in the configuration @ /etc/syslog.conf . Mea culpa,
> I've got to have a look at this *again*, but at least
> we do not have the segfault anymore.

Yes, but the sed syntax was butchered. I fixed in a new commit:

  https://cgit.ipfire.org/ipfire-2.x.git/commit/?h=next&id=3925a0db6cc0ca91fe34eb7c0366f72b0af39ab6

However, it appears that our syslog daemon doesn't support TCP at all.

...

-Michael

> 
> Apart from that, everything changed in 118 seems to work.
> 
> Best regards,
> Peter Müller

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Testing Core Update 118

[ummeegge]

[-- Attachment #1: Type: text/plain, Size: 528 bytes --]

Hi all,

Am 31.01.2018 um 17:35 schrieb Michael Tremer:

> I wouldn't mind changing to rsyslog, but would this be just a drop-in
> replacement or cause some serious work?

longer time ago i have integrated Rsyslog into IPFire --> https://forum.ipfire.org/viewtopic.php?t=16669 which was not that difficult but there was the need for some new packages. As far as i remember i worked there with

libtasn1
libestr
json-c
liblogging
libfastjson
librelp

as a beneath info.

> 
> -Michael


Greetings,

Erik

Re: Testing Core Update 118

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 1064 bytes --]

Hi,

nothing of that sounds too bad - except that it sounds a bit extortionate for
just some syslogging.

Maybe it is worth trying it if can be built without json at least.

However, since we are on the topic of logging reliably, it might be interesting
to have support for SSL in there.

Have you tried if the old configuration file works or does it come with a new
configuration file format?

Best,
-Michael

On Thu, 2018-02-01 at 09:41 +0100, ummeegge wrote:
> Hi all,
> 
> Am 31.01.2018 um 17:35 schrieb Michael Tremer:
> 
> > I wouldn't mind changing to rsyslog, but would this be just a drop-in
> > replacement or cause some serious work?
> 
> longer time ago i have integrated Rsyslog into IPFire --> https://forum.ipfire
> .org/viewtopic.php?t=16669 which was not that difficult but there was the need
> for some new packages. As far as i remember i worked there with
> 
> libtasn1
> libestr
> json-c
> liblogging
> libfastjson
> librelp
> 
> as a beneath info.
> 
> > 
> > -Michael
> 
> 
> Greetings,
> 
> Erik

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Testing Core Update 118

[ummeegge]

[-- Attachment #1: Type: text/plain, Size: 2259 bytes --]

Hello,

Am 01.02.2018 um 12:37 schrieb Michael Tremer:

> Hi,
> 
> nothing of that sounds too bad - except that it sounds a bit extortionate for
> just some syslogging.
> 
> Maybe it is worth trying it if can be built without json at least.

Have tried yesterday for fast checkout if it builds but the compilation quits causing a non existing libfastjson. Also the Rsyslog documentation points json-c as a requirement --> http://www.rsyslog.com/doc/master/installation/install_from_source.html#build-requirements out, not sure if can get around it . Some of the other listed libs at the bottom might not be a requirement but possibly a nice one to have even the logs should be send to a remote log server.
Rsyslog do provides really a lot of extensions which might be worth for a look over...

> 
> However, since we are on the topic of logging reliably, it might be interesting
> to have support for SSL in there.

This should be no problem i think. Rsyslog can be linked against libgcrypt, GnuTLS and OpenSSL. A build is currently running on a openssl-11 basis whereby i think OpenSSL should be preferred ?

> 
> Have you tried if the old configuration file works or does it come with a new
> configuration file format?

The new version 8.32 is in building process will check this if it is ready. On my first tries with Rsyslog which is no also 1+ year ago, i needed to make two changes on syslog.conf --> https://forum.ipfire.org/viewtopic.php?t=16669#p98481 and after that i renamed it to rsyslog.conf and everything worked just fine.

> 
> Best,
> -Michael
> 
> On Thu, 2018-02-01 at 09:41 +0100, ummeegge wrote:
>> Hi all,
>> 
>> Am 31.01.2018 um 17:35 schrieb Michael Tremer:
>> 
>>> I wouldn't mind changing to rsyslog, but would this be just a drop-in
>>> replacement or cause some serious work?
>> 
>> longer time ago i have integrated Rsyslog into IPFire --> https://forum.ipfire
>> .org/viewtopic.php?t=16669 which was not that difficult but there was the need
>> for some new packages. As far as i remember i worked there with
>> 
>> libtasn1
>> libestr
>> json-c
>> liblogging
>> libfastjson
>> librelp
>> 
>> as a beneath info.
>> 
>>> 
>>> -Michael
>> 
>> 
>> Greetings,
>> 
>> Erik

Re: Testing Core Update 118

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 3599 bytes --]

Hi,

On Thu, 2018-02-01 at 15:43 +0100, ummeegge wrote:
> Hello,
> 
> Am 01.02.2018 um 12:37 schrieb Michael Tremer:
> 
> > Hi,
> > 
> > nothing of that sounds too bad - except that it sounds a bit extortionate for
> > just some syslogging.
> > 
> > Maybe it is worth trying it if can be built without json at least.
> 
> Have tried yesterday for fast checkout if it builds but the compilation quits causing a non existing libfastjson. Also the Rsyslog documentation points json-c as a requirement --> http://www.rsyslog.com/doc/master/installation/install_from_source.html#build-requirements out, not sure if can get around it . Some of the other listed libs at the bottom might not be a requirement but possibly a nice one to have even the logs should be send to a remote log server.
> Rsyslog do provides really a lot of extensions which might be worth for a look over...

I am not sure if we need any of the extensions since we are basically
using the bare minimum at the moment - sysklogd cannot do much more.
And so far nobody has really complained about this.

json-c is fine. Some other packages need this too and I am sure that I
have built this before somewhere.

The rest: I think the rsyslogd people are doing this a bit wrong. It is
quite an excessive amount of libraries and I do not get why some basic
functionality like a syslog daemon needs that. But I guess we don't
have any other choice here than packaging all of them.

> > However, since we are on the topic of logging reliably, it might be interesting
> > to have support for SSL in there.
> 
> This should be no problem i think. Rsyslog can be linked against libgcrypt, GnuTLS and OpenSSL. A build is currently running on a openssl-11 basis whereby i think OpenSSL should be preferred ?

OpenSSL should be preferred.

> > 
> > Have you tried if the old configuration file works or does it come with a new
> > configuration file format?
> 
> The new version 8.32 is in building process will check this if it is ready. On my first tries with Rsyslog which is no also 1+ year ago, i needed to make two changes on syslog.conf --> https://forum.ipfire.org/viewtopic.php?t=16669#p98481 and after that i renamed it to rsyslog.conf and everything worked just fine.

That sounds good. They have their own configuration file format for
some modules stuff but since we don't intend to use any of that, we
should be fine here.

Would you update those packages and rebase the branch on next since you
have already been working on this? Please send this over to Peter for
review then and let him test the TCP logging capabilities since he was
the one who required this in the first place.

But let's only start working on this when the current OpenVPN stuff is
done.

Best,
-Michael

> 
> > 
> > Best,
> > -Michael
> > 
> > On Thu, 2018-02-01 at 09:41 +0100, ummeegge wrote:
> > > Hi all,
> > > 
> > > Am 31.01.2018 um 17:35 schrieb Michael Tremer:
> > > 
> > > > I wouldn't mind changing to rsyslog, but would this be just a drop-in
> > > > replacement or cause some serious work?
> > > 
> > > longer time ago i have integrated Rsyslog into IPFire --> https://forum.ipfire
> > > .org/viewtopic.php?t=16669 which was not that difficult but there was the need
> > > for some new packages. As far as i remember i worked there with
> > > 
> > > libtasn1
> > > libestr
> > > json-c
> > > liblogging
> > > libfastjson
> > > librelp
> > > 
> > > as a beneath info.
> > > 
> > > > 
> > > > -Michael
> > > 
> > > 
> > > Greetings,
> > > 
> > > Erik
> 
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Testing Core Update 118

[ummeegge]

[-- Attachment #1: Type: text/plain, Size: 4787 bytes --]

Hi all,
did some tests now with Rsyslog which needed some entries in rsyslog.conf more then before but the old sysklogd format can nevertheless be used and the logging results are the same then sysklogd´s .
Have build Rsyslog also in openssl-11 environment which looks good for the building process, a test on a machine without OpenSSL-1.1.0g looked like this:

-> /etc/init.d/rsyslog start
Starting system logger... 
/usr/sbin/rsyslogd: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory   [  OK  ]

which looks good. Started with a configuration file and an initscript whereby both are functional but this is a fast shot and there should be more possible.

Am 01.02.2018 um 16:16 schrieb Michael Tremer:

> Hi,
> 
> On Thu, 2018-02-01 at 15:43 +0100, ummeegge wrote:
>> Hello,
>> 
>> Am 01.02.2018 um 12:37 schrieb Michael Tremer:
>> 
>>> Hi,
>>> 
>>> nothing of that sounds too bad - except that it sounds a bit extortionate for
>>> just some syslogging.
>>> 
>>> Maybe it is worth trying it if can be built without json at least.
>> 
>> Have tried yesterday for fast checkout if it builds but the compilation quits causing a non existing libfastjson. Also the Rsyslog documentation points json-c as a requirement --> http://www.rsyslog.com/doc/master/installation/install_from_source.html#build-requirements out, not sure if can get around it . Some of the other listed libs at the bottom might not be a requirement but possibly a nice one to have even the logs should be send to a remote log server.
>> Rsyslog do provides really a lot of extensions which might be worth for a look over...
> 
> I am not sure if we need any of the extensions since we are basically
> using the bare minimum at the moment - sysklogd cannot do much more.
> And so far nobody has really complained about this.

I think we need in a minimum 'imuxsock.so' and 'imklog.so' to get the same logging results then sysklod provides it. But there is more and it might be nice if someone jump into this here and complains more about possible extensions.
It might also be possible to make a rsyslog-mysql or rsyslog-elasticsearch packet available via Pakfire ? But this only as an idea from here...

> 
> json-c is fine. Some other packages need this too and I am sure that I
> have built this before somewhere.

Yes i do remember that too ?? 

> 
> The rest: I think the rsyslogd people are doing this a bit wrong. It is
> quite an excessive amount of libraries and I do not get why some basic
> functionality like a syslog daemon needs that. But I guess we don't
> have any other choice here than packaging all of them.

Think so, lot´s of new packages event they aren´t that heavy weights but as mentioned some of them can possibly also be dropped but may they are also nice ones to have ?
Possibly Peter can also take a look into that.

> 
>>> However, since we are on the topic of logging reliably, it might be interesting
>>> to have support for SSL in there.
>> 
>> This should be no problem i think. Rsyslog can be linked against libgcrypt, GnuTLS and OpenSSL. A build is currently running on a openssl-11 basis whereby i think OpenSSL should be preferred ?
> 
> OpenSSL should be preferred.

Yes.

> 
>>> 
>>> Have you tried if the old configuration file works or does it come with a new
>>> configuration file format?
>> 
>> The new version 8.32 is in building process will check this if it is ready. On my first tries with Rsyslog which is no also 1+ year ago, i needed to make two changes on syslog.conf --> https://forum.ipfire.org/viewtopic.php?t=16669#p98481 and after that i renamed it to rsyslog.conf and everything worked just fine.
> 
> That sounds good. They have their own configuration file format for
> some modules stuff but since we don't intend to use any of that, we
> should be fine here.

Think so.

> 
> Would you update those packages and rebase the branch on next since you
> have already been working on this? Please send this over to Peter for
> review then and let him test the TCP logging capabilities since he was
> the one who required this in the first place.

No problem but i think it might be a little more then a review only since some more stuff needs to be checked especially for the config but also the initscript.
Build it currently as packages (easier to install) 32bit are ready and a little tested am currently on a 64 bit build.

> 
> But let's only start working on this when the current OpenVPN stuff is
> done.

For an OK is it too late now :D but i won´t do much more on this since it works and i think this might be a good state to work further on it. Will push it to Git then ?

Greetings,

Erik

Re: Testing Core Update 118

[ummeegge]

[-- Attachment #1: Type: text/plain, Size: 7292 bytes --]

Hi all,
first version for Rsyslogd has been pushed and can be found in here --> https://git.ipfire.org/?p=people/ummeegge/ipfire-2.x.git;a=commit;h=9adcfd745418bcb6e2acc5d1f99743f25282d2f8 .

There is currently no make.sh diff cause on this machine are lot´s of other developments too but this should be no problem to supplement or to extend this by your own.
Have added so far:
- GSSAPI (Kerberos) support which might also be interesting for encryption.
- OpenSSL should be used, libgcyrpt too.
- The usertools which includes also 'rscryutil' --> https://www.mankier.com/1/rscryutil has been added .
- Needed to set the CFLAG line in LFS cause Zlib has not been found while compiling, this can be made may nicer. Rsyslog can also be linked against LZ4 (new compression lib which should be available with Core 118 i think).
- There are possibly a lot more modules available in the ROOTFILE  then needed but for testing purposes possibly nice ? Can be reduced to the really needed after testings ...
- A new directory to extend the configuration has been added and is findable under /etc/rsyslog.d .
- Rsyslogd process drops privileges now to user/group 'syslogd' cause user/group already exists and it might be nice to not operate as root i thought.
    --> There was the need for a usermod and to add the user 'syslogd' to the group 'tty' . Initscript do this if not already done <-- A think about this might also be nice .
- 'RepeatedMsgReduction' is in config file but commented.
- A work directory for spool and state files are under /var/spool/rsyslogd (will be created via init script if not already there) but needs also some testings (queueing may?).
- Bootlog block from sysklogd has been integrated into start) sequence in the rsyslogd init script.
- logrotate.conf needed to be adapted but the rotation has not been tested until now. 

The init script do not start if sysklod is still active. A fast idea to integrate rsyslog into the run levels without creating the symlinks is to simply rename 'rsyslogd' to 'sysklogd' findable under /etc/rc.d/init.d .
I think there is no need for json-c and libfastjson (libfastjson might be a better solution) but this needs to be check either.

Have surely forgot a lot but for the first something from here (happy testing).

Greetings,

Erik


Am 02.02.2018 um 20:49 schrieb ummeegge:

> Hi all,
> did some tests now with Rsyslog which needed some entries in rsyslog.conf more then before but the old sysklogd format can nevertheless be used and the logging results are the same then sysklogd´s .
> Have build Rsyslog also in openssl-11 environment which looks good for the building process, a test on a machine without OpenSSL-1.1.0g looked like this:
> 
> -> /etc/init.d/rsyslog start
> Starting system logger... 
> /usr/sbin/rsyslogd: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory   [  OK  ]
> 
> which looks good. Started with a configuration file and an initscript whereby both are functional but this is a fast shot and there should be more possible.
> 
> Am 01.02.2018 um 16:16 schrieb Michael Tremer:
> 
>> Hi,
>> 
>> On Thu, 2018-02-01 at 15:43 +0100, ummeegge wrote:
>>> Hello,
>>> 
>>> Am 01.02.2018 um 12:37 schrieb Michael Tremer:
>>> 
>>>> Hi,
>>>> 
>>>> nothing of that sounds too bad - except that it sounds a bit extortionate for
>>>> just some syslogging.
>>>> 
>>>> Maybe it is worth trying it if can be built without json at least.
>>> 
>>> Have tried yesterday for fast checkout if it builds but the compilation quits causing a non existing libfastjson. Also the Rsyslog documentation points json-c as a requirement --> http://www.rsyslog.com/doc/master/installation/install_from_source.html#build-requirements out, not sure if can get around it . Some of the other listed libs at the bottom might not be a requirement but possibly a nice one to have even the logs should be send to a remote log server.
>>> Rsyslog do provides really a lot of extensions which might be worth for a look over...
>> 
>> I am not sure if we need any of the extensions since we are basically
>> using the bare minimum at the moment - sysklogd cannot do much more.
>> And so far nobody has really complained about this.
> 
> I think we need in a minimum 'imuxsock.so' and 'imklog.so' to get the same logging results then sysklod provides it. But there is more and it might be nice if someone jump into this here and complains more about possible extensions.
> It might also be possible to make a rsyslog-mysql or rsyslog-elasticsearch packet available via Pakfire ? But this only as an idea from here...
> 
>> 
>> json-c is fine. Some other packages need this too and I am sure that I
>> have built this before somewhere.
> 
> Yes i do remember that too ?? 
> 
>> 
>> The rest: I think the rsyslogd people are doing this a bit wrong. It is
>> quite an excessive amount of libraries and I do not get why some basic
>> functionality like a syslog daemon needs that. But I guess we don't
>> have any other choice here than packaging all of them.
> 
> Think so, lot´s of new packages event they aren´t that heavy weights but as mentioned some of them can possibly also be dropped but may they are also nice ones to have ?
> Possibly Peter can also take a look into that.
> 
>> 
>>>> However, since we are on the topic of logging reliably, it might be interesting
>>>> to have support for SSL in there.
>>> 
>>> This should be no problem i think. Rsyslog can be linked against libgcrypt, GnuTLS and OpenSSL. A build is currently running on a openssl-11 basis whereby i think OpenSSL should be preferred ?
>> 
>> OpenSSL should be preferred.
> 
> Yes.
> 
>> 
>>>> 
>>>> Have you tried if the old configuration file works or does it come with a new
>>>> configuration file format?
>>> 
>>> The new version 8.32 is in building process will check this if it is ready. On my first tries with Rsyslog which is no also 1+ year ago, i needed to make two changes on syslog.conf --> https://forum.ipfire.org/viewtopic.php?t=16669#p98481 and after that i renamed it to rsyslog.conf and everything worked just fine.
>> 
>> That sounds good. They have their own configuration file format for
>> some modules stuff but since we don't intend to use any of that, we
>> should be fine here.
> 
> Think so.
> 
>> 
>> Would you update those packages and rebase the branch on next since you
>> have already been working on this? Please send this over to Peter for
>> review then and let him test the TCP logging capabilities since he was
>> the one who required this in the first place.
> 
> No problem but i think it might be a little more then a review only since some more stuff needs to be checked especially for the config but also the initscript.
> Build it currently as packages (easier to install) 32bit are ready and a little tested am currently on a 64 bit build.
> 
>> 
>> But let's only start working on this when the current OpenVPN stuff is
>> done.
> 
> For an OK is it too late now :D but i won´t do much more on this since it works and i think this might be a good state to work further on it. Will push it to Git then ?
> 
> Greetings,
> 
> Erik
> 
> 

Re: Testing Core Update 118

[ummeegge]

[-- Attachment #1: Type: text/plain, Size: 1367 bytes --]

Hello,
i wanted to ask if it is possible to add the OpenSSL configuration for OpenVPN ovpn.cnf --> https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=config/ovpn/openssl/ovpn.cnf;h=40daf2a0a886dd4957df00c3303d3504f8cb0bc0;hb=refs/heads/core115 again to Core 118 update ? 
Since there are a couple of users in the IPFire forum which reports the same issue --> https://forum.ipfire.org/viewtopic.php?f=16&t=19738p112874#p112874 and --> https://forum.ipfire.org/viewtopic.php?f=27&t=20180 , it seems that  ovpn.cnf has been missing with the Core 115 update but the ovpnmain.cgi changes has been delivered.

Might be great if we can deliver this file again to complete this patch otherwise it does not work...


Am 01.02.2018 um 16:16 schrieb Michael Tremer:
> 
> That sounds good. They have their own configuration file format for
> some modules stuff but since we don't intend to use any of that, we
> should be fine here.
> 
> Would you update those packages and rebase the branch on next since you
> have already been working on this? Please send this over to Peter for
> review then and let him test the TCP logging capabilities since he was
> the one who required this in the first place.
> 

Did that now and have changed also the topic to here --> https://lists.ipfire.org/pipermail/development/2018-February/004011.html .


Best,

Erik

Re: Testing Core Update 118

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 1822 bytes --]

Hello,

On Tue, 2018-02-06 at 10:37 +0100, ummeegge wrote:
> Hello,
> i wanted to ask if it is possible to add the OpenSSL configuration for OpenVPN
> ovpn.cnf --> https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=config/ovpn/ope
> nssl/ovpn.cnf;h=40daf2a0a886dd4957df00c3303d3504f8cb0bc0;hb=refs/heads/core115
>  again to Core 118 update ? 
> Since there are a couple of users in the IPFire forum which reports the same
> issue --> https://forum.ipfire.org/viewtopic.php?f=16&t=19738p112874#p112874
> and --> https://forum.ipfire.org/viewtopic.php?f=27&t=20180 , it seems
> that  ovpn.cnf has been missing with the Core 115 update but the ovpnmain.cgi
> changes has been delivered.
> 
> Might be great if we can deliver this file again to complete this patch
> otherwise it does not work...

Technically this could be done, but the update is already built and on the
servers. Not sure if it is worth rebuilding it since this does not seem to be a
pressing issue when this has been around for months.

Please send a patch for this again when we have branched Core Update 119.

Best,
-Michael

> 
> 
> Am 01.02.2018 um 16:16 schrieb Michael Tremer:
> > 
> > That sounds good. They have their own configuration file format for
> > some modules stuff but since we don't intend to use any of that, we
> > should be fine here.
> > 
> > Would you update those packages and rebase the branch on next since you
> > have already been working on this? Please send this over to Peter for
> > review then and let him test the TCP logging capabilities since he was
> > the one who required this in the first place.
> > 
> 
> Did that now and have changed also the topic to here --> https://lists.ipfire.
> org/pipermail/development/2018-February/004011.html .
> 
> 
> Best,
> 
> Erik
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]