Hi All,
At the last video call we agreed to test out openvpn and ipsec with the openssl-3.2.0 version that is in next.
I cloned a vm and updated it to unstable (CU183) and ran my existing openvpn connections on it that had been created with an older version of openssl-3.x. Everything worked without any problems.
I then created new connections with openssl-3.2.0 and tested them out. Again the connection was successfully made and I could access the remote green machine with no problems.
So for openvpn there looks to be no issues with openssl-3.2.0 from my testing.
Regards, Adolf.
Hello Adolf,
Thank you very much for testing.
I believe that I might have a small regression from OpenSSL 3.2.0 - at least I think it is that:
https://bugzilla.ipfire.org/show_bug.cgi?id=13527
Apache won’t start if a system has been upgraded for a long time and is using an older RSA key.
I could not find any indication in the change log of OpenSSL, but since we did not touch Apache itself in this update, I cannot come up with any other idea.
Since we are already using ECDSA keys as well as RSA keys, how about dropping the RSA keys altogether to solve this problem?
-Michael
On 16 Jan 2024, at 14:18, Adolf Belka adolf.belka@ipfire.org wrote:
Hi All,
At the last video call we agreed to test out openvpn and ipsec with the openssl-3.2.0 version that is in next.
I cloned a vm and updated it to unstable (CU183) and ran my existing openvpn connections on it that had been created with an older version of openssl-3.x. Everything worked without any problems.
I then created new connections with openssl-3.2.0 and tested them out. Again the connection was successfully made and I could access the remote green machine with no problems.
So for openvpn there looks to be no issues with openssl-3.2.0 from my testing.
Regards, Adolf.
-- Sent from my laptop
Hi Michael & all,
On 17/01/2024 11:22, Michael Tremer wrote:
Hello Adolf,
Thank you very much for testing.
I believe that I might have a small regression from OpenSSL 3.2.0 - at least I think it is that:
https://bugzilla.ipfire.org/show_bug.cgi?id=13527
Apache won’t start if a system has been upgraded for a long time and is using an older RSA key.
I could not find any indication in the change log of OpenSSL, but since we did not touch Apache itself in this update, I cannot come up with any other idea.
When I raised the patch I looked through the logs and didn't find anything that sprung out to me as being a problem. When Arne raised that bug I went back and had another look at the logs and searched though them with various phrases and couldn't find anything either related to it.
When I did my unstable update Apache did not stop for me. I had another look at it just now and my RSA cert has a 4096 bit key. I must have re4-created it myself at some time in the past. The original version of the vm is probably around 6 to 12 months old from when I had to re-install it due to some problem.
My production system has a 2048 bit key. Maybe that makes the difference.
I will do some clones of my vm and re-create the Apache server certs with 1024 and 2048 bit certs and test doing the update and see if I get the same problem with either of those two sizes.
Since we are already using ECDSA keys as well as RSA keys, how about dropping the RSA keys altogether to solve this problem?
We could do that but I would think 4096 bit keys are still okay for RSA.
Will let you know what I find with my testing.
Regards,
Adolf.
-Michael
On 16 Jan 2024, at 14:18, Adolf Belka adolf.belka@ipfire.org wrote:
Hi All,
At the last video call we agreed to test out openvpn and ipsec with the openssl-3.2.0 version that is in next.
I cloned a vm and updated it to unstable (CU183) and ran my existing openvpn connections on it that had been created with an older version of openssl-3.x. Everything worked without any problems.
I then created new connections with openssl-3.2.0 and tested them out. Again the connection was successfully made and I could access the remote green machine with no problems.
So for openvpn there looks to be no issues with openssl-3.2.0 from my testing.
Regards, Adolf.
-- Sent from my laptop
Hi Michael & all,
On 17/01/2024 15:25, Adolf Belka wrote:
Hi Michael & all,
On 17/01/2024 11:22, Michael Tremer wrote:
Hello Adolf,
Thank you very much for testing.
I believe that I might have a small regression from OpenSSL 3.2.0 - at least I think it is that:
https://bugzilla.ipfire.org/show_bug.cgi?id=13527
Apache won’t start if a system has been upgraded for a long time and is using an older RSA key.
I could not find any indication in the change log of OpenSSL, but since we did not touch Apache itself in this update, I cannot come up with any other idea.
When I raised the patch I looked through the logs and didn't find anything that sprung out to me as being a problem. When Arne raised that bug I went back and had another look at the logs and searched though them with various phrases and couldn't find anything either related to it.
When I did my unstable update Apache did not stop for me. I had another look at it just now and my RSA cert has a 4096 bit key. I must have re4-created it myself at some time in the past. The original version of the vm is probably around 6 to 12 months old from when I had to re-install it due to some problem.
My production system has a 2048 bit key. Maybe that makes the difference.
I will do some clones of my vm and re-create the Apache server certs with 1024 and 2048 bit certs and test doing the update and see if I get the same problem with either of those two sizes.
Okay have done my testing and some more searching. 1024 bit rsa key works fine with CU182 but Apache fails to start with CU183.
2048 bit rsa key works fine with both CU182 & CU183.
I have found a mention that openssl-3.2.0 has increased the security level from 1 to 2.
With security level 2 a minimum of 2048 bit keys are required to be used. Anything smaller is not accepted.
Security level 1 accepted a minimum key length of 1024 bits.
So that looks to be the cause of the effect found by Arne. It means the system must have had a 1024 bit rsa key length.
It is in the changelog but you would have to read it closely and it is not very near the top of the changelog. This is what is said:-
The default SSL/TLS security level has been changed from 1 to 2. RSA, DSA and DH keys of 1024 bits and above and less than 2048 bits and ECC keys of 160 bits and above and less than 224 bits were previously accepted by default but are now no longer allowed. By default TLS compression was already disabled in previous OpenSSL versions. At security level 2 it cannot be enabled.
There seem to be suggestions I have read that you can change the security level to a different value to the default value if you want to, so we could put it back to 1 if so desired.
Personally I would be okay with it being at 2 but then we will need to check if the certificate has a 1024 bit rsa key and if so to re-generate it.
I am not sure if this regeneration will then cause the self signed certificate notification to be re-shown to users and need to be accepted again.
If so then this would need to be flagged up in the release notes so users are not surprised by that.
Regards,
Adolf.
Since we are already using ECDSA keys as well as RSA keys, how about dropping the RSA keys altogether to solve this problem?
We could do that but I would think 4096 bit keys are still okay for RSA.
Will let you know what I find with my testing.
Regards,
Adolf.
-Michael
On 16 Jan 2024, at 14:18, Adolf Belka adolf.belka@ipfire.org wrote:
Hi All,
At the last video call we agreed to test out openvpn and ipsec with the openssl-3.2.0 version that is in next.
I cloned a vm and updated it to unstable (CU183) and ran my existing openvpn connections on it that had been created with an older version of openssl-3.x. Everything worked without any problems.
I then created new connections with openssl-3.2.0 and tested them out. Again the connection was successfully made and I could access the remote green machine with no problems.
So for openvpn there looks to be no issues with openssl-3.2.0 from my testing.
Regards, Adolf.
-- Sent from my laptop
Hello Adolf, hello Michael,
Hi Michael & all,
On 17/01/2024 15:25, Adolf Belka wrote:
Hi Michael & all,
On 17/01/2024 11:22, Michael Tremer wrote:
Hello Adolf,
Thank you very much for testing.
I believe that I might have a small regression from OpenSSL 3.2.0 - at least I think it is that:
https://bugzilla.ipfire.org/show_bug.cgi?id=13527
Apache won’t start if a system has been upgraded for a long time and is using an older RSA key.
I could not find any indication in the change log of OpenSSL, but since we did not touch Apache itself in this update, I cannot come up with any other idea.
When I raised the patch I looked through the logs and didn't find anything that sprung out to me as being a problem. When Arne raised that bug I went back and had another look at the logs and searched though them with various phrases and couldn't find anything either related to it.
When I did my unstable update Apache did not stop for me. I had another look at it just now and my RSA cert has a 4096 bit key. I must have re4-created it myself at some time in the past. The original version of the vm is probably around 6 to 12 months old from when I had to re-install it due to some problem.
My production system has a 2048 bit key. Maybe that makes the difference.
I will do some clones of my vm and re-create the Apache server certs with 1024 and 2048 bit certs and test doing the update and see if I get the same problem with either of those two sizes.
Okay have done my testing and some more searching. 1024 bit rsa key works fine with CU182 but Apache fails to start with CU183.
2048 bit rsa key works fine with both CU182 & CU183.
I have found a mention that openssl-3.2.0 has increased the security level from 1 to 2.
With security level 2 a minimum of 2048 bit keys are required to be used. Anything smaller is not accepted.
Security level 1 accepted a minimum key length of 1024 bits.
So that looks to be the cause of the effect found by Arne. It means the system must have had a 1024 bit rsa key length.
To be honest, I am surprised that modern web browsers still accept 1k RSA keys. I thought at least Firefox has started rejecting these a while ago.
It is in the changelog but you would have to read it closely and it is not very near the top of the changelog. This is what is said:-
The default SSL/TLS security level has been changed from 1 to 2. RSA, DSA and DH keys of 1024 bits and above and less than 2048 bits and ECC keys of 160 bits and above and less than 224 bits were previously accepted by default but are now no longer allowed. By default TLS compression was already disabled in previous OpenSSL versions. At security level 2 it cannot be enabled.
There seem to be suggestions I have read that you can change the security level to a different value to the default value if you want to, so we could put it back to 1 if so desired.
Personally I would be okay with it being at 2 but then we will need to check if the certificate has a 1024 bit rsa key and if so to re-generate it.
I am not sure if this regeneration will then cause the self signed certificate notification to be re-shown to users and need to be accepted again.
Yes, this will cause such effects, because web browsers take a certificates' fingerprint into account when storing a certificate exception.
It is a tricky situation: I think we should definitely do something about this, and going back to security level 1 isn't a good option to me.
Personally, I would be happy with taking this as an opportunity of dropping RSA for the web interface altogether. That might break some very outdated clients who do not understand anything about ECC, but we can't help it.
It will also prompt certificate warnings to users whose clients previously preferred RSA over ECC, but since the server does the latter for performance reason, I'd dismiss that as a corner case.
If so then this would need to be flagged up in the release notes so users are not surprised by that.
Fully agree. Past changes of this kind have shown us how important communication is. :-)
Thanks, and best regards, Peter Müller
Regards,
Adolf.
Since we are already using ECDSA keys as well as RSA keys, how about dropping the RSA keys altogether to solve this problem?
We could do that but I would think 4096 bit keys are still okay for RSA.
Will let you know what I find with my testing.
Regards,
Adolf.
-Michael
On 16 Jan 2024, at 14:18, Adolf Belka adolf.belka@ipfire.org wrote:
Hi All,
At the last video call we agreed to test out openvpn and ipsec with the openssl-3.2.0 version that is in next.
I cloned a vm and updated it to unstable (CU183) and ran my existing openvpn connections on it that had been created with an older version of openssl-3.x. Everything worked without any problems.
I then created new connections with openssl-3.2.0 and tested them out. Again the connection was successfully made and I could access the remote green machine with no problems.
So for openvpn there looks to be no issues with openssl-3.2.0 from my testing.
Regards, Adolf.
-- Sent from my laptop
Hi Peter & Michael,
On 19/01/2024 09:50, Peter Müller wrote:
Hello Adolf, hello Michael,
Hi Michael & all,
On 17/01/2024 15:25, Adolf Belka wrote:
Hi Michael & all,
On 17/01/2024 11:22, Michael Tremer wrote:
Hello Adolf,
Thank you very much for testing.
I believe that I might have a small regression from OpenSSL 3.2.0 - at least I think it is that:
https://bugzilla.ipfire.org/show_bug.cgi?id=13527
Apache won’t start if a system has been upgraded for a long time and is using an older RSA key.
I could not find any indication in the change log of OpenSSL, but since we did not touch Apache itself in this update, I cannot come up with any other idea.
When I raised the patch I looked through the logs and didn't find anything that sprung out to me as being a problem. When Arne raised that bug I went back and had another look at the logs and searched though them with various phrases and couldn't find anything either related to it.
When I did my unstable update Apache did not stop for me. I had another look at it just now and my RSA cert has a 4096 bit key. I must have re4-created it myself at some time in the past. The original version of the vm is probably around 6 to 12 months old from when I had to re-install it due to some problem.
My production system has a 2048 bit key. Maybe that makes the difference.
I will do some clones of my vm and re-create the Apache server certs with 1024 and 2048 bit certs and test doing the update and see if I get the same problem with either of those two sizes.
Okay have done my testing and some more searching. 1024 bit rsa key works fine with CU182 but Apache fails to start with CU183.
2048 bit rsa key works fine with both CU182 & CU183.
I have found a mention that openssl-3.2.0 has increased the security level from 1 to 2.
With security level 2 a minimum of 2048 bit keys are required to be used. Anything smaller is not accepted.
Security level 1 accepted a minimum key length of 1024 bits.
So that looks to be the cause of the effect found by Arne. It means the system must have had a 1024 bit rsa key length.
To be honest, I am surprised that modern web browsers still accept 1k RSA keys. I thought at least Firefox has started rejecting these a while ago.
Firefox might well do so but as we have the ECDSA certificate as well all my Firefox systems are taking that in preference. I have not been able to find any way to make IPFire try and take only the 1024 bit RSA that I created. If I remove the ECDSA certs from IPFire then apache won't restart as it looks for both ecdsa and rsa certs to be present.
It seems to me that anyone that has a 1024 bit rsa certificate won't be using it on Firefox. I don't know if all browsers take the ecdsa cert in preference if it is available.
So with my vm setup changing the 1024 bit rsa key to 4096 did not cause me to have the message about unsigned certificate as my firefox used the ecdsa.
It is in the changelog but you would have to read it closely and it is not very near the top of the changelog. This is what is said:-
The default SSL/TLS security level has been changed from 1 to 2. RSA, DSA and DH keys of 1024 bits and above and less than 2048 bits and ECC keys of 160 bits and above and less than 224 bits were previously accepted by default but are now no longer allowed. By default TLS compression was already disabled in previous OpenSSL versions. At security level 2 it cannot be enabled.
There seem to be suggestions I have read that you can change the security level to a different value to the default value if you want to, so we could put it back to 1 if so desired.
Personally I would be okay with it being at 2 but then we will need to check if the certificate has a 1024 bit rsa key and if so to re-generate it.
I am not sure if this regeneration will then cause the self signed certificate notification to be re-shown to users and need to be accepted again.
Yes, this will cause such effects, because web browsers take a certificates' fingerprint into account when storing a certificate exception.
It is a tricky situation: I think we should definitely do something about this, and going back to security level 1 isn't a good option to me.
Personally, I would be happy with taking this as an opportunity of dropping RSA for the web interface altogether. That might break some very outdated clients who do not understand anything about ECC, but we can't help it.
It will also prompt certificate warnings to users whose clients previously preferred RSA over ECC, but since the server does the latter for performance reason, I'd dismiss that as a corner case.
Do you know which clients might prefer rsa over ecdsa and how to set that preference. In Firefox it looks to just use ecdsa.
If so then this would need to be flagged up in the release notes so users are not surprised by that.
Fully agree. Past changes of this kind have shown us how important communication is. :-)
I think this is good to mention but we should indicate it will only happen if users have browser clients that have been set to prefer rsa over ecdsa.
I can also confirm that the patch commits that Arne has merged work. I ran a CU182 vm with a 1024 bit rsa cert and did the update to CU183 and everything went without a problem and the rsa cert was 4096 bit afterwards.
Regards,
Adolf.
Thanks, and best regards, Peter Müller
Regards,
Adolf.
Since we are already using ECDSA keys as well as RSA keys, how about dropping the RSA keys altogether to solve this problem?
We could do that but I would think 4096 bit keys are still okay for RSA.
Will let you know what I find with my testing.
Regards,
Adolf.
-Michael
On 16 Jan 2024, at 14:18, Adolf Belka adolf.belka@ipfire.org wrote:
Hi All,
At the last video call we agreed to test out openvpn and ipsec with the openssl-3.2.0 version that is in next.
I cloned a vm and updated it to unstable (CU183) and ran my existing openvpn connections on it that had been created with an older version of openssl-3.x. Everything worked without any problems.
I then created new connections with openssl-3.2.0 and tested them out. Again the connection was successfully made and I could access the remote green machine with no problems.
So for openvpn there looks to be no issues with openssl-3.2.0 from my testing.
Regards, Adolf.
-- Sent from my laptop
Hello,
On 19 Jan 2024, at 11:50, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Peter & Michael,
On 19/01/2024 09:50, Peter Müller wrote:
Hello Adolf, hello Michael,
Hi Michael & all,
On 17/01/2024 15:25, Adolf Belka wrote:
Hi Michael & all,
On 17/01/2024 11:22, Michael Tremer wrote:
Hello Adolf,
Thank you very much for testing.
I believe that I might have a small regression from OpenSSL 3.2.0 - at least I think it is that:
https://bugzilla.ipfire.org/show_bug.cgi?id=13527
Apache won’t start if a system has been upgraded for a long time and is using an older RSA key.
I could not find any indication in the change log of OpenSSL, but since we did not touch Apache itself in this update, I cannot come up with any other idea.
When I raised the patch I looked through the logs and didn't find anything that sprung out to me as being a problem. When Arne raised that bug I went back and had another look at the logs and searched though them with various phrases and couldn't find anything either related to it.
When I did my unstable update Apache did not stop for me. I had another look at it just now and my RSA cert has a 4096 bit key. I must have re4-created it myself at some time in the past. The original version of the vm is probably around 6 to 12 months old from when I had to re-install it due to some problem.
My production system has a 2048 bit key. Maybe that makes the difference.
I will do some clones of my vm and re-create the Apache server certs with 1024 and 2048 bit certs and test doing the update and see if I get the same problem with either of those two sizes.
Okay have done my testing and some more searching. 1024 bit rsa key works fine with CU182 but Apache fails to start with CU183.
2048 bit rsa key works fine with both CU182 & CU183.
I have found a mention that openssl-3.2.0 has increased the security level from 1 to 2.
With security level 2 a minimum of 2048 bit keys are required to be used. Anything smaller is not accepted.
Security level 1 accepted a minimum key length of 1024 bits.
So that looks to be the cause of the effect found by Arne. It means the system must have had a 1024 bit rsa key length.
To be honest, I am surprised that modern web browsers still accept 1k RSA keys. I thought at least Firefox has started rejecting these a while ago.
Firefox might well do so but as we have the ECDSA certificate as well all my Firefox systems are taking that in preference. I have not been able to find any way to make IPFire try and take only the 1024 bit RSA that I created. If I remove the ECDSA certs from IPFire then apache won't restart as it looks for both ecdsa and rsa certs to be present.
You can simply remove the ECDSA certificates from the configuration file in /etc/httpd/conf/vhosts.d/ipfire-interface-ssl.conf.
Otherwise there is no way to convince your browser to use old crypto as far as I know. Some browsers have feature flags where certain things can be enabled/disabled but that usually is for development/testing newer features or things that have just been released to the masses.
It seems to me that anyone that has a 1024 bit rsa certificate won't be using it on Firefox. I don't know if all browsers take the ecdsa cert in preference if it is available.
To put it simply, the IPFire websites do not use RSA for several years now. We only use ECDSA certificates there. So browsers need to be *very* old to not support that. Hence I would support to remove RSA from the´ web UI.
So with my vm setup changing the 1024 bit rsa key to 4096 did not cause me to have the message about unsigned certificate as my firefox used the ecdsa.
I believe virtually nobody will see it as they will only use ECDSA.
It is in the changelog but you would have to read it closely and it is not very near the top of the changelog. This is what is said:-
The default SSL/TLS security level has been changed from 1 to 2. RSA, DSA and DH keys of 1024 bits and above and less than 2048 bits and ECC keys of 160 bits and above and less than 224 bits were previously accepted by default but are now no longer allowed. By default TLS compression was already disabled in previous OpenSSL versions. At security level 2 it cannot be enabled.
There seem to be suggestions I have read that you can change the security level to a different value to the default value if you want to, so we could put it back to 1 if so desired.
Personally I would be okay with it being at 2 but then we will need to check if the certificate has a 1024 bit rsa key and if so to re-generate it.
I am not sure if this regeneration will then cause the self signed certificate notification to be re-shown to users and need to be accepted again.
Yes, this will cause such effects, because web browsers take a certificates' fingerprint into account when storing a certificate exception. It is a tricky situation: I think we should definitely do something about this, and going back to security level 1 isn't a good option to me. Personally, I would be happy with taking this as an opportunity of dropping RSA for the web interface altogether. That might break some very outdated clients who do not understand anything about ECC, but we can't help it. It will also prompt certificate warnings to users whose clients previously preferred RSA over ECC, but since the server does the latter for performance reason, I'd dismiss that as a corner case.
Do you know which clients might prefer rsa over ecdsa and how to set that preference. In Firefox it looks to just use ecdsa.
I tried to figure this out by finding some report on browser compatibility/usage, but I didn’t find anything that is recent enough.
I believe that no recent OS/browser would struggle with this.
If so then this would need to be flagged up in the release notes so users are not surprised by that.
Fully agree. Past changes of this kind have shown us how important communication is. :-)
I think this is good to mention but we should indicate it will only happen if users have browser clients that have been set to prefer rsa over ecdsa.
I can also confirm that the patch commits that Arne has merged work. I ran a CU182 vm with a 1024 bit rsa cert and did the update to CU183 and everything went without a problem and the rsa cert was 4096 bit afterwards.
Thank you for confirming. I am still not entirely sure if we should leave this in, or whether we should remove RSA entirely. I do not see what RSA provides at this point.
-Michael
Regards,
Adolf.
Thanks, and best regards, Peter Müller
Regards,
Adolf.
Since we are already using ECDSA keys as well as RSA keys, how about dropping the RSA keys altogether to solve this problem?
We could do that but I would think 4096 bit keys are still okay for RSA.
Will let you know what I find with my testing.
Regards,
Adolf.
-Michael
On 16 Jan 2024, at 14:18, Adolf Belka adolf.belka@ipfire.org wrote:
Hi All,
At the last video call we agreed to test out openvpn and ipsec with the openssl-3.2.0 version that is in next.
I cloned a vm and updated it to unstable (CU183) and ran my existing openvpn connections on it that had been created with an older version of openssl-3.x. Everything worked without any problems.
I then created new connections with openssl-3.2.0 and tested them out. Again the connection was successfully made and I could access the remote green machine with no problems.
So for openvpn there looks to be no issues with openssl-3.2.0 from my testing.
Regards, Adolf.
-- Sent from my laptop
-
Adolf Belka -
Michael Tremer -
Peter Müller