Hi All,
Status update.
On 07/06/2023 14:24, Michael Tremer wrote:
On 7 Jun 2023, at 11:17, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michel,
On 07/06/2023 10:50, Michael Tremer wrote:
Hello,
On 6 Jun 2023, at 22:51, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Peter,
On 06/06/2023 22:58, Peter Müller wrote:
Hello Adolf, hello Michael, thank you very much for verifying the OpenVPN functionality for PKCS12 containers imported from IPFire systems running OpenSSL 1.1.1x. As discussed yesterday, I tested the same functionality for IPsec, both for N2N and roadwarrior connections. For both cases, an IPFire installtion running Core-Update 175 Development Build master/3d2beee7 was able to import PKCS12 containers created on an IPFire installation running Core Update 174 without any error message whatsoever, and the IPsec connections could be established afterwards. As far as I am aware, all blocking issues with Core Update 175 are now resolved, and this update can finally be released! @Michael, a draft for its release announcement should be with you already.
I am afraid there is still some more work I am involved in. I realised that we did not have the providers legacy default line in the roadwarrior client configs. After a discussion with Michael, we also decide that we don't want everything to be running legacy, otherwise it will stay like that for ever.
So I am working on some patches that will identify the connection certificates that were created with openssl-1.1.1x and those with openssl-3.x and this will be stored as a key for each connection. Then when using any of the openssl pkcs12 commands, those that are uploading existing certificates will use -legacy if the key is openssl-1.1.1x and not for the openssl-3.x If -legacy is used then providers legacy default will be added to the end of the .ovpn profile file.
You don’t need to store this in the CSV file. You can simply create a new function in the CGI file that takes the path of the p12 file and runs the command with it:
I can see that but the benefit of having it in the ovpnconfig csv file is that in future you can easily see if you have an old based certificate if newer problems occur with the old version.
Hmm, is this something we want to expose to the user?
I was just thinking that then they know which ones they need to update rather than at some time in the future openssl stops even accepting legacy option and then all the old ones are left broken.
I have done the work without adding anything to the csv so we can always discuss this later (maybe a topic for our get together).
openssl pkcs12 -info -nodes -in file.p12 -passin pass: -noout You will then need to parse the output and if you find “MAC: sha1” at the beginning of any line (might be worth breaking the loop after the first iteration still) then you will return True, otherwise False. Then you can just call the function when you need to know whether this is a legacy certificate. It’s not super fast when iterating over hundreds of connections, but we currently only need this when exporting the ZIP file which does not create that much extra overhead.
It also needs to be run if the user uploads a client certificate into IPFire. That is an option within IPFire when creating a client connection, even if the vast majority of people don't use it. That uploading expects a pkcs12 file which it then extracts the required certs, key etc from. If that openssl command does not have -legacy in it then the extraction will fail.
Should we actually accept those certificates any more? Are they not considered cryptographically weak?
That is a good point. I was just thinking we had accepted them in the past but maybe this is where we just say, if you are uploading a new client connection certificate it is the same as if you are creating it on IPFire. It needs to be in line with Openssl-3.x Will work on that basis and make upload only without the -legacy option.
If -legacy is there in the cert and key extraction routines does that still work if the uploaded certificate has been created with openssl-3.x? I will test that out. If yes then the upload openssl commands could have -legacy as standard as it wouldn't make any difference then.
For the openssl pkcs12 command that is creating a new connection certificate, it will always not use -legacy so that any new certificates that are created will always be the newer ones. If you use -legacy with the pkcs12 creation command then the version will be the same as was created with openssl-1.1.1x even though openssl-3.x is running on IPFire.
When existing connection certificates expire (default is 760 days) then when a new connection certificate is created it will also always be the openssl-3.x version.
Over time the old openssl-1.1.1x certificates will disappear.
I will also look at doing the same thing with the net2net connection certificates so only old ones imported will have the providers legacy default added at the end of the n2nconf file.
You can use the same function as above, as it does not care whether the PKCS12 file belongs to a RW or N2N connection.
The patches I will supply will be based on the nightly build from last night so none of my existing patches need to be reverted.
I will also go back and modify the backup.pl restore function so that it also checks the openssl generation of all the connection certs and where required adds the providers legacy default to the .ovpn config.
If you use the function, you can skip that step I believe.
You are saying that it will also be covered by the download the client n2n package flow. I will check that out in my testing.
Cool. Let us know if we can help :)
I was close to asking for help, or suggesting someone else pick it up as I was getting very frustrated. My knowledge of perl syntax is not very good and I made a few basic errors and I couldn't understand what was going wrong. Figured it out eventually. An Oh Duh moment.
I now have the code downloading net2net client packages or roadwarrior packages (secure and insecure) with providers legacy default only on the ones that have legacy certificates. The ones created with openssl-3.x are downloading without the providers legacy default option in the config file.
I have confirmed that the backup saves the n2n client config file without the providers legacy default line. For the roadwarrior it is only the .pem and .p12 files that are backed up. The .ovpn are created from the settings file when being downloaded.
Tested out restoring a backup from yesterday without all the changes and the legacy certificates end up with providers legacy default in the .ovpn file.
So looks like everything is working as expected.
When people download a legacy certificate connection package they will have providers legacy default added to the .ovpn profile. Openssl3.x certificate connection package won't get that.
Any new certificate created will be with the openssl-3.x algorithms - confirmed for both secure and insecure connections after making all the code changes.
Patch coming shortly.
Regards,
Adolf.
-Michael
Regards, Adolf.
I am working to get it done as soon as possible.
-Michael
Regards,
Adolf.
To echo Erik, many many thanks for all the hard work, and dealing with the massive headache the transition to OpenSSL 3.x has induced! All the best, Peter Müller
True, thanks to all for this.
Best,
Erik
Am Dienstag, dem 06.06.2023 um 17:23 +0200 schrieb Adolf Belka: > Hi Michael, > > I tested out a newly created openvpn connection on Openssl-3.x and > installed that on my laptop with the existing legacy providers lines > in > the openssl.cnf file commented out. that connection was successfully > made. > > So what we discussed is the case. Any new connection from CU175 > onwards > will work with a system with OpenSSL-3.x > > It was what we hoped for and expected but it is good to have it > confirmed. > > Regards, > > Adolf. > > On 06/06/2023 11:33, Michael Tremer wrote: >> Thank you. >> >> I just created a page to explain this problem: >> >> https://wiki.ipfire.org/configuration/services/openvpn/openssl-3-transition >> >> Please feel free to edit if I forgot to mention something. >> >> We should include this in the change log for the release, but >> highlight that this breaking change has not been introduced in >> c175, but c175 has the fix. >> >> -Michael >> >>> On 4 Jun 2023, at 20:14, Adolf Belka adolf.belka@ipfire.org >>> wrote: >>> >>> Hi All, >>> >>> I Have submitted a patch set for fixing the bug#13137 related to >>> OpenVPN. This patch set together with the one for bug#13138 >>> related to IPSec that I shipped yesterday fix the major bugs I >>> found for CU175. >>> >>> Sorry it took a bit longer than I had hoped for. >>> >>> Please let me know if there are any problems with what I have >>> produced and I will work to sort them out. >>> >>> I am not aware of any other blocking bugs for CU175 at this point >>> in time. :crossed_fingers: >>> >>> >>> Regards, >>> >>> Adolf. >>> >>> >>> On 02/06/2023 21:35, Adolf Belka wrote: >>>> The poster found the following reference >>>> >>>> https://serverfault.com/questions/857131/odd-error-while-using-openssl >>>> >>>> which suggests that openssl can sometimes give spurious error >>>> messages if unique_subject is not explicitly defined as yes but >>>> it doesn't give any suggestion for why this would have occurred >>>> with the move to openssl-3.x >>>> >>>> >>>> Regards, >>>> >>>> Adolf. >>>> >>>> >>>> On 02/06/2023 21:28, Adolf Belka wrote: >>>>> Hi All, >>>>> >>>>> On 02/06/2023 19:54, Adolf Belka wrote: >>>>>> Hi All, >>>>>> >>>>>> Just when you thought it was safe to get back in the water >>>>>> - aaaaah. >>>>>> >>>>>> >>>>>> I suspected that IPSec would also suffer from the same >>>>>> problem of openssl-3.x vs openssl-1.1.1x for the opening of >>>>>> .p12 certificates. >>>>>> >>>>>> Today, from feedback from Roberto Pena in the forum, it >>>>>> turns out that trying to create a Root/Host certificate set >>>>>> for IPSec causes an error message to be created. >>>>>> >>>>>> I reproduced this on CU175 Testing build master/9797af30 >>>>>> and confirmed that it did not happen in CU174. >>>>>> >>>>>> >>>>>> I added the -legacy option into every openssl command in >>>>>> vpnmain.cgi dealing with .p12 files but this did not solve >>>>>> this problem. I then added -legacy to every openssl >>>>>> command. This solved the problem. >>>>>> >>>>> Someone else in that thread about the IPSec Root/Host >>>>> certificate creation has suggested adding >>>>> >>>>> "unique_subject = yes" >>>>> >>>>> into /var/ipfire/certs/index.txt.attr >>>>> >>>>> have tried that and it does solve the problem. >>>>> >>>>> However now have the dilemma of which solution to use, the >>>>> unique_subject or the legacy option in the openssl commands. >>>>> >>>>> I have asked the person who suggested the unique_subject fix, >>>>> why this is the correct fix to this problem. I searched on >>>>> the internet but only found comments that the default value >>>>> of that variable is yes anyway and no reference to it needing >>>>> to be changed because of a changed from openssl-1.1.1x to >>>>> openssl-3.x >>>>> >>>>> I will see what input the poster provides but does anyone on >>>>> this mailing list have any suggestion as to how the >>>>> unique_subject = yes fix works and why it is needed with >>>>> openssl-3.x or any input on how to make the decision between >>>>> the two approaches or if both should be used? >>>>> >>>>> Regards, >>>>> >>>>> Adolf. >>>>> >>>>>> >>>>>> I then tried only adding the legacy option to each non .p12 >>>>>> openssl command one at a time but this also did not fix the >>>>>> problem. So more than one of the non .p12 commands needs to >>>>>> be defined as legacy at the same time. >>>>>> >>>>>> I thought if every openssl command was going to have the >>>>>> legacy option then the same effect should be possible by >>>>>> adding the legacy provider into the openssl.cnf file. Tried >>>>>> this but it didn't solve the error message when creating >>>>>> the Root/Host IPSec certificate set. So there must be some >>>>>> sort of difference between the command line legacy option >>>>>> and the legacy provider option in the openssl.cnf file. >>>>>> >>>>>> >>>>>> So I will also submit a patch set for the vpnmain.cgi file >>>>>> where all openssl commands have the legacy option added. >>>>>> >>>>>> >>>>>> The same will be done for the ovpnmain.cgi file to be on >>>>>> the safe side. Currently I had only added it to the openssl >>>>>> commands running on .p12 files. >>>>>> >>>>>> I did test creating the Root/Host certificate set on >>>>>> OpenVPN with openssl-3.x and that worked successfully but I >>>>>> think having the legacy option on every openssl command >>>>>> should not cause anything to break. >>>>>> >>>>>> >>>>>> Hope to provide a patch set for the various IPSec/OpenVPN >>>>>> openssl issues found over the weekend, presuming no one >>>>>> finds any further issues. As Michael has said in the past. >>>>>> Better to get any patch fix correct than to have a faulty >>>>>> one quickly. >>>>>> >>>>>> >>>>>> Regards, >>>>>> >>>>>> Adolf. >>>>>> >>>>>> >>>>>> On 31/05/2023 20:53, Adolf Belka wrote: >>>>>>> Hi Michael, >>>>>>> >>>>>>> On 31/05/2023 20:43, Michael Tremer wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Does this mean that OpenVPN cannot read those files >>>>>>>> either? I had that suspicion. >>>>>>> I believe that is the case. The failure I found has no >>>>>>> openssl command in the cgi file and adding "providers >>>>>>> legacy default" to the end of the client n2n conf file >>>>>>> made everything work again. >>>>>>> >>>>>>> Your idea of making a modification to the openssl conf >>>>>>> file is a good one and I will try that out and see if >>>>>>> that works. If so then that would be the better approach >>>>>>> as then any certificates would be covered by the legacy >>>>>>> approach if required. >>>>>>> >>>>>>> Regards, >>>>>>> Adolf. >>>>>>>> >>>>>>>> I will halt the update for now and let’s take some more >>>>>>>> time to figure this one out. >>>>>>>> >>>>>>>> Best, >>>>>>>> -Michael >>>>>>>> >>>>>>>>> On 31 May 2023, at 19:42, Adolf Belka >>>>>>>>> adolf.belka@ipfire.org wrote: >>>>>>>>> >>>>>>>>> Hi All, >>>>>>>>> >>>>>>>>> I have figured out how to solve the problem. I have >>>>>>>>> to add "providers legacy default" into the n2n client >>>>>>>>> configuration file. >>>>>>>>> >>>>>>>>> I have raised a bug on this >>>>>>>>> https://bugzilla.ipfire.org/show_bug.cgi?id=13137 and >>>>>>>>> will submit a patch as soon as feasible. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Adolf. >>>>>>>>> >>>>>>>>> >>>>>>>>> On 31/05/2023 17:07, Adolf Belka wrote: >>>>>>>>>> Hi All, >>>>>>>>>> >>>>>>>>>> I have a blocking bug. >>>>>>>>>> >>>>>>>>>> On 29/05/2023 23:34, Adolf Belka wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 26/05/2023 16:28, Michael Tremer wrote: >>>>>>>>>>>> Thank you very much for letting me know. I >>>>>>>>>>>> wasn’t aware at all. >>>>>>>>>>>> >>>>>>>>>>>> I reverted those changes in next and if the >>>>>>>>>>>> build goes through I will merge the branch back >>>>>>>>>>>> into master again. >>>>>>>>>>> >>>>>>>>>>> Just as a reminder the reversion changes went >>>>>>>>>>> through on next but are not yet reverted in >>>>>>>>>>> master. This needs to be done before CU175 is >>>>>>>>>>> released otherwise n2n connections will fail to >>>>>>>>>>> work properly. >>>>>>>>>>> >>>>>>>>>> The reversions have been applied to master (Core >>>>>>>>>> Update 175) and I did an update but have found that >>>>>>>>>> the n2n connection when the end that had the client >>>>>>>>>> upload has stopped working. This is definitely not >>>>>>>>>> because of bug#11048 as all of that has been >>>>>>>>>> reverted (confirmed by looking at the ovpnmain.cgi >>>>>>>>>> file) >>>>>>>>>> >>>>>>>>>> I have realised that what is happening is the same >>>>>>>>>> as for the insecure roadwarrior certificate that >>>>>>>>>> was created with openssl-1.1.1x would not work with >>>>>>>>>> openssl-3.x. I fixed that with the addition of the >>>>>>>>>> legacy option into the openssl command for that >>>>>>>>>> client package download. >>>>>>>>>> >>>>>>>>>> The same thing is happening with the n2n >>>>>>>>>> connection. I will find the command that access the >>>>>>>>>> .p12 file in the zip package and add the legacy >>>>>>>>>> option to that. Presuming it works I will raise and >>>>>>>>>> submit a patch to fix it. >>>>>>>>>> >>>>>>>>>> I will also raise a bug for it and assign myself to >>>>>>>>>> it. >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> Adolf. >>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Adolf. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -Michael >>>>>>>>>>>> >>>>>>>>>>>>> On 26 May 2023, at 14:33, Adolf Belka >>>>>>>>>>>>> adolf.belka@ipfire.org wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> On 24/05/2023 11:02, Michael Tremer wrote: >>>>>>>>>>>>>> Hello, >>>>>>>>>>>>>>> On 24 May 2023, at 09:47, Adolf Belka >>>>>>>>>>>>>>> adolf.belka@ipfire.org wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Michael, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 24/05/2023 10:07, Michael Tremer >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> It looks like we might not want to >>>>>>>>>>>>>>>> release the forthcoming Core Update >>>>>>>>>>>>>>>> before this. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I did not hear any rumours about what >>>>>>>>>>>>>>>> might be the issue, but I would say >>>>>>>>>>>>>>>> that it wouldn’t hurt us to wait. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> What other outstanding issues do we >>>>>>>>>>>>>>>> have that are currently blocking the >>>>>>>>>>>>>>>> update? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The fix for Bug#13117 has been merged >>>>>>>>>>>>>>> into master so that is no longer >>>>>>>>>>>>>>> blocking. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> As mentioned to Peter, I recommend >>>>>>>>>>>>>>> reverting my fix for Bug#11048 as some >>>>>>>>>>>>>>> issues were found by myself (missed in my >>>>>>>>>>>>>>> own testing) plus from other testers >>>>>>>>>>>>>>> reporting in the forum. I am making >>>>>>>>>>>>>>> progress on this but there are still some >>>>>>>>>>>>>>> bits outstanding. The bug has been around >>>>>>>>>>>>>>> for a long time so it won't hurt for it >>>>>>>>>>>>>>> to wait till Core Update 176. >>>>>>>>>>>>>> Okay. Let’s rather have the right fix than >>>>>>>>>>>>>> a quick one. I agree! >>>>>>>>>>>>> The reversion of the update.sh script was >>>>>>>>>>>>> done two days ago but there are 4 other >>>>>>>>>>>>> commits for the same bug fix that also need >>>>>>>>>>>>> to be reverted. >>>>>>>>>>>>> >>>>>>>>>>>>> https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=762c88ec4d85e3a4f7265b88... >>>>>>>>>>>>> https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=82822934ba769bca4235cd2a... >>>>>>>>>>>>> https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=070abb0d011ff71e5aefd170... >>>>>>>>>>>>> https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=18bece0edbd817933f48fdbf... >>>>>>>>>>>>> >>>>>>>>>>>>> just to make sure that those don't get >>>>>>>>>>>>> missed. >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Adolf. >>>>>>>>>>>>>>> I haven't found anything else that was a >>>>>>>>>>>>>>> problem and I haven't seen any other >>>>>>>>>>>>>>> issues mentioned in the forum that look >>>>>>>>>>>>>>> to be caused by CU175. >>>>>>>>>>>>>> That sounds good then! >>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>> Adolf. >>>>>>>>>>>>>>>> -Michael >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Begin forwarded message: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> From: Tomas Mraz tomas@openssl.org >>>>>>>>>>>>>>>>> Subject: Forthcoming OpenSSL Releases >>>>>>>>>>>>>>>>> Date: 24 May 2023 at 05:06:12 BST >>>>>>>>>>>>>>>>> To: "openssl-project@openssl.org" >>>>>>>>>>>>>>>>> openssl-project@openssl.org, >>>>>>>>>>>>>>>>> "openssl-users@openssl.org" >>>>>>>>>>>>>>>>> openssl-users@openssl.org, >>>>>>>>>>>>>>>>> openssl-announce@openssl.org >>>>>>>>>>>>>>>>> Reply-To: openssl-users@openssl.org >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> The OpenSSL project team would like >>>>>>>>>>>>>>>>> to announce the forthcoming release >>>>>>>>>>>>>>>>> of OpenSSL versions 3.0.9, 1.1.1u and >>>>>>>>>>>>>>>>> 1.0.2zh. Note that OpenSSL 1.0.2 >>>>>>>>>>>>>>>>> is End Of Life and so 1.0.2zh will be >>>>>>>>>>>>>>>>> available to premium support >>>>>>>>>>>>>>>>> customers only. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> These releases will be made available >>>>>>>>>>>>>>>>> on Tuesday 30th May 2023 >>>>>>>>>>>>>>>>> between 1300-1700 UTC. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> These are security-fix releases. The >>>>>>>>>>>>>>>>> highest severity issue fixed in >>>>>>>>>>>>>>>>> each of these three releases is >>>>>>>>>>>>>>>>> Moderate: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> https://www.openssl.org/policies/secpolicy.html >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yours >>>>>>>>>>>>>>>>> The OpenSSL Project Team >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> Sent from my laptop >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>> >> >
-- Sent from my laptop