Re: Core Update 161 (testing) report

[Bernhard Bitsch <bbitsch@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 7887 bytes --]

Hi,

Am 18.11.2021 um 10:58 schrieb Michael Tremer:
> Hello,
> 
>> On 15 Nov 2021, at 14:09, Bernhard Bitsch <bbitsch(a)ipfire.org> wrote:
>>
>> Hi,
>>
>> Am 14.11.2021 um 11:52 schrieb Bernhard Bitsch:
>>> Am 12.11.2021 um 23:33 schrieb Bernhard Bitsch:
>>>> Hi,
>>>>
>>>> as far as I saw in the code, the new CGI tries the refreshing of the tail -f also. But it is never displayed.
>>>> I tried to search by test prints, but had no success, yet.
>>> Reinstalled my test prints.
>>> The processing I saw till now:
>>> - update is called
>>>    waiting for lock ( 7 x sleep(1) )
>>> - no output( lockfile does not exist )
>>> Each - block describes a call of the .cgi
>>
>> I think, there's a problem with the refreshing of the page.
>> I'm no HTML guru, but I suppose the refreshing  only works on open pages. If do not exit the cgi script, but just go to the display of the logs, I managed to get a second box with the log snippet.
>> Could somebody with more experience in web design look at this?
> 
> Is this regarding the solution before the latest patch or after?
> 
> -Michael
> 

I think this was ( and is ) the behaviour of the update/upgrade part.
We didn't look accurately at this until now. I noticed it just on 
testing the patch for the new processing.

- Bernhard

>>
>>> Next I'll add some timing information.
>>> Update: time between calls ~35-40s
>>
>> - Bernhard
>>
>>>> Because I didn't test the real CU 161, I'm not sure I've implemented all changes ( especially these new systemxxx functions). So I decided to stop this research.
>>>> I'll give a new try next days.
>>>>
>>>> Regards,
>>>> Bernhard
>>>>
>>>> Am 12.11.2021 um 19:54 schrieb Kienker, Fred:
>>>>> Peter - the behavior you describe also happens on all our testing systems. It took us several tries to realize the systems hand not just
>>>>> locked up.
>>>>>
>>>>> Michael - this is a regression from previous behavior.
>>>>>
>>>>> There is never any indication to the user the update processing has been
>>>>> completed. The tailf of the update log provided an indication of when
>>>>> the processing is completed.
>>>>>
>>>>> Best regards,
>>>>> Fred
>>>>>
>>>>> Please note: Although we may sometimes respond to email, text and phone
>>>>> calls instantly at all hours of the day and night, our regular business
>>>>> hours are 9:00 AM - 6:00 PM ET, Monday thru Friday.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Peter Müller <peter.mueller(a)ipfire.org>
>>>>> Sent: Friday, November 12, 2021 12:32 PM
>>>>> To: Michael Tremer <michael.tremer(a)ipfire.org>
>>>>> Cc: IPFire: Development <development(a)lists.ipfire.org>
>>>>> Subject: Re: Core Update 161 (testing) report
>>>>>
>>>>> Hello Michael,
>>>>>
>>>>> thanks for your mail. Please excuse my tardy reply - I currently have a
>>>>> lot of other things on my plate, and 24 hours per day are not sufficient
>>>>> to get them done.
>>>>>
>>>>> [Insert personal load average graph here]
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>>> On 2 Nov 2021, at 08:01, Peter Müller <peter.mueller(a)ipfire.org>
>>>>> wrote:
>>>>>>>
>>>>>>> Hello *,
>>>>>>>
>>>>>>> Core Update 161 (testing; no release announcement or changelog has
>>>>>>> been published, yet) is running here for about 12 hours by now
>>>>> without any major issues known so far.
>>>>>>
>>>>>> Yay \o/
>>>>>>
>>>>>>> During the upgrade, I noticed the Pakfire CGI still does not display
>>>>>>> log messages as it used to do, but at least there is now a spinning
>>>>>>> loading icon displaying the message that an operation is currently in
>>>>> progress. From a UX perspective, this is okay I guess.
>>>>>>
>>>>>> What is different about it?
>>>>> The older CGI used to print a "tail -f"-like output of Pakfire's log,
>>>>> reloading the page every few seconds so the user could see the actual
>>>>> process of the ongoing operation.
>>>>>
>>>>> Nowadays, it only gives a spinning GIF image and a text note - better
>>>>> than nothing, but the user has no idea what is going on behind the
>>>>> scenes and how long it will take to be completed.
>>>>>
>>>>>>
>>>>>>> The reconnection necessary for upgrading pppd went smooth, albeit
>>>>>>> Pakfire could not download add-on upgrades afterwards since the VPN
>>>>>>> did not came back in time, so I had to do this manually.
>>>>>>
>>>>>> Normally people dont download packages over a VPN. So I can live
>>>>> with this.
>>>>>>
>>>>>>> To my surprise, some IPsec N2N connections did not reconnect
>>>>>>> automatically, even after rebooting the testing machine. After
>>>>>>> manually clicking on one of the "restart" buttons on the IPsec CGI,
>>>>> they came back instantly, and have been stable ever since.
>>>>>>
>>>>>> Anything in the logs? It should come back automatically.
>>>>> Unfortunately, I did not yet have time to look at this.
>>>>>
>>>>>>
>>>>>>> This affected N2N connections not being in the "on-demand" mode only.
>>>>>
>>>>>>> While it is not really a show-stopper if someone is sitting in front
>>>>>>> of his/her/its IPFire machine, remote upgrades might be tricky.
>>>>>>
>>>>>> Indeed. Could you please investigate further whether this is or is not
>>>>> a regression introduced in this update?
>>>>>
>>>>> Will do.
>>>>>
>>>>>>
>>>>>>> Apart from that, this update looks quite good to me. The IPS changes
>>>>>>> are really noticeable, and bring a throughput I think I never
>>>>>>> experienced with IPFire and the IPS turned on. :-) This is certainly
>>>>>>> worth mentioning, as it finally makes the IPS suitable for everyone,
>>>>> hence massively increasing security without worrying too much of
>>>>> performance impacts.
>>>>>>>
>>>>>>> (For the sake of completeness: Unfortunately I did not yet have time
>>>>>>> do conduct a penetration test against this. Personally, I can imagine
>>>>>
>>>>>>> the IPS changes permitting some attacks after Suricata decided it
>>>>>>> cannot analyse a connection further. Switching protocols might be an
>>>>> issue, starting with TLS, while using something completely different
>>>>> afterwards.
>>>>>>
>>>>>> I expected you to bring this up a lot earlier and it is indeed a
>>>>> concern. Although I think it is a theoretical one:
>>>>>>
>>>>>> * You cannot really change back from a TLS connection on any
>>>>>> application that I am aware of
>>>>>> * Suricata only does this if it is very very certain that the
>>>>> connection can be bypassed and just hope the guys over there know what
>>>>> they are doing.
>>>>> Yes. Again, things are quite packet on my end - sorry.
>>>>>
>>>>> Indeed, it is a rather theoretical setup: If an attacker already got a
>>>>> TLS connection established so far that Suricata cannot look into it
>>>>> anymore, why not use that connection to conduct the malicious
>>>>> activities? There is no need to do protocol obfuscation anymore.
>>>>>
>>>>> Thanks, and best regards,
>>>>> Peter Müller
>>>>>
>>>>>>
>>>>>>> While I do not really consider this to be a critical attack surface,
>>>>>>> I wanted to look deeper into this as soon as I have some spare time
>>>>>>> to do so.)
>>>>>>>
>>>>>>> Tested IPFire functionalities in detail:
>>>>>>> - PPPoE dial-up via a DSL connection
>>>>>>> - IPsec (N2N connections only)
>>>>>>> - Squid (authentication enabled, using an upstream proxy)
>>>>>>> - OpenVPN (RW connections only)
>>>>>>> - IPS/Suricata (with Emerging Threats community ruleset enabled)
>>>>>>> - Guardian
>>>>>>> - Quality of Service
>>>>>>> - DNS (using DNS over TLS and strict QNAME minimisation)
>>>>>>> - Dynamic DNS
>>>>>>> - Tor (relay mode)
>>>>>>>
>>>>>>> I am looking forward to the release of Core Update 161.
>>>>>>>
>>>>>>> Thanks, and best regards,
>>>>>>> Peter Müller
>>>>>>
>>>>>> -Michael
>>>>>>
>>>>>
>>>>>
>