From mboxrd@z Thu Jan  1 00:00:00 1970
From: Adolf Belka <adolf.belka@ipfire.org>
To: development@lists.ipfire.org
Subject: Re: [PATCH 1/2] header.pl: Add utf-8 handling into cleanhtml command
Date: Wed, 17 Jul 2024 10:58:41 +0200
Message-ID: <ed6316b9-2a60-4452-894b-8fa328226216@ipfire.org>
In-Reply-To: <20240617111236.2926-1-adolf.belka@ipfire.org>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============2666131410462033072=="
List-Id: <development.lists.ipfire.org>

--===============2666131410462033072==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Have tested this patch set out on all the menu items that have remarks and ar=
e using cleanhtml in Core Update 187 Testing.

All remarks were able to have full range of diacritical characters.
I tested with

=C3=9F =D0=A4 =D0=A7 < > =D3=A6 =C3=BC =C2=A3 =CE=BC =C3=B4 =C3=B2 =C3=B3 =C3=
=B5 =C3=A5 =C3=A4 =C3=A3 =C3=A2 =C3=A1 =C3=A0

as the remark and it worked everywhere tested, which it did not without this =
patch set.

Regards,
Adolf.

On 17/06/2024 13:12, Adolf Belka wrote:
> - existing cleanhtml command does not handle diacritical charcters such as =
umlauts, acute,
>     grave and circumflex accents.
> - In bug 12395 the problem was resolved by adding decode before and encode =
after the
>     cleanhtml command in dns.cgi
> - Suggestion from @Michael Tremer was to add the decode and encode sections=
 into the
>     actual cleanhtml subroutine in header.pl
> - This patch submission is the execution of that suggestion.
> - This will ensure that whenever cleanhtml is used for any remark in a WUI =
page it will
>     handle diacritical charcters.
> - Tested out on my vm testbed system and confirmed to be working when clean=
html has the
>     encode and decode lines.
> - Combined with this patch is another one that changes the dns.cgi to remov=
e the decode
>     and encode entries added into the cgi code.
>=20
> Suggested-by: Michael Tremer <michael.tremer(a)ipfire.org>
> Tested-by: Adolf Belka <adolf.belka(a)ipfire.org>
> Signed-off-by: Adolf Belka <adolf.belka(a)ipfire.org>
> ---
>   config/cfgroot/header.pl | 10 ++++++++--
>   1 file changed, 8 insertions(+), 2 deletions(-)
>=20
> diff --git a/config/cfgroot/header.pl b/config/cfgroot/header.pl
> index a67ff92ee..66b49e411 100644
> --- a/config/cfgroot/header.pl
> +++ b/config/cfgroot/header.pl
> @@ -16,6 +16,7 @@ use File::Basename;
>   use HTML::Entities();
>   use Socket;
>   use Time::Local;
> +use Encode;
>  =20
>   our %color =3D ();
>   &General::readhash("/srv/web/ipfire/html/themes/ipfire/include/colors.txt=
", \%color);
> @@ -365,8 +366,13 @@ sub escape($) {
>   sub cleanhtml {
>   	my $outstring =3D$_[0];
>   	$outstring =3D~ tr/,/ / if not defined $_[1] or $_[1] ne 'y';
> -
> -	return escape($outstring);
> +	# decode the UTF-8 text so that characters with diacritical marks such as
> +	# umlauts are treated correctly by the escape command
> +	$outstring =3D &Encode::decode("UTF-8",$outstring);
> +	escape($outstring);
> +	# encode the text back to UTF-8 after running the escape command
> +	$outstring =3D &Encode::encode("UTF-8",$outstring);
> +	return $outstring;
>   }
>  =20
>   sub connectionstatus

--===============2666131410462033072==--