[PATCH 1/2] header.pl: Add utf-8 handling into cleanhtml command

[PATCH 1/2] header.pl: Add utf-8 handling into cleanhtml command

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 2044 bytes --]

- existing cleanhtml command does not handle diacritical charcters such as umlauts, acute,
   grave and circumflex accents.
- In bug 12395 the problem was resolved by adding decode before and encode after the
   cleanhtml command in dns.cgi
- Suggestion from @Michael Tremer was to add the decode and encode sections into the
   actual cleanhtml subroutine in header.pl
- This patch submission is the execution of that suggestion.
- This will ensure that whenever cleanhtml is used for any remark in a WUI page it will
   handle diacritical charcters.
- Tested out on my vm testbed system and confirmed to be working when cleanhtml has the
   encode and decode lines.
- Combined with this patch is another one that changes the dns.cgi to remove the decode
   and encode entries added into the cgi code.

Suggested-by: Michael Tremer <michael.tremer(a)ipfire.org>
Tested-by: Adolf Belka <adolf.belka(a)ipfire.org>
Signed-off-by: Adolf Belka <adolf.belka(a)ipfire.org>
---
 config/cfgroot/header.pl | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/config/cfgroot/header.pl b/config/cfgroot/header.pl
index a67ff92ee..66b49e411 100644
--- a/config/cfgroot/header.pl
+++ b/config/cfgroot/header.pl
@@ -16,6 +16,7 @@ use File::Basename;
 use HTML::Entities();
 use Socket;
 use Time::Local;
+use Encode;
 
 our %color = ();
 &General::readhash("/srv/web/ipfire/html/themes/ipfire/include/colors.txt", \%color);
@@ -365,8 +366,13 @@ sub escape($) {
 sub cleanhtml {
 	my $outstring =$_[0];
 	$outstring =~ tr/,/ / if not defined $_[1] or $_[1] ne 'y';
-
-	return escape($outstring);
+	# decode the UTF-8 text so that characters with diacritical marks such as
+	# umlauts are treated correctly by the escape command
+	$outstring = &Encode::decode("UTF-8",$outstring);
+	escape($outstring);
+	# encode the text back to UTF-8 after running the escape command
+	$outstring = &Encode::encode("UTF-8",$outstring);
+	return $outstring;
 }
 
 sub connectionstatus
-- 
2.45.2

[PATCH 2/2] dns.cgi: Remove the decode and encode lines as now integrated in header.pl

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 1498 bytes --]

- decode and encode lines have now been integrated into the cleanhtml subroutine in
   header.pl so that all uses of cleanhtml will be able to handle diacritical characters

Tested-by: Adolf Belka <adolf.belka(a)ipfire.org>
Signed-off-by: Adolf Belka <adolf.belka(a)ipfire.org>
---
 html/cgi-bin/dns.cgi | 11 -----------
 1 file changed, 11 deletions(-)

diff --git a/html/cgi-bin/dns.cgi b/html/cgi-bin/dns.cgi
index 1181523d4..0d3b14797 100644
--- a/html/cgi-bin/dns.cgi
+++ b/html/cgi-bin/dns.cgi
@@ -21,7 +21,6 @@
 
 use strict;
 use IO::Socket;
-use Encode;
 
 # enable only the following on debugging purpose
 #use warnings;
@@ -143,18 +142,8 @@ if (($cgiparams{'SERVERS'} eq $Lang::tr{'save'}) || ($cgiparams{'SERVERS'} eq $L
 	# Go further if there was no error.
 	if ( ! $errormessage) {
 		# Check if a remark has been entered.
-
-		# decode the UTF-8 text so that characters with diacritical marks such as
-		# umlauts are treated correctly by the following cleanhtml command
-		$cgiparams{'REMARK'} = decode("UTF-8", $cgiparams{'REMARK'});
-
-		# run the REMARK text through cleanhtml to ensure all unsafe html characters
-		# are correctly encoded to their html entities
 		$cgiparams{'REMARK'} = &Header::cleanhtml($cgiparams{'REMARK'});
 
-		# encode the text back to UTF-8 after running the cleanhtml command
-		$cgiparams{'REMARK'} = encode("UTF-8", $cgiparams{'REMARK'});
-
 		my %dns_servers = ();
 		my $id;
 		my $status;
-- 
2.45.2

Re: [PATCH 1/2] header.pl: Add utf-8 handling into cleanhtml command

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 2569 bytes --]

Have tested this patch set out on all the menu items that have remarks and are using cleanhtml in Core Update 187 Testing.

All remarks were able to have full range of diacritical characters.
I tested with

ß Ф Ч < > Ӧ ü £ μ ô ò ó õ å ä ã â á à

as the remark and it worked everywhere tested, which it did not without this patch set.

Regards,
Adolf.

On 17/06/2024 13:12, Adolf Belka wrote:
> - existing cleanhtml command does not handle diacritical charcters such as umlauts, acute,
>     grave and circumflex accents.
> - In bug 12395 the problem was resolved by adding decode before and encode after the
>     cleanhtml command in dns.cgi
> - Suggestion from @Michael Tremer was to add the decode and encode sections into the
>     actual cleanhtml subroutine in header.pl
> - This patch submission is the execution of that suggestion.
> - This will ensure that whenever cleanhtml is used for any remark in a WUI page it will
>     handle diacritical charcters.
> - Tested out on my vm testbed system and confirmed to be working when cleanhtml has the
>     encode and decode lines.
> - Combined with this patch is another one that changes the dns.cgi to remove the decode
>     and encode entries added into the cgi code.
> 
> Suggested-by: Michael Tremer <michael.tremer(a)ipfire.org>
> Tested-by: Adolf Belka <adolf.belka(a)ipfire.org>
> Signed-off-by: Adolf Belka <adolf.belka(a)ipfire.org>
> ---
>   config/cfgroot/header.pl | 10 ++++++++--
>   1 file changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/config/cfgroot/header.pl b/config/cfgroot/header.pl
> index a67ff92ee..66b49e411 100644
> --- a/config/cfgroot/header.pl
> +++ b/config/cfgroot/header.pl
> @@ -16,6 +16,7 @@ use File::Basename;
>   use HTML::Entities();
>   use Socket;
>   use Time::Local;
> +use Encode;
>   
>   our %color = ();
>   &General::readhash("/srv/web/ipfire/html/themes/ipfire/include/colors.txt", \%color);
> @@ -365,8 +366,13 @@ sub escape($) {
>   sub cleanhtml {
>   	my $outstring =$_[0];
>   	$outstring =~ tr/,/ / if not defined $_[1] or $_[1] ne 'y';
> -
> -	return escape($outstring);
> +	# decode the UTF-8 text so that characters with diacritical marks such as
> +	# umlauts are treated correctly by the escape command
> +	$outstring = &Encode::decode("UTF-8",$outstring);
> +	escape($outstring);
> +	# encode the text back to UTF-8 after running the escape command
> +	$outstring = &Encode::encode("UTF-8",$outstring);
> +	return $outstring;
>   }
>   
>   sub connectionstatus