[Mojolicious] Mojo::Log and UTF-8

[Alex Povolotsky]

Hello

How do I send UTF-8 constants to Mojo::Log? It forcefully encodes
everything so UTF-8 goes with double encoding and unreadable. Trying to
decode UTF-8 cat

$log->info(decode('UTF-8', 'тест лПга'));

results in

Use of uninitialized value $_[0] in join or string at
/usr/local/lib/perl5/site_perl/Mojo/Log.pm line 55.

Looks like something is wrong...

[Alex Povolotsky]

Quite interesting. The simple script

=== cut mojolog ===

#!/usr/bin/env perl
use strict;
use utf8;
use Mojo::Log;
my $log = new Mojo::Log;
print "ПрПстП прОМт\n";
$log->info('тест лПга');
=== cut mojolog ===

yields fine results on all my servers but one.

On it, I'm getting

% perl mojolog
ПрПстП прОМт
[Sat Oct 6 15:22:43 2018] [info] ÑￜеÑￜÑￜ лПга

The second line is clearly wrong and seems to be badly encoded. Same
version of OS, Perl, Mojolicious, same set of environment variables. What
else could influence unicode processing?

суббПта, 6 Пктября 2018 г., 13:35:48 UTC+3 пПльзПватель Alex Povolotsky

Post by Alex Povolotsky
Hello
How do I send UTF-8 constants to Mojo::Log? It forcefully encodes
everything so UTF-8 goes with double encoding and unreadable. Trying to
decode UTF-8 cat
$log->info(decode('UTF-8', 'тест лПга'));
results in
Use of uninitialized value $_[0] in join or string at
/usr/local/lib/perl5/site_perl/Mojo/Log.pm line 55.
Looks like something is wrong...

[Dan Book]

The script is not quite correct. I suspect if you add "use warnings" you
will see a "Wide character in print" warning, because you are not encoding
your unicode characters to UTF-8 before printing them. However, on the
server where you are seeing the double encoding, perhaps you have a
PERL5OPT environment variable which is setting something like "-CSAD" and
thus encoding all output handles by default (this should not be globally
set, because it can mess with modules that don't expect it like this).

-Dan

Post by Alex Povolotsky
Quite interesting. The simple script
=== cut mojolog ===
#!/usr/bin/env perl
use strict;
use utf8;
use Mojo::Log;
my $log = new Mojo::Log;
print "ПрПстП прОМт\n";
$log->info('тест лПга');
=== cut mojolog ===
yields fine results on all my servers but one.
On it, I'm getting
% perl mojolog
ПрПстП прОМт
[Sat Oct 6 15:22:43 2018] [info] ÑￜеÑￜÑￜ лПга
The second line is clearly wrong and seems to be badly encoded. Same
version of OS, Perl, Mojolicious, same set of environment variables. What
else could influence unicode processing?
суббПта, 6 Пктября 2018 г., 13:35:48 UTC+3 пПльзПватель Alex Povolotsky

Post by Alex Povolotsky
Hello
How do I send UTF-8 constants to Mojo::Log? It forcefully encodes
everything so UTF-8 goes with double encoding and unreadable. Trying to
decode UTF-8 cat
$log->info(decode('UTF-8', 'тест лПга'));
results in
Use of uninitialized value $_[0] in join or string at
/usr/local/lib/perl5/site_perl/Mojo/Log.pm line 55.
Looks like something is wrong...

--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.

[Alex Povolotsky]

So I should not binmode STDERR, ':utf8' if I'm using Mojo::Log and always
explicitly encode() data?

Post by Dan Book
The script is not quite correct. I suspect if you add "use warnings" you
will see a "Wide character in print" warning, because you are not encoding
your unicode characters to UTF-8 before printing them. However, on the
server where you are seeing the double encoding, perhaps you have a
PERL5OPT environment variable which is setting something like "-CSAD" and
thus encoding all output handles by default (this should not be globally
set, because it can mess with modules that don't expect it like this).
-Dan

Post by Alex Povolotsky
Quite interesting. The simple script
=== cut mojolog ===
#!/usr/bin/env perl
use strict;
use utf8;
use Mojo::Log;
my $log = new Mojo::Log;
print "ПрПстП прОМт\n";
$log->info('тест лПга');
=== cut mojolog ===
yields fine results on all my servers but one.
On it, I'm getting
% perl mojolog
ПрПстП прОМт
[Sat Oct 6 15:22:43 2018] [info] ÑￜеÑￜÑￜ лПга
The second line is clearly wrong and seems to be badly encoded. Same
version of OS, Perl, Mojolicious, same set of environment variables. What
else could influence unicode processing?
суббПта, 6 Пктября 2018 г., 13:35:48 UTC+3 пПльзПватель Alex Povolotsky

Post by Alex Povolotsky
Hello
How do I send UTF-8 constants to Mojo::Log? It forcefully encodes
everything so UTF-8 goes with double encoding and unreadable. Trying to
decode UTF-8 cat
$log->info(decode('UTF-8', 'тест лПга'));
results in
Use of uninitialized value $_[0] in join or string at
/usr/local/lib/perl5/site_perl/Mojo/Log.pm line 55.
Looks like something is wrong...

--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.

[Dan Book]

Mojo::Log already encodes data by default. Anything you send to stderr
outside of that should be encoded if necessary.

-Dan

Post by Alex Povolotsky
So I should not binmode STDERR, ':utf8' if I'm using Mojo::Log and always
explicitly encode() data?

Post by Dan Book
The script is not quite correct. I suspect if you add "use warnings" you
will see a "Wide character in print" warning, because you are not encoding
your unicode characters to UTF-8 before printing them. However, on the
server where you are seeing the double encoding, perhaps you have a
PERL5OPT environment variable which is setting something like "-CSAD" and
thus encoding all output handles by default (this should not be globally
set, because it can mess with modules that don't expect it like this).
-Dan

Post by Alex Povolotsky
Quite interesting. The simple script
=== cut mojolog ===
#!/usr/bin/env perl
use strict;
use utf8;
use Mojo::Log;
my $log = new Mojo::Log;
print "ПрПстП прОМт\n";
$log->info('тест лПга');
=== cut mojolog ===
yields fine results on all my servers but one.
On it, I'm getting
% perl mojolog
ПрПстП прОМт
[Sat Oct 6 15:22:43 2018] [info] ÑￜеÑￜÑￜ лПга
The second line is clearly wrong and seems to be badly encoded. Same
version of OS, Perl, Mojolicious, same set of environment variables. What
else could influence unicode processing?
суббПта, 6 Пктября 2018 г., 13:35:48 UTC+3 пПльзПватель Alex Povolotsky

Post by Alex Povolotsky
Hello
How do I send UTF-8 constants to Mojo::Log? It forcefully encodes
everything so UTF-8 goes with double encoding and unreadable. Trying to
decode UTF-8 cat
$log->info(decode('UTF-8', 'тест лПга'));
results in
Use of uninitialized value $_[0] in join or string at
/usr/local/lib/perl5/site_perl/Mojo/Log.pm line 55.
Looks like something is wrong...

--
You received this message because you are subscribed to the Google
Groups "Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.