[Mojolicious] Reducing daemon memory footprint?

[Charlie Brady]

Some Mojolicious::Lite code I've written has been adapted by another group
in my company to run on their embedded controller (limited CPU and limited
memory). They are now asking for my advice on how to reduce the memory
footprint of their daemon. It's consuming about 50MB at the moment.

Does anyone have some advice on where to start looking at where memory is
being used and how to minimise it?

Thanks

[Stefan Adams]

I shouldn't involve myself in this topic because I haven't the knowledge,
but it sounds fascinating. Wouldn't the majority of the 50MB memory
footprint be from the Perl interpreter itself? Wouldn't the question be,
how to reduce the size of the Perl interpreter? Different compile options?
e.g. I remember once compiling a "minimal" perl that was 2MB. I imagine
that would probably be insufficient to run Mojolicious, though I really
haven't a clue, but it suggests to me that it's possible to reduce the size
of the perl interpreter and, therefore it seems, the size of the
Mojolicious daemon?

On Wed, Sep 19, 2018, 8:19 PM Charlie Brady <

Post by Charlie Brady
Some Mojolicious::Lite code I've written has been adapted by another group
in my company to run on their embedded controller (limited CPU and limited
memory). They are now asking for my advice on how to reduce the memory
footprint of their daemon. It's consuming about 50MB at the moment.
Does anyone have some advice on where to start looking at where memory is
being used and how to minimise it?
Thanks

[Charlie Brady]

Hi Stefan, thanks for trying to help.

I know the perl executable will make some contribution to the memory
footprint, but not much.

I can measure that separately by running a minimum perl program and
looking at its resource usage. If I run 'perl -e "<>"', I see this in
/proc/$pid/status:

...
VmPeak: 5476 kB
VmSize: 5476 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 3184 kB
VmRSS: 3184 kB
VmData: 468 kB
VmStk: 132 kB
VmExe: 4 kB
VmLib: 4232 kB
VmPTE: 24 kB
VmPMD: 0 kB
VmSwap: 0 kB
Threads: 1
...

The same data from my M::L daemon is:

...
VmPeak: 135548 kB
VmSize: 133540 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 67016 kB
VmRSS: 65988 kB
VmData: 86940 kB
VmStk: 176 kB
VmExe: 4 kB
VmLib: 31664 kB
VmPTE: 132 kB
VmPMD: 0 kB
VmSwap: 0 kB
Threads: 6
...

CPAN has a few modules which might help gather some stats
(Memory::Process, Memory::Stats, Memory::Usage). '-Dm' flag to perl might
help too, if perl is compiled with DEBUG enabled.

I notice that I can disable Mojolicious response cacheing; it'll be
interesting to see what difference that makes.

Post by Stefan Adams
I shouldn't involve myself in this topic because I haven't the knowledge,
but it sounds fascinating. Wouldn't the majority of the 50MB memory
footprint be from the Perl interpreter itself? Wouldn't the question be,
how to reduce the size of the Perl interpreter? Different compile options?
e.g. I remember once compiling a "minimal" perl that was 2MB. I imagine
that would probably be insufficient to run Mojolicious, though I really
haven't a clue, but it suggests to me that it's possible to reduce the size
of the perl interpreter and, therefore it seems, the size of the
Mojolicious daemon?
On Wed, Sep 19, 2018, 8:19 PM Charlie Brady <

Post by Charlie Brady
Some Mojolicious::Lite code I've written has been adapted by another group
in my company to run on their embedded controller (limited CPU and limited
memory). They are now asking for my advice on how to reduce the memory
footprint of their daemon. It's consuming about 50MB at the moment.
Does anyone have some advice on where to start looking at where memory is
being used and how to minimise it?
Thanks

[Charlie Brady]

There's some good general advice from our friend Brian D Foy here:

https://stackoverflow.com/questions/9733146/tips-for-keeping-perl-memory-usage-low

To bring this more closely focussed on Mojolicious, my daemon is loading a
ew dozen other Mojolicious::Lite applications using the Mount plugin. And
all of those are using a common plugin. I'm wondering how much duplication
there is within the sandboxing of each mounted application, and what might
be possible to optimize the sharing of data/compiled code between the
instances.

Post by Charlie Brady
Hi Stefan, thanks for trying to help.
I know the perl executable will make some contribution to the memory
footprint, but not much.
I can measure that separately by running a minimum perl program and
looking at its resource usage. If I run 'perl -e "<>"', I see this in
...
VmPeak: 5476 kB
VmSize: 5476 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 3184 kB
VmRSS: 3184 kB
VmData: 468 kB
VmStk: 132 kB
VmExe: 4 kB
VmLib: 4232 kB
VmPTE: 24 kB
VmPMD: 0 kB
VmSwap: 0 kB
Threads: 1
...
...
VmPeak: 135548 kB
VmSize: 133540 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 67016 kB
VmRSS: 65988 kB
VmData: 86940 kB
VmStk: 176 kB
VmExe: 4 kB
VmLib: 31664 kB
VmPTE: 132 kB
VmPMD: 0 kB
VmSwap: 0 kB
Threads: 6
...
CPAN has a few modules which might help gather some stats
(Memory::Process, Memory::Stats, Memory::Usage). '-Dm' flag to perl might
help too, if perl is compiled with DEBUG enabled.
I notice that I can disable Mojolicious response cacheing; it'll be
interesting to see what difference that makes.

Post by Stefan Adams
I shouldn't involve myself in this topic because I haven't the knowledge,
but it sounds fascinating. Wouldn't the majority of the 50MB memory
footprint be from the Perl interpreter itself? Wouldn't the question be,
how to reduce the size of the Perl interpreter? Different compile options?
e.g. I remember once compiling a "minimal" perl that was 2MB. I imagine
that would probably be insufficient to run Mojolicious, though I really
haven't a clue, but it suggests to me that it's possible to reduce the size
of the perl interpreter and, therefore it seems, the size of the
Mojolicious daemon?
On Wed, Sep 19, 2018, 8:19 PM Charlie Brady <

Post by Charlie Brady
Some Mojolicious::Lite code I've written has been adapted by another group
in my company to run on their embedded controller (limited CPU and limited
memory). They are now asking for my advice on how to reduce the memory
footprint of their daemon. It's consuming about 50MB at the moment.
Does anyone have some advice on where to start looking at where memory is
being used and how to minimise it?
Thanks

[Stefan Adams]

Wow! That's fascinating! Thanks for sharing! I'm surprised by how much
the memory footprint grows for a minimal perl program to one running M::L!

On Thu, Sep 20, 2018 at 2:57 PM Charlie Brady <

Post by Charlie Brady
Hi Stefan, thanks for trying to help.
I know the perl executable will make some contribution to the memory
footprint, but not much.
I can measure that separately by running a minimum perl program and
looking at its resource usage. If I run 'perl -e "<>"', I see this in
...
VmPeak: 5476 kB
VmSize: 5476 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 3184 kB
VmRSS: 3184 kB
VmData: 468 kB
VmStk: 132 kB
VmExe: 4 kB
VmLib: 4232 kB
VmPTE: 24 kB
VmPMD: 0 kB
VmSwap: 0 kB
Threads: 1
...
...
VmPeak: 135548 kB
VmSize: 133540 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 67016 kB
VmRSS: 65988 kB
VmData: 86940 kB
VmStk: 176 kB
VmExe: 4 kB
VmLib: 31664 kB
VmPTE: 132 kB
VmPMD: 0 kB
VmSwap: 0 kB
Threads: 6
...
CPAN has a few modules which might help gather some stats
(Memory::Process, Memory::Stats, Memory::Usage). '-Dm' flag to perl might
help too, if perl is compiled with DEBUG enabled.
I notice that I can disable Mojolicious response cacheing; it'll be
interesting to see what difference that makes.

Post by Stefan Adams
I shouldn't involve myself in this topic because I haven't the knowledge,
but it sounds fascinating. Wouldn't the majority of the 50MB memory
footprint be from the Perl interpreter itself? Wouldn't the question be,
how to reduce the size of the Perl interpreter? Different compile

options?

Post by Stefan Adams
e.g. I remember once compiling a "minimal" perl that was 2MB. I imagine
that would probably be insufficient to run Mojolicious, though I really
haven't a clue, but it suggests to me that it's possible to reduce the

size

Post by Stefan Adams
of the perl interpreter and, therefore it seems, the size of the
Mojolicious daemon?
On Wed, Sep 19, 2018, 8:19 PM Charlie Brady <

Post by Charlie Brady
Some Mojolicious::Lite code I've written has been adapted by another

group

Post by Stefan Adams

Post by Charlie Brady
in my company to run on their embedded controller (limited CPU and

limited

Post by Stefan Adams

Post by Charlie Brady
memory). They are now asking for my advice on how to reduce the memory
footprint of their daemon. It's consuming about 50MB at the moment.
Does anyone have some advice on where to start looking at where memory

is

Post by Stefan Adams

Post by Charlie Brady
being used and how to minimise it?
Thanks

[Charlie Brady]

Post by Stefan Adams
Wow! That's fascinating! Thanks for sharing! I'm surprised by how
much the memory footprint grows for a minimal perl program to one
running M::L!

Bear in mind this is one M::L program which has loaded (via the Mount
plugin) a few dozen other M::L programs and then run for a few months
(mostly idle).

FWIW, this is on a PPC processor (perl 5.22.0, Mojolicious 7.33).

Post by Stefan Adams
On Thu, Sep 20, 2018 at 2:57 PM Charlie Brady <

Post by Charlie Brady
Hi Stefan, thanks for trying to help.
I know the perl executable will make some contribution to the memory
footprint, but not much.
I can measure that separately by running a minimum perl program and
looking at its resource usage. If I run 'perl -e "<>"', I see this in
...
VmPeak: 5476 kB
VmSize: 5476 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 3184 kB
VmRSS: 3184 kB
VmData: 468 kB
VmStk: 132 kB
VmExe: 4 kB
VmLib: 4232 kB
VmPTE: 24 kB
VmPMD: 0 kB
VmSwap: 0 kB
Threads: 1
...
...
VmPeak: 135548 kB
VmSize: 133540 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 67016 kB
VmRSS: 65988 kB
VmData: 86940 kB
VmStk: 176 kB
VmExe: 4 kB
VmLib: 31664 kB
VmPTE: 132 kB
VmPMD: 0 kB
VmSwap: 0 kB
Threads: 6
...
CPAN has a few modules which might help gather some stats
(Memory::Process, Memory::Stats, Memory::Usage). '-Dm' flag to perl might
help too, if perl is compiled with DEBUG enabled.
I notice that I can disable Mojolicious response cacheing; it'll be
interesting to see what difference that makes.

Post by Stefan Adams
I shouldn't involve myself in this topic because I haven't the knowledge,
but it sounds fascinating. Wouldn't the majority of the 50MB memory
footprint be from the Perl interpreter itself? Wouldn't the question be,
how to reduce the size of the Perl interpreter? Different compile

options?

Post by Stefan Adams
e.g. I remember once compiling a "minimal" perl that was 2MB. I imagine
that would probably be insufficient to run Mojolicious, though I really
haven't a clue, but it suggests to me that it's possible to reduce the

size

Post by Stefan Adams
of the perl interpreter and, therefore it seems, the size of the
Mojolicious daemon?
On Wed, Sep 19, 2018, 8:19 PM Charlie Brady <

Post by Charlie Brady
Some Mojolicious::Lite code I've written has been adapted by another

group

Post by Stefan Adams

Post by Charlie Brady
in my company to run on their embedded controller (limited CPU and

limited

Post by Stefan Adams

Post by Charlie Brady
memory). They are now asking for my advice on how to reduce the memory
footprint of their daemon. It's consuming about 50MB at the moment.
Does anyone have some advice on where to start looking at where memory

is

Post by Stefan Adams

Post by Charlie Brady
being used and how to minimise it?
Thanks

[Charlie Brady]

This is why I need to run a persistent daemon rather than use CGI mode.

***@jy-cx:~# time perl -e exit

real 0m0.035s
user 0m0.000s
sys 0m0.030s
***@jy-cx:~# time perl -MMojolicious::Lite -e exit

real 0m11.011s
user 0m8.690s
sys 0m0.400s
***@jy-cx:~#

Looking at an strace there doesn't seem to be a single culprit for the
slow startup - there's just a lot of files to find, open, read, parse.

***@jy-cx:~# grep open ml.trace | wc -l
211
***@jy-cx:~#

...
***@jy-cx:~# grep open ml.trace | grep 12:21:26 | wc -l
16
***@jy-cx:~# grep open ml.trace | grep 12:21:27 | wc -l
28
***@jy-cx:~# grep open ml.trace | grep 12:21:28 | wc -l
16
***@jy-cx:~# grep open ml.trace | grep 12:21:29 | wc -l
19
***@jy-cx:~# grep open ml.trace | grep 12:21:30 | wc -l
13
***@jy-cx:~# grep open ml.trace | grep 12:21:31 | wc -l
2
***@jy-cx:~# grep open ml.trace | grep 12:21:32 | wc -l
15
***@jy-cx:~# grep open ml.trace | grep 12:21:33 | wc -l
8
***@jy-cx:~# grep open ml.trace | grep 12:21:34 | wc -l
18
***@jy-cx:~# grep open ml.trace | grep 12:21:35 | wc -l
24
***@jy-cx:~# grep open ml.trace | grep 12:21:36 | wc -l
8
***@jy-cx:~# grep open ml.trace | grep 12:21:37 | wc -l
19
***@jy-cx:~# grep open ml.trace | grep 12:21:38 | wc -l
11
***@jy-cx:~# grep open ml.trace | grep 12:21:39 | wc -l
14
***@jy-cx:~# grep open ml.trace | grep 12:21:40 | wc -l
0
***@jy-cx:~#
...

Initialising Time::HiRes::HiRes seems to be the single individual
parse/init step:

12:21:30.734184 open("/usr/lib/perl/5.22.0/Pod/Simple/BlackBox.pm",
O_RDONLY|O_L
ARGEFILE) = 16
12:21:31.607709 open("/usr/lib/perl/5.22.0/Time/HiRes.pm",
O_RDONLY|O_LARGEFILE) = 12
12:21:31.643254 open("/usr/lib/perl/5.22.0//auto/Time/HiRes/HiRes.so",
O_RDONLY|O_CLOEXEC) = 12
12:21:32.356811 open("/usr/lib/perl/vendor_perl/5.22.0/Mojo/Log.pm",
O_RDONLY|O_LARGEFILE) = 7

Post by Charlie Brady

Post by Stefan Adams
Wow! That's fascinating! Thanks for sharing! I'm surprised by how
much the memory footprint grows for a minimal perl program to one
running M::L!

Bear in mind this is one M::L program which has loaded (via the Mount
plugin) a few dozen other M::L programs and then run for a few months
(mostly idle).
FWIW, this is on a PPC processor (perl 5.22.0, Mojolicious 7.33).

Post by Stefan Adams
On Thu, Sep 20, 2018 at 2:57 PM Charlie Brady <

Post by Charlie Brady
Hi Stefan, thanks for trying to help.
I know the perl executable will make some contribution to the memory
footprint, but not much.
I can measure that separately by running a minimum perl program and
looking at its resource usage. If I run 'perl -e "<>"', I see this in
...
VmPeak: 5476 kB
VmSize: 5476 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 3184 kB
VmRSS: 3184 kB
VmData: 468 kB
VmStk: 132 kB
VmExe: 4 kB
VmLib: 4232 kB
VmPTE: 24 kB
VmPMD: 0 kB
VmSwap: 0 kB
Threads: 1
...
...
VmPeak: 135548 kB
VmSize: 133540 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 67016 kB
VmRSS: 65988 kB
VmData: 86940 kB
VmStk: 176 kB
VmExe: 4 kB
VmLib: 31664 kB
VmPTE: 132 kB
VmPMD: 0 kB
VmSwap: 0 kB
Threads: 6
...
CPAN has a few modules which might help gather some stats
(Memory::Process, Memory::Stats, Memory::Usage). '-Dm' flag to perl might
help too, if perl is compiled with DEBUG enabled.
I notice that I can disable Mojolicious response cacheing; it'll be
interesting to see what difference that makes.

Post by Stefan Adams
I shouldn't involve myself in this topic because I haven't the knowledge,
but it sounds fascinating. Wouldn't the majority of the 50MB memory
footprint be from the Perl interpreter itself? Wouldn't the question be,
how to reduce the size of the Perl interpreter? Different compile

options?

Post by Stefan Adams
e.g. I remember once compiling a "minimal" perl that was 2MB. I imagine
that would probably be insufficient to run Mojolicious, though I really
haven't a clue, but it suggests to me that it's possible to reduce the

size

Post by Stefan Adams
of the perl interpreter and, therefore it seems, the size of the
Mojolicious daemon?
On Wed, Sep 19, 2018, 8:19 PM Charlie Brady <

Post by Charlie Brady
Some Mojolicious::Lite code I've written has been adapted by another

group

Post by Stefan Adams

Post by Charlie Brady
in my company to run on their embedded controller (limited CPU and

limited

Post by Stefan Adams

Post by Charlie Brady
memory). They are now asking for my advice on how to reduce the memory
footprint of their daemon. It's consuming about 50MB at the moment.
Does anyone have some advice on where to start looking at where memory

is

Post by Stefan Adams

Post by Charlie Brady
being used and how to minimise it?
Thanks

[sri]

Post by Charlie Brady
This is why I need to run a persistent daemon rather than use CGI mode.
real 0m0.035s
user 0m0.000s
sys 0m0.030s
real 0m11.011s
user 0m8.690s
sys 0m0.400s
Looking at an strace there doesn't seem to be a single culprit for the
slow startup - there's just a lot of files to find, open, read, parse.

If that's the case, why not just fatpack your mojo app?

--
sebastian

[Charlie Brady]

Post by sri
If that's the case, why not just fatpack your mojo app?

Um, because I didn't know about fatpack? :-)

Thanks, looks a good suggestion. The system will continue to be slow, but
not as slow...