[luatex] Lua module cache manager (a solution)

[Kalrish Bäakjen]

> 1. Do you have any benchmarks to encourage LuaTeX users to try it? On
> your test file, I get a speedup of about 400ms, which is not nothing, so
> I think it's a promising improvement!

No, sorry, I don't have any proper benchmarks. I just noticed a
similar improvement and that convinced me that it was worth sharing.

> 2. I admit I'm not really familiar with the --lua command line argument
> of LuaTeX

LuaTeX can run a Lua script at startup, while it is still parsing the
command line and before the typesetting process has started, even
before the everyjob is executed (that's why tex.sprint and friends are
not available and the jobname isn't yet known). This is explained in
detail in section 2.1.3 of the LuaTeX manual. The --lua option simply
specifies a script to run.

> is there any way to turn this code into a regular package?

It would be possible to turn this code into a LaTeX package; it would
just require swapping callback.register for the LaTeX kernel wrapper
and redoing option handling. I can do that when I have time again.
However, that would imply losing the ability to catch some modules,
because, by the time the code set up its trick, some modules (namely
ltluatex and expl3) would have been loaded already. The ltluatex
module, for instance, is set to be loaded in everyjob by the LaTeX
kernel; it is catched by this system only because it [this system] is
set up at the Lua initialization script stage (before everyjob is
executed and, thus, before ltluatex is require()d).

On the other hand, turning this code in a LaTeX package wouldn't be
very useful, because it would require changes to every document. I
think it would be better that the LaTeX kernel itself set it to be
loaded in everyjob, just as it does with ltluatex (before it, of
course, so as to cache it as well). This would also allow to use the
jobname to compute the cache file name; that way, there could be a
"${jobname}.lmc" (where 'lmc' stands for "Lua Module Cache") just as
there is a "${jobname}.aux".

That said, I consider this to be nothing more than a hack. If this
cache manager were somehow used in all documents by default, it would
cache roughly the same modules (luaotfload, fontspec,...) again and
again. Moreover, this system doesn't know if a Lua source file
changed; this is best handled by a build system. In my opinion, the
proper solution is that the distribution itself byte-compiles Lua
files at installation time (taking into account which bytecode formats
are needed) and somehow adds a Lua bytecode searcher with priority 2
(just as this cache manager does) to load them. If the Lua source
searcher is not replaced, this would actually be innocuous and
backwards-compatible.

> And maybe using a Git(hub?) repository? We can make one in the lualatex Github organisation if you want

I have just uploaded everything to a GitHub repository[1]. Feel free
to use it however you want. A few fixes went to the script, by the
way, and I put some comments here and there :).

Thank you!

Kalrish


1:  https://github.com/kalrish/luatex-lua-module-cache-manager