[luatex] Lua module cache manager (a solution)

[Kalrish Bäakjen]

Hello,

I have written a Lua initialization script that manages a cache for Lua
modules. It adds a new module searcher that looks up the required module's
name in the cache and, if it finds the module loader there, retrieves the
associated bytecode, thus skipping the steps between Lua source and Lua
bytecode and making document building a bit faster. The script places this
new module searcher at position 2, sending all default searchers with a
priority of 2 or more one step back. The second default searcher,
responsible for loading Lua modules (from .lua files, I mean, and not from
C libraries), is thus put at position 3, and reached only if the required
Lua module wasn't cached (in which case it would have been loaded by the
new cache searcher). The script therefore hooks into it so as to keep track
of those uncached modules and dump them later. The cache is itself a Lua
table in which the keys are the module names and the values are their
bytecodes, and works exactly like the aux file (as far as my understanding
goes, at least), that is, it is read, and updated if it's needed, every run.

The script is attached, along with a test file. (I hope the attachments are
ok. Gmail is quite impolite, someone should really teach him manners…)

To use it:

$  lualatex --interaction=nonstopmode --lua=lua-module-cache-manager.lua --
main.tex

It also works with LuaJITTeX. Both version 0.80, revision 5238.

You may also want to consider byte-compiling the script:

$  texluac -s -o lua-module-cache-manager.texluabc
-- lua-module-cache-manager.lua
$  texluajitc -b lua-module-cache-manager.lua
lua-module-cache-manager.texluabc

all that's needed then is to change the argument to the --lua parameter to
match that of the output file. This is best handled by a build system, of
course.

The parameters of the script are:

 *  --lua-module-cache-file=/path/to/the/cache/file.extension
 (default=lua-module-cache.bin)
 *  --lua-module-cache-mode=b/t  (default=b)
       Whether the cache should be stored as Lua source (t) or Lua bytecode
(b), which should be a bit faster.

You can check that Lua module files (.lua) aren't read by passing
--recorder to the TeX engine. If you see .lua files (e.g.
lualibs-basic.lua) in the resulting .fls log, that's because they aren't
loaded with require and thus can't be catched with this system. They are
probably loaded with dofile or perhaps loadfile, but they could also be
loaded with io.read and load or some other crazy combination.

There are some limitations and caveats. Some of them are explained in
comments. I hope to improve the quality of the source.

In the hope that this will be useful or interesting to you,
Kalrish
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tug.org/pipermail/luatex/attachments/20160212/1463f04f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lua-module-cache-manager.lua
Type: application/octet-stream
Size: 7758 bytes
Desc: not available
URL: <http://tug.org/pipermail/luatex/attachments/20160212/1463f04f/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: main.tex
Type: application/x-tex
Size: 777 bytes
Desc: not available
URL: <http://tug.org/pipermail/luatex/attachments/20160212/1463f04f/attachment-0001.tex>