[luatex] Allowing or switching to string indexes in Lua bytecode registers

[Kalrish Bäakjen]

Currently, all array elements of the lua.bytecode table, which must be
functions, are dumped as Lua bytecode to the format file by iniTeX. This
functionality is useful if one is dumping his own format from a split
preamble (to decrease compilation time) and uses custom Lua code: to avoid
loading it every time, one can assign consecutive slots of lua.bytecode to
the functions returned by loadfile, and then, in the main document, execute
those functions to recreate the code. More information about this can be
found in section 4.8.1 ("LUA bytecode registers") of the LuaTeX Reference
Manual[1] and in this StackExchange question[2].

I think this functionality is also interesting for packages because it
would allow for a clean way of dumping them to a format, if the user so
desires. Currently, packages which use Lua code, such as fontspec or
polyglossia, can't be "preloaded" in a format file because the Lua code
that they load is not saved and, thus, cannot be restored in the actual
run. The same person who asked [2] already considered using \everyjob (in
combination with the \directlua call that loads the code) to solve this
issue. While that solution would indeed make it possible to dump those
packages, it would be a trick in the sense that Lua code would be loaded
every run. Since users (usually) dump their own format to save time, this
doesn't seem to fit.

However, as far as I know, this functionality is restricted to user code
and can't be effectively leveraged by packages, because the indexes of
lua.bytecode are numbers and there is no reasonable relation between
numbers and package names. Not even hashes of the names could be used, as
far as I understand, because iniLuaTeX only dumps the array part of the
table (that is: the consecutive non-nil slots, starting from 1).

Therefore, I suggest that this mechanism (the Lua bytecode registers
dumping logic) is either extended to allow the use of string indexes, or
entirely switched to them (only strings allowed as indexes). That way,
packages could do (Lua code follows):

-- If there are no errors, loadfile returns the compiled chunk as a
function; otherwise, it returns nil. assert returns all its arguments on
success. The combination of the two is a convenient idiom for
unconditionally loading Lua chunks. See [3] and [4]. So, if everything goes
well, assert shall return what loadfile returned, that is, the chunk as a
function (which should be executed later to "recreate" the chunk)
local shiny_package_chunk = assert( loadfile( 'my_things.lua' ) )
-- This is for savvy users: it stores the "magical function" in a bytecode
register. This function shall be dumped into the format so that, later, it
could be loaded and run
lua.setbytecode( 'shiny_package' , shiny_package_chunk )
-- Execute the "magical bootstrapping function" to load the chunk; this is
for users that don't dump their own format
shiny_package_chunk()

Then, on the TeX side, just this little piece for the adventurers:
% On every run, execute that function to load the chunk that was processed
earlier
\everyjob\expandafter{ \the\everyjob\directlua{
lua.getbytecode('shiny_package')() } }

Please, excuse any problems with this code; I have taken it from my project
and am no TeX expert.

I have tried to summarize it all. I'm aware that my approach to the problem
possibly conflicts with the approach taken for module loading by the
luatexbase-modutils package[5], but am hopeful that the aforementioned
package could be made to work with this method (I think that only the
\RequireLuaModule macro would have to be touched). I have also tried to
find the most about this matter so as to better know it (by the way,
there's also this conversation[6], which may be of interest to you).
Please, excuse any faults and share your ideas on this.

1: http://www.luatex.org/svn/trunk/manual/luatexref-t.pdf
2:
http://tex.stackexchange.com/questions/121728/how-can-i-dump-lua-code-to-a-format-file
3: http://www.lua.org/manual/5.2/manual.html#pdf-assert
4: http://www.lua.org/manual/5.2/manual.html#pdf-loadfile
5:
http://osl.ugr.es/CTAN/macros/luatex/generic/luatexbase/luatexbase-modutils.pdf
6:
http://compgroups.net/comp.text.tex/-luatex-possible-issue-with-graphicx-and-format/1937166
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tug.org/pipermail/luatex/attachments/20150905/f9d3eb9f/attachment.html>