[luatex] Allowing or switching to string indexes in Lua bytecode registers

[David Carlisle]

On 4 September 2015 at 23:43, Kalrish Bäakjen <kalrish.baakjen at gmail.com> wrote:
> Currently, all array elements of the lua.bytecode table, which must be
> functions, are dumped as Lua bytecode to the format file by iniTeX. This
> functionality is useful if one is dumping his own format from a split
> preamble (to decrease compilation time) and uses custom Lua code: to avoid
> loading it every time, one can assign consecutive slots of lua.bytecode to
> the functions returned by loadfile, and then, in the main document, execute
> those functions to recreate the code. More information about this can be
> found in section 4.8.1 ("LUA bytecode registers") of the LuaTeX Reference
> Manual[1] and in this StackExchange question[2].
>
> I think this functionality is also interesting for packages because it would
> allow for a clean way of dumping them to a format, if the user so desires.
> Currently, packages which use Lua code, such as fontspec or polyglossia,
> can't be "preloaded" in a format file because the Lua code that they load is
> not saved and, thus, cannot be restored in the actual run. The same person
> who asked [2] already considered using \everyjob (in combination with the
> \directlua call that loads the code) to solve this issue. While that
> solution would indeed make it possible to dump those packages, it would be a
> trick in the sense that Lua code would be loaded every run. Since users
> (usually) dump their own format to save time, this doesn't seem to fit.
>
> However, as far as I know, this functionality is restricted to user code and
> can't be effectively leveraged by packages, because the indexes of
> lua.bytecode are numbers and there is no reasonable relation between numbers
> and package names. Not even hashes of the names could be used, as far as I
> understand, because iniLuaTeX only dumps the array part of the table (that
> is: the consecutive non-nil slots, starting from 1).
>
> Therefore, I suggest that this mechanism (the Lua bytecode registers dumping
> logic) is either extended to allow the use of string indexes, or entirely
> switched to them (only strings allowed as indexes). That way, packages could
> do (Lua code follows):


The engine does not need to change if  tex allocation macros are available for
bytecodes as they are for other register types, as then any name/number mapping
is available in TeX code and so will be dumped into the format.


In the ltluatex code that is planned to be part of the next LaTeX
release and form the basis of
a luatexbase update, there was already such an allocator for lua
functions, but the
bytecode allocator was just in lua.  We have just adjusted it so that
the allocation count
for bytecodes is also stored in a tex count register.

I think this addresses your use case, you can see the code for the
core ltluatex in the latex svn
and for a luatexbase update based on this in github.


http://latex-project.org/svnroot/latex2e-public/trunk/base/

and

https://github.com/josephwright/ltluatexsupp

Like the original luatexbase code, the ltluatex code is designed to
run in plain TeX despite its latex roots.

David