[luatex] Allowing or switching to string indexes in Lua bytecode registers

[Kalrish Bäakjen]

Sorry for taking this long. I have been hacking on a "solution", but
it's not (yet, hopefully) working; I think it's because of a conflict
between the luatex and the luatexbase packages, but I have no clue
about why it's happening. It's not properly documented either;
basically, one has to run build.sh under bash. So this is more of a
status update than a proper response. I have uploaded the files in a
tarball[1], nonetheless; if you didn't mind having a look at them,
perhaps you could point me in the right direction.

After tinkering with this, I feel that so many hacks may not be worth
the benefit. After all, even if Lua code can be dumped behind the
scenes without requiring any change on the packages' side, calls that
change Lua state can't be dumped (with the current design). Hopefully,
a guru can shed some light on the matter :-).

Have a nice week!

1: https://drive.google.com/file/d/0B6J2lxNkmOp2blM1elZXMVdCRXM/view?usp=sharing

On Sun, Sep 6, 2015 at 12:43 AM, Hans Hagen <pragma at wxs.nl> wrote:
> On 9/5/2015 10:39 PM, Kalrish Bäakjen wrote:
>
>> Why can't arbitrary code be byte-compiled? Is what you refer to
>> related to what's mentioned in the LuaTeX Reference Manual (see
>> below)?
>>
>>> Section 4.8.1 (LUA bytecode registers)
>>> Note: The function must not contain any upvalues. Currently, functions
>>> containing upvalues can be stored (and their upvalues are set to nil), but
>>> this is an artifact of the current Lua implementation and thus subject to
>>> change.
>>
>>
>> I had been wondering what the consequences of this were.
>
>
> It means that you have to make sure that you initialize your code properly
> as you cannot assume upvalues to be saved for you (just like a normal lua
> run will end with forgetting all its variables, luatex will forget them and
> loading a couple of initializations/settings runtime is no big deal as lua
> is quite fast. Of course data you initialize in your code blob is
> initialized properly because the bytecode is just normal lua code. Only the
> code writer knows what is worth initializing anyway so saving upvalues some
> way would introduce issues.)
>
>>> On the other hand a more general scheme could probably work although
>>> I'd need to try building a test case to follow the details below.
>>> I see the general direction you are suggesting but some of the details
>>> escape me:-)
>>
>>
>> I can write some code, but not today :-). If the "custom searcher"
>> solution fits, I think that something at the core of LuaTeX (either
>> LuaTeX itself or the LuaTeX format) would require changes, which would
>> indeed not be something to decide lightly. Feel free to discuss
>> further.
>
>
> Nothing will change in that department i.e. the team will not change the
> loader (if only because different macro packages have different demands).
>
> If you want a different loader you can write it in lua and hook it into the
> normal loader. Keep in mind that luatex operates in tds and organization of
> files in there is well defined and controlled by variables in the
> configuration so that different macro packages don't clash and also loading
> can be restricted.
>
> Hans
>
> -----------------------------------------------------------------
>                                           Hans Hagen | PRAGMA ADE
>               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
>     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
>                                              | www.pragma-pod.nl
> -----------------------------------------------------------------