[luatex] LuaTeX precompilation

[Axel Kittenberger]

I got an alternative idea which would be a reasonably sized hack-project to
achieve what I suppose Henrik ultimately wants: to minimize the latency
from hitting the compile button in an TeX-IDE like TeXstudio until being
able to view the PDF.

Instead of dumping/restoring the state of luatex, just keep preparing a
process ahead of time and pause it, until you want it to compile the bulk
of the TeX code past the preamble you are currently working on.

The best of this is: (theoretically) you would not need to change/patch
luatex at all!

To have it wait for say a HUP signal (or USR1) you could just use
\directlua{} to require the luaposix module (
https://github.com/luaposix/luaposix) and at the place you want it to
pause, do a \directlua{} and call the C-API function pselect() without any
file descriptors to watch but only a respective signal mask.

Just note due to the way file descriptors work, it will likely keep the
main file the way it was.
so your main document should look something like this (supposing LaTeX)
\documentclass{..}
[...preamble with lots of \usepackage call ]
\directlua{..pselect() stuff to wait for signal.. }
\input{myactualmain.tex}

and then not change this header at all (or if you do remember that the
prepared luatex process has not yet prepared the changed preamble and you
need to compile twice).

The bulk work of this hack is writing two programs (in any language of your
choice):
* one user systemd service daemon that will open a named unix socket,
starts the luatex process and then waits for the TeX wrapper (2nd program)
to connect to that socket, then sends the signal to the paused luatex
process and keeps forwarding its stdout/stderr streams and eventually the
exit code to the wrapper.
* the second program does this wrapping, to be replaced as the "tex
compiler" in the TeX-IDE, but instead of processing TeX, it will connect to
that socket and forwards its stdout/stderr to the IDE.

I think this forwarding is a tiny bit more elaborate than just piping
streams, since you'd want to stream stdout and stderr over one fd and also
eventually inform of the proper exit code for the wrapper to exit too.
However it should be manageable, I'd suggest a super simple packet based
approach, where the the first byte is one ascii letter '1' for stdout, '2'
for stderr and 'E' for exited following with a number (however you want to
encode it) on how many bytes are to follow (or in case of 'E' the exitcode
value).

Kind regards, Axel

PS: I tried now for over an hour for a minimal proof of concept to have
lualatex successfully require('posix'), but I just can't get this damn
paths to work.
LUA_PATH=/usr/share/lua/5.3 LUA_CPATH=/usr/lib/x86_64-linux-gnu/lua/5.3
lualatex --shell-escape luahack.tex

with

\documentclass{article}
\usepackage{luapackageloader}
\usepackage{luacode}
\begin{luacode*}
print( 'X' );
local posix = require( 'posix' );
print( 'X', posix );
\end{luacode*}
\input{mymain.tex}

will still result into "module 'posix' not found:" (to plain lua
interpreter requires this without issues so it is well installed,
everything 5.3)

albeit all complaints of not finding posix/init.lua and posix.so are gone
by setting the paths.
Okay, this is how far I can go with this.

So sorrry no proof of concept from me. At worst you'll have to make your
own Lua c binary .so module to link to to get that pselect() call, still
better than patching lualatex in my opinion, and yes I did think of
--shell-escape which should be required.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://tug.org/pipermail/luatex/attachments/20231221/8ef87188/attachment.htm>