[tlbuild] luatex-1.17.0 update

[Max Chernoff]

Hi Karl, Mojca, 


>     context.exe and mtxrun.exe are copies of luametatex.exe for Windows.
>     Please update context.exe and mtxrun.exe.
> 
> Right, thanks. Done (r67046).
> 
> Mojca, it would be nice if {context,mtxrun}.exe could be some kind of
> wrapper instead of duplicating a 3MB binary, but I suppose Hans is
> uninterested in worrying about that.


Hans talks about this a little in the "Following Up" manual:

   https://www.pragma-ade.nl/general/manuals/followingup.pdf#page=82
   
   Now when we move to LMTX we need to make sure that the method that we
   choose is acceptable for distributions but also nicely consistent over
   platforms. We only have one binary luametatex with all messy logic
   removed and no second face like metaluatex. When it is copied to another
   instance (or linked) it will load the script with its own name when it
   finds one. So on Windows we now have:
   
   texmf-win64/bin/luametatex.exe		medium program
   texmf-win64/bin/mtxrun.exe		copy (or link) of luametatex
   texmf-win64/bin/mtxrun.lua		large lua file
   texmf-win64/bin/context.exe		copy (or link) of luametatex
   texmf-win64/bin/context.lua		small lua file
   
   and in Unix:
   
   texmf-linux-64/bin/luametatex		medium program
   texmf-linux-64/bin/mtxrun		copy (or link) of luametatex
   texmf-linux-64/bin/mtxrun.lua		large lua file
   texmf-linux-64/bin/context		copy (or link) of luametatex
   texmf-linux-64/bin/context.lua		small lua file
   
   So, luametatex[.exe], mtxrun[.exe] and context[.exe] are all the same.
   On both platforms there is mtxrun.lua (with suffix) and on both we also
   use the same runner approach. The context.lua script is really small and
   just sets the script command line argument before loading mtxrun.lua
   from the same path. In the case of copied binaries: keep in mind that
   the three copies together are not (much) larger than the luatex and
   texlua pair (especially when you take additional libraries into
   account).
   
   The disadvantage of using copies is that one can forget to copy with an
   update, but the fact that one can use them might be easier for
   installers. It's up to those who create the installers.

> I wonder if our standard wrapper would magically work. Probably not.


It probably would, although I'm not really sure. From the same manual:

   Just for the record: we cannot really use batch files here because we
   need to know the original command, and when run from a script that is
   normally not known. It works to some extend but for instance when
   started indirectly from an editor it can fail, depending on how that
   editor is calling programs. Therefore the stub is the most robust
   method.
   
I think that we may be able to just use symlinks on Windows now though.
If I remember correctly, one of the Windows 10 updates a few years ago
enabled unprivileged symlinks by default. I believe that this was 4ish
years ago, so it should be included in all currently supported Windows
versions.

At the very least, Windows Vista onwards supports symlinks, but they do
require Admin privileges to create. The initial release of Windows 10
includes a registry/group policy setting that you can enable to permit
unprivileged symlinks. So this would for sure work with Admin/system
installations, although we probably don't want different logic for
Admin/system versus user installations.

This is all based off of memory, so you'll probably want to check with
someone with more recent Windows experience. Maybe I'm imagining all of
this :)

>    [mtxrun.lua in Windows]
> 
> Right, done now.
> 
> This requirement of having data files (mtxrun.lua, context.lua) in the
> bin directories is error-prone, as you can see, besides being
> philosophically wrong. Any chance of having the binaries find their .lua
> files in the normal tree, that is, along TEXMFSCRIPTS in kpse-land and
> whatever the equivalent is in ConTeXt-land?


All the file-finding code is in mtxrun.lua, so how would luametatex know
where to look? It doesn't use kpathsea, so something would need to be
hardcoded in the binary, but the point of LuaMetaTeX is to include as
little as possible in the binary.

I agree that the current mtxrun.lua/context.lua situation is a bit of a
mess, but it is kind of nice how much core behaviour you can change
without recompiling anything. It's a trade-off; not necessarily a bad
one, but not really compatible with the rest of TL.

The current code looks something like this (actually written in C, but
I'll use Lua here):

   local script = "./" .. arg[0] .. ".lua"
   if file.is_readable(script) then
       dofile(script)
   else
       error(script .. " not found")
   end
   
Hans might be willing to do something like:

   local script = arg[0] .. ".lua"
   if file.is_readable("./" .. script) then
       dofile("./" .. script)
   elseif file.is_readable("./kpsewhich")
       script = io.popen("./kpsewhich --format=texmfscripts " .. script):read("*a")
   end
   
   if file.is_readable(script) then
       dofile(script)
   else
       error(script .. " not found")
   end

   
That should be sufficient for our purposes, and it shouldn't affect the
LMTX standalone at all. I can propose this to Hans if this seems like a
good idea to you all.

(Also, here's another vaguely similar issue that I reported:
   https://www.mail-archive.com/ntg-context@ntg.nl/msg104600.html
)

Thanks,
-- Max