[luatex] Discussion about packaging binary modules for LuaTeX (& ConTeXt Meeting 2015 in France)

[Mojca Miklavec]

(was: lltxplatform integration)

Dear Élie,

Assuming that you might be French (judging from your name) and given
that this year's ConTeXt Meeting is going to be in France (so
hopefully not too far from where you live):

    http://meeting.contextgarden.net/2015/

this might be an excellent opportunity for you to come to the meeting
and schedule a discussion about different ways (with pros and cons)
about packaging binary modules for LuaTeX.

We've been discussing some kind of automated builds of LuaTeX (and
other engines) for a while already, and planned to discuss how to
integrate automated "swiglib" builds as well. We don't have all the
answers ready at the moment, but some clever combination of
[semi-]automated builds (either using buildbots or humans who press a
button every now and then – which already works well for building the
latest releases of LuaTeX, MetaPost and XeTeX), TLContrib and some
smart scripting hackery might result in some acceptable user-friendly
way to get the desired functionality to the users.

You should consider joining us and hope for a fruitful discussion
together with all the "players" (LuaTeX developers, TLContrib hacker,
ConTeXt garden packagers etc.).

Mojca


On Thu, Jun 4, 2015 at 9:25 AM, Élie Roux wrote:
>> Before we include any such .so, there should be an option in luatex
>> about loading .so's (maybe it already exists?), and the default has to
>> be off.  Otherwise our efforts at security are lost, so far as I can see.
>
> That's a difficult question...
>
>> Which means, it seems to me, that the feature is only useful for
>> particular users doing particular things, not for shipping stuff to be
>> used by everyone.
>
> It is useful for Windows users
>
>> As I understand it, ConTeXt gets the list of OS fonts by running a
>> script at install time, or really mtxrun --generate time.  I expect we
>> could figure out something similar for lualatex.  Ideally even re-using
>> the same file if it exists ...
>
> Sadly, Windows has a mechanism that stocks some installed fonts only in
> the registry, and not in a standard directory. Fonts installed this way
> are not handled by ConTeXt. There aren't many cases where users install
> fonts this way, but (AFAIK) the mechanism is quite recent and there
> might be more and more demand on this...
>
> There are currently two ways to access the registry in LuaTeX:
>  - loading a .so file that calls the Windows API (lua-winreg,
> lualatex-platform, etc.)
>  - calling the "reg" command
>
> The second is already forbidden by shell_escape_commands (which is
> good), and if you forbid the first one, this means that users will never
> be able to have all their fonts found by TeX.
>
> The only other way this could be dealt with would be through fontconfig,
> but fontconfig doesn't support the fonts in the registry, doesn't plan
> to, and making a patch is not easy (fonts not in a standard directory
> doesn't fit well with fontconfig architecture).[1][2]
>
>> Overall, I think just about any alternative -- a script, a system call,
>> a whole separate program, whatever -- is preferable to a .so.
>
> This means that all modules people need will have to be statically
> linked with LuaTeX? This seems a difficult choice, and I'm not sure the
> LuaTeX team is willing to do this... This will also forbid experiments
> with harfbuzz, etc. Maybe there could be a list of allowed extensions,
> that would respect openin_any and openout_any (lualatex-platform does,
> it doesn't read nor write any file), and also removing "." from CLUAINPUTS ?
>
>> They'd only be updated once a year, or more precisely when the binaries
>> are updated.  There is no miracle way to get them compiled on all
>> platforms on demand, and I don't want to impose repeated recompiling on
>> our builders (or manage such an effort).
>
> Ok
>
>> I also expect it will take
>> quite some effort to get figure out the first one.
>
> Indeed, hence my question...
>
> Thank you!
> --
> Elie
>
> [1]: http://lists.freedesktop.org/archives/fontconfig/2015-May/005510.html
> [2]: https://bugs.freedesktop.org/show_bug.cgi?id=90813