[luatex] [lltx] Speeding up otf lua cache file load

[Paul Isambert]

Reinhard Kotucha <reinhard.kotucha at web.de> a écrit:
> I copied this thread from [lltx] to [luatex] because I fear that the
> luaotfload maintainer is not listening on [lltx]
>
>
> On 2013-03-23 at 17:59:23 +0400, LS wrote:
>
>  >  Hi, all,
>
>  > I think, that compiling  with otf font would be very quick if it
>  > would be possible  to create a separate "temp-<font-name>.lua" in a
>  > current directory only with the project used glyphs.  It could be a
>  > package f.e. "glyphstoluahash.sty" witch would do all that. Maybe,
>  > on the first run it would be slow, while it will pick all these
>  > glyphs, but on other runs, if  "temp-<font-name>.lua" exists in a
>  > current directory, it would do nothing, but luaotfload
would use
>  > this local hash file.  How many different glyphs uses articles,
>  > probably less than 500 from one font. So it would be a huge
>  > improvement.
>  >
>  > I see in a otfl-luat-dum.lua that $TEXMFCACHE can be set for the
>  > cache look up, but when I set it to: TEXMFCACHE=.//;$TEXMFVAR
>  > luatex crates a local empty directories ./luatex-cache/generic. I
>  > tried to copy font cache file in local directory and in a created
>  > one, but it still loads cache files from 
>  > $TEXMFVAR\luatex-cache\generic\fonts\otf\
>  >
>  > So, what do you think about that idea?
>
> A better approach is to pre-compile the Lua files.  A pre-compiled Lua
> file is loaded significantly faster because it doesn't have to be
> parsed.  I observed that loading a .luc file is up to 30 times as fast
> as loading .lua file.
>
> You can try yourself.  Since Lua recognizes whether a file is
> pre-compiled, regardless of the extension, you can do
>
>   texluac -s -o foo.luc foo.lua
>   move foo.luc foo.lua
>
> for testing.
>
> What I nowadys do in my own programs in order to load large files with
> dofile() is to provide a wrapper, let's call it do_file().  The
> wrapper takes a file name as an argument.  If the file name has an
> extension, then the argument is passed directly to dofile().
> If no extension is given, the wrapper looks for foo.luc and foo.lua.
> If both are present, the newer one is used.
>
> I think that this approach can be used by luaotfload too without much
> effort.  The filename cache can be large, but the Lua representation
> of OTFs can be huge.  Pre-compiling the Lua files helps in both cases.
>
> Vafa, AFAIR you offered to maintain luaotfload some time ago.  What do
> you think about this suggestion?  See the function do_file() below.
>
> BTW, luaotfload creates files containing absolute paths.  In my
> current application it's a pain.  Thus the function below is using
> kpathsea in order to locate files.  It's the preferred way to locate
> files in TeX Live anyway.

I've never really tried to make font loading faster, but in my
experience the bulk of the time spent processing comes from setting
kern tables to the demanded size. Loading Minion Pro twenty times
with:

  \directlua{T = os.time()}
  % Loading the font 20 times.
  \directlua{texio.write_nl("TIME: " .. os.time()-T)}

returns "2" if kern tables are set, "0" otherwise. Using pre-compiled
Lua files doesn't make that faster (the loading of the cache file is
negligible compared to its processing). A font that is lighter on kern
tables (e.g. Palatino Linotype) also loads much faster.

One solution I've considered but never implemented is to cache fonts
at a given size, the one most used by the user (e.g. 10pt); then it
would be faster by default, unless loading at a different size (the
font could also be cached with several sizes too).

Of course, reducing cache files to a subset of the glyphs would also
make things faster, provided only the relevant parts of the kern table
are processed. How much faster it would be, I don't know, though.

Best,
Paul