[luatex] [lltx] Speeding up otf lua cache file load

Paul Isambert zappathustra at free.fr
Fri Mar 29 23:35:54 CET 2013


Reinhard Kotucha <reinhard.kotucha at web.de> a écrit:
> On 2013-03-28 at 10:09:18 +0100, Paul Isambert wrote:
>
>  > Reinhard Kotucha <reinhard.kotucha at web.de> a écrit:
>  > > I copied this thread from [lltx] to [luatex] because I fear that the
>  > > luaotfload maintainer is not listening on [lltx]
>  > >
>  > >
>  > > On 2013-03-23 at 17:59:23 +0400, LS wrote:
>  > >
>  > >  >  Hi, all,
>  > >
>  > >  > I think, that compiling  with otf font would be very quick if
>  > >  > it would be possible  to create a separate
>  > >  > "temp-<font-name>.lua" in a current directory only with the
>  > >  > project used glyphs.  It could be a package
>  > >  > f.e. "glyphstoluahash.sty" witch would do all that. Maybe, on
>  > >  > the first run it would be slow, while it will pick all these
>  > >  > glyphs, but on other runs, if  "temp-<font-name>.lua" exists
>  > >  > in a current directory, it would do nothing, but
>  > >  > luaotfload > would use this local hash file.
>  > >  > How many different glyphs uses articles, probably less than
>  > >  > 500 from one font. So it would be a huge improvement.
>  > >  >
>  > >  > I see in a otfl-luat-dum.lua that $TEXMFCACHE can be set for
>  > >  > the cache look up, but when I set it to:
>  > >  > TEXMFCACHE=.//;$TEXMFVAR luatex crates a local empty
>  > >  > directories ./luatex-cache/generic. I tried to copy font cache
>  > >  > file in local directory and in a created one, but it still
>  > >  > loads cache files from 
>  > >  > $TEXMFVAR\luatex-cache\generic\fonts\otf\
>  > >  >
>  > >  > So, what do you think about that idea?
>  > >
>  > > A better approach is to pre-compile the Lua files.  A
>  > > pre-compiled Lua file is loaded significantly faster because it
>  > > doesn't have to be parsed.  I observed that loading a .luc file
>  > > is up to 30 times as fast as loading .lua file.
>  > >
>  > > You can try yourself.  Since Lua recognizes whether a file is
>  > > pre-compiled, regardless of the extension, you can do
>  > >
>  > >   texluac -s -o foo.luc foo.lua
>  > >   move foo.luc foo.lua
>  > >
>  > > for testing.
>  > >
>  > > What I nowadys do in my own programs in order to load large files with
>  > > dofile() is to provide a wrapper, let's call it do_file().  The
>  > > wrapper takes a file name as an argument.  If the file name has an
>  > > extension, then the argument is passed directly to dofile().
>  > > If no extension is given, the wrapper looks for foo.luc and foo.lua.
>  > > If both are present, the newer one is used.
>  > >
>  > > I think that this approach can be used by luaotfload too without much
>  > > effort.  The filename cache can be large, but the Lua representation
>  > > of OTFs can be huge.  Pre-compiling the Lua files helps in both cases.
>  > >
>  > > Vafa, AFAIR you offered to maintain luaotfload some time ago.  What do
>  > > you think about this suggestion?  See the function do_file() below.
>  > >
>  > > BTW, luaotfload creates files containing absolute paths.  In my
>  > > current application it's a pain.  Thus the function below is using
>  > > kpathsea in order to locate files.  It's the preferred way to locate
>  > > files in TeX Live anyway.
>  >
>  > I've never really tried to make font loading faster, but in my
>  > experience the bulk of the time spent processing comes from setting
>  > kern tables to the demanded size. Loading Minion Pro twenty times
>  > with:
>  >
>  >   \directlua{T = os.time()}
>  >   % Loading the font 20 times.
>  >   \directlua{texio.write_nl("TIME: " .. os.time()-T)}
>  >
>  > returns "2" if kern tables are set, "0" otherwise. Using pre-compiled
>  > Lua files doesn't make that faster (the loading of the cache file is
>  > negligible compared to its processing). A font that is lighter on kern
>  > tables (e.g. Palatino Linotype) also loads much faster.
>
> Linas tried the function I attached to my previous mail and sent me
> the result:
>
>  | Here's what I've got testing one font:
>  |
>  | Lua size: 13MB
>  | Luc size: ~8MB.
>  |
>  | Time loading lua: ~5.8s.
>  | Time loading luc: ~1.5s.
>
> He only loaded the file and didn't process it further.
> However, I can't reproduce this at all.  Pre-compiling *all* cached
> fonts takes 2.4s normally and 3.6s with an empty filesystem cache.
>
> Loading:
>
>   temp-charissilr.lua  2.4MB  160ms
>   temp-charissilr.luc  1.5MB   45ms
>
> Thus, on my system there will be only a noticable difference if many
> large fonts are used.  However, I can't neglect Linas' experience,
> though I have no idea why it's *that* slow there.

I hadn't tried large fonts; I've just tested Gabriola, and indeed it
runs faster when pre-compiled, although Linas's figures are quite
surprising.

>  > One solution I've considered but never implemented is to cache fonts
>  > at a given size, the one most used by the user (e.g. 10pt); then it
>  > would be faster by default, unless loading at a different size (the
>  > font could also be cached with several sizes too).
>
> The program I wrote recently had to be optimized for speed because
> it's run in the background and people expect a reaction immediately
> after pressing a button.  The solution was to load luaotfload and
> declare the fonts when the format file is created.  I only need two
> fonts (CharisSIL, Regular and Bold) at three different sizes.  Thus,
> this approach is possible.  I currently avoid fontspec.  It blows up
> the format file significantly and slows down everything.
>
> When the format file is loaded, os.clock() returns 0.4s.  If I remember
> correctly, whithout the preloaded fonts it returned 0.25s.  The
> CharisSIL fonts and their Lua representations are quite large, much
> larger than Latin Modern.  So I'm quite satisfied with the result.
> Generating a PDF file takes 0.6 or 0.7 seconds, depending on its
> content.  The former version (fontspec and without a dedicated format
> file) took at least 1.6s.  I can't determine the total time under
> Windows, but it's slower there, especially if it's run the first time.

Dumping formats in order to save time is probably a very good
solution, even though not very widespread as far as I can tell (I
myself don't use it, but I'm constantly modifying the files I use...).

> There is not much text in the PDF files, just a few small tables.
> Maybe this is the reason why the font subsetting doesn't take much
> time.  The main content of the file are ECG plots created with
> \pdfliteral.  BTW, my first attempt was to use Metapost but it was
> much too slow.  Processing data with Lua and writing PDF stuff
> directly into the PDF file is extremely fast.  It's similar to what
> you described in a TUGboat article.  The difference is that your
> approach is more Metapost orientated while mine is more PostScript
> orientated, just because I don't need a user interface.

Nice to know that article rang some bells. I think there's a lot to do
in that area, and I'll even bet the next new graphic kid on the TeX
block will be in Lua.

>  > Of course, reducing cache files to a subset of the glyphs would
>  > also make things faster, provided only the relevant parts of the
>  > kern table are processed. How much faster it would be, I don't
>  > know, though.
>
> What would this subset contain?  The glyphs you're currently using in
> a particular document or all the glyphs needed for a particular
> language?

I don't know (probably the latter), it was the OP's idea. My idea of
saving time is to stop compiling and checking the PDF after every sentence. :)

Best,
Paul


More information about the luatex mailing list