[luatex] [lltx] Speeding up otf lua cache file load
Paul Isambert
zappathustra at free.fr
Fri Mar 29 23:35:54 CET 2013
Reinhard Kotucha <reinhard.kotucha at web.de> a écrit:
> On 2013-03-28 at 10:09:18 +0100, Paul Isambert wrote:
>
> > Reinhard Kotucha <reinhard.kotucha at web.de> a écrit:
> > > I copied this thread from [lltx] to [luatex] because I fear that the
> > > luaotfload maintainer is not listening on [lltx]
> > >
> > >
> > > On 2013-03-23 at 17:59:23 +0400, LS wrote:
> > >
> > > > Hi, all,
> > >
> > > > I think, that compiling with otf font would be very quick if
> > > > it would be possible to create a separate
> > > > "temp-<font-name>.lua" in a current directory only with the
> > > > project used glyphs. It could be a package
> > > > f.e. "glyphstoluahash.sty" witch would do all that. Maybe, on
> > > > the first run it would be slow, while it will pick all these
> > > > glyphs, but on other runs, if "temp-<font-name>.lua" exists
> > > > in a current directory, it would do nothing, but
> > > > luaotfload > would use this local hash file.
> > > > How many different glyphs uses articles, probably less than
> > > > 500 from one font. So it would be a huge improvement.
> > > >
> > > > I see in a otfl-luat-dum.lua that $TEXMFCACHE can be set for
> > > > the cache look up, but when I set it to:
> > > > TEXMFCACHE=.//;$TEXMFVAR luatex crates a local empty
> > > > directories ./luatex-cache/generic. I tried to copy font cache
> > > > file in local directory and in a created one, but it still
> > > > loads cache files from
> > > > $TEXMFVAR\luatex-cache\generic\fonts\otf\
> > > >
> > > > So, what do you think about that idea?
> > >
> > > A better approach is to pre-compile the Lua files. A
> > > pre-compiled Lua file is loaded significantly faster because it
> > > doesn't have to be parsed. I observed that loading a .luc file
> > > is up to 30 times as fast as loading .lua file.
> > >
> > > You can try yourself. Since Lua recognizes whether a file is
> > > pre-compiled, regardless of the extension, you can do
> > >
> > > texluac -s -o foo.luc foo.lua
> > > move foo.luc foo.lua
> > >
> > > for testing.
> > >
> > > What I nowadys do in my own programs in order to load large files with
> > > dofile() is to provide a wrapper, let's call it do_file(). The
> > > wrapper takes a file name as an argument. If the file name has an
> > > extension, then the argument is passed directly to dofile().
> > > If no extension is given, the wrapper looks for foo.luc and foo.lua.
> > > If both are present, the newer one is used.
> > >
> > > I think that this approach can be used by luaotfload too without much
> > > effort. The filename cache can be large, but the Lua representation
> > > of OTFs can be huge. Pre-compiling the Lua files helps in both cases.
> > >
> > > Vafa, AFAIR you offered to maintain luaotfload some time ago. What do
> > > you think about this suggestion? See the function do_file() below.
> > >
> > > BTW, luaotfload creates files containing absolute paths. In my
> > > current application it's a pain. Thus the function below is using
> > > kpathsea in order to locate files. It's the preferred way to locate
> > > files in TeX Live anyway.
> >
> > I've never really tried to make font loading faster, but in my
> > experience the bulk of the time spent processing comes from setting
> > kern tables to the demanded size. Loading Minion Pro twenty times
> > with:
> >
> > \directlua{T = os.time()}
> > % Loading the font 20 times.
> > \directlua{texio.write_nl("TIME: " .. os.time()-T)}
> >
> > returns "2" if kern tables are set, "0" otherwise. Using pre-compiled
> > Lua files doesn't make that faster (the loading of the cache file is
> > negligible compared to its processing). A font that is lighter on kern
> > tables (e.g. Palatino Linotype) also loads much faster.
>
> Linas tried the function I attached to my previous mail and sent me
> the result:
>
> | Here's what I've got testing one font:
> |
> | Lua size: 13MB
> | Luc size: ~8MB.
> |
> | Time loading lua: ~5.8s.
> | Time loading luc: ~1.5s.
>
> He only loaded the file and didn't process it further.
> However, I can't reproduce this at all. Pre-compiling *all* cached
> fonts takes 2.4s normally and 3.6s with an empty filesystem cache.
>
> Loading:
>
> temp-charissilr.lua 2.4MB 160ms
> temp-charissilr.luc 1.5MB 45ms
>
> Thus, on my system there will be only a noticable difference if many
> large fonts are used. However, I can't neglect Linas' experience,
> though I have no idea why it's *that* slow there.
I hadn't tried large fonts; I've just tested Gabriola, and indeed it
runs faster when pre-compiled, although Linas's figures are quite
surprising.
> > One solution I've considered but never implemented is to cache fonts
> > at a given size, the one most used by the user (e.g. 10pt); then it
> > would be faster by default, unless loading at a different size (the
> > font could also be cached with several sizes too).
>
> The program I wrote recently had to be optimized for speed because
> it's run in the background and people expect a reaction immediately
> after pressing a button. The solution was to load luaotfload and
> declare the fonts when the format file is created. I only need two
> fonts (CharisSIL, Regular and Bold) at three different sizes. Thus,
> this approach is possible. I currently avoid fontspec. It blows up
> the format file significantly and slows down everything.
>
> When the format file is loaded, os.clock() returns 0.4s. If I remember
> correctly, whithout the preloaded fonts it returned 0.25s. The
> CharisSIL fonts and their Lua representations are quite large, much
> larger than Latin Modern. So I'm quite satisfied with the result.
> Generating a PDF file takes 0.6 or 0.7 seconds, depending on its
> content. The former version (fontspec and without a dedicated format
> file) took at least 1.6s. I can't determine the total time under
> Windows, but it's slower there, especially if it's run the first time.
Dumping formats in order to save time is probably a very good
solution, even though not very widespread as far as I can tell (I
myself don't use it, but I'm constantly modifying the files I use...).
> There is not much text in the PDF files, just a few small tables.
> Maybe this is the reason why the font subsetting doesn't take much
> time. The main content of the file are ECG plots created with
> \pdfliteral. BTW, my first attempt was to use Metapost but it was
> much too slow. Processing data with Lua and writing PDF stuff
> directly into the PDF file is extremely fast. It's similar to what
> you described in a TUGboat article. The difference is that your
> approach is more Metapost orientated while mine is more PostScript
> orientated, just because I don't need a user interface.
Nice to know that article rang some bells. I think there's a lot to do
in that area, and I'll even bet the next new graphic kid on the TeX
block will be in Lua.
> > Of course, reducing cache files to a subset of the glyphs would
> > also make things faster, provided only the relevant parts of the
> > kern table are processed. How much faster it would be, I don't
> > know, though.
>
> What would this subset contain? The glyphs you're currently using in
> a particular document or all the glyphs needed for a particular
> language?
I don't know (probably the latter), it was the OP's idea. My idea of
saving time is to stop compiling and checking the PDF after every sentence. :)
Best,
Paul
More information about the luatex
mailing list