[luatex] LuaLaTeX's performance

[Khaled Hosny]

On Fri, Nov 19, 2010 at 09:23:46PM +0100, Pander wrote:
> On 2010-11-19 21:15, Khaled Hosny wrote:
> >> The other thing is the performance. Now it could be that the current
> >> executable is not optimised and is doing loads of debug logging.
> >> However, my font catalogue is a good test case.
> >>
> >> XeLaTeX can process it in two parts because other it runs out of memory.
> >> However, LuaLaTeX is hitting some sort of performance barrier halfway my
> >> document and after using up almost 6 GB of memory I have to kill off the
> >> process. Please see attached screenshots of memory usage.
> > 
> > LuaTeX will happily use all memory available to it (unlike most other
> > TeX engines which have hard-coded memory limit, IIUC). The current font
> > loading code is not very memory efficient and for certain fonts e.g.
> > fonts with huge kerning tables (some font developers like to say their
> > fonts have tens of thousands of kerning pairs, like if it is a feature
> > or something). I think post-texlive2010 releases of luatex made some
> > progress in this area, but I'm not sure if Hans started making use of it
> > or not.
> > 
> >> How can LuaLaTeX's performance be improved? Is the current executable in
> >> TLContrib a debug build with maximum logging?
> > 
> > I don't think so, I don't think it would be that relevant either. If you
> > can isolate the most offending fonts it might help in identifying and
> > possibly fixing the cause.
> 
> It it difficult to isolate since gradually memory gets eaten. Can I run
> it with profiling or logging to report which font is using how much memory?

Here is a quickly hacked script that takes a list of fonts and reports,
in kb, the memory used for each:

for _,v in ipairs(arg) do
    local b, m, f, t
    b = string.explode(v, "/")
    b = b[#b]
    m = collectgarbage("count")
    f = fontloader.open(v)
    if f then
        t = fontloader.to_table(f)
        fontloader.close(f)
        print(math.floor(collectgarbage("count") - m), b)
        t = nil
        collectgarbage('collect')
    end
end

Save it to some file and cal it something like:

$ texlua foo.lua `find /path/to/fonts`

Regards,
 Khaled

-- 
 Khaled Hosny
 Arabic localiser and member of Arabeyes.org team
 Free font developer