[luatex] [lltx] Speeding up otf lua cache file load

[Reinhard Kotucha]

I copied this thread from [lltx] to [luatex] because I fear that the
luaotfload maintainer is not listening on [lltx]


On 2013-03-23 at 17:59:23 +0400, LS wrote:

 >  Hi, all,

 > I think, that compiling  with otf font would be very quick if it
 > would be possible  to create a separate "temp-<font-name>.lua" in a
 > current directory only with the project used glyphs.  It could be a
 > package f.e. "glyphstoluahash.sty" witch would do all that. Maybe,
 > on the first run it would be slow, while it will pick all these
 > glyphs, but on other runs, if  "temp-<font-name>.lua" exists in a
 > current directory, it would do nothing, but luaotfload would use
 > this local hash file.  How many different glyphs uses articles,
 > probably less than 500 from one font. So it would be a huge
 > improvement.
 > 
 > I see in a otfl-luat-dum.lua that $TEXMFCACHE can be set for the
 > cache look up, but when I set it to: TEXMFCACHE=.//;$TEXMFVAR
 > luatex crates a local empty directories ./luatex-cache/generic. I
 > tried to copy font cache file in local directory and in a created
 > one, but it still loads cache files from 
 > $TEXMFVAR\luatex-cache\generic\fonts\otf\
 > 
 > So, what do you think about that idea?

A better approach is to pre-compile the Lua files.  A pre-compiled Lua
file is loaded significantly faster because it doesn't have to be
parsed.  I observed that loading a .luc file is up to 30 times as fast
as loading .lua file.

You can try yourself.  Since Lua recognizes whether a file is
pre-compiled, regardless of the extension, you can do

  texluac -s -o foo.luc foo.lua
  move foo.luc foo.lua

for testing.

What I nowadys do in my own programs in order to load large files with
dofile() is to provide a wrapper, let's call it do_file().  The
wrapper takes a file name as an argument.  If the file name has an
extension, then the argument is passed directly to dofile().  
If no extension is given, the wrapper looks for foo.luc and foo.lua.
If both are present, the newer one is used.

I think that this approach can be used by luaotfload too without much
effort.  The filename cache can be large, but the Lua representation
of OTFs can be huge.  Pre-compiling the Lua files helps in both cases.

Vafa, AFAIR you offered to maintain luaotfload some time ago.  What do
you think about this suggestion?  See the function do_file() below.

BTW, luaotfload creates files containing absolute paths.  In my
current application it's a pain.  Thus the function below is using
kpathsea in order to locate files.  It's the preferred way to locate
files in TeX Live anyway.

Regards,
  Reinhard

------------------------------------------------------------------
function do_file (file)
  if file:find('\.lu[ac]$') then -- extension provided
     return dofile(kpse.find_file(file))
  end

  -- no extension:

  local function mtime (f)
    return lfs.attributes(f).modification
  end

  local luafile=kpse.find_file(file..'.lua') or false
  local lucfile=kpse.find_file(file..'.luc') or false

  local function exists (f) 
     if f then return true end 
  end 

  local filename=luafile

  if exists(lucfile) then 
     if exists(luafile) then -- compare timestamps, the newer file wins
        if mtime(lucfile) > mtime(luafile) then
	   filename=lucfile
        end
     else -- no .lua file
        filename=lucfile
     end
  end
  return dofile(filename)
end
------------------------------------------------------------------


-- 
----------------------------------------------------------------------------
Reinhard Kotucha                                      Phone: +49-511-3373112
Marschnerstr. 25
D-30167 Hannover                              mailto:reinhard.kotucha at web.de
----------------------------------------------------------------------------
Microsoft isn't the answer. Microsoft is the question, and the answer is NO.
----------------------------------------------------------------------------