[lltx] luaotfload-database.lua

[Reinhard Kotucha]

On 2013-05-25 at 13:53:59 +0200, Philipp Gesang wrote:

 > ááá<Datum: Saturday, 25. May 2013>ááá<Von: Reinhard Kotucha>ááá
 > 
 > > On 2013-05-24 at 12:21:56 +0200, Philipp Gesang wrote:
 > > 
 > >  > Cygwin should now be handled the same way as linux. Could you
 > >  > check if the changes address your point and report back? The code
 > >  > should be on CTAN in soon.
 > > 
 > > Yes, it looks good now.
 > 
 > Great.
 > 
 > > BTW, there is another thing I'd like to mention:
 > > 
 > > I've written a program using luaotfload.  This program is called by
 > > another program and runs in the background, i.e. users don't see what
 > > LuaTeX prints to screen.  I develop under Linux but the program is
 > > usually run under Windows.
 > > 
 > > The first version invoked luaotfload at run time.  It turned out that
 > > the font cache is problematic for many reasons in this application.
 > > Everything has to work out-of-the-box, I can't expect that users
 > > (physicians) are able to fix problems with file permissions or so
 > > themselves and the fact that some system fonts have to be blacklisted
 > > manually is even more problematic.
 > 
 > I agree, itÕs unfortunate. But since faulty fonts freeze the
 > interpeter there isnÕt much we can do about it at the Lua end.
 > (Context has the same problem, but afaics it doesnÕt occur while
 > building the database because only .otf files are analyzed with
 > fontloader.open().)
 > 
 > Btw. you sound like this happens more often than it we are aware
 > of. Do you by any chance have some candidates for the blacklist
 > that youÕd like to share?

Hi Philipp,
no, there are no problems on *my* system, but I don't have access to
the systems the program actually runs on.

 > > The current version runs luaotfload (and reads the cache) when the
 > > format file is created.
 > 
 > I donÔt understand. There shouldnÕt be any cache files at that
 > time. Also, which format are you using?

I create my own format file.  luaotfload is loaded just after LaTeX.
All Open Type fonts I need are also defined in the format file.  Thus,
the font cache is used only when the format file is created.

The problem occurs at run time when the PDF file is created.  Since
luaotfload defines the font with an absolute path, the program isn't
portable anymore.  However, if I remove the path component then
kpathsea will find the font.  At least if it's in a TEXMF tree.  But
this is always the case here because everything else isn't portable.

 > > This solves many problems.  Everything is extremely fast and the
 > > cache isn't used at all (it even doesn't exist in the runtime
 > > environment).
 > 
 > I donÕt think thatÕs the case. It should be darned slow without
 > the cache.

It's even faster because everything which is time consuming is done
when the format file is created.  At run time the font is loaded again
in order to put a subset into the PDF file.  This takes some time but
can't be avoided anyway.

Please note that *I* create the format file under Linux and then run
the TeX file from a program which traces system calls (open, stat) in
order to create a tiny TeX system in another directory.  This
directory contains everything users need and it's what users get.

 > > However, at run time fonts are loaded with absolute paths and I
 > > had to patch otfl-font-def.lua in order to strip the path
 > > component from the font filename.  Then the font is found by
 > > kpathsea at run time.  The patch (see attachment) works fine for
 > > me, but in general it would
 > 
 > 1) Use the patch_font callback for this kind of hack.
 > 2) fontdata.filename = fontdata.filename and string.gsub(fontdata.filename, "^.*/", "")
 > 3) With a file: and path: lookup this would be mapped back to an
 >    absolute path for the reason listed below, so itÕs kind of
 >    pointless.

I'll have to investigate which solution is best.  With the old version
of luaotfload I found out that I couldn't change anything after the
fonts had been specified:

  \font\foo=bar.ttf at 12pt

And before that there were no paths to strip.  So I think that using a
callback is the only solution to avoid patching luaotfload.

 > > make more sense to check in luaotfload whether a font is in a system
 > > font directory or in a TEXMF tree and strip the path component only if
 > > the latter is true.
 > 
 > Can you give the current Luaotfload a try (in the pretest)? We
 > donÕt actually require kpse to load fonts anymore, only to
 > collect their whereabouts for the database. File names are hashed
 > and the corresponding absolute paths are retrieved from the
 > index. At runtime there is no distinction between texmf and
 > system fonts whatsoever anymore.

I tried the version from CTAN now and when I create the format file I
get the message (in the log file and on stdout)

  \font\foo=/usr/share/fonts/bar.ttf at 12pt

whereas with the _patched_ old version of luaotfload I got

  \font\foo=bar.ttf at 12pt

 > > It doesn't make a difference when luaotfload is loaded at run time
 > > (because the cache is re-generated if necessary) but it makes a
 > > difference if it's in a format file.
 > > 
 > > In short:  Everything in TEXMF trees should be found by kpathsea.
 > 
 > Why? The files are already indexed.

I suppose that my previous mail didn't describe clearly what I'm
actually doing.  But I still think that it's better to strip the path
component from fonts in TEXMF directories.  It probably doesn't matter
when fonts are loaded by luaotfload at run time.

There is another, quite nasty, problem:

If you have the same font installed in a TEXMF tree and in a system
font directory, luaotfload is using the system font.  This is very bad.   

There might be different versions of a particular font with the same
file name.  CentOS switched from teTeX to TeXLive-2007 recently.  You
can imagine how old the system fonts are on this system.  If somebody
installs TL from tug.org, I suppose that he wants to use the versions
provided by TeX Live.  Or even newer versions in own TEXMF trees.

I'm not familiar with the luaotfload code, but I suppose that this
behavior is not intended.  I suppose that it has something to do with
hashes.  If a font name is used as a key in a hash, everything depends
on the way the hash is created.

If font names are used as keys in a hash and system fonts are scanned
_after_ fonts in TEXMF trees, it's clear that system fonts always win
because they overwrite existing keys.

Anyway, IMO it's a severe bug if luaotfload ignores everything set up
in texmf.cnf and uses (probably ancient) system fonts instead.

Regards,
  Reinhard

-- 
----------------------------------------------------------------------------
Reinhard Kotucha                                      Phone: +49-511-3373112
Marschnerstr. 25
D-30167 Hannover                              mailto:reinhard.kotucha at web.de
----------------------------------------------------------------------------
Microsoft isn't the answer. Microsoft is the question, and the answer is NO.
----------------------------------------------------------------------------