[lltx] luaotfload-database.lua

[Philipp Gesang]

Hi Reinhard!

···<Datum: Sunday, 26. May 2013>···<Von: Reinhard Kotucha>···

>  > > The current version runs luaotfload (and reads the cache) when the
>  > > format file is created.
>  > 
>  > I don‘t understand. There shouldn’t be any cache files at that
>  > time. Also, which format are you using?
> 
> I create my own format file.

Okay, that was the missing bit.

>                               luaotfload is loaded just after LaTeX.
> All Open Type fonts I need are also defined in the format file.  Thus,
> the font cache is used only when the format file is created.
> 
> The problem occurs at run time when the PDF file is created.  Since
> luaotfload defines the font with an absolute path, the program isn't
> portable anymore.  However, if I remove the path component then
> kpathsea will find the font.  At least if it's in a TEXMF tree.  But
> this is always the case here because everything else isn't portable.

There is also aux.scan_external_dir() for including fonts from directories
that weren’t indexed.

>  > > This solves many problems.  Everything is extremely fast and the
>  > > cache isn't used at all (it even doesn't exist in the runtime
>  > > environment).
>  > 
>  > I don’t think that’s the case. It should be darned slow without
>  > the cache.
> 
> It's even faster because everything which is time consuming is done
> when the format file is created.  At run time the font is loaded again
> in order to put a subset into the PDF file.  This takes some time but
> can't be avoided anyway.
> 
> Please note that *I* create the format file under Linux and then run
> the TeX file from a program which traces system calls (open, stat) in
> order to create a tiny TeX system in another directory.  This
> directory contains everything users need and it's what users get.
> 
>  > > However, at run time fonts are loaded with absolute paths and I
>  > > had to patch otfl-font-def.lua in order to strip the path
>  > > component from the font filename.  Then the font is found by
>  > > kpathsea at run time.  The patch (see attachment) works fine for
>  > > me, but in general it would
>  > 
>  > 1) Use the patch_font callback for this kind of hack.
>  > 2) fontdata.filename = fontdata.filename and string.gsub(fontdata.filename, "^.*/", "")
>  > 3) With a file: and path: lookup this would be mapped back to an
>  >    absolute path for the reason listed below, so it’s kind of
>  >    pointless.
> 
> I'll have to investigate which solution is best.  With the old version
> of luaotfload I found out that I couldn't change anything after the
> fonts had been specified:
> 
>   \font\foo=bar.ttf at 12pt

Better: \font\foo="file:bar.ttf" at 12pt

> And before that there were no paths to strip.  So I think that using a
> callback is the only solution to avoid patching luaotfload.

You could also replace the file resolver.

    fonts.definers.resolvers.file = function (spec)
      --- rewrite spec.name here
    end

>  > > make more sense to check in luaotfload whether a font is in a system
>  > > font directory or in a TEXMF tree and strip the path component only if
>  > > the latter is true.
>  > 
>  > Can you give the current Luaotfload a try (in the pretest)? We
>  > don’t actually require kpse to load fonts anymore, only to
>  > collect their whereabouts for the database. File names are hashed
>  > and the corresponding absolute paths are retrieved from the
>  > index. At runtime there is no distinction between texmf and
>  > system fonts whatsoever anymore.
> 
> I tried the version from CTAN now and when I create the format file I
> get the message (in the log file and on stdout)
> 
>   \font\foo=/usr/share/fonts/bar.ttf at 12pt
> 
> whereas with the _patched_ old version of luaotfload I got
> 
>   \font\foo=bar.ttf at 12pt

Yes, that is expected. As I said, we keep the paths in the
database so there isn’t a need for the extra calls to kpse.

>  > > It doesn't make a difference when luaotfload is loaded at run time
>  > > (because the cache is re-generated if necessary) but it makes a
>  > > difference if it's in a format file.
>  > > 
>  > > In short:  Everything in TEXMF trees should be found by kpathsea.
>  > 
>  > Why? The files are already indexed.
> 
> I suppose that my previous mail didn't describe clearly what I'm
> actually doing.  But I still think that it's better to strip the path
> component from fonts in TEXMF directories.  It probably doesn't matter
> when fonts are loaded by luaotfload at run time.
> 
> There is another, quite nasty, problem:
> 
> If you have the same font installed in a TEXMF tree and in a system
> font directory, luaotfload is using the system font.  This is very bad.   

Well, isn’t that the reason why I installed that font in the
first place?

> There might be different versions of a particular font with the same
> file name.  CentOS switched from teTeX to TeXLive-2007 recently.

Which would mean an outdated *texmf*.

>                                                                   You
> can imagine how old the system fonts are on this system.

Usually, when someone asks for help with their distro’s tex
packages, the first thing they’re told is to drop it in favor of
texlive (or the minimals). This is not limited to tex, btw., but
quite common with other languages that come with dedicated
package management.

>                                                           If somebody
> installs TL from tug.org, I suppose that he wants to use the versions
> provided by TeX Live.  Or even newer versions in own TEXMF trees.

Or those in ~/.fonts, which is *the* canonical directory to
install your preferred fonts in.

> I'm not familiar with the luaotfload code, but I suppose that this
> behavior is not intended.  I suppose that it has something to do with
> hashes.  If a font name is used as a key in a hash, everything depends
> on the way the hash is created.
> 
> If font names are used as keys in a hash and system fonts are scanned
> _after_ fonts in TEXMF trees, it's clear that system fonts always win
> because they overwrite existing keys.

That’s pretty much it, yes. Order matters.

> Anyway, IMO it's a severe bug if luaotfload ignores everything set up
> in texmf.cnf and uses (probably ancient) system fonts instead.

Or an ancient texlive on an up-to-date system. Stranger things
have happened.

Since you are loading fonts by their filename and don’t care for
system fonts, you probably have no need for a names database. We
could make it optional, I guess, by installing a kpse-only file:
resolver and omitting the others. This could be toggled by
setting a flag in config.luaotfload.

The texmf vs. system fonts matter could be made configurable as
well, allowing the user to prioritize as they see fit. This would
require either a dotfile or separate tables in the database for
each of the search paths. Either way, it’s a more complex
addition so it’ll probably require testing.

For a start, you can test with this branch:

    https://github.com/phi-gamma/luaotfload/commits/texmf-fonts

it restores the previous behavior of not indexing full paths for
texmf fonts and also prefers them over system fonts.

Best
Philipp


-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://tug.org/pipermail/lualatex-dev/attachments/20130526/1e9ad3a6/attachment.bin>