[luatex] Identifying why LuaTeX 2023 is slower than LuaTeX 2022

[Hans Hagen]

On 8/30/2023 1:16 AM, Reinhard Kotucha wrote:
> On 2023-08-29 at 15:38:35 +0200, Hans Hagen wrote:
> 
>   > So guess what: as i mentioned before there has been a discussion about
>   > portable fmt files (irr a latex request) and the normal luatex build
>   > script has --no-dump-share but araiks tex live doesn't do that (any
>   > longer) so then we get 0.020 sec per run more (2+ sec on 100 runes).
>   >
>   > (plain has a very small format so one can wonder if it's the same there
>   > as in other macropackages)
> 
> Hi Hans,
> as far as plain TeX is concerned, I don't think that the endianmess is
> the main problem.  As of 2016 TeX Live loads the file UnicodeData.txt
> (1.9 MB) into the format file.

i didn't say that it was for plain, although one should realize that a 
format file is compressed so a the real one is a bit bigger .. reports 
about 'preformance here' are mixed: plain (basically only measuring 
start up time) and as well as reports on latex performance drops (which 
involves loading more i guess)

> If you remove the line
> 
>    \input load-unicode-data.tex
> 
> from luatex.ini you probably save much more time than with
> --disable-dump-share.

well, i don't have or load that file at all .. we have a plain setup 
right from the beginning of the project that e use; fyi: format file 
size 361K compressed, 1.44 M uncompressed. And that's what i measured with

the fact that you mention some large file being loaded (maybe that's an 
inefficient one indeed) already indicates that we're talking macro 
package specifics here which for me makes firther tests irrelevant

>   > Of course there can be other factors in a non context universe like the
>   > time needed to load a file database (wasn't there a upper / lowercase
>   > checking change recently?), fonts etc.
> 
> There are currently more than 200,000 files in texmf-dist.  I've
> written a script based on strace(1) which creates a tiny subset of TeX
> Live in another directory containing only the files needed to process
> that particular document.  It's invoked like this:

I dont' have that many files although plenty fonts.

>     tinytex luatex myfile.tex
> 
> When I compile a plain TeX document this way the tiny TeX distribution
> created by my script usually contains less than 30 files.

Plain doesn't need much.

> Kpathsea needs a bit more time to locate files within a system
> containing 200,000 files than within a system containing only 30 files.
> The difference is measurable but negligible.  Kpathsea is extremely
> fast.

Good that this has been made better because in the past a full tex live 
installation was good for adding seconds to startup time. SSD's and OS 
file caching probably have been of help ere too.

btw, the reports hereare for linux, windows is a bit different because 
there using a native vs a cross compiled binary make a difference but 
let's not dive into that

>   > Then when we talk macro packages,
> 
> We shouldn't.  If we are interested in the speed of an *engine* we
> have to avoid macro packages and LaTeX by all means.  LaTeX introduces
> too many rooms of freedom and is a moving target.

well, now you assume all plains being the same but there is no standard 
plain here and i haven't seen timings that are pure engine based apart 
from startup time

> For benchmarking it's best to stick with plain TeX.  It's pretty easy
> to create a 500+ pages lorem ipsum with a halfways reasonable text
> editor.
Well, that's what I did: 1000 ipsums. But usign a plain that is not that 
different from a few years back. And 0.04 sec start up difference is no 
big deal given possible changes in libraries, maybe kpse, the compiler, 
etc. In the early days of luatex we even noticed that sometimes adding 
somethign to the engine made for a change in performance (cache hits 
which was then solved by upgrading hardward) so ... also a bit of a gamble.

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------