[luatex] Identifying why LuaTeX 2023 is slower than LuaTeX 2022

[Paulo Roberto Massa Cereda]

Thanks for the feedback, Hans! Apologies for the "vague" context, I
was in a hurry. I will try to provide more details as follows:

My initial test was:

--------8<--------
\documentclass{article}

\usepackage{fontspec}

\setmainfont[
    Extension=.ttf,
    Path=./fonts/,
    UprightFont=*-Regular,
    BoldFont=*-Bold,
    ItalicFont=*-Italic,
    BoldItalicFont=*-BoldItalic
]{JetBrainsMono}

\begin{document}

Hello world.

\end{document}
--------8<--------

JetBrains Mono was obtained from here: https://www.jetbrains.com/lp/mono/

By "cold cache" (and apologies in advance for probably using the wrong
term for describing), I was referring to deleting

texmf-var/luatex-cache

before each benchmark run. I did this with both TL 2022 and TL 2023.

Then I tried with an actual document. I had Lorem Ipsum paragraphs
replicated more than 1000 times:

➜ wc -l test.tex
2499 test.tex

➜ hyperfine -L year 2022,2023 --warmup 2
'texlive/{year}/bin/x86_64-linux/lualatex test.tex'
Benchmark 1: texlive/2022/bin/x86_64-linux/lualatex test.tex
  Time (mean ± σ):      2.881 s ±  0.021 s    [User: 2.771 s, System: 0.108 s]
  Range (min … max):    2.848 s …  2.907 s    10 runs

Benchmark 2: texlive/2023/bin/x86_64-linux/lualatex test.tex
  Time (mean ± σ):      5.814 s ±  0.059 s    [User: 5.675 s, System: 0.128 s]
  Range (min … max):    5.760 s …  5.910 s    10 runs

Summary
  texlive/2022/bin/x86_64-linux/lualatex test.tex ran
    2.02 ± 0.03 times faster than
texlive/2023/bin/x86_64-linux/lualatex test.tex

 For HarfBuzz, I had the OTF font example above with:

\setmainfont[
    Extension=.ttf,
    Path=./fonts/,
    UprightFont=*-Regular,
    BoldFont=*-Bold,
    ItalicFont=*-Italic,
    BoldItalicFont=*-BoldItalic,
    Renderer=HarfBuzz
]{JetBrainsMono}

For the normalize thing one, I had:

\setmainfont[
    Extension=.ttf,
    Path=./fonts/,
    UprightFont=*-Regular,
    BoldFont=*-Bold,
    ItalicFont=*-Italic,
    BoldItalicFont=*-BoldItalic,
    RawFeature=-normalize
]{JetBrainsMono}

Marcel suggested me to update TL2022 luaotfload-tool to the latest
version as well, so I did:

➜ texlive/2022/bin/x86_64-linux/luaotfload-tool --version | grep
"luaotfload-tool version"
luaotfload-tool version: "3.24"

➜ texlive/2023/bin/x86_64-linux/luaotfload-tool --version | grep
"luaotfload-tool version"
luaotfload-tool version: "3.24"

➜ hyperfine -L year 2022,2023 --warmup 2
'texlive/{year}/bin/x86_64-linux/lualatex test.tex'
Benchmark 1: texlive/2022/bin/x86_64-linux/lualatex test.tex
  Time (mean ± σ):     689.6 ms ±   2.0 ms    [User: 588.9 ms, System: 100.1 ms]
  Range (min … max):   687.3 ms … 693.6 ms    10 runs

Benchmark 2: texlive/2023/bin/x86_64-linux/lualatex test.tex
  Time (mean ± σ):      1.264 s ±  0.007 s    [User: 1.151 s, System: 0.112 s]
  Range (min … max):    1.254 s …  1.277 s    10 runs

Summary
  texlive/2022/bin/x86_64-linux/lualatex test.tex ran
    1.83 ± 0.01 times faster than
texlive/2023/bin/x86_64-linux/lualatex test.tex

I also tried with \usepackage[T1]{fontenc}:

➜ hyperfine -L year 2022,2023 --warmup 2
'texlive/{year}/bin/x86_64-linux/lualatex test.tex'
Benchmark 1: texlive/2022/bin/x86_64-linux/lualatex test.tex
  Time (mean ± σ):     457.4 ms ±   3.1 ms    [User: 380.7 ms, System: 76.3 ms]
  Range (min … max):   452.8 ms … 463.6 ms    10 runs

Benchmark 2: texlive/2023/bin/x86_64-linux/lualatex test.tex
  Time (mean ± σ):     893.9 ms ±   8.4 ms    [User: 805.5 ms, System: 88.0 ms]
  Range (min … max):   886.8 ms … 915.7 ms    10 runs

Summary
  texlive/2022/bin/x86_64-linux/lualatex test.tex ran
    1.95 ± 0.02 times faster than
texlive/2023/bin/x86_64-linux/lualatex test.tex

I can create a GitHub repository with some tests for reproducibilty,
if it helps.

I will try to compile the sources. Will report soon.

Thanks!

Cheerio,

Paulo

Em seg., 28 de ago. de 2023 às 12:30, Hans Hagen <j.hagen at xs4all.nl> escreveu:
>
> On 8/28/2023 4:49 PM, Paulo Roberto Massa Cereda wrote:
> > Dear friends,
> >
> > I beseech your wisdom. :) In my tests, I noticed LuaTeX 2023 is
> > significantly slower than the 2022 counterpart. Here's a MWE:
> >
> > --------8<--------
> > Lorem ipsum dolor sit amet, consectetur adipiscing elit,
> > sed do eiusmod tempor incididunt ut labore et dolore magna
> > aliqua. Ut enim ad minim veniam, quis nostrud exercitation
> > ullamco laboris nisi ut aliquip ex ea commodo consequat.
> > Duis aute irure dolor in reprehenderit in voluptate velit
> > esse cillum dolore eu fugiat nulla pariatur. Excepteur
> > sint occaecat cupidatat non proident, sunt in culpa qui
> > officia deserunt mollit anim id est laborum.
> > \bye
> > --------8<--------
> >
> > Benchmark 1: texlive/2022/bin/x86_64-linux/luatex test.tex
> >    Time (mean ± σ):     177.7 ms ±   1.4 ms    [User: 140.9 ms, System: 36.6 ms]
> >    Range (min … max):   175.8 ms … 180.3 ms    16 runs
> >
> > Benchmark 2: texlive/2023/bin/x86_64-linux/luatex test.tex
> >    Time (mean ± σ):     286.7 ms ±   2.3 ms    [User: 246.1 ms, System: 40.3 ms]
> >    Range (min … max):   284.0 ms … 291.2 ms    10 runs
> >
> > Summary
> >    texlive/2022/bin/x86_64-linux/luatex test.tex ran
> >      1.61 ± 0.02 times faster than texlive/2023/bin/x86_64-linux/luatex test.tex
> >
> > I initially had tried with LuaLaTeX and got similar results. Ulrike
> > Fischer and Marcel Krüger kindly helped me find out what was going on,
> > but we could not find anything relevant. I've tried:
> >
> > - loading a custom OTF font: 2022 was 1.53 ± 0.02 (cold cache) and
> > 1.83 ± 0.03 (existing cache) times faster
> > - loading the default OTF font: 2022 was 1.68 ± 0.02 (cold cache) and
> > 2.02 ± 0.03 (existing cache) times faster
> > - HarfBuzz: 2022 was 1.57 ± 0.02 times faster (cold cache) and 1.84 ±
> > 0.05 times faster (existing cache)
> > - raw feature (normalize): 2022 was 1.81 ± 0.03 times faster
> > - updated luaotfload-tool in TL2022: 2022 was 1.83 ± 0.01 times faster
> > - no OTF: 2022 was 1.95 ± 0.02 times faster
> >
> > I was wondering if someone could shed some light into this. :) Thanks!
> I can't test it here (no 2022 installed and i;m also nto going to fdo
> that) but you need to define
>
> - cold cache
> - custom OTF
> - default OTF
> - raw feature
>
> as i have no clue what that means here. Also, a simple single paragraph
> is no real test. How does it look for 50 pages. What for a more complex
> document of say 300 pages. In a simple test startup time (whatever needs
> to be initialized) kicks in. So, what if no fonts are loaded ta all. If
> you use plain, what fonts gets loaded by default.
>
> I can't really compare (laptop windows 10) but using my plain version it
> needs .5 sec for a single li, 1.2 seconds for 100 pages of it, 4.48
> seconds for 500 pages.
>
> As a comparison, context needs 5.2 sec for 500 pages in node mode and
> 3.65 sec in base mode (luametatex: 5.15 and 3.45 with 1.6 sec spend in
> backend). Anywway over 80 pps for a simpel document like this which will
> of course drop down to 20-40 pps for a more complex document.
>
> Some 0.1 seconds loss between versions is not dramatic unless if
> multiplies with largers runs.
>
> (btw, i'd expect windows to be a bit more sensitive between years
> depending on native vs crosscompiled and tighter mem protection between
> versions.)
>
> So, a question is: what is the fontless baseline? What if you compile a
> binary for your system yourself? Is this only plain or also other macro
> packages?
>
> Another thing I can think of is that format loading time changed maybe
> because of a change in endian related storage. I remember some
> discussion about that (portable formats) and on intel byte swapping then
> kicks in. But again that's only startup time related.
>
> Hans
>
>
> -----------------------------------------------------------------
>                                            Hans Hagen | PRAGMA ADE
>                Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
>         tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
> -----------------------------------------------------------------
>