[luatex] Identifying why LuaTeX 2023 is slower than LuaTeX 2022

Wed Aug 30 09:44:59 CEST 2023

Hi Hans,

> So, what are we now looking at? A simple run test.tex with probably one 
> paragraph and a few fonts loaded (as it's plain).

Correct. I'm just using the example file from Paulo's initial email:

   Lorem ipsum dolor sit amet, consectetur adipiscing elit,
   sed do eiusmod tempor incididunt ut labore et dolore magna
   aliqua. Ut enim ad minim veniam, quis nostrud exercitation
   ullamco laboris nisi ut aliquip ex ea commodo consequat.
   Duis aute irure dolor in reprehenderit in voluptate velit
   esse cillum dolore eu fugiat nulla pariatur. Excepteur
   sint occaecat cupidatat non proident, sunt in culpa qui
   officia deserunt mollit anim id est laborum.
   \bye

> Different versions of 
> luatex, so there can be small changes, unlikely in the frontend, maybe 
> some in the backend but we're talking little output here. 

But it isn't really different versions of LuaTeX here. From the tests in
my earlier reply, "2023-initial" is LuaTeX v1.16.0, "2023" is LuaTeX
v1.16.0 with the debug/popen bug patched, and "sys" is LuaTeX v1.16.0
with the debug/popen bug patched and socket disabled (aka v1.17.0).

None of these patches should have any performance implications, yet
"sys" is ~40% slower than either "2023" or "2023-initial", which makes
me think that the LuaTeX currently in TL23 ("sys") was erroneously
compiled with "-O0" or something.

> Is the same 
> fontloader used for every test (which assume stability over 2017 - 2023)?

Yes.

> Is the luatex build.sh used or some tex live one?

I used the LuaTeX "build.sh" for all the "20xx" binaries, albeit with a
few tiny patches. Very detailed build instructions here:

   https://tug.org/~mseven/luatex.html#binary-details

"2023-initial" would have been built with the standard TL build script.
"sys" was probably built with the TL script, but it's a special ad-hoc
release so something different might have happened.

> Is the same lua used? 

Between the 3 different 2023 versions, yes.

> These 
> .40 ms are noise to me anc can originate in several places (going into 
> details about possible bottlenecks in tex is bit off topic here) so the 
> question is how this goes for a 1000 ipsum paragraphs.
> 
> The simplest startup test i can come upwith (using context) is \stoptext 
> which gives me with the garden binaries:
> 
> [...]
> 
> So guess what: as i mentioned before there has been a discussion about 
> portable fmt files (irr a latex request) and the normal luatex build 
> script has --no-dump-share but araiks tex live doesn't do that (any 
> longer) so then we get 0.020 sec per run more (2+ sec on 100 runes).

The test below rebuilds the LaTeX format files, so it is much more
intensive and it doesn't load any format files. Start by running the
script from my previous reply, then:

   $ hyperfine --warmup 1 \
       -L year 2022,2023,2023-initial,sys \
       'PATH=/tmp/texlive-testing/{year}/bin/x86_64-linux:/bin/ fmtutil-sys --byfmt lualatex'

   Benchmark 1: PATH=/tmp/texlive-testing/2022/bin/x86_64-linux:/bin/ fmtutil-sys --byfmt lualatex
     Time (mean ± σ):      4.766 s ±  0.023 s    [User: 4.529 s, System: 0.238 s]
     Range (min … max):    4.718 s …  4.795 s    10 runs

   Benchmark 2: PATH=/tmp/texlive-testing/2023/bin/x86_64-linux:/bin/ fmtutil-sys --byfmt lualatex
     Time (mean ± σ):      4.908 s ±  0.035 s    [User: 4.675 s, System: 0.233 s]
     Range (min … max):    4.864 s …  4.998 s    10 runs

   Benchmark 3: PATH=/tmp/texlive-testing/2023-initial/bin/x86_64-linux:/bin/ fmtutil-sys --byfmt lualatex
     Time (mean ± σ):      4.784 s ±  0.037 s    [User: 4.545 s, System: 0.240 s]
     Range (min … max):    4.738 s …  4.873 s    10 runs

   Benchmark 4: PATH=/tmp/texlive-testing/sys/bin/x86_64-linux:/bin/ fmtutil-sys --byfmt lualatex
     Time (mean ± σ):      7.344 s ±  0.039 s    [User: 7.104 s, System: 0.240 s]
     Range (min … max):    7.291 s …  7.426 s    10 runs

   Summary
     PATH=/tmp/texlive-testing/2022/bin/x86_64-linux:/bin/ fmtutil-sys --byfmt lualatex ran
       1.00 ± 0.01 times faster than PATH=/tmp/texlive-testing/2023-initial/bin/x86_64-linux:/bin/ fmtutil-sys --byfmt lualatex
       1.03 ± 0.01 times faster than PATH=/tmp/texlive-testing/2023/bin/x86_64-linux:/bin/ fmtutil-sys --byfmt lualatex
       1.54 ± 0.01 times faster than PATH=/tmp/texlive-testing/sys/bin/x86_64-linux:/bin/ fmtutil-sys --byfmt lualatex

Or this test which computes 12! 10000 times in ini mode (see
attachment):

   $ hyperfine --warmup 1 \
       -L year 2022,2023,2023-initial,sys \
       'PATH=/tmp/texlive-testing/{year}/bin/x86_64-linux:/bin/ luatex -ini factorial.tex'

   Benchmark 1: PATH=/tmp/texlive-testing/2022/bin/x86_64-linux:/bin/ luatex -ini factorial.tex
     Time (mean ± σ):      3.936 s ±  0.011 s    [User: 3.913 s, System: 0.024 s]
     Range (min … max):    3.918 s …  3.952 s    10 runs

   Benchmark 2: PATH=/tmp/texlive-testing/2023/bin/x86_64-linux:/bin/ luatex -ini factorial.tex
     Time (mean ± σ):      3.926 s ±  0.023 s    [User: 3.899 s, System: 0.026 s]
     Range (min … max):    3.877 s …  3.953 s    10 runs

   Benchmark 3: PATH=/tmp/texlive-testing/2023-initial/bin/x86_64-linux:/bin/ luatex -ini factorial.tex
     Time (mean ± σ):      3.958 s ±  0.039 s    [User: 3.933 s, System: 0.024 s]
     Range (min … max):    3.908 s …  4.031 s    10 runs

   Benchmark 4: PATH=/tmp/texlive-testing/sys/bin/x86_64-linux:/bin/ luatex -ini factorial.tex
     Time (mean ± σ):      5.519 s ±  0.054 s    [User: 5.489 s, System: 0.029 s]
     Range (min … max):    5.464 s …  5.589 s    10 runs

   Summary
     PATH=/tmp/texlive-testing/2023/bin/x86_64-linux:/bin/ luatex -ini factorial.tex ran
       1.00 ± 0.01 times faster than PATH=/tmp/texlive-testing/2022/bin/x86_64-linux:/bin/ luatex -ini factorial.tex
       1.01 ± 0.01 times faster than PATH=/tmp/texlive-testing/2023-initial/bin/x86_64-linux:/bin/ luatex -ini factorial.tex
       1.41 ± 0.02 times faster than PATH=/tmp/texlive-testing/sys/bin/x86_64-linux:/bin/ luatex -ini factorial.tex

All 4 tests use the exact same TL23 texmf trees, so the TeX code, Lua
code, and all the other binaries are identical; only the "luahbtex" and
"luatex" binaries are different. The only difference between "2023-
initial" and "sys" binaries should be the socket and debug/popen
patches; they (probably) used the same compilers and the source should
be otherwise identical. 

Yet the current TL23 LuaTeX binary ("sys") is 50% slower than the
initially-released TL23 LuaTeX binary ("2023-initial"), so something
weird is definitely going on. My guess would be that the current LuaTeX
binaries were compiled with -O0 while all the other binaries use -O3, or
something similar. Just a guess though.

Thanks,
-- Max

-------------- next part --------------
A non-text attachment was scrubbed...
Name: factorial.tex
Type: text/x-tex
Size: 488 bytes
Desc: not available
URL: <https://tug.org/pipermail/luatex/attachments/20230830/4c3021aa/attachment.bin>