[tlbuild] report on building locally on an old macos and a query

[jfbu]

Hello Max,

> Le 15 mars 2025 à 09:01, Max Chernoff <tex at maxchernoff.ca> a écrit :
> 
> Hi Jean-François,
> 
> On Fri, 2025-03-14 at 17:20 +0100, jfbu wrote:
>> +-------------+------------+-------+-------+
>>> binaries    | datestamp  | tex   | etex  |
>> +-------------+------------+-------+-------+
>>> upstream    | 2022/03/09 | 1.57s | 1.77s |
>>> local (O3)  | 2022/05/28 | 1.37s | 1.44s |
>>> local (O3)  | 2023/03/30 | 1.44s | 1.48s |
>>> local (O3)  | 2024/03/30 | 1.45s | 1.50s |
>>> local (O3 + | 2025/03/11 | 1.42s | 1.43s |
>>>  sys libs) |            |       |       |
>>> local (O3)  | 2025/03/13 | 1.43s | 1.43s |
>>> local (O3 + | 2025/03/14 | 1.44s | 1.51s |
>>> sys libs + |            |       |       |
>>>   clang17) |            |       |       |
>>> local (O3 + | 2025/03/14 | 1.42s | 1.44s |
>>> sys libs + |            |       |       |
>>>    gcc14)  |            |       |       |
>>> upstream    | 2025/03/01 | 1.61s | 1.74s |
>> +-------------+------------+-------+-------+
> 
> Hmm, the fact that the upstream binaries are so much slower than your
> locally-compiled binaries somewhat reminds me of the following thread:
> 
>    https://tug.org/pipermail/luatex/2023-August/007864.html
> 
>    https://tug.org/pipermail/luatex/2023-August/007877.html
> 
> The problem there was that the binaries were compiled with "-O0" and/or
> didn't have the debugging info stripped. If you compile the binaries
> locally with "-O0" and/or "-g", do you get the similar timings as
> "upstream 2025"?

Much to my dismay it seems at my locale that using "-O0" or "-g" produces
more efficient (!) tex/etex than the ones I had gotten with "-O3".

But first, my benchmark is ridiculously not a serious one, however most
of my time having been devoted this morning
to produce as fast as possible tex and pdftex
binaries and formats and organize things for testing,
I have not at all tried
to improve the testing itself (any advice appreciated on what to process
through tex/etex for more robust benchmarking).

Also, it seems generally speaking most everything is about 1% faster,
this morning, perhaps my computer had a good night of sleep, or
maybe it has to do with the fact that now things are organized with
separate directories containing binaries and format files, but the other
day, if I remember correctly I was doing the tests with the binaries and
formats kept in /usr/local/texlive/2025.

Also different is that TEXMFROOT is now set in the environment
(formerly I only had TEXMFDIST and two or three others set from my
.bash_profile).  Also different is that the binaries built today were done
without having set STRIP in the environment.

Anyway I re-did the (silly) benchmark for some of the binaries already
tested the other day, getting roughly comparable results (albeit a bit
faster as indicated above) and for some new ones built with compiler
options "-O0" or "-g". (did not try both at same time).

There is quite some variability here in the hundredths of second, but,
roughly:

- built the other day and timed anew today:

gcc14 with -03 from  1.38s (tex) 1.41s (etex)

 (thus for some reason the same binaries and formats act a bit faster but 0.03s is of the order of intrinsic variability across 5 runs anyhow)

clang17 with -03  1.44s (tex) 1.50s (etex)

- newly built today (most everything perform the same!):

gcc14 with -O0 1.37s (tex) 1.40s (etex)
gcc14 with -g  1.36s (tex) 1.40s (etex)
clang17 with -O0 1.37s (tex) 1.40s (etex)
clang17 with -g  1.37s (tex) 1.40s (etex)

I did not try with gcc11. For the binaries built today I did not
disable TeXLive native as I was building only tex and pdftex.
The gcc14 and clang17 built binaries from the other day
were using --disable-native-texlive-build and had set
STRIP to "strip -u -r"

(I was forced to do --disable-native-texlive-build the other day
for the main ./Build
as at my locale there was a problem building xetex/bibtex/...
which use TeXLive ICU, I had to use the ICU from my MacPorts
configuration).

> If so, you could probably poke the darwinlegacy builder
> to rebuild with optimizations enabled.

In view of my adventures this morning I think I should not
too much burden Mojca with trying this out...

It could be the main difference is probably that I am building
on a macos 10.13.6 machine whereas "upstream" is on 10.6 and using c99.

Maybe I am simply wrong to try to fiddle around using Optimization
options...

In passing, as people may wonder if by any chance I got at some point
completely confused between compilers and formats and perhaps
started mis-attributing some benchmarks, I have the following observation:

1. it would be nice for the .log file to not only give the day of creation
of the format but also its creation time (if that data is kept anywhere).

2. also, it would be nice if that info made it not only to the log file
but also to stdout/stderr: my benchmarking script captures the stdout+stderr
which only will tell the name of the format, not its creation day (and
even less so the creation time).

Points 1/ and 2/ are for messy benchmarkers like myself who may at some
point wonder which binary/format was actually executed.  But I did
take some measures to get reassurance I was not using accidentally
something else than intended.

side remarks:
- the --fmtdir option for fmtutil can not be set to the
directory where pdftex resides because fmtutil will try to create there
a pdftex/ sub-directory...

- it is a bit disconcerting that fmtutil 
complains even if with --fmtdir option and one has to use fmtutil-sys/user
which is a bit counter-intuitive (especially that fmtutil-sys works,
but I understand it would be a pain to check if --fmtdir is among options
and allow fmtutil without -sys or -user only in that special niche case).


> 
> I've never used macOS though, so this is all just a wild guess.

It would feel exactly the same here as with any Unixen machine.
All the work is done in a command-line interface and I use Emacs
for all text aspects.  The shell on my system is bash, but I understand
newer macos'es use another shell. Currently this system is still supported
by MacPorts so I can install libraries if needed for example GNU utilities
such as gtime (not used in the above benchmarks, but which I used
this week for something else).

Best,
JF

> 
> Thanks,
> -- Max