[tlbuild] CMake

[Mojca Miklavec]

On Sun, 23 Dec 2018 at 08:10, Norbert Preining wrote:
>
> Hi Mojca,
>
> as one who has recently written a autoconf/automake based system, which
> is just plain incompatible with how github releases work, and then also
> built a meson system, I agree with your feeling, and myself I would be
> happy to see something different.

I would like to see support for either system as well. In particular
this could have simplified Windows builds (and hopefully not break
cross-compiling builds too much).

> BUT ... Karl is completely right

I'm still waiting for the answer about what the real limitations are.

> - many different packages that needs inclusion
>   do you want to write meson/cmake for all the libs/progs etc that are
>   actually developed outside of TL and mostly come with auto* build
>   systems?

Well, this counts as "lack of manpower" in my dictionary :)
(The answer would be "Yes", of course.)
This is not a REAL limitation, just a lot of work that needs to be
done at some point.

(I don't want to know how many hours Akira has spent on writing and
maintaining visual studio project files to keep W32TeX running. And
yes, I believe he had to write a separate project file for every
single dependency. That was certainly orders of magnitude more work
than one with equivalent level of expertise would need to invest into
CMake or meson definitions.)

Both meson and recent versions of CMake can natively read .pc files
for the sake of finding external dependencies, but of course this
doesn't scale too well for Windows, so if we would want to do it
right, we would need to write all the missing support for all the
packages that we ship with the project (not 100% how to do it right
for external dependencies). On the plus side, if well written, either
CMake or meson support could eventually be included upstream,
benefitting everyone else outside of the TeX community.

We could start with LuaTeX for example. Something that's still complex
enough, actively maintained, but doesn't have **too** many
dependencies.

> - man power
>   I think if someone starts a meson based system that would be great,
>   and it could be included in the TL sources without interference of
>   file names. ANyone having the time, please go ahead. But I doubt.

Yes, we would need to have two systems running in parallel (for at
least a couple of years). There's no reason to remove autotools
support even if someone steps up and adds support for a new build
system.

> - old systems
>   esp meson, but also cmake are simply a PITA to get on older systems

(This does count as a real limitation to me.)

I have no idea how many platforms/architectures are covered by ninja,
but certainly there are lots of exotic ones which are not. And I have
absolutely no idea how difficult it is to install python 3 on a random
old box.

Here's a list of CMake versions that come with some OSes:
    https://cliutils.gitlab.io/modern-cmake/chapters/intro/dodonot.html

CMake now requires C++11 which lead to quite some bootstrapping
headaches in MacPorts on older OSes (clang requires CMake to build,
and CMake requires C++11 support which can only be achieved by getting
a newer compiler first; the workaround is using gcc for bootstrapping,
but ...). However we require C++11 for other packages in TeX already,
so the really old systems are out of question already.

Other than C++11 requirement, is CMake really so much probelmatic to
install on older systems? Except of course the pain to have to do that
in the first place as opposed to simply running ./configure and
getting it work out of the box?

The macOS doesn't ship with CMake for example, so one needs to install
it manually or via a package manager in any case. It doesn't ship with
python3 or ninja either. (I wonder if the existence of package
managers on mac just removed pressure from Apple to include some
reasonable tools in their OS, or if they simply don't care about
developers at all.)

> So as much as I would like to see a more simple build system, I think
> for now auto* is still the best/stable way to go.
>
> But, if someone wants to work on a cmake or meson system that does not
> interfere with the current status, (s)he is heartly welcome I would say.

Definitely if we find sufficient workforce to work on this (I'm
willing to spend some time, but there's absolutely no way I would
manage to understand it all and do it all, and if I only do 10% of the
work, that's equal to nothing).

> I forgot - my preference would be for meson. I *hate* cmake.

When meson came out / when I heard of it, I was totally enthusiastic :) :) :)
I probably spent two days trying to get cross-compilation to work, and
failed. Then I did the same in meson (while writing my first project
ever) in probably half an hour.

I tried to suggest the usage of meson in my company, but my idea was
rejected with a pointer to
    https://www.rojtberg.net/1481/do-not-use-meson/

After listening to quite some "modern CMake" talks I agree that
version 3+ is "light-years" better than the old approach, even if the
syntax is still weird. I have yet to experience how "modern
FindFoo.cmake" works. It was absolutely unbearable from the package
management perspective in the past, and I gave up trying to patch
those Find packages years ago (I didn't try recently). I absolutely
hated that every project had to reinvent the wheel writing custom and
totally buggy FindFoo-s. Other concepts now make much more sense to me
than they did in the past, I assume that support for cross-compilation
is meanwhile better.

Another thing I hate with CMake is that unless the authors hardcoded
the knowledge about the compiler at hand into the "internal CMake
know-it-all about compilers", most of the stuff simply won't work. In
contrast to autotools where each feature is tested for separately,
CMake "knows it all" and hardly ever tests for stuff. In MacPorts this
is a serious issue since we are building our own version of clang on
top of what Apple ships by default, and CMake has absolutely no idea
what features that compiler has. I think that CMake also supports some
kind of tests for features, but I never tested that.

I recently asked one package author to modernize the CMake in his
super small project, but he said this was no option as he had to
support older systems which don't yet have a recent enough version of
CMake installed (at which point you loose all the benefits).

Probably the most painful part of CMake is that if you google for
solutions to your questions / problems nowadays, you'll get super
highly-ranked wrong (i.e. outdated) answers on StackExchange more than
90% of the time. So one needs to read a book or two first before
starting to write something.

>From the point of view of treating bug reports: meson is still in a
bit of a beta phase, there are a number of issues (until a few months
ago it was literally useless on mac), but they do respond very quickly
(last time I checked it was mainly one developer + a few helping out a
bit though). CMake didn't fix any single bug of those I reported (you
cannot even compile it on OS X 10.7 for example, for a long while it
was impossible to compile fortran project on mac until gcc decided not
to throw "unknown flag" errors on "-arch i386" any longer), and when
they switched to a new bug tracker, they simply closed all reports and
asked you to resubmit if you still thought it was relevant.

(I started having warm feelings for CMake last week when I figured out
how CUDA support worked literally out of the box, and three lines of
code replaced probably hours of manual fiddling of project files.)

Meson is a really nice and clean language, I like the syntax a lot
more than CMake's. From the perspective of momentum, I do fear in a
way that it might get "eaten up" by CMake in a similar way as
mercurial was "defeated" by git (not comparing features here, just
frequency of use), meaning that lack of manpower in one project could
no longer cut it at some point, but I cannot predict the future.

If build system of TeX Live was rewritten, I'm relatively ambivalent
about the choice of the tool.

Mojca