[tlbuild] Bug in kpse-icu-flags.m4: a long-standing OpenBSD build failure

[Mojca Miklavec]

Hi,

Compiling TeX Live fails on OpenBSD.

This is a super old issue:

http://tug.org/pipermail/tex-live/2007-January/011870.html
http://tug.org/pipermail/tlbuild/2012q2/002290.html
http://tug.org/pipermail/tlbuild/2016q2/003508.html
http://tug.org/pipermail/tlbuild/2016q2/003533.html

One way to work around this is to add a bunch of explicit -lpthread
calls to Makefiles or to manually repeat the failed commands with the
addition of -lpthread. This list is not entirely complete (it's
sufficient to build TL, but then the same flags are missing when
building tests):

--- libs/icu/Makefile.am        (revision 43959)
+++ libs/icu/Makefile.am        (working copy)
@@ -84,7 +84,7 @@

 icutest_CPPFLAGS = -Iinclude

-LDADD = icu-build/lib/libicuuc.a icu-build/lib/libicudata.a
+LDADD = icu-build/lib/libicuuc.a icu-build/lib/libicudata.a -lpthread

 # Rebuild
 rebuild_prereq =
--- texk/bibtex-x/Makefile.am   (revision 43959)
+++ texk/bibtex-x/Makefile.am   (working copy)
@@ -42,7 +42,7 @@
 nodist_EXTRA_bibtexu_SOURCES = dummy.cxx
 bibtexu_CPPFLAGS = $(AM_CPPFLAGS) $(ICU_INCLUDES) -DUTF_8
 $(bibtexu_OBJECTS): $(KPATHSEA_DEPEND) $(ICU_DEPEND)
-bibtexu_LDADD = $(KPATHSEA_LIBS) $(ICU_LIBS)
+bibtexu_LDADD = $(KPATHSEA_LIBS) $(ICU_LIBS) -lpthread

 ## Rebuild libkpathsea
 @KPATHSEA_RULE@
--- texk/upmendex/Makefile.am   (revision 43959)
+++ texk/upmendex/Makefile.am   (working copy)
@@ -35,7 +35,7 @@

 $(upmendex_OBJECTS): $(KPATHSEA_DEPEND) $(ICU_DEPEND)

-LDADD = $(KPATHSEA_LIBS) $(ICU_LIBS)
+LDADD = $(KPATHSEA_LIBS) $(ICU_LIBS) -lpthread

 ## Rebuild libkpathsea
 @KPATHSEA_RULE@



But then I noticed some "pattern" in this. Basically anything where
$(ICU_LIBS) is present lacks an "-lpthread" flag.


I went on exploring and realized that these flags are (supposed to be)
determined from:

    ICU_LIBS=`$ICU_CONFIG --ldflags-searchpath --ldflags-icuio
--ldflags-libsonly --ldflags-system`

but then I also found traces of

    checking for icu-config... no

in configure logs (this returs "icu-config" on one of the linux boxes
where I probably have icu installed, but that's probably not the
desired one as it would give the wrong flags anyway). That was for
dvisvgm, but nonetheless.

I suspected that icu-config was "broken". It took me a while to figure
out how to call it and I eventually figured out:

    ./Work/libs/icu/icu-build/config/icu-config \
        --prefix=/path/to/Work/libs/icu/icu-build \
        --ldflags-system

The thing is: this *does* return

    -lpthread -lm

so my only conclusion was that TL failed to call (the right)
icu-config during configuration of bibtex-u and upmendex and others
and thus failed to add the "-lpthread" flag when it is needed.

I don't have the patch yet, but the problem seems to be somewhere in
    m4/kpse-icu-flags.m4

most likely in this part:

# LIBNAME and OPTIONS as for _KPSE_LIB_FLAGS().
# MORE-ICU-LIBS: icu libraries from the TL tree in addition to icuuc
and icudata.
m4_define([_KPSE_ICU_FLAGS], [dnl
_KPSE_LIB_FLAGS([icu], [$1], [$2],
                [-DU_STATIC_IMPLEMENTATION -IBLD/libs/icu/include],
                m4_bpatsubst([$3 icuuc icudata],
                             [icu\([18a-z]*\)],
                             [BLD/libs/icu/icu-build/lib/libicu\1.a]),
                [],
                [],
[${top_builddir}/../../libs/icu/include/unicode/uversion.h])[]dnl
]) # _KPSE_ICU_FLAGS

Here's the simplified output of

    KPSE_ICU_FLAGS([icuio icui18n], [--ldflags-icuio])

being called in upmendex's configure:

    if $ICU_CONFIG --version >/dev/null 2>&1; then
      ICU_INCLUDES=`$ICU_CONFIG --cppflags`
      ICU_LIBS=`$ICU_CONFIG --ldflags-searchpath --ldflags-icuio
--ldflags-libsonly --ldflags-system`
    elif test "x$need_icu:$with_system_icu" = xyes:yes; then
      as_fn_error $? "did not find icu-config required for system icu
libraries" "$LINENO" 5
    fi
    else
      ICU_INCLUDES="-DU_STATIC_IMPLEMENTATION -I$kpse_BLD/libs/icu/include"
      ICU_LIBS="$kpse_BLD/libs/icu/icu-build/lib/*.a"
      ...
    fi

This means that when a built-in version of ICU is being used, the part with

    icu-config --ldflags-system

is entirely ignored. TL just fills in all the libraries, but not the
requirements that ICU is aware of and would tell you about them if
being asked.


Nelson wrote:

> The problem is that the contents of -lpthread are in -lc
> on many flavors of Unix

so this problem does not show up on any other platform we are building for.

Note that my package manager on OS X also says:

    > icu-config --ldflags-system
    -lpthread -lm

but building mac binaries still works.


What's a bit confusing to me is why the OpenBSD needs this patch for
the packaged TeX Live
(http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/print/texlive/base/patches/patch-texk_bibtex-x_Makefile_in?rev=1.3&content-type=text/x-cvsweb-markup).
According to the above analysis it would seem to me that the correct
flags should be applied when compiling against the system-wide icu.

Karl or any other automagic expert: would you be willing to come up
with a patch for m4/ that would add the call to (a local) icu-config,
ask for flags and add them to ICU_LIBS?

Thank you very much,
    Mojca