[tlbuild] Bug in kpse-icu-flags.m4: a long-standing OpenBSD build failure

[Mojca Miklavec]

Hi,

Just a short follow-up and a short question. Edd confirmed that the
compilation works out of the box when using the system ICU.

Another citation regarding pthread:

On 22 April 2017 at 19:35, Edd Barrett wrote:
>
> -pthread is preferred, since it let's the system choose which
> implementation of pthreads to use, whereas -lpthread forces linking of a
> library called libpthread.so. They should be equivalent on OpenBSD
> though.

I only mention that because "icu-config" says that "-lm -lpthread" is
needed rather than perhaps "-pthread" (and -pthread didn't work for me
for some mysterious reason, but I didn't investigate in details, it
could also be that I didn't apply something properly or that it failed
somewhere else). Not too relevant at this point.


My main question (for Karl or any other autotools expert) is the following:

I would like to add (read as: hardcode) "-lpthread" to ICU_LIBS. How
exactly should I modify the main configure script to do that? I'm
slightly lost in the super complex icu-related code in configure.

If someone can point me to the right track or the right patch, I'll
check whether it works out of the box for OpenBSD.

Thank you very much,
    Mojca


On 22 April 2017 at 04:39, Mojca Miklavec wrote:
> Hi,
>
> Compiling TeX Live fails on OpenBSD.
>
> This is a super old issue:
>
> http://tug.org/pipermail/tex-live/2007-January/011870.html
> http://tug.org/pipermail/tlbuild/2012q2/002290.html
> http://tug.org/pipermail/tlbuild/2016q2/003508.html
> http://tug.org/pipermail/tlbuild/2016q2/003533.html
>
> One way to work around this is to add a bunch of explicit -lpthread
> calls to Makefiles or to manually repeat the failed commands with the
> addition of -lpthread. This list is not entirely complete (it's
> sufficient to build TL, but then the same flags are missing when
> building tests):
>
> --- libs/icu/Makefile.am        (revision 43959)
> +++ libs/icu/Makefile.am        (working copy)
> @@ -84,7 +84,7 @@
>
>  icutest_CPPFLAGS = -Iinclude
>
> -LDADD = icu-build/lib/libicuuc.a icu-build/lib/libicudata.a
> +LDADD = icu-build/lib/libicuuc.a icu-build/lib/libicudata.a -lpthread
>
>  # Rebuild
>  rebuild_prereq =
> --- texk/bibtex-x/Makefile.am   (revision 43959)
> +++ texk/bibtex-x/Makefile.am   (working copy)
> @@ -42,7 +42,7 @@
>  nodist_EXTRA_bibtexu_SOURCES = dummy.cxx
>  bibtexu_CPPFLAGS = $(AM_CPPFLAGS) $(ICU_INCLUDES) -DUTF_8
>  $(bibtexu_OBJECTS): $(KPATHSEA_DEPEND) $(ICU_DEPEND)
> -bibtexu_LDADD = $(KPATHSEA_LIBS) $(ICU_LIBS)
> +bibtexu_LDADD = $(KPATHSEA_LIBS) $(ICU_LIBS) -lpthread
>
>  ## Rebuild libkpathsea
>  @KPATHSEA_RULE@
> --- texk/upmendex/Makefile.am   (revision 43959)
> +++ texk/upmendex/Makefile.am   (working copy)
> @@ -35,7 +35,7 @@
>
>  $(upmendex_OBJECTS): $(KPATHSEA_DEPEND) $(ICU_DEPEND)
>
> -LDADD = $(KPATHSEA_LIBS) $(ICU_LIBS)
> +LDADD = $(KPATHSEA_LIBS) $(ICU_LIBS) -lpthread
>
>  ## Rebuild libkpathsea
>  @KPATHSEA_RULE@
>
>
>
> But then I noticed some "pattern" in this. Basically anything where
> $(ICU_LIBS) is present lacks an "-lpthread" flag.
>
>
> I went on exploring and realized that these flags are (supposed to be)
> determined from:
>
>     ICU_LIBS=`$ICU_CONFIG --ldflags-searchpath --ldflags-icuio
> --ldflags-libsonly --ldflags-system`
>
> but then I also found traces of
>
>     checking for icu-config... no
>
> in configure logs (this returs "icu-config" on one of the linux boxes
> where I probably have icu installed, but that's probably not the
> desired one as it would give the wrong flags anyway). That was for
> dvisvgm, but nonetheless.
>
> I suspected that icu-config was "broken". It took me a while to figure
> out how to call it and I eventually figured out:
>
>     ./Work/libs/icu/icu-build/config/icu-config \
>         --prefix=/path/to/Work/libs/icu/icu-build \
>         --ldflags-system
>
> The thing is: this *does* return
>
>     -lpthread -lm
>
> so my only conclusion was that TL failed to call (the right)
> icu-config during configuration of bibtex-u and upmendex and others
> and thus failed to add the "-lpthread" flag when it is needed.
>
> I don't have the patch yet, but the problem seems to be somewhere in
>     m4/kpse-icu-flags.m4
>
> most likely in this part:
>
> # LIBNAME and OPTIONS as for _KPSE_LIB_FLAGS().
> # MORE-ICU-LIBS: icu libraries from the TL tree in addition to icuuc
> and icudata.
> m4_define([_KPSE_ICU_FLAGS], [dnl
> _KPSE_LIB_FLAGS([icu], [$1], [$2],
>                 [-DU_STATIC_IMPLEMENTATION -IBLD/libs/icu/include],
>                 m4_bpatsubst([$3 icuuc icudata],
>                              [icu\([18a-z]*\)],
>                              [BLD/libs/icu/icu-build/lib/libicu\1.a]),
>                 [],
>                 [],
> [${top_builddir}/../../libs/icu/include/unicode/uversion.h])[]dnl
> ]) # _KPSE_ICU_FLAGS
>
> Here's the simplified output of
>
>     KPSE_ICU_FLAGS([icuio icui18n], [--ldflags-icuio])
>
> being called in upmendex's configure:
>
>     if $ICU_CONFIG --version >/dev/null 2>&1; then
>       ICU_INCLUDES=`$ICU_CONFIG --cppflags`
>       ICU_LIBS=`$ICU_CONFIG --ldflags-searchpath --ldflags-icuio
> --ldflags-libsonly --ldflags-system`
>     elif test "x$need_icu:$with_system_icu" = xyes:yes; then
>       as_fn_error $? "did not find icu-config required for system icu
> libraries" "$LINENO" 5
>     fi
>     else
>       ICU_INCLUDES="-DU_STATIC_IMPLEMENTATION -I$kpse_BLD/libs/icu/include"
>       ICU_LIBS="$kpse_BLD/libs/icu/icu-build/lib/*.a"
>       ...
>     fi
>
> This means that when a built-in version of ICU is being used, the part with
>
>     icu-config --ldflags-system
>
> is entirely ignored. TL just fills in all the libraries, but not the
> requirements that ICU is aware of and would tell you about them if
> being asked.
>
>
> Nelson wrote:
>
>> The problem is that the contents of -lpthread are in -lc
>> on many flavors of Unix
>
> so this problem does not show up on any other platform we are building for.
>
> Note that my package manager on OS X also says:
>
>     > icu-config --ldflags-system
>     -lpthread -lm
>
> but building mac binaries still works.
>
>
> What's a bit confusing to me is why the OpenBSD needs this patch for
> the packaged TeX Live
> (http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/print/texlive/base/patches/patch-texk_bibtex-x_Makefile_in?rev=1.3&content-type=text/x-cvsweb-markup).
> According to the above analysis it would seem to me that the correct
> flags should be applied when compiling against the system-wide icu.
>
> Karl or any other automagic expert: would you be willing to come up
> with a patch for m4/ that would add the call to (a local) icu-config,
> ask for flags and add them to ICU_LIBS?
>
> Thank you very much,
>     Mojca