[tlbuild] Stripping the luatex binary

[Philipp Stephani]

Am 04.04.2011 um 10:26 schrieb Peter Breitenlohner:

> On Fri, 1 Apr 2011, Philipp Stephani wrote:
> 
>>> (3) thus "libtool --mode=link" is used to build the binaries, for luatex
>>> with '--export-dynamic' but for pdftex without, and "libtool --mode=install"
>>> to install them.
>> 
>> The OS X libtool has neither --mode nor --export-dynamic flags, nor does
>> the linker.  Do you use a custom libtool and/or linker?
> 
> Hi Philipp,
> 
> no, I was talking about GNU libtool (and didn't know that there is an OS X
> binary with the same name.
> 
> GNU libtool is a shell script created by configure (from ltmain.sh and test
> results) used to build shared and/or static "libtool libraries" and
> executables using such libraries and allows to run uninstalled such programs
> using equally uninstalled shared libraries, e.g., in order to test the
> libraries plus programs before installing them.
> 
> This libtool script does have both the --mode option and --export-dynamic
> flag, the later is supposed to
>     Allow symbols from OUTPUT-FILE to be resolved with `dlsym'

OK, I downloaded the TeX Live source and generated the libtool script by running configure, so I hope it is the same thing we're talking about. This script indeed accepts --export-dynamic, but that option is apparently a no-op: it appends $export_dynamic_flag_spec to the linker command, which is empty on my system.

> 
> The libtool script is supposed to pass suitable flags to the system linker
> to achieve that and it is nowhere said that 'strip' would remove such exported
> symbols.  From my experience there seems to be a distinction between
> exported symbols and debug symbols (but that may not so for OS X, causing
> all the problems).

>From what I've tried so far, the concept of exported symbols seems indeed to be completely absent from OS X.

> 
>>> (4) Autoconf likes to compile with '-g' unless the user specifies CFLAGS (or
>>> CXXFLAGS).
>>> 
>>> (5) the programs are stripped upon installation when this is done via "make
>>> install-strip" and it seems this is always done with '-s'.
>> 
>> Do you mean -S (capital S)? If so, that's fine and also supported by OS X strip and ld.
> 
> certainly not, I mean '-s' (small s).

Oh, I confused "install" with "strip".

> 
>>> To see how all this works for i686-linux-gnu (and probably linux in general)
>>> I have inspected the unstripped versions in the build tree as well as the
>>> stripped installed ones with 'nm' and grepped for 'write_png':
>>> 
>>> 	unstripped luatex or pdftex:	one 'T write_png'
>>> 	stripped luatex or pdftex:	nothing
>>> 
>>> Next I have grepped the output of 'strings':
>>> 
>>> 	unstripped or stripped luatex:	two times 'write_png'
>>> 	unstripped or stripped pdftex:	nothing
>> 
>> On some x86-64 bit Linux system, nm says "no symbols" for both lua and
>> luatex.  Maybe the loader ABI is again different for 64-bit Linux systems.
> 
> I think the test above demonstrates this.  nm sees lots of symbols in the
> unstripped versions of both binaries and none in the stripped versions. However, the string 'write_png' (together with many others, I just wanted a
> symbol present in both programs) is present in the luatex binary and absent
> in pdftex (stripped or not).

Yes, but why not on the 64-bit Linux? Anyway, that is not relevant here.

> 
>>> If all that doesn't work for OS X, this is IMHO a libtool + OS X problem
>> 
>> The underlying program (ld) works as expected, and if I compile with
>> "build.sh --nostrip" from the LuaTeX repository, I get a working binary
>> even though the makefile uses libtool to call ld.  But LuaTeX uses a
>> custom libtool, maybe that makes a difference.
> 
> More precisely: a "GNU libtool" + "OS X ld" + "OS X strip" problem.

Or even an "install-sh" problem: install has no way of knowing that it shouldn't call strip for one specific binary.

> 
>>> (either the OS X ld doesn't have the required capabilities
> 
>> No special capabilities are required since all symbols are exported by
>> default.  It works (at least with the binary from the LuaTeX repository)
>> if you call strip with -S -x.  Libtool itself doesn't seem to call strip
>> at all (at least when compiled with LuaTeX's build script).  To summarize:
>> On OS X, call "ld -S -x -o luatex", or call "strip -S -x luatex"
>> afterward, but not "strip luatex".
> 
> With the present TL build system that would mean that no OS X binaries can
> be stripped. That might be an option provided they are compiled without
> debugging symbols.

Or you could use a kludge: Replace "strip" by a shell script that strips everything but luatex.
On the other hand, you probably don't gain that much from removing the symbol table. So compiling with -g0 and/or linking with -S -x might indeed suffice.

> 
> NB: why would luatex want to to load, e.g., lsqlite3.so?  Is this because
> texlua (-> luatex) is used as general purpose lua interpreter?

No but because lua is incomplete without extension modules. The ability to load extension modules is documented in the LuaTeX manual, and it would be sad if that didn't hold just because of problems with the compilation process. ConTeXt already requires several extension modules built into the binary itself, but loading of further modules should be supported (the user might still want to disable it for security reasons).