[tlbuild] sparc-linux (bigendian?) failure in icu

[Jonathan Kew]

On 13/4/12 01:24, Karl Berry wrote:
> The build fails for me on sparc-linux, like this:
> ..
> LD_LIBRARY_PATH=../lib:../stubdata:../tools/ctestfw:$LD_LIBRARY_PATH  ../bin/icupkg -tb /u/guest/karl/tl/Build/source/Work/libs/icu/../../../libs/icu/icu-49.1/data/in/icudt49l.dat ./out/icudt49b.dat
> udata_swapDataHeader(): header size mismatch - headerSize 32768 infoSize 5120 length 1024
> make[6]: *** [out/icudt49b.dat] Error 8
> make[6]: Leaving directory `/home/1007/gu/karl/tl/Build/source/Work/libs/icu/icu-build/data'
>
> Some painful debugging showed that the function
> icu-49.1/tools/toolutil/package.cpp:readFile() is calling
> getTypeEnumForInputData() to read the file
> icu-49.1/data/in/icudt49l.dat.  And that .dat file apparently embeds the
> endianness of where the file was created (namely LittleEndian, on
> Peter's machine).

Doesn't the "l" suffix on the filename there indicate littleendianness? 
Is that file part of the original ICU source distribution? I'd have 
expected it to be created - appropriately for the target platform - as 
part of the ICU build process.

http://userguide.icu-project.org/icudata#TOC-Sharing-ICU-Data-Between-Platforms

(Though I don't recall using the .dat file approach to ICU data 
packaging; I think I always used the library configuration.)

>
> And then it seems that that endianness is used in further processing on
> the current machine (BigEndian on sparc-linux), and that causes things
> to be (not) byte-swapped where they need to be.  Or something like that.
> I didn't get to the bitter end of it today.
>
> Specifically, `length' ends up being 1024, which is less than headerSize
> (32768), which causes the bailing-out above (in
> icu-49.1/common/udataswp.c line 268).
>
> So far, this appears to be basically unrelated to the previous endian
> problem with ICU, although there are so many changes it's hard to tell.
>
> I'm not sure if rearranging the build process to generate icudt49l.dat
> at build time would work, or if it's the best thing to do, in any case.

For a bigendian target, I think it should be building icudt49b.dat. And 
yes, it'd have to know what endianness it's building for; you can't 
build the data file once and then use it everywhere, as per the doc above.

>
> Jonathan, if you have any ideas about any of this, they would be most
> welcome.
>
> Dick, I'm surprised you didn't run into this on powerpc-darwin.
>
> Vladimir, watch out on powerpc-aix.
>
> karl
>
>
> P.S. For the record, turns out to be necessary to be use -mcpu=sparcv9
> on the new sparc-linux host at Utah (gentoo 3.2.1), else get the
> nefarious:
> ..
> libTECkit_Compiler.a(Compiler.o): In function `__atomic_add':
> /usr/lib/gcc/sparc-unknown-linux-gnu/4.5.3/include/g++-v4/ext/atomicity.h:50: undefined reference to `__sync_fetch_and_add_4'
> make[5]: *** [teckit_compile] Error 1
> make[5]: Leaving directory `/home/1007/gu/karl/tl/Build/source/Work/libs/teckit'
>