[luatex] BOM

[Ulrike Fischer]

Am Fri, 15 May 2009 10:12:30 +0200 schrieb Javier Bezos:

>    ... except if we first set how the file should be read -- since
>    the BOM must be the very first thing in the file, this means we
>    need to do some kind of preprocessing, which is not always
>    desirable or convenient.
> 
> Indeed, after the preprocessing it's irrelevant, but then
> the catcode is irrelevant, too, because the BOM string of
> bytes may be discarded at that stage. ¿Can this preprocessing be
> done from inside luatex?

As far as I remember, xetex does such a preprocessing. It reads the
first byte(s) to identify the encoding (it can handle utf8 and both
utf16) and while doing it also removes the BOM. 

But are you all sure that luatex chokes on the BOM? I know it did -
I even added on another PC some code to a format to get around it,
but currently I can't reproduce it. pdflatex chokes, lualatex not. 

This is LuaTeX, Version snapshot-0.39.0-2009041523


-- 
Ulrike Fischer