<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, 14 Jan 2023 at 12:53, Vítek Novotný <<a href="mailto:witiko@mail.muni.cz">witiko@mail.muni.cz</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Sat, Jan 14, 2023 at 10:50:21AM +0100, luigi scarso wrote:<br>
> On Sat, 14 Jan 2023 at 10:27, luigi scarso <<a href="mailto:luigi.scarso@gmail.com" target="_blank">luigi.scarso@gmail.com</a>> wrote:<br>
> <br>
> ><br>
> ><br>
> > On Fri, 13 Jan 2023 at 22:36, Vítek Novotný <<a href="mailto:witiko@mail.muni.cz" target="_blank">witiko@mail.muni.cz</a>> wrote:<br>
> ><br>
> >> Dear LuaTeX developers,<br>
> >><br>
> >> assume the following plain TeX document `example.tex`:<br>
> >><br>
> >> \newwrite\outfile<br>
> >> \openout\outfile\jobname.out<br>
> >> \write\outfile{^^^^fffd}<br>
> >> \closeout\outfile<br>
> >> \bye<br>
> >><br>
> >> Running `luatex example` will correctly produce file `example.out` with<br>
> >> the<br>
> >> UTF-8 encoding of U+FFFD: 0xEF, 0xBF, and 0xBD.<br>
> >><br>
> >> $ hexdump -C<br>
> >> 00000000 ef bf bd 0a |....|<br>
> >> 00000004<br>
> >><br>
> >> Now, let's change `example.tex` as follows:<br>
> >><br>
> >> \input\jobname.out<br>
> >> \bye<br>
> >><br>
> >> Running `luatex example` produces the following error:<br>
> >><br>
> >> ! String contains an invalid utf-8 sequence.<br>
> >><br>
> >> I would expect that LuaTeX would treat ^^^^fffd and the byte sequence<br>
> >> 0xEF,<br>
> >> 0xBF, and 0xBD the same. This issue was co-discovered by @lostenderman at<br>
> >> <<a href="https://github.com/lostenderman/markdown/issues/34" rel="noreferrer" target="_blank">https://github.com/lostenderman/markdown/issues/34</a>>.<br>
> >><br>
> >><br>
> > hm, checking it now.<br>
> ><br>
> ><br>
> hm I am not able to reproduce the error... My log says,<br>
> Missing character: There is no � (U+FFFD) in font cmr10!<br>
> but luatex exits fine<br>
> <br>
> $ luatex --credits<br>
> This is LuaTeX, Version 1.15.1 (TeX Live 2023/dev)<br>
><br>
> The LuaTeX team is Hans Hagen, Hartmut Henkel, Taco Hoekwater, Luigi Scarso.<br>
> <br>
> LuaTeX merges and builds upon (parts of) the code from these projects:<br>
> <br>
> tex : Donald Knuth<br>
> etex : Peter Breitenlohner, Phil Taylor and friends<br>
> omega : John Plaice and Yannis Haralambous<br>
> aleph : Giuseppe Bilotta<br>
> pdftex : Han The Thanh and friends<br>
> kpathsea : Karl Berry, Olaf Weber and others<br>
> lua : Roberto Ierusalimschy, Waldemar Celes and Luiz Henrique de<br>
> Figueiredo<br>
> metapost : John Hobby, Taco Hoekwater, Luigi Scarso, Hans Hagen and friends<br>
> pplib : Paweł Jackowski<br>
> fontforge : George Williams (partial)<br>
> luajit : Mike Pall (used in LuajitTeX)<br>
> <br>
> Compiled with libpng 1.6.39; using 1.6.39<br>
> Compiled with lua version 5.3.6<br>
> Compiled with mplib version 2.02<br>
> Compiled with zlib 1.2.13; using 1.2.13<br>
> <br>
> Development id: 7554<br>
<br>
Here is mine:<br>
<br>
# luatex --credits<br>
This is LuaTeX, Version 1.15.0 (TeX Live 2022)<br>
<br>
The LuaTeX team is Hans Hagen, Hartmut Henkel, Taco Hoekwater, Luigi Scarso.<br>
<br>
LuaTeX merges and builds upon (parts of) the code from these projects:<br>
<br>
tex : Donald Knuth<br>
etex : Peter Breitenlohner, Phil Taylor and friends<br>
omega : John Plaice and Yannis Haralambous<br>
aleph : Giuseppe Bilotta<br>
pdftex : Han The Thanh and friends<br>
kpathsea : Karl Berry, Olaf Weber and others<br>
lua : Roberto Ierusalimschy, Waldemar Celes and Luiz Henrique de Figueiredo<br>
metapost : John Hobby, Taco Hoekwater, Luigi Scarso, Hans Hagen and friends<br>
pplib : Paweł Jackowski<br>
fontforge : George Williams (partial)<br>
luajit : Mike Pall (used in LuajitTeX)<br>
<br>
Compiled with libpng 1.6.37; using 1.6.37<br>
Compiled with lua version 5.3.6<br>
Compiled with mplib version 2.02<br>
Compiled with zlib 1.2.11; using 1.2.11<br>
<br>
Development id: 7509<br>
<br>
Here is the full sequence of commands to reproduce the issue:<br>
<br>
$ docker run --rm -it texlive/texlive<br>
<br>
# cat > example.tex << EOF<br>
> \newwrite\outfile<br>
> \openout\outfile\jobname.out<br>
> \write\outfile{^^^^fffd}<br>
> \closeout\outfile<br>
> \bye<br>
> EOF<br>
<br>
# luatex example<br>
This is LuaTeX, Version 1.15.0 (TeX Live 2022)<br>
restricted system commands enabled.<br>
(./example.tex [1{/usr/local/texlive/2022/texmf-var/fonts/map/pdftex/updmap/pdf<br>
tex.map}])</usr/local/texlive/2022/texmf-dist/fonts/type1/public/amsfonts/cm/cm<br>
r10.pfb><br>
Output written on example.pdf (1 page, 8143 bytes).<br>
Transcript written on example.log.<br>
<br>
# cat > example.tex << EOF<br>
> \input\jobname.out<br>
> \bye<br>
> EOF<br>
<br>
# luatex example<br>
This is LuaTeX, Version 1.15.0 (TeX Live 2022)<br>
restricted system commands enabled.<br>
(./example.tex (./example.out<br>
! String contains an invalid utf-8 sequence.<br>
l.1<br>
?<br>
? q<br>
OK, entering \batchmode<br>
<br>
I don't have the opportunity to test the latest LuaTeX but if I<br>
understand you correctly, it seems that this issue has been fixed<br>
sumetime between Development id 7509 and 7554 and will no longer be<br>
present in TeX Live 2023.<br>
<br>
Best,<br>
Vit<br></blockquote><div><br></div><div>yes, maybe this patch we have done fixes the issue:</div><div>2022-08-16 Luigi Scarso <<a href="mailto:luigi.scarso@gmail.com">luigi.scarso@gmail.com</a>> <br> * Accent 0xFFFD but still error on invalid utf (compatible) (H.Hagen)<br> * omitinfodict added: \pdfvariable omitinfodict 1 omit Info dicttionary (H.Hagen)<br></div><div> </div><div>--</div><div>luigi</div></div></div>