[luatex] LuaTeX garbles special tokens in terminal output

[Joseph Wright]

On 24/12/2018 08:34, Henri Menke wrote:
> Dear devs,
> 
> LuaTeX somehow garbles special tokens when printing to the log using
> \show.  For example when you define a macro with active end of lines,
> I'd expect ^^M to be printed in the log.  However, LuaTeX replaces ^^M
> with \r, mashing all the lines together into one.
> 
> Consider the follwing MWE:
> 
> \begingroup
> \obeylines
> \gdef\lines{
>    Hello LuaTeX!
> 
>    Why do you garble newlines in
>    terminal output?
> }
> \endgroup
> 
> \show\lines
> 
> \bye
> 
> ------------------------------------------------
> 
> Here is the log with pdfTeX.
> 
> ------------------------------------------------
> 
> $ pdftex test.tex
> This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018)
> (preloaded format=pdftex)
>   restricted \write18 enabled.
> entering extended mode
> (./test.tex
>> \lines=macro:
> ->^^MHello LuaTeX!^^M^^MWhy do you garble newlines in^^Mterminal output?^^M.
> l.11 \show\lines
> 
> ?
>   )
> No pages of output.
> Transcript written on test.log.
> 
> ------------------------------------------------
> 
> And here is the log with LuaTeX.
> 
> ------------------------------------------------
> 
> $ luatex test.tex
> This is LuaTeX, Version 1.09.0 (TeX Live 2019/dev)
>   restricted system commands enabled.
> (./test.tex
>> \lines=macro:
> .erminal output?e newlines in
> l.11 \show\lines
> 
> ?
> )
> warning  (pdf backend): no pages of output.
> Transcript written on test.log.
> 
> ------------------------------------------------
> 
> Cheers, Henri
> 

This is by-design: see page 12 of the current manual:

Output to the terminal uses ^^ notation for the lower control range (𝑐 
< 32), with the exception
of ^^I, ^^J and ^^M. These are considered ‘safe’ and therefore printed 
as-is.

For l3build (which uses log comparison), we do a sweep over the log to 
turn returns into ^^M to get 'closer' to pdfTeX for exactly this reason.

Joseph