[luatex] Adding a callback before trailing spaces are removed from a line of input

[Eduardo Ochs]

Hi Vitek,

I am the author of this package,

  http://angg.twu.net/dednat6/tug-slides.pdf
  http://angg.twu.net/dednat6/tugboat-rev2.pdf
  http://angg.twu.net/dednat6.html

that uses Lua to read material from the _comments_ of the current .tex
file. I think that what you need can be implemented easily on top of
this single-file version of the core of dednat6,

  http://angg.twu.net/dednat6/dednat6/minimalcore.lua.html
  http://angg.twu.net/dednat6/dednat6/minimalcore.lua
  http://angg.twu.net/dednat6/demo-core.tex.html
  http://angg.twu.net/dednat6/demo-core.tex
  http://angg.twu.net/dednat6/demo-core.pdf

that only implements the code that interprets each block of lines
starting with "%L" as a chunk of Lua code...

If this looks interesting to you, please get in touch! I took a
quick look at your markdown.dtx and I got the impression that 1) it
would be trivial to make my minimalcore.lua read the markdown code in
fencedCode blocks and make TeX skip over it, and that 2) it would be
easy to port the ideas from the prototype made with minimalcore.lua to
your markdown package.

  Cheers,
    Eduardo Ochs
    http://angg.twu.net/contact.html



On Sat, 28 Aug 2021 at 19:57, Vítek Novotný <witiko at mail.muni.cz> wrote:

> Hello all,
>
> in Knuth's TeX, trailing spaces are removed very early on when a line is
> being put to the input buffer. [1]  According to Eijkhout's TeX by
> Topic, this is because "these spaces are hard to see in an editor" [2].
>
>  [1]: https://texdoc.org/serve/tex.pdf/0#page=15
>  [2]: http://mirrors.ctan.org/info/texbytopic/TeXbyTopic.pdf
>
> I develop and maintain the Markdown package [3] for plain TeX, ConTeXt,
> and LaTeX. The package makes it possible to use the lightweight markup
> of markdown [4] in TeX documents. In markdown, a hard line break can be
> inserted by ending a line with two or more spaces. However, since
> trailing spaces are removed by TeX, hard breaks are only recognized when
> we' are inserting an external markdown file, not when markdown is typed
> in the top-level document. This deficiency is known and documented [5],
> but I am hoping we could resolve it with LuaTeX.
>
>  [3]: https://github.com/witiko/markdown
>  [4]: https://daringfireball.net/projects/markdown
>  [5]:
> https://mirrors.ctan.org/macros/generic/markdown/markdown.pdf#page=20
>
> In LuaTeX, the `process_input_buffer` callback [6] can be used to
> intercept the text coming *out* of the input buffer. However, the
> trailing spaces have already been removed by this point.
>
> By adding a callback right after a line has entered the input buffer
> [1], we could either replace the trailing space characters with tabs,
> or place a character such as the zero-width non-joiner (U+200C) to the
> right of the trailing spaces.
>
>  [6]: https://www.pragma-ade.com/general/manuals/luatex.pdf#page=176
>
> Is this something you would consider---if not for LuaTeX then perhaps
> for LuaMetaTeX?
>
> Best regards,
> Vítek Novotný
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://tug.org/pipermail/luatex/attachments/20210829/063a74bb/attachment.html>