[luatex] callbacks in luatex 0.89

[Will Robertson]

On 12 Feb 2016, at 10:08 PM, Hans Hagen <pragma at wxs.nl> wrote:
> 
> injecting stuff (at the user level) in the input stream in an uncontrolled way is messy anyway but if one really wants to do that, one can easily kick in a callback when reading the file and then add commands or whatever at unmarked script/language boundaries (after all it's mostly trying to make structure from non structure we're talking of)

Working directly before the input stream is processed seems unlikely to be able address things like

    \def\x{x}
    \def\y{<chinese>}
    \x\y

And using virtual fonts to add Chinese to a roman font is not enough because you also have to switch the language setup. Oh I see Ulrike already said that :)

Since XeTeX’s interchartoken feature will never be provided directly in LuaTeX (I assume), it would be nice at least if it could be emulated by 3rd party authors so that XeTeX and LuaTeX code can be shared. So I’d support this proposal:

> we might add a token.print function (if so it will be around version 0.95) in which case one can use token.get_tok in an endless loop to pick up tokens and print them back but (as with the callback) it will be very inefficient

… even if it is inefficient :)

Cheers,
Will