[luatex] token_filter

Paul Isambert zappathustra at free.fr
Sat Apr 27 01:15:03 CEST 2013

Khaled wrote:
> > I’d just go with that from start, no need to replicate XeTeX’s
> > implementation. I believe Arthur was doing some work in this area, so
> > you may want to check with him.

I second that; going for token_filter will only result in hacks, I’m afraid.
It’s just so easy with nodes. I think Taco’s page on the wiki was just for fun.

Arthur wrote:
>   Yes, I already communicated with Élie about that, but just in case
> anyone cares, my experiments from last summer are at
> https://github.com/reutenauer/inter-char-node

l.28 of your luaintercharnode.lua, you say:

    Why the hell can’t I index a table with a table?

Well, you can :)

    local t = { [{1, 1}] = emspace }
    t[{a = b}] = c

Elie’s original questions:
>   - where is the ouptut of token.command_name documented?

Err, in the manual?

>   - how can I know if the token I get is inside a csname? (ex: in a file 
> I have \foobar. When, in the token_filter, I get the token b, how can I 
> know it's part of a csname name ?)

“b” in “\foobar” isn’t a token; the token is the entire command.

>   - same question for other characters, for example '<' in \ifnum\foo<0

If I understand your question correctly as “How can I know whether ‘<’
is used as an operator in an \ifnum-conditional”, then the answer is:
keep track of what has come before :) (Or, actually, perhaps some
tracing is possible in Lua: “tex.tracingifs”... “tex.currentconditional”...?)

Somebody (Patrick, if I remember correctly) once asked on this list
what token_filter was good for; honestly, I don’t know. It’s nice to
have it around when you really don’t know how to spend/waste a few
hours at your computer, but apart from that, I find it neither useful
nor usable.


More information about the luatex mailing list