[luatex] token_filter

[Élie Roux]

Dear all,

I'm currently struggling to get polyglossia working with french 
punctuation[1]. As it is implemented with interchartokens in XeTeX, I 
thought I could achieve something similar with the implementation taken 
here: http://wiki.luatex.org/index.php?title=Token_filter. It works 
quite well for very simple things, but I really cannot understand a few 
things:
  - where is the ouptut of token.command_name documented?
  - how can I know if the token I get is inside a csname? (ex: in a file 
I have \foobar. When, in the token_filter, I get the token b, how can I 
know it's part of a csname name ?)
  - same question for other characters, for example '<' in \ifnum\foo<0

It seems XeTeX handles it well, but it is possible to get this 
fine-grained filtering in the token_filter, without having to expand and 
inspect all tokens "by hand"?

If there is no simple solution, I'll try something with the nodes, I 
think it will be more appropriate...

What do you think?

Thank you,
-- 
Elie

[1] https://github.com/eroux/polyglossia