[luatex] \Ucharcat

[Hans Hagen]

On 5/22/2015 11:02 PM, David Carlisle wrote:
> Hi,
>
> xetex in the tl2015 release has added a new primitive \Ucharcat
> which is like Uchar but takes a second integer specifying a catcode.
>
> \Ucharcat 65 11  expands to letter A
> \Ucharcat 65 12  expands to a character A, etc.
>
> This is rather more useful than \Uchar as it allows arbitrary character
> tokens to be generated via expansion.
>
> Would there be interest in adding a matching primitive in luatex?
>
> It can be implemented in lua at the cost of one extra expansion step
> (although the same is true of \Uchar of course)
>
> Whether or not it gets implemented as a primitive, I wonder if this is the best
> lua implementation? Comments welcome:-)
>
> \ifx\newcatcodetable\@undefined
>    \chardef\ucharcat at table"7000
> \else
>    \newcatcodetable\ucharcat at table
> \fi
> \luatexinitcatcodetable\ucharcat at table
> \directlua{%
> function UcharcatLua()
>    local mych = newtoken.scan_int()
>    local mycat = newtoken.scan_int()
>    tex.setcatcode(\the\numexpr\ucharcat at table\relax,mych,mycat)
>    tex.sprint(\the\numexpr\ucharcat at table\relax,unicode.utf8.char(mych))
> end
> }
> \def\Ucharcat{\directlua{UcharcatLua()}}
>
> I'd hoped to avoid dedicating a catcode table table for this
> and instead just set the catcode, print, and set back.
> However it seems that the printing happens with the catcode
> table value in force at the end of \directua not the value at
> the point of the tex.sprint. (The manual could be more explicit
> on that point but I think that's right?)

well, the whole idea of luatex and having lua is that we can avoid all 
kind of extensions (i can think of a truckload of handy helpers for 
context but deliberately don't implement them in luatex because it can 
done with help of lua and is also macro package specific)

your lua solution looks ok to me, if it's real critital you can speed it 
up a bit with locals (normally lua is pretty fast)

we have on the (low priority) agenda a cleanup of some of the input 
handler code (your case relates to input handling and lua prints get 
injected, currently a bit more complex than needed because we started 
out with multiple lua instances)

at some point i can imagine an extra lua print function that print a 
string with given catcode (something tex.cprint(catcode,string,..) but 
for that we need to carry an extra around) so it's then part of the 
cleanup (so that would simplify the \Ucharcat definition then) ... i've 
added a reminder in the tracker

you can consider passing the catcodetable so you can define the function 
before the table

\directlua{%
function UcharcatLua()
    local myct  = newtoken.scan_int()
    local mych  = newtoken.scan_int()
    local mycat = newtoken.scan_int()
    tex.setcatcode(myct,mych,mycat)
    tex.sprint(myct,unicode.utf8.char(mych))
end
}

\newcatcodetable\Ucharcatcodetable

\initcatcodetable\Ucharcatcodetable

\def\Ucharcat{\directlua{UcharcatLua()}\Ucharcatcodetable}

A:\the\catcode65:\Ucharcat 65 11:A:\the\catcode65\par
A:\the\catcode65:\Ucharcat 65  5:A:\the\catcode65\par
A:\the\catcode65:\Ucharcat 65 11:A:\the\catcode65\par

the following avoids the catcodetable:

\directlua{%
local cct = nil
local chr = nil
function UcharcatLua()
    chr = newtoken.scan_int()
    cct = tex.getcatcode(chr)
    tex.setcatcode(chr,newtoken.scan_int())
    tex.sprint(unicode.utf8.char(chr),
      "\\directlua{UcharcatLuaFollowup()}")
end
function UcharcatLuaFollowup()
    tex.setcatcode(chr,cct)
end
}
\def\Ucharcat{\directlua{UcharcatLua()}}

A:\the\catcode65:\Ucharcat 65 11:A:\the\catcode65\par
A:\the\catcode65:\Ucharcat 65  5:A:\the\catcode65\par
A:\the\catcode65:\Ucharcat 65 11:A:\the\catcode65\par


Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------