[luatex] Getting all symbols

[Joseph Wright]

On 13/06/2012 07:35, Arno Trautmann wrote:
> Ulrike Fischer wrote:
>> Am Thu, 7 Jun 2012 17:20:48 +0200 schrieb Arno Trautmann:
>>>> I'm trying to find *at runtime* all the symbols allowed in a latex
>>>> environment. So, given a tex file, I'd like to access
>>>> - the complete predefined commands/symbols/definitions;
>>>> - the complete list of commands/symbols/definitions defined in
>>>> packages.
>>
>>> This may not really help, but maybe give a hint: I think I had asked a
>>> similar question some time ago, Tako answered that he wanted to
>>> implement something, and Hans answered that he [Tako] already did.
>>> Unfortunately I cannot find the corresponding mails, sorry.
>>
>> The thread starts here
>> http://tug.org/mailman/htdig/luatex/2011-July/003115.html
>>
>> And Hans answer is here
>> http://tug.org/mailman/htdig/luatex/2011-July/003125.html
> 
> Thanks for finding and posting this again!
> 
>> If I process the following document with
>>
>> "lualatex test > output.txt"
>>
>> I get a list of about 3500 commands in output.txt
>>
>> \documentclass{article}
>>
>> \begin{document}
>> \directlua{
>>   for name in pairs(tex.hashtokens()) do
>>       print(name)
>>   end}
>> \end{document}
>>
>> Addings fontspec leds to 14.000 commands.
> 
> I tried your example with fontspec and am somewhat puzzled with the
> result. The L3 syntax seems to lead to some strange entries, e.g.:
> 
> pdfdecimaldigits
> key code > fontspec/Script/Braille
> @ifdot
> key var > fontspec/Script/osma.req
> key var > fontspec/CharacterWidth/Quarter.default
> 
> How do these key code and key var expressions make it into the
> hashtokens? I guess it's because of the ~ that is used in L3, but I
> don't the exact reasoning.
> 
> Cheers
> Arno

As Patrick has said, any \csname construct can lead to 'odd' looking
names. For example, the LaTeX kernel uses spaces in names to produce
'robust' commands. So in the hash listing you will find both "cite" and
"cite " :-)

On why the names have spaces in them here, the idea is to make things
more 'human readable' in the log and so forth. It also means that these
names are 'out of the way' in namespacing terms, as they are quite
distinct from 'normal' LaTeX2e and LaTeX3 names. You see similar 'odd'
naming approaches for example with the NFSS (things like
"<5><6><7><8><9><10><10.95><12><14.4><17.28><20.74><24.88>cmcsc10" and
"OT1/cmr/m/n/7"), or with pgf (e.g. "pgfk@/pgfplots/table/header/has
colnames/. at cmd").

The thing to remember here is that these names are not really meant for
direct access: not everything in the hash table is a user command.
-- 
Joseph Wright