[luatex] Hash tokens meaning

[Sensei]

On 5/28/13 6:11pm, Arthur Reutenauer wrote:
>    You really could have simply mentioned you were using tex.hashtokens()
> from the LuaTeX library.  "Hash token" is not part of the standard TeX
> vocabulary, but "token" is, with a different meaning.

As I said, I didn't post to TeX but to Lua because I used LuaLaTeX, and 
the subject of this post was about Hash tokens. I thought it was a 
LuaTeX feature, not a TeX one (no "\" in any token, so I didn't 
recognize TeX except for the names which are the same, except for "\"), 
and I thought it was clear. So, again as I said, I was wrong, I wasn't 
clear.

>    OK, but you do have the source code here: why refuse to use it?

Where did I ever refuse to use it? I said I would get myself the code 
and the TeX book.

>> Because it's new. I thought of it as an escaping in C, a sort, let's
>> say this, of protection of the next character, as in \% (the same
>> way in C for \").
>
>    That's understandable, but TeX's control sequence mechanism is
> different from escaping.  A control sequence is anything starting with a
> character having category code 0 -- that character is usually \, but can
> actually be anything -- and that translates to something else during
> macro expansion.  And you can even make \ a part of a control sequence's
> name, if you want, but that's a little bit convoluted (and not very
> useful).


I understand now.

>    And I did leave several parts out, read the TeXbook for the full story.

Yes, I am reading it.

>> Yes, I knew that once opened the hash file with an hex editor. I
>> knew TeX didn't have support for unicode, and I thought that
>> lualatex translated into TeX, which produced an output. So a unicode
>> string was unexpected, and I thought I messed up with my dump code.
>
>    That's another misconception, I'm afraid.  LuaTeX doesn't translate to
> TeX, it is an extension of TeX.  It augments the original code base with
> many new features -- including, obviously, Lua -- and its input and
> output are in UTF-8 (by default).
>
>    (And the code base has been converted from Pascal to C, etc., etc.,
> there are many subtopics to that aspect.)

A common misconception that I've fallen into.


>> Pardon me, but I'm used to write code in C, assembly, C++, or
>> whatever other programming language (mainly those three, in that
>> order). TeX is very, very different.
>
>    Yes, and it is a programming language too (specifically, a macro
> language).  It would be useful if you would stop resisting the notion.

Where did I resist? I accept the notion, and I am just trying to explain 
*WHY* I thought otherwise. It's not that I am saying that I was right.


>>>     Far more interesting are the different commands defined by LaTeX when
>>> \label is called, look for "ltxref.dtx" in latex.ltx.  The letter "r"
>>> (in \r at something) is introduced in a macro called \newlabel (line 3881
>>> of my copy of latex.ltx), and "@" in \@newl at bel, one line above it.
>>
>> That is awesome, I now have a place to start!
>
>    I still don't see what's exciting about these few macros, but if it
> makes you happy ...

I start walking, then I run. Since I don't know for instance where the 
equivalent of "main()" is in the LaTeX code, you provided me a


>> Anyway, at some point there *is* a static version of a code
>> somewhere, otherwise there would be no output.
>
>    I'm not sure that makes sense, but let's just leave it at that for the
> time being.

There is a static version because somewhere the codes are in RAM, 
somewhere the final version of a LaTeX command can be found.

>>                                                 Yes, I am biased by
>> my job and education, but I find hard to grasp the opposition to
>> this approach.
>
>    I can't believe I'm reading that.  *You* are the one coming towards
> TeX with a huge bias and a host of misconceptions, and you're talking
> about opposition to your approach on *our* part?  You freely admit that
> you don't understand the way a source file is structured, and you want
> us to help you analyse a file that you've produced according to your own
> taste, without even showing us what the input was?  You won't tell us
> your name, and use a pseudonym that, when you're in the position of a
> student, is at best snooty?  If it wasn't for our expertise, your
> approach wouldn't have led you anywhere.


I never asked to analyze a file, reread my post. I asked about specific 
tokens.

I am quite sure of my knowledge to recognize that I am not able to make 
it on my own.

So, since I recognize my bias, I asked for something that in the 
programming language community is clear. If I have a function in C 
called "main()" how will this be translated? Well, you'll end up with a 
symbol "_main". If you don't see why this connection makes sense... but 
anyway, if you read my posts with calm and zen, you will see that I 
always said "I was wrong".

Should I flog myself for being wrong? It's the only way to improve.

Again, I never had an attitude of someone saying "you pricks don't know 
s*it and I do LOL". If you ever see in one of my posts that, I didn't do 
that *INTENTIONALLY*.


>  Sensei my *ss.

And that's very mature.

I understand you know everything on every topic, and don't need to ask 
questions to those who know more than you. Well, I am sorry I am not 
skilled on these matters, that's why I asked.


A "RTFM" attitude won't make you any more respected.