[luatex] Hash tokens meaning

[Reinhard Kotucha]

On 2013-05-28 at 15:09:56 +0200, Sensei wrote:

 > I will get deeper into latex source code!

I fear that you have to read the TeXbook first.  LaTeX is a macro
package written in TeX.  The TeXbook describes the programming
language.  TeX is quite different from C and similar languages.
You'll certainly fail if you're not aware of its basic concepts.

If you prefer to start at the bottom, then forget LaTeX.  It's sitting
on top of TeX and adds an extra level of complexity which is
definitely not negligible.  And every macro package contributes to the
complexity.  If someone encounters a bug, we ask him to provide a
*minimal* example which reproduces the error, otherwise it's too
painful to debug it.  In your case I think that if you want to start
at the bottom than it's better to avoid LaTeX due to its complexity.
You can investigate LaTeX later.

Robin already pointed you to the LaTeX sources.  They are documented
somehow but the documentation assumes that the reader is familiar with
the content of the TeXbook.  In short:  The TeXbook is unavoidable.

 > That is not my purpose, and by the way, yes I look at the memory
 > when trying to figure out how a piece of software works 

Arthur already explained that you can't compare TeX with C.  C is a
low-level language and if you disassemble a program, you can certainly
see how it works.  However, TeX is extremely complex and you don't
achieve anything without having read the TeXbook before. 

 > (it's part of my job), especially when you assume that you don't
 > have the source code.

I hope that what you do is legal and you don't end up in prison one
day.  :)

However, the TeX source code is available and there shouldn't be a
need to reverse engineer things.  It's shipped with TeX Live.

I just created a PDF file for you and put it on my server:

  http://ms25.ath.cx/tex/tex.pdf

It describes Knuth's TeX, thus no PDF or Lua related code inside.

BTW, I understand that you want to find out how TeX works.  This is
understandable.  However, I'm convinced that the hash table is by far
the worst starting point.  I can't imagine a worse one.  The hash
table is meaningless unless you know where the entries come from and
what they are good for.

IMO the best approach to make TeX's internals visible is provided by
Patrick Gundlach.  His tool examines TeX's internals and uses Graphviz
in order to display them graphically.

I'm absolutely convinced that everything you're doing at this low
level is absolutely useless and you waste your time unless you've read
the TeXbook before.

Regards,
  Reinhard

-- 
----------------------------------------------------------------------------
Reinhard Kotucha                                      Phone: +49-511-3373112
Marschnerstr. 25
D-30167 Hannover                              mailto:reinhard.kotucha at web.de
----------------------------------------------------------------------------
Microsoft isn't the answer. Microsoft is the question, and the answer is NO.
----------------------------------------------------------------------------