[luatex] Where can I learn more about catcodes in LuaTeX?

[Eduardo Ochs]

Hi list,

I just sent a question about catcodes that had the wrong tone... let
me try to rewrite it completely.

I am the author of a package called "Dednat6" for drawing diagrams
with luatex/lualatex. It is very extensible, and, among other things,
it provides a way to run Lua code embedded in specially-marked
comments - more precisely, in what I call "%L"-blocks. This is
explained in sections 3 and 4 of the TUGBoat article about Dednat6:

  https://tug.org/TUGboat/tb39-3/tb123ochs-dednat.pdf

I am trying to convince more category theorists to try Dednat6, but I
am afraid that they will find LuaTeX too hard to learn, as I did, and
they will give up. One of the things that I am trying to do to remedy
this is to collect pointers on where they can learn things.



The message about catcodes that I sent yesterday is here:

  https://tug.org/pipermail/luatex/2020-June/007364.html

It contains a simple test that reveals - I mean, to an imaginary
newbie reading it - that the TeX interpreter and tex.print use two
different catcode tables.

One obvious place to learn more about these different catcode tables
is the LuaTeX reference manual. Let me use a trick from eev to point
to sections of the manual. The trick is explained here,

  http://angg.twu.net/eev-intros/find-pdf-like-intro.html#7
  (find-pdf-like-intro "7. Shorter hyperlinks to PDF files")

and - for the sake of completeness, and for Emacs users only - there
is an introduction of how to use Emacs and eev to create and share
"executable notes" here:

  http://angg.twu.net/emacsconf2019.html

To make a long story short, after running these sexps in Emacs,

(code-pdf-page "luatexref"
"/usr/local/texlive/2019/texmf-dist/doc/context/documents/general/manuals/luatex.pdf")
(code-pdf-text "luatexref"
"/usr/local/texlive/2019/texmf-dist/doc/context/documents/general/manuals/luatex.pdf"
4)

I can use the sexp hyperlinks below to point to specific sections of
the manual; I believe that they are easy to understand.

  (find-luatexrefpage        5  "Contents")
  (find-luatexreftext        5  "Contents")
  (find-luatexrefpage (+ 4  24) "Note that the expansion of \\directlua")
  (find-luatexreftext (+ 4  24) "Note that the expansion of \\directlua")
  (find-luatexrefpage (+ 4  27) "2.5 Catcode tables")
  (find-luatexreftext (+ 4  27) "2.5 Catcode tables")
  (find-luatexrefpage (+ 4 197) "10.3.14.1 print")
  (find-luatexreftext (+ 4 197) "10.3.14.1 print")

So, how can I do things like inspecting catcodes and changing them
from Lua? I saw that there is a brief mention of a "tex.catcodetable"
here,

  (find-luatexrefpage (+ 4 186) "10.3.2.1 Integer parameters")
  (find-luatexreftext (+ 4 186) "10.3.2.1 Integer parameters")
  (find-luatexreftext (+ 4 186) "10.3.2.1 Integer parameters"
"tex.catcodetable")

but I didn't find there the ways to inspect and change entries of
catcode tables from Lua...




I am quite outsider-ish in the LuaTeX community, and I have the
sensation that I don't even know where are the forums in which the
discussions are happening... so:

Please, people, if you have any pointers that you believe that may be
useful to me, I'd like to see them - and private messages are welcome.

I *guess* that the material - both "reference" and "discussions" -
about catcodes in LuaTeX is scattered through several mailing lists,
and in comments in the source code of LuaTeX itself, in the source
code of packages that use catcodes in luatex-ish ways, in TUGBoat
articles, in slides, in posts in stackexchange... and I don't know
where else, and I don't know if there are chat channels about LuaTeX
nowadays - did they migrate from IRC to things like Slack and
Zulip?...

  Cheers and thanks in advance,
    Eduardo Ochs
    http://angg.twu.net/dednat6.html
    http://angg.twu.net/dednat6.html#hacking