[luatex] macros to invoke lua in LaTeX

[Taco Hoekwater]

Hi Karl,

A not too short mail follows that attempts to answer both of your
email messages at once.

Karl Berry wrote:
> Not just right now, but at any time in the future: if a new primitive
> \foo is introduced, any existing package that happens to use \foo is
> potentially affected.  

This is the best argument ever -- for not developing luatex at all.

Until version luatex 1.0, we reserve the right to make any change.
Because if we were to promise not to remove old primitives and to
avoid adding conflicting primitives, you get exactly what you have
already: pdftex. Which isn't going away.

People that are hung up on compatibility should stick with pdftex
in any case, because luatex's *output* is potentially different
(and will remain to be so, as that is an unavoidable consequence
of the unwrapping of the engine internals). The name of some primitives
is really a minor point in view of these other engine changes.

In the particular case of \pdfprimitive, I don't mind promising 
(herewith) that that will remain in the engine and that it will keep
doing what it did in pdftex, with just possibly a bug fix here or there.
But we won't go as far as making that promise in the general case.

For example, right at the start of luatex, we threw out a bunch of
primitives from Heiko Oberdiek and myself that provide various scripting
subroutines for use inside pdftex. Why? Because in luatex that sort of
stuff should be done in lua, not in tex macros. If you want to use
Heiko's subroutines in their \primitive form, simply keep using pdftex.

About primitive names:

The core problem is that the \newcommand macro in latex does not
allow *any* new primitives: it assumes the engine is totally stable.
Adding some special prefix does not solve this, it merely attempts at
making clashes less likely by increasing the hash key length. I will
not accept that as a general solution to the problem.

If there was a functional proposal for namespaces, I would gladly
implement that and put the new primitives in a dedicated namespace.
But any language extension to allow for namespaces would likely be
just as incompatible, so I highly doubt that would solve your issue.

Prefixes with a generic engine prefix are just silly. In the new Unicode
math support I've been adding a lot of primitives (more than 100).  A
whole bunch of them are identical to xetex counterparts (\XeTeXmathchar
and friends). But I couldn't keep their names, because then a package
could believe it was running xetex instead of luatex (!). Some talk with
JK and Will resulted in the \U prefix and I am happy with that, it is
a good compromise to have \U... for all the new math-related primitives
now.

But that does not mean that I should now start creating say \Udirectlua,
because \directlua has nothing to do with Unicode. Just like
\pdfprimitive has nothing to do with pdf. When choosing a primitive
name, I believe that fitness to purpose should weigh more heavily
than conceptual compatibity concerns.

Actual known clashes are a different issue, and we do try quite hard
to avoid such names: for example, luatex will never get \begin as a
new primitive.

Best wishes,
Taco