[XeTeX] newcommand with optional arguments

[Morten Høgholm]

On Tue, Mar 4, 2008 at 11:37 AM, Bruno Voisin <bvoisin at mac.com> wrote:

> Even after reading this several times, and trying to figure out how
> the LaTeX 3 experimental code converts to LaTeX 2e, I still don't get
> it: why all these \expandafter, what \\ (which in LaTeX means carriage
> return I think and in plain TeX can mean anything)

If you find out what \\ means in plain TeX, you'll realize how big of
a jester Knuth is. :-)

> and why a \long definition (in plain TeX a long def is one
> that includes carriage returns as part of its definition if I'm not
> mistaken, or one whose argument can include a carriage return, I'm not
> sure).

as this is for a generic function testing arbitrary token lists (which
may be handed down to it through several layers), it is better to make
it \long.

The usual trick for testing is to do
  \ifx\arbitrarymarker #1\arbitrarymarker <true> \else <false> \fi
If #1 is empty, the \ifx comparison is
\ifx\arbitrarymarker\arbitrarymarker = TRUE, else it is false. Now,
the problem is that #1 is inserted raw here and the test may fail if
the beginning of #1 is the marker chosen. The second and very serious
problem when used in low-level TeX is that #1 may be an \ifx test
itself such as "\ifx\aa\bb \else (diff)\fi". Then we get
  \ifx\arbitrarymarker \ifx\aa\bb \else (diff)\fi\arbitrarymarker
<true> \else <false> \fi
which compares \arbitrarymarker with \ifx and decides they are
different and TeX starts looking for an \else or \fi to finish this
conditional. So it now executes (diff), exits via that first \fi and
then executes "\arbitrarymarker <true> \else <false> \fi".

The advantage of using \detokenize is that we turn the tokens into
harmless characters. Thus, after the first expansion step we have
  \ifx\\\aa\bb \else (diff)\fi\\
thus comparing the token "\\" and the character "\". The harmless
charactertokens and the \\ are then skipped and the false branch
executed.

>
> I'm also surprised at seeing
>
> \expandafter\ifx\expandafter\\\detokenize{#1}\\
>
> I thought in such situations a cascade of \expandafter was needed, like
>
> \expandafter\expandafter\expandafter\ifx\expandafter\\\detokenize{#1}\\
>
> to get \detokenize{#1} expanded first, then \\, then \ifx.

We don't want \\ expanded, it just serves as a marker which is not a
character, that is all. In reality, it could even be a letter such as
  \expandafter\ifx\expandafter A\detokenize{#1}A%
because the letter A has catcode 11 but the one coming out of
\detokenize is catcode 12.

> But I must confess I never really got the hang of this \expandafter
> stuff, even after reading Appendix D of the TeXBook several times.

And you should not have to either IMO. The expl3 code is specifically
designed to alleviate the user from having to insert \expandafter all
over the place.

> If higher-level abstractions allowing that type of construction aren't
> provided to the user, I doubt the above will ever be used.

The \tlist_if_empty:nTF function (from l3tlp) is the higher-level
function: test if an arbitrary token list is empty. Then there is
\tlist_if_empty:oTF which expands the token list once and then hands
it over to the base function \tlist_if_empty:nTF.

-- 
Morten