[latex3-commits] [git/LaTeX3-latex3-latex3] cs-delimited-args: Further explanation on argument delimiters (9f066640c)

[PhelypeOleinik]

Repository : https://github.com/latex3/latex3
On branch  : cs-delimited-args
Link       : https://github.com/latex3/latex3/commit/9f066640c5bb1b5f083f53d34b429c22fbbc967a

>---------------------------------------------------------------

commit 9f066640c5bb1b5f083f53d34b429c22fbbc967a
Author: PhelypeOleinik <tex.phelype at gmail.com>
Date:   Sat Feb 29 00:20:07 2020 -0300

    Further explanation on argument delimiters


>---------------------------------------------------------------

9f066640c5bb1b5f083f53d34b429c22fbbc967a
 l3packages/xparse/xparse.dtx | 65 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 65 insertions(+)

diff --git a/l3packages/xparse/xparse.dtx b/l3packages/xparse/xparse.dtx
index 5f04632c4..d9e6dfd90 100644
--- a/l3packages/xparse/xparse.dtx
+++ b/l3packages/xparse/xparse.dtx
@@ -489,6 +489,71 @@
 %     is given: \texttt{G}\marg{default}.
 % \end{itemize}
 %
+% \subsection{Details about argument delimiters}
+%
+% In normal (non-expandable) commands, the delimited types look for the
+% initial delimiter by peeking ahead (using \pkg{expl3}'s |\peek_...|
+% functions) looking for the delimiter token.  The token has to have the
+% same meaning and \enquote{shape} of the token defined as delimiter.
+% There are three possible cases of delimiters: character tokens, control
+% sequence tokens, and active character tokens.  For all practical purposes
+% of this description, active character tokens will behave exactly as
+% control sequence tokens.
+%
+% \subsubsection{Character tokens}
+%
+% A character token is characterised by its character code, and its meaning
+% is the category code~(|\catcode|).  When a command is defined, the meaning
+% of the character token is fixed into the definition of the command and
+% cannot change.  A command will correctly see an argument delimiter if
+% the open delimiter has the same character and category codes as at the
+% time of the definition.  For example in:
+% \begin{verbatim}
+%   \NewDocumentCommand { \foobar } { D<>{default} } {(#1)}
+%   \foobar <hello> \par
+%   \char_set_catcode_letter:N <
+%   \foobar <hello>
+% \end{verbatim}
+% the output would be:
+% \begin{verbatim}
+%   (hello)
+%   (default)<hello>
+% \end{verbatim}
+% as the open-delimter |<| changed in meaning between the two calls to
+% |\foobar|, so the second one doesn't see the |<| as a valid delimiter.
+% Commands assume that if a valid open-delimiter was found, a matching
+% close-delimiter will also be there.  If it is not (either by being
+% omitted or by changing in meaning), a low-level \TeX{} error is raised
+% and the command call is aborted.
+%
+% \subsubsection{Control sequence tokens}
+%
+% A control sequence (or control character) token is characterised by is
+% its name, and its meaning is its definition.  For example, after
+% |\cs_set:Npn \tmp_A:w {abc}| and |\cs_set:Npn \tmp_B:w {abc}|, the
+% two control sequence tokens |\tmp_A:w| and |\tmp_B:w| have the same
+% meaning (|abc|).  A token cannot have two different meanings at the
+% same time.  When a control sequence is used as delimiter in a command,
+% both its name (string representation) and meaning have to match when
+% the command is scanning for the delimiter.  However,
+% different from the character token, the meaning of a control sequence
+% is not fixed at the time the command is defined, so if it is redefined
+% later, the command still finds the delimiter as long as the control
+% sequence name is the same:  For example in:
+% \begin{verbatim}
+%   \cs_set:Npn \x { abc }
+%   \NewDocumentCommand { \foobar } { D\x\y{default} } {(#1)}
+%   \foobar \x hello\y \par
+%   \cs_set:Npn \x { def }
+%   \foobar \x hello\y
+% \end{verbatim}
+% the output would be:
+% \begin{verbatim}
+%   (hello)
+%   (hello)
+% \end{verbatim}
+% with both calls to the command seeing the delimiter |\x|.
+%
 % \section{Declaring commands and environments}
 %
 % With the concept of an argument specifier defined, it is now