[latex3-commits] [l3svn] r5889 - Document that rescanning applies 'current' catcodes

[noreply at latex-project.org]

Author: joseph
Date: 2015-08-25 09:42:30 +0200 (Tue, 25 Aug 2015)
New Revision: 5889

Modified:
   trunk/l3kernel/l3tl.dtx
Log:
Document that rescanning applies 'current' catcodes

I've also added a 'warning note' here: rescanning is pretty risky.


Modified: trunk/l3kernel/l3tl.dtx
===================================================================
--- trunk/l3kernel/l3tl.dtx	2015-08-24 17:30:59 UTC (rev 5888)
+++ trunk/l3kernel/l3tl.dtx	2015-08-25 07:42:30 UTC (rev 5889)
@@ -323,6 +323,14 @@
 %
 % \section{Reassigning token list category codes}
 %
+% These functions allow the rescanning of tokens: re-apply \TeX{}'s
+% tokenization process to apply category codes different from those
+% in force when the tokens were absorbed. Whilst this functionality is
+% supported, it is often preferable to find alternative approaches 
+% to achieving outcomes rather than rescanning tokens (for example
+% construction of token lists token-by-token with intervening category
+% code changes).
+%
 % \begin{function}[updated = 2015-08-11]
 %   {
 %     \tl_set_rescan:Nnn,  \tl_set_rescan:Nno,  \tl_set_rescan:Nnx,
@@ -335,11 +343,14 @@
 %   \end{syntax}
 %   Sets \meta{tl~var} to contain \meta{tokens}, applying the category
 %   code r\'{e}gime specified in the \meta{setup} before carrying out
-%   the assignment.  This allows the \meta{tl~var} to contain material
+%   the assignment. (Category codes applied to tokens not explicitly covered
+%   by the \meta{setup} will be those in force at the point of use of
+%   \cs{tl_set_rescan:Nnn}.)
+%   This allows the \meta{tl~var} to contain material
 %   with category codes other than those that apply when \meta{tokens}
 %   are absorbed.  The \meta{setup} is run within a group and may
 %   contain any valid input, although only changes in category codes
-%   are relevant.  See also \cs{tl_rescan:nn}.
+%   are relevant. See also \cs{tl_rescan:nn}.
 %   \begin{texnote}
 %     The \meta{tokens} are first turned into a string (using
 %     \cs{tl_to_str:n}).  If the string contains one or more characters
@@ -362,7 +373,10 @@
 %   \end{syntax}
 %   Rescans \meta{tokens} applying the category code r\'{e}gime
 %   specified in the \meta{setup}, and leaves the resulting tokens in
-%   the input stream.  The \meta{setup} is run within a group and may
+%   the input stream. (Category codes applied to tokens not explicitly covered
+%   by the \meta{setup} will be those in force at the point of use of
+%   \cs{tl_rescan:nn}.)
+%   The \meta{setup} is run within a group and may
 %   contain any valid input, although only changes in category codes
 %   are relevant.  See also \cs{tl_set_rescan:Nnn}, which is more
 %   robust than using \cs{tl_set:Nn} in the \meta{tokens} argument of