[latex3-commits] [git/LaTeX3-latex3-latex3] master: Make \char_generate:nn take two steps in all cases (03eab0c)
Bruno Le Floch
bruno at le-floch.fr
Wed Jan 16 00:15:41 CET 2019
Repository : https://github.com/latex3/latex3
On branch : master
Link : https://github.com/latex3/latex3/commit/03eab0cb1c836a2066729986b1a01eeaa6ba3194
>---------------------------------------------------------------
commit 03eab0cb1c836a2066729986b1a01eeaa6ba3194
Author: Bruno Le Floch <bruno at le-floch.fr>
Date: Wed Jan 16 00:15:41 2019 +0100
Make \char_generate:nn take two steps in all cases
>---------------------------------------------------------------
03eab0cb1c836a2066729986b1a01eeaa6ba3194
l3kernel/CHANGELOG.md | 4 ++++
l3kernel/l3token.dtx | 34 +++++++++++++++----------------
l3kernel/testfiles/m3char001.luatex.tlg | 16 +++++++++++++++
l3kernel/testfiles/m3char001.lvt | 29 +++++++++++++++++++++++++-
l3kernel/testfiles/m3char001.tlg | 16 +++++++++++++++
l3kernel/testfiles/m3char001.xetex.tlg | 26 +++++++++++++++++++++++
6 files changed, 107 insertions(+), 18 deletions(-)
diff --git a/l3kernel/CHANGELOG.md b/l3kernel/CHANGELOG.md
index c6eff81..a0d6799 100644
--- a/l3kernel/CHANGELOG.md
+++ b/l3kernel/CHANGELOG.md
@@ -7,6 +7,10 @@ this project uses date-based 'snapshot' version identifiers.
## [Unreleased]
+### Changed
+
+- `\char_generate:nn` now always takes exactly two expansions
+
### Fixed
- In (u)platex: detection of spaces in `\tl_rescan:nn` and related functions
diff --git a/l3kernel/l3token.dtx b/l3kernel/l3token.dtx
index 6fb3279..5f9fef3 100644
--- a/l3kernel/l3token.dtx
+++ b/l3kernel/l3token.dtx
@@ -123,7 +123,7 @@
% be an active character.
% \end{function}
%
-% \begin{function}[EXP, added = 2015-09-09, updated = 2018-04-19]
+% \begin{function}[EXP, added = 2015-09-09, updated = 2019-01-16]
% {\char_generate:nn}
% \begin{syntax}
% \cs{char_generate:nn} \Arg{charcode} \Arg{catcode}
@@ -146,6 +146,9 @@
% and other values raise an error. The \meta{charcode} may be any one valid
% for the engine in use.
% Active characters cannot be generated in older versions of \XeTeX{}.
+% \begin{texnote}
+% Exactly two expansions are needed to produce the character.
+% \end{texnote}
% \end{function}
%
% \begin{variable}[added = 2011-09-05]{\c_catcode_other_space_tl}
@@ -1417,7 +1420,9 @@
% Engine-dependent definitions are now needed for the implementation. For
% \LuaTeX{} and \XeTeX{} there is engine-level support.
% They can do cases that macro emulation can't. All of those are filtered
-% out here using a primitive-based boolean expression for speed.
+% out here using a primitive-based boolean expression to avoid fixing
+% the category code of the null character used in the false branch
+% (for 8-bit engines).
% The final level is the basic definition at the engine level: the arguments
% here are integers so there is no need to worry about them too much.
% Older versions of \XeTeX{} cannot generate active characters so we filter
@@ -1467,7 +1472,7 @@
}
\else:
% \end{macrocode}
-% For engines where \tn{Ucharcat} isn't available (or emulated) then we have
+% For engines where \tn{Ucharcat} isn't available or emulated, we have
% to work in macros, and cover only the $8$-bit range. The first stage is
% to build up a |tl| containing |^^@| with each category code that can
% be accessed in this way, with an error set up for the other cases. This
@@ -1485,21 +1490,8 @@
\tl_set:Nx \l_@@_tmp_tl { \l_@@_tmp_tl }
\char_set_catcode_math_toggle:n { 0 }
\tl_put_right:Nn \l_@@_tmp_tl { \or: ^^@ }
-% \end{macrocode}
-% As \TeX{} is very unhappy if if finds an alignment character inside
-% a primitive \tn{halign} even when skipping false branches, some precautions
-% are required. \TeX{} is happy if the token is hidden inside
-% \tn{unexpanded} (which needs to be the primitive). The expansion chain here
-% is required so that the conditional gets cleaned up correctly (other code
-% assumes there is exactly one token to skip during the clean-up).
-% \begin{macrocode}
\char_set_catcode_alignment:n { 0 }
- \tl_put_right:Nn \l_@@_tmp_tl
- {
- \or:
- \__kernel_exp_not:w \exp_after:wN
- { \exp_after:wN ^^@ \exp_after:wN }
- }
+ \tl_put_right:Nn \l_@@_tmp_tl { \or: ^^@ }
\tl_put_right:Nn \l_@@_tmp_tl { \or: }
\char_set_catcode_parameter:n { 0 }
\tl_put_right:Nn \l_@@_tmp_tl { \or: ^^@ }
@@ -1552,15 +1544,23 @@
%<*initex>
\int_step_function:nnN { 0 } { 255 } \@@_tmp:n
%</initex>
+% \end{macrocode}
+% As \TeX{} is very unhappy if if finds an alignment character inside
+% a primitive \tn{halign} even when skipping false branches, some
+% precautions are required. \TeX{} is happy if the token is hidden
+% between braces within \cs{if_false:} \dots{} \cs{fi:}.
+% \begin{macrocode}
\cs_new:Npn \@@_generate_aux:nnw #1#2#3 \exp_end:
{
#3
+ \if_false: { \fi:
\exp_after:wN \exp_after:wN
\exp_after:wN \exp_end:
\exp_after:wN \exp_after:wN
\if_case:w #2
\exp_last_unbraced:Nv \exp_stop_f:
{ c_@@_ \@@_int_to_roman:w #1 _tl }
+ \or: }
\fi:
}
\fi:
diff --git a/l3kernel/testfiles/m3char001.luatex.tlg b/l3kernel/testfiles/m3char001.luatex.tlg
index e19a7f2..1e4c5de 100644
--- a/l3kernel/testfiles/m3char001.luatex.tlg
+++ b/l3kernel/testfiles/m3char001.luatex.tlg
@@ -474,6 +474,8 @@ l. ... }
============================================================
TEST 6: Use inside \halign
============================================================
+cell 2
+C
============================================================
============================================================
TEST 7: \char_codepoint_to_bytes:n
@@ -483,3 +485,17 @@ TEST 7: \char_codepoint_to_bytes:n
{225}{136}{128}{}
{240}{144}{128}{128}
============================================================
+============================================================
+TEST 8: Number of expansions
+============================================================
+begin-group character A
+end-group character A
+math shift character A
+alignment tab character A
+macro parameter character A
+superscript character A
+subscript character A
+the letter A
+the character A
+undefined
+============================================================
diff --git a/l3kernel/testfiles/m3char001.lvt b/l3kernel/testfiles/m3char001.lvt
index 7d23f6c..5d36e5f 100644
--- a/l3kernel/testfiles/m3char001.lvt
+++ b/l3kernel/testfiles/m3char001.lvt
@@ -1,5 +1,5 @@
%
-% Copyright (C) 2015,2016,2018 The LaTeX Project
+% Copyright (C) 2015,2016,2018,2019 The LaTeX Project
%
\documentclass{minimal}
\input{regression-test}
@@ -132,6 +132,11 @@
{
\tex_halign:D
{ # \tex_cr:D A \char_generate:nn { `A } { 12 } \tex_cr:D }
+ \tex_halign:D
+ {
+ # & \TYPE{cell~2} \TYPE # \tex_cr:D
+ A \char_generate:nn { `B } { 4 } C \tex_cr:D
+ }
}
\TESTEXP { \char_codepoint_to_bytes:n }
@@ -142,4 +147,26 @@
\char_codepoint_to_bytes:n { "10000 }
}
+\OMIT
+\cs_gset:Npn \test:nn #1#2
+ {
+ \exp_after:wN \exp_after:wN \exp_after:wN \token_to_meaning:N
+ \char_generate:nn {#1} {#2} \NEWLINE
+ }
+\TIMO
+
+\TESTEXP { Number~of~expansions }
+ {
+ \test:nn { 65 } { 1 }
+ \test:nn { 65 } { 2 }
+ \test:nn { 65 } { 3 }
+ \test:nn { 65 } { 4 }
+ \test:nn { 65 } { 6 }
+ \test:nn { 65 } { 7 }
+ \test:nn { 65 } { 8 }
+ \test:nn { 65 } { 11 }
+ \test:nn { 65 } { 12 }
+ \test:nn { 65 } { 13 }
+ }
+
\END
diff --git a/l3kernel/testfiles/m3char001.tlg b/l3kernel/testfiles/m3char001.tlg
index c6d6f67..4887919 100644
--- a/l3kernel/testfiles/m3char001.tlg
+++ b/l3kernel/testfiles/m3char001.tlg
@@ -509,6 +509,8 @@ l. ... }
============================================================
TEST 6: Use inside \halign
============================================================
+cell 2
+C
============================================================
============================================================
TEST 7: \char_codepoint_to_bytes:n
@@ -518,3 +520,17 @@ TEST 7: \char_codepoint_to_bytes:n
{225}{136}{128}{}
{240}{144}{128}{128}
============================================================
+============================================================
+TEST 8: Number of expansions
+============================================================
+begin-group character A
+end-group character A
+math shift character A
+alignment tab character A
+macro parameter character A
+superscript character A
+subscript character A
+the letter A
+the character A
+undefined
+============================================================
diff --git a/l3kernel/testfiles/m3char001.xetex.tlg b/l3kernel/testfiles/m3char001.xetex.tlg
index b4f558c..7ac1dd1 100644
--- a/l3kernel/testfiles/m3char001.xetex.tlg
+++ b/l3kernel/testfiles/m3char001.xetex.tlg
@@ -475,6 +475,8 @@ l. ... }
============================================================
TEST 6: Use inside \halign
============================================================
+cell 2
+C
============================================================
============================================================
TEST 7: \char_codepoint_to_bytes:n
@@ -484,3 +486,27 @@ TEST 7: \char_codepoint_to_bytes:n
{225}{136}{128}{}
{240}{144}{128}{128}
============================================================
+============================================================
+TEST 8: Number of expansions
+============================================================
+! Undefined control sequence.
+<argument> \LaTeX3 error:
+ Cannot generate active chars.
+l. ... }
+The control sequence at the end of the top line
+of your error message was never \def'ed. If you have
+misspelled it (e.g., `\hobx'), type `I' and the correct
+spelling (e.g., `I\hbox'). Otherwise just continue,
+and I'll forget about whatever was undefined.
+begin-group character A
+end-group character A
+math shift character A
+alignment tab character A
+macro parameter character A
+superscript character A
+subscript character A
+the letter A
+the character A
+macro:->\TYPE {
+}
+============================================================
More information about the latex3-commits
mailing list