[latex3-commits] [git/LaTeX3-latex3-latex3] main: Fix \peek_analysis_map_inline:n for not-expanded tokens (fix #1113) (34bf4776f)

[Bruno Le Floch]

Repository : https://github.com/latex3/latex3
On branch  : main
Link       : https://github.com/latex3/latex3/commit/34bf4776f89fc0d5c37f15ecb164a132e7ad92e6

>---------------------------------------------------------------

commit 34bf4776f89fc0d5c37f15ecb164a132e7ad92e6
Author: Bruno Le Floch <blflatex at gmail.com>
Date:   Mon Oct 3 22:16:12 2022 +0200

    Fix \peek_analysis_map_inline:n for not-expanded tokens (fix #1113)
    
    When followed by an implicit special token, the code saves the
    next token using \futurelet, then stringifies the first token and
    collects characters until hitting the saved next token.  If that
    token was in a "notexpanded" state then the collection loop never
    terminated.  Now fixed by simply running \futurelet twice to get
    rid of the "notexpanded" state.  One could conceivably want to
    keep track of it and restore it.


>---------------------------------------------------------------

34bf4776f89fc0d5c37f15ecb164a132e7ad92e6
 l3kernel/CHANGELOG.md      |  5 +++--
 l3kernel/l3tl-analysis.dtx | 10 +++++++++-
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/l3kernel/CHANGELOG.md b/l3kernel/CHANGELOG.md
index a76cafb94..2db59e1bb 100644
--- a/l3kernel/CHANGELOG.md
+++ b/l3kernel/CHANGELOG.md
@@ -13,8 +13,9 @@ this project uses date-based 'snapshot' version identifiers.
 
 ### Fixed
 - `\peek_analysis_map_inline:n` support for macro parameter characters (issue
-  [\#1109](https://github.com/latex3/latex3/issues/1109)) and for many
-  expandable tokens (issue [\#1110](https://github.com/latex3/latex3/issues/1110))
+  [\#1109](https://github.com/latex3/latex3/issues/1109)), for many expandable
+  tokens (issue [\#1110](https://github.com/latex3/latex3/issues/1110)), and an
+  esoteric case (issue [\#1113](https://github.com/latex3/latex3/issues/1113))
 
 ## [2022-09-28]
 
diff --git a/l3kernel/l3tl-analysis.dtx b/l3kernel/l3tl-analysis.dtx
index 8228bff64..cb5df54c4 100644
--- a/l3kernel/l3tl-analysis.dtx
+++ b/l3kernel/l3tl-analysis.dtx
@@ -1183,7 +1183,8 @@
 %     \@@_peek_analysis_nonexp:N, \@@_peek_analysis_cs:N,
 %     \@@_peek_analysis_char:N, \@@_peek_analysis_char:w,
 %     \@@_peek_analysis_special:, \@@_peek_analysis_retest:,
-%     \@@_peek_analysis_next:, \@@_peek_analysis_str:,
+%     \@@_peek_analysis_next:, \@@_peek_analysis_nextii:,
+%     \@@_peek_analysis_str:,
 %     \@@_peek_analysis_str:w, \@@_peek_analysis_str:n,
 %     \@@_peek_analysis_active_str:n, \@@_peek_analysis_explicit:n,
 %     \@@_peek_analysis_escape:, \@@_peek_analysis_collect:w,
@@ -1420,6 +1421,8 @@
 %   active character with the same character code as its meaning or as
 %   the escape character).  Now look at the \meta{next token} following
 %   it using a combination of \tn{afterassignment} and \tn{futurelet}.
+%   (In fact look twice to reset an internal \TeX{} flag in case the
+%   \meta{next token} had been hit with \cs{exp_not:N}.)
 %   The syntax of this primitive is \tn{futurelet} \meta{peek token}
 %   \meta{first token} \meta{next token}, and it sets \meta{peek token}
 %   equal to \meta{next token}.  Traditionally, one takes \meta{first
@@ -1436,6 +1439,11 @@
   {
     \tl_if_empty:oT { \tex_the:D \tex_everyeof:D }
       { \tex_everyeof:D { \scan_stop: } }
+    \tex_afterassignment:D \@@_peek_analysis_nextii:
+    \tex_futurelet:D \l_@@_analysis_next_token
+  }
+\cs_new_protected:Npn \@@_peek_analysis_nextii:
+  {
     \tex_afterassignment:D \@@_peek_analysis_str:
     \tex_futurelet:D \l_@@_analysis_next_token
   }