Apparent bug in pdftex/xetex (but not luatex)

[Marcel Krüger]

On Tue, Apr 27, 2021 at 02:41:33PM -0500, Don Hosek wrote:
> Raising this here at the suggestion of David Carlisle. Original post at tex.se <http://tex.se/>: https://tex.stackexchange.com/questions/594702/apparent-bug-in-pdflatex-xelatex-but-not-lualatex <https://tex.stackexchange.com/questions/594702/apparent-bug-in-pdflatex-xelatex-but-not-lualatex>
> 
> test.tex:
> 
> \def\zz#1{#1}
> \zz{\input test.l2e }
> 
> text.l2e:
> 
> \oops
> Hit E at the error prompt and an error along the lines of:
> 
> pdflatex:calledit: unexpected if_ptr=5 not in range 1..2,from input_stack[2].namefield=2378
> xelatex:calledit: unexpected if_ptr=6 not in range 1..2,from input_stack[2].namefield=2230090
> 
> will appear.
> 
> I’ve not had a chance to dig into source to look for the issue. David’s diagnosis from stackexchange:
> 
> > Note you need the wrapping macro, if you just call \input directly it works, presumably related to the input_stack[2] in the error when it fails.
> -dh

I attached a patch which fixes both this issue and a similar problem
which is triggered doing the same with test.tex containing

    \scantokens{\input test.l2e }

Best regards,
Marcel
-------------- next part --------------
>From 4c72847515c123dc729d6588f660072428f4909e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marcel=20Fabian=20Kr=C3=BCger?= <tex at 2krueger.de>
Date: Tue, 27 Apr 2021 22:57:00 +0200
Subject: [PATCH] Fix bugs in calledit

---
 texk/web2c/lib/ChangeLog |  5 +++++
 texk/web2c/lib/texmfmp.c | 15 ++++++++++++---
 2 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/texk/web2c/lib/ChangeLog b/texk/web2c/lib/ChangeLog
index baa54a525..6caf64e44 100644
--- a/texk/web2c/lib/ChangeLog
+++ b/texk/web2c/lib/ChangeLog
@@ -1,3 +1,8 @@
+2021-04-27  Marcel Krüger  <tex at 2krueger.de>
+
+	* texmfmp.c: Skip \scantokens pseudo files and token lists in
+	calledit.
+
 2021-03-23  Karl Berry  <karl at tug.org>
 
 	* TL'21.
diff --git a/texk/web2c/lib/texmfmp.c b/texk/web2c/lib/texmfmp.c
index eeb32eaf2..8a2935fdb 100644
--- a/texk/web2c/lib/texmfmp.c
+++ b/texk/web2c/lib/texmfmp.c
@@ -2627,10 +2627,13 @@ calledit (packedASCIIcode *filename,
      looking for name_field values >17, which correspond to open
      files, and then the index_field value of that entry tells us the
      corresponding element of input_file, which is what we need to close.
+     Additionally we have to skip all entries with state_field 0 since these
+     correspond to token lists and not input files.
 
-     We test for >17 because name_field=0 means the terminal,
+     We test for >19 because name_field=0 means the terminal,
      name_field=1..16 means \openin stream n - 1,
-     name_field=17 means an invalid stream number (for read_toks).
+     name_field=17 means an invalid stream number (for read_toks),
+     name_field=18..19 means \scantokens pseudo files
      Although ... seems like we should close any opened \openin files also.
      Whoever is reading this, please implement that? Sigh.
      
@@ -2643,7 +2646,13 @@ calledit (packedASCIIcode *filename,
  {  
   int is_ptr; /* element of input_stack, 0 < input_ptr */  
   for (is_ptr = 0; is_ptr < inputptr; is_ptr++) {
-    if (inputstack[is_ptr].namefield <= 17) {
+#ifdef TeX
+    if (inputstack[is_ptr].statefield == 0 || inputstack[is_ptr].namefield <= 19) {
+#elif defined(MF)
+    if (inputstack[is_ptr].namefield <= 2) {
+#else
+#error "Unable to identify program"
+#endif
         ; /* fprintf (stderr, "calledit: skipped input_stack[%d], ", is_ptr);
              fprintf (stderr, "namefield=%d <= 17\n",
                       inputstack[is_ptr].namefield); */
-- 
2.31.1