Apparent bug in pdftex/xetex (but not luatex)

Karl Berry karl at
Thu Apr 29 00:55:39 CEST 2021

    first 256 strings are statically initialized for internal purposes

Yeah. Indeed, I see that Knuth used 256 in {tex,mf}.web in his
non-implementation of editing:

    "E": if base_ptr>0 then if input_stack[base_ptr].name_field>=256 then
      begin print_nl("You want to edit file ");
    @.You want to edit file x@>

Therefore it seems we may as well follow suit?  So I adjusted the patch
and my rather verbose commentary about the code.  I probably messed up
somehow.  If any suggestions, let me know. I'll commit tomorrow barring

    MetaPost doesn't use the code

Right, I forgot. Living in the past.

    implies that the filename texmfmp.c is at least misleading, 

Yeah, I guess we (I) could/should rename it. --thanks, karl.

Index: lib/ChangeLog
--- lib/ChangeLog	(revision 59012)
+++ lib/ChangeLog	(working copy)
@@ -1,3 +1,16 @@
+2021-04-28  Marcel Fabian Krueger  <tex at>
+	    Hironori KITAGAWA <h_kitagawa2001 at>
+	    Karl Berry <karl at
+	* texmfmp.c (calledit): Skip \scantokens pseudo-files, indeed all
+	name_field values <=255 (following tex.web), and token lists
+	(state_field==0). Original reports from Don Hosek,
+	and Hironori (plus patch),
+	and Marcel (plus patch) for \scantokens,
 2021-03-23  Karl Berry  <karl at>
 	* TL'21.
Index: lib/texmfmp.c
--- lib/texmfmp.c	(revision 59012)
+++ lib/texmfmp.c	(working copy)
@@ -2624,29 +2624,49 @@ calledit (packedASCIIcode *filename,
      and a non-file for the insert. 
      Therefore, we have to traverse down input_stack (not input_file),
-     looking for name_field values >17, which correspond to open
-     files, and then the index_field value of that entry tells us the
+     looking for large enough name_field values corresponding to open
+     files. Then the index_field value of that entry tells us the
      corresponding element of input_file, which is what we need to close.
+     Additionally we have to skip all entries with state_field 0 since these
+     correspond to token lists and not input files.
-     We test for >17 because name_field=0 means the terminal,
+     We test for name_field<=255, following tex.web, because the first
+     256 strings are static, initialized by TeX. (Well, many more
+     strings are initialized, but we'll follow tex.web.)
+     For the record, name_field=0 means the terminal,
      name_field=1..16 means \openin stream n - 1,
-     name_field=17 means an invalid stream number (for read_toks).
-     Although ... seems like we should close any opened \openin files also.
-     Whoever is reading this, please implement that? Sigh.
+     name_field=17 means an invalid stream number (for read_toks),
+     name_field=18..19 means \scantokens pseudo-files (except for
+     original TeX of course). But 255 suffices for us.
-     Description in modules 300--304 of tex.web: "Input stacks and states."
      Here, we do not have to look at cur_input, the global variable
      which is effectively the top of input_stack, because it will always
      be a terminal (non-file) interaction -- the one where the user
-     typed "e" to start the edit.  */
+     typed "e" to start the edit.
+     In addition, state_field will be zero for token lists. Skip those too.
+     (Does not apply to Metafont.)
+     Description in modules 300--304 of tex.web: "Input stacks and states".
+     We should close any opened \openin files also. Whoever is reading
+     this, please implement that?  */
   int is_ptr; /* element of input_stack, 0 < input_ptr */  
   for (is_ptr = 0; is_ptr < inputptr; is_ptr++) {
-    if (inputstack[is_ptr].namefield <= 17) {
+#ifdef TeX
+    if (inputstack[is_ptr].statefield == 0 /* token list */
+        || inputstack[is_ptr].namefield <= 255) { /* can't be filename */
+#elif defined(MF)
+    if (inputstack[is_ptr].namefield <= 255) {
+#error "Unable to identify program" /* MetaPost doesn't use this file */
         ; /* fprintf (stderr, "calledit: skipped input_stack[%d], ", is_ptr);
-             fprintf (stderr, "namefield=%d <= 17\n",
-                      inputstack[is_ptr].namefield); */
+             fprintf (stderr, "namefield=%d <= 255 or statefield=%d == 0\n",
+                      inputstack[is_ptr].namefield,
+                      inputstack[is_ptr].statefield); */
     } else {
       FILE *f;
       /* when name_field > 17, index_field specifies the element of

More information about the tex-live mailing list.