[tlbuild] [tex-live] TL2010 pretest luatex fails on Solaris 10 sparc

[Vladimir Volovich]

Hi Taco and George,

"TH" == Taco Hoekwater writes:

 >> * why would luatex not interpret the command line, but display a "*"
 >> prompt?

 TH> Because the topenin() function in luatex.c fails to copy the
 TH> command line to the internal buffer, I assume.  But as to why? I
 TH> have no idea.

 >> * is it possible to gain more information on why it stops with
 >> "emergency stop"?

 TH> Probably because input_line() in luatex.c also fails to copy the
 TH> line to the internal buffer. I still do not have any idea why,
 TH> though.

i've made some debugging of luatex compiled on sparc solaris using sun
studio.

my understanding of why the bug is happening is as follows:

inputstack.h contains:

typedef struct in_state_record {
    halfword start_field;
    halfword loc_field;
    halfword limit_field;
    halfword name_field;
    halfword ocp_lstack_field;  /* used for omega translation processes */
    int synctex_tag_field;      /* stack the tag of the current file */
    halfword ocp_no_field:16;   /* used for omega translation processes */
    int cattable_field:16;      /* category table used by the current line (see textoken.c) */
    quarterword state_field:8;
    quarterword index_field:8;
    boolean partial_field:8;    /* is the current line partial? (see textoken.c) */
    boolean nofilter_field:8;   /* used by token filtering */
} in_state_record;
[...]
extern in_state_record cur_input;       /* the ``top'' input state */
[...]
#  define  line_catcode_table cur_input.cattable_field

notice "int cattable_field:16;"

textoken.w contains:

#define do_get_cat_code(a) do {                                         \
    if (line_catcode_table!=DEFAULT_CAT_TABLE)                          \
      a=get_cat_code(line_catcode_table,cur_chr);                       \
    else                                                                \
      a=get_cat_code(cat_code_table,cur_chr);                           \
  } while (0)

now, when running

luatex -ini -jobname=luatex -progname=luatex luatex.ini

the function get_cat_code is called several times, and the first two
calls look like:

1) on linux:

(gdb) where
#0  get_cat_code (h=0, n=88) at ../../../texk/web2c/luatexdir/tex/textcodes.w:159
#1  0x0000000000495218 in main_body () at ../../../texk/web2c/luatexdir/tex/mainbody.w:468
#2  0x000000000044ef54 in main (ac=5, av=0x7fffffffe258) at ../../../texk/web2c/luatexdir/luatex.c:461

(gdb) where
#0  get_cat_code (h=0, n=88) at ../../../texk/web2c/luatexdir/tex/textcodes.w:159
#1  0x00000000004a7ea2 in get_next_file () at ../../../texk/web2c/luatexdir/tex/textoken.w:831
#2  0x00000000004aa11c in get_next () at ../../../texk/web2c/luatexdir/tex/textoken.w:1447
#3  0x00000000004aa22e in get_token_lua () at ../../../texk/web2c/luatexdir/tex/textoken.w:1515
#4  0x000000000046d45b in get_x_token () at ../../../texk/web2c/luatexdir/tex/expand.w:379
#5  0x000000000048dbcd in start_input () at ../../../texk/web2c/luatexdir/tex/texfileio.w:843
#6  0x0000000000495221 in main_body () at ../../../texk/web2c/luatexdir/tex/mainbody.w:470
#7  0x000000000044ef54 in main (ac=5, av=0x7fffffffe258) at ../../../texk/web2c/luatexdir/luatex.c:461

2) on solaris:

(dbx) where
=>[1] get_cat_code(h = 0, n = 88), line 159 in "textcodes.w"
  [2] main_body(), line 469 in "mainbody.w"
  [3] main(ac = 5, av = 0xffbef974), line 461 in "luatex.c"

(dbx) where
=>[1] get_cat_code(h = 65535, n = 88), line 159 in "textcodes.w"
  [2] get_next_file(), line 831 in "textoken.w"
  [3] get_next(), line 1447 in "textoken.w"
  [4] get_token_lua(), line 1515 in "textoken.w"
  [5] get_x_token(), line 379 in "expand.w"
  [6] start_input(), line 843 in "texfileio.w"
  [7] main_body(), line 470 in "mainbody.w"
  [8] main(ac = 5, av = 0xffbef974), line 461 in "luatex.c"

please notice the second call, and the value of the first parameter:
on linux, i get h = 0, and on solaris, i get h = 65535.

this is because the condition
  if (line_catcode_table!=DEFAULT_CAT_TABLE)
in the do_get_cat_code expands to false on linux (-1 != -1) and to true
on solaris (65535 != -1)

i tested it by creating a function

int x_get_cat_code(void) {
  int a;
  if (line_catcode_table!=DEFAULT_CAT_TABLE) {
    int i1 = line_catcode_table;
    int i2 = DEFAULT_CAT_TABLE;
    a=get_cat_code(line_catcode_table,cur_chr);
  } else {
    int idx = int_base+cat_code_table_code;
    int h = eqtb[idx].ii.CINT0;
    a=get_cat_code(h,cur_chr);
  }
  return a;
}

and replacing
            do_get_cat_code(cur_cmd);
with
            cur_cmd = x_get_cat_code();
in textoken.w.

then, when debugging, i get:

(dbx) cont
stopped in get_next_file at line 839 in file "textoken.w"
  839       if (iloc <= ilimit) {       /* current line not yet finished */
(dbx) cont
stopped in x_get_cat_code at line 823 in file "textoken.w"
  823     if (line_catcode_table!=DEFAULT_CAT_TABLE) {
(dbx) next
stopped in x_get_cat_code at line 824 in file "textoken.w"
  824       int i1 = line_catcode_table;
(dbx) next
stopped in x_get_cat_code at line 825 in file "textoken.w"
  825       int i2 = DEFAULT_CAT_TABLE;
(dbx) next
stopped in x_get_cat_code at line 826 in file "textoken.w"
  826       a=get_cat_code(line_catcode_table,cur_chr);
(dbx) print i1
i1 = 65535
(dbx) print i2
i2 = -1
(dbx) print cur_input
cur_input = {
    start_field       = 1
    loc_field         = 2
    limit_field       = 3
    name_field        = 0
    ocp_lstack_field  = 0
    synctex_tag_field = 0
    ocp_no_field      = 0
    cattable_field    = -1
    state_field       = 33U
    index_field       = 0
    partial_field     = 0
    nofilter_field    = 0
}
(dbx) quit

as you see, line_catcode_table is treated as 65535 instead of -1,
apparently because of width specifier (:16).

i'm able to get a working luatex when i change
  int cattable_field:16;
to
  short cattable_field;
in inputstack.h.

i'm not sure if the behavior of the compiler is buggy, or the code is
not 100% portable.

please have a look at this, and if you need some more information, or
you'd like to debug yourself on my solaris box, please let me know.

Best,
v.