[luatex] node.rangedimensions on vertical lists

[Esger Renkema]

Dear list,

The function node.rangedimensions(), when applied to vertical lists, 
includes contributions of (vertical) glue nodes in the (horizontal) 
width. A simple example:

-------------------------------------------------------------
\setbox0 = \vbox{
     \vskip 10sp
     \hrule width 1sp height 2sp depth 3sp }
\directlua{
     texio.write_nl (string.format(
         '\csstring\%s, \csstring\%s, \csstring\%s',
         node.rangedimensions(tex.box[0], tex.box[0].head))) }
-------------------------------------------------------------

-------------------------------------------------------------
This is LuaTeX, Version 1.12.0 (TeX Live 2020/Debian)
  restricted system commands enabled.
(./bug.tex
11, 2, 3)
warning  (pdf backend): no pages of output.
Transcript written on bug.log.
-------------------------------------------------------------

where the fourth line should be 1, 12, 3.

The cause of this problem seems to be the function natural_sizes() 
defined in packaging.c; it looks like it was written with only 
horizontal lists in mind.

This assumption also causes trouble with horizontal rules:

-------------------------------------------------------------
\setbox0 = \vbox{
     \hrule height 1sp width 2sp
     \hrule height 1sp }
\setbox1 = \vbox{
     \hrule height 1sp width 2sp
     \hrule height 1sp width 2sp }
\directlua{
     texio.write_nl (string.format(
         '\csstring\%s, \csstring\%s, \csstring\%s',
         node.rangedimensions(tex.box[0], tex.box[0].head)))
     texio.write_nl (string.format(
         '\csstring\%s, \csstring\%s, \csstring\%s',
         node.rangedimensions(tex.box[1], tex.box[1].head))) }
-------------------------------------------------------------

-------------------------------------------------------------
This is LuaTeX, Version 1.12.0 (TeX Live 2020/Debian)
  restricted system commands enabled.
(./bug.tex
-1073741822, 1, 0
4, 1, 0)
warning  (pdf backend): no pages of output.
Transcript written on bug.log.
-------------------------------------------------------------

where the reported numbers should in both cases be 2, 2, 0.

Regards,
Esger Renkema