[XeTeX] misaligned math if \smash used in subscripts

[David Carlisle]

The following first reported at the unicode-math github
https://github.com/wspr/unicode-math/issues/383
but the problem can be seem with plain tex.


In the tex below there are six repeated instances of a subscripted bracket
they should all be the same but the first one is misaligned.

It appears to be a misbehaviour of \mathchoice as if I change the size of the
scriptscript font the effect changes, even though there is no visible
script script text.

The attached images show the output with the file as below and with
the scriptscript font changed to

at 25pt

I've tried it with fam 0,1,2 all set up as the same font, or as here
with just fam0
using a opentype font.

David


\font\m= "[latinmodern-math.otf]/OT:script=math;language=DFLT;" at 10pt
\font\sm= "[latinmodern-math.otf]/OT:script=math;language=DFLT;" at 7pt
\font\ssm= "[latinmodern-math.otf]/OT:script=math;language=DFLT;" at 5pt

\textfont0=\m
\scriptfont0=\sm
\scriptscriptfont0=\ssm


\Umathcode`.= 0 0 `.

\hrule
$$
\hbox{$\left)\vbox to15pt{}\right.$}_{\smash.}
\hbox{$\left)\vbox to15pt{}\right.$}_{\smash.}
\hbox{$\left)\vbox to15pt{}\right.$}_{\smash.}
$$
\hrule
$$
\hbox{$\left)\vbox to15pt{}\right.$}_{\smash.}
\hbox{$\left)\vbox to15pt{}\right.$}_{\smash.}
\hbox{$\left)\vbox to15pt{}\right.$}_{\smash.}
$$
\hrule


\bye
-------------- next part --------------
A non-text attachment was scrubbed...
Name: b1.png
Type: image/png
Size: 5751 bytes
Desc: not available
URL: <http://tug.org/pipermail/xetex/attachments/20170214/9bac4d41/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: b2.png
Type: image/png
Size: 4666 bytes
Desc: not available
URL: <http://tug.org/pipermail/xetex/attachments/20170214/9bac4d41/attachment-0003.png>