<html><head></head><body><div class="ydp7e84bdfeyahoo-style-wrap" style="font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-size:13px;"><div></div>
<div dir="ltr" data-setdir="false">Hello Karl et al,</div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">I've moved to texlive-20240312 source tarball and have taken another look. I was wrong about the rounding function itself so started taking a look in xetex for where sum(round()) != round(sum()).</div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">I found a place where I think this effective happens (or can happen) in the function <span>measure_native_node (in XeTeX_ext.c) around line 2100. Below is the code in question and my (very hacky) changes.<br></span></div><div dir="ltr" data-setdir="false"><span><br></span></div><div dir="ltr" data-setdir="false"><span><br></span></div><div dir="ltr" data-setdir="false"><div dir="ltr" data-setdir="false"> uint32_t advances_fixed = 0; // Sum rounded advances<br><br> if (totalGlyphCount > 0) {<br> int i;<br><br> glyph_info = xcalloc(totalGlyphCount, native_glyph_info_size);<br> locations = (FixedPoint*)glyph_info;<br> glyphIDs = (uint16_t*)(locations + totalGlyphCount);<br> glyphAdvances = (Fixed*) xcalloc(totalGlyphCount, sizeof(Fixed));<br> for (i = 0; i < totalGlyphCount; ++i) {<br> glyphIDs[i] = glyphs[i];<br> glyphAdvances[i] = D2Fix(advances[i]);<br> locations[i].x = D2Fix(positions[i].x);<br> locations[i].y = D2Fix(positions[i].y);<br> advances_fixed += D2Fix(advances[i]); // Sum the advances to get the width<br> }<br> width = positions[totalGlyphCount].x;<br> }<br> printf("%d %d\n", advances_fixed, D2Fix(width));<br><br> // node_width(node) = D2Fix(width);<br><br> node_width(node) = advances_fixed; // Use summed advances rather than rounded sum<br><br></div></div><div dir="ltr" data-setdir="false">The printf does show that differences between rounding the final position (positions[totalGlyphCount].x) and sum(glyphAdvances) does occur. With the change your example (see below), using xetex (<div>This is XeTeX, Version 3.141592653-2.6-0.999996 (TeX Live 2024) (preloaded format=xetex)) no longer breaks the boxes.<br></div><div><br></div><div dir="ltr" data-setdir="false">I would be good if somebody can sanity check my logic and confirm (or deny) if this on the right track. If so the BIDI_MIXED case will also need to be reviewed.<br></div><div><br></div><div><br></div><div dir="ltr" data-setdir="false"><div dir="ltr" data-setdir="false"><div>\parindent=0pt<br>\def\test#1{%<br> \setbox0=\hbox{#1}<br> \vbox{\hsize=\wd0 \leftskip0pt plus 1fil #1} % emulate \raggedleft etc.<br> \vbox{\hsize=\wd0 \rightskip0pt plus 1fil #1}<br> \vbox{\hsize=\wd0 \leftskip0pt plus 1fil \rightskip\leftskip #1}<br> \par<br>}<br><br>\font\1="[lmroman10-regular]" at 10pt \1<br>\test{Dest-Addr} % ok<br><br>\font\1="[lmroman10-regular]" at 10.95pt \1<br><br>\test{Dest-Addr} % unwanted line breaks right after hyphen char<br>% none of 10.94pt nor 10.96pt can reproduce the problem<br><br>\font\1="[lmroman10-regular]" at 11pt \1<br>\test{Dest-Addr} % ok<br><br>\bye<br><br></div></div><div dir="ltr" data-setdir="false">Regards and thank you for help,</div><div dir="ltr" data-setdir="false">Ross<br></div><br></div></div><div><br></div>
</div><div id="yahoo_quoted_2108836538" class="yahoo_quoted">
<div style="font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;font-size:13px;color:#26282a;">
<div>
On Monday, 12 February 2024 at 17:04:16 GMT, Karl Berry <karl@freefriends.org> wrote:
</div>
<div><br></div>
<div><br></div>
<div><div dir="ltr">Ross and all - regarding the rounding vs. truncation question in XeTeX<br clear="none">https://sourceforge.net/p/xetex/bugs/185/.<br clear="none"><br clear="none">The answer does not seem so simple to me. The bottom line is that I'd<br clear="none">like to understand specifically why those minutely different values<br clear="none">cause different breaks. I think something is different "downstream" from<br clear="none">the D2Fix computation. But I don't have the time or energy to follow up,<br clear="none">sorry; hope you do, or someone does.<br clear="none"><br clear="none">Details ...<br clear="none"><br clear="none">Agreed that changing XeTeX's D2Fixed to do truncation fixes this<br clear="none">particular line break difference. I could reproduce that.<br clear="none"><br clear="none">However, I don't think the rounding is an error, per se, because the<br clear="none">other engines do rounding. Citing Thanh about pdfTeX:<br clear="none"><br clear="none">thanh> pdftex uses ext_xn_over_d (defined as extxnoverd in utils.c) for<br clear="none">thanh> calculations that involve real numbers. It does rounding, not <br clear="none">thanh> truncating.<br clear="none"><br clear="none">LuaTeX also rounds (just) like XeTeX, at least in some places:<br clear="none">scarso> #define lua_roundnumber(a,b) (int)floor((double)lua_tonumber(a,b)+0.5)<br clear="none"><br clear="none">So just forcing truncation does not feel right. It fixes this particular<br clear="none">xetex+10.95pt case, but who knows what other changes will ensue? We<br clear="none">could add something like a \XeTeXfloatconversion={1,0} switch so that a<br clear="none">document can choose, but ... before we blindly make such a broad change,<br clear="none">I think the underlying question is what happens with the line breaking<br clear="none">after truncation and why the different outcomes are happening in the<br clear="none">first place.<br clear="none"><br clear="none">--<br clear="none"><br clear="none">Additionally: as Nelson pointed out, there is an error in XeTeX's and<br clear="none">LuaTeX's rounding of simply adding 0.5; with negative numbers, that will<br clear="none">not round to the nearest integer (it's necessary to subtract 0.5, not<br clear="none">add, when the argument is negative). pdfTeX does not have this error;<br clear="none">the pdftex code looks like (essentially):<br clear="none"> if (r > DBL_EPSILON) r += 0.5;<br clear="none"> else r -= 0.5;<br clear="none"> ...<br clear="none"> return (scaled) r;<br clear="none"><br clear="none">Unfortunately, when I changed D2Fix to round negative numbers correctly,<br clear="none">it made no difference in the Dest-Addr example. Not surprisingly, since<br clear="none">almost all numbers coming from fonts are positive. So that is not the<br clear="none">answer in practice.<br clear="none"><br clear="none">--<br clear="none"><br clear="none">Additionally: the engine comparisons are problematic.<br clear="none"><br clear="none">1) pdftex is using cmr10.tfm(+pfb) for the font. With traditional TeX<br clear="none">line breaking, floating point is not involved at all.<br clear="none"><br clear="none">2) With XeTeX, on the other hand, the test document is reading<br clear="none">lmroman10-regular.otf, a completely different font rendered in a<br clear="none">completely different way. One immediate question is whether the metrics<br clear="none">of lmroman10-regular.otf match *exactly* the metrics of cmr10.tfm.<br clear="none">And whether otf vs. tfm makes a difference wrt hyphenation.<br clear="none"><br clear="none">3) With LuaTeX, lmroman10-regular.otf is also being read, so XeTeX and<br clear="none">LuaTeX at least have that basis of comparison. However, there are many<br clear="none">differences in how luatex and xetex operate. E.g., for one thing,<br clear="none">LuaTeX gets the "bad" Dest-<linebreak>Addr output with many font sizes,<br clear="none">I tried from 10.90 to 10.99 and they all broke after the hyphen, unlike<br clear="none">XeTeX where (as you noted) it has to be 10.95 exactly.<br clear="none"><br clear="none">3b) With LuaTeX, in principle there is also the question of whether the<br clear="none">"native" rendering is used (luatex) or harfbuzz (luahbtex). However, I<br clear="none">found no difference in the test document either way.<br clear="none"><br clear="none">Another possible difference is that XeTeX might "snap" to the limited<br clear="none">number of heights/depths/widths available in tfm format, even when using<br clear="none">otf? Not sure. LuaTeX does not do this.<br clear="none"><br clear="none">Although I can't find it now, I seem to recall that Ulrike(?) wrote a<br clear="none">TUGboat article about differences between LuaTeX and XeTeX, e.g., LuaTeX<br clear="none">computing a lot more in floating point before converting to fixed, and<br clear="none">plenty of computations happen in Lua.<br clear="none"><br clear="none"><br clear="none">Finally, in case anyone does want to take this further: I tweaked the<br clear="none">test document to run under -ini, since it's much easier to experiment<br clear="none">with engine changes without having to bother building a .fmt. Also added<br clear="none">a conditional so it would run with both xetex and luatex. Appended below<br clear="none">and also posted to the xetex bug report.<br clear="none"><br clear="none">Thanks,<br clear="none">Karl<br clear="none"><br clear="none"><br clear="none">%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%<br clear="none">% <a shape="rect" href="https://sourceforge.net/p/xetex/bugs/185/" target="_blank">https://sourceforge.net/p/xetex/bugs/185/</a><br clear="none"><br clear="none">% ini doesn't work without setting more line breaking params.<br clear="none">\catcode`\{=1 \catcode`\}=2 \catcode`\#=6 <br clear="none">\hsize=6.5in \vsize=9in<br clear="none">\parfillskip=0pt plus1fil<br clear="none">\hyphenpenalty=50<br clear="none">\exhyphenpenalty=50<br clear="none">\doublehyphendemerits=10000<br clear="none">\finalhyphendemerits=5000<br clear="none">\defaulthyphenchar=`\-<br clear="none">\lefthyphenmin=2 \righthyphenmin=3 % disallow x- or -xx breaks<br clear="none"><br clear="none">\catcode`@=11<br clear="none">\chardef\@ne=1<br clear="none">\chardef\tw@=2<br clear="none"><br clear="none">\def\loggingall{\tracingcommands\tw@\tracingstats\tw@<br clear="none"> \tracingpages\@ne\tracingoutput\@ne\tracinglostchars\@ne<br clear="none"> \tracingmacros\tw@\tracingparagraphs\@ne\tracingrestores\@ne}<br clear="none">% maxdimen errors \showboxbreadth\maxdimen\showboxdepth\maxdimen}<br clear="none"><br clear="none">\def\test#1{%<br clear="none"> \setbox0=\hbox{#1}<br clear="none"> \vbox{\hsize=\wd0 \leftskip0pt plus 1fil #1} % emulate \raggedleft etc.<br clear="none"> \vskip5pt<br clear="none"> \vbox{\hsize=\wd0 \rightskip0pt plus 1fil #1}<br clear="none"> \vskip5pt<br clear="none"> \vbox{\hsize=\wd0 \leftskip0pt plus 1fil \rightskip\leftskip #1}<br clear="none">}<br clear="none"><br clear="none">%\font\1="[lmroman10-regular]" at 10pt \1<br clear="none">%\test{Dest-Addr} % ok<br clear="none"><br clear="none">\ifx\directlua\undefined<br clear="none"> \font\1="[lmroman10-regular.otf]"<br clear="none">\else<br clear="none"> \let\dump\relax \input luatex.ini % so we can:<br clear="none"> \input luaotfload.sty<br clear="none"> \font\1="[lmroman10-regular.otf]"<br clear="none">\fi<br clear="none"> at 10.90pt<br clear="none">\loggingall<div class="yqt6497268031" id="yqtfd47231"><br clear="none">\1</div><br clear="none">\test{Dest-Addr} % unwanted line breaks right after hyphen char<br clear="none">% none of 10.94pt nor 10.96pt can reproduce the problem with xetex.<br clear="none"><br clear="none">%\font\1="[lmroman10-regular]" at 11pt \1<br clear="none">%\test{Dest-Addr} % ok<br clear="none"><br clear="none">\end<div class="yqt6497268031" id="yqtfd60131"><br clear="none"></div></div></div>
</div>
</div></body></html>