[XeTeX] Line-breaking algorithms in XeTeX

[Jonathan Kew]

On 26 Apr 2009, at 14:57, John Was wrote:

> Dear All
>
> Since starting to use (plain) XeTeX I've noticed something strange  
> with the paragraphing/line-breaking mechanism which has never  
> happened during the ten years or so during which I have used  
> traditional TeX.  It is cropping up in the fourth issue of a  
> periodical that I have set with XeTeX, so I'm pretty sure that it's  
> not a random fluke.
>
> (1) I sometimes get an overfull rule (i.e. rectangular box) at the  
> right-hand side which will disappear when I either (a) attach the  
> word causing the problem to the next word with ~, forcing it over (I  
> sometimes have to put the word in an \hbox{} as well); or (b) when I  
> increase the line-count by giving \looseness1 for the paragraph.  In  
> the past, plain TeX would always make such decisions for itself and  
> never generate an overfull rule when it could find a way to justify  
> the paragraph without doing so.  This happens most frequently in the  
> reviews section of the periodical, where  \looseness is set to -1 by  
> default to save as much space as possible:  but until I started to  
> use XeTeX, it was always the case that if the paragraph could not  
> lose a line, then the negative looseness was ignored and the  
> paragraph was set successfully with normal looseness  (i.e.  
> \looseness = 0).  It was never (I think) the case that a tight  
> looseness which generated an overfull box would get through and need  
> manual intervention from me.  So has something altered in the way  
> XeTeX is handling the line-breaks, giving priority to the looseness  
> command even at the expense of generating an overfull rule, and even  
> when zero looseness would cause that error to disappear?
>
> (2) This is even more puzzling (and more of an nuisance).  For the  
> purpose of sending contributors proofs of their reviews I start each  
> review on a new page so that they don't also receive the tops and  
> tails of adjacent reviews, but while initially typesetting I have  
> the reviews running on consecutively, as they will do in the final  
> published version.  There is a switch at the end of each review  
> which generates a \vfill \eject when \ifseparatereviews is true,  
> otherwise it just produces a \vskip: there is no other difference.   
> Yet I sometimes get overfull rules showing up (at random points)  
> when the reviews are separated out, even though the same paragraph  
> typeset without error while the reviews were set to run on  
> continuously.  The problem almost (but not entirely) disappears if I  
> double the \hfuzz when the \ifseparatereviews switch is true, but  
> that is no more than a quick fix to prevent authors receiving proofs  
> with worrying blobs at the right-hand side.  This seems  
> incomprehensible, but as it has happened with four out of four  
> periodical issues I can't be imagining it - and the commands are  
> precisely the same as the ones I used when the periodical was  
> typeset using traditional plain TeX, with no new parameters such as  
> alteration to \spaceskip or anything else that might cause this to  
> happen.
>
> (1) and (2) seem likely to be part of the same problem (though not  
> necessarily so).  Any ideas, or at least insight into what XeTeX is  
> doing that old plain TeX didn't?

No obvious ideas come to mind; any chance you could come up with a  
small test-case that illustrates the problem(s), and doesn't require  
proprietary fonts, etc.?

JK