[texhax] overfull lines, hyphenation - setup for automatic processing

[Uwe Lück]

At 15:30 06.07.09, james mourton wrote:
>I'd like to generate documentation from various XML sources
>and create a PDF output using TeX (preferably ConTeXt, possibly LaTeX).
>I need the system to run automatically, without user interaction.
[...]
>Could you please advise, how to tell TeX not to be so fussy
>and rather work reliably -- no overfulls or other errors, even if the result
>is not "perfect"? (the "manuscript" is generated, so I don't care
>how complicated or unreadable it is)

At 12:33 07.07.09, Donald Arseneau wrote:
>In LaTeX, declare \sloppy.  The opposite is -- you guessed it -- \fussy.

I guess this is the essential answer to the original posting. Answering it 
seems to need much guessing indeed. I guess the misconceptions of TeX's 
hyphenation are irrelevant, and if the original poster sees the result of 
\sloppy, he will be happy and no longer insist on his hyphenation 
suggestions. The only remaining question is that of \bTABLE, I am unable to 
comment it. Regarding \sloppy, a ConTeXt user may need the definition of 
\sloppy (I don't know whether ConTeXt has something similar):

     \def\sloppy{%
       \tolerance 9999%
       \emergencystretch 3em%
       \hfuzz .5\p@
       \vfuzz\hfuzz}

I think I have never seen an overful \hbox with \sloppy -- until I now tried

     mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm

I wonder whether such cases are relevant.

The rest is mere theory.

At 15:30 06.07.09, james mourton wrote:
>By default, if TeX does not know how to hyphenate words and break lines
>nicely, it leaves it to the user.

This was criticized, but it is correct in that the user can force a 
hyphenation that TeX would not do by its hyphenation algorithm. The wording 
however, creates an impression that there were something like a TeX message 
`I am unable to hyphenate this nicely, please do it for me!' This idea 
resembles the idea below that if one kind of hyphenating fails, TeX might 
try another; special case here: ask the user for doing the hyphenation. The 
user may force a hyphenation that TeX will not perform, and indeed 
sometimes TeX does not hyhenate as wanted; it is just strange to say `TeX 
does not know how to ...' then. Indeed TeX may not know that certain 
hyphenations are wanted. Real life examples happen almost each paragraph in 
German with umlauts and OT1 encoding.

At 15:30 06.07.09, james mourton wrote:
>1) Making larger gaps
>How can I set it up, shall I redefine
>\tolerance and \emergencystretch ? To what values? What is the scope of 
>the settings?
>Is there any other important option?

Indeed, the answer `\sloppy' seems to answer the previous sufficiently.

>2) Hyphenation
>Is there any way how to fall-back to more aggressive hyphenation rules
>if the default algorithm fails?
>If the word is unbreakable, could TeX just tear off the extra letters it 
>and put them to the next line?
>(I could theoretically put \discretionary to every word, but I'd prefer 
>TeX to use its own algorithms first)

The idea here resembles the fact that TeX first tries linebreaking without 
hyphenation and leaves it this way depending on \pretolerance. But there is 
no built-in mechanism to try another linebreaking with different 
(hyphenation) settings if the first try was "too bad". This may perhaps be 
implemented, however, as a macro storing the code for a paragraph in order 
to typeset it under different settings and somehow choose among them then. 
But I would need some hours for this and won't try.

A "more aggressive" hyphenation could be implemented by defining another 
language and associate it with a hyphenation pattern list that allows all 
hyphenations of the first language, but adds some "arbitrary" hyphenations. 
Then you could switch to the second language for another try at 
linebreaking according to the idea above. But who will generate such a 
pattern list, and is it worth it? I am not inclined to try this and to 
improve my knowledge of the matter for it.

"More aggressive" hyphenation may as well include (varying Pierre MacKay's 
comment) \lefthyphenmin=1, \righthyphenmin=1 and \hyphenpenalty=0.

>3) Error reporting
>Another issue is, that texexec
>creates very ugly table (\bTABLE) with overlapping text in the cells,
>but does not report any problem.
>Is there any way how to make it fail loudly, so the problems can
>be detected?

Sorry, I don't know about texexec and \bTABLE ...

-- Uwe Lueck.