[luatex] Hyphenation in plain TeX

[Mojca Miklavec]

On Fri, Oct 8, 2010 at 11:04, Robin Fairbairns wrote:
> Mojca Miklavec wrote:
>
>> This should be fixed in format one day though, but if you want to use
>> it now, this is what you need to do ...
>
> what should be fixed in the format?  lccode tables for all unicode
> ranges covering alphabetic characters?

Exactly.

> seems extreme:

Well, it's done by:
- plain XeTeX & XeLaTeX
  /usr/local/texlive/2010/texmf-dist/tex/generic/xetexconfig/unicode-letters.tex
- ConTeXt when using XeTeX
  /usr/local/texlive/2010/texmf-dist/tex/context/base/xetx-utf.mkii
- ConTeXt MKIV
  /usr/local/texlive/2010/texmf-dist/tex/context/base/char-def.lua

I don't know why it would be any more extreme to do it in plain LuaTeX format.

> i would think it
> more sensible to provide the tables in hyphenation files

Btw, there is
    /usr/local/texlive/2010/texmf-dist/tex/generic/hyph-utf8/patterns/txt/hyph-ru.chr.txt
but it is not used anywhere.

> that way
> there would be some chance that the tables were constributed by people
> who know the language.

Now imagine user A using both German and French patterns. When he
switches from German to French - should we switch all the German
lccodes for äöü back to undefined? If we don't, there's a chance that
the user will get different hyphenation for French (let's say that the
text contains a name with some German letters) depending on whether he
was typesetting a French-only or mixed language document.

Now imagine user B using some package that deals with Unicode and sets
all the lccodes for the whole Unicode range. If we do change the
lccodes back to undefined when switching from German to French, the
user will have all the letters for the whole Unicode range properly
defined, except for some umlauts which we modified without user's
consent.

I simply don't think that changing any codes inside hyphenation files
is the right thing to do. Yes, we can solve many users' problems
(simply hyphenation will work), but create many ugly new ones.

Mojca