[tex-hyphen] German patterns in TL2010 (was: How and where to generate language.dat.lua?)

[Manuel Pégourié-Gonnard]

Le 24/05/2010 18:33, Stephan Hennig a écrit :
> [
>    * starting a new thread,
>    * summing up the current state of German patterns,
>    * long
> ]
> 
> Am 03.05.2010 20:06, schrieb Manuel Pégourié-Gonnard:
>> Le 03/05/2010 11:08, Mojca Miklavec a écrit :
>>
>>>> The down side is, when german-x is updated, hyph-utf8 needs to
>>>> be updated too
>>>
>>> Whet german-x is updated, they'll probably want to update patterns
>>> in hyph-utf8 anyway.
>>>
>> The actual patterns are in hyph-utf8? I was under the impression
>> that dehyph-exptl was a separate package, both on CTAN and in TL.
> 
> Everything said is correct.
> 
Ok. Now that I understand the situation a bit better (or so I hope), back to
Mojca's comment:

>>>> The down side is, when german-x is updated, hyph-utf8 needs to
>>>> be updated too

But it's already the case, isn't it? I mean, it has nothing to do with the new
luatex loading system?

>    * Werner has converted the patterns to UTF-8.  He also provided a
>      .tex pattern wrapper, that converts patterns back into T1
>      encoding, if an 8-bit engines is recognized.  8-bit TeX engines
>      require the dehyph-exptl package with their time-stamped patterns
>      to be installed for this.  This package is already part of TL2009
>      and the patterns are enabled in language.dat, by default, making
>      them available via languages 'german-x-2009-06-19' etc.
> 
>    * XeTeX loads the unmodified UTF-8 patterns provided by hyph-utf8,
>      which are the same as in dehyph-exptl, but uses its own pattern
>      wrapper, though.  As long as the patterns for XeTeX and LuaTeX
>      aren't frozen, this means hyph-utf8 is updated whenever we provide
>      new patterns.
> 
>    * As for LuaTeX, I have no idea what patterns LuaTeX from TL2009
>      actually loads.
> 
Thanks for this very clear explanation.

> For TeX Live 2010, I hope we can agree on the following goals:
> 
>    * 8-bit TeX
>        No change (ever).  Load traditional patterns in the format.
>        Experimental patterns are provided by package dehyph-exptl.
> 
Agreed.

>    * XeTeX
>        No change.  Load experimental patterns, by default.  Make them
>        available as traditional languages 'german' and 'ngerman'.
> 
Agreed. Though, since the language.dat file is shared by all engines, they will
also be available as german-x-latest and ngerman-x-latest (without the languages
being properly defined as synonyms, they will end up loading the same patterns).

>    * LuaTeX:
>        Load experimental patterns from new language.dat.lua, by default.
>        Make them available as traditional languages 'german' and
>        'ngerman'.
> 
We can easily make the situation the same as for XeTeX, which is the best
solution now IMO. In the entries for (n)german-x, I can just add txthyph nd
txtpatt fields, so that the plain text version from hyph-utf8 is used. That way,
german-x-latest will be available under the three engines and mean the same thing.

Or...

>      The question is, whether there should be an entry for languages
>      'german-x-<date>' etc.  I'd say no and I'll emphasize in our
>      documentation, that package dehyph-exptl is not required for
>      LuaTeX (and XeTeX).
> 
we can desactivate (n)german-x-* in LuaTeX. (Note that in the current state of
things, we're not desactivating it for XeTeX.)

Well, your choice. Here, as a co-author of luatex-hyphen, and as a TL developer,
my goal is only to provide solutions (and possibly advice about how they can be
used), but the decision is yours.

>      The only thing for us to do, is to remember to provide patterns as
>      text versions, too, in future releases.  Did I miss something?
> 
I don't think you missed anything.

> I'm sorry for the confusion about the state of German patterns.  There
> must have been some.  At least, I have learnt much about pattern loading
> during the last weeks.  Comments and corrections are welcome (hence the
> lengthy mail)!
> 
Thanks very much for taking the time to explain the situation. It's much more
clearer now.

Manuel.