[lltx] Re: A new Babel?

[François Charette]

Thanks Elie for this post. As the author of polyglossia I will add a
few notices below from my perspective, and will leave the lualatex
aspects of the discussions to others.

On Jan 14, 10:37 am, Élie Roux <elie.r... at telecom-bretagne.eu> wrote:
> Hello,
>
> recently I got interested in polyglossia and had interesting
> discussions with François Charette. The reasons I got interested in
> polyglossia were:
>  - having (the very romanian) ș instead of (the very turkish) ş in
> romanian titles, which is not possible in standard 8-bit TeX (and I
> don't really like tricky TeX patches)
>  - having a modern Unicode+OTF solution without all the 8-bit+T1 heavy machinery
>
> The discussions I had with François were on these topics:
>  - is it possible to adapt polyglossia to LuaTeX? currently it uses
> things like \XeTeXintercharclass that are not accessible in LuaTeX.
> This means we have two choices: (a) adapt \XeTeXintercharclass and
> stuff to LuaTeX, which would be quite funny, (b) copy-paste bunches of
> code from Babel with active characters.
>  - why isn't it possible in polyglossia to have some features from
> babel (especially in french) like modification of the shape of the
> lists, of the indentation, etc. that are really mandatory for me
> (english typography is so awful when you're used to french one). The
> answer is: "it's on the TODO list"

Just a side-note: when I initiated polyglossia, I had expected that
the "xelatex community" would actually develop and improve the various
language modules.. I have received a number of corrections for various
languages over the year, but funnily, none at all for French and very
few for German, two languages with a large user base. If someone had
nagged me to improve the french module, I would certainly have devoted
more time to it :) Overall the contribution from the community has
been rather limited (with the exception of some important bugfixes and
stylistic improvements sent by Enrico Gregorio, the author of the
Italian module for Babel). One way to improve this situation, perhaps,
would be to create a more abstract user-interface for setting up the
most common bells and whistles that are found in most gloss-*.ldf
files (i.e. hyphenation pattern + hyphenmins, name of language, script
and direction, captions and date, settings for frenchindent and
frenchspacing, etc. Hypotherical example:

\PolyglossiaSetup{<lang>}{%
   Script=Latin, %default=Latin
   Direction=LTR, %default=LTR
%   LanguageString=<lang>, %default=<lang>
%   Aliases={Foo,Bar},
   hyphenmins={x,y} %default={2,2} ?  {2,3} in babel...
   frenchspacing=true, %default=false
   frenchindent=true %default=false
   }

That would make it much easier for average users to customize
polyglossia...

> I'd like to know what you think about a new babel. First you can read
> the discussions on the XeTeX list that lead to the birth of
> polyglossia:http://www.tug.org/pipermail/xetex/2007-April/thread.html
> in the thread "[XeTeX] "new-babel", was: Ancient Greek hyphenation".
> The README is also useful to see the functionalities:http://mirror.unl.edu/ctan/macros/xetex/latex/polyglossia/README.
> These readings are really interesting and hard to sum up, but the main
> open points are:
>
>  (1) should it set a font for each language or just set the language
> option in the current font:
>     - for french and german, the same font can obviously be used
>     - for arabic and japanese obviously not, so for close languages it
> shoudn't be changed, when it should be for far languages... but what
> is the limit... should we make groups of languages?
>     - for german and polish the ligatures are not the same even though
> the languages are close, do all the fonts change the ligatures
> according to the language option?
>     - I must say I have no preference for this... except that we
> should allow the user to do whatever he want by providing the good
> macros.

FWIW, my approach has been to assume that, when switching to a new
language, the same font is kept, unless \<lang>font or \<script>font
has been explicitly defined.

>
> (2) should it include typographic rules or should they be in another
> package? in a different file? For me it should as an option, activated
> by default.
>
> (3) Is all the babel machinery included in Unicode+OTF? Do we still
> need active characters and things like that? I think we still need
> some active characters for some languages, or to implement the \XeTeX
> machinery... but I think the babel active characters have been more
> tested and work in more cases.

Yes, active characters are needed in German, Dutch and some other
languages to fine-tune various aspects of hyphenation, etc. See page 8
of the documentation (latest version of polyglossia.pdf).

I have recently imported the relevant code for active characters from
babel into polyglossia:
http://github.com/fc7/polyglossia/blob/master/tex/babelsh.def
It is currently only used for German, but they should also become
available for Dutch (hopefully in the next version I shall upload to
CTAN).

> (4 - not discussed on the XeTeX list) how far should it be from babel?
> I was thinking about loading the babel files and then overriding the
> translated strings to Unicode, so that we have the same result as
> babel... This is a point that would need further inquiries for me. But
> we shouldn't forget that some language files change very often
> (frenchb for example, which often breaks backward compatibility), and
> maybe we should be aligned with babel to confuse less users...

Following the brain-storming we had on the xetex list in 2007, I
decided against this and chose to use antomega as a starting point for
polyglossia, and later added a bunch of additional modules based on
the ones from Babel. The problem, obviously, is that those have been
updated since then, and it's hard to keep in sync. One solution would
be to have common files for use by babel, polyglossia (and potentially
other packages), in which things such as caption strings are set. And
these should of course include both the Babel representation and the
UTF-8 strings side-by-side. I am not sure the Babel people would
welcome this idea with joy, but it is certainly worth a try :)

> Even though Will has already participated in the XeTeX discussions,
> I'd like to have your opinions on these topics. The main reason is
> that François gives up polyglossia and proposed me to take the
> leadership on the package, allowing me to change a lot of things... If
> we manage to agree on something I'll move the discussion to a wider
> range, including the babel team, and the luatex and xetex
> mailing-lists.

As I wrote to Elie, I am no longer able to properly maintain
polyglossia, and I am very happy he has accepted to take it over!

Best regards to all,
FC