[lltx] Fwd: LuaLaTeX for TeXLive 2013

[Élie Roux]

Dear all,

Here is a copy of a mail I just sent by mistake to 
lualatex-dev at googlegroups.com instead of this address. If you answer it, 
can you please send your reply also to other people outside this group?

Thank you,
--
Elie

-------- Original Message --------
Subject: LuaLaTeX for TeXLive 2013
Date: Thu, 11 Apr 2013 19:29:46 +0200
From: Élie Roux <elie.roux at telecom-bretagne.eu>
To: lualatex-dev at googlegroups.com,  Philipp Gesang 
<philipp.gesang at alumni.uni-heidelberg.de>, George Nurser 
<gnurser at gmail.com>, وفا خلیقی، Vafa Khalighi <vafa at live.com.au>, 
Khaled Hosny <khaledhosny at eglug.org>, Paul Isambert 
<zappathustra at free.fr>,  Mojca Miklavec <mojca.miklavec.lists at gmail.com>

Dear all,

After some years of pause in my TeX activities, I'm considering to come
back to it, especially since LuaTeX 0.75 breaks the LuaLaTeX packages
and I need them to work for another project.

As I hardly followed the lists, I don't really know who is involved in
all this, so I'm writing to lualatex-dev at googlegroups.com and to other
people I know are interested and involved. There are certainly more
people outside, please consider forwarding this mail to them. I didn't
manage to find the administration interface of the googlegroup so I do
not know who subscribed, sorry if some of you receive this email twice
or if I forgot someone.

I took the initiative, after some discussions, to start working on this,
with the great and precious help of Philipp Gesang without who the
update of luaotfload would certainly not be possible.

The purpose of this quite long mail is to discuss, inform, take advices
and critics, and welcome help on ideas, development and testing.

Here is the current state :
  - luaotfload v1 [1] and v2 [2] break with LuaTeX་ 0.75
  - luatexbase [3] breaks with LuaTeX 0.75
  - fontspec [4] and unicode-math [5] break with luaotfload v2
  - lualibs [6] seems ok, but it might be a good idea to update it

The following ideas have appeared and are started, do not hesitate to
comment, on the ideas or their implementations:
  1. adapt luaotfload v1 and v2 to LuaTeX 0.75
  2. adapt luatexbase to LuaTeX 0.75
  3. make a luaotfload v3 (or v2.1) with code updated from the ConTeXt
     code that will ship in TeXLive 2013
  4. update lualibs with the same ConTeXt code, keeping backward
     compatibility by adding a backward compatibility file
  5. make fontspec and unicode-math compatible with luaotfload v2 and v3
     (keeping backward compatibility)
  6. make a more developped lualibs with other libraries

Different developments on these ideas:

  1. updates of luaotfload v1 and v2 for compatibility with LuaTeX 0.75
     are available on [7] and [8], to be tested further. It also fixes
     some small issues. Note that this just adapts the old code for
     compatibility with new LuaTeX, it does not really update the code.

  2. update of luatexbase is available on [9], to be tested further.
     I think what I did is not really what should be done and I'll update
     it, but it's quite a small update...

  3. Phlipp Gesang is working on luaotfload v3, in collaboration with
     Hans Hagen on [10], and has an update of fontspec on [11]. The new
     luaotfload will certainly use a copy of luatex-fonts.lua, an
     integrated file made by Hans in order to make his fontloader code
     useable outside ConTeXt. The code is from the beta currently in
     TeXLive 2013.

  4. Philipp will work on the update of lualibs when the font code will
     be stable.

  5. For fontspec adaptation, most of the work had already been done by
     Khaled, I just merged it with the current master and made it
     backward compatible with luaotfload v1, it's on [12]. For
     unicode-math, the adaptation is quite simple and is one [13]. I'm
     not sure it works with luaotfload v3.

  6. Philipp had a very interesting idea: it was to rename the current
     lualibs into lualibs-essentials and to make a lualibs-extended with
     goodies from ConTeXt like introspection, instant serialization, a
     json parser, converters for dimensions, etc. I personally think it
     is a very good idea, but I would like to have your opinion on this.

The main questions are: what do you think about all this? Would you have
time to test or contribute? How do you think we should proceed?

Here is a proposal, but I'm not really sure about it, so do not hesitate
to criticize it:

  - first we could update luaotfload v1 and luatexbase on CTAN, for
    people testing TeXLive 2013, as these are really mandatory for
    LuaTeX 0.75. Once I'll have it finished, I'll ask Manuel to
    review and upload on CTAN. Manuel, does it seem ok?

  - then update fontspec and unicode-math on CTAN too, so that people
    can test the new luaotfload easily. This would require some further
    testing and synchronization between [11] and [12], but it can be
    quite fast. Once it will be done, we would have already something
    useable and a little updated in TeXLive! Will, what do you think?

  - the most interesting but difficult part: test luaotfload v3
    extensively, against many uses (complex scripts, maths, etc.), when
    Philipp will consider his code stable enough. This will require tests
    by people with experience in the domain... I was, of course,
    thinking about Khaled. Khaled, I know you don't have time to be
    involved in all this, but would you have some very hard documents or
    test files on arabic or math? Of course, everyone is kindly
    encouraged to test with all possible documents!

Thank you all for everything,
--
Elie

[1] https://github.com/khaledhosny/luaotfload/tree/texlive2011 (it is
     the version currently on CTAN and in TeXLive 2012)
[2] https://github.com/khaledhosny/luaotfload/
[3] https://github.com/mpg/luatexbase/
[4] https://github.com/wspr/fontspec/
[5] https://github.com/wspr/unicode-math
[6] https://github.com/mpg/lualibs
[7] https://github.com/eroux/luaotfload/
[8] https://github.com/eroux/luaotfload/tree/texlive2011
[9] https://github.com/eroux/luatexbase
[10] https://github.com/phi-gamma/luaotfload/tree/experimental
[11] https://github.com/phi-gamma/fontspec/tree/experimental
[12] https://github.com/eroux/fontspec/tree/otfl-unstable
[13] https://github.com/eroux/unicode-math