[kadingira] Fwd: Babel status

[Deepak Jois]

> Broadly agree, and if we could get HarfBuzz properly into LuaTeX I'd go
> with it. The fact Hans wants to write a separate font shaper is a real
> issue.
>From the discussions on LuaTeX mailing lists (see [1] for example), I
guess Hans is of the opinion that concepts like font shapers and BiDi
belong outside the core LuaTeX engine. The LuaTeX engine could support
the design of such packages by exposing the underlying bits (by means
of callbacks etc.), but LuaTeX will be agnostic to different ways to
build these things.

Currently, there is font shaping code that exists in ConTeXt, that is
extracted regularly into luaotfload. Khaled Hosny also wrote
context-bidi[2] a long time ago. But both these solutions are
inadequate. luaotfload has a lot of bugs, and last I checked it was
pretty bad for typesetting the Arabic script (see [3] for example).
context-bidi isn’t really maintained and Khaled got disillusioned with
the TeX ecosystem and moved away from any substantive TeX development
a long time ago. I remember he was a pretty passionate advocate of
getting Harfbuzz into LuaTeX or ConTeXt (I can’t remember which).

A few months ago, I was looking for good open source tools for
non-Latin typesetting (my specific interest was in Indic and Arabic
texts), so I started investigating LuaTeX. I have so far written:

* luaharfbuzz[4] - Lua bindings for Harfbuzz

* luaucdn[5] - Lua bindings for the UCDN C library, which provides
basic Unicode support

* luabidi[6] - A pure Lua port of the Unicode BiDi algorithm reference
implementation in Java. I just started writing this package, and it is
not complete.

The idea was to develop them as independent Lua packages (hosted on
LuaRocks), so that they have wider applicability beyond the TeX
ecosystem as well. There have been a few experiments integrating
LuaTeX with luaharfbuzz. There is Michal Hoftich’s
luatex-harfbuzz-shaper[7]. There is also some sample code[8] in the
luaharfbuzz repo, although it is not very straightforward to run it.

I hope the summary above is useful to people interested in these
issues. Please do correct me if the above information is not accurate.
I  also hope somebody can use the packages above and bring some
quality font shaping and BiDi support to LuaTeX. I will keep
contributing to the ecosystem when I have time, although I expect
things to move slowly. Once I am done with luabidi, I hope to write a
Lua based font handling library (an equivalent of the fontloader
module in LuaTeX), as a complement to luaharfbuzz. Someday after that,
I might try to write my own typesetting system :), if things in TeX
land haven’t improved.

Regards
Deepak


[1]: http://tug.org/pipermail/luatex/2011-July/003147.html
[2]: https://github.com/khaledhosny/context-bidi
[3]: https://github.com/lualatex/luaotfload/issues/292
[4]: https://github.com/deepakjois/luaharfbuzz
[5]: https://github.com/deepakjois/luaucdn
[6]: https://github.com/deepakjois/luabidi
[7]: https://github.com/michal-h21/luatex-harfbuzz-shaper
[8]: https://github.com/deepakjois/luaharfbuzz/tree/master/examples/luatex