[luatex] About the level of bidi implementation

[Idris Samawi Hamid ادريس سماوي ح]

Hi Yannis,

On Fri, 10 Apr 2009 10:17:40 -0600, Yannis Haralambous  
<yannis.haralambous at telecom-bretagne.eu> wrote:

> Le 10 avr. 09 à 15:02, Idris Samawi Hamid ادريس سماوي
> حامد a écrit :
>
>> There is a difference of philosophy here: It is not the job of the
>> engine to hardwire the bidi algorithm or any other implementation.
>> This is the job of the macropackage. Then one has full control
>> without having to fiddle with the engine.
>
> I do not agree. Bidi is something fundamental, just like contextual
> analysis. It should be treated on a low level.

> **But** it should be deactivetable: by default bidi should be active,
> but the user should be able to deactivate it.
>
> This will make the difference with XeTeX. XeTeX uses Uniscribe, which
> is a black box leaving no freedom whatsoever to the user. This is
> fine, except when we want to do something different. For example, for
> a very specific application (2D barcodes) I need Arabic typeset left-
> to-right. Under Windows it is impossible. Under luaTeX it should be
> possible. Another example: contextual analysis of Urdu is quite more
> complicated than the one of regular Arabic script, it is not possible
> in the presence of Uniscribe.

do you mean nasta`liq?

[By the way -- and this is by no means directed at you :-) -- SIL and  
others have really confused things by talking about "the Urdu script". As  
you know so well, Urdu uses Arabic script; it just happens to prefer the  
nastaliq style to the naskh style.]

But maybe I misunderstand you: I get working nastaliq under uniscribe...  
am I missing something? [I'd be happy for us to start another thread on  
this topic]

> For reasons of efficiency, whatever is done by Uniscribe or Pango (for
> example, Indic reordering, Korean hangul composition, etc.) should be
> hardcoded into luatex (written in C, etc.) so that it happens very
> quickly. But one should be able to deactivate it using a primitive or
> something of the kind (otherwise we may as well use XeTeX...).
>
> To reply to Idris: if you use a macropackage to do bidi, it's like
> adopting the ArabTeX philosophy: it's like using macros to do the most
> fundamental operations, and it's a pity.

Ok, let's distinguish two types of macropackage:

1. the traditional TeX/LaTeX etc macropackage. I agree with most of what  
you said above in that case

2. lua extensionality. what lua provides is a way of extending the  
functionality of the engine without changing the engine. So bidi, opentype  
etc can be implemented as extensions of the engine without fiddling with  
the engine itself. The macropackage writers can, well, package those  
extensions as they see fit, and even change them, without bothering the  
main engine.

Of course, there are twilight zones: Where exactly do the boundaries  
between engine, lua extensionality of the engine, and macropackage lie? In  
the case of bidi my understanding is that it fits in the  
lua-extensionality-to-the-engine component. lua-extensionality is the  
perfect balance between hardwiring certain behavior and the purgatory of  
arabtex-style macrowriting. If you and I come up with improvements to the  
bidi algorithm, or need to make changes in places -- not a far-fetched  
possibility btw -- then the lua-extensionality is there, and no need to  
break all the work Taco and Hartmut have done lately to fix rtl bugs  
etc... Of course, if there is a real logical bug in the engine, that has  
to be fixed.

:-)

Best wishes
Idris

-- 
Professor Idris Samawi Hamid, Editor-in-Chief
International Journal of Shi`i Studies
Department of Philosophy
Colorado State University
Fort Collins, CO 80523