[lltx] bidi package extended for LuaLaTeX

[Khaled Hosny]

On Fri, Apr 02, 2010 at 11:33:30AM +0200, Manuel Pégourié-Gonnard wrote:
> [quotations re-ordered]
> 
> Khaled Hosny a écrit :
> > On Fri, Apr 02, 2010 at 11:37:45AM +1100, Vafa Khalighi wrote:
> >> 1- Tex macro programming is not a disadvantage, it actually is an art to do
> >> macro programming.
> > 
> > Any one who wrote or maintained a reasonably complex tex macro code
> > knows it is pain to write, pain to understand and pain to maintain, the
> > main idea behind luatex is to give tex a sane programming language which
> > we all know tex mcros isn't.
> > 
> I tend to disagree with this restrictive vision of LuaTeX. To me, the biggest
> advantage of LuaTeX over all other engines is to open up some internals of TeX.
> I would have called the project OpenTeX :-) Having a saner programming language
> is only the second-biggest advantage to me. (And of course, native support for
> Unicode, including math, is also a enormous advantage over pdfTeX.)

That 'is' in 'main idea behind luatex is' was meant to be 'was', and I
can't but agree with you here, what is the biggest advantage of luatex
wasn't the point I was trying to make.

> By the way, I agree with you that in general Lua is a "better" programming
> language than TeX (which wasn't precisely designed as a programming language
> btw), but:
> - Great things have been done and successfully maintained with TeX. It's a
> tricky language, but a lot of tricks are now pretty standard, if you take the
> time to read and study code of experienced TeX programmers before starting
> writing your own (which is a good idea in any language btw).

Great things were written in assembly too (and I know at least one
modern OS written completely in it), it is not about the possibility,
it is about in what time and what degree of complexity one can do
things. So, basically what you are saying is that tex is Turing
complete, a fact I don't find very interesting by itself.

> - Most experienced programmers agree that the quality of a code depends much
> more on the skills of its author than the language it is written in.

True, but again there are languages that force writing ugly code on you,
and from my experience almost all complex tex macro that I see are ugly
and unreadable, I for example don't know any major tex package actively
maintained by anyone other than the original developer unless it was
just rewritten, and there must be a reason for this.

> Therefore, I quite strongly disagree with your habit of saying "it's written in
> TeX, it can't be good, it should be rewritten in Lua". (Ok, I'm quite quick at
> calling it an habit, it's only the second occurrence, the first being microtype.)

Yes and no, yes I strongly dislike programing in tex macros, or,
precisely, I plainly hate it, no I never said "it's written in TeX, it
can't be good, it should be rewritten in Lua", but rather "it is written
in tex, so I'l never be able to hack it, so I'd rather rewrite it in
lua"; I'm not into the position of telling anyone what to do, I'm rather
talking about what can I do. The original question was if I'm interested
in bidi package or not, and my answer was literally "*I*'m not
interested in bidi package since *I* feel it is too overloaded by
legacies of TeX macros for *me* to manage", emphasis added.

> By the way, concerning the advantages/disavantages of LuaTeX, it's worth noting
> that LuaTeX's overall complexity is bigger that other engines, since you need to
> understand /both/ TeX and Lua languages in order to be a good LuaTeX programmer.
> (Though maybe you don't need to be as good with TeX as previously, you just
> can't ignore TeX rules for lexical analysis (catcodes etc) and development.)

No, I need not to understand tex macros, well nothing more than the very
superficial understanding, the rest I simply ignore and I think I'm
doing fine so far.

> >> 2- bidi is not overloaded by TeX macros.
> > 
> > It is, more than 7000 lines of tex macro to implement such basic thing
> > as reversing document layout is a bit overload to me, and one still
> > needs to mark rtl or ltr runs explicitly.
> > 
> Here you're basically telling Vafa that what he did is very basic but that he
> nevertheless used a lot of code to do it. Maybe you're thinking that it's TeX's
> fault, not Vafa's, but maybe you're implying that Vafa is a bad programmer (I
> hope the first option is correct).

Surely it is the limitation of tex macros and tex-xet bidirectional
extension, what this discussion is all about.

> >> 3- bidi does not copy codes from other packages verbatim.
> > 
> > It does, or what all those *-bidi.def files, with thousands of lines
> > redefining internal macros of other packages then?
> > 
> It's not copied verbatim (unless we have different definitions of verbatim), the

"Verbatim" was not my word, it is Vafa's, what I said was "and I really
hate latex packages duplicating code of other packages", which is what
the redefined, rtl-aware internal macros is all about.

> whole point is to modify the macros, in order to insert bidi instructions where
> they are needed. It implies copying some of the code, and writing in TeX, yes,
> but what else do you want to do?

Use luatex, plain and simple, you get a vastly advanced bidirectional
typesetting support for free and 99% of the "redefined" macros will go
away, without writing a single line in lua (most of the remaining 1% can
be implemented in lua).

> It may displease you, but that's the way things go in the LaTeX world. Most
> packages don't anticipate future developments and don't provide hooks for other
> packages, so that big fundamental packages such as bidi, tex4ht, hyperref and
> others need to redefine other package's internals in order to get the job done.
> (Ideally a lot of this should be in the kernel, but it is not and you have to
> make things work anyway.) At first glance, the way bidi does this is the most
> correct way to do it.

But then you are walking on a field of mines, and since many of the
redefined packages are in active development, things can break any time
(and it does), I know it is a "standard" latex practice, but that want
make me feel positive about it (but it doesn't matter, latex is flawed
in so many ways anyway).

> I see it as a big advantage of bidi, from a user's point of view (and users are
> more important than programmers), since it means things "just work". Taking care
> of this require time and usually quite a lot of user feedback, which is of great
> value. Again, this value would probably be lost by rewriting form scratch. This
> is why it is almost always a bad idea to rewrite things from scratch, from a
> software engineering point of view. (And, mind you, saying "I don't understand
> why things are done this way, it looks too complicated to me, and it's not
> written in my favourite language, so I'll just rewrite it" is a common mistake
> of young software developers.) (You may argue that some engineering paradigms
> such as extreme programming suggest frequent rewrite from scratch, but they
> require to do it in a way that accumulated experience doesn't get lost in the
> process.)

It doesn't work for me, and the users I care about, otherwise I would
have been using it and not bothering myself with luatex, why I need to
keep saying this?

So? I'm not learning tex macros and I'm not writing any code in it, and
if I need certain functionality I'll write it in lua, I don't see why I
need to justify this.

> > I were not aware that criticizing the shortcomings of tex macros (that
> > everyone knows and admit) and old engines is not showing respect for
> > others work. If bidi, xetex and programming in tex was so good, I would
> > have saved my self the trouble and just used xetex.
> > 
> See, you're again implying that bidi is no so good...

It is, for my own needs.

> Well, to sum up: from a technical point of view, I'm against rewriting from
> scratch without a good reason. In general, I think it's best, when something for
> XeTeX already exists, to write a "compatibility layer" (such as luaotfload), and
> then reuse existing macros (such as fontspec). Don't forget that user interface
> and interaction with other packages always take much more lines of code than one
> would think (and, unfortunately for you, most of it has to be done in TeX).

You are free to take the position you like, but if it is me who is to
write the code, I think you already know my position.

> >From a personnal point of view:
> 
> > On Fri, Apr 02, 2010 at 11:37:45AM +1100, Vafa Khalighi wrote:
> >> Thanks. Just to answer your insulting email:
> 
> I wouldn't call it insulting, but I nevertheless find it offending, and I think
> you (Khaled) should try to be more diplomatic in the way you write things,
> especially when it comes to the value of other's work. (From a technical point
> of view, I think you're underestimating it as I explained, but even if Vafa's
> work actually wasn't valuable, a little bit of diplomacy never hurts.)

I'm not interested in diplomacy, it is hard enough for me to communicate
in English, so I'd rather keep things simple and go strait forward to
the point.

Regards,
 Khaled

-- 
 Khaled Hosny
 Arabic localiser and member of Arabeyes.org team
 Free font developer


-- 
To unsubscribe, reply using "remove me" as the subject.