[XeTeX] Problem with typesetting table with Arabic characters with TeX Live 2009 on Ubuntu Jaunty 9.04

[Jonathan Kew]

On 12 Aug 2009, at 12:32, Edin Salkovic wrote:

> I'm generating (processing) some XeTeX files in Pyhon, and Python
> prints out the line ends according to the underlying OS.  I have
> attached the Windows version of the minimal.tex file that works fine
> on Windows.  I haven't tried this version on GNU/Linux.
>
> I still don't understand why the line end makes a difference?

The kind of line-end, by itself, shouldn't make a difference; AFAIK  
all current TeX implementations are line-ending-neutral, and accept  
either DOS/Windows or Unix-style line endings without fuss.

However, your "Windows version" of the file actually seems to contain  
a somewhat broken mixture of conventions. Hex-dumping shows:

00000000  5c 64 6f 63 75 6d 65 6e  74 63 6c 61 73 73 7b 61  | 
\documentclass{a|
00000010  72 74 69 63 6c 65 7d 0a  5c 75 73 65 70 61 63 6b  |rticle}. 
\usepack|
00000020  61 67 65 7b 62 69 64 69  7d 0a 5c 75 73 65 70 61  |age{bidi}. 
\usepa|
00000030  63 6b 61 67 65 7b 66 6f  6e 74 73 70 65 63 7d 0a  | 
ckage{fontspec}.|
00000040  5c 6e 65 77 66 6f 6e 74  66 61 6d 69 6c 79 7b 5c  | 
\newfontfamily{\|
00000050  61 72 61 62 69 63 66 6f  6e 74 7d 5b 53 63 72 69  | 
arabicfont}[Scri|
00000060  70 74 3d 41 72 61 62 69  63 2c 20 53 63 61 6c 65  | 
pt=Arabic, Scale|
00000070  3d 31 2e 32 5d 7b 53 63  68 65 68 65 72 61 7a 61  |=1.2] 
{Scheheraza|
00000080  64 65 7d 0a 0a 5c 6e 65  77 63 6f 6d 6d 61 6e 64  |de}.. 
\newcommand|
00000090  7b 5c 61 72 7d 5b 31 5d  7b 5c 52 4c 7b 5c 61 72  |{\ar}[1] 
{\RL{\ar|
000000a0  61 62 69 63 66 6f 6e 74  23 31 7d 7d 0a 5c 6e 65  | 
abicfont#1}}.\ne|
000000b0  77 63 6f 6d 6d 61 6e 64  7b 5c 61 72 63 65 6c 6c  | 
wcommand{\arcell|
000000c0  7d 5b 31 5d 7b 5c 61 72  7b 5c 4c 61 72 67 65 23  |}[1] 
{\ar{\Large#|
000000d0  31 7d 7d 0a 0a 5c 62 65  67 69 6e 7b 64 6f 63 75  |1}}.. 
\begin{docu|
000000e0  6d 65 6e 74 7d 0a 0a 5c  62 65 67 69 6e 7b 74 61  |ment}.. 
\begin{ta|
000000f0  62 6c 65 7d 0a 5c 62 65  67 69 6e 7b 63 65 6e 74  |ble}. 
\begin{cent|
00000100  65 72 7d 0a 5c 63 61 70  74 69 6f 6e 7b 54 68 65  |er}. 
\caption{The|
00000110  20 63 6f 6d 6d 61 6e 64  7d 0a 5c 62 65 67 69 6e  | command}. 
\begin|
00000120  7b 74 61 62 75 6c 61 72  7d 5b 76 61 6c 69 67 6e  |{tabular} 
[valign|
00000130  3d 63 65 6e 74 65 72 5d  7b 63 20 63 7d 0a 5c 68  |=center] 
{c c}.\h|
00000140  6c 69 6e 65 0a 5c 61 72  63 65 6c 6c 7b d9 84 d9  |line. 
\arcell{...|
00000150  90 d8 aa d9 8f d9 81 d9  92 d8 b9 d9 8e d9 84 d9   
|................|
00000160  92 7d 26 5c 61 72 63 65  6c 6c 7b d8 a7 d9 90 d9  |.}& 
\arcell{.....|
00000170  81 d9 92 d8 b9 d9 8e d9  84 d9 92 7d 5c 5c 0d 0d   
|...........}\\..|
00000180  0a 5b 59 6f 75 5d 20 42  65 20 64 6f 6e 65 21 26  |.[You] Be  
done!&|
00000190  5b 59 6f 75 5d 20 44 6f  21 5c 5c 0d 0d 0a 5c 65  |[You] Do!\ 
\...\e|
000001a0  6e 64 7b 74 61 62 75 6c  61 72 7d 0a 5c 65 6e 64  | 
nd{tabular}.\end|
000001b0  7b 63 65 6e 74 65 72 7d  0a 5c 65 6e 64 7b 74 61  |{center}. 
\end{ta|
000001c0  62 6c 65 7d 0a 0a 5c 65  6e 64 7b 64 6f 63 75 6d  |ble}.. 
\end{docum|
000001d0  65 6e 74 7d 0a                                    |ent}.|

Looking at the data preceding the table shows that the line-endings  
are just LF (0x0A), i.e., Unix-style. But the lines of your table,  
which are the crucial part here, end with CR CR LF (look at offsets  
017E..0180 and 019B..019D). Here, there is not only a CR/LF pair  
(which TeX, regardless of platform, should read correctly as a single  
newline) but ALSO an extra preceding CR, which will be read as ANOTHER  
newline. So here, TeX will actually see two successive newlines  
following the \\ command.... that is, a blank line.... that is, \par.  
And that's what stops it looking ahead and seeing [You] as an optional  
argument.

(I'm a bit surprised that \par within the tabular environment doesn't  
cause some other kind of trouble. But as I don't use that stuff, I  
don't really know anything about it. Perhaps tabular ignores it.)

JK