[XeTeX] XeTeX interchar token mechanism vs. Mapping

Aleksandar Zec zealeksanman at gmail.com
Tue Nov 16 09:39:19 CET 2010

Hello again,

Again, many thanks to Arthur for his explanation regarding XeTeX
interchar token mechanism.

The problem: XeTeX seems to take into account \XeTeXcharclass of a
particular character *before* Mapping is done. Please take a look at
the following example: the word "baino" should appear equally
non-hyphenated in both cases, however it doesn't, depending on whether
U+0301 is inserted directly or via Mapping.

To get it correctly hyphenated, you have to set \XeTeXcharclass for
'/', which is not as it should be.

------- test.map -------
LHSName "Testing XeTeX interchar token mechanism"
U+002F U+0060 U+0060 <> U+0301 ; i.e. /`` -> combining acute

------- test2.tex -------
\setmainfont[Mapping=test]{DejaVu Sans}

\XeTeXinterchartoks 0 \combiningchar = {\penalty10000}

    \hspace*{2.33cm}βαι/``νο κτλ.\\ % incorrect hyphenation
    \hspace*{2.33cm}βαι\char"0301 νο κτλ. % correct hyphenation


More information about the XeTeX mailing list