xunicode expansion problem (was Re: [XeTeX] Hyphenation of "--" with tex-text mapping on)

[Ross Moore]

Hi Bruno,

On 25/11/2005, at 2:35 AM, Bruno Voisin wrote:

> Le 24 nov. 05 à 14:29, Bruno Voisin a écrit :

> With the LaTeX report class, in order to get, in the table of  
> contents, adjustable space for chapter numbers etc. depending on  
> the font, size, series etc. used in this table, I've defined, for  
> example, a typical chapter number width:
>
> \newlength{\chapnowd}
> \settowidth{\chapnowd}{\normalfont\normalsize\bfseries A.9 -- }

> If now I use \textendash instead of --, namely:
>
> \settowidth{\chapnowd}{\normalfont\normalsize\bfseries A.9  
> \textendash\ }
>
> things still work OK. But if I add the xunicode package on top of  
> this, then it seems \textendash is not taken into account when  
> \chapnowd is measured (in \settowidth), and the chapter numbers in  
> the toc are superposed partly on the chapter titles.

> I imagine this is caused by the way \textendash is expanded by  
> \settowidth, causing the redefinition of \textendash by  
> xunicode.sty to fail. How can this be avoided?

Would you please supply an example, and the .log file.

I'm not at all convinced that xunicode is causing the problem.
Rather, it may be that it is always wrong *without* xunicode.

Using the following coding, I'll describe what I get in some detail.

\documentclass[11pt]{article}
%\usepackage{xunicode}
\DeclareFontFamily{U}{lucida}{}
\DeclareFontShape{U}{lucida}{m}{n}
    {<-> "Lucida\space Grande:mapping=tex-text"}{}
\DeclareFontShape{U}{lucida}{b}{n}
    {<-> "Lucida\space Grande\space Bold:mapping=tex-text"}{}
\DeclareFontShape{U}{lucida}{bx}{n}
    {<-> ssub * lucida/b/n}{}
\renewcommand{\rmdefault}{lucida}
\renewcommand{\familydefault}{\rmdefault}
\renewcommand{\encodingdefault}{U}

\newlength{\chapnowd}
\begin{document}
{%\tracingall
\settowidth{\chapnowd}{\normalfont\normalsize\bfseries A.9 -- }
\showthe\chapnowd
\settowidth{\chapnowd}{\normalfont\normalsize\bfseries A.9 \textendash 
\ }
\showthe\chapnowd
}
\end{document}


Do you agree that this should allow us to see the width obtained
in both cases, without creating any output.
All the info resides in the .log file (console window).

OK, from the above I get:
  (Type a <return> in the "TeX Input" at the ?  prompt.)


 > 30.68454pt.
l.19 \showthe\chapnowd

?

LaTeX Font Warning: Font shape `OT1/lucida/bx/n' undefined
(Font)              using `OT1/cmr/m/n' instead
(Font)              for symbol `textendash' on input line 20.

 > 30.68456pt.
l.21 \showthe\chapnowd

?

   ... which shows just a slightly different width.

***However***, the "LaTeX Font Warning" indicates that the endash
character used was from  cmr  not from  Lucida Grande !



To try to get a better idea of what we are really getting,
uncomment the  \tracingall .
The places that really matter now show:

\font at name ->\U/lucida/bx/n/10.95
{select font "Lucida Grande Bold:mapping=tex-text" at 10.95pt}
{\relax}
{\relax}
{the letter A}
{blank space  }
{the character -}
{blank space  }
{end-group character }}
{restoring current font=\U/lucida/m/n/10.95}

... and ...

\font at name ->\OT1/cmr/m/n/10.95
{select font cmr10 at 10.95pt}
  ... lots of stuff here ...
\textendash ->\OT1-cmd \textendash \OT1\textendash

\OT1-cmd #1->\ifx \protect \@typeset at protect \@inmathwarn #1\else  
\noexpand #1\
expandafter \@gobble \fi
#1<-\textendash
{\ifx}
{true}

\@inmathwarn #1->\ifmmode \@latex at warning {Command \protect #1  
invalid in math
mode}\fi
#1<-\textendash
{\ifmmode}
{false}
{\else}
{\char"7B}
{end-group character }}
{restoring \cf at encoding=macro:->U}
{restoring \default at shape=macro:->n}
{restoring \default at series=macro:->m}
{restoring \default at family=macro:->cmr}
{restoring \OT1-cmd=macro:#1#2->\ifx \protect \@typeset at protect \ETC.}
{restoring \U-cmd=macro:#1->\ifx \protect \@typeset at protect \ETC.}
{restoring current font=\U/lucida/b/n/10.95}
{restoring \OT1/lucida/bx/n/10.95=undefined}
{restoring \f at encoding=macro:->U}
{restoring \@wrong at font@char=macro:->}
{\ }
{end-group character }}
{retaining \U\textendash=\long macro:->\UseTextSymbol {OT1}\textendash }


Thus we are getting  \char"7B  from font cmr10 at 10.95pt .
The {\ } is done with  \U/lucida/b/n/10.95  as the current font.



OK, now lets try this again using  \usepackage{xunicode} .

Without tracing, I get exactly equal lengths:


 > 30.68454pt.
l.18 \showthe\chapnowd

?
 > 30.68454pt.
l.20 \showthe\chapnowd

?


With \tracingall the relevant details are:


\font at name ->\U/lucida/bx/n/10.95
{select font "Lucida Grande Bold:mapping=tex-text" at 10.95pt}
{\relax}
{\relax}
{the letter A}
{blank space  }
{the character -}
{blank space  }
{end-group character }}
{restoring current font=\U/lucida/m/n/10.95}

... and ...

\font at name ->\U/lucida/bx/n/10.95
{select font "Lucida Grande Bold:mapping=tex-text" at 10.95pt}
{\relax}
{\relax}
{the letter A}
{blank space  }

\textendash ->\OT1-cmd \textendash \OT1\textendash
  ... other stuff here ...
\U\textendash ->\char "2013\relax
{\char}
{\relax}
{\ }
{end-group character }}
{restoring current font=\U/lucida/m/n/10.95}



With the lengths agreeing to 5 decimal places, then surely
the same font characters are being used, and the visual
output is identical.


Summarising, I cannot reproduce your problem this way; though I have
seen differences using other ways to declare the document's fonts.


Here's one such example.
Now we use  fontspec.sty  in the preamble:

\documentclass[11pt]{article}
\usepackage{xunicode}
\usepackage{fontspec}
\setromanfont{Lucida Grande:mapping=tex-text}
\newlength{\chapnowd}
\begin{document}


The output is dramatically different from previously.
Maybe I'm not using  fontspec  correctly.


Without xunicode :

 > 37.54433pt.
l.20 \showthe\chapnowd

?

LaTeX Font Warning: Font shape `OT1/LucidaGrande:mapping=tex-text/bx/ 
n' undefin
ed
(Font)              using `OT1/cmr/m/n' instead
(Font)              for symbol `textendash' on input line 21.

 > 30.34773pt.
l.22 \showthe\chapnowd

?


With xunicode :

 > 37.54433pt.
l.20 \showthe\chapnowd

?
 > 30.34772pt.
l.22 \showthe\chapnowd


I'll explore a bit further, but really I need to see an example
from you before being able to point-the-finger at a culprit.
(So far it doesn't seem to be  xunicode .)



>
> Then the subsidairy question is: why do I always have to confronted  
> with bugs or features, as soon as I'm starting to use software  
> seriously ;-(
>
> For example earlier today I had problems getting correct bookmarks  
> in a hyperref document where I had defined subsubsections to be  
> unnumbered but to appear in the toc and bookmarks using:
>
> \setcounter{secnumdepth}{2}
> \setcounter{tocdepth}{3}
>
> This turned out to be linked with cryptic error messages such as:
>
> Package hyperref Warning: The anchor of a bookmark and its parent's  
> must not
> (hyperref)                be the same. Added a new anchor on input  
> line 748.

I always have trouble with bookmarks, in documents which
have a structure that differs from the simple LaTeX classes.
It's invariably connected with getting certain counters to
have the right value at a critical place within the processing.

Examine the .out file to determine *what* you want.
Then do a bit of strategic \tracingall  to locate *where* the
incorrect value comes from. Then devise a way to fix it.

>
> Eventually, after many unsuccessful attempts, I ended up searching  
> comp.text.tex in Google groups to find a post from December 2000  
> <http://groups.google.com/group/comp.text.tex/browse_thread/thread/ 
> 5f5632c9cdd5f2a2/9689fc0b4365dc53?q=Package+hyperref+Warning%3A+The 
> +anchor+of+a+bookmark+and+its 
> +parent's&rnum=1&auth=DQAAAGQAAABbYvlv0bjyVsORMqk5DWQJPUIgz6rUiAmaexfo 
> v1qCaq9n7yLl9gZeBVehQSfpYehFUFk-aOmzdEWN_-mDS6- 
> wfaKXa60w1HS0XHL6N4IgKpAKs7V9LRYHRcgBzAk_Wqr7HhMw1x3GqWcim_Jgs9Yh>,  
> mentioning the hyperref command \phantomsection which fixed the  
> problem but seems to be mentioned nowhere except inside the source  
> code hyperref.sty itself.

I remember conversing with Heiko at about this time.
He created some other commands:

\newcommand{\currentpdfbookmark}{%
\newcommand{\subpdfbookmark}{%
\newcommand{\belowpdfbookmark}[2]{%

  ... none of which are documented either.

>
> Bruno Voisin


Cheers,

	Ross

------------------------------------------------------------------------
Ross Moore                                         ross at maths.mq.edu.au
Mathematics Department                             office: E7A-419
Macquarie University                               tel: +61 +2 9850 8955
Sydney, Australia  2109                            fax: +61 +2 9850 8114
------------------------------------------------------------------------