xunicode expansion problem (was Re: [XeTeX] Hyphenation of "--" with tex-text mapping on)

[Bruno Voisin]

Le 26 nov. 05 à 02:46, Ross Moore a écrit :

> On 25/11/2005, at 2:35 AM, Bruno Voisin wrote:
>
>> For example earlier today I had problems getting correct bookmarks  
>> in a hyperref document where I had defined subsubsections to be  
>> unnumbered but to appear in the toc and bookmarks using:
>>
>> \setcounter{secnumdepth}{2}
>> \setcounter{tocdepth}{3}
>>
>> This turned out to be linked with cryptic error messages such as:
>>
>> Package hyperref Warning: The anchor of a bookmark and its  
>> parent's must not
>> (hyperref)                be the same. Added a new anchor on input  
>> line 748.
>
> I always have trouble with bookmarks, in documents which
> have a structure that differs from the simple LaTeX classes.
> It's invariably connected with getting certain counters to
> have the right value at a critical place within the processing.
>
> Examine the .out file to determine *what* you want.
> Then do a bit of strategic \tracingall  to locate *where* the
> incorrect value comes from. Then devise a way to fix it.
>
>> Eventually, after many unsuccessful attempts, I ended up searching  
>> comp.text.tex in Google groups to find a post from December 2000  
>> [...], mentioning the hyperref command \phantomsection which fixed  
>> the problem but seems to be mentioned nowhere except inside the  
>> source code hyperref.sty itself.
>
> I remember conversing with Heiko at about this time.
> He created some other commands:
>
> \newcommand{\currentpdfbookmark}{%
> \newcommand{\subpdfbookmark}{%
> \newcommand{\belowpdfbookmark}[2]{%
>
>  ... none of which are documented either.

Hi Ross,

An elucidation of the bug, revealing it was, again, entirely my fault.

In order to get customized section numbers written to the table of  
contents, I had redefined the control sequence \@sect from latex.ltx,  
to:

\def\@sect#1#2#3#4#5#6[#7]#8{%
   [...]
   \ifdim \@tempskipa>\z@
     [...]
     \addcontentsline{toc}{#1}{%
       \ifnum #2>\c at secnumdepth \else
         \protect\numberline{\csname the#1\endcsname\ \textendash}%
       [...]
   \else
     [...]
       \addcontentsline{toc}{#1}{%
         \ifnum #2>\c at secnumdepth \else
           \protect\numberline{\csname the#1\endcsname\ \textendash}%
         [...]

Given I was using the hyperref package, I had looked (quickly) at  
hyperref.sty, verified it wasn't fussing with \@sect and deduced I  
wasn't doing anything wrong.

However, it turns out the hyperref package *does* fuss with \@sect,  
not in hyperref.sty but in the driver definition files, as may be  
seen by looking at the default driver file hypertex.def upon which  
all other driver are based:

\let\H at old@sect\@sect
\def\@sect#1#2#3#4#5#6[#7]#8{%
   \ifnum #2>\c at secnumdepth
     \Hy at GlobalStepCount\Hy at linkcounter
     \xdef\@currentHref{section*.\the\Hy at linkcounter}%
   \fi
   \H at old@sect{#1}{#2}{#3}{#4}{#5}{#6}[{#7}]{#8}%
   \ifnum #2>\c at secnumdepth
     \Hy at raisedlink{\hyper at anchorstart{\@currentHref}\hyper at anchorend}%
   \fi
}

Thus it turns out the standard \@sect is stored in a control sequence  
\H at old@sect, and then redefined to add to this standard definition  
some additional instructions dealing with bookmarks. And, no  
surprise, these additional instructions are exactly the content of  
\phantomsection from hyperref.sty:

\def\phantomsection{%
\Hy at GlobalStepCount\Hy at linkcounter
\xdef\@currentHref{section*.\the\Hy at linkcounter}%
\Hy at raisedlink{\hyper at anchorstart{\@currentHref}\hyper at anchorend}%
}

Thus by making my redefinition apply to \H at old@sect instead of  
\@sect, I was able to solve the problem.

Not quite, actually: with the above definition of \@sect in  
hypertex.def (and hpdftex.def etc. as well):

- For numbered sections the bookmarks are placed immediately before  
the section titles.

- For unnumbered sections the bookmarks are placed immediately before  
the text following the section titles, but after these titles which  
are thus hidden.

That's not what I was looking for: I was willing bookmarks to be  
placed immediately before the section titles, whether the sections  
are numbered or not. This was cured by redefining \@sect to include  
the two parts of \phantomsection *before* \H at old@sect, instead of  
spreading them partly before and partly after it:

\def\@sect#1#2#3#4#5#6[#7]#8{%
   \ifnum #2>\c at secnumdepth
     \Hy at GlobalStepCount\Hy at linkcounter
     \xdef\@currentHref{section*.\the\Hy at linkcounter}%
     \Hy at raisedlink{\hyper at anchorstart{\@currentHref}\hyper at anchorend}%
   \fi
   \H at old@sect{#1}{#2}{#3}{#4}{#5}{#6}[{#7}]{#8}%
}

So far this seems to work, and there is no need to insert  
\phantomsection explicitly any longer in the LaTeX input alongside  
with \section.

I realized all the above by getting the documented source code  
hyperref.dtx from CTAN, and typesetting it to get the full hyperref  
documentation. Then looking for \phantomsection in the resulting  
hyperref.pdf I found out (1) a whole bunch of sectioning commands  
such as \@sect and \@chapter are modified from their LaTeX defaults  
and (2) the redefinitions take place in hypertex.def not hyperref.sty.

Again, sorry for your time possibly lost on that.

Bruno