[XeTeX] Running out of memory in XeSearch while trying to XeIndex on largish latex files.

[Paul Isambert]

Le 10/02/2011 04:39, Ross Moore a écrit :
> Hi Michael, and Heiko,
>
> On 10/02/2011, at 1:49 PM, Michael Joyner wrote:
>
>>
>> On Wed, Feb 9, 2011 at 9:30 PM, Ross Moore<ross.moore at mq.edu.au>  wrote:
>>
>>
>> See how large you can set the  "save size" parameter.
>> Multiply by 10, or 100, or 1000... .
>>
>> I'd say you are exploring to the boundaries of what XeTeX
>> is capable of doing.
>>
>>
>> save size won't go over 80,000 :(
> OK.
>
> But we don't need it now.
> Here's the cause of the problem.
>
> The package source  xesearch.sty   has a technical problem.
> The macros  \xs at String  and  \xs at Stack  are used as variables,
> repeatedly changing their expansions. However, sometimes the code
> uses \edef\xs at String{...} but mostly it uses \xdef\xs at String{...}.
>
> This mixture of local/global scope is what causes the loss of
> string space, because an \edef instance requires the previous \xdef
> instance to be retained, not discarded. Then comes another \xdef
> which may release the previous \edef's memory, but not that of the
>   \xdef  prior to the \edef .  Hence memory usage grows.
>
> By making all instances become global, I now get your document to
> finish, along with the Index page.
>
> Here's the memory usage:
>
>>>> Here is how much of TeX's memory you used:
>>>>   26092 strings out of 494542
>>>>   451878 string characters out of 3157455
>>>>   480737 words of memory out of 3000000
>>>>   29083 multiletter control sequences out of 15000+200000
>>>>   8574 words of font info for 51 fonts, out of 3000000 for 9000
>>>>   669 hyphenation exceptions out of 8191
>>>>   40i,7n,43p,1687b,9339s stack positions out of 5000i,500n,10000p,200000b,50000s
>
>
>
> Here's the patch needed to modify  xesearch.sty .
>
>
> [GlenMorangie:tex/xetex/xesearch] rossmoor% diff xesearch-*.sty
> 169c169
> <          \edef\xs at String{\xs at unexpanded\expandafter{\xs at String} }%
> ---
>>          \xdef\xs at String{\xs at unexpanded\expandafter{\xs at String} }%
> 178c178
> <            \edef\xs at String{\xs at unexpanded\expandafter{\xs at String} }%
> ---
>>            \xdef\xs at String{\xs at unexpanded\expandafter{\xs at String} }%
> 687c687
> <  \def\xs at Stack{}
> ---
>> \xdef\xs at Stack{}
> 692c692
> <    \def\xs at String{}%
> ---
>>    \xdef\xs at String{}%
> 696c696
> <    \def\xs at Stack{}%
> ---
>>    \xdef\xs at Stack{}%
> 911c911
> <              \def\xs at Stack{}%
> ---
>>              \xdef\xs at Stack{}%
> 978c978
> <    \def\xs at String{#2}%
> ---
>>    \xdef\xs at String{#2}%
> 986c986
> <    \def\xs at Stack{#2}%
> ---
>>    \xdef\xs at Stack{#2}%
>
> I'll leave it to the author to verify that these changes
> do not affect the functionality of the package in any
> detrimental way.
> Or if not, to modify  xesearch.sty  differently, while
> avoiding the mistake of mixing local and global definitions
> of the save macro-name.

I'll be happy to include the patch if it works. I generally use 
local/global assignment with a purpose, but I might have been fussy (I 
remember XeSearch being one painful gestation). And I'll admit I'm 
under-educated re those memory stack things or whatever.
Anyway, many thanks Ross for taking the trouble to look into that 
spaghetti code and spotting the problem.

Best,
Paul