[XeTeX] Running out of memory in XeSearch while trying to XeIndex on largish latex files.

[Ross Moore]

Hi Michael, and Heiko,

On 10/02/2011, at 1:49 PM, Michael Joyner wrote:

> 
> 
> On Wed, Feb 9, 2011 at 9:30 PM, Ross Moore <ross.moore at mq.edu.au> wrote:
> 
> 
> See how large you can set the  "save size" parameter.
> Multiply by 10, or 100, or 1000... .
> 
> I'd say you are exploring to the boundaries of what XeTeX
> is capable of doing.
> 
> 
> save size won't go over 80,000 :(

OK.

But we don't need it now.
Here's the cause of the problem.

The package source  xesearch.sty   has a technical problem.
The macros  \xs at String  and  \xs at Stack  are used as variables, 
repeatedly changing their expansions. However, sometimes the code
uses \edef\xs at String{...} but mostly it uses \xdef\xs at String{...}.

This mixture of local/global scope is what causes the loss of 
string space, because an \edef instance requires the previous \xdef
instance to be retained, not discarded. Then comes another \xdef 
which may release the previous \edef's memory, but not that of the
 \xdef  prior to the \edef .  Hence memory usage grows.

By making all instances become global, I now get your document to
finish, along with the Index page.

Here's the memory usage:

>>> Here is how much of TeX's memory you used:
>>>  26092 strings out of 494542
>>>  451878 string characters out of 3157455
>>>  480737 words of memory out of 3000000
>>>  29083 multiletter control sequences out of 15000+200000
>>>  8574 words of font info for 51 fonts, out of 3000000 for 9000
>>>  669 hyphenation exceptions out of 8191
>>>  40i,7n,43p,1687b,9339s stack positions out of 5000i,500n,10000p,200000b,50000s




Here's the patch needed to modify  xesearch.sty .


[GlenMorangie:tex/xetex/xesearch] rossmoor% diff xesearch-*.sty
169c169
<         \edef\xs at String{\xs at unexpanded\expandafter{\xs at String} }%
---
>         \xdef\xs at String{\xs at unexpanded\expandafter{\xs at String} }%
178c178
<           \edef\xs at String{\xs at unexpanded\expandafter{\xs at String} }%
---
>           \xdef\xs at String{\xs at unexpanded\expandafter{\xs at String} }%
687c687
< \def\xs at Stack{}
---
> \xdef\xs at Stack{}
692c692
<   \def\xs at String{}%
---
>   \xdef\xs at String{}%
696c696
<   \def\xs at Stack{}%
---
>   \xdef\xs at Stack{}%
911c911
<             \def\xs at Stack{}%
---
>             \xdef\xs at Stack{}%
978c978
<   \def\xs at String{#2}%
---
>   \xdef\xs at String{#2}%
986c986
<   \def\xs at Stack{#2}%
---
>   \xdef\xs at Stack{#2}%


I'll leave it to the author to verify that these changes
do not affect the functionality of the package in any
detrimental way.
Or if not, to modify  xesearch.sty  differently, while
avoiding the mistake of mixing local and global definitions
of the save macro-name.


> 



Hope this helps,

	Ross

------------------------------------------------------------------------
Ross Moore                                       ross.moore at mq.edu.au 
Mathematics Department                           office: E7A-419      
Macquarie University                             tel: +61 (0)2 9850 8955
Sydney, Australia  2109                          fax: +61 (0)2 9850 8114
------------------------------------------------------------------------