[XeTeX] Running out of memory in XeSearch while trying to XeIndex on largish latex files.

[Michael Joyner]

On Thu, Feb 10, 2011 at 8:21 AM, Paul Isambert <zappathustra at free.fr> wrote:

> Le 10/02/2011 04:39, Ross Moore a écrit :
>
>  Hi Michael, and Heiko,
>>
>> On 10/02/2011, at 1:49 PM, Michael Joyner wrote:
>>
>>
>>> On Wed, Feb 9, 2011 at 9:30 PM, Ross Moore<ross.moore at mq.edu.au>  wrote:
>>>
>>>
>>> See how large you can set the  "save size" parameter.
>>> Multiply by 10, or 100, or 1000... .
>>>
>>> I'd say you are exploring to the boundaries of what XeTeX
>>> is capable of doing.
>>>
>>>
>>> save size won't go over 80,000 :(
>>>
>> OK.
>>
>> But we don't need it now.
>> Here's the cause of the problem.
>>
>> The package source  xesearch.sty   has a technical problem.
>> The macros  \xs at String  and  \xs at Stack  are used as variables,
>> repeatedly changing their expansions. However, sometimes the code
>> uses \edef\xs at String{...} but mostly it uses \xdef\xs at String{...}.
>>
>> This mixture of local/global scope is what causes the loss of
>> string space, because an \edef instance requires the previous \xdef
>> instance to be retained, not discarded. Then comes another \xdef
>> which may release the previous \edef's memory, but not that of the
>>  \xdef  prior to the \edef .  Hence memory usage grows.
>>
>> By making all instances become global, I now get your document to
>> finish, along with the Index page.
>>
>> Here's the memory usage:
>>
>>  Here is how much of TeX's memory you used:
>>>>>  26092 strings out of 494542
>>>>>  451878 string characters out of 3157455
>>>>>  480737 words of memory out of 3000000
>>>>>  29083 multiletter control sequences out of 15000+200000
>>>>>  8574 words of font info for 51 fonts, out of 3000000 for 9000
>>>>>  669 hyphenation exceptions out of 8191
>>>>>  40i,7n,43p,1687b,9339s stack positions out of
>>>>> 5000i,500n,10000p,200000b,50000s
>>>>>
>>>>
>>
>>
>> Here's the patch needed to modify  xesearch.sty .
>>
>>
>>
Does this look right?

diff -u xesearch*sty
--- xesearch-orig.sty 2011-02-10 09:56:49.000000000 -0500
+++ xesearch.sty 2011-02-10 09:56:25.000000000 -0500
@@ -166,7 +166,7 @@
         \xdef\xs at Stack{%
           \xs at String\noexpand\xs at end\xs at unexpanded\expandafter{\xs at Stack}%
           }%
-        \edef\xs at String{\xs at unexpanded\expandafter{\xs at String} }%
+        \xdef\xs at String{\xs at unexpanded\expandafter{\xs at String} }%
         \XeTeXinterchartokenstate1
       \else
         \expandafter\xs at Lowercase\expandafter{\xs at String}\xs at lcString
@@ -175,7 +175,7 @@
           \xdef\xs at Stack{%
             \xs at String\noexpand\xs at end\xs at unexpanded\expandafter{\xs at Stack
}%
             }%
-          \edef\xs at String{\xs at unexpanded\expandafter{\xs at String} }%
+          \xdef\xs at String{\xs at unexpanded\expandafter{\xs at String} }%
           \XeTeXinterchartokenstate1
         \else
           \expandafter\expandafter\expandafter\xs at EndString
@@ -684,16 +684,16 @@
     \fi
   \fi\xs at next
   }
-\def\xs at Stack{}
+\xdef\xs at Stack{}
 \def\xs at Remainder{}
 \def\xs at StartString{%
   \xs at Stringtrue
   \let\xs at StartString\relax
-  \def\xs at String{}%
+  \xdef\xs at String{}%
   \def\PrefixFound{}%
   \def\SuffixFound{}%
   \def\AffixFound{}%
-  \def\xs at Stack{}%
+  \xdef\xs at Stack{}%
   \def\xs at Remainder{}%
   \xs at Phrasefalse
   \setbox\xs at Box=\hbox\bgroup
@@ -908,7 +908,7 @@
           \let\xs at next\xs at Evaluate
         \else
           \ifxs at Phrase
-            \def\xs at Stack{}%
+            \xdef\xs at Stack{}%
             \def\xs at next{\xs at String\xs at Restore}%
           \else
             \def\xs at next{\unhbox\xs at Box\xs at Restore}%
@@ -975,7 +975,7 @@
   \XeTeXinterchartokenstate=1
   }
 \def\xs at PopWord#1\xs at end#2\xs at end{%
-  \def\xs at String{#2}%
+  \xdef\xs at String{#2}%
   \def\xs@@PopWord#2##1\xs at end{%
     \edef\xs at Remainder{##1\xs at unexpanded\expandafter{\xs at Remainder}%
       }%
@@ -983,7 +983,7 @@
   \xs@@PopWord#1\xs at end
   }
 \def\xs at PopStack#1\xs at end#2\xs@@end{%
-  \def\xs at Stack{#2}%
+  \xdef\xs at Stack{#2}%
   \expandafter\xs at PopWord\xs at String\xs at end#1\xs at end
   }
 \def\SortByLength#1{%
[seeksort at www sty]$
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tug.org/pipermail/xetex/attachments/20110210/b040cdac/attachment.html>