[XeTeX] Running out of memory in XeSearch while trying to XeIndex on largish latex files.

[Ross Moore]

Hi Michael,

On 10/02/2011, at 12:01 PM, Michael Joyner wrote:

> 
> 
> On Wed, Feb 9, 2011 at 4:16 PM, Ross Moore <ross.moore at mq.edu.au> wrote:
> Hello Michael,
> That will not help.
> 
> What causes  hyperref to load?  Is it up-to-date?
> 
> Latest TexLive install.
> 
> Let's see more of your .log file/Console output.
> 
> I will send you a copy off list. (size) 

Thanks for those.
I've done some testing, and get different results to you.
That is, the document fails at a different place, since 
I have a different "save size" parameter.
viz.

>> ! TeX capacity exceeded, sorry [save size=50000].

Upon tracing the processing, I cannot see any looping,
but do understand (to some extent) why it is this portion 
of memory that is being exhausted.

It is not due to hyperref's \pdfstringdef , but simply due 
to TeX actually running out of memory while trying to do 
automatic indexing on a (quite) long document.


The process of automatic indexing --- which is cunningly 
implemented by using \XeTeXinterchartoks  to call a macro 
at every inter-word space, and presumably at punctuation
also --- results in multiple new macro definitions, \let-assignments 
and \catcode changes for every single letter and word within 
the body of your document.

Each new \def takes up memory in the "save stack", but I'm
not sure whether any of that memory gets released when the
\def  goes out of scope due to exiting a grouping level.
Clearly it is not *all* being released for reuse.

This ultimately takes a toll on memory usage, resulting
in the failure at some point.

It would be nice to hear from the XeIndex authors about this.


In the meantime, you'll have to accept that there is a physical
limit on the size of a document that can be processed this way.

See how large you can set the  "save size" parameter.
Multiply by 10, or 100, or 1000... .

I'd say you are exploring to the boundaries of what XeTeX 
is capable of doing.

> 
> 
>> 
>> We would really appreciate any advice on this.
>> 
>> Thanks!
>> -Mike


Hope this helps,

      Ross


------------------------------------------------------------------------
Ross Moore                                       ross.moore at mq.edu.au 
Mathematics Department                           office: E7A-419      
Macquarie University                             tel: +61 (0)2 9850 8955
Sydney, Australia  2109                          fax: +61 (0)2 9850 8114
------------------------------------------------------------------------