[luatex] parallel computing

[Taco Hoekwater]

On 03/14/2011 08:31 AM, Arno Trautmann wrote:
> Hi all,
>
> I think this has been discussed a few times: An a multi-core computer,
> *TeX can only use one core as the typesetting process is linear.

In fact I think I have replied to some of these threads already.

> While the breaking of paragraphs is linear,

Actually it is not, there are up to three passes over the paragraph
during breaking (1: no hyphens, 2: hyphens, 3: emergency), and these
could be run in parallel. If 1) is good enough, the results of 2)
and 3) can be discarded, and similar for 2) and 3). But it remains
to be seen whether the extra setup does not in practise make the
process slower instead of faster.

> there is a second process
> that starts only after paragraph breaking has been done, which is page
> building. Now I could imagine a *TeX that lets one core do the paragraph
> breaking and another one do pagebuilding. Depending on which one of
> those is faster, the paragraph breaker has to be stopped at a
> (specified) number of lines it is in advance of the page builder. If the
> page builder is faster, everything is fine from start on.

That won't work. There are no lines coming put of the paragraph builder
at all until it is completely finished. What could be made to work is
to let the tokenizer engine start on reading a second paragraph already
while the first paragraph is still being broken. There are a number of
commands that would create a pause, but most paragraphs do not contain
them (querying \prevdepth or \pagetotal, for example).

But the most useful multicore operation we can do is to put the pdf
generator into a separate core so that generating pdf from a finished
page one can run in parallel with the creation of a new page two. And
that is the only multi-core optimization that has a chance of being
included before version 1.0.

Best wishes,
Taco