[tex-k] Does TeX's Input Processor Tokenize The Entire Input File First?
rokicki at gmail.com
Wed Apr 17 09:41:33 CEST 2019
Use the source, Luke. I mean Joseph.
On Tue, Apr 16, 2019 at 11:13 PM Joseph Wright <
joseph.wright at morningstar2.co.uk> wrote:
> On 17/04/2019 04:25, Jon Forrest wrote:
> > I'm trying to understand TeX internals.
> > One thing I haven't found a definitive explanation of is whether the
> > input processor tokenizes the entire input file before the execution
> > processor gets the token list. TexbyTopic says "... all [processor]
> > levels are simultaneously active, and there is interaction between
> > them.". Given the primitive computing environment TeX was developed in,
> > it's hard to imagine how the 4 processors could run simultaneously. As
> > far as I can see, TeX has no multi-threading ability so it would be up
> > to TeX to "process" switch.
> > So, how and why do the processors run simultaneously? How much of the
> > input file gets tokenized before the token list is handed to the
> > expansion processor?
> > I wonder if Knuth were writing TeX today would he use multi-threading?
> > Cordially,
> > Jon Forrest
> TeX tokenizes one item at a time. That's seen of course in the fact that
> one can change catcodes during a document
> \catcode`~=13 %
> \catcode`~=12 %
> ~ world
> As such, the different stages of TeX processing cannot be run in parallel.
-- http://cube20.org/ -- http://golly.sf.net/ --
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the tex-k