[pdftex] Caching intermediate compilation results for near-real-time PDF re-renders during editing

[Jamie Vicary]

Hi all,

My understanding of pdftex is that it operates on a per-page basis.
One page is rendered and sent to the output, then the next, and so on,
with the result being that the tool can render large documents, with
an essentially constant memory usage, which does not grow
proportionally with the number of pages in the final document.

To me this suggests a possible caching approach which would allow
near-real-time PDF rendering during file editing. It work something
like this.

At first the entire document is compiled. During this process, when
each page n is compiled, the state of the various memory registers
(macros, lengths, etc) is stored as S(n).

Suppose a small change is then made to the latex source, such that the
compiler determines this change would first affect page k. Then the
compiler could recover state S(k-1) from disk, and only re-render page
k. This would of course be extremely fast. The cache S(k) is
re-computed, and the values S(k+1), S(k+2) etc are deleted, to be
re-computed in the future when later page view is requested.

For the user, who has open on screen both the source code and a PDF
preview, this would be extremely convenient, as upon saving their
file, they would see the result of their code changes extremely
quickly.

My knowledge of the architecture of pdflatex is poor, so it is
possible that some part of this description above is suboptimal,
please forgive that. My question is whether it could be possible in
principle to design a caching pdflatex tool that would operate in this
way, and how difficult this would be.

The wider context to my question is the following: I know this is
possible in principle because this is the basis of operation of BaKoMa
TeX, which I have used for years. Sadly the developer has recently
passed away and as a closed-source tool, it can no longer be
maintained.

I believe very strongly in the benefits of ultra-fast recompilation
during editing for efficient authoring of latex documents, and will
happily discuss that. But I am asking here specifically about the
technical achievability of this as an extension of pdflatex.

Best wishes,
Jamie