[luatex] LuaTeX precompilation

[Hans Hagen]

On 12/22/2023 1:21 AM, Reinhard Kotucha via luatex wrote:
> On 2023-12-21 at 10:27:48 +0100, Hans Hagen wrote:
> 
>   > I stop here as I've written plenty about performance the last
>   > decades in various docs,
> 
> Hi Hans,
> please let me mention this though:
> 
> A few months ago I uncompressed the lualatex format file and
> compressed it with lz4.  The lz4-compressed file was smaller than the
> original format file.  Though I know that luatex doesn't use maximum
> gzip compression, I didn't really expect this.
> 
>   26702648  lualatex.fmt-uncompressed
>   12201683  lualatex.fmt
>    9926586  lualatex.fmt.lz4
> 
> But what I absolutely didn't expect is that de-compression of lz4
> files is about 7.5 times as fast as de-compression of the gzip'ed
> format file.
> 
> In order to determine speed I directed the de-compressed output to
> /dev/null because when writing to disk results differ much more
> between successive runs and a format file is extracted in memory and
> not written to disk.
> 
> Maybe it's worthwhile to keep an eye on lz4.
Ok, it's a bit off topic but ... in luametatex (context lmtx) we don't 
zip the format at all. One reason is that is not really needed because 
the format file smaller and also because there is that little overhead 
involved (actually endian swapping also takes time but in a thread on 
this list a while ago that was not deemed relevant).

Anyway, I played with (and interface to) zip (gz), lzo, lz4 and zstd for 
various purposes (one is that i can have a real fast texlive files 
uncompression when i need to see what is in these xz files, esp when it 
comes to fonts).    	

cont-en.fmt:

             uncompressed  gzip -3    format
luametatex: 19,387,682    6,483,137  (we don't compress)
luatex    : 19,819,371    6,281,775  11,433,340

(there is more code in luametatex but we save some elsewhere in tex)

So, actually luates doesn't compress agressive (11 vs 6 MB). I have no 
clue how large other macro package format files are (compressed or 
uncompressed).

(bit as mentioned, in earlier threads about performance formats when i 
mentioned formats it was concluded that that had little impact)

it's very unlikely that we will change luatex in this respect,

Hans
  	 	

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------