texlive[59973] trunk: checking fmt sharability across architectures

commits+karl at tug.org commits+karl at tug.org
Sun Jul 18 19:16:30 CEST 2021


Revision: 59973
          http://tug.org/svn/texlive?view=revision&revision=59973
Author:   karl
Date:     2021-07-18 19:16:29 +0200 (Sun, 18 Jul 2021)
Log Message:
-----------
checking fmt sharability across architectures

Modified Paths:
--------------
    trunk/Build/source/texk/web2c/doc/web2c.info
    trunk/Build/source/texk/web2c/doc/web2c.texi
    trunk/Master/tlpkg/doc/releng.txt

Added Paths:
-----------
    trunk/Master/tlpkg/bin/tl-check-fmtshare

Modified: trunk/Build/source/texk/web2c/doc/web2c.info
===================================================================
--- trunk/Build/source/texk/web2c/doc/web2c.info	2021-07-18 12:09:19 UTC (rev 59972)
+++ trunk/Build/source/texk/web2c/doc/web2c.info	2021-07-18 17:16:29 UTC (rev 59973)
@@ -1,4 +1,4 @@
-This is web2c.info, produced by makeinfo version 6.7 from web2c.texi.
+This is web2c.info, produced by makeinfo version 6.8 from web2c.texi.
 
 This file documents the installation and use of the programs in Web2c,
 an implementation of Donald Knuth's TeX system.
@@ -58,7 +58,7 @@
 
 This document describes how to install and use the programs in the Web2c
 implementation of the TeX system, especially for Unix systems.  It
-corresponds to Web2c version 2021, released in May 2021.
+corresponds to Web2c version 2021, released in July 2021.
 
 * Menu:
 
@@ -83,7 +83,7 @@
 1 Introduction
 **************
 
-This manual corresponds to version 2021 of Web2c, released in May 2021.
+This manual corresponds to version 2021 of Web2c, released in July 2021.
 
    "Web2c" is the name of a TeX implementation, originally for Unix, but
 now also running under various operating systems.  By "TeX
@@ -774,33 +774,38 @@
 4.3.3 Hardware and memory dumps
 -------------------------------
 
-By default, memory dump files are generally sharable between
-architectures of different types; specifically, on machines of different
-endianness (*note (libc)Byte order::).  (This is a feature of the Web2c
-implementation, and is not true of all TeX implementations.)  If you
-specify '--disable-dump-share' to 'configure', however, memory dumps
-will be endian-dependent.
+By default, memory dump files are sharable between architectures of
+different types; specifically, on machines of different endianness
+(*note (libc)Byte order::) and with different word sizes (4-byte 'long'
+vs. 8-byte 'long').  This is a feature of the Web2c implementation, and
+is not true of all TeX implementations.
 
-   The reason to do this is speed.  To achieve endian-independence, the
-reading of memory dumps on LittleEndian architectures, such as PC's and
-DEC architectures, is somewhat slowed (all the multibyte values have to
-be swapped).  Usually, this is not noticeable, and the advantage of
-being able to share memory dumps across all platforms at a site far
-outweighs the speed loss.  But if you're installing Web2c for use on
-LittleEndian machines only, perhaps on a PC being used only by you, you
-may wish to get maximum speed.
+   The script 'tl-check-fmtshare' in the TeX Live source tree
+('Master/tlpkg/bin') provides a relatively easy way to test that a
+'.fmt' built on the local host can be loaded by a TeX engine built on
+some remote host.
 
+   If you specify '--disable-dump-share' to 'configure', however, memory
+dumps will be endian-dependent.  The reason to do this is speed.  To
+achieve endian-independence, the reading of memory dumps on LittleEndian
+architectures, such as PC's and DEC architectures, is somewhat slowed
+(all the multibyte values have to be swapped).  Usually, this is not
+noticeable, and the advantage of being able to share memory dumps across
+all platforms at a site far outweighs the speed loss.  But if you're
+trying to squeeze out every possible bit of performance, you may wish to
+do this.
+
    TeXnically, even without '--disable-dump-share', sharing of '.fmt'
 files cannot be guaranteed to work.  Floating-point values are always
 written in native format, and hence will generally not be readable
 across platforms.  Fortunately, TeX uses floating point only to
-represent glue ratios, and all common formats (plain, LaTeX, AMSTeX,
-...) do not do any glue setting at '.fmt'-creation time.  Metafont does
+represent glue ratios, and none of the common formats (plain, LaTeX,
+AMSTeX, ...) do any glue setting at '.fmt'-creation time.  Metafont does
 not use floating point in any dumped value at all.
 
    Incidentally, different memory dump files will never compare equal
-byte-for-byte, because the program always dumps the current date and
-time.  So don't be alarmed by just a few bytes difference.
+byte-for-byte, because the programs dump the current date and time.  So
+don't be alarmed by a few bytes difference.
 
    If you don't know what endianness your machine is, and you're
 curious, here is a little C program to tell you.  (The 'configure'
@@ -827,6 +832,9 @@
        exit (u.c[sizeof (long) - 1] == 1);
      }
 
+   You can add 'printf("long %d\n", sizeof(long));' to see the size of
+the 'long' data type.
+
 
 File: web2c.info,  Node: Editor invocation,  Next: \input filenames,  Prev: Memory dumps,  Up: Three programs
 
@@ -3502,7 +3510,7 @@
 * - starts option names:                 Option conventions.  (line  11)
 * -- starts option names:                Option conventions.  (line  11)
 * --disable-dump-share configure option: Hardware and memory dumps.
-                                                              (line   6)
+                                                              (line  17)
 * --disable-mf-nowin:                    Online Metafont graphics.
                                                               (line  65)
 * --enable-epsfwin:                      Online Metafont graphics.
@@ -3534,7 +3542,7 @@
 * -base=BASE:                            Determining the memory dump to use.
                                                               (line  15)
 * -base=DUMPNAME:                        Common options.      (line  39)
-* -change=CHFILE:                        mft invocation.      (line  63)
+* -change=CHFILE:                        mft invocation.      (line  62)
 * -charcode-format=TYPE:                 tftopl invocation.   (line  27)
 * -charcode-format=TYPE <1>:             vftovp invocation.   (line  30)
 * -cnf-line:                             Common options.      (line  25)
@@ -3596,7 +3604,7 @@
 * -shell-restricted:                     tex invocation.      (line 122)
 * -show-opcodes:                         dvitype invocation.  (line  52)
 * -strict:                               tangle invocation.   (line  46)
-* -style=MFTFILE:                        mft invocation.      (line  67)
+* -style=MFTFILE:                        mft invocation.      (line  66)
 * -T:                                    mpost invocation.    (line 111)
 * -terse:                                bibtex invocation.   (line  28)
 * -tex=TEXPROGRAM:                       mpost invocation.    (line 114)
@@ -3629,6 +3637,10 @@
 * .Xresources:                           Online Metafont graphics.
                                                               (line  69)
 * 2602gf:                                mf invocation.       (line  48)
+* 32-bit architecture:                   Hardware and memory dumps.
+                                                              (line   6)
+* 64-bit architecture:                   Hardware and memory dumps.
+                                                              (line   6)
 * 8 bit clean:                           Common options.      (line 110)
 * 8 bit clean output, specifying:        Common options.      (line 110)
 * 8-bit characters:                      TCX files.           (line   6)
@@ -3738,11 +3750,11 @@
 * byte position:                         pktype invocation.   (line  40)
 * byte position <1>:                     gftype invocation.   (line  76)
 * byte swapping:                         Hardware and memory dumps.
-                                                              (line  13)
+                                                              (line  17)
 * c-sources Makefile target:             Additional targets.  (line  12)
 * caveats for \input filenames:          \input filename caveats.
                                                               (line   6)
-* change files, and MFT:                 mft invocation.      (line  63)
+* change files, and MFT:                 mft invocation.      (line  62)
 * change files, and Tangle:              tangle invocation.   (line  11)
 * change files, and Weave:               weave invocation.    (line  14)
 * changing error messages style:         Common options.      (line  31)
@@ -3762,7 +3774,7 @@
                                                               (line  69)
 * cm.base:                               Initial Metafont.    (line  26)
 * cmbase.mf:                             Initial Metafont.    (line  26)
-* cmbase.mft:                            mft invocation.      (line  73)
+* cmbase.mft:                            mft invocation.      (line  72)
 * cmmf.base not recommended:             Initial Metafont.    (line  26)
 * color, in DVItoMP:                     dvitomp invocation.  (line  15)
 * comments, in TCX files:                TCX files.           (line  70)
@@ -3775,7 +3787,7 @@
                                                               (line   6)
 * Computer Modern fonts, and Troff:      mpost invocation.    (line  61)
 * Computer Modern macros:                Initial Metafont.    (line  26)
-* Computer Modern Typefaces, production of: mft invocation.   (line  73)
+* Computer Modern Typefaces, production of: mft invocation.   (line  72)
 * configuration:                         Installation.        (line   6)
 * configuration file reading:            Path searching.      (line   6)
 * configuration file values:             Runtime options.     (line   6)
@@ -3805,7 +3817,7 @@
 * Cweb:                                  WEB.                 (line  12)
 * CWEB:                                  WEB.                 (line  12)
 * date and time, in memory dumps:        Hardware and memory dumps.
-                                                              (line  30)
+                                                              (line  35)
 * debugging DVI utilities:               dvitype invocation.  (line  52)
 * debugging flags, specifying:           Common options.      (line  64)
 * decimal character codes, in TCX files: TCX files.           (line  77)
@@ -3847,7 +3859,7 @@
 * e response at error prompt:            Editor invocation.   (line   6)
 * e-circumflex:                          \charsubdef.         (line  20)
 * e-TeX:                                 TeX extensions.      (line  10)
-* e.mft:                                 mft invocation.      (line  73)
+* e.mft:                                 mft invocation.      (line  72)
 * EC fonts:                              tex invocation.      (line  41)
 * EC fonts <1>:                          mf invocation.       (line  24)
 * editing of bitmap fonts:               Font utilities available elsewhere.
@@ -3896,7 +3908,7 @@
 * floating-point arithmetic:             Compile-time options.
                                                               (line  21)
 * floating-point values:                 Hardware and memory dumps.
-                                                              (line  22)
+                                                              (line  27)
 * fmt file, determining:                 Determining the memory dump to use.
                                                               (line   6)
 * fmt files:                             Initial TeX.         (line   6)
@@ -3944,7 +3956,7 @@
 * gftype GF validation:                  gftype invocation.   (line   6)
 * gftype.web:                            Font utilities.      (line  11)
 * glue ratio representations:            Hardware and memory dumps.
-                                                              (line  22)
+                                                              (line  27)
 * glue, memory for:                      Runtime options.     (line  20)
 * glyph substitutions:                   MLTeX.               (line   6)
 * gray font:                             gftodvi invocation.  (line  29)
@@ -3954,7 +3966,7 @@
 * gsftopk:                               Font utilities available elsewhere.
                                                               (line  35)
 * Harbison, Samuel P.:                   Hardware and memory dumps.
-                                                              (line  34)
+                                                              (line  39)
 * hardware and memory dumps:             Hardware and memory dumps.
                                                               (line   6)
 * headerbyte information:                tftopl invocation.   (line  95)
@@ -4041,6 +4053,8 @@
                                                               (line   6)
 * location of texmf.cnf in TeX Live:     Runtime options.     (line  56)
 * log file, BibTeX:                      bibtex invocation.   (line  14)
+* long data type, size of:               Hardware and memory dumps.
+                                                              (line   6)
 * Lua:                                   TeX extensions.      (line  30)
 * LuaTeX:                                TeX extensions.      (line  30)
 * machine dependencies:                  Hardware and memory dumps.
@@ -4070,7 +4084,7 @@
 * memory dumps and hardware:             Hardware and memory dumps.
                                                               (line   6)
 * memory dumps, contain date and time:   Hardware and memory dumps.
-                                                              (line  30)
+                                                              (line  35)
 * memory dumps, creating:                Creating memory dumps.
                                                               (line   6)
 * mems Make target:                      Additional targets.  (line  34)
@@ -4095,7 +4109,7 @@
                                                               (line  15)
 * MetaPost input files:                  mpost invocation.    (line  26)
 * MetaPost invocation:                   mpost invocation.    (line   6)
-* MetaPost source, prettyprinting:       mft invocation.      (line  79)
+* MetaPost source, prettyprinting:       mft invocation.      (line  77)
 * MetaPost, initial:                     Initial MetaPost.    (line   6)
 * MetaPost, TeX, and Metafont:           Three programs.      (line   6)
 * metatype1:                             Font utilities available elsewhere.
@@ -4212,7 +4226,7 @@
 * plain.bst:                             Basic BibTeX style files.
                                                               (line  11)
 * plain.fmt:                             Initial TeX.         (line  13)
-* plain.mft:                             mft invocation.      (line  67)
+* plain.mft:                             mft invocation.      (line  66)
 * pltotf:                                pltotf invocation.   (line   6)
 * pool file, writing:                    tangle invocation.   (line  16)
 * Poole, Simon:                          Online Metafont graphics.
@@ -4314,7 +4328,7 @@
 * starting page:                         dvicopy invocation.  (line  28)
 * starting page <1>:                     dvitype invocation.  (line  46)
 * Steele Jr., Guy L.:                    Hardware and memory dumps.
-                                                              (line  34)
+                                                              (line  39)
 * stopping at the first error:           Common options.      (line  46)
 * strategy, overall:                     Introduction.        (line  16)
 * string numbers, displaying:            pooltype invocation. (line   6)
@@ -4321,7 +4335,7 @@
 * string pool, writing:                  tangle invocation.   (line  16)
 * string representation:                 pooltype invocation. (line  30)
 * style design, for BibTeX:              bibtex invocation.   (line  49)
-* style files:                           mft invocation.      (line  67)
+* style files:                           mft invocation.      (line  66)
 * substitutions of font glyphs:          MLTeX.               (line   6)
 * sun:                                   Online Metafont graphics.
                                                               (line  42)
@@ -4333,7 +4347,7 @@
                                                               (line  43)
 * swap space, as array limit:            Runtime options.     (line  45)
 * swapping bytes:                        Hardware and memory dumps.
-                                                              (line  13)
+                                                              (line  17)
 * syntax of TCX files:                   TCX files.           (line  65)
 * system C library function:             Shell escapes.       (line   6)
 * system command:                        Shell escapes.       (line   6)
@@ -4394,8 +4408,10 @@
 * tftopl:                                tftopl invocation.   (line   6)
 * three programs:                        Three programs.      (line   6)
 * time and date, in memory dumps:        Hardware and memory dumps.
-                                                              (line  30)
+                                                              (line  35)
 * title font:                            gftodvi invocation.  (line  33)
+* tl-check-fmtshare:                     Hardware and memory dumps.
+                                                              (line  12)
 * tokenization, and \input filenames:    \input filename caveats.
                                                               (line   6)
 * toolkits, X:                           Online Metafont graphics.
@@ -4505,78 +4521,78 @@
 
 Tag Table:
 Node: Top2759
-Node: Introduction3829
-Node: Installation7165
-Node: configure options9209
-Node: Compile-time options11231
-Node: Additional targets12333
-Node: Triptrap13610
-Node: Commonalities15113
-Node: Option conventions15646
-Node: Common options16869
-Node: Path searching21487
-Node: Output file location22459
-Node: Three programs23761
-Node: Runtime options24468
-Node: Initial and virgin27609
-Node: Memory dumps28558
-Node: Creating memory dumps29010
-Node: Determining the memory dump to use30217
-Node: Hardware and memory dumps31794
-Node: Editor invocation34240
-Node: \input filenames35103
-Node: \input quoted filename36654
-Node: \input braced filename37431
-Node: \input filename caveats38361
-Node: TeX39875
-Node: tex invocation41043
-Node: Initial TeX48029
-Node: Formats49393
-Node: Languages and hyphenation52210
-Node: MLTeX52640
-Node: \charsubdef54268
-Node: \tracingcharsubdef56591
-Node: TCX files57166
-Node: patgen invocation62606
-Node: Shell escapes63313
-Node: IPC and TeX66893
-Node: TeX extensions67463
-Node: Metafont69492
-Node: mf invocation70780
-Node: Initial Metafont74617
-Node: Modes76247
-Node: Online Metafont graphics78497
-Node: gftodvi invocation81922
-Node: mft invocation84742
-Node: MetaPost88740
-Node: mpost invocation89576
-Node: Initial MetaPost94612
-Node: dvitomp invocation95534
-Node: BibTeX96199
-Node: bibtex invocation96560
-Node: Basic BibTeX style files99034
-Node: WEB100364
-Node: tangle invocation101573
-Node: weave invocation103697
-Node: pooltype invocation105104
-Node: DVI utilities106234
-Node: dvicopy invocation107168
-Node: dvitype invocation108453
-Node: dvitype output example110784
-Node: Font utilities113837
-Node: Font file formats115017
-Node: gftopk invocation118284
-Node: pktogf invocation119475
-Node: pktype invocation120641
-Node: gftype invocation123464
-Node: tftopl invocation127967
-Node: pltotf invocation132560
-Node: vftovp invocation133611
-Node: vptovf invocation135846
-Node: Font utilities available elsewhere136873
-Node: Legalisms139078
-Node: References140525
-Node: Index144658
+Node: Introduction3830
+Node: Installation7167
+Node: configure options9211
+Node: Compile-time options11233
+Node: Additional targets12335
+Node: Triptrap13612
+Node: Commonalities15115
+Node: Option conventions15648
+Node: Common options16871
+Node: Path searching21489
+Node: Output file location22461
+Node: Three programs23763
+Node: Runtime options24470
+Node: Initial and virgin27611
+Node: Memory dumps28560
+Node: Creating memory dumps29012
+Node: Determining the memory dump to use30219
+Node: Hardware and memory dumps31796
+Node: Editor invocation34546
+Node: \input filenames35409
+Node: \input quoted filename36960
+Node: \input braced filename37737
+Node: \input filename caveats38667
+Node: TeX40181
+Node: tex invocation41349
+Node: Initial TeX48335
+Node: Formats49699
+Node: Languages and hyphenation52516
+Node: MLTeX52946
+Node: \charsubdef54574
+Node: \tracingcharsubdef56897
+Node: TCX files57472
+Node: patgen invocation62912
+Node: Shell escapes63619
+Node: IPC and TeX67199
+Node: TeX extensions67769
+Node: Metafont69798
+Node: mf invocation71086
+Node: Initial Metafont74923
+Node: Modes76553
+Node: Online Metafont graphics78803
+Node: gftodvi invocation82228
+Node: mft invocation85048
+Node: MetaPost89046
+Node: mpost invocation89882
+Node: Initial MetaPost94918
+Node: dvitomp invocation95840
+Node: BibTeX96505
+Node: bibtex invocation96866
+Node: Basic BibTeX style files99340
+Node: WEB100670
+Node: tangle invocation101879
+Node: weave invocation104003
+Node: pooltype invocation105410
+Node: DVI utilities106540
+Node: dvicopy invocation107474
+Node: dvitype invocation108759
+Node: dvitype output example111090
+Node: Font utilities114143
+Node: Font file formats115323
+Node: gftopk invocation118590
+Node: pktogf invocation119781
+Node: pktype invocation120947
+Node: gftype invocation123770
+Node: tftopl invocation128273
+Node: pltotf invocation132866
+Node: vftovp invocation133917
+Node: vptovf invocation136152
+Node: Font utilities available elsewhere137179
+Node: Legalisms139384
+Node: References140831
+Node: Index144964
 
 End Tag Table
 

Modified: trunk/Build/source/texk/web2c/doc/web2c.texi
===================================================================
--- trunk/Build/source/texk/web2c/doc/web2c.texi	2021-07-18 12:09:19 UTC (rev 59972)
+++ trunk/Build/source/texk/web2c/doc/web2c.texi	2021-07-18 17:16:29 UTC (rev 59973)
@@ -12,7 +12,7 @@
 @end tex
 
 @set version 2021
- at set month-year May 2021
+ at set month-year July 2021
 
 @c Define new indices for commands in auxiliary files, filenames, and options.
 @defcodeindex cm
@@ -789,24 +789,34 @@
 @cindex machine dependencies
 @cindex architecture dependencies
 @cindex dependencies, hardware
- at opindex --disable-dump-share configure @r{option}
-By default, memory dump files are generally sharable between
-architectures of different types; specifically, on machines of different
-endianness (@pxref{Byte order,,,libc,GNU C Library}).  (This is a
+ at cindex 32-bit architecture
+ at cindex 64-bit architecture
+ at cindex @code{long} data type, size of
+By default, memory dump files are sharable between architectures of
+different types; specifically, on machines of different endianness
+(@pxref{Byte order,,, libc, GNU C Library}) and with different word
+sizes (4-byte @code{long} vs.@: 8-byte @code{long}).  This is a
 feature of the Web2c implementation, and is not true of all @TeX{}
-implementations.)  If you specify @samp{--disable-dump-share} to
- at code{configure}, however, memory dumps will be endian-dependent.
+implementations.
 
+ at pindex tl-check-fmtshare
+The script @code{tl-check-fmtshare} in the @TeX{} Live source tree
+(@file{Master/tlpkg/bin}) provides a relatively easy way to test that
+a @file{.fmt} built on the local host can be loaded by a @TeX{} engine
+built on some remote host.
+
 @cindex byte swapping
 @cindex swapping bytes
-The reason to do this is speed.  To achieve endian-independence, the
-reading of memory dumps on LittleEndian architectures, such as PC's and
-DEC architectures, is somewhat slowed (all the multibyte values have to
-be swapped).  Usually, this is not noticeable, and the advantage of
-being able to share memory dumps across all platforms at a site far
-outweighs the speed loss.  But if you're installing Web2c for use on
-LittleEndian machines only, perhaps on a PC being used only by you, you
-may wish to get maximum speed.
+ at opindex --disable-dump-share configure @r{option}
+If you specify @samp{--disable-dump-share} to @code{configure},
+however, memory dumps will be endian-dependent.  The reason to do this
+is speed.  To achieve endian-independence, the reading of memory dumps
+on LittleEndian architectures, such as PC's and DEC architectures, is
+somewhat slowed (all the multibyte values have to be swapped).
+Usually, this is not noticeable, and the advantage of being able to
+share memory dumps across all platforms at a site far outweighs the
+speed loss.  But if you're trying to squeeze out every possible bit of
+performance, you may wish to do this.
 
 @cindex floating-point values
 @cindex glue ratio representations
@@ -814,8 +824,8 @@
 @file{.fmt} files cannot be guaranteed to work.  Floating-point values
 are always written in native format, and hence will generally not be
 readable across platforms.  Fortunately, @TeX{} uses floating point
-only to represent glue ratios, and all common formats (plain,
- at LaTeX{}, AMS at TeX{}, @dots{}) do not do any glue setting at
+only to represent glue ratios, and none of the common formats (plain,
+ at LaTeX{}, AMS at TeX{}, @dots{}) do any glue setting at
 @file{.fmt}-creation time.  Metafont does not use floating point in
 any dumped value at all.
 
@@ -823,8 +833,8 @@
 @cindex time and date, in memory dumps
 @cindex memory dumps, contain date and time
 Incidentally, different memory dump files will never compare equal
-byte-for-byte, because the program always dumps the current date and
-time. So don't be alarmed by just a few bytes difference.
+byte-for-byte, because the programs dump the current date and time. So
+don't be alarmed by a few bytes difference.
 
 @cindex Harbison, Samuel P.
 @cindex Steele Jr., Guy L.
@@ -855,7 +865,10 @@
 @}
 @end example
 
+You can add @code{printf("long %d\n", sizeof(long));} to see the size
+of the @code{long} data type.
 
+
 @node Editor invocation
 @section Editor invocation
 

Added: trunk/Master/tlpkg/bin/tl-check-fmtshare
===================================================================
--- trunk/Master/tlpkg/bin/tl-check-fmtshare	                        (rev 0)
+++ trunk/Master/tlpkg/bin/tl-check-fmtshare	2021-07-18 17:16:29 UTC (rev 59973)
@@ -0,0 +1,268 @@
+#!/bin/sh
+# Public domain. Originally written by Karl Berry, 2021.
+# Test that a .fmt file built on one system can be used on another.
+
+version='$Id$'
+renice 20 $$ >/dev/null 2>&1
+unset CDPATH
+LC_ALL=C; export LC_ALL
+umask 077 # since we write in a tmp dir (unsafe still, but oh well)
+
+fmtutil_path=`cd /tmp && which fmtutil-sys 2>/dev/null`
+if test -z "$fmtutil_path"; then
+  echo "$0: no fmtutil-sys in PATH, goodbye." >&2
+  exit 1
+fi
+bindir=`dirname "$fmtutil_path"`
+mydir=`cd \`dirname $0\` && pwd`	# Master/tlpkg/bin
+#echo "fmtutil_path=$fmtutil_path, bindir=$bindir, mydir=$mydir"
+
+Master=`cd "$bindir"/../.. && pwd`
+enginedir=`cd "$mydir"/../../../Build/source/Work/texk/web2c 2>/dev/null&& pwd`
+outdir=/tmp/fmtshare.`id -u`
+default_fmts="tex.fmt pdftex.fmt pdflatex.fmt xetex.fmt xelatex.fmt
+		     luatex.fmt optex.fmt euptex.fmt" # not lualatex, see below
+fmts=
+remote=
+
+usage="Usage: $0 [OPTION]... RHOST:RDIR
+Build TeX .fmt files locally, then copy them to RHOST:RDIR and try to
+load them with the TeX engines there.
+
+The idea is to support testing for .fmts being sharable among different
+system architectures, as intended; for instance, 4-byte long vs. 8-byte
+long and BigEndian vs. LittleEndian.
+
+The further idea is to be able to test new binaries in the Build
+directory (since that's where the compiled sources change), while using
+support files from a Master directory (since that's where they are
+updated).
+
+fmtutil-sys is run locally (found using PATH) to discern the engine and
+full command line used for building the given .fmts.
+
+The remote execution is done with \"ssh -n RHOST && cd RDIR && ...\",
+so ssh has to be working.
+
+The default for the Build directory is to be relative to where this
+script is executed. The default for the Master directory is to be
+relative to where fmtutil-sys is found in PATH. These may or may not be
+the same.
+
+The output directory is not cleaned before starting, in case older logs
+are helpful in debugging new failures.
+
+Special case: unfortunately, lualatex.fmt cannot be easily tested, and
+it is omitted by default.  Loading the fmt requires finding many Lua
+files, which are unlikely to be available on the remote machine.  The
+script does not try to handle this.
+
+Options (must use --, must use space to separate option from value, sorry):
+--Master MDIR     use MDIR for local support files
+                    [$Master]
+--enginedir EDIR  use EDIR for local engine binaries
+                    [$enginedir]
+--outdir ODIR     build locally in ODIR [$outdir]
+--fmt FMT         build FMT; can be given more than once
+                    [$default_fmts]
+
+--help            display this help and exit
+--version         display version information and exit
+
+Doc: https://tug.org/texinfohtml/web2c.html#Hardware-and-memory-dumps
+Bug reports, discussion: tex-live at tug.org
+Version: $version"
+
+while test $# -gt 0; do
+ case $1 in
+  --Master)    shift; Master=$1;;
+  --enginedir) shift; enginedir=$1;;
+  --fmt)       shift; fmts="$fmts $1";;
+  --outdir)    shift; outdir=$1;;
+  #
+  --help)      echo "$usage"; exit 0;;
+  --version)   echo "$version"; exit 0;;
+  -*)          echo "$0: goodbye, unrecognized option: $1" >&2; exit 1;;
+  #
+  *)           if test -n "$remote"; then
+                 echo "$0: remote argument already seen: $remote" >&2
+                 echo "$0:   now have second: $1" >&2
+                 exit 1
+               fi
+               remote=$1;;
+ esac
+ shift
+done
+
+if test -z "$remote"; then
+  echo "$0: no remote argument given, goodbye." >&2
+  exit 1
+elif echo "$remote" | grep -v : >/dev/null; then
+  echo "$0: remote argument must have form HOST:DIR, not like: $remote" >&2
+  exit 1
+elif test `echo "$remote" | tr -cd : | wc -c` != 1 >/dev/null; then
+  # not worth the fancier parsing until we need it.
+  echo "$0: remote argument must have only one colon: $remote" >&2
+  exit 1
+fi
+remotesys=`echo "$remote" | sed 's/:.*//'`
+remotedir=`echo "$remote" | sed 's/.*://'`
+#
+if test ! -d "$Master/texmf-dist/web2c"; then
+  echo "$0: no texmf-dist/web2c subdir of Master: $Master" >&2
+  ls "$Master" >&2
+  exit 1
+fi
+#
+if test ! -d "$enginedir"; then
+  echo "$0: enginedir not a directory: $enginedir" >&2
+  ls "$enginedir" >&2
+  exit 1
+fi
+
+# create output directory if needed.
+test -d "$outdir" || mkdir "$outdir" || exit 1
+
+# default fmt list.
+test -z "$fmts" && fmts=$default_fmts
+
+# Our function to create a format locally:
+#   mkfmt FMT ENGINEDIR MASTERDIR OUTDIR
+# Generates FMT using the binary from ENGINEDIR,
+# support files from MASTERDIR, and leaving FMT in OUTDIR.
+# 
+# Leaves current directory as OUTDIR.
+#
+# On success, returns zero and outputs two lines to stdout:
+# 1) the full path to the newly-generated fmt file; and
+# 2) the basename of the engine binary that was used to build it.
+# 
+# On failure, returns nonzero and outputs nothing to stdout; issues
+# diagnostics to stderr.
+# 
+mkfmt ()
+{
+  fmt=$1
+  enginedir=$2
+  Master=$3
+  outdir=$4
+
+  # Our goal is to build a .fmt using a binary from the Build tree
+  # while still using the support files from the Master tree.
+  # The idea being that after changing the source and rebuilding the
+  # binary, we want to easily test whether the new binary is ok.
+  #
+  # First, we run fmtutil-sys (assumed to be in PATH) without generating
+  # anything (--dry-run), to garner the command line needed to build # FMT.
+  # 
+  fcmd="fmtutil-sys --dry-run --no-engine-subdir --fmtdir $outdir --byfmt $fmt"
+  ffot=$outdir/`basename $fmt .fmt`-fmtutil.fot
+  #
+  echo "$0: running: $fcmd" >"$ffot"
+  if $fcmd >>"$ffot" 2>&1; then :;
+    echo "$0: could not get cmdline to build $fmt" >&2
+    echo "$0:  fmtutil command failed: $fcmd" >&2
+    echo "$0:  see transcript: $ffot" >&2
+    return 1
+  fi
+  
+  # Extract the build command from the fmtutil output, which has a line like:
+  #   fmtutil: running `pdftex -ini [more options] *pdfetex.ini' ...
+  # We want what's between the quotes. Format names and options can't
+  # contain quote or other special characters.
+  lq='`'
+  rq="'"
+  ecmd=`sed -n "s/^fmtutil.*running $lq\(.*\)$rq.*/\1/p" "$ffot"`
+  if test -z "$ecmd"; then
+    echo "$0: could not extract cmdline to build $fmt" >&2
+    echo "$0:  from fmtutil output; see: $ffot" >&2
+    return 1
+  fi
+  
+  # Generate everything in OUTDIR.
+  cd "$outdir" || return 1
+  
+  # Set environment variables so the given Master tree is used, and
+  # prepend the given engine directory to PATH, and run the command.
+  env="env PATH=$enginedir:$PATH \
+         TEXMFCNF=$Master/texmf-dist/web2c \
+         TEXMFROOT=$Master"
+  efot=$outdir/`basename $fmt .fmt`-engine.fot
+  echo "$0: running: $ecmd" >"$efot"
+  if $env $ecmd >>"$efot" 2>&1 && test -s "$outdir/$fmt"; then
+    sed -n '2p;q' "$efot"
+    echo "$outdir/$fmt"
+    echo "$ecmd" | awk '{print $1}' # engine that was used
+  else
+    echo "$0: could not build $fmt" >&2
+    echo "$0: engine command failed: $ecmd" >&2
+    echo "$0: see transcript: $efot" >&2
+    return 1
+  fi
+}
+
+# 
 Loop over all given fmts.
+#
+for fmt in $fmts; do
+  if echo "$fmt" | grep '\.fmt$' >/dev/null; then :; else
+    echo "$0: fmt does not end with .fmt, goodbye: $fmt" >&2
+    exit 1
+  fi
+  
+  # first build fmt on local machine.
+  set - `mkfmt "$fmt" "$enginedir" "$Master" "$outdir"`
+  fmtfile=$1
+  engine=$2
+  #
+  if test -z "$fmtfile"; then
+    # error messages already given, but give another just to be sure.
+    echo "$0: mkfmt failed: $fmt " \
+              "(enginedir $enginedir) (Master $Master) (outdir $outdir)" >&2  
+    exit 1
+  elif test -z "$engine"; then
+    echo "$0: should not happen, mkfmt returned fmtfile: $fmtfile" >&2
+    echo "$0: but no engine value?!" >&2
+    exit 1
+  elif test ! -s "$fmtfile"; then
+    echo "$0: should not happen, returned fmtfile is empty: $fmtfile" >&2
+    exit 1
+  fi
+  echo "$0: (`date`) built fmtfile: `ls -l $fmtfile`"
+
+  # The TeX \command to exit a job immediately. Assume a LaTeX fmt
+  # if "latex" is in the name, else plain (enough).
+  if echo "$fmt" | grep latex >/dev/null; then
+    endcmd='\stop'
+  else
+    endcmd='\end'
+  fi
+
+  # copy to remote machine.
+  echo "$0: copying $fmtfile to $remotesys:$remotedir..."
+  scp -pq "$fmtfile" "$remotesys:$remotedir" || exit 1
+
+  # load on remote machine.
+  echo "$0: running $engine in $remotesys:$remotedir..."
+  #
+  # On the remote side, we need to find a texmf.cnf or pdftex gets the
+  # mysterious "Must increase the hyph_size"; assume the
+  # TL source directory structure to find kpathsea/texmf.cnf.
+  # 
+  # Do not try to dynamically create the .fmt.
+  # 
+  remoteenv="env MKTEXFMT=0 TEXMFCNF=../../../texk/kpathsea"
+  remotecmd="cd $remotedir && $remoteenv ./$engine -fmt=./$fmt '$endcmd'"
+  #
+  rfot=$outdir/`basename $fmt .fmt`-$remotesys.fot
+  echo "$0: running on $remotesys: $remotecmd" >$rfot
+  ssh -n $remotesys "$remotecmd" </dev/null >>$rfot 2>&1
+  if test $? -ne 0; then
+    echo "$0: fmt load failed on $remotesys: $fmt" >&2
+    echo "$0: see transcript: $rfot" >&2
+    exit 1
+  else
+    echo "$0: fmt load ok on $remotesys: $fmt"
+  fi
+done
+
+exit 0


Property changes on: trunk/Master/tlpkg/bin/tl-check-fmtshare
___________________________________________________________________
Added: svn:executable
## -0,0 +1 ##
+*
\ No newline at end of property
Added: svn:keywords
## -0,0 +1 ##
+Date Author Id Revision
\ No newline at end of property
Modified: trunk/Master/tlpkg/doc/releng.txt
===================================================================
--- trunk/Master/tlpkg/doc/releng.txt	2021-07-18 12:09:19 UTC (rev 59972)
+++ trunk/Master/tlpkg/doc/releng.txt	2021-07-18 17:16:29 UTC (rev 59973)
@@ -273,11 +273,20 @@
     tl-update-txi
   Don't forget to commit everything (texmf-dist/doc, Build/source, ...).
 
-N. Really test during pretest (really! don't leave it for Knuth to do!):
-  .fmt's sharable across endian-different platforms (including luatex).
-  interactive mf '\drawdot (50,50); showit;'  % and from * prompt
+N. Really test during and at end of pretest that
+    (really! don't leave it for post-release failure)
+  .fmt's sharable are across endian/word-different platforms,
+  including luatex:
+    tlpkg/bin/tl-check-fmtshare RHOST:RDIR
+  where RHOST is a host you can ssh to with different specifications
+  than the local host, and RDIR is a TL:Build/source/Work/texk/web2c/
+  build directory. (Must build TL there first before running, naturally.)
+  Nelson Beebe has a wide variety of hosts available.
 
-O. Two days before the final pretest (i.e., to-be-published) build:
+O. Test that basic interactive Metafont is still working:
+    mf '\drawdot (50,50); showit;'  % and from * prompt
+
+W. Two days before the final pretest (i.e., to-be-published) build:
   Put date into NEWS and ChangeLog files above.
   Update Build/source/tardate.ac && reautoconf.
   tlmgr version number in texmf-dist/scripts/texlive/NEWS.
@@ -291,13 +300,13 @@
   Tell developers no more commits: tldev +
     hirata, hironori, luigi, mojca, takuji, 
 
-P. The day before the final pretest:
+X. The day before the final pretest:
   In cron.tl, catalogue_compare=false to avoid skew.
   In tl-update-tlnet, option adjustrepo 1, else install-tl-unx won't go to ctan
   Update Master/source (if not earlier):
     xz wget w32tex-src texlive*source mactexdoc install-tl-unx.tar.gz 
 
-Q. The day of the final tlnet-from-branch build, which may or may not
+Y. The day of the final tlnet-from-branch build, which may or may not
    also be the day of the final tlpretest-from-trunk build:
  Set frozen=>1 in branch TLConfig.pm.
  Freeze of tlnet should happen at that night's rebuild.



More information about the tex-live-commits mailing list.