texlive[60569] Build/source: hitexdir: initial import, omitted from
commits+karl at tug.org
commits+karl at tug.org
Tue Sep 21 18:46:46 CEST 2021
Revision: 60569
http://tug.org/svn/texlive?view=revision&revision=60569
Author: karl
Date: 2021-09-21 18:46:46 +0200 (Tue, 21 Sep 2021)
Log Message:
-----------
hitexdir: initial import, omitted from build by default
Modified Paths:
--------------
trunk/Build/source/configure
trunk/Build/source/doc/tlbuild.info
trunk/Build/source/doc/tlbuild.texi
trunk/Build/source/libs/configure
trunk/Build/source/texk/configure
trunk/Build/source/texk/web2c/ChangeLog
trunk/Build/source/texk/web2c/Makefile.am
trunk/Build/source/texk/web2c/Makefile.in
trunk/Build/source/texk/web2c/ac/web2c.ac
trunk/Build/source/texk/web2c/c-auto.in
trunk/Build/source/texk/web2c/configure
trunk/Build/source/texk/web2c/configure.ac
trunk/Build/source/utils/configure
Added Paths:
-----------
trunk/Build/source/texk/web2c/hitexdir/
trunk/Build/source/texk/web2c/hitexdir/am/
trunk/Build/source/texk/web2c/hitexdir/am/hitex.am
trunk/Build/source/texk/web2c/hitexdir/format.w
trunk/Build/source/texk/web2c/hitexdir/hitex.w
trunk/Build/source/texk/web2c/hitexdir/htex.w
trunk/Build/source/texk/web2c/hitexdir/lex.sed
trunk/Build/source/texk/web2c/hitexdir/yacc.sed
Modified: trunk/Build/source/configure
===================================================================
--- trunk/Build/source/configure 2021-09-20 23:48:29 UTC (rev 60568)
+++ trunk/Build/source/configure 2021-09-21 16:46:46 UTC (rev 60569)
@@ -852,6 +852,7 @@
enable_euptex
enable_euptex_synctex
enable_aleph
+enable_hitex
enable_pdftex
enable_pdftex_synctex
enable_luatex
@@ -1669,6 +1670,7 @@
--disable-euptex do not compile and install e-upTeX
--disable-euptex-synctex build e-upTeX without SyncTeX support
--disable-aleph do not compile and install Aleph
+ --enable-hitex compile and install HiTeX
--disable-pdftex do not compile and install pdfTeX
--disable-pdftex-synctex build pdfTeX without SyncTeX support
--disable-luatex do not compile and install LuaTeX
@@ -5488,6 +5490,21 @@
*) :
enable_aleph=yes ;;
esac
+# Check whether --enable-hitex was given.
+if test ${enable_hitex+y}
+then :
+ enableval=$enable_hitex;
+fi
+case $enable_hitex in #(
+ yes | no) :
+ ;; #(
+ *) :
+ enable_hitex=no ;;
+esac
+
+test "x$enable_web2c:$enable_hitex" = xyes:yes && {
+ need_zlib=yes
+}
# Check whether --enable-pdftex was given.
if test ${enable_pdftex+y}
then :
Modified: trunk/Build/source/doc/tlbuild.info
===================================================================
(Binary files differ)
Modified: trunk/Build/source/doc/tlbuild.texi
===================================================================
--- trunk/Build/source/doc/tlbuild.texi 2021-09-20 23:48:29 UTC (rev 60568)
+++ trunk/Build/source/doc/tlbuild.texi 2021-09-21 16:46:46 UTC (rev 60569)
@@ -1619,6 +1619,7 @@
@menu
* Adding a new program module::
+* Adding a new engine::
* Adding a new generic library module::
* Adding a new @TeX{}-specific library module::
@end menu
@@ -1751,15 +1752,38 @@
After final success, don't forget to commit. (Or email the TL
maintainers with the patch.)
+ at node Adding a new engine
+ at subsection Adding a new engine
+
+ at cindex adding a new engine
@cindex engine, adding new
-Caveat: adding a new @TeX{} engine is not completely different, but
-it's not all that similar, either. In that case, the work is done
-inside a new subdirectory of @file{texk/web2c/}. Many things are
-common to all the engines, other things need to be copied and possibly
-modified for each one, yet others are unique to each. No general
-recipe is possible.
+Adding a new @TeX{} engine is not completely different from adding a
+program, but it's not all that similar, either. In this case, the main
+work is done by creating a new subdirectory of @file{texk/web2c/} for
+the engine. The subdirectory is conventionally named ending in
+ at file{dir}, like @file{pdftexdir} and @file{xetexdir}.
+The source files for the new engine should be put in this
+ at file{@var{newengine}dir} subdirectory. Also, a file
+ at file{@var{newengine}dir/am/@var{newengine}.am} (e.g.,
+ at file{pdftexdir/am/pdftex.am} is needed with the Makefile fragment
+needed to build it.
+
+The overall @file{web2c/Makefile.am} needs to have an @file{include}
+statement added to insert that @file{@var{newengine}.am} file.
+
+In @file{web2c/ac/web2c.ac}, a line needs to be added in the
+definition of the @code{kpse_tex_progs} variable to include it in the
+build. That line specifies whether the new engine is built by default,
+and the additional libraries requires.
+
+For examples of building engines in CWEB, you can check the existing
+ at file{hitexdir} and @file{mplibdir} directories; these are somewhat
+simpler than Lua at TeX{}. Of course, every engine will have its own
+unique features and requirements, so existing examples will only take
+you so far.
+
@node Adding a new generic library module
@subsection Adding a new generic library module
Modified: trunk/Build/source/libs/configure
===================================================================
--- trunk/Build/source/libs/configure 2021-09-20 23:48:29 UTC (rev 60568)
+++ trunk/Build/source/libs/configure 2021-09-21 16:46:46 UTC (rev 60569)
@@ -765,6 +765,7 @@
enable_euptex
enable_euptex_synctex
enable_aleph
+enable_hitex
enable_pdftex
enable_pdftex_synctex
enable_luatex
@@ -1560,6 +1561,7 @@
--disable-euptex do not compile and install e-upTeX
--disable-euptex-synctex build e-upTeX without SyncTeX support
--disable-aleph do not compile and install Aleph
+ --enable-hitex compile and install HiTeX
--disable-pdftex do not compile and install pdfTeX
--disable-pdftex-synctex build pdfTeX without SyncTeX support
--disable-luatex do not compile and install LuaTeX
@@ -4560,6 +4562,21 @@
*) :
enable_aleph=yes ;;
esac
+# Check whether --enable-hitex was given.
+if test ${enable_hitex+y}
+then :
+ enableval=$enable_hitex;
+fi
+case $enable_hitex in #(
+ yes | no) :
+ ;; #(
+ *) :
+ enable_hitex=no ;;
+esac
+
+test "x$enable_web2c:$enable_hitex" = xyes:yes && {
+ need_zlib=yes
+}
# Check whether --enable-pdftex was given.
if test ${enable_pdftex+y}
then :
Modified: trunk/Build/source/texk/configure
===================================================================
--- trunk/Build/source/texk/configure 2021-09-20 23:48:29 UTC (rev 60568)
+++ trunk/Build/source/texk/configure 2021-09-21 16:46:46 UTC (rev 60569)
@@ -765,6 +765,7 @@
enable_euptex
enable_euptex_synctex
enable_aleph
+enable_hitex
enable_pdftex
enable_pdftex_synctex
enable_luatex
@@ -1560,6 +1561,7 @@
--disable-euptex do not compile and install e-upTeX
--disable-euptex-synctex build e-upTeX without SyncTeX support
--disable-aleph do not compile and install Aleph
+ --enable-hitex compile and install HiTeX
--disable-pdftex do not compile and install pdfTeX
--disable-pdftex-synctex build pdfTeX without SyncTeX support
--disable-luatex do not compile and install LuaTeX
@@ -4560,6 +4562,21 @@
*) :
enable_aleph=yes ;;
esac
+# Check whether --enable-hitex was given.
+if test ${enable_hitex+y}
+then :
+ enableval=$enable_hitex;
+fi
+case $enable_hitex in #(
+ yes | no) :
+ ;; #(
+ *) :
+ enable_hitex=no ;;
+esac
+
+test "x$enable_web2c:$enable_hitex" = xyes:yes && {
+ need_zlib=yes
+}
# Check whether --enable-pdftex was given.
if test ${enable_pdftex+y}
then :
Modified: trunk/Build/source/texk/web2c/ChangeLog
===================================================================
--- trunk/Build/source/texk/web2c/ChangeLog 2021-09-20 23:48:29 UTC (rev 60568)
+++ trunk/Build/source/texk/web2c/ChangeLog 2021-09-21 16:46:46 UTC (rev 60569)
@@ -1,3 +1,9 @@
+2021-09-21 Karl Berry <karl at freefriends.org>
+
+ * ac/web2c.ac (kpse_tex_progs): include hitex, disabled by default.
+ * Makefile.am: include hitexdir/am/hitex.am.
+ * hitexdir: new directory.
+
2021-09-16 Andreas Scherer <https://ascherer.github.io>
* Makefile.in,
Modified: trunk/Build/source/texk/web2c/Makefile.am
===================================================================
--- trunk/Build/source/texk/web2c/Makefile.am 2021-09-20 23:48:29 UTC (rev 60568)
+++ trunk/Build/source/texk/web2c/Makefile.am 2021-09-21 16:46:46 UTC (rev 60569)
@@ -274,6 +274,9 @@
## e-upTeX
include $(srcdir)/euptexdir/am/euptex.am
+## hiTeX
+include $(srcdir)/hitexdir/am/hitex.am
+
## pdfTeX
include $(srcdir)/pdftexdir/am/libpdftex.am
include $(srcdir)/pdftexdir/am/pdftex.am
@@ -289,12 +292,9 @@
include $(srcdir)/luatexdir/am/luaffi.am
include $(srcdir)/luatexdir/am/luatex.am
-
## luahbTeX
include $(srcdir)/luatexdir/am/luaharfbuzz.am
-
-
## XeTeX
include $(srcdir)/xetexdir/am/xetex.am
Modified: trunk/Build/source/texk/web2c/Makefile.in
===================================================================
--- trunk/Build/source/texk/web2c/Makefile.in 2021-09-20 23:48:29 UTC (rev 60568)
+++ trunk/Build/source/texk/web2c/Makefile.in 2021-09-21 16:46:46 UTC (rev 60569)
@@ -112,7 +112,8 @@
mfluajit$(EXEEXT) mfluajit-nowin$(EXEEXT) mpost$(EXEEXT) \
pmpost$(EXEEXT) upmpost$(EXEEXT) etex$(EXEEXT) ptex$(EXEEXT) \
$(am__EXEEXT_2) eptex$(EXEEXT) uptex$(EXEEXT) $(am__EXEEXT_3) \
- euptex$(EXEEXT) pdftex$(EXEEXT) ttf2afm$(EXEEXT) \
+ euptex$(EXEEXT) hitex$(EXEEXT) hishrink$(EXEEXT) \
+ histretch$(EXEEXT) pdftex$(EXEEXT) ttf2afm$(EXEEXT) \
pdftosrc$(EXEEXT) luatex$(EXEEXT) luajittex$(EXEEXT) \
luahbtex$(EXEEXT) luajithbtex$(EXEEXT) xetex$(EXEEXT) \
$(am__EXEEXT_4) aleph$(EXEEXT) synctex$(EXEEXT) \
@@ -128,22 +129,23 @@
$(am__EXEEXT_21) $(am__EXEEXT_22) $(am__EXEEXT_23) \
$(am__EXEEXT_24) $(am__EXEEXT_25) $(am__EXEEXT_26) \
$(am__EXEEXT_27) $(am__EXEEXT_28) $(am__EXEEXT_29) \
- $(am__EXEEXT_30) $(am__EXEEXT_31) $(am__EXEEXT_32)
+ $(am__EXEEXT_30) $(am__EXEEXT_31) $(am__EXEEXT_32) \
+ $(am__EXEEXT_33)
noinst_PROGRAMS = tangleboot$(EXEEXT) ctangleboot$(EXEEXT) \
- $(am__EXEEXT_33) $(am__EXEEXT_34) $(am__EXEEXT_35) \
- $(am__EXEEXT_36) $(am__EXEEXT_37) $(am__EXEEXT_38) \
- $(am__EXEEXT_39) $(am__EXEEXT_40) $(am__EXEEXT_41) \
- $(am__EXEEXT_42)
-TESTS = tangle.test $(am__EXEEXT_45) ctiedir/ctie.test \
+ $(am__EXEEXT_34) $(am__EXEEXT_35) $(am__EXEEXT_36) \
+ $(am__EXEEXT_37) $(am__EXEEXT_38) $(am__EXEEXT_39) \
+ $(am__EXEEXT_40) $(am__EXEEXT_41) $(am__EXEEXT_42) \
+ $(am__EXEEXT_43)
+TESTS = tangle.test $(am__EXEEXT_46) ctiedir/ctie.test \
cwebdir/cweave.test tiedir/tie.test $(am__append_6) \
$(am__append_16) $(am__append_25) $(am__append_34) \
$(am__append_42) $(am__append_55) $(am__append_56) \
$(am__append_58) $(am__append_63) $(am__append_66) \
$(am__append_68) $(am__append_73) $(am__append_76) \
- $(am__append_78) $(am__append_83) $(am__append_104) \
- $(am__append_105) $(am__append_106) $(am__append_107) \
- $(am__append_115) $(am__append_117) $(am__append_119) \
- $(am__append_153) libmd5/md5.test
+ $(am__append_78) $(am__append_84) $(am__append_105) \
+ $(am__append_106) $(am__append_107) $(am__append_108) \
+ $(am__append_116) $(am__append_118) $(am__append_120) \
+ $(am__append_154) libmd5/md5.test
@WEB_TRUE at am__append_1 = $(web_programs)
@WEB_TRUE at am__append_2 = $(web_tests)
@TEX_TRUE at am__append_3 = tex
@@ -228,45 +230,46 @@
@EUPTEX_TRUE at am__append_78 = $(euptex_tests)
@EUPTEX_TRUE at am__append_79 = euptrip.diffs
@EUPTEX_TRUE at am__append_80 = euptrip-clean
- at MINGW32_TRUE@am__append_81 = \
+ at HITEX_TRUE@am__append_81 = hitex hishrink histretch
+ at MINGW32_TRUE@am__append_82 = \
@MINGW32_TRUE@ pdftexdir/regex/regex.c \
@MINGW32_TRUE@ pdftexdir/regex/regex.h
- at PDFTEX_TRUE@am__append_82 = pdftex ttf2afm pdftosrc
- at PDFTEX_TRUE@am__append_83 = $(pdftex_tests) $(ttf2afm_tests) \
+ at PDFTEX_TRUE@am__append_83 = pdftex ttf2afm pdftosrc
+ at PDFTEX_TRUE@am__append_84 = $(pdftex_tests) $(ttf2afm_tests) \
@PDFTEX_TRUE@ $(pdftosrc_tests)
- at LUATEX_TRUE@am__append_84 = luatex
- at LUATEX_TRUE@@WIN32_TRUE at am__append_85 = call_luatex
- at LUATEX_TRUE@@WIN32_TRUE at am__append_86 = install-luatex-links
- at LUATEX_TRUE@@WIN32_TRUE at am__append_87 = uninstall-luatex-links
- at LUATEX_TRUE@@WIN32_FALSE at am__append_88 = luatex$(EXEEXT):texlua luatex$(EXEEXT):texluac
- at LUAHBTEX_TRUE@am__append_89 = luahbtex
- at LUAHBTEX_TRUE@@WIN32_TRUE at am__append_90 = call_luahbtex
- at LUAHBTEX_TRUE@@WIN32_TRUE at am__append_91 = install-luahbtex-links
- at LUAHBTEX_TRUE@@WIN32_TRUE at am__append_92 = uninstall-luahbtex-links
+ at LUATEX_TRUE@am__append_85 = luatex
+ at LUATEX_TRUE@@WIN32_TRUE at am__append_86 = call_luatex
+ at LUATEX_TRUE@@WIN32_TRUE at am__append_87 = install-luatex-links
+ at LUATEX_TRUE@@WIN32_TRUE at am__append_88 = uninstall-luatex-links
+ at LUATEX_TRUE@@WIN32_FALSE at am__append_89 = luatex$(EXEEXT):texlua luatex$(EXEEXT):texluac
+ at LUAHBTEX_TRUE@am__append_90 = luahbtex
+ at LUAHBTEX_TRUE@@WIN32_TRUE at am__append_91 = call_luahbtex
+ at LUAHBTEX_TRUE@@WIN32_TRUE at am__append_92 = install-luahbtex-links
+ at LUAHBTEX_TRUE@@WIN32_TRUE at am__append_93 = uninstall-luahbtex-links
# keep texlua[c] as links to luatex unless luatex is not installed.
- at LUAHBTEX_TRUE@@LUATEX_FALSE@@WIN32_FALSE at am__append_93 = luahbtex$(EXEEXT):texlua luahbtex$(EXEEXT):texluac
- at LUAJITTEX_TRUE@am__append_94 = luajittex
- at LUAJITTEX_TRUE@@WIN32_TRUE at am__append_95 = call_luajittex
- at LUAJITTEX_TRUE@@WIN32_TRUE at am__append_96 = install-luajittex-links
- at LUAJITTEX_TRUE@@WIN32_TRUE at am__append_97 = uninstall-luajittex-links
- at LUAJITTEX_TRUE@@WIN32_FALSE at am__append_98 = luajittex$(EXEEXT):texluajit luajittex$(EXEEXT):texluajitc
- at LUAJITHBTEX_TRUE@am__append_99 = luajithbtex
- at LUAJITHBTEX_TRUE@@WIN32_TRUE at am__append_100 = call_luajithbtex
- at LUAJITHBTEX_TRUE@@WIN32_TRUE at am__append_101 = install-luajithbtex-links
- at LUAJITHBTEX_TRUE@@WIN32_TRUE at am__append_102 = uninstall-luajithbtex-links
+ at LUAHBTEX_TRUE@@LUATEX_FALSE@@WIN32_FALSE at am__append_94 = luahbtex$(EXEEXT):texlua luahbtex$(EXEEXT):texluac
+ at LUAJITTEX_TRUE@am__append_95 = luajittex
+ at LUAJITTEX_TRUE@@WIN32_TRUE at am__append_96 = call_luajittex
+ at LUAJITTEX_TRUE@@WIN32_TRUE at am__append_97 = install-luajittex-links
+ at LUAJITTEX_TRUE@@WIN32_TRUE at am__append_98 = uninstall-luajittex-links
+ at LUAJITTEX_TRUE@@WIN32_FALSE at am__append_99 = luajittex$(EXEEXT):texluajit luajittex$(EXEEXT):texluajitc
+ at LUAJITHBTEX_TRUE@am__append_100 = luajithbtex
+ at LUAJITHBTEX_TRUE@@WIN32_TRUE at am__append_101 = call_luajithbtex
+ at LUAJITHBTEX_TRUE@@WIN32_TRUE at am__append_102 = install-luajithbtex-links
+ at LUAJITHBTEX_TRUE@@WIN32_TRUE at am__append_103 = uninstall-luajithbtex-links
# keep texluajit[c] as links to luajittex unless luajittex is not installed.
- at LUAJITHBTEX_TRUE@@LUAJITTEX_FALSE@@WIN32_FALSE at am__append_103 = luajithbtex$(EXEEXT):texluajit luajithbtex$(EXEEXT):texluajitc
- at LUATEX_TRUE@am__append_104 = $(luatex_tests)
- at LUAHBTEX_TRUE@am__append_105 = $(luahbtex_tests)
- at LUAJITTEX_TRUE@am__append_106 = $(luajittex_tests)
- at LUAJITHBTEX_TRUE@am__append_107 = $(luajithbtex_tests)
- at XETEX_TRUE@am__append_108 = xetex
- at XETEX_MACOSX_TRUE@am__append_109 = -DXETEX_MAC
- at XETEX_MACOSX_TRUE@am__append_110 = -std=c++11
- at XETEX_MACOSX_FALSE@am__append_111 = $(FONTCONFIG_INCLUDES)
- at XETEX_MACOSX_FALSE@am__append_112 = $(FONTCONFIG_LIBS)
- at XETEX_MACOSX_TRUE@am__append_113 = \
+ at LUAJITHBTEX_TRUE@@LUAJITTEX_FALSE@@WIN32_FALSE at am__append_104 = luajithbtex$(EXEEXT):texluajit luajithbtex$(EXEEXT):texluajitc
+ at LUATEX_TRUE@am__append_105 = $(luatex_tests)
+ at LUAHBTEX_TRUE@am__append_106 = $(luahbtex_tests)
+ at LUAJITTEX_TRUE@am__append_107 = $(luajittex_tests)
+ at LUAJITHBTEX_TRUE@am__append_108 = $(luajithbtex_tests)
+ at XETEX_TRUE@am__append_109 = xetex
+ at XETEX_MACOSX_TRUE@am__append_110 = -DXETEX_MAC
+ at XETEX_MACOSX_TRUE@am__append_111 = -std=c++11
+ at XETEX_MACOSX_FALSE@am__append_112 = $(FONTCONFIG_INCLUDES)
+ at XETEX_MACOSX_FALSE@am__append_113 = $(FONTCONFIG_LIBS)
+ at XETEX_MACOSX_TRUE@am__append_114 = \
@XETEX_MACOSX_TRUE@ xetexdir/XeTeXFontInst_Mac.cpp \
@XETEX_MACOSX_TRUE@ xetexdir/XeTeXFontInst_Mac.h \
@XETEX_MACOSX_TRUE@ xetexdir/XeTeXFontMgr_Mac.mm \
@@ -273,105 +276,105 @@
@XETEX_MACOSX_TRUE@ xetexdir/XeTeXFontMgr_Mac.h \
@XETEX_MACOSX_TRUE@ xetexdir/XeTeX_mac.c
- at XETEX_MACOSX_FALSE@am__append_114 = \
+ at XETEX_MACOSX_FALSE@am__append_115 = \
@XETEX_MACOSX_FALSE@ xetexdir/XeTeXFontMgr_FC.cpp \
@XETEX_MACOSX_FALSE@ xetexdir/XeTeXFontMgr_FC.h
- at XETEX_TRUE@am__append_115 = $(xetex_tests)
- at OTANGLE_TRUE@am__append_116 = $(omegaware_programs)
- at OTANGLE_TRUE@am__append_117 = $(OTANGLE_tests) $(OMFONTS_tests)
- at ALEPH_TRUE@am__append_118 = aleph
- at ALEPH_TRUE@am__append_119 = $(aleph_tests)
- at SYNCTEX_TRUE@am__append_120 = synctex
- at SYNCTEX_TRUE@am__append_121 = $(LTLIBSYNCTEX)
- at SYNCTEX_TRUE@am__append_122 = $(LIBSYNCTEX)
- at MINGW32_TRUE@am__append_123 = -lshlwapi
+ at XETEX_TRUE@am__append_116 = $(xetex_tests)
+ at OTANGLE_TRUE@am__append_117 = $(omegaware_programs)
+ at OTANGLE_TRUE@am__append_118 = $(OTANGLE_tests) $(OMFONTS_tests)
+ at ALEPH_TRUE@am__append_119 = aleph
+ at ALEPH_TRUE@am__append_120 = $(aleph_tests)
+ at SYNCTEX_TRUE@am__append_121 = synctex
+ at SYNCTEX_TRUE@am__append_122 = $(LTLIBSYNCTEX)
+ at SYNCTEX_TRUE@am__append_123 = $(LIBSYNCTEX)
@MINGW32_TRUE at am__append_124 = -lshlwapi
- at TEX_SYNCTEX_TRUE@am__append_125 = -I$(srcdir)/synctexdir \
+ at MINGW32_TRUE@am__append_125 = -lshlwapi
+ at TEX_SYNCTEX_TRUE@am__append_126 = -I$(srcdir)/synctexdir \
@TEX_SYNCTEX_TRUE@ $(ZLIB_INCLUDES) -D__SyncTeX__ \
@TEX_SYNCTEX_TRUE@ -DSYNCTEX_ENGINE_H=\"synctex-tex.h\"
- at TEX_SYNCTEX_TRUE@am__append_126 = $(ZLIB_LIBS)
- at TEX_SYNCTEX_TRUE@am__append_127 = $(ZLIB_DEPEND)
- at TEX_SYNCTEX_TRUE@am__append_128 = \
+ at TEX_SYNCTEX_TRUE@am__append_127 = $(ZLIB_LIBS)
+ at TEX_SYNCTEX_TRUE@am__append_128 = $(ZLIB_DEPEND)
+ at TEX_SYNCTEX_TRUE@am__append_129 = \
@TEX_SYNCTEX_TRUE@ synctexdir/synctex.c \
@TEX_SYNCTEX_TRUE@ synctexdir/synctex.h \
@TEX_SYNCTEX_TRUE@ synctexdir/synctex-common.h \
@TEX_SYNCTEX_TRUE@ synctexdir/synctex-tex.h
- at ETEX_SYNCTEX_TRUE@am__append_129 = -I$(srcdir)/synctexdir \
+ at ETEX_SYNCTEX_TRUE@am__append_130 = -I$(srcdir)/synctexdir \
@ETEX_SYNCTEX_TRUE@ $(ZLIB_INCLUDES) -D__SyncTeX__ \
@ETEX_SYNCTEX_TRUE@ -DSYNCTEX_ENGINE_H=\"synctex-etex.h\"
- at ETEX_SYNCTEX_TRUE@am__append_130 = $(ZLIB_LIBS)
- at ETEX_SYNCTEX_TRUE@am__append_131 = $(ZLIB_DEPEND)
- at ETEX_SYNCTEX_TRUE@am__append_132 = \
+ at ETEX_SYNCTEX_TRUE@am__append_131 = $(ZLIB_LIBS)
+ at ETEX_SYNCTEX_TRUE@am__append_132 = $(ZLIB_DEPEND)
+ at ETEX_SYNCTEX_TRUE@am__append_133 = \
@ETEX_SYNCTEX_TRUE@ synctexdir/synctex.c \
@ETEX_SYNCTEX_TRUE@ synctexdir/synctex.h \
@ETEX_SYNCTEX_TRUE@ synctexdir/synctex-common.h \
@ETEX_SYNCTEX_TRUE@ synctexdir/synctex-etex.h
- at PTEX_SYNCTEX_TRUE@am__append_133 = -I$(srcdir)/synctexdir \
+ at PTEX_SYNCTEX_TRUE@am__append_134 = -I$(srcdir)/synctexdir \
@PTEX_SYNCTEX_TRUE@ $(ZLIB_INCLUDES) -D__SyncTeX__ \
@PTEX_SYNCTEX_TRUE@ -DSYNCTEX_ENGINE_H=\"synctex-ptex.h\"
- at PTEX_SYNCTEX_TRUE@am__append_134 = $(ZLIB_LIBS)
- at PTEX_SYNCTEX_TRUE@am__append_135 = $(ZLIB_DEPEND)
- at PTEX_SYNCTEX_TRUE@am__append_136 = \
+ at PTEX_SYNCTEX_TRUE@am__append_135 = $(ZLIB_LIBS)
+ at PTEX_SYNCTEX_TRUE@am__append_136 = $(ZLIB_DEPEND)
+ at PTEX_SYNCTEX_TRUE@am__append_137 = \
@PTEX_SYNCTEX_TRUE@ synctexdir/synctex.c \
@PTEX_SYNCTEX_TRUE@ synctexdir/synctex.h \
@PTEX_SYNCTEX_TRUE@ synctexdir/synctex-common.h \
@PTEX_SYNCTEX_TRUE@ synctexdir/synctex-ptex.h
- at UPTEX_SYNCTEX_TRUE@am__append_137 = -I$(srcdir)/synctexdir \
+ at UPTEX_SYNCTEX_TRUE@am__append_138 = -I$(srcdir)/synctexdir \
@UPTEX_SYNCTEX_TRUE@ $(ZLIB_INCLUDES) -D__SyncTeX__ \
@UPTEX_SYNCTEX_TRUE@ -DSYNCTEX_ENGINE_H=\"synctex-uptex.h\"
- at UPTEX_SYNCTEX_TRUE@am__append_138 = $(ZLIB_LIBS)
- at UPTEX_SYNCTEX_TRUE@am__append_139 = $(ZLIB_DEPEND)
- at UPTEX_SYNCTEX_TRUE@am__append_140 = \
+ at UPTEX_SYNCTEX_TRUE@am__append_139 = $(ZLIB_LIBS)
+ at UPTEX_SYNCTEX_TRUE@am__append_140 = $(ZLIB_DEPEND)
+ at UPTEX_SYNCTEX_TRUE@am__append_141 = \
@UPTEX_SYNCTEX_TRUE@ synctexdir/synctex.c \
@UPTEX_SYNCTEX_TRUE@ synctexdir/synctex.h \
@UPTEX_SYNCTEX_TRUE@ synctexdir/synctex-common.h \
@UPTEX_SYNCTEX_TRUE@ synctexdir/synctex-uptex.h
- at EPTEX_SYNCTEX_TRUE@am__append_141 = -I$(srcdir)/synctexdir \
+ at EPTEX_SYNCTEX_TRUE@am__append_142 = -I$(srcdir)/synctexdir \
@EPTEX_SYNCTEX_TRUE@ $(ZLIB_INCLUDES) -D__SyncTeX__ \
@EPTEX_SYNCTEX_TRUE@ -DSYNCTEX_ENGINE_H=\"synctex-eptex.h\"
- at EPTEX_SYNCTEX_TRUE@am__append_142 = $(ZLIB_LIBS)
- at EPTEX_SYNCTEX_TRUE@am__append_143 = $(ZLIB_DEPEND)
- at EPTEX_SYNCTEX_TRUE@am__append_144 = \
+ at EPTEX_SYNCTEX_TRUE@am__append_143 = $(ZLIB_LIBS)
+ at EPTEX_SYNCTEX_TRUE@am__append_144 = $(ZLIB_DEPEND)
+ at EPTEX_SYNCTEX_TRUE@am__append_145 = \
@EPTEX_SYNCTEX_TRUE@ synctexdir/synctex.c \
@EPTEX_SYNCTEX_TRUE@ synctexdir/synctex.h \
@EPTEX_SYNCTEX_TRUE@ synctexdir/synctex-common.h \
@EPTEX_SYNCTEX_TRUE@ synctexdir/synctex-eptex.h
- at EUPTEX_SYNCTEX_TRUE@am__append_145 = -I$(srcdir)/synctexdir \
+ at EUPTEX_SYNCTEX_TRUE@am__append_146 = -I$(srcdir)/synctexdir \
@EUPTEX_SYNCTEX_TRUE@ $(ZLIB_INCLUDES) -D__SyncTeX__ \
@EUPTEX_SYNCTEX_TRUE@ -DSYNCTEX_ENGINE_H=\"synctex-euptex.h\"
- at EUPTEX_SYNCTEX_TRUE@am__append_146 = $(ZLIB_LIBS)
- at EUPTEX_SYNCTEX_TRUE@am__append_147 = $(ZLIB_DEPEND)
- at EUPTEX_SYNCTEX_TRUE@am__append_148 = \
+ at EUPTEX_SYNCTEX_TRUE@am__append_147 = $(ZLIB_LIBS)
+ at EUPTEX_SYNCTEX_TRUE@am__append_148 = $(ZLIB_DEPEND)
+ at EUPTEX_SYNCTEX_TRUE@am__append_149 = \
@EUPTEX_SYNCTEX_TRUE@ synctexdir/synctex.c \
@EUPTEX_SYNCTEX_TRUE@ synctexdir/synctex.h \
@EUPTEX_SYNCTEX_TRUE@ synctexdir/synctex-common.h \
@EUPTEX_SYNCTEX_TRUE@ synctexdir/synctex-euptex.h
- at PDFTEX_SYNCTEX_TRUE@am__append_149 = -I$(srcdir)/synctexdir \
+ at PDFTEX_SYNCTEX_TRUE@am__append_150 = -I$(srcdir)/synctexdir \
@PDFTEX_SYNCTEX_TRUE@ -D__SyncTeX__ \
@PDFTEX_SYNCTEX_TRUE@ -DSYNCTEX_ENGINE_H=\"synctex-pdftex.h\"
- at PDFTEX_SYNCTEX_TRUE@am__append_150 = \
+ at PDFTEX_SYNCTEX_TRUE@am__append_151 = \
@PDFTEX_SYNCTEX_TRUE@ synctexdir/synctex.c \
@PDFTEX_SYNCTEX_TRUE@ synctexdir/synctex.h \
@PDFTEX_SYNCTEX_TRUE@ synctexdir/synctex-common.h \
@PDFTEX_SYNCTEX_TRUE@ synctexdir/synctex-pdftex.h
- at XETEX_SYNCTEX_TRUE@am__append_151 = -I$(srcdir)/synctexdir \
+ at XETEX_SYNCTEX_TRUE@am__append_152 = -I$(srcdir)/synctexdir \
@XETEX_SYNCTEX_TRUE@ -D__SyncTeX__ \
@XETEX_SYNCTEX_TRUE@ -DSYNCTEX_ENGINE_H=\"synctex-xetex.h\"
- at XETEX_SYNCTEX_TRUE@am__append_152 = \
+ at XETEX_SYNCTEX_TRUE@am__append_153 = \
@XETEX_SYNCTEX_TRUE@ synctexdir/synctex.c \
@XETEX_SYNCTEX_TRUE@ synctexdir/synctex.h \
@XETEX_SYNCTEX_TRUE@ synctexdir/synctex-common.h \
@XETEX_SYNCTEX_TRUE@ synctexdir/synctex-xetex.h
- at SYNCTEX_TRUE@am__append_153 = $(synctex_tests)
+ at SYNCTEX_TRUE@am__append_154 = $(synctex_tests)
subdir = .
ACLOCAL_M4 = $(top_srcdir)/aclocal.m4
am__aclocal_m4_deps = $(top_srcdir)/m4/web2c-disable.m4 \
@@ -461,30 +464,32 @@
@UPTEX_TRUE at am__EXEEXT_21 = uptex$(EXEEXT)
@UPWEB_TRUE at am__EXEEXT_22 = $(am__EXEEXT_3)
@EUPTEX_TRUE at am__EXEEXT_23 = euptex$(EXEEXT)
- at PDFTEX_TRUE@am__EXEEXT_24 = pdftex$(EXEEXT) ttf2afm$(EXEEXT) \
+ at HITEX_TRUE@am__EXEEXT_24 = hitex$(EXEEXT) hishrink$(EXEEXT) \
+ at HITEX_TRUE@ histretch$(EXEEXT)
+ at PDFTEX_TRUE@am__EXEEXT_25 = pdftex$(EXEEXT) ttf2afm$(EXEEXT) \
@PDFTEX_TRUE@ pdftosrc$(EXEEXT)
- at LUATEX_TRUE@am__EXEEXT_25 = luatex$(EXEEXT)
- at LUAHBTEX_TRUE@am__EXEEXT_26 = luahbtex$(EXEEXT)
- at LUAJITTEX_TRUE@am__EXEEXT_27 = luajittex$(EXEEXT)
- at LUAJITHBTEX_TRUE@am__EXEEXT_28 = luajithbtex$(EXEEXT)
- at XETEX_TRUE@am__EXEEXT_29 = xetex$(EXEEXT)
- at OTANGLE_TRUE@am__EXEEXT_30 = $(am__EXEEXT_4)
- at ALEPH_TRUE@am__EXEEXT_31 = aleph$(EXEEXT)
- at SYNCTEX_TRUE@am__EXEEXT_32 = synctex$(EXEEXT)
+ at LUATEX_TRUE@am__EXEEXT_26 = luatex$(EXEEXT)
+ at LUAHBTEX_TRUE@am__EXEEXT_27 = luahbtex$(EXEEXT)
+ at LUAJITTEX_TRUE@am__EXEEXT_28 = luajittex$(EXEEXT)
+ at LUAJITHBTEX_TRUE@am__EXEEXT_29 = luajithbtex$(EXEEXT)
+ at XETEX_TRUE@am__EXEEXT_30 = xetex$(EXEEXT)
+ at OTANGLE_TRUE@am__EXEEXT_31 = $(am__EXEEXT_4)
+ at ALEPH_TRUE@am__EXEEXT_32 = aleph$(EXEEXT)
+ at SYNCTEX_TRUE@am__EXEEXT_33 = synctex$(EXEEXT)
am__installdirs = "$(DESTDIR)$(bindir)" "$(DESTDIR)$(libdir)" \
"$(DESTDIR)$(libdir)" "$(DESTDIR)$(man1dir)" \
"$(DESTDIR)$(man5dir)" "$(DESTDIR)$(pkgconfigdir)" \
"$(DESTDIR)$(syncincludedir)"
- at MF_TRUE@@WIN32_TRUE at am__EXEEXT_33 = call_mf$(EXEEXT)
- at MFLUA_TRUE@@WIN32_TRUE at am__EXEEXT_34 = call_mflua$(EXEEXT)
- at MFLUAJIT_TRUE@@WIN32_TRUE at am__EXEEXT_35 = call_mfluajit$(EXEEXT)
- at MP_TRUE@@WIN32_TRUE at am__EXEEXT_36 = call_mpost$(EXEEXT)
- at PMP_TRUE@@WIN32_TRUE at am__EXEEXT_37 = call_pmpost$(EXEEXT)
- at UPMP_TRUE@@WIN32_TRUE at am__EXEEXT_38 = call_upmpost$(EXEEXT)
- at LUATEX_TRUE@@WIN32_TRUE at am__EXEEXT_39 = call_luatex$(EXEEXT)
- at LUAHBTEX_TRUE@@WIN32_TRUE at am__EXEEXT_40 = call_luahbtex$(EXEEXT)
- at LUAJITTEX_TRUE@@WIN32_TRUE at am__EXEEXT_41 = call_luajittex$(EXEEXT)
- at LUAJITHBTEX_TRUE@@WIN32_TRUE at am__EXEEXT_42 = \
+ at MF_TRUE@@WIN32_TRUE at am__EXEEXT_34 = call_mf$(EXEEXT)
+ at MFLUA_TRUE@@WIN32_TRUE at am__EXEEXT_35 = call_mflua$(EXEEXT)
+ at MFLUAJIT_TRUE@@WIN32_TRUE at am__EXEEXT_36 = call_mfluajit$(EXEEXT)
+ at MP_TRUE@@WIN32_TRUE at am__EXEEXT_37 = call_mpost$(EXEEXT)
+ at PMP_TRUE@@WIN32_TRUE at am__EXEEXT_38 = call_pmpost$(EXEEXT)
+ at UPMP_TRUE@@WIN32_TRUE at am__EXEEXT_39 = call_upmpost$(EXEEXT)
+ at LUATEX_TRUE@@WIN32_TRUE at am__EXEEXT_40 = call_luatex$(EXEEXT)
+ at LUAHBTEX_TRUE@@WIN32_TRUE at am__EXEEXT_41 = call_luahbtex$(EXEEXT)
+ at LUAJITTEX_TRUE@@WIN32_TRUE at am__EXEEXT_42 = call_luajittex$(EXEEXT)
+ at LUAJITHBTEX_TRUE@@WIN32_TRUE at am__EXEEXT_43 = \
@LUAJITHBTEX_TRUE@@WIN32_TRUE@ call_luajithbtex$(EXEEXT)
PROGRAMS = $(bin_PROGRAMS) $(noinst_PROGRAMS)
am__vpath_adj_setup = srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`;
@@ -1382,12 +1387,27 @@
gftype_OBJECTS = $(nodist_gftype_OBJECTS)
gftype_LDADD = $(LDADD)
gftype_DEPENDENCIES = $(proglib) $(am__DEPENDENCIES_1)
+nodist_hishrink_OBJECTS = hishrink-hformat.$(OBJEXT) \
+ hishrink-hishrink.$(OBJEXT) hishrink-shrink-lexer.$(OBJEXT) \
+ hishrink-shrink-parser.$(OBJEXT)
+hishrink_OBJECTS = $(nodist_hishrink_OBJECTS)
+hishrink_DEPENDENCIES = $(am__DEPENDENCIES_1)
+nodist_histretch_OBJECTS = histretch-hformat.$(OBJEXT) \
+ histretch-histretch.$(OBJEXT)
+histretch_OBJECTS = $(nodist_histretch_OBJECTS)
+histretch_DEPENDENCIES = $(am__DEPENDENCIES_1)
+am__objects_37 = hitex-htex.$(OBJEXT)
+am__objects_38 = hitex-hitex.$(OBJEXT)
+nodist_hitex_OBJECTS = hitex-hformat.$(OBJEXT) hitex-hput.$(OBJEXT) \
+ $(am__objects_37) $(am__objects_38)
+hitex_OBJECTS = $(nodist_hitex_OBJECTS)
+hitex_DEPENDENCIES = $(am__DEPENDENCIES_1) $(am__DEPENDENCIES_1)
nodist_initex_OBJECTS = initex-callexe.$(OBJEXT)
initex_OBJECTS = $(nodist_initex_OBJECTS)
initex_DEPENDENCIES =
-am__objects_37 = luatexdir/luahbtex-luatex.$(OBJEXT) \
+am__objects_39 = luatexdir/luahbtex-luatex.$(OBJEXT) \
mplibdir/luahbtex-lmplib.$(OBJEXT)
-nodist_luahbtex_OBJECTS = $(am__objects_37)
+nodist_luahbtex_OBJECTS = $(am__objects_39)
luahbtex_OBJECTS = $(nodist_luahbtex_OBJECTS)
am__DEPENDENCIES_6 = $(am__DEPENDENCIES_1)
am__DEPENDENCIES_7 = libmplibcore.a $(am__DEPENDENCIES_1) \
@@ -1398,16 +1418,16 @@
luahbtex_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CXX $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXXLD) $(luahbtex_CXXFLAGS) \
$(CXXFLAGS) $(luahbtex_LDFLAGS) $(LDFLAGS) -o $@
-am__objects_38 = luatexdir/luajithbtex-luatex.$(OBJEXT) \
+am__objects_40 = luatexdir/luajithbtex-luatex.$(OBJEXT) \
mplibdir/luajithbtex-lmplib.$(OBJEXT)
-nodist_luajithbtex_OBJECTS = $(am__objects_38)
+nodist_luajithbtex_OBJECTS = $(am__objects_40)
luajithbtex_OBJECTS = $(nodist_luajithbtex_OBJECTS)
luajithbtex_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CXX $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXXLD) $(luajithbtex_CXXFLAGS) \
$(CXXFLAGS) $(luajithbtex_LDFLAGS) $(LDFLAGS) -o $@
-am__objects_39 = luatexdir/luajittex-luatex.$(OBJEXT) \
+am__objects_41 = luatexdir/luajittex-luatex.$(OBJEXT) \
mplibdir/luajittex-lmplib.$(OBJEXT)
-nodist_luajittex_OBJECTS = $(am__objects_39)
+nodist_luajittex_OBJECTS = $(am__objects_41)
luajittex_OBJECTS = $(nodist_luajittex_OBJECTS)
am__DEPENDENCIES_8 = libmplibcore.a $(am__DEPENDENCIES_1) \
$(am__DEPENDENCIES_1) $(am__DEPENDENCIES_1) \
@@ -1416,9 +1436,9 @@
luajittex_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \
$(luajittex_LDFLAGS) $(LDFLAGS) -o $@
-am__objects_40 = luatexdir/luatex-luatex.$(OBJEXT) \
+am__objects_42 = luatexdir/luatex-luatex.$(OBJEXT) \
mplibdir/luatex-lmplib.$(OBJEXT)
-nodist_luatex_OBJECTS = $(am__objects_40)
+nodist_luatex_OBJECTS = $(am__objects_42)
luatex_OBJECTS = $(nodist_luatex_OBJECTS)
luatex_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \
@@ -1444,8 +1464,8 @@
mflua_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \
$(mflua_LDFLAGS) $(LDFLAGS) -o $@
-am__objects_41 = mfluadir/mflua_nowin-mfluaextra.$(OBJEXT)
-nodist_mflua_nowin_OBJECTS = $(am__objects_41)
+am__objects_43 = mfluadir/mflua_nowin-mfluaextra.$(OBJEXT)
+nodist_mflua_nowin_OBJECTS = $(am__objects_43)
mflua_nowin_OBJECTS = $(nodist_mflua_nowin_OBJECTS)
mflua_nowin_DEPENDENCIES = libmflua.a libmfluaotfcc.a \
$(am__DEPENDENCIES_2) $(windowlib) $(am__DEPENDENCIES_6)
@@ -1460,8 +1480,8 @@
mfluajit_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \
$(mfluajit_LDFLAGS) $(LDFLAGS) -o $@
-am__objects_42 = mfluajitdir/mfluajit_nowin-mfluajitextra.$(OBJEXT)
-nodist_mfluajit_nowin_OBJECTS = $(am__objects_42)
+am__objects_44 = mfluajitdir/mfluajit_nowin-mfluajitextra.$(OBJEXT)
+nodist_mfluajit_nowin_OBJECTS = $(am__objects_44)
mfluajit_nowin_OBJECTS = $(nodist_mfluajit_nowin_OBJECTS)
mfluajit_nowin_DEPENDENCIES = libmfluajit.a libmfluaotfcc.a \
$(am__DEPENDENCIES_2) $(windowlib) $(am__DEPENDENCIES_1)
@@ -1473,8 +1493,8 @@
mft_OBJECTS = $(nodist_mft_OBJECTS)
mft_LDADD = $(LDADD)
mft_DEPENDENCIES = $(proglib) $(am__DEPENDENCIES_1)
-am__objects_43 = mpost-mpxout.$(OBJEXT)
-nodist_mpost_OBJECTS = mpost-mpost.$(OBJEXT) $(am__objects_43)
+am__objects_45 = mpost-mpxout.$(OBJEXT)
+nodist_mpost_OBJECTS = mpost-mpost.$(OBJEXT) $(am__objects_45)
mpost_OBJECTS = $(nodist_mpost_OBJECTS)
mpost_DEPENDENCIES = libmplibcore.a libmplibextramath.a \
libmplibbackends.a $(am__DEPENDENCIES_1) $(am__DEPENDENCIES_1) \
@@ -1507,12 +1527,12 @@
pdftexdir/etex_version.h synctexdir/synctex.c \
synctexdir/synctex.h synctexdir/synctex-common.h \
synctexdir/synctex-pdftex.h
- at PDFTEX_SYNCTEX_TRUE@am__objects_44 = \
+ at PDFTEX_SYNCTEX_TRUE@am__objects_46 = \
@PDFTEX_SYNCTEX_TRUE@ synctexdir/pdftex-synctex.$(OBJEXT)
dist_pdftex_OBJECTS = pdftexdir/pdftex-pdftexextra.$(OBJEXT) \
- $(am__objects_44)
-am__objects_45 = pdftex-pdftexini.$(OBJEXT) pdftex-pdftex0.$(OBJEXT)
-nodist_pdftex_OBJECTS = $(am__objects_45) pdftex-pdftex-pool.$(OBJEXT)
+ $(am__objects_46)
+am__objects_47 = pdftex-pdftexini.$(OBJEXT) pdftex-pdftex0.$(OBJEXT)
+nodist_pdftex_OBJECTS = $(am__objects_47) pdftex-pdftex-pool.$(OBJEXT)
pdftex_OBJECTS = $(dist_pdftex_OBJECTS) $(nodist_pdftex_OBJECTS)
am__DEPENDENCIES_11 = $(am__DEPENDENCIES_1) $(am__DEPENDENCIES_1) \
$(am__DEPENDENCIES_1) libmd5.a
@@ -1539,20 +1559,20 @@
pltotf_OBJECTS = $(nodist_pltotf_OBJECTS)
pltotf_LDADD = $(LDADD)
pltotf_DEPENDENCIES = $(proglib) $(am__DEPENDENCIES_1)
-am__objects_46 = pmpost-pmp.$(OBJEXT)
-am__objects_47 = pmpost-pmpmath.$(OBJEXT)
-am__objects_48 = pmpost-pmpmathbinary.$(OBJEXT)
-am__objects_49 = pmpost-pmpmathdecimal.$(OBJEXT)
-am__objects_50 = pmpost-pmpmathdouble.$(OBJEXT)
-am__objects_51 = pmpost-pmpstrings.$(OBJEXT)
-am__objects_52 = pmpost-pmpxout.$(OBJEXT)
-am__objects_53 = pmpost-ppngout.$(OBJEXT)
-am__objects_54 = pmpost-ppsout.$(OBJEXT)
-am__objects_55 = pmpost-psvgout.$(OBJEXT)
-nodist_pmpost_OBJECTS = $(am__objects_46) $(am__objects_47) \
- $(am__objects_48) $(am__objects_49) $(am__objects_50) \
- pmpost-pmpost.$(OBJEXT) $(am__objects_51) $(am__objects_52) \
- $(am__objects_53) $(am__objects_54) $(am__objects_55) \
+am__objects_48 = pmpost-pmp.$(OBJEXT)
+am__objects_49 = pmpost-pmpmath.$(OBJEXT)
+am__objects_50 = pmpost-pmpmathbinary.$(OBJEXT)
+am__objects_51 = pmpost-pmpmathdecimal.$(OBJEXT)
+am__objects_52 = pmpost-pmpmathdouble.$(OBJEXT)
+am__objects_53 = pmpost-pmpstrings.$(OBJEXT)
+am__objects_54 = pmpost-pmpxout.$(OBJEXT)
+am__objects_55 = pmpost-ppngout.$(OBJEXT)
+am__objects_56 = pmpost-ppsout.$(OBJEXT)
+am__objects_57 = pmpost-psvgout.$(OBJEXT)
+nodist_pmpost_OBJECTS = $(am__objects_48) $(am__objects_49) \
+ $(am__objects_50) $(am__objects_51) $(am__objects_52) \
+ pmpost-pmpost.$(OBJEXT) $(am__objects_53) $(am__objects_54) \
+ $(am__objects_55) $(am__objects_56) $(am__objects_57) \
pmpost-ptfmin.$(OBJEXT)
pmpost_OBJECTS = $(nodist_pmpost_OBJECTS)
pmpost_DEPENDENCIES = $(am__DEPENDENCIES_1) $(am__DEPENDENCIES_1) \
@@ -1570,10 +1590,10 @@
ptexdir/ptex_version.h synctexdir/synctex.c \
synctexdir/synctex.h synctexdir/synctex-common.h \
synctexdir/synctex-ptex.h
- at PTEX_SYNCTEX_TRUE@am__objects_56 = synctexdir/ptex-synctex.$(OBJEXT)
-dist_ptex_OBJECTS = ptexdir/ptex-ptexextra.$(OBJEXT) $(am__objects_56)
-am__objects_57 = ptex-ptexini.$(OBJEXT) ptex-ptex0.$(OBJEXT)
-nodist_ptex_OBJECTS = $(am__objects_57) ptex-ptex-pool.$(OBJEXT)
+ at PTEX_SYNCTEX_TRUE@am__objects_58 = synctexdir/ptex-synctex.$(OBJEXT)
+dist_ptex_OBJECTS = ptexdir/ptex-ptexextra.$(OBJEXT) $(am__objects_58)
+am__objects_59 = ptex-ptexini.$(OBJEXT) ptex-ptex0.$(OBJEXT)
+nodist_ptex_OBJECTS = $(am__objects_59) ptex-ptex-pool.$(OBJEXT)
ptex_OBJECTS = $(dist_ptex_OBJECTS) $(nodist_ptex_OBJECTS)
@PTEX_SYNCTEX_TRUE at am__DEPENDENCIES_12 = $(am__DEPENDENCIES_1)
am_ptftopl_OBJECTS =
@@ -1594,10 +1614,10 @@
am__dist_tex_SOURCES_DIST = texextra.c synctexdir/synctex.c \
synctexdir/synctex.h synctexdir/synctex-common.h \
synctexdir/synctex-tex.h
- at TEX_SYNCTEX_TRUE@am__objects_58 = synctexdir/tex-synctex.$(OBJEXT)
-dist_tex_OBJECTS = tex-texextra.$(OBJEXT) $(am__objects_58)
-am__objects_59 = tex-texini.$(OBJEXT) tex-tex0.$(OBJEXT)
-nodist_tex_OBJECTS = $(am__objects_59) tex-tex-pool.$(OBJEXT)
+ at TEX_SYNCTEX_TRUE@am__objects_60 = synctexdir/tex-synctex.$(OBJEXT)
+dist_tex_OBJECTS = tex-texextra.$(OBJEXT) $(am__objects_60)
+am__objects_61 = tex-texini.$(OBJEXT) tex-tex0.$(OBJEXT)
+nodist_tex_OBJECTS = $(am__objects_61) tex-tex-pool.$(OBJEXT)
tex_OBJECTS = $(dist_tex_OBJECTS) $(nodist_tex_OBJECTS)
@TEX_SYNCTEX_TRUE at am__DEPENDENCIES_14 = $(am__DEPENDENCIES_1)
tex_DEPENDENCIES = $(am__DEPENDENCIES_2) $(am__DEPENDENCIES_1) \
@@ -1622,21 +1642,21 @@
nodist_updvitype_OBJECTS = updvitype-updvitype.$(OBJEXT)
updvitype_OBJECTS = $(am_updvitype_OBJECTS) \
$(nodist_updvitype_OBJECTS)
-am__objects_60 = upmpost-pmp.$(OBJEXT)
-am__objects_61 = upmpost-pmpmath.$(OBJEXT)
-am__objects_62 = upmpost-pmpmathbinary.$(OBJEXT)
-am__objects_63 = upmpost-pmpmathdecimal.$(OBJEXT)
-am__objects_64 = upmpost-pmpmathdouble.$(OBJEXT)
-am__objects_65 = upmpost-pmpstrings.$(OBJEXT)
-am__objects_66 = upmpost-pmpxout.$(OBJEXT)
-am__objects_67 = upmpost-ppngout.$(OBJEXT)
-am__objects_68 = upmpost-ppsout.$(OBJEXT)
-am__objects_69 = upmpost-psvgout.$(OBJEXT)
-am__objects_70 = $(am__objects_60) $(am__objects_61) $(am__objects_62) \
- $(am__objects_63) $(am__objects_64) upmpost-pmpost.$(OBJEXT) \
- $(am__objects_65) $(am__objects_66) $(am__objects_67) \
- $(am__objects_68) $(am__objects_69) upmpost-ptfmin.$(OBJEXT)
-nodist_upmpost_OBJECTS = $(am__objects_70)
+am__objects_62 = upmpost-pmp.$(OBJEXT)
+am__objects_63 = upmpost-pmpmath.$(OBJEXT)
+am__objects_64 = upmpost-pmpmathbinary.$(OBJEXT)
+am__objects_65 = upmpost-pmpmathdecimal.$(OBJEXT)
+am__objects_66 = upmpost-pmpmathdouble.$(OBJEXT)
+am__objects_67 = upmpost-pmpstrings.$(OBJEXT)
+am__objects_68 = upmpost-pmpxout.$(OBJEXT)
+am__objects_69 = upmpost-ppngout.$(OBJEXT)
+am__objects_70 = upmpost-ppsout.$(OBJEXT)
+am__objects_71 = upmpost-psvgout.$(OBJEXT)
+am__objects_72 = $(am__objects_62) $(am__objects_63) $(am__objects_64) \
+ $(am__objects_65) $(am__objects_66) upmpost-pmpost.$(OBJEXT) \
+ $(am__objects_67) $(am__objects_68) $(am__objects_69) \
+ $(am__objects_70) $(am__objects_71) upmpost-ptfmin.$(OBJEXT)
+nodist_upmpost_OBJECTS = $(am__objects_72)
upmpost_OBJECTS = $(nodist_upmpost_OBJECTS)
am__DEPENDENCIES_16 = $(am__DEPENDENCIES_1) $(am__DEPENDENCIES_1) \
$(am__DEPENDENCIES_1) $(am__DEPENDENCIES_1) \
@@ -1650,12 +1670,12 @@
uptexdir/uptexextra.h uptexdir/uptex_version.h \
synctexdir/synctex.c synctexdir/synctex.h \
synctexdir/synctex-common.h synctexdir/synctex-uptex.h
- at UPTEX_SYNCTEX_TRUE@am__objects_71 = \
+ at UPTEX_SYNCTEX_TRUE@am__objects_73 = \
@UPTEX_SYNCTEX_TRUE@ synctexdir/uptex-synctex.$(OBJEXT)
dist_uptex_OBJECTS = uptexdir/uptex-uptexextra.$(OBJEXT) \
- $(am__objects_71)
-am__objects_72 = uptex-uptexini.$(OBJEXT) uptex-uptex0.$(OBJEXT)
-nodist_uptex_OBJECTS = $(am__objects_72) uptex-uptex-pool.$(OBJEXT)
+ $(am__objects_73)
+am__objects_74 = uptex-uptexini.$(OBJEXT) uptex-uptex0.$(OBJEXT)
+nodist_uptex_OBJECTS = $(am__objects_74) uptex-uptex-pool.$(OBJEXT)
uptex_OBJECTS = $(dist_uptex_OBJECTS) $(nodist_uptex_OBJECTS)
@UPTEX_SYNCTEX_TRUE at am__DEPENDENCIES_17 = $(am__DEPENDENCIES_1)
am_uptftopl_OBJECTS =
@@ -1694,12 +1714,12 @@
xetexdir/xetex_version.h synctexdir/synctex.c \
synctexdir/synctex.h synctexdir/synctex-common.h \
synctexdir/synctex-xetex.h
- at XETEX_SYNCTEX_TRUE@am__objects_73 = \
+ at XETEX_SYNCTEX_TRUE@am__objects_75 = \
@XETEX_SYNCTEX_TRUE@ synctexdir/xetex-synctex.$(OBJEXT)
dist_xetex_OBJECTS = xetexdir/xetex-xetexextra.$(OBJEXT) \
- $(am__objects_73)
-am__objects_74 = xetex-xetexini.$(OBJEXT) xetex-xetex0.$(OBJEXT)
-nodist_xetex_OBJECTS = $(am__objects_74) xetex-xetex-pool.$(OBJEXT)
+ $(am__objects_75)
+am__objects_76 = xetex-xetexini.$(OBJEXT) xetex-xetex0.$(OBJEXT)
+nodist_xetex_OBJECTS = $(am__objects_76) xetex-xetex-pool.$(OBJEXT)
xetex_OBJECTS = $(dist_xetex_OBJECTS) $(nodist_xetex_OBJECTS)
@XETEX_MACOSX_FALSE at am__DEPENDENCIES_18 = $(am__DEPENDENCIES_1)
am__DEPENDENCIES_19 = $(libxetex) $(am__DEPENDENCIES_1) \
@@ -1750,7 +1770,15 @@
./$(DEPDIR)/euptex-euptex-pool.Po \
./$(DEPDIR)/euptex-euptex0.Po ./$(DEPDIR)/euptex-euptexini.Po \
./$(DEPDIR)/gftodvi.Po ./$(DEPDIR)/gftopk.Po \
- ./$(DEPDIR)/gftype.Po ./$(DEPDIR)/initex-callexe.Po \
+ ./$(DEPDIR)/gftype.Po ./$(DEPDIR)/hishrink-hformat.Po \
+ ./$(DEPDIR)/hishrink-hishrink.Po \
+ ./$(DEPDIR)/hishrink-shrink-lexer.Po \
+ ./$(DEPDIR)/hishrink-shrink-parser.Po \
+ ./$(DEPDIR)/histretch-hformat.Po \
+ ./$(DEPDIR)/histretch-histretch.Po \
+ ./$(DEPDIR)/hitex-hformat.Po ./$(DEPDIR)/hitex-hitex.Po \
+ ./$(DEPDIR)/hitex-hput.Po ./$(DEPDIR)/hitex-htex.Po \
+ ./$(DEPDIR)/initex-callexe.Po \
./$(DEPDIR)/libluahbtexspecific_a-luainit-hb.Po \
./$(DEPDIR)/libluahbtexspecific_a-luastuff-hb.Po \
./$(DEPDIR)/libluahbtexspecific_a-printing-hb.Po \
@@ -2433,6 +2461,15 @@
am__v_CXXLD_ = $(am__v_CXXLD_ at AM_DEFAULT_V@)
am__v_CXXLD_0 = @echo " CXXLD " $@;
am__v_CXXLD_1 =
+ at MAINTAINER_MODE_FALSE@am__skiplex = test -f $@ ||
+LEXCOMPILE = $(LEX) $(AM_LFLAGS) $(LFLAGS)
+LTLEXCOMPILE = $(LIBTOOL) $(AM_V_lt) $(AM_LIBTOOLFLAGS) \
+ $(LIBTOOLFLAGS) --mode=compile $(LEX) $(AM_LFLAGS) $(LFLAGS)
+AM_V_LEX = $(am__v_LEX_ at AM_V@)
+am__v_LEX_ = $(am__v_LEX_ at AM_DEFAULT_V@)
+am__v_LEX_0 = @echo " LEX " $@;
+am__v_LEX_1 =
+YLWRAP = $(top_srcdir)/../../build-aux/ylwrap
OBJCXXCOMPILE = $(OBJCXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) \
$(AM_CPPFLAGS) $(CPPFLAGS) $(AM_OBJCXXFLAGS) $(OBJCXXFLAGS)
LTOBJCXXCOMPILE = $(LIBTOOL) $(AM_V_lt) $(AM_LIBTOOLFLAGS) \
@@ -2451,6 +2488,16 @@
am__v_OBJCXXLD_ = $(am__v_OBJCXXLD_ at AM_DEFAULT_V@)
am__v_OBJCXXLD_0 = @echo " OBJCXXLD" $@;
am__v_OBJCXXLD_1 =
+ at MAINTAINER_MODE_FALSE@am__skipyacc = test -f $@ ||
+am__yacc_c2h = sed -e s/cc$$/hh/ -e s/cpp$$/hpp/ -e s/cxx$$/hxx/ \
+ -e s/c++$$/h++/ -e s/c$$/h/
+YACCCOMPILE = $(YACC) $(AM_YFLAGS) $(YFLAGS)
+LTYACCCOMPILE = $(LIBTOOL) $(AM_V_lt) $(AM_LIBTOOLFLAGS) \
+ $(LIBTOOLFLAGS) --mode=compile $(YACC) $(AM_YFLAGS) $(YFLAGS)
+AM_V_YACC = $(am__v_YACC_ at AM_V@)
+am__v_YACC_ = $(am__v_YACC_ at AM_DEFAULT_V@)
+am__v_YACC_0 = @echo " YACC " $@;
+am__v_YACC_1 =
SOURCES = $(libff_a_SOURCES) $(libkanji_a_SOURCES) \
$(nodist_libluaffi_a_SOURCES) $(libluaharfbuzz_a_SOURCES) \
$(nodist_libluahbtexspecific_a_SOURCES) \
@@ -2493,8 +2540,10 @@
$(dist_euptex_SOURCES) $(nodist_euptex_SOURCES) \
$(gftodvi_SOURCES) $(nodist_gftodvi_SOURCES) \
$(nodist_gftopk_SOURCES) $(nodist_gftype_SOURCES) \
- $(nodist_initex_SOURCES) $(nodist_luahbtex_SOURCES) \
- $(nodist_EXTRA_luahbtex_SOURCES) $(nodist_luajithbtex_SOURCES) \
+ $(nodist_hishrink_SOURCES) $(nodist_histretch_SOURCES) \
+ $(nodist_hitex_SOURCES) $(nodist_initex_SOURCES) \
+ $(nodist_luahbtex_SOURCES) $(nodist_EXTRA_luahbtex_SOURCES) \
+ $(nodist_luajithbtex_SOURCES) \
$(nodist_EXTRA_luajithbtex_SOURCES) \
$(nodist_luajittex_SOURCES) $(nodist_luatex_SOURCES) \
$(md5main_SOURCES) $(dist_mf_SOURCES) $(dist_mf_nowin_SOURCES) \
@@ -2755,14 +2804,14 @@
bases=`echo $$bases`
AM_TESTSUITE_SUMMARY_HEADER = ' for $(PACKAGE_STRING)'
RECHECK_LOGS = $(TEST_LOGS)
-am__EXEEXT_43 = bibtex.test dvicopy.test dvitype.test gftodvi.test \
+am__EXEEXT_44 = bibtex.test dvicopy.test dvitype.test gftodvi.test \
gftopk.test gftype.test mft.test patgen.test pktogf.test \
pktype.test pltotf.test pooltype.test tftopl.test vftovp.test \
vptovf.test weave.test
-am__EXEEXT_44 = $(am__EXEEXT_43) tests/bibtex-openout-test.pl \
+am__EXEEXT_45 = $(am__EXEEXT_44) tests/bibtex-openout-test.pl \
tests/bibtex-longline-test.pl tests/bibtex-mem.test \
tests/bibtex-bigauth.test tests/bibtex-auxinclude.test
- at WEB_TRUE@am__EXEEXT_45 = $(am__EXEEXT_44)
+ at WEB_TRUE@am__EXEEXT_46 = $(am__EXEEXT_45)
TEST_SUITE_LOG = test-suite.log
am__test_logs1 = $(TESTS:=.log)
am__test_logs2 = $(am__test_logs1:@EXEEXT at .log=.log)
@@ -2790,7 +2839,8 @@
$(srcdir)/am/cweb.am $(srcdir)/am/texmf.am $(srcdir)/am/web.am \
$(srcdir)/c-auto.in $(srcdir)/ctangleboot-sh.in \
$(srcdir)/eptexdir/am/eptex.am $(srcdir)/etexdir/am/etex.am \
- $(srcdir)/euptexdir/am/euptex.am $(srcdir)/libmd5/am/md5.am \
+ $(srcdir)/euptexdir/am/euptex.am \
+ $(srcdir)/hitexdir/am/hitex.am $(srcdir)/libmd5/am/md5.am \
$(srcdir)/luatexdir/am/libluatex.am \
$(srcdir)/luatexdir/am/libunilib.am \
$(srcdir)/luatexdir/am/luaffi.am \
@@ -2822,6 +2872,7 @@
$(top_srcdir)/../../build-aux/ltmain.sh \
$(top_srcdir)/../../build-aux/missing \
$(top_srcdir)/../../build-aux/test-driver \
+ $(top_srcdir)/../../build-aux/ylwrap \
$(top_srcdir)/../texlive/w32_wrapper/callexe.c \
$(top_srcdir)/luatexdir/luafontloader/ff-config.in \
$(top_srcdir)/synctexdir/synctex.pc.in ../../build-aux/ar-lib \
@@ -2985,7 +3036,7 @@
NMEDIT = @NMEDIT@
OBJCXX = @OBJCXX@
OBJCXXDEPMODE = @OBJCXXDEPMODE@
-OBJCXXFLAGS = @OBJCXXFLAGS@ $(am__append_110)
+OBJCXXFLAGS = @OBJCXXFLAGS@ $(am__append_111)
OBJDUMP = @OBJDUMP@
OBJEXT = @OBJEXT@
OTANGLE = @OTANGLE@
@@ -3127,8 +3178,8 @@
libluajithbtexspecific.a libluaffi.a libluaharfbuzz.a \
libluajitharfbuzz.a libxetex.a libsynctex.a libmd5.a
EXTRA_LTLIBRARIES = libsynctex.la
-lib_LIBRARIES = $(am__append_122)
-lib_LTLIBRARIES = $(am__append_121)
+lib_LIBRARIES = $(am__append_123)
+lib_LTLIBRARIES = $(am__append_122)
dist_man_MANS = synctexdir/man1/synctex.1 synctexdir/man5/synctex.5
nodist_man_MANS =
TEST_EXTENSIONS = .pl .test
@@ -3245,21 +3296,21 @@
euptexdir/euptrip/euptrip.diffs euptexdir/euptrip/texmf.cnf \
euptexdir/pdfprimitive.test \
eptexdir/tests/pdfprimitive-test.tex \
- euptexdir/tests/pdfprimitive-euptex.log \
- pdftexdir/regex/COPYING.LIB pdftexdir/regex/README \
- $(pdftex_ch_srcs) pdftexdir/pdftex.defines pdftexdir/ChangeLog \
- pdftexdir/NEWS pdftexdir/README pdftexdir/change-files.txt \
- $(pdftex_tests) tests/wprob.tex pdftexdir/tests/pdfimage.tex \
- tests/1-4.jpg tests/B.pdf tests/basic.tex \
- tests/lily-ledger-broken.png tests/expanded.tex \
- tests/expanded.txt tests/cnfline.tex tests/partoken-ok.tex \
- tests/partoken-xfail.tex $(ttf2afm_tests) \
- pdftexdir/tests/postV3.afm pdftexdir/tests/postV3.ttf \
- pdftexdir/tests/postV7.afm pdftexdir/tests/postV7.ttf \
- $(pdftosrc_tests) pdftexdir/tests/test-13.pdf \
- pdftexdir/tests/test-13.xref pdftexdir/tests/test-15.pdf \
- pdftexdir/tests/test-15.xref $(libluasocket_sources) \
- luatexdir/luasocket/src/ftp_lua.c \
+ euptexdir/tests/pdfprimitive-euptex.log hitexdir/ChangeLog \
+ $(hitex_web) lex.sed yacc.sed pdftexdir/regex/COPYING.LIB \
+ pdftexdir/regex/README $(pdftex_ch_srcs) \
+ pdftexdir/pdftex.defines pdftexdir/ChangeLog pdftexdir/NEWS \
+ pdftexdir/README pdftexdir/change-files.txt $(pdftex_tests) \
+ tests/wprob.tex pdftexdir/tests/pdfimage.tex tests/1-4.jpg \
+ tests/B.pdf tests/basic.tex tests/lily-ledger-broken.png \
+ tests/expanded.tex tests/expanded.txt tests/cnfline.tex \
+ tests/partoken-ok.tex tests/partoken-xfail.tex \
+ $(ttf2afm_tests) pdftexdir/tests/postV3.afm \
+ pdftexdir/tests/postV3.ttf pdftexdir/tests/postV7.afm \
+ pdftexdir/tests/postV7.ttf $(pdftosrc_tests) \
+ pdftexdir/tests/test-13.pdf pdftexdir/tests/test-13.xref \
+ pdftexdir/tests/test-15.pdf pdftexdir/tests/test-15.xref \
+ $(libluasocket_sources) luatexdir/luasocket/src/ftp_lua.c \
luatexdir/luasocket/src/headers_lua.c \
luatexdir/luasocket/src/http_lua.c \
luatexdir/luasocket/src/ltn12_lua.c \
@@ -3568,7 +3619,10 @@
uptests/yuparse.* uptests/ygkhuge*.* uptrip.diffs \
$(nodist_euptex_SOURCES) euptex.web euptex.ch euptex-web2c \
euptex.p euptex.pool euptex-tangle euptrip.diffs \
- pdfprimitive-euptex.* $(nodist_pdftex_SOURCES) pdftex-final.ch \
+ pdfprimitive-euptex.* $(nodist_hitex_SOURCES) \
+ $(nodist_hishrink_SOURCES) $(nodist_histretch_SOURCES) \
+ format-tangle htex-tangle hitex-tangle shrink-parser.* \
+ shrink-lexer.* $(nodist_pdftex_SOURCES) pdftex-final.ch \
pdftex-web2c pdftex.p pdftex.pool pdftex-tangle pwprob.log \
pwprob.tex pdfimage.fmt pdfimage.log pdfimage.pdf expanded.log \
cnfline.log partoken-ok.log partoken-xfail.log postV3.afm \
@@ -3607,16 +3661,16 @@
$(am__append_79)
bin_links = $(am__append_5) $(am__append_14) $(am__append_15) \
$(am__append_24) $(am__append_33) $(am__append_41) \
- $(am__append_49) $(am__append_54) $(am__append_88) \
- $(am__append_93) $(am__append_98) $(am__append_103)
+ $(am__append_49) $(am__append_54) $(am__append_89) \
+ $(am__append_94) $(am__append_99) $(am__append_104)
install_exe_links = $(am__append_12) $(am__append_22) $(am__append_31) \
$(am__append_39) $(am__append_47) $(am__append_52) \
- $(am__append_86) $(am__append_91) $(am__append_96) \
- $(am__append_101)
+ $(am__append_87) $(am__append_92) $(am__append_97) \
+ $(am__append_102)
uninstall_exe_links = $(am__append_13) $(am__append_23) \
$(am__append_32) $(am__append_40) $(am__append_48) \
- $(am__append_53) $(am__append_87) $(am__append_92) \
- $(am__append_97) $(am__append_102)
+ $(am__append_53) $(am__append_88) $(am__append_93) \
+ $(am__append_98) $(am__append_103)
NEVER_DIST = `find . $(NEVER_NAMES)` cwebdir/cweave.log \
cwebdir/cweave.trs cwebdir/ctwill.log cwebdir/ctwill.trs \
cwebdir/refsort.log cwebdir/refsort.trs cwebdir/twinx.log \
@@ -3787,18 +3841,18 @@
initex_CPPFLAGS = -DEXEPROG=\"tex.exe\"
nodist_initex_SOURCES = callexe.c
initex_LDADD =
-tex_CPPFLAGS = $(AM_CPPFLAGS) $(am__append_125)
+tex_CPPFLAGS = $(AM_CPPFLAGS) $(am__append_126)
# With --enable-ipc, TeX may need to link with -lsocket.
-tex_LDADD = $(LDADD) $(ipc_socketlibs) $(am__append_126)
+tex_LDADD = $(LDADD) $(ipc_socketlibs) $(am__append_127)
# TeX C sources
tex_c_h = texini.c tex0.c texcoerce.h texd.h
nodist_tex_SOURCES = $(tex_c_h) tex-pool.c
-dist_tex_SOURCES = texextra.c $(am__append_128)
+dist_tex_SOURCES = texextra.c $(am__append_129)
# We must create texd.h before building the tex_OBJECTS.
-tex_prereq = texd.h $(am__append_127)
+tex_prereq = texd.h $(am__append_128)
tex_ch_srcs = \
tex.web \
tex.ch \
@@ -4274,10 +4328,10 @@
mplibdir/pngout.w mplibdir/mpmath.w mplibdir/mpmathbinary.w \
mplibdir/mpmathdecimal.w mplibdir/mpmathdouble.w \
mplibdir/mpstrings.w mplibdir/tfmin.w
-etex_CPPFLAGS = $(AM_CPPFLAGS) $(ZLIB_INCLUDES) $(am__append_129)
+etex_CPPFLAGS = $(AM_CPPFLAGS) $(ZLIB_INCLUDES) $(am__append_130)
# With --enable-ipc, e-TeX may need to link with -lsocket.
-etex_LDADD = $(LDADD) $(ZLIB_LIBS) $(ipc_socketlibs) $(am__append_130)
+etex_LDADD = $(LDADD) $(ZLIB_LIBS) $(ipc_socketlibs) $(am__append_131)
etex_DEPENDENCIES = $(ZLIB_DEPEND)
# e-TeX C sources
@@ -4284,11 +4338,11 @@
etex_c_h = etexini.c etex0.c etexcoerce.h etexd.h
nodist_etex_SOURCES = $(etex_c_h) etex-pool.c
dist_etex_SOURCES = etexdir/etexextra.c etexdir/etexextra.h \
- etexdir/etex_version.h $(am__append_132)
+ etexdir/etex_version.h $(am__append_133)
# We must create etexd.h and etexdir/etex_version.h before building the
# etex_OBJECTS.
-etex_prereq = etexd.h etexdir/etex_version.h $(am__append_131)
+etex_prereq = etexd.h etexdir/etex_version.h $(am__append_132)
etex_web_srcs = \
tex.web \
etexdir/etex.ch
@@ -4317,10 +4371,10 @@
pproglib = lib/libp.a
libkanji_a_SOURCES = ptexdir/kanji.c ptexdir/kanji.h ptexdir/kanji_dump.c
libkanji_a_CPPFLAGS = $(ptex_cppflags)
-ptex_CPPFLAGS = $(ptex_cppflags) $(am__append_133)
+ptex_CPPFLAGS = $(ptex_cppflags) $(am__append_134)
# With --enable-ipc, pTeX may need to link with -lsocket.
-ptex_LDADD = $(ptex_ldadd) $(ipc_socketlibs) $(am__append_134)
+ptex_LDADD = $(ptex_ldadd) $(ipc_socketlibs) $(am__append_135)
ptex_DEPENDENCIES = $(ptex_dependencies)
# pTeX C sources
@@ -4327,10 +4381,10 @@
ptex_c_h = ptexini.c ptex0.c ptexcoerce.h ptexd.h
nodist_ptex_SOURCES = $(ptex_c_h) ptex-pool.c
dist_ptex_SOURCES = ptexdir/ptexextra.c ptexdir/ptexextra.h \
- ptexdir/ptex_version.h $(am__append_136)
+ ptexdir/ptex_version.h $(am__append_137)
# We must create ptexd.h and ptexdir/ptex_version.h before building the ptex_OBJECTS.
-ptex_prereq = ptexd.h ptexdir/ptex_version.h $(am__append_135)
+ptex_prereq = ptexd.h ptexdir/ptex_version.h $(am__append_136)
ptex_web_srcs = \
tex.web \
tex.ch \
@@ -4374,11 +4428,11 @@
ptexdir/nissya.test ptexdir/sample.test ptexdir/yokotate.test \
ptexdir/skipjfmp.test
eptex_CPPFLAGS = $(PTEXENC_INCLUDES) $(AM_CPPFLAGS) -I$(srcdir)/libmd5 \
- $(ZLIB_INCLUDES) $(am__append_141)
+ $(ZLIB_INCLUDES) $(am__append_142)
# With --enable-ipc, e-pTeX may need to link with -lsocket.
eptex_LDADD = libkanji.a $(pproglib) $(PTEXENC_LIBS) $(LDADD) \
- $(ipc_socketlibs) libmd5.a $(ZLIB_LIBS) $(am__append_142)
+ $(ipc_socketlibs) libmd5.a $(ZLIB_LIBS) $(am__append_143)
eptex_DEPENDENCIES = libkanji.a $(pproglib) $(PTEXENC_DEPEND) $(default_dependencies) libmd5.a $(ZLIB_DEPEND)
# e-pTeX C sources
@@ -4385,11 +4439,11 @@
eptex_c_h = eptexini.c eptex0.c eptexcoerce.h eptexd.h
nodist_eptex_SOURCES = $(eptex_c_h) eptex-pool.c
dist_eptex_SOURCES = eptexdir/eptexextra.c eptexdir/eptexextra.h \
- eptexdir/eptex_version.h $(am__append_144)
+ eptexdir/eptex_version.h $(am__append_145)
# We must create eptexd.h and eptexdir/eptex_version.h before building the eptex_OBJECTS.
eptex_prereq = eptexd.h etexdir/etex_version.h ptexdir/ptex_version.h \
- eptexdir/eptex_version.h $(am__append_143)
+ eptexdir/eptex_version.h $(am__append_144)
eptex_web_srcs = \
tex.web \
etexdir/etex.ch \
@@ -4423,10 +4477,10 @@
upweb_programs = upbibtex updvitype uppltotf uptftopl
libukanji_a_SOURCES = uptexdir/kanji.c uptexdir/kanji.h uptexdir/kanji_dump.c
libukanji_a_CPPFLAGS = $(uptex_cppflags)
-uptex_CPPFLAGS = $(uptex_cppflags) $(am__append_137)
+uptex_CPPFLAGS = $(uptex_cppflags) $(am__append_138)
# With --enable-ipc, upTeX may need to link with -lsocket.
-uptex_LDADD = $(uptex_ldadd) $(ipc_socketlibs) $(am__append_138)
+uptex_LDADD = $(uptex_ldadd) $(ipc_socketlibs) $(am__append_139)
uptex_DEPENDENCIES = $(uptex_dependencies)
# upTeX C sources
@@ -4433,11 +4487,11 @@
uptex_c_h = uptexini.c uptex0.c uptexcoerce.h uptexd.h
nodist_uptex_SOURCES = $(uptex_c_h) uptex-pool.c
dist_uptex_SOURCES = uptexdir/uptexextra.c uptexdir/uptexextra.h \
- uptexdir/uptex_version.h $(am__append_140)
+ uptexdir/uptex_version.h $(am__append_141)
# We must create uptexd.h and uptexdir/uptex_version.h before building the uptex_OBJECTS.
uptex_prereq = uptexd.h ptexdir/ptex_version.h \
- uptexdir/uptex_version.h $(am__append_139)
+ uptexdir/uptex_version.h $(am__append_140)
uptex_web_srcs = \
tex.web \
tex.ch \
@@ -4485,11 +4539,11 @@
uptexdir/gkhuge.test
euptex_CPPFLAGS = $(PTEXENC_INCLUDES) $(AM_CPPFLAGS) \
- -I$(srcdir)/libmd5 $(ZLIB_INCLUDES) $(am__append_145)
+ -I$(srcdir)/libmd5 $(ZLIB_INCLUDES) $(am__append_146)
# With --enable-ipc, e-upTeX may need to link with -lsocket.
euptex_LDADD = libukanji.a $(pproglib) $(PTEXENC_LIBS) $(LDADD) \
- $(ipc_socketlibs) libmd5.a $(ZLIB_LIBS) $(am__append_146)
+ $(ipc_socketlibs) libmd5.a $(ZLIB_LIBS) $(am__append_147)
euptex_DEPENDENCIES = libukanji.a $(pproglib) $(PTEXENC_DEPEND) $(default_dependencies) libmd5.a $(ZLIB_DEPEND)
# e-upTeX C sources
@@ -4496,12 +4550,12 @@
euptex_c_h = euptexini.c euptex0.c euptexcoerce.h euptexd.h
nodist_euptex_SOURCES = $(euptex_c_h) euptex-pool.c
dist_euptex_SOURCES = euptexdir/euptexextra.c euptexdir/euptexextra.h \
- $(am__append_148)
+ $(am__append_149)
# We must create euptexd.h and [eu]ptexdir/[eu]ptex_version.h before building the euptex_OBJECTS.
euptex_prereq = euptexd.h etexdir/etex_version.h \
ptexdir/ptex_version.h eptexdir/eptex_version.h \
- uptexdir/uptex_version.h $(am__append_147)
+ uptexdir/uptex_version.h $(am__append_148)
euptex_web_srcs = \
tex.web \
etexdir/etex.ch \
@@ -4532,6 +4586,47 @@
# e-upTeX Tests
#
euptex_tests = euptexdir/euptriptest.test euptexdir/pdfprimitive.test euptexdir/eupver.test
+
+#kb hitex_CFLAGS = $(AM_CFLAGS) \
+#kb -Wno-parentheses -Wno-maybe-uninitialized -Wno-unused-function -Wno-unused-variable
+hitex_CPPFLAGS = $(AM_CPPFLAGS) $(ZLIB_INCLUDES)
+hitex_LDADD = $(KPATHSEA_LIBS) $(ZLIB_LIBS)
+#
+histretch_CPPFLAGS = $(AM_CPPFLAGS) $(ZLIB_INCLUDES)
+histretch_LDADD = $(ZLIB_LIBS)
+#
+hishrink_CPPFLAGS = $(AM_CPPFLAGS) $(ZLIB_INCLUDES)
+hishrink_LDADD = $(ZLIB_LIBS)
+
+# We want debugging for the parser, and we use features not in POSIX yacc.
+# I guess the result is that we assume bison? Not sure if that's ok.
+AM_YFLAGS = -d -v -Wno-yacc
+
+# HiTeX CWEB sources
+hitex_web = hitexdir/format.w hitexdir/htex.w hitexdir/hitex.w
+
+# Creating several files: need stamp file and two rules with identical recipes
+hi_ctangle_sh = CWEBINPUTS=$(srcdir)/hitexdir AM_V_P=$(AM_V_P) $(SHELL) ./tangle-sh $@ $(CTANGLE)
+
+# HiTeX C/yacc/lex sources generated using ctangle.
+# For each cweb (.w) source file, list the files generated.
+format_c_h_l_y = basetypes.h error.h hformat.h \
+ mkhformat.c \
+ hput.c hput.h \
+ hget.c hget.h \
+ shrink.lex-in shrink.yacc-in \
+ hishrink.c histretch.c
+
+htex_c_h = htex.c htex.h
+hitex_c_h = hitex.c hitex.h
+
+# Other web2c/*dir/am/*.am engine Makefile fragments define a bunch of
+# _OBJECTS variables. This does not seem right to me. Instead, we
+# correctly define the _SOURCES.
+#
+nodist_hitex_SOURCES = hformat.c hput.c $(htex_c_h) $(hitex_c_h)
+nodist_histretch_SOURCES = hformat.c histretch.c
+nodist_hishrink_SOURCES = hformat.c hishrink.c shrink-lexer.l shrink-parser.y
libpdftex_a_CPPFLAGS = $(REGEX_INCLUDES) $(pdftex_cppflags)
libpdftex_a_CXXFLAGS = $(WARNING_CXXFLAGS)
libpdftex_a_SOURCES = pdftexdir/avl.c pdftexdir/avl.h \
@@ -4545,7 +4640,7 @@
pdftexdir/writejbig2.c pdftexdir/writejpg.c \
pdftexdir/writepng.c pdftexdir/writet1.c pdftexdir/writet3.c \
pdftexdir/writettf.c pdftexdir/writettf.h pdftexdir/writezip.c \
- $(am__append_81)
+ $(am__append_82)
@MINGW32_TRUE at REGEX_INCLUDES = -I$(srcdir)/pdftexdir/regex
EXTRA_libpdftex_a_SOURCES = pdftexdir/macnames.c \
pdftexdir/regex/regcomp.c pdftexdir/regex/regex_internal.c \
@@ -4561,7 +4656,7 @@
# Force Automake to use CXXLD for linking
nodist_EXTRA_pdftex_SOURCES = dummy.cxx
pdf_tangle = WEBINPUTS=.:$(srcdir)/pdftexdir AM_V_P=$(AM_V_P) $(SHELL) ./tangle-sh $@ $(TANGLE)
-pdftex_CPPFLAGS = $(pdftex_cppflags) $(am__append_149)
+pdftex_CPPFLAGS = $(pdftex_cppflags) $(am__append_150)
pdftex_CXXFLAGS = $(WARNING_CXXFLAGS)
# With --enable-ipc, pdfTeX may need to link with -lsocket.
@@ -4574,7 +4669,7 @@
nodist_pdftex_SOURCES = $(pdftex_c_h) pdftex-pool.c
dist_pdftex_SOURCES = pdftexdir/pdftexextra.c pdftexdir/pdftexextra.h \
pdftexdir/pdftex_version.h pdftexdir/etex_version.h \
- $(am__append_150)
+ $(am__append_151)
pdftex_ch_srcs = \
pdftexdir/pdftex.web \
pdftexdir/tex.ch0 \
@@ -5248,18 +5343,18 @@
xetex_cppflags = $(AM_CPPFLAGS) -I$(srcdir)/xetexdir $(ICU_INCLUDES) \
$(FREETYPE2_INCLUDES) $(TECKIT_INCLUDES) $(HARFBUZZ_INCLUDES) \
$(GRAPHITE2_INCLUDES) $(LIBPNG_INCLUDES) $(ZLIB_INCLUDES) \
- $(PPLIB_INCLUDES) -I$(srcdir)/libmd5 $(am__append_109) \
- $(am__append_111)
+ $(PPLIB_INCLUDES) -I$(srcdir)/libmd5 $(am__append_110) \
+ $(am__append_112)
xetex_ldadd = $(libxetex) $(HARFBUZZ_LIBS) $(GRAPHITE2_LIBS) \
$(ICU_LIBS) $(ICU_LIBS_EXTRA) $(TECKIT_LIBS) $(LIBPNG_LIBS) \
$(FREETYPE2_LIBS) $(PPLIB_LIBS) $(ZLIB_LIBS) libmd5.a \
- $(am__append_112)
+ $(am__append_113)
xetex_dependencies = $(proglib) $(KPATHSEA_DEPEND) $(ICU_DEPEND) \
$(TECKIT_DEPEND) $(HARFBUZZ_DEPEND) $(GRAPHITE2_DEPEND) \
$(LIBPNG_DEPEND) $(FREETYPE2_DEPEND) $(ZLIB_DEPEND) \
$(PPLIB_DEPEND) libmd5.a
@XETEX_MACOSX_TRUE at xetex_LDFLAGS = -framework ApplicationServices -framework Cocoa
-xetex_CPPFLAGS = $(xetex_cppflags) $(am__append_151)
+xetex_CPPFLAGS = $(xetex_cppflags) $(am__append_152)
xetex_CFLAGS = $(WARNING_CFLAGS)
xetex_CXXFLAGS = # $(WARNING_CXXFLAGS)
xetex_LDADD = $(xetex_ldadd) $(LDADD) $(ipc_socketlibs)
@@ -5268,7 +5363,7 @@
nodist_xetex_SOURCES = $(xetex_c_h) xetex-pool.c
dist_xetex_SOURCES = xetexdir/xetexextra.c xetexdir/xetexextra.h \
xetexdir/etex_version.h xetexdir/xetex_version.h \
- $(am__append_152)
+ $(am__append_153)
xetex_ch_srcs = \
xetexdir/xetex.web \
xetexdir/tex.ch0 \
@@ -5299,7 +5394,7 @@
xetexdir/image/jpegimage.h xetexdir/image/mfileio.c \
xetexdir/image/mfileio.h xetexdir/image/numbers.c \
xetexdir/image/numbers.h xetexdir/image/pngimage.c \
- xetexdir/image/pngimage.h $(am__append_113) $(am__append_114)
+ xetexdir/image/pngimage.h $(am__append_114) $(am__append_115)
# We must create xetexd.h etc. before building the libxetex_a_OBJECTS.
libxetex_prereq = xetexd.h $(xetex_dependencies)
@@ -5404,12 +5499,12 @@
synctexdir/synctex_main.c
synctex_CPPFLAGS = -I$(srcdir)/synctexdir
-synctex_LDADD = $(libsynctex) $(ZLIB_LIBS) $(am__append_123)
+synctex_LDADD = $(libsynctex) $(ZLIB_LIBS) $(am__append_124)
libsynctex = $(LTLIBSYNCTEX) $(LIBSYNCTEX)
libsynctex_la_CPPFLAGS = -I$(srcdir)/synctexdir $(ZLIB_INCLUDES) -DSYNCTEX_USE_LOCAL_HEADER
libsynctex_a_CPPFLAGS = $(libsynctex_la_CPPFLAGS)
libsynctex_la_LDFLAGS = -rpath @libdir@ -bindir @bindir@ -no-undefined -version-info $(SYNCTEX_LT_VERSINFO)
-libsynctex_la_LIBADD = $(ZLIB_LIBS) $(am__append_124)
+libsynctex_la_LIBADD = $(ZLIB_LIBS) $(am__append_125)
libsynctex_la_SOURCES = \
synctexdir/synctex_parser.c \
synctexdir/synctex_parser_local.h \
@@ -5523,10 +5618,10 @@
$(MAKE) $(AM_MAKEFLAGS) all-recursive
.SUFFIXES:
-.SUFFIXES: .c .cc .cin .cpp .cxx .h .hin .lo .log .mm .o .obj .p .pin .pl .pl$(EXEEXT) .test .test$(EXEEXT) .trs
+.SUFFIXES: .c .cc .cin .cpp .cxx .h .hin .l .lo .log .mm .o .obj .p .pin .pl .pl$(EXEEXT) .test .test$(EXEEXT) .trs .y
am--refresh: Makefile
@:
-$(srcdir)/Makefile.in: @MAINTAINER_MODE_TRUE@ $(srcdir)/Makefile.am $(srcdir)/../../am/dist_hook.am $(srcdir)/am/bootstrap.am $(srcdir)/am/web.am $(srcdir)/am/cweb.am $(srcdir)/am/texmf.am $(srcdir)/mfluadir/am/mflua.am $(srcdir)/mfluadir/am/mfluaotfcc.am $(srcdir)/mfluajitdir/am/mfluajit.am $(srcdir)/mplibdir/am/mplib.am $(srcdir)/pmpostdir/am/pmpost.am $(srcdir)/mplibdir/am/libmputil.am $(srcdir)/mplibdir/am/libmplib.am $(srcdir)/etexdir/am/etex.am $(srcdir)/ptexdir/am/ptex.am $(srcdir)/eptexdir/am/eptex.am $(srcdir)/uptexdir/am/uptex.am $(srcdir)/euptexdir/am/euptex.am $(srcdir)/pdftexdir/am/libpdftex.am $(srcdir)/pdftexdir/am/pdftex.am $(srcdir)/pdftexdir/am/ttf2afm.am $(srcdir)/pdftexdir/am/pdftosrc.am $(srcdir)/luatexdir/am/luasocket.am $(srcdir)/luatexdir/am/luamisc.am $(srcdir)/luatexdir/am/libunilib.am $(srcdir)/luatexdir/am/luafontforge.am $(srcdir)/luatexdir/am/libluatex.am $(srcdir)/luatexdir/am/luaffi.am $(srcdir)/luatexdir/am/luatex.am $(srcdir)/luatexdir/am/luaharfbuzz.am $(srcdir)/xetexdir/am/xetex.am $(srcdir)/omegaware/am/omegaware.am $(srcdir)/alephdir/am/aleph.am $(srcdir)/synctexdir/am/synctex.am $(srcdir)/libmd5/am/md5.am $(srcdir)/../../am/bin_links.am $(am__configure_deps)
+$(srcdir)/Makefile.in: @MAINTAINER_MODE_TRUE@ $(srcdir)/Makefile.am $(srcdir)/../../am/dist_hook.am $(srcdir)/am/bootstrap.am $(srcdir)/am/web.am $(srcdir)/am/cweb.am $(srcdir)/am/texmf.am $(srcdir)/mfluadir/am/mflua.am $(srcdir)/mfluadir/am/mfluaotfcc.am $(srcdir)/mfluajitdir/am/mfluajit.am $(srcdir)/mplibdir/am/mplib.am $(srcdir)/pmpostdir/am/pmpost.am $(srcdir)/mplibdir/am/libmputil.am $(srcdir)/mplibdir/am/libmplib.am $(srcdir)/etexdir/am/etex.am $(srcdir)/ptexdir/am/ptex.am $(srcdir)/eptexdir/am/eptex.am $(srcdir)/uptexdir/am/uptex.am $(srcdir)/euptexdir/am/euptex.am $(srcdir)/hitexdir/am/hitex.am $(srcdir)/pdftexdir/am/libpdftex.am $(srcdir)/pdftexdir/am/pdftex.am $(srcdir)/pdftexdir/am/ttf2afm.am $(srcdir)/pdftexdir/am/pdftosrc.am $(srcdir)/luatexdir/am/luasocket.am $(srcdir)/luatexdir/am/luamisc.am $(srcdir)/luatexdir/am/libunilib.am $(srcdir)/luatexdir/am/luafontforge.am $(srcdir)/luatexdir/am/libluatex.am $(srcdir)/luatexdir/am/luaffi.am $(srcdir)/luatexdir/am/luatex.am $(srcdir)/luatexdir/am/luaharfbuzz.am $(srcdir)/xetexdir/am/xetex.am $(srcdir)/omegaware/am/omegaware.am $(srcdir)/alephdir/am/aleph.am $(srcdir)/synctexdir/am/synctex.am $(srcdir)/libmd5/am/md5.am $(srcdir)/../../am/bin_links.am $(am__configure_deps)
@for dep in $?; do \
case '$(am__configure_deps)' in \
*$$dep*) \
@@ -5548,7 +5643,7 @@
echo ' cd $(top_builddir) && $(SHELL) ./config.status $@ $(am__maybe_remake_depfiles)'; \
cd $(top_builddir) && $(SHELL) ./config.status $@ $(am__maybe_remake_depfiles);; \
esac;
-$(srcdir)/../../am/dist_hook.am $(srcdir)/am/bootstrap.am $(srcdir)/am/web.am $(srcdir)/am/cweb.am $(srcdir)/am/texmf.am $(srcdir)/mfluadir/am/mflua.am $(srcdir)/mfluadir/am/mfluaotfcc.am $(srcdir)/mfluajitdir/am/mfluajit.am $(srcdir)/mplibdir/am/mplib.am $(srcdir)/pmpostdir/am/pmpost.am $(srcdir)/mplibdir/am/libmputil.am $(srcdir)/mplibdir/am/libmplib.am $(srcdir)/etexdir/am/etex.am $(srcdir)/ptexdir/am/ptex.am $(srcdir)/eptexdir/am/eptex.am $(srcdir)/uptexdir/am/uptex.am $(srcdir)/euptexdir/am/euptex.am $(srcdir)/pdftexdir/am/libpdftex.am $(srcdir)/pdftexdir/am/pdftex.am $(srcdir)/pdftexdir/am/ttf2afm.am $(srcdir)/pdftexdir/am/pdftosrc.am $(srcdir)/luatexdir/am/luasocket.am $(srcdir)/luatexdir/am/luamisc.am $(srcdir)/luatexdir/am/libunilib.am $(srcdir)/luatexdir/am/luafontforge.am $(srcdir)/luatexdir/am/libluatex.am $(srcdir)/luatexdir/am/luaffi.am $(srcdir)/luatexdir/am/luatex.am $(srcdir)/luatexdir/am/luaharfbuzz.am $(srcdir)/xetexdir/am/xetex.am $(srcdir)/omegaware/am/omegaware.am $(srcdir)/alephdir/am/aleph.am $(srcdir)/synctexdir/am/synctex.am $(srcdir)/libmd5/am/md5.am $(srcdir)/../../am/bin_links.am $(am__empty):
+$(srcdir)/../../am/dist_hook.am $(srcdir)/am/bootstrap.am $(srcdir)/am/web.am $(srcdir)/am/cweb.am $(srcdir)/am/texmf.am $(srcdir)/mfluadir/am/mflua.am $(srcdir)/mfluadir/am/mfluaotfcc.am $(srcdir)/mfluajitdir/am/mfluajit.am $(srcdir)/mplibdir/am/mplib.am $(srcdir)/pmpostdir/am/pmpost.am $(srcdir)/mplibdir/am/libmputil.am $(srcdir)/mplibdir/am/libmplib.am $(srcdir)/etexdir/am/etex.am $(srcdir)/ptexdir/am/ptex.am $(srcdir)/eptexdir/am/eptex.am $(srcdir)/uptexdir/am/uptex.am $(srcdir)/euptexdir/am/euptex.am $(srcdir)/hitexdir/am/hitex.am $(srcdir)/pdftexdir/am/libpdftex.am $(srcdir)/pdftexdir/am/pdftex.am $(srcdir)/pdftexdir/am/ttf2afm.am $(srcdir)/pdftexdir/am/pdftosrc.am $(srcdir)/luatexdir/am/luasocket.am $(srcdir)/luatexdir/am/luamisc.am $(srcdir)/luatexdir/am/libunilib.am $(srcdir)/luatexdir/am/luafontforge.am $(srcdir)/luatexdir/am/libluatex.am $(srcdir)/luatexdir/am/luaffi.am $(srcdir)/luatexdir/am/luatex.am $(srcdir)/luatexdir/am/luaharfbuzz.am $(srcdir)/xetexdir/am/xetex.am $(srcdir)/omegaware/am/omegaware.am $(srcdir)/alephdir/am/aleph.am $(srcdir)/synctexdir/am/synctex.am $(srcdir)/libmd5/am/md5.am $(srcdir)/../../am/bin_links.am $(am__empty):
$(top_builddir)/config.status: $(top_srcdir)/configure $(CONFIG_STATUS_DEPENDENCIES)
$(SHELL) ./config.status --recheck
@@ -7950,7 +8045,22 @@
gftype$(EXEEXT): $(gftype_OBJECTS) $(gftype_DEPENDENCIES) $(EXTRA_gftype_DEPENDENCIES)
@rm -f gftype$(EXEEXT)
$(AM_V_CCLD)$(LINK) $(gftype_OBJECTS) $(gftype_LDADD) $(LIBS)
+shrink-parser.h: shrink-parser.c
+ @if test ! -f $@; then rm -f shrink-parser.c; else :; fi
+ @if test ! -f $@; then $(MAKE) $(AM_MAKEFLAGS) shrink-parser.c; else :; fi
+hishrink$(EXEEXT): $(hishrink_OBJECTS) $(hishrink_DEPENDENCIES) $(EXTRA_hishrink_DEPENDENCIES)
+ @rm -f hishrink$(EXEEXT)
+ $(AM_V_CCLD)$(LINK) $(hishrink_OBJECTS) $(hishrink_LDADD) $(LIBS)
+
+histretch$(EXEEXT): $(histretch_OBJECTS) $(histretch_DEPENDENCIES) $(EXTRA_histretch_DEPENDENCIES)
+ @rm -f histretch$(EXEEXT)
+ $(AM_V_CCLD)$(LINK) $(histretch_OBJECTS) $(histretch_LDADD) $(LIBS)
+
+hitex$(EXEEXT): $(hitex_OBJECTS) $(hitex_DEPENDENCIES) $(EXTRA_hitex_DEPENDENCIES)
+ @rm -f hitex$(EXEEXT)
+ $(AM_V_CCLD)$(LINK) $(hitex_OBJECTS) $(hitex_LDADD) $(LIBS)
+
initex$(EXEEXT): $(initex_OBJECTS) $(initex_DEPENDENCIES) $(EXTRA_initex_DEPENDENCIES)
@rm -f initex$(EXEEXT)
$(AM_V_CCLD)$(LINK) $(initex_OBJECTS) $(initex_LDADD) $(LIBS)
@@ -8331,6 +8441,16 @@
@AMDEP_TRUE@@am__include@ @am__quote at ./$(DEPDIR)/gftodvi.Po at am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote at ./$(DEPDIR)/gftopk.Po at am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote at ./$(DEPDIR)/gftype.Po at am__quote@ # am--include-marker
+ at AMDEP_TRUE@@am__include@ @am__quote at ./$(DEPDIR)/hishrink-hformat.Po at am__quote@ # am--include-marker
+ at AMDEP_TRUE@@am__include@ @am__quote at ./$(DEPDIR)/hishrink-hishrink.Po at am__quote@ # am--include-marker
+ at AMDEP_TRUE@@am__include@ @am__quote at ./$(DEPDIR)/hishrink-shrink-lexer.Po at am__quote@ # am--include-marker
+ at AMDEP_TRUE@@am__include@ @am__quote at ./$(DEPDIR)/hishrink-shrink-parser.Po at am__quote@ # am--include-marker
+ at AMDEP_TRUE@@am__include@ @am__quote at ./$(DEPDIR)/histretch-hformat.Po at am__quote@ # am--include-marker
+ at AMDEP_TRUE@@am__include@ @am__quote at ./$(DEPDIR)/histretch-histretch.Po at am__quote@ # am--include-marker
+ at AMDEP_TRUE@@am__include@ @am__quote at ./$(DEPDIR)/hitex-hformat.Po at am__quote@ # am--include-marker
+ at AMDEP_TRUE@@am__include@ @am__quote at ./$(DEPDIR)/hitex-hitex.Po at am__quote@ # am--include-marker
+ at AMDEP_TRUE@@am__include@ @am__quote at ./$(DEPDIR)/hitex-hput.Po at am__quote@ # am--include-marker
+ at AMDEP_TRUE@@am__include@ @am__quote at ./$(DEPDIR)/hitex-htex.Po at am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote at ./$(DEPDIR)/initex-callexe.Po at am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote at ./$(DEPDIR)/libluahbtexspecific_a-luainit-hb.Po at am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote at ./$(DEPDIR)/libluahbtexspecific_a-luastuff-hb.Po at am__quote@ # am--include-marker
@@ -17248,6 +17368,146 @@
@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(euptex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o euptex-euptex-pool.obj `if test -f 'euptex-pool.c'; then $(CYGPATH_W) 'euptex-pool.c'; else $(CYGPATH_W) '$(srcdir)/euptex-pool.c'; fi`
+hishrink-hformat.o: hformat.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hishrink_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT hishrink-hformat.o -MD -MP -MF $(DEPDIR)/hishrink-hformat.Tpo -c -o hishrink-hformat.o `test -f 'hformat.c' || echo '$(srcdir)/'`hformat.c
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/hishrink-hformat.Tpo $(DEPDIR)/hishrink-hformat.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='hformat.c' object='hishrink-hformat.o' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hishrink_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o hishrink-hformat.o `test -f 'hformat.c' || echo '$(srcdir)/'`hformat.c
+
+hishrink-hformat.obj: hformat.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hishrink_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT hishrink-hformat.obj -MD -MP -MF $(DEPDIR)/hishrink-hformat.Tpo -c -o hishrink-hformat.obj `if test -f 'hformat.c'; then $(CYGPATH_W) 'hformat.c'; else $(CYGPATH_W) '$(srcdir)/hformat.c'; fi`
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/hishrink-hformat.Tpo $(DEPDIR)/hishrink-hformat.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='hformat.c' object='hishrink-hformat.obj' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hishrink_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o hishrink-hformat.obj `if test -f 'hformat.c'; then $(CYGPATH_W) 'hformat.c'; else $(CYGPATH_W) '$(srcdir)/hformat.c'; fi`
+
+hishrink-hishrink.o: hishrink.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hishrink_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT hishrink-hishrink.o -MD -MP -MF $(DEPDIR)/hishrink-hishrink.Tpo -c -o hishrink-hishrink.o `test -f 'hishrink.c' || echo '$(srcdir)/'`hishrink.c
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/hishrink-hishrink.Tpo $(DEPDIR)/hishrink-hishrink.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='hishrink.c' object='hishrink-hishrink.o' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hishrink_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o hishrink-hishrink.o `test -f 'hishrink.c' || echo '$(srcdir)/'`hishrink.c
+
+hishrink-hishrink.obj: hishrink.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hishrink_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT hishrink-hishrink.obj -MD -MP -MF $(DEPDIR)/hishrink-hishrink.Tpo -c -o hishrink-hishrink.obj `if test -f 'hishrink.c'; then $(CYGPATH_W) 'hishrink.c'; else $(CYGPATH_W) '$(srcdir)/hishrink.c'; fi`
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/hishrink-hishrink.Tpo $(DEPDIR)/hishrink-hishrink.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='hishrink.c' object='hishrink-hishrink.obj' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hishrink_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o hishrink-hishrink.obj `if test -f 'hishrink.c'; then $(CYGPATH_W) 'hishrink.c'; else $(CYGPATH_W) '$(srcdir)/hishrink.c'; fi`
+
+hishrink-shrink-lexer.o: shrink-lexer.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hishrink_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT hishrink-shrink-lexer.o -MD -MP -MF $(DEPDIR)/hishrink-shrink-lexer.Tpo -c -o hishrink-shrink-lexer.o `test -f 'shrink-lexer.c' || echo '$(srcdir)/'`shrink-lexer.c
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/hishrink-shrink-lexer.Tpo $(DEPDIR)/hishrink-shrink-lexer.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='shrink-lexer.c' object='hishrink-shrink-lexer.o' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hishrink_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o hishrink-shrink-lexer.o `test -f 'shrink-lexer.c' || echo '$(srcdir)/'`shrink-lexer.c
+
+hishrink-shrink-lexer.obj: shrink-lexer.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hishrink_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT hishrink-shrink-lexer.obj -MD -MP -MF $(DEPDIR)/hishrink-shrink-lexer.Tpo -c -o hishrink-shrink-lexer.obj `if test -f 'shrink-lexer.c'; then $(CYGPATH_W) 'shrink-lexer.c'; else $(CYGPATH_W) '$(srcdir)/shrink-lexer.c'; fi`
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/hishrink-shrink-lexer.Tpo $(DEPDIR)/hishrink-shrink-lexer.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='shrink-lexer.c' object='hishrink-shrink-lexer.obj' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hishrink_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o hishrink-shrink-lexer.obj `if test -f 'shrink-lexer.c'; then $(CYGPATH_W) 'shrink-lexer.c'; else $(CYGPATH_W) '$(srcdir)/shrink-lexer.c'; fi`
+
+hishrink-shrink-parser.o: shrink-parser.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hishrink_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT hishrink-shrink-parser.o -MD -MP -MF $(DEPDIR)/hishrink-shrink-parser.Tpo -c -o hishrink-shrink-parser.o `test -f 'shrink-parser.c' || echo '$(srcdir)/'`shrink-parser.c
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/hishrink-shrink-parser.Tpo $(DEPDIR)/hishrink-shrink-parser.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='shrink-parser.c' object='hishrink-shrink-parser.o' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hishrink_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o hishrink-shrink-parser.o `test -f 'shrink-parser.c' || echo '$(srcdir)/'`shrink-parser.c
+
+hishrink-shrink-parser.obj: shrink-parser.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hishrink_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT hishrink-shrink-parser.obj -MD -MP -MF $(DEPDIR)/hishrink-shrink-parser.Tpo -c -o hishrink-shrink-parser.obj `if test -f 'shrink-parser.c'; then $(CYGPATH_W) 'shrink-parser.c'; else $(CYGPATH_W) '$(srcdir)/shrink-parser.c'; fi`
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/hishrink-shrink-parser.Tpo $(DEPDIR)/hishrink-shrink-parser.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='shrink-parser.c' object='hishrink-shrink-parser.obj' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hishrink_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o hishrink-shrink-parser.obj `if test -f 'shrink-parser.c'; then $(CYGPATH_W) 'shrink-parser.c'; else $(CYGPATH_W) '$(srcdir)/shrink-parser.c'; fi`
+
+histretch-hformat.o: hformat.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(histretch_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT histretch-hformat.o -MD -MP -MF $(DEPDIR)/histretch-hformat.Tpo -c -o histretch-hformat.o `test -f 'hformat.c' || echo '$(srcdir)/'`hformat.c
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/histretch-hformat.Tpo $(DEPDIR)/histretch-hformat.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='hformat.c' object='histretch-hformat.o' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(histretch_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o histretch-hformat.o `test -f 'hformat.c' || echo '$(srcdir)/'`hformat.c
+
+histretch-hformat.obj: hformat.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(histretch_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT histretch-hformat.obj -MD -MP -MF $(DEPDIR)/histretch-hformat.Tpo -c -o histretch-hformat.obj `if test -f 'hformat.c'; then $(CYGPATH_W) 'hformat.c'; else $(CYGPATH_W) '$(srcdir)/hformat.c'; fi`
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/histretch-hformat.Tpo $(DEPDIR)/histretch-hformat.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='hformat.c' object='histretch-hformat.obj' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(histretch_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o histretch-hformat.obj `if test -f 'hformat.c'; then $(CYGPATH_W) 'hformat.c'; else $(CYGPATH_W) '$(srcdir)/hformat.c'; fi`
+
+histretch-histretch.o: histretch.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(histretch_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT histretch-histretch.o -MD -MP -MF $(DEPDIR)/histretch-histretch.Tpo -c -o histretch-histretch.o `test -f 'histretch.c' || echo '$(srcdir)/'`histretch.c
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/histretch-histretch.Tpo $(DEPDIR)/histretch-histretch.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='histretch.c' object='histretch-histretch.o' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(histretch_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o histretch-histretch.o `test -f 'histretch.c' || echo '$(srcdir)/'`histretch.c
+
+histretch-histretch.obj: histretch.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(histretch_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT histretch-histretch.obj -MD -MP -MF $(DEPDIR)/histretch-histretch.Tpo -c -o histretch-histretch.obj `if test -f 'histretch.c'; then $(CYGPATH_W) 'histretch.c'; else $(CYGPATH_W) '$(srcdir)/histretch.c'; fi`
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/histretch-histretch.Tpo $(DEPDIR)/histretch-histretch.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='histretch.c' object='histretch-histretch.obj' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(histretch_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o histretch-histretch.obj `if test -f 'histretch.c'; then $(CYGPATH_W) 'histretch.c'; else $(CYGPATH_W) '$(srcdir)/histretch.c'; fi`
+
+hitex-hformat.o: hformat.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hitex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT hitex-hformat.o -MD -MP -MF $(DEPDIR)/hitex-hformat.Tpo -c -o hitex-hformat.o `test -f 'hformat.c' || echo '$(srcdir)/'`hformat.c
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/hitex-hformat.Tpo $(DEPDIR)/hitex-hformat.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='hformat.c' object='hitex-hformat.o' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hitex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o hitex-hformat.o `test -f 'hformat.c' || echo '$(srcdir)/'`hformat.c
+
+hitex-hformat.obj: hformat.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hitex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT hitex-hformat.obj -MD -MP -MF $(DEPDIR)/hitex-hformat.Tpo -c -o hitex-hformat.obj `if test -f 'hformat.c'; then $(CYGPATH_W) 'hformat.c'; else $(CYGPATH_W) '$(srcdir)/hformat.c'; fi`
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/hitex-hformat.Tpo $(DEPDIR)/hitex-hformat.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='hformat.c' object='hitex-hformat.obj' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hitex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o hitex-hformat.obj `if test -f 'hformat.c'; then $(CYGPATH_W) 'hformat.c'; else $(CYGPATH_W) '$(srcdir)/hformat.c'; fi`
+
+hitex-hput.o: hput.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hitex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT hitex-hput.o -MD -MP -MF $(DEPDIR)/hitex-hput.Tpo -c -o hitex-hput.o `test -f 'hput.c' || echo '$(srcdir)/'`hput.c
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/hitex-hput.Tpo $(DEPDIR)/hitex-hput.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='hput.c' object='hitex-hput.o' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hitex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o hitex-hput.o `test -f 'hput.c' || echo '$(srcdir)/'`hput.c
+
+hitex-hput.obj: hput.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hitex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT hitex-hput.obj -MD -MP -MF $(DEPDIR)/hitex-hput.Tpo -c -o hitex-hput.obj `if test -f 'hput.c'; then $(CYGPATH_W) 'hput.c'; else $(CYGPATH_W) '$(srcdir)/hput.c'; fi`
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/hitex-hput.Tpo $(DEPDIR)/hitex-hput.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='hput.c' object='hitex-hput.obj' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hitex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o hitex-hput.obj `if test -f 'hput.c'; then $(CYGPATH_W) 'hput.c'; else $(CYGPATH_W) '$(srcdir)/hput.c'; fi`
+
+hitex-htex.o: htex.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hitex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT hitex-htex.o -MD -MP -MF $(DEPDIR)/hitex-htex.Tpo -c -o hitex-htex.o `test -f 'htex.c' || echo '$(srcdir)/'`htex.c
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/hitex-htex.Tpo $(DEPDIR)/hitex-htex.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='htex.c' object='hitex-htex.o' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hitex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o hitex-htex.o `test -f 'htex.c' || echo '$(srcdir)/'`htex.c
+
+hitex-htex.obj: htex.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hitex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT hitex-htex.obj -MD -MP -MF $(DEPDIR)/hitex-htex.Tpo -c -o hitex-htex.obj `if test -f 'htex.c'; then $(CYGPATH_W) 'htex.c'; else $(CYGPATH_W) '$(srcdir)/htex.c'; fi`
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/hitex-htex.Tpo $(DEPDIR)/hitex-htex.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='htex.c' object='hitex-htex.obj' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hitex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o hitex-htex.obj `if test -f 'htex.c'; then $(CYGPATH_W) 'htex.c'; else $(CYGPATH_W) '$(srcdir)/htex.c'; fi`
+
+hitex-hitex.o: hitex.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hitex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT hitex-hitex.o -MD -MP -MF $(DEPDIR)/hitex-hitex.Tpo -c -o hitex-hitex.o `test -f 'hitex.c' || echo '$(srcdir)/'`hitex.c
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/hitex-hitex.Tpo $(DEPDIR)/hitex-hitex.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='hitex.c' object='hitex-hitex.o' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hitex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o hitex-hitex.o `test -f 'hitex.c' || echo '$(srcdir)/'`hitex.c
+
+hitex-hitex.obj: hitex.c
+ at am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hitex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT hitex-hitex.obj -MD -MP -MF $(DEPDIR)/hitex-hitex.Tpo -c -o hitex-hitex.obj `if test -f 'hitex.c'; then $(CYGPATH_W) 'hitex.c'; else $(CYGPATH_W) '$(srcdir)/hitex.c'; fi`
+ at am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/hitex-hitex.Tpo $(DEPDIR)/hitex-hitex.Po
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='hitex.c' object='hitex-hitex.obj' libtool=no @AMDEPBACKSLASH@
+ at AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ at am__fastdepCC_FALSE@ $(AM_V_CC at am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(hitex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o hitex-hitex.obj `if test -f 'hitex.c'; then $(CYGPATH_W) 'hitex.c'; else $(CYGPATH_W) '$(srcdir)/hitex.c'; fi`
+
initex-callexe.o: callexe.c
@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(initex_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT initex-callexe.o -MD -MP -MF $(DEPDIR)/initex-callexe.Tpo -c -o initex-callexe.o `test -f 'callexe.c' || echo '$(srcdir)/'`callexe.c
@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/initex-callexe.Tpo $(DEPDIR)/initex-callexe.Po
@@ -18636,6 +18896,13 @@
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepCXX_FALSE@ $(AM_V_CXX at am__nodep@)$(LTCXXCOMPILE) -c -o $@ $<
+.l.c:
+ $(AM_V_LEX)$(am__skiplex) $(SHELL) $(YLWRAP) $< $(LEX_OUTPUT_ROOT).c $@ -- $(LEXCOMPILE)
+
+shrink-lexer.c: shrink-lexer.l
+ $(AM_V_LEX) \
+ $(SHELL) $(YLWRAP) `test -f 'shrink-lexer.l' || echo '$(srcdir)/'`shrink-lexer.l $(LEX_OUTPUT_ROOT).c shrink-lexer.c -- $(LEX) $(AM_LFLAGS) $(LFLAGS)
+
.mm.o:
@am__fastdepOBJCXX_TRUE@ $(AM_V_OBJCXX)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\
@am__fastdepOBJCXX_TRUE@ $(OBJCXXCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\
@@ -18674,6 +18941,13 @@
@AMDEP_TRUE@@am__fastdepOBJCXX_FALSE@ DEPDIR=$(DEPDIR) $(OBJCXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepOBJCXX_FALSE@ $(AM_V_OBJCXX at am__nodep@)$(OBJCXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libxetex_a_CPPFLAGS) $(CPPFLAGS) $(libxetex_a_OBJCXXFLAGS) $(OBJCXXFLAGS) -c -o xetexdir/libxetex_a-XeTeXFontMgr_Mac.obj `if test -f 'xetexdir/XeTeXFontMgr_Mac.mm'; then $(CYGPATH_W) 'xetexdir/XeTeXFontMgr_Mac.mm'; else $(CYGPATH_W) '$(srcdir)/xetexdir/XeTeXFontMgr_Mac.mm'; fi`
+.y.c:
+ $(AM_V_YACC)$(am__skipyacc) $(SHELL) $(YLWRAP) $< y.tab.c $@ y.tab.h `echo $@ | $(am__yacc_c2h)` y.output $*.output -- $(YACCCOMPILE)
+
+shrink-parser.c: shrink-parser.y
+ $(AM_V_YACC) \
+ $(SHELL) $(YLWRAP) `test -f 'shrink-parser.y' || echo '$(srcdir)/'`shrink-parser.y y.tab.c shrink-parser.c y.tab.h `echo shrink-parser.c | $(am__yacc_c2h)` y.output shrink-parser.output -- $(YACC) $(AM_YFLAGS) $(YFLAGS)
+
mostlyclean-libtool:
-rm -f *.lo
@@ -19324,6 +19598,9 @@
-test -z "$(TEST_SUITE_LOG)" || rm -f $(TEST_SUITE_LOG)
clean-generic:
+ -rm -f shrink-lexer.c
+ -rm -f shrink-parser.c
+ -rm -f shrink-parser.h
-test -z "$(CLEANFILES)" || rm -f $(CLEANFILES)
distclean-generic:
@@ -19511,6 +19788,16 @@
-rm -f ./$(DEPDIR)/gftodvi.Po
-rm -f ./$(DEPDIR)/gftopk.Po
-rm -f ./$(DEPDIR)/gftype.Po
+ -rm -f ./$(DEPDIR)/hishrink-hformat.Po
+ -rm -f ./$(DEPDIR)/hishrink-hishrink.Po
+ -rm -f ./$(DEPDIR)/hishrink-shrink-lexer.Po
+ -rm -f ./$(DEPDIR)/hishrink-shrink-parser.Po
+ -rm -f ./$(DEPDIR)/histretch-hformat.Po
+ -rm -f ./$(DEPDIR)/histretch-histretch.Po
+ -rm -f ./$(DEPDIR)/hitex-hformat.Po
+ -rm -f ./$(DEPDIR)/hitex-hitex.Po
+ -rm -f ./$(DEPDIR)/hitex-hput.Po
+ -rm -f ./$(DEPDIR)/hitex-htex.Po
-rm -f ./$(DEPDIR)/initex-callexe.Po
-rm -f ./$(DEPDIR)/libluahbtexspecific_a-luainit-hb.Po
-rm -f ./$(DEPDIR)/libluahbtexspecific_a-luastuff-hb.Po
@@ -20280,6 +20567,16 @@
-rm -f ./$(DEPDIR)/gftodvi.Po
-rm -f ./$(DEPDIR)/gftopk.Po
-rm -f ./$(DEPDIR)/gftype.Po
+ -rm -f ./$(DEPDIR)/hishrink-hformat.Po
+ -rm -f ./$(DEPDIR)/hishrink-hishrink.Po
+ -rm -f ./$(DEPDIR)/hishrink-shrink-lexer.Po
+ -rm -f ./$(DEPDIR)/hishrink-shrink-parser.Po
+ -rm -f ./$(DEPDIR)/histretch-hformat.Po
+ -rm -f ./$(DEPDIR)/histretch-histretch.Po
+ -rm -f ./$(DEPDIR)/hitex-hformat.Po
+ -rm -f ./$(DEPDIR)/hitex-hitex.Po
+ -rm -f ./$(DEPDIR)/hitex-hput.Po
+ -rm -f ./$(DEPDIR)/hitex-htex.Po
-rm -f ./$(DEPDIR)/initex-callexe.Po
-rm -f ./$(DEPDIR)/libluahbtexspecific_a-luainit-hb.Po
-rm -f ./$(DEPDIR)/libluahbtexspecific_a-luastuff-hb.Po
@@ -21825,6 +22122,48 @@
euptrip-clean:
rm -rf euptripdir
+$(format_c_h_l_y): format-tangle
+ $(hi_ctangle_sh) format
+#
+format-tangle: ctangle$(EXEEXT) hitexdir/format.w tangle-sh
+ $(hi_ctangle_sh) format
+
+$(hitex_c_h): hitex-tangle
+ $(hi_ctangle_sh) hitex
+#
+hitex-tangle: ctangle$(EXEEXT) hitexdir/hitex.w tangle-sh
+ $(hi_ctangle_sh) hitex
+
+# htex.c needs hitex.h, so depend on the hitex tangle.
+$(htex_c_h): htex-tangle hitex-tangle
+ $(hi_ctangle_sh) htex
+#
+htex-tangle: ctangle$(EXEEXT) hitexdir/htex.w tangle-sh hitex-tangle
+ $(hi_ctangle_sh) htex
+
+# Generating hformat.c using mkhformat.
+hformat.c: mkhformat
+ ./mkhformat >hformat.c || { rm -f hformat.c; exit 1; }
+
+# Postprocessing lex and yacc files generated by ctangle.
+# (SED is defined with AC_PROG_SED)
+shrink-parser.y: $(srcdir)/hitexdir/yacc.sed shrink.yacc-in
+ $(SED) -f $(srcdir)/hitexdir/yacc.sed shrink.yacc-in >shrink-parser.y \
+ || { rm -f shrink-parser.y; exit 1; }
+#
+shrink-lexer.l: $(srcdir)/hitexdir/lex.sed shrink.lex-in
+ $(SED) -f $(srcdir)/hitexdir/lex.sed shrink.lex-in >shrink-lexer.l \
+ || { rm -f shrink-lexer.l; exit 1; }
+
+# shrink needs the yacc header. It's actually the .o that should be the
+# target here, seems to me, but hishrink.o is not used for the object
+# name, it's hishrink-hishrink.o (because the program is also named
+# hishrink), and adding that here ends up trying to link with
+# hishrink-hishrink.o (I don't know), which doesn't exist. Sigh.
+hishrink.c: shrink-parser.h shrink-parser.c
+
+# still missing
+
# We must create pdftexd.h (and xpdf...) before building the libpdftex_a_OBJECTS.
$(libpdftex_a_OBJECTS): pdftexd.h $(XPDF_DEPEND) $(ZLIB_DEPEND) $(LIBPNG_DEPEND)
$(pdftex_OBJECTS): $(pdftex_prereq)
Modified: trunk/Build/source/texk/web2c/ac/web2c.ac
===================================================================
--- trunk/Build/source/texk/web2c/ac/web2c.ac 2021-09-20 23:48:29 UTC (rev 60568)
+++ trunk/Build/source/texk/web2c/ac/web2c.ac 2021-09-21 16:46:46 UTC (rev 60569)
@@ -1,7 +1,7 @@
dnl $Id$
# texk/web2c/ac/web2c.ac: configure.ac fragment for TL subdir
dnl
-dnl Copyright 2015-2020 Karl Berry <tex-live at tug.org>
+dnl Copyright 2015-2021 Karl Berry <tex-live at tug.org>
dnl Copyright 2009-2015 Peter Breitenlohner <tex-live at tug.org>
dnl You may freely use, modify and/or distribute this file.
dnl
@@ -36,6 +36,7 @@
[[uptex], [yes], [yes], [upTeX], [ptexenc zlib]],
[[euptex], [yes], [yes], [e-upTeX], [ptexenc zlib]],
[[aleph], [yes], [], [Aleph], []],
+[[hitex], [no], [], [HiTeX], [zlib]],
[[pdftex], [yes], [yes], [pdfTeX], [xpdf libpng zlib]],
[[luatex], [yes], [], [LuaTeX], [pplib libpng zziplib lua53]],
[[luajittex], [yes], [], [LuaJITTeX], [pplib libpng zziplib luajit]],
Modified: trunk/Build/source/texk/web2c/c-auto.in
===================================================================
--- trunk/Build/source/texk/web2c/c-auto.in 2021-09-20 23:48:29 UTC (rev 60568)
+++ trunk/Build/source/texk/web2c/c-auto.in 2021-09-21 16:46:46 UTC (rev 60569)
@@ -2,7 +2,7 @@
/* w2c/c-auto.h: defines for web2c, as determined by configure.
- Copyright 1994-97, 2008-2012 Karl Berry.
+ Copyright 1994-97, 2008-2021 Karl Berry.
Copyright 1997-99, 2002, 2005 Olaf Weber.
This program is free software: you can redistribute it and/or modify
Modified: trunk/Build/source/texk/web2c/configure
===================================================================
--- trunk/Build/source/texk/web2c/configure 2021-09-20 23:48:29 UTC (rev 60568)
+++ trunk/Build/source/texk/web2c/configure 2021-09-21 16:46:46 UTC (rev 60569)
@@ -822,6 +822,8 @@
PDFTEX_SYNCTEX_TRUE
PDFTEX_FALSE
PDFTEX_TRUE
+HITEX_FALSE
+HITEX_TRUE
ALEPH_FALSE
ALEPH_TRUE
EUPTEX_SYNCTEX_FALSE
@@ -1045,6 +1047,7 @@
enable_euptex
enable_euptex_synctex
enable_aleph
+enable_hitex
enable_pdftex
enable_pdftex_synctex
enable_luatex
@@ -1784,6 +1787,7 @@
--disable-euptex do not compile and install e-upTeX
--disable-euptex-synctex build e-upTeX without SyncTeX support
--disable-aleph do not compile and install Aleph
+ --enable-hitex compile and install HiTeX
--disable-pdftex do not compile and install pdfTeX
--disable-pdftex-synctex build pdfTeX without SyncTeX support
--disable-luatex do not compile and install LuaTeX
@@ -19811,6 +19815,21 @@
*) :
enable_aleph=yes ;;
esac
+# Check whether --enable-hitex was given.
+if test ${enable_hitex+y}
+then :
+ enableval=$enable_hitex;
+fi
+case $enable_hitex in #(
+ yes | no) :
+ ;; #(
+ *) :
+ enable_hitex=no ;;
+esac
+
+test "x$enable_web2c:$enable_hitex" = xyes:yes && {
+ need_zlib=yes
+}
# Check whether --enable-pdftex was given.
if test ${enable_pdftex+y}
then :
@@ -22526,6 +22545,13 @@
ALEPH_TRUE='#'
ALEPH_FALSE=
fi
+ if test "x$enable_hitex" = xyes; then
+ HITEX_TRUE=
+ HITEX_FALSE='#'
+else
+ HITEX_TRUE='#'
+ HITEX_FALSE=
+fi
if test "x$enable_pdftex" = xyes; then
PDFTEX_TRUE=
PDFTEX_FALSE='#'
@@ -28375,6 +28401,10 @@
as_fn_error $? "conditional \"ALEPH\" was never defined.
Usually this means the macro was only invoked conditionally." "$LINENO" 5
fi
+if test -z "${HITEX_TRUE}" && test -z "${HITEX_FALSE}"; then
+ as_fn_error $? "conditional \"HITEX\" was never defined.
+Usually this means the macro was only invoked conditionally." "$LINENO" 5
+fi
if test -z "${PDFTEX_TRUE}" && test -z "${PDFTEX_FALSE}"; then
as_fn_error $? "conditional \"PDFTEX\" was never defined.
Usually this means the macro was only invoked conditionally." "$LINENO" 5
Modified: trunk/Build/source/texk/web2c/configure.ac
===================================================================
--- trunk/Build/source/texk/web2c/configure.ac 2021-09-20 23:48:29 UTC (rev 60568)
+++ trunk/Build/source/texk/web2c/configure.ac 2021-09-21 16:46:46 UTC (rev 60569)
@@ -326,7 +326,7 @@
AH_TOP([/* w2c/c-auto.h: defines for web2c, as determined by configure.
- Copyright 1994-97, 2008-2012 Karl Berry.
+ Copyright 1994-97, 2008-2021 Karl Berry.
Copyright 1997-99, 2002, 2005 Olaf Weber.
This program is free software: you can redistribute it and/or modify
Added: trunk/Build/source/texk/web2c/hitexdir/am/hitex.am
===================================================================
--- trunk/Build/source/texk/web2c/hitexdir/am/hitex.am (rev 0)
+++ trunk/Build/source/texk/web2c/hitexdir/am/hitex.am 2021-09-21 16:46:46 UTC (rev 60569)
@@ -0,0 +1,100 @@
+## $Id$
+## texk/web2c/hitexdir/am/hitex.am: Makefile fragment for HiTeX
+##
+## Copyright 2021 Martin Ruckert <ruckert at cs.hm.edu>
+## You may freely use, modify and/or distribute this file.
+
+if HITEX
+bin_PROGRAMS += hitex hishrink histretch
+endif HITEX
+EXTRA_PROGRAMS += hitex hishrink histretch
+
+#kb hitex_CFLAGS = $(AM_CFLAGS) \
+#kb -Wno-parentheses -Wno-maybe-uninitialized -Wno-unused-function -Wno-unused-variable
+hitex_CPPFLAGS = $(AM_CPPFLAGS) $(ZLIB_INCLUDES)
+hitex_LDADD = $(KPATHSEA_LIBS) $(ZLIB_LIBS)
+#
+histretch_CPPFLAGS = $(AM_CPPFLAGS) $(ZLIB_INCLUDES)
+histretch_LDADD = $(ZLIB_LIBS)
+#
+hishrink_CPPFLAGS = $(AM_CPPFLAGS) $(ZLIB_INCLUDES)
+hishrink_LDADD = $(ZLIB_LIBS)
+
+# We want debugging for the parser, and we use features not in POSIX yacc.
+# I guess the result is that we assume bison? Not sure if that's ok.
+AM_YFLAGS = -d -v -Wno-yacc
+
+# HiTeX CWEB sources
+hitex_web = hitexdir/format.w hitexdir/htex.w hitexdir/hitex.w
+
+# Creating several files: need stamp file and two rules with identical recipes
+hi_ctangle_sh = CWEBINPUTS=$(srcdir)/hitexdir AM_V_P=$(AM_V_P) $(SHELL) ./tangle-sh $@ $(CTANGLE)
+
+# HiTeX C/yacc/lex sources generated using ctangle.
+# For each cweb (.w) source file, list the files generated.
+format_c_h_l_y = basetypes.h error.h hformat.h \
+ mkhformat.c \
+ hput.c hput.h \
+ hget.c hget.h \
+ shrink.lex-in shrink.yacc-in \
+ hishrink.c histretch.c
+htex_c_h = htex.c htex.h
+hitex_c_h = hitex.c hitex.h
+
+$(format_c_h_l_y): format-tangle
+ $(hi_ctangle_sh) format
+#
+format-tangle: ctangle$(EXEEXT) hitexdir/format.w tangle-sh
+ $(hi_ctangle_sh) format
+
+$(hitex_c_h): hitex-tangle
+ $(hi_ctangle_sh) hitex
+#
+hitex-tangle: ctangle$(EXEEXT) hitexdir/hitex.w tangle-sh
+ $(hi_ctangle_sh) hitex
+
+# htex.c needs hitex.h, so depend on the hitex tangle.
+$(htex_c_h): htex-tangle hitex-tangle
+ $(hi_ctangle_sh) htex
+#
+htex-tangle: ctangle$(EXEEXT) hitexdir/htex.w tangle-sh hitex-tangle
+ $(hi_ctangle_sh) htex
+
+# Generating hformat.c using mkhformat.
+hformat.c: mkhformat
+ ./mkhformat >hformat.c || { rm -f hformat.c; exit 1; }
+
+# Postprocessing lex and yacc files generated by ctangle.
+# (SED is defined with AC_PROG_SED)
+shrink-parser.y: $(srcdir)/hitexdir/yacc.sed shrink.yacc-in
+ $(SED) -f $(srcdir)/hitexdir/yacc.sed shrink.yacc-in >shrink-parser.y \
+ || { rm -f shrink-parser.y; exit 1; }
+#
+shrink-lexer.l: $(srcdir)/hitexdir/lex.sed shrink.lex-in
+ $(SED) -f $(srcdir)/hitexdir/lex.sed shrink.lex-in >shrink-lexer.l \
+ || { rm -f shrink-lexer.l; exit 1; }
+
+# Other web2c/*dir/am/*.am engine Makefile fragments define a bunch of
+# _OBJECTS variables. This does not seem right to me. Instead, we
+# correctly define the _SOURCES.
+#
+nodist_hitex_SOURCES = hformat.c hput.c $(htex_c_h) $(hitex_c_h)
+nodist_histretch_SOURCES = hformat.c histretch.c
+nodist_hishrink_SOURCES = hformat.c hishrink.c shrink-lexer.l shrink-parser.y
+
+# shrink needs the yacc header. It's actually the .o that should be the
+# target here, seems to me, but hishrink.o is not used for the object
+# name, it's hishrink-hishrink.o (because the program is also named
+# hishrink), and adding that here ends up trying to link with
+# hishrink-hishrink.o (I don't know), which doesn't exist. Sigh.
+hishrink.c: shrink-parser.h shrink-parser.c
+
+EXTRA_DIST += hitexdir/ChangeLog $(hitex_web) lex.sed yacc.sed
+
+DISTCLEANFILES += $(nodist_hitex_SOURCES)
+DISTCLEANFILES += $(nodist_hishrink_SOURCES) $(nodist_histretch_SOURCES)
+DISTCLEANFILES += format-tangle htex-tangle hitex-tangle
+DISTCLEANFILES += shrink-parser.* shrink-lexer.*
+
+## HiTeX tests
+# still missing
Property changes on: trunk/Build/source/texk/web2c/hitexdir/am/hitex.am
___________________________________________________________________
Added: svn:eol-style
## -0,0 +1 ##
+native
\ No newline at end of property
Added: svn:keywords
## -0,0 +1 ##
+Date Author Id Revision
\ No newline at end of property
Added: trunk/Build/source/texk/web2c/hitexdir/format.w
===================================================================
--- trunk/Build/source/texk/web2c/hitexdir/format.w (rev 0)
+++ trunk/Build/source/texk/web2c/hitexdir/format.w 2021-09-21 16:46:46 UTC (rev 60569)
@@ -0,0 +1,10509 @@
+% This file is part of HINT
+% Copyright 2017-2021 Martin Ruckert
+%
+% Permission is hereby granted, free of charge, to any person obtaining a copy
+% of this software and associated documentation files (the "Software"), to deal
+% in the Software without restriction, including without limitation the rights
+% to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+% copies of the Software, and to permit persons to whom the Software is
+% furnished to do so, subject to the following conditions:
+%
+% The above copyright notice and this permission notice shall be
+% included in all copies or substantial portions of the Software.
+%
+% THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+% IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+% FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+% COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+% WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT
+% OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+% THE SOFTWARE.
+%
+% Except as contained in this notice, the name of the copyright holders shall
+% not be used in advertising or otherwise to promote the sale, use or other
+% dealings in this Software without prior written authorization from the
+% copyright holders.
+
+\input ../hint.sty
+
+%% defining how to display certain C identifiers
+
+ at s int8_t int
+ at s uint8_t int
+ at s int16_t int
+ at s uint16_t int
+ at s uint32_t int
+ at s int32_t int
+ at s uint64_t int
+ at s bool int
+
+@
+
+\makeindex
+\maketoc
+\makecode
+%\makefigindex
+\titletrue
+
+
+\def\setrevision$#1: #2 ${\gdef\lastrevision{#2}}
+\setrevision$Revision: 2485 $
+\def\setdate$#1(#2) ${\gdef\lastdate{#2}}
+\setdate$Date: 2021-08-19 19:37:13 +0200 (Thu, 19 Aug 2021) $
+
+\null
+
+\font\largetitlefont=cmssbx10 scaled\magstep4
+\font\Largetitlefont=cmssbx10 at 40pt
+\font\hugetitlefont=cmssbx10 at 48pt
+\font\smalltitlefontit=cmbxti10 scaled\magstep3
+\font\smalltitlefont=cmssbx10 scaled\magstep3
+
+%halftitle
+\def\raggedleft{\leftskip=0pt plus 5em\parfillskip=0pt
+\spaceskip=.3333em \xspaceskip=0.5em \emergencystretch=1em\relax
+\hyphenpenalty=1000\exhyphenpenalty=1000\pretolerance=10000\linepenalty=5000
+}
+\hbox{}
+\vskip 0pt plus 1fill
+{ \baselineskip=60pt
+ \hugetitlefont\hfill HINT:\par
+ \Largetitlefont\raggedleft The File Format\par
+}
+\vskip 0pt plus 5fill
+\eject
+% verso of half title
+\titletrue
+\null
+\vfill
+\eject
+
+% title
+\titletrue
+\hbox{}
+\vskip 0pt plus 1fill
+{
+ \baselineskip=1cm\parindent=0pt
+ {\largetitlefont\raggedright HINT: The File Format}\par
+ \leftline{\smalltitlefont Version 1.3}
+ \vskip 10pt plus 0.5fill
+ \leftline{\smalltitlefont Reflowable}
+ \vskip-3pt
+ \leftline{\smalltitlefont Output}
+ \vskip-3pt
+ \leftline{\smalltitlefont for \TeX}
+ \vskip 10pt plus 0.5fill
+ \hskip 0pt plus 2fill{\it F\"ur meine Mutter}\hskip 0pt plus 0.5fill\hbox{}
+ \bigskip
+ \vskip 10pt plus 3fill
+ \raggedright\baselineskip=12pt
+ {\bf MARTIN RUCKERT} \ {\it Munich University of Applied Sciences}\par
+ \bigskip
+ \leftline{Second edition}
+ \bigskip
+% \leftline{\bf Eigendruck im Selbstverlag}
+% \bigskip
+}
+\eject
+
+% verso of title
+% copyright page (ii)
+\titletrue
+\begingroup
+\figrm
+\parindent=0pt
+%\null
+{\raggedright\advance\rightskip 3.5pc
+The author has taken care in the preparation of this book,
+but makes no expressed or implied warranty of any kind and assumes no
+responsibility for errors or omissions. No liability is assumed for
+incidental or consequential damages in connection with or arising out
+of the use of the information or programs contained herein.
+
+\bigskip
+{\figtt\obeylines\obeyspaces\baselineskip=11pt
+Ruckert, Martin.
+ HINT: The File Format
+ Includes index.
+ ISBN 979-854992684-4
+}
+\bigskip
+
+{\raggedright\advance\rightskip 3.5pc
+\def\:{\discretionary{}{}{}}
+Internet page {\tt http:\://hint.\:userweb.\:mwn.\:de/\:hint/\:format.html}
+may contain current information about this book, downloadable software,
+and news.
+
+\vfill
+Copyright $\copyright$ 2019, 2021 by Martin Ruckert
+\smallskip
+All rights reserved.
+Printed by Kindle Direct Publishing.
+This publication is protected by copyright, and permission must be
+obtained prior to any prohibited reproduction, storage in
+a~retrieval system, or transmission in any form or by any means, electronic,
+mechanical, photocopying, recording, or likewise.
+To obtain permission to use material from this work, please submit a written
+request to Martin Ruckert,
+Hochschule M\"unchen,
+Fakult\"at f\"ur Informatik und Mathematik,
+Lothstrasse 64,
+80335 M\"unchen,
+Germany.
+\medskip
+{\tt ruckert\:@@cs.hm.edu}
+\medskip
+ISBN-13: 979-854992684-4\par
+\medskip
+First printing: August 2019\par
+Second edition: August 2021\par
+\medskip
+Revision: \lastrevision,\quad Date: \lastdate\par
+}
+}
+\endgroup
+
+
+\frontmatter
+
+
+
+\plainsection{Preface}
+Late in summer 2017, with my new \CEE\ based {\tt cweb} implementation
+of \TeX\cite{Knuth:tex} in hand\cite{MR:webtocweb}\cite{MR:tug38}\cite{MR:web2w}, I started to write
+the first prototype of the \HINT\ viewer. I basically made two copies
+of \TeX: In the first copy, I replaced the |build_page| procedure by
+an output routine which used more or less the printing routines
+already available in \TeX. This was the beginning of the
+\HINT\ file format.
+In the second copy, I replaced \TeX's main loop by an input routine
+that would feed the \HINT\ file more or less directly to \TeX's
+|build_page| procedure. And after replacing \TeX's |ship_out|
+procedure by a modified rendering routine of a dvi viewer that I had
+written earlier for my experiments with \TeX's Computer Modern
+fonts\cite{MR:tug37}, I had my first running \HINT\ viewer. My
+sabbatical during the following Fall term gave me time for ``rapid
+prototyping'' various features that I considered necessary for
+reflowable \TeX\ output\cite{MR:tug39}.
+
+The textual output format derived from the original \TeX\ debugging
+routines proved to be insufficient when I implemented a ``page up''
+button because it did not support reading the page content
+``backwards''. As a consequence, I developed a compact binary file
+format that could be parsed easily in both directions. The \HINT\
+short file format war born. I stopped an initial attempt at
+eliminating the old textual format because it was so much nicer when
+debugging. Instead, I converted the long textual format into the short
+binary format as a preliminary step in the viewer. This was not a long
+term solution. When opening a big file, as produced from a 1000
+pages \TeX\ file, the parsing took several seconds before the first
+page would appear on screen. This delay, observed on a fast desktop
+PC, is barley tolerable, and the delay one would expect on a low-cost,
+low-power, mobile device seemed prohibitive. The consequence is
+simple: The viewer will need an input file in the short format; and to
+support debugging (or editing), separate programs are needed to
+translate the short format into the long format and back again. But
+for the moment, I did not bother to implement any of this but
+continued with unrestricted experimentation.
+
+With the beginning of the Spring term 2018, I stopped further
+experiments with the \HINT\ viewer and decided that I had to write
+down a clean design of the \HINT\ file format. Or of both file
+formats? Professors are supposed to do research, and hence I tried an
+experiment: Instead of writing down a traditional language
+specification, I decided to stick with the ``literate programming''
+paradigm\cite{Knuth:lp} and write the present book. It describes and implements
+the \.{stretch} and \.{shrink} programs translating one file format
+into the other. As a side effect, it contains the underlying language
+specification. Whether this experiment is a success as a language
+specification remains to be seen, and you should see for yourself. But
+the only important measure for the value of a scientific experiment is
+how much you can learn form it---and I learned a lot.
+
+The whole project turned out to be much more difficult than I had
+expected. Early on, I decided that I would use a recursive descent
+parser for the short format and an LR($k$) parser for the long
+format. Of course, I would use {\tt lex}/{\tt flex} and {\tt yacc}/{\tt bison}
+to generate the LR($k$) parser, and so I had to extend the {\tt
+cweb} tools\cite{Knuth:cweb} to support the corresponding source files.
+
+About in mid May, after writing down about 100 pages, the first
+problems emerged that could not be resolved with my current
+approach. I had started to describe font definitions containing
+definitions of the interword glue and the default hyphen, and the
+declarative style of my exposition started to conflict with the
+sequential demands of writing an output file. So it was time for a
+first complete redesign. Two more passes over the whole book were
+necessary to find the concepts and the structure that would allow me
+to go forward and complete the book as you see it now.
+
+While rewriting was on its way, many ``nice ideas'' were pruned from
+the book. For example, the initial idea of optimizing the \HINT\ file
+while translating it was first reduced to just gathering statistics
+and then disappeared completely. The added code and complexity was
+just too distracting.
+
+What you see before you is still a snapshot of the \HINT\ file format
+because its development is still under way. We will know what
+features are needed for a reflowable \TeX\ file format only after many
+people have started using the format. To use the format, the end-user
+will need implementations, and the implementer will need a language
+specification. The present book is the first step in an attempt to
+solve this ``chicken or egg'' dilemma.
+
+
+\vskip 1cm
+\noindent {\it M\"unchen\hfil\break
+August 20, 2019 \hfill Martin Ruckert}
+
+
+\tableofcontent
+%\thefigindex
+
+
+\mainmatter
+
+\section{Introduction}\label{intro}
+This book defines a file format for reflowable text.
+Actually it describes two file formats: a long format
+that optimizes readability for human beings, and
+a short format that optimizes readability for machines
+and the use of storage space. Both formats use the concept of nodes and lists of
+nodes to describe the file content. Programs that process these nodes
+will likely want to convert the compressed binary representation of a
+node---the short format---or the lengthy textual representation of a
+node---the long format---into a convenient internal representation.
+So most of what follows is just a description of these nodes: their short format,
+their long format and sometimes their internal representation.
+Where as the description of the long and short external format is part
+of the file specification, the description of the internal representation
+is just informational. Different internal representations can be chosen
+based on the individual needs of the program.
+
+While defining the format, I illustrate the processing of long and short format
+files by implementing two utilities: \.{shrink} and \.{stretch}.
+\.{shrink} converts the long format into the short format and \.{stretch}
+goes the other way.
+
+There is also a prototype viewer for this
+file format and a special version of \TeX\cite{DK:texbook} to produce output
+in this format. Both are not described here; a survey describing
+them can be found in \cite{MR:tug39}.
+
+\subsection{Glyphs}
+Let's start with a simple and very common kind of node: a node describing
+a character.
+Because we describe a format that is used to display text,
+we are not so much interested in the
+character itself but we are interested in the specific glyph\index{glyph}.
+In typography, a glyph is a unique mark to be placed on the page representing
+a character. For example the glyph representing the character `a' can have
+many forms among them `{\it a\/}', `{\bf a}', or `{\tenss a}'.
+Such glyphs come in collections, called fonts, representing every character
+of the alphabet in a consistent way.
+
+The long format of a node describing the glyph `a'
+ might look like this:`` \.{<glyph} \.{97} \.{*1>}''.
+Here ``\.{97}'' is the character code which
+happens to be the ASCII code of the letter `a' and ``{\tt *1}'' is a font reference
+that stands for ``Computer Modern Roman 10pt''.
+Reference numbers, as you can see,
+start with an asterisk reminiscent of references in the \CEE\ programming language.
+The Astrix enables us to distinguish between ordinary numbers like ``\.{1}'' and references like ``{\tt *1}''.
+
+To make this node more readable, we will see in section~\secref{chars} that it is also
+possible to write `` \.{<glyph 'a' (cmr10) *1>}''.
+The latter form uses a comment ``\.{(cmr10)}'', enclosed in parentheses, to
+give an indication of what kind of font happens to be font 1, and it uses ``\.{'a'}'',
+the character enclosed in single quotes to denote the ASCII code of `a'.
+But let's keep things simple for now and stick with the decimal notation of the character code.
+
+The rest is common for all nodes: a keyword, here ``\.{glyph}'', and a pair of pointed brackets ``\.{<}\dots\.{>}''.
+
+Internally, we represent a glyph by the font number
+and the character number or character code.
+To store the internal representation of a glyph node,
+we define an appropriate structure type, named after the node with a trailing {\dots\bf\_t}.
+@<hint types@>=
+typedef struct {@+ uint32_t c;@+ uint8_t f; @+} glyph_t;
+@
+
+Let us now look at the program \.{shrink} and see how it will convert the long format description
+to the internal representation of the glyph and finally to a short format description.
+
+
+\subsection{Scanning the Long Format}
+First, \.{shrink} reads the input file and extracts a sequence of tokens. This is called ``scanning''\index{scanning}.
+We generate the procedure to do the scanning using the program \.{flex}\cite{JL:flexbison}\index{flex+{\tt flex}} which is the
+GNU version of the common UNIX tool \.{lex}\cite{JL:lexyacc}\index{lex+{\tt lex}}.
+
+The input to \.{flex} is a list of pattern/\kern -1pt action rules where the pattern is a regular
+expression and the action is a piece of \CEE\ code.
+Most of the time, the \CEE\ code is very simple: it just returns the right token\index{token} number
+to the parser which we consider shortly.
+
+The code that defines the tokens will be marked with a line ending in ``\redsymbol''.
+This symbol\index{symbol} stands for ``{\it Reading the long format\/}''.
+These code sequences define the syntactical elements of the long format and at the same time
+implement the reading process. All sections where that happens are preceded by a similar heading
+and for reference they are conveniently listed together starting on page~\pageref{codeindex}.
+
+\codesection{\redsymbol}{Reading the Long Format}\redindex{1}{2}{Glyphs}
+ at s START symbol
+ at s END symbol
+ at s GLYPH symbol
+ at s UNSIGNED symbol
+ at s REFERENCE symbol
+
+@<symbols@>=
+%token START "<"
+%token END ">"
+%token GLYPH "glyph"
+%token <u> UNSIGNED
+%token <u> REFERENCE
+@
+You might notice that a small caps font is used for |START|, |END| or |GLYPH|.
+These are ``terminal symbols'' or ``tokens''.
+Next are the scanning rules which define the connection between tokens and their
+textual representation.
+
+@<scanning rules@>=
+::@="<"@> :< SCAN_START; return START; >:
+::@=">"@> :< SCAN_END; return END; >:
+::@=glyph@> :< return GLYPH; >:
+::@=0|[1-9][0-9]*@> :< SCAN_UDEC(yytext); return UNSIGNED; >:
+::@=\*(0|[1-9][0-9]*)@> :< SCAN_UDEC(yytext+1); return REFERENCE; >:
+::@=[[:space:]]@> :< ; >:
+::@=\([^()\n]*[)\n]@> :< ; >:
+@
+
+As we will see later, the macros starting with |SCAN_|\dots\ are scanning macros.
+Here |SCAN_UDEC| is a macro that converts the decimal representation
+that did match the given pattern to an unsigned integer value; it is explained in
+section~\secref{integers}.
+The macros |SCAN_START| and |SCAN_END| are explained in section~\secref{text}.
+
+
+The action ``{\tt ;}'' is a ``do nothing'' action; here it causes spaces or comments\index{comment}
+to be ignored. Comments start with an opening parenthesis and are terminated by a
+closing parenthesis or the end of line character.
+The pattern ``\.{[\^()\\n]}'' is a negated
+character class that matches all characters except parentheses and the newline
+character. These are not allowed inside comments. For detailed information about
+the patterns used in a \.{flex} program, see the \.{flex} user manual\cite{JL:flexbison}.
+
+\subsection{Parsing the Long Format}
+\label{parse_glyph}
+Next, the tokens produced by the scanner are assembled into larger entities.
+This is called ``parsing''\index{parsing}.
+We generate the procedure to do the parsing using the program \.{bison}\cite{JL:flexbison}\index{bison+{\tt bison}} which is
+the GNU version of the common UNIX tool \.{yacc}\cite{JL:lexyacc}\index{yacc+{\tt yacc}}.
+
+The input to \.{bison} is a list of parsing rules, called a ``grammar''\index{grammar}.
+The rules describe how to build larger entities from smaller entities.
+For a simple glyph node like `` \.{<glyph 97 *1>}'', we need just these rules:
+\codesection{\redsymbol}{Reading the Long Format}%\redindex{1}{2}{Glyphs}
+ at s content_node symbol
+ at s node symbol
+ at s glyph symbol
+ at s glyph_t int
+ at s start symbol
+@<symbols@>=
+%type <u> start
+%type <c> glyph
+@
+
+@<parsing rules@>=@/
+glyph: UNSIGNED REFERENCE @/{ $$.c=$1; REF(font_kind,$2); $$.f=$2; };
+content_node: start GLYPH glyph END { hput_tags($1,hput_glyph(&($3))); };
+start: START {HPUTNODE; $$=(uint32_t)(hpos++-hstart);}
+@
+
+You might notice that a slanted font is used for |glyph|, |content_node|, or |start|.
+These are ``nonterminal symbols' and occur on the left hand side of a rule. On the
+right hand side of a rule you find nonterminal symbols, as well as terminal\index{terminal symbol} symbols
+and \CEE\ code enclosed in braces.
+
+Within the \CEE\ code, the expressions |$1| and |$2| refer to the variables on the parse stack
+that are associated with the first and second symbol on the right hand side of the rule.
+In the case of our glyph node, these will be the values 97 and 1, respectively, as produced
+by the macro |SCAN_UDEC|.
+|$$| refers to the variable associated with the left hand side of the rule.
+These variables contain the internal representation of the object in question.
+The type of the variable is specified by a mandatory {\bf token} or optional {\bf type} clause
+when we define the symbol.
+In the above {\bf type} clause for |start| and |glyph| , the identifiers |u| and |c| refer to
+the |union| declaration of the parser (see page~\pageref{union})
+where we find |uint32_t u| and |glyph_t c|. The macro |REF| tests a reference number for
+its valid range.
+
+
+Reading a node is usually split into the following sequence of steps:
+\itemize
+\item Reading the node specification, here a |glyph|
+ consisting of an |UNSIGNED| value and a |REFERENCE| value.
+\item Creating the internal representation in the variable |$$|
+ based on the values of |$1|, |$2|, \dots\ Here the character
+ code field |c| is initialized using the |UNSIGNED| value
+ stored in |$1| and the font field |f| is initialized using
+ |$2| after checking the reference number for the proper range.
+\item A |content_node| rule explaining that |start| is followed by |GLYPH|,
+ the keyword that directs the parser to |glyph|, the
+ node specification, and a final |END|.
+\item Parsing |start|, which is defined as the token |START| will assign
+ to the corresponding variable |p| on the parse stack the current
+ position |hpos| in the output and increments that position
+ to make room for the start byte, which we will discuss shortly.
+\item At the end of the |content_node| rule, the \.{shrink} program calls
+ a {\it hput\_\dots\/} function, here |hput_glyph|, to write the short
+ format of the node as given by its internal representation to the output
+ and return the correct tag value.
+\item Finally the |hput_tags| function will add the tag as a start byte and end byte
+ to the output stream.
+\enditemize
+
+Now let's see how writing the short format works in detail.
+
+
+\subsection{Writing the Short Format}
+A content node in short form begins with a start\index{start byte} byte. It tells us what kind of node it is.
+To describe the content of a short \HINT\ file, 32 different kinds\index{kind} of nodes are defined.
+Hence the kind of a node can be stored in 5 bits and the remaining bits of the start byte
+can be used to contain a 3 bit ``info''\index{info} value.
+
+We define an enumeration type to give symbolic names to the kind-values.
+The exact numerical values are of no specific importance;
+we will see in section~\secref{text}, however, that the assignment chosen below,
+has certain advantages.
+
+Because the usage of kind-values in content nodes is
+slightly different from the usage in definition nodes, we define alternative names for some kind-values.
+To display readable names instead of numerical values when debugging,
+we define two arrays of strings as well. Keeping the definitions consistent
+is achieved by creating all definitions from the same list
+of identifiers using different definitions of the macro |DEF_KIND|.
+
+@<hint basic types@>=
+#define DEF_KIND(C,D,N) @[C##_kind=N@]
+typedef enum {@+@<kinds@>@+, at + @<alternative kind names@> @+} kind_t;
+#undef DEF_KIND
+@
+
+@<define |content_name| and |definition_name|@>=
+
+#define DEF_KIND(C,D,N) @[#C@]
+const char *content_name[32]=@+{@+@<kinds@>@;@+}@+;
+#undef DEF_KIND@#
+#define DEF_KIND(C,D,N) @[#D@]
+const char *definition_name[0x20]=@+{@+@<kinds@>@;@+}@+;
+#undef DEF_KIND
+@
+
+@<print |content_name| and |definition_name|@>=
+printf("const char *content_name[32]={");
+for (k=0; k<= 31;k++)
+{ printf("\"%s\"",content_name[k]);
+ if (k<31) printf(", ");
+}
+printf("};\n\n");
+printf("const char *definition_name[32]={");
+for (k=0; k<= 31;k++)
+{ printf("\"%s\"",definition_name[k]);
+ if (k<31) printf(", ");
+}
+printf("};\n\n");
+@
+
+
+
+
+
+
+\goodbreak
+\index{glyph kind+\\{glyph\_kind}}
+\index{font kind+\\{font\_kind}}
+\index{penalty kind+\\{penalty\_kind}}
+\index{int kind+\\{int\_kind}}
+\index{kern kind+\\{kern\_kind}}
+\index{xdimen kind+\\{xdimen\_kind}}
+\index{ligature kind+\\{ligature\_kind}}
+\index{disc kind+\\{disc\_kind}}
+\index{glue kind+\\{glue\_kind}}
+\index{language kind+\\{language\_kind}}
+\index{rule kind+\\{rule\_kind}}
+\index{image kind+\\{image\_kind}}
+\index{baseline kind+\\{baseline\_kind}}
+\index{dimen kind+\\{dimen\_kind}}
+\index{hbox kind+\\{hbox\_kind}}
+\index{vbox kind+\\{vbox\_kind}}
+\index{par kind+\\{par\_kind}}
+\index{math kind+\\{math\_kind}}
+\index{table kind+\\{table\_kind}}
+\index{item kind+\\{item\_kind}}
+\index{hset kind+\\{hset\_kind}}
+\index{vset kind+\\{vset\_kind}}
+\index{hpack kind+\\{hpack\_kind}}
+\index{vpack kind+\\{vpack\_kind}}
+\index{stream kind+\\{stream\_kind}}
+\index{page kind+\\{page\_kind}}
+\index{range kind+\\{range\_kind}}
+\index{adjust kind+\\{adjust\_kind}}
+\index{param kind+\\{param\_kind}}
+\index{text kind+\\{text\_kind}}
+\index{list kind+\\{list\_kind}}
+\label{kinddef}
+@<kinds@>=
+DEF_KIND(t@&ext,t@&ext,0),@/
+DEF_KIND(l@&ist,l@&ist,1),@/
+DEF_KIND(p@&aram,p@&aram,2),@/
+DEF_KIND(x@&dimen,x@&dimen,3),@/
+DEF_KIND(a@&djust,a@&djust,4),@/
+DEF_KIND(g@&lyph, f@&ont,5),@/
+DEF_KIND(k@&ern,d@&imen,6),@/
+DEF_KIND(g@&lue,g@&lue,7),@/
+DEF_KIND(l@&igature,l@&igature,8),@/
+DEF_KIND(d@&isc,d@&isc,9),@/
+DEF_KIND(l@&anguage,l@&anguage,10),@/
+DEF_KIND(r@&ule,r@&ule,11),@/
+DEF_KIND(i@&mage,i@&mage,12),@/
+DEF_KIND(l@&eaders,l@&eaders,13),@/
+DEF_KIND(b@&aseline,b@&aseline,14),@/
+DEF_KIND(h@&b@&ox,h@&b@&ox,15),@/
+DEF_KIND(v@&b@&ox,v@&b@&ox,16),@/
+DEF_KIND(p@&ar,p@&ar,17),@/
+DEF_KIND(m@&ath,m@&ath,18),@/
+DEF_KIND(t@&able,t@&able,19),@/
+DEF_KIND(i@&tem,i@&tem,20),@/
+DEF_KIND(h@&set,h@&set,21),@/
+DEF_KIND(v@&set,v@&set,22),@/
+DEF_KIND(h@&pack,h@&pack,23),@/
+DEF_KIND(v@&pack,v@&pack,24),@/
+DEF_KIND(s@&tream,s@&tream,25),@/
+DEF_KIND(p@&age,p@&age,26),@/
+DEF_KIND(r@&ange,r@&ange,27),@/
+DEF_KIND(l@&ink,l@&abel,28),@/
+DEF_KIND(u@&ndefined2,u@&ndefined2,29),@/
+DEF_KIND(u@&ndefined3,u@&ndefined3,30),@/
+DEF_KIND(p@&enalty, i@&nt,31)
+ at t@>
+@
+
+For a few kind-values we have
+alternative names; we will use them
+to express different intentions when using them.
+@<alternative kind names@>=
+font_kind=glyph_kind,int_kind=penalty_kind, dimen_kind=kern_kind, label_kind=link_kind, outline_kind=link_kind@/@t{}@>
+@
+
+The info\index{info value} values can be used to represent numbers in the range 0 to 7; for an example
+see the |hput_glyph| function later in this section.
+Mostly, however, the individual bits are used as flags indicating the presence
+or absence of immediate parameter values. If the info bit is set, it
+means the corresponding parameter is present as an immediate value; if it
+is zero, it means that there is no immediate parameter value present, and
+the node specification will reveal what value to use instead.
+In some cases there is a common default value that can be used, in other
+cases a one byte reference number is used to select a predefined value.
+
+To make the binary
+representation of the info bits more readable, we define an
+enumeration type.
+
+\index{b000+\\{b000}}
+\index{b001+\\{b001}}
+\index{b010+\\{b010}}
+\index{b011+\\{b011}}
+\index{b100+\\{b100}}
+\index{b101+\\{b101}}
+\index{b110+\\{b110}}
+\index{b111+\\{b111}}
+@<hint basic types@>=
+typedef enum {@+ b000=0,b001=1,b010=2,b011=3,b100=4,b101=5,b110=6,b111=7 at + } info_t;
+@
+
+
+After the start byte follows the node content and it is the purpose of
+the start byte to reveal the exact syntax and semantics of the node
+content. Because we want to be able to read the short form of a \HINT\
+file in forward direction and in backward direction, the start byte is
+duplicated after the content as an end\index{end byte} byte.
+
+
+We store a kind and an info value in one byte and call this a tag.
+The following macros are used to assemble and disassemble tags:\index{TAG+\.{TAG}}
+@<hint macros@>=
+#define @[KIND(T)@] (((T)>>3)&0x1F)
+#define @[NAME(T)@] @[content_name[KIND(T)]@]
+#define @[INFO(T)@] ((T)&0x7)
+#define @[TAG(K,I)@] (((K)<<3)|(I))
+@
+
+Writing a short format \HINT\ file is implemented by a collection of {\it hput\_\kern 1pt\dots\/} functions;
+they follow most of the time the same schema:
+\itemize
+\item First, we define a variable for |info|.
+\item Then follows the main part of the function body, where we
+decide on the output format, do the actual output and set the |info| value accordingly.
+\item We combine the info value with the kind-value and return the correct tag.
+\item The tag value will be passed to |hput_tags| which generates
+debugging information, if requested, and stores the tag before and after the node content.
+\enditemize
+
+
+After these preparations, we turn our attention again to the |hput_glyph| function.
+The font number in a glyph node is between 0 and 255 and fits nicely in one byte,
+but the character code is more difficult: we want to store the most common character
+codes as a single byte and less frequent codes with two, three, or even four byte.
+Naturally, we use the |info| bits to store the number of bytes needed for the character code.
+
+\codesection{\putsymbol}{Writing the Short Format}\putindex{1}{2}{Glyphs}
+@<put functions@>=
+static uint8_t hput_n(uint32_t n)
+{@+ if (n<=0xFF) @+
+ {@+HPUT8(n);@+ return 1;@+}
+ else if (n<=0xFFFF) @+
+ {@+HPUT16(n);@+ return 2;@+}
+ else if (n<=0xFFFFFF)@+
+ {@+HPUT24(n);@+ return 3;@+}
+ else @+
+ {@+HPUT32(n);@+ return 4;@+}
+}
+
+uint8_t hput_glyph(glyph_t *g)
+{ info_t info;
+ info = hput_n(g->c);
+ HPUT8(g->f);@/
+ return TAG(glyph_kind,info);
+}
+@
+The |hput_tags| function is called after the node content has been written to the
+stream. It gets a the position of the start byte and the tag. With this information
+it writes the start byte at the given position and the end byte at the current stream position.
+@<put functions@>=
+void hput_tags(uint32_t pos, uint8_t tag)
+{ DBGTAG(tag,hstart+pos);DBGTAG(tag,hpos);
+ HPUTX(1); *(hstart+pos)=*(hpos++)=tag; @+
+}
+@
+
+
+
+The variables |hpos| and |hstart|, the macros |HPUT8|, |HPUT16|,
+|HPUT24|, |HPUT32|, and |HPUTX| are all defined in
+section~\secref{HPUT}; they put 8, 16, 24, or 32 bits into the output
+stream and check for sufficient space in the output buffer. The macro
+|DBGTAG| writes debugging output; its definition is found in
+section~\secref{error_section}.
+
+Now that we have seen the general outline of the \.{shrink} program,
+starting with a long format file and ending with a short format file,
+we will look at the program \.{stretch} that reverses this
+transformation.
+
+
+\subsection{Parsing the Short Format}
+The inverse of writing the short format with a {\it hput\_\kern 1pt\dots\/} function
+is reading the short format with a {\it hget\_\kern 1pt\dots\/} function.
+
+The schema of {\it hget\_\kern 1pt\dots\/} functions reverse the schema of {\it hput\_\kern 1pt\dots\/} functions.
+Here is the code for the initial and final part of a get function:
+
+@<read the start byte |a|@>=
+uint8_t a,z; /* the start and the end byte*/
+uint32_t node_pos=hpos-hstart;
+if (hpos>=hend) QUIT("Attempt to read a start byte at the end of the section");
+HGETTAG(a);@/@t{}@>
+@
+
+@<read and check the end byte |z|@>=
+HGETTAG(z);@+
+if (a!=z)
+ QUIT(@["Tag mismatch [%s,%d]!=[%s,%d] at 0x%x to " SIZE_F "\n"@],@|
+ NAME(a),INFO(a),NAME(z),INFO(z),@|node_pos, hpos-hstart-1);
+@
+
+
+The central routine to parse\index{parsing} the content section of a short format
+file is the function |hget_content_node| which calls |hget_content| to
+do most of the processing.
+
+|hget_content_node| will read a content node in short format and write
+it out in long format: It reads the start\index{start byte} byte |a|, writes the |START|
+token using the function |hwrite_start|, and based on |KIND(a)|, it
+writes the node's keyword found in the |content_name| array. Then it
+calls |hget_content| to read the node's content and write it out.
+Finally it reads the end\index{end byte} byte, checks it against the start byte, and
+finishes up the content node by writing the |END| token using the
+|hwrite_end| function. The function returns the tag byte so that
+the calling function might check that the content node meets its requirements.
+
+|hget_content| uses the start byte |a|, passed as a parameter, to
+branch directly to the reading routine for the given combination of
+kind and info value. The reading routine will read the data and store
+its internal representation in a variable. All that the \.{stretch}
+program needs to do with this internal representation is writing it in
+the long format. As we will see, the call to the proper
+{\it hwrite\_\kern 1pt\dots} function is included as final part of the the
+reading routine (avoiding another switch statement).
+
+
+\codesection{\getsymbol}{Reading the Short Format}\getindex{1}{2}{Content Nodes}
+@<get functions@>=
+void hget_content(uint8_t a);
+uint8_t hget_content_node(void)
+{ @<read the start byte |a|@>@;@+ hwrite_start();
+ hwritef("%s",content_name[KIND(a)]);
+ hget_content(a);@/
+ @<read and check the end byte |z|@>@; hwrite_end();
+ return a;
+}
+
+void hget_content(uint8_t a)
+{@+
+ switch (a)@/
+ {@+
+ @<cases to get content@>@;@t\1@>@/
+ default:
+ TAGERR(a);
+ break;@t\2@>@/
+ }
+}
+@
+
+We implement the code to read a glyph node in two stages.
+First we define a general reading macro |HGET_GLYPH(I,G)| that reads a glyph node with info value |I| into
+a |glyph_t| variable |G|; then we insert this macro
+in the above switch statement for all cases where it applies.
+Knowing the function |hput_glyph|, the macro |HGET_GLYPH| should not be a surprise.
+It reverses |hput_glyph|, storing the glyph node in its internal representation.
+After that, the \.{stretch} program calls |hwrite_glyph| to produce the glyph
+node in long format.
+
+\codesection{\getsymbol}{Reading the Short Format}\getindex{1}{2}{Glyphs}
+@<get macros@>=
+#define @[HGET_N(I,X)@] \
+ if ((I)==1) (X)=HGET8;\
+ else if ((I)==2) HGET16(X);\
+ else if ((I)==3) HGET24(X);\
+ else if ((I)==4) HGET32(X);
+
+#define @[HGET_GLYPH(I,G)@] \
+ HGET_N(I,(G).c); (G).f=HGET8; @+REF_RNG(font_kind,(G).f);@/\
+ hwrite_glyph(&(G));\
+@
+
+Note that we allow a glyph to reference a font even before that font is defined.
+This is necessary because fonts usually contain definitions---for example
+the fonts hyphen character---that reference this or other fonts.
+
+
+@<cases to get content@>=
+ at t\1\kern1em@>case TAG(glyph_kind,1): @+{@+glyph_t g;@+ HGET_GLYPH(1,g);@+}@+break;
+case TAG(glyph_kind,2): @+{@+glyph_t g;@+ HGET_GLYPH(2,g);@+}@+break;
+case TAG(glyph_kind,3): @+{@+glyph_t g;@+ HGET_GLYPH(3,g);@+}@+break;
+case TAG(glyph_kind,4): @+{@+glyph_t g;@+ HGET_GLYPH(4,g);@+}@+break;
+@
+
+If this two stage method seems strange to you, consider what the \CEE\ compiler will
+do with it. It will expand the |HGET_GLYPH| macro four times inside the switch
+statement. The macro is, however, expanded with a constant |I| value, so the expansion
+of the |if| statement in |HGET_GLYPH(1,g)|, for example,
+will become ``|if (1==1)| \dots\ |else if (1==2)| \dots''
+and the compiler will have no difficulties eliminating the constant tests and
+the dead branches altogether. This is the most effective use of the switch statement:
+a single jump takes you to a specialized code to handle just the given combination
+of kind and info value.
+
+Last not least, we implement the function |hwrite_glyph| to write a
+glyph node in long form---that is: in a form that is as readable as possible.
+
+\subsection{Writing the Long Format}
+
+The |hwrite_glyph| function inverts the scanning and parsing process we have described
+at the very beginning of this chapter.
+To implement the |hwrite_glyph| function, we use the function |hwrite_charcode|
+to write the character code.
+Besides writing the character code as a decimal number, this function can handle also other
+representations of character codes as fully explained in section~\secref{chars}.
+We split off the writing of the opening and the closing pointed bracket, because
+we will need this function very often and because it will keep track of the |nesting|
+of nodes and indent them accordingly. The |hwrite_range| and |hwrite_label| functions
+used in |hwrite_end| are discussed in section~\secref{range} and~\secref{hwritelabel}.
+
+\codesection{\wrtsymbol}{Writing the Long Format}\wrtindex{1}{2}{Glyphs}
+@<write functions@>=
+int nesting=0;
+void hwrite_nesting(void)
+{ int i;
+ hwritec('\n');
+ for (i=0;i<nesting;i++) hwritec(' ');
+}
+
+void hwrite_start(void)
+{ @+hwrite_nesting();@+ hwritec('<');@+ nesting++;
+}
+
+void hwrite_range(void);
+void hwrite_label(void);
+
+void hwrite_end(void)
+{ nesting--; hwritec('>');
+ if (section_no==2)
+ { if (nesting==0) hwrite_range();
+ hwrite_label();
+ }
+}
+
+void hwrite_comment(char *str)
+{ char c;
+ if (str==NULL) return;
+ hwritef(" (");
+ while ((c=*str++)!=0)
+ if (c=='(' || c==')') hwritec('_');
+ else if (c=='\n') hwritef("\n(");
+ else hwritec(c);
+ hwritec(')');
+}
+
+void hwrite_charcode(uint32_t c);
+void hwrite_ref(int n);
+
+void hwrite_glyph(glyph_t *g)
+{ char *n=hfont_name[g->f];
+ hwrite_charcode(g->c);
+ hwrite_ref(g->f);
+ if (n!=NULL) hwrite_comment(n);
+}
+@
+
+The two primitive operations to write the long format file are defined
+as macros:
+
+@<write macros@>=
+#define @[hwritec(c)@] @[putc(c,hout)@]
+#define @[hwritef(...)@] @[fprintf(hout,__VA_ARGS__)@]
+@
+
+
+Now that we have completed the round trip of shrinking and stretching
+glyph nodes, we continue the description of the \HINT\ file formats
+in a more systematic way.
+
+
+\section{Data Types}\hascode
+\subsection{Integers}
+\label{integers}
+We have already seen the pattern/\kern -1pt action rule for unsigned decimal\index{decimal number} numbers. It remains
+to define the macro |SCAN_UDEC| which converts a string containing an unsigned\index{unsigned} decimal
+number into an unsigned integer\index{integer}.
+We use the \CEE\ library function | strtoul|:
+
+\readcode
+@<scanning macros@>=
+#define @[SCAN_UDEC(S)@] @[yylval.u=strtoul(S,NULL,10)@]
+@
+Unsigned integers can be given in hexadecimal\index{hexadecimal} notation as well.
+@<scanning definitions@>=
+::@=HEX@> :< @=[0-9A-F]@> >:
+@
+
+@<scanning rules@>=
+::@=0x{HEX}+@> :< SCAN_HEX(yytext+2); return UNSIGNED; >:
+@
+
+Note that the pattern above allows only upper case letters in the
+hexadecimal notation for integers.
+
+@<scanning macros@>=
+#define @[SCAN_HEX(S)@] @[yylval.u=strtoul(S,NULL,16)@]
+@
+
+Last not least, we add rules for signed\index{signed integer} integers.
+ at s SIGNED symbol
+ at s number symbol
+ at s integer symbol
+
+@<symbols@>=
+%token <i> SIGNED
+%type <i> integer
+@
+
+@<scanning rules@>=
+::@=[+-](0|[1-9][0-9]*)@> :< SCAN_DEC(yytext); return SIGNED; >:
+@
+
+@<scanning macros@>=
+#define @[SCAN_DEC(S)@] @[yylval.i=strtol(S,NULL,10)@]
+@
+
+@<parsing rules@>=
+integer: SIGNED @+| UNSIGNED { RNG("number",$1,0,INT32_MAX);};
+@
+
+To preserve the ``signedness'' of an integer also for positive signed integers
+in the long format, we implement the function |hwrite_signed|.
+
+\writecode
+@<write functions@>=
+void hwrite_signed(int32_t i)
+{ if (i<0) hwritef(" -%d",-i);
+ else hwritef(" +%d",+i);
+}
+@
+
+Reading and writing integers in the short format is done directly with the {\tt HPUT} and {\tt HGET}
+macros.
+
+
+\subsection{Strings}
+\label{strings}
+Strings\index{string} are needed in the definition part of a \HINT\
+file to specify names of objects, and in the long file format, we also use them for file\index{file name} names.
+In the long format, strings are sequences of characters delimited by single quote\index{single quote} characters;
+for example: ``\.{'Hello'}'' or ``\.{'cmr10-600dpi.tfm'}''; in the short format, strings are
+byte sequences terminated by a zero byte.
+Because file names are system dependent, we no not allow arbitrary characters in strings
+but only printable ASCII codes which we can reasonably expect to be available on most operating systems.
+If your file names in a long format \HINT\ file are supposed to be portable,
+you should probably be even more restrictive. For example you should avoid characters like
+``\.{\\}'' or ``\.{/}'' which are used in different ways for directories.
+
+The internal representation of a string is a simple zero terminated \CEE\ string.
+When scanning a string, we copy it to the |str_buffer| keeping track
+of its length in |str_length|. When done,
+we make a copy for permanent storage and return the pointer to the parser.
+To operate on the |str_buffer|, we define a few macros.
+The constant |MAX_STR| determines the maximum size of a string (including the zero byte) to be $2^{10}$ byte.
+This restriction is part of the \HINT\ file format specification.
+
+@<scanning macros@>=
+#define MAX_STR (1<<10) /* $2^{10}$ Byte or 1kByte */
+static char str_buffer[MAX_STR];
+static int str_length;
+#define STR_START @[(str_length=0)@]
+#define @[STR_PUT(C)@] @[(str_buffer[str_length++]=(C))@]
+#define @[STR_ADD(C)@] @[STR_PUT(C);RNG("String length",str_length,0,MAX_STR-1)@]
+#define STR_END @[str_buffer[str_length]=0@]
+#define SCAN_STR @[yylval.s=str_buffer@]
+@
+
+
+To scan a string, we switch the scanner to |STR| mode when we find a quote character,
+then we scan bytes in the range |0x20| to |0x7E|, which is the range of printable ASCII
+characters, until we find the closing single\index{single quote} quote.
+Quote characters inside the string are written as two consecutive single quote characters.
+
+\readcode
+ at s STRING symbol
+ at s STR symbol
+ at s INITIAL symbol
+
+@<scanning definitions@>=
+%x STR
+@
+
+@<symbols@>=
+%token <s> STRING
+@
+
+@<scanning rules@>=
+::@='@> :< STR_START; BEGIN(STR); >:
+<STR>{
+::@='@> :< STR_END; SCAN_STR; BEGIN(INITIAL); return STRING; >:
+::@=''@> :< STR_ADD('\''); >:
+::@=[\x20-\x7E]@> :< STR_ADD(yytext[0]); >:
+::@=.@> :< RNG("String character",yytext[0],0x20,0x7E); >:
+::@=\n@> :< QUIT("Unterminated String in line %d",yylineno); >:
+}
+
+
+@
+The function |hwrite_string| reverses this process; it must take care of the quote symbols.
+\writecode
+@<write functions@>=
+void hwrite_string(char *str)
+{@+hwritec(' ');
+ if (str==NULL) hwritef("''");
+ else@/
+ { hwritec('\'');
+ while (*str!=0)@/
+ { @+if (*str=='\'') hwritec('\'');
+ hwritec(*str++);
+ }
+ hwritec('\'');
+ }
+}
+
+
+@
+In the short format, a string is just a byte sequence terminated by a zero byte.
+This makes the function |hput_string|, to write a string, and the macro |HGET_STRING|,
+to read a string in short format, very simple. Note that after writing an unbounded
+string to the output buffer, the macro |HPUTNODE| will make sure that there is enough
+space left to write the remainder of the node.
+
+\putcode
+@<put functions@>=
+void hput_string(char *str)
+{ char *s=str;
+ if (s!=NULL)
+ { do {
+ HPUTX(1);
+ HPUT8(*s);
+ } while (*s++!=0);
+ HPUTNODE;
+ }
+ else HPUT8(0);
+}
+@
+
+\getcode
+@<shared get macros@>=
+#define @[HGET_STRING(S)@] @[S=(char*)hpos;\
+ while(hpos<hend && *hpos!=0) { RNG("String character",*hpos,0x20,0x7E); hpos++;}\
+ hpos++;
+@
+
+\subsection{Character Codes}
+\label{chars}
+We have already seen in the introduction that character\index{character code} codes can be written as decimal numbers
+and section~\secref{integers} adds the possibility to use hexadecimal numbers as well.
+
+It is, however, in most cases more readable if we represent character codes directly
+using the characters themselves. Writing ``\.{a}'' is just so much better than writing ``\.{97}''.
+To distinguish the character ``\.{9}'' from the number ``\.{9}'', we use the common technique
+of enclosing characters within single\index{single quote} quotes. So ``\.{'9'}'' is the character code and
+``\.{9}'' is the number.
+Therefore we will define |CHARCODE| tokens and complement the parsing rules of section~\secref{parse_glyph}
+with the following rule:
+\readcode
+@<parsing rules@>=
+glyph: CHARCODE REFERENCE @|{ $$.c=$1; REF(font_kind,$2); $$.f=$2; };
+@
+
+
+
+If the character codes are small, we can represent them using
+ASCII character codes. We do not offer a special notation for very small
+character codes that map to the non-printable ASCII control codes; for them, the decimal
+or hexadecimal notation will suffice.
+For larger character codes, we use the multibyte encoding scheme known from UTF8\index{UTF8} as
+follows. Given a character code~|c|:
+
+\itemize
+\item
+Values in the range |0x00| to |0x7f| are encoded as a single byte with a leading bit of 0.
+
+@<scanning definitions@>=
+::@=UTF8_1@> :< @=[\x00-\x7F]@> >:
+@
+@<scanning macros@>=
+#define @[SCAN_UTF8_1(S)@] @[yylval.u=((S)[0]&0x7F)@]
+@
+
+
+\item
+Values in the range |0x80| to |0x7ff| are encoded in two byte with the first byte
+having three high bits |110|, indicating a two byte sequence, and the lower five bits equal
+to the five high bits of |c|. It is followed by a continuation byte having two high bits |10|
+and the lower six bits
+equal to the lower six bits of |c|.
+
+@<scanning definitions@>=
+::@=UTF8_2@> :< @=[\xC0-\xDF][\x80-\xBF]@> >:
+@
+
+@<scanning macros@>=
+#define @[SCAN_UTF8_2(S)@] @[yylval.u=(((S)[0]&0x1F)<<6)+((S)[1]&0x3F)@]
+@
+
+\item
+Values in the range |0x800| to |0xFFFF| are encoded in three byte with the first byte
+having the high bits |1110| indicating a three byte sequence followed by two continuation bytes.
+
+@<scanning definitions@>=
+::@=UTF8_3@> :< @=[\xE0-\xEF][\x80-\xBF][\x80-\xBF]@> >:
+@
+
+@<scanning macros@>=
+#define @[SCAN_UTF8_3(S)@] @[yylval.u=(((S)[0]&0x0F)<<12)+(((S)[1]&0x3F)<<6)+((S)[2]&0x3F)@]
+@
+
+\item
+Values in the range |0x1000| to |0x1FFFFF| are encoded in four byte with the first byte
+having the high bits |11110| indicating a four byte sequence followed by three continuation bytes.
+
+@<scanning definitions@>=
+::@=UTF8_4@> :< @=[\xF0-\xF7][\x80-\xBF][\x80-\xBF][\x80-\xBF]@> >:
+@
+
+@<scanning macros@>=
+#define @[SCAN_UTF8_4(S)@] @[yylval.u=(((S)[0]&0x03)<<18)+(((S)[1]&0x3F)<<12)+@|(((S)[2]&0x3F)<<6)+((S)[3]&0x3F)@]
+@
+
+\enditemize
+
+In the long format file, we enclose a character code in single\index{single quote} quotes, just as we do for strings.
+This is convenient but it has the downside that we must exercise special care when giving the
+scanning rules in order
+not to confuse character codes with strings. Further we must convert character codes back into strings
+in the rare case where the parser expects a string and gets a character code because the string
+was only a single character long.
+
+Let's start with the first problem:
+The scanner might confuse a string\index{string} and a character code if the first or second
+character of the string is a quote character which is written as two consecutive quotes.
+For example \.{'a''b'} is a string with three characters, ``\.{a}'',
+``\.{'}'', and ``\.{b}''. Two character codes would need a space to separate
+them like this: \.{'a' 'b'}.
+
+
+ at s CHARCODE symbol
+@<symbols@>=
+%token <u> CHARCODE
+@
+
+@<scanning rules@>=
+::@='''@> :< STR_START; STR_PUT('\''); BEGIN(STR); >:
+::@=''''@> :< SCAN_UTF8_1(yytext+1); return CHARCODE; >:
+::@='[\x20-\x7E]''@> :< STR_START; STR_PUT(yytext[1]); STR_PUT('\''); BEGIN(STR); >:
+::@='''''@> :< STR_START; STR_PUT('\''); STR_PUT('\''); BEGIN(STR); >:
+::@='{UTF8_1}'@> :< SCAN_UTF8_1(yytext+1); return CHARCODE; >:
+::@='{UTF8_2}'@> :< SCAN_UTF8_2(yytext+1); return CHARCODE; >:
+::@='{UTF8_3}'@> :< SCAN_UTF8_3(yytext+1); return CHARCODE; >:
+::@='{UTF8_4}'@> :< SCAN_UTF8_4(yytext+1); return CHARCODE; >:
+@
+
+If needed, the parser can convert character codes back to single character strings.
+
+ at s string symbol
+
+@<symbols@>=
+%type <s> string
+@
+
+@<parsing rules@>=
+string: STRING @+ | CHARCODE { static char s[2];
+ RNG("String element",$1,0x20,0x7E);
+ s[0]=$1; s[1]=0; $$=s;};
+@
+
+
+The function |hwrite_charcode| will write a character code. While ASCII codes are handled directly,
+larger character codes are passed to the function |hwrite_utf8|.
+It returns the number of characters written.
+
+\writecode
+@<write functions@>=
+int hwrite_utf8(uint32_t c)
+{@+ if (c<0x80)
+ { hwritec(c); @+return 1;@+ }
+ else if (c<0x800)@/
+ { hwritec(0xC0|(c>>6));@+ hwritec(0x80|(c&0x3F));@+ return 2;@+}
+ else if (c<0x10000)@/
+ { hwritec(0xE0|(c>>12)); hwritec(0x80|((c>>6)&0x3F));@+ hwritec(0x80|(c&0x3F)); return 3; }
+ else if (c<0x200000)@/
+ { hwritec(0xF0|(c>>18));@+ hwritec(0x80|((c>>12)&0x3F));
+ hwritec(0x80|((c>>6)&0x3F));@+ hwritec(0x80|(c&0x3F)); return 4;}
+ else
+ RNG("character code",c,0,0x1FFFFF);
+ return 0;
+}
+
+void hwrite_charcode(uint32_t c)
+{ @+if (c < 0x20)
+ { if (option_hex) hwritef(" 0x%02X",c); /* non printable ASCII */
+ else hwritef(" %u",c);
+ }
+ else if (c=='\'') hwritef(" ''''");
+ else if (c<=0x7E) hwritef(" \'%c\'",c); /* printable ASCII */
+ else if (option_utf8) { hwritef(" \'"); @+ hwrite_utf8(c); @+ hwritec('\'');@+}
+ else if (option_hex) hwritef(" 0x%04X",c);
+ else hwritef(" %u",c);
+}
+@
+
+\getcode
+@<shared get functions@>=
+#define @[HGET_UTF8C(X)@] (X)=HGET8;@+ if ((X&0xC0)!=0x80) \
+ QUIT(@["UTF8 continuation byte expected at " SIZE_F " got 0x%02X\n"@],hpos-hstart-1,X)@;
+
+uint32_t hget_utf8(void)
+{ uint8_t a;
+ a=HGET8;
+ if (a<0x80) return a;
+ else
+ { if ((a&0xE0)==0xC0) @/
+ { uint8_t b; @+ HGET_UTF8C(b);
+ return ((a&~0xE0)<<6)+(b&~0xC0);
+ }
+ else if ((a&0xF0)==0xE0) @/
+ { uint8_t b,c; @+ HGET_UTF8C(b); @+ HGET_UTF8C(c);
+ return ((a&~0xF0)<<12)+((b&~0xC0)<<6)+(c&~0xC0);
+ }
+ else if ((a&0xF8)==0xF0) @/
+ { uint8_t b,c,d; @+ HGET_UTF8C(b); @+ HGET_UTF8C(c); @+ HGET_UTF8C(d);
+ return ((a&~0xF8)<<18)@|+ ((b&~0xC0)<<12)+((c&~0xC0)<<6)+(d&~0xC0);
+ }
+ else QUIT("UTF8 byte sequence expected");
+ }
+}
+@
+\putcode
+@<put functions@>=
+void hput_utf8(uint32_t c)
+{ @+HPUTX(4);
+ if (c<0x80)
+ HPUT8(c);
+ else if (c<0x800)
+ { HPUT8(0xC0|(c>>6));@+ HPUT8(0x80|(c&0x3F));@+ }
+ else if (c<0x10000)@/
+ { HPUT8(0xE0|(c>>12)); HPUT8(0x80|((c>>6)&0x3F));@+ HPUT8(0x80|(c&0x3F)); }
+ else if (c<0x200000)@/
+ { HPUT8(0xF0|(c>>18));@+ HPUT8(0x80|((c>>12)&0x3F));
+ HPUT8(0x80|((c>>6)&0x3F));@+ HPUT8(0x80|(c&0x3F)); }
+ else
+ RNG("character code",c,0,0x1FFFFF);
+}
+@
+
+\subsection{Floating Point Numbers}
+You know a floating point numbers\index{floating point number} when you see it because it features a radix\index{radix point} point.
+The optional exponent\index{exponent} allows you to ``float'' the point.
+
+\readcode
+ at s FPNUM symbol
+ at s number symbol
+
+@<symbols@>=
+%token <f> FPNUM
+%type <f> number
+@
+@<scanning rules@>=
+::@=[+-]?[0-9]+\.[0-9]+(e[+-]?[0-9])?@> :< SCAN_DECFLOAT; return FPNUM; >:
+@
+
+The layout of floating point variables of type |double|
+or |float| typically follows the IEEE754\index{IEEE754} standard\cite{IEEE754-1985}\cite{IEEE754-2008}.
+We use the following definitions:
+
+\index{float32 t+\&{float32\_t}}
+\index{float64 t+\&{float64\_t}}
+
+@<hint basic types@>=
+
+#define FLT_M_BITS 23
+#define FLT_E_BITS 8
+#define FLT_EXCESS 127
+
+#define DBL_M_BITS 52
+#define DBL_E_BITS 11
+#define DBL_EXCESS 1023
+
+@
+
+ at s float32_t int
+ at s float64_t int
+
+We expect a variable of type |float64_t| to have a binary representation using 64 bit.
+The most significant bit is the sign bit, then follow $|DBL_E_BITS|=11$ bits for
+the exponent\index{exponent}, and $|DBL_M_BITS|=52$ bits for the mantissa\index{mantissa}.
+The sign\index{sign bit} bit is 1 for a negative number and 0 for a positive number.
+A floating point number is stored in normalized\index{normalization} form which means that
+the mantissa is shifted such that it has exactly 52+1 bit not counting leading zeros.
+The leading bit is then always 1 and there is no need to store it. So 52 bits suffice.
+To store the exponent, the excess $q=1023$ is added and the result is stored as an
+unsigned 11 bit number.
+For example if we regard the exponent bits and the mantissa bits as unsigned binary numbers
+$e$ and $m$ then the absolute value of such a floating point number can be expressed
+as $(1+m*2^{-52})\cdot2^{e-1023}$.
+We make similar assumptions about variables of type |float32_t| using the constants as defined above.
+
+
+
+
+
+To convert the decimal representation of a floating point number to binary values of type |float64_t|,
+we use a \CEE\ library function.
+
+@<scanning macros@>=
+#define SCAN_DECFLOAT @[yylval.f=atof(yytext)@]
+@
+
+When the parser expects a floating point number and gets an integer number,
+it converts it. So whenever in the long format a floating point number is expected,
+an integer number will do as well.
+
+@<parsing rules@>=
+number: UNSIGNED {$$=(float64_t)$1; } | SIGNED {$$=(float64_t)$1; } | FPNUM;
+@
+
+Unfortunately the decimal representation is not optimal for floating point numbers
+since even simple numbers in decimal notation like $0.1$ do not have an exact
+representation as a binary floating point number.
+So if we want a notation that allows an exact representation
+of binary floating point numbers, we must use a hexadecimal\index{hexadecimal} representation.
+Hexadecimal floating point numbers start with an optional sign, then as usual the two characters ``{\tt 0x}'',
+then follows a sequence of hex digits, a radix point, more hex digits, and an optional exponent.
+The optional exponent starts with the character ``{\tt x}'', followed by an optional sign, and some more
+hex digits. The hexadecimal exponent is given as a base 16 number and it is interpreted as an exponent
+with the base 16. As an example an exponent of ``{\tt x10}'', would multiply the mantissa by $16^{16}$.
+In other words it would shift any mantissa 16 hexadecimal digits to the left. Here are the exact rules:
+
+@<scanning rules@>=
+::@=[+-]?0x{HEX}+\.{HEX}+(x[+-]?{HEX}+)?@> :< SCAN_HEXFLOAT; return FPNUM; >:
+@
+
+@<scanning macros@>=
+#define SCAN_HEXFLOAT @[yylval.f=xtof(yytext)@]
+@
+There is no function in the \CEE\ library for hexadecimal floating point notation
+so we have to write our own conversion routine.
+The function |xtof| converts a string matching the above regular expression to
+its binary representation. Its outline is very simple:
+
+@<scanning functions@>=
+
+float64_t xtof(char *x)
+{ int sign, digits, exp;
+ uint64_t mantissa=0;
+ DBG(DBGFLOAT,"converting %s:\n",x);
+ @<read the optional sign@>@;
+ x=x+2; /* skip ``\.{0x}'' */
+ @<read the mantissa@>@;
+ @<normalize the mantissa@>@;
+ @<read the optional exponent@>@;
+ @<return the binary representation@>@;
+}
+@
+
+Now the pieces:
+
+@<read the optional sign@>=
+ if (*x=='-') { sign=-1;@+ x++;@+ }
+ else if (*x=='+') { sign=+1;@+ x++;@+ }
+ else @+sign=+1;
+ DBG(DBGFLOAT,"\tsign=%d\n",sign);
+@
+
+When we read the mantissa, we use the temporary variable |mantissa|, keep track
+of the number of digits, and adjust the exponent while reading the fractional part.
+@<read the mantissa@>=
+ digits=0;
+ while (*x=='0') x++; /*ignore leading zeros*/
+ while (*x!='.')@/
+ { mantissa=mantissa<<4;
+ if (*x<'A') mantissa=mantissa+*x-'0';
+ else mantissa=mantissa+*x-'A'+10;
+ x++;
+ digits++;
+ }
+ x++; /* skip ``\.{.}'' */
+ exp=0;
+ while (*x!=0 && *x!='x')@/
+ { mantissa=mantissa<<4;
+ exp=exp-4;
+ if (*x<'A') mantissa=mantissa+*x-'0';
+ else mantissa=mantissa+*x-'A'+10;
+ x++;
+ digits++;
+ }
+ DBG(DBGFLOAT,"\tdigits=%d mantissa=0x%" PRIx64 ", exp=%d\n",@|digits,mantissa,exp);
+@
+
+To normalize the mantissa, first we shift it to place exactly one nonzero hexadecimal
+digit to the left of the radix point. Then we shift it right bit-wise until there is
+just a single 1 bit to the left of the radix point.
+To compensate for the shifting, we adjust the exponent accordingly.
+Finally we remove the most significant bit because it is
+not stored.
+
+@<normalize the mantissa@>=
+if (mantissa==0) return 0.0;
+{ int s;
+ s = digits-DBL_M_BITS/4;
+ if (s>1)
+ mantissa=mantissa>>(4*(s-1));
+ else if (s<1)
+ mantissa=mantissa<<(4*(1-s));
+ exp=exp+4*(digits-1);
+ DBG(DBGFLOAT,"\tdigits=%d mantissa=0x%" PRIx64 ", exp=%d\n",@|digits,mantissa,exp);
+ while ((mantissa>>DBL_M_BITS)>1)@/ { mantissa=mantissa>>1; @+ exp++;@+ }
+ DBG(DBGFLOAT,"\tdigits=%d mantissa=0x%" PRIx64 ", exp=%d\n",@|digits,mantissa,exp);
+ mantissa=mantissa&~((uint64_t)1<<DBL_M_BITS);
+ DBG(DBGFLOAT,"\tdigits=%d mantissa=0x%" PRIx64 ", exp=%d\n",@|digits,mantissa,exp);
+}
+@
+
+In the printed representation,
+the exponent is an exponent with base 16. For example, an exponent of 2 shifts
+the hexadecimal mantissa two hexadecimal digits to the left, which corresponds to a
+multiplication by ${16}^2$.
+
+@<read the optional exponent@>=
+ if (*x=='x')@/
+ { int s;
+ x++; /* skip the ``\.{x}'' */
+ if (*x=='-') {s=-1;@+x++;@+}
+ else if (*x=='+') {s=+1;@+x++;@+}
+ else s=+1;
+ DBG(DBGFLOAT,"\texpsign=%d\n",s);
+ DBG(DBGFLOAT,"\texp=%d\n",exp);
+ while (*x!=0 )
+ { if (*x<'A') exp=exp+4*s*(*x-'0');
+ else exp=exp+4*s*(*x-'A'+10);
+ x++;
+ DBG(DBGFLOAT,"\texp=%d\n",exp);
+ }
+ }
+ RNG("Floating point exponent",@|exp,-DBL_EXCESS,DBL_EXCESS);
+@
+
+To assemble the binary representation, we use a |union| of a |float64_t| and |uint64_t|.
+
+
+@<return the binary representation@>=
+{ union {@+float64_t d; @+uint64_t bits; @+} u;
+ if (sign<0) sign=1;@+ else at + sign=0; /* the sign bit */
+ exp=exp+DBL_EXCESS; /* the exponent bits */
+ u.bits=((uint64_t)sign<<63)@/
+ | ((uint64_t)exp<<DBL_M_BITS) | mantissa;
+ DBG(DBGFLOAT," return %f\n",u.d);
+ return u.d;
+}
+@
+
+The inverse function is |hwrite_float64|. It strives to print floating point numbers
+as readable as possible. So numbers without fractional part are written as integers.
+Numbers that can be represented exactly in decimal notation are represented in
+decimal notation. All other values are written as hexadecimal floating point numbers.
+We avoid an exponent if it can be avoided by using up to |MAX_HEX_DIGITS|
+
+\writecode
+@<write functions@>=
+#define MAX_HEX_DIGITS 12
+void hwrite_float64(float64_t d)
+{ uint64_t bits, mantissa;
+ int exp, digits;
+ hwritec(' ');
+ if (floor(d)==d)
+ { hwritef("%d",(int)d);@+ return;@+}
+ if (floor(10000.0*d)==10000.0*d)
+ { hwritef("%g",d); @+return;@+}
+ DBG(DBGFLOAT,"Writing hexadecimal float %f\n",d);
+ if (d<0) { hwritec('-');@+ d=-d;@+}
+ hwritef("0x");
+ @<extract mantissa and exponent@>@;
+ if (exp>MAX_HEX_DIGITS)
+ @<write large numbers@>@;
+ else if (exp>=0) @<write medium numbers@>@;
+ else @<write small numbers@>@;
+}
+@
+
+The extraction just reverses the creation of the binary representation.
+
+@<extract mantissa and exponent@>=
+{ union {@+float64_t d; @+ uint64_t bits; @+} u;
+ u.d=d; @+ bits=u.bits;
+}
+ mantissa= bits&(((uint64_t)1<<DBL_M_BITS)-1);
+ mantissa=mantissa+((uint64_t)1<<DBL_M_BITS);
+ exp= ((bits>>DBL_M_BITS)&((1<<DBL_E_BITS)-1))-DBL_EXCESS;
+ digits=DBL_M_BITS+1;
+ DBG(DBGFLOAT,"\tdigits=%d mantissa=0x%" PRIx64 " binary exp=%d\n",@|digits,mantissa,exp);
+@
+
+After we have obtained the binary exponent,
+we round it down, and convert it to a hexadecimal
+exponent.
+@<extract mantissa and exponent@>=
+ { int r;
+ if (exp>=0)
+ { r= exp%4;
+ if (r>0)
+ { mantissa=mantissa<<r; @+exp=exp-r; @+digits=digits+r; @+}
+ }
+ else
+ { r=(-exp)%4;
+ if (r>0)
+ { mantissa=mantissa>>r; @+exp=exp+r; @+digits=digits-r;@+}
+ }
+ }
+ exp=exp/4;
+ DBG(DBGFLOAT,"\tdigits=%d mantissa=0x%" PRIx64 " hex exp=%d\n",@|digits,mantissa,exp);
+@
+
+In preparation for writing,
+we shift the mantissa to the left so that the leftmost hexadecimal
+digit of it will occupy the 4 leftmost bits of the variable |bits| .
+
+@<extract mantissa and exponent@>=
+ mantissa=mantissa<<(64-DBL_M_BITS-4); /* move leading digit to leftmost nibble */
+@
+
+If the exponent is larger than |MAX_HEX_DIGITS|, we need to
+use an exponent even if the mantissa uses only a few digits.
+When we use an exponent, we always write exactly one digit preceding the radix point.
+
+@<write large numbers@>=
+{ DBG(DBGFLOAT,"writing large number\n");
+ hwritef("%X.",(uint8_t)(mantissa>>60));
+ mantissa=mantissa<<4;
+ do {
+ hwritef("%X",(uint8_t)(mantissa>>DBL_M_BITS)&0xF);
+ mantissa=mantissa<<4;
+ } while (mantissa!=0);
+ hwritef("x+%X",exp);
+}
+@
+If the exponent is small and non negative, we can write the
+number without an exponent by writing the radix point at the
+appropriate place.
+ @<write medium numbers@>=
+ { DBG(DBGFLOAT,"writing medium number\n");
+ do {
+ hwritef("%X",(uint8_t)(mantissa>>60));
+ mantissa=mantissa<<4;
+ if (exp--==0) hwritec('.');
+ } while (mantissa!=0 || exp>=-1);
+ }
+@
+Last non least, we write numbers that would require additional zeros after the
+radix point with an exponent, because it keeps the mantissa shorter.
+@<write small numbers@>=
+ { DBG(DBGFLOAT,"writing small number\n");
+ hwritef("%X.",(uint8_t)(mantissa>>60));
+ mantissa=mantissa<<4;
+ do {
+ hwritef("%X",(uint8_t)(mantissa>>60));
+ mantissa=mantissa<<4;
+ } while (mantissa!=0);
+ hwritef("x-%X",-exp);
+ }
+@
+
+Compared to the complications of long format floating point numbers,
+the short format is very simple because we just use the binary representation.
+Since 32 bit floating point numbers offer sufficient precision we use only
+the |float32_t| type.
+It is however not possible to just write |HPUT32(d)| for a |float32_t| variable |d|
+or |HPUT32((uint32_t)d)| because in the \CEE\ language this would imply
+rounding the floating point number to the nearest integer.
+But we have seen how to convert floating point values to bit pattern before.
+
+@<put functions@>=
+void hput_float32(float32_t d)
+{ union {@+float32_t d; @+ uint32_t bits; @+} u;
+ u.d=d; @+ HPUT32(u.bits);
+}
+@
+
+@<shared get functions@>=
+float32_t hget_float32(void)
+{ union {@+float32_t d; @+ uint32_t bits; @+} u;
+ HGET32(u.bits);
+ return u.d;
+}
+@
+
+\subsection{Fixed Point Numbers}
+\TeX\ internally represents most real numbers as fixed\index{fixed point number} point numbers or ``scaled integers''\index{scaled integer}.
+The type {\bf scaled\_t} is defined as a signed 32 bit integer, but we consider it as a fixed point number
+with the binary radix point just in the middle with sixteen bits before and sixteen bits after it.
+To convert an integer into a scaled number, we multiply it by |ONE|; to convert a floating point number
+into a scaled number, we multiply it by |ONE| and |ROUND| the result to the nearest integer;
+to convert a scaled number to a floating point number we divide it by |(float64_t)ONE|.
+
+\noindent
+@<hint basic types@>=
+typedef int32_t scaled_t;
+#define ONE ((scaled_t)(1<<16))
+@
+
+@<hint macros@>=
+#define ROUND(X) ((int)((X)>=0.0?floor((X)+0.5):ceil((X)-0.5)))
+@
+
+\writecode
+@<write functions@>=
+void hwrite_scaled(scaled_t x)
+{ hwrite_float64(x/(float64_t)ONE);
+}
+@
+
+\subsection{Dimensions}
+In the long format,
+the dimensions\index{dimension} of characters, boxes, and other things can be given
+in three units: \.{pt}, \.{in}, and \.{mm}.
+
+\readcode
+ at s PT symbol
+ at s MM symbol
+ at s INCH symbol
+ at s dimension symbol
+ at s DIMEN symbol
+@<symbols@>=
+%token DIMEN "dimen"
+%token PT "pt"
+%token MM "mm"
+%token INCH "in"
+%type <d> dimension
+@
+
+@<scanning rules@>=
+::@=dimen@> :< return DIMEN; >:
+::@=pt@> :< return PT; >:
+::@=mm@> :< return MM; >:
+::@=in@> :< return INCH; >:
+@
+
+The unit \.{pt} is a printers point\index{point}\index{pt+{\tt pt}}.
+The unit ``\.{in}'' stands for inches\index{inch}\index{in+{\tt in}} and we have $1\.{in}= 72.27\,\.{pt}$.
+The unit ``\.{mm}'' stands for millimeter\index{millimeter}\index{mm+{\tt mm}} and we have $1\.{in}= 25.4\,\.{mm}$.
+
+The definition of a printers\index{printers point} point given above follows the definition used in
+\TeX\ which is slightly larger than the official definition of a printer's
+point which was defined to equal exactly 0.013837\.{in} by the American Typefounders
+Association in~1886\cite{DK:texbook}.
+
+We follow the tradition of \TeX\ and
+store dimensions as ``scaled points''\index{scaled point} that is a dimension of $d$ points is
+stored as $d\cdot2^{16}$ rounded to the nearest integer.
+The maximum absolute value of a dimension is $(2^{30}-1)$ scaled points.
+
+@<hint basic types@>=
+typedef scaled_t dimen_t;
+#define MAX_DIMEN ((dimen_t)(0x3FFFFFFF))
+@
+
+@<parsing rules@>=
+dimension: number PT @|{$$=ROUND($1*ONE); RNG("Dimension",$$,-MAX_DIMEN,MAX_DIMEN); }
+ | number INCH @|{$$=ROUND($1*ONE*72.27); RNG("Dimension",$$,-MAX_DIMEN,MAX_DIMEN);@+}
+ | number MM @|{$$=ROUND($1*ONE*(72.27/25.4)); RNG("Dimension",$$,-MAX_DIMEN,MAX_DIMEN);@+};
+@
+
+When \.{stretch} is writing dimensions in the long format,
+for simplicity it always uses the unit ``\.{pt}''.
+\writecode
+@<write functions@>=
+void hwrite_dimension(dimen_t x)
+{ hwrite_scaled(x);
+ hwritef("pt");
+}
+@
+
+In the short format, dimensions are stored as 32 bit scaled point values without conversion.
+\getcode
+@<get functions@>=
+void hget_dimen(uint8_t a)
+{ if (INFO(a)==b000)
+ {uint8_t r; r=HGET8; REF(dimen_kind,r); hwrite_ref(r);}
+ else
+ { uint32_t d; HGET32(d); hwrite_dimension(d); }
+}
+@
+
+\putcode
+@<put functions@>=
+
+uint8_t hput_dimen(dimen_t d)
+{ HPUT32(d);
+ return TAG(dimen_kind, b001);
+}
+@
+
+
+
+\subsection{Extended Dimensions}\index{extended dimension}\index{hsize+{\tt hsize}}\index{vsize+{\tt vsize}}
+The dimension that is probably used most frequently in a \TeX\ file is {\tt hsize}:
+the ho\-ri\-zon\-tal size of a line of text. Common are also assignments
+like \.{\\hsize=0.5\\hsize} \.{\\advance\\hsize by -10pt}, for example to
+get two columns with lines almost half as wide as usual, leaving a small gap
+between left and right column. Similar considerations apply to {\tt vsize}.
+
+Because we aim at a reflowable format for \TeX\ output, we have to postpone
+such computations until the values of \.{hsize} and \.{vsize} are known in the viewer.
+Until then, we do symbolic computations on linear functions\index{linear function} of \.{hsize} and \.{vsize}.
+We call such a linear function $w+h\cdot\.{hsize}+v\cdot\.{vsize}$
+an extended dimension and represent it by the three numbers $w$, $h$, and $v$.
+
+@<hint basic types@>=
+typedef struct {@+
+dimen_t w; @+ float32_t h, v; @+
+} xdimen_t;
+@
+Since very often a component of an extended dimension is zero, we
+store in the short format only the nonzero components and use the
+info bits to mark them: |b100| implies $|w|\ne0$,
+|b010| implies $|h|\ne 0$, and |b001| implies $|v|\ne 0$.
+
+\readcode
+ at s XDIMEN symbol
+ at s xdimen symbol
+ at s xdimen_node symbol
+ at s H symbol
+ at s V symbol
+@<symbols@>=
+%token XDIMEN "xdimen"
+%token H "h"
+%token V "v"
+%type <xd> xdimen
+@
+@<scanning rules@>=
+::@=xdimen@> :< return XDIMEN; >:
+::@=h@> :< return H; >:
+::@=v@> :< return V; >:
+@
+
+
+@<parsing rules@>=
+xdimen: dimension number H number V { $$.w=$1; @+$$.h=$2; @+$$.v=$4; }
+ | dimension number H { $$.w=$1; @+$$.h=$2; @+$$.v=0.0; }
+ | dimension number V { $$.w=$1; @+$$.h=0.0; @+$$.v=$2; }
+ | dimension { $$.w=$1; @+$$.h=0.0; @+$$.v=0.0; };
+
+
+
+xdimen_node: start XDIMEN xdimen END { hput_tags($1,hput_xdimen(&($3))); };
+@
+
+\writecode
+@<write functions@>=
+void hwrite_xdimen(xdimen_t *x)
+{ hwrite_dimension(x->w);
+ if (x->h!=0.0) {hwrite_float64(x->h); @+hwritec('h');@+}
+ if (x->v!=0.0) {hwrite_float64(x->v); @+hwritec('v');@+}
+}
+
+void hwrite_xdimen_node(xdimen_t *x)
+{ hwrite_start(); hwritef("xdimen"); hwrite_xdimen(x); hwrite_end();}
+@
+
+\getcode
+
+@<get macros@>=
+#define @[HGET_XDIMEN(I,X)@] \
+ if((I)&b100) HGET32((X).w);@+ else (X).w=0;\
+ if((I)&b010) (X).h=hget_float32(); @+ else (X).h=0.0;\
+ if((I)&b001) (X).v=hget_float32(); @+else (X).v=0.0;
+@
+
+@<get functions@>=
+void hget_xdimen(uint8_t a, xdimen_t *x)
+{ switch(a)
+ {
+ case TAG(xdimen_kind,b001): HGET_XDIMEN(b001,*x);@+break;
+ case TAG(xdimen_kind,b010): HGET_XDIMEN(b010,*x);@+break;
+ case TAG(xdimen_kind,b011): HGET_XDIMEN(b011,*x);@+break;
+ case TAG(xdimen_kind,b100): HGET_XDIMEN(b100,*x);@+break;
+ case TAG(xdimen_kind,b101): HGET_XDIMEN(b101,*x);@+break;
+ case TAG(xdimen_kind,b110): HGET_XDIMEN(b110,*x);@+break;
+ case TAG(xdimen_kind,b111): HGET_XDIMEN(b111,*x);@+break;
+ default:
+ QUIT("Extent expected got [%s,%d]",NAME(a),INFO(a));
+ }
+ }
+@
+
+Note that the info value |b000|, usually indicating a reference,
+is not supported for extended dimensions.
+Most nodes that need an extended dimension offer the opportunity to give
+a reference directly without the start and end byte. An exception is the glue node,
+but glue nodes that need an extended width are rare.
+
+@<get functions@>=
+void hget_xdimen_node(xdimen_t *x)
+{ @<read the start byte |a|@>@;
+ if (KIND(a)==xdimen_kind)
+ hget_xdimen(a,x);
+ else
+ QUIT("Extent expected at 0x%x got %s",node_pos,NAME(a));
+ @<read and check the end byte |z|@>@;
+}
+@
+
+
+
+\putcode
+@<put functions@>=
+uint8_t hput_xdimen(xdimen_t *x)
+{ info_t info=b000;
+ if (x->w==0 && x->h==0.0 && x->v==0.0){ HPUT32(0); @+info|=b100; @+}
+ else
+ { if (x->w!=0) { HPUT32(x->w); @+info|=b100; @+}
+ if (x->h!=0.0) { hput_float32(x->h); @+info|=b010; @+}
+ if (x->v!=0.0) { hput_float32(x->v); @+info|=b001; @+}
+ }
+ return TAG(xdimen_kind,info);
+}
+void hput_xdimen_node(xdimen_t *x)
+{ uint32_t p=hpos++-hstart;
+ hput_tags(p, hput_xdimen(x));
+}
+
+
+@
+
+
+
+\subsection{Stretch and Shrink}\label{stretch}
+In section~\secref{glue}, we will consider glue\index{glue} which
+is something that can stretch and shrink.
+The stretchability\index{stretchability} and shrinkability\index{shrinkability} of the
+glue can be given in ``\.{pt}'' like a dimension,
+but there are three more units: \.{fil}, \.{fill}, and \.{filll}.
+A glue with a stretchability of $1\,\hbox{\tt fil}$ will stretch infinitely more
+than a glue with a stretchability of $1\,\hbox{\tt pt}$. So if you stretch both glues
+together, the first glue will do all the stretching and the latter will not stretch
+at all. The ``\.{fil}'' glue has simply a higher order of infinity.
+You might guess that ``\.{fill}'' glue and ``\.{filll}'' glue have even higher
+orders of infinite stretchability.
+The order of infinity is 0 for \.{pt}, 1 for \.{fil}, 2 for \.{fill}, and 3 for \.{filll}.
+
+The internal representation of a stretch is a variable of type |stretch_t|.
+It stores the floating point value and the order of infinity separate as a |float64_t| and a |uint8_t|.
+
+
+The short format tries to be space efficient and because it is not necessary to give the
+stretchability with a precision exceeding about six decimal digits,
+we use a single 32 bit floating point value.
+To write a |float32_t| value and an order value as one 32 bit value,
+we round the two lowest bit of the |float32_t| variable to zero
+using ``round to even'' and store the order of infinity in these bits.
+We define a union type \&{stch\_t} to simplify conversion.
+
+@<hint basic types@>=
+typedef enum { @+ normal_o=0, fil_o=1, fill_o=2, filll_o=3 at +} order_t;
+typedef struct {@+ float64_t f;@+ order_t o; @+} stretch_t;
+typedef union {@+float32_t f; @+ uint32_t u; @+} stch_t;
+@
+
+\putcode
+@<put functions@>=
+void hput_stretch(stretch_t *s)
+{ uint32_t mantissa, lowbits, sign, exponent;
+ stch_t st;
+ st.f=s->f;
+ DBG(DBGFLOAT,"joining %f->%f(0x%X),%d:",s->f,st.f,st.u,s->o);
+ mantissa = st.u &(((uint32_t)1<<FLT_M_BITS)-1);
+ lowbits = mantissa&0x7; /* lowest 3 bits */
+ exponent=(st.u>>FLT_M_BITS)&(((uint32_t)1<<FLT_E_BITS)-1);
+ sign=st.u & ((uint32_t)1<<(FLT_E_BITS+FLT_M_BITS));
+ DBG(DBGFLOAT,"s=%d e=0x%x m=0x%x",sign, exponent, mantissa);
+ switch (lowbits) /* round to even */
+ { @+case 0: break; /* no change */
+ case 1: mantissa = mantissa -1; @+break;/* round down */
+ case 2: mantissa = mantissa -2; @+break;/* round down to even */
+ case 3: mantissa = mantissa +1; @+break; /* round up */
+ case 4: break; /* no change */
+ case 5: mantissa = mantissa -1; @+break;/* round down */
+ case 6: mantissa = mantissa +1; /* round up to even, fall through */
+ case 7: mantissa = mantissa +1; /* round up to even */
+ if (mantissa >= ((uint32_t)1<<FLT_M_BITS))@/
+ {exponent++; /* adjust exponent */
+ RNG("Float32 exponent",exponent,1,2*FLT_EXCESS);
+ @+mantissa=mantissa>>1;
+ }
+ break;
+ }
+ DBG(DBGFLOAT," round s=%d e=0x%x m=0x%x",sign, exponent, mantissa);
+ st.u=sign| (exponent<<FLT_M_BITS) | mantissa | s->o;
+ DBG(DBGFLOAT,"float %f hex 0x%x\n",st.f, st.u);
+ HPUT32(st.u);
+}
+@
+
+\getcode
+@<get macros@>=
+#define @[HGET_STRETCH(S)@] { stch_t st; @+ HGET32(st.u);@+ S.o=st.u&3; st.u&=~3; S.f=st.f; @+}
+@
+
+\readcode
+ at s FIL symbol
+ at s FILL symbol
+ at s FILLL symbol
+ at s order symbol
+
+@<symbols@>=
+%token FIL "fil"
+%token FILL "fill"
+%token FILLL "filll"
+%type <st> stretch
+%type <o> order
+@
+
+@<scanning rules@>=
+::@=fil@> :< return FIL; >:
+::@=fill@> :< return FILL; >:
+::@=filll@> :< return FILLL; >:
+@
+
+ at s stretch symbol
+ at s stretch_t int
+@<parsing rules@>=
+
+order: PT {$$=normal_o;} | FIL {$$=fil_o;} @+| FILL {$$=fill_o;} @+| FILLL {$$=filll_o;};
+
+stretch: number order { $$.f=$1; $$.o=$2; };
+@
+
+\writecode
+
+@<write functions@>=
+void hwrite_order(order_t o)
+{ switch (o)
+ { case normal_o: hwritef("pt"); @+break;
+ case fil_o: hwritef("fil"); @+break;
+ case fill_o: hwritef("fill"); @+break;
+ case filll_o: hwritef("filll"); @+break;
+ default: QUIT("Illegal order %d",o); @+ break;
+ }
+}
+
+void hwrite_stretch(stretch_t *s)
+{ hwrite_float64(s->f);
+ hwrite_order(s->o);
+}
+@
+
+\section{Simple Nodes}\hascode
+\label{simple}
+\subsection{Penalties}
+Penalties\index{penalty} are very simple nodes. They specify the cost of breaking a
+line or page at the present position. For the internal representation
+we use an |int32_t|. The full range of integers is, however, not
+used. Instead penalties must be between -20000 and +20000.
+(\TeX\ specifies a range of -10000 to +10000, but plain \TeX\ uses the value -20000
+when it defines the supereject control sequence.)
+The more general node is called an integer node;
+it shares the same kind-value |int_kind=penalty_kind|
+but allows the full range of values.
+The info value of a penalty node is 1 or 2 and indicates the number of bytes
+used to store the integer. The info value 4 can be used for general
+integers (see section~\secref{definitions}) that need four byte of storage.
+
+\readcode
+ at s PENALTY symbol
+ at s INTEGER symbol
+ at s penalty symbol
+@<symbols@>=
+%token PENALTY "penalty"
+%token INTEGER "int"
+%type <i> penalty
+@
+
+@<scanning rules@>=
+::@=penalty@> :< return PENALTY; >:
+::@=int@> :< return INTEGER; >:
+@
+
+@<parsing rules@>=
+penalty: integer {RNG("Penalty",$1,-20000,+20000);$$=$1;};
+content_node: start PENALTY penalty END { hput_tags($1,hput_int($3));@+};
+@
+
+\getcode
+@<cases to get content@>=
+ at t\1\kern1em@>case TAG(penalty_kind,1): @+{int32_t p;@+ HGET_PENALTY(1,p);@+} @+break;
+case TAG(penalty_kind,2): @+{int32_t p;@+ HGET_PENALTY(2,p);@+} @+break;
+@
+
+@<get macros@>=
+#define @[HGET_PENALTY(I,P)@] \
+if (I==1) {int8_t n=HGET8; @+P=n;@+ } \
+else {int16_t n;@+ HGET16(n);@+RNG("Penalty",n,-20000,+20000); @+ P=n; @+}\
+hwrite_signed(P);
+@
+
+\putcode
+@<put functions@>=
+uint8_t hput_int(int32_t n)
+{ info_t info;
+ if (n>=0) @/
+ { @+if (n<0x80) { @+HPUT8(n); @+info=1;@+ }
+ else if (n<0x8000) {@+ HPUT16(n);@+ info=2;@+ }
+ else {@+ HPUT32(n);@+ info=4;@+ }
+ }
+ else@/
+ {@+ if (n>=-0x80) {@+ HPUT8(n);@+ info=1;@+ }
+ else if (n>=-0x8000) {@+ HPUT16(n);@+ info=2;@+ }
+ else {@+ HPUT32(n);@+ info=4;@+ }
+ }
+ return TAG(int_kind,info);
+}
+@
+
+
+
+
+\subsection{Languages}
+To render a \HINT\ file on screen, information about the language is not necessary.
+Knowing the language is, however, very important for language translation and
+text to speech conversion which makes texts accessible to the visually-impaired.
+For this reason, \HINT\ offers the opportunity to add this information
+and encourages authors to supply this information.
+
+Language information by itself is not sufficient to decode text. It must be supplemented
+by information about the character encoding (see section~\secref{fonts}).
+
+To represent language information, the world wide web has set universally
+accepted standards. The Internet Engineering Task Force IETF has defined tags for identifying
+languages\cite{rfc5646}: short strings like ``en'' for English
+or ``de'' for Deutsch, and longer ones like ``sl-IT-nedis'', for the specific variant of
+the Nadiza dialect of Slovenian that is spoken in Italy.
+We assume that any \HINT\ file
+will contain only a small number of different languages and all language nodes can be
+encoded using a reference to a predefined node from the
+definition section (see section~\secref{reference}).
+In the definition section, a language node will just
+contain the language tag as given in~\cite{iana:language} (see section~\secref{definitions}).
+
+\readcode
+ at s LANGUAGE symbol
+ at s language symbol
+
+@<symbols@>=
+%token LANGUAGE "language"
+@
+
+@<scanning rules@>=
+::@=language@> :< return LANGUAGE; >:
+@
+
+When encoding language nodes in the short format,
+we use the info value |b000| for language nodes in the definition section
+and for language nodes in the content section that contain just a one-byte
+reference (see section~\secref{reference}).
+We use the info value |1| to |7| as a shorthand for
+the references {\tt *0} and {\tt *6} to the predefined language nodes.
+
+
+\goodbreak
+\vbox{\getcode\vskip -\baselineskip\writecode}
+@<cases to get content@>=
+ at t\1\kern1em@>case TAG(language_kind,1): REF(language_kind,0); @+hwrite_ref(0); @+break;
+case TAG(language_kind,2): REF(language_kind,1); @+hwrite_ref(1); @+break;
+case TAG(language_kind,3): REF(language_kind,2); @+hwrite_ref(2); @+break;
+case TAG(language_kind,4): REF(language_kind,3); @+hwrite_ref(3); @+break;
+case TAG(language_kind,5): REF(language_kind,4); @+hwrite_ref(4); @+break;
+case TAG(language_kind,6): REF(language_kind,5); @+hwrite_ref(5); @+break;
+case TAG(language_kind,7): REF(language_kind,6); @+hwrite_ref(6); @+break;
+@
+
+\putcode
+@<put functions@>=
+uint8_t hput_language(uint8_t n)
+{ if (n<7) return TAG(language_kind,n+1);
+ HPUT8(n); return TAG(language_kind,0);
+}
+@
+
+
+
+\subsection{Rules}
+Rules\index{rule} are simply black rectangles having a height, a depth, and a
+width. All of these dimensions can also be negative but a rule will
+not be visible unless its width is positive and its height plus depth
+is positive.
+
+As a specialty, rules can have ``running dimensions''\index{running dimension}. If any of the
+three dimensions is a running dimension, its actual value will be
+determined by running the rule up to the boundary of the innermost
+enclosing box. The width is never running in an horizontal\index{horizontal list} list; the
+height and depth are never running in a vertical\index{vertical list} list. In the long
+format, we use a vertical bar ``{\tt \VB}'' or a horizontal bar
+``{\tt \_}'' (underscore character) to indicate a running
+dimension. Of course the vertical bar is meant to indicate a running
+height or depth while the horizontal bar stands for a running
+width. The parser, however, makes no distinction between the two and
+you can use either of them. In the short format, we follow \TeX\ and
+implement a running dimension by using the special value
+$-2^{30}=|0xC0000000|$.
+
+
+@<hint macros@>=
+#define RUNNING_DIMEN 0xC0000000
+@
+
+It could have been possible to allow extended dimensions in a rule node,
+but in most circumstances, the mechanism of running dimensions is sufficient
+and simpler to use. If a rule is needed that requires an extended dimension as
+its length, it is always possible to put it inside a suitable box and use a
+running dimension.
+
+
+To make the short format encoding more compact, the first info bit
+|b100| will be zero to indicate a running height, bit |b010| will be
+zero to indicate a running depth, and bit |b001| will be zero to
+indicate a running width.
+
+Because leaders\index{leaders} (see section~\secref{leaders}) may contain a rule
+node, we also provide functions to read and write a complete rule
+node. While parsing the symbol ``{\sl rule\/}'' will just initialize a variable of type
+\&{rule\_t} (the writing is done with a separate routine),
+parsing a {\sl rule\_node\/} will always include writing it.
+
+% Currently no predefined rules.
+%Further, a {\sl rule\_node} will permit the
+%use of a predefined rule (see section~\secref{reference}),
+
+
+@<hint types@>=
+typedef struct {@+
+dimen_t h,d,w; @+
+} rule_t;
+@
+
+\readcode
+ at s RULE symbol
+ at s RUNNING symbol
+ at s rule_dimension symbol
+ at s rule symbol
+ at s rule_node symbol
+@<symbols@>=
+%token RULE "rule"
+%token RUNNING "|"
+%type <d> rule_dimension
+%type <r> rule
+@
+
+@<scanning rules@>=
+::@=rule@> :< return RULE; >:
+::@="|"@> :< return RUNNING; >:
+::@="_"@> :< return RUNNING; >:
+@
+
+@<parsing rules@>=
+rule_dimension: dimension at + | RUNNING {$$=RUNNING_DIMEN;};
+rule: rule_dimension rule_dimension rule_dimension @/
+ { $$.h=$1; @+ $$.d=$2; @+ $$.w=$3;
+ if ($3==RUNNING_DIMEN && ($1==RUNNING_DIMEN || $2==RUNNING_DIMEN))
+ QUIT("Incompatible running dimensions 0x%x 0x%x 0x%x",@|$1,$2,$3); };
+rule_node: start RULE rule END { hput_tags($1,hput_rule(&($3))); };
+content_node: rule_node;
+@
+
+\writecode
+@<write functions@>=
+static void hwrite_rule_dimension(dimen_t d, char c)
+{ @+if (d==RUNNING_DIMEN) hwritef(" %c",c);
+ else hwrite_dimension(d);
+}
+
+void hwrite_rule(rule_t *r)
+{ @+hwrite_rule_dimension(r->h,'|');
+ hwrite_rule_dimension(r->d,'|');
+ hwrite_rule_dimension(r->w,'_');
+}
+@
+\getcode
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(rule_kind,b011): {rule_t r;@+ HGET_RULE(b011,r); @+hwrite_rule(&(r));@+ } @+break;
+case TAG(rule_kind,b101): {rule_t r;@+ HGET_RULE(b101,r); @+hwrite_rule(&(r));@+ } @+break;
+case TAG(rule_kind,b001): {rule_t r;@+ HGET_RULE(b001,r); @+hwrite_rule(&(r));@+ } @+break;
+case TAG(rule_kind,b110): {rule_t r;@+ HGET_RULE(b110,r); @+hwrite_rule(&(r));@+ } @+break;
+case TAG(rule_kind,b111): {rule_t r;@+ HGET_RULE(b111,r); @+hwrite_rule(&(r));@+ } @+break;
+@
+
+@<get macros@>=
+#define @[HGET_RULE(I,R)@]@/\
+if ((I)&b100) HGET32((R).h); @+else (R).h=RUNNING_DIMEN;\
+if ((I)&b010) HGET32((R).d); @+else (R).d=RUNNING_DIMEN;\
+if ((I)&b001) HGET32((R).w); @+else (R).w=RUNNING_DIMEN;
+@
+
+@<get functions@>=
+void hget_rule_node(void)
+{ @<read the start byte |a|@>@;
+ if (KIND(a)==rule_kind) @/
+ { @+rule_t r; @+HGET_RULE(INFO(a),r); @/
+ hwrite_start();@+ hwritef("rule"); @+hwrite_rule(&r); @+hwrite_end();
+ }
+ else
+ QUIT("Rule expected at 0x%x got %s",node_pos,NAME(a));
+ @<read and check the end byte |z|@>@;
+}
+@
+
+\putcode
+@<put functions@>=
+uint8_t hput_rule(rule_t *r)
+{ info_t info=b000;
+ if (r->h!=RUNNING_DIMEN) { HPUT32(r->h); @+info|=b100; @+}
+ if (r->d!=RUNNING_DIMEN) { HPUT32(r->d); @+info|=b010; @+}
+ if (r->w!=RUNNING_DIMEN) { HPUT32(r->w); @+info|=b001; @+}
+ return TAG(rule_kind,info);
+}
+@
+
+
+\subsection{Kerns}
+A kern\index{kern} is a bit of white space with a certain length. If the kern is part of a
+horizontal list, the length is measured in the horizontal direction,
+if it is part of a vertical list, it is measured in the vertical
+direction. The length of a kern is mostly given as a dimension
+but provisions are made to use extended dimensions as well.
+
+The typical
+use of a kern is its insertion between two characters to make the natural
+distance between them a bit wider or smaller. In the latter case, the kern
+has a negative length. The typographic optimization just described is called
+``kerning'' and has given the kern node its name.
+Kerns inserted from font information or math mode calculations are normal kerns,
+while kerns inserted from \TeX's {\tt \BS kern} or {\tt \BS/}
+commands are explicit kerns.
+Kern nodes do not disappear at a line break unless they are explicit\index{explicit kern}.
+
+In the long format, explicit kerns are marked with an ``!'' sign
+and in the short format with the |b100| info bit.
+The two low order info bits are: 0 for a reference to a dimension, 1 for a reference to
+an extended dimension, 2 for an immediate dimension, and 3 for an immediate extended dimension node.
+To distinguish in the long format between a reference to a dimension and a reference to an extended dimension,
+the latter is prefixed with the keyword ``{\tt xdimen}'' (see section~\secref{reference}).
+
+@<hint types@>=
+typedef struct {@+
+bool x;@+
+xdimen_t d;@+
+} kern_t;
+@
+
+\readcode
+ at s KERN symbol
+ at s EXPLICIT symbol
+ at s kern symbol
+ at s explicit symbol
+@<symbols@>=
+%token KERN "kern"
+%token EXPLICIT "!"
+%type <b> explicit
+%type <kt> kern
+@
+
+@<scanning rules@>=
+::@=kern@> :< return KERN; >:
+::@=!@> :< return EXPLICIT; >:
+@
+
+@<parsing rules@>=
+explicit: {$$=false;} @+| EXPLICIT {$$=true;};
+kern: explicit xdimen {$$.x=$1; $$.d=$2;};
+content_node: start KERN kern END { hput_tags($1,hput_kern(&($3)));}
+@
+
+\writecode
+@<write functions@>=
+void hwrite_explicit(bool x)
+{ @+if (x) hwritef(" !"); @+}
+
+void hwrite_kern(kern_t *k)
+{ @+hwrite_explicit(k->x);
+ if (k->d.h==0.0 && k->d.v==0.0 && k->d.w==0) hwrite_ref(zero_dimen_no);
+ else hwrite_xdimen(&(k->d));
+}
+@
+
+
+\getcode
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(kern_kind,b010): @+ {@+kern_t k; @+HGET_KERN(b010,k);@+ } @+break;
+case TAG(kern_kind,b011): @+ {@+kern_t k; @+HGET_KERN(b011,k);@+ } @+break;
+case TAG(kern_kind,b110): @+ {@+kern_t k; @+HGET_KERN(b110,k);@+ } @+break;
+case TAG(kern_kind,b111): @+ {@+kern_t k; @+HGET_KERN(b111,k);@+ } @+break;
+@
+
+@<get macros@>=
+#define @[HGET_KERN(I,K)@] \
+K.x=(I)&b100;\
+if (((I)&b011)==2) {HGET32(K.d.w);@+ K.d.h=K.d.v=0.0;@+}\
+else if (((I)&b011)==3) hget_xdimen_node(&(K.d));\
+hwrite_kern(&k);
+@
+
+\putcode
+@<put functions@>=
+uint8_t hput_kern(kern_t *k)
+{ info_t info;
+ if (k->x) info=b100; @+else info=b000;
+ if (k->d.h==0.0 && k->d.v==0.0)
+ { if (k->d.w==0) HPUT8(zero_dimen_no);
+ else {HPUT32(k->d.w); info=info|2;@+}
+ }
+ else {hput_xdimen_node(&(k->d));info=info|3;@+}
+ return TAG(kern_kind,info);
+}
+@
+
+
+
+\subsection{Glue}\label{glue}
+
+%Glue considerations
+
+%So what are the cases:
+%\itemize
+%\item reference to a dimen (common)
+%\item reference to a xdimen
+%\item reference to a dimen plus and minus
+%\item reference to a xdimen plus and minus
+%\item reference to a dimen plus
+%\item reference to a xdimen plus
+%\item reference to a dimen minus
+%\item reference to a xdimen minus
+%\item dimen
+%\item xdimen
+%\item dimen plus and minus
+%\item xdimen plus and minus (covers all other cases)
+%\item dimen plus
+%\item xdimen plus
+%\item dimen minus
+%\item xdimen minus
+%\item plus and minus
+%\item plus
+%\item minus
+%\item zero glue (rare, can be replaced by a reference to the zero glue)
+%\item reference to a predefined glue (common)
+%\enditemize
+%This is a total of 21 cases. Can we use the info bits to specify 7 common
+%cases and one catch all? First the use of an extended dimension in a glue
+%is probably not very common. More typically is the use of a fill glue
+%that extends to the boundaries of the enclosing box.
+
+%Here is the statistics for ctex:
+%total 58937 glue entries
+%total 49 defined glues (so 200 still available)
+%There are three font specific glues defined for each font used in texts.
+%The explicit glue nodes are the following:
+%\itemize
+%\item 35\% is predefined zero glue
+%\item 30\% are 39 other predefined glue most of them less than 1%
+%\item 8\% (4839) is one glue with 25pt pure stretch with order 0
+%\item 25\% (14746) is one glue with 100pt stretch and 10pt shrink with order 0
+%\item 2\% (1096) is one glue with 10pt no stretch and shrink
+%\item 0\% (13) are 7 different glues with no stretch and shrink
+%\item 0\% (3) different glues with width!=0 and some stretch of order 0
+%\item 0\% (27) 20 different glues with stretch and shrink
+%\enditemize
+
+%Some more glue with 1fil is insider 55 leaders
+%one vset has an extent 1 no stretch and shrink
+%56 hset specify an extent 2 and 1 fil stretch
+
+
+We have seen in section~\secref{stretch} how to deal with
+stretchability\index{stretchability} and
+shrinkability\index{shrinkability} and we will need this now.
+Glue\index{glue} has a natural width---which in general can be an
+extended dimension---and in addition it can stretch and shrink. It
+might have been possible to allow an extended dimension also for the
+stretch\-ability or shrink\-ability of a glue, but this seems of
+little practical relevance and so simplicity won over generality.
+Even with that restriction, it is an understatement to regard glue
+nodes as "simple" nodes.
+%, and we could equally well list them in
+%section~\secref{composite} as composite nodes.
+
+To use the info bits in the short format wisely, I collected some
+statistical data using the \TeX book as an example. It turns out that
+about 99\% of all the 58937 glue nodes (not counting the interword
+glues used inside texts) could be covered with only 43 predefined
+glues. So this is by far the most common case; we reserve the info
+value |b000| to cover it and postpone the description of such glue
+nodes until we describe references in section~\secref{reference}.
+
+We expect the remaining cases to contribute not too much to the file
+size, and hence, simplicity is a more important aspect than efficiency
+when allocating the remaining info values.
+
+Looking at the glues in more detail, we find that the most common
+cases are those where either one, two, or all three glue components
+are zero. We use the two lowest bits to indicate the presence of a
+nonzero stretchability or shrinkability and reserve the info values
+|b001|, |b010|, and |b011| for those cases where the width of the glue
+is zero. The zero glue, where all components are zero, is defined as
+a fixed, predefined glue instead of reserving a special info value for
+it. The cost of one extra byte when encoding it seems not too high a
+price to pay. After reserving the info value |b111| for the most
+general case of a glue, we have only three more info values left:
+|b100|, |b101|, and |b110|. Keeping things simple implies using the
+two lowest info bits---as before---to indicate a nonzero
+stretchability or shrinkability. For the width, three choices remain:
+using a reference to a dimension, using a reference to an extended
+dimension, or using an immediate value. Since references to glues are
+already supported, an immediate width seems best for glues that are
+not frequently reused, avoiding the overhead of references.
+
+% It also makes parsing simpler because we avoid the confusion
+% between references to dimensions
+% and references to glues and references to extended dimensions.
+
+Here is a summary of the info bits and the implied layout
+of glue nodes in the short format:
+\itemize
+\item |b000|: reference to a predefined glue
+\item |b001|: zero width and nonzero shrinkability
+\item |b010|: zero width and nonzero stretchability
+\item |b011|: zero width and nonzero stretchability and shrinkability
+\item |b100|: nonzero width
+\item |b101|: nonzero width and nonzero shrinkability
+\item |b110|: nonzero width and nonzero stretchability
+\item |b111|: extended dimension and nonzero stretchability and shrinkability
+\enditemize
+
+
+@<hint basic types@>=
+typedef struct {@+
+xdimen_t w; @+
+stretch_t p, m;@+
+} glue_t;
+@
+
+
+To test for a zero glue,
+we implement a macro:
+@<hint macros@>=
+#define @[ZERO_GLUE(G)@] ((G).w.w==0 && (G).w.h==0.0 && (G).w.v==0.0 && (G).p.f==0.0 && (G).m.f==0.0)
+@
+
+Because other nodes (leaders, baselines, and fonts)
+contain glue nodes as parameters, we provide functions
+to read and write a complete glue node in the same way as we did
+for rule nodes.
+Further, such an internal {\sl glue\_node\/} has the special property that
+in the short format a node for the zero glue might be omitted entirely.
+
+\readcode
+ at s GLUE symbol
+ at s glue symbol
+ at s glue_node symbol
+ at s PLUS symbol
+ at s MINUS symbol
+ at s plus symbol
+ at s minus symbol
+
+@<symbols@>=
+%token GLUE "glue"
+%token PLUS "plus"
+%token MINUS "minus"
+%type <g> glue
+%type <b> glue_node
+%type <st> plus minus
+@
+
+@<scanning rules@>=
+::@=glue@> :< return GLUE; >:
+::@=plus@> :< return PLUS; >:
+::@=minus@> :< return MINUS; >:
+@
+
+@<parsing rules@>=
+plus: { $$.f=0.0; $$.o=0; } | PLUS stretch {$$=$2;};
+minus: { $$.f=0.0; $$.o=0; } | MINUS stretch {$$=$2;};
+glue: xdimen plus minus {$$.w=$1; $$.p=$2; $$.m=$3; };
+content_node: start GLUE glue END {if (ZERO_GLUE($3)) {HPUT8(zero_skip_no);
+ hput_tags($1,TAG(glue_kind,0)); } else hput_tags($1,hput_glue(&($3))); };
+glue_node: start GLUE glue END @/
+ {@+ if (ZERO_GLUE($3)) { hpos--; $$=false;@+}@/
+ else { hput_tags($1,hput_glue(&($3))); $$=true;@+}@+ };
+@
+
+\writecode
+@<write functions@>=
+void hwrite_plus(stretch_t *p)
+{ @+if (p->f!=0.0) { hwritef(" plus");@+hwrite_stretch(p); @+}
+}
+void hwrite_minus(stretch_t *m)
+{@+ if (m->f!=0.0) { hwritef(" minus");@+hwrite_stretch(m); @+}
+}
+
+void hwrite_glue(glue_t *g)
+{ hwrite_xdimen(&(g->w)); @+
+ hwrite_plus(&g->p); @+hwrite_minus(&g->m);
+}
+
+void hwrite_ref_node(kind_t k, uint8_t n);
+void hwrite_glue_node(glue_t *g)
+{@+
+ if (ZERO_GLUE(*g)) hwrite_ref_node(glue_kind,zero_skip_no);
+ else @+{ hwrite_start(); @+hwritef("glue"); @+hwrite_glue(g); @+hwrite_end();@+}
+}
+@
+
+\getcode
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(glue_kind,b001): { glue_t g;@+ HGET_GLUE(b001,g);@+ hwrite_glue(&g);@+}@+break;
+case TAG(glue_kind,b010): { glue_t g;@+ HGET_GLUE(b010,g);@+ hwrite_glue(&g);@+}@+break;
+case TAG(glue_kind,b011): { glue_t g;@+ HGET_GLUE(b011,g);@+ hwrite_glue(&g);@+}@+break;
+case TAG(glue_kind,b100): { glue_t g;@+ HGET_GLUE(b100,g);@+ hwrite_glue(&g);@+}@+break;
+case TAG(glue_kind,b101): { glue_t g;@+ HGET_GLUE(b101,g);@+ hwrite_glue(&g);@+}@+break;
+case TAG(glue_kind,b110): { glue_t g;@+ HGET_GLUE(b110,g);@+ hwrite_glue(&g);@+}@+break;
+case TAG(glue_kind,b111): { glue_t g;@+ HGET_GLUE(b111,g);@+ hwrite_glue(&g);@+}@+break;
+@
+
+@<get macros@>=
+#define @[HGET_GLUE(I,G)@] {\
+ if((I)!=b111) { if ((I)&b100) HGET32((G).w.w);@+ else (G).w.w=0;}\
+ if((I)&b010) HGET_STRETCH((G).p) @+else (G).p.f=0.0, (G).p.o=0;\
+ if((I)&b001) HGET_STRETCH((G).m) @+else (G).m.f=0.0, (G).m.o=0;\
+ if((I)==b111) hget_xdimen_node(&((G).w)); else (G).w.h=(G).w.v=0.0;@+}
+@
+
+The |hget_glue_node| can cope with a glue node that is omitted and
+will supply a zero glue instead.
+
+@<get functions@>=
+void hget_glue_node(void)
+{ @<read the start byte |a|@>@;
+ if (KIND(a)!=glue_kind)
+ {@+ hpos--; hwrite_ref_node(glue_kind,zero_skip_no);@+return; @+}
+ if (INFO(a)==b000)
+ { uint8_t n=HGET8;@+ REF(glue_kind,n);@+hwrite_ref_node(glue_kind,n); @+}
+ else
+ { @+glue_t g; @+HGET_GLUE(INFO(a),g);@+ hwrite_glue_node(&g);@+}
+ @<read and check the end byte |z|@>@;
+}
+@
+
+
+\putcode
+@<put functions@>=
+uint8_t hput_glue(glue_t *g)
+{ info_t info=b000;
+ if (ZERO_GLUE(*g)) { HPUT8(zero_skip_no); @+ info=b000; }
+ else if ( (g->w.w==0 && g->w.h==0.0 && g->w.v==0.0))
+ { if (g->p.f!=0.0) { hput_stretch(&g->p); @+info|=b010; @+}
+ if (g->m.f!=0.0) { hput_stretch(&g->m); @+info|=b001; @+}
+ }
+ else if ( g->w.h==0.0 && g->w.v==0.0 && (g->p.f==0.0 || g->m.f==0.0))
+ { HPUT32(g->w.w); @+ info=b100;
+ if (g->p.f!=0.0) { hput_stretch(&g->p); @+info|=b010; @+}
+ if (g->m.f!=0.0) { hput_stretch(&g->m); @+info|=b001; @+}
+ }
+ else@/
+ { hput_stretch(&g->p);@+ hput_stretch(&g->m);
+ hput_xdimen_node(&(g->w));
+ info=b111;
+ }
+ return TAG(glue_kind,info);
+}
+@
+
+\section{Lists}\hascode\label{lists}
+When a node contains multiple other nodes, we package these nodes into
+a list\index{list} node. It is important to note that list nodes
+never occur as individual nodes, they only occur as parts of other
+nodes. In total, we have three different types of lists: plain lists
+that use the kind-value |list_kind|, text\index{text} lists that use
+the kind-value |text_kind|, and parameter\index{parameter} lists that use the
+kind-value |param_kind|. A description of the first two types of
+lists follows here. Parameter lists are described in section~\secref{paramlist}.
+
+Because lists are of variable size, it is not possible in the short
+format to tell from the kind and info bits of a tag byte the size of
+the list node. So advancing from the beginning of a list node to the
+next node after the list is not as simple as usual. To solve this
+problem, we store the size of the list immediately after the start
+byte and before the end byte. Alternatively we could require programs
+to traverse the entire list. The latter solution is more compact but
+inefficient for list with many nodes; our solution will cost some
+extra bytes, but the amount of extra bytes will only grow
+logarithmically with the size of the \HINT\ file. It would be
+possible to allow both methods so that a \HINT\ file could balance
+size and time trade-offs by making small lists---where the size can be
+determined easily by reading the entire list---without size
+information and making large lists with size information so that they
+can be skipped easily without reading them. But the added complexity
+seems too high a price to pay.
+
+
+Now consider the problem of reading a content stream starting at an arbitrary
+position $i$ in the middle of the stream. This situation occurs
+naturally when resynchronizing\index{resynchronization} a content stream after
+an error has been detected, but implementing links poses a similar problem.
+We can inspect the byte at position $i$ and see
+if it is a valid tag. If yes, we are faced with the problem of
+verifying that this is not a mere coincidence.
+So we determine the size $s$ of the node. If the byte in question is a start byte,
+we should find a matching byte $s$ bytes later in the stream; if it is an end byte,
+we should find the matching byte $s$ bytes earlier in the stream; if we
+find no matching byte, this was neither a start nor an end byte.
+If we find exactly one matching byte, we can be quite confident (error
+probability 1/256 if assuming equal probability of all byte values)
+that we have found a tag, and we know whether
+it is the beginning or the end tag. If we find two matching byte, we
+have most likely the start or the end of a node, but we do not know which
+of the two. To find out which of the two possibilities is true
+or to reduce the probability of an error, we can
+check the start and end byte of the node immediately preceding a start byte or
+immediately following an end byte in a similar way.
+By testing two more byte, this additional check will reduce the error
+probability further to $1/2^{24}$ (under the same assumption as before). So
+checking more nodes is rarely necessary. This whole schema
+would, however, not work if we happen to find a tag byte that indicated
+either the begin or the end of a list without specifying the size
+of the list. Sure, we can verify the bytes before and after it to
+find out whether the byte following it is the begin of a node and the
+byte preceding it is the end of a node, but we still don't know if the
+byte itself starts a node list or ends a node list. Even reading along
+in either direction until finding a matching tag will not answer the
+question. The situation is better if we specify a
+size: we can read the suspected size after or before the tag and check if we
+find a matching tag and size at the position indicated.
+In the short format, we use the |info| value to indicate the number of
+byte used to store the list size: A list with $0<|info|\le 5$
+uses $|info|-1$ byte to store the size.
+The info value zero is reserved for references to predefined lists
+(which are currently not implemented).
+
+Storing the list size immediately preceding the end tag creates a new
+problem: If we try to recover from an error, we might not know the
+size of the list and searching for the end of a list, we might be
+unable to tell the difference between the bytes that encode the list
+size and the start tag of a possible next node. If we parse the
+content backward, the problem is completely symmetric.
+
+To solve the problem, we insert an additional byte immediately before
+the final size and after the initial size marking the size boundary.
+We choose the byte values |0xFF|, |0xFE|, |0xFD|, and |0xFC| which can
+not be confused with valid tag bytes and indicate that the size is
+stored using 1, 2, 3, or 4 byte respectively. Under regular
+circumstances, these bytes are simply skipped. When searching for the
+list end (or start) these bytes would correspond to
+|TAG(penalty_kind,i)| with $7 \ge \hbox{|i|} \ge 4$ and can not be
+confused with valid penalty nodes which use only the info values 0, 1,
+and~2. An empty list uses the info value 1 and has neither a size bytes
+nor boundary bytes; it consists only of the two tags.
+
+We are a bit lazy when it comes to the internal representation of a list.
+Since we need the representation as a short format byte sequence anyway,
+it consists of the position |p| of the start of the byte sequence
+combined with an integer |s| giving the size of the byte sequence.
+If the list is empty, |s| is zero.
+
+@<hint types@>=
+typedef struct {@+
+kind_t k; @+
+uint32_t p;@+
+uint32_t s;@+
+} list_t;
+@
+
+The major drawback of this choice of representation is that it ties
+together the reading of the long format and the writing of the short
+format; these are no longer independent.
+So starting with the present section, we have to take the short format
+representation of a node into account already when we parse the long
+format representation.
+
+In the long format, we may start a list node with an
+estimate\index{estimate} of the size needed to store the list in the
+short format. We do not want to require the exact size because this
+would make editing of long format \HINT\ files almost impossible. Of
+course this makes it also impossible to derive the exact |s| value of
+the internal representation from the long format
+representation. Therefore we start by parsing the estimate of the list
+size and use it to reserve the necessary number of byte to store the
+size. Then we parse the |content_list|. As a side effect---and this
+is an important point---this will write the list content in short
+format into the output buffer. As mentioned above, whenever a node
+contains a list, we need to consider this side effect when we give the
+parsing rules. We will see examples for this in
+section~\secref{composite}.
+
+The function |hput_list| will be called {\it after} the short format
+of the list is written to the output. Before we pass the internal
+representation of the list to the |hput_list|
+function, we update |s| and |p|. Further, we pass the position in the stream where the
+list size and its boundary mark is supposed to be.
+Before |hput_list| is called, space for the tag, the size, and the boundary mark
+is allocated based on the estimate. The function
+|hsize_bytes| computes the number of byte required to store the list
+size, and the function |hput_list_size| will later write the list
+size. If the estimate turns out to be wrong, the list data can be moved
+to make room for a larger or smaller size field.
+
+
+If the long format does not specify a size estimate, a suitable default must be chosen.
+A statistical analysis shows
+%
+%statistics about list sizes using my old prototype
+%
+%name type size_byte list_count total_size
+%hello.hnt text 1 6 748
+% text 2 2 1967
+% list 1 65 3245
+% list 2 1 352
+%web2w.hnt text 1 1043 121925
+% text 2 1344 859070
+% list 1 19780 725514
+% list 2 487 199243
+%ctex.hnt text 1 9121 4241128
+% text 2 12329 7872687
+% text 3 1 75010
+% list 1 121557 4600743
+% list 2 222 147358
+%
+that most plain lists need only a single byte to store the size; and even the
+total amount of data contained in these lists exceeds the amount of data stored
+in longer lists by a factor of about 3. Hence if we do not have an estimate,
+we reserve only a single byte to store the size of a list.
+The statistics looks different for lists stored as a text: The number of texts
+that require two byte for the size is slightly larger than the number of texts that
+need only one byte, and the total amount of data stored in these texts is larger
+by a factor of 2 to 7 than the total amount of data found in all other texts.
+Hence as a default, we reserve two byte to store the size for texts.
+
+
+\subsection{Plain Lists}\label{plainlists}
+Plain list nodes start and end with a tag of kind |list_kind|.
+
+Not uncommon are empty\index{empty list} lists; these are the only lists that can be
+stored using $|info|=1$; such a list has zero bytes of size
+information, and no boundary bytes either; implicitly its size is zero.
+The |info| value 0 is not used since we do not use predefined plain lists.
+
+Writing the long format uses the fact that the function
+|hget_content_node|, as implemented in the \.{stretch} program, will
+output the node in the long format.
+
+\readcode
+ at s list symbol
+ at s content_list symbol
+ at s estimate symbol
+ at s position symbol
+
+@<symbols@>=
+%type <l> list
+%type <u> position content_list
+@
+
+@<parsing rules@>=
+position: {$$=hpos-hstart;};
+content_list: @+ position @+
+ | content_list content_node;
+estimate: {hpos+=2; } @+
+ | UNSIGNED {hpos+=hsize_bytes($1)+1; } ;
+list: start estimate content_list END @/
+ {@+$$.k=list_kind;@+ $$.p=$3; @+ $$.s=(hpos-hstart)-$3;
+ hput_tags($1,hput_list($1+1, &($$)));@+};
+@
+
+\writecode
+@<write functions@>=
+void hwrite_list(list_t *l)
+{ uint32_t h=hpos-hstart, e=hend-hstart; /* save |hpos| and |hend| */
+ hpos=l->p+hstart;@+ hend=hpos+l->s;
+ if (l->k==list_kind ) @<write a list@>@;
+ else if (l->k==text_kind) @<write a text@>@;
+ else QUIT("List expected got %s", content_name[l->k]);
+ hpos=hstart+h;@+ hend=hstart+e; /* restore |hpos| and |hend| */
+}
+@
+
+@<write a list@>=
+{@+if (l->s==0) hwritef(" <>");@/
+ else@/
+ {@+DBG(DBGNODE,"Write list at 0x%x size=%u\n", l->p, l->s);
+ @+hwrite_start();@+
+ if (section_no==2) hwrite_label();
+ if (l->s>0xFF) hwritef("%d",l->s);
+ while(hpos<hend)
+ hget_content_node();
+ hwrite_end();
+ }
+}
+@
+\getcode
+@<shared get functions@>=
+void hget_size_boundary(info_t info)
+{ uint32_t n;
+ if (info<2) return;
+ n=HGET8;
+ if (n-1!=0x100-info) QUIT(@["Size boundary byte 0x%x with info value %d at " SIZE_F@],
+ n, info,hpos-hstart-1);
+}
+
+uint32_t hget_list_size(info_t info)
+{ uint32_t n=0;
+ if (info==1) return 0;
+ else if (info==2) n=HGET8;
+ else if (info==3) HGET16(n);
+ else if (info==4) HGET24(n);
+ else if (info==5) HGET32(n);
+ else QUIT("List info %d must be 1, 2, 3, 4, or 5",info);
+ return n;
+}
+
+void hget_list(list_t *l)
+{@+if (KIND(*hpos)!=list_kind && @/
+ KIND(*hpos)!=text_kind &&@| KIND(*hpos)!=param_kind) @/
+ QUIT("List expected at 0x%x", (uint32_t)(hpos-hstart));
+ else
+ {
+ @<read the start byte |a|@>@;
+ l->k=KIND(a);
+ HGET_LIST(INFO(a),*l);
+ @<read and check the end byte |z|@>@;
+ DBG(DBGNODE,"Get list at 0x%x size=%u\n", l->p, l->s);
+ }
+}
+@
+
+@<shared get macros@>=
+#define @[HGET_LIST(I,L)@] \
+ (L).s=hget_list_size(I); hget_size_boundary(I);\
+ (L).p=hpos-hstart; \
+ hpos=hpos+(L).s; hget_size_boundary(I);\
+ { uint32_t s=hget_list_size(I); \
+ if (s!=(L).s) \
+ QUIT(@["List sizes at 0x%x and " SIZE_F " do not match 0x%x != 0x%x"@],node_pos+1,hpos-hstart-I-1,(L).s,s);}
+@
+
+\putcode
+
+@<put functions@>=
+uint8_t hsize_bytes(uint32_t n)
+{ @+if (n==0) return 0;
+ else if (n<0x100) return 1;
+ else if (n<0x10000) return 2;
+ else if (n<0x1000000) return 3;
+ else return 4;
+}
+
+void hput_list_size(uint32_t n, int i)
+{ @+if (i==0) ;
+ else if (i==1) HPUT8(n);
+ else if (i==2) HPUT16(n);
+ else if (i==3) HPUT24(n);
+ else HPUT32(n);
+}
+
+uint8_t hput_list(uint32_t start_pos, list_t *l)
+{ @+if (l->s==0)
+ { hpos=hstart+start_pos; return TAG(l->k,1);@+}
+ else@/
+ { uint32_t list_end=hpos-hstart;
+ int i=l->p -start_pos-1; /* number of byte allocated for size */
+ int j=hsize_bytes(l->s); /* number of byte needed for size */
+ DBG(DBGNODE,"Put list at 0x%x size=%u\n", l->p, l->s);
+ if (i>j && l->s> 0x100) j=i; /* avoid moving large lists */
+ if (i!=j)@/
+ { int d= j-i;
+ DBG(DBGNODE,"Moving %u byte by %d\n", l->s,d);
+ if (d>0) HPUTX(d);
+ memmove(hstart+l->p+d,hstart+l->p,l->s);
+ @<adjust label positions after moving a list@>@;
+ l->p=l->p+d;@+
+ list_end=list_end+d;
+ }
+ hpos=hstart+start_pos; @+ hput_list_size(l->s,j);@+ HPUT8(0x100-j);
+ hpos=hstart+list_end;@+ HPUT8(0x100-j);@+ hput_list_size(l->s,j);
+ return TAG(l->k,j+1);
+ }
+}
+
+@
+
+
+
+\subsection{Texts}\label{text}
+A Text\index{text} is a list of nodes with a representation optimized
+for character nodes. In the long format, a sequence of characters
+like ``{\tt Hello}'' is written ``\.{<glyph 'H'} \.{*0>} \.{<glyph} \.{'e'}
+\.{*0>} \.{<glyph 'l' *0>} \.{<glyph 'l' *0>} \.{<glyph 'o' *0>}'', and
+even in the short format it requires 4 byte per character! As a text,
+the same sequence is written ``{\tt\,"Hello"\,}'' in the long format and the
+short format requires usually just 1 byte per character. Indeed
+except the bytes with values from |0x00| to |0x20|, which are
+considered control\index{control code} codes, all bytes and all
+\hbox{UTF-8}\index{UTF8} multibyte sequences are simply considered
+character\index{character code} codes. They are equivalent to a glyph
+node in the ``current font''. The current\index{current font}
+font\index{font} is font number 0 at the beginning of a text and it
+can be changed using the control codes. We introduce the concept of a
+``current font'' because we do not expect the font to change too
+often, and it allows for a more compact representation if we do not
+store the font with every character code. It has an important
+disadvantage though: storing only font changes prevents us from
+parsing a text backwards; we always have to start at the beginning of
+the text, where the font is known to be font number~0.
+
+Defining a second format for encoding lists of nodes adds another
+difficulty to the problem we had discussed at the beginning of
+section~\secref{lists}. When we try to recover from an error and start
+reading a content stream at an arbitrary position, the first thing we
+need to find out is whether at this position we have the tag byte of
+an ordinary node or whether we have a position inside a text.
+
+Inside a text, character nodes start with a byte in the range
+|0x21|--|0xF7|. This is a wide range and it overlaps considerably with
+the range of valid tag bytes. It is however possible to choose the
+kind-values in such a way that the control codes do not overlap with
+the valid tag bytes that start a node. For this reason, the values
+|text_kind==0|, |list_kind==1|, |param_kind==2|, |xdimen_kind==3|, and
+|adjust_kind==4| were chosen on page~\pageref{kinddef}. Texts, lists,
+parameter lists, and extended dimensions occur only {\it inside} of
+content nodes, but are not content nodes in their own right; so the
+values |0x00| to |0x1F| are not used as tag bytes of content
+nodes. The value |0x20| would, as a tag byte, indicate an adjust node
+(|adjust_kind==4|) with info value zero. Because there are no
+predefined adjustments, |0x20| is not used as a tag byte either.
+(An alternative choice would be to use the kind value 4 for paragraph
+nodes because there are no predefined paragraphs.)
+
+The largest byte that starts an UTF8 code is |0xF7|; hence, there are
+eight possible control codes, from |0xF8| to |0xFF|, available. The
+first three values |0xF8|, |0xF9|, and |0xFA| are actually used for
+penalty nodes with info values, 0, 1, and 2. The last four |0xFC|,
+|0xFD|, |0xFE|, and |0xFF| are used as boundary marks for the text
+size and therefore we use only |0xFB| as control code.
+
+In the long format, we do not provide a syntax for specifying a size
+estimate\index{estimate} as we did for plain lists, because we expect
+text to be quite short. We allocate two byte for the size and hope
+that this will prove to be sufficient most of the time. Further, we
+will disallow the use of non-printable ASCII codes, because these
+are---by definition---not very readable, and we will give special
+meaning to some of the printable ASCII codes because we will need a
+notation for the beginning and ending of a text, for nodes inside a
+text, and the control codes.
+
+Here are the details:
+\itemize
+
+\item In the long format, a text starts and ends with a
+double\index{double quote} quote character ``{\tt "}''. In the short
+format, texts are encoded similar to lists using the kind-value
+|text_kind|.
+
+\item Arbitrary nodes can be embedded inside a text. In the long
+format, they are enclosed in pointed brackets \.{<} \dots \.{>} as
+usual. In the short format, an arbitrary node can follow the control
+code $|txt_node|=|0x1E|$. Because text may occur in nodes, the scanner
+needs to be able to parse texts nested inside nodes nested inside
+nodes nested inside texts \dots\ To accomplish this, we use the
+``stack'' option of \.{flex} and include the pushing and popping of the
+stack in the macros |SCAN_START| and |SCAN_END|.
+
+\item The space\index{space character} character ``\.{\ }'' with ASCII
+value |0x20| stands in both formats for the font specific interword
+glue node (control code |txt_glue|).
+
+\item The hyphen\index{hyphen character} character ``\.{-}'' in the
+long format and the control code $|txt_hyphen|=|0x1F|$ in the short
+format stand for the font specific discretionary hyphenation node.
+
+\item In the long format, the backslash\index{backslash} character
+``\.{\\}'' is used as an escape character. It is used to introduce
+notations for control codes, as described below, and to access the
+character codes of those ASCII characters that otherwise carry a
+special meaning. For example ``{\tt \BS "}'' denotes the character code
+of the double quote character ``{\tt "}''; and similarly ``\.{\\\\}'',
+``\.{\\<}'', ``\.{\\>}'', ``\.{\\\ }'', and ``\.{\\-}'' denote the
+character codes of ``\.{\\}'', ``\.{<}'', ``\.{>}'', ``\.{\ }'', and
+``\.{-}'' respectively.
+
+
+\item In the long format, a TAB-character (ASCII code
+|0x09|)\index{tab character} is silently converted to a
+space\index{space character} character (ASCII code |0x20|);
+a NL-character\index{newline character} (ASCII code |0x0A|), together
+with surrounding spaces, TAB-characters,
+and CR-characters\index{carriage return character} (ASCII code |0x0D|),
+is silently converted to a single space character. All other ASCII
+characters in the range |0x00| to |0x1F| are not allowed inside a
+text. This rule avoids the problems arising from ``invisible''
+characters embedded in a text and it allows to break texts into lines,
+even with indentation\index{indentation}, at word boundaries.
+
+To allow breaking a text into lines without inserting spaces, a
+NL-character together with surrounding spaces, TAB-characters, and
+CR-characters is completely ignored if the whole group of spaces,
+TAB-characters, CR-characters, and the NL-character is preceded by a
+backslash character.
+
+For example, the text ``\.{"There\ is\ no\ more\ gas\ in\ the\
+tank."}''\hfil\break can be written as \medskip
+
+\qquad\vbox{\hsize=0.5\hsize\noindent
+\.{"There\ is\ }\hfil\break
+\.{\hbox to 2em {$\rightarrow$\hfill}no more g\\\ \ }\hfil\break
+\.{\hbox to 2em {$\rightarrow$\hfill}as in the tank."}
+}\hss
+
+To break long lines when writing a long format file, we use the
+variable |txt_length| to keep track of the approximate length of the
+current line.
+
+\item The control codes $|txt_font|=|0x00|$, |0x01|, |0x02|, \dots,
+and |0x07| are used to change the current font to font
+number 0, 1, 2, \dots, and 7. In the long format these control
+codes are written \.{\\0}, \.{\\1}, \.{\\2}, \dots, and \.{\\7}.
+
+\item The control code $|txt_global|=|0x08|$ is followed by a second
+parameter byte. If the value of the parameter byte is $n$, it will set
+the current font to font number $n$. In the long format, the two byte
+sequence is written ``\.{\\F}$n$\.{\\}'' where $n$ is the decimal
+representation of the font number.
+
+
+\item The control codes |0x09|, |0x0A|, |0x0B|, |0x0C|, |0x0E|,
+|0x0E|, |0x0F|, and |0x10| are also followed by a second parameter
+byte. They are used to reference the global definitions of
+penalty\index{penalty}, kern\index{kern}, ligature\index{ligature},
+disc\index{discretionary hyphen}, glue\index{glue}, language\index{language},
+rule\index{rule}, and image\index{image} nodes. The parameter byte
+contains the reference number. For example, the byte sequence |0x09|
+|0x03| is equivalent to the node \.{<penalty *3>}.
+In the long format these two-byte sequences are written,
+``\.{\\P}$n$\.{\\}'' (penalty),
+``\.{\\K}$n$\.{\\}'' (kern),
+``\.{\\L}$n$\.{\\}'' (ligature),
+``\.{\\D}$n$\.{\\}'' (disc),
+``\.{\\G}$n$\.{\\}'' (glue),
+``\.{\\S}$n$\.{\\}'' (speak or German ``Sprache''),
+``\.{\\R}$n$\.{\\}'' (rule), and
+``\.{\\I}$n$\.{\\}'' (image), where $n$ is the decimal representation
+ of the parameter value.
+
+
+\item The control codes from $|txt_local|=|0x11|$ to |0x1C| are used
+to reference one of the 12 font specific parameters\index{font
+parameter}. In the long format they are written ``\.{\\a}'',
+``\.{\\b}'', ``\.{\\c}'', \dots, ``\.{\\j}'', ``\.{\\k}'',``\.{\\l}''.
+
+
+\item The control code $|txt_cc|=|0x1D|$ is used as a prefix for an
+arbitrary character code represented as an UTF-8 multibyte sequence.
+Its main purpose is providing a method for including character codes
+less or equal to |0x20| which otherwise would be considered control
+codes. In the long format, the byte sequence is written ``\.{\\C}$n$\.{\\}''
+where $n$ is the decimal representation of the character code.
+
+
+\item The control code $|txt_node|=|0x1E|$ is used as a prefix for an
+arbitrary node in short format. In the long format, it is written
+``\.{<}'' and is followed by the node content in long format
+terminated by ``\.{>}''.
+
+\item The control code $|txt_hyphen|=|0x1F|$ is used to access the
+font specific discretionary hyphen\index{hyphen}. In the long format
+it is simply written as ``\.{-}''.
+
+\item The control code $|txt_glue|=|0x20|$ is the space character, it
+is used to access the font specific interword\index{interword glue}
+glue. In the long format, we use the space character\index{space
+character} ``\.{\ }'' as well.
+
+\item The control code $|txt_ignore|=|0xFB|$ is ignored, its position
+can be used in a link to specify a position between two characters. In
+the long format it is written as ``\.{\\@@}''.
+
+\enditemize
+For the control codes, we define an enumeration type
+and for references, a reference type.
+@<hint types@>=
+typedef enum { @+txt_font=0x00, txt_global=0x08, txt_local=0x11,
+ txt_cc=0x1D, txt_node=0x1E, txt_hyphen=0x1F,
+ txt_glue=0x20, txt_ignore=0xFB} txt_t;
+@
+
+\readcode
+ at s TXT symbol
+ at s TXT_START symbol
+ at s TXT_END symbol
+ at s TXT_FONT symbol
+ at s TXT_LOCAL symbol
+ at s TXT_GLOBAL symbol
+ at s TXT_FONT_GLUE symbol
+ at s TXT_FONT_HYPHEN symbol
+ at s TXT_CC symbol
+ at s TXT_IGNORE symbol
+ at s text symbol
+@<scanning definitions@>=
+%x TXT
+@
+
+@<symbols@>=
+%token TXT_START TXT_END TXT_IGNORE
+%token TXT_FONT_GLUE TXT_FONT_HYPHEN
+%token <u> TXT_FONT TXT_LOCAL
+%token <rf> TXT_GLOBAL
+%token <u> TXT_CC
+%type <u> text
+@
+
+@<scanning rules@>=
+::@=\"@> :< SCAN_TXT_START; return TXT_START; >:
+
+<TXT>{
+::@=\"@> :< SCAN_TXT_END; return TXT_END; >:
+
+::@="<"@> :< SCAN_START; return START; >:
+::@=">"@> :< QUIT("> not allowed in text mode");>:
+
+::@=\\\\@> :< yylval.u='\\'; return TXT_CC; >:
+::@=\\\"@> :< yylval.u='"'; return TXT_CC; >:
+::@=\\"<"@> :< yylval.u='<'; return TXT_CC; >:
+::@=\\">"@> :< yylval.u='>'; return TXT_CC; >:
+::@=\\" "@> :< yylval.u=' '; return TXT_CC; >:
+::@=\\"-"@> :< yylval.u='-'; return TXT_CC; >:
+::@=\\"@@"@> :< return TXT_IGNORE; >:
+
+::@=[ \t\r]*(\n[ \t\r]*)+@> :< return TXT_FONT_GLUE; >:
+::@=\\[ \t\r]*\n[ \t\r]*@> :< ; >:
+
+::@=\\[0-7]@> :< yylval.u=yytext[1]-'0'; return TXT_FONT; >:
+
+::@=\\F[0-9]+\\@> :< SCAN_REF(font_kind); return TXT_GLOBAL; >:
+::@=\\P[0-9]+\\@> :< SCAN_REF(penalty_kind); return TXT_GLOBAL; >:
+::@=\\K[0-9]+\\@> :< SCAN_REF(kern_kind); return TXT_GLOBAL; >:
+::@=\\L[0-9]+\\@> :< SCAN_REF(ligature_kind); return TXT_GLOBAL; >:
+::@=\\D[0-9]+\\@> :< SCAN_REF(disc_kind); return TXT_GLOBAL; >:
+::@=\\G[0-9]+\\@> :< SCAN_REF(glue_kind); return TXT_GLOBAL; >:
+::@=\\S[0-9]+\\@> :< SCAN_REF(language_kind); return TXT_GLOBAL; >:
+::@=\\R[0-9]+\\@> :< SCAN_REF(rule_kind); return TXT_GLOBAL; >:
+::@=\\I[0-9]+\\@> :< SCAN_REF(image_kind); return TXT_GLOBAL; >:
+
+
+::@=\\C[0-9]+\\@> :< SCAN_UDEC(yytext+2); return TXT_CC; >:
+
+::@=\\[a-l]@> :< yylval.u=yytext[1]-'a'; return TXT_LOCAL; >:
+::@=" "@> :< return TXT_FONT_GLUE; >:
+::@="-"@> :< return TXT_FONT_HYPHEN; >:
+
+::@={UTF8_1}@> :< SCAN_UTF8_1(yytext); return TXT_CC; >:
+::@={UTF8_2}@> :< SCAN_UTF8_2(yytext); return TXT_CC; >:
+::@={UTF8_3}@> :< SCAN_UTF8_3(yytext); return TXT_CC; >:
+::@={UTF8_4}@> :< SCAN_UTF8_4(yytext); return TXT_CC; >:
+}
+@
+
+@<scanning macros@>=
+#define @[SCAN_REF(K)@] @[yylval.rf.k=K;@+ yylval.rf.n=atoi(yytext+2)@;@]
+static int scan_level=0;
+#define SCAN_START @[yy_push_state(INITIAL);@+if (1==scan_level++) hpos0=hpos;@]
+#define SCAN_END @[if (scan_level--) yy_pop_state(); @/else QUIT("Too many '>' in line %d",yylineno)@]
+#define SCAN_TXT_START @[BEGIN(TXT)@;@]
+#define SCAN_TXT_END @[BEGIN(INITIAL)@;@]
+@
+ at s txt symbol
+
+@<parsing rules@>=
+list: TXT_START position @|
+ {hpos+=4; /* start byte, two size byte, and boundary byte */ }
+ text TXT_END@|
+ { $$.k=text_kind;$$.p=$4; $$.s=(hpos-hstart)-$4;
+ hput_tags($2,hput_list($2+1, &($$)));@+};
+text: position @+| text txt;
+
+txt: TXT_CC { hput_txt_cc($1); }
+ | TXT_FONT { REF(font_kind,$1); hput_txt_font($1); }
+ | TXT_GLOBAL { REF($1.k,$1.n); hput_txt_global(&($1)); }
+ | TXT_LOCAL { RNG("Font parameter",$1,0,11); hput_txt_local($1); }
+ | TXT_FONT_GLUE { HPUTX(1); HPUT8(txt_glue); }
+ | TXT_FONT_HYPHEN { HPUTX(1);HPUT8(txt_hyphen); }
+ | TXT_IGNORE { HPUTX(1);HPUT8(txt_ignore); }
+ | { HPUTX(1); HPUT8(txt_node);} content_node;
+@
+
+The following function keeps track of the position in the current line.
+It the line gets too long it will break the text at the next space
+character. If no suitable space character comes along,
+the line will be broken after any regular character.
+
+\writecode
+@<write a text@>=
+{@+if (l->s==0) hwritef(" \"\"");
+ else@/
+ { int pos=nesting+20; /* estimate */
+ hwritef(" \"");
+ while(hpos<hend)@/
+ { int i=hget_txt();
+ if (i<0)
+ { if (pos++<70) hwritec(' ');
+ else hwrite_nesting(), pos=nesting;
+ }
+ else if (i==1 && pos>=100)@/
+ { hwritec('\\'); @+hwrite_nesting(); @+pos=nesting; @+}
+ else
+ pos+=i;
+ }
+ hwritec('"');
+ }
+}
+@
+
+
+The function returns the number of characters written
+because this information is needed in |hget_txt| below.
+
+@<write functions@>=
+int hwrite_txt_cc(uint32_t c)
+{@+ if (c<0x20)
+ return hwritef("\\C%d\\",c);
+ else at +
+ switch(c)
+ { case '\\': return hwritef("\\\\");
+ case '"': return hwritef("\\\"");
+ case '<': return hwritef("\\<");
+ case '>': return hwritef("\\>");
+ case ' ': return hwritef("\\ ");
+ case '-': return hwritef("\\-");
+ default: return option_utf8?hwrite_utf8(c):hwritef("\\C%d\\",c);
+ }
+}
+@
+
+\getcode
+@<get macros@>=
+#define @[HGET_GREF(K,S)@] {uint8_t n=HGET8;@+ REF(K,n); @+ return hwritef("\\" S "%d\\",n);@+}
+
+@
+
+The function |hget_txt| reads a text element and writes it immediately.
+To enable the insertion of line breaks when writing a text, we need to keep track
+of the number of characters in the current line. For this purpose
+the function |hget_txt| returns the number of characters written.
+It returns $-1$ if a space character needs to be written
+providing a good opportunity for a break.
+
+@<get functions@>=
+int hget_txt(void)
+{@+ if (*hpos>=0x80 && *hpos<=0xF7)
+ { if (option_utf8)
+ return hwrite_utf8(hget_utf8());
+ else
+ return hwritef("\\C%d\\",hget_utf8());
+ }
+ else @/
+ { uint8_t a;
+ a=HGET8;
+ switch (a)
+ { case txt_font+0: return hwritef("\\0");
+ case txt_font+1: return hwritef("\\1");
+ case txt_font+2: return hwritef("\\2");
+ case txt_font+3: return hwritef("\\3");
+ case txt_font+4: return hwritef("\\4");
+ case txt_font+5: return hwritef("\\5");
+ case txt_font+6: return hwritef("\\6");
+ case txt_font+7: return hwritef("\\7");
+ case txt_global+0: HGET_GREF(font_kind,"F");
+ case txt_global+1: HGET_GREF(penalty_kind,"P");
+ case txt_global+2: HGET_GREF(kern_kind,"K");
+ case txt_global+3: HGET_GREF(ligature_kind,"L");
+ case txt_global+4: HGET_GREF(disc_kind,"D");
+ case txt_global+5: HGET_GREF(glue_kind,"G");
+ case txt_global+6: HGET_GREF(language_kind,"S");
+ case txt_global+7: HGET_GREF(rule_kind,"R");
+ case txt_global+8: HGET_GREF(image_kind,"I");
+ case txt_local+0: return hwritef("\\a");
+ case txt_local+1: return hwritef("\\b");
+ case txt_local+2: return hwritef("\\c");
+ case txt_local+3: return hwritef("\\d");
+ case txt_local+4: return hwritef("\\e");
+ case txt_local+5: return hwritef("\\f");
+ case txt_local+6: return hwritef("\\g");
+ case txt_local+7: return hwritef("\\h");
+ case txt_local+8: return hwritef("\\i");
+ case txt_local+9: return hwritef("\\j");
+ case txt_local+10: return hwritef("\\k");
+ case txt_local+11: return hwritef("\\l");
+ case txt_cc: return hwrite_txt_cc(hget_utf8());
+ case txt_node: { int i;
+ @<read the start byte |a|@>@;
+ i=hwritef("<");
+ i+= hwritef("%s",content_name[KIND(a)]);@+ hget_content(a);
+ @<read and check the end byte |z|@>@;
+ hwritec('>');@+ return i+10; /* just an estimate */
+ }
+ case txt_hyphen: hwritec('-'); @+return 1;
+ case txt_glue: return -1;
+ case '<': return hwritef("\\<");
+ case '>': return hwritef("\\>");
+ case '"': return hwritef("\\\"");
+ case '-': return hwritef("\\-");
+ case txt_ignore: return hwritef("\\@@");
+ default: hwritec(a); @+return 1;
+ }
+ }
+}
+
+@
+
+
+\putcode
+@<put functions@>=
+
+void hput_txt_cc(uint32_t c)
+{ @+ if (c<=0x20) { HPUTX(2); HPUT8(txt_cc);@+ HPUT8(c); @+ }
+ else hput_utf8(c);
+}
+
+void hput_txt_font(uint8_t f)
+{@+ if (f<8) HPUTX(1),HPUT8(txt_font+f);
+ else QUIT("Use \\F%d\\ instead of \\%d for font %d in a text",f,f,f);
+}
+
+void hput_txt_global(ref_t *d)
+{ @+ HPUTX(2);
+ switch (d->k)
+ { case font_kind: HPUT8(txt_global+0);@+ break;
+ case penalty_kind: HPUT8(txt_global+1);@+ break;
+ case kern_kind: HPUT8(txt_global+2);@+ break;
+ case ligature_kind: HPUT8(txt_global+3);@+ break;
+ case disc_kind: HPUT8(txt_global+4);@+ break;
+ case glue_kind: HPUT8(txt_global+5);@+ break;
+ case language_kind: HPUT8(txt_global+6);@+ break;
+ case rule_kind: HPUT8(txt_global+7);@+ break;
+ case image_kind: HPUT8(txt_global+8);@+ break;
+ default: QUIT("Kind %s not allowed as a global reference in a text",NAME(d->k));
+ }
+ HPUT8(d->n);
+}
+
+void hput_txt_local(uint8_t n)
+{ HPUTX(1);
+ HPUT8(txt_local+n);
+}
+@
+
+
+@<hint types@>=
+typedef struct { @+kind_t k; @+int n; @+} ref_t;
+@
+
+
+\section{Composite Nodes}\hascode
+\label{composite}
+The nodes that we consider in this section can contain one or more list nodes.
+When we implement the parsing\index{parsing} routines
+for composite nodes in the long format, we have to take into account
+that parsing such a list node will already write the list node
+to the output. So we split the parsing of composite nodes into several parts
+and store the parts immediately after parsing them. On the parse stack, we will only
+keep track of the info value.
+This new strategy is not as transparent as our previous strategy used for
+simple nodes where we had a clean separation of reading and writing:
+reading would store the internal representation of a node and writing the internal
+representation to output would start only after reading is completed.
+The new strategy, however, makes it easier to reuse
+the grammar\index{grammar} rules for the component nodes.
+
+Another rule applies to composite nodes: in the short format, the subnodes
+will come at the end of the node, and especially a list node that contains content nodes
+comes last. This helps when traversing the content section as we will see in
+appendix~\secref{fastforward}.
+
+\subsection{Boxes}\label{boxnodes}
+The central structuring elements of \TeX\ are boxes\index{box}.
+Boxes have a height |h|, a depth |d|, and a width |w|.
+The shift amount |a| shifts the contents of the box, the glue ratio\index{glue ratio} |r| is a factor
+applied to the glue inside the box, the glue order |o| is its order of stretchability\index{stretchability},
+and the glue sign |s| is $-1$ for shrinking\index{shrinkability}, 0 for rigid, and $+1$ for stretching.
+Most importantly, a box contains a list |l| of content nodes inside the box.
+
+
+@<hint types@>=
+typedef struct @/{@+ dimen_t h,d,w,a;@+ float32_t r;@+ int8_t s,o; @+list_t l; @+} box_t;
+@
+
+There are two types of boxes: horizontal\index{horizontal box} boxes
+and vertical\index{vertical box} boxes.
+The difference between the two is simple:
+a horizontal box aligns the reference\index{reference point}
+points of its content nodes horizontally, and a positive shift amount\index{shift amount} |a|
+shifts the box down;
+a vertical box aligns\index{alignment} the reference\index{reference point}
+points vertically, and a positive shift amount |a| shifts the box right.
+
+Not all box parameters are used frequently. In the short format, we use the info bits
+to indicated which of the parameters are used.
+Where as the width of a horizontal box is most of the time (80\%) nonzero,
+other parameters are most of the time zero,
+like the shift amount (99\%) or the glue settings (99.8\%).
+The depth is zero in about 77\%, the height in about 53\%,
+and both together are zero in about 47\%. The results for vertical boxes,
+which constitute about 20\% of all boxes, are similar,
+except that the depth is zero in about 89\%,
+but the height and width are almost never zero.
+For this reason we use bit |b001| to indicate a nonzero depth,
+bit |b010| for a nonzero shift amount, and |b100| for nonzero glue settings.
+Glue sign and glue order can be packed as two nibbles in a single byte.
+% A different use of the info bits for vertical and horizontal boxes is possible,
+% but does not warrant the added complexity.
+
+
+
+\goodbreak
+\readcode
+ at s HBOX symbol
+ at s VBOX symbol
+ at s box symbol
+ at s boxparams symbol
+ at s hbox_node symbol
+ at s vbox_node symbol
+ at s box_dimen symbol
+ at s box_shift symbol
+ at s box_glue_set symbol
+@<symbols@>=
+%token HBOX "hbox"
+%token VBOX "vbox"
+%token SHIFTED "shifted"
+%type <info> box box_dimen box_shift box_glue_set
+
+@
+@<scanning rules@>=
+::@=hbox@> :< return HBOX; >:
+::@=vbox@> :< return VBOX; >:
+::@=shifted@> :< return SHIFTED; >:
+@
+
+@<parsing rules@>=@/
+
+box_dimen: dimension dimension dimension @/
+ {$$= hput_box_dimen($1,$2,$3); };
+box_shift: {$$=b000;} @+
+ | SHIFTED dimension {$$=hput_box_shift($2);};
+
+box_glue_set: {$$=b000;}
+ | PLUS stretch { $$=hput_box_glue_set(+1,$2.f,$2.o); }
+ | MINUS stretch { $$=hput_box_glue_set(-1,$2.f,$2.o); };
+
+
+box: box_dimen box_shift box_glue_set list {$$=$1|$2|$3; };
+
+hbox_node: start HBOX box END { hput_tags($1, TAG(hbox_kind,$3)); };
+vbox_node: start VBOX box END { hput_tags($1, TAG(vbox_kind,$3)); };
+content_node: hbox_node @+ | vbox_node;
+@
+
+\writecode
+@<write functions@>=
+void hwrite_box(box_t *b)
+{ hwrite_dimension(b->h);
+ hwrite_dimension(b->d);
+ hwrite_dimension(b->w);
+ if (b->a!=0) { hwritef(" shifted"); @+hwrite_dimension(b->a); @+}
+ if (b->r!=0.0 && b->s!=0 )@/
+ { @+if (b->s>0) @+hwritef(" plus"); @+else @+hwritef(" minus");
+ @+hwrite_float64(b->r); @+hwrite_order(b->o);
+ }
+ hwrite_list(&(b->l));
+}
+@
+
+\getcode
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(hbox_kind,b000): {box_t b; @+HGET_BOX(b000,b); @+hwrite_box(&b);@+} @+ break;
+case TAG(hbox_kind,b001): {box_t b; @+HGET_BOX(b001,b); @+hwrite_box(&b);@+} @+ break;
+case TAG(hbox_kind,b010): {box_t b; @+HGET_BOX(b010,b); @+hwrite_box(&b);@+} @+ break;
+case TAG(hbox_kind,b011): {box_t b; @+HGET_BOX(b011,b); @+hwrite_box(&b);@+} @+ break;
+case TAG(hbox_kind,b100): {box_t b; @+HGET_BOX(b100,b); @+hwrite_box(&b);@+} @+ break;
+case TAG(hbox_kind,b101): {box_t b; @+HGET_BOX(b101,b); @+hwrite_box(&b);@+} @+ break;
+case TAG(hbox_kind,b110): {box_t b; @+HGET_BOX(b110,b); @+hwrite_box(&b);@+} @+ break;
+case TAG(hbox_kind,b111): {box_t b; @+HGET_BOX(b111,b); @+hwrite_box(&b);@+} @+ break;
+case TAG(vbox_kind,b000): {box_t b; @+HGET_BOX(b000,b); @+hwrite_box(&b);@+} @+ break;
+case TAG(vbox_kind,b001): {box_t b; @+HGET_BOX(b001,b); @+hwrite_box(&b);@+} @+ break;
+case TAG(vbox_kind,b010): {box_t b; @+HGET_BOX(b010,b); @+hwrite_box(&b);@+} @+ break;
+case TAG(vbox_kind,b011): {box_t b; @+HGET_BOX(b011,b); @+hwrite_box(&b);@+} @+ break;
+case TAG(vbox_kind,b100): {box_t b; @+HGET_BOX(b100,b); @+hwrite_box(&b);@+} @+ break;
+case TAG(vbox_kind,b101): {box_t b; @+HGET_BOX(b101,b); @+hwrite_box(&b);@+} @+ break;
+case TAG(vbox_kind,b110): {box_t b; @+HGET_BOX(b110,b); @+hwrite_box(&b);@+} @+ break;
+case TAG(vbox_kind,b111): {box_t b; @+HGET_BOX(b111,b); @+hwrite_box(&b);@+} @+ break;
+@
+
+@<get macros@>=
+#define @[HGET_BOX(I,B)@] \
+HGET32(B.h);\
+if ((I)&b001) HGET32(B.d); @+ else B.d=0;\
+HGET32(B.w);\
+if ((I)&b010) HGET32(B.a); @+else B.a=0;\
+if ((I)&b100) @/{ B.r=hget_float32();@+ B.s=HGET8; @+ B.o=B.s&0xF; @+B.s=B.s>>4;@+ }\
+else { B.r=0.0;@+ B.o=B.s=0;@+ }\
+hget_list(&(B.l));
+@
+
+@<get functions@>=
+void hget_hbox_node(void)
+{ box_t b;
+ @<read the start byte |a|@>@;
+ if (KIND(a)!=hbox_kind) QUIT("Hbox expected at 0x%x got %s",node_pos,NAME(a));
+ HGET_BOX(INFO(a),b);@/
+ @<read and check the end byte |z|@>@;
+ hwrite_start();@+
+ hwritef("hbox");@+
+ hwrite_box(&b);@+
+ hwrite_end();
+}
+
+
+void hget_vbox_node(void)
+{ box_t b;
+ @<read the start byte |a|@>@;
+ if (KIND(a)!=vbox_kind) QUIT("Vbox expected at 0x%x got %s",node_pos,NAME(a));
+ HGET_BOX(INFO(a),b);@/
+ @<read and check the end byte |z|@>@;
+ hwrite_start();@+
+ hwritef("vbox");@+
+ hwrite_box(&b);@+
+ hwrite_end();
+}
+@
+
+\putcode
+@<put functions@>=
+
+info_t hput_box_dimen(dimen_t h, dimen_t d, dimen_t w)
+{ info_t i;
+ @+HPUT32(h);
+ if (d!=0) { HPUT32(d); @+i=b001;@+ } @+else at + i=b000;
+ HPUT32(w);
+ return i;
+}
+info_t hput_box_shift(dimen_t a)
+{ @+if (a!=0) { @+ HPUT32(a); @+return @+ b010;@+} @+ else @+return b000;
+}
+
+info_t hput_box_glue_set(int8_t s, float32_t r, order_t o)
+{ @+if (r!=0.0 && s!=0 )
+ { hput_float32(r);@+
+ HPUT8((s<<4)|o);@+
+ return b100;@+
+ }
+ else return b000;
+}
+
+@
+
+\subsection{Extended Boxes}
+Hi\TeX\ produces two kinds of extended\index{extended box} horizontal
+boxes, |hpack_kind| and |hset_kind|, and the same for vertical boxes
+using |vpack_kind| and |vset_kind|. Let us focus on horizontal boxes;
+the handling of vertical boxes is completely parallel.
+
+The \\{hpack} procedure of Hi\TeX\ produces an extended box of |hset_kind|
+either if it is given an extended\index{extended dimension} dimension as its width
+or if it discovers that the width of its content is an extended
+dimension. After the final width of the box has been computed in the
+viewer, it just remains to set the glue; a very simple operation
+indeed.
+
+If the \\{hpack} procedure of Hi\TeX\ can not determine the natural
+dimensions of the box content because it contains
+paragraphs\index{paragraph} or other extended boxes, it produces a box
+of |hpack_kind|. Now the viewer needs to traverse the list of content
+nodes to determine the natural\index{natural dimension}
+dimensions. Even the amount of stretchability\index{stretchability}
+and shrinkability\index{shrinkability} has to be determined in the
+viewer. For example, the final stretchability of a paragraph with some
+stretchability in the baseline\index{baseline skip} skip will depend
+on the number of lines which, in turn, depends on \.{hsize}. It is
+not possible to merge these traversals of the box content with the
+traversal necessary when displaying the box. The latter needs to
+convert glue nodes into positioning instructions which requires a
+fixed glue\index{glue ratio} ratio. The computation of the glue ratio,
+however, requires a complete traversal of the content.
+
+In the short format of a box node of type |hset_kind|, |vset_kind|,
+|hpack_kind|, or |vpack_kind|, the info bit |b100| indicates, if set,
+a complete extended dimension, and if unset, a reference to a
+predefined extended dimension for the target size; the info bit |b010|
+indicates a nonzero shift amount. For a box of type |hset_kind| or
+|vset_kind|, the info bit |b001| indicates, if set, a nonzero depth.
+For a box of type |hpack_kind| or |vpack_kind|, the info bit |b001|
+indicates, if set, an additional target size, and if unset, an exact
+target size. For a box of type |vpack_kind| also the maximum depth is
+given.
+
+\readcode
+ at s xbox symbol
+ at s hpack symbol
+ at s vpack symbol
+ at s box_goal symbol
+ at s HPACK symbol
+ at s HSET symbol
+ at s VPACK symbol
+ at s VSET symbol
+ at s TO symbol
+ at s ADD symbol
+ at s box_flex symbol
+ at s vxbox_node symbol
+ at s hxbox_node symbol
+ at s DEPTH symbol
+
+@<symbols@>=
+%token HPACK "hpack"
+%token HSET "hset"
+%token VPACK "vpack"
+%token VSET "vset"
+%token DEPTH "depth"
+%token ADD "add"
+%token TO "to"
+%type <info> xbox box_goal hpack vpack
+@
+
+@<scanning rules@>=
+::@=hpack@> :< return HPACK; >:
+::@=hset@> :< return HSET; >:
+::@=vpack@> :< return VPACK; >:
+::@=vset@> :< return VSET; >:
+::@=add@> :< return ADD; >:
+::@=to@> :< return TO; >:
+::@=depth@> :< return DEPTH; >:
+@
+
+@<parsing rules@>=
+box_flex: plus minus { hput_stretch(&($1));hput_stretch(&($2)); };
+xbox: box_dimen box_shift box_flex xdimen_ref list {$$=$1|$2;}
+ | box_dimen box_shift box_flex xdimen_node list {$$=$1|$2|b100;};
+
+box_goal: TO xdimen_ref {$$=b000;}
+ | ADD xdimen_ref {$$=b001;}
+ | TO xdimen_node {$$=b100;}
+ | ADD xdimen_node {$$=b101;};
+
+hpack: box_shift box_goal list {$$=$2;};
+vpack: box_shift MAX DEPTH dimension {HPUT32($4);} @/ box_goal list {$$= $1|$6;};
+
+vxbox_node: start VSET xbox END { hput_tags($1, TAG(vset_kind,$3)); }
+ | start VPACK vpack END { hput_tags($1, TAG(vpack_kind,$3)); };
+
+
+hxbox_node: start HSET xbox END { hput_tags($1, TAG(hset_kind,$3)); }
+ | start HPACK hpack END { hput_tags($1, TAG(hpack_kind,$3)); };
+
+content_node: vxbox_node | hxbox_node;
+ @
+
+\getcode
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(hset_kind,b000): HGET_SET(b000); @+ break;
+case TAG(hset_kind,b001): HGET_SET(b001); @+ break;
+case TAG(hset_kind,b010): HGET_SET(b010); @+ break;
+case TAG(hset_kind,b011): HGET_SET(b011); @+ break;
+case TAG(hset_kind,b100): HGET_SET(b100); @+ break;
+case TAG(hset_kind,b101): HGET_SET(b101); @+ break;
+case TAG(hset_kind,b110): HGET_SET(b110); @+ break;
+case TAG(hset_kind,b111): HGET_SET(b111); @+ break;@#
+
+case TAG(vset_kind,b000): HGET_SET(b000); @+ break;
+case TAG(vset_kind,b001): HGET_SET(b001); @+ break;
+case TAG(vset_kind,b010): HGET_SET(b010); @+ break;
+case TAG(vset_kind,b011): HGET_SET(b011); @+ break;
+case TAG(vset_kind,b100): HGET_SET(b100); @+ break;
+case TAG(vset_kind,b101): HGET_SET(b101); @+ break;
+case TAG(vset_kind,b110): HGET_SET(b110); @+ break;
+case TAG(vset_kind,b111): HGET_SET(b111); @+ break;@#
+
+case TAG(hpack_kind,b000): HGET_PACK(hpack_kind,b000); @+ break;
+case TAG(hpack_kind,b001): HGET_PACK(hpack_kind,b001); @+ break;
+case TAG(hpack_kind,b010): HGET_PACK(hpack_kind,b010); @+ break;
+case TAG(hpack_kind,b011): HGET_PACK(hpack_kind,b011); @+ break;
+case TAG(hpack_kind,b100): HGET_PACK(hpack_kind,b100); @+ break;
+case TAG(hpack_kind,b101): HGET_PACK(hpack_kind,b101); @+ break;
+case TAG(hpack_kind,b110): HGET_PACK(hpack_kind,b110); @+ break;
+case TAG(hpack_kind,b111): HGET_PACK(hpack_kind,b111); @+ break;@#
+
+case TAG(vpack_kind,b000): HGET_PACK(vpack_kind,b000); @+ break;
+case TAG(vpack_kind,b001): HGET_PACK(vpack_kind,b001); @+ break;
+case TAG(vpack_kind,b010): HGET_PACK(vpack_kind,b010); @+ break;
+case TAG(vpack_kind,b011): HGET_PACK(vpack_kind,b011); @+ break;
+case TAG(vpack_kind,b100): HGET_PACK(vpack_kind,b100); @+ break;
+case TAG(vpack_kind,b101): HGET_PACK(vpack_kind,b101); @+ break;
+case TAG(vpack_kind,b110): HGET_PACK(vpack_kind,b110); @+ break;
+case TAG(vpack_kind,b111): HGET_PACK(vpack_kind,b111); @+ break;
+@
+
+
+@<get macros@>=
+#define @[HGET_SET(I)@] @/\
+ { dimen_t h; @+HGET32(h); @+hwrite_dimension(h);@+}\
+ { dimen_t d; @+if ((I)&b001) HGET32(d); @+ else d=0;@+hwrite_dimension(d); @+}\
+ { dimen_t w; @+HGET32(w); @+hwrite_dimension(w);@+} \
+if ((I)&b010) { dimen_t a; @+HGET32(a); hwritef(" shifted"); @+hwrite_dimension(a);@+}\
+ { stretch_t p; @+HGET_STRETCH(p);@+hwrite_plus(&p);@+}\
+ { stretch_t m; @+HGET_STRETCH(m);@+hwrite_minus(&m);@+}\
+ if ((I)&b100) {xdimen_t x;@+ hget_xdimen_node(&x); @+hwrite_xdimen_node(&x);@+} else HGET_REF(xdimen_kind)@;\
+ { list_t l; @+hget_list(&l);@+ hwrite_list(&l); @+}
+@#
+
+#define @[HGET_PACK(K,I)@] @/\
+ if ((I)&b010) { dimen_t d; @+HGET32(d); hwritef(" shifted"); @+hwrite_dimension(d); @+ }\
+ if (K==vpack_kind) { dimen_t d; @+HGET32(d); hwritef(" max depth");@+hwrite_dimension(d); @+ }\
+ if ((I)&b001) hwritef(" add");@+ else hwritef(" to");\
+ if ((I)&b100) {xdimen_t x;@+ hget_xdimen_node(&x);@+hwrite_xdimen_node(&x);@+}\
+ else @+HGET_REF(xdimen_kind);\
+ { list_t l; @+hget_list(&l);@+ hwrite_list(&l); @+}
+@
+
+
+\subsection{Leaders}\label{leaders}
+Leaders\index{leaders} are a special type of glue that is best explained by a few
+examples.
+Where as ordinary glue fills its designated space with \hfil\ whiteness,\break
+leaders fill their designated space with either a rule \xleaders\hrule\hfil\ or\break
+some sort of repeated\leaders\hbox to 15pt{$\hss.\hss$}\hfil content.\break
+In multiple leaders, the dots\leaders\hbox to 15pt{$\hss.\hss$}\hfil are usually aligned\index{alignment} across lines,\break
+as in the last\leaders\hbox to 15pt{$\hss.\hss$}\hfil three lines.\break
+Unless you specify centered\index{centered}\cleaders\hbox to 15pt{$\hss.\hss$}\hfil leaders\break
+or you specify expanded\index{expanded}\xleaders\hbox to 15pt{$\hss.\hss$}\hfil leaders.\break
+The former pack the repeated content tight and center
+the repeated content in the available space, the latter distributes
+the extra space between all the repeated instances.
+
+In the short format, the two lowest info bits store the type
+of leaders: 1 for aligned, 2 for centered, and 3 for expanded.
+The |b100| info bit is usually set and only zero in the unlikely
+case that the glue is zero and therefore not present.
+
+\readcode
+ at s LEADERS symbol
+ at s ALIGN symbol
+ at s CENTER symbol
+ at s EXPAND symbol
+ at s leaders symbol
+ at s ltype symbol
+@<symbols@>=
+%token LEADERS "leaders"
+%token ALIGN "align"
+%token CENTER "center"
+%token EXPAND "expand"
+%type <info> leaders
+%type <info> ltype
+@
+
+@<scanning rules@>=
+::@=leaders@> :< return LEADERS; >:
+::@=align@> :< return ALIGN; >:
+::@=center@> :< return CENTER; >:
+::@=expand@> :< return EXPAND; >:
+@
+@<parsing rules@>=
+ltype: {$$=1;} | ALIGN {$$=1;} @+| CENTER {$$=2;} @+| EXPAND {$$=3;};
+leaders: glue_node ltype rule_node {@+if ($1) $$=$2|b100;@+else $$=$2; @+}
+ | glue_node ltype hbox_node {@+if ($1) $$=$2|b100;@+else $$=$2;@+}
+ | glue_node ltype vbox_node {@+if ($1) $$=$2|b100;@+else $$=$2;@+};
+content_node: start LEADERS leaders END @| {@+ hput_tags($1, TAG(leaders_kind, $3));}
+@
+
+\writecode
+@<write functions@>=
+void hwrite_leaders_type(int t)
+{@+
+ if (t==2) hwritef(" center");
+ else if (t==3) hwritef(" expand");
+}
+@
+
+\getcode
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(leaders_kind,1): @+ HGET_LEADERS(1); @+break;
+case TAG(leaders_kind,2): @+ HGET_LEADERS(2); @+break;
+case TAG(leaders_kind,3): @+ HGET_LEADERS(3); @+break;
+case TAG(leaders_kind,b100|1): @+ HGET_LEADERS(b100|1); @+break;
+case TAG(leaders_kind,b100|2): @+ HGET_LEADERS(b100|2); @+break;
+case TAG(leaders_kind,b100|3): @+ HGET_LEADERS(b100|3); @+break;
+@
+@<get macros@>=
+#define @[HGET_LEADERS(I)@]@/ \
+if ((I)&b100) hget_glue_node();\
+hwrite_leaders_type((I)&b011);\
+if (KIND(*hpos)==rule_kind) hget_rule_node(); \
+else if (KIND(*hpos)==hbox_kind) hget_hbox_node(); \
+else hget_vbox_node();
+@
+
+\subsection{Baseline Skips}
+Baseline\index{baseline skip} skips are small amounts of glue inserted
+between two consecutive lines of text. To get nice looking pages, the
+amount of glue\index{glue} inserted must take into account the depth
+of the line above the glue and the height of the line below the glue
+to achieve a constant distance of the baselines. For example, if we
+have the lines
+\medskip
+
+\qquad\vbox{\hsize=0.5\hsize\noindent
+``There is no\hfil\break
+more gas\hfil\break
+in the tank.''
+}\hss
+
+\medskip\noindent
+\TeX\ will insert 7.69446pt of baseline skip between the first and the
+second line and 3.11111pt of baseline skip between the second and the
+third line. This is due to the fact that the first line has no
+descenders, its depth is zero, the second line has no ascenders but
+the ``g'' descends below the baseline, and the third line has
+ascenders (``t'', ``h'',\dots) so it is higher than the second line.
+\TeX's choice of baseline skips ensures that the baselines are exactly
+12pt apart in both cases.
+
+Things get more complicated if the text contains mathematical formulas because then
+a line can get so high or deep that it is impossible to keep the distance between
+baselines constant without two adjacent lines touching each other. In such cases,
+\TeX\ will insert a small minimum line skip glue\index{line skip glue}.
+
+For the whole computation, \TeX\ uses three parameters: {\tt base\-line\-skip},
+{\tt line\-skip\-limit},\index{line skip limit} and
+{\tt lineskip}. {\tt baselineskip} is a glue value; its size is the
+normal distance of two baselines. \TeX\ adjusts the size of the
+{\tt baselineskip} glue for the height and the depth of the two lines and
+then checks the result against {\tt lineskiplimit}. If the result is
+smaller than {\tt lineskiplimit} it will use the {\tt lineskip} glue
+instead.
+
+Because the depth and the height of lines depend on the outcome
+of the line breaking\index{line breaking}
+routine, baseline computations must be done in the viewer.
+The situation gets even more complicated because \TeX\ can manipulate the insertion
+of baseline skips in various ways. Therefore \HINT\ requires the insertion of
+baseline nodes wherever the viewer is supposed to perform a baseline skip
+computation.
+
+In the short format of a baseline definition, we store only
+the nonzero components and use the
+info bits to mark them: |b100| implies $|bs|\ne0$,
+|b010| implies $|ls|\ne 0$, and |b001| implies $|lslimit|\ne 0$.
+If the baseline has only zero components, we put a reference to baseline number 0
+in the output.
+
+@<hint basic types@>=
+typedef struct {@+
+glue_t bs, ls;@+
+dimen_t lsl;@+
+} baseline_t;
+@
+
+
+
+\readcode
+ at s BASELINE symbol
+ at s baseline symbol
+@<symbols@>=
+%token BASELINE "baseline"
+%type <info> baseline
+@
+@<scanning rules@>=
+::@=baseline@> :< return BASELINE; >:
+@
+
+@<parsing rules@>=
+baseline: dimension { if ($1!=0) HPUT32($1); }
+ glue_node glue_node @/{ $$=b000; if ($1!=0) $$|=b001;
+ if ($3) $$|=b100;
+ if ($4) $$|=b010;
+ @+};
+content_node: start BASELINE baseline END @/
+{ @+if ($3==b000) HPUT8(0); @+hput_tags($1,TAG(baseline_kind, $3)); };
+@
+
+\getcode
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(baseline_kind,b001): { baseline_t b;@+ HGET_BASELINE(b001,b);@+ }@+break;
+case TAG(baseline_kind,b010): { baseline_t b;@+ HGET_BASELINE(b010,b);@+ }@+break;
+case TAG(baseline_kind,b011): { baseline_t b;@+ HGET_BASELINE(b011,b);@+ }@+break;
+case TAG(baseline_kind,b100): { baseline_t b;@+ HGET_BASELINE(b100,b);@+ }@+break;
+case TAG(baseline_kind,b101): { baseline_t b;@+ HGET_BASELINE(b101,b);@+ }@+break;
+case TAG(baseline_kind,b110): { baseline_t b;@+ HGET_BASELINE(b110,b);@+ }@+break;
+case TAG(baseline_kind,b111): { baseline_t b;@+ HGET_BASELINE(b111,b);@+ }@+break;
+@
+
+@<get macros@>=
+#define @[HGET_BASELINE(I,B)@] \
+ if((I)&b001) HGET32((B).lsl); @+else B.lsl=0; hwrite_dimension(B.lsl);\
+ if((I)&b100) hget_glue_node(); \
+ else {B.bs.p.o=B.bs.m.o=B.bs.w.w=0; @+B.bs.w.h=B.bs.w.v=B.bs.p.f=B.bs.m.f=0.0; @+hwrite_glue_node(&(B.bs));@+}\
+ if((I)&b010) hget_glue_node(); \
+ else {B.ls.p.o=B.ls.m.o=B.ls.w.w=0; @+B.ls.w.h=B.ls.w.v=B.ls.p.f=B.ls.m.f=0.0; @+hwrite_glue_node(&(B.ls));@+}
+@
+
+
+\putcode
+@<put functions@>=
+uint8_t hput_baseline(baseline_t *b)
+{ info_t info=b000;
+ if (!ZERO_GLUE(b->bs)) @+info|=b100;
+ if (!ZERO_GLUE(b->ls)) @+ info|=b010;
+ if (b->lsl!=0) { @+ HPUT32(b->lsl); @+info|=b001; @+}
+ return TAG(baseline_kind,info);
+}
+@
+
+
+
+\subsection{Ligatures}
+Ligatures\index{ligature} occur only in horizontal lists. They specify characters
+that combine the glyphs of several characters into one specialized
+glyph. For example in the word ``{\it difficult\/}'' the three letters
+``{\it f{}f{}i\/}'' are combined into the ligature ``{\it ffi\/}''.
+Hence, a ligature is very similar to a simple glyph node; the
+characters that got replaced are, however, retained in the ligature
+because they might be needed for example to support searching. Since
+ligatures are therefore only specialized list of characters and since
+we have a very efficient way to store such lists of characters, namely
+as a |text|, input and output of ligatures is quite simple.
+
+The info value zero is reserved for references to a ligature. If the
+info value is between 1 and 6, it gives the number of bytes used to encode
+the characters in UTF8. Note that a ligature will always include a
+glyph byte, so the minimum size is 1. A typical ligature like ``{\it fi\/}''
+will need 3 byte: the ligature character ``{\it fi\/}'', and
+the replacement characters ``f'' and ''i''. More byte might be
+required if the character codes exceed |0x7F| since we use the UTF8
+encoding scheme for larger character codes. If the info value is 7,
+a full text node follows the font byte. In the long
+format, we give the font, the character code, and then the replacement
+characters represented as a text.
+
+@<hint types@>=
+typedef struct{@+uint8_t f; @+list_t l;@+} lig_t;
+@
+
+\readcode
+ at s ref symbol
+ at s LIGATURE symbol
+ at s ligature symbol
+ at s cc_list symbol
+ at s lig_cc symbol
+@<symbols@>=
+%token LIGATURE "ligature"
+%type <u> lig_cc
+%type <lg> ligature
+%type <u> ref
+@
+@<scanning rules@>=
+::@=ligature@> :< return LIGATURE; >:
+@
+
+@<parsing rules@>=@/
+cc_list:@+ | cc_list TXT_CC { hput_utf8($2); };
+lig_cc: UNSIGNED {RNG("UTF-8 code",$1,0,0x1FFFFF);$$=hpos-hstart; hput_utf8($1); };
+lig_cc: CHARCODE {$$=hpos-hstart; hput_utf8($1); };
+ref: REFERENCE { HPUT8($1); $$=$1; };
+ligature: ref { REF(font_kind,$1);} lig_cc TXT_START cc_list TXT_END @/
+ { $$.f=$1; $$.l.p=$3; $$.l.s=(hpos-hstart)-$3;
+ RNG("Ligature size",$$.l.s,0,255);};
+content_node: start LIGATURE ligature END {hput_tags($1,hput_ligature(&($3)));};
+@
+
+\writecode
+@<write functions@>=
+void hwrite_ligature(lig_t *l)
+{ uint32_t pos=hpos-hstart;
+ hwrite_ref(l->f);
+ hpos=l->l.p+hstart;
+ hwrite_charcode(hget_utf8());
+ hwritef(" \"");
+ while (hpos<hstart+l->l.p+l->l.s)
+ hwrite_txt_cc(hget_utf8());
+ hwritec('"');
+ hpos=hstart+pos;
+}
+@
+
+\getcode
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(ligature_kind,1):@+ {lig_t l; @+HGET_LIG(1,l);@+} @+break;
+case TAG(ligature_kind,2):@+ {lig_t l; @+HGET_LIG(2,l);@+} @+break;
+case TAG(ligature_kind,3):@+ {lig_t l; @+HGET_LIG(3,l);@+} @+break;
+case TAG(ligature_kind,4):@+ {lig_t l; @+HGET_LIG(4,l);@+} @+break;
+case TAG(ligature_kind,5):@+ {lig_t l; @+HGET_LIG(5,l);@+} @+break;
+case TAG(ligature_kind,6):@+ {lig_t l; @+HGET_LIG(6,l);@+} @+break;
+case TAG(ligature_kind,7):@+ {lig_t l; @+HGET_LIG(7,l);@+} @+break;
+@
+@<get macros@>=
+#define @[HGET_LIG(I,L)@] @/\
+(L).f=HGET8;REF(font_kind,(L).f);\
+if ((I)==7) hget_list(&((L).l)); \
+else { (L).l.s=(I); (L).l.p=hpos-hstart; @+ hpos+=(L).l.s;} \
+hwrite_ligature(&(L));
+@
+
+\putcode
+@<put functions@>=
+uint8_t hput_ligature(lig_t *l)
+{ @+if (l->l.s < 7) return TAG(ligature_kind,l->l.s);
+ else@/
+ { uint32_t pos=l->l.p;
+ hput_tags(pos,hput_list(pos+1, &(l->l)));
+ return TAG(ligature_kind,7);
+ }
+}
+@
+
+
+\subsection{Discretionary breaks}\label{discbreak}\index{discretionary break}
+\HINT\ is capable to break lines into paragraphs. It does this
+primarily at interword spaces but it might also break a line in the
+middle of a word if it finds a discretionary\index{discretionary break}
+line break there. These discretionary breaks are usually
+provided by an automatic hyphenation algorithm but they might be also
+explicitly\index{explicit} inserted by the author of a
+document.
+
+When a line break occurs at such a discretionary break, the line
+before the break ends with a |pre_break| list of nodes, the line after
+the break starts with a |post_break| list of nodes, and the next
+|replace_count| nodes after the discretionary break will be
+ignored. Both lists must consist entirely of glyphs\index{glyph},
+kerns\index{kern}, boxes\index{box}, rules\index{rule}, or
+ligatures\index{ligature}. For example, an ordinary discretionary
+break will have a |pre_break| list containing ``-'', an empty
+|post_break| list, and a |replace_count| of zero.
+
+The long format starts with an optional ``{\tt !}'', indicating an
+explicit discretionary break, followed by the replace-count.
+Then comes the pre-break list followed by the post-break list.
+The replace-count can be omitted if it is zero;
+an empty post-break list may be omitted as well.
+Both list may be omitted only if both are empty.
+
+In the short format, the three components of a disc node are stored
+in this order: |replace_count|, |pre_break| list, and |post_break| list.
+The |b100| bit in the info value indicates the presence of a replace-count,
+the |b010| bit the presence of a |pre_break| list,
+and the |b001| bit the presence of a |post_break| list.
+Since the info value |b000| is reserved for references, at least one
+of these must be specified; so we represent a node with empty lists
+and a replace\index{replace count} count of zero using the info value
+|b100| and a zero byte for the replace count.
+
+Replace counts must be in the range 0 to 31; so the short format can
+set the high bit of the replace count to indicate an explicit\index{explicit} break.
+
+@<hint types@>=
+typedef struct disc_t at + {@+ bool x; @+list_t p,q;@+ uint8_t r;@+ } disc_t;
+@
+
+
+\readcode
+ at s DISC symbol
+ at s disc symbol
+ at s disc_node symbol
+ at s replace_count symbol
+
+@<symbols@>=
+%token DISC "disc"
+%type <dc> disc
+%type <u> replace_count
+@
+@<scanning rules@>=
+::@=disc@> :< return DISC; >:
+@
+
+@<parsing rules@>=@/
+replace_count: explicit {@+ if ($1) {$$=0x80; HPUT8(0x80);@+}@+ else $$=0x00;@+}
+ | explicit UNSIGNED { RNG("Replace count",$2,0,31);
+ $$=($2)|(($1)?0x80:0x00); @+ if ($$!=0) HPUT8($$);@+};
+disc: replace_count list list { $$.r=$1;$$.p=$2; $$.q=$3;
+ if ($3.s==0) { hpos=hpos-2;@+ if ($2.s==0) hpos=hpos-2; @+}@+}
+ | replace_count list { $$.r=$1;$$.p=$2; if ($2.s==0) hpos=hpos-2;@+ $$.q.s=0; }
+ | replace_count { $$.r=$1;$$.p.s=0; $$.q.s=0; };
+
+
+disc_node: start DISC disc END
+ {hput_tags($1,hput_disc(&($3)));};
+
+content_node: disc_node;
+@
+
+\writecode
+@<write functions@>=
+void hwrite_disc(disc_t *h)
+{ @+hwrite_explicit(h->x);
+ if (h->r!=0) hwritef(" %d",h->r);
+ if (h->p.s!=0 || h->q.s!=0) hwrite_list(&(h->p));
+ if (h->q.s!=0) hwrite_list(&(h->q));
+}
+void hwrite_disc_node(disc_t *h)
+{ @+ hwrite_start(); @+hwritef("disc"); @+ hwrite_disc(h); @+hwrite_end();}
+@
+
+\getcode
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(disc_kind,b001): {disc_t h; @+HGET_DISC(b001,h);@+ hwrite_disc(&h); @+} @+break;
+case TAG(disc_kind,b010): {disc_t h; @+HGET_DISC(b010,h);@+ hwrite_disc(&h); @+} @+break;
+case TAG(disc_kind,b011): {disc_t h; @+HGET_DISC(b011,h);@+ hwrite_disc(&h); @+} @+break;
+case TAG(disc_kind,b100): {disc_t h; @+HGET_DISC(b100,h);@+ hwrite_disc(&h); @+} @+break;
+case TAG(disc_kind,b101): {disc_t h; @+HGET_DISC(b101,h);@+ hwrite_disc(&h); @+} @+break;
+case TAG(disc_kind,b110): {disc_t h; @+HGET_DISC(b110,h);@+ hwrite_disc(&h); @+} @+break;
+case TAG(disc_kind,b111): {disc_t h; @+HGET_DISC(b111,h);@+ hwrite_disc(&h); @+} @+break;
+@
+
+@<get macros@>=
+#define @[HGET_DISC(I,Y)@]\
+if ((I)&b100) {uint8_t r=HGET8; (Y).r=r&0x7F; @+ RNG("Replace count",(Y).r,0,31); @+(Y).x=(r&0x80)!=0; @+}\
+ at +else { (Y).r=0; @+ (Y).x=false;@+}\
+if ((I)&b010) hget_list(&((Y).p)); else { (Y).p.p=hpos-hstart; @+(Y).p.s=0; @+(Y).p.k=list_kind; @+}\
+if ((I)&b001) hget_list(&((Y).q)); else { (Y).q.p=hpos-hstart; @+(Y).q.s=0; @+(Y).q.k=list_kind; @+}
+@
+
+@<get functions@>=
+void hget_disc_node(disc_t *h)
+{ @<read the start byte |a|@>@;
+ if (KIND(a)!=disc_kind || INFO(a)==b000)
+ QUIT("Hyphen expected at 0x%x got %s,%d",node_pos,NAME(a),INFO(a));
+ HGET_DISC(INFO(a),*h);
+ @<read and check the end byte |z|@>@;
+}
+@
+
+When |hput_disc| is called, the node is already written to the output,
+but empty lists might have been deleted, and the info value needs to be determined.
+Because the info value |b000| is reserved for references, a zero reference
+count is written to avoid this case.
+\putcode
+@<put functions@>=
+uint8_t hput_disc(disc_t *h)
+{ info_t info=b000;
+ if (h->r!=0) info|=b100;
+ if (h->q.s!=0) info|=b011;
+ else if (h->p.s!=0) info|=b010;
+ if (info==b000) { @+info|=b100; @+HPUT8(0);@+}
+ return TAG(disc_kind,info);
+}
+@
+\subsection{Paragraphs}
+The most important procedure that the \HINT\ viewer inherits from \TeX\ is the
+line breaking routine. If the horizontal size of the paragraph is not known,
+breaking the paragraph\index{paragraph} into lines must be postponed and this is done by creating
+a paragraph node. The paragraph node must contain all information that \TeX's
+line breaking\index{line breaking} algorithm needs to do its job.
+
+Besides the horizontal list describing the content of the paragraph and
+the extended dimension describing the horizontal size,
+this is the set of parameters that guide the line breaking algorithm:
+
+\itemize
+\item
+Integer parameters:\hfill\break
+{\tt pretolerance} (badness tolerance before hyphenation),\hfill\break
+{\tt tolerance} (badness tolerance after hyphenation),\hfill\break
+{\tt line\_penalty} (added to the badness of every line, increase to get fewer lines),\hfill\break
+{\tt hy\-phen\_pe\-nal\-ty} (penalty for break after hyphenation break),\hfill\break
+{\tt ex\_hy\-phen\_pe\-nal\-ty} (penalty for break after explicit\index{explicit} break),\hfill\break
+{\tt doub\-le\_hy\-phen\_de\-merits} (demerits for double hyphen break),\hfill\break
+{\tt final\_hyphen\_de\-me\-rits} (demerits for final hyphen break),\hfill\break
+{\tt adj\_demerits} (demerits for adjacent incompatible lines),\hfill\break
+{\tt looseness} (make the paragraph that many lines longer than its optimal size),\hfill\break
+{\tt inter\_line\_penalty} (additional penalty between lines),\hfill\break
+{\tt club\_pe\-nal\-ty} (penalty for creating a club line),\hfill\break
+{\tt widow\_penalty} (penalty for creating a widow line),\hfill\break
+{\tt display\_widow\_penalty} (ditto, just before a display),\hfill\break
+{\tt bro\-ken\_pe\-nal\-ty} (penalty for breaking a page at a broken line),\hfill\break
+{\tt hang\_af\-ter} (start/end hanging indentation at this line).
+\item
+Dimension parameters:\hfill\break
+{\tt line\_skip\_limit} (threshold for {\tt line\_skip} instead of {\tt base\-line\_skip}),\hfill\break
+{\tt hang\_in\-dent} (amount of hanging indentation),\hfill\break
+{\tt emergency\_stretch} (stretchability added to every line in the final pass of line breaking).
+\item
+Glue parameters:\hfill\break
+{\tt baseline\_skip} (desired glue between baselines),\hfill\break
+{\tt line\_skip} (interline glue if {\tt baseline\_skip} is infeasible),\hfill\break
+{\tt left\_skip} (glue at left of justified lines),\hfill\break
+{\tt right\_skip} (glue at right of justified lines),\hfill\break
+{\tt par\_fill\_skip} (glue on last line of paragraph).
+\enditemize
+
+
+For a detailed explanation of these parameters and how they influence
+line breaking, you should consult the {\TeX}book\cite{DK:texbook};
+\TeX's {\tt parshape} feature is currently not implemented. There are
+default values for all of these parameters (see section~\secref{defaults}),
+and therefore it might not be necessary to specify any of them.
+Any local adjustments are contained in a list of
+parameters contained in the paragraph node.
+
+A further complication arises from displayed\index{displayed formula} formulas
+that interrupt a paragraph. Such displays are described in the next
+section.
+
+To summarize, a paragraph node in the long format specifies an
+extended dimension, a parameter list,
+and a node list. The extended dimension is given either as an
+|xdimen| node (info bit |b100|) or as a reference; similarly the parameter list
+can be embedded in the node (info bit |b010|) or again it is given by a reference.
+
+
+\readcode
+ at s PAR symbol
+ at s par symbol
+ at s xdimen_ref symbol
+ at s param_ref symbol
+ at s par_dimen symbol
+
+
+@<symbols@>=
+%token PAR "par"
+%type <info> par
+@
+
+@<scanning rules@>=
+::@=par@> :< return PAR; >:
+@
+
+
+The following parsing rules are slightly more complicated than I would like them to be, but it seems more important
+to achieve a regular layout of the short format nodes where all sub nodes are located at the end of a node.
+In this case, I want to put a |param_ref| before an |xdimen| node, but otherwise have
+the |xdimen_ref| before a |param_list|.
+The |par_dimen| rule is introduced only to avoid a reduce/reduce conflict in the parser.
+The parsing of |empty_param_list| and |non_empty_param_list| is explained in
+section~\secref{paramlist}.
+
+@<parsing rules@>=
+par_dimen: xdimen { hput_xdimen_node(&($1)); };
+par: xdimen_ref param_ref list {$$=b000;}
+ | xdimen_ref empty_param_list non_empty_param_list list { $$=b010;}
+ | xdimen_ref empty_param_list list { $$=b010;}
+ | xdimen param_ref { hput_xdimen_node(&($1)); } list { $$=b100;}
+ | par_dimen empty_param_list non_empty_param_list list { $$=b110;}
+ | par_dimen empty_param_list list { $$=b110;};
+
+content_node: start PAR par END { hput_tags($1,TAG(par_kind,$3));};
+@
+
+\getcode
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(par_kind,b000): @+HGET_PAR(b000);@+break;
+case TAG(par_kind,b010): @+HGET_PAR(b010);@+break;
+case TAG(par_kind,b100): @+HGET_PAR(b100);@+break;
+case TAG(par_kind,b110): @+HGET_PAR(b110);@+break;
+@
+
+@<get macros@>=
+#define @[HGET_PAR(I)@] @/\
+{ uint8_t n;\
+ if ((I)==b100) {n=HGET8; @+REF(param_kind,n);@+}\
+ if ((I)&b100) {xdimen_t x; @+hget_xdimen_node(&x); @+hwrite_xdimen(&x);@+} else HGET_REF(xdimen_kind);\
+ if ((I)&b010) { list_t l; @+hget_param_list(&l); @+hwrite_param_list(&l); @+} \
+ else if ((I)!=b100) HGET_REF(param_kind)@; else hwrite_ref(n);\
+ { list_t l; @+hget_list(&l);@+ hwrite_list(&l); @+}}
+@
+
+
+\subsection{Mathematics}\index{Mathematics}\index{displayed formula}
+\gdef\subcodetitle{Displayed Math}
+Being able to handle mathematics\index{mathematics} nicely is one
+of the primary features of \TeX\ and
+so you should expect the same from \HINT.
+We start here with the more complex case---displayed equations---and finish with the
+simpler case of mathematical formulas that are part of the normal flow of text.
+
+Displayed equations occur inside a paragraph\index{paragraph}
+node. They interrupt normal processing of the paragraph and the
+paragraph processing is resumed after the display. Positioning of the
+display depends on several parameters, the shape of the paragraph, and
+the length of the last line preceding the display. Displayed formulas
+often feature an equation number which can be placed either left or
+right of the formula. Also the size of the equation number will
+influence the placement of the formula.
+
+In a \HINT\ file, the parameter list is followed by a list of content
+nodes, representing the formula, and an optional horizontal box
+containing the equation number.
+
+In the short format, we use the info bit |b100| to indicate the
+presence of a parameter list (which might be empty---so it's actually the absence of a
+reference to a parameter list); the info bit |b010| to indicate the presence of
+a left equation number; and the info bit |b001| for a right
+equation\index{equation number} number.
+
+In the long format, we use ``{\tt eqno}'' or ``{\tt left eqno}'' to indicate presence and
+placement of the equation number.
+
+\readcode
+ at s MATH symbol
+ at s math symbol
+@<symbols@>=
+%token MATH "math"
+%type <info> math
+@
+
+@<scanning rules@>=
+::@=math@> :< return MATH; >:
+@
+
+@<parsing rules@>=
+math: param_ref list {$$=b000;}
+ | param_ref list hbox_node {$$=b001;}
+ | param_ref hbox_node list {$$=b010;}
+ | empty_param_list list {$$=b100;}
+ | empty_param_list list hbox_node {$$=b101;}
+ | empty_param_list hbox_node list {$$=b110;}
+ | empty_param_list non_empty_param_list list {$$=b100;}
+ | empty_param_list non_empty_param_list list hbox_node {$$=b101;}
+ | empty_param_list non_empty_param_list hbox_node list {$$=b110;};
+
+content_node: start MATH math END @/{ hput_tags($1,TAG(math_kind,$3));};
+@
+
+\getcode
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(math_kind,b000): HGET_MATH(b000); @+ break;
+case TAG(math_kind,b001): HGET_MATH(b001); @+ break;
+case TAG(math_kind,b010): HGET_MATH(b010); @+ break;
+case TAG(math_kind,b100): HGET_MATH(b100); @+ break;
+case TAG(math_kind,b101): HGET_MATH(b101); @+ break;
+case TAG(math_kind,b110): HGET_MATH(b110); @+ break;
+@
+
+@<get macros@>=
+#define @[HGET_MATH(I)@] \
+if ((I)&b100) { list_t l; @+hget_param_list(&l); @+hwrite_param_list(&l); @+} \
+else HGET_REF(param_kind);\
+if ((I)&b010) hget_hbox_node(); \
+{ list_t l; @+hget_list(&l);@+ hwrite_list(&l); @+} \
+if ((I)&b001) hget_hbox_node();
+@
+
+\gdef\subcodetitle{Text Math}
+Things are much simpler if mathematical formulas are embedded in regular text.
+Here it is just necessary to mark the beginning and the end of the formula
+because glue inside a formula is not a possible point for a line break.
+To break the line within a formula you can insert a penalty node.
+
+In the long format, such a simple math node just consists of the keyword ``on''
+or ``off''. In the short format, there are two info values still unassigned:
+we use |b011| for ``off'' and |b111| for ``on''.
+
+
+\readcode
+ at s ON symbol
+ at s OFF symbol
+ at s on_off symbol
+@<symbols@>=
+%token ON "on"
+%token OFF "off"
+%type <i> on_off
+@
+
+@<scanning rules@>=
+::@=on@> :< return ON; >:
+::@=off@> :< return OFF; >:
+@
+
+@<parsing rules@>=
+on_off: ON {$$=1;} | OFF {$$=0;};
+math: on_off { $$=b011|($1<<2); };
+@
+
+\getcode
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(math_kind,b111): hwritef(" on");@+break;
+case TAG(math_kind,b011): hwritef(" off");@+break;
+@
+
+Note that \TeX\ allows math nodes to specify a width using the current value of
+mathsurround. If this width is nonzero, it is equivalent to inserting a
+kern node before the math on node and after the math off node.
+
+\subsection{Adjustments}\label{adjust}
+An adjustment\index{adjustment} occurs only in paragraphs\index{paragraph}.
+When the line breaking\index{line breaking} routine finds an adjustment, it inserts
+the vertical material contained in the adjustment node right after the current line.
+Adjustments simply contain a list node.
+
+\vbox{\readcode\vskip -\baselineskip\putcode}
+ at s ADJUST symbol
+@<symbols@>=
+%token ADJUST "adjust"
+@
+
+@<scanning rules@>=
+::@=adjust@> :< return ADJUST; >:
+@
+
+@<parsing rules@>=
+content_node: start ADJUST list END { hput_tags($1,TAG(adjust_kind,1));};
+@
+
+\vbox{\getcode\vskip -\baselineskip\writecode}
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(adjust_kind,1):@+ { list_t l;@+hget_list(&l); @+ hwrite_list(&l); @+} @+ break;
+@
+
+\subsection{Tables}\index{alignment}
+As long as a table contains no dependencies on \.{hsize} and \.{vsize},
+Hi\TeX\ can expand an alignment into a set of nested horizontal and
+vertical boxes and no special processing is required.
+As long as only the size of the table itself but neither the tabskip
+glues nor the table content depends on \.{hsize} or \.{vsize}, the table
+just needs an outer node of type |hset_kind| or |vset_kind|. If there
+is non aligned material inside the table that depends on \.{hsize} or
+\.{vsize}, a vpack or hpack node is still sufficient.
+
+While it is reasonable to restrict the tabskip glues to be ordinary
+glue values without \.{hsize} or \.{vsize} dependencies, it might be
+desirable to have content in the table that does depend on \.{hsize} or
+\.{vsize}. For the latter case, we need a special kind of table
+node. Here is why:
+
+As soon as the dimension of an item in the table is an extended
+dimension, it is no longer possible to compute the maximum natural with
+of a column, because it is not possible to compare extended dimensions
+without knowing \.{hsize} and \.{vsize}. Hence the computation of maximum
+widths needs to be done in the viewer. After knowing the width of the columns,
+the setting of tabskip glues is easy to compute.
+
+To implement these extended tables, we will need a table node that
+specifies a direction, either horizontal or vertical; a list of
+tabskip glues, with the provision that the last tabskip glue in the
+list is repeated as long as necessary; and a list of table content.
+The table's content is stacked, either vertical or
+horizontal, orthogonal to the alignment direction of the table.
+The table's content consists of nonaligned content, for example extra glue
+or rules, and aligned content.
+Each element of aligned content
+is called an outer item and it consist of a list of inner items.
+For example in a horizontal alignment, each row is an outer item
+and each table entry in that row is an inner item.
+An inner item contains a box node (of kind |hbox_kind|, |vbox_kind|,
+|hset_kind|, |vset_kind|, |hpack_kind|, or |vpack_kind|) followed by
+an optional span count.
+
+The glue of the boxes in the inner items will be reset so that all boxes in the same
+column reach the same maximum column with. The span counts will be replaced by
+the appropriate amount of empty boxes and tabskip glues. Finally the
+glue in the outer item will be set to obtain the desired size
+of the table.
+
+The definitions below specify just a |list| for the list of tabskip glues and a
+list for the outer table items.
+This is just for convenience; the first list must contain glue
+nodes and the second list must contain nonaligned content and inner item nodes.
+
+We reuse the |H| and |V| tokens, defined as part of the specification
+of extended dimensions, to indicate the alignment direction of the
+table. To tell a reference to an extended dimension from a reference
+to an ordinary dimension, we prefix the former with an |XDIMEN| token;
+for the latter, the |DIMEN| token is optional. The scanner will
+recognize not only ``item'' as an |ITEM| token but also ``row'' and
+''column''. This allows a more readable notation, for example by
+marking the outer items as rows and the inner items as columns.
+
+In the short format, the |b010| bit is used to mark a vertical table
+and the |b101| bits indicate how the table size is specified; an outer
+item node has the info value |b000|, an inner item node with info
+value |b111| contains an extra byte for the span count, otherwise the
+info value is equal to the span count.
+
+
+
+
+
+
+\readcode
+ at s TABLE symbol
+ at s ITEM symbol
+ at s table symbol
+ at s span_count symbol
+
+@<symbols@>=
+%token TABLE "table"
+%token ITEM "item"
+%type <info> table span_count
+@
+
+@<scanning rules@>=
+::@=table@> :< return TABLE; >:
+::@=item@> :< return ITEM; >:
+::@=row@> :< return ITEM; >:
+::@=column@> :< return ITEM; >:
+@
+
+@<parsing rules@>=
+span_count: UNSIGNED { $$=hput_span_count($1); };
+content_node: start ITEM content_node END { hput_tags($1,TAG(item_kind,1)); };
+content_node: start ITEM span_count content_node END {@+ hput_tags($1,TAG(item_kind,$3));};
+content_node: start ITEM list END { hput_tags($1,TAG(item_kind,b000));};
+
+table: H box_goal list list {$$=$2;};
+table: V box_goal list list {$$=$2|b010;};
+
+content_node: start TABLE table END { hput_tags($1,TAG(table_kind,$3));};
+@
+
+\getcode
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(table_kind,b000): @+ HGET_TABLE(b000); @+ break;
+case TAG(table_kind,b001): @+ HGET_TABLE(b001); @+ break;
+case TAG(table_kind,b010): @+ HGET_TABLE(b010); @+ break;
+case TAG(table_kind,b011): @+ HGET_TABLE(b011); @+ break;
+case TAG(table_kind,b100): @+ HGET_TABLE(b100); @+ break;
+case TAG(table_kind,b101): @+ HGET_TABLE(b101); @+ break;
+case TAG(table_kind,b110): @+ HGET_TABLE(b110); @+ break;
+case TAG(table_kind,b111): @+ HGET_TABLE(b111); @+ break;@#
+
+case TAG(item_kind,b000): @+{@+ list_t l;@+ hget_list(&l);@+ hwrite_list(&l);@+ } @+ break;
+case TAG(item_kind,b001): hget_content_node(); @+ break;
+case TAG(item_kind,b010): hwritef(" 2");@+hget_content_node(); @+ break;
+case TAG(item_kind,b011): hwritef(" 3");@+hget_content_node(); @+ break;
+case TAG(item_kind,b100): hwritef(" 4");@+hget_content_node(); @+ break;
+case TAG(item_kind,b101): hwritef(" 5");@+hget_content_node(); @+ break;
+case TAG(item_kind,b110): hwritef(" 6");@+hget_content_node(); @+ break;
+case TAG(item_kind,b111): hwritef(" %u",HGET8);@+hget_content_node(); @+ break;
+@
+
+@<get macros@>=
+#define @[HGET_TABLE(I)@] \
+if(I&b010) hwritef(" v"); @+else hwritef(" h"); \
+if ((I)&b001) hwritef(" add");@+ else hwritef(" to");\
+if ((I)&b100) {xdimen_t x; hget_xdimen_node(&x); @+hwrite_xdimen_node(&x);@+} else HGET_REF(xdimen_kind)@;\
+{@+ list_t l; @+hget_list(&l);@+ hwrite_list(&l);@+ } /* tabskip */ \
+{@+ list_t l; @+hget_list(&l);@+ hwrite_list(&l);@+ } /* items */
+@
+
+
+\putcode
+@<put functions@>=
+info_t hput_span_count(uint32_t n)
+{ if (n==0) QUIT("Span count in item must not be zero");
+ else if (n<7) return n;
+ else if (n>0xFF) QUIT("Span count %d must be less than 255",n);
+ else
+ { HPUT8(n); return 7; }
+}
+@
+\section{Extensions to \TeX}\hascode
+
+\subsection{Images}
+Images behave pretty much like glue\index{glue}. They can stretch (or shrink)
+together with the surrounding glue to fill a horizontal or vertical box.
+Like glue, they stretch in the horizontal direction when filling an horizontal box
+and they stretch in the vertical direction as part of a vertical box.
+Stretchability and shrinkability are optional parts of an image node.
+
+Unlike glue, images have both a width and a height.
+The relation of height to width, the aspect ratio, is preserved by stretching and shrinking.
+
+While glue often has a zero width, images usually have a nonzero natural size and making
+them much smaller is undesirable.
+The natural width and height of an image are optional parts of an image node;
+typically this information is contained in the image data.
+
+The only required part of an image node is the number of the auxiliary section
+where the image data can be found.
+
+@<hint types@>=
+typedef struct {@+
+uint16_t n;@+
+dimen_t w,h;@+
+stretch_t p,m;@+
+} image_t;
+@
+
+
+\readcode
+ at s IMAGE symbol
+ at s image symbol
+ at s image_dimen symbol
+@<symbols@>=
+%token IMAGE "image"
+%type <x> image image_dimen
+@
+
+@<scanning rules@>=
+::@=image@> :< return IMAGE; >:
+@
+
+@<parsing rules@>=
+image_dimen: dimension dimension {$$.w=$1; $$.h=$2;} | {$$.w=$$.h=0; };
+image: UNSIGNED image_dimen plus minus { $$.w=$2.w; $$.h=$2.h; $$.p=$3; $$.m=$4; RNG("Section number",$1,3,max_section_no);$$.n=$1; };
+content_node: start IMAGE image END { hput_tags($1,hput_image(&($3)));}
+@
+
+\writecode
+@<write functions@>=
+void hwrite_image(image_t *x)
+{ hwritef(" %u",x->n);
+ if (x->w!=0 ||x->h!=0) { hwrite_dimension(x->w); hwrite_dimension(x->h);@+}
+ hwrite_plus(&x->p);
+ hwrite_minus(&x->m);
+}
+@
+
+\getcode
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(image_kind,b100): @+ { image_t x;@+HGET_IMAGE(b100,x);@+}@+break;
+case TAG(image_kind,b101): @+ { image_t x;@+HGET_IMAGE(b101,x);@+}@+break;
+case TAG(image_kind,b110): @+ { image_t x;@+HGET_IMAGE(b110,x);@+}@+break;
+case TAG(image_kind,b111): @+ { image_t x;@+HGET_IMAGE(b111,x);@+}@+break;
+@
+
+@<get macros@>=
+#define @[HGET_IMAGE(I,X)@] @/\
+HGET16((X).n);RNG("Section number",(X).n,3,max_section_no); \
+if (I&b010) {HGET32((X).w);HGET32((X).h);@+} \
+else (X).w=(X).h=0;\
+if (I&b001) {HGET_STRETCH((X).p);HGET_STRETCH((X).m);@+}\
+else { (X).p.f=(X).m.f=0.0; (X).p.o=(X).m.o=normal_o;@+}\
+hwrite_image(&(X));
+@
+
+
+\putcode
+@<put functions@>=
+uint8_t hput_image(image_t *x)
+{ info_t i=b100;
+ HPUT16(x->n);
+ if (x->w!=0 || x->h!=0)@+ {HPUT32(x->w); HPUT32(x->h); i|=b010;@+ }
+ if (x->p.f!=0.0 || x->m.f!=0.0)@+ {hput_stretch(&x->p); hput_stretch(&x->m); i|=b001;@+ }
+ return TAG(image_kind,i);
+}
+@
+
+
+\subsection{Positions, Outlines, Links, and Labels}\label{labels}
+\index{position}\index{outline}\index{link}\index{label}
+A viewer can usually not display the entire content section of
+a \HINT\ file. Instead it will display a page of content and will give
+its user various means to change the page. This might be as simple as
+a ``page down'' or ``page up'' button (or gesture) and as
+sophisticated as searching using regular expressions. More
+traditional ways to navigate the content include the use of a table of
+content or an index of keywords. All these methods of changing a page
+have in common that a part of the content that fits nicely in the
+screen area provided by the output device must be rendered given a
+position inside the content section.
+
+
+Let's assume that the viewer uses a \HINT\ file in short
+format---after all that's the format designed for precisely this use.
+A position inside the content section is then the position of the
+starting byte of a node. Such a position can be stored as a 32 bit
+number. Because even the smallest node contains two tag bytes,
+the position of any node is strictly smaller than the maximum 32 bit
+number which we can conveniently use as a ``non position''.
+
+@<hint macros@>=
+#define HINT_NO_POS 0xFFFFFFFF
+@
+
+To render a page starting at a given position is not difficult:
+We just read content nodes, starting at the given position and feed
+them to \TeX's page builder until the page is complete. To implement a
+``clickable'' table of content this is good enough. We store with
+every entry in the table of content the position of the section
+header, and when the user clicks the entry, the viewer can display a
+new page starting exactly with that section header.
+
+Things are slightly more complex if we want to implement a ``page
+down'' button. If we press this button, we want the next page to
+start exactly where the current page has ended. This is
+typically in the middle of a paragraph node, and it might even be in
+the middle of an hyphenated word in that paragraph. Fortunately,
+paragraph and table nodes are the only nodes that can be broken across page
+boundaries. But broken paragraph nodes are a common case non the less,
+and unless we want to search for the enclosing node, we need to
+augment in this case the primary 32 bit position inside the content
+section with a secondary position. Most of the
+time, 16 bit will suffice for this secondary position if we give it
+relative to the primary position. Further, if the list of nodes forming the
+paragraph is given as a text, we need to know the current font at the
+secondary position. Of course, the viewer can find it by scanning the
+initial part of the text, but when we think of a page down button, the
+viewer might already know it from rendering the previous page.
+
+Similar is the case of a ``page up'' button. Only here we need a page
+that ends precisely where our current page starts. Possibly even with
+the initial part of a hyphenated word. Here we need a reverse version
+of \TeX's page builder that assembles a ``good'' page from the bottom
+up instead of from the top down. Sure the viewer can cache the start
+position of the previous page (or the rendering of the entire page) if
+the reader has reached the current page using the page down
+button. But this is not possible in all cases. The reader might have
+reached the current page using the table of content or even an index
+or a search form.
+
+This is the most complex case to consider: a link from an index or a
+search form to the position of a keyword in the main text. Let's assume
+someone looks up the word ``M\"unchen''. Should the viewer then
+generate a page that starts in the middle of a sentence with the word
+``M\"unchen''? Probably not! We want a page that shows at least the whole sentence if
+not the whole paragraph. Of course the program that generates the
+link could specify the position of the start of the paragraph instead
+of the position of the word. But that will not solve the problem. Just
+imagine reading the groundbreaking masterpiece of a German philosopher
+on a small hand-held device: the paragraph will most likely be very
+long and perhaps only part of the first sentence will fit on the small
+screen. So the desired keyword might not be found on the page that
+starts with the beginning of the paragraph; it might not even be on
+the next or next to next page. Only the viewer can decide what is the
+best fragment of content to display around the position of the given
+keyword.
+
+To summarize, we need three different ways to render a page for a given position:
+\itemize
+\item A page that starts exactly at the given position.
+\item A page that ends exactly at the given position.
+\item The ``best'' page that contains the given position somewhere in the middle.
+\enditemize
+
+\noindent
+A possible way to find the ``best'' page for the latter case
+could be the following:
+\itemize
+\item If the position is inside a paragraph, break the paragraph
+ into lines. One line will contain
+ the given position. Let's call this the destination line.
+\item If the paragraph will not fit entirely on the page,
+ start the page with the beginning of the
+ paragraph if that will place the destination line on the page, otherwise
+ start with a line in the paragraph that is about half a page
+ before the destination line.
+\item Else traverse the content list backward for about $2/3$ of the
+ page height and forward for about $2/3$ of the page height, searching
+ for the smallest negative penalty node. Use the penalty node found as
+ either the beginning or ending of the page.
+\item If there are several equally low negative penalty nodes. Prefer
+ penalties preceding the destination line over penalty nodes following
+ it. A good page start is more important than a good page end.
+\item If there are are still several equally low negative penalty
+ nodes, choose the one whose distance to the destination line is closest
+ to $1/2$ of the page height.
+\item If no negative penalty nodes could be found, start the page with
+ the paragraph containing the destination line.
+\item Once the page start (or end) is found, use \TeX's page builder
+ (or its reverse variant) to complete the page.
+\enditemize
+
+We call content nodes that reference some position inside the content section
+``link'' nodes. The position that is referenced is called the destination of the link.
+Link nodes occur always in pairs of an ``on'' link
+followed by a corresponding ``off'' link that both reference the same position
+%, the same nesting level, % not sure!
+and no other link nodes between them.
+The content between the two will constitute the visible part of the link.
+
+To encode a position inside the content section that can be used
+as the destination of a link node, an other kind of node is needed which
+we call a ``label''.
+
+Links are not the only way to navigate inside a large
+document. The user interface can also present an ``outline''
+of the document that can be used for navigation.
+An outline node implements an association between a name displayed by the
+user interface of the \HINT\ viewer and the destination position in the \HINT\ document.
+
+It is possible though that outline nodes, link nodes, and label nodes can share
+the same kind-value and we have |outline_kind==link_kind==label_kind|.
+To distinguish an outline node from a label node---both occur
+in the short format definition section---the |b100| info bit is set in an
+outline node.
+
+
+@<get functions@>=
+void hget_outline_or_label_def(info_t i, uint32_t node_pos)
+{ @+if (i&b100)
+ @<get and write an outline node@>@;
+ else
+ @<get and store a label node@>@;
+}
+@
+
+The next thing we need to implement is a new maximum number
+for outline nodes. We store this number in the variable
+|max_outline| and limit it to a 16 bit value.
+
+In the short format, the value of |max_outline| is stored with the
+other maximum values using the kind value |outline_kind==label_kind| and the info
+value |b100| for single byte and |b101| for a two byte value.
+
+\codesection{\getsymbol}{Reading the Short Format}\getindex{1}{7}{Special Maximum Values}
+@<cases of getting special maximum values@>=
+ at t\1\kern1em@>
+case TAG(outline_kind,b100):
+case TAG(outline_kind,b101): max_outline=n;
+ DBG(DBGDEF|DBGLABEL,"max(outline) = %d\n",max_outline); break;
+@
+
+\codesection{\putsymbol}{Writing the Short Format}\putindex{1}{7}{Special Maximum Values}
+@<cases of putting special maximum values@>=
+if (max_outline>-1)
+{ uint32_t pos=hpos++-hstart;
+ DBG(DBGDEF|DBGLABEL,"max(outline) = %d\n",max_outline);
+ hput_tags(pos,TAG(outline_kind,b100|(hput_n(max_outline)-1)));
+}
+@
+
+\codesection{\wrtsymbol}{Writing the Long Format}\wrtindex{1}{7}{Special Maximum Values}
+@<cases of writing special maximum values@>=
+ at t\1\kern1em@>
+case label_kind:
+if (max_ref[label_kind]>-1)@/
+{ hwrite_start();
+ hwritef("label %d",max_ref[label_kind]);
+ hwrite_end();@+
+}
+if (max_outline>-1)@/
+{ hwrite_start();
+ hwritef("outline %d", max_outline);
+ hwrite_end();@+
+}
+break;
+@
+
+\codesection{\redsymbol}{Reading the Long Format}\redindex{1}{7}{Special Maximum Values}
+@<parsing rules@>=
+max_value: OUTLINE UNSIGNED { max_outline=$2;
+ RNG("max outline",max_outline,0, 0xFFFF);
+ DBG(DBGDEF|DBGLABEL,"Setting max outline to %d\n",max_outline);
+ };
+@
+
+After having seen the maximum values, we now explain labels, then links,
+and finally outlines.
+
+
+To store labels, we define a data type |label_t| and an array |labels|
+indexed by the labels reference number.
+
+@<hint basic types@>=
+typedef struct
+{@+ uint32_t pos; /* position */
+ uint8_t where; /* where on the rendered page */
+ bool used; /* label used in a link or an outline */
+ int next; /* reference in a linked list */
+ uint32_t pos0;@+ uint8_t f; /* secondary position */
+} label_t;
+@
+
+The |where| field indicates where the label position
+should be on the rendered page: at the top,
+at the bottom, or somewhere in the middle.
+An undefined label has |where| equal to zero.
+
+@<hint macros@>=
+#define LABEL_UNDEF 0
+#define LABEL_TOP 1
+#define LABEL_BOT 2
+#define LABEL_MID 3
+@
+
+@<common variables@>=
+label_t *labels;
+int first_label=-1;
+@
+The variable |first_label| will be used together with the |next| field of
+a label to construct a linked list of labels.
+
+@<initialize definitions@>=
+if (max_ref[label_kind]>=0)@/
+ ALLOCATE(labels,max_ref[label_kind]+1,label_t);
+@
+
+The implementation of labels has to solve the
+problem of forward links:
+a link node that references a label
+that is not yet defined.
+We solve this problem by
+keeping all labels in the definition section.
+So for every label at least a definition is available
+before we start with the content section and we can fill
+in the position when the label is found.
+If we restrict labels to the definition section and
+do not have an alternative representation, the number of possible references
+is a hard limit on the number of labels in a document.
+Therefore label references are allowed to use 16 bit reference numbers.
+In the short format,
+the |b001| bit indicates a two byte reference number if set, and a one byte
+reference number otherwise.
+
+In the short format, the complete information about a label is in the definition section.
+In the long format, this is not possible because we do not have node positions.
+Therefore we will put label nodes at appropriate points in the content section
+and compute the label position when writing the short format.
+
+\gdef\subcodetitle{Labels}
+\readcode
+ at s LABEL symbol
+ at s BOT symbol
+ at s MID symbol
+ at s placement symbol
+
+@<symbols@>=
+%token LABEL "label"
+%token BOT "bot"
+%token MID "mid"
+%type <i> placement
+@
+
+@<scanning rules@>=
+::@=label@> :< return LABEL; >:
+::@=bot@> :< return BOT; >:
+::@=mid@> :< return MID; >:
+@
+
+A label node specifies the reference number and a placement.
+
+@<parsing rules@>=
+placement: TOP {$$=LABEL_TOP;} | BOT {$$=LABEL_BOT;} | MID {$$=LABEL_MID;} | {$$=LABEL_MID;};
+content_node: START LABEL REFERENCE placement END @|
+ { hset_label($3,$4); @+}
+@
+
+
+After parsing a label, the function |hset_label| is called.
+
+@<put functions@>=
+void hset_label(int n,int w )
+{ label_t *t;
+ REF_RNG(label_kind,n);
+ t=labels+n;@/
+ if (t->where!=LABEL_UNDEF)
+ MESSAGE("Duplicate definition of label %d\n",n);
+ t->where=w;
+ t->pos=hpos-hstart;
+ t->pos0=hpos0-hstart;
+ t->next=first_label; first_label=n;
+}
+@
+
+
+All that can be done by the above function
+is storing the data obtained in the |labels| array.
+The generation of the short format output is
+postponed until the entire content section has been parsed and
+the positions of all labels are known.
+
+One more complication needs to be considered: The |hput_list| function
+is allowed to move lists in the output stream and if positions
+inside the list were recorded in a label, these labels need an
+adjustment. To find out quickly if any labels are affected,
+the |hset_label| function
+constructs a linked list of labels starting with the reference number
+of the most recent label in |first_label| and the
+reference number of the label preceding label |i| in |labels[i].next|.
+Because labels are recorded with increasing positions,
+the list will be sorted with positions decreasing.
+
+@<adjust label positions after moving a list@>=
+{ int i;
+ for (i=first_label;i>=0 && labels[i].pos>=l->p;i=labels[i].next)
+ { DBG(DBGNODE|DBGLABEL,"Moving label *%d by %d\n", i,d);@/
+ labels[i].pos+=d;
+ if (labels[i].pos0>=l->p) labels[i].pos0+=d;
+ }
+}
+@
+
+
+The |hwrite_label| function\label{hwritelabel} is the reverse of the above parsing rule.
+Note that it is different from the
+usual |hwrite_|\dots\ functions. And we will see shortly why that is so.
+
+%see |hwrite_range|
+\writecode
+@<write functions@>=
+void hwrite_label(void) /* called in |hwrite_end| and at the start of a list */
+{@+ while (first_label>=0 && (uint32_t)(hpos-hstart)>=labels[first_label].pos)@/
+ { label_t *t=labels+first_label;
+ DBG(DBGLABEL,"Inserting label *%d\n", first_label);
+ hwrite_start();
+ hwritef("label *%d",first_label);
+ if (t->where==LABEL_TOP) hwritef(" top");
+ else if (t->where==LABEL_BOT) hwritef(" bot");
+ nesting--;hwritec('>'); /* avoid a recursive call to |hwrite_end| */
+ first_label=labels[first_label].next;
+ }
+}
+@
+
+The short format specifies the label positions in the definition section.
+This is not possible in the long format because there are no ``positions''
+in the long format. Therefore long format label nodes must
+be inserted in the content section just before those nodes
+that should come after the label. The function |hwrite_label| is called
+in |hwrite_end|. At that point |hpos| is the position of the next node
+and it can be compared with the positions of the labels taken from
+the definition section.
+Because |hpos| is strictly increasing while reading the content section,
+the comparison can be made efficient by sorting the labels.
+The sorting uses the |next| field in the
+array of |labels| to construct a linked list. After sorting, the value of
+|first_label| is the index of the label with the smallest position;
+and for each |i|, the value of |labels[i].next| is the index of
+the label with the next bigger position. If |labels[i].next| is negative,
+there is no next bigger position.
+Currently a simple insertion sort is used.
+The insertion sort will work well if the labels are already
+mostly in ascending order.
+If we expect lots of labels in random order,
+a more sophisticated sorting algorithm might be appropriate.
+
+
+
+@<write functions@>=
+void hsort_labels(void)
+{ int i;
+ if (max_ref[label_kind]<0)
+ { first_label=-1; return; @+} /* empty list */
+ first_label=max_ref[label_kind];
+ while (first_label>=0 && labels[first_label].where==LABEL_UNDEF)
+ first_label--;
+ if (first_label<0) return; /* no defined labels */
+ labels[first_label].next=-1;
+ DBG(DBGLABEL,"Sorting %d labels\n",first_label+1);
+ for (i=first_label-1; i>=0; i--) /* insert label |i| */
+ if (labels[i].where!=LABEL_UNDEF)@/
+ { uint32_t pos=labels[i].pos;
+ if (labels[first_label].pos >= pos)@/
+ { labels[i].next= first_label; first_label=i;@+ } /* new smallest */
+ else @/
+ { int j;
+ for (j= first_label;
+ labels[j].next>=0 && labels[labels[j].next].pos<pos;
+ j=labels[j].next) continue;
+ labels[i].next=labels[j].next; labels[j].next=i;
+ }
+ }
+}
+@
+
+
+The following code is used to get label information from the
+definition section and store it in the |labels| array.
+The |b010| bit indicates the presence of a secondary position for the label.
+
+\getcode
+@<get and store a label node@>=
+{ label_t *t;
+ int n;
+ if (i&b001) HGET16(n); @+else n=HGET8;
+ REF_RNG(label_kind,n);
+ t=labels+n;
+ if (t->where!=LABEL_UNDEF)
+ DBG(DBGLABEL,"Duplicate definition of label %d at 0x%x\n",n, node_pos);
+ HGET32(t->pos);
+ t->where=HGET8;
+ if (t->where==LABEL_UNDEF || t->where>LABEL_MID)
+ DBG(DBGLABEL,"Label %d where value invalid: %d at 0x%x\n",n,t->where,node_pos);
+ if (i&b010) /* secondary position */
+ { HGET32(t->pos0); t->f=HGET8;@+}
+ else t->pos0=t->pos;
+ DBG(DBGLABEL,"Defining label %d at 0x%x\n",n,t->pos);
+}
+@
+
+
+The function |hput_label| is simply the reverse of the above code.
+
+\putcode
+@<put functions@>=
+uint8_t hput_label(int n, label_t *l)
+{ info_t i=b000;
+ HPUTX(13);
+ if (n>0xFF) {i|=b001; HPUT16(n);@+}@+ else HPUT8(n);
+ HPUT32(l->pos);
+ HPUT8(l->where);
+ if (l->pos!=l->pos0)
+ { i|=b010; HPUT32(l->pos0); HPUT8(l->f); @+}
+ return TAG(label_kind,i);
+}
+@
+
+|hput_label_defs| is called by the parser after the entire content
+section has been processed; it appends the label definitions
+to the definition section.
+%Using the fact that the linked list
+%starting at |first_label| already contains all labels in
+%order of descending position, we could easily output the
+%labels in sorted order and reconstruct the sorting while reading
+%in the labels. The \HINT\ format however does not require
+%label nodes to be sorted and the |hsort_labels| function
+%can not be avoided.
+The outlines are stored after the labels because they reference the labels.
+@<put functions@>=
+extern void hput_definitions_end(void);
+extern uint8_t hput_outline(outline_t *t);
+void hput_label_defs(void)
+{ int n;
+ section_no=1;
+ hstart=dir[1].buffer;
+ hend=hstart+ dir[1].bsize;
+ hpos=hstart+dir[1].size;@/
+ @<output the label definitions@>@;
+ @<output the outline definitions@>@;
+ hput_definitions_end();
+}
+@
+
+@<output the label definitions@>=
+ for (n=0; n<=max_ref[label_kind]; n++)@/
+ { label_t *l=labels+n;
+ uint32_t pos;
+ if (l->used)@/
+ { pos=hpos++-hstart;
+ hput_tags(pos,hput_label(n,l));
+ if (l->where==LABEL_UNDEF)
+ MESSAGE("WARNING: Label *%d is used but not defined\n",n);
+ else
+ DBG(DBGDEF|DBGLABEL,"Label *%d defined 0x%x\n",n,pos);@/
+ }
+ else
+ { if (l->where!=LABEL_UNDEF)
+ { pos=hpos++-hstart;
+ hput_tags(pos,hput_label(n,l));
+ DBG(DBGDEF|DBGLABEL,"Label *%d defined but not used 0x%x\n",n,pos);@/
+ }
+ }
+ }
+@
+
+Links are simpler than labels. They are found only in the
+content section and resemble pretty much what we have seen for other
+content nodes. Let's look at them next.
+When reading a short format link node,
+we use again the |b001| info bit to indicate a 16 bit reference
+number to a label. The |b010| info bit indicates an ``on'' link.
+\gdef\subcodetitle{Links}
+\getcode
+@<get macros@>=
+#define @[HGET_LINK(I)@] @/\
+{ int n; if (I&b001) HGET16(n);@+ else n=HGET8; @+ hwrite_link(n,I&b010); @+}
+@
+
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(link_kind,b000): @+ HGET_LINK(b000);@+ break;
+case TAG(link_kind,b001): @+ HGET_LINK(b001);@+ break;
+case TAG(link_kind,b010): @+ HGET_LINK(b010);@+ break;
+case TAG(link_kind,b011): @+ HGET_LINK(b011);@+ break;
+@
+
+The function |hput_link| will insert the link in the output stream and return
+the appropriate tag.
+
+\putcode
+@<put functions@>=
+uint8_t hput_link(int n, int on)
+{ info_t i;
+ REF_RNG(label_kind,n);
+ labels[n].used=true;
+ if (on) i=b010;@+ else i=b000;
+ if (n>0xFF) { i|=b001; HPUT16(n);@+} @+else HPUT8(n);
+ return TAG(link_kind,i);
+}
+@
+
+\readcode
+ at s LINK symbol
+@<symbols@>=
+%token LINK "link"
+@
+@<scanning rules@>=
+::@=link@> :< return LINK; >:
+@
+
+@<parsing rules@>=
+content_node:start LINK REFERENCE on_off END
+ {@+ hput_tags($1,hput_link($3,$4));@+ };
+@
+
+\writecode
+@<write functions@>=
+void hwrite_link(int n, uint8_t on)
+{ REF_RNG(label_kind,n);
+ if (labels[n].where==LABEL_UNDEF)
+ MESSAGE("WARNING: Link to an undefined label %d\n",n);
+ hwrite_ref(n);
+ if (on) hwritef(" on");
+ else hwritef(" off");
+}
+@
+
+Now we look at the
+outline nodes which are found only in the definition section.
+Every outline node is associated with a label node, giving the position in the
+document, and a unique title that should tell the user
+what to expect when navigating to this position. For example
+an item with the title ``Table of Content'' should navigate
+to the page that shows the table of content.
+The sequence of outline nodes found in the definition section
+gets a tree structure by assigning to each item a depth level.
+
+@<hint types@>=
+typedef struct {@+
+uint8_t *t; /* title */
+int s; /* title size */
+int d; /* depth */
+uint16_t r; /* reference to a label */
+} outline_t;
+@
+
+@<shared put variables@>=
+outline_t *outlines;
+@
+
+@<initialize definitions@>=
+if (max_outline>=0)@/
+ ALLOCATE(outlines,max_outline+1,outline_t);
+@
+
+Child items follow their parent item and have a bigger depth level.
+In the short format, the first item must be a root item, with
+a depth level of 0. Further, if any item has the depth $d$, then the
+item following it must have either the same depth $d$ in which
+case it is a sibling, or the depth $d+1$ in which case it is a child,
+or a depth $d^\prime$ with $0\le d^\prime<d$ in which case it is a sibling
+of the latest ancestor with depth $d^\prime$. Because the depth is
+stored in a single byte, the maximum depth is |0xFF|.
+
+In the long format, the depth assignments are more flexible.
+We allow any signed integer, but insist that the depth
+assignments can be compressed to depth levels for the
+short format using the following algorithm:
+
+@<compress long format depth levels@>=
+n=0;@+
+while (n<=max_outline)
+ n=hcompress_depth(n,0);
+@
+Outline items must be listed in the order
+in which they should be displayed.
+The function |hcompress_depth(n,c)| will compress the subtree starting at
+|n| with root level |d| to a new tree with the same structure
+and root level |c|. It returns the outline number of the
+following subtree.
+
+@<put functions@>=
+int hcompress_depth(int n, int c)
+{ int d=outlines[n].d;
+ if (c>0xFF)
+ QUIT("Outline %d, depth level %d to %d out of range",n,d,c);
+ while (n<=max_outline)
+ if (outlines[n].d==d)
+ outlines[n++].d=c;
+ else if (outlines[n].d>d)
+ n=hcompress_depth(n,c+1);
+ else break;
+ return n;
+}
+@
+
+For an outline node, the |b001| bit indicates a two byte reference to a label.
+There is no reference number for an outline item itself:
+it is never referenced anywhere in an \HINT\ file.
+
+\gdef\subcodetitle{Outlines}
+\vbox{\getcode\vskip -\baselineskip\writecode}
+
+@<get and write an outline node@>=
+ { int r,d;
+ list_t l;
+ static int outline_no=-1;
+ hwrite_start();@+hwritef("outline");
+ ++outline_no;
+ RNG("outline",outline_no, 0, max_outline);
+ if (i&b001) HGET16(r);@+ else r=HGET8;
+ REF_RNG(link_kind,r);
+ if (labels[r].where==LABEL_UNDEF)@/
+ MESSAGE("WARNING: Outline with undefined label %d at 0x%x\n",@|r, node_pos);
+ hwritef(" *%d",r);@/
+ d=HGET8; hwritef(" %d",d);@/
+ hget_list(&l);hwrite_list(&l);@/
+ hwrite_end();
+ }
+@
+
+When parsing an outline definition in the long format,
+we parse the outline title as a |list| which will
+write the representation of the list to the output stream.
+Writing the outline definitions, however, must be postponed
+until the label have found their way into the definition
+section. So we save the list's representation in the
+outline node for later use and remove it again from the
+output stream.
+
+\readcode
+ at s OUTLINE symbol
+
+@<symbols@>=
+%token OUTLINE "outline"
+@
+
+@<scanning rules@>=
+::@=outline@> :< return OUTLINE; >:
+@
+
+@<parsing rules@>=
+def_node: START OUTLINE REFERENCE integer position list END {
+ static int outline_no=-1;
+ $$.k=outline_kind; $$.n=$3;
+ if ($6.s==0) QUIT("Outline with empty title in line %d",yylineno);
+ outline_no++;
+ hset_outline(outline_no,$3,$4,$5);
+ };
+@
+
+@<put functions@>=
+void hset_outline(int m, int r, int d, uint32_t pos)
+{ outline_t *t;
+ RNG("Outline",m,0,max_outline);
+ t=outlines+m;
+ REF_RNG(label_kind,r);
+ t->r=r;
+ t->d=d;
+ t->s=hpos-(hstart+pos);
+ hpos=(hstart+pos);
+ ALLOCATE(t->t,t->s,uint8_t);
+ memmove(t->t,hpos,t->s);
+ labels[r].used=true;
+}
+@
+To output the title, we need to move the list back to the output stream.
+Before doing so, we allocate space (and make sure there is room left for the
+end tag of the outline node), and after doing so, we release
+the memory used to save the title.
+
+@<output the title of outline |*t|@>=
+ memmove(hpos,t->t,t->s);
+ hpos=hpos+t->s;
+ free(t->t);
+@
+
+We output all outline definitions from 0 to |max_outline| and
+check that every one of them has a title. Thereby we make sure
+that in the short format |max_outline| matches the number of
+outline definitions.
+
+\putcode
+@<put functions@>=
+uint8_t hput_outline(outline_t *t)
+{ info_t i=b100;
+ HPUTX(t->s+4);
+ if (t->r>0xFF) {i|=b001; @+HPUT16(t->r);@+} @+else HPUT8(t->r);
+ labels[t->r].used=true;
+ HPUT8(t->d);
+ @<output the title of outline |*t|@>@;
+ return TAG(outline_kind,i);
+}
+@
+
+@<output the outline definitions@>=
+@<compress long format depth levels@>@;
+for (n=0;n<=max_outline;n++)
+{ outline_t *t=outlines+n;
+ uint32_t pos;
+ pos=hpos++-hstart;
+ if (t->s==0 || t->t==NULL)
+ QUIT("Definition of outline %d has an empty title",n);
+ DBG(DBGDEF|DBGLABEL,"Outline *%d defined\n",n);@/
+ hput_tags(pos,hput_outline(t));
+}
+@
+
+\subsection{Colors}
+Colors\index{color} are certainly one of the features you will find in the final \HINT\ file format.
+Here some remarks must suffice.
+
+A \HINT\ viewer must be capable of rendering a page given just any valid
+position inside the content section. Therefore \HINT\ files are stateless;
+there is no need to search for preceding commands that might change a state
+variable.
+As a consequence, we can not just define a ``color change node''.
+Colors could be specified as an optional parameter of a glyph node, but the
+amount of data necessary would be considerable. In texts, on the other hand,
+a color change control code would be possible because we parse texts only in forward
+direction. The current font would then become a current color and font with the appropriate
+changes for positions.
+
+A more attractive alternative would be to specify colored fonts.
+This would require an optional
+color argument for a font. For example one could have a cmr10 font in black as
+font number 3, and a cmr10 font in blue as font number 4. Having 256 different fonts,
+this is definitely a possibility because rarely you would need that many fonts
+or that many colors. If necessary and desired, one could allow 16 bit font numbers
+of overcome the problem.
+
+Background colors could be associated with boxes as an optional parameter.
+
+
+\section{Replacing \TeX's Page Building Process}
+
+\TeX\ uses an output\index{output routine} routine to finalize the page. It uses the accumulated material
+from the page builder, found in {\tt box255}, attaches headers, footers, and floating material
+like figures, tables, and footnotes. The latter material is specified by insert nodes
+while headers and footers are often constructed using mark nodes.
+Running an output routine requires the full power of the \TeX\ engine and will not be
+part of the \HINT\ viewer. Therefore, \HINT\ replaces output routines by page templates\index{template}.
+As \TeX\ can use different output routines for different parts of a book---for example
+the index might use a different output routine than the main body of text---\HINT\
+will allow multiple page templates. To support different output media, the page
+templates will be named and a suitable user interface may offer the user a selection
+of possible page layouts. In this way, the page layout remains in the hands of the
+book designer, and the user has still the opportunity to pick a layout that best fits
+the display device.
+
+\TeX\ uses insertions to describe floating content that is not necessarily displayed
+where it is specified. Three examples may illustrate this:
+\itemize
+\item Footnotes\footnote*{Like this one.} are specified in the middle of the text but are displayed at the
+bottom of the page. Several
+footnotes\index{footnote} on the same page are collected and displayed together. The
+page layout may specify a short rule to separate footnotes from the
+main text, and if there are many short footnotes, it may use two columns
+to display them. In extreme cases, the page layout may demand a long
+footnote to be split and continued on the next page.
+
+\item Illustrations\index{illustration} may be displayed exactly where specified if there is enough
+room on the page, but may move to the top of the page, the bottom of the page,
+the top of next page, or a separate page at the end of the chapter.
+
+\item Margin notes\index{margin note} are displayed in the margin on the same page starting at the top
+of the margin.
+\enditemize
+
+\HINT\ uses page templates and content streams to achieve similar effects.
+But before I describe the page building\index{page building} mechanisms of \HINT, let me summarize \TeX's page builder.
+
+\TeX's page builder ignores leading glue\index{glue}, kern\index{kern}, and penalty\index{penalty} nodes until the first
+box\index{box} or rule\index{rule} is encountered;
+whatsit\index{whatsit node} nodes do not really contribute anything to a page; mark\index{mark node} nodes are recorded for later use.
+Once the first box, rule, or insert\index{insert node} arrives, \TeX\ makes copies of all parameters
+that influence the page building process and uses these copies. These parameters
+are the |page_goal| and the |page_max_depth|. Further, the variables
+|page_total|, |page_shrink|, |page_stretch|, |page_depth|,
+and {\it insert\_pe\-nal\-ties\/} are initialized to zero.
+The top skip\index{top skip} adjustment is made
+when the first box or rule arrives---possibly after an insert.
+
+Now the page builder accumulates material: normal material goes into {\tt box255}\index{box 255} and will change |page_total|, |page_shrink|,
+|page_stretch|, and |page_depth|. The latter is adjusted so that
+is does not exceed |page_max_depth|.
+
+The handling of inserts\index{insert node} is more complex.
+\TeX\ creates an insert class using \.{newinsert}. This reserves a number $n$
+and four registers: {\tt box\hair$n$} for the inserted material, {\tt count\hair$n$} for the
+magnification factor $f$, {\tt dimen\hair$n$} for the maximum size per page $d$, and {\tt skip\hair$n$} for the
+extra space needed on a page if there are any insertions of class $n$.
+
+For example plain \TeX\ allocates $n=254$ for footnotes\index{footnote} and sets
+{\tt count254} to~$1000$, {\tt dimen254} to 8in, and {\tt skip254} to {\tt \BS bigskipamount}.
+
+An insertion node will specify the insertion class $n$, some vertical material,
+its natural height plus depth $x$, a {\it split\-\_top\-\_skip}, a {\it split\-\_max\_depth},
+and a {\it floa\-ting\-\_pe\-nal\-ty}.
+
+
+Now assume that an insert node with subtype 254 arrives at the page builder.
+If this is the first such insert, \TeX\ will decrease the |page_goal|
+by the width of skip254 and adds its stretchability and shrinkability
+to the total stretchability and shrinkability of the page. Later,
+the output routine will add some space and the footnote rule to fill just that
+much space and add just that much shrinkability and stretchability to the page.
+Then \TeX\ will normally add the vertical material in the insert node to
+box254 and decrease the |page_goal| by $x\times f/1000$.
+
+Special processing is required if \TeX\ detects that there is not enough space on
+the current page to accommodate the complete insertion.
+If already a previous insert did not fit on the page, simply the |floating_penalty|
+as given in the insert node is added to the total |insert_penalties|.
+Otherwise \TeX\ will test that the total natural height plus depth of box254
+including $x$ does not exceed the maximum size $d$ and that the
+$|page_total| + |page_depth| + x\times f/1000 - |page_shrink| \le |page_goal|$.
+If one of these tests fails, the current insertion
+is split in such a way as to make the size of the remaining insertions just pass the tests
+just stated.
+
+Whenever a glue node, or penalty node, or a kern node that is followed by glue arrives
+at the page builder, it rates the current position as a possible end of the page based on
+the shrinkability of the page and the difference between |page_total| and |page_goal|.
+As the page fills, the page breaks tend to become better and better until the
+page starts to get overfull and the page breaks get worse and worse until
+they reach the point where they become |awful_bad|. At that point,
+the page builder returns to the best page break found so far and fires up the
+output routine.
+
+Let's look next at the problems that show up when implementing a replacement mechanism for \HINT.
+
+\enumerate
+\item
+An insertion node can not always specify its height $x$ because insertions may contain paragraphs that need
+to be broken in lines and the height of a paragraph depends in some non obvious way on
+its width.
+
+\item
+Before the viewer can compute the height $x$, it needs to know the width of the insertion. Just imagine
+displaying footnotes in two columns or setting notes in the margin. Knowing the width, it
+can pack the vertical material and derive its height and depth.
+
+\item
+\TeX's plain format provides an insert macro that checks whether there is still space
+on the current page, and if so, it creates a contribution to the main text body, otherwise it
+creates a topinsert. Such a decision needs to be postponed to the \HINT\ viewer.
+
+\item
+\HINT\ has no output routines that would specify something like the space and the rule preceding the footnote.
+
+\item
+\TeX's output routines have the ability to inspect the content of the boxes,
+split them, and distribute the content over the page.
+For example, the output routine for an index set in two column format might
+expect a box containing index entries up to a height of $2\times\.{vsize}$.
+It will split this box in the middle and display the top part in the left
+column and the bottom part in the right column. With this approach, the
+last page will show two partly filled columns of about equal size.
+
+\item
+\HINT\ has no mark nodes that could be used to create page headers or footers.
+Marks, like output routines, contain token lists and need the full \TeX\ interpreter
+for processing them. Hence, \HINT\ does not support mark nodes.
+\endenumerate
+
+Here now is the solution I have chosen for \HINT:
+
+Instead of output routines, \HINT\ will use page templates.
+Page templates are basically vertical boxes with placeholders marking the
+positions where the content of the box registers, filled by the page builder,
+should appear.
+To output the page, the viewer traverses the page template,
+replaces the placeholders by the appropriate box content, and
+sets the glue. Inside the page template, we can use insert nodes to act
+as placeholders.
+
+It is only natural to treat the page's main body, the
+inserts, and the marks using the same mechanism. We call this
+mechanism a content stream\index{stream}.
+Content streams are identified by a stream number in the range 0 to 254;
+the number 255 is used to indicate an invalid stream number.
+The stream number 0 is reserved for the main content stream; it is always defined.
+Besides the main content stream, there are three types of streams:
+\itemize
+\item normal streams correspond to \TeX's inserts and accumulate content on the page,
+\item first\index{first stream} streams correspond to \TeX's first marks and will contain only the first insertion of the page,
+\item last\index{last stream} streams correspond to \TeX's bottom marks and will contain only the last insertion of the page, and
+\item top\index{top stream} streams correspond to \TeX's top marks. Top streams are not yet implemented.
+\enditemize
+
+Nodes from the content section are considered contributions to stream 0 except
+for insert nodes which will specify the stream number explicitly.
+If the stream is not defined or is not used in the current page template, its content is simply ignored.
+
+The page builder needs a mechanism to redirect contributions from one content
+stream to another content stream based on the availability of space.
+Hence a \HINT\ content stream can optionally specify a preferred stream number,
+where content should go if there is still space available, a next stream number,
+where content should go if the present stream has no more space available, and
+a split ratio if the content is to be split between these two streams before
+filling in the template.
+
+Various stream parameters govern the treatment of contributions to the stream
+and the page building process.
+
+\itemize
+\item The magnification factor $f$: Inserting a box of height $h$ to this stream will contribute $h\times f/1000$
+to the height of the page under construction. For example, a stream
+that uses a two column format will have an $f$ value of 500; a stream
+that specifies notes that will be displayed in the page margin will
+have an $f$ value of zero.
+
+\item The height $h$: The extended dimension $h$ gives the maximum height this
+stream is allowed to occupy on the current page.
+To continue the previous example, a stream that will be split into two columns
+will have $h=2\cdot\.{vsize}$ , and a stream that specifies
+notes that will be displayed in the page margin will have
+$h=1\cdot\.{vsize}$. You can restrict the amount of space occupied by
+footnotes to the bottom quarter by setting the corresponding $h$ value
+to $h=0.25\cdot\.{vsize}$.
+
+\item The depth $d$: The dimension $d$ gives the maximum depth this
+stream is allowed to have after formatting.
+
+\item The width $w$: The extended dimension $w$ gives the width of this stream
+when formatting its content. For example margin notes
+should have the width of the margin less some surrounding space.
+
+\item The ``before'' list $b$: If there are any contributions to this
+stream on the current page, the material in list $b$
+is inserted {\it before\/} the material from the stream itself. For
+example, the short line that separates the footnotes from the main
+page will go, together with some surrounding space, into the list~$b$.
+
+\item The top skip glue $g$: This glue is inserted between the material
+from list $b$ and the first box of the stream, reduced
+by the height of the first box. Hence it specifies the distance between
+the material in $b$ and the first baseline of the stream content.
+
+\item The ``after'' list $a$: The list $a$ is treated like list $b$ but
+its material is placed {\it after\/} the material from the stream itself.
+
+\item The ``preferred'' stream number $p$: If $p\ne 255$, it is the number of
+the {\it preferred\/} stream. If stream $p$ has still
+enough room to accommodate the current contribution, move the
+contribution to stream $p$, otherwise keep it. For example, you can
+move an illustration to the main content stream, provided there is
+still enough space for it on the current page, by setting $p=0$.
+
+\item The ``next'' stream number $n$: If $n\ne 255$, it is the number of the
+{\it next\/} stream. If a contribution can not be
+accommodated in stream $p$ nor in the current stream, treat it as an
+insertion to stream $n$. For example, you can move contributions to
+the next column after the first column is full, or move illustrations
+to a separate page at the end of the chapter.
+
+\item The split ratio\index{split ratio} $r$: If $r$ is positive, both $p$ and $n$ must
+be valid stream numbers and contents is not immediately moved to stream $p$ or $n$ as described before.
+Instead the content is kept in the stream itself until the current page is complete.
+Then, before inserting the streams into the page template, the content of
+this stream is formatted as a vertical box, the vertical box is
+split into a top fraction and a bottom fraction in the ratio $r/1000$
+for the top and $(1000-r)/1000$ for the bottom, and finally the top
+fraction is moved to stream $p$ and the bottom fraction to stream
+$n$. You can use this feature for example to implement footnotes
+arranged in two columns of about equal size. By collecting all the
+footnotes in one stream and then splitting the footnotes with $r=500$
+before placing them on the page into a right and left column. Even
+three or more columns can be implemented by cascades of streams using
+this mechanism.
+\enditemize
+
+\subsection{Stream Definitions}
+\index{stream}
+There are four types of streams: normal streams that work like \TeX's inserts;
+and first, last, and top streams that work like \TeX's marks.
+For the latter types, the long format uses a matching keyword and the
+short format the two least significant info bits. All stream definitions
+start with the stream number.
+In definitions of normal streams after the number follows in this order
+\itemize
+\item the maximum insertion height,
+\item the magnification factor, and
+\item information about splitting the stream.
+ It consists of: a preferred stream, a next stream, and a split ratio.
+ An asterisk indicates a missing stream reference, in the
+ short format the stream number 255 serves the same purpose.
+\enditemize
+All stream definitions finish with
+\itemize
+\item the ``before'' list,
+\item an extended dimension node specifying the width of the inserted material,
+\item the top skip glue,
+\item the ``after'' list,
+\item and the total height, stretchability, and shrinkability of the material in
+ the ``before'' and ``after'' list.
+\enditemize
+
+A special case is the stream definition for stream 0, the main content stream.
+None of the above information is necessary for it so it is omitted.
+Stream definitions, including the definition of stream 0,
+occur only inside page template definitions\index{template}
+where they occur twice in two different roles:
+In the stream definition list, they define properties of the stream
+and in the template they mark the insertion point (see section~\secref{page}).
+In the latter case, stream nodes just contain the stream number.
+Because a template looks like ordinary vertical material,
+we like to use the same functions for parsing it.
+But stream definitions are very different from stream content
+nodes. To solve the problem for the long format,
+the scanner will return two different tokens
+when it sees the keyword ``{\tt stream}''.
+In the definition section, it will return
+|STREAMDEF| and in the content section |STREAM|.
+The same problem is solved in the short format
+by using the |b100| bit to mark a definition.
+
+\goodbreak
+\vbox{\readcode\vskip -\baselineskip\putcode}
+
+ at s STREAM symbol
+ at s STREAMDEF symbol
+ at s TOP symbol
+ at s FIRST symbol
+ at s LAST symbol
+ at s NOREFERENCE symbol
+ at s stream_type symbol
+ at s stream_info symbol
+ at s stream_split symbol
+ at s stream_link symbol
+ at s stream_def_node symbol
+ at s stream_ins_node symbol
+ at s stream_ref symbol
+
+
+@<symbols@>=
+%token STREAM "stream"
+%token STREAMDEF "stream (definition)"
+%token FIRST "first"
+%token LAST "last"
+%token TOP "top"
+%token NOREFERENCE "*"
+%type <info> stream_type
+%type <u> stream_ref
+%type <rf> stream_def_node
+@
+
+@<scanning rules@>=
+::@=stream@> :< if (section_no==1) return STREAMDEF; else return STREAM;@+ >:
+::@=first@> :< return FIRST; >:
+::@=last@> :< return LAST; >:
+::@=top@> :< return TOP; >:
+::@=\*@> :< return NOREFERENCE; >:
+@
+
+@<parsing rules@>=
+stream_link: ref { REF_RNG(stream_kind,$1); } | NOREFERENCE {HPUT8(255);};
+stream_split: stream_link stream_link UNSIGNED @/{RNG("split ratio",$3,0,1000); HPUT16($3);};
+stream_info: xdimen_node UNSIGNED @/{RNG("magnification factor",$2,0,1000); HPUT16($2);} stream_split;
+
+stream_type: stream_info {$$=0;} |FIRST {$$=1;} @+ | LAST {$$=2;} @+ |TOP {$$=3;} ;
+
+stream_def_node: start STREAMDEF ref stream_type @/
+ list xdimen_node glue_node list glue_node END @/
+ {@+ DEF($$,stream_kind,$3); @+ hput_tags($1,TAG(stream_kind,$4|b100));};
+
+stream_ins_node: start STREAMDEF ref END@/
+ { RNG("Stream insertion",$3,0,max_ref[stream_kind]); hput_tags($1,TAG(stream_kind,b100));};
+
+content_node: stream_def_node @+ | stream_ins_node;
+@
+
+
+\goodbreak
+\vbox{\getcode\vskip -\baselineskip\writecode}
+
+
+
+@<get stream information for normal streams@>=
+{ xdimen_t x;
+ uint16_t f,r;
+ uint8_t n;
+ DBG(DBGDEF,"Defining normal stream %d at " SIZE_F "\n",*(hpos-1),hpos-hstart-2);
+ hget_xdimen_node(&x); @+hwrite_xdimen_node(&x);
+ HGET16(f); @+RNG("magnification factor",f,0,1000);@+ hwritef(" %d",f);
+ n=HGET8; if (n==255) hwritef(" *"); else { REF_RNG(stream_kind,n);@+hwrite_ref(n);@+}
+ n=HGET8; if (n==255) hwritef(" *"); else { REF_RNG(stream_kind,n);@+hwrite_ref(n);@+}
+ HGET16(r); RNG("split ratio",r,0,1000); hwritef(" %d",r);
+}
+@
+
+@<get functions@>=
+static bool hget_stream_def(void)
+{@+ if (KIND(*hpos)!=stream_kind || !(INFO(*hpos)&b100))
+ return false;
+ else
+ { ref_t df;
+ @<read the start byte |a|@>@;
+ DBG(DBGDEF,"Defining stream %d at " SIZE_F "\n",*hpos,hpos-hstart-1);
+ DEF(df,stream_kind,HGET8);
+ hwrite_start();@+hwritef("stream");@+ at +hwrite_ref(df.n);
+ if (df.n>0)
+ { xdimen_t x; @+ list_t l;
+ if (INFO(a)==b100) @<get stream information for normal streams@>@;
+ else if (INFO(a)==b101) hwritef(" first");
+ else if(INFO(a)==b110) hwritef(" last");
+ else if (INFO(a)==b111) hwritef(" top");
+ hget_list(&l);@+ hwrite_list(&l);
+ hget_xdimen_node(&x); @+hwrite_xdimen_node(&x);
+ hget_glue_node();@+
+ hget_list(&l);@+ hwrite_list(&l);@+
+ hget_glue_node();
+ }
+ @<read and check the end byte |z|@>@;
+ hwrite_end();
+ return true;
+ }
+}
+
+@
+
+When stream definitions are part of the page template, we call them
+stream insertion points.
+They contain only the stream reference and
+are parsed by the usual content parsing functions.
+
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(stream_kind,b100): {uint8_t n=HGET8;@+ REF_RNG(stream_kind,n); @+hwrite_ref(n); @+ break; @+}
+@
+
+
+\subsection{Stream Content}
+Stream\index{stream} nodes occur in the content section where they
+must not be inside other nodes except toplevel
+paragraph\index{paragraph} nodes. A normal stream node contains in this
+order: the stream reference number, the optional stream parameters,
+and the stream content. The content is either a vertical box or an
+extended vertical box. The stream parameters consists of the
+|floating_penalty|, the |split_max_depth|, and the
+|split_top_skip|. The parameterlist can be given
+explicitly or as a reference.
+
+In the short format, the info bits |b010| indicate
+a normal stream content node with an explicit parameter list
+and the info bits |b000| a normal stream with a parameter list reference.
+
+If the info bit |b001| is set, we have a content node of type top, first,
+or last. In this case, the short format has instead of the parameter list
+a single byte indicating the type.
+These types are currently not yet implemented.
+
+\goodbreak
+\vbox{\readcode\vskip -\baselineskip\putcode}
+
+ at s stream symbol
+
+@<symbols@>=
+%type <info> stream
+@
+
+@<parsing rules@>=
+stream: empty_param_list list {$$=b010;}
+ | empty_param_list non_empty_param_list list {$$=b010;}
+ | param_ref list {$$=b000;};
+content_node: start STREAM stream_ref stream END
+ @/{@+hput_tags($1,TAG(stream_kind,$4)); @+};
+@
+
+\goodbreak
+\vbox{\getcode\vskip -\baselineskip\writecode}
+
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(stream_kind,b000): HGET_STREAM(b000); @+ break;
+case TAG(stream_kind,b010): HGET_STREAM(b010); @+ break;
+@
+
+When we read stream numbers, we relax the define before use policy.
+We just check, that the stream number is in the correct range.
+\goodbreak
+@<get macros@>=
+#define @[HGET_STREAM(I)@] @/\
+ {uint8_t n=HGET8;@+ REF_RNG(stream_kind,n); @+hwrite_ref(n);@+}\
+if ((I)&b010) { list_t l; @+hget_param_list(&l); @+hwrite_param_list(&l); @+} \
+else HGET_REF(param_kind);\
+{ list_t l; @+hget_list(&l);@+ hwrite_list(&l); @+}
+@
+
+
+
+
+\subsection{Page Template Definitions}\label{page}
+A \HINT\ file can define multiple page templates\index{template}. Not only
+might an index demand a different page layout than the main body of text,
+also the front page or the chapter headings might use their own page templates.
+Further, the author of a \HINT\ file might define a two column format as
+an alternative to a single column format to be used if the display area
+is wide enough.
+
+To help in selecting the right page template, page template definitions start with
+a name and an optional priority\index{priority}; the default priority is 1.
+The names might appear in a menu from which the user
+can select a page layout that best fits her taste.
+Without user interaction, the
+system can pick the template with the highest priority. Of course,
+a user interface might provide means to alter priorities. Future
+versions might include sophisticated feature-vectors that
+identify templates that are good for large or small displays,
+landscape or portrait mode, etc \dots
+
+After the priority follows a glue node to specify the topskip glue
+and the dimension of the maximum page depth,
+an extended dimension to specify the page height and
+an extended dimension to specify the page width.
+
+Then follows the main part of a page template definition: the template.
+The template consists of a list of vertical material.
+To construct the page, this list will be placed
+into a vertical box and the glue will be set.
+But of course before doing so, the viewer will
+scan the list and replace all stream insertion points
+by the appropriate content streams.
+
+Let's call the vertical box obtained this way ``the page''.
+The page will fill the entire display area top to bottom and left to right.
+It defines not only the appearance of the main body of text
+but also the margins, the header, and the footer.
+Because the \.{vsize} and \.{hsize} variables of \TeX\ are used for
+the vertical and horizontal dimension of the main body of text---they
+do not include the margins---the page will usually be wider than \.{hsize}
+and taller than \.{vsize}. The dimensions of the page are part
+of the page template. The viewer, knowing the actual dimensions
+of the display area, can derive from them the actual values of \.{hsize}
+and \.{vsize}.
+
+Stream definitions are listed after the template.
+
+The page template with number 0 is always defined and has priority 0.
+It will display just the main content stream. It puts a small margin
+of $\.{hsize}/8 -4.5\hbox{pt}$ all around it.
+Given a letter size page, 8.5 inch wide, this formula yields a margin of 1 inch,
+matching \TeX's plain format. The margin will be positive as long as
+the page is wider than $1/2$ inch. For narrower pages, there will be no
+margin at all. In general, the \HINT\ viewer will never set {\tt hsize} larger
+than the width of the page and {\tt vsize} larger than its height.
+
+%8.5 in should give 1 inch margin 2/17
+%612pt should give 72pt margin
+%72pt = 612/8-4.5pt
+%This would give a positive margin starting at 36pt or 1/2 inch
+
+\goodbreak
+\vbox{\readcode\vskip -\baselineskip\putcode}
+
+ at s PAGE symbol
+ at s page_priority symbol
+ at s page symbol
+ at s stream_def_list symbol
+
+@<symbols@>=
+%token PAGE "page"
+@
+
+@<scanning rules@>=
+::@=page@> :< return PAGE; >:
+@
+
+@<parsing rules@>=
+page_priority: { HPUT8(1); }
+ | UNSIGNED { RNG("page priority",$1,0,255); HPUT8($1); };
+
+stream_def_list: | stream_def_list stream_def_node;
+
+page: string { hput_string($1);} page_priority glue_node dimension {@+HPUT32($5);@+}
+ xdimen_node xdimen_node
+ list stream_def_list ;
+@
+
+\goodbreak
+\vbox{\getcode\vskip -\baselineskip\writecode}
+@<get functions@>=
+void hget_page(void)
+{ char *n; uint8_t p; xdimen_t x; list_t l;
+ HGET_STRING(n);@+ hwrite_string(n);
+ p=HGET8; @+ if (p!=1) hwritef(" %d",p);
+ hget_glue_node();
+ hget_dimen(TAG(dimen_kind,b001));
+ hget_xdimen_node(&x); @+hwrite_xdimen_node(&x); /* page height */
+ hget_xdimen_node(&x); @+hwrite_xdimen_node(&x); /* page width */
+ hget_list(&l);@+ hwrite_list(&l);
+ while (hget_stream_def()) continue;
+}
+@
+
+\subsection{Page Ranges}\label{range}\index{page range}
+Not every template\index{template} is necessarily valid for the entire content
+section. A page range specifies a start position $a$ and an end
+position $b$ in the content section and the page template is valid if
+the start position $p$ of the page is within that range: $a\le p < b$.
+If paging backward this definition might cause problems because the
+start position of the page is known only after the page has been
+build. In this case, the viewer might choose a page template based on
+the position at the bottom of the page. If it turns out that this ``bottom template''
+is no longer valid when the page builder has found the start of the
+page, the viewer might display the page anyway with the bottom
+template, it might just display the page with the new ``top
+template'', or rerun the whole page building process using this time
+the ``top template''. Neither of these alternatives is guaranteed to
+produce a perfect result because changing the page template might
+change the amount of material that fits on the page. A good page
+template design should take this into account.
+
+The representation of page ranges differs significantly for the short
+format and the long format. The short format will include a list of page
+ranges in the definition section which consist of a page template number,
+a start position, and an end position. In the long format, the start
+and end position of a page
+range is marked with a page range node switching the availability of a
+page template on and off. Such a page range node must be a top level node.
+It is an error, to switch a page template
+off that was not switched on, or to switch a page template on that was
+already switched on. It is permissible to omit switching off a page
+template at the very end of the content section.
+
+While we parse a long format \HINT\ file, we store page ranges and generate
+the short format after reaching the end of the content section.
+While we parse a short format \HINT\ file,
+we check at the end of each top level node whether we should insert a
+page range node into the output.
+For the \.{shrink} program, it is best
+to store the start and end positions of all page ranges
+in an array sorted by the position\footnote*{For a \HINT\ viewer,
+a data structure which allows fast retrieval of all
+valid page templates for a given position is needed.}.
+To check the restrictions on the switching of page templates, we
+maintain for every page template an index into the range array
+which identifies the position where the template was switched on.
+A zero value instead of an index will identify templates that
+are currently invalid. When switching a range off again, we
+link the two array entries using this index. These links
+are useful when producing the range nodes in short format.
+
+A range node in short format contains the template number, the
+start position and the end position.
+A zero start position
+is not stored, the info bit |b100| indicates a nonzero start position.
+An end position equal to |HINT_NO_POS| is not stored,
+the info bit |b010| indicates a smaller end position.
+The info bit |b001| indicates that positions are stored using 2 byte
+otherwise 4 byte are used for the positions.
+
+@<hint types@>=
+typedef
+struct {@+uint8_t pg; @+uint32_t pos; @+ bool on; @+int link;@+} range_pos_t;
+@
+
+@<common variables@>=
+range_pos_t *range_pos;
+int next_range=1, max_range;
+int *page_on;
+@
+
+@<initialize definitions@>=
+ALLOCATE(page_on,max_ref[page_kind]+1,int);
+ALLOCATE(range_pos,2*(max_ref[range_kind]+1),range_pos_t);
+@
+
+@<hint macros@>=
+#define @[ALLOCATE(R,S,T)@] @/((R)=@[(T *)calloc((S),sizeof(T)),\
+ (((R)==NULL)?QUIT("Out of memory for " #R):0))
+#define @[REALLOCATE(R,S,T)@] @/((R)=@[(T *)realloc((R),(S)*sizeof(T)),\
+ (((R)==NULL)?QUIT("Out of memory for " #R):0))
+@
+
+\readcode
+ at s RANGE symbol
+@<symbols@>=
+%token RANGE "range"
+@
+
+@<scanning rules@>=
+::@=range@> :< return RANGE; >:
+@
+@<parsing rules@>=
+
+content_node: START RANGE REFERENCE ON END @/{ REF(page_kind,$3);hput_range($3,true);}
+ | START RANGE REFERENCE OFF END @/{ REF(page_kind,$3);hput_range($3,false);};
+@
+
+
+\writecode
+@<write functions@>=
+void hwrite_range(void) /* called in |hwrite_end| */
+{ uint32_t p=hpos-hstart;
+ DBG(DBGRANGE,"Range check at pos 0x%x next at 0x%x\n",p,range_pos[next_range].pos);
+ while (next_range<max_range && range_pos[next_range].pos <= p)
+ { hwrite_start();
+ hwritef("range *%d ",range_pos[next_range].pg);
+ if (range_pos[next_range].on) hwritef("on"); else hwritef("off");
+ nesting--; @+hwritec('>'); /* avoid a recursive call to |hwrite_end| */
+ next_range++;
+ }
+}
+@
+
+\getcode
+@<get functions@>=
+void hget_range(info_t info, uint8_t pg)
+{ uint32_t from, to;
+ REF(page_kind,pg);
+ REF(range_kind,(next_range-1)/2);
+ if (info&b100) @+
+ { @+ if (info&b001) HGET32(from); @+else HGET16(from); @+}
+ else from=0;
+ if (info&b010) @+
+ { @+if (info&b001) HGET32(to); @+else HGET16(to); @+}
+ else to=HINT_NO_POS;
+ range_pos[next_range].pg=pg;
+ range_pos[next_range].on=true;
+ range_pos[next_range].pos=from;
+ DBG(DBGRANGE,"Range *%d from 0x%x\n",pg,from);
+ DBG(DBGRANGE,"Range *%d to 0x%x\n",pg,to);
+ next_range++;
+ if (to!=HINT_NO_POS) @/
+ { range_pos[next_range].pg=pg;
+ range_pos[next_range].on=false;
+ range_pos[next_range].pos=to;
+ next_range++;
+ }
+}
+@
+
+@<write functions@>=
+void hsort_ranges(void) /* simple insert sort by position */
+{ int i;
+ DBG(DBGRANGE,"Range sorting %d positions\n",next_range-1);
+ for(i=3; i<next_range; i++)@/
+ { int j = i-1;
+ if (range_pos[i].pos < range_pos[j].pos) @/
+ { range_pos_t t;
+ t= range_pos[i];
+ do {
+ range_pos[j+1] = range_pos[j];
+ j--;
+ } while (range_pos[i].pos < range_pos[j].pos);
+ range_pos[j+1] = t;
+ }
+ }
+ max_range=next_range; @+next_range=1; /* prepare for |hwrite_range| */
+}
+@
+
+\putcode
+@<put functions@>=
+void hput_range(uint8_t pg, bool on)
+{ if (((next_range-1)/2)>max_ref[range_kind])
+ QUIT("Page range %d > %d",(next_range-1)/2,max_ref[range_kind]);
+ if (on && page_on[pg]!=0)
+ QUIT(@["Template %d is switched on at 0x%x and " SIZE_F@],@|
+ pg, range_pos[page_on[pg]].pos, hpos-hstart);
+ else if (!on && page_on[pg]==0)
+ QUIT(@["Template %d is switched off at " SIZE_F " but was not on"@],@|
+ pg, hpos-hstart);
+ DBG(DBGRANGE,@["Range *%d %s at " SIZE_F "\n"@],pg,on?"on":"off",hpos-hstart);
+ range_pos[next_range].pg=pg;
+ range_pos[next_range].pos=hpos-hstart;
+ range_pos[next_range].on=on;
+ if (on) page_on[pg]=next_range;
+ else @/{ range_pos[next_range].link =page_on[pg];
+ range_pos[page_on[pg]].link=next_range;
+ page_on[pg]=0; }
+ next_range++;
+}
+
+void hput_range_defs(void)
+{ int i;
+ section_no=1;
+ hstart=dir[1].buffer;
+ hend=hstart+ dir[1].bsize;
+ hpos=hstart+dir[1].size;
+ for (i=1; i< next_range;i++)
+ if (range_pos[i].on)@/
+ { info_t info=b000;
+ uint32_t p=hpos++-hstart;
+ uint32_t from, to;
+ HPUT8(range_pos[i].pg);
+ from= range_pos[i].pos;
+ if (range_pos[i].link!=0) to = range_pos[range_pos[i].link].pos;
+ else to=HINT_NO_POS;
+ if (from!=0) @/
+ { info=info|b100;@+ if (from>0xFFFF) @+ info=info|b001;@+}
+ if (to!=HINT_NO_POS) @/
+ { info=info|b010;@+ if (to>0xFFFF) info=info|b001;@+ }
+ if (info & b100) @/
+ { @+if (info & b001) HPUT32(from); @+else HPUT16(from); @+}
+ if (info & b010) @/
+ { @+if (info & b001) HPUT32(to); @+else HPUT16(to); @+}
+ DBG(DBGRANGE,"Range *%d from 0x%x to 0x%x\n",@|range_pos[i].pg,from, to);
+ hput_tags(p,TAG(range_kind,info));
+ }
+ hput_definitions_end();
+ }
+@
+
+
+\section{File Structure}\hascode
+All \HINT\ files\index{file} start with a banner\index{banner} as
+described below. After that, they contain three mandatory
+sections\index{section}: the directory\index{directory section}
+section, the definition\index{definition section} section, and the
+content\index{content section} section. Usually, further
+optional\index{optional section} sections follow. In short format
+files, these contain auxiliary\index{auxiliary file} files
+(fonts\index{font}, images\index{image}, \dots) necessary for
+rendering the content. In long format files, the directory section
+will simply list the file names of the auxiliary files.
+
+
+
+\subsection{Banner}
+All \HINT\ files start with a banner\index{banner}. The banner contains only
+printable ASCII characters and spaces;
+its end is marked with a newline character\index{newline character}.
+The first four byte are the ``magic'' number by which you recognize a \HINT\
+file. It consists of the four ASCII codes `{\tt H}', `{\tt I}', `{\tt N}',
+and `{\tt T}' in the long format and `{\tt h}', `{\tt i}', `{\tt n}',
+and `{\tt t}' in the short format. Then follows a space, then
+the version number, a dot, the sub-version number, and another
+space. Both numbers are encoded as decimal ASCII strings. The
+remainder of the banner is simply ignored but may be used to contain
+other useful information about the file. The maximum size of the
+banner is 256 byte.
+@<hint macros@>=
+#define HINT_VERSION 1
+#define HINT_SUB_VERSION 3
+#define MAX_BANNER 256
+@
+
+\goodbreak
+To check the banner, we have the function |hcheck_banner|;
+it returns |true| if successful.
+
+@<common variables@>=
+char hbanner[MAX_BANNER+1];
+@
+
+@<function to check the banner@>=
+
+bool hcheck_banner(char *magic)
+{ int hbanner_size=0;
+ int v;
+ char *t;
+ t=hbanner;
+ if (strncmp(magic,hbanner,4)!=0)
+ { MESSAGE("This is not a %s file",magic); return false; }
+ else t+=4;
+ hbanner_size=(int)strnlen(hbanner,MAX_BANNER);
+ if(hbanner[hbanner_size-1]!='\n')
+ { MESSAGE("Banner exceeds maximum size=0x%x",MAX_BANNER); return false; }
+ if (*t!=' ')
+ { MESSAGE("Space expected in banner after %s",magic); return false; }
+ else t++;
+ v=strtol(t,&t,10);
+ if (v!=HINT_VERSION)
+ { MESSAGE("Wrong HINT version: got %d, expected %d",v,HINT_VERSION); return false; }
+ if (*t!='.')
+ { MESSAGE("Dot expected in banner after HINT version number"); return false; }
+ else t++;
+ v=strtol(t,&t,10);
+ if (v!=HINT_SUB_VERSION)
+ { MESSAGE("Wrong HINT subversion: got %d, expected %d",v,HINT_SUB_VERSION); return false; }
+ if (*t!=' ' && *t!='\n')
+ { MESSAGE("Space expected in banner after HINT subversion"); return false; }
+ LOG("%s file version %d.%d:%s",magic,HINT_VERSION, HINT_SUB_VERSION, t);
+ DBG(DBGDIR,"banner size=0x%x\n",hbanner_size);
+ return true;
+}
+@
+
+To read a short format file, we use the macro |HGET8|. It returns a single byte.
+We read the banner knowing that it ends with a newline character
+and is at most |MAX_BANNER| byte long. Because this is the first access to a yet unknown file,
+we are very careful and make sure we do not read past the end of the file.
+Checking the banner is a separate step.
+
+\getcode
+@<get file functions@>=
+void hget_banner(void)
+{ int i;
+ for (i=0;i<MAX_BANNER && hpos<hend;i++)@/
+ { hbanner[i]=HGET8;
+ if (hbanner[i]=='\n') break;
+ }
+ hbanner[++i]=0;
+}
+@
+
+To read a long format file, we use the function |fgetc|.
+\readcode
+@<read the banner@>=
+{ int i,c;
+ for (i=0;i<MAX_BANNER && (c=fgetc(hin))!=EOF;i++)@/
+ { hbanner[i]=c;
+ if (hbanner[i]=='\n') break;
+ }
+ hbanner[++i]=0;
+}
+@
+
+Writing the banner to a short format file is accomplished by calling
+|hput_banner| with the ``magic'' string |"hint"| as a first argument
+and a (short) comment as the second argument.
+\putcode
+@<function to write the banner@>=
+
+static size_t hput_banner(char *magic, char *s)
+{ return fprintf(hout,"%s %d.%d %s\n",magic,HINT_VERSION,HINT_SUB_VERSION,s);
+}
+@
+
+
+\writecode
+Writing the banner of a long format file is essentially the same as for a short
+format file calling |hput_banner| with |"HINT"| as a first argument.
+
+\subsection{Long Format Files}\gdef\subcodetitle{Banner}%
+
+After reading and checking the banner, reading a long format file is
+simply done by calling |yyparse|. The following rule gives the big picture:
+\readcode
+ at s hint symbol
+ at s content_section symbol
+
+@<parsing rules@>=
+hint: directory_section definition_section content_section ;
+@
+
+
+\subsection{Short Format Files}\gdef\subcodetitle{Primitives}%
+A short format\index{short format} file starts with the banner and continues
+with a list of sections. Each section has a maximum size
+of $2^{32}$ byte or 4GByte. This restriction ensures that positions\index{position}
+inside a section can be stored as 32 bit integers, a feature that
+we will need only for the so called ``content'' section, but it
+is also nice for implementers to know in advance what sizes to expect.
+The big picture is captured by the |put_hint| function:
+
+@<put functions@>=
+static size_t hput_root(void);
+static size_t hput_section(uint16_t n);
+static void hput_optional_sections(void);
+
+void hput_hint(char * str)
+{ size_t s;
+ DBG(DBGBASIC,"Writing hint output %s\n",str);
+ s=hput_banner("hint",str);
+ DBG(DBGDIR,@["Root entry at " SIZE_F "\n"@],s);
+ s+=hput_root();
+ DBG(DBGDIR,@["Directory section at " SIZE_F "\n"@],s);
+ s+=hput_section(0);
+ DBG(DBGDIR,@["Definition section at " SIZE_F "\n"@],s);
+ s+=hput_section(1);
+ DBG(DBGDIR,@["Content section at " SIZE_F "\n"@],s);
+ s+=hput_section(2);
+ DBG(DBGDIR,@["Auxiliary sections at " SIZE_F "\n"@],s);
+ hput_optional_sections();
+}
+@
+
+
+When we work on a section, we will have the entire section in
+memory and use three variables to access it: |hstart|
+points to the first byte of the section, |hend| points
+to the byte after the last byte of the section, and |hpos| points to the
+current position inside the section.\label{hpos}
+The auxiliary variable |hpos0| contains the |hpos| value of the
+last content node on nesting level zero.
+
+@<common variables@>=
+uint8_t *hpos=NULL, *hstart=NULL, *hend=NULL, *hpos0=NULL;
+@
+
+There are two sets of macros that read or write binary data at the current position
+and advance the stream position accordingly.\label{HPUT}\label{HGET}
+
+\getcode
+@<shared get macros@>=
+#define HGET_ERROR @/ QUIT(@["HGET overrun in section %d at " SIZE_F "\n"@],@|section_no,hpos-hstart)
+#define @[HEND@] @[((hpos<=hend)?0:(HGET_ERROR,0))@]
+
+#define @[HGET8@] ((hpos<hend)?*(hpos++):(HGET_ERROR,0))
+#define @[HGET16(X)@] ((X)=(hpos[0]<<8)+hpos[1],hpos+=2,HEND)
+#define @[HGET24(X)@] ((X)=(hpos[0]<<16)+(hpos[1]<<8)+hpos[2],hpos+=3,HEND)
+#define @[HGET32(X)@] ((X)=(hpos[0]<<24)+(hpos[1]<<16)+(hpos[2]<<8)+hpos[3],hpos+=4,HEND)
+#define @[HGETTAG(A)@] @[A=HGET8,DBGTAG(A,hpos-1)@]
+@
+
+\putcode
+@<put functions@>=
+void hput_error(void)
+{@+if (hpos<hend) return;
+ QUIT(@["HPUT overrun section %d pos=" SIZE_F "\n"@],@|section_no,hpos-hstart);
+}
+@
+
+@<put macros@>=
+extern void hput_error(void);
+#define @[HPUT8(X)@] (hput_error(),*(hpos++)=(X))
+#define @[HPUT16(X)@] (HPUT8(((X)>>8)&0xFF),HPUT8((X)&0xFF))
+#define @[HPUT24(X)@] (HPUT8(((X)>>16)&0xFF),HPUT8(((X)>>8)&0xFF),HPUT8((X)&0xFF))
+#define @[HPUT32(X)@] (HPUT8(((X)>>24)&0xFF),HPUT8(((X)>>16)&0xFF),HPUT8(((X)>>8)&0xFF),HPUT8((X)&0xFF))
+@
+
+The above macros test for buffer overruns\index{buffer overrun};
+allocating sufficient buffer space is done separately.
+
+Before writing a node, we will insert a test and increase the buffer if necessary.
+@<put macros@>=
+void hput_increase_buffer(uint32_t n);
+#define @[HPUTX(N)@] @[(((hend-hpos) < (N))? hput_increase_buffer(N):(void)0)@]
+#define HPUTNODE @[HPUTX(MAX_TAG_DISTANCE)@]
+#define @[HPUTTAG(K,I)@] @|@[(HPUTNODE, at +DBGTAG(TAG(K,I),hpos), at +HPUT8(TAG(K,I)))@]
+@
+
+Fortunately the only data types that have an unbounded size are
+strings\index{string} and texts\index{text}.
+For these we insert specific tests. For all other cases a relatively
+small upper bound on the maximum distance between two tags can be determined.
+Currently the maximum distance between tags is 26 byte as can be determined
+from the |hnode_size| array described in appendix~\secref{fastforward}.
+The definition below uses a slightly larger value leaving some room
+for future changes in the design of the short file format.
+
+@<hint macros@>=
+#define MAX_TAG_DISTANCE 32
+@
+
+\subsection{Mapping a Short Format File to Memory}
+In the following, we implement two alternatives to map a file into memory.
+The first implementation, opens the file, gets its size, allocates memory,
+and reads the file. The second implementation uses a call to |mmap|.
+
+Since modern computers with 64bit hardware have a huge address space,
+using |mmap| to map the entire file into virtual memory is the most efficient way
+to access a large file. ``Mapping'' is not the same as ``reading'' and it is
+not the same as allocating precious memory, all that is done by the
+operating system when needed. Mapping just reserves addresses.
+There is one disadvantage of mapping: it typically locks the underlying file
+and will not allow a separate process to modify it. This prevents using
+this method for previewing a \HINT\ file while editing and recompiling it.
+In this case, the first implementation, which has a copy of the file in memory,
+is the better choice. To select the second implementation, define the macro |USE_MMAP|.
+
+The following functions map and unmap a short format input
+file setting |hin_addr| to its address and |hin_size| to its size.
+The value |hin_addr==NULL| indicates, that no file is open.
+The variable |hin_time| is set to the time when the file was last modified.
+It can be used to detect modifications of the file and reload it.\label{map}
+
+@<common variables@>=
+char *hin_name=NULL;
+uint64_t hin_size=0;
+uint8_t *hin_addr=NULL;
+uint64_t hin_time=0;
+@
+
+@<map functions@>=
+#ifndef USE_MMAP
+void hget_unmap(void)
+{@+ if (hin_addr!=NULL) free(hin_addr);
+ hin_addr=NULL;
+ hin_size=0;
+}
+bool hget_map(void)
+{ FILE *f;
+ struct stat st;
+ size_t s,t,u;
+ f= fopen(hin_name,"rb");
+ if (f==NULL)@/
+ { MESSAGE("Unable to open file: %s", hin_name);@+ return false;@+ }
+ if (stat(hin_name,&st)<0)
+ { MESSAGE("Unable to obtain file size: %s", hin_name);
+ fclose(f);
+ return false;
+ }
+ if (st.st_size==0)
+ { MESSAGE("File %s is empty", hin_name);
+ fclose(f);
+ return false;
+ }
+ if (hin_addr!=NULL) hget_unmap();
+ hin_addr=malloc(st.st_size);
+ if (hin_addr==NULL)
+ { MESSAGE("Unable to allocate %ld byte for File %s", st.st_size,hin_name);
+ fclose(f);
+ return 0;
+ }
+ t=0;@+
+ u=st.st_size;
+ do{
+ s=fread(hin_addr+t,1,u,f);
+ if (s<=0)
+ { MESSAGE("Unable to read file %s",hin_name);
+ fclose(f);
+ free(hin_addr);
+ hin_addr=NULL;
+ return false;
+ }
+ t=t+s;@+
+ u=u-s;
+ } while (u>0);
+ hin_size=st.st_size;
+ hin_time=st.st_mtime;
+ return true;
+}
+
+#else
+
+#include <sys/mman.h>
+
+void hget_unmap(void)
+{@+ munmap(hin_addr,hin_size);
+ hin_addr=NULL;
+ hin_size=0;
+}
+
+bool hget_map(void)
+{ struct stat st;
+ int fd;
+ fd = open(hin_name, O_RDONLY, 0);
+ if (fd<0)@/
+ { MESSAGE("Unable to open file %s", hin_name);@+ return false;@+ }
+ if (fstat(fd, &st)<0)
+ { MESSAGE("Unable to get file size");
+ close(fd);
+ return false;
+ }
+ if (st.st_size==0)
+ { MESSAGE("File %s is empty",hin_name);
+ close(fd);
+ return false;
+ }
+ if (hin_addr!=NULL) hget_unmap();
+ hin_size=st.st_size;
+ hin_time=st.st_mtime;
+ hin_addr= mmap(NULL,hin_size,PROT_READ,MAP_PRIVATE,fd, 0);
+ if (hin_addr==MAP_FAILED)
+ { close(fd);
+ hin_addr=NULL;hin_size=0;
+ MESSAGE("Unable to map file into memory");
+ return 0;
+ }
+ close(fd);
+ return hin_size;
+}
+#endif
+
+@
+\subsection{Compression}
+The short file format offers the possibility to store sections in
+compressed\index{compression} form. We use the {\tt zlib}\index{zlib+{\tt zlib}} compression library\cite{zlib}\cite{RFC1950}
+to deflate\index{deflate} and inflate\index{inflate} individual sections. When one of the following
+functions is called, we can get the section buffer, the buffer size
+and the size actually used from the directory entry. If a section
+needs to be inflated, its size after decompression is found in the
+|xsize| field; if a section needs to be deflated, its size after
+compression will be known after deflating it.
+
+ at s z_stream int
+
+@<get file functions@>=
+
+static void hdecompress(uint16_t n)
+{ z_stream z; /* decompression stream */
+ uint8_t *buffer;
+ int i;
+
+ DBG(DBGCOMPRESS,"Decompressing section %d from 0x%x to 0x%x byte\n",@|n, dir[n].size, dir[n].xsize);
+ z.zalloc = (alloc_func)0;@+
+ z.zfree = (free_func)0;@+
+ z.opaque = (voidpf)0;
+ z.next_in = hstart;
+ z.avail_in = hend-hstart;
+ if (inflateInit(&z)!=Z_OK)
+ QUIT("Unable to initialize decompression: %s",z.msg);
+ ALLOCATE(buffer,dir[n].xsize+MAX_TAG_DISTANCE,uint8_t);
+ DBG(DBGBUFFER,"Allocating output buffer size=0x%x, margin=0x%x\n",dir[n].xsize,MAX_TAG_DISTANCE);
+ z.next_out = buffer;
+ z.avail_out =dir[n].xsize+MAX_TAG_DISTANCE;
+ i= inflate(&z, Z_FINISH);
+ DBG(DBGCOMPRESS,"in: avail/total=0x%x/0x%lx "@|"out: avail/total=0x%x/0x%lx, return %d;\n",@|
+ z.avail_in,z.total_in, z.avail_out, z.total_out,i);
+ if (i!=Z_STREAM_END)
+ QUIT("Unable to complete decompression: %s",z.msg);
+ if (z.avail_in != 0)
+ QUIT("Decompression missed input data");
+ if (z.total_out != dir[n].xsize)
+ QUIT("Decompression output size mismatch 0x%lx != 0x%x",z.total_out, dir[n].xsize );
+ if (inflateEnd(&z)!=Z_OK)
+ QUIT("Unable to finalize decompression: %s",z.msg);
+ dir[n].buffer=buffer;
+ dir[n].bsize=dir[n].xsize;
+ hpos0=hpos=hstart=buffer;
+ hend=hstart+dir[n].xsize;
+}
+@
+
+
+@<put functions@>=
+static void hcompress(uint16_t n)
+{ z_stream z; /* compression stream */
+ uint8_t *buffer;
+ int i;
+ if (dir[n].size==0) { dir[n].xsize=0;@+ return; @+}
+ DBG(DBGCOMPRESS,"Compressing section %d of size 0x%x\n",n, dir[n].size);
+ z.zalloc = (alloc_func)0;@+
+ z.zfree = (free_func)0;@+
+ z.opaque = (voidpf)0;
+ if (deflateInit(&z,Z_DEFAULT_COMPRESSION)!=Z_OK)
+ QUIT("Unable to initialize compression: %s",z.msg);
+ ALLOCATE(buffer,dir[n].size+MAX_TAG_DISTANCE,uint8_t);
+ z.next_out = buffer;
+ z.avail_out = dir[n].size+MAX_TAG_DISTANCE;
+ z.next_in = dir[n].buffer;
+ z.avail_in = dir[n].size;
+ i=deflate(&z, Z_FINISH);
+ DBG(DBGCOMPRESS,"deflate in: avail/total=0x%x/0x%lx out: avail/total=0x%x/0x%lx, return %d;\n",@|
+ z.avail_in,z.total_in, z.avail_out, z.total_out,i);
+ if (z.avail_in != 0)
+ QUIT("Compression missed input data");
+ if (i!=Z_STREAM_END)
+ QUIT("Compression incomplete: %s",z.msg);
+ if (deflateEnd(&z)!=Z_OK)
+ QUIT("Unable to finalize compression: %s",z.msg);
+ DBG(DBGCOMPRESS,"Compressed 0x%lx byte to 0x%lx byte\n",@|z.total_in,z.total_out);
+ free(dir[n].buffer);
+ dir[n].buffer=buffer;
+ dir[n].bsize=dir[n].size+MAX_TAG_DISTANCE;
+ dir[n].xsize=dir[n].size;
+ dir[n].size=z.total_out;
+}
+@
+
+
+
+\subsection{Reading Short Format Sections}
+\gdef\subcodetitle{Sections}%
+
+After mapping the file at address |hin_addr| access to sections of the
+file is provided by decompressing them if necessary and
+setting the three pointers |hpos|, |hstart|, and
+|hend|.
+
+To read sections of a short format input file, we use the function |hget_section|.
+
+\getcode
+%\codesection{\getsymbol}\getindex{1}{3}{Files}
+
+@<get file functions@>=
+void hget_section(uint16_t n)
+{ DBG(DBGDIR,"Reading section %d\n",n);
+ RNG("Section number",n,0,max_section_no);
+ if (dir[n].buffer!=NULL && dir[n].xsize>0)
+ { hpos0=hpos=hstart=dir[n].buffer;
+ hend=hstart+dir[n].xsize;
+ }
+ else
+ { hpos0=hpos=hstart=hin_addr+dir[n].pos;
+ hend=hstart+dir[n].size;
+ if (dir[n].xsize>0) hdecompress(n);
+ }
+}
+@
+\subsection{Writing Short Format Sections}
+\gdef\subcodetitle{Sections}%
+
+To write a short format file, we allocate for each of the first three sections a
+suitable buffer\index{buffer}, then fill these buffers, and finally write them
+out in sequential order.
+
+@<put functions@>=
+#define BUFFER_SIZE 0x400
+void new_output_buffers(void)
+{ dir[0].bsize=dir[1].bsize=dir[2].bsize=BUFFER_SIZE;
+ DBG(DBGBUFFER,"Allocating output buffer size=0x%x, margin=0x%x\n",BUFFER_SIZE,MAX_TAG_DISTANCE);
+ ALLOCATE(dir[0].buffer,dir[0].bsize+MAX_TAG_DISTANCE,uint8_t);
+ ALLOCATE(dir[1].buffer,dir[1].bsize+MAX_TAG_DISTANCE,uint8_t);
+ ALLOCATE(dir[2].buffer,dir[2].bsize+MAX_TAG_DISTANCE,uint8_t);
+}
+
+void hput_increase_buffer(uint32_t n)
+{ size_t bsize;
+ uint32_t pos, pos0;
+ const double buffer_factor=1.4142136; /* $\sqrt 2$ */
+ pos=hpos-hstart; pos0=hpos0-hstart;
+ bsize=dir[section_no].bsize*buffer_factor+0.5;
+ if (bsize<pos+n) bsize=pos+n;
+ if (bsize>=HINT_NO_POS) bsize=HINT_NO_POS;
+ if (bsize<pos+n) QUIT(@["Unable to increase buffer size " SIZE_F " by 0x%x byte"@],@|hpos-hstart,n);
+ DBG(DBGBUFFER,@["Reallocating output buffer "@|" for section %d from 0x%x to " SIZE_F " byte\n"@],
+ section_no,dir[section_no].bsize,bsize);
+ REALLOCATE(dir[section_no].buffer,bsize,uint8_t);
+ dir[section_no].bsize=(uint32_t)bsize;
+ hstart=dir[section_no].buffer;
+ hend=hstart+bsize;
+ hpos0=hstart+pos0; hpos=hstart+pos;
+}
+
+static size_t hput_data(uint16_t n, uint8_t *buffer, uint32_t size)
+{ size_t s;
+ s=fwrite(buffer,1,size,hout);
+ if (s!=size)
+ QUIT(@["short write " SIZE_F " < %d in section %d"@],s,size,n);
+ return s;
+}
+
+static size_t hput_section(uint16_t n)
+{ return hput_data(n, dir[n].buffer,dir[n].size);
+}
+@
+
+
+
+
+\section{Directory Section}
+A \HINT\ file is subdivided in sections and
+each section can be identified by its section number.
+The first three sections, numbered 0, 1, and 2, are mandatory:
+directory\index{directory section} section, definition section, and content section.
+The directory section, which we explain now, lists all sections
+that make up a \HINT\ file.
+
+A document will often contain not only plain text but also other media
+for example illustrations. Illustrations are produced with specialized
+tools and stored in specialized files. Because a \HINT\ file in short format
+should be self contained, these special files are embedded in the \HINT\ file
+as optional sections.
+Because a \HINT\ file in long format should be readable, these special files
+are written to disk and only the file names are retained in the directory.
+Writing special files to disk has also the advantage that you can modify
+them individually before embedding them in a short format file.
+
+
+\subsection{Directories in Long Format}\gdef\subcodetitle{Directory Section}%
+The directory\index{directory section} section of a long format \HINT\ file starts
+with the ``\.{directory}'' keyword; then follows the maximum section number used and
+a list of directory entries, one for each optional section numbered 3 and above.
+Each entry consists of the keyword ``\.{section}'' followed by the
+section number, followed by the file name.
+The section numbers must be unique and fit into 16 bit.
+The directory entries must be ordered with strictly increasing section numbers.
+Keeping section numbers consecutive is recommended because it reduces the
+memory footprint if directories are stored as arrays indexed by the section
+number as we will do below.
+
+\readcode
+ at s directory_section symbol
+ at s entry_list symbol
+ at s entry symbol
+ at s DIRECTORY symbol
+ at s SECTION symbol
+
+@<symbols@>=
+%token DIRECTORY "directory"
+%token SECTION "entry"
+@
+
+@<scanning rules@>=
+::@=directory@> :< return DIRECTORY; >:
+::@=section@> :< return SECTION; >:
+@
+
+@<parsing rules@>=
+directory_section: START DIRECTORY UNSIGNED @|{new_directory($3+1); new_output_buffers();} entry_list END ;
+entry_list: @, at + | entry_list entry;
+entry: START SECTION UNSIGNED string END @/
+ { RNG("Section number",$3,3,max_section_no); hset_entry(&(dir[$3]), $3,0,0,$4);};
+@
+
+
+
+We use a dynamically allocated array
+of directory entries to store the directory.
+
+@<directory entry type@>=
+typedef struct {
+uint64_t pos;
+uint32_t size, xsize;
+uint16_t section_no;
+char *file_name;
+uint8_t *buffer;
+uint32_t bsize;
+} entry_t;
+@
+
+
+The function |new_directory| allocates the directory.
+
+@<directory functions@>=
+entry_t *dir=NULL;
+uint16_t section_no, max_section_no;
+void new_directory(uint32_t entries)
+{ DBG(DBGDIR,"Creating directory with %d entries\n", entries);
+ RNG("Directory entries",entries,3,0x10000);
+ max_section_no=entries-1;@+
+ ALLOCATE(dir,entries,entry_t);
+ dir[0].section_no=0; @+ dir[1].section_no=1; @+ dir[2].section_no=2;
+}
+@
+
+The function |hset_entry| fills in the appropriate entry.
+@<directory functions@>=
+void hset_entry(entry_t *e, uint16_t i, uint32_t size, uint32_t xsize, @|char *file_name)
+{ e->section_no=i;
+ e->size=size; @+e->xsize=xsize;
+ if (file_name==NULL || *file_name==0)
+ e->file_name=NULL;
+ else
+ e->file_name=strdup(file_name);
+ DBG(DBGDIR,"Creating entry %d: \"%s\" size=0x%x xsize=0x%x\n",@|i,file_name,size,xsize);
+}
+@
+
+
+Writing the auxiliary files depends on the {\tt -f} and the {\tt -g}
+option.
+
+@<without {\tt -f} skip writing an existing file@>=
+ if ( !option_force && access(file_name,F_OK)==0)
+ { MESSAGE("File '%s' exists.\n"@| "To rewrite the file use the -f option.\n",
+ file_name);
+ continue;
+ }
+@
+
+The above code uses the |access| function, and we need to make sure it is defined:
+@<make sure |access| is defined@>=
+#ifdef WIN32
+#include <io.h>
+#define @[access(N,M)@] @[_access(N, M )@]
+#define F_OK 0
+#else
+#include <unistd.h>
+#endif
+@
+
+With the {\tt -g} option, filenames are considered global, and files
+are written to the filesystem possibly overwriting the existing files.
+For example a font embedded in a \HINT\ file might replace a font of
+the same name in some operating systems font folder.
+If the \HINT\ file is {\tt shrink}ed on one system and
+{\tt stretch}ed on another system, this is usually not the desired behavior.
+Without the {\tt -g} option,\label{absrel} the files will be written in two local directories.
+The names of these directories are derived from the output file name,
+replacing the extension ``{\tt .HINT}'' with ``{\tt .abs}'' if the original
+filename contained an absolute path, and replacing it with ``{\tt .rel}''
+if the original filename contained a relative path. Inside these directories,
+the path as given in the filename is retained.
+When {\tt shrink}ing a \HINT\ file without the {\tt -g} option,
+the original filenames can be reconstructed.
+
+@<without {\tt -g} compute a local |file_name|@>=
+if (!option_global)
+{ int path_length=(int)strlen(file_name);
+ @<determine whether |file_name| is absolute or relative@>@;
+ if (file_name_length<stem_length+ext_length+path_length)
+ { file_name_length=stem_length+ext_length+path_length;
+ REALLOCATE(stem_name, file_name_length+1,char);
+ }
+ strcpy(stem_name+stem_length,aux_ext[name_type]);
+ strcpy(stem_name+stem_length+ext_length,file_name);
+ DBG(DBGDIR,"Replacing auxiliary file name:\n\t%s\n->\t%s\n",file_name,stem_name);
+ file_name=stem_name;
+}
+@
+
+@<determine whether |file_name| is absolute or relative@>=
+ enum {absolute=0, relative=1} name_type;
+ char *aux_ext[2]={".abs/",".rel/"};
+ int ext_length=5;
+ if (file_name[0]=='/')
+ { name_type=absolute;
+ file_name++; path_length--;
+ }
+ else if (path_length>3 && isalpha(file_name[0]) &&
+ file_name[1]==':' && file_name[2]=='/')
+ { name_type=absolute;
+ file_name[1]='_';
+ }
+ else
+ name_type=relative;
+@
+It remains to create the directories along the path we might have constructed.
+@<make sure the path in |file_name| exists@>=
+{ char *path_end;
+ path_end=file_name+1;
+ while (*path_end!=0)
+ { if(*path_end=='/')
+ { struct stat s;
+ *path_end=0;
+ if (stat(file_name,&s)==-1)
+ {
+#ifdef WIN32
+ if (mkdir(file_name)!=0)
+#else
+ @t\2\kern-1em@>if (mkdir(file_name,0777)!=0)
+#endif
+ QUIT("Unable to create directory %s",file_name);
+ DBG(DBGDIR,"Creating directory %s\n",file_name);
+ } else if (!(S_IFDIR&s.st_mode))
+ QUIT("Unable to create directory %s, file exists",file_name);
+ *path_end='/';
+ }
+ path_end++;
+ }
+}
+
+
+@
+
+\writecode
+@<write functions@>=
+@<make sure |access| is defined@>@;
+extern char *stem_name;
+extern int stem_length;
+
+void hget_section(uint16_t n);
+void hwrite_aux_files(void)
+{ int i;
+ DBG(DBGBASIC|DBGDIR,"Writing %d aux files\n",max_section_no-2);
+ for (i=3;i<=max_section_no;i++)
+ { FILE *f;
+ char * file_name=dir[i].file_name;
+ int file_name_length=0;
+
+
+ @<without {\tt -g} compute a local |file_name|@>@;
+ @<without {\tt -f} skip writing an existing file@>@;
+ @<make sure the path in |file_name| exists@>@;
+
+ f=fopen(file_name,"wb");
+ if (f==NULL)
+ QUIT("Unable to open file '%s' for writing",file_name);
+ else
+ { size_t s;
+ hget_section(i);
+ DBG(DBGDIR,"Writing file %s\n",file_name);
+ s=fwrite(hstart,1,dir[i].size,f);
+ if (s!=dir[i].size) QUIT("writing file %s",file_name);
+ fclose(f);
+ }
+ }
+}
+@
+
+We write the directory, and the directory entries
+in long format using the following functions.
+@<write functions@>=
+static void hwrite_entry(int i)
+{ hwrite_start();
+ hwritef("section %u",dir[i].section_no);@+ hwrite_string(dir[i].file_name);
+ hwrite_end();
+}
+
+void hwrite_directory(void)
+{ int i;
+ if (dir==NULL) QUIT("Directory not allocated");
+ section_no=0;
+ hwritef("<directory %u", max_section_no);@/
+ for (i=3;i<=max_section_no;i++)
+ hwrite_entry(i);
+ hwritef("\n>\n");
+}
+@
+
+\subsection{Directories in Short Format}
+The directory\index{directory section} section of a short format file contains entries
+for all sections including the directory section itself. After reading the
+directory section, enough information---position and size---is available to
+access any section directly. As usual, a directory entry starts and ends with
+a tag byte. The kind part of an entry's tag is not used; it is always zero.
+The value $s$ of the two least significant bits of the info part indicate
+that sizes are stored using $s+1$ byte. The most significant bit of the info
+part is 1 if the section is stored in compressed\index{compression} form. In this case the size
+of the section is followed by the size of the section after decompressing it.
+After the tag byte follows the section number. In the short format file,
+section numbers must be strictly increasing and consecutive. This is redundant but helps
+with checking. Then follows the size---or the sizes---of the section. After the size
+follows the file name terminated by a zero byte. The file name might be an empty
+string in which case there is just the zero byte. After the zero byte follows
+a copy of the tag byte.
+
+Here is the macro and function to read a directory\index{directory entry} entry:
+\gdef\subcodetitle{Directory Entries}%
+\getcode
+
+@<shared get macros@>=
+#define @[HGET_SIZE(I)@] \
+ if ((I)&b100) { \
+ if (((I)&b011)==0) s=HGET8,xs=HGET8; \
+ else if (((I)&b011)==1) HGET16(s),HGET16(xs); \
+ else if (((I)&b011)==2) HGET24(s),HGET24(xs); \
+ else if (((I)&b011)==3) HGET32(s),HGET32(xs); \
+ } \
+ else { \
+ if (((I)&b011)==0) s=HGET8; \
+ else if (((I)&b011)==1) HGET16(s); \
+ else if (((I)&b011)==2) HGET24(s); \
+ else if (((I)&b011)==3) HGET32(s); \
+ }
+
+#define @[HGET_ENTRY(I,E)@] \
+{ uint16_t i; \
+ uint32_t s=0,xs=0; \
+ char *file_name; \
+ HGET16(i); @+HGET_SIZE(I); @+HGET_STRING(file_name); @/\
+ hset_entry(&(E),i,s,xs,file_name); \
+}
+@
+
+@<get file functions@>=
+void hget_entry(entry_t *e)
+{ @<read the start byte |a|@>@;
+ DBG(DBGDIR,"Reading directory entry\n");
+ switch(a)
+ { case TAG(0,b000+0): HGET_ENTRY(b000+0,*e);@+ break;
+ case TAG(0,b000+1): HGET_ENTRY(b000+1,*e);@+ break;
+ case TAG(0,b000+2): HGET_ENTRY(b000+2,*e);@+ break;
+ case TAG(0,b000+3): HGET_ENTRY(b000+3,*e);@+ break;
+ case TAG(0,b100+0): HGET_ENTRY(b100+0,*e);@+ break;
+ case TAG(0,b100+1): HGET_ENTRY(b100+1,*e);@+ break;
+ case TAG(0,b100+2): HGET_ENTRY(b100+2,*e);@+ break;
+ case TAG(0,b100+3): HGET_ENTRY(b100+3,*e);@+ break;
+ default: TAGERR(a); @+ break;
+ }
+ @<read and check the end byte |z|@>@;
+}
+@
+
+Because the first entry in the directory section describes the
+directory section itself, we can not check its info bits in advance to determine
+whether it is compressed or not. Therefore the directory section
+starts with a root entry, which is always uncompressed. It describes
+the remainder of the directory which follows.
+There are two differences between the root entry and a normal entry:
+it starts with the maximum section number instead of the section number zero,
+and we set its position to the position of the
+entry for section 1 (which might already be compressed).
+The name of the directory section must be the empty string.
+\gdef\subcodetitle{Directory Section}%
+\getcode
+@<get file functions@>=
+static void hget_root(entry_t *root)
+{ DBG(DBGDIR,"Root entry at " SIZE_F "\n",hpos-hstart);
+ hget_entry(root);
+ root->pos=hpos-hstart;
+ max_section_no=root->section_no;
+ root->section_no=0;
+ if (max_section_no<2) QUIT("Sections 0, 1, and 2 are mandatory");
+}
+
+void hget_directory(void)
+{ int i;
+ entry_t root={0};
+ hget_root(&root);
+ DBG(DBGDIR,"Directory\n");
+ new_directory(max_section_no+1);
+ dir[0]=root;
+ DBG(DBGDIR,"Directory entry 1 at 0x%"PRIx64"\n",dir[0].pos);
+ hget_section(0);
+ for (i=1;i<=max_section_no;i++)@/
+ { hget_entry(&(dir[i]));@+
+ dir[i].pos=dir[i-1].pos +dir[i-1].size;@+
+ DBG(DBGDIR,"Section %d at 0x%"PRIx64"\n",i,dir[i].pos);
+ }
+}
+
+void hclear_dir(void)
+{ int i;
+ if (dir==NULL) return;
+ for (i=0;i<3;i++) /* currently the only compressed sections */
+ if (dir[i].xsize>0 && dir[i].buffer!=NULL) free(dir[i].buffer);
+ free(dir); dir=NULL;
+}
+
+@
+
+When the \.{shrink} program writes the directory section in the short format,
+it needs to know the sizes of all the sections---including the optional sections.
+These sizes are not provided in the long format because it is safer and more
+convenient to let the machine figure out the file sizes\index{file size}.
+
+@<set the file sizes for optional sections@>=
+{ int i;
+ for (i=3;i<=max_section_no;i++)
+ { struct stat s;
+ char *file_name=dir[i].file_name;
+ int file_name_length=0;
+ @<without {\tt -g} compute a local |file_name|@>@;
+ if (stat(file_name,&s)!=0)
+ QUIT("Unable to obtain file size for '%s'",dir[i].file_name);
+ dir[i].size=s.st_size;
+ dir[i].xsize=0;
+ }
+}
+@
+
+The computation of the sizes of the mandatory sections will be
+explained later.
+Armed with these preparations, we can put the directory into the \HINT\ file.
+
+\gdef\subcodetitle{Directory Section}%
+\putcode
+@<put functions@>=
+static void hput_entry(entry_t *e)
+{ uint8_t b;
+ if (e->size<0x100 && e->xsize<0x100) b=0;
+ else if (e->size<0x10000 &&e->xsize<0x10000) b=1;
+ else if (e->size<0x1000000 &&e->xsize<0x1000000) b=2;
+ else b=3;
+ if (e->xsize!=0) b =b|b100;
+ DBG(DBGTAGS,"Directory entry no=%d size=0x%x xsize=0x%x\n",e->section_no, e->size, e->xsize);
+ HPUTTAG(0,b);@/
+ HPUT16(e->section_no);
+ switch (b) {
+ case 0: HPUT8(e->size);@+break;
+ case 1: HPUT16(e->size);@+break;
+ case 2: HPUT24(e->size);@+break;
+ case 3: HPUT32(e->size);@+break;
+ case b100|0: HPUT8(e->size);@+HPUT8(e->xsize);@+break;
+ case b100|1: HPUT16(e->size);@+HPUT16(e->xsize);@+break;
+ case b100|2: HPUT24(e->size);@+HPUT24(e->xsize);@+break;
+ case b100|3: HPUT32(e->size);@+HPUT32(e->xsize);@+break;
+ default: QUIT("Can't happen");@+ break;
+ }
+ hput_string(e->file_name);@/
+ DBGTAG(TAG(0,b),hpos);@+HPUT8(TAG(0,b));
+}
+
+static void hput_directory_start(void)
+{ DBG(DBGDIR,"Directory Section\n");
+ section_no=0;
+ hpos=hstart=dir[0].buffer;
+ hend=hstart+dir[0].bsize;
+}
+static void hput_directory_end(void)
+{ dir[0].size=hpos-hstart;
+ DBG(DBGDIR,"End Directory Section size=0x%x\n",dir[0].size);
+}
+
+static size_t hput_root(void)
+{ uint8_t buffer[MAX_TAG_DISTANCE];
+ size_t s;
+ hpos=hstart=buffer;
+ hend=hstart+MAX_TAG_DISTANCE;
+ dir[0].section_no=max_section_no;
+ hput_entry(&dir[0]);
+ s=hput_data(0, hstart,hpos-hstart);
+ DBG(DBGDIR,@["Writing root size=" SIZE_F "\n"@],s);
+ return s;
+}
+
+extern int option_compress;
+void hput_directory(void)
+{ int i;
+ @<set the file sizes for optional sections@>@;
+ if (option_compress) { hcompress(1); @+hcompress(2); @+}
+ hput_directory_start();
+ for (i=1; i<=max_section_no; i++)
+ { dir[i].pos=dir[i-1].pos+dir[i-1].size;
+ DBG(DBGDIR,"writing entry %u at 0x%" PRIx64 "\n",i, dir[i].pos);
+ hput_entry(&dir[i]);
+ }
+ hput_directory_end();
+ if (option_compress) hcompress(0);
+}
+
+@
+
+
+To conclude this section, here is the function that adds the files that
+are described in the directory entries 3 and above to a \HINT\ file in short format.
+Where these files are found depends on the {\tt -g} option.
+With that option given, the file names of the directory entries are used unchanged.
+Without that option, the files are found in the {|hin_name|\tt .abs} and {|hin_name|\tt .rel}
+directories, as described in section~\secref{absrel}.
+
+\gdef\subcodetitle{Optional Sections}%
+\putcode
+@<put functions@>=
+static void hput_optional_sections(void)
+{ int i;
+ DBG(DBGDIR,"Optional Sections\n");
+ for (i=3; i<=max_section_no; i++)@/
+ { FILE *f;
+ size_t fsize;
+ char *file_name=dir[i].file_name;
+ int file_name_length=0;
+ DBG(DBGDIR,"file %d: %s\n",dir[i].section_no,file_name);
+ if (dir[i].xsize!=0) @/
+ DBG(DBGDIR,"Compressing of auxiliary files currently not supported");
+ @<without {\tt -g} compute a local |file_name|@>@;
+ f=fopen(file_name,"rb");
+ if (f==NULL) QUIT("Unable to read section %d, file %s",
+ dir[i].section_no,file_name);
+ fsize=0;
+ while (!feof(f))@/
+ { size_t s,t;
+ char buffer[1<<13]; /* 8kByte */
+ s=fread(buffer,1,1<<13,f);@/
+ t=fwrite(buffer,1,s,hout);
+ if (s!=t) QUIT("writing file %s",file_name);
+ fsize=fsize+t;
+ }
+ fclose(f);
+ if (fsize!=dir[i].size)
+ QUIT(@["File size " SIZE_F " does not match directory size %u"@],@|fsize,dir[i].size);
+ }
+}
+@
+
+
+
+\section{Definition Section}\index{definition section}
+\label{defsection}
+In a typical \HINT\ file, there are many things that are used over and over again.
+For example the interword glue of a specific font or the indentation of
+the first line of a paragraph. The definition section contains this information so that
+it can be referenced in the content section by a simple reference number.
+In addition there are a few parameters that guide the routines of \TeX.
+An example is the ``above display skip'', which controls the amount of white space
+inserted above a displayed equation, or the ``hyphen penalty'' that tells \TeX\
+the ``\ae sthetic cost'' of ending a line with a hyphenated word. These parameters
+also get their values in the definition section as explained in section~\secref{defaults}.
+
+
+The most simple way to store these definitions is to store them in an array indexed by the
+reference numbers.
+To simplify the dynamic allocation of these arrays, the list of definitions
+will always start with the list of maximum\index{maximum values} values: a list that contains
+for each node type the maximum reference number used.
+
+In the long format, the definition section starts with the keyword \.{definitions},
+followed by the list of maximum values,
+followed by the definitions proper.
+
+When writing the short format, we start by positioning the output stream at the beginning of
+the definition buffer and we end with recording the size of the definition section
+in the directory.
+
+\readcode
+ at s definition_section symbol
+ at s definition_list symbol
+ at s definition symbol
+ at s DEFINITIONS symbol
+@<symbols@>=
+%token DEFINITIONS "definitions"
+@
+
+@<scanning rules@>=
+::@=definitions@> :< return DEFINITIONS; >:
+@
+
+@<parsing rules@>=
+definition_section: START DEFINITIONS { hput_definitions_start();}@/
+ max_definitions definition_list @/
+ END {hput_definitions_end();};
+definition_list: @+ | definition_list def_node;
+@
+
+\writecode
+@<write functions@>=
+void hwrite_definitions_start(void)
+{ section_no=1; @+hwritef("<definitions");
+}
+
+void hwrite_definitions_end(void)
+{ hwritef("\n>\n");
+}
+@
+
+
+
+@<get functions@>=
+void hget_definition_section(void)
+{ DBG(DBGBASIC|DBGDEF,"Definitions\n");
+ hget_section(1);
+ hwrite_definitions_start();
+ DBG(DBGDEF,"List of maximum values\n");
+ hget_max_definitions();
+ @<initialize definitions@>@;
+ hwrite_max_definitions();
+ DBG(DBGDEF,"List of definitions\n");
+ while (hpos<hend)
+ hget_def_node();
+ hwrite_definitions_end();
+}
+@
+
+\putcode
+@<put functions@>=
+void hput_definitions_start(void)
+{ DBG(DBGDEF,"Definition Section\n");
+ section_no=1;
+ hpos=hstart=dir[1].buffer;
+ hend=hstart+dir[1].bsize;
+}
+void hput_definitions_end(void)
+{ dir[1].size=hpos-hstart;
+ DBG(DBGDEF,"End Definition Section size=0x%x\n",dir[1].size);
+}
+@
+\gdef\codetitle{Definitions}
+\hascode
+\subsection{Maximum Values}\index{maximum values}
+To help implementations allocating the right amount of memory for the
+definitions, the definition section starts with a list of maximum
+values. For each kind of node, we store the maximum valid reference
+number in the array |max_ref| which is indexed by the kind-values.
+For a reference number |n| and kind-value $k$ we have
+$0\le n\le |max_ref[k]|$.
+To make sure that a hint file without any definitions
+will work, some definitions have default values.
+The initialization of default and maximum values is described
+in section~\secref{defaults}. The maximum
+reference number that has a default value is stored in the array
+|max_default|.
+We have $-1 \le |max_default[k]| \le |max_ref[k]| < 2^{16}$,
+and for most $k$ even $|max_ref[k]| < 2^{8}$.
+Specifying maximum values that are lower than the
+default\index{default value} values is not allowed in the short
+format; in the long format, lower values are silently ignored. Some
+default values are permanently fixed; for example the zero glue with
+reference number |zero_skip_no| must never change. The array
+|max_fixed| stores the maximum reference number that has a fixed value for a
+given kind. Definitions with reference numbers less or equal than the
+corresponding |max_fixed[k]| number are disallowed. Usually we have
+$-1 \le |max_fixed[k]| \le |max_default[k]| $, but if for a kind-value
+$k$ no definitions, and hence no maximum values are allowed, we set
+$|max_fixed[k]|=|0x10000|>|max_default[k]| $.
+
+
+We use the |max_ref| array whenever we find a
+reference number in the input to check if it is within the proper range.
+
+@<debug macros@>=
+#define @[REF_RNG(K,N)@] if ((int)(N)>max_ref[K]) QUIT("Reference %d to %s out of range [0 - %d]",\
+ (N),definition_name[K],max_ref[K])
+@
+
+In the long format file, the list of maximum values starts with
+ ``\.{<max }'', then follow pairs of keywords and numbers like
+ ``\.{<glue 57>}'', and it ends with ``\.{>}''. In the short format,
+we start the list of maximums with a |list_kind| tag and end it with
+a |list_kind| tag. Each maximum value is preceded and followed by a
+tag byte with the appropriate kind-value. The info value has its |b001| bit
+cleared if the maximum value is in the range 0 to |0xFF| and fits into a
+single byte; the info value hast its |b001| bit set if it fits into two byte.
+Currently only the |label_kind| may need to use two byte.
+@<debug macros@>=
+#define MAX_REF(K) ((K)==label_kind?0xFFFF:0xFF)
+@
+
+Other info values are reserved for future extensions.
+After reading the maximum values, we initialize the data structures for
+the definitions.
+
+
+\readcode
+ at s max_list symbol
+ at s max_value symbol
+ at s max_definitions symbol
+ at s MAX symbol
+@<symbols@>=
+%token MAX "max"
+@
+
+@<scanning rules@>=
+::@=max@> :< return MAX; >:
+@
+@<parsing rules@>=
+max_definitions: START MAX max_list END @|
+ { @<initialize definitions@>@;@+ hput_max_definitions(); };
+
+max_list:@+ | max_list START max_value END;
+
+max_value: FONT UNSIGNED { hset_max(font_kind,$2); }
+ | INTEGER UNSIGNED { hset_max(int_kind,$2); }
+ | DIMEN UNSIGNED { hset_max(dimen_kind,$2); }
+ | LIGATURE UNSIGNED { hset_max(ligature_kind,$2); }
+ | DISC UNSIGNED { hset_max(disc_kind,$2); }
+ | GLUE UNSIGNED { hset_max(glue_kind,$2); }
+ | LANGUAGE UNSIGNED { hset_max(language_kind,$2); }
+ | RULE UNSIGNED { hset_max(rule_kind,$2); }
+ | IMAGE UNSIGNED { hset_max(image_kind,$2); }
+ | LEADERS UNSIGNED { hset_max(leaders_kind,$2); }
+ | BASELINE UNSIGNED { hset_max(baseline_kind,$2); }
+ | XDIMEN UNSIGNED { hset_max(xdimen_kind,$2); }
+ | PARAM UNSIGNED { hset_max(param_kind,$2); }
+ | STREAMDEF UNSIGNED { hset_max(stream_kind,$2); }
+ | PAGE UNSIGNED { hset_max(page_kind,$2); }
+ | RANGE UNSIGNED { hset_max(range_kind,$2); }
+ | LABEL UNSIGNED { hset_max(label_kind,$2); };
+
+@
+
+@<parsing functions@>=
+void hset_max(kind_t k, int n)
+{ DBG(DBGDEF,"Setting max %s to %d\n",definition_name[k],n);
+ RNG("Maximum",n,max_fixed[k]+1,MAX_REF(k));
+ if (n>max_ref[k])
+ max_ref[k]=n;
+}
+@
+
+\writecode
+@<write functions@>=
+void hwrite_max_definitions(void)
+{ kind_t k;
+ hwrite_start();@+
+ hwritef("max");
+ for (k=0; k<32;k++)
+ if (max_ref[k]>max_default[k])@/
+ {@+ switch (k)
+ { @<cases of writing special maximum values@>@;
+ default:
+ hwrite_start();
+ hwritef("%s %d",definition_name[k], max_ref[k]);
+ hwrite_end();
+ break;
+ }
+ }
+ hwrite_end();
+}
+@
+
+\getcode
+@<get file functions@>=
+void hget_max_definitions(void)
+{ kind_t k;
+ @<read the start byte |a|@>@;
+ if (a!=TAG(list_kind,0)) QUIT("Start of maximum list expected");
+ for(k= 0;k<32;k++)max_ref[k]= max_default[k]; max_outline=-1;
+ while (true) @/
+ { int n;
+ if (hpos>=hend) QUIT("Unexpected end of maximum list");
+ node_pos=hpos-hstart;
+ HGETTAG(a);@+
+ k=KIND(a);@+
+ if (k==list_kind) break;
+ if (INFO(a)&b001) HGET16(n); @+else n=HGET8;
+ switch (a)
+ { @<cases of getting special maximum values@>@;
+ default:
+ if (max_fixed[k]>max_default[k])
+ QUIT("Maximum value for kind %s not supported",definition_name[k]);
+ RNG("Maximum number",n,max_default[k],MAX_REF(k));
+ max_ref[k]=n;
+ DBG(DBGDEF,"max(%s) = %d\n",definition_name[k],max_ref[k]);
+ break;
+ }
+ @<read and check the end byte |z|@>@;
+ }
+ if (INFO(a)!=0) QUIT("End of maximum list with info %d", INFO(a));
+}
+@
+
+\putcode
+
+@<put functions@>=
+void hput_max_definitions(void)
+{ kind_t k;
+ DBG(DBGDEF,"Writing Max Definitions\n");
+ HPUTTAG(list_kind,0);
+ for (k=0; k<32; k++)
+ if (max_ref[k]>max_default[k])
+ { uint32_t pos=hpos++-hstart;
+ DBG(DBGDEF,"max(%s) = %d\n",definition_name[k],max_ref[k]);
+ hput_tags(pos,TAG(k,hput_n(max_ref[k])-1));
+ }
+ @<cases of putting special maximum values@>@;
+ HPUTTAG(list_kind,0);
+ DBG(DBGDEF,"Writing Max Definitions End\n");
+}
+@
+
+
+\subsection{Definitions}\label{definitions}
+A definition\index{definition section} associates a reference number
+with a content node. Here is an example: A glue definition associates
+a glue number, for example 71, with a glue specification. In the long
+format this might look like ``{\tt \.{<}glue *71 4pt plus 5pt minus
+0.5pt\.{>}}'' which makes glue number 71 refer to a 4pt glue with a
+stretchability of 5pt and a shrinkability of 0.5pt.
+Such a glue definition differs from a normal glue node just by an extra
+byte value immediately following the keyword respectively start byte.
+
+Whenever we need this glue in the content section, we can say
+``{\tt \.{<}glue *71\.{>}}''. Because we restrict the number of glue definitions
+to at most 256, a single byte is sufficient to
+store the reference number. The \.{shrink} and \.{stretch} programs
+will, however, not bother to store glue definitions. Instead they will
+write them in the new format immediately to the output.
+
+The parser will handle definitions in any order, but the order is relevant
+if a definition references another definition, and of course,
+it never does any harm to present definitions in a systematic way.
+
+As a rule, the definition of a reference must always precede the
+use of that reference. While this is always the case for
+references in the content section, it restricts the use of
+references inside the definition section.
+
+The definitions for integers, dimensions, extended dimensions,
+ languages, rules, ligatures, and images are ``simple''.
+They never contain references and so it is always possible to list them first.
+The definition of glues may contain extended dimensions,
+the definitions of baselines may reference glue nodes, and
+the definitions of parameter lists contain definitions of integers, dimensions,
+and glues. So these definitions should follow in this order.
+
+The definitions of leaders and discretionary breaks allow boxes.
+While these boxes are usually
+quite simple, they may contain arbitrary references---including again
+references to leaders and discretionary breaks. So, at least in principle,
+they might impose complex (or even unsatisfiable) restrictions
+on the order of those definitions.
+
+The definitions of fonts contain not only ``simple'' definitions
+but also the definitions of interword glues and hyphens
+introducing additional ordering restrictions.
+The definition of hyphens regularly contain glyphs which in turn reference
+a font---typically the font that just gets defined.
+Therefore we relax the define before use policy for glyphs:
+Glyphs may reference a font before the font is defined.
+
+The definitions of page templates contain lists of arbitrary content
+nodes, and while the boxes inside leaders or discretionary breaks tend to be simple,
+the content of page templates is often quite complex.
+Page templates are probably the source of most ordering restrictions.
+Placing page templates towards the end of the list of definitions
+might be a good idea.
+%
+A special case are stream definitions. These occur only as part of
+the corresponding page template definition and are listed at its end.
+So references to them will occur in the page template always before their
+definition.
+%
+Finally, the definitions of page ranges always reference a page template
+and they should come after the page template definitions.
+For technical reasons explained in section~\secref{labels},
+definitions of labels and outlines come last.
+
+To avoid complex dependencies, an application can always choose not to
+use references in the definition section. There are only three types of
+nodes where references can not be avoided: fonts are referenced in glyph nodes,
+labels are referenced in outlines,
+and languages are referenced in boxes or page templates.
+Possible ordering restrictions can be satisfied if languages are defined early.
+To check the define before use policy, we use an array of bitvectors,
+but we limit checking to the first 256 references.
+We have for every reference number $|N|<256$ and every kind |K| a single
+bit which is set if and only if the corresponding reference is defined.
+
+@<definition checks@>=
+uint32_t definition_bits[0x100/32][32]={{0}};
+
+#define @[SET_DBIT(N,K)@] ((N)>0xFF?1:(definition_bits[N/32][K]|=(1<<((N)&(32-1)))))
+#define @[GET_DBIT(N,K)@] ((N)>0xFF?1:((definition_bits[N/32][K]>>((N)&(32-1)))&1))
+#define @[DEF(D,K,N)@] (D).k=K;@+ (D).n=(N);@+SET_DBIT((D).n,(D).k);\
+ DBG(DBGDEF,"Defining %s %d\n",definition_name[(D).k],(D).n);\
+ RNG("Definition",(D).n,max_fixed[(D).k]+1,max_ref[(D).k]);
+#define @[REF(K,N)@] REF_RNG(K,N);if(!GET_DBIT(N,K)) \
+ QUIT("Reference %d to %s before definition",(N),definition_name[K])
+@
+
+@<initialize definitions@>=
+definition_bits[0][int_kind]=(1<<(MAX_INT_DEFAULT+1))-1;
+definition_bits[0][dimen_kind]=(1<<(MAX_DIMEN_DEFAULT+1))-1;
+definition_bits[0][xdimen_kind]=(1<<(MAX_XDIMEN_DEFAULT+1))-1;
+definition_bits[0][glue_kind]=(1<<(MAX_GLUE_DEFAULT+1))-1;
+definition_bits[0][baseline_kind]=(1<<(MAX_BASELINE_DEFAULT+1))-1;
+definition_bits[0][page_kind]=(1<<(MAX_PAGE_DEFAULT+1))-1;
+definition_bits[0][stream_kind]=(1<<(MAX_STREAM_DEFAULT+1))-1;
+definition_bits[0][range_kind]=(1<<(MAX_RANGE_DEFAULT+1))-1;
+@
+
+\goodbreak
+\vbox{\readcode\vskip -\baselineskip\putcode}
+
+
+ at s font symbol
+@<symbols@>=
+
+%type <rf> def_node
+@
+
+@<parsing rules@>=
+def_node:
+ start FONT ref font END @| { DEF($$,font_kind,$3);@+ hput_tags($1,$4);@+}
+| start INTEGER ref integer END @| { DEF($$,int_kind,$3);@+ hput_tags($1,hput_int($4));@+}
+| start DIMEN ref dimension END @| { DEF($$,dimen_kind,$3);@+ hput_tags($1,hput_dimen($4));}
+| start LANGUAGE ref string END @| { DEF($$,language_kind,$3);@+hput_string($4); hput_tags($1,TAG(language_kind,0));}
+| start GLUE ref glue END @| { DEF($$,glue_kind,$3);@+ hput_tags($1,hput_glue(&($4)));}
+| start XDIMEN ref xdimen END @| { DEF($$,xdimen_kind,$3);@+ hput_tags($1,hput_xdimen(&($4)));}
+| start RULE ref rule END @| { DEF($$,rule_kind,$3);@+ hput_tags($1,hput_rule(&($4)));}
+| start LEADERS ref leaders END @| { DEF($$,leaders_kind,$3);@+ hput_tags($1,TAG(leaders_kind, $4));}
+| start BASELINE ref baseline END @| { DEF($$,baseline_kind,$3);@+hput_tags($1,TAG(baseline_kind, $4));@+}
+| start LIGATURE ref ligature END @| { DEF($$,ligature_kind,$3);@+hput_tags($1,hput_ligature(&($4)));}
+| start DISC ref disc END @| { DEF($$,disc_kind,$3);@+ hput_tags($1,hput_disc(&($4)));}
+| start IMAGE ref image END @| { DEF($$,image_kind,$3);@+ hput_tags($1,hput_image(&($4)));}
+| start PARAM ref parameters END @| { DEF($$,param_kind,$3);@+ hput_tags($1,hput_list($1+2,&($4)));}
+| start PAGE ref page END @| { DEF($$,page_kind,$3);@+ hput_tags($1,TAG(page_kind,0));};
+@
+
+There are a few cases where one wants to define a reference by a reference.
+For example, a \HINT\ file may want to set the {\tt parfillskip} glue to zero.
+While there are multiple ways to define the zero glue, the canonical way is a reference
+using the |zero_glue_no|. All these cases have in common that the reference to be defined
+is one of the default references and the defining reference is one of the fixed references.
+We add a few parsing rules and a testing macro for those cases where the number
+of default definitions is greater than the number of fixed definitions.
+
+@<definition checks@>=
+#define @[DEF_REF(D,K,M,N)@] DEF(D,K,M);\
+if ((M)>max_default[K]) QUIT("Defining non default reference %d for %s",M,definition_name[K]); \
+if ((N)>max_fixed[K]) QUIT("Defining reference %d for %s by non fixed reference %d",M,definition_name[K],N);
+@
+
+@<parsing rules@>=
+def_node:
+ start INTEGER ref ref END @/{DEF_REF($$,int_kind,$3,$4); hput_tags($1,TAG(int_kind,0)); }
+| start DIMEN ref ref END @/{DEF_REF($$,dimen_kind,$3,$4); hput_tags($1,TAG(dimen_kind,0)); }
+| start GLUE ref ref END @/{DEF_REF($$,glue_kind,$3,$4); hput_tags($1,TAG(glue_kind,0)); };
+@
+
+
+
+
+\goodbreak
+\vbox{\getcode\vskip -\baselineskip\writecode}
+
+@<get functions@>=
+void hget_definition(int n, uint8_t a, uint32_t node_pos)
+{@+ switch(KIND(a))
+ { case font_kind: hget_font_def(n);@+ break;
+ case param_kind:
+ {@+ list_t l; @+HGET_LIST(INFO(a),l); @+hwrite_parameters(&l); @+ break;@+}
+ case page_kind: hget_page(); @+break;
+ case dimen_kind: hget_dimen(a); @+break;
+ case xdimen_kind:
+ {@+ xdimen_t x; @+hget_xdimen(a,&x); @+hwrite_xdimen(&x); @+break;@+ }
+ case language_kind:
+ if (INFO(a)!=b000)
+ QUIT("Info value of language definition must be zero");
+ else
+ { char *n; HGET_STRING(n);@+ hwrite_string(n); }
+ break;
+ default:
+ hget_content(a); @+break;
+ }
+}
+
+
+void hget_def_node()
+{ kind_t k;
+
+ @<read the start byte |a|@>@;
+ k=KIND(a);
+ if (k==label_kind)
+ hget_outline_or_label_def(INFO(a),node_pos);
+ else
+ { int n;
+ n=HGET8;
+ if (k!=range_kind) REF_RNG(k,n);
+ SET_DBIT(n,k);
+ if (k==range_kind)
+ hget_range(INFO(a),n);
+ else
+ { hwrite_start(); @+hwritef("%s *%d",definition_name[k],n);
+ hget_definition(n,a,node_pos);
+ hwrite_end();
+ }
+ if(n>max_ref[k] || n <= max_fixed[k])
+ QUIT("Definition %d for %s out of range [%d - %d]",@|
+ n, definition_name[k],max_fixed[k]+1,max_ref[k]);
+ }
+ if (max_fixed[k]>max_default[k])
+ QUIT("Definitions for kind %s not supported", definition_name[k]);
+ @<read and check the end byte |z|@>@;
+}
+@
+
+
+
+\subsection{Parameter Lists}\label{paramlist}\index{parameter list}
+Because the content section is a ``stateless'' list of nodes, the
+definitions we see in the definition section can never change. It is
+however necessary to make occasionally local modifications of some of
+these definitions, because some definitions are parameters of the
+algorithms borrowed from \TeX. Nodes that need such modifications, for
+example the paragraph nodes that are passed to \TeX's line breaking
+algorithm, contain a list of local definitions called parameters.
+Typically sets of related parameters are needed. To facilitate a
+simple reference to such a set of parameters, we allow predefined
+parameter lists that can be referenced by a single number. The
+parameters of \TeX's routines are quite basic---integers\index{integer},
+dimensions\index{dimension}, and glues\index{glue}---and all
+of them have default values.
+Therefore we restrict the definitions in parameter lists to such
+basic definitions.
+
+@<parsing functions@>=
+void check_param_def(ref_t *df)
+{ if(df->k!=int_kind && df->k!=dimen_kind && @| df->k!=glue_kind)
+ QUIT("Kind %s not allowed in parameter list", definition_name[df->k]);
+ if(df->n<=max_fixed[df->k] || max_default[df->k]<df->n)
+ QUIT("Parameter %d for %s not allowed in parameter list", df->n, definition_name[df->k]);
+}
+@
+
+The definitions below repeat the definitions we have seen for lists in section~\secref{plainlists}
+with small modifications. For example we use the kind-value |param_kind|. An empty parameter list
+is omitted in the long format as well as in the short format.
+
+\goodbreak
+\vbox{\readcode\vskip -\baselineskip\putcode}
+
+ at s PARAM symbol
+ at s def_list symbol
+ at s parameters_node symbol
+ at s def_node symbol
+ at s parameters symbol
+ at s empty_param_list symbol
+ at s non_empty_param_list symbol
+
+@<symbols@>=
+%token PARAM "param"
+%type <u> def_list
+%type <l> parameters
+@
+
+@<scanning rules@>=
+::@=param@> :< return PARAM; >:
+@
+@<parsing rules@>=
+def_list: position @+
+ | def_list def_node {check_param_def(&($2));};
+parameters: estimate def_list { $$.p=$2; $$.k=param_kind; $$.s=(hpos-hstart)-$2;};
+@
+
+Using a parsing rule like
+``\nts{param\_list}: \nts{start} \ts{PARAM} \nts{parameters} \ts{END}'',
+an empty parameter list will be written as ``\.{<param>}''.
+This looks ugly and seems like unnecessary syntax. It would be nicer
+if an empty parameter list could simply be omitted.
+Generating an empty parameter list for an omitted parameter list
+is however a bit tricky.
+Consider the sequence ``\.{<param\dots>} \.{<hbox\dots>}'' versus
+the sequence ``\.{<hbox\dots>}''. In the latter case,
+the parser will notice the missing parameter list
+when it encounters the \.{hbox} token.
+Of course it is not a good idea to augment the rules for the \.{hbox} with
+a special test for the missing empty parameter list.
+It is better to insert an empty parameter list before parsing the first ``\.{<}'' token
+and remove it again if a non-empty parameter list has been detected.
+This can be accomplished by the following two rules.
+
+@<parsing rules@>=
+empty_param_list: position { HPUTX(2); hpos++; hput_tags($1,TAG(param_kind,1));};
+non_empty_param_list: start PARAM {hpos=hpos-2;} parameters END @/
+ { @+ hput_tags($1-2,hput_list($1-1,&($4)));@+};
+@
+
+\writecode
+@<write functions@>=
+void hwrite_parameters(list_t *l)
+{ uint32_t h=hpos-hstart, e=hend-hstart; /* save |hpos| and |hend| */
+ hpos=l->p+hstart;@+ hend=hpos+l->s;
+ if (l->s>0xFF) hwritef(" %d",l->s);
+ while(hpos<hend) hget_def_node();
+ hpos=hstart+h;@+ hend=hstart+e; /* restore |hpos| and |hend| */
+}
+void hwrite_param_list(list_t *l)
+{ @+if (l->s!=0) @/
+ { hwrite_start();@+
+ hwritef("param");
+ hwrite_parameters(l);
+ hwrite_end();
+ }
+}
+@
+
+\getcode
+@<get functions@>=
+void hget_param_list(list_t *l)
+{ @+if (KIND(*hpos)!=param_kind) @/
+ QUIT("Parameter list expected at 0x%x", (uint32_t)(hpos-hstart));
+ else hget_list(l);
+}
+@
+
+
+
+\subsection{Fonts}\label{fonts}
+Another definition that has no corresponding content node is the
+font\index{font} definition. Fonts by themselves do not constitute
+content, instead they are used in glyph\index{glyph} nodes.
+Further, fonts are never directly embedded in a content node; in a content node, a
+font is always specified by its font number. This limits the number of
+fonts that can be used in a \HINT\ file to at most 256.
+
+A long format font definition starts with the keyword ``\.{font}'' and
+is followed by the font number, as usual prefixed by an asterisk. Then
+comes the font specification with the font size, the font
+name, the section number of the \TeX\ font metric file, and the
+section number of the file containing the glyphs for the font.
+The \HINT\ format supports \.{.pk} files, the traditional font format
+for \TeX, and the more modern PostScript Type 1 fonts,
+TrueType fonts, and OpenType fonts.
+
+The format of font definitions will probably change in future
+versions of the \HINT\ file format.
+For example, \.{.pk} files might be replaced entirely by PostScript Type 1 fonts.
+Also \HINT\ needs the \TeX\ font metric files only to obtain the sizes
+of characters when running \TeX's line breaking algorithm.
+But for many TrueType fonts there are no \TeX\ font metric files,
+while the necessary information about character sizes should be easy
+to obtain.
+Another information, that is currently missing from font definitions,
+is the fonts character encoding.
+
+In a \HINT\ file, text is represented as a sequence of numbers called
+character codes. \HINT\ files use the UTF-8 character encoding
+scheme (CES) to map these numbers to their representation as byte
+sequences. For example the number ``|0xE4|'' is encoded as the byte
+sequence ``|0xC3| |0xA4|''. The same number |0xE4| now can represent
+different characters depending on the coded character set (CCS). For
+example in the common ISO-8859-1 (Latin 1) encoding the number |0xE4|
+is the umlaut ``\"a'' where as in the ISO-8859-7 (Latin/Greek) it is
+the Greek letter ``$\delta$'' and in the EBCDIC encoding, used on IBM
+mainframes, it is the upper case letter ``U''.
+
+The character encoding is
+irrelevant for rendering a \HINT\ file as long as the character codes
+in the glyph nodes are consistent with the character codes used in the font
+file, but the character encoding is necessary for all programs that
+need to ``understand'' the content of the \HINT\ file. For example
+programs that want to translate a \HINT\ document to a different language,
+or for text-to-speech conversion.
+
+The Internet Engineering Task Force IETF has established a character set
+registry\cite{ietf:charset-mib} that defines an enumeration of all
+registered coded character sets\cite{iana:charset-mib}. The coded
+character set numbers are in the range 1--2999.
+This encoding number, as given in~\cite{iana:charset},
+might be one possibility for specifying the font encoding as
+part of a font definition.
+
+Currently, it is only required that a font specifies
+an interword glue and a default discretionary break. After that comes
+a list of up to 12 font specific parameters.
+
+The font size specifies the desired ``at size''\index{font at size}
+which might be different from the ``design size''\index{font design size}
+of the font as stored in the \.{.tfm} file.
+
+In the short format, the font specification is given in the same order
+as in the long format.
+
+Our internal representation of a font just stores the font name
+because in the long format we add the font name as a comment to glyph
+nodes.
+
+
+@<common variables@>=
+char **hfont_name; /* dynamically allocated array of font names */
+@
+
+@<hint basic types@>=
+#define MAX_FONT_PARAMS 11
+@
+
+@<initialize definitions@>=
+ALLOCATE(hfont_name,max_ref[font_kind]+1,char *);
+@
+
+
+
+\readcode
+ at s FONT symbol
+ at s fref symbol
+ at s font_param_list symbol
+ at s font_param symbol
+ at s font_head symbol
+
+@<symbols@>=
+%token FONT "font"
+%type <info> font font_head
+@
+
+@<scanning rules@>=
+::@=font@> :< return FONT; >:
+@
+
+Note that we set the definition bit early because the definition of font |f|
+might involve glyphs that reference font |f| (or other fonts).
+
+@<parsing rules@>=@/
+
+font: font_head font_param_list;
+
+font_head: string dimension UNSIGNED UNSIGNED @/
+ {uint8_t f=$<u>@&0; SET_DBIT(f,font_kind); @+hfont_name[f]=strdup($1); $$=hput_font_head(f,hfont_name[f],$2,$3,$4);};
+
+font_param_list: glue_node disc_node @+ | font_param_list font_param ;
+
+font_param: @/
+ start PENALTY fref penalty END { hput_tags($1,hput_int($4));}
+| start KERN fref kern END { hput_tags($1,hput_kern(&($4)));}
+| start LIGATURE fref ligature END { hput_tags($1,hput_ligature(&($4)));}
+| start DISC fref disc END { hput_tags($1,hput_disc(&($4)));}
+| start GLUE fref glue END { hput_tags($1,hput_glue(&($4)));}
+| start LANGUAGE fref string END { hput_string($4);hput_tags($1,TAG(language_kind,0));}
+| start RULE fref rule END { hput_tags($1,hput_rule(&($4)));}
+| start IMAGE fref image END { hput_tags($1,hput_image(&($4)));};
+
+fref: ref @/{ RNG("Font parameter",$1,0,MAX_FONT_PARAMS); };
+@
+
+\goodbreak
+\vbox{\getcode\vskip -\baselineskip\writecode}
+
+@<get functions@>=
+static void hget_font_params(void)
+{ disc_t h;
+ hget_glue_node();
+ hget_disc_node(&(h));@+ hwrite_disc_node(&h);
+ DBG(DBGDEF,"Start font parameters\n");
+ while (KIND(*hpos)!=font_kind)@/
+ { ref_t df;
+ @<read the start byte |a|@>@;
+ df.k=KIND(a);
+ df.n=HGET8;
+ DBG(DBGDEF,"Reading font parameter %d: %s\n",df.n, definition_name[df.k]);
+ if (df.k!=penalty_kind && df.k!=kern_kind && df.k!=ligature_kind && @|
+ df.k!=disc_kind && df.k!=glue_kind && df.k!=language_kind && @| df.k!=rule_kind && df.k!=image_kind)
+ QUIT("Font parameter %d has invalid type %s",df.n, content_name[df.n]);
+ RNG("Font parameter",df.n,0,MAX_FONT_PARAMS);
+ hwrite_start(); @+ hwritef("%s *%d",content_name[KIND(a)],df.n);
+ hget_definition(df.n,a,node_pos);
+ hwrite_end();
+ @<read and check the end byte |z|@>@;
+ }
+ DBG(DBGDEF,"End font parameters\n");
+}
+
+
+void hget_font_def(uint8_t f)
+{ char *n; @+dimen_t s=0;@+uint16_t m,y;
+ HGET_STRING(n);@+ hwrite_string(n);@+ hfont_name[f]=strdup(n);
+ HGET32(s); @+ hwrite_dimension(s);
+ DBG(DBGDEF,"Font %s size 0x%x\n", n, s);
+ HGET16(m); @+RNG("Font metrics",m,3,max_section_no);
+ HGET16(y); @+RNG("Font glyphs",y,3,max_section_no);
+ hwritef(" %d %d",m,y);
+ hget_font_params();
+ DBG(DBGDEF,"End font definition\n");
+}
+@
+
+\putcode
+@<put functions@>=
+uint8_t hput_font_head(uint8_t f, char *n, dimen_t s, @| uint16_t m, uint16_t y)
+{ info_t i=b000;
+ DBG(DBGDEF,"Defining font %d (%s) size 0x%x\n", f, n, s);
+ hput_string(n);
+ HPUT32(s);@+
+ HPUT16(m); @+HPUT16(y);
+ return TAG(font_kind,i);
+}
+@
+
+
+
+\subsection{References}\label{reference}
+We have seen how to make definitions, now let's see how to
+reference\index{reference} them. In the long form, we can simply
+write the reference number, after the keyword like this:
+``{\tt \.{<}glue *17\.{>}}''.
+The asterisk\index{asterisk} is necessary to keep apart,
+for example, a penalty with value 50,
+written ``{\tt \.{<}penalty 50\.{>}}'',
+from a penalty referencing the integer
+definition number 50, written ``{\tt \.{<}penalty *50\.{>}}''.
+
+\goodbreak
+\vbox{\readcode\vskip -\baselineskip\putcode}
+
+@<parsing rules@>=
+xdimen_ref: ref { REF(xdimen_kind,$1);};
+param_ref: ref { REF(param_kind,$1); };
+stream_ref: ref { REF_RNG(stream_kind,$1); };
+
+
+content_node:
+ start PENALTY ref END @/{ REF(penalty_kind,$3); @+ hput_tags($1,TAG(penalty_kind,0)); }
+|start KERN explicit ref END @/
+ { REF(dimen_kind,$4); @+ hput_tags($1,TAG(kern_kind,($3)?b100:b000)); }
+|start KERN explicit XDIMEN ref END @/
+ { REF(xdimen_kind,$5); @+hput_tags($1,TAG(kern_kind,($3)?b101:b001)); }
+|start GLUE ref END @/{ REF(glue_kind,$3); @+ hput_tags($1,TAG(glue_kind,0)); }
+|start LIGATURE ref END @/{ REF(ligature_kind,$3); @+ hput_tags($1,TAG(ligature_kind,0)); }
+|start DISC ref END @/{ REF(disc_kind,$3); @+ hput_tags($1,TAG(disc_kind,0)); }
+|start RULE ref END @/{ REF(rule_kind,$3); @+ hput_tags($1,TAG(rule_kind,0)); }
+|start IMAGE ref END @/{ REF(image_kind,$3);@+ hput_tags($1,TAG(image_kind,0)); }
+|start LEADERS ref END @/{ REF(leaders_kind,$3); @+ hput_tags($1,TAG(leaders_kind,0)); }
+|start BASELINE ref END @/{ REF(baseline_kind,$3);@+ hput_tags($1,TAG(baseline_kind,0)); }
+|start LANGUAGE REFERENCE END @/{ REF(language_kind,$3);@+ hput_tags($1,hput_language($3)); };
+
+glue_node: start GLUE ref END @/{ REF(glue_kind,$3);
+if ($3==zero_skip_no) { hpos=hpos-2; $$=false;@+ }
+else {hput_tags($1,TAG(glue_kind,0)); $$=true;@t\2@>@+}};
+
+@
+
+\getcode
+@<cases to get content@>=
+ at t\1\kern1em@>
+case TAG(penalty_kind,0): HGET_REF(penalty_kind); @+break;
+case TAG(kern_kind,b000): HGET_REF(dimen_kind); @+break;
+case TAG(kern_kind,b100): hwritef(" !"); @+HGET_REF(dimen_kind); @+break;
+case TAG(kern_kind,b001): @| hwritef(" xdimen");@+ HGET_REF(xdimen_kind); @+break;
+case TAG(kern_kind,b101): @| hwritef(" ! xdimen");@+ HGET_REF(xdimen_kind); @+break;
+case TAG(ligature_kind,0): HGET_REF(ligature_kind); @+break;
+case TAG(disc_kind,0): HGET_REF(disc_kind); @+break;
+case TAG(glue_kind,0): HGET_REF(glue_kind); @+break;
+case TAG(language_kind,b000): HGET_REF(language_kind); @+break;
+case TAG(rule_kind,0): HGET_REF(rule_kind); @+break;
+case TAG(image_kind,0): HGET_REF(image_kind); @+break;
+case TAG(leaders_kind,0): HGET_REF(leaders_kind); @+break;
+case TAG(baseline_kind,0): HGET_REF(baseline_kind); @+break;
+@
+
+@<get macros@>=
+#define @[HGET_REF(K)@] {uint8_t n=HGET8;@+ REF(K,n); @+hwrite_ref(n);@+}
+@
+\writecode
+@<write functions@>=
+void hwrite_ref(int n)
+{hwritef(" *%d",n);@+}
+
+void hwrite_ref_node(kind_t k, uint8_t n)
+{ hwrite_start(); @+hwritef("%s",content_name[k]);@+ hwrite_ref(n); @+hwrite_end();}
+@
+
+
+
+\section{Defaults}\label{defaults}\index{default value}
+Several of the predefined values found in the definition section are used
+as parameters for the routines borrowed from \TeX\ to display the content
+of a \HINT\ file. These values must be defined, but it is inconvenient if
+the same standard definitions need to be placed in each and every \HINT\ file.
+Therefore we specify in this chapter reasonable default values.
+As a consequence, even a \HINT\ file without any definitions should
+produce sensible results when displayed.
+
+The definitions that have default values are integers, dimensions,
+extended dimensions, glues, baselines, labels, page templates,
+streams, and page ranges.
+Each of these defaults has its own subsection below.
+Actually the defaults for extended dimensions, baselines, and labels
+ are not needed by \TeX's routines, but it is nice to have default
+values for the extended dimensions that represent
+\.{hsize}, \.{vsize}, a zero baseline skip, and a label for the table
+of content.
+
+The array |max_default| contains for each kind-value the maximum number of
+the default values. The function |hset_max| is used to initialize them.
+
+The programs \.{shrink} and \.{stretch} actually do not use the defaults,
+but it would be possible to suppress definitions if the defined value
+is the same as the default value.
+%
+We start by setting |max_default[k]==-1|, meaning no defaults,
+and |max_fixed[k]==0x10000|, meaning no definitions.
+The following subsections will then overwrite these values for
+all kinds of definitions that have defaults.
+It remains to reset |max_fixed| to $-1$ for all those kinds
+that have no defaults but allow definitions.
+
+@<take care of variables without defaults@>=
+ for (k=0; k<32; k++) max_default[k]=-1,max_fixed[k]=0x10000;
+ @/@t}$\hangindent=1em${@>max_fixed[font_kind]= max_fixed[ligature_kind]= max_fixed[disc_kind]
+ @|=max_fixed[language_kind]=max_fixed[rule_kind]= max_fixed[image_kind]
+ @|= max_fixed[leaders_kind]= max_fixed[param_kind]=max_fixed[label_kind]@|= -1;
+@
+
+
+\subsection{Integers}
+Integers\index{integer} are very simple objects, and it might be tempting not to
+use predefined integers at all. But the \TeX\ typesetting engine,
+which is used by \HINT, uses many integer parameters to fine tune
+its operations. As we will see, all these integer parameters have a predefined
+integer number that refers to an integer definition.
+
+Integers and penalties\index{penalty} share the same kind-value. So a penalty node that references
+one of the predefined penalties, simply contains the integer number as a reference
+number.
+
+The following integer numbers are predefined.
+The zero integer is fixed with integer number zero. %It is never redefined.
+The default values are taken from {\tt plain.tex}.
+
+@<default names@>=
+typedef enum {@t}$\hangindent=2em${@>
+ zero_int_no=0,
+ pretolerance_no=1,
+ tolerance_no=2,
+ line_penalty_no=3,
+ hyphen_penalty_no=4,
+ ex_hyphen_penalty_no=5,
+ club_penalty_no=6,
+ widow_penalty_no=7,
+ display_widow_penalty_no=8,
+ broken_penalty_no=9,
+ pre_display_penalty_no=10,
+ post_display_penalty_no=11,
+ inter_line_penalty_no=12,
+ double_hyphen_demerits_no=13,
+ final_hyphen_demerits_no=14,
+ adj_demerits_no=15,
+ looseness_no=16,
+ time_no=17,
+ day_no=18,
+ month_no=19,
+ year_no=20,
+ hang_after_no=21,
+ floating_penalty_no=22
+} int_no_t;
+#define MAX_INT_DEFAULT floating_penalty_no
+@
+
+@<define |int_defaults|@>=
+max_default[int_kind]=MAX_INT_DEFAULT;
+max_fixed[int_kind]=zero_int_no;
+int_defaults[zero_int_no]=0;
+int_defaults[pretolerance_no]=100;
+int_defaults[tolerance_no]=200;
+int_defaults[line_penalty_no]=10;
+int_defaults[hyphen_penalty_no]=50;
+int_defaults[ex_hyphen_penalty_no]=50;
+int_defaults[club_penalty_no]=150;
+int_defaults[widow_penalty_no]=150;
+int_defaults[display_widow_penalty_no]=50;
+int_defaults[broken_penalty_no]=100;
+int_defaults[pre_display_penalty_no]=10000;
+int_defaults[post_display_penalty_no]=0;
+int_defaults[inter_line_penalty_no]=0;
+int_defaults[double_hyphen_demerits_no]=10000;
+int_defaults[final_hyphen_demerits_no]=5000;
+int_defaults[adj_demerits_no]=10000;
+int_defaults[looseness_no]=0;
+int_defaults[time_no]=720;
+int_defaults[day_no]=4;
+int_defaults[month_no]=7;
+int_defaults[year_no]=1776;
+int_defaults[hang_after_no]=1;
+int_defaults[floating_penalty_no]=20000;
+@#
+
+printf("int32_t int_defaults[MAX_INT_DEFAULT+1]={");
+for (i=0; i<= max_default[int_kind];i++)@/
+{ printf("%d",int_defaults[i]);@+
+ if (i<max_default[int_kind]) printf(", ");@+
+}
+printf("};\n\n");
+@
+
+\subsection{Dimensions}
+
+Notice that there are default values for the two dimensions \.{hsize} and \.{vsize}.
+These are the ``design sizes'' for the hint file. While it might not be possible
+to display the \HINT\ file using these values of \.{hsize} and \.{vsize},
+these are the author's recommendation for the best ``viewing experience''.
+
+\noindent
+@<default names@>=
+typedef enum {@t}$\hangindent=2em${@>
+zero_dimen_no=0,
+hsize_dimen_no=1,
+vsize_dimen_no=2,
+line_skip_limit_no=3,
+max_depth_no=4,
+split_max_depth_no=5,
+hang_indent_no=6,
+emergency_stretch_no=7,
+quad_no=8,
+math_quad_no=9
+} dimen_no_t;
+#define MAX_DIMEN_DEFAULT math_quad_no
+@
+
+@<define |dimen_defaults|@>=
+max_default[dimen_kind]=MAX_DIMEN_DEFAULT;
+max_fixed[dimen_kind]=zero_dimen_no;@#
+dimen_defaults[zero_dimen_no]=0;
+dimen_defaults[hsize_dimen_no]=(dimen_t)(6.5*72.27*ONE);
+dimen_defaults[vsize_dimen_no]=(dimen_t)(8.9*72.27*ONE);
+dimen_defaults[line_skip_limit_no]=0;
+dimen_defaults[split_max_depth_no]=(dimen_t)(3.5*ONE);
+dimen_defaults[hang_indent_no]=0;
+dimen_defaults[emergency_stretch_no]=0;
+dimen_defaults[quad_no]=10*ONE;
+dimen_defaults[math_quad_no]=10*ONE;@#
+
+printf("dimen_t dimen_defaults[MAX_DIMEN_DEFAULT+1]={");
+for (i=0; i<= max_default[dimen_kind];i++)
+{ printf("0x%x",dimen_defaults[i]);
+ if (i<max_default[dimen_kind]) printf(", ");
+}
+printf("};\n\n");
+@
+
+\subsection{Extended Dimensions}
+Extended dimensions\index{extended dimension} can be used in a variety of nodes for example
+kern\index{kern} and box\index{box} nodes.
+We define three fixed extended dimensions: zero, hsize, and vsize.
+In contrast to the \.{hsize} and \.{vsize} dimensions defined in the previous
+section, the extended dimensions defined here are linear functions that always evaluate
+to the current horizontal and vertical size in the viewer.
+
+@<default names@>=
+typedef enum {
+zero_xdimen_no=0,
+hsize_xdimen_no=1,
+vsize_xdimen_no=2
+} xdimen_no_t;
+#define MAX_XDIMEN_DEFAULT vsize_xdimen_no
+@
+
+@<define |xdimen_defaults|@>=
+max_default[xdimen_kind]=MAX_XDIMEN_DEFAULT;
+max_fixed[xdimen_kind]=vsize_xdimen_no;@#
+
+printf("xdimen_t xdimen_defaults[MAX_XDIMEN_DEFAULT+1]={"@/
+"{0x0, 0.0, 0.0}, {0x0, 1.0, 0.0}, {0x0, 0.0, 1.0}"@/
+"};\n\n");
+@
+
+
+\subsection{Glue}
+
+There are predefined glue\index{glue} numbers that correspond to the skip parameters of \TeX.
+The default values are taken from {\tt plain.tex}.
+
+@<default names@>=
+typedef enum {@t}$\hangindent=2em${@>
+zero_skip_no=0,
+fil_skip_no=1,
+fill_skip_no=2,
+line_skip_no=3,
+baseline_skip_no=4,
+above_display_skip_no=5,
+below_display_skip_no=6,
+above_display_short_skip_no=7,
+below_display_short_skip_no=8,
+left_skip_no=9,
+right_skip_no=10,
+top_skip_no=11,
+split_top_skip_no=12,
+tab_skip_no=13,
+par_fill_skip_no=14
+} glue_no_t;
+#define MAX_GLUE_DEFAULT par_fill_skip_no
+@
+
+@<define |glue_defaults|@>=
+max_default[glue_kind]=MAX_GLUE_DEFAULT;
+max_fixed[glue_kind]=fill_skip_no;
+
+glue_defaults[fil_skip_no].p.f=1.0;
+glue_defaults[fil_skip_no].p.o=fil_o;
+
+glue_defaults[fill_skip_no].p.f=1.0;
+glue_defaults[fill_skip_no].p.o=fill_o;@#
+
+glue_defaults[line_skip_no].w.w=1*ONE;
+glue_defaults[baseline_skip_no].w.w=12*ONE;
+
+glue_defaults[above_display_skip_no].w.w=12*ONE;
+glue_defaults[above_display_skip_no].p.f=3.0;
+glue_defaults[above_display_skip_no].p.o=normal_o;
+glue_defaults[above_display_skip_no].m.f=9.0;
+glue_defaults[above_display_skip_no].m.o=normal_o;
+
+glue_defaults[below_display_skip_no].w.w=12*ONE;
+glue_defaults[below_display_skip_no].p.f=3.0;
+glue_defaults[below_display_skip_no].p.o=normal_o;
+glue_defaults[below_display_skip_no].m.f=9.0;
+glue_defaults[below_display_skip_no].m.o=normal_o;
+
+glue_defaults[above_display_short_skip_no].p.f=3.0;
+glue_defaults[above_display_short_skip_no].p.o=normal_o;
+
+glue_defaults[below_display_short_skip_no].w.w=7*ONE;
+glue_defaults[below_display_short_skip_no].p.f=3.0;
+glue_defaults[below_display_short_skip_no].p.o=normal_o;
+glue_defaults[below_display_short_skip_no].m.f=4.0;
+glue_defaults[below_display_short_skip_no].m.o=normal_o;
+
+glue_defaults[top_skip_no].w.w=10*ONE;
+glue_defaults[split_top_skip_no].w.w=(dimen_t)8.5*ONE;
+
+glue_defaults[par_fill_skip_no].p.f=1.0;
+glue_defaults[par_fill_skip_no].p.o=fil_o;
+
+#define @[PRINT_GLUE(G)@] \
+ @[printf("{{0x%x, %f, %f},{%f, %d},{%f, %d}}",\
+ G.w.w, G.w.h, G.w.v, G.p.f, G.p.o, G.m.f,G.m.o)@]@#
+
+printf("glue_t glue_defaults[MAX_GLUE_DEFAULT+1]={\n");
+for (i=0; i<= max_default[glue_kind];i++)@/
+{ PRINT_GLUE(glue_defaults[i]); @+
+ if (i<max_default[int_kind]) printf(",\n");
+}
+printf("};\n\n");
+@
+
+We fix the glue definition with number zero to be the ``zero glue'': a
+glue with width zero and zero stretchability and shrinkability. Here
+is the reason: In the short format, the info bits of a glue node
+indicate which components of a glue are nonzero. Therefore the zero
+glue should have an info value of zero---which on the other hand is
+reserved for a reference to a glue definition. Hence, the best way to
+represent a zero glue is as a predefined glue.
+
+
+\subsection{Baseline Skips}
+
+The zero baseline\index{baseline skip} which inserts no baseline skip is predefined.
+
+@<default names@>=
+typedef enum {@+
+zero_baseline_no=0 at +
+} baseline_no_t;
+#define MAX_BASELINE_DEFAULT zero_baseline_no
+@
+@<define |baseline_defaults|@>=
+max_default[baseline_kind]=MAX_BASELINE_DEFAULT;
+max_fixed[baseline_kind]=zero_baseline_no;@#
+{ baseline_t z={{{0}}};
+ printf("baseline_t baseline_defaults[MAX_BASELINE_DEFAULT+1]={{");
+ PRINT_GLUE(z.bs); @+printf(", "); @+PRINT_GLUE(z.ls); printf(", 0x%x}};\n\n",z.lsl);
+}
+@
+
+\subsection{Labels}
+The zero label\index{label} is predefined. It should point to the
+``home'' position of the document which should be the position
+where a user can start reading or navigating the document.
+For a short document this is usually the start of the document,
+and hence, the default is the first position of the content section.
+For a larger document, the home position could point to the
+table of content where a reader will find links to other parts
+of the document.
+
+@<default names@>=
+typedef enum {@+
+zero_label_no=0 at +
+} label_no_t;
+#define MAX_LABEL_DEFAULT zero_label_no
+@
+@<define |label_defaults|@>=
+max_default[label_kind]=MAX_LABEL_DEFAULT;
+printf("label_t label_defaults[MAX_LABEL_DEFAULT+1]="@|"{{0,LABEL_TOP,true,0,0,0}};\n\n");
+@
+
+
+\subsection{Streams}
+The zero stream\index{stream} is predefined for the main content.
+@<default names@>=
+typedef enum {@+
+zero_stream_no=0 at +
+} stream_no_t;
+#define MAX_STREAM_DEFAULT zero_stream_no
+@
+
+@<define stream defaults@>=
+max_default[stream_kind]=MAX_STREAM_DEFAULT;
+max_fixed[stream_kind]=zero_stream_no;
+@
+
+
+\subsection{Page Templates}
+
+The zero page template\index{template} is a predefined, built-in page template.
+@<default names@>=
+typedef enum {@+
+zero_page_no=0 at +
+} page_no_t;
+#define MAX_PAGE_DEFAULT zero_page_no
+@
+
+@<define page defaults@>=
+max_default[page_kind]=MAX_PAGE_DEFAULT;
+max_fixed[page_kind]=zero_page_no;
+@
+
+\subsection{Page Ranges}
+
+The page\index{page range} range for the zero page template is
+the entire content section.
+
+@<default names@>=
+typedef enum {@+
+zero_range_no=0 at +
+} range_no_t;
+#define MAX_RANGE_DEFAULT zero_range_no
+@
+
+@<define range defaults@>=
+max_default[range_kind]=MAX_RANGE_DEFAULT;
+max_fixed[range_kind]=zero_range_no;
+@
+
+
+\section{Content Section}
+The content section\index{content section} is just a list of nodes.
+Within the \.{shrink} program,
+reading a node in long format will trigger writing the node in short format.
+Similarly within the \.{stretch} program, reading a node
+in short form will cause writing it in long format. As a consequence,
+the main task of writing the content section in long format is accomplished
+by calling |get_content| and writing it in the short format
+is accomplished by parsing the |content_list|.
+
+%\readcode
+\codesection{\redsymbol}{Reading the Long Format}\redindex{1}{6}{Content Section}
+\label{content}%
+ at s CONTENT symbol
+@<symbols@>=
+%token CONTENT "content"
+@
+
+@<scanning rules@>=
+::@=content@> :< return CONTENT; >:
+@
+
+
+@<parsing rules@>=
+content_section: START CONTENT { hput_content_start(); } @| content_list END @|
+ { hput_content_end(); hput_range_defs(); hput_label_defs(); };
+@
+
+%\writecode
+\codesection{\wrtsymbol}{Writing the Long Format}\wrtindex{1}{6}{Content Section}
+
+@<write functions@>=
+void hwrite_content_section(void)
+{ section_no=2;
+ hwritef("<content");
+ hsort_ranges();
+ hsort_labels();
+ hget_content_section();
+ hwritef("\n>\n");
+}
+@
+
+%\getcode
+\codesection{\getsymbol}{Reading the Short Format}\getindex{1}{6}{Content Section}
+@<get functions@>=
+void hget_content_section()
+{ DBG(DBGBASIC|DBGDIR,"Content\n");
+ hget_section(2);
+ hwrite_range();
+ hwrite_label();
+ while(hpos<hend)
+ hget_content_node();
+}
+@
+
+%\putcode
+\codesection{\putsymbol}{Writing the Short Format}\putindex{1}{6}{Content Section}
+@<put functions@>=
+void hput_content_start(void)
+{ DBG(DBGDIR,"Content Section\n");
+ section_no=2;
+ hpos0=hpos=hstart=dir[2].buffer;
+ hend=hstart+dir[2].bsize;
+
+}
+void hput_content_end(void)
+{
+ dir[2].size=hpos-hstart; /* Updating the directory entry */
+ DBG(DBGDIR,"End Content Section, size=0x%x\n", dir[2].size);
+}
+@
+
+
+\section{Processing the Command Line}
+The following code explains the command line\index{command line}
+parameters and options\index{option}\index{debugging}.
+It tells us what to expect in the rest of this section.
+{\def\SP{\hskip .5em}
+@<explain usage@>=
+ fprintf(stderr,
+ "Usage: %s [options] filename%s\n",prog_name, in_ext);@/
+ fprintf(stderr,
+ "Options:\n"@/
+ "\t -o file\t specify an output file name\n"@/
+ "\t -g \t assume global names for auxiliary files\n"@/
+ "\t -l \t redirect stderr to a log file\n"@/
+ "\t -u \t enable writing utf8 character codes\n"@/
+ "\t -x \t enable writing hexadecimal character codes\n"@/
+ "\t -c \t enable compression of section 1 and 2\n");@/
+#ifdef DEBUG
+fprintf(stderr,"\t -d XXX \t hexadecimal value. OR together these values:\n");@/
+fprintf(stderr,"\t\t\t XX=%03X basic debugging\n", DBGBASIC);@/
+fprintf(stderr,"\t\t\t XX=%03X tag debugging\n", DBGTAGS);@/
+fprintf(stderr,"\t\t\t XX=%03X node debugging\n",DBGNODE);@/
+fprintf(stderr,"\t\t\t XX=%03X definition debugging\n", DBGDEF);@/
+fprintf(stderr,"\t\t\t XX=%03X directory debugging\n", DBGDIR);@/
+fprintf(stderr,"\t\t\t XX=%03X range debugging\n",DBGRANGE);@/
+fprintf(stderr,"\t\t\t XX=%03X float debugging\n", DBGFLOAT);@/
+fprintf(stderr,"\t\t\t XX=%03X compression debugging\n", DBGCOMPRESS);@/
+fprintf(stderr,"\t\t\t XX=%03X buffer debugging\n", DBGBUFFER);@/
+fprintf(stderr,"\t\t\t XX=%03X flex debugging\n", DBGFLEX);@/
+fprintf(stderr,"\t\t\t XX=%03X bison debugging\n", DBGBISON);@/
+fprintf(stderr,"\t\t\t XX=%03X TeX debugging\n", DBGTEX);@/
+fprintf(stderr,"\t\t\t XX=%03X Page debugging\n", DBGPAGE);@/
+fprintf(stderr,"\t\t\t XX=%03X Font debugging\n", DBGFONT);@/
+fprintf(stderr,"\t\t\t XX=%03X Render debugging\n", DBGRENDER);@/
+fprintf(stderr,"\t\t\t XX=%03X Label debugging\n", DBGLABEL);@/
+#endif
+@
+}
+We define constants for different debug flags.
+@<debug constants@>=
+#define DBGNONE 0x0
+#define DBGBASIC 0x1
+#define DBGTAGS 0x2
+#define DBGNODE 0x4
+#define DBGDEF 0x8
+#define DBGDIR 0x10
+#define DBGRANGE 0x20
+#define DBGFLOAT 0x40
+#define DBGCOMPRESS 0x80
+#define DBGBUFFER 0X100
+#define DBGFLEX 0x200
+#define DBGBISON 0x400
+#define DBGTEX 0x800
+#define DBGPAGE 0x1000
+#define DBGFONT 0x2000
+#define DBGRENDER 0x4000
+#define DBGLABEL 0x8000
+@
+
+Next we define common variables that are
+needed in all three programs defined here.
+
+@<common variables@>=
+unsigned int debugflags=DBGNONE;
+int option_utf8=false;
+int option_hex=false;
+int option_force=false;
+int option_global=false;
+int option_compress=false;
+char *stem_name=NULL;
+int stem_length=0;
+@
+The variable |stem_name| contains the name of the input file
+not including the extension. The space allocated for it
+is large enough to append an extension with up to five characters.
+It can be used with the extension {\tt .log} for the log file,
+with {\tt .HINT} or {\tt .hnt} for the output file,
+and with {\tt .abs} or {\tt .rel} when writing or reading the auxiliary sections.
+The {\tt stretch} program will overwrite the |stem_name|
+using the name of the output file if it is set with the {\tt -o}
+option.
+
+
+Next are the variables that are local in the |main| program.
+@<local variables in |main|@>=
+char *prog_name;
+char *in_ext;
+char *out_ext;
+char *file_name=NULL;
+int file_name_length=0;
+int option_log=false;
+@
+
+Processing the command line looks for options and then sets the
+input file name\index{file name}.
+
+@<process the command line@>=
+ debugflags=DBGBASIC;
+ prog_name=argv[0];
+ if (argc < 2) goto explain_usage;
+ argv++; /* skip the program name */
+ while (*argv!=NULL)
+ { if ((*argv)[0]=='-')
+ { char option=(*argv)[1];
+ switch(option)
+ { default: goto explain_usage;
+ case 'o': argv++;
+ file_name_length=(int)strlen(*argv);
+ ALLOCATE(file_name,file_name_length+6,char); /*plus extension*/
+ strcpy(file_name,*argv);@+ break;
+ case 'l': option_log=true; @+break;
+ case 'u': option_utf8=true;@+break;
+ case 'x': option_hex=true;@+break;
+ case 'f': option_force=true; @+break;
+ case 'g': option_global=true; @+break;
+ case 'c': option_compress=true; @+break;
+ case 'd': @/
+ argv++; if (*argv==NULL) goto explain_usage;
+ debugflags=strtol(*argv,NULL,16);
+ break;
+ }
+ }
+ else /* the input file name */
+ { int path_length=(int)strlen(*argv);
+ int ext_length=(int)strlen(in_ext);
+ ALLOCATE(hin_name,path_length+ext_length+1,char);
+ strcpy(hin_name,*argv);
+ if (path_length<ext_length
+ || strncmp(hin_name+path_length-ext_length,in_ext,ext_length)!=0)
+ { strcat(hin_name,in_ext);
+ path_length+=ext_length;
+ }
+ stem_length=path_length-ext_length;
+ ALLOCATE(stem_name,stem_length+6,char);
+ strncpy(stem_name,hin_name,stem_length);
+ stem_name[stem_length]=0;
+ if (*(argv+1)!=NULL) goto explain_usage;
+ }
+ argv++;
+ }
+ if (hin_name==NULL) goto explain_usage;
+@
+
+After the command line has been processed, three file streams need to be opened:
+The input file |hin|\index{input file} and the output file |hout|\index{output file}.
+Further we need a log file |hlog|\index{log file} if debugging is enabled.
+For technical reasons, the scanner\index{scanning} generated by \.{flex} needs
+an input file |yyin|\index{input file} which is set to |hin|
+and an output file |yyout| (which is not used).
+
+@<common variables@>=
+FILE *hin=NULL, *hout=NULL, *hlog=NULL;
+@
+
+
+The log file is opened first because
+this is the place where error messages\index{error message}
+should go while the other files are opened.
+It inherits its name from the input file name.
+
+@<open the log file@> =
+#ifdef DEBUG
+ if (option_log)
+ {
+ strcat(stem_name,".log");
+ hlog=freopen(stem_name,"w",stderr);
+ if (hlog==NULL)
+ { fprintf(stderr,"Unable to open logfile %s",stem_name);
+ hlog=stderr;
+ }
+ stem_name[stem_length]=0;
+ }
+ else
+ hlog=stderr;
+#else
+ hlog=stderr;
+#endif
+@
+
+Once we have established logging, we can try to open the other files.
+@<open the input file@>=
+ hin=fopen(hin_name,"rb");
+ if (hin==NULL) QUIT("Unable to open input file %s",hin_name);
+@
+
+@<open the output file@>=
+ if (file_name!=NULL)
+ { int ext_length=(int)strlen(out_ext);
+ if (file_name_length<=ext_length
+ || strncmp(file_name+file_name_length-ext_length,out_ext,ext_length)!=0)
+ { strcat(file_name,out_ext); file_name_length+=ext_length; }
+ }
+ else
+ { file_name_length=stem_length+(int)strlen(out_ext);
+ ALLOCATE(file_name,file_name_length+1,char);
+ strcpy(file_name,stem_name);@+
+ strcpy(file_name+stem_length,out_ext);
+ }
+ @<make sure the path in |file_name| exists@>@;
+ hout=fopen(file_name,"wb");
+ if (hout==NULL) QUIT("Unable to open output file %s",file_name);
+@
+
+The {\tt stretch} program will replace the |stem_name| using the stem of the
+output file.
+@<determine the |stem_name| from the output |file_name|@>=
+stem_length=file_name_length-(int)strlen(out_ext);
+ALLOCATE(stem_name,stem_length+6,char);
+strncpy(stem_name,file_name,stem_length);
+stem_name[stem_length]=0;
+@
+
+At the very end, we will close the files again.
+@<close the input file@>=
+if (hin_name!=NULL) free(hin_name);
+if (hin!=NULL) fclose(hin);
+@
+@<close the output file@>=
+if (file_name!=NULL) free(file_name);
+if (hout!=NULL) fclose(hout);
+@
+@<close the log file@>=
+if (hlog!=NULL) fclose(hlog);
+if (stem_name!=NULL) free(stem_name);
+@
+
+
+
+\section{Error Handling and Debugging}\label{error_section}
+There is no good program without good error handling\index{error message}\index{debugging}.
+To print messages\index{message} or indicate errors, I define the following macros:
+\index{MESSAGE+\.{MESSAGE}}\index{QUIT+\.{QUIT}}
+
+@(error.h@>=
+#ifndef _ERROR_H
+#define _ERROR_H
+#include <stdlib.h>
+#include <stdio.h>
+extern FILE *hlog;
+extern uint8_t *hpos, *hstart;
+#define @[LOG(...)@] @[(fprintf(hlog,__VA_ARGS__),fflush(hlog))@]
+#define @[MESSAGE(...)@] @[(fprintf(hlog,__VA_ARGS__),fflush(hlog))@]
+#define @[QUIT(...)@] (MESSAGE("ERROR: " __VA_ARGS__),fprintf(hlog,"\n"),exit(1))
+
+#endif
+@
+
+
+The amount of debugging\index{debugging} depends on the debugging flags.
+For portability, we first define the output specifier for expressions of type |size_t|.
+\index{DBG+\.{DBG}}\index{SIZE F+\.{SIZE\_F}}\index{DBGTAG+\.{DBGTAG}}
+\index{RNG+\.{RNG}}\index{TAGERR+\.{TAGERR}}
+@<debug macros@>=
+#ifdef WIN32
+#define SIZE_F "0x%x"
+#else
+#define SIZE_F "0x%zx"
+#endif
+#ifdef DEBUG
+#define @[DBG(FLAGS,...)@] ((debugflags & (FLAGS))?LOG(__VA_ARGS__):0)
+#else
+#define @[DBG(FLAGS,...)@] 0
+#endif
+#define @[DBGTAG(A,P)@] @[DBG(DBGTAGS,@["tag [%s,%d] at " SIZE_F "\n"@],@|NAME(A),INFO(A),(P)-hstart)@]
+
+#define @[RNG(S,N,A,Z)@] @/\
+ if ((int)(N)<(int)(A)||(int)(N)>(int)(Z)) QUIT(S@, " %d out of range [%d - %d]",N,A,Z)
+
+#define @[TAGERR(A)@] @[QUIT(@["Unknown tag [%s,%d] at " SIZE_F "\n"@],NAME(A),INFO(A),hpos-hstart)@]
+@
+
+The \.{bison} generated parser will need a function |yyerror| for
+error reporting. We can define it now:
+
+@<parsing functions@>=
+extern int yylineno;
+int yyerror(const char *msg)
+{ QUIT(" in line %d %s",yylineno,msg);
+ return 0;
+}
+@
+
+To enable the generation of debugging code \.{bison} needs also the following:
+@<enable bison debugging@>=
+#ifdef DEBUG
+#define YYDEBUG 1
+extern int yydebug;
+#else
+#define YYDEBUG 0
+#endif
+@
+
+
+\appendix
+
+\section{Traversing Short Format Files}\label{fastforward}
+For applications like searching or repositioning a file after reloading
+a possibly changed version of a file, it is useful to have a fast way
+of getting from one content node to the next.
+For quite some nodes, it is possible to know the size of the
+node from the tag. So the fastest way to get to the next node
+is looking up the node size in a table.
+
+Other important nodes, for example hbox, vbox, or par nodes, end with a
+list node and it is possible to know the size of the node up to the final
+list. With that knowledge it is possible to skip the initial
+part of the node, then skip the list, and finally skip the tag byte.
+The size of the initial part can be stored in the same node size table
+using negated values. What works for lists,
+of course, will work for other kinds of nodes as well.
+So we use the lowest two bits of the values in the size table
+to store the number of embedded nodes that follow after the initial part.
+
+For list nodes neither of these methods works and these nodes can be marked
+with a zero entry in the node size table.
+
+This leads to the following code for a ``fast forward'' function
+for |hpos|:
+
+@<shared skip functions@>=
+uint32_t hff_list_pos=0, hff_list_size=0;
+uint8_t hff_tag;
+void hff_hpos(void)
+{ signed char i,k;
+ hff_tag=*hpos;@+
+ DBGTAG(hff_tag,hpos);
+ i= hnode_size[hff_tag];
+ if (i>0) {hpos=hpos+i; @+return;@+ }
+ else if (i<0)
+ { k=1+(i&0x3);@+ i=i>>2;
+ hpos=hpos-i; /* skip initial part */
+ while (k>0)
+ { hff_hpos(); @+k--; @+} /* skip trailing nodes */
+ hpos++;/* skip end byte */
+ return;
+ }
+ else if (hff_tag <=TAG(param_kind,5))
+ @<advance |hpos| over a list@>@;
+ TAGERR(hff_tag);
+}
+@
+
+
+We will put the |hnode_size| variable into the {\tt hformat.c} file
+using the following function. We add some comments and
+split negative values into their components, to make the result more
+readable.
+
+@<print the |hnode_size| variable@>=
+ printf("signed char hnode_size[0x100]= {\n");
+ for (i=0; i<=0xff; i++)@/
+ { signed char s = hnode_size[i];
+ if (s>=0) printf("%d",s); else printf("-4*%d+%d",-(s>>2),s&3);
+ if (i<0xff) printf(","); else printf("};");
+ if ((i&0x7)==0x7) printf(" /* %s */\n", content_name[KIND(i)]);
+ }
+ printf("\n\n");
+@
+
+\subsection{Lists}\index{list}\index{text}\index{parameters}
+List don't follow the usual schema of nodes. They have a variable size
+that is stored in the node. We keep position and size in global variables
+so that the list that ends a node can be conveniently located.
+
+@<advance |hpos| over a list@>=
+switch (INFO(hff_tag)){
+case 1: hff_list_pos=hpos-hstart+1;hff_list_size=0; hpos=hpos+2;@+ return;
+case 2: hpos++;@+ hff_list_size=HGET8;@+ hff_list_pos=hpos-hstart+1; hpos=hpos+1+hff_list_size+1+1+1;@+ return;
+case 3: hpos++;@+ HGET16(hff_list_size);@+hff_list_pos=hpos-hstart+1; hpos=hpos+1+hff_list_size+1+2+1;@+ return;
+case 4: hpos++;@+ HGET24(hff_list_size);@+hff_list_pos=hpos-hstart+1; hpos=hpos+1+hff_list_size+1+3+1;@+ return;
+case 5: hpos++;@+ HGET32(hff_list_size);@+hff_list_pos=hpos-hstart+1; hpos=hpos+1+hff_list_size+1+4+1;@+ return;
+}
+@
+
+Now let's consider the different kinds of nodes.
+
+\subsection{Glyphs}\index{glyph}
+We start with the glyph nodes. All glyph nodes
+have a start and an end tag, one byte for the font,
+and depending on the info from 1 to 4 bytes for the character code.
+
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(glyph_kind,1)] = 1+1+1+1;
+hnode_size[TAG(glyph_kind,2)] = 1+1+2+1;
+hnode_size[TAG(glyph_kind,3)] = 1+1+3+1;
+hnode_size[TAG(glyph_kind,4)] = 1+1+4+1;
+@
+
+\subsection{Penalties}\index{penalty}
+Penalty nodes either contain a one byte reference, a one byte number, or a two byte number.
+
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(penalty_kind,0)] = 1+1+1;
+hnode_size[TAG(penalty_kind,1)] = 1+1+1;
+hnode_size[TAG(penalty_kind,2)] = 1+2+1;
+@
+
+\subsection{Kerns}\index{kern}
+Kern nodes can contain a reference (either to a dimension or an extended dimension)
+a dimension, or an extended dimension node.
+
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(kern_kind,b000)] = 1+1+1;
+hnode_size[TAG(kern_kind,b001)] = 1+1+1;
+hnode_size[TAG(kern_kind,b010)] = 1+4+1;
+hnode_size[TAG(kern_kind,b011)] = I_T(1,1);
+hnode_size[TAG(kern_kind,b100)] = 1+1+1;
+hnode_size[TAG(kern_kind,b101)] = 1+1+1;
+hnode_size[TAG(kern_kind,b110)] = 1+4+1;
+hnode_size[TAG(kern_kind,b111)] = I_T(1,1);
+@
+
+For the two cases where a kern node contains an extended dimension,
+we use the following macro to combine the size of the initial part
+with the number of trailing nodes:
+@<skip macros@>=
+#define @[I_T(I,T)@] (((-(I))<<2)|((T)-1))
+@
+
+\subsection{Extended Dimensions}\index{extended dimension}
+Extended dimensions contain either one two or three 4 byte values depending
+on the info bits.
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(xdimen_kind,b100)] = 1+4+1;
+hnode_size[TAG(xdimen_kind,b010)] = 1+4+1;
+hnode_size[TAG(xdimen_kind,b001)] = 1+4+1;
+hnode_size[TAG(xdimen_kind,b110)] = 1+4+4+1;
+hnode_size[TAG(xdimen_kind,b101)] = 1+4+4+1;
+hnode_size[TAG(xdimen_kind,b011)] = 1+4+4+1;
+hnode_size[TAG(xdimen_kind,b111)] = 1+4+4+4+1;
+@
+
+\subsection{Language}\index{language}
+Language nodes either code the language in the info value or they contain
+a reference byte.
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(language_kind,b000)] = 1+1+1;
+hnode_size[TAG(language_kind,1)] = 1+1;
+hnode_size[TAG(language_kind,2)] = 1+1;
+hnode_size[TAG(language_kind,3)] = 1+1;
+hnode_size[TAG(language_kind,4)] = 1+1;
+hnode_size[TAG(language_kind,5)] = 1+1;
+hnode_size[TAG(language_kind,6)] = 1+1;
+hnode_size[TAG(language_kind,7)] = 1+1;
+@
+
+\subsection{Rules}\index{rule}
+Rules usually contain a reference, otherwise
+they contain either one, two, or three 4 byte values depending
+on the info bits.
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(rule_kind,b000)] = 1+1+1;
+hnode_size[TAG(rule_kind,b100)] = 1+4+1;
+hnode_size[TAG(rule_kind,b010)] = 1+4+1;
+hnode_size[TAG(rule_kind,b001)] = 1+4+1;
+hnode_size[TAG(rule_kind,b110)] = 1+4+4+1;
+hnode_size[TAG(rule_kind,b101)] = 1+4+4+1;
+hnode_size[TAG(rule_kind,b011)] = 1+4+4+1;
+hnode_size[TAG(rule_kind,b111)] = 1+4+4+4+1;
+@
+
+\subsection{Glue}\index{glue}
+Glues usually contain a reference or
+they contain either one two or three 4 byte values depending
+on the info bits, and possibly even an extended dimension node followed
+by two 4 byte values.
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(glue_kind,b000)] = 1+1+1;
+hnode_size[TAG(glue_kind,b100)] = 1+4+1;
+hnode_size[TAG(glue_kind,b010)] = 1+4+1;
+hnode_size[TAG(glue_kind,b001)] = 1+4+1;
+hnode_size[TAG(glue_kind,b110)] = 1+4+4+1;
+hnode_size[TAG(glue_kind,b101)] = 1+4+4+1;
+hnode_size[TAG(glue_kind,b011)] = 1+4+4+1;
+hnode_size[TAG(glue_kind,b111)] = I_T(1+4+4,1);
+@
+
+
+\subsection{Boxes}\index{box}
+The layout of boxes is quite complex and explained in section~\secref{boxnodes}.
+All boxes contain height and width, some contain a depth, some a shift amount,
+and some a glue setting together with glue sign and glue order.
+The last item in a box is a node list.
+
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(hbox_kind,b000)] = I_T(1+4+4,1); /* tag, height, width*/
+hnode_size[TAG(hbox_kind,b001)] = I_T(1+4+4+4,1); /* and depth */
+hnode_size[TAG(hbox_kind,b010)] = I_T(1+4+4+4,1); /* or shift */
+hnode_size[TAG(hbox_kind,b011)] = I_T(1+4+4+4+4,1); /* or both */
+hnode_size[TAG(hbox_kind,b100)] = I_T(1+4+4+5,1); /* and glue setting*/
+hnode_size[TAG(hbox_kind,b101)] = I_T(1+4+4+4+5,1); /* and depth */
+hnode_size[TAG(hbox_kind,b110)] = I_T(1+4+4+4+5,1); /* or shift */
+hnode_size[TAG(hbox_kind,b111)] = I_T(1+4+4+4+4+5,1); /*or both */
+hnode_size[TAG(vbox_kind,b000)] = I_T(1+4+4,1); /* same for vbox*/
+hnode_size[TAG(vbox_kind,b001)] = I_T(1+4+4+4,1);
+hnode_size[TAG(vbox_kind,b010)] = I_T(1+4+4+4,1);
+hnode_size[TAG(vbox_kind,b011)] = I_T(1+4+4+4+4,1);
+hnode_size[TAG(vbox_kind,b100)] = I_T(1+4+4+5,1);
+hnode_size[TAG(vbox_kind,b101)] = I_T(1+4+4+4+5,1);
+hnode_size[TAG(vbox_kind,b110)] = I_T(1+4+4+4+5,1);
+hnode_size[TAG(vbox_kind,b111)] = I_T(1+4+4+4+4+5,1);
+@
+
+\subsection{Extended Boxes}\index{extended box}
+Extended boxes start with height, width, depth, stretch, or shrink components.
+Then follows an extended dimension either as a reference or a node.
+The node ends with a list.
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(hset_kind,b000)] = I_T(1+4+4+4+4+1,1);
+hnode_size[TAG(hset_kind,b001)] = I_T(1+4+4+4+4+4+1,1);
+hnode_size[TAG(hset_kind,b010)] = I_T(1+4+4+4+4+4+1,1);
+hnode_size[TAG(hset_kind,b011)] = I_T(1+4+4+4+4+4+4+1,1);
+hnode_size[TAG(vset_kind,b000)] = I_T(1+4+4+4+4+1,1);
+hnode_size[TAG(vset_kind,b001)] = I_T(1+4+4+4+4+4+1,1);
+hnode_size[TAG(vset_kind,b010)] = I_T(1+4+4+4+4+4+1,1);
+hnode_size[TAG(vset_kind,b011)] = I_T(1+4+4+4+4+4+4+1,1);
+
+hnode_size[TAG(hset_kind,b100)] = I_T(1+4+4+4+4,2);
+hnode_size[TAG(hset_kind,b101)] = I_T(1+4+4+4+4+4,2);
+hnode_size[TAG(hset_kind,b110)] = I_T(1+4+4+4+4+4,2);
+hnode_size[TAG(hset_kind,b111)] = I_T(1+4+4+4+4+4+4,2);
+hnode_size[TAG(vset_kind,b100)] = I_T(1+4+4+4+4,2);
+hnode_size[TAG(vset_kind,b101)] = I_T(1+4+4+4+4+4,2);
+hnode_size[TAG(vset_kind,b110)] = I_T(1+4+4+4+4+4,2);
+hnode_size[TAG(vset_kind,b111)] = I_T(1+4+4+4+4+4+4,2);
+@
+
+The hpack and vpack nodes start with a shift amount and in case of vpack a depth.
+Then again an extended dimension and a list.
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(hpack_kind,b000)] = I_T(1+1,1);
+hnode_size[TAG(hpack_kind,b001)] = I_T(1+1,1);
+hnode_size[TAG(hpack_kind,b010)] = I_T(1+4+1,1);
+hnode_size[TAG(hpack_kind,b011)] = I_T(1+4+1,1);
+hnode_size[TAG(vpack_kind,b000)] = I_T(1+4+1,1);
+hnode_size[TAG(vpack_kind,b001)] = I_T(1+4+1,1);
+hnode_size[TAG(vpack_kind,b010)] = I_T(1+4+4+1,1);
+hnode_size[TAG(vpack_kind,b011)] = I_T(1+4+4+1,1);
+
+hnode_size[TAG(hpack_kind,b100)] = I_T(1,2);
+hnode_size[TAG(hpack_kind,b101)] = I_T(1,2);
+hnode_size[TAG(hpack_kind,b110)] = I_T(1+4,2);
+hnode_size[TAG(hpack_kind,b111)] = I_T(1+4,2);
+hnode_size[TAG(vpack_kind,b100)] = I_T(1+4,2);
+hnode_size[TAG(vpack_kind,b101)] = I_T(1+4,2);
+hnode_size[TAG(vpack_kind,b110)] = I_T(1+4+4,2);
+hnode_size[TAG(vpack_kind,b111)] = I_T(1+4+4,2);
+@
+
+\subsection{Leaders}\index{leaders}
+Most leader nodes will use a reference.
+Otherwise they contain a glue node followed by a box or rule node.
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(leaders_kind,b000)] = 1+1+1;
+hnode_size[TAG(leaders_kind,1)] = I_T(1,1);
+hnode_size[TAG(leaders_kind,2)] = I_T(1,1);
+hnode_size[TAG(leaders_kind,3)] = I_T(1,1);
+hnode_size[TAG(leaders_kind,b100|1)] = I_T(1,2);
+hnode_size[TAG(leaders_kind,b100|2)] = I_T(1,2);
+hnode_size[TAG(leaders_kind,b100|3)] = I_T(1,2);
+@
+
+\subsection{Baseline Skips}\index{baseline skip}
+Here we expect either a reference or two optional glue nodes followed by an optional dimension.
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(baseline_kind,b000)] = 1+1+1;
+hnode_size[TAG(baseline_kind,b001)] = 1+4+1;
+hnode_size[TAG(baseline_kind,b010)] = I_T(1,1);
+hnode_size[TAG(baseline_kind,b100)] = I_T(1,1);
+hnode_size[TAG(baseline_kind,b110)] = I_T(1,2);
+
+hnode_size[TAG(baseline_kind,b011)] = I_T(1+4,1);
+hnode_size[TAG(baseline_kind,b101)] = I_T(1+4,1);
+hnode_size[TAG(baseline_kind,b111)] = I_T(1+4,2);
+@
+
+
+\subsection{Ligatures}\index{ligature}
+As usual a reference is possible, otherwise the font is followed by character bytes
+as given by the info. Only if the info value is 7, the number of character bytes
+is stored separately.
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(ligature_kind,b000)] = 1+1+1;
+hnode_size[TAG(ligature_kind,1)] = 1+1+1+1;
+hnode_size[TAG(ligature_kind,2)] = 1+1+2+1;
+hnode_size[TAG(ligature_kind,3)] = 1+1+3+1;
+hnode_size[TAG(ligature_kind,4)] = 1+1+4+1;
+hnode_size[TAG(ligature_kind,5)] = 1+1+5+1;
+hnode_size[TAG(ligature_kind,6)] = 1+1+6+1;
+hnode_size[TAG(ligature_kind,7)] = I_T(1+1,1);
+@
+
+\subsection{Discretionary breaks}\index{discretionary break}
+The simple cases here are references, discretionary breaks
+with empty pre- and post-list, or with a zero line skip limit
+Otherwise one or two lists are followed by an optional replace count.
+
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(disc_kind,b000)] = 1+1+1;
+hnode_size[TAG(disc_kind,b010)] = I_T(1,1);
+hnode_size[TAG(disc_kind,b011)] = I_T(1,2);
+hnode_size[TAG(disc_kind,b100)] = 1+1+1;
+hnode_size[TAG(disc_kind,b110)] = I_T(1+1,1);
+hnode_size[TAG(disc_kind,b111)] = I_T(1+1,2);
+@
+
+\subsection{Paragraphs}\index{paragraph}
+Paragraph nodes contain an extended dimension, an parameter list and a list.
+The first two can be given as a reference.
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(par_kind,b000)] = I_T(1+1+1,1);
+hnode_size[TAG(par_kind,b010)] = I_T(1+1,2);
+hnode_size[TAG(par_kind,b110)] = I_T(1,3);
+hnode_size[TAG(par_kind,b100)] = I_T(1+1,2);
+@
+
+\subsection{Mathematics}\index{mathematics}\index{displayed formula}
+Displayed math needs a parameter list, either as list or as reference
+followed by an optional left or right equation number and a list.
+Text math is simpler: the only information is in the info value.
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(math_kind,b000)] = I_T(1+1,1);
+hnode_size[TAG(math_kind,b001)] = I_T(1+1,2);
+hnode_size[TAG(math_kind,b010)] = I_T(1+1,2);
+hnode_size[TAG(math_kind,b100)] = I_T(1,2);
+hnode_size[TAG(math_kind,b101)] = I_T(1,3);
+hnode_size[TAG(math_kind,b110)] = I_T(1,3);
+hnode_size[TAG(math_kind,b111)] = 1+1;
+hnode_size[TAG(math_kind,b011)] = 1+1;
+@
+
+\subsection{Adjustments}\index{adjustment}
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(adjust_kind,1)] = I_T(1,1);
+@
+
+\subsection{Tables}\index{alignment}
+Tables have an extended dimension either as a node or as a reference followed
+by two lists.
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(table_kind,b000)] = I_T(1+1,2);
+hnode_size[TAG(table_kind,b001)] = I_T(1+1,2);
+hnode_size[TAG(table_kind,b010)] = I_T(1+1,2);
+hnode_size[TAG(table_kind,b011)] = I_T(1+1,2);
+hnode_size[TAG(table_kind,b100)] = I_T(1,3);
+hnode_size[TAG(table_kind,b101)] = I_T(1,3);
+hnode_size[TAG(table_kind,b110)] = I_T(1,3);
+hnode_size[TAG(table_kind,b111)] = I_T(1,3);
+@
+Outer item nodes are lists of inner item nodes, inner item nodes are box nodes
+followed by an optional span count.
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(item_kind,b000)] = I_T(1,1); /* outer */
+hnode_size[TAG(item_kind,1)] = I_T(1,1); /* inner */
+hnode_size[TAG(item_kind,2)] = I_T(1,1);
+hnode_size[TAG(item_kind,3)] = I_T(1,1);
+hnode_size[TAG(item_kind,4)] = I_T(1,1);
+hnode_size[TAG(item_kind,5)] = I_T(1,1);
+hnode_size[TAG(item_kind,6)] = I_T(1,1);
+hnode_size[TAG(item_kind,6)] = I_T(2,1);
+@
+
+\subsection{Images}\index{image}
+If not given by a reference, images contain a section reference and optional dimensions, stretch, and shrink.
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(image_kind,b000)] = 1+1+1;
+hnode_size[TAG(image_kind,b100)] = 1+2+1;
+hnode_size[TAG(image_kind,b101)] = 1+2+4+4+1;
+hnode_size[TAG(image_kind,b110)] = 1+2+4+4+1;
+hnode_size[TAG(image_kind,b111)] = 1+2+4+4+4+4+1;
+@
+
+\subsection{Links}\index{link}
+Links contain either a 2 byte or a 1 byte reference.
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(link_kind,b000)] = 1+1+1;
+hnode_size[TAG(link_kind,b001)] = 1+2+1;
+hnode_size[TAG(link_kind,b010)] = 1+1+1;
+hnode_size[TAG(link_kind,b011)] = 1+2+1;
+@
+
+\subsection{Stream Nodes}\index{stream}
+After the stream reference follows a parameter list, either as reference
+or as a list, and then a content list.
+@<initialize the |hnode_size| array@>=
+hnode_size[TAG(stream_kind,b000)] = I_T(1+1+1,1);
+hnode_size[TAG(stream_kind,b010)] = I_T(1+1,2);
+@
+
+
+\section{Reading Short Format Files Backwards}
+This section is not really part of the file format definition, but it
+illustrates an important property of the content section in short
+format files: it can be read in both directions. This is important
+because we want to be able to start at an arbitrary point in the
+content and from there move pagewise backward.
+
+The program {\tt skip}\index{skip+{\tt skip}} described in this
+section does just that. As wee see in appendix~\secref{skip}, its
+|main| program is almost the same as the |main| program of the program
+{\tt stretch} in appendix~\secref{stretchmain}.
+The major difference is the removal of an output file
+and the replacement of the call to |hwrite_content_section| by
+a call to |hteg_content_section|.
+
+@<skip functions@>=
+static void hteg_content_section(void)
+{ hget_section(2);
+ hpos=hend;
+ while(hpos>hstart)
+ hteg_content_node();
+}
+@
+
+The functions |hteg_content_section| and |hteg_content_node| above are
+reverse versions of the functions |hget_content_section| and
+|hget_content_node|. Many such ``reverse functions'' will follow now
+and we will consistently use the same naming scheme: replacing
+``{\it get\/}`` by ``{\it teg\/}'' or ``{\tt GET}'' by ``{\tt TEG}''.
+The {\tt skip} program does not do much input
+checking; it will just extract enough information from a content node
+to skip a node and ``advance'' or better ``retreat'' to the previous
+node.
+
+@<skip functions@>=
+static void hteg_content_node(void)
+{ @<skip the end byte |z|@>@;
+ hteg_content(z);
+ @<skip and check the start byte |a|@>@;
+}
+
+static void hteg_content(uint8_t z)
+{@+ switch (z)@/
+ {
+ @<cases to skip content@>@;@t\1@>@/
+ default:
+ TAGERR(z);
+ break;@t\2@>@/
+ }
+}
+@
+
+The code to skip the end\index{end byte} byte |z| and to check the start\index{start byte} byte |a| is used repeatedly.
+
+@<skip the end byte |z|@>=
+ uint8_t a,z; /* the start and the end byte*/
+ uint32_t node_pos=hpos-hstart;
+ if (hpos<=hstart) return;
+ HTEGTAG(z);
+@
+
+@<skip and check the start byte |a|@>=
+ HTEGTAG(a);
+ if (a!=z) QUIT(@["Tag mismatch [%s,%d]!=[%s,%d] at " SIZE_F " to 0x%x\n"@],@|NAME(a),INFO(a),NAME(z),INFO(z),@|
+ hpos-hstart,node_pos-1);
+@
+
+We replace the ``{\tt GET}'' macros by the following ``{\tt TEG}'' macros:
+
+@<shared get macros@>=
+#define @[HBACK(X)@] @[((hpos-(X)<hstart)?(QUIT("HTEG underflow\n"),NULL):(hpos-=(X)))@]
+
+#define @[HTEG8@] (HBACK(1),hpos[0])
+#define @[HTEG16(X)@] (HBACK(2),(X)=(hpos[0]<<8)+hpos[1])
+#define @[HTEG24(X)@] (HBACK(3),(X)=(hpos[0]<<16)+(hpos[1]<<8)+hpos[2])
+#define @[HTEG32(X)@] (HBACK(4),(X)=(hpos[0]<<24)+(hpos[1]<<16)+(hpos[2]<<8)+hpos[3])
+#define @[HTEGTAG(X)@] @[X=HTEG8,DBGTAG(X,hpos)@]
+@
+
+Now we review step by step the different kinds of nodes.
+\subsection{Floating Point Numbers}\index{floating point number}
+\noindent
+@<shared skip functions@>=
+float32_t hteg_float32(void)
+{ union {@+float32_t d; @+ uint32_t bits; @+} u;
+ HTEG32(u.bits);
+ return u.d;
+}
+@
+
+
+\subsection{Extended Dimensions}\index{extended dimension}
+\noindent
+@<skip macros@>=
+#define @[HTEG_XDIMEN(I,X)@] \
+ if((I)&b001) HTEG32((X).v); \
+ if((I)&b010) HTEG32((X).h);\
+ if((I)&b100) HTEG32((X).w);
+@
+
+@<skip functions@>=
+static void hteg_xdimen_node(xdimen_t *x)
+{ @<skip the end byte |z|@>@;
+ switch(z)
+ {
+#if 0
+/* currently the info value 0 is not supported */
+case TAG(xdimen_kind,b000): /* see section~\secref{reference} */
+ {uint8_t n;@+ n=HTEG8;} @+ break;
+#endif
+ case TAG(xdimen_kind,b001): HTEG_XDIMEN(b001,*x);@+break;
+ case TAG(xdimen_kind,b010): HTEG_XDIMEN(b010,*x);@+break;
+ case TAG(xdimen_kind,b011): HTEG_XDIMEN(b011,*x);@+break;
+ case TAG(xdimen_kind,b100): HTEG_XDIMEN(b100,*x);@+break;
+ case TAG(xdimen_kind,b101): HTEG_XDIMEN(b101,*x);@+break;
+ case TAG(xdimen_kind,b110): HTEG_XDIMEN(b110,*x);@+break;
+ case TAG(xdimen_kind,b111): HTEG_XDIMEN(b111,*x);@+break;
+ default:
+ QUIT("Extent expected at 0x%x got %s",node_pos,NAME(z)); @+ break;
+ }
+@<skip and check the start byte |a|@>@;
+}
+@
+
+
+\subsection{Stretch and Shrink}\index{stretchability}\index{shrinkability}
+\noindent
+@<skip macros@>=
+#define @[HTEG_STRETCH(S)@] { stch_t st; @+ HTEG32(st.u);@+ S.o=st.u&3;@+ st.u&=~3;@+ S.f=st.f; @+}
+@
+
+\subsection{Glyphs}\index{glyph}
+\noindent
+@<skip macros@>=
+#define HTEG_GLYPH(I,G) \
+ (G).f=HTEG8; \
+ if (I==1) (G).c=HTEG8;\
+ else if (I==2) HTEG16((G).c);\
+ else if (I==3) HTEG24((G).c);\
+ else if (I==4) HTEG32((G).c);
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(glyph_kind,1): @+{@+glyph_t g;@+ HTEG_GLYPH(1,g);@+}@+break;
+case TAG(glyph_kind,2): @+{@+glyph_t g;@+ HTEG_GLYPH(2,g);@+}@+break;
+case TAG(glyph_kind,3): @+{@+glyph_t g;@+ HTEG_GLYPH(3,g);@+}@+break;
+case TAG(glyph_kind,4): @+{@+glyph_t g;@+ HTEG_GLYPH(4,g);@+}@+break;
+@
+
+
+\subsection{Penalties}\index{penalty}
+\noindent
+@<skip macros@>=
+#define @[HTEG_PENALTY(I,P)@] \
+if (I==1) {int8_t n; @+n=HTEG8; @+P=n;@+ } \
+else {int16_t n;@+ HTEG16(n); @+ P=n; @+}\
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(penalty_kind,1): @+{int32_t p;@+ HTEG_PENALTY(1,p);@+} @+break;
+case TAG(penalty_kind,2): @+{int32_t p;@+ HTEG_PENALTY(2,p);@+} @+break;
+@
+
+
+\subsection{Kerns}\index{kern}
+\noindent
+@<skip macros@>=
+#define @[HTEG_KERN(I,X)@] @[if (((I)&b011)==2) HTEG32(X.w); else if (((I)&b011)==3) hteg_xdimen_node(&(X))@]
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(kern_kind,b010): @+ {@+xdimen_t x; @+HTEG_KERN(b010,x);@+ } @+break;
+case TAG(kern_kind,b011): @+ {@+xdimen_t x; @+HTEG_KERN(b011,x);@+ } @+break;
+case TAG(kern_kind,b110): @+ {@+xdimen_t x; @+HTEG_KERN(b110,x);@+ } @+break;
+case TAG(kern_kind,b111): @+ {@+xdimen_t x; @+HTEG_KERN(b111,x);@+ } @+break;
+@
+
+\subsection{Language}\index{language}
+\noindent
+@<cases to skip content@>=
+ at t\kern1em@>case TAG(language_kind,1):
+case TAG(language_kind,2):
+case TAG(language_kind,3):
+case TAG(language_kind,4):
+case TAG(language_kind,5):
+case TAG(language_kind,6):
+case TAG(language_kind,7):@+break;
+@
+
+\subsection{Rules}\index{rule}
+\noindent
+@<skip macros@>=
+#define @[HTEG_RULE(I,R)@]@/\
+if ((I)&b001) HTEG32((R).w); @+else (R).w=RUNNING_DIMEN;\
+if ((I)&b010) HTEG32((R).d); @+else (R).d=RUNNING_DIMEN;\
+if ((I)&b100) HTEG32((R).h); @+else (R).h=RUNNING_DIMEN;
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(rule_kind,b011): @+ {rule_t r;@+ HTEG_RULE(b011,r);@+ }@+ break;
+case TAG(rule_kind,b101): @+ {rule_t r;@+ HTEG_RULE(b101,r);@+ }@+ break;
+case TAG(rule_kind,b001): @+ {rule_t r;@+ HTEG_RULE(b001,r);@+ }@+ break;
+case TAG(rule_kind,b110): @+ {rule_t r;@+ HTEG_RULE(b110,r);@+ }@+ break;
+case TAG(rule_kind,b111): @+ {rule_t r;@+ HTEG_RULE(b111,r);@+ }@+ break;
+@
+
+@<skip functions@>=
+static void hteg_rule_node(void)
+{ @<skip the end byte |z|@>@;
+ if (KIND(z)==rule_kind) { @+rule_t r; @+HTEG_RULE(INFO(z),r); @+}
+ else
+ QUIT("Rule expected at 0x%x got %s",node_pos,NAME(z));
+ @<skip and check the start byte |a|@>@;
+}
+@
+\subsection{Glue}\index{glue}
+\noindent
+@<skip macros@>=
+#define @[HTEG_GLUE(I,G)@] @/\
+ if(I==b111) hteg_xdimen_node(&((G).w)); else (G).w.h=(G).w.v=0.0;\
+ if((I)&b001) HTEG_STRETCH((G).m) @+else (G).m.f=0.0, (G).m.o=0; \
+ if((I)&b010) HTEG_STRETCH((G).p) @+else (G).p.f=0.0, (G).p.o=0;\
+ if((I)!=b111) { @+if ((I)&b100) HTEG32((G).w.w);@+ else (G).w.w=0;@+ }
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(glue_kind,b001): @+{ glue_t g;@+ HTEG_GLUE(b001,g);@+}@+break;
+case TAG(glue_kind,b010): @+{ glue_t g;@+ HTEG_GLUE(b010,g);@+}@+break;
+case TAG(glue_kind,b011): @+{ glue_t g;@+ HTEG_GLUE(b011,g);@+}@+break;
+case TAG(glue_kind,b100): @+{ glue_t g;@+ HTEG_GLUE(b100,g);@+}@+break;
+case TAG(glue_kind,b101): @+{ glue_t g;@+ HTEG_GLUE(b101,g);@+}@+break;
+case TAG(glue_kind,b110): @+{ glue_t g;@+ HTEG_GLUE(b110,g);@+}@+break;
+case TAG(glue_kind,b111): @+{ glue_t g;@+ HTEG_GLUE(b111,g);@+}@+break;
+@
+
+@<skip functions@>=
+static void hteg_glue_node(void)
+{ @<skip the end byte |z|@>@;
+ if (INFO(z)==b000) HTEG_REF(glue_kind);
+ else
+ { @+glue_t g; @+HTEG_GLUE(INFO(z),g);@+}
+ @<skip and check the start byte |a|@>@;
+}
+@
+
+\subsection{Boxes}\index{box}
+\noindent
+@<skip macros@>=
+#define @[HTEG_BOX(I,B)@] \
+hteg_list(&(B.l));\
+if ((I)&b100) @/{ B.s=HTEG8; @+ B.r=hteg_float32();@+ B.o=B.s&0xF; @+B.s=B.s>>4;@+ }\
+else { B.r=0.0;@+ B.o=B.s=0;@+ }\
+if ((I)&b010) HTEG32(B.a); @+else B.a=0;\
+HTEG32(B.w);\
+if ((I)&b001) HTEG32(B.d); @+ else B.d=0;\
+HTEG32(B.h);\
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@> case TAG(hbox_kind,b000): @+{box_t b; @+HTEG_BOX(b000,b);@+} @+ break;
+case TAG(hbox_kind,b001): @+{box_t b; @+HTEG_BOX(b001,b);@+} @+ break;
+case TAG(hbox_kind,b010): @+{box_t b; @+HTEG_BOX(b010,b);@+} @+ break;
+case TAG(hbox_kind,b011): @+{box_t b; @+HTEG_BOX(b011,b);@+} @+ break;
+case TAG(hbox_kind,b100): @+{box_t b; @+HTEG_BOX(b100,b);@+} @+ break;
+case TAG(hbox_kind,b101): @+{box_t b; @+HTEG_BOX(b101,b);@+} @+ break;
+case TAG(hbox_kind,b110): @+{box_t b; @+HTEG_BOX(b110,b);@+} @+ break;
+case TAG(hbox_kind,b111): @+{box_t b; @+HTEG_BOX(b111,b);@+} @+ break;
+case TAG(vbox_kind,b000): @+{box_t b; @+HTEG_BOX(b000,b);@+} @+ break;
+case TAG(vbox_kind,b001): @+{box_t b; @+HTEG_BOX(b001,b);@+} @+ break;
+case TAG(vbox_kind,b010): @+{box_t b; @+HTEG_BOX(b010,b);@+} @+ break;
+case TAG(vbox_kind,b011): @+{box_t b; @+HTEG_BOX(b011,b);@+} @+ break;
+case TAG(vbox_kind,b100): @+{box_t b; @+HTEG_BOX(b100,b);@+} @+ break;
+case TAG(vbox_kind,b101): @+{box_t b; @+HTEG_BOX(b101,b);@+} @+ break;
+case TAG(vbox_kind,b110): @+{box_t b; @+HTEG_BOX(b110,b);@+} @+ break;
+case TAG(vbox_kind,b111): @+{box_t b; @+HTEG_BOX(b111,b);@+} @+ break;
+@
+
+@<skip functions@>=
+static void hteg_hbox_node(void)
+{ box_t b;
+ @<skip the end byte |z|@>@;
+ if (KIND(z)!=hbox_kind) QUIT("Hbox expected at 0x%x got %s",node_pos,NAME(z));
+ HTEG_BOX(INFO(z),b);@/
+ @<skip and check the start byte |a|@>@;
+}
+
+static void hteg_vbox_node(void)
+{ box_t b;
+ @<skip the end byte |z|@>@;
+ if (KIND(z)!=vbox_kind) QUIT("Vbox expected at 0x%x got %s",node_pos,NAME(z));
+ HTEG_BOX(INFO(z),b);@/
+ @<skip and check the start byte |a|@>@;
+}
+@
+
+
+\subsection{Extended Boxes}\index{extended box}
+\noindent
+@<skip macros@>=
+#define @[HTEG_SET(I)@] @/\
+{ list_t l; @+hteg_list(&l); @+} \
+ if ((I)&b100) {xdimen_t x;@+ hteg_xdimen_node(&x); @+} \
+ else HTEG_REF(xdimen_kind);\
+{ stretch_t m; @+HTEG_STRETCH(m);@+}\
+{ stretch_t p; @+HTEG_STRETCH(p);@+}\
+if ((I)&b010) { dimen_t a; @+HTEG32(a);@+} \
+ { dimen_t w; @+HTEG32(w);@+} \
+ { dimen_t d; @+if ((I)&b001) HTEG32(d); @+ else d=0;@+}\
+ { dimen_t h; @+HTEG32(h);@+}
+@#
+
+#define @[HTEG_PACK(K,I)@] @/\
+ { list_t l; @+hteg_list(&l); @+} \
+ if ((I)&b100) {xdimen_t x; hteg_xdimen_node(&x);@+} @+ else HTEG_REF(xdimen_kind);\
+ if (K==vpack_kind) { dimen_t d; @+HTEG32(d); @+ }\
+ if ((I)&b010) { dimen_t d; @+HTEG32(d); @+ }
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(hset_kind,b000): HTEG_SET(b000); @+ break;
+case TAG(hset_kind,b001): HTEG_SET(b001); @+ break;
+case TAG(hset_kind,b010): HTEG_SET(b010); @+ break;
+case TAG(hset_kind,b011): HTEG_SET(b011); @+ break;
+case TAG(hset_kind,b100): HTEG_SET(b100); @+ break;
+case TAG(hset_kind,b101): HTEG_SET(b101); @+ break;
+case TAG(hset_kind,b110): HTEG_SET(b110); @+ break;
+case TAG(hset_kind,b111): HTEG_SET(b111); @+ break;@#
+
+case TAG(vset_kind,b000): HTEG_SET(b000); @+ break;
+case TAG(vset_kind,b001): HTEG_SET(b001); @+ break;
+case TAG(vset_kind,b010): HTEG_SET(b010); @+ break;
+case TAG(vset_kind,b011): HTEG_SET(b011); @+ break;
+case TAG(vset_kind,b100): HTEG_SET(b100); @+ break;
+case TAG(vset_kind,b101): HTEG_SET(b101); @+ break;
+case TAG(vset_kind,b110): HTEG_SET(b110); @+ break;
+case TAG(vset_kind,b111): HTEG_SET(b111); @+ break;@#
+
+case TAG(hpack_kind,b000): HTEG_PACK(hpack_kind,b000); @+ break;
+case TAG(hpack_kind,b001): HTEG_PACK(hpack_kind,b001); @+ break;
+case TAG(hpack_kind,b010): HTEG_PACK(hpack_kind,b010); @+ break;
+case TAG(hpack_kind,b011): HTEG_PACK(hpack_kind,b011); @+ break;
+case TAG(hpack_kind,b100): HTEG_PACK(hpack_kind,b100); @+ break;
+case TAG(hpack_kind,b101): HTEG_PACK(hpack_kind,b101); @+ break;
+case TAG(hpack_kind,b110): HTEG_PACK(hpack_kind,b110); @+ break;
+case TAG(hpack_kind,b111): HTEG_PACK(hpack_kind,b111); @+ break;@#
+
+case TAG(vpack_kind,b000): HTEG_PACK(vpack_kind,b000); @+ break;
+case TAG(vpack_kind,b001): HTEG_PACK(vpack_kind,b001); @+ break;
+case TAG(vpack_kind,b010): HTEG_PACK(vpack_kind,b010); @+ break;
+case TAG(vpack_kind,b011): HTEG_PACK(vpack_kind,b011); @+ break;
+case TAG(vpack_kind,b100): HTEG_PACK(vpack_kind,b100); @+ break;
+case TAG(vpack_kind,b101): HTEG_PACK(vpack_kind,b101); @+ break;
+case TAG(vpack_kind,b110): HTEG_PACK(vpack_kind,b110); @+ break;
+case TAG(vpack_kind,b111): HTEG_PACK(vpack_kind,b111); @+ break;
+@
+
+
+\subsection{Leaders}\index{leaders}
+\noindent
+@<skip macros@>=
+#define @[HTEG_LEADERS(I)@]@/ \
+if (KIND(hpos[-1])==rule_kind) hteg_rule_node(); \
+else if (KIND(hpos[-1])==hbox_kind) hteg_hbox_node();\
+else hteg_vbox_node();\
+if ((I)&b100) hteg_glue_node();
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(leaders_kind,1): @+ HTEG_LEADERS(1); @+break;
+case TAG(leaders_kind,2): @+ HTEG_LEADERS(2); @+break;
+case TAG(leaders_kind,3): @+ HTEG_LEADERS(3); @+break;
+case TAG(leaders_kind,b100|1): @+ HTEG_LEADERS(b100|1); @+break;
+case TAG(leaders_kind,b100|2): @+ HTEG_LEADERS(b100|2); @+break;
+case TAG(leaders_kind,b100|3): @+ HTEG_LEADERS(b100|3); @+break;
+@
+
+\subsection{Baseline Skips}\index{baseline skip}
+\noindent
+@<skip macros@>=
+#define @[HTEG_BASELINE(I,B)@] \
+ if((I)&b010) hteg_glue_node(); \
+ else {B.ls.p.o=B.ls.m.o=B.ls.w.w=0; @+B.ls.w.h=B.ls.w.v=B.ls.p.f=B.ls.m.f=0.0;@+}\
+ if((I)&b100) hteg_glue_node(); \
+ else {B.bs.p.o=B.bs.m.o=B.bs.w.w=0; @+B.bs.w.h=B.bs.w.v=B.bs.p.f=B.bs.m.f=0.0;@+}\
+ if((I)&b001) HTEG32((B).lsl); @+else B.lsl=0;
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(baseline_kind,b001): @+{ baseline_t b;@+ HTEG_BASELINE(b001,b);@+ }@+break;
+case TAG(baseline_kind,b010): @+{ baseline_t b;@+ HTEG_BASELINE(b010,b);@+ }@+break;
+case TAG(baseline_kind,b011): @+{ baseline_t b;@+ HTEG_BASELINE(b011,b);@+ }@+break;
+case TAG(baseline_kind,b100): @+{ baseline_t b;@+ HTEG_BASELINE(b100,b);@+ }@+break;
+case TAG(baseline_kind,b101): @+{ baseline_t b;@+ HTEG_BASELINE(b101,b);@+ }@+break;
+case TAG(baseline_kind,b110): @+{ baseline_t b;@+ HTEG_BASELINE(b110,b);@+ }@+break;
+case TAG(baseline_kind,b111): @+{ baseline_t b;@+ HTEG_BASELINE(b111,b);@+ }@+break;
+@
+\subsection{Ligatures}\index{ligature}
+\noindent
+@<skip macros@>=
+#define @[HTEG_LIG(I,L)@] @/\
+if ((I)==7) hteg_list(&((L).l)); \
+else {(L).l.s=(I); @+hpos-=(L).l.s; @+ (L).l.p=hpos-hstart;@+} \
+(L).f=HTEG8;
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(ligature_kind,1):@+ {lig_t l; @+HTEG_LIG(1,l);@+} @+break;
+case TAG(ligature_kind,2):@+ {lig_t l; @+HTEG_LIG(2,l);@+} @+break;
+case TAG(ligature_kind,3):@+ {lig_t l; @+HTEG_LIG(3,l);@+} @+break;
+case TAG(ligature_kind,4):@+ {lig_t l; @+HTEG_LIG(4,l);@+} @+break;
+case TAG(ligature_kind,5):@+ {lig_t l; @+HTEG_LIG(5,l);@+} @+break;
+case TAG(ligature_kind,6):@+ {lig_t l; @+HTEG_LIG(6,l);@+} @+break;
+case TAG(ligature_kind,7):@+ {lig_t l; @+HTEG_LIG(7,l);@+} @+break;
+@
+
+
+\subsection{Discretionary breaks}\index{discretionary breaks}
+\noindent
+@<skip macros@>=
+#define @[HTEG_DISC(I,H)@]\
+if ((I)&b001) hteg_list(&((H).q)); else { (H).q.p=hpos-hstart; @+(H).q.s=0; @+(H).q.k=list_kind; @+}\
+if ((I)&b010) hteg_list(&((H).p)); else { (H).p.p=hpos-hstart; @+(H).p.s=0; @+(H).p.k=list_kind; @+} \
+if ((I)&b100) (H).r=HTEG8; @+else (H).r=0;
+@
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(disc_kind,b001): @+{disc_t h; @+HTEG_DISC(b001,h); @+} @+break;
+case TAG(disc_kind,b010): @+{disc_t h; @+HTEG_DISC(b010,h); @+} @+break;
+case TAG(disc_kind,b011): @+{disc_t h; @+HTEG_DISC(b011,h); @+} @+break;
+case TAG(disc_kind,b100): @+{disc_t h; @+HTEG_DISC(b100,h); @+} @+break;
+case TAG(disc_kind,b101): @+{disc_t h; @+HTEG_DISC(b101,h); @+} @+break;
+case TAG(disc_kind,b110): @+{disc_t h; @+HTEG_DISC(b110,h); @+} @+break;
+case TAG(disc_kind,b111): @+{disc_t h; @+HTEG_DISC(b111,h); @+} @+break;
+@
+
+
+\subsection{Paragraphs}\index{paragraph}
+\noindent
+@<skip macros@>=
+#define @[HTEG_PAR(I)@] @/\
+ { list_t l; @+hteg_list(&l); @+} \
+ if ((I)&b010) { list_t l; @+hteg_param_list(&l); @+} else if ((I)!=b100) HTEG_REF(param_kind);\
+ if ((I)&b100) {xdimen_t x; @+ hteg_xdimen_node(&x); @+} else HTEG_REF(xdimen_kind);\
+ if ((I)==b100) HTEG_REF(param_kind);
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(par_kind,b000): @+HTEG_PAR(b000);@+break;
+case TAG(par_kind,b010): @+HTEG_PAR(b010);@+break;
+case TAG(par_kind,b100): @+HTEG_PAR(b100);@+break;
+case TAG(par_kind,b110): @+HTEG_PAR(b110);@+break;
+@
+
+
+\subsection{Mathematics}\index{mathematics}\index{displayed formula}%
+\noindent
+@<skip macros@>=
+#define @[HTEG_MATH(I)@] \
+if ((I)&b001) hteg_hbox_node();\
+{ list_t l; @+hteg_list(&l); @+} \
+if ((I)&b010) hteg_hbox_node(); \
+if ((I)&b100) { list_t l; @+hteg_param_list(&l); @+} @+ else HTEG_REF(param_kind);
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(math_kind,b000): HTEG_MATH(b000); @+ break;
+case TAG(math_kind,b001): HTEG_MATH(b001); @+ break;
+case TAG(math_kind,b010): HTEG_MATH(b010); @+ break;
+case TAG(math_kind,b100): HTEG_MATH(b100); @+ break;
+case TAG(math_kind,b101): HTEG_MATH(b101); @+ break;
+case TAG(math_kind,b110): HTEG_MATH(b110); @+ break;
+case TAG(math_kind,b011):
+case TAG(math_kind,b111): @+ break;
+@
+
+\subsection{Images}\index{image}
+\noindent
+@<skip macros@>=
+#define @[HTEG_IMAGE(I,X)@] @/\
+if (I&b001) {HTEG_STRETCH((X).m);HTEG_STRETCH((X).p);@+}\
+else { (X).p.f=(X).m.f=0.0; (X).p.o=(X).m.o=normal_o;@+}\
+if (I&b010) {HTEG32((X).h);HTEG32((X).w);@+} \
+else (X).w=(X).h=0;\
+HTEG16((X).n);
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(image_kind,b100): @+ { image_t x;@+HTEG_IMAGE(b100,x);@+}@+break;
+case TAG(image_kind,b101): @+ { image_t x;@+HTEG_IMAGE(b101,x);@+}@+break;
+case TAG(image_kind,b110): @+ { image_t x;@+HTEG_IMAGE(b110,x);@+}@+break;
+case TAG(image_kind,b111): @+ { image_t x;@+HTEG_IMAGE(b111,x);@+}@+break;
+@
+
+\subsection{Links and Labels}
+\noindent
+@<skip macros@>=
+#define @[HTEG_LINK(I)@] @/\
+{ uint16_t n; if (I&b001) HTEG16(n);@+ else n=HTEG8; @+}
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(link_kind,b000): @+ HTEG_LINK(b000); @+break;
+case TAG(link_kind,b001): @+ HTEG_LINK(b001); @+break;
+case TAG(link_kind,b010): @+ HTEG_LINK(b010); @+break;
+case TAG(link_kind,b011): @+ HTEG_LINK(b011); @+break;
+@
+
+
+\subsection{Plain Lists, Texts, and Parameter Lists}\index{list}
+
+\noindent
+@<shared skip functions@>=
+void hteg_size_boundary(info_t info)
+{ uint32_t n;
+ if (info<2) return;
+ n=HTEG8;
+ if (n-1!=0x100-info) QUIT(@["List size boundary byte 0x%x does not match info value %d at " SIZE_F@],
+ n, info,hpos-hstart);
+}
+
+uint32_t hteg_list_size(info_t info)
+{ uint32_t n;
+ if (info==1) return 0;
+ else if (info==2) n=HTEG8;
+ else if (info==3) HTEG16(n);
+ else if (info==4) HTEG24(n);
+ else if (info==5) HTEG32(n);
+ else QUIT("List info %d must be 1, 2, 3, 4, or 5",info);
+ return n;
+}
+
+void hteg_list(list_t *l)
+{ @<skip the end byte |z|@>@,
+ @+if (KIND(z)!=list_kind && KIND(z)!=text_kind &&@| KIND(z)!=param_kind) @/
+ QUIT("List expected at 0x%x", (uint32_t)(hpos-hstart));
+ else
+ { uint32_t s;
+ l->k=KIND(z);
+ l->s=hteg_list_size(INFO(z));
+ hteg_size_boundary(INFO(z));
+ hpos=hpos-l->s;
+ l->p=hpos-hstart;
+ hteg_size_boundary(INFO(z));
+ s=hteg_list_size(INFO(z));
+ if (s!=l->s) QUIT(@["List sizes at " SIZE_F " and 0x%x do not match 0x%x != 0x%x"@],
+ hpos-hstart,node_pos-1,s,l->s);
+ @<skip and check the start byte |a|@>@;
+ }
+}
+
+void hteg_param_list(list_t *l)
+{ @+if (KIND(*(hpos-1))!=param_kind) return;
+ hteg_list(l);
+}
+
+
+@
+
+\subsection{Adjustments}\index{adjustment}
+\noindent
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(adjust_kind,b001): @+ { list_t l; @+hteg_list(&l);@+ } @+ break;
+@
+
+\subsection{Tables}\index{table}
+\noindent
+@<skip macros@>=
+#define @[HTEG_TABLE(I)@] \
+{@+ list_t l; @+ hteg_list(&l);@+}\
+{@+ list_t l; @+ hteg_list(&l);@+}\
+if ((I)&b100) {xdimen_t x;@+ hteg_xdimen_node(&x);@+} else HTEG_REF(xdimen_kind)@;
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(table_kind,b000): @+ HTEG_TABLE(b000); @+ break;
+case TAG(table_kind,b001): @+ HTEG_TABLE(b001); @+ break;
+case TAG(table_kind,b010): @+ HTEG_TABLE(b010); @+ break;
+case TAG(table_kind,b011): @+ HTEG_TABLE(b011); @+ break;
+case TAG(table_kind,b100): @+ HTEG_TABLE(b100); @+ break;
+case TAG(table_kind,b101): @+ HTEG_TABLE(b101); @+ break;
+case TAG(table_kind,b110): @+ HTEG_TABLE(b110); @+ break;
+case TAG(table_kind,b111): @+ HTEG_TABLE(b111); @+ break;@#
+
+case TAG(item_kind,b000): @+{@+ list_t l; @+hteg_list(&l);@+ } @+ break;
+case TAG(item_kind,b001): hteg_content_node(); @+ break;
+case TAG(item_kind,b010): hteg_content_node(); @+ break;
+case TAG(item_kind,b011): hteg_content_node(); @+ break;
+case TAG(item_kind,b100): hteg_content_node(); @+ break;
+case TAG(item_kind,b101): hteg_content_node(); @+ break;
+case TAG(item_kind,b110): hteg_content_node(); @+ break;
+case TAG(item_kind,b111): hteg_content_node(); @+{uint8_t n;@+ n=HTEG8;@+}@+ break;
+@
+
+
+
+\subsection{Stream Nodes}\index{stream}
+@<skip macros@>=
+#define @[HTEG_STREAM(I)@] @/\
+{ list_t l; @+hteg_list(&l); @+}\
+if ((I)&b010) { list_t l; @+hteg_param_list(&l); @+} @+ else HTEG_REF(param_kind);\
+HTEG_REF(stream_kind);
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(stream_kind,b000): HTEG_STREAM(b000); @+ break;
+case TAG(stream_kind,b010): HTEG_STREAM(b010); @+ break;
+@
+
+
+
+\subsection{References}\index{reference}
+\noindent
+@<skip macros@>=
+#define @[HTEG_REF(K)@] do at +{uint8_t n; @+ n=HTEG8;@+} @+ while (false)
+@
+
+@<cases to skip content@>=
+ at t\1\kern1em@>case TAG(penalty_kind,0): HTEG_REF(penalty_kind); @+break;
+case TAG(kern_kind,b000): HTEG_REF(dimen_kind); @+break;
+case TAG(kern_kind,b100): HTEG_REF(dimen_kind); @+break;
+case TAG(kern_kind,b001): HTEG_REF(xdimen_kind); @+break;
+case TAG(kern_kind,b101): HTEG_REF(xdimen_kind); @+break;
+case TAG(ligature_kind,0): HTEG_REF(ligature_kind); @+break;
+case TAG(disc_kind,0): HTEG_REF(disc_kind); @+break;
+case TAG(glue_kind,0): HTEG_REF(glue_kind); @+break;
+case TAG(language_kind,0): HTEG_REF(language_kind); @+break;
+case TAG(rule_kind,0): HTEG_REF(rule_kind); @+break;
+case TAG(image_kind,0): HTEG_REF(image_kind); @+break;
+case TAG(leaders_kind,0): HTEG_REF(leaders_kind); @+break;
+case TAG(baseline_kind,0): HTEG_REF(baseline_kind); @+break;
+@
+
+
+\section{Code and Header Files}\index{code file}\index{header file}
+
+\subsection{{\tt basetypes.h}}
+To define basic types in a portable way, we create an include file.
+The macro |_MSC_VER| (Microsoft Visual C Version)\index{Microsoft Visual C}
+is defined only if using the respective compiler.
+\index{false+\\{false}}\index{true+\\{true}}\index{bool+\&{bool}}
+@(basetypes.h@>=
+#ifndef __BASETYPES_H__
+#define __BASETYPES_H__
+#include <stdlib.h>
+#include <stdio.h>
+#ifndef _STDLIB_H
+#define _STDLIB_H
+#endif
+#ifdef _MSC_VER
+#include <windows.h>
+#define uint8_t UINT8
+#define uint16_t UINT16
+#define uint32_t UINT32
+#define uint64_t UINT64
+#define int8_t INT8
+#define int16_t INT16
+#define int32_t INT32
+#define bool BOOL
+#define true (0==0)
+#define false (!true)
+#define __SIZEOF_FLOAT__ 4
+#define __SIZEOF_DOUBLE__ 8
+#define INT32_MAX (2147483647)
+#define PRIx64 "I64x"
+#pragma @[warning( disable : @[4244@]@t @> @[4996@]@t @> @[4127@])@]
+#else
+#include <stdint.h>
+#include <stdbool.h>
+#include <inttypes.h>
+#include <unistd.h>
+#ifdef WIN32
+#include <io.h>
+#endif
+#endif
+typedef float float32_t;
+typedef double float64_t;
+#if __SIZEOF_FLOAT__!=4
+#error @=float32 type must have size 4@>
+#endif
+#if __SIZEOF_DOUBLE__!=8
+#error @=float64 type must have size 8@>
+#endif
+#endif
+@
+
+
+
+\subsection{{\tt hformat.h}}\index{hformat.h+{\tt hformat.h}}
+The \.{hformat.h} file contains definitions of types, macros, variables and functions
+that are needed in other compilation units.
+
+@(hformat.h@>=
+#ifndef _HFORMAT_H_
+#define _HFORMAT_H_
+@<debug macros@>@;
+@<debug constants@>@;
+@<hint macros@>@;
+@<hint basic types@>@;
+@<default names@>@;
+
+extern const char *content_name[32];
+extern const char *definition_name[32];
+extern unsigned int debugflags;
+extern FILE *hlog;
+extern int max_fixed[32], max_default[32], max_ref[32], max_outline;
+extern int32_t int_defaults[MAX_INT_DEFAULT+1];
+extern dimen_t dimen_defaults[MAX_DIMEN_DEFAULT+1];
+extern xdimen_t xdimen_defaults[MAX_XDIMEN_DEFAULT+1];
+extern glue_t glue_defaults[MAX_GLUE_DEFAULT+1];
+extern baseline_t baseline_defaults[MAX_BASELINE_DEFAULT+1];
+extern label_t label_defaults[MAX_LABEL_DEFAULT+1];
+extern signed char hnode_size[0x100];
+
+#endif
+@
+\subsection{{\tt hformat.c}}\index{hformat.c+{\tt hformat.c}}\index{mkhformat.c+{\tt mkhformat.c}}
+For maximum flexibility and efficiency, the file {\tt hformat.c}
+is generated by a \CEE\ program.
+Here is the |main| program of {\tt mkhformat}:
+
+@(mkhformat.c@>=
+#include <stdio.h>
+#include "basetypes.h"
+#include "hformat.h"
+@<skip macros@>@;
+
+int max_fixed[32], max_default[32];
+
+int32_t int_defaults[MAX_INT_DEFAULT+1]={0};
+dimen_t dimen_defaults[MAX_DIMEN_DEFAULT+1]={0};
+xdimen_t xdimen_defaults[MAX_XDIMEN_DEFAULT+1]={{0}};
+glue_t glue_defaults[MAX_GLUE_DEFAULT+1]={{{0}}};
+baseline_t baseline_defaults[MAX_BASELINE_DEFAULT+1]={{{{0}}}};
+
+signed char hnode_size[0x100]={0};
+@<define |content_name| and |definition_name|@>@;
+int main(void)
+{ kind_t k;
+ int i;
+
+
+ printf("#include \"basetypes.h\"\n"@/
+ "#include \"hformat.h\"\n\n");@/
+
+ @<print |content_name| and |definition_name|@>@;
+
+ printf("int max_outline=-1;\n\n");
+
+ @<take care of variables without defaults@>@;
+ @<define |int_defaults|@>@;
+ @<define |dimen_defaults|@>@;
+ @<define |glue_defaults|@>@;
+ @<define |xdimen_defaults|@>@;
+ @<define |baseline_defaults|@>@;
+ @<define page defaults@>@;
+ @<define stream defaults@>@;
+ @<define range defaults@>@;
+ @<define |label_defaults|@>@;
+ @<print defaults@>@;
+
+ @<initialize the |hnode_size| array@>@;
+ @<print the |hnode_size| variable@>@;
+ return 0;
+}
+@
+
+The following code prints the arrays containing the default values.
+
+@<print defaults@>=
+ printf("int max_fixed[32]= {");
+ for (k=0; k<32; k++)@/
+ { printf("%d",max_fixed[k]);@+
+ if (k<31) printf(", ");@+
+ }
+ printf("};\n\n");@#
+ printf("int max_default[32]= {");
+ for (k=0; k<32; k++)@/
+ { printf("%d",max_default[k]);@+
+ if (k<31) printf(", ");@+
+ }
+ printf("};\n\n");
+ printf("int max_ref[32]= {");
+ for (k=0; k<32; k++)@/
+ { printf("%d",max_default[k]);@+
+ if (k<31) printf(", ");@+
+ }
+ printf("};\n\n");
+@
+
+
+\subsection{{\tt hget.h}}\index{hget.h+{\tt hget.h}}
+The \.{hget.h} file contains function prototypes for all the functions
+that read the short format.
+
+@(hget.h@>=
+@<hint types@>@;
+@<directory entry type@>@;
+@<shared get macros@>@;
+
+extern entry_t *dir;
+extern uint16_t section_no, max_section_no;
+extern uint8_t *hpos, *hstart, *hend, *hpos0;
+extern uint64_t hin_size, hin_time;
+extern uint8_t *hin_addr;
+
+extern label_t *labels;
+extern char *hin_name;
+extern bool hget_map(void);
+extern void hget_unmap(void);
+
+extern void new_directory(uint32_t entries);
+extern void hset_entry(entry_t *e, uint16_t i, @|uint32_t size, uint32_t xsize, char *file_name);
+
+extern void hget_banner(void);
+extern void hget_section(uint16_t n);
+extern void hget_entry(entry_t *e);
+extern void hget_directory(void);
+extern void hclear_dir(void);
+extern bool hcheck_banner(char *magic);
+
+extern void hget_max_definitions(void);
+extern uint32_t hget_utf8(void);
+extern void hget_size_boundary(info_t info);
+extern uint32_t hget_list_size(info_t info);
+extern void hget_list(list_t *l);
+extern uint32_t hget_utf8(void);
+extern float32_t hget_float32(void);
+extern float32_t hteg_float32(void);
+extern void hteg_size_boundary(info_t info);
+extern uint32_t hteg_list_size(info_t info);
+extern void hteg_list(list_t *l);
+extern void hff_hpos(void);
+extern uint32_t hff_list_pos, hff_list_size;
+extern uint8_t hff_tag;
+@
+
+
+
+\subsection{{\tt hget.c}}\index{hget.c+{\tt hget.c}}
+@(hget.c@>=
+#include "basetypes.h"
+#include <string.h>
+#include <math.h>
+#include <zlib.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+
+#include "error.h"
+#include "hformat.h"
+#include "hget.h"
+
+@<common variables@>@;
+
+@<map functions@>@;
+@<function to check the banner@>@;
+@<directory functions@>@;
+
+@<get file functions@>@;
+@<shared get functions@>@;
+@<shared skip functions@>@;
+@
+
+\subsection{{\tt hput.h}}\index{hput.h+{\tt hput.h}}
+The \.{hput.h} file contains function prototypes for all the functions
+that write the short format.
+
+
+@(hput.h@>=
+@<put macros@>@;
+@<hint macros@>@;
+@<hint types@>@;
+@<directory entry type@>@;
+extern entry_t *dir;
+extern uint16_t section_no, max_section_no;
+extern uint8_t *hpos, *hstart, *hend, *hpos0;
+extern int next_range;
+extern range_pos_t *range_pos;
+extern int *page_on;
+extern label_t *labels;
+extern int first_label;
+extern int max_outline;
+extern outline_t *outlines;
+
+
+extern FILE *hout;
+extern void new_directory(uint32_t entries);
+extern void new_output_buffers(void);
+
+/* declarations for the parser */
+extern void hput_definitions_start(void);
+extern void hput_definitions_end(void);
+extern void hput_content_start(void);
+extern void hput_content_end(void);
+
+extern void hset_label(int n,int w);
+extern uint8_t hput_link(int n, int on);
+extern void hset_outline(int m, int r, int d, uint32_t p);
+extern void hput_label_defs(void);
+
+extern void hput_tags(uint32_t pos, uint8_t tag);
+extern uint8_t hput_glyph(glyph_t *g);
+extern uint8_t hput_xdimen(xdimen_t *x);
+extern uint8_t hput_int(int32_t p);
+extern uint8_t hput_language(uint8_t n);
+extern uint8_t hput_rule(rule_t *r);
+extern uint8_t hput_glue(glue_t *g);
+extern uint8_t hput_list(uint32_t size_pos, list_t *y);
+extern uint8_t hsize_bytes(uint32_t n);
+extern void hput_txt_cc(uint32_t c);
+extern void hput_txt_font(uint8_t f);
+extern void hput_txt_global(ref_t *d);
+extern void hput_txt_local(uint8_t n);
+extern info_t hput_box_dimen(dimen_t h, dimen_t d, dimen_t w);
+extern info_t hput_box_shift(dimen_t a);
+extern info_t hput_box_glue_set(int8_t s, float32_t r, order_t o);
+extern void hput_stretch(stretch_t *s);
+extern uint8_t hput_kern(kern_t *k);
+extern void hput_utf8(uint32_t c);
+extern uint8_t hput_ligature(lig_t *l);
+extern uint8_t hput_disc(disc_t *h);
+extern info_t hput_span_count(uint32_t n);
+extern uint8_t hput_image(image_t *x);
+extern void hput_string(char *str);
+extern void hput_range(uint8_t pg, bool on);
+extern void hput_max_definitions(void);
+extern uint8_t hput_dimen(dimen_t d);
+extern uint8_t hput_font_head(uint8_t f, char *n, dimen_t s,@| uint16_t m, uint16_t y);
+extern void hput_range_defs(void);
+extern void hput_xdimen_node(xdimen_t *x);
+extern void hput_directory(void);
+extern void hput_hint(char * str);
+extern void hput_list_size(uint32_t n, int i);
+extern int hcompress_depth(int n, int c);
+@
+
+
+\subsection{{\tt hput.c}}\label{writeshort}\index{hput.c+{\tt hput.c}}
+\noindent
+@(hput.c@>=
+#include "basetypes.h"
+#include <string.h>
+#include <ctype.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <zlib.h>
+#include "error.h"
+#include "hformat.h"
+#include "hput.h"
+
+@<common variables@>@;
+@<shared put variables@>@;
+@<directory functions@>@;
+@<function to write the banner@>@;
+@<put functions@>@;
+@
+
+\subsection{{\tt shrink.l}}\index{shrink.l+{\tt shrink.l}}\index{scanning}
+The definitions for lex are collected in the file {\tt shrink.l}
+
+@(shrink.lex-in@>=
+%{
+#include "basetypes.h"
+#include "error.h"
+#include "hformat.h"
+#include "hput.h"
+
+@<enable bison debugging@>@;
+#include "shrink-parser.h"
+
+@<scanning macros@>@;@+
+@<scanning functions@>@;
+int yywrap (void )@+{ return 1;@+}
+#ifdef _MSC_VER
+#pragma warning( disable : 4267)
+#endif
+%}
+
+%option yylineno batch stack
+%option debug
+%option nounistd nounput noinput noyy_top_state
+
+@<scanning definitions@>@/
+
+%%
+
+@<scanning rules@>@/
+::@=[a-z]+@> :< QUIT("Unexpected keyword '%s' in line %d",@|yytext,yylineno); >:
+::@=.@> :< QUIT("Unexpected character '%c' (0x%02X) in line %d",@|yytext[0]>' '?yytext[0]:' ',yytext[0],yylineno); >:
+
+%%
+@
+
+
+
+\subsection{{\tt shrink.y}}\index{shrink.y+{\tt shrink.y}}\index{parsing}
+
+The grammar rules for bison are collected in the file {\tt shrink.y}.
+% for the option %token-table use the command line parameter -k
+
+
+@(shrink.yacc-in@>=
+%{
+#include "basetypes.h"
+#include <string.h>
+#include <math.h>
+#include "error.h"
+#include "hformat.h"
+#include "hput.h"
+extern char **hfont_name; /* in common variables */
+
+@<definition checks@>@;
+
+extern void hset_entry(entry_t *e, uint16_t i, @|uint32_t size,
+ uint32_t xsize, char *file_name);
+
+@<enable bison debugging@>@;
+extern int yylex(void);
+
+@<parsing functions@>@;
+
+%}
+
+
+ at t{\label{union}\index{union}\index{parsing}}@>
+
+
+%union {uint32_t u; @+ int32_t i; @+ char *s; @+ float64_t f; @+ glyph_t c;
+ @+ dimen_t @+d; stretch_t st; @+ xdimen_t xd; @+ kern_t kt;
+ @+ rule_t r; @+ glue_t g; @+ @+ image_t x;
+ @+ list_t l; @+ box_t h; @+ disc_t dc; @+ lig_t lg;
+ @+ ref_t rf; @+ info_t info; @+ order_t o; bool at + b;
+ }
+
+ at t{}@>
+
+%error_verbose
+%start hint
+ at t@>
+@<symbols@>@/
+%%
+@<parsing rules@>@;
+%%
+@
+
+\subsection{{\tt hishrink.c}}\index{hishrink.c+{\tt hishrink.c}}
+
+\.{shrink} is a \CEE\ program translating a \HINT\ file in long format into a \HINT\ file in short format.
+
+@(hishrink.c@>=
+#include "basetypes.h"
+#include <string.h>
+#include <ctype.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#ifdef WIN32
+#include <direct.h>
+#endif
+#include <zlib.h>
+
+#include "error.h"
+#include "hformat.h"
+#include "hput.h"
+#include "shrink-parser.h"
+
+extern void yyset_debug(int lex_debug);
+extern int yylineno;
+extern FILE *yyin, *yyout;
+extern int yyparse(void);
+
+@<put macros@>@;
+
+@<common variables@>@;
+@<shared put variables@>@;
+@<function to check the banner@>@;
+@<directory functions@>@;
+@<function to write the banner@>@;
+@<put functions@>@;
+
+int main(int argc, char *argv[])
+{ @<local variables in |main|@>@;
+ in_ext=".HINT";
+ out_ext=".hnt";
+ @<process the command line@>@;
+
+ if (debugflags&DBGFLEX) yyset_debug(1); else yyset_debug(0);
+#if YYDEBUG
+ if (debugflags&DBGBISON) yydebug=1;
+ else yydebug=0;
+#endif
+ @<open the log file@>@;
+ @<open the input file@>@;
+ @<open the output file@>@;
+
+ yyin=hin;
+ yyout=hlog;
+ @<read the banner@>@;
+ if (!hcheck_banner("HINT")) QUIT("Invalid banner");
+ yylineno++;
+ DBG(DBGBISON|DBGFLEX,"Parsing Input\n");
+ yyparse();
+
+ hput_directory();
+
+ hput_hint("shrink");
+
+ @<close the output file@>@;
+ @<close the input file@>@;
+ @<close the log file@>@;
+ return 0;
+explain_usage:
+ @<explain usage@>@;
+ return 1;
+}
+@
+
+
+
+\subsection{{\tt histretch.c}}\label{stretchmain}\index{histretch.c+{\tt histretch.c}}
+\.{histretch} is a \CEE\ program translating a \HINT\ file in short
+format into a \HINT\ file in long format.
+
+@(histretch.c@>=
+#include "basetypes.h"
+#include <math.h>
+#include <string.h>
+#include <ctype.h>
+#include <zlib.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#ifdef WIN32
+#include <direct.h>
+#endif
+#include <fcntl.h>
+#include "error.h"
+#include "hformat.h"
+#include "hget.h"
+
+@<get macros@>@;
+@<write macros@>@;
+@<common variables@>@;
+@<shared put variables@>@;
+@<map functions@>@;
+@<function to check the banner@>@;
+@<function to write the banner@>@;
+@<directory functions@>@;
+
+@<definition checks@>@;
+@<get function declarations@>@;
+@<write functions@>@;
+@<get file functions@>@;
+@<shared get functions@>@;
+@<get functions@>@;
+
+int main(int argc, char *argv[])
+{ @<local variables in |main|@>@;
+
+ in_ext=".hnt";
+ out_ext=".HINT";
+ @<process the command line@>@;
+ @<open the log file@>@;
+ @<open the output file@>@;
+ @<determine the |stem_name| from the output |file_name|@>@;
+ if (!hget_map()) QUIT("Unable to map the input file");
+ hpos=hstart=hin_addr;
+ hend=hstart+hin_size;
+ hget_banner();
+ if (!hcheck_banner("hint")) QUIT("Invalid banner");
+ hput_banner("HINT","stretch");
+ hget_directory();
+ hwrite_directory();
+ hget_definition_section();
+ hwrite_content_section();
+ hwrite_aux_files();
+ hget_unmap();
+ @<close the output file@>@;
+ DBG(DBGBASIC,"End of Program\n");
+ @<close the log file@>@;
+ return 0;
+explain_usage:
+ @<explain usage@>@;
+ return 1;}
+@
+
+In the above program, the get functions call the write functions
+and the write functions call some get functions. This requires
+function declarations to satisfy the define before use requirement
+of \CEE. Some of the necessary function declarations are already
+contained in {\tt hget.h}. The remaining declarations are these:
+
+@<get function declarations@>=
+extern void hget_xdimen_node(xdimen_t *x);
+extern void hget_def_node(void);
+extern void hget_font_def(uint8_t f);
+extern void hget_content_section(void);
+extern uint8_t hget_content_node(void);
+extern void hget_glue_node(void);
+extern void hget_rule_node(void);
+extern void hget_hbox_node(void);
+extern void hget_vbox_node(void);
+extern void hget_param_list(list_t *l);
+extern int hget_txt(void);
+@
+
+
+\subsection{{\tt skip.c}}\label{skip}\index{skip.c+{\tt skip.c}}
+\.{skip} is a \CEE\ program reading the content section of a \HINT\ file in short format
+backwards.
+
+@(skip.c@>=
+#include "basetypes.h"
+#include <string.h>
+#include <zlib.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include "error.h"
+#include "hformat.h"
+@<hint types@>@;
+
+@<common variables@>@;
+
+@<map functions@>@;
+@<function to check the banner@>@;
+@<directory entry type@>@;
+@<directory functions@>@;
+@<shared get macros@>@;
+@<get file functions@>@;
+
+@<skip macros@>@;
+@<skip function declarations@>@;
+@<shared skip functions@>@;
+@<skip functions@>@;
+
+int main(int argc, char *argv[])
+{ @<local variables in |main|@>@;
+ in_ext=".hnt";
+ out_ext=".bak";
+
+ @<process the command line@>@;
+ @<open the log file@>@;
+ if (!hget_map()) QUIT("Unable to map the input file");
+ hpos=hstart=hin_addr;
+ hend=hstart+hin_size;
+ hget_banner();
+ if (!hcheck_banner("hint")) QUIT("Invalid banner");
+ hget_directory();
+ DBG(DBGBASIC,"Skipping Content Section\n");
+ hteg_content_section();
+ DBG(DBGBASIC,"Fast forward Content Section\n");
+ hpos=hstart;
+ while(hpos<hend)
+ { hff_hpos();
+ if (KIND(*(hpos-1))==par_kind && KIND(hff_tag)==list_kind && hff_list_size>0)
+ { uint8_t *p=hpos,*q;
+ DBG(DBGTAGS,"Fast forward list at 0x%x, size %d",hff_list_pos,hff_list_size);
+ hpos=hstart+hff_list_pos;
+ q=hpos+hff_list_size;
+ while (hpos<q)
+ hff_hpos();
+ hpos=p;
+ }
+ }
+ hget_unmap();
+ @<close the log file@>@;
+ return 0;
+explain_usage:
+ @<explain usage@>@;
+ return 1;
+}
+@
+
+As we have seen already in the {\tt stretch} program, a few
+function declarations are necessary to satisfy the define before
+use requirement of \CEE.
+
+@<skip function declarations@>=
+static void hteg_content_node(void);
+static void hteg_content(uint8_t z);
+static void hteg_xdimen_node(xdimen_t *x);
+static void hteg_list(list_t *l);
+static void hteg_param_list(list_t *l);
+static float32_t hteg_float32(void);
+static void hteg_rule_node(void);
+static void hteg_hbox_node(void);
+static void hteg_vbox_node(void);
+static void hteg_glue_node(void);
+@
+
+\thecodeindex
+
+\crosssections
+
+\plainsection{References}
+
+{\baselineskip=11pt
+\def\bfblrm{\small\rm}%
+\def\bblem{\small\it}%
+\bibliography{../hint}
+\bibliographystyle{plain}
+}
+
+\plainsection{Index}
+{
+\def\_{{\tt \UL}} % underline in a string
+\catcode`\_=\active \let_=\_ % underline is a letter
+\input format.ind
+}
+
+\write\cont{} % ensure that the contents file isn't empty
+% \write\cont{\catcode `\noexpand\@=12\relax} % \makeatother
+\closeout\cont% the contents information has been fully gathered
Property changes on: trunk/Build/source/texk/web2c/hitexdir/format.w
___________________________________________________________________
Added: svn:eol-style
## -0,0 +1 ##
+native
\ No newline at end of property
Added: trunk/Build/source/texk/web2c/hitexdir/hitex.w
===================================================================
--- trunk/Build/source/texk/web2c/hitexdir/hitex.w (rev 0)
+++ trunk/Build/source/texk/web2c/hitexdir/hitex.w 2021-09-21 16:46:46 UTC (rev 60569)
@@ -0,0 +1,4753 @@
+% This file is part of HINT
+% Copyright 2017-2021 Martin Ruckert
+%
+% Permission is hereby granted, free of charge, to any person obtaining a copy
+% of this software and associated documentation files (the "Software"), to deal
+% in the Software without restriction, including without limitation the rights
+% to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+% copies of the Software, and to permit persons to whom the Software is
+% furnished to do so, subject to the following conditions:
+%
+% The above copyright notice and this permission notice shall be
+% included in all copies or substantial portions of the Software.
+%
+% THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+% IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+% FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+% COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+% WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT
+% OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+% THE SOFTWARE.
+%
+% Except as contained in this notice, the name of the copyright holders shall
+% not be used in advertising or otherwise to promote the sale, use or other
+% dealings in this Software without prior written authorization from the
+% copyright holders.
+
+\input ../hint.sty
+\input ../changefile.sty
+
+%% defining how to display certain C identifiers
+ at s uint8_t int
+ at s uint16_t int
+ at s uint32_t int
+ at s uint64_t int
+ at s int8_t int
+ at s int16_t int
+ at s int32_t int
+ at s float64_t int
+ at s float32_t int
+ at s scaled int
+ at s pointer int
+ at s memory_word int
+ at s ASCII_code int
+ at s small_number int
+ at s info_t int
+ at s lig_t int
+ at s disc_t int
+ at s math_t int
+ at s glyph_t int
+ at s glue_t int
+ at s rule_t int
+ at s list_t int
+ at s kind_t int
+ at s image_t int
+ at s glue_ord int
+ at s four_quarters int
+ at s eight_bits int
+ at s internal_font_number int
+ at s explicit normal
+ at s str_number int
+ at s dimen_t int
+ at s loop else
+ at s kpse_file_format_type int
+ at s font_t int
+
+
+\makeindex
+\maketoc
+
+%\makefigindex
+\titletrue
+
+\def\lastrevision{${}$Revision: 2485 ${}$}
+\def\lastdate{${}$Date: 2021-08-19 19:37:13 +0200 (Thu, 19 Aug 2021) ${}$}
+\eject
+\input titlepage.tex
+
+
+\frontmatter
+
+@
+
+
+\plainsection{Preface}
+To be written
+
+\vskip 1cm
+\noindent {\it M\"unchen\hfil\break
+August 20, 2018 \hfill Martin Ruckert}
+
+
+\tableofcontent
+%\thefigindex
+
+
+\mainmatter
+
+\section{Introduction}\label{intro}
+Hi\TeX\ is a modified version of \TeX\ that replaces the \.{DVI}
+(device independent) output format by the \HINT\ format. The \HINT\
+format, described in \cite{MR:format}, was designed with two
+objectives: being able to support reflowing pages using the \TeX\
+typesetting engine and making it simple to use it as output format of
+the \TeX\ programm. The present book tries to convince its readers
+that the latter claim is true. To this end, this book implements a
+version of \TeX\ that produces \HINT\ output---to be precise: a short
+format \HINT\ file.
+
+The book is written as a literate program\cite{Knuth:lp} using ``The
+\.{CWEB} System of Structured Documentation''\cite{Knuth:cweb}. The
+largest part of this program is, of course, \TeX\cite{Knuth:tex}
+itself. Therefore a significant part of Hi\TeX\ consists of
+modifications of the \TeX\ source code, which itself is written as a
+literate program using ``The \.{WEB} System of Structured
+Documentation''\cite{Knuth:WEB}. The tiny, but significant,
+difference between \.{WEB}\index{WEB+{\tt WEB}}
+and \.{CWEB}\index{CWEB+{\tt CWEB}} is the transition from
+\Pascal\ to \CEE/ as target language. To bridge the gap between
+\.{WEB} and \.{CWEB}, Hi\TeX\ is not based on the original \.{WEB}
+implementation of \TeX\ but on the \.{CWEB} implementation of \TeX\
+as described in~\cite{MR:webtocweb} and~\cite{MR:web2w}.
+Further, it does not use the plain {\tt ctex.w} translation of {\tt tex.web}
+but the {\tt ktex.w} file which extends \TeX\ with
+ the features of $\epsilon$-\TeX, the file searching
+mechanisms of the {\tt kpathsearch} library, and the command line
+conventions of \TeX\ Live.
+
+To accomplish modifications of a \.{CWEB} file, the \.{CWEB} tools
+support the use of ``change files''. These change files are lists of
+code changes optionally embellished with explanatory text. Each code
+change consists of two parts: a literal copy of the original source
+code followed by the replacement text. When a \.{CWEB} tool applies
+such a change file to a cweb file, it will read both files
+sequentially and applies the code changes in the order given: Whenever
+it finds a section in the \.{CWEB} file that matches the first part of
+the current code change, it will replace it by the second part of the
+code change, and advance to the next code change.
+
+The present book makes an attempt to present these code changes in
+``literate programming style'' and had to overcome two obstacles:
+First, in a literate program, the exposition determines the order of
+appearance of the code sections; and second, the tools that generate
+the documentation from a \.{CWEB} file are not able to generate
+documentation for a change file.
+
+The first problem can be solved by using the program {\tt
+tie}\cite{kg:tie}\cite{jg:ctie}. It allows splitting a single change
+file into multiple change files, and while within each change file,
+the order of changes is still determined by the original cweb file,
+changes that belong together can be grouped into separate files.
+
+The second problem is solved by a simple preprocessor, that converts
+change files into cweb files which then can be converted into
+\TeX. With the help of some modifications of the macros in {\tt
+cwebmac.tex} these \TeX\ files are used to present change files in
+this book.
+
+Let's look at an examples to explain and illustrate the presentation
+of code changes in this book: the {\tt display\_node.ch} change file.
+
+\subsection{The Function |display_node|}
+It is the purpose of the {\tt display\_node.ch} change file to make
+the \TeX\ code contained in the section ``\<Display node |p|>''
+available to other parts of Hi\TeX\ as a function.
+The function is then used to display debugging output.
+
+This requires the following changes:
+
+\changestyle{displaynode.tex}
+
+As you can see, the two parts of a code change are enclosed in square brackets
+where the opening top bracket is labeled {\it old\/} or {\it new\/}.
+While a full understanding of these code changes requires a look at the original \TeX\ code,
+at least they document what was done and why.
+
+There is still one thing missing before we can use the |display_node|
+function in the {\tt hitex.c} file we are about to generate:
+we need an |extern| declaration of |display_node|.
+In fact, the Hi\TeX\ program contains a whole bunch of functions
+that will need access to \TeX's internals and the most readable
+and convenient way to gain this access is a header file.
+
+Generating such a header file can be accomplished by using the {\tt -h}
+and {\tt -e} options of the {\tt web2w} program when converting the {\tt WEB}
+source of \TeX\ to its {\tt cweb} source. We just need to modify the
+process slightly for our purposes.
+
+{
+\changestyle{types.tex}
+}
+
+
+\section{Modifying \TeX}\label{modifyingTeX}
+In this section, we explain the different change files that modify \TeX.
+Larger changes are accomplished by replacing entire functions.
+
+
+\subsection{The |main| Program}
+We add two function calls to \TeX's |main| program:
+|hint_open|, and |hint_close|.
+
+{
+\changestyle{main.tex}
+}
+The functions to open and close the \HINT\ file follow in
+section~\secref{output}.
+
+
+\subsection{The Command Line}
+Hi\TeX\ has a few additional command line options that we define next.
+
+{
+\changestyle{command_line.tex}
+}
+
+Above we used the function |hint_debug_help|. It is defined as:
+
+@<Hi\TeX\ routines@>=
+#ifdef DEBUG
+void hint_debug_help(void)
+{
+fprintf(stderr,@/
+ "To generate HINT format debug output use the option\n"
+ " -debug=XX "@/
+ at t\qquad@>"\t XX is a hexadecimal value. OR together these values:\n");@/
+fprintf(stderr,"\t\t\t XX=%04X \t basic debugging\n", DBGBASIC);@/
+fprintf(stderr,"\t\t\t XX=%04X \t tag debugging\n", DBGTAGS);@/
+fprintf(stderr,"\t\t\t XX=%04X \t node debugging\n",DBGNODE);@/
+fprintf(stderr,"\t\t\t XX=%04X \t definition debugging\n", DBGDEF);@/
+fprintf(stderr,"\t\t\t XX=%04X \t directory debugging\n", DBGDIR);@/
+fprintf(stderr,"\t\t\t XX=%04X \t range debugging\n",DBGRANGE);@/
+fprintf(stderr,"\t\t\t XX=%04X \t float debugging\n", DBGFLOAT);@/
+fprintf(stderr,"\t\t\t XX=%04X \t compression debugging\n", DBGCOMPRESS);@/
+fprintf(stderr,"\t\t\t XX=%04X \t buffer debugging\n", DBGBUFFER);@/
+fprintf(stderr,"\t\t\t XX=%04X \t TeX debugging\n", DBGTEX);@/
+fprintf(stderr,"\t\t\t XX=%04X \t page debugging\n", DBGPAGE);@/
+fprintf(stderr,"\t\t\t XX=%04X \t font debugging\n", DBGFONT);@/
+exit(0);
+}
+#endif
+@
+
+
+
+\subsection{The Page Builder}\label{pagebuilder}
+The point where Hi\TeX\ goes an entirely different path than \TeX\ is the page builder:
+Instead of building a page, Hi\TeX\ writes a \HINT\ file.
+
+{
+\changestyle{build.tex}
+}
+
+
+
+\subsection{Adding new {\tt whatsit} Nodes}
+\TeX\ has a mechanism to extend it: the {\tt whatsit} nodes.
+For new concepts like baseline specifications, paragraphs, and boxes with unknown dimensions,
+we define now special {\tt whatsit} nodes. The new node definitions are supplemented by
+procedures to print them, copy them, delete them, and handle them in various contexts.
+
+{
+\changestyle{whatsit.tex}
+}
+
+\subsubsection{Freeing}
+Because some of the new nodes occur at places where \TeX\ originaly only
+handles box nodes,
+there are many places in \TeX\ where we can no longer just say |free_node(b, box_node_size)|
+to free a (box) node.
+Instead, we have to use the more general routine |flush_node_list|.
+This leads to a long series of simple replacements:
+
+{
+\changestyle{free.tex}
+}
+
+\subsubsection{Unpacking}
+In the function |unpackage| the pointer |p| might now point to an
+hset, vset, hpack or vpack node instead of a vbox or hbox node.
+We have to adapte the following test to this new situation.
+
+{
+\changestyle{unpackage.tex}
+}
+
+We continue to define auxiliar functions to create the new nodes.
+
+\subsubsection{Creating}
+
+The following functions create nodes for paragraphs, displayed equations, baseline skips,
+hpack nodes, vpack nodes, hset nodes, vset nodes, and image nodes.
+
+@<Hi\TeX\ routines@>=
+pointer new_graf_node(void)
+{ @+ pointer p;
+ p=get_node(graf_node_size);
+ type(p)=whatsit_node;
+ subtype(p)=graf_node;
+ graf_params(p)=null;
+ graf_list(p)=null;
+ return p;
+}
+
+
+pointer new_disp_node(void)
+{ @+ pointer p;
+ p=get_node(disp_node_size);
+ type(p)=whatsit_node;
+ subtype(p)=disp_node;
+ display_params(p)=null;
+ display_formula(p)=null;
+ display_eqno(p)=null;
+ return p;
+}
+
+pointer new_baseline_node(pointer bs, pointer ls, scaled lsl)
+{ @+pointer p;
+ p=get_node(baseline_node_size);
+ type(p)=whatsit_node;
+ subtype(p)=baseline_node;
+ baseline_node_no(p)=hget_baseline_no(bs, ls, lsl);
+ return p;
+}
+
+pointer new_pack_node(void)
+{ @+pointer p;
+ p=get_node(pack_node_size);
+ type(p)=whatsit_node;
+ subtype(p)=hpack_node;
+ width(p)=depth(p)=height(p)=shift_amount(p)=0;
+ pack_limit(p)=max_dimen;
+ list_ptr(p)=null;
+ return p;
+}
+pointer new_set_node(void)
+{ @+pointer p;
+ p=get_node(set_node_size);
+ type(p)=whatsit_node;
+ subtype(p)=hset_node;
+ width(p)=depth(p)=height(p)=shift_amount(p)=set_stretch(p)=set_shrink(p)=set_extent(p)=0;
+ list_ptr(p)=null;
+ return p;
+}
+
+
+
+
+pointer new_image_node( str_number n, char *a, char *e)
+{ pointer p;
+ int i;
+ char *fn;
+ int l;
+
+ p=get_node(image_node_size);type(p)=whatsit_node;subtype(p)=image_node;
+ image_name(p)=n;
+ image_area(p)=s_no(a);
+ image_ext(p)=s_no(e);
+ fn=hfile_name(n,a,e);
+#if 0
+ fn=kpse_find_tex(fn);
+#endif
+ i=hnew_file_section(fn);
+#if 0
+ free(fn);
+#endif
+ image_no(p)=i;
+ image_width(p)=image_height(p)=image_stretch(p)=image_shrink(p)=0;
+ image_shrink_order(p)=image_stretch_order(p)=normal;@/
+ return p;
+}
+
+
+
+@
+
+\subsubsection{Parameter nodes}
+
+Parameter nodes are added to the current list using the |add_par_node| function.
+It should be possible to check the parameter values against those
+stored in the definition section and remove the ones that
+are unchanged. It would make the parameter lists shorter, saving
+some time when setting and restoring them later.
+There is probably not much savings in memory space, because
+most of the times a reference number is found for the parameter list.
+@<Create the parameter node@>=
+ p=get_node(par_node_size);
+ type(p)=whatsit_node;
+ subtype(p)=par_node;
+ par_type(p)=t;
+ par_number(p)=n;
+@
+
+@<Initialize the parameter node@>=
+ if (t==int_type) par_value(p).i=v;
+ else if (t==dimen_type) par_value(p).sc=v;
+ else if (t==glue_type)
+ {@+ par_value(p).i=v;add_glue_ref(par_value(p).i); @+}
+ else
+ { free_node(p, par_node_size);
+ QUIT("Undefined parameter type %d",t);
+ }
+@
+
+@<Hi\TeX\ routines@>=
+void add_par_node(uint8_t t, uint8_t n, int v)
+{ @+pointer p;
+ @<Create the parameter node@>@;
+ @<Initialize the parameter node@>@;
+ link(p)=link(temp_head);
+ link(temp_head)=p;
+}
+@
+
+
+\subsection{Extended Dimensions}
+An extended dimension is a linar function of {\tt hsize} and {\tt vsize}, and
+whenever \TeX\ works with a dimension, we want it to be able to deal with
+an extended dimension. This implies many changes throughout \TeX's sources.
+So let's get started.
+
+{
+\changestyle{xdimen.tex}
+}
+
+
+
+
+\subsection{Hyphenation}
+While the breaking of a paragraph into lines must be postponed because
+{\tt hsize} is not known, hyphenation should be done as part of Hi\TeX\
+because we want to keep hyphenation out of the viewer. Therefore
+Hi\TeX\ will do hyphenation for all words within a paragraph.
+
+There is a fine point to observe here: \TeX\ will consider a word as
+a candidate for automatic hyphenation only if the world ``follows'' after a
+glue. (For the exact rules, see the \TeX book\cite{DK:texbook}, Appendix H.)
+As a consequence, \TeX\ usually does not submit the first word of a
+paragraph to its hyphenation routine.
+Viewing paragraphs that start with a lengthy word on a narrow display
+therefore often look more unsightly than necessary: the long word sticks out
+into the right margin as much as it can. To remedy this situation,
+Hi\TeX\ has a ``{\tt -f}'' option. If set Hi\TeX\ will deviate from
+\TeX's rules and submit the first word of a paragraph to the hyphenation algorithm.
+
+The next problem arises from \TeX's multipass approach to line breaking
+and the attempt to have Hi\TeX\ choose exactly the same line breaks as
+\TeX\ does:
+\TeX\ distingishes between discretionary breaks inserted by the author of a
+text, and discretionary breaks discovered by the hyphenation routine.
+The latter, called here ``automatic'', are used only in pass two and three
+of the line breaking routine.
+
+The change file that follows below contains mostly three types of changes:
+\itemize
+\item Hi\TeX\ restricts the |replace_count| to seven bit to make room for an
+extra bit to mark automatic discretionary breaks.
+Assignments to the replace count are therefore replaced by a macro that
+protects the automatic bit.
+\item The \TeX\ code for ``\<Try to hyphenate the following word>''
+is packaged into a function |hyphenate_word|, so that it can be invoked as needed.
+\item And calls of the function |line_break| are replaced by calls to |hline_break|
+to have a hook for inserting Hi\TeX specific changes.
+\enditemize
+
+{
+\changestyle{disc.tex}
+}
+
+The function |hline_break| follows:
+
+
+@<Hi\TeX\ routines@>=
+void hline_break(int final_widow_penalty)
+{@+ bool auto_breaking; /*is node |cur_p| outside a formula?*/
+ pointer r, s ; /*miscellaneous nodes of temporary interest*/
+ pointer pp;
+ bool par_shape_fix=false;
+ bool first_word=true;
+ if (DBGTEX&debugflags)
+ { print_ln();print("Before hline_break:\n");
+ breadth_max=200;
+ depth_threshold=200;
+ show_node_list(link(head));print_ln();
+ }
+ if (dimen_par_hfactor(hsize_code)==0 && dimen_par_vfactor(hsize_code)==0)
+ { line_break(final_widow_penalty); /* the easy case */
+ return;
+ }
+ /* Get ready to start line breaking */
+ pp=new_graf_node();
+ graf_penalty(pp)=final_widow_penalty;
+ if(par_shape_ptr==null)
+ graf_extent(pp)=new_xdimen(dimen_par(hsize_code),
+ dimen_par_hfactor(hsize_code),dimen_par_vfactor(hsize_code));
+ else
+ @<fix the use of parshape = 1 indent length@>@;
+ link(temp_head)=link(head);
+ if (is_char_node(tail))
+ { tail_append(new_penalty(inf_penalty))@;
+ tail_append(new_param_glue(par_fill_skip_code));
+ }
+ else if (type(tail)!=whatsit_node || subtype(tail)!=disp_node)
+ { if (type(tail)!=glue_node) tail_append(new_penalty(inf_penalty))@;
+ else
+ {@+type(tail)=penalty_node;delete_glue_ref(glue_ptr(tail));
+ flush_node_list(leader_ptr(tail));penalty(tail)=inf_penalty;
+ }
+ link(tail)=new_param_glue(par_fill_skip_code);
+ }
+ DBG(DBGTEX,"\nCalling line_break:\n"
+ "hang_indent=0x%08X hang_after=%d",hang_indent,hang_after);
+ if (line_skip_limit!=0)
+ DBG(DBGTEX," line_skip_limit=0x%08X",line_skip_limit);
+ DBG(DBGTEX," prev_graf=0x%08X",prev_graf);
+
+ init_cur_lang=prev_graf%0200000;
+ init_l_hyf=prev_graf/020000000;
+ init_r_hyf=(prev_graf/0200000)%0100;
+ pop_nest();
+ DBG(DBGTEX," prev_graf=0x%08X",prev_graf);
+
+ /* Initialize for hyphenating...*/
+#ifdef INIT
+ if (trie_not_ready) init_trie();
+#endif
+ cur_lang=init_cur_lang;l_hyf=init_l_hyf;r_hyf=init_r_hyf;
+ if (DBGTEX&debugflags)
+ { print_ln();print("Before hyphenation:\n");
+ breadth_max=200;
+ depth_threshold=200;
+ show_node_list(link(temp_head));print_ln();
+ }
+ auto_breaking=true;
+ cur_p=temp_head;
+ if (option_hyphen_first && is_char_node(link(cur_p)))
+ { hyphenate_word(); first_word=false; }
+ cur_p=link(cur_p);
+ while (cur_p!=null)
+ { /*Call |try_break| if |cur_p| is a legal breakpoint...*/
+ if (is_char_node(cur_p))
+ { /* Advance |cur_p| to the node following the present string...*/
+ do {
+ cur_p=link(cur_p);
+ } while (is_char_node(cur_p));
+ if (cur_p==null) goto done5; /* mr: no glue and penalty at the end */
+ }
+ switch (type(cur_p))
+ { case whatsit_node:
+ adv_past(cur_p);
+ break;
+ case glue_node:
+ if (auto_breaking) /* Try to hyphenate the following word*/
+ hyphenate_word();
+ break;
+ case ligature_node:
+ break;
+ case disc_node:
+ /* Try to break after a discretionary fragment...*/
+ r=replace_count(cur_p);s=link(cur_p);
+ while (r > 0)
+ { decr(r);s=link(s);
+ }
+ cur_p=s;
+ goto done5;
+ case math_node:
+ auto_breaking=(subtype(cur_p)==after);
+ break;
+ default:
+ break;
+ }
+ if (option_hyphen_first && first_word && is_char_node(link(cur_p)))
+ { hyphenate_word(); first_word=false; }
+ cur_p=link(cur_p);
+done5:;
+ }
+ if (DBGTEX&debugflags)
+ { print_ln();print("After hline_break:\n");
+ breadth_max=200;
+ depth_threshold=200;
+ show_node_list(link(temp_head));print_ln();
+ }
+ graf_list(pp)=link(temp_head);
+ /* adding parameter nodes */
+ link(temp_head)=null;
+
+ add_par_node(int_type,pretolerance_code,pretolerance);
+ add_par_node(int_type,tolerance_code,tolerance);
+ add_par_node(dimen_type,emergency_stretch_code,emergency_stretch);
+
+ add_par_node(int_type,line_penalty_code,line_penalty);
+ add_par_node(int_type,hyphen_penalty_code,hyphen_penalty);
+ add_par_node(int_type,ex_hyphen_penalty_code,ex_hyphen_penalty);
+ add_par_node(int_type,club_penalty_code,club_penalty);
+ add_par_node(int_type,widow_penalty_code,widow_penalty);
+ add_par_node(int_type,broken_penalty_code,broken_penalty);
+ add_par_node(int_type,inter_line_penalty_code,inter_line_penalty);
+ add_par_node(int_type,double_hyphen_demerits_code,double_hyphen_demerits);
+ add_par_node(int_type,final_hyphen_demerits_code,final_hyphen_demerits);
+ add_par_node(int_type,adj_demerits_code,adj_demerits);
+ add_par_node(int_type,looseness_code,looseness);
+
+ if (par_shape_fix)
+ { add_par_node(int_type,hang_after_code,0);
+ add_par_node(dimen_type,hang_indent_code,second_indent);
+ }
+ else
+ { add_par_node(int_type,hang_after_code,hang_after);
+ add_par_node(dimen_type,hang_indent_code, hang_indent);
+ }
+
+ add_par_node(dimen_type,line_skip_limit_code,line_skip_limit);
+ add_par_node(glue_type,line_skip_code,line_skip);
+ add_par_node(glue_type,baseline_skip_code,baseline_skip);
+
+ add_par_node(glue_type,left_skip_code,left_skip);
+ add_par_node(glue_type,right_skip_code,right_skip);
+ add_par_node(glue_type,par_fill_skip_code,par_fill_skip);
+
+
+ /* |par_shape| is not yet supported */
+ graf_params(pp)=link(temp_head);
+ link(temp_head)=null;
+ append_to_vlist(pp);
+}
+@
+
+Currently Hi\TeX\ dos not implement the parshape feature of \TeX.
+The implementation of {\tt \BS list} in \LaTeX\ does however depend
+on a simple us of parshape where all lines have the same length
+and indentation. We cover this special case be using a hanging
+indentation and adjusting the paragraph width by the difference
+of the normal {\tt \BS hsize} and the given length.
+
+@<fix the use of parshape = 1 indent length@>=
+{ last_special_line= info(par_shape_ptr)-1;
+ if (last_special_line!=0)
+ DBG(DBGTEX,"Warning parshape with n=%d not yet implemented",info(par_shape_ptr));
+ second_width= mem[par_shape_ptr+2*(last_special_line+1)].sc;
+ second_indent= mem[par_shape_ptr+2*last_special_line+1].sc;
+
+ graf_extent(pp)=new_xdimen(second_indent+second_width,
+ par_shape_hfactor,par_shape_vfactor);
+ second_width=second_width+ round((double)par_shape_hfactor*hhsize/unity
+ +(double)par_shape_vfactor*hvsize/unity);
+ par_shape_fix=true;
+}
+@
+
+\subsection{Baseline Skips}
+
+\TeX\ will automatically insert a baseline skip between two boxes in a
+vertical list. The baseline skip is computed to make the distance
+between the baselines of the two boxes exactly equal to the value of
+{\tt \BS baselineskip}. And if that is not possible, because it would
+make the distance between the descenders of the upper box and the
+ascenders of the lower box smaller than {\tt \BS lineskiplimit}, it
+will insert at least a small amount of glue of size {\tt \BS
+lineskip}. As a further complication, if the depth of the upper box
+has the special value |ignore_depth| the insertion of a baseline skip
+is suppressed. It is also common practice that authors manipulate the
+values of {\tt \BS baselineskip}, {\tt \BS lineskiplimit}, {\tt \BS
+lineskip}, and {\tt \BS prevdepth} to change the baseline calculations
+of \TeX.
+
+Of course a prerequisite of the whole computation of a baseline skip
+is the knowledge of the depth of the upper box and the height of the
+lower box. With the new types of boxes introduced by Hi\TeX\ neither
+of them might be known. In these cases, the baseline skip computation
+must be defered to the viewer by inserting a whatsit node of subtype
+|baseline_node|.
+
+
+{
+\changestyle{baseline.tex}
+}
+
+
+\subsection{Displayed Formulas}
+
+\TeX\ enters into math mode when it finds a math shift character ``\$''.
+Then it calls |init_math| which checks for an aditional math shift
+character to call either
+ \<Go into ordinary math mode> or \<Go into display math mode>.
+We remove most of the differences from the latter because we will
+treat both cases very similar here and will consider the necessary
+differences in the \HINT\ viewer.
+
+When the shift character that terminates math more is encountered,
+\TeX\ calls |after_math| which ends with
+either \<Finish math in text> or \<Finish displayed math> .
+We need to modify the latter, because positioning a displayed
+equation usually depends on {\tt hsize},
+and must be postponed until the viewer knows its value.
+So all Hi\TeX\ can do in this case is create a new whatsit node
+with subtype |disp_node| and
+insert it proceeding as if the formula had occured in ordinary math mode.
+
+
+{
+\changestyle{display.tex}
+}
+
+
+
+
+\subsection{Alignments}
+
+
+{
+\changestyle{align.tex}
+}
+
+
+\subsection{Inserts}
+\TeX\ uses |vpack| to determine the natural height of inserted
+material. In Hi\TeX, the function |vpack| will not always return
+a |vlist_node| with a fixed height because the computation of the
+height might fail. In the latter case, |vpack| will return a |vset_node|
+or |vpack_node|. Therfore Hi\TeX\ will not use the |height| field of
+a an |ins_node|.
+
+As a result we have the following changes:
+
+
+{
+\changestyle{insert.tex}
+}
+
+
+
+
+\subsection{Implementing Hi\TeX\ Specific Primitives}\label{primitive}
+Now we implement new \TeX\ primitives:
+one to include images, a few more to define page teamplates,
+and one to test whether the current instance of \TeX\ is
+the new Hi\TeX.
+To implement a primitive, we call the |primitive| function.
+Since it needs the name of the new primitive as a \TeX\ string,
+we need to add the strings to the \TeX\ string pool.
+
+Note that primitives are put into \TeX's memory when a \TeX\ format file is read.
+So adding or changing the code below implies regenerating the format files.
+To generate a format file run a command like
+
+{\tt\obeylines
+hinitex
+**plain \BS dump
+}
+
+\noindent
+Then find out where the \kpse\ library looks for format files
+(for example by running
+
+{\tt\obeylines
+ kpsewhich latex.fmt
+}
+
+\noindent
+and then place the new format file in the same directory.
+
+
+{
+\changestyle{primitive.tex}
+}
+
+
+
+
+\section{Hi\TeX}
+
+\subsection{Images}
+The handling of images is an integral part of Hi\TeX. In
+section~\secref{imageext} on page~\pageref{imageext} the {\tt image}
+primitive was defined. It requires a filename for the image and
+allows the specification, of width, height, stretch and shrink of an
+image. If either width or height is not given, Hi\TeX\ tries to
+extract this information from the image file itself calling the
+function |hget_image_information| with the pointer |p| to the image
+node as parameter.
+
+@<Hi\TeX\ routines@>=
+void hget_image_information(pointer p)
+{ char *fn;
+ FILE *f;
+ if (image_width(p)!=0 && image_height(p)!=0) return;
+ fn=dir[image_no(p)].file_name;
+ f=fopen(fn,"rb");
+ if (f==NULL) QUIT("Unable to open image file %s.", fn);
+ MESSAGE("(%s",fn);
+ img_buf_size=0;
+ if (!get_BMP_info(f,fn,p)&&!get_PNG_info(f,fn,p)&&!get_JPG_info(f,fn,p))
+ QUIT("Unable to obtain width and height information for image %s",fn);
+ fclose(f);
+ MESSAGE(" width= %fpt height= %fpt)",image_width(p)/(double)ONE,image_height(p)/(double)ONE);
+}
+@
+
+When we have found the width and height of the stored image, we can
+supplement the information given in the \TeX\ file. Occasionaly, the
+image file will not specify the absolute dimensions of the image. In
+this case, we can still compute the aspect ratio and supplement either
+the width based on the height or vice versa. This is accomplished by
+calling the |set_image_dimensions| function. It returns true on
+success and false otherwise.
+
+@<Hi\TeX\ auxiliar routines@>=
+static bool set_image_dimensions(pointer p, double w, double h, bool absolute)
+{ if (image_width(p)!=0)
+ { double aspect=h/w;
+ image_height(p)=round(image_width(p)*aspect);
+ }
+ else if (image_height(p)!=0)
+ { double aspect=w/h;
+ image_width(p)=round(image_height(p)*aspect);
+ }
+ else
+ { if (!absolute) return false;
+ image_width(p)=round(unity*w);
+ image_height(p)=round(unity*h);
+ }
+ return true;
+}
+@
+We call the following routines with the image buffer partly filled
+with the start of the image file.
+
+@<Hi\TeX\ auxiliar routines@>=
+#define IMG_BUF_MAX 54
+#define IMG_HEAD_MAX 2
+static unsigned char img_buf[IMG_BUF_MAX];
+static size_t img_buf_size;
+#define @[LittleEndian32(X)@] (img_buf[(X)]+(img_buf[(X)+1]<<8)+\
+ (img_buf[(X)+2]<<16)+(img_buf[(X)+3]<<24))
+
+#define @[BigEndian16(X)@] (img_buf[(X)+1]+(img_buf[(X)]<<8))
+
+#define @[BigEndian32(X)@] (img_buf[(X)+3]+(img_buf[(X)+2]<<8)+\
+ (img_buf[(X)+1]<<16)+(img_buf[(X)]<<24))
+
+#define Match2(X,A,B) ((img_buf[(X)]==(A)) && (img_buf[(X)+1]==(B)))
+#define Match4(X,A,B,C,D) (Match2(X,A,B)&&Match2((X)+2,C,D))
+
+#define @[GET_IMG_BUF(X)@] \
+if (img_buf_size<X) \
+ { size_t i=fread(img_buf+img_buf_size,1,(X)-img_buf_size,f); \
+ if (i<0) QUIT("Unable to read image %s",fn); \
+ else if (i==0) QUIT("Unable to read image header %s",fn); \
+ else img_buf_size+=i; \
+ }
+@
+
+\subsubsection{BMP}
+We start with Windows Bitmaps. A Windows bitmap file usually has the extension {\tt .bmp}
+but the better way to check for a Windows bitmap file ist to examine the first two byte
+of the file: the ASCII codes for `B' and `M'.
+Once we have verified the file type, we find the width and height of the bitmap in pixels
+at offsets |0x12| and |0x16| stored as little-endian 32 bit integers. At offsets |0x26| and |0x2A|,
+we find the horizontal and vertical resolution in pixel per meter stored in the same format.
+This is sufficient to compute the true width and height of the image in scaled points.
+If either the width or the height was given in the \TeX\ file, we just compute the aspect ratio
+and compute the missing value.
+
+The Windows Bitmap format is easy to process but not very efficient. So the support for this
+format in Hi\TeX\ is deprecated and will disappear. You should use one of the formats described next.
+
+@<Hi\TeX\ auxiliar routines@>=
+static bool get_BMP_info(FILE *f, char *fn, pointer p)
+{ double w,h;
+ double xppm,yppm;
+ GET_IMG_BUF(2);
+ if (!Match2(0,'B','M')) return false;
+ GET_IMG_BUF(0x2E);
+ w=(double)LittleEndian32(0x12); /*width in pixel*/
+ h=(double)LittleEndian32(0x16); /*height in pixel*/
+ xppm=(double)LittleEndian32(0x26); /* horizontal pixel per meter*/
+ yppm=(double)LittleEndian32(0x2A); /* vertical pixel per meter*/
+ return set_image_dimensions(p,(72.27*1000.0/25.4)*w/xppm,(72.27*1000.0/25.4)*h/yppm,true);
+}
+@
+
+\subsubsection{PNG}
+Now we repeat this process for image files in using the Portable Network Graphics\cite{png2nd} file
+format. This file format is well suited to simple graphics that do not use color gradients.
+These images usually have the extension {\tt .png} and start with an eight byte signature:
+|0x89| followed by the ASCII Codes `P', `N', `G', followd by a carriage return (|0x0D| and line feed (|0x0A|),
+an DOS end-of-file character (|0x1A|) and final line feed (|0x0A|).
+After the signature follows a list of chunks. The first chunk is the image header chunk.
+Each chunk starts with the size of the chunk stored as big-endian 32 bit integer, followed by the chunk name
+stored as four ASCII codes followed by the chunk data and a CRC. The size as stored in the chunk
+does not include the size itself, the name or the CRC.
+The first chunk is the IHDR chunk.
+The chunk data of the IHDR chunk starts with the width and the height of the image in pixels
+stored as 32 bit big-endian integers.
+
+Finding the image resolution takes some more effort. The image resolution is stored in an optional chunk
+named ``pHYs'' for the physical pixel dimensions.
+All we know is that this chunk, if it exists, will appear after the IHDR
+chunk and before the (required) IDAT chunk. The pHYs chunk contains two 32 bit big-endian integers,
+giving the horizontal and vertical pixels per unit, and a one byte unit specifier, which is either 0
+for an undefined unit or 1 for the meter as unit. With an undefined unit only the aspect ratio
+of the pixels anh hence the aspect ratio of the image can be determined.
+
+
+@<Hi\TeX\ auxiliar routines@>=
+
+static bool get_PNG_info(FILE *f, char *fn, pointer p)
+{ int pos, size;
+ double w,h;
+ double xppu,yppu;
+ int unit;
+ GET_IMG_BUF(24);
+ if (!Match4(0, 0x89, 'P', 'N', 'G') ||
+ !Match4(4, 0x0D, 0x0A, 0x1A, 0x0A)) return false;
+ size=BigEndian32(8);
+ if (!Match4(12,'I', 'H', 'D', 'R')) return false;
+ w=(double)BigEndian32(16);
+ h=(double)BigEndian32(20);
+ pos=20+size;
+ while (true)
+ { if (fseek(f,pos,SEEK_SET)!=0) return false;
+ img_buf_size=0;
+ GET_IMG_BUF(17);
+ size=BigEndian32(0);
+ if (Match4(4,'p', 'H', 'Y', 's'))
+ { xppu =(double)BigEndian32(8);
+ yppu =(double)BigEndian32(12);
+ unit=img_buf[16];
+ if (unit==0)
+ return set_image_dimensions(p,w/xppu,h/yppu,false);
+ else if (unit==1)
+ return set_image_dimensions(p,(72.27/0.0254)*w/xppu,(72.27/0.0254)*h/yppu,true);
+ else
+ return false;
+ }
+ else if (Match4(4,'I', 'D', 'A', 'T'))
+ return set_image_dimensions(p,w,h,false);
+ else
+ pos=pos+12+size;
+ }
+ return false;
+}
+@
+
+\subsubsection{JPG}
+For photographs, the JPEG File Interchange Format (JFIF)\cite{Ham92}\cite{JPEG93} is more appropriate.
+JPEG files come with all sorts of file extensions like {\tt .jpg}, {\tt .jpeg}, or {\tt .jfif}.
+We check the file siganture: it starts with the the SOI (Start of Image) marker |0xFF|, |0xD8|
+followed by the JIFI-Tag. The JIFI-Tag starts with the segment marker APP0 (|0xFF|, |0xE0|) followed by the
+2 byte segment size, followed by the ASCII codes `J', `F', `I', `F' followed by a zero byte.
+Next is a two byte version number which we do not read.
+Before the resolution proper there is a resolution unit indicator byte (0 = no units,
+1 = dots per inch, 2 = dots per cm) and then comes the horizontal and vertical resolution both
+as 16 Bit big-endian integers.
+To find the actual width and height, we have to search for a start of frame marker (|0xFF|, |0xC0|+$n$ with $0\le n\le 15$). Which is followed by the 2 byte segment size, the 1 byte sample precission, the
+2 byte height and the 2 byte width.
+
+
+@<Hi\TeX\ auxiliar routines@>=
+
+static bool get_JPG_info(FILE *f, char *fn, pointer p)
+{ int pos, size;
+ double w,h;
+ double xppu,yppu;
+ int unit;
+ GET_IMG_BUF(18);
+
+ if (!Match4(0, 0xFF,0xD8, 0xFF, 0xE0)) return false;
+ size=BigEndian16(4);
+ if (!Match4(6,'J', 'F', 'I', 'F')) return false;
+ if (img_buf[10] != 0) return false;
+ unit=img_buf[13];
+ xppu=(double)BigEndian16(14);
+ yppu=(double)BigEndian16(16);
+ pos=4+size;
+ while (true)
+ { if (fseek(f,pos,SEEK_SET)!=0) return false;
+ img_buf_size=0;
+ GET_IMG_BUF(10);
+ if (img_buf[0] != 0xFF) return false; /* Not the start of a segment */
+ if ( (img_buf[1]&0xF0) == 0xC0) /* Start of Frame */
+ { h =(double)BigEndian16(5);
+ w =(double)BigEndian16(7);
+ if (unit==0)
+ return set_image_dimensions(p,w/xppu,h/yppu,false);
+ else if (unit==1)
+ return set_image_dimensions(p,72.27*w/xppu,72.27*h/yppu,true);
+ else if (unit==2)
+ return set_image_dimensions(p,(72.27/2.54)*w/xppu,(72.27/2.54)*h/yppu,true);
+ else
+ return false;
+ }
+ else
+ { size= BigEndian16(2);
+ pos=pos+2+size;
+ }
+ }
+ return false;
+}
+@
+
+\subsubsection{SVG}
+There is still one image format missing: scalable vector graphics.
+In the moment, I tend not to include a further image format into
+the definition of the \HINT\ file format but instead use the
+PostScript subset that is used for Type 1 fonts to encode
+vector graphics. Any \HINT\ viewer must support Type 1
+PostScript fonts and hence it has already the necessary interpreter.
+So it seems reasonable to put the burden of converting vector graphics
+into a Type 1 PostScript font on the generator of \HINT\ files
+and keep the \HINT\ viewer as small and simple as possible.
+
+
+\subsection{Links, Labels, and Outlines}
+The \HINT\ format knows about labels, links, and outlines.
+When generating a short format \HINT\ file, links are part of
+the content section, where as labels and outlines are found in
+the definition section. Because labels are defined while
+writing the content section, the writing of labels and outlines, which
+reference the labels, must be postponed. For that reason,
+we store information about labels and outlines in dynamic arrays,
+and map labels, which are identified by a name or a number,
+to their index using a dynamic hash table.
+
+We start with two functions that allocate new entries in the
+dynamic arrays increasing their size if necessary.
+@<Hi\TeX\ auxiliar routines@>=
+static int next_label(void)
+{ static int label_no=-1;
+ static int labels_allocated =0;
+ label_no++;
+ if (label_no>0xFFFF)
+ overflow("labels",0xFFFF);
+ if (label_no>=labels_allocated)
+ { if (labels_allocated==0)
+ { labels_allocated=32; ALLOCATE(labels,labels_allocated,label_t); }
+ else RESIZE(labels,labels_allocated,label_t);
+ }
+ max_ref[label_kind]=label_no;
+ return label_no;
+}
+
+static int next_outline(void)
+{ static int outlines_allocated =0;
+ static int outline_no=-1;
+ outline_no++;
+ if (outline_no>0xFFFF)
+ overflow("outlines",0xFFFF);
+ if (outline_no>=outlines_allocated)
+ { if (outlines_allocated==0)
+ { outlines_allocated=32; ALLOCATE(outlines,outlines_allocated,outline_t); }
+ else RESIZE(outlines,outlines_allocated,outline_t);
+ }
+ max_outline=outline_no;
+ return outline_no;
+}
+@
+
+While processing the content nodes, access to the labels is provided either
+by name or by number through a hash table. We store tabel entries in linked
+lists starting with a reasonably sized table of pointers. This keeps
+the fixed costs low and guards against overflow and rapidly increasing
+inefficiency. We start with a function to insert a new entry into
+the hash table.
+
+@<Hi\TeX\ auxiliar routines@>=
+typedef struct hash_entry
+{int num; char *nom; uint16_t n; struct hash_entry *next;} hash_entry_t;
+#define LABEL_HASH 1009 /* MIX a prime number */
+static hash_entry_t *label_hash[LABEL_HASH]={NULL};
+
+static int insert_hash(int h, int num, char *nom)
+{ hash_entry_t *e;
+ ALLOCATE(e,1,hash_entry_t);
+ e->n= next_label();
+ if (nom!=NULL) e->nom=strdup(nom);
+ else e->num=num;
+ e->next= label_hash[h];
+ label_hash[h]=e;
+ if (e->nom!=NULL)
+ DBG(DBGLABEL,"Creating new label *%d: name='%s'\n",e->n,e->nom);
+ else
+ DBG(DBGLABEL,"Creating new label *%d: num=%d\n",e->n,e->num);
+ return e->n;
+}
+@
+
+There are two cases: finding a label by name or by number.
+We start with the simpler case where the number is given.
+The process is straigth forward:
+
+@<Hi\TeX\ auxiliar routines@>=
+static int find_label_by_number(int p)
+{ unsigned int h=(unsigned int)p%LABEL_HASH;
+ hash_entry_t *e= label_hash[h];
+ while (e!=NULL)
+ if (e->nom==NULL && e->num==p) return e->n;
+ else e=e->next;
+ return insert_hash(h,p,NULL);
+}
+@
+
+To look up a label by its name as given by a token list,
+we prepare ourselfs by implementing two functions:
+one to extract the character codes from the token list
+forming the ``name''
+and one to compute the hash value for a name.
+The routine to find the label by name is then equivalent to the
+routine we have just seen. Given a pointer |p| to
+either a label, a link, or an outline node, the last function
+returns the correct label reference.
+Currently, we limit label names to at most 255 byte not counting the
+zero byte.
+
+@<Hi\TeX\ auxiliar routines@>=
+static char *tokens_to_name(pointer p)
+{ static char s[256];
+ int i=0;
+ bool skip_space=0;
+ while (i<255 && p!=0)@/
+ { int m = info(p)/0400;@+
+ int c = info(p)%0400;
+ if (m==spacer && ! skip_space) @/
+ { s[i++]=' '; skip_space=true;@+}
+ else if ((m==letter || m==other_char) && ' '< c && c < 0x7F)@/
+ {@+ s[i++]=c; skip_space=false;@+}
+ p=link(p);
+ }
+ s[i]=0;
+ return s;
+}
+
+static unsigned int name_hash(char *s)
+{ unsigned int h=0;
+ while (*s!=0)
+ h=(h<<2)+*(s++);
+ return h;
+}
+
+static int find_label_by_name(pointer p)
+{ char *s=tokens_to_name(link(p));
+ unsigned int h=name_hash(s)%LABEL_HASH;
+ hash_entry_t *e= label_hash[h];
+ while (e!=NULL)
+ if (e->nom!=NULL && strcmp(e->nom,s)==0) return e->n;
+ else e=e->next;
+ return insert_hash(h,0,s);
+}
+@
+
+We combine both ways of finding a label reference in the following function:
+
+@<Hi\TeX\ auxiliar routines@>=
+static int find_label(pointer p)
+{@+ if (label_has_name(p)) return find_label_by_name(label_ptr(p));
+ else return find_label_by_number(label_ptr(p));
+}
+@
+
+After these preparations, we can implement the functions needed
+when labels, links, and outlines are delivered to the page builder.
+
+We start with looking at the labels:
+When a labels is defined, the current position is recorded.
+Further labels are linked together in order of descending positions,
+to allow the efficient adjustment of label positions when
+moving lists.
+
+@<Hi\TeX\ auxiliar routines@>=
+static void new_label(pointer p)
+{ int n=find_label(p);
+ if (n!=zero_label_no && labels[n].where!=LABEL_UNDEF)
+ { MESSAGE("WARNING: Ignoring duplicate definition of label ");
+ if (label_has_name(p)) MESSAGE("name %s\n",tokens_to_name(link(label_ptr(p))));
+ else MESSAGE("num %d\n",label_ptr(p));
+ }
+ else
+ { labels[n].where=label_where(p);
+ labels[n].pos=hpos-hstart;
+ labels[n].pos0=hpos0-hstart;
+ labels[n].next=first_label;
+ first_label=n;
+ DBG(DBGLABEL,"Defining label *%d: pos=0x%x\n",n,labels[n].pos);
+ }
+}
+@
+
+When a link node is written to the output, we can check
+that start links and end links properly match.
+
+@<Hi\TeX\ auxiliar routines@>=
+static int last_link=-1;
+static int new_start_link(pointer p)
+{ int n=find_label(p);
+ if (last_link>=0)
+ fatal_error("Missing end link before start link");
+ labels[n].used=true;
+ last_link=n;
+ DBG(DBGLABEL,"New link to label *%d\n",n);
+ return n;
+}
+
+static int new_end_link(void)
+{ int n;
+ if (last_link<0)
+ fatal_error("Missing start link before end link");
+ n=last_link;
+ last_link=-1;
+ return n;
+}
+@
+
+For outline nodes, we use the next two functions.
+The node list representing the title can be an arbitrary
+list in horizontal mode.
+In general, the front end should be able to render such a
+horizontal list, but at least it should be able to extract
+the UTF8 characters codes and display those.
+
+@<Hi\TeX\ auxiliar routines@>=
+static void new_outline(pointer p)
+{ int r=find_label(p);
+ int m=next_outline();
+ list_t l;
+ uint32_t pos;
+ pos=hpos-hstart;
+ l.k=list_kind; /* this eventually should be |text_kind| */
+ hout_list_node(outline_ptr(p),pos,&l);
+ hset_outline(m,r,outline_depth(p),pos);
+ DBG(DBGLABEL,"New outline for label *%d\n",r);
+}
+@
+
+One last function is needed, which better schould be in the {\tt htex.w}
+file. It is called when the |outline_group| ends that was started
+when scanning the {\tt\BS HINToutline} primitive.
+@<Hi\TeX\ routines@>=
+void hfinish_outline_group(void)
+{ pointer s=link(head);
+ unsave();
+ pop_nest();
+ outline_ptr(tail)=s;
+}
+@
+
+\subsection{The New Page Builder}\label{buildpage}
+Here is the new |build_page| routine of Hi\TeX:
+
+@<Hi\TeX\ routines@>=
+void build_page(void)
+{ static bool initial=true;
+ if(link(contrib_head)==null||output_active)return;
+ do
+ { pointer p= link(contrib_head);
+ pointer q=null; /* for output nodes */
+ pointer *t; /*the tail of the output nodes*/
+ @<Record the bottom mark@>@;
+ @<Suppress empty pages if requested@>@;
+ link(contrib_head)= link(p);link(p)= null;
+ if (link(contrib_head)==null)
+ { if(nest_ptr==0) tail= contrib_head;
+ else contrib_tail= contrib_head;
+ }
+ update_last_values(p);
+ @<Freeze the page specs if called for@>@;
+ page_goal=0x3fffffff; /* maximum dimension */
+ t=collect_output(&p,&q);
+ if (p!=null)
+ { hpos0=hpos; hout_node(p); }
+recycle_p:
+ flush_node_list(p);
+ if (q!=null)
+ @<Fire up the output routine for |q|@>@;
+ } while(link(contrib_head)!=null);
+ DBG(DBGBUFFER,"after build page dyn_used= %d\n", dyn_used);
+}
+@
+
+@<Freeze the page specs if called for@>=
+if (page_contents<box_there)
+{ switch(type(p))
+ { case whatsit_node:
+ if (subtype(p)==baseline_node) goto recycle_p;
+ else if (subtype(p)!=hset_node && subtype(p)!=vset_node &&
+ subtype(p)!=hpack_node && subtype(p)!=vpack_node &&
+ subtype(p)!=graf_node && subtype(p)!=disp_node &&
+ subtype(p)!=image_node && subtype(p)!=align_node)
+ break; /* else fall through */
+ case hlist_node: case vlist_node: case rule_node:
+ if (page_contents==empty)
+ { freeze_page_specs(box_there);
+ hfix_defaults();
+ }
+ else page_contents=box_there;
+ break;
+ case ins_node:
+ if (page_contents==empty)
+ { freeze_page_specs(inserts_only);
+ hfix_defaults();
+ }
+ break;
+ case kern_node:
+ case penalty_node:
+ case glue_node: goto recycle_p;
+ default:
+ break;
+ }
+}
+@
+
+Users of \TeX\ often force the generation of empty pages for example to start
+a new chapter on a right hand page with an odd page number.
+This makes sense for a printed book but not for a screen reader where
+there are no page numbers nor right or left hand pages.
+Using a screen reader, empty pages are just annoying.
+The common way to achive an empty page is the use of {\tt \BS eject}
+followed by a an empty box, a fill glue, and another {\tt \BS eject}.
+
+The following code tries to detect such a sequence of nodes and will eliminate
+them if requested. To do so, we delay the output of nodes after
+an eject penalty until either something gets printed on the page or
+another eject penalty comes along. To override the delayed output,
+a penalty less or equal to a double |eject_penalty| can be used.
+The function |its_all_over| (see section~\secref{itsallover})
+is an example for such a use.
+
+
+@<Suppress empty pages if requested@>=
+if (option_no_empty_page &&
+ ((type(p)==penalty_node &&
+ penalty(p)<=eject_penalty && penalty(p)>2*(eject_penalty)) ||
+ (page_contents==empty && !is_visible(p))))
+{ pointer q= link(p);
+ while (true)
+ { if (q==null) return;
+ else if (is_visible(q)) break;
+ else if (type(q)==penalty_node && penalty(q)<=eject_penalty)
+ { while (p!=q)
+ { pointer r=p;
+ DBG(DBGPAGE,"Eliminating node (%d,%d)\n",
+ type(p), type(p)==penalty_node?penalty(p):subtype(p));
+ p=link(p);
+ link(r)=null;
+ flush_node_list(r);
+ }
+ link(contrib_head)= p;
+ DBG(DBGPAGE,"Eliminating empty page done\n");
+ if (penalty(q)<=2*(eject_penalty)) break;
+ }
+ q=link(q);
+ }
+}
+@
+It remains to test a node for visibility. This is a quick (and dirty) test
+because the test will not look inside boxes; it simply tests whether
+the list pointer is |null|. We consider an |open_node|, |write_node|,
+|close_node|, |label_node|, or |outline_node| a as visible,
+because deleting them could cause unwanted
+side effects. Possibly it would be better to regard them as invisible,
+but still pass them on to the rest of the output routine.
+
+@<Hi\TeX\ auxiliar routines@>=
+static bool is_visible(pointer p)
+{ switch (type(p))
+ { case penalty_node:
+ case kern_node:
+ case glue_node:
+ case mark_node:
+ return false;
+ case ins_node:
+ return ins_ptr(p)!=null;
+ case adjust_node:
+ return adjust_ptr(p)!=null;
+ case hlist_node:
+ case vlist_node:
+ return list_ptr(p)!=null;
+ case whatsit_node:
+ if (subtype(p)==image_node || subtype(p)==align_node || subtype(p)==disp_node ||
+ subtype(p)==open_node ||subtype(p)==write_node ||subtype(p)==close_node ||
+ subtype(p)==label_node || subtype(p)==outline_node )
+ return true;
+ else if (subtype(p)==hset_node || subtype(p)==vset_node ||
+ subtype(p)==hpack_node || subtype(p)==vpack_node)
+ return list_ptr(p)!=null;
+ else if (subtype(p)==graf_node)
+ return graf_list(p)!=null;
+ else
+ return false;
+ default: return true;
+ }
+}
+@
+
+An important feature of the new routine is the call to
+|hfix_defaults|. It occurs when the first ``visible mark'' is placed
+on the page. At that point we record the current values of \TeX's
+parameters which we will use to generate the definition section of the
+\HINT\ file. It is still possible to specify alternative values for
+these parameters by using parameter lists but only at an additional
+cost in space and time.
+
+Furthermore, this is the point where we freeze the definition of
+|hsize| and |vsize|. The current values will be regarded as the sizes
+as recommended by the author.
+
+From then on |hsize| and |vsize| are replaced by the equivalent
+extended dimensions and any attempt to modify them on the global level
+will be ignored. |hhsize| and |hvsize| will contain the sizes that a
+regular \TeX\ engine would use.
+
+We also compute the total page size from the page template defined
+last.
+
+
+@<Compute the page size@>=
+{ pointer p;
+ p=link(setpage_head);
+ if (p==null)
+ { scaled margin;
+ if (hhsize<hvsize) margin=hhsize; else margin=hvsize;
+ margin = margin/6 -6*unity;
+ if (margin<0) margin=0;
+ page_h=hhsize+2*margin;
+ page_v=hvsize+2*margin;
+ }
+ else
+ { pointer x;
+ x=setpage_height(p);
+ page_v=xdimen_width(x)
+ +round(((double)xdimen_hfactor(x)*hhsize+(double)xdimen_vfactor(x)*hvsize)/unity);
+ x=setpage_width(p);
+ page_h=xdimen_width(x)
+ +round(((double)xdimen_hfactor(x)*hhsize+(double)xdimen_vfactor(x)*hvsize)/unity);
+ }
+}
+@
+
+@<Hi\TeX\ variables@>=
+static scaled page_h, page_v;
+@
+
+@<Switch |hsize| and |vsize| to extended dimensions@>=
+ hsize=0; vsize=0;
+ dimen_par_hfactor(hsize_code)= unity;
+ dimen_par_vfactor(vsize_code)= unity;
+@
+
+There is one point where we can not simpy forego the
+output routine: \.{\\write} commands. Unless the \.{\\write} is
+decorated with an \.{\\immediate}, the whatsit node generated from it
+will lay dormant in the contribution list and then the page until
+the output routine passes it inside the finished page to the |ship_out|
+routine where it will come to life and write its token list out.
+The whatsit nodes from \.{\\openout} and \.{\\closeout} commands
+behave similary.
+
+It is not possible to ignore the output routine
+because the output routine may change the environment in which the
+token list of a \.{\\write} will be expanded.
+For example \LaTeX\ redefines \.{\\protect} to be \.{\\noexpand}.
+As a consequence we have to implement a simplified version
+of \TeX's usual process to fire up the output routine.
+
+The |collect_output| routine takes a node list |p|,
+removes the output nodes and appends them to |q|, with |q|
+always pointing to the tail pointer.
+
+@<Hi\TeX\ auxiliar routines@>=
+static pointer *collect_output(pointer *p, pointer *q)
+{ while (*p!=null)
+ { @<Collect output nodes from |*p|@>@;
+ p=&(link(*p));
+ }
+ return q;
+}
+@
+
+\TeX\ does not permit output nodes in leaders, so we dont check them;
+further we do not check the pre- and post-break lists of
+discretionary breaks.
+
+@<Collect output nodes from |*p|@>=
+if (!is_char_node(*p))
+{ pointer r=*p;
+ switch (type(r))
+ { case whatsit_node:
+ switch (subtype(r))
+ { case open_node: case write_node: case close_node:
+ { *p=link(r); link(r)=null; *q=r; q=&(link(r));
+ if (*p==null) return q;
+ }
+ break;
+ case graf_node: q=collect_output(&graf_list(r),q);
+ break;
+ case disp_node:
+ if (display_left(r)) q=collect_output(&display_eqno(r),q);
+ q=collect_output(&display_formula(r),q);
+ if (!display_left(r)) q=collect_output(&display_eqno(r),q);
+ break;
+ case hset_node: case vset_node: case hpack_node: case vpack_node:
+ q=collect_output(&list_ptr(r),q);
+ break;
+ case align_node:
+ q=collect_output(&align_list(r),q);
+ break;
+ default: break;
+ }
+ break;
+ case hlist_node: case vlist_node:
+ q=collect_output(&list_ptr(r),q);
+ break;
+ case ins_node:
+ q=collect_output(&ins_ptr(r),q);
+ break;
+ case adjust_node:
+ q=collect_output(&adjust_ptr(r),q);
+ break;
+ default: break;
+ }
+}
+@
+
+
+
+@<Fire up the output routine for |q|@>=
+{ pointer r=new_null_box();type(r)=vlist_node;
+ subtype(r)=0;shift_amount(r)=0;height(r)=hvsize;
+ list_ptr(r)=q;
+ *t=new_glue(ss_glue);
+ flush_node_list(box(255)); /* just in case \dots */
+ box(255)=r;
+ if (output_routine!=null)
+ {@+output_active=true;
+ if (bot_mark!=null)
+ {@+if (top_mark!=null) delete_token_ref(top_mark);
+ top_mark=bot_mark;add_token_ref(top_mark);
+ if (first_mark!=null) delete_token_ref(first_mark);
+ first_mark=bot_mark;add_token_ref(first_mark);
+ }
+ push_nest();mode=-vmode;prev_depth=ignore_depth;mode_line=-line;
+ begin_token_list(output_routine, output_text);
+ new_save_level(output_group);normal_paragraph();
+ scan_left_brace();
+ return;
+ }
+ else
+ {
+ ship_out(box(255)); box(255)=null;
+ }
+}
+@
+
+The |ship_out| routine just calls |execute_output|.
+Because the output routine might have added plenty
+of decorations around the list of output nodes,
+we have to find them again.
+
+@<Hi\TeX\ routines@>=
+void execute_output(pointer p)
+{@+while (p!=null)
+ { @<Execute output nodes from |p|@>@;
+ p=link(p);
+ }
+}
+@
+
+@<Execute output nodes from |p|@>=
+if (!is_char_node(p))
+ switch (type(p))
+ { case whatsit_node:
+ switch (subtype(p))
+ { case open_node: case write_node: case close_node:
+ out_what(p);
+ break;
+ case graf_node: execute_output(graf_list(p));
+ break;
+ case disp_node:
+ if (display_left(p)) execute_output(display_eqno(p));
+ execute_output(display_formula(p));
+ if (!display_left(p)) execute_output(display_eqno(p));
+ break;
+ case hset_node: case vset_node: case hpack_node: case vpack_node:
+ execute_output(list_ptr(p));
+ break;
+ case align_node:
+ execute_output(align_list(p));
+ break;
+ default: break;
+ }
+ break;
+ case hlist_node: case vlist_node:
+ execute_output(list_ptr(p));
+ break;
+ case ins_node:
+ execute_output(ins_ptr(p));
+ break;
+ case adjust_node:
+ execute_output(adjust_ptr(p));
+ break;
+ default: break;
+ }
+@
+
+Invoking the user's output routine is a risky endeavor
+if marks are not initialized properly. In our case
+we will have always |top_mark| equal to |first_mark| and
+|bot_mark|.
+
+@<Record the bottom mark@>=
+if (type(p)==mark_node)
+{ if (bot_mark!=null) delete_token_ref(bot_mark);
+ bot_mark=mark_ptr(p);add_token_ref(bot_mark);
+}
+@
+
+
+\subsection{Replacing {\tt hpack} and {\tt vpack}}
+
+
+@<Hi\TeX\ routines@>=
+
+pointer hpack(pointer p,scaled w, scaled hf, scaled vf, small_number m)
+{
+ pointer r; /*the box node that will be returned*/
+ pointer prev_p; /*trails behind |p|*/
+ scaled h,d,x; /*height, depth, and natural width*/
+ scaled s; /*shift amount*/
+ pointer g; /*points to a glue specification*/
+ glue_ord sto, sho; /*order of infinity*/
+ internal_font_number f; /*the font in a |char_node|*/
+ four_quarters i; /*font information about a |char_node|*/
+ eight_bits hd; /*height and depth indices for a character*/
+ last_badness= 0;r= get_node(box_node_size);type(r)= hlist_node;
+ subtype(r)= min_quarterword;shift_amount(r)= 0;
+ prev_p= r+list_offset;link(prev_p)= p;
+ h= 0; d= 0; x= 0;
+ total_stretch[normal]= 0;total_shrink[normal]= 0;
+ total_stretch[fil]= 0;total_shrink[fil]= 0;
+ total_stretch[fill]= 0;total_shrink[fill]= 0;
+ total_stretch[filll]= 0;total_shrink[filll]= 0;
+ while(p!=null)
+ {
+ reswitch:
+ while(is_char_node(p))
+ { f= font(p);i= char_info(f, character(p));hd= height_depth(i);
+ x= x+char_width(f, i);
+ s= char_height(f, hd);if (s> h)h= s;
+ s= char_depth(f, hd);if (s> d)d= s;
+ p= link(p);
+ }
+ if (p!=null)
+ { switch(type(p)){
+ case hlist_node:case vlist_node:case rule_node:case unset_node:
+ { x= x+width(p);
+ if (type(p)>=rule_node)s= 0;else s= shift_amount(p);
+ if (height(p)-s> h)h= height(p)-s;
+ if (depth(p)+s> d)d= depth(p)+s;
+ }
+ break;
+ case ins_node:case mark_node:case adjust_node:if (adjust_tail!=null)
+ { while(link(prev_p)!=p)prev_p= link(prev_p);
+ if (type(p)==adjust_node)
+ { link(adjust_tail)= adjust_ptr(p);
+ while(link(adjust_tail)!=null)adjust_tail= link(adjust_tail);
+ p= link(p);free_node(link(prev_p),small_node_size);
+ }
+ else
+ { link(adjust_tail)= p;adjust_tail= p;p= link(p);
+ }
+ link(prev_p)= p;p= prev_p;
+ }
+ break;
+ case whatsit_node:
+ if (subtype(p)==graf_node)
+ goto repack;
+ else if (subtype(p)==disp_node )
+ goto repack;
+ else if (subtype(p)==vpack_node )
+ goto repack;
+ else if (subtype(p)==hpack_node )
+ goto repack;
+ else if (subtype(p)==hset_node )
+ goto repack;
+ else if (subtype(p)==vset_node )
+ goto repack;
+ else if (subtype(p)==stream_node )
+ goto repack;
+ else if (subtype(p)==image_node)
+ { glue_ord o;
+ if (image_height(p)> h) h= image_height(p);
+ x= x+image_width(p);
+ o= image_stretch_order(p);total_stretch[o]= total_stretch[o]+image_stretch(p);
+ o= image_shrink_order(p);total_shrink[o]= total_shrink[o]+image_shrink(p);
+ }
+ break;
+ break;
+ case glue_node:
+ { glue_ord o;
+ g= glue_ptr(p);x= x+width(g);
+ o= stretch_order(g);total_stretch[o]= total_stretch[o]+stretch(g);
+ o= shrink_order(g);total_shrink[o]= total_shrink[o]+shrink(g);
+ if (subtype(p)>=a_leaders)
+ { g= leader_ptr(p);
+ if (height(g)> h)h= height(g);
+ if (depth(g)> d)d= depth(g);
+ }
+ }
+ break;
+ case kern_node:case math_node:x= x+width(p);break;
+ case ligature_node:
+ { mem[lig_trick]= mem[lig_char(p)];link(lig_trick)= link(p);
+ p= lig_trick;goto reswitch;
+ }
+ default:do_nothing;
+ }
+ p= link(p);
+ }
+ }
+ if (adjust_tail!=null) link(adjust_tail)= null;
+ height(r)= h;depth(r)= d;
+ if (total_stretch[filll]!=0)sto= filll;
+ else if (total_stretch[fill]!=0)sto= fill;
+ else if (total_stretch[fil]!=0)sto= fil;
+ else sto= normal;
+
+ if (total_shrink[filll]!=0)sho= filll;
+ else if (total_shrink[fill]!=0)sho= fill;
+ else if (total_shrink[fil]!=0)sho= fil;
+ else sho= normal;
+
+ if (hf!=0 || vf!=0 ) /* convert to a hset node */
+ { pointer q;
+ q=new_set_node();
+ subtype(q)=hset_node;
+ height(q)=h;
+ depth(q)=d;
+ width(q)=x; /* the natural width */
+ shift_amount(q)=shift_amount(r);
+ list_ptr(q)=list_ptr(r);
+ list_ptr(r)=null;
+ free_node(r, box_node_size);
+ if (m==exactly)
+ set_extent(q)=new_xdimen(w,hf,vf);
+ else
+ set_extent(q)=new_xdimen(x+w,hf,vf);
+ set_stretch_order(q)=sto;
+ set_shrink_order(q)=sho;
+ set_stretch(q)=total_stretch[sto];
+ set_shrink(q)=total_shrink[sho];
+ return q;
+ }
+
+
+
+
+ if (m==additional) w= x+w;
+ width(r)= w;x= w-x; /*now |x| is the excess to be made up*/
+
+ if (x==0)
+ { glue_sign(r)= normal; glue_order(r)= normal;
+ set_glue_ratio_zero(glue_set(r));
+ goto end;
+ }
+ else if (x> 0)
+ {
+ glue_order(r)= sto;glue_sign(r)= stretching;
+ if (total_stretch[sto]!=0)glue_set(r)= unfloat(x/(double)total_stretch[sto]);
+ else
+ { glue_sign(r)= normal;
+ set_glue_ratio_zero(glue_set(r));
+ }
+ if (sto==normal)
+ { if (list_ptr(r)!=null)
+ { last_badness= badness(x,total_stretch[normal]);
+ if (last_badness> hbadness)
+ { print_ln();
+ if (last_badness> 100)
+ print_nl("Underfull");else print_nl("Loose");
+ print(" \\hbox (badness ");print_int(last_badness);
+ goto common_ending;
+ }
+ }
+ }
+ goto end;
+ }
+ else
+ {
+ glue_order(r)= sho;glue_sign(r)= shrinking;
+ if (total_shrink[sho]!=0)
+ glue_set(r)= unfloat((-x)/(double)total_shrink[sho]);
+ else
+ { glue_sign(r)= normal;
+ set_glue_ratio_zero(glue_set(r));
+ }
+ if ((total_shrink[sho]<-x)&&(sho==normal)&&(list_ptr(r)!=null))
+ { last_badness= 1000000;
+ set_glue_ratio_one(glue_set(r));
+ if ((-x-total_shrink[normal]> hfuzz)||(hbadness<100))
+ { if ((overfull_rule> 0)&&(-x-total_shrink[normal]> hfuzz))
+ { while(link(prev_p)!=null)prev_p= link(prev_p);
+ link(prev_p)= new_rule();
+ width(link(prev_p))= overfull_rule;
+ }
+ print_ln();print_nl("Overfull \\hbox (");
+ print_scaled(-x-total_shrink[normal]);print("pt too wide");
+ goto common_ending;
+ }
+ }
+ else if (sho==normal)
+ { if (list_ptr(r)!=null)
+ { last_badness= badness(-x,total_shrink[normal]);
+ if (last_badness> hbadness)
+ { print_ln();print_nl("Tight \\hbox (badness ");print_int(last_badness);
+ goto common_ending;
+ }
+ }
+ }
+ goto end;
+ }
+ common_ending:
+ if (pack_begin_line!=0)
+ { if (pack_begin_line> 0)print(") in paragraph at lines ");
+ else print(") in alignment at lines ");
+ print_int(abs(pack_begin_line));
+ print("--");
+ }
+ else print(") detected at line ");
+ print_int(line);
+ print_ln();
+ font_in_short_display= null_font;short_display(list_ptr(r));print_ln();
+ begin_diagnostic();show_box(r);end_diagnostic(true);
+ end:return r;
+
+
+repack:
+ { /* convert the box to a |hpack_node| */
+ pointer q;
+ q=new_pack_node();
+ height(q)=h;
+ depth(q)=d;
+ width(q)=x;
+ subtype(q)=hpack_node;
+ list_ptr(q)=list_ptr(r);
+ list_ptr(r)=null;
+ free_node(r, box_node_size);
+ pack_limit(q)=max_dimen; /* no limit, not used */
+ pack_m(q)=m;
+ pack_extent(q)=new_xdimen(w,hf,vf);
+ return q;
+ }
+}
+
+@
+
+@<Hi\TeX\ routines@>=
+pointer vpackage(pointer p, scaled h, scaled hf, scaled vf, small_number m, scaled l)
+{ pointer r; /*the box node that will be returned*/
+ scaled w,d,x; /*width, depth, and natural height*/
+ scaled s=0; /*shift amount*/
+ pointer g; /*points to a glue specification*/
+ glue_ord sho, sto; /*order of infinity*/
+ last_badness= 0; r= get_node(box_node_size); type(r)= vlist_node;
+ subtype(r)= min_quarterword; shift_amount(r)= 0;
+ list_ptr(r)= p;
+ w= 0;
+ d= 0;x= 0;
+ total_stretch[normal]= 0;total_shrink[normal]= 0;
+ total_stretch[fil]= 0;total_shrink[fil]= 0;
+ total_stretch[fill]= 0;total_shrink[fill]= 0;
+ total_stretch[filll]= 0;total_shrink[filll]= 0;
+ while(p!=null)
+ { if (is_char_node(p))
+ confusion("vpack");
+ else
+ switch(type(p))
+ { case hlist_node:case vlist_node:case rule_node:case unset_node:
+ x= x+d+height(p);d= depth(p);
+ if (type(p)>=rule_node) s= 0;
+ else s= shift_amount(p);
+ if (width(p)+s> w) w= width(p)+s;
+ break;
+ case whatsit_node:
+ if (subtype(p)==graf_node)
+ goto repack;
+ else if (subtype(p)==disp_node )
+ goto repack;
+ else if (subtype(p)==vpack_node )
+ goto repack;
+ else if (subtype(p)==hpack_node )
+ goto repack;
+ else if (subtype(p)==hset_node )
+ goto repack;
+ else if (subtype(p)==vset_node )
+ goto repack;
+ else if (subtype(p)==stream_node )
+ goto repack;
+ else if (subtype(p)==image_node)
+ { glue_ord o;
+ if (image_width(p)> w) w= image_width(p);
+ x= x+d+image_height(p);d=0;
+ o= image_stretch_order(p);total_stretch[o]= total_stretch[o]+image_stretch(p);
+ o= image_shrink_order(p);total_shrink[o]= total_shrink[o]+image_shrink(p);
+ }
+ break;
+ case glue_node:
+ { glue_ord o;
+ x= x+d;d= 0;
+ g= glue_ptr(p);x= x+width(g);
+ o= stretch_order(g); total_stretch[o]= total_stretch[o]+stretch(g);
+ o= shrink_order(g); total_shrink[o]= total_shrink[o]+shrink(g);
+ if (subtype(p)>=a_leaders)
+ { g= leader_ptr(p);
+ if (width(g)> w) w= width(g);
+ }
+ }
+ break;
+ case kern_node:
+ x= x+d+width(p);d= 0;
+ break;
+ default:do_nothing;
+ }
+ p= link(p);
+ }
+ width(r)= w;
+
+
+ if (total_stretch[filll]!=0) sto= filll;
+ else if (total_stretch[fill]!=0) sto= fill;
+ else if (total_stretch[fil]!=0) sto= fil;
+ else sto= normal;
+
+ if (total_shrink[filll]!=0) sho= filll;
+ else if (total_shrink[fill]!=0) sho= fill;
+ else if (total_shrink[fil]!=0) sho= fil;
+ else sho= normal;
+
+ if (hf!=0 || vf!=0) /* convert to a vset node */
+ { pointer q;
+ q=new_set_node();
+ subtype(q)=vset_node;
+ width(q)=w;
+ if (d> l)
+ { x= x+d-l;depth(r)= l;
+ }
+ else depth(r)= d;
+ height(q)=x;
+ depth(q)=d;
+ shift_amount(q)=shift_amount(r);
+ list_ptr(q)=list_ptr(r);
+ list_ptr(r)=null;
+ free_node(r, box_node_size);
+ if (m==exactly)
+ set_extent(q)=new_xdimen(h,hf,vf);
+ else
+ set_extent(q)=new_xdimen(x+h,hf,vf);
+ set_stretch_order(q)=sto;
+ set_shrink_order(q)=sho;
+ set_stretch(q)=total_stretch[sto];
+ set_shrink(q)=total_shrink[sho];
+ return q;
+ }
+
+ if (d> l)
+ { x= x+d-l;depth(r)= l;
+ }
+ else depth(r)= d;
+ if (m==additional)
+ h= x+h;
+ height(r)= h; x= h-x; /*now |x| is the excess to be made up*/
+if (x==0)
+ { glue_sign(r)= normal; glue_order(r)= normal;
+ set_glue_ratio_zero(glue_set(r));
+ goto end;
+ }
+ else if (x> 0)
+ { glue_order(r)= sto;glue_sign(r)= stretching;
+ if (total_stretch[sto]!=0)glue_set(r)= unfloat(x/(double)total_stretch[sto]);
+ else
+ { glue_sign(r)= normal;
+ set_glue_ratio_zero(glue_set(r));
+ }
+ if (sto==normal)
+ { if (list_ptr(r)!=null)
+ { last_badness= badness(x,total_stretch[normal]);
+ if (last_badness> vbadness)
+ { print_ln();
+ if (last_badness> 100)print_nl("Underfull");else print_nl("Loose");
+ print(" \\vbox (badness ");print_int(last_badness);
+ goto common_ending;
+ }
+ }
+ }
+ goto end;
+ }
+ else /* if (x<0) */
+ {
+ glue_order(r)= sho;glue_sign(r)= shrinking;
+ if (total_shrink[sho]!=0)glue_set(r)= unfloat((-x)/(double)total_shrink[sho]);
+ else
+ { glue_sign(r)= normal;
+ set_glue_ratio_zero(glue_set(r));
+ }
+ if ((total_shrink[sho]<-x)&&(sho==normal)&&(list_ptr(r)!=null))
+ { last_badness= 1000000;
+ set_glue_ratio_one(glue_set(r));
+ if ((-x-total_shrink[normal]> vfuzz)||(vbadness<100))
+ { print_ln();print_nl("Overfull \\vbox (");
+ print_scaled(-x-total_shrink[normal]);print("pt too high");
+ goto common_ending;
+ }
+ }
+ else if (sho==normal)
+ { if (list_ptr(r)!=null)
+ { last_badness= badness(-x,total_shrink[normal]);
+ if (last_badness> vbadness)
+ { print_ln();print_nl("Tight \\vbox (badness ");print_int(last_badness);
+ goto common_ending;
+ }
+ }
+ }
+ goto end;
+ }
+
+
+ common_ending:
+ if (pack_begin_line!=0)
+ { print(") in alignment at lines ");
+ print_int(abs(pack_begin_line));
+ print("--");
+ }
+ else
+ print(") detected at line ");
+ print_int(line);
+ print_ln();
+
+ begin_diagnostic();show_box(r);end_diagnostic(true);
+ end:
+ return r;
+
+
+repack:
+ { /* convert the box to a |vpack_node| */
+ pointer q;
+ q=new_pack_node();
+ subtype(q)=vpack_node;
+ height(q)=x;
+ depth(q)=d;
+ width(q)=w;
+ list_ptr(q)=list_ptr(r);
+ list_ptr(r)=null;
+ free_node(r, box_node_size);
+ pack_limit(q)=l;
+ pack_m(q)=m;
+ pack_extent(q)=new_xdimen(h,hf,vf);
+ return q;
+ }
+}
+
+@
+
+\subsection{Streams}
+\HINT\ stream numbers start at 0 for the main text and continue
+upwards. \TeX, on the other hand, numbers insertions starting with
+{\tt box255} for the main text and continues downwards. Some mapping is
+needed, and we use the array |insert2stream| to map \TeX's insert
+numbers to \HINT\ stream numbers.
+The predefined stream for the main content has stream number 0.
+
+@<Hi\TeX\ variables@>=
+int insert2stream[0x100]={0};
+@
+
+The following function returns the stream number for a given insert number $i$
+with $255>|i|\ge 0$. A new stream number is allocated if necessary.
+Note that no overflow test is necessary since \TeX\ allocates less
+than 233 inserts.
+The initial value of |max_ref[stream_kind]| is 0 and therefore
+stream number 0, reserved for the main content
+is never allocated. Stream definitions might also be loaded
+as part of a format file. Then the maximum stream number ist stored in |max_stream|.
+So if we do not find a stream number
+in the |insert2stream| array, we scan the stream definitions
+once and cache the associations found there.
+
+@<Hi\TeX\ routines@>=
+int hget_stream_no(int i)
+{ static bool init=false;
+ int s;
+ if (i==0) return 0;
+ s=insert2stream[i];
+ if (s!=0) return s;
+ if (!init)
+ { pointer t,s;
+ for (t=link(setpage_head); t!=null; t=link(t))
+ for(s=setpage_streams(t); s!=null; s=link(s))
+ insert2stream[setstream_insertion(s)]=setstream_number(s);
+ max_ref[stream_kind]=max_stream;
+ init=true;
+ }
+ s=insert2stream[i];
+ if (s==0)
+ s=insert2stream[i]=max_ref[stream_kind]=++max_stream;
+ return s;
+}
+@
+
+\subsection{Stream Definitions}
+
+A stream definition is stored as a whatsit node with subtype |setstream_node| as
+defined in section~\secref{whatsit}.
+Given a pointer |p| to such a node, here are the macros used to access the data stored there:
+\itemize
+\item |setstream_number(p)| the \HINT\ stream number $n$.
+\item |setstream_insertion(p)| the corresponding \TeX\ insertion number $i$.
+\item |setstream_max(p)| the maximum height $x$: This extended dimension is the maximum size
+ per page for this insertion.
+\item |setstream_mag(p)| the magnification factor $f$:
+ Inserting a box of height $h$ will contribute $h*f/1000$
+ to the main page.
+\item |setstream_prefered(p)| the prefered stream $p$:
+ If $p\ge0$ we move the insert to stream $p$ if possible.
+\item |setstream_next(p)| the next stream $n$:
+ If $n\ge0$ we move the insert to stream $n$ if it can not be
+ accomodated otherwise.
+\item |setstream_ratio(p)| the split ratio $r$:
+ If $r>0$ split the final contribution of this streams between
+ stream $p$ and $n$ in the ratio $r/1000$ for $p$ and $1-r/1000$ for $n$
+ before contributing streams $p$ and $r$ to the page.
+\item |setstream_before(p)| the ``before'' list $b$:
+ For a nonempty stream the material that is added before the stream content.
+\item |setstream_after(p)| the ``after'' list $a$:
+ For a nonempty stream, material that is added after the stream conten.
+\item |setstream_topskip(p)| the top skip glue $t$: This glue is inserted between
+ the $b$ list and the stream content and ajusted for the height for the first box
+ of the stream content.
+\item |setstream_width(p)| the width $w$:
+ This extended dimension is the width used for example
+ to break paragraphs in the stream content into lines.
+\item |setstream_height(p)| a glue specification $h$ reflecting the total height,
+ strechability and shrinkability of the material in lists $a$ and $b$.
+\enditemize
+
+Currently Hi\TeX\ handles only normal streams. First or last streams will come later.
+
+The stream definition nodes are created and initialized with the following function:
+@<Hi\TeX\ routines@>=
+pointer new_setstream_node(uint8_t n)
+{ pointer p=get_node(setstream_node_size);
+ type(p)=whatsit_node;subtype(p)=setstream_node;
+ setstream_insertion(p)=n;
+ setstream_number(p)=hget_stream_no(n);
+ setstream_mag(p)=1000;
+ setstream_prefered(p)=255;
+ setstream_next(p)=255;
+ setstream_ratio(p)=0;
+ setstream_max(p)=new_xdimen(0,0,ONE);
+ setstream_width(p)=new_xdimen(0,ONE,0);
+ setstream_topskip(p)=zero_glue; add_glue_ref(zero_glue);
+ setstream_height(p)=zero_glue; add_glue_ref(zero_glue);
+ setstream_before(p)=null;
+ setstream_after(p)=null;
+
+ return p;
+}
+@
+
+
+The prefered stream, the next stream, and the split ratio are scanned as part of the
+{\tt \BS setstream} primitive as described in section~\secref{primitive}.
+When \TeX\ finds the right brace that terminates the stream definition,
+it calls |handle_right_brace|. Then it is time to obtain the remaining parts of the
+stream definition.
+For insertion class $i$,
+we can extract the maximum height $x$ of the insertions from
+the corresponding {\tt dimen$i$} register
+the magnification factor $f$ from the {\tt count$i$} register,
+and the total height $h$ from the {\tt skip$i$} register.
+The width $w$ is taken from {\tt \BS hsize} and the
+topskip $t$ from {\tt \BS topskip}.
+
+@<Hi\TeX\ routines@>=
+void hfinish_stream_group(void)
+{ pointer s;
+ end_graf();
+ s=hget_current_stream();
+ if (s!=null)
+ { pointer t;
+ uint8_t i;
+ i= setstream_insertion(s);
+ setstream_mag(s)=count(i);
+ setstream_width(s)=new_xdimen(dimen_par(hsize_code),
+ dimen_par_hfactor(hsize_code),dimen_par_vfactor(hsize_code));
+ t=zero_glue;add_glue_ref(t);delete_glue_ref(setstream_topskip(s)); setstream_topskip(s)=t;
+ t=skip(i);add_glue_ref(t);delete_glue_ref(setstream_height(s)); setstream_height(s)=t;
+ setstream_max(s)=new_xdimen(dimen(i),
+ dimen_hfactor(i),dimen_vfactor(i));
+ }
+ unsave();
+ flush_node_list(link(head));
+ pop_nest();
+}
+@
+
+The before list $b$ and the after list $a$ are defined using the
+{\tt \BS before} and {\tt \BS after} primitives. When the corresponding list
+has ended with a right brace, \TeX\ calls |handle_right_brace| and we can store
+the lists.
+
+@<Hi\TeX\ routines@>=
+void hfinish_stream_before_group(void)
+{ pointer s;
+ end_graf();
+ s=hget_current_stream();
+ if (s!=null)
+ setstream_before(s)=link(head);
+ unsave();
+ pop_nest();
+}
+
+void hfinish_stream_after_group(void)
+{ pointer s;
+ end_graf();
+ s=hget_current_stream();
+ if (s!=null)
+ setstream_after(s)=link(head);
+ unsave();
+ pop_nest();
+}
+@
+
+\subsection{Page Template Definitions}
+
+The data describing a page template is stored in a whatsit node with subtype
+|setpage_node| as defined insection~\secref{whatsit}.
+Given a pointer |p| to such a node, here are the macros used to access the data stored there:
+\itemize
+\item |setpage_name(p)|: The name of the page template
+ can be used in the user interface of a \HINT\ viewer.
+\item |setpage_number(p)|: The number of the page template that is used in the \HINT\
+ file to reference this page template.
+\item |setpage_id(p)|: The number of the page template that is used in \TeX\
+ to reference this page template.
+\item |setpage_priority(p)|: The priority helps in selecting a page template.
+\item |setpage_topskip(p)|: The topskip glue is added at the top of a page and
+ adjusted by the height of the first bos on the page.
+\item |setpage_height(p)|: The height of the full page including the margins.
+\item |setpage_width(p)|: The width of the full page including the margins.
+\item |setpage_depth(p)|: The maximum depth of the page content. If the last box is deeper
+than this maximum, the difference is subtracted from the height of the page body.
+\item |setpage_list(p)|: The list that defines the page template. After the page builder
+has completed a page this list is scanned and page body and nonempty streams
+are added at the corresponding insertion points.
+\item |setpage_streams(p)|: The list of stream definitions that
+belong to this page template.
+\enditemize
+
+To allow \TeX\ to use arbitrary numbers between 1 and 255 for the page templates
+while in \HINT\ the numbers of page templates are best consecutive
+from 1 to |max_ref[page_kind]==max_page|, we let \TeX\ assign an id and generate
+the template number. Because templates might be in format files, the variable
+|max_page| will hold the true number.
+
+The function |new_setpage_node| is called with the page template id
+$0<|i|<256$ and a string number for the name |n|.
+It allocates and initializes a node if necessary and moves it to the front of the
+list of templates.
+
+@<Hi\TeX\ routines@>=
+pointer new_setpage_node(uint8_t i, str_number n)
+{ pointer p, prev_p;
+ prev_p=setpage_head;
+ for (p=link(prev_p); p!=null; prev_p=p,p=link(p))
+ if (setpage_id(p)==i) break;
+ if (p==null)
+ @<allocate a new |setpage_node| |p|@>@;
+ else
+ link(prev_p)=link(p);
+ link(p)=link(setpage_head);
+ link(setpage_head)=p;
+ return p;
+}
+@
+
+@<allocate a new |setpage_node| |p|@>=
+{ p=get_node(setpage_node_size);type(p)=whatsit_node;subtype(p)=setpage_node;
+ setpage_number(p)=max_ref[page_kind]=++max_page;
+ setpage_id(p)=i;
+ setpage_name(p)=n;
+ setpage_priority(p)=1;
+ setpage_topskip(p)=zero_glue; add_glue_ref(zero_glue);
+ setpage_height(p)=new_xdimen(0,0,ONE);
+ setpage_width(p)=new_xdimen(0,ONE,0);
+ setpage_depth(p)=max_depth;
+ setpage_list(p)=null;
+ setpage_streams(p)=null;
+}
+@
+
+The default values are replaced by parameters given to the {\tt\BS setpage}
+primitive (see section~\secref{pageext}) and by the current values of certain
+\TeX\ registers when finishing the page template.
+page template itself,
+
+@<Hi\TeX\ routines@>=
+void hfinish_page_group(void)
+{ uint8_t k;
+ pointer p,q,r;
+ end_graf();
+ p=hget_current_page();
+ if (p!=null)
+ { delete_glue_ref(setpage_topskip(p));
+ setpage_topskip(p)=top_skip;add_glue_ref(top_skip);
+ setpage_depth(p)=max_depth;
+ flush_node_list(setpage_list(p));
+ setpage_list(p)=link(head);
+ }
+ unsave();
+ pop_nest();
+}
+@
+@<Hi\TeX\ auxiliar routines@>=
+
+static pointer hget_current_page(void)
+{ pointer p=link(setpage_head);
+ if (p==null)
+ print_err("end of output group without setpage node");
+ return p;
+}
+
+static pointer hget_current_stream(void)
+{ pointer p,s;
+ p=hget_current_page();
+ if (p==null) return null;
+ s=setpage_streams(p);
+ if (s==null)
+ print_err("end of setstream group without setstream node");
+ return s;
+}
+@
+
+
+
+\section{\HINT\ Output}
+
+\subsection{Initialization}
+\noindent
+
+@<Hi\TeX\ routines@>=
+static void hout_init(void)
+{
+ new_directory(dir_entries);
+ new_output_buffers();
+ max_section_no=2;
+ hdef_init();
+ hput_content_start();
+ @<insert an initial language node@>@;
+}
+
+extern int option_global;
+void hint_open(void)
+{ if (job_name==0) open_log_file();
+ pack_job_name(".hnt");
+ while (!(hout=open_out((char *)name_of_file+1,"wb")))
+ prompt_file_name("file name for output",".hnt");
+ hlog=stderr;
+ option_global=true;
+ hout_init();
+}
+@
+
+\subsection{Termination}
+
+@<Hi\TeX\ routines@>=
+#define HITEX_VERSION "1.1"
+static void hput_definitions();
+static void hout_terminate(void)
+{ hput_content_end();
+ hput_definitions();
+ hput_directory();
+ hput_hint("created by HiTeX Version " HITEX_VERSION);
+}
+
+void hint_close(void)
+{ hout_terminate();
+ if (hout!=NULL)
+ fclose(hout);
+ hout=NULL;
+}
+@
+
+
+\subsection{\HINT\ Directory}
+There is not much to do here: some code to find a new or existing directory entry,
+a variable to hold the number of directory entries allocated,
+a function to allocate a new file section, and an auxiliar function to
+convert \TeX's file names to ordinary \CEE/ strings.
+
+@<Find an existing directory entry@>=
+for (i=3; i<= max_section_no;i++)
+ if (dir[i].file_name!=NULL && strcmp(dir[i].file_name,file_name)==0)
+ return i;
+@
+
+@<Allocate a new directory entry@>=
+ i = max_section_no;
+ i++;
+ if (i>0xFFFF) QUIT("Too many file sections");
+ if (i>=dir_entries)
+ RESIZE(dir,dir_entries,entry_t);
+ max_section_no=i;
+ if (max_section_no>0xFFFF) QUIT("Too many sections");
+ dir[i].section_no=i;
+@
+
+@<Hi\TeX\ macros@>=
+#define @[RESIZE(P,S,T)@] \
+{ int _n = (S)*1.4142136 +0.5; \
+ if (_n<32) _n=32; \
+ { REALLOCATE(P,_n,T); \
+ memset((P)+(S),0,(_n-(S))*sizeof(T)); \
+ (S)=_n; \
+ } \
+}
+@
+
+@<Hi\TeX\ variables@>=
+static int dir_entries=4;
+@
+
+@<Hi\TeX\ auxiliar routines@>=
+static uint16_t hnew_file_section(char *file_name)
+{ uint16_t i;
+ @<Find an existing directory entry@>@;
+ @<Allocate a new directory entry@>@;
+ dir[i].file_name=strdup(file_name);
+ return i;
+}
+@
+
+The following function uses \TeX's function |pack_file_name|
+to create a new filename from a name |n|, a direcory or ``area'' |a|,
+and an extension |e|. \TeX\ will truncate the new filename
+to |file_name_size| characters without warning. The new function
+will take a |name_length| equal to | file_name_size| as an
+indication that truncation has taken place and terminates the
+program. The return value converts a \Pascal\ array, starting with index 1,
+into a \CEE/ array starting with index 0.
+
+@<Hi\TeX\ auxiliar routines@>=
+static char *hfile_name(str_number n, char *a, char *e)
+{ pack_file_name(n,a,e);
+ if (name_length>=file_name_size)
+ QUIT("File name too long %d >= %d",name_length,file_name_size);
+ return (char *)name_of_file+1;
+}
+
+@
+
+\section{\HINT\ Definitions}\label{definitions}
+Definitions are used for two reasons: they provide default values for the parameters
+that drive \TeX's algorithms running in the \HINT\ viewer, and they provide a compact notation
+for \HINT\ content nodes.
+
+To find the optimal coding for a \HINT\ file, a global knowledge of the \HINT\ file is necessary.
+This would require a two pass process:
+in the first pass Hi\TeX\ could gather statistics on the use of parameter values and content
+nodes as a basis for making definitions and in the second pass it could encode the content using
+these definitions. I consider it, however, more reasonable to write such a two pass optimizer
+as a separate program which can be used on any \HINT\ file.
+Hence Hi\TeX\ uses a much simpler one pass approach:
+\itemize
+
+\item Hi\TeX\ generates definitions for \TeX-parameters using the
+ values they have when the first non discardable item appears in
+ |build_page|. This is usually the case after initial style files have
+ been processed and we can expect that they set useful default values.
+
+The procedure that generates these definitions is called |hfix_defaults|:
+
+@<Hi\TeX\ auxiliar routines@>=
+
+void hfix_defaults(void)
+{ @+int i;
+ DBG(DBGDEF,"Freezing HINT file defaults\n");
+ @<Compute the page size@>@;
+ @<Fix definitions for integer parameters@>@;
+ @<Fix definitions for dimension parameters@>@;
+ @<Fix definitions for glue parameters@>@;
+ @<Fix definitions of page templates@>@;
+}
+
+@
+
+\item Hi\TeX\ generates definitions to be used in content nodes on the fly:
+Whenever a routine outputs an item for which a definition might be available,
+it calls a {\it hget\_\dots\_no} function. This function returns, if possible,
+the reference number of a suitable definition.
+If no definition is available, the function will try to allocate a new one,
+only if all reference numbers from 0 to |0xFF| are already in use, a $-1$ is
+returned to indicate failure.
+
+There are two possible problems with this approach: We might miss a
+very common item because it occurs for the first time late in the
+input when all reference numbers are already in use. For example an
+extensive index might repeat a certain pattern for each entry. And second, we
+might make a definition for an item that occurs only once. Taken
+together the definition plus the reference to it requires more space
+than the same item without a definition.
+
+We can hope that the first effect does not occur too often, especially if the
+\TeX\ file is short, and we know that the second effect is limited by the
+total number of definitions we can make plus four byte of overhead per instance.
+\enditemize
+
+Here we initialize the necessary data structures for definitions.
+@<Hi\TeX\ auxiliar routines@>=
+static void hdef_init(void)
+{@+ int i;
+ @<Switch |hsize| and |vsize| to extended dimensions@>@;
+ @<Initialize definitions for extended dimensions@>@;
+ @<Initialize definitions for baseline skips@>@;
+ @<Initialize definitions for fonts@>@;
+ @<Initialize definitions for labels@>@;
+#if 0
+ overfull_rule=0; /* no overfull rules please */
+#endif
+}
+@
+
+After all definitions are ready, we write them using the function
+|hput_definitions|. When we output the definitions,
+we have to make sure to define references before we use them.
+This is achived by using a specific ordering of the
+definitions in the function |hput_definitions| and by preventing
+the allocation of new definitions as soon as the output of the definition
+section has started. The latter has the aditional benefit that the
+maximum values do no longer change.
+
+@<Hi\TeX\ routines@>=
+static void hput_definitions()
+/* write the definitions into the definitions buffer */
+{ int i;
+ uint32_t d, m, s;
+ hput_definitions_start();
+ hput_max_definitions();
+ @<Output language definitions@>@;
+ @<Output font definitions@>@;
+ @<Output integer definitions@>@;
+ @<Output dimension definitions@>@;
+ @<Output extended dimension definitions@>@;
+ @<Output glue definitions@>@;
+ @<Output baseline skip definitions@>@;
+ @<Output parameter list definitions@>@;
+ @<Output discretionary break definitions@>@;
+ @<Output page template definitions@>@;
+ hput_definitions_end();
+ hput_range_defs(); /* expects the definitions section to be ended */
+ hput_label_defs();
+}
+@
+
+
+In the following, we present for each node type the code to generate
+the definitions, using a common schema: We define a data structure
+called {\it\dots\_defined}, to hold the definitions; we define, if
+applicable, the \TeX-parameters; we add an {\it hget\_\dots\_no}
+function to allocate new definitions; and we finish with the code to
+output the collected definitions.
+
+Lets start with the most simple case: integers.
+
+\subsection{Integers}
+\subsubsection{Data}
+The data structure to hold the integer definitions is a simple array with |0x100| entries.
+A more complex data structure, for example a hash table, could speed up searching for
+existing definitions but lets keep things simple for now.
+
+@<Hi\TeX\ variables@>=
+static int32_t int_defined[0x100]={0};
+@
+\subsubsection{Mapping}
+Before we can generate definitions for \TeX-parameters, we have to map \TeX's
+parameter numbers to \HINT\ definition numbers. While it seems more convenient here
+to have the reverse mapping, we need the mapping only once to record parameter definitions,
+but we will need it repeatedly in the function |hdef_param_node| and the overhead here does
+not warant having the mapping in both directions.
+
+@<Hi\TeX\ variables@>=
+static const int hmap_int[] ={@/
+pretolerance_no, /* |pretolerance_code| 0 */
+tolerance_no, /* |tolerance_code| 1 */
+line_penalty_no, /* |line_penalty_code| 2 */
+hyphen_penalty_no, /* |hyphen_penalty_code| 3 */
+ex_hyphen_penalty_no, /* |ex_hyphen_penalty_code| 4 */
+club_penalty_no, /* |club_penalty_code| 5 */
+widow_penalty_no, /* |widow_penalty_code| 6 */
+display_widow_penalty_no, /* |display_widow_penalty_code| 7 */
+broken_penalty_no, /* |broken_penalty_code| 8 */
+-1, /* |bin_op_penalty_code| 9 */
+-1, /* |rel_penalty_code| 10 */
+pre_display_penalty_no, /* |pre_display_penalty_code| 11 */
+post_display_penalty_no, /* |post_display_penalty_code| 12 */
+inter_line_penalty_no, /* |inter_line_penalty_code| 13 */
+double_hyphen_demerits_no, /* |double_hyphen_demerits_code| 14 */
+final_hyphen_demerits_no, /* |final_hyphen_demerits_code| 15 */
+adj_demerits_no, /* |adj_demerits_code| 16 */
+-1, /* |mag_code| 17 */
+-1, /* |delimiter_factor_code| 18 */
+looseness_no, /* |looseness_code| 19 */
+time_no, /* |time_code| 20 */
+day_no, /* |day_code| 21 */
+month_no, /* |month_code| 22 */
+year_no, /* |year_code| 23 */
+-1, /* |show_box_breadth_code| 24 */
+-1, /* |show_box_depth_code| 25 */
+-1, /* |hbadness_code| 26 */
+-1, /* |vbadness_code| 27 */
+-1, /* |pausing_code| 28 */
+-1, /* |tracing_online_code| 29 */
+-1, /* |tracing_macros_code| 30 */
+-1, /* |tracing_stats_code| 31 */
+-1, /* |tracing_paragraphs_code| 32 */
+-1, /* |tracing_pages_code| 33 */
+-1, /* |tracing_output_code| 34 */
+-1, /* |tracing_lost_chars_code| 35 */
+-1, /* |tracing_commands_code| 36 */
+-1, /* |tracing_restores_code| 37 */
+-1, /* |uc_hyph_code| 38 */
+-1, /* |output_penalty_code| 39 */
+-1, /* |max_dead_cycles_code| 40 */
+hang_after_no, /* |hang_after_code| 41*/
+floating_penalty_no /* |floating_penalty_code| 42*/
+};
+
+
+@
+\subsubsection{Parameters}
+Now we can generate the definitions for integer parameters:
+
+@<Fix definitions for integer parameters@>=
+ int_defined[zero_int_no]=0;
+ for (i=pretolerance_code; i<=hang_after_code;i++)
+ if ( hmap_int[i]>=0) int_defined[hmap_int[i]]=int_par(i);
+ max_ref[int_kind]=MAX_INT_DEFAULT;
+@
+
+
+\subsubsection{Allocation}
+The function |hget_int_no| tries to allocate a predefined integer number;
+if not successful, it returns $-1$.
+
+@<Hi\TeX\ auxiliar routines@>=
+static int hget_int_no(int32_t n)
+{ int i;
+ int m =max_ref[int_kind];
+ for (i=0; i<=m; i++)
+ if (n== int_defined[i]) return i;
+ if (m<0xFF && section_no==2)
+ { m=++max_ref[int_kind]; int_defined[m]=n; return m; }
+ else
+ return -1;
+}
+@
+
+\subsubsection{Output}
+Before we give the code to output an integer definition, we declare a macro that
+is usefull for all the definitions. |HPUTDEF| takes a function |F| and a reference number |R|.
+It is assumed that |F| writes a definition into the output and returns a tag. The macro
+will then add the reference number and both tags to the output.
+@<Hi\TeX\ macros@>=
+#define HPUTDEF(F,R) \
+ { uint32_t _p; \
+ uint8_t _t; \
+ HPUTNODE; /* allocate */ \
+ _p=hpos-hstart; \
+ HPUT8(0); /* tag */ \
+ HPUT8(R); /* reference */ \
+ _t=F; \
+ hstart[_p]=_t; DBGTAG(_t,hstart+_p); \
+ DBGTAG(_t,hpos); HPUT8(_t); \
+ }
+@
+
+Definitions are written to the output only if they differ from Hi\TeX's built in defaults.
+@<Output integer definitions@>=
+ DBG(DBGDEF,"Maximum int reference: %d\n",max_ref[int_kind]);
+ for (i=max_fixed[int_kind]+1;i<=max_default[int_kind]; i++)
+ { if (int_defined[i]!=int_defaults[i])@/
+ HPUTDEF(hput_int(int_defined[i]),i);
+ }
+ for (;i<=max_ref[int_kind]; i++)@/
+ HPUTDEF(hput_int(int_defined[i]),i);
+@
+
+
+\subsection{Dimensions}
+We proceed as we did for integers, starting with the array that holds the defined dimensions.
+\subsubsection{Data}
+@<Hi\TeX\ variables@>=
+static scaled dimen_defined[0x100]={0};
+@
+\subsubsection{Mapping}
+@<Hi\TeX\ variables@>=
+static const int hmap_dimen[] ={@/
+ -1, /* |par_indent_code| 0 */
+ -1, /* |math_surround_code| 1 */
+ line_skip_limit_no, /* |line_skip_limit_code| 2 */
+ hsize_dimen_no, /* |hsize_code| 3 */
+ vsize_dimen_no, /* |vsize_code| 4 */
+ max_depth_no, /* |max_depth_code| 5 */
+ split_max_depth_no, /* |split_max_depth_code| 6 */
+ -1, /* |box_max_depth_code| 7 */
+ -1, /* |hfuzz_code| 8 */
+ -1, /* |vfuzz_code| 9 */
+ -1, /* |delimiter_shortfall_code| 10 */
+ -1, /* |null_delimiter_space_code| 11 */
+ -1, /* |script_space_code| 12 */
+ -1, /* |pre_display_size_code| 13 */
+ -1, /* |display_width_code| 14 */
+ -1, /* |display_indent_code| 15 */
+ -1, /* |overfull_rule_code| 16 */
+ hang_indent_no, /* |hang_indent_code| 17 */
+ -1, /* |h_offset_code| 18 */
+ -1, /* |v_offset_code| 19 */
+ emergency_stretch_no /* |emergency_stretch_code| 20 */
+};
+@
+
+\subsubsection{Parameters}
+@<Fix definitions for dimension parameters@>=
+ dimen_defined[zero_dimen_no]=0;
+ for (i=par_indent_code; i<=emergency_stretch_code;i++)
+ if ( hmap_dimen[i]>=0) dimen_defined[hmap_dimen[i]]=dimen_par(i);
+ dimen_defined[hsize_dimen_no]=page_h;
+ dimen_defined[vsize_dimen_no]=page_v;
+ dimen_defined[quad_no]=quad(cur_font);
+ dimen_defined[math_quad_no]=math_quad(text_size);
+ max_ref[dimen_kind]=MAX_DIMEN_DEFAULT;
+@
+
+\subsubsection{Allocation}
+@<Hi\TeX\ auxiliar routines@>=
+static int hget_dimen_no(scaled s)
+/* tries to allocate a predefined dimension number in the range 0 to 0xFF
+ if not successful return -1 */
+{ int i;
+ int m =max_ref[dimen_kind];
+ for (i=0; i<=m; i++)
+ if (s== dimen_defined[i]) return i;
+ if (m<0xFF && section_no==2)
+ { m=++max_ref[dimen_kind]; dimen_defined[m]=s; return m; }
+ else
+ return -1;
+}
+@
+
+\subsubsection{Output}
+@<Output dimension definitions@>=
+ DBG(DBGDEF,"Maximum dimen reference: %d\n",max_ref[dimen_kind]);
+ for (i=max_fixed[dimen_kind]+1;i<=max_default[dimen_kind]; i++)
+ { if (dimen_defined[i]!=dimen_defaults[i])
+ HPUTDEF(hput_dimen(dimen_defined[i]),i);
+ }
+ for (;i<=max_ref[dimen_kind]; i++)
+ HPUTDEF(hput_dimen(dimen_defined[i]),i);
+@
+
+
+\subsection{Extended Dimensions}
+
+\subsubsection{Data}
+@<Hi\TeX\ variables@>=
+static struct {
+scaled w,h,v; } xdimen_defined[0x100];
+@
+
+\subsubsection{Initialization}
+@<Initialize definitions for extended dimensions@>=
+ for (i=0; i<=max_fixed[xdimen_kind]; i++)
+ { xdimen_defined[i].w = xdimen_defaults[i].w;
+ xdimen_defined[i].h = ONE*xdimen_defaults[i].h;
+ xdimen_defined[i].v = ONE*xdimen_defaults[i].v;
+ }
+@
+
+\subsubsection{Allocation}
+To obtain a reference number for an extended dimension, we search the
+array and if no match was found, we allocate a new entry,
+reallocating the array if needed.
+We use the variable |rover| to mark the place where the
+last entry was inserted, because
+quite often we repeatedly search for the same values.
+
+@<Hi\TeX\ auxiliar routines@>=
+int hget_xdimen_no(pointer p)
+{ int i;
+ for (i=0;i<=max_ref[xdimen_kind];i++)
+ { if (xdimen_defined[i].w== xdimen_width(p) &&
+ xdimen_defined[i].h== xdimen_hfactor(p) &&
+ xdimen_defined[i].v== xdimen_vfactor(p))
+ return i;
+ }
+ if (section_no!=2) return -1;
+ if (i>=0x100) return -1;
+ max_ref[xdimen_kind]=i;
+ xdimen_defined[i].w= xdimen_width(p);
+ xdimen_defined[i].h= xdimen_hfactor(p);
+ xdimen_defined[i].v= xdimen_vfactor(p);
+ return i;
+}
+@
+
+@<Hi\TeX\ routines@>=
+pointer new_xdimen(scaled w, scaled h, scaled v)
+{ pointer p=get_node(xdimen_node_size);
+ type(p)=whatsit_node;subtype(p)=xdimen_node;
+ xdimen_width(p)=w;
+ xdimen_hfactor(p)=h;
+ xdimen_vfactor(p)=v;
+ return p;
+}
+@
+
+\subsubsection{Output}
+@<Output extended dimension definitions@>=
+ DBG(DBGDEF,"Maximum xdimen reference: %d\n",max_ref[xdimen_kind]);
+ for (i=max_fixed[xdimen_kind]+1;i<=max_default[xdimen_kind]; i++)
+ { xdimen_t x;
+ x.w=xdimen_defined[i].w;
+ x.h=xdimen_defined[i].h/(double)ONE;
+ x.v=xdimen_defined[i].v/(double)ONE;
+
+ if (x.w!=xdimen_defaults[i].w ||
+ x.h!=xdimen_defaults[i].h ||
+ x.v!=xdimen_defaults[i].v)
+ HPUTDEF(hput_xdimen(&x),i);
+ }
+ for (;i<=max_ref[xdimen_kind]; i++)
+ { xdimen_t x;
+ x.w=xdimen_defined[i].w;
+ x.h=xdimen_defined[i].h/(double)ONE;
+ x.v=xdimen_defined[i].v/(double)ONE;
+ HPUTDEF(hput_xdimen(&x),i);
+ }
+@
+
+
+@
+\subsection{Glues}
+In general there are two choices on how to store a definition: We can use the data structures used by \TeX\
+or we can use the data structures defined by \HINT. If we are lucky, both of them are the same
+as we have seen for integers and dimensions. For extended dimensions, we had to use the \HINT\ data type
+|xdimen_t| because \TeX\ has no corresponding data type and uses only reference numbers.
+In the case of glue, we definitely have a choice. We decide to use \TeX's pointers to glue specifications
+in the hope to save some work when comparing glues for equality, because \TeX\ already reuses
+glue specifications and often a simple comparison of pointers might suffice.
+\subsubsection{Data}
+@<Hi\TeX\ variables@>=
+static pointer glue_defined[0x100];
+@
+\subsubsection{Mapping}
+@<Hi\TeX\ variables@>=
+static int hmap_glue[] ={
+
+line_skip_no, /* |line_skip_code| 0 */
+baseline_skip_no, /* |baseline_skip_code| 1 */
+-1, /* |par_skip_code| 2 */
+above_display_skip_no, /* |above_display_skip_code| 3 */
+below_display_skip_no, /* |below_display_skip_code| 4 */
+above_display_short_skip_no, /* |above_display_short_skip_code| 5 */
+below_display_short_skip_no, /* |below_display_short_skip_code| 6 */
+left_skip_no, /* |left_skip_code| 7 */
+right_skip_no, /* |right_skip_code| 8 */
+top_skip_no, /* |top_skip_code| 9 */
+split_top_skip_no, /* |split_top_skip_code| 10 */
+tab_skip_no, /* |tab_skip_code| 11 */
+-1, /* |space_skip_code| 12 */
+-1, /* |xspace_skip_code| 13 */
+par_fill_skip_no /* |par_fill_skip_code| 14 */
+};
+@
+\subsubsection{Parameters}
+@<Fix definitions for glue parameters@>=
+ glue_defined[zero_skip_no]=zero_glue; incr(glue_ref_count(zero_glue));
+ for (i=line_skip_code; i<=par_fill_skip_code;i++)
+ if ( hmap_glue[i]>=0)
+ { glue_defined[hmap_glue[i]]=glue_par(i); incr(glue_ref_count(glue_par(i)));}
+ max_ref[glue_kind]=MAX_GLUE_DEFAULT;
+@
+
+
+\subsubsection{Allocation}
+Next we define some auxiliar routines to compare glues for equality and to convert glues beween the different representations.
+
+@<Hi\TeX\ auxiliar routines@>=
+int glue_spec_equal(pointer p, pointer q)
+{ return (width(q)==width(p) && stretch(q)==stretch(p) && shrink(q)==shrink(p) &&
+ (stretch_order(q)==stretch_order(p) || stretch(q)==0) &&
+ (shrink_order(q)==shrink_order(p)|| shrink(q)==0));
+}
+
+int glue_equal(pointer p, pointer q)
+{ return p==q || glue_spec_equal(p,q);
+}
+
+int glue_t_equal(glue_t *p, glue_t *q)
+{ return(p->w.w==q->w.w && p->w.h==q->w.h && p->w.v==q->w.v &&
+ p->p.f== q->p.f && p->m.f==q->m.f &&
+ (p->p.o==q->p.o || p->p.f==0.0) &&
+ (p->m.o==q->m.o || q->m.f==0.0));
+}
+
+@
+
+To find a matching glue we make two passes over the defined glues:
+on the first pass we just compare pointers and on the second pass
+we also compare values. An alternative approach to speed up searching
+is used in section~\secref{paramlist} below.
+
+@<Hi\TeX\ auxiliar routines@>=
+static int hget_glue_no(pointer p)
+{ static int rover=0;
+ int i;
+ if (p==zero_glue) return zero_skip_no;
+ for(i=0; i<= max_ref[glue_kind] ; i++)
+ { if (p==glue_defined[rover]) return rover;
+ else if (rover==0)
+ rover=max_ref[glue_kind];
+ else
+ rover--;
+ }
+ for(i=0; i<= max_ref[glue_kind] ; i++)
+ { pointer q=glue_defined[rover];
+ if (glue_spec_equal(p,q))
+ return rover;
+ else if (rover==0)
+ rover=max_ref[glue_kind];
+ else
+ rover--;
+ }
+ if (max_ref[glue_kind]<0xFF && section_no==2)
+ { rover=++max_ref[glue_kind];
+ glue_defined[rover]=p;
+ incr(glue_ref_count(p));
+ DBG(DBGDEF,"Defining new glue %d\n",rover);
+ return rover;
+ }
+ else
+ return -1;
+}
+@
+\subsubsection{Output}
+@<Output glue definitions@>=
+ DBG(DBGDEF,"Maximum glue reference: %d\n",max_ref[glue_kind]);
+ for (i=max_fixed[glue_kind]+1;i<=max_default[glue_kind]; i++)
+ { glue_t g;
+ to_glue_t(glue_defined[i],&g);
+ if (!glue_t_equal(&g,&glue_defaults[i]))
+ HPUTDEF(hput_glue(&g),i);
+ }
+ for (;i<=max_ref[glue_kind]; i++)
+ HPUTDEF(hout_glue_spec(glue_defined[i]),i);
+@
+
+The above code uses the following conversion routine.
+While \HINT\ supports glue that depends on {\tt hsize} and {\tt vsize},
+this is currently not supported by Hi\TeX.
+Future versions of Hi\TeX\ should extend glue spec nodes (and kern nodes)
+by fields for |hfactor| and |vfactor| which are zero by default.
+This would leave most parts of \TeX\ unchanged.
+As a work-around one can combine a box with an extended dimension with
+a regular glue or kern.
+% Care should be taken for the statically allocated glue specs
+% deallocation of glue specs is relatively simple
+% extended glue and kern values can be restricted to non math mode
+@<Hi\TeX\ auxiliar routines@>=
+void to_glue_t(pointer p, glue_t *g)
+{ g->w.w=width(p);
+ g->w.h=g->w.v=0.0;
+ g->p.f=stretch(p)/(double)ONE; g->p.o= stretch_order(p);
+ g->m.f=shrink(p)/(double)ONE; g->m.o= shrink_order(p);
+}
+@
+
+\subsection{Baseline Skips}
+TeX's baseline nodes just store a baseline skip reference number.
+We have seen this sitation before when dealing with extended dimensions
+and the solution here is the same: a dynamicaly allocated array.
+\subsubsection{Data}
+@<Hi\TeX\ variables@>=
+typedef struct {
+ pointer ls, bs; /* line skip and baselineskip gluespecs */
+ scaled lsl; /* lineskip limit */
+} bl_defined_t;
+
+static bl_defined_t *bl_defined=NULL;
+static int bl_used=0,bl_allocated=0;
+@
+
+\subsubsection{Initialization}
+The zero baseline skip is predefined which prevents an ambiguous info value of zero
+in a baseline node.
+
+@<Initialize definitions for baseline skips@>=
+ bl_allocated=8;
+ ALLOCATE(bl_defined,bl_allocated,bl_defined_t);
+ bl_defined[zero_baseline_no].bs=zero_glue; incr(glue_ref_count(zero_glue));
+ bl_defined[zero_baseline_no].ls=zero_glue; incr(glue_ref_count(zero_glue));
+ bl_defined[zero_baseline_no].lsl=0;
+ bl_used= MAX_BASELINE_DEFAULT+1;
+ max_ref[baseline_kind]= MAX_BASELINE_DEFAULT;
+@
+\subsubsection{Allocation}
+@<Hi\TeX\ auxiliar routines@>=
+int hget_baseline_no(pointer bs, pointer ls, scaled lsl)
+{
+ static int rover=0;
+ int i;
+ for(i=0; i< bl_used; i++) /* search for an existing spec */
+ { bl_defined_t *q=&(bl_defined[rover]);
+ if (glue_equal(bs,q->bs) && glue_equal(ls,q->ls) && lsl==q->lsl)
+ return rover;
+ else if (rover==0)
+ rover=bl_used-1;
+ else
+ rover--;
+ }
+ if (bl_used>=bl_allocated)
+ RESIZE(bl_defined,bl_allocated, bl_defined_t);
+ rover=bl_used++;
+ if (rover<0x100 && section_no==2) max_ref[baseline_kind]=rover;
+ if (glue_equal(bs,zero_glue))
+ { bl_defined[rover].bs=zero_glue; incr(glue_ref_count(zero_glue)); }
+ else
+ { bl_defined[rover].bs=bs; incr(glue_ref_count(bs));}
+ if (glue_equal(ls,zero_glue))
+ { bl_defined[rover].ls=zero_glue; incr(glue_ref_count(zero_glue)); }
+ else
+ { bl_defined[rover].ls=ls; incr(glue_ref_count(ls));}
+ bl_defined[rover].lsl=lsl;
+ return rover;
+}
+
+@
+\subsubsection{Output}
+
+The following routine does not allocate a new glue definition, because the
+baselinedefinitions are output after the glue definitions. This is not perfect.
+@<Hi\TeX\ auxiliar routines@>=
+static uint8_t hout_glue_spec(pointer p);
+uint8_t hout_baselinespec(int n)
+{ info_t i=b000;
+ pointer p;
+ scaled s;
+ s=bl_defined[n].lsl;
+ if (s!=0) {HPUT32(s); i|=b001;}
+ p=bl_defined[n].bs;
+ if (p!=zero_glue)
+{ uint8_t *pos;
+ uint8_t tag;
+ HPUTNODE; /* allocate */
+ pos=hpos;
+ hpos++; /* tag */
+ tag=hout_glue_spec(p);
+ *pos=tag; DBGTAG(tag,pos);
+ DBGTAG(tag,hpos); HPUT8(tag);
+ i|=b100;
+}
+ p=bl_defined[n].ls;
+ if (p!=zero_glue)
+{ uint8_t *pos;
+ uint8_t tag;
+ HPUTNODE; /* allocate */
+ pos=hpos;
+ hpos++; /* tag */
+ tag=hout_glue_spec(p);
+ *pos=tag; DBGTAG(tag,pos);
+ DBGTAG(tag,hpos); HPUT8(tag);
+ i|=b010;
+}
+ return TAG(baseline_kind,i);
+}
+@
+
+@<Output baseline skip definitions@>=
+ DBG(DBGDEF,"Defining %d baseline skips\n",max_ref[baseline_kind]);
+ for (i=1;i<=max_ref[baseline_kind]; i++)
+ { uint32_t pos=hpos-hstart;
+ uint8_t tag;
+ hpos++; /* space for the tag */
+ HPUT8(i); /* reference */
+ tag=hout_baselinespec(i);
+ hstart[pos]=tag;
+ HPUT8(tag);
+ }
+@
+\subsubsection{Printing}
+The following function is needed in Hi\TeX\ to produce debugging output if needed.
+@<Hi\TeX\ routines@>=
+void print_baseline_skip(int i)
+{ if (0<=i && i < bl_used)
+ { print_spec(bl_defined[i].bs,0); print_char(',');
+ print_spec(bl_defined[i].ls,0); print_char(',');
+ print_scaled(bl_defined[i].lsl);
+ }
+ else
+ print("unknown");
+}
+@
+
+
+
+\subsection{Discretionary breaks}
+\subsubsection{Data}
+For discretionary breaks, we use again the pointer representation.
+@<Hi\TeX\ variables@>=
+static pointer dc_defined[0x100];
+@
+\subsubsection{Allocation}
+There are no predefined discretionary breaks and so we start with
+three auxiliar functions and the
+function to get a ``disc'' number.
+
+The first two routines are used to compare discretionary breaks
+in order to reuse already defined disc numbers.
+The pre and post break lists must consist entirely of character,
+kern, box, rule, and ligature nodes.
+Unfortunately a box node might contain all kinds of nodes
+and its content might be huge and deeply nested.
+The following routine will not make a complete comparison but will give
+up if the box content is ``too complex''.
+
+@<Hi\TeX\ auxiliar routines@>=
+static bool list_equal(pointer p, pointer q);
+static bool node_equal(pointer p, pointer q)
+{ if (is_char_node(p) && is_char_node(q) &&
+ font(p)==font(q) && character(p)==character(q))
+ return true;
+ if (!is_char_node(p) && !is_char_node(q))
+ { if (type(p)!=type(q)) return false;
+ if (type(p)==kern_node &&
+ subtype(p)==subtype(q) && width(p)==width(q))
+ return true;
+ if (type(p)==ligature_node &&
+ character(lig_char(p)) == character(lig_char(q)) &&
+ font(lig_char(p)) == font(lig_char(q)))
+ return true;
+ if (type(p)==rule_node &&
+ width(p)==width(q) && height(p)==height(q) && depth(p)==depth(q))
+ return true;
+ if ((type(p)==hlist_node || type(p)==vlist_node) &&
+ width(p)==width(q) && height(p)==height(q) && depth(p)==depth(q) &&
+ shift_amount(p)==shift_amount(q) &&
+ glue_sign(p) == glue_sign(q) &&
+ glue_order(p) == glue_order(q) &&
+ glue_set(p) == glue_set(q) &&
+ list_equal(list_ptr(p),list_ptr(q)))
+ return true;
+ }
+ return false;
+}
+
+static bool list_equal(pointer p, pointer q)
+{@+while (true)
+ { if (p==q) return true;
+ if (p==null || q==null) return false;
+ if (!node_equal(p,q)) return false;
+ p=link(p);q=link(q);
+ }
+}
+
+static pointer copy_disc_node(pointer p)
+{ pointer q;
+ q=get_node(small_node_size);
+ pre_break(q)=copy_node_list(pre_break(p));
+ post_break(q)=copy_node_list(post_break(p));
+ type(q)=type(p);
+ subtype(q)=subtype(p); /* replace count and explicit bit */
+ return q;
+}
+@
+
+@<Hi\TeX\ routines@>=
+int hget_disc_no(pointer p)
+{
+ static int rover=0;
+ int i;
+ for(i=0; i<= max_ref[disc_kind]; i++)
+ { pointer q=dc_defined[rover];
+ if ( is_auto_disc(p)==is_auto_disc(q) && replace_count(p)==replace_count(q) &&
+ list_equal(pre_break(p),pre_break(q)) &&
+ list_equal(post_break(p),post_break(q)))
+ return rover;
+ else if (rover==0)
+ rover=max_ref[disc_kind];
+ else
+ rover--;
+ }
+ if (max_ref[disc_kind]>=0xFF || section_no!=2) return -1;
+
+ rover=++max_ref[disc_kind];
+ dc_defined[rover]=copy_disc_node(p);
+ @<Allocate font numbers for glyphs in the pre- and post-break lists@>@;
+ return rover;
+}
+@
+
+When we allocate disc numbers we might have fonts inside the pre-
+or post-break list, that never show up anywhere else in the content.
+These fonts would then be undefined once we start the definition section.
+So we have to make sure, all necessary fonts get defined.
+
+@<Allocate font numbers for glyphs in the pre- and post-break lists@>=
+ensure_font_no(pre_break(p));
+ensure_font_no(post_break(p));
+@
+
+\subsubsection{Output}
+@<Output discretionary break definitions@>=
+ DBG(DBGDEF,"Maximum disc reference: %d\n",max_ref[disc_kind]);
+ for (i=0;i<=max_ref[disc_kind]; i++)
+ HPUTDEF(hout_disc(dc_defined[i]),i);
+@
+
+
+
+\subsection{Parameter Lists}\label{paramlist}
+\subsubsection{Data}
+We store predefined parameter lists in a hash table in order to speed up
+finding existing parameter lists. The parameter list itself is stored as
+a byte sequence using the short \HINT\ file format.
+We link the table entries in order of increasing reference numbers to be able
+to output them in a more ``orderly'' fashion.
+
+@<Hi\TeX\ variables@>=
+
+#define PLH_SIZE 313 /* a prime number $\approx2^8\times 1.2$. */
+
+struct {int l; /* link */
+ uint32_t h; /* hash */
+ uint32_t n; /* number */
+ uint32_t s; /* size */
+ uint8_t *p; /* pointer */} pl_defined[PLH_SIZE]={{0}};
+int pl_head=-1, *pl_tail=&pl_head;
+@
+
+\subsubsection{Allocation}
+Next we define three short auxiliar routines and the |hget_param_list_no| function.
+
+@<Hi\TeX\ routines@>=
+
+static uint32_t hparam_list_hash(list_t *l)
+{ uint32_t h=0;
+ uint32_t i;
+ for (i=0;i<l->s;i++)
+ h=3*h+hstart[l->p+i];
+ return i;
+}
+
+static bool pl_equal(list_t *l, uint8_t *p)
+{ uint8_t *q=hstart+l->p;
+ uint32_t i;
+ for (i=0; i<l->s; i++)
+ if (q[i]!=p[i]) return false;
+ return true;
+}
+
+static void pl_copy(list_t *l, uint8_t *p)
+{ uint8_t *q=hstart+l->p;
+ memcpy(p,q,l->s);
+}
+
+int hget_param_list_no(list_t *l)
+{ uint32_t h;
+ int i;
+ if (l->s<=0) return -1;
+ h= hparam_list_hash(l);
+ i = h%PLH_SIZE;
+ while (pl_defined[i].p!=NULL)
+ { if (pl_defined[i].h==h && pl_equal(l,pl_defined[i].p))
+ return pl_defined[i].n;
+ i=i+199; /* some other prime */
+ if (i>=PLH_SIZE) i=i-PLH_SIZE;
+ }
+ if (max_ref[param_kind]>=0xFF || section_no!=2) return -1;
+ pl_defined[i].n=++max_ref[param_kind];
+ *pl_tail=i; pl_tail=&(pl_defined[i].l);
+ pl_defined[i].l=-1;
+ pl_defined[i].h=h;
+ pl_defined[i].s=l->s;
+ ALLOCATE(pl_defined[i].p,l->s,uint8_t);
+ pl_copy(l,pl_defined[i].p);
+ return pl_defined[i].n;
+}
+@
+
+\subsubsection{Output}
+To output parameter lists, we need a function to output a parameter node:
+
+@<Hi\TeX\ routines@>=
+void hdef_param_node(int ptype, int pnumber,int pvalue)
+{
+ if (ptype==int_type)
+ { if (pvalue==int_defined[hmap_int[pnumber]]) return;
+ else HPUTDEF(hput_int(pvalue),hmap_int[pnumber]);
+ }
+ else if (ptype==dimen_type)
+ { if (pvalue==dimen_defined[hmap_dimen[pnumber]]) return;
+ else HPUTDEF(hput_dimen(pvalue),hmap_dimen[pnumber]);
+ }
+ else if (ptype==glue_type)
+ { if (glue_equal(pvalue,glue_defined[hmap_glue[pnumber]])) return;
+ else HPUTDEF(hout_glue_spec((pointer)pvalue),hmap_glue[pnumber]);
+ }
+ else QUIT("Unexpected parameter type %d",ptype);
+}
+@
+
+
+Now we use the linked list starting with |pl_head| to output the predefined
+parameter lists sorted by their reference number.
+
+ @<Output parameter list definitions@>=
+ DBG(DBGDEF,"Defining %d parameter lists\n",max_ref[param_kind]+1);
+ for (i=pl_head;i>=0;i=pl_defined[i].l)
+ { int j;
+ DBG(DBGDEF,"Defining parameter list %d, size 0x%x\n",i,pl_defined[i].s);
+ j=hsize_bytes(pl_defined[i].s);
+ HPUTX(1+1+j+1+pl_defined[i].s+1+j+1);
+ HPUTTAG(param_kind,j+1);
+ HPUT8(pl_defined[i].n);
+ hput_list_size(pl_defined[i].s,j);
+ HPUT8(0x100-j);
+ memcpy(hpos,pl_defined[i].p,pl_defined[i].s);
+ hpos=hpos+pl_defined[i].s;
+ HPUT8(0x100-j);
+ hput_list_size(pl_defined[i].s,j);
+ HPUTTAG(param_kind,j+1);
+ }
+@
+
+\subsection{Fonts}
+
+It seems I need: (see email by Karl)
+
+
+|kpse_find_file(name,kpse_fontmap_format,false);|
+with name "ps2pk.map" or "psfonts.map" or "ttfonts.map" or cmfonts.map
+to get from the tfm name the postscript name.
+then I get the |psfont_name| ususaly its the same font name
+with .pfb appended
+so I can skip it
+%#ifdef HAVE_KPSE_ENC_FORMATS
+% enc_file=kpse_find_file(encoding,kpse_enc_format,false);
+%#else
+% enc_file=kpse_find_file(encoding,kpse_tex_ps_header_format,false);
+%#endif
+%reading the encoding is found in dvipng/dvipnd-src/enc.c
+
+
+
+\subsubsection{Data}
+To store a font definition, we define the data type |font_t|
+and an array |hfonts| of pointers indexed by \HINT\ font numbers.
+To map \HINT\ font numbers to \TeX\ font numbers, the |font_t| contains
+the |i| field; to map \TeX\ font numbers to \HINT\ font numbers,
+we use the array |hmap_font|.
+
+@<Hi\TeX\ variables@>=
+#define MAX_FONTS 0x100
+
+typedef struct {
+ uint8_t i; /* the \TeX\ font number */
+ pointer g; /* space glue */
+ pointer h; /* default hyphen */
+ pointer p[MAX_FONT_PARAMS]; /* font parameters */
+ uint16_t m; /* section number of font metric file */
+ uint16_t y; /* section number of font glyph file */
+} font_t;
+
+static font_t *hfonts[MAX_FONTS]={NULL};
+static int hmap_font[MAX_FONTS];
+@
+
+
+
+@<Initialize definitions for fonts@>=
+ for (i=0;i<0x100;i++) hmap_font[i]=-1;
+ max_ref[font_kind]=-1;
+@
+
+\subsubsection{Allocation}
+Allocation of a |font_t| record takes place when we translate a \TeX\ font
+number to a \HINT\ font number using the function |hget_font_no|, and while
+doing so discover that the corresponding \HINT\ font number does not yet exist.
+Because the |font_t| structure must be initialized after allocating it,
+we start with some auxiliar routines for that purpose.
+
+@<Hi\TeX\ auxiliar routines@>=
+static pointer find_space_glue(internal_font_number f)
+{@+font_index @!main_k;
+ pointer main_p=font_glue[f];
+if (main_p==null)
+ {@+main_p=new_spec(zero_glue);main_k=param_base[f]+space_code;
+ width(main_p)=font_info[main_k].sc; /*that's |space(f)|*/
+ stretch(main_p)=font_info[main_k+1].sc; /*and |space_stretch(f)|*/
+ shrink(main_p)=font_info[main_k+2].sc; /*and |space_shrink(f)|*/
+ font_glue[f]=main_p;
+ }
+ return main_p;
+}
+static pointer hget_font_space(uint8_t f)
+{ pointer p;
+ if (space_skip==zero_glue)
+ p = find_space_glue(f);
+ else
+ p=glue_par(space_skip_code);
+ add_glue_ref(p);
+ return p;
+}
+
+
+static pointer hget_font_hyphen(uint8_t f)
+{ pointer p;
+ int c;
+ p=new_disc();
+ c= hyphen_char[f];
+ if (c >= 0 && c < 256) pre_break(p)=new_character(f, c);
+ return p;
+}
+
+
+static void hdef_font_params(pointer p[MAX_FONT_PARAMS])
+{ /* used only for texts */
+}
+@
+\goodbreak
+
+In the following code, |f| is a \TeX\ internal font number
+and |g| is the corresponding \HINT\ font number.
+\TeX's null-font, a kind of undefined font containing no characters
+is replaced by \HINT's font number zero. Actually the nullfont should
+never appear in the output, but if it does so, either an error message
+or a more sensible replacement font might be in order.
+
+@<Hi\TeX\ auxiliar routines@>=
+
+static char *hfind_glyphs(char *filename)
+{ char *fname=NULL;
+ kpse_glyph_file_type file_ret;
+ fname=kpse_find_file(filename,kpse_type1_format,true);
+ if (fname==NULL) fname=kpse_find_file(filename,kpse_truetype_format,true);
+ if (fname==NULL) fname=kpse_find_file(filename,kpse_opentype_format,true);
+ if (fname==NULL) fname = kpse_find_glyph(filename, option_dpi,kpse_pk_format, &file_ret);
+ if (fname==NULL)
+ fprintf(stderr,"Unable to find glyph data for font %s\n",filename),exit(1);
+ return fname;
+}
+
+static uint8_t hget_font_no(uint8_t f)
+
+{ int g;
+ char *n,*fn;
+ int l;
+ if (f==0)
+ { DBG(DBGFONT,"TeX nullfont -> 0\n");
+ return 0;@+
+ }
+ g=hmap_font[f];
+ DBG(DBGFONT,"Mapping TeX font %d->%d\n",f,g);
+ if (g>=0) return g;
+ DBG(DBGDEF,"New TeX font %d\n",f);
+ if (max_ref[font_kind]>=0x100)
+ QUIT("too many fonts in use");
+ g = ++(max_ref[font_kind]);
+ ALLOCATE(hfonts[g],1,font_t);
+ hfonts[g]->i=f;
+ hmap_font[f]=g;
+ hfonts[g]->g=hget_font_space(f);
+ hfonts[g]->h=hget_font_hyphen(f);
+ pack_file_name(font_name[f], "",".tfm");
+ n = kpse_find_tfm((char*)name_of_file+1);
+ if (n==NULL)
+ QUIT("Unable to find .tfm file for font %s",(char*)name_of_file+1);
+ hfonts[g]->m= hnew_file_section(n);
+ free(n);
+ pack_file_name(font_name[f], "","");
+ n= hfind_glyphs((char*)name_of_file+1);
+ if (n==NULL)
+ QUIT("Unable to find glyph file for font %s",(char*)name_of_file+1);
+ hfonts[g]->y= hnew_file_section(n);
+ free(n);
+ return g;
+}
+@
+
+Surprisingly, not all characters that occur in a \HINT\ file are inside the
+content section; some characters might hide in the definition section
+inside the pre- or post-break list of a predefined discretionary break.
+To make sure that the fonts necessary for these characters are included
+in the final \HINT\ file, we check these lists to make sure all \TeX\ font
+numbers have a corresponting \HINT\ font number.
+
+@<Hi\TeX\ auxiliar routines@>=
+static void ensure_font_no(pointer p)
+{ while (p!=null)
+ { if (is_char_node(p))
+ hget_font_no(font(p));
+ else if (type(p)==hlist_node||type(p)==vlist_node)
+ ensure_font_no(list_ptr(p));
+ p=link(p);
+ }
+}
+@
+
+\subsubsection{Output}
+@<Output font definitions@>=
+{ int f;
+ DBG(DBGDEF,"Defining %d fonts\n",max_ref[font_kind]+1);
+ for (f=0;f<=max_ref[font_kind];f++)
+ { font_t *hf=hfonts[f];
+ internal_font_number g=hf->i;
+ uint32_t pos=hpos-hstart;
+ info_t i= b000;
+ DBG(DBGDEF,"Defining font %d size 0x%x\n",f,font_size[g]);
+ hpos++; HPUTNODE; /* space for the tag and the node */
+ HPUT8(f); /* reference */
+ hout_string(font_id_text(g));
+ if(font_size[g]>0) HPUT32(font_size[g]);
+ else HPUT32(font_dsize[g]);
+ HPUT16(hf->m);HPUT16(hf->y);
+ DBG(DBGDEF,"Defining font space\n");
+ HPUTCONTENT(hout_glue_spec,hf->g);
+ DBG(DBGDEF,"Defining font hyphen\n");
+ HPUTCONTENT(hout_disc,hf->h);
+ hdef_font_params(hf->p);
+ DBG(DBGDEF,"End of font %d\n",f);
+ hput_tags(pos,TAG(font_kind,i));
+ }
+}
+@
+
+We used the following function to write a \TeX\ string to the \HINT\ file:
+@<Hi\TeX\ auxiliar routines@>=
+
+void hout_string(int s)
+{ pool_pointer j;
+ uint8_t c;
+ j= str_start[s];
+ while(j<str_start[s+1])
+ { c= so(str_pool[j++]);
+ if (c=='%' || c < 0x20 ||c >= 0x7F)
+ { char str[4];
+ snprintf(str,4,"%%%02X",c);/* convert to printable ASCII */
+ HPUTX(3);
+ HPUT8(str[0]); HPUT8(str[1]); HPUT8(str[2]);
+ }
+ else
+ { HPUTX(1);
+ HPUT8(c);
+ }
+ }
+ HPUT8(0);
+}
+@
+
+
+We used the following macro to add tags around the font glue and the font hyphen:
+
+
+@<Hi\TeX\ macros@>=
+
+#define HPUTCONTENT(F,D) \
+ { uint8_t *_p; \
+ uint8_t _t; \
+ HPUTNODE; /* allocate */ \
+ _p=hpos++; /* tag */ \
+ _t=F(D); \
+ *_p=_t; DBGTAG(_t,_p); \
+ DBGTAG(_t,hpos); HPUT8(_t); \
+ }
+@
+
+\subsection{Labels}
+The only label that must exist always is the zero label used
+to mark the ``home'' position of a document.
+
+\subsubsection{Initialization}
+We allocate the zero label with the first call to |next_label|
+and initialize is with the value from |label_defaults|.
+We then make sure it can be found under the name ``HINT.home''.
+
+@<Initialize definitions for labels@>=
+{ char nom[]="HINT.home";
+ unsigned int h=name_hash(nom)%LABEL_HASH;
+ int i=insert_hash(h,0,nom);
+ if (i!=zero_label_no)
+ QUIT("Trying to allocate the zero label, got %d",i);
+ labels[zero_label_no]=label_defaults[zero_label_no];
+ labels[zero_label_no].next=first_label;
+ first_label=zero_label_no;
+ DBG(DBGLABEL,"Defining zero label: pos=0x%x\n",labels[zero_label_no].pos);
+}
+@
+
+\subsection{Page Templates}
+
+Once we start producing content nodes, we update the maximum numbers
+of page templates and streams from |max_page| and |max_stream|.
+These values might have changed because the templates were stored in a
+format file.
+
+@<Fix definitions of page templates@>=
+max_ref[page_kind]=max_page;
+max_ref[stream_kind]=max_stream;
+@
+
+
+\subsubsection{Output}
+
+As part of a page template, we will see stream insertion nodes.
+When we encounter an |stream_node| inside a template definition,
+we output a stream insertion point.
+
+@<cases to output whatsit content nodes@>=
+ case stream_node:
+ HPUT8(setstream_number(p));
+ tag=TAG(stream_kind,b100);
+ break;
+@
+
+
+@<Output page template definitions@>=
+ DBG(DBGDEF,"Maximum page template reference: %d\n",max_page);
+ { pointer t;
+ for (t=link(setpage_head);t!=null;t=link(t))
+ { uint32_t pos=hpos-hstart;
+ DBG(DBGDEF,"Defining page template %d\n",setpage_number(i));@/
+ hpos++; HPUTNODE; /* space for the tag and the node */
+ HPUT8(setpage_number(t));
+ hout_string(setpage_name(t));
+ HPUT8(setpage_priority(t));
+ hout_glue_node(setpage_topskip(t));
+ hput_dimen(setpage_depth(t));
+ hout_xdimen_node(setpage_height(t));
+ hout_xdimen_node(setpage_width(t));
+ hout_list_node2(setpage_list(t));
+ @<output stream definitions@>@;
+ hput_tags(pos,TAG(page_kind,0));
+ }
+}
+@
+As part of the output of page template definitions, we output
+stream definitions:
+
+@<output stream definitions@>=
+{ pointer p,q;
+ p = setpage_streams(t);
+ while (p!=null)
+ { uint8_t n;
+ n=setstream_number(p);
+ DBG(DBGDEF,"Defining stream %d at " SIZE_F "\n",n,hpos-hstart);
+ HPUTTAG(stream_kind,b100);
+ HPUT8(n);
+ hout_xdimen_node(setstream_max(p)); /* maximum height */
+ HPUT16(setstream_mag(p)); /* factor */
+ HPUT8(setstream_prefered(p)); /* prefered */
+ HPUT8(setstream_next(p)); /* next */
+ HPUT16(setstream_ratio(p)); /* ratio */
+ q=setstream_before(p);setstream_before(p)=null;
+ hout_list_node2(q);flush_node_list(q);
+ hout_xdimen_node(setstream_width(p));
+ q=setstream_topskip(p);
+ hout_glue_node(q); delete_glue_ref(q);
+ q=setstream_after(p);setstream_after(p)=null;
+ hout_list_node2(q);flush_node_list(q);
+ q=setstream_height(p);
+ hout_glue_node(q); delete_glue_ref(q);
+ HPUTTAG(stream_kind,b100);
+ p=link(p);
+ }
+}
+@
+
+
+
+\section{\HINT\ Content}\label{output}
+\TeX\ puts content nodes on the contribution list and once in a while calls |build_page| to
+move nodes from the contribution list to the current page. Hi\TeX\ has a special version of
+|build_page| that will simply remove nodes from the contribution list and passe them
+to the function |hout_node|.
+
+\noindent
+@<Hi\TeX\ routines@>=
+static void hout_node(pointer p)
+{ uint32_t pos=hpos-hstart;
+ uint8_t tag;
+ HPUTNODE;
+ hpos++;
+ if(is_char_node(p))
+ @<output a character node@>@;
+ else switch(type(p)) @/
+ { @<cases to output content nodes@>@t\1@>@;
+ default:
+ MESSAGE("\nOutput of node type=%d subtype=%d not implemented\n", type(p),subtype(p));
+ display_node(p);
+ MESSAGE("End of node");
+ hpos--;
+ return;@t\2@>@;
+ }
+ hput_tags(pos,tag);
+}
+@
+@<Hi\TeX\ function declarations@>=
+static void hout_node(pointer p);
+@
+
+To output \HINT\ nodes, we use the functions defined in {\tt hput.c} from the {\tt shrink} program
+(see~\cite{MR:format}).
+
+
+Let's start with character nodes.
+
+\subsection{Characters}
+The processing of a character node consist of three steps: checking for definitions, converting the
+\TeX\ node pointed to by |p| to a \HINT\ data type, here a |glyph_t|, and using the
+corresponding {\tt hput\_\dots} function to output the node and return the |tag|.
+In the following, we will see the same approach in many
+small variations for all kinds of nodes.
+
+@<output a character node@>=
+{ glyph_t g;
+ g.f=hget_font_no(font(p));
+ g.c = character(p);
+ tag=hput_glyph(&g);
+}
+@
+
+\subsection{Penalties}
+Integer nodes, which as content nodes are used for penalties, come next.
+Except for the embedding between |case| and |break|, the
+processing of penalty nodes follows the same pattern we have just seen.
+ @<cases to output content nodes@>=
+ case penalty_node:
+ { int n,i;
+ i = penalty(p);
+ if (i>10000) i=10000;
+ else if (i<-10000) i=-10000;
+ n=hget_int_no(i);
+ if (n<0) tag=hput_int(i);
+ else { HPUT8(n); tag=TAG(penalty_kind,0);}
+ }
+ break;
+@
+
+\subsection{Kerns}
+The kern nodes of \TeX\ contain a single dimension and a flag to mark ``explicit'' kerns.
+ @<cases to output content nodes@>=
+ case kern_node:
+ { int n;
+ n=hget_dimen_no(width(p));
+ if (n<0)
+ { kern_t k;
+ k.x=(subtype(p)==explicit);
+ k.d.w=width(p);
+ k.d.h=k.d.v=0.0;
+ tag=hput_kern(&k);
+ }
+ else
+ { HPUT8(n);
+ if (subtype(p)==explicit) tag=TAG(kern_kind,b100); else tag=TAG(kern_kind,b000);
+ }
+ }
+ break;
+@
+
+\subsection{Extended Dimensions}
+Extended dimensions do not consitute content on their own, but nodes
+containing an extended dimension are part of other nodes. Here we
+define an auxiliar function that checks for a predefined extended
+dimension and if found outputs the reference number and returns false;
+otherwise it outputs the extended dimension and returns true.
+
+
+@<Hi\TeX\ auxiliar routines@>=
+void hout_xdimen_node(pointer p)
+{ xdimen_t x;
+ x.w=xdimen_width(p);
+ x.h=xdimen_hfactor(p)/(double)ONE;
+ x.v=xdimen_vfactor(p)/(double)ONE;
+ hput_xdimen_node(&x);
+}
+bool hout_xdimen(pointer p)
+{int n = hget_xdimen_no(p);
+ if (n>=0) { HPUT8(n); return false; @+}
+ else
+ { hout_xdimen_node(p); return true;@+}
+}
+
+@
+
+\subsection{Languages}
+We map the language numbers of \TeX\ to \HINT\ language numbers
+using the |hlanguage| array.
+@<Hi\TeX\ variables@>=
+static struct {
+ uint8_t n;
+ str_number s;
+} hlanguage[0x100];
+@
+
+For any language number of \TeX, the following function returns
+the corresponding \HINT\ language number.
+Since \TeX\ knowns about a maximum of 255 languages, there is
+no need for overflow checking. The next function writes a language
+node to the output stream.
+
+@<Hi\TeX\ auxiliar routines@>=
+uint8_t hget_language_no(uint8_t n)
+{ int i;
+ for (i=0;i<=max_ref[language_kind]; i++)
+ if (hlanguage[i].n==n) return i;
+ i=++max_ref[language_kind];
+ hlanguage[i].n=n;
+ hlanguage[i].s=0; /* language unknown*/
+ return i;
+}
+
+uint8_t hout_language(uint8_t n)
+{ n=hget_language_no(n);
+ if (n<7) return TAG(language_kind,n+1);
+ else
+ { HPUT8(n); return TAG(language_kind,0); }
+}
+@
+
+After these preparations, the output of a language node is
+simple:
+
+@<cases to output whatsit content nodes@>=
+case language_node:
+ tag=hout_language(what_lang(p));
+ break;
+@
+
+Normaly \TeX\ does not produce an initial language node and
+then the language in the \HINT\ file would not be known until
+it changes for the first time.
+
+@<insert an initial language node@>=
+{ uint32_t pos = hpos-hstart;
+ hpos++;
+ hput_tags(pos,hout_language(language));
+}
+@
+
+\TeX\ offers currently no simple way to obtain a standardized
+language identifier for the current language. So if the
+string number of the language is zero, we output the string |"unknown"|;
+if somehow the language is known, we output the corresponding string
+from \TeX's string pool.
+
+@<Output language definitions@>=
+ DBG(DBGDEF,"Maximum language reference: %d\n",max_ref[language_kind]);
+ for (i=max_fixed[language_kind]+1;i<=max_ref[language_kind]; i++)
+ { HPUTNODE;
+ HPUT8(TAG(language_kind,0));
+ HPUT8(i);
+ if (hlanguage[i].s==0)
+ hput_string("unknown");
+ else
+ hout_string(hlanguage[i].s);
+ HPUT8(TAG(language_kind,0));
+ }
+@
+
+
+
+\subsection{Mathematics}
+\TeX's math nodes have an optional width---a copy of the mathsurround parameter---while
+\HINT\ math nodes do not. Therefore we have to add an explicit kern node if
+the width is nonzero. We add it before a ``math on'' node or after a ``math off''
+to get the same behavior in respect to line breaking.
+
+@<cases to output content nodes@>=
+ case math_node:
+ { kern_t k;
+ k.x=true;
+ k.d.w=width(p);
+ k.d.h=k.d.v=0.0;
+ if (subtype(p)==before)
+ { tag=TAG(math_kind,b111);
+ if (width(p)!=0)
+ { hput_tags(pos,hput_kern(&k));
+ pos=hpos-hstart;
+ HPUTNODE;
+ hpos++;
+ }
+ }
+ else
+ { tag=TAG(math_kind,b011);
+ if (width(p)!=0)
+ { hput_tags(pos,tag);
+ pos=hpos-hstart;
+ HPUTNODE;
+ hpos++;
+ tag=hput_kern(&k);
+ }
+ }
+ }
+ break;
+@
+
+\subsection{Glue and Leaders}
+Because glue specifications and glue nodes are sometimes part of other
+nodes, we start with three auxiliar functions: The first simply
+converts a Hi\TeX\ glue node into a \HINT\ |glue_t|, outputs it and
+returns the tag; the second checks for predefined glues, and the third
+outputs a complete glue node including tags.
+@<Hi\TeX\ auxiliar routines@>=
+
+static uint8_t hout_glue_spec(pointer p)
+{ @+glue_t g;
+ to_glue_t(p,&g);
+ return hput_glue(&g);@+
+}
+
+static uint8_t hout_glue(pointer p)
+{ int n;
+ n = hget_glue_no(p);
+ if (n<0)
+ return hout_glue_spec(p);
+ else
+ {@+ HPUT8(n); return TAG(glue_kind,0);@+}
+}
+
+static void hout_glue_node(pointer p)
+{ uint8_t *pos;
+ uint8_t tag;
+ HPUTNODE; /* allocate */
+ pos=hpos;
+ hpos++; /* tag */
+ tag=hout_glue(p);
+ *pos=tag; DBGTAG(tag,pos);
+ DBGTAG(tag,hpos); HPUT8(tag);
+}
+@
+
+Since \TeX\ implements leaders as a kind of glue, we have one case statement covering glue and leaders.
+
+@<cases to output content nodes@>=
+ case glue_node:
+ if (subtype(p)< cond_math_glue) /* normal glue */
+ tag= hout_glue(glue_ptr(p));
+ else if (a_leaders<=subtype(p) && subtype(p)<=x_leaders) /*leaders */
+ { hout_glue_node(glue_ptr(p));
+ { bool outer_doing_leaders=doing_leaders;
+ doing_leaders=true;
+ hout_node(leader_ptr(p));
+ doing_leaders=outer_doing_leaders;
+ }
+ tag=TAG(leaders_kind,b100|(subtype(p)-a_leaders+1));
+ }
+ else
+ QUIT("glue subtype %d not implemented\n", subtype(p));
+ break;
+@
+
+\subsection{Discretionary breaks}
+Discretionary breaks are needed in font descriptions (see section~\secref{fontdef}).
+Therefore we define a function that converts \TeX's |disc_node| pointers
+to \HINT's |disc_t|, outputs the discretionary break, and returns the tag.
+
+@<Hi\TeX\ auxiliar routines@>=
+
+uint8_t hout_disc(pointer p)
+{ disc_t h;
+ h.x=!is_auto_disc(p);
+ h.r=replace_count(p);
+ if (h.x) h.r|=0x80;
+ if (h.r!=0) HPUT8(h.r);
+ if (pre_break(p)==null && post_break(p)==null)
+ h.p.s=h.q.s=0;
+ else
+ { uint32_t lpos;
+ lpos=hpos-hstart;
+ h.p.k=list_kind;
+ hout_list_node(pre_break(p),lpos,&(h.p));
+ if (post_break(p)==null)
+ h.q.s=0;
+ else
+ { uint32_t lpos;
+ lpos=hpos-hstart;
+ h.q.k=list_kind;
+ hout_list_node(post_break(p),lpos,&(h.q));
+ }
+ }
+ return hput_disc(&h);
+}
+
+@
+
+
+@<cases to output content nodes@>=
+ case disc_node:
+ { int n;
+ n=hget_disc_no(p);
+ if (n<0)
+ tag=hout_disc(p);
+ else { HPUT8(n); tag=TAG(disc_kind,0);}
+ }
+ break;
+@
+
+\subsection{Ligatures}
+The subtype giving information on left and right boundary characters
+is ignored since the \HINT\ viewer will not do ligature or kerning
+programms and neither attempt hyphenation.
+
+@<cases to output content nodes@>=
+ case ligature_node:
+ { lig_t l;
+ pointer q;
+ l.f=hget_font_no(font(lig_char(p)));
+ HPUT8(l.f);
+ l.l.p=hpos-hstart;
+ hput_utf8(qo(character(lig_char(p))));
+ q=lig_ptr(p);
+ while(q> null)
+ { hput_utf8(qo(character(q)));
+ q=link(q);
+ }
+ l.l.s=(hpos-hstart)-l.l.p;
+ tag=hput_ligature(&l);
+ }
+ break;
+@
+\subsection{Rules}
+@<cases to output content nodes@>=
+ case rule_node:
+ { rule_t r;
+ if (is_running(height(p))) r.h=RUNNING_DIMEN; else r.h=height(p);
+ if (is_running(depth(p))) r.d=RUNNING_DIMEN; else r.d=depth(p);
+ if (is_running(width(p))) r.w=RUNNING_DIMEN; else r.w=width(p);
+ tag=hput_rule(&r);
+ }
+ break;
+@
+
+\subsection{Boxes}
+@<cases to output content nodes@>=
+ case hlist_node:
+ case vlist_node:
+ if (type(p)==hlist_node) tag=TAG(hbox_kind,0);
+ else tag=TAG(vbox_kind,0);
+ tag |= hput_box_dimen(height(p),depth(p),width(p));
+ tag |= hput_box_shift(shift_amount(p));
+ tag |= hput_box_glue_set((glue_sign(p)==stretching)?+1:-1,glue_set(p),glue_order(p));
+ hout_list_node2(list_ptr(p));
+ break;
+ @
+
+\subsection{Adjustments}
+@<cases to output content nodes@>=
+ case adjust_node:
+ hout_list_node2(adjust_ptr(p));
+ tag=TAG(adjust_kind,1);
+ break;
+@
+
+
+\subsection{Insertions}
+\TeX's insertions are mapped to \HINT\ streams.
+
+@<cases to output content nodes@>=
+case ins_node:
+@<output stream content@>@;
+break;
+@
+
+Here we consider stream content and come back to stream
+definitions in section~\secref{streamdef}.
+In a \HINT\ stream content node the stream
+parameters |floating_penalty|, |split_max_depth|, and
+|split_top_skip| are optional. If omitted, the defaults from the stream definition are
+used. This is probably also for \TeX\ the most common situation.
+It is, however, possible to supply more than one page template with different defaults
+and while not very common, \TeX\ might change the parameters at any time.
+Because we dont know which is the current page template,
+it is not posible to compare the current parameter values against the defaults,
+and we have to supply all the parameters always.
+In a future version, we might have a \TeX\ primitive that allows us to
+signal ``use the defaults''.
+
+@<output stream content@>=
+{ int k,n;
+ uint32_t pos;
+ list_t l;
+ info_t i=b000;
+ k=subtype(p);
+ n=hget_stream_no(k);
+ HPUT8(n);
+ link(temp_head)=null;
+ add_par_node(int_type,floating_penalty_code,float_cost(p));
+ add_par_node(dimen_type,split_max_depth_code,depth(p));
+ add_par_node(glue_type,split_top_skip_code,split_top_ptr(p));
+ pos=hpos-hstart;
+ l.k=param_kind;
+ n=hout_param_list(link(temp_head),pos,&l);
+ flush_node_list(link(temp_head));@+ link(temp_head)=null;
+ if (n>=0) HPUT8(n); else i=b010;
+ hout_list_node2(ins_ptr(p));
+ tag=TAG(stream_kind,i);
+}
+@
+
+
+\subsection{Marks}
+We currently ignore Marks.
+
+@<cases to output content nodes@>=
+ case mark_node: hpos--; return;
+@
+
+\subsection{Whatsit Nodes}\label{whatsit}
+We have added custom whatsit nodes and now we switch based on the subtype.
+
+@<cases to output content nodes@>=
+ case whatsit_node:
+ switch(subtype(p))
+ { @<cases to output whatsit content nodes@>@;
+ default:
+ MESSAGE("\nOutput of whatsit nodes subtype=%d not implemented\n", subtype(p));
+ display_node(p);
+ MESSAGE("End of node");
+ hpos--;
+ return;
+ }
+ break;
+@
+
+For \TeX's whatsit nodes that handle output files, no code is generated;
+hence, we call |out_what| and simply remove the tag byte that is already
+in the output.
+When the \.{\\write} node arrives here, it is normaly handled
+in |hlist_out| or |vlist_out| in an environment determined by
+the output routine. For example \LaTeX\ redefines \.{\\protect}
+as \.{\\noexpand} and these redefinitions need to be made
+before calling |out_what| which expands the token list.
+We should therefore add the definitions contained in the output routine
+to mimic expanding inside an output routine.
+
+
+@<cases to output whatsit content nodes@>=
+ case open_node:
+ case write_node:
+ case close_node: out_what(p);
+ case special_node: hpos--; return;
+@
+
+\subsection{Paragraphs}
+When we output a paragraph node, we have to consider a special case:
+The parameter list is given by a reference number but the extended dimension
+needs an |xdimen| node. In this case the reference number for the parameter
+list comes first, while otherwise the extended dimension would come first.
+To determine whether ther is a reference number for the parameter list,
+the function |hout_param_list| is writing the parameter list to the output.
+\noindent
+@<cases to output whatsit content nodes@>=
+case graf_node:
+ { uint32_t pos, xpos, xsize;
+ list_t l;
+ pointer q;
+ int n,m;
+ info_t i=b000;
+ q=graf_extent(p);
+ n=hget_xdimen_no(q);
+ if (n>=0) HPUT8(n);
+ else
+ { xpos=hpos-hstart; hout_xdimen_node(p); xsize=(hpos-hstart)-xpos; i|=b100; }
+ pos=hpos-hstart;
+ l.k=param_kind;
+ m=hout_param_list(graf_params(p),pos,&l);
+ if (m>=0)
+ { if (i&b100)
+ { HPUTX(1);
+ memmove(hstart+xpos+1,hstart+xpos,xsize);
+ hpos++;
+ hstart[xpos]=m;
+ }
+ else
+ HPUT8(m);
+ }
+ else i|=b010;
+ pos=hpos-hstart;
+ l.k=list_kind;
+ hout_list_node(graf_list(p),pos,&l);
+ tag=TAG(par_kind,i);
+ }
+ break;
+@
+
+\subsection{Baseline Skips}
+@<cases to output whatsit content nodes@>=
+ case baseline_node:
+ { int n;
+ n= baseline_node_no(p);
+ if (n>0xFF) tag=hout_baselinespec(n);
+ else
+ { HPUT8(n);
+ tag=TAG(baseline_kind,b000);
+ }
+ }
+ break;
+ @
+
+\subsection{Displayed Equations}
+@<cases to output whatsit content nodes@>=
+ case disp_node:
+ { uint32_t pos;
+ list_t l;
+ int n;
+ info_t i=b000;
+ pos=hpos-hstart;
+ l.k=param_kind;
+ n=hout_param_list(display_params(p),pos,&l);
+ if (n>=0) HPUT8(n); else i|=b100;
+ if (display_eqno(p)!=null && display_left(p))
+ { hout_node(display_eqno(p)); i|=b010; }
+ pos=hpos-hstart;
+ l.k=list_kind;
+ hout_list_node(display_formula(p),pos,&l);
+ if (display_eqno(p)!=null && !display_left(p))
+ { hout_node(display_eqno(p)); i|=b001; }
+ tag=TAG(math_kind,i);
+ /* the |display_no_bs(p)| tells whether the baseline skip is ignored */
+ }
+ break;
+@
+\subsection{Extended Boxes}
+When we output an extended box, we have to consider a special case: the page templates.
+Page templates are boxes that contain insertion points. These insertion points look
+like regular insertions but with an empty content list. As a result the |hpack| and
+|vpackage| routines might belive that they can compute the dimensions of the box
+conntent when in fact they can not.
+
+
+@<cases to output whatsit content nodes@>=
+ case hset_node:
+ case vset_node:
+ { kind_t k= subtype(p)==hset_node?hset_kind:vset_kind;
+ info_t i=b000;
+ stretch_t s;
+ int n=set_extent(p);
+ i|=hput_box_dimen(height(p),depth(p),width(p));
+ i|=hput_box_shift(shift_amount(p));
+ s.f=set_stretch(p)/(double)ONE; s.o=set_stretch_order(p);
+ hput_stretch(&s);
+ s.f=set_shrink(p)/(double)ONE; s.o=set_shrink_order(p);
+ hput_stretch(&s);
+ if (hout_xdimen(n)) i|=b001;
+ hout_list_node2(list_ptr(p));
+ tag=TAG(k,i);
+ }
+ break;
+@
+
+@<cases to output whatsit content nodes@>=
+ case hpack_node:
+ case vpack_node:
+ { kind_t k= (subtype(p)==hpack_node?hpack_kind:vpack_kind);
+ info_t i=b000;
+ int n=pack_extent(p);
+ if (pack_m(p)==additional) i|=b001;
+ if (shift_amount(p)!=0) { HPUT32(shift_amount(p)); i|=b010; }
+ if (k==vpack_kind) HPUT32(pack_limit(p));
+ if (hout_xdimen(n)) i|=b100;
+ hout_list_node2(list_ptr(p));
+ tag=TAG(k,i);
+ }
+ break;
+@
+
+\subsection{Extended Alignments}
+@<cases to output whatsit content nodes@>=
+case align_node:
+ { info_t i=b000;
+ if (align_m(p)==additional) i|=b001;
+ if (align_v(p)) i|=b010;
+ if (hout_xdimen(align_extent(p))) i|=b100;
+ hout_preamble(align_preamble(p));
+ hout_align_list(align_list(p),align_v(p));
+ tag=TAG(table_kind,i);
+ }
+break;
+@
+
+In the preamble, we remove the unset nodes and retain only the list of tabskip glues.
+@<Hi\TeX\ auxiliar routines@>=
+void hout_preamble(pointer p)
+{ pointer q,r;
+ DBG(DBGBASIC,"Writing Preamble\n");
+ q=p;
+ if (q!=null) r=link(q); else r=null;
+ while (r!=null)
+ { if (type(r)==unset_node)
+ { link(q)=link(r);
+ link(r)=null; flush_node_list(r);
+ }
+ else
+ q=r;
+ r=link(q);
+ }
+ hout_list_node2(p);
+ DBG(DBGBASIC,"End Preamble\n");
+}
+@
+
+In the |align_list| we have to convert the unset nodes back to box nodes or extended box nodes
+packaged inside an item node.
+When the viewer reads an item node, it will package the extended boxes to their natural size.
+This is the size that is needed to compute the maximum width of a column.
+
+@<Hi\TeX\ auxiliar routines@>=
+
+static void hout_item(pointer p, uint8_t t, uint8_t s)
+{ info_t i=b000;
+ uint8_t n;
+ n=span_count(p)+1;
+ DBG(DBGBASIC,"Writing Item %d/%d->%d/%d\n",type(p),n,t,s);
+ display_node(p);
+ if (n==0) QUIT("Span count of item must be positive");
+ if (n<7) i=n; else i=7;
+ HPUTTAG(item_kind,i);
+ if (i==7) HPUT8(n);
+ type(p)=t; subtype(p)=s;
+ hout_node(p);
+ HPUTTAG(item_kind,i);
+ DBG(DBGBASIC,"End Item\n");
+}
+
+
+static void hout_item_list(pointer p, bool v)
+{ list_t l;
+ uint32_t pos;
+ DBG(DBGBASIC,"Writing Item List\n");
+ l.k=list_kind;
+ HPUTTAG(item_kind,b000);
+ pos=hpos-hstart;
+ HPUTX(2);
+ HPUT8(0); /* space for the list tag */
+ HPUT8(0); /* space for the list size */
+ l.p=hpos-hstart;
+ while(p> mem_min)
+ { if (is_char_node(p)) hout_node(p);
+ else if (type(p)==unset_node) hout_item(p,v?vlist_node:hlist_node,0);
+ else if (type(p)==unset_set_node) hout_item(p,whatsit_node,v?vset_node:hset_node);
+ else if (type(p)==unset_pack_node) hout_item(p,whatsit_node,v?vpack_node:hpack_node);
+ else hout_node(p);
+ p=link(p);
+ }
+ l.s=(hpos-hstart)-l.p;
+ hput_tags(pos,hput_list(pos+1,&l));
+ HPUTTAG(item_kind,b000);
+ DBG(DBGBASIC,"End Item List\n");
+}
+
+void hout_align_list(pointer p, bool v)
+{ list_t l;
+ uint32_t pos;
+ DBG(DBGBASIC,"Writing Align List\n");
+ l.k=list_kind;
+ pos=hpos-hstart;
+ HPUTX(2);
+ HPUT8(0); /* space for the tag */
+ HPUT8(0); /* space for the list size */
+ l.p=pos+2;
+ while(p> mem_min)
+ { if (!is_char_node(p) && (type(p)==unset_node||type(p)==unset_set_node||type(p)==unset_pack_node))
+ hout_item_list(list_ptr(p),v);
+ else
+ hout_node(p);
+ p=link(p);
+ }
+ l.s=(hpos-hstart)-l.p;
+ hput_tags(pos,hput_list(pos+1,&l));
+ DBG(DBGBASIC,"End Align List\n");
+}
+@
+
+Inside the alignment list we will find various types of unset nodes, we convert them back
+to regular nodes and put them inside an item node.
+
+@<cases to output content nodes@>=
+case unset_node:
+case unset_set_node:
+case unset_pack_node:
+
+
+@
+
+\subsection{Lists}
+Two functions are provided here: |hout_list| will write a list given by the pointer |p| to the
+output at the current position |hpos|. After the list has finished, it will move the list, if necessary,
+and add the size information so that the final list will be at position |pos|;
+|hout_list_node| uses |hout_list| but adds the tags to form a complete node.
+
+
+@<Hi\TeX\ routines@>=
+
+static uint8_t hout_list(pointer p, uint32_t pos, list_t *l)
+{ l->p=hpos-hstart;
+ while(p> mem_min)
+ { hout_node(p);
+ p=link(p);
+ }
+ l->s=(hpos-hstart)-l->p;
+ return hput_list(pos,l);
+}
+
+static void hout_list_node(pointer p, uint32_t pos, list_t *l)
+/* p is a pointer to a node list,
+ output the node list at position pos (thats where the tag goes)
+ using l->k as list kind,
+ and set l->p and l->s.
+*/
+{
+ hpos=hstart+pos;
+ HPUTX(3);
+ HPUT8(0); /* space for the tag */
+ HPUT8(0); /* space for the list size */
+ HPUT8(0); /* space for the size boundary byte */
+ hput_tags(pos,hout_list(p,pos+1,l));
+}
+
+
+static void hout_list_node2(pointer p)
+{ list_t l;
+ uint32_t pos;
+ pos=hpos-hstart;
+ l.k=list_kind;
+ hout_list_node(p,pos,&l);
+}
+@
+
+@<Hi\TeX\ function declarations@>=
+static void hout_list_node(pointer p, uint32_t pos, list_t *l);
+static void hout_list_node2(pointer p);
+static uint8_t hout_list(pointer p, uint32_t pos, list_t *l);
+@
+
+\subsection{Parameter Lists}
+
+The next function is like |hout_list_node| but restricted to parameter nodes.
+
+@<Hi\TeX\ routines@>=
+
+static int hout_param_list(pointer p, uint32_t pos, list_t *l)
+/* p is a pointer to a param node list,
+ either find a reference number to a predefined parameter list
+ and return the reference number or
+ output the node list at position pos (thats where the tag goes)
+ and set l->k, l->p and l->s,
+ and return -1;
+*/
+{ int n;
+ hpos=hstart+pos;
+ if (p==null)
+ {HPUTX(2); hpos++;hput_tags(pos,TAG(param_kind,1)); l->s=0; return -1;}
+ HPUTX(3);
+ HPUT8(0); /* space for the tag */
+ HPUT8(0); /* space for the list size */
+ HPUT8(0); /* space for the size boundary byte*/
+ l->p=hpos-hstart;
+ while(p> mem_min)
+ { hdef_param_node(par_type(p),par_number(p),par_value(p).i);
+ p=link(p);
+ }
+ l->s=(hpos-hstart)-l->p;
+ n=hget_param_list_no(l);
+ if (n>=0)
+ hpos=hstart+pos;
+ else
+ hput_tags(pos,hput_list(pos+1,l));
+ return n;
+}
+@
+
+@<Hi\TeX\ function declarations@>=
+static int hout_param_list(pointer p, uint32_t pos, list_t *l);
+@
+
+\subsection{Labels, Links, and Outlines}
+\indent
+@<cases to output whatsit content nodes@>=
+case label_node: hpos--; new_label(p); return;
+case start_link_node:
+{ info_t i;
+ int n=new_start_link(p);
+ i=b010;
+ if (n>0xFF) { i|=b001; HPUT16(n);@+} @+else HPUT8(n);
+ tag= TAG(link_kind,i);
+}
+break;
+case end_link_node:
+{ info_t i;
+ int n=new_end_link();
+ i=b000;
+ if (n>0xFF) { i|=b001; HPUT16(n);@+} @+else HPUT8(n);
+ tag= TAG(link_kind,i);
+}
+break;
+case outline_node: hpos--; new_outline(p); return;
+@
+
+The routines to put labels and outlines into the definition section
+are defined in {\tt hput.c}.
+
+\subsection{Images}
+\indent
+@<cases to output whatsit content nodes@>=
+ case image_node:
+ { image_t i;
+ i.n=image_no(p);
+ i.w=image_width(p);
+ i.h=image_height(p);
+ i.p.f=image_stretch(p)/(double)ONE;
+ i.p.o=image_stretch_order(p);
+ i.m.f=image_shrink(p)/(double)ONE;
+ i.m.o=image_shrink_order(p);
+ tag=hput_image(&i);
+ }
+ break;
+@
+
+
+
+\subsection{Text}
+The routines in this section are not yet ready.
+
+@<Hi\TeX\ routines@>=
+
+#if 0
+static void hchange_text_font(internal_font_number f)
+{ uint8_t g;
+ if (f!=hfont)
+ { g=get_font_no(f);
+ if (g<8)
+ hputcc(FONT0_CHAR+g);
+ else
+ { hputcc(FONTN_CHAR);
+ hputcc(g);
+ }
+ hfont=f;
+ }
+}
+
+static void hprint_text_char(pointer p)
+{ uint8_t f,c;
+ f = font(p);
+ c = character(p);
+ hchange_text_font(f);
+ if (c<=SPACE_CHAR) hputcc(ESC_CHAR);
+ hputcc(c);
+}
+
+
+static void hprint_text_node(pointer p)
+{ switch(type(p))
+ { case hlist_node:
+ /* this used to be the |par_indent| case */
+ goto nodex;
+ case glue_node:
+ if (subtype(p)>= cond_math_glue) goto nodex;
+ else
+ { pointer q=glue_ptr(p);
+ int i;
+ if (glue_equal(f_space_glue[hfont],q))
+ { hputc(SPACE_CHAR); return; }
+ if (glue_equal(f_xspace_glue[hfont],q))
+ { hputcc(XSPACE_CHAR); return; }
+ if (f_1_glue[hfont]==0 && (subtype(p)-1==space_skip_code))
+ { pointer r=glue_par(subtype(p)-1);
+ add_glue_ref(r);
+ f_1_glue[hfont]=r;
+ }
+ if (f_1_glue[hfont]!=0 && glue_equal(f_1_glue[hfont],q))
+ { hputcc(GLUE1_CHAR); return; }
+ if (f_2_glue[hfont]==0 && (subtype(p)-1==space_skip_code || subtype(p)-1==xspace_skip_code))
+ { pointer r=glue_par(subtype(p)-1);
+ add_glue_ref(r);
+ f_2_glue[hfont]=r;
+ }
+ if (f_2_glue[hfont]!=0 && glue_equal(f_2_glue[hfont],q))
+ { hputcc(GLUE2_CHAR); return; }
+ if (f_3_glue[hfont]==0)
+ { f_3_glue[hfont]=q;
+ add_glue_ref(q);
+ }
+ if (f_3_glue[hfont]!=0 && glue_equal(f_3_glue[hfont],q))
+ { hputcc(GLUE3_CHAR); return; }
+ i = hget_glue_no(q);
+ if (i>=0)
+ { hputcc(GLUEN_CHAR); hputcc(i); return; }
+ }
+ break;
+ case ligature_node:
+ { int n;
+ pointer q;
+ for (n=0,q=lig_ptr(p); n<5 && q!=null; n++,q=link(q)) continue;
+ if (n==2) hputcc(LIG2_CHAR);
+ else if (n==3) hputcc(LIG3_CHAR);
+ else if (n==0) hputcc(LIG0_CHAR);
+ else goto nodex;
+ hprint_text_char(lig_char(p));
+ for (q=lig_ptr(p);q!=null; q=link(q)) hprint_text_char(q);
+ return;
+ }
+ case disc_node:
+ if (post_break(p)==null && pre_break(p)!=null && replace_count(p)==0)
+ { pointer q;
+ q=pre_break(p);
+ if (is_char_node(q) && link(q)==null && font(q)==hfont && character(q)== hyphen_char[hfont])
+ { if (is_auto_disc(p)) hputcc(DISC1_CHAR);
+ else hputcc(DISC2_CHAR);
+ return;
+ }
+ }
+ else if (post_break(p)==null && pre_break(p)==null && replace_count(p)==0 && !is_auto_disc(p))
+ { hputcc(DISC3_CHAR); return; }
+ break;
+ case math_node:
+ if(width(p)!=0) goto nodex;
+ if(subtype(p)==before) hputcc(MATHON_CHAR);
+ else hputcc(MATHOFF_CHAR);
+ return;
+ default:
+ break;
+ }
+nodex:
+ hout_node(p);
+}
+
+static void hprint_text(pointer p)
+{ internal_font_number f=hfont;
+ nesting++;
+ hprint_nesting();
+ hprintf("<text ");
+ while(p> mem_min)
+ { if (is_char_node(p))
+ hprint_text_char(p);
+ else
+ hprint_text_node(p);
+ p=link(p);
+ }
+ hchange_text_font(f);
+ hprintf(">\n");
+ nesting--;
+}
+#endif
+
+@
+
+
+\section{\HINT\ Limitations}
+\itemize
+\item Kerns and glues using a width that depends on \.{\\hsize} or
+ \.{\\vsize} are not yet supported.
+\item Tables where the width of a column depends on \.{\\hsize} or
+ \.{\\vsize} are not tested and probably not yet supported.
+\item The encoding of horizontal lists as texts is not yet supported,
+ but it would make the \HINT\ file shorter and much better to read
+ when stretched into long \HINT\ format.
+\enditemize
+
+
+\appendix
+
+\section{Source Files}
+
+\subsection{Basic types}
+To define basic types we use the file {\tt bastypes.h} from~\cite{MR:format}.
+
+\subsection{Hi\TeX\ routines: \tt hitex.c}
+\indent
+ at p
+#include <stdio.h>
+#include <kpathsea/kpathsea.h>
+#include "basetypes.h"
+#include "error.h"
+#include "hformat.h"
+#include "hput.h"
+#include "htex.h"
+#include "hitex.h"
+
+/* from ktex.ch */
+
+@<Hi\TeX\ macros@>@;
+@<Hi\TeX\ variables@>@;
+@<Hi\TeX\ function declarations@>@;
+
+@<Hi\TeX\ auxiliar routines@>@;
+
+@<Hi\TeX\ routines@>@;
+
+@
+
+
+\subsection{Hi\TeX\ prototypes: \tt hitex.h}
+@(hitex.h@>=
+extern void hint_open(void);
+extern void hint_close(void);@#
+
+extern pointer new_xdimen(scaled w, scaled h, scaled v);
+extern void hget_image_information(pointer p);
+extern pointer new_baseline_node(pointer bs, pointer ls, scaled lsl);
+extern pointer new_set_node(void);
+extern void hline_break(int final_widow_penalty);
+extern pointer new_disp_node(void);
+extern void add_par_node(eight_bits t, eight_bits n, int v);
+extern pointer new_image_node(str_number n, char *a, char *e);
+extern pointer new_setpage_node(eight_bits k, str_number n);
+extern int hget_stream_no(int i);
+extern pointer new_setstream_node(eight_bits n);
+extern void hfinish_stream_group(void);
+extern void hfinish_page_group(void);
+extern void hfinish_stream_before_group(void);
+extern void hfinish_stream_after_group(void);
+extern void hfinish_outline_group(void);
+extern void execute_output(pointer p);
+extern void hint_debug_help(void);
+@
+
+\crosssections
+
+\plainsection{References}
+
+{\baselineskip=11pt
+\def\bfblrm{\small\rm}%
+\def\bblem{\small\it}%
+\bibliography{../hint}
+\bibliographystyle{plain}
+}
+
+\plainsection{Index}
+{
+\def\_{{\tt \UL}} % underline in a string
+\catcode`\_=\active \let_=\_ % underline is a letter
+\input hitex.ind
+}
+
+ \write\cont{} % ensure that the contents file isn't empty
+% \write\cont{\catcode `\noexpand\@=12\relax} % \makeatother
+ \closeout\cont% the contents information has been fully gathered
Property changes on: trunk/Build/source/texk/web2c/hitexdir/hitex.w
___________________________________________________________________
Added: svn:eol-style
## -0,0 +1 ##
+native
\ No newline at end of property
Added: trunk/Build/source/texk/web2c/hitexdir/htex.w
===================================================================
--- trunk/Build/source/texk/web2c/hitexdir/htex.w (rev 0)
+++ trunk/Build/source/texk/web2c/hitexdir/htex.w 2021-09-21 16:46:46 UTC (rev 60569)
@@ -0,0 +1,29180 @@
+% This file is part of HINT
+% Copyright 2017-2021 Martin Ruckert
+%
+% Permission is hereby granted, free of charge, to any person obtaining a copy
+% of this software and associated documentation files (the "Software"), to deal
+% in the Software without restriction, including without limitation the rights
+% to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+% copies of the Software, and to permit persons to whom the Software is
+% furnished to do so, subject to the following conditions:
+%
+% The above copyright notice and this permission notice shall be
+% included in all copies or substantial portions of the Software.
+%
+% THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+% IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+% FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+% COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+% WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT
+% OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+% THE SOFTWARE.
+%
+% Except as contained in this notice, the name of the copyright holders shall
+% not be used in advertising or otherwise to promote the sale, use or other
+% dealings in this Software without prior written authorization from the
+% copyright holders.
+
+% The HINT program is derived from Donald E. Knuth's TeX and the
+% subsequent e-TeX extension of TeX.
+
+% e-TeX is copyright (C) 1999-2012 by P. Breitenlohner (1994,98 by the NTS
+% team); all rights are reserved. Copying of this file is authorized only if
+% (1) you are P. Breitenlohner, or if (2) you make absolutely no changes to
+% your copy. (Programs such as TIE allow the application of several change
+% files to tex.web; the master files tex.web and etex.ch should stay intact.)
+
+% See etex_gen.tex for hints on how to install this program.
+% And see etripman.tex for details about how to validate it.
+
+% This program is directly derived from Donald E. Knuth's TeX;
+% the change history which follows and the reward offered for finders of
+% bugs refer specifically to TeX; they should not be taken as referring
+% to e-TeX, although the change history is relevant in that it
+% demonstrates the evolutionary path followed. This program is not TeX;
+% that name is reserved strictly for the program which is the creation
+% and sole responsibility of Professor Knuth.
+
+% Version 0 was released in September 1982 after it passed a variety of tests.
+% Version 1 was released in November 1983 after thorough testing.
+% Version 1.1 fixed ``disappearing font identifiers'' et alia (July 1984).
+% Version 1.2 allowed `0' in response to an error, et alia (October 1984).
+% Version 1.3 made memory allocation more flexible and local (November 1984).
+% Version 1.4 fixed accents right after line breaks, et alia (April 1985).
+% Version 1.5 fixed \the\toks after other expansion in \edefs (August 1985).
+% Version 2.0 (almost identical to 1.5) corresponds to "Volume B" (April 1986).
+% Version 2.1 corrected anomalies in discretionary breaks (January 1987).
+% Version 2.2 corrected "(Please type...)" with null \endlinechar (April 1987).
+% Version 2.3 avoided incomplete page in premature termination (August 1987).
+% Version 2.4 fixed \noaligned rules in indented displays (August 1987).
+% Version 2.5 saved cur_order when expanding tokens (September 1987).
+% Version 2.6 added 10sp slop when shipping leaders (November 1987).
+% Version 2.7 improved rounding of negative-width characters (November 1987).
+% Version 2.8 fixed weird bug if no \patterns are used (December 1987).
+% Version 2.9 made \csname\endcsname's "relax" local (December 1987).
+% Version 2.91 fixed \outer\def\a0{}\a\a bug (April 1988).
+% Version 2.92 fixed \patterns, also file names with complex macros (May 1988).
+% Version 2.93 fixed negative halving in allocator when mem_min<0 (June 1988).
+% Version 2.94 kept open_log_file from calling fatal_error (November 1988).
+% Version 2.95 solved that problem a better way (December 1988).
+% Version 2.96 corrected bug in "Infinite shrinkage" recovery (January 1989).
+% Version 2.97 corrected blunder in creating 2.95 (February 1989).
+% Version 2.98 omitted save_for_after at outer level (March 1989).
+% Version 2.99 caught $$\begingroup\halign..$$ (June 1989).
+% Version 2.991 caught .5\ifdim.6... (June 1989).
+% Version 2.992 introduced major changes for 8-bit extensions (September 1989).
+% Version 2.993 fixed a save_stack synchronization bug et alia (December 1989).
+% Version 3.0 fixed unusual displays; was more \output robust (March 1990).
+% Version 3.1 fixed nullfont, disabled \write{\the\prevgraf} (September 1990).
+% Version 3.14 fixed unprintable font names and corrected typos (March 1991).
+% Version 3.141 more of same; reconstituted ligatures better (March 1992).
+% Version 3.1415 preserved nonexplicit kerns, tidied up (February 1993).
+% Version 3.14159 allowed fontmemsize to change; bulletproofing (March 1995).
+% Version 3.141592 fixed \xleaders, glueset, weird alignments (December 2002).
+% Version 3.1415926 was a general cleanup with minor fixes (February 2008).
+% Version 3.14159265 was similar (January 2014).
+
+% A reward of $327.68 will be paid to the first finder of any remaining bug.
+
+% A preliminary version of TeX--XeT was released in April 1992.
+% TeX--XeT version 1.0 was released in June 1992,
+% version 1.1 prevented arith overflow in glue computation (Oct 1992).
+% A preliminary e-TeX version 0.95 was operational in March 1994.
+% Version 1.0beta was released in May 1995.
+% Version 1.01beta fixed bugs in just_copy and every_eof (December 1995).
+% Version 1.02beta allowed 256 mark classes (March 1996).
+% Version 1.1 changed \group{type,level} -> \currentgroup{type,level},
+% first public release (October 1996).
+% Version 2.0 development was started in March 1997;
+% fixed a ligature-\beginR bug in January 1998;
+% was released in March 1998.
+% Version 2.1 fixed a \marks bug (when min_halfword<>0) (January 1999).
+% Version 2.2 development was started in Feb 2003; released in Oct 2004.
+% fixed a bug in sparse array handling (0=>null), Jun 2002;
+% fixed a bug in \lastnodetype (cur_val=>cur_val_level)
+% reported by Hartmut Henkel <hartmut_henkel@@gmx.de>,
+% fix by Fabrice Popineau <Fabrice.Popineau@@supelec.fr>,
+% Jan 2004;
+% another bug in sparse array handling (cur_ptr=>cur_chr)
+% reported by Taco Hoekwater <taco@@elvenkind.com>, Jul 2004;
+% fixed a sparse array reference count bug (\let,\futurelet),
+% fix by Bernd Raichle <berd@@dante.de>, Aug 2004;
+% reorganized handling of banner, additional token list and
+% integer parameters, and similar in order to reduce the
+% interference between eTeX, pdfTeX, and web2c change files.
+% adapted to tex.web 3.141592, revised glue rounding for mixed
+% direction typesetting;
+% fixed a bug in the revised glue rounding code, detected by
+% Tigran Aivazian <tigran@@aivazian.fsnet.co.uk>, Oct 2004.
+% Version 2.3 development was started in Feb 2008; released in Apr 2011.
+% fixed a bug in hyph_code handling (\savinghyphcodes)
+% reported by Vladimir Volovich <vvv@@vsu.ru>, Feb 2008.
+% fixed the error messages for improper use of \protected,
+% reported by Heiko Oberdiek
+% <heiko.oberdiek@@googlemail.com>, May 2010.
+% some rearrangements to reduce interferences between
+% e-TeX and pTeX, in part suggested by Hironori Kitagawa
+% <h_kitagawa2001@@yahoo.co.jp>, Mar 2011.
+% Version 2.4 fixed an uninitialized line number bug, released in May 2012.
+% Version 2.5 development was started in Aug 2012; released in Feb 2013.
+% better tracing of font definitions, reported by
+% Bruno Le Floch <blflatex@@gmail.com>, Jul 2012.
+% Version 2.6 development was started in Mar 2013; released in ??? 201?.
+% enable hyphenation of text between \beginL and \endL or
+% between \beginR and \endR, problem reported by
+% Vafa Khalighi <vafalgk@@gmail.com>, Nov 2013.
+% better handling of right-to-left text -- to be done.
+
+% Although considerable effort has been expended to make the e-TeX program
+% correct and reliable, no warranty is implied; the author disclaims any
+% obligation or liability for damages, including but not limited to
+% special, indirect, or consequential damages arising out of or in
+% connection with the use or performance of this software. This work has
+% been a ``labor of love'' and the author hopes that users enjoy it.
+
+% Here is TeX material that gets inserted after \input webmac
+\def\hang{\hangindent 3em\noindent\ignorespaces}
+\def\hangg#1 {\hang\hbox{#1 }}
+\def\textindent#1{\hangindent2.5em\noindent\hbox to2.5em{\hss#1 }\ignorespaces}
+\font\ninerm=cmr9
+\let\mc=\ninerm % medium caps for names like SAIL
+\def\eTeX{$\varepsilon$-\TeX}
+\font\revrm=xbmc10 % for right-to-left text
+% to generate xbmc10 (i.e., reflected cmbx10) use a file
+% xbmc10.mf containing:
+%+++++++++++++++++++++++++++++++++++++++++++++++++
+% if unknown cmbase: input cmbase fi
+% extra_endchar := extra_endchar &
+% "currentpicture:=currentpicture " &
+% "reflectedabout((.5[l,r],0),(.5[l,r],1));";
+% input cmbx10
+%+++++++++++++++++++++++++++++++++++++++++++++++++
+\ifx\beginL\undefined % this is TeX
+ \def\XeT{X\kern-.125em\lower.5ex\hbox{E}\kern-.1667emT}
+ \def\TeXeT{\TeX-\hbox{\revrm \XeT}} % for TeX-XeT
+ \def\TeXXeT{\TeX-\hbox{\revrm -\XeT}} % for TeX--XeT
+\else
+ \ifx\eTeXversion\undefined % this is \TeXeT
+ \def\TeXeT{\TeX-{\revrm\beginR\TeX\endR}} % for TeX-XeT
+ \def\TeXXeT{\TeX-{\revrm\beginR\TeX-\endR}} % for TeX--XeT
+ \else % this is \eTeX
+ \def\TeXeT{\TeX-{\TeXXeTstate=1\revrm\beginR\TeX\endR}} % for TeX-XeT
+ \def\TeXXeT{\TeX-{\TeXXeTstate=1\revrm\beginR\TeX-\endR}} % for TeX--XeT
+ \fi
+\fi
+\def\PASCAL{Pascal}
+\def\ph{\hbox{Pascal-H}}
+\def\pct!{{\char`\%}} % percent sign in ordinary text
+\def\grp{\.{\char'173...\char'175}}
+\font\logo=logo10 % font used for the METAFONT logo
+\def\MF{{\logo META}\-{\logo FONT}}
+\def\<#1>{$\langle#1\rangle$}
+\def\section{\mathhexbox278}
+
+\def\(#1){} % this is used to make section names sort themselves better
+\def\9#1{} % this is used for sort keys in the index via @@:sort key}{entry@@>
+
+\let\@@=\relax % we want to be able to \write a \?
+
+\def\title{k\TeX}
+% system dependent redefinitions of \title should come later
+% and should use:
+% \toks0=\expandafter{\title}
+% \edef\title{...\the\toks0...}
+\let\maybe=\iffalse % print only changed modules
+\def\topofcontents{\hsize 5.5in
+ \vglue 0pt plus 1fil minus 1.5in
+ \def\@@##1]{\hbox to 1in{\hfil##1.\ }}
+ }
+\def\botofcontents{\vskip 0pt plus 1fil minus 1.5in}
+\pageno=3
+\def\glob{13} % this should be the section number of "<Global...>"
+\def\gglob{20, 26} % this should be the next two sections of "<Global...>"
+
+\def\.#1{\leavevmode\hbox{\tentex % typewriter type for strings
+ \let\\=\BS % backslash in a string
+ \let\'=\RQ % right quote in a string
+ \let\`=\LQ % left quote in a string
+ \let\{=\LB % left brace in a string
+ \let\}=\RB % right brace in a string
+ \let\~=\TL % tilde in a string
+ \let\ =\SP % space in a string
+ \let\_=\UL % underline in a string
+ \let\&=\AM % ampersand in a string
+ #1\kern.05em}}
+\def\^{\ifmmode\mathchar"222 \else\char`^ \fi} % pointer or hat
+\def\LQ{{\tt\char'22}} % left quote in a string
+\def\RQ{{\tt\char'23}} % right quote in a string
+\def\dotdot{\mathrel{.\,.}} % double dot, used only in math mode
+ at s dotdot TeX
+@* Introduction.
+This is \eTeX, a program derived from and extending the capabilities of
+\TeX, a document compiler intended to produce typesetting of high
+quality.
+The \PASCAL\ program that follows is the definition of \TeX82, a standard
+@:PASCAL}{\PASCAL@>
+@!@:TeX82}{\TeX82@>
+version of \TeX\ that is designed to be highly portable so that identical output
+will be obtainable on a great variety of computers.
+
+The main purpose of the following program is to explain the algorithms of \TeX\
+as clearly as possible. As a result, the program will not necessarily be very
+efficient when a particular \PASCAL\ compiler has translated it into a
+particular machine language. However, the program has been written so that it
+can be tuned to run efficiently in a wide variety of operating environments
+by making comparatively few changes. Such flexibility is possible because
+the documentation that follows is written in the \.{WEB} language, which is
+at a higher level than \PASCAL; the preprocessing step that converts \.{WEB}
+to \PASCAL\ is able to introduce most of the necessary refinements.
+Semi-automatic translation to other languages is also feasible, because the
+program below does not make extensive use of features that are peculiar to
+\PASCAL.
+
+A large piece of software like \TeX\ has inherent complexity that cannot
+be reduced below a certain level of difficulty, although each individual
+part is fairly simple by itself. The \.{WEB} language is intended to make
+the algorithms as readable as possible, by reflecting the way the
+individual program pieces fit together and by providing the
+cross-references that connect different parts. Detailed comments about
+what is going on, and about why things were done in certain ways, have
+been liberally sprinkled throughout the program. These comments explain
+features of the implementation, but they rarely attempt to explain the
+\TeX\ language itself, since the reader is supposed to be familiar with
+{\sl The \TeX book}.
+ at .WEB@>
+@:TeXbook}{\sl The \TeX book@>
+
+@ The present implementation has a long ancestry, beginning in the summer
+of~1977, when Michael~F. Plass and Frank~M. Liang designed and coded
+a prototype
+@^Plass, Michael Frederick@>
+@^Liang, Franklin Mark@>
+@^Knuth, Donald Ervin@>
+based on some specifications that the author had made in May of that year.
+This original proto\TeX\ included macro definitions and elementary
+manipulations on boxes and glue, but it did not have line-breaking,
+page-breaking, mathematical formulas, alignment routines, error recovery,
+or the present semantic nest; furthermore,
+it used character lists instead of token lists, so that a control sequence
+like \.{\\halign} was represented by a list of seven characters. A
+complete version of \TeX\ was designed and coded by the author in late
+1977 and early 1978; that program, like its prototype, was written in the
+{\mc SAIL} language, for which an excellent debugging system was
+available. Preliminary plans to convert the {\mc SAIL} code into a form
+somewhat like the present ``web'' were developed by Luis Trabb~Pardo and
+@^Trabb Pardo, Luis Isidoro@>
+the author at the beginning of 1979, and a complete implementation was
+created by Ignacio~A. Zabala in 1979 and 1980. The \TeX82 program, which
+@^Zabala Salelles, Ignacio Andr\'es@>
+was written by the author during the latter part of 1981 and the early
+part of 1982, also incorporates ideas from the 1979 implementation of
+@^Guibas, Leonidas Ioannis@>
+@^Sedgewick, Robert@>
+@^Wyatt, Douglas Kirk@>
+\TeX\ in {\mc MESA} that was written by Leonidas Guibas, Robert Sedgewick,
+and Douglas Wyatt at the Xerox Palo Alto Research Center. Several hundred
+refinements were introduced into \TeX82 based on the experiences gained with
+the original implementations, so that essentially every part of the system
+has been substantially improved. After the appearance of ``Version 0'' in
+September 1982, this program benefited greatly from the comments of
+many other people, notably David~R. Fuchs and Howard~W. Trickey.
+A final revision in September 1989 extended the input character set to
+eight-bit codes and introduced the ability to hyphenate words from
+different languages, based on some ideas of Michael~J. Ferguson.
+@^Fuchs, David Raymond@>
+@^Trickey, Howard Wellington@>
+@^Ferguson, Michael John@>
+
+No doubt there still is plenty of room for improvement, but the author
+is firmly committed to keeping \TeX82 ``frozen'' from now on; stability
+and reliability are to be its main virtues.
+
+On the other hand, the \.{WEB} description can be extended without changing
+the core of \TeX82 itself, and the program has been designed so that such
+extensions are not extremely difficult to make.
+The |banner| string defined here should be changed whenever \TeX\
+undergoes any modifications, so that it will be clear which version of
+\TeX\ might be the guilty party when a problem arises.
+@^extensions to \TeX@>
+@^system dependencies@>
+
+This program contains code for various features extending \TeX,
+therefore this program is called `\eTeX' and not
+`\TeX'; the official name `\TeX' by itself is reserved
+for software systems that are fully compatible with each other.
+A special test suite called the ``\.{TRIP} test'' is available for
+helping to determine whether a particular implementation deserves to be
+known as `\TeX' [cf.~Stanford Computer Science report CS1027,
+November 1984].
+
+A similar test suite called the ``\.{e-TRIP} test'' is available for
+helping to determine whether a particular implementation deserves to be
+known as `\eTeX'.
+
+ at d eTeX_version 2 /* \.{\\eTeXversion} */
+ at d eTeX_revision ".6" /* \.{\\eTeXrevision} */
+ at d eTeX_version_string "-2.6" /*current \eTeX\ version*/
+@#
+ at d eTeX_banner "This is e-TeX, Version 3.14159265" " (HINT) " eTeX_version_string
+ /*printed when \eTeX\ starts*/
+@#
+ at d TeX_banner "This is TeX, Version 3.14159265 (HINT)" /*printed when \TeX\ starts*/
+@#
+ at d banner eTeX_banner
+@#
+ at d TEX ETEX /*change program name into |ETEX|*/
+@#
+ at d eTeX_states 1 /*number of \eTeX\ state variables in |eqtb|*/
+
+@ Different \PASCAL s have slightly different conventions, and the present
+@!@:PASCAL H}{\ph@>
+program expresses \TeX\ in terms of the \PASCAL\ that was
+available to the author in 1982. Constructions that apply to
+this particular compiler, which we shall call \ph, should help the
+reader see how to make an appropriate interface for other systems
+if necessary. (\ph\ is Charles Hedrick's modification of a compiler
+@^Hedrick, Charles Locke@>
+for the DECsystem-10 that was originally developed at the University of
+Hamburg; cf.\ {\sl SOFTWARE---Practice \AM\ Experience \bf6} (1976),
+29--42. The \TeX\ program below is intended to be adaptable, without
+extensive changes, to most other versions of \PASCAL, so it does not fully
+use the admirable features of \ph. Indeed, a conscious effort has been
+made here to avoid using several idiosyncratic features of standard
+\PASCAL\ itself, so that most of the code can be translated mechanically
+into other high-level languages. For example, the `\&{with}' and `\\{new}'
+features are not used, nor are pointer types, set types, or enumerated
+scalar types; there are no `\&{var}' parameters, except in the case of files
+--- \eTeX, however, does use `\&{var}' parameters for the |reverse| function;
+there are no tag fields on variant records; there are no assignments
+|double=int|; no procedures are declared local to other procedures.)
+
+The portions of this program that involve system-dependent code, where
+changes might be necessary because of differences between \PASCAL\ compilers
+and/or differences between
+operating systems, can be identified by looking at the sections whose
+numbers are listed under `system dependencies' in the index. Furthermore,
+the index entries for `dirty \PASCAL' list all places where the restrictions
+of \PASCAL\ have not been followed perfectly, for one reason or another.
+@!@^system dependencies@>
+@!@^dirty \PASCAL@>
+
+Incidentally, \PASCAL's standard |round| function can be problematical,
+because it disagrees with the IEEE floating-point standard.
+Many implementors have
+therefore chosen to substitute their own home-grown rounding procedure.
+
+@ The following is an outline of the program, whose
+components will be filled in later, using the conventions of \.{WEB}.
+ at .WEB@>
+For example, the portion of the program called `\X\glob:Global
+variables\X' below will be replaced by a sequence of variable declarations
+that starts in $\section\glob$ of this documentation. In this way, we are able
+to define each individual global variable when we are prepared to
+understand what it means; we do not have to define all of the globals at
+once. Cross references in $\section\glob$, where it says ``See also
+sections \gglob, \dots,'' also make it possible to look at the set of
+all global variables, if desired. Similar remarks apply to the other
+portions of the program.
+
+ at p @<Header files and function declarations@>@;
+#include "htex.h"
+#include "hitex.h"
+@<Global variables@>@;
+@#
+
+static void initialize(void) /*this procedure gets things started properly*/
+ {@+@<Local variables for initialization@>@;
+ @<Initialize whatever \TeX\ might access@>;
+ } @#
+@<Basic printing procedures@>@;
+@<Error handling procedures@>@;
+
+@ The overall \TeX\ program begins with the heading just shown, after which
+comes a bunch of procedure declarations and function declarations.
+Finally we will get to the main program, which begins with the
+comment `|start_here|'. If you want to skip down to the
+main program now, you can look up `|start_here|' in the index.
+But the author suggests that the best way to understand this program
+is to follow pretty much the order of \TeX's components as they appear in the
+\.{WEB} description you are now reading, since the present ordering is
+intended to combine the advantages of the ``bottom up'' and ``top down''
+approaches to the problem of understanding a somewhat complicated system.
+
+@ There is no need to declare labels in \CEE/, but occasionally
+it is necessary to insert header files and declare functions
+very early in the program. For example, the function |s_no|
+which uses |make_string| to convert a \CEE/ string into
+a string number is used in |initialize| and needs a forward
+declaration.
+
+@<Header files and function declarations@>=
+extern int s_no(const char *str);
+
+@ Some of the code below is intended to be used only when diagnosing the
+strange behavior that sometimes occurs when \TeX\ is being installed or
+when system wizards are fooling around with \TeX\ without quite knowing
+what they are doing. Such code will not normally be compiled; it is
+delimited by the codewords `$|debug|\ldots|debug|$', with apologies
+to people who wish to preserve the purity of English.
+
+Similarly, there is some conditional code delimited by
+`$|stat|\ldots|tats|$' that is intended for use when statistics are to be
+kept about \TeX's memory usage. The |stat| $\ldots$ |tats| code also
+implements diagnostic information for \.{\\tracingparagraphs} and
+\.{\\tracingpages}.
+@^debugging@>
+
+@ This program has two important variations: (1) There is a long and slow
+version called \.{INITEX}, which does the extra calculations needed to
+ at .INITEX@>
+initialize \TeX's internal tables; and (2)~there is a shorter and faster
+production version, which cuts the initialization to a bare minimum.
+Parts of the program that are needed in (1) but not in (2) are delimited by
+|#ifdef| |INIT|\dots\ |#endif|.
+
+\TeX\ Live has established the common practice
+to select the initialization code at runtime
+using the |iniversion| variable.
+
+@<Initialize whatever...@>=
+@<Set initial values of key variables@>@/
+#ifdef @!INIT
+if (iniversion) {@+@<Initialize table entries (done by \.{INITEX} only)@>@;@+}
+#endif
+
+@ Most of the external declarations needed are included using standard
+header files.
+
+ at s uint8_t int
+ at s int16_t int
+ at s uint16_t int
+ at s int32_t int
+ at s uint32_t int
+ at s halfword int
+ at s in TeX
+ at s line normal
+ at s to do
+
+@<Header files and function declarations@>=
+#include <stdint.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <math.h>
+
+@ Further it is necessary to define some build in primitives of
+\PASCAL\ that are otherwise not available in \CEE/.
+@:PASCAL H}{\ph@>
+
+ at d odd(X) ((X)&1)
+ at d chr(X) ((unsigned char)(X))
+ at d ord(X) ((unsigned int)(X))
+ at d abs(X) ((X)>-(X)?(X):-(X))
+ at d round(X) ((int)((X)>=0.0?floor((X)+0.5):ceil((X)-0.5)))
+
+@ The following parameters can be changed at compile time to extend or
+reduce \TeX's capacity. They may have different values in \.{INITEX} and
+in production versions of \TeX.
+ at .INITEX@>
+@^system dependencies@>
+
+@<Constants...@>=
+@!mem_max=5000000, /*greatest index in \TeX's internal |mem| array;
+ must be strictly less than |max_halfword|;
+ must be equal to |mem_top| in \.{INITEX}, otherwise | >= mem_top|*/
+@!mem_min=0, /*smallest index in \TeX's internal |mem| array;
+ must be |min_halfword| or more;
+ must be equal to |mem_bot| in \.{INITEX}, otherwise | <= mem_bot|*/
+@!buf_size=2000000, /*maximum number of characters simultaneously present in
+ current lines of open files and in control sequences between
+ \.{\\csname} and \.{\\endcsname}; must not exceed |max_halfword|*/
+@!error_line=79, /*width of context lines on terminal error messages*/
+@!half_error_line=50, /*width of first lines of contexts in terminal
+ error messages; should be between 30 and |error_line-15|*/
+@!max_print_line=79, /*width of longest text lines output; should be at least 60*/
+@!stack_size=5000, /*maximum number of simultaneous input sources*/
+@!max_in_open=15, /*maximum number of input files and error insertions that
+ can be going on simultaneously*/
+@!font_max=255, /*maximum internal font number; must not exceed |max_quarterword|
+ and must be at most |font_base+256|*/
+@!font_mem_size=8000000, /*number of words of |font_info| for all fonts*/
+@!param_size=10000, /*maximum number of simultaneous macro parameters*/
+@!nest_size=500, /*maximum number of semantic levels simultaneously active*/
+@!max_strings=500000, /*maximum number of strings; must not exceed |max_halfword|*/
+@!string_vacancies=90000, /*the minimum number of characters that should be
+ available for the user's control sequences and font names,
+ after \TeX's own error messages are stored*/
+@!pool_size=6250000, /*maximum number of characters in strings, including all
+ error messages and help texts, and the names of all fonts and
+ control sequences; must exceed |string_vacancies| by the total
+ length of \TeX's own strings, which is currently about 23000*/
+@!save_size=100000, /*space for saving values outside of current group; must be
+ at most |max_halfword|*/
+@!trie_size=1000000, /*space for hyphenation patterns; should be larger for
+ \.{INITEX} than it is in production versions of \TeX*/
+@!trie_op_size=35111, /*space for ``opcodes'' in the hyphenation patterns*/
+@!dvi_buf_size=16384, /*size of the output buffer; must be a multiple of 8*/
+@!file_name_size=1024, /*file names shouldn't be longer than this*/
+@!empty_string=256 /*the empty string follows after 256 characters*/
+
+@ Like the preceding parameters, the following quantities can be changed
+at compile time to extend or reduce \TeX's capacity. But if they are changed,
+it is necessary to rerun the initialization program \.{INITEX}
+ at .INITEX@>
+to generate new tables for the production \TeX\ program.
+One can't simply make helter-skelter changes to the following constants,
+since certain rather complex initialization
+numbers are computed from them. They are defined here using
+\.{WEB} macros, instead of being put into \PASCAL's |const| list, in order to
+emphasize this distinction.
+
+ at d mem_bot 0 /*smallest index in the |mem| array dumped by \.{INITEX};
+ must not be less than |mem_min|*/
+ at d mem_top 5000000 /*largest index in the |mem| array dumped by \.{INITEX};
+ must be substantially larger than |mem_bot|
+ and not greater than |mem_max|*/
+ at d font_base 0 /*smallest internal font number; must not be less
+ than |min_quarterword|*/
+ at d hash_size 15000 /*maximum number of control sequences; it should be at most
+ about |(mem_max-mem_min)/(double)10|*/
+ at d hash_prime 8501 /*a prime number equal to about 85\pct! of |hash_size|*/
+ at d hyph_size 8191 /*another prime; the number of \.{\\hyphenation} exceptions*/
+@^system dependencies@>
+
+@ In case somebody has inadvertently made bad settings of the ``constants,''
+\TeX\ checks them using a global variable called |bad|.
+
+This is the first of many sections of \TeX\ where global variables are
+defined.
+
+@<Glob...@>=
+static int @!bad; /*is some ``constant'' wrong?*/
+
+@ Later on we will say `\ignorespaces|if (mem_max >= max_halfword) bad=14|',
+or something similar. (We can't do that until |max_halfword| has been defined.)
+
+@<Check the ``constant'' values for consistency@>=
+bad=0;
+if ((half_error_line < 30)||(half_error_line > error_line-15)) bad=1;
+if (max_print_line < 60) bad=2;
+if (dvi_buf_size%8!=0) bad=3;
+if (mem_bot+1100 > mem_top) bad=4;
+if (hash_prime > hash_size) bad=5;
+if (max_in_open >= 128) bad=6;
+if (mem_top < 256+11) bad=7; /*we will want |null_list > 255|*/
+
+@ Labels are given symbolic names by the following definitions, so that
+occasional |goto| statements will be meaningful. We insert the label
+`|end|' just before the `\ignorespaces|} |\unskip' of a procedure in
+which we have used the `|goto end|' statement defined below; the label
+`|restart|' is occasionally used at the very beginning of a procedure; and
+the label `|reswitch|' is occasionally used just prior to a |case|
+statement in which some cases change the conditions and we wish to branch
+to the newly applicable case. Loops that are set up with the |loop|
+construction defined below are commonly exited by going to `|done|' or to
+`|found|' or to `|not_found|', and they are sometimes repeated by going to
+`|resume|'. If two or more parts of a subroutine start differently but
+end up the same, the shared code may be gathered together at
+`|common_ending|'.
+
+Incidentally, this program never declares a label that isn't actually used,
+because some fussy \PASCAL\ compilers will complain about redundant labels.
+
+@ Here are some macros for common programming idioms.
+
+ at d incr(A) A=A+1 /*increase a variable by unity*/
+ at d decr(A) A=A-1 /*decrease a variable by unity*/
+ at d negate(A) A=-A /*change the sign of a variable*/
+ at d loop @+while (true) @+ /*repeat over and over until a |goto| happens*/
+ at f loop else
+ /*\.{WEB}'s |else| acts like `\ignorespaces|while true do|\unskip'*/
+ at d do_nothing /*empty statement*/
+ at d empty 0 /*symbolic name for a null constant*/
+
+@* The character set.
+In order to make \TeX\ readily portable to a wide variety of
+computers, all of its input text is converted to an internal eight-bit
+code that includes standard ASCII, the ``American Standard Code for
+Information Interchange.'' This conversion is done immediately when each
+character is read in. Conversely, characters are converted from ASCII to
+the user's external representation just before they are output to a
+text file.
+
+Such an internal code is relevant to users of \TeX\ primarily because it
+governs the positions of characters in the fonts. For example, the
+character `\.A' has ASCII code $65=0101$, and when \TeX\ typesets
+this letter it specifies character number 65 in the current font.
+If that font actually has `\.A' in a different position, \TeX\ doesn't
+know what the real position is; the program that does the actual printing from
+\TeX's device-independent files is responsible for converting from ASCII to
+a particular font encoding.
+@^ASCII code@>
+
+\TeX's internal code also defines the value of constants
+that begin with a reverse apostrophe; and it provides an index to the
+\.{\\catcode}, \.{\\mathcode}, \.{\\uccode}, \.{\\lccode}, and \.{\\delcode}
+tables.
+
+@ Characters of text that have been converted to \TeX's internal form
+are said to be of type |ASCII_code|, which is a subrange of the integers.
+
+@<Types...@>=
+typedef uint8_t ASCII_code; /*eight-bit numbers*/
+
+@ The original \PASCAL\ compiler was designed in the late 60s, when six-bit
+character sets were common, so it did not make provision for lowercase
+letters. Nowadays, of course, we need to deal with both capital and small
+letters in a convenient way, especially in a program for typesetting;
+so the present specification of \TeX\ has been written under the assumption
+that the \PASCAL\ compiler and run-time system permit the use of text files
+with more than 64 distinguishable characters. More precisely, we assume that
+the character set contains at least the letters and symbols associated
+with ASCII codes 040 through 0176; all of these characters are now
+available on most computer terminals.
+
+Since we are dealing with more characters than were present in the first
+\PASCAL\ compilers, we have to decide what to call the associated data
+type. Some \PASCAL s use the original name |unsigned char| for the
+characters in text files, even though there now are more than 64 such
+characters, while other \PASCAL s consider |unsigned char| to be a 64-element
+subrange of a larger data type that has some other name.
+
+In order to accommodate this difference, we shall use the name |text_char|
+to stand for the data type of the characters that are converted to and
+from |ASCII_code| when they are input and output. We shall also assume
+that |text_char| consists of the elements |chr(first_text_char)| through
+|chr(last_text_char)|, inclusive. The following definitions should be
+adjusted if necessary.
+@^system dependencies@>
+
+ at s text_char char
+ at d text_char unsigned char /*the data type of characters in text files*/
+ at d first_text_char 0 /*ordinal number of the smallest element of |text_char|*/
+ at d last_text_char 255 /*ordinal number of the largest element of |text_char|*/
+
+@<Local variables for init...@>=
+int @!i;
+
+@ The \TeX\ processor converts between ASCII code and
+the user's external character set by means of arrays |xord| and |xchr|
+that are analogous to \PASCAL's |ord| and |chr| functions.
+
+@<Glob...@>=
+static ASCII_code @!xord[256];
+ /*specifies conversion of input characters*/
+static text_char @!xchr[256];
+ /*specifies conversion of output characters*/
+
+@ Since we are assuming that our \PASCAL\ system is able to read and
+write the visible characters of standard ASCII (although not
+necessarily using the ASCII codes to represent them), the following
+assignment statements initialize the standard part of the |xchr| array
+properly, without needing any system-dependent changes. On the other
+hand, it is possible to implement \TeX\ with less complete character
+sets, and in such cases it will be necessary to change something here.
+@^system dependencies@>
+
+@<Set init...@>=
+xchr[040]=' ';
+xchr[041]='!';
+xchr[042]='"';
+xchr[043]='#';
+xchr[044]='$';
+xchr[045]='%';
+xchr[046]='&';
+xchr[047]='\'';@/
+xchr[050]='(';
+xchr[051]=')';
+xchr[052]='*';
+xchr[053]='+';
+xchr[054]=',';
+xchr[055]='-';
+xchr[056]='.';
+xchr[057]='/';@/
+xchr[060]='0';
+xchr[061]='1';
+xchr[062]='2';
+xchr[063]='3';
+xchr[064]='4';
+xchr[065]='5';
+xchr[066]='6';
+xchr[067]='7';@/
+xchr[070]='8';
+xchr[071]='9';
+xchr[072]=':';
+xchr[073]=';';
+xchr[074]='<';
+xchr[075]='=';
+xchr[076]='>';
+xchr[077]='?';@/
+xchr[0100]='@@';
+xchr[0101]='A';
+xchr[0102]='B';
+xchr[0103]='C';
+xchr[0104]='D';
+xchr[0105]='E';
+xchr[0106]='F';
+xchr[0107]='G';@/
+xchr[0110]='H';
+xchr[0111]='I';
+xchr[0112]='J';
+xchr[0113]='K';
+xchr[0114]='L';
+xchr[0115]='M';
+xchr[0116]='N';
+xchr[0117]='O';@/
+xchr[0120]='P';
+xchr[0121]='Q';
+xchr[0122]='R';
+xchr[0123]='S';
+xchr[0124]='T';
+xchr[0125]='U';
+xchr[0126]='V';
+xchr[0127]='W';@/
+xchr[0130]='X';
+xchr[0131]='Y';
+xchr[0132]='Z';
+xchr[0133]='[';
+xchr[0134]='\\';
+xchr[0135]=']';
+xchr[0136]='^';
+xchr[0137]='_';@/
+xchr[0140]='`';
+xchr[0141]='a';
+xchr[0142]='b';
+xchr[0143]='c';
+xchr[0144]='d';
+xchr[0145]='e';
+xchr[0146]='f';
+xchr[0147]='g';@/
+xchr[0150]='h';
+xchr[0151]='i';
+xchr[0152]='j';
+xchr[0153]='k';
+xchr[0154]='l';
+xchr[0155]='m';
+xchr[0156]='n';
+xchr[0157]='o';@/
+xchr[0160]='p';
+xchr[0161]='q';
+xchr[0162]='r';
+xchr[0163]='s';
+xchr[0164]='t';
+xchr[0165]='u';
+xchr[0166]='v';
+xchr[0167]='w';@/
+xchr[0170]='x';
+xchr[0171]='y';
+xchr[0172]='z';
+xchr[0173]='{';
+xchr[0174]='|';
+xchr[0175]='}';
+xchr[0176]='~';@/
+
+@ Some of the ASCII codes without visible characters have been given symbolic
+names in this program because they are used with a special meaning.
+
+ at d null_code 00 /*ASCII code that might disappear*/
+ at d carriage_return 015 /*ASCII code used at end of line*/
+ at d invalid_code 0177 /*ASCII code that many systems prohibit in text files*/
+
+@ The ASCII code is ``standard'' only to a certain extent, since many
+computer installations have found it advantageous to have ready access
+to more than 94 printing characters. Appendix~C of {\sl The \TeX book\/}
+gives a complete specification of the intended correspondence between
+characters and \TeX's internal representation.
+@:TeXbook}{\sl The \TeX book@>
+
+If \TeX\ is being used
+on a garden-variety \PASCAL\ for which only standard ASCII
+codes will appear in the input and output files, it doesn't really matter
+what codes are specified in |xchr[0 dotdot 037]|, but the safest policy is to
+blank everything out by using the code shown below.
+
+However, other settings of |xchr| will make \TeX\ more friendly on
+computers that have an extended character set, so that users can type things
+like `\.^^Z' instead of `\.{\\ne}'. People with extended character sets can
+assign codes arbitrarily, giving an |xchr| equivalent to whatever
+characters the users of \TeX\ are allowed to have in their input files.
+It is best to make the codes correspond to the intended interpretations as
+shown in Appendix~C whenever possible; but this is not necessary. For
+example, in countries with an alphabet of more than 26 letters, it is
+usually best to map the additional letters into codes less than~040.
+To get the most ``permissive'' character set, change |' '| on the
+right of these assignment statements to |chr(i)|.
+@^character set dependencies@>
+@^system dependencies@>
+
+@<Set init...@>=
+for (i=0; i<=037; i++) xchr[i]=chr(i); /* k\TeX\ */
+for (i=0177; i<=0377; i++) xchr[i]=chr(i); /* k\TeX\ */
+
+@ The following system-independent code makes the |xord| array contain a
+suitable inverse to the information in |xchr|. Note that if |xchr[i]==xchr[j]|
+where |i < j < 0177|, the value of |xord[xchr[i]]| will turn out to be
+|j| or more; hence, standard ASCII code numbers will be used instead of
+codes below 040 in case there is a coincidence.
+
+@<Set init...@>=
+for (i=first_text_char; i<=last_text_char; i++) xord[chr(i)]=invalid_code;
+for (i=0200; i<=0377; i++) xord[xchr[i]]=i;
+for (i=0; i<=0176; i++) xord[xchr[i]]=i;
+
+@* Input and output.
+The bane of portability is the fact that different operating systems treat
+input and output quite differently, perhaps because computer scientists
+have not given sufficient attention to this problem. People have felt somehow
+that input and output are not part of ``real'' programming. Well, it is true
+that some kinds of programming are more fun than others. With existing
+input/output conventions being so diverse and so messy, the only sources of
+joy in such parts of the code are the rare occasions when one can find a
+way to make the program a little less bad than it might have been. We have
+two choices, either to attack I/O now and get it over with, or to postpone
+I/O until near the end. Neither prospect is very attractive, so let's
+get it over with.
+
+The basic operations we need to do are (1)~inputting and outputting of
+text, to or from a file or the user's terminal; (2)~inputting and
+outputting of eight-bit bytes, to or from a file; (3)~instructing the
+operating system to initiate (``open'') or to terminate (``close'') input or
+output from a specified file; (4)~testing whether the end of an input
+file has been reached.
+
+\TeX\ needs to deal with two kinds of files.
+We shall use the term |alpha_file| for a file that contains textual data,
+and the term |byte_file| for a file that contains eight-bit binary information.
+These two types turn out to be the same on many computers, but
+sometimes there is a significant distinction, so we shall be careful to
+distinguish between them. Standard protocols for transferring
+such files from computer to computer, via high-speed networks, are
+now becoming available to more and more communities of users.
+
+The program actually makes use also of a third kind of file, called a
+|word_file|, when dumping and reloading base information for its own
+initialization. We shall define a word file later; but it will be possible
+for us to specify simple operations on word files before they are defined.
+
+@<Types...@>=
+typedef uint8_t eight_bits; /*unsigned one-byte quantity*/
+typedef struct {@+FILE *f;@+text_char@,d;@+} alpha_file; /*files that contain textual data*/
+typedef struct {@+FILE *f;@+eight_bits@,d;@+} byte_file; /*files that contain binary data*/
+
+@ Most of what we need to do with respect to input and output can be handled
+by the I/O facilities that are standard in \PASCAL, i.e., the routines
+called |get|, |put|, |eof|, and so on. But
+standard \PASCAL\ does not allow file variables to be associated with file
+names that are determined at run time, so it cannot be used to implement
+\TeX; some sort of extension to \PASCAL's ordinary |reset| and |rewrite|
+is crucial for our purposes. We shall assume that |name_of_file| is a variable
+of an appropriate type such that the \PASCAL\ run-time system being used to
+implement \TeX\ can open a file whose external name is specified by
+|name_of_file|.
+@^system dependencies@>
+
+@<Glob...@>=
+unsigned char @!name_of_file0[file_name_size+1]={0}, *const @!name_of_file = @!name_of_file0-1;@;@/
+ /*on some systems this may be a \&{record} variable*/
+int @!name_length;@/ /*this many characters are actually
+ relevant in |name_of_file| (the rest are blank)*/
+
+@ k\TeX\ uses the {\tt kpathsearch} library to implement access to files.
+To do so we declare \TeX's functions here and postpone the
+actual implementations. These functions do not to issue their own
+error messages if something goes wrong. If a file identified by
+|name_of_file| cannot be found,
+or if such a file cannot be opened for some other reason
+(e.g., someone may already be trying to write the same file)
+\TeX's file-opening procedures return |false|.
+This allows \TeX\ to undertake appropriate corrective action.
+@^system dependencies@>
+
+ at p static bool a_open_in(alpha_file *f); /*open a text file for input*/
+static bool a_open_out(alpha_file *f); /*open a text file for output*/
+static bool b_open_in(byte_file *f); /*open a binary file for input*/
+static bool b_open_out(byte_file *f); /*open a binary file for output*/
+static bool w_open_in(word_file *f); /*open a word file for input*/
+#ifdef @!INIT
+static bool w_open_out(word_file *f); /*open a word file for output*/
+#endif
+
+@ Files can be closed with the \ph\ routine `|pascal_close(f)|', which
+@:PASCAL H}{\ph@>
+@^system dependencies@>
+should be used when all input or output with respect to |f| has been completed.
+This makes |f| available to be opened again, if desired; and if |f| was used for
+output, the |pascal_close| operation makes the corresponding external file appear
+on the user's area, ready to be read.
+
+These procedures should not generate error messages if a file is
+being closed before it has been successfully opened.
+
+ at p static void a_close(alpha_file *f) /*close a text file*/
+{@+pascal_close((*f));
+}
+@#
+static void b_close(byte_file *f) /*close a binary file*/
+{@+pascal_close((*f));
+}
+@#
+static void w_close(word_file *f) /*close a word file*/
+{@+pascal_close((*f));
+}
+
+@ Binary input and output are done with \PASCAL's ordinary |get| and |put|
+procedures, so we don't have to make any other special arrangements for
+binary~I/O. Text output is also easy to do with standard \PASCAL\ routines.
+The treatment of text input is more difficult, however, because
+of the necessary translation to |ASCII_code| values.
+\TeX's conventions should be efficient, and they should
+blend nicely with the user's operating environment.
+
+@ Input from text files is read one line at a time, using a routine called
+|input_ln|. This function is defined in terms of global variables called
+|buffer|, |first|, and |last| that will be described in detail later; for
+now, it suffices for us to know that |buffer| is an array of |ASCII_code|
+values, and that |first| and |last| are indices into this array
+representing the beginning and ending of a line of text.
+
+@<Glob...@>=
+static ASCII_code @!buffer[buf_size+1]; /*lines of characters being read*/
+static int @!first; /*the first unused position in |buffer|*/
+static int @!last; /*end of the line just input to |buffer|*/
+static int @!max_buf_stack; /*largest index used in |buffer|*/
+
+@ The |input_ln| function brings the next line of input from the specified
+file into available positions of the buffer array and returns the value
+|true|, unless the file has already been entirely read, in which case it
+returns |false| and sets |last=first|. In general, the |ASCII_code|
+numbers that represent the next line of the file are input into
+|buffer[first]|, |buffer[first+1]|, \dots, |buffer[last-1]|; and the
+global variable |last| is set equal to |first| plus the length of the
+line. Trailing blanks are removed from the line; thus, either |last==first|
+(in which case the line was entirely blank) or |buffer[last-1]!=' '|.
+
+An overflow error is given, however, if the normal actions of |input_ln|
+would make |last >= buf_size|; this is done so that other parts of \TeX\
+can safely look at the contents of |buffer[last+1]| without overstepping
+the bounds of the |buffer| array. Upon entry to |input_ln|, the condition
+|first < buf_size| will always hold, so that there is always room for an
+``empty'' line.
+
+The variable |max_buf_stack|, which is used to keep track of how large
+the |buf_size| parameter must be to accommodate the present job, is
+also kept up to date by |input_ln|.
+
+If the |bypass_eoln| parameter is |true|, |input_ln| will do a |get|
+before looking at the first character of the line; this skips over
+an |eoln| that was in |f.d|. The procedure does not do a |get| when it
+reaches the end of the line; therefore it can be used to acquire input
+from the user's terminal as well as from ordinary text files.
+
+Standard \PASCAL\ says that a file should have |eoln| immediately
+before |eof|, but \TeX\ needs only a weaker restriction: If |eof|
+occurs in the middle of a line, the system function |eoln| should return
+a |true| result (even though |f.d| will be undefined).
+
+Since the inner loop of |input_ln| is part of \TeX's ``inner loop''---each
+character of input comes in at this place---it is wise to reduce system
+overhead by making use of special routines that read in an entire array
+of characters at once, if such routines are available. The following
+code uses standard \PASCAL\ to illustrate what needs to be done, but
+finer tuning is often possible at well-developed \PASCAL\ sites.
+@^inner loop@>
+
+ at p static bool input_ln(alpha_file *f, bool @!bypass_eoln)
+ /*inputs the next line or returns |false|*/
+{@+int last_nonblank; /*|last| with trailing blanks removed*/
+if (bypass_eoln) if (!eof((*f))) get((*f));
+ /*input the first character of the line into |f.d|*/
+last=first; /*cf.\ Matthew 19\thinspace:\thinspace30*/
+if (eof((*f))) return false;
+else{@+last_nonblank=first;
+ while (!eoln((*f)))
+ {@+if (last >= max_buf_stack)
+ {@+max_buf_stack=last+1;
+ if (max_buf_stack==buf_size)
+ @<Report overflow of the input buffer, and abort@>;
+ }
+ buffer[last]=xord[(*f).d];get((*f));incr(last);
+ if (buffer[last-1]!=' ') last_nonblank=last;
+ }
+ last=last_nonblank;return true;
+ }
+}
+
+@ The user's terminal acts essentially like other files of text, except
+that it is used both for input and for output. When the terminal is
+considered an input file, the file variable is called |term_in|, and when it
+is considered an output file the file variable is |term_out|.
+@^system dependencies@>
+
+@<Glob...@>=
+static alpha_file @!term_in; /*the terminal as an input file*/
+static alpha_file @!term_out; /*the terminal as an output file*/
+
+@ Here is how to open the terminal files
+in \ph. The `\.{/I}' switch suppresses the first |get|.
+@:PASCAL H}{\ph@>
+@^system dependencies@>
+
+ at d t_open_in term_in.f=stdin /*open the terminal for text input*/
+ at d t_open_out term_out.f=stdout /*open the terminal for text output*/
+
+@ Sometimes it is necessary to synchronize the input/output mixture that
+happens on the user's terminal, and three system-dependent
+procedures are used for this
+purpose. The first of these, |update_terminal|, is called when we want
+to make sure that everything we have output to the terminal so far has
+actually left the computer's internal buffers and been sent.
+The second, |clear_terminal|, is called when we wish to cancel any
+input that the user may have typed ahead (since we are about to
+issue an unexpected error message). The third, |wake_up_terminal|,
+is supposed to revive the terminal if the user has disabled it by
+some instruction to the operating system. The following macros show how
+these operations can be specified in \ph:
+@:PASCAL H}{\ph@>
+@^system dependencies@>
+
+ at d update_terminal fflush(term_out.f) /*empty the terminal output buffer*/
+ at d clear_terminal fflush(term_in.f) /*clear the terminal input buffer*/
+ at d wake_up_terminal do_nothing /*cancel the user's cancellation of output*/
+
+@ We need a special routine to read the first line of \TeX\ input from
+the user's terminal. This line is different because it is read before we
+have opened the transcript file; there is sort of a ``chicken and
+egg'' problem here. If the user types `\.{\\input paper}' on the first
+line, or if some macro invoked by that line does such an \.{\\input},
+the transcript file will be named `\.{paper.log}'; but if no \.{\\input}
+commands are performed during the first line of terminal input, the transcript
+file will acquire its default name `\.{texput.log}'. (The transcript file
+will not contain error messages generated by the first line before the
+first \.{\\input} command.)
+ at .texput@>
+
+The first line is even more special if we are lucky enough to have an operating
+system that treats \TeX\ differently from a run-of-the-mill \PASCAL\ object
+program. It's nice to let the user start running a \TeX\ job by typing
+a command line like `\.{tex paper}'; in such a case, \TeX\ will operate
+as if the first line of input were `\.{paper}', i.e., the first line will
+consist of the remainder of the command line, after the part that invoked
+\TeX.
+
+The first line is special also because it may be read before \TeX\ has
+input a format file. In such cases, normal error messages cannot yet
+be given. The following code uses concepts that will be explained later.
+(If the \PASCAL\ compiler does not support non-local |@!goto|\unskip, the
+@^system dependencies@>
+statement `|goto exit(0)|' should be replaced by something that
+quietly terminates the program.)
+
+@<Report overflow of the input buffer, and abort@>=
+if (format_ident==0)
+ {@+write_ln(term_out,"Buffer size exceeded!");exit(0);
+ at .Buffer size exceeded@>
+ }
+else{@+cur_input.loc_field=first;cur_input.limit_field=last-1;
+ overflow("buffer size", buf_size);
+@:TeX capacity exceeded buffer size}{\quad buffer size@>
+ }
+
+@ Different systems have different ways to get started. But regardless of
+what conventions are adopted, the routine that initializes the terminal
+should satisfy the following specifications:
+
+\yskip\textindent{1)}It should open file |term_in| for input from the
+ terminal. (The file |term_out| will already be open for output to the
+ terminal.)
+
+\textindent{2)}If the user has given a command line, this line should be
+ considered the first line of terminal input. Otherwise the
+ user should be prompted with `\.{**}', and the first line of input
+ should be whatever is typed in response.
+
+\textindent{3)}The first line of input, which might or might not be a
+ command line, should appear in locations |first| to |last-1| of the
+ |buffer| array.
+
+\textindent{4)}The global variable |loc| should be set so that the
+ character to be read next by \TeX\ is in |buffer[loc]|. This
+ character should not be blank, and we should have |loc < last|.
+
+\yskip\noindent(It may be necessary to prompt the user several times
+before a non-blank line comes in. The prompt is `\.{**}' instead of the
+later `\.*' because the meaning is slightly different: `\.{\\input}' need
+not be typed immediately after~`\.{**}'.)
+
+ at d loc cur_input.loc_field /*location of first unread character in |buffer|*/
+
+@ The following program calls |input_command_line|
+to retrieve a possible command line.
+@^system dependencies@>
+
+ at p static bool init_terminal(void) /*gets the terminal input started*/
+{@+
+t_open_in;
+if (input_command_line()) return true; /* k\TeX\ */
+loop at +{@+wake_up_terminal;pascal_write(term_out,"**");update_terminal;
+ at .**@>
+ if (!input_ln(&term_in, true)) /*this shouldn't happen*/
+ {@+write_ln(term_out);
+ pascal_write(term_out,"! End of file on the terminal... why?");
+ at .End of file on the terminal@>
+ return false;
+ }
+ loc=first;
+ while ((loc < last)&&(buffer[loc]==' ')) incr(loc);
+ if (loc < last)
+ {@+return true;
+ /*return unless the line was all blank*/
+ }
+ write_ln(term_out,"Please type the name of your input file.");
+ }
+}
+
+@* String handling.
+Control sequence names and diagnostic messages are variable-length strings
+of eight-bit characters. Since \PASCAL\ does not have a well-developed string
+mechanism, \TeX\ does all of its string processing by homegrown methods.
+
+Elaborate facilities for dynamic strings are not needed, so all of the
+necessary operations can be handled with a simple data structure.
+The array |str_pool| contains all of the (eight-bit) ASCII codes in all
+of the strings, and the array |str_start| contains indices of the starting
+points of each string. Strings are referred to by integer numbers, so that
+string number |s| comprises the characters |str_pool[j]| for
+|str_start[s] <= j < str_start[s+1]|. Additional integer variables
+|pool_ptr| and |str_ptr| indicate the number of entries used so far
+in |str_pool| and |str_start|, respectively; locations
+|str_pool[pool_ptr]| and |str_start[str_ptr]| are
+ready for the next string to be allocated.
+
+String numbers 0 to 255 are reserved for strings that correspond to single
+ASCII characters. This is in accordance with the conventions of \.{WEB},
+ at .WEB@>
+which converts single-character strings into the ASCII code number of the
+single character involved, while it converts other strings into integers
+and builds a string pool file. Thus, when the string constant \.{"."} appears
+in the program below, \.{WEB} converts it into the integer 46, which is the
+ASCII code for a period, while \.{WEB} will convert a string like \.{"hello"}
+into some integer greater than~255. String number 46 will presumably be the
+single character `\..'; but some ASCII codes have no standard visible
+representation, and \TeX\ sometimes needs to be able to print an arbitrary
+ASCII character, so the first 256 strings are used to specify exactly what
+should be printed for each of the 256 possibilities.
+
+Elements of the |str_pool| array must be ASCII codes that can actually
+be printed; i.e., they must have an |xchr| equivalent in the local
+character set. (This restriction applies only to preloaded strings,
+not to those generated dynamically by the user.)
+
+Some \PASCAL\ compilers won't pack integers into a single byte unless the
+integers lie in the range |-128 dotdot 127|. To accommodate such systems
+we access the string pool only via macros that can easily be redefined.
+@^system dependencies@>
+
+ at d si(A) A /*convert from |ASCII_code| to |packed_ASCII_code|*/
+ at d so(A) A /*convert from |packed_ASCII_code| to |ASCII_code|*/
+
+@<Types...@>=
+typedef int32_t pool_pointer; /*for variables that point into |str_pool|*/
+typedef int32_t str_number; /*for variables that point into |str_start|*/
+typedef uint8_t packed_ASCII_code; /*elements of |str_pool| array*/
+
+@ @<Glob...@>=
+packed_ASCII_code @!str_pool[pool_size+1]; /*the characters*/
+pool_pointer @!str_start[max_strings+1]; /*the starting pointers*/
+static pool_pointer @!pool_ptr; /*first unused position in |str_pool|*/
+static str_number @!str_ptr; /*number of the current string being created*/
+static pool_pointer @!init_pool_ptr; /*the starting value of |pool_ptr|*/
+static str_number @!init_str_ptr; /*the starting value of |str_ptr|*/
+
+@ Several of the elementary string operations are performed using \.{WEB}
+macros instead of \PASCAL\ procedures, because many of the
+operations are done quite frequently and we want to avoid the
+overhead of procedure calls. For example, here is
+a simple macro that computes the length of a string.
+ at .WEB@>
+
+ at d length(A) (str_start[A+1]-str_start[A]) /*the number of characters
+ in string number \#*/
+
+@ The length of the current string is called |cur_length|:
+
+ at d cur_length (pool_ptr-str_start[str_ptr])
+
+@ Strings are created by appending character codes to |str_pool|.
+The |append_char| macro, defined here, does not check to see if the
+value of |pool_ptr| has gotten too high; this test is supposed to be
+made before |append_char| is used. There is also a |flush_char|
+macro, which erases the last character appended.
+
+To test if there is room to append |l| more characters to |str_pool|,
+we shall write |str_room(l)|, which aborts \TeX\ and gives an
+apologetic error message if there isn't enough room.
+
+ at d append_char(A) /*put |ASCII_code| \# at the end of |str_pool|*/
+{@+str_pool[pool_ptr]=si(A);incr(pool_ptr);
+}
+ at d flush_char decr(pool_ptr) /*forget the last character in the pool*/
+ at d str_room(A) /*make sure that the pool hasn't overflowed*/
+ {@+if (pool_ptr+A > pool_size)
+ overflow("pool size", pool_size-init_pool_ptr);
+@:TeX capacity exceeded pool size}{\quad pool size@>
+ }
+
+@ Once a sequence of characters has been appended to |str_pool|, it
+officially becomes a string when the function |make_string| is called.
+This function returns the identification number of the new string as its
+value.
+
+ at p static str_number make_string(void) /*current string enters the pool*/
+{@+if (str_ptr==max_strings)
+ overflow("number of strings", max_strings-init_str_ptr);
+@:TeX capacity exceeded number of strings}{\quad number of strings@>
+incr(str_ptr);str_start[str_ptr]=pool_ptr;
+return str_ptr-1;
+}
+
+@ To destroy the most recently made string, we say |flush_string|.
+
+ at d flush_string {@+decr(str_ptr);pool_ptr=str_start[str_ptr];
+ }
+
+@ The following subroutine compares string |s| with another string of the
+same length that appears in |buffer| starting at position |k|;
+the result is |true| if and only if the strings are equal.
+Empirical tests indicate that |str_eq_buf| is used in such a way that
+it tends to return |true| about 80 percent of the time.
+
+ at p static bool str_eq_buf(str_number @!s, int @!k)
+ /*test equality of strings*/
+{@+ /*loop exit*/
+pool_pointer j; /*running index*/
+bool @!result; /*result of comparison*/
+j=str_start[s];
+while (j < str_start[s+1])
+ {@+if (so(str_pool[j])!=buffer[k])
+ {@+result=false;goto not_found;
+ }
+ incr(j);incr(k);
+ }
+result=true;
+not_found: return result;
+}
+
+@ Here is a similar routine, but it compares two strings in the string pool,
+and it does not assume that they have the same length.
+
+ at p static bool strn_eq_str(str_number @!s, str_number @!t)
+ /*test equality of strings*/
+{@+ /*loop exit*/
+pool_pointer j, @!k; /*running indices*/
+bool @!result; /*result of comparison*/
+result=false;
+if (length(s)!=length(t)) goto not_found;
+j=str_start[s];k=str_start[t];
+while (j < str_start[s+1])
+ {@+if (str_pool[j]!=str_pool[k]) goto not_found;
+ incr(j);incr(k);
+ }
+result=true;
+not_found: return result;
+}
+
+static bool str_eq_str(str_number @!s, char *@!t)
+ /*test equality of strings*/
+{@+int j, @!k; /*running indices*/
+if (length(s)!=(int)strlen(t)) return false;
+j=str_start[s];k=0;
+while (j < str_start[s+1])
+ if (str_pool[j++]!=t[k++]) return false;
+return true;
+}
+
+@ The initial values of |str_pool|, |str_start|, |pool_ptr|,
+and |str_ptr| are computed by the \.{INITEX} program, based in part
+on the information that \.{WEB} has output while processing \TeX.
+ at .INITEX@>
+@^string pool@>
+
+ at p
+static void get_strings_started(void) /*initializes the string pool*/
+{@+
+int k, @!l; /*small indices or counters*/
+pool_ptr=0;str_ptr=0;str_start[0]=0;
+@<Make the first 256 strings@>;
+@<Add the empty string to the string pool@>;
+}
+
+@ @d app_lc_hex(A) l=A;
+ if (l < 10) append_char(l+'0')@;@+else append_char(l-10+'a')
+
+@<Make the first 256...@>=
+for (k=0; k<=255; k++)
+ {@+if ((@<Character |k| cannot be printed@>))
+ {@+append_char('^');append_char('^');
+ if (k < 0100) append_char(k+0100)@;
+ else if (k < 0200) append_char(k-0100)@;
+ else{@+app_lc_hex(k/16);app_lc_hex(k%16);
+ }
+ }
+ else append_char(k);
+ make_string();
+ }
+
+@ The first 128 strings will contain 95 standard ASCII characters, and the
+other 33 characters will be printed in three-symbol form like `\.{\^\^A}'
+unless a system-dependent change is made here. Installations that have
+an extended character set, where for example |xchr[032]==@t\.{\'^^Z\'}@>|,
+would like string 032 to be the single character 032 instead of the
+three characters 0136, 0136, 0132 (\.{\^\^Z}). On the other hand,
+even people with an extended character set will want to represent string
+015 by \.{\^\^M}, since 015 is |carriage_return|; the idea is to
+produce visible strings instead of tabs or line-feeds or carriage-returns
+or bell-rings or characters that are treated anomalously in text files.
+
+Unprintable characters of codes 128--255 are, similarly, rendered
+\.{\^\^80}--\.{\^\^ff}.
+
+The boolean expression defined here should be |true| unless \TeX\
+internal code number~|k| corresponds to a non-troublesome visible
+symbol in the local character set. An appropriate formula for the
+extended character set recommended in {\sl The \TeX book\/} would, for
+example, be `|k in[0, 010 dotdot 012, 014, 015, 033, 0177 dotdot 0377]|'.
+If character |k| cannot be printed, and |k < 0200|, then character |k+0100| or
+|k-0100| must be printable; moreover, ASCII codes |[041 dotdot 046,
+060 dotdot 071, 0136, 0141 dotdot 0146, 0160 dotdot 0171]| must be printable.
+Thus, at least 81 printable characters are needed.
+@:TeXbook}{\sl The \TeX book@>
+@^character set dependencies@>
+@^system dependencies@>
+
+@<Character |k| cannot be printed@>=
+ (k < ' ')||(k > '~')
+
+@ The |pool_file| variable is no longer needed and has been removed.
+
+@ Instead of reading the other strings from the \.{TEX.POOL} file,
+it is sufficient here to add the empty string.
+@<Add the empty string to the string pool@>=
+make_string();
+
+@ Without a string pool file there is no need for a pool check sum either.
+But this is a convenient place to define the function |s_no| that will
+add literal strings to the string pool at runtime, thereby obtaining their
+string number.
+
+ at p int s_no(const char *str)
+{@+
+ if (str[0]==0) return empty_string;
+ if (str[1]==0) return str[0];
+ str_room(strlen(str));
+ while (*str!=0) append_char(*str++);
+ return make_string();
+}
+
+@* On-line and off-line printing.
+Messages that are sent to a user's terminal and to the transcript-log file
+are produced by several `|print|' procedures. These procedures will
+direct their output to a variety of places, based on the setting of
+the global variable |selector|, which has the following possible
+values:
+
+\yskip
+\hang |term_and_log|, the normal setting, prints on the terminal and on the
+ transcript file.
+
+\hang |log_only|, prints only on the transcript file.
+
+\hang |term_only|, prints only on the terminal.
+
+\hang |no_print|, doesn't print at all. This is used only in rare cases
+ before the transcript file is open.
+
+\hang |pseudo|, puts output into a cyclic buffer that is used
+ by the |show_context| routine; when we get to that routine we shall discuss
+ the reasoning behind this curious mode.
+
+\hang |new_string|, appends the output to the current string in the
+ string pool.
+
+\hang 0 to 15, prints on one of the sixteen files for \.{\\write} output.
+
+\yskip
+\noindent The symbolic names `|term_and_log|', etc., have been assigned
+numeric codes that satisfy the convenient relations |no_print+1==term_only|,
+|no_print+2==log_only|, |term_only+2==log_only+1==term_and_log|.
+
+Three additional global variables, |tally| and |term_offset| and
+|file_offset|, record the number of characters that have been printed
+since they were most recently cleared to zero. We use |tally| to record
+the length of (possibly very long) stretches of printing; |term_offset|
+and |file_offset|, on the other hand, keep track of how many characters
+have appeared so far on the current line that has been output to the
+terminal or to the transcript file, respectively.
+
+ at d no_print 16 /*|selector| setting that makes data disappear*/
+ at d term_only 17 /*printing is destined for the terminal only*/
+ at d log_only 18 /*printing is destined for the transcript file only*/
+ at d term_and_log 19 /*normal |selector| setting*/
+ at d pseudo 20 /*special |selector| setting for |show_context|*/
+ at d new_string 21 /*printing is deflected to the string pool*/
+ at d max_selector 21 /*highest selector setting*/
+
+@<Glob...@>=
+static alpha_file @!log_file; /*transcript of \TeX\ session*/
+static int @!selector; /*where to print a message*/
+static int8_t @!dig[23]; /*digits in a number being output*/
+static int @!tally; /*the number of characters recently printed*/
+static int @!term_offset;
+ /*the number of characters on the current terminal line*/
+static int @!file_offset;
+ /*the number of characters on the current file line*/
+static ASCII_code @!trick_buf[error_line+1]; /*circular buffer for
+ pseudoprinting*/
+static int @!trick_count; /*threshold for pseudoprinting, explained later*/
+static int @!first_count; /*another variable for pseudoprinting*/
+
+@ @<Initialize the output routines@>=
+selector=term_only;tally=0;term_offset=0;file_offset=0;
+
+@ Macro abbreviations for output to the terminal and to the log file are
+defined here for convenience. Some systems need special conventions
+for terminal output, and it is possible to adhere to those conventions
+by changing |wterm|, |wterm_ln|, and |wterm_cr| in this section.
+@^system dependencies@>
+
+@<Basic printing procedures@>=
+#define put(F) @[fwrite(&((F).d),sizeof((F).d),1,(F).f)@]
+#define get(F) @[fread(&((F).d),sizeof((F).d),1,(F).f)@]
+
+#define pascal_close(F) @[fclose((F).f)@]
+#define eof(F) @[feof((F).f)@]
+#define eoln(F) @[((F).d=='\n'||eof(F))@]
+#define erstat(F) @[((F).f==NULL?-1:ferror((F).f))@]
+
+#define pascal_read(F,X) @[((X)=(F).d,get(F))@]
+#define read_ln(F) @[do get(F); while (!eoln(F))@]
+
+#define pascal_write(F, FMT,...) @[fprintf(F.f,FMT,## __VA_ARGS__)@]
+#define write_ln(F,...) @[pascal_write(F,__VA_ARGS__"\n")@]
+
+#define wterm(FMT,...) @[pascal_write(term_out,FMT, ## __VA_ARGS__)@]
+#define wterm_ln(FMT,...) @[wterm(FMT "\n", ## __VA_ARGS__)@]
+#define wterm_cr @[pascal_write(term_out,"\n")@]
+#define wlog(FMT, ...) @[pascal_write(log_file,FMT, ## __VA_ARGS__)@]
+#define wlog_ln(FMT, ...) @[wlog(FMT "\n", ## __VA_ARGS__)@]
+#define wlog_cr @[pascal_write(log_file,"\n")@]
+
+@ To end a line of text output, we call |print_ln|.
+
+@<Basic print...@>=
+void print_ln(void) /*prints an end-of-line*/
+{@+switch (selector) {
+case term_and_log: {@+wterm_cr;wlog_cr;
+ term_offset=0;file_offset=0;
+ } @+break;
+case log_only: {@+wlog_cr;file_offset=0;
+ } @+break;
+case term_only: {@+wterm_cr;term_offset=0;
+ } @+break;
+case no_print: case pseudo: case new_string: do_nothing;@+break;
+default:write_ln(write_file[selector]);
+} @/
+} /*|tally| is not affected*/
+
+@ The |print_char| procedure sends one character to the desired destination,
+using the |xchr| array to map it into an external character compatible with
+|input_ln|. All printing comes through |print_ln| or |print_char|.
+
+@<Basic printing...@>=
+void print_char(ASCII_code @!s) /*prints a single character*/
+{@+
+if (@<Character |s| is the current new-line character@>)
+ if (selector < pseudo)
+ {@+print_ln();return;
+ }
+switch (selector) {
+case term_and_log: {@+wterm("%c",xchr[s]);wlog("%c",xchr[s]);
+ incr(term_offset);incr(file_offset);
+ if (term_offset==max_print_line)
+ {@+wterm_cr;term_offset=0;
+ }
+ if (file_offset==max_print_line)
+ {@+wlog_cr;file_offset=0;
+ }
+ } @+break;
+case log_only: {@+wlog("%c",xchr[s]);incr(file_offset);
+ if (file_offset==max_print_line) print_ln();
+ } @+break;
+case term_only: {@+wterm("%c",xchr[s]);incr(term_offset);
+ if (term_offset==max_print_line) print_ln();
+ } @+break;
+case no_print: do_nothing;@+break;
+case pseudo: if (tally < trick_count) trick_buf[tally%error_line]=s;@+break;
+case new_string: {@+if (pool_ptr < pool_size) append_char(s);
+ } @+break; /*we drop characters if the string space is full*/
+default:pascal_write(write_file[selector],"%c", xchr[s]);
+} @/
+incr(tally);
+}
+
+@ An entire string is output by calling |print|. Note that if we are outputting
+the single standard ASCII character \.c, we could call |print('c')|, since
+|'c'==99| is the number of a single-character string, as explained above. But
+|print_char('c')| is quicker, so \TeX\ goes directly to the |print_char|
+routine when it knows that this is safe. (The present implementation
+assumes that it is always safe to print a visible ASCII character.)
+@^system dependencies@>
+
+@<Basic print...@>=
+void print(char *s) /* the simple version */
+{ while (*s!=0) print_char(*s++);@+
+}
+
+static void printn(int @!s) /*prints string |s|*/
+{@+
+pool_pointer j; /*current character code position*/
+int @!nl; /*new-line character to restore*/
+if (s >= str_ptr) {print("???"); return;}/*this can't happen*/
+ at .???@>
+else if (s < 256)
+ if (s < 0) { print("???");return; } /*can't happen*/
+ else{@+if (selector > pseudo)
+ {@+print_char(s);return; /*internal strings are not expanded*/
+ }
+ if ((@<Character |s| is the current new-line character@>))
+ if (selector < pseudo)
+ {@+print_ln();return;
+ }
+ nl=new_line_char;new_line_char=-1;
+ /*temporarily disable new-line character*/
+ j=str_start[s];
+ while (j < str_start[s+1])
+ {@+print_char(so(str_pool[j]));incr(j);
+ }
+ new_line_char=nl;return;
+ }
+j=str_start[s];
+while (j < str_start[s+1])
+ {@+print_char(so(str_pool[j]));incr(j);
+ }
+}
+
+@ Control sequence names, file names, and strings constructed with
+\.{\\string} might contain |ASCII_code| values that can't
+be printed using |print_char|. Therefore we use |slow_print| for them:
+
+@<Basic print...@>=
+static void slow_print(int @!s) /*prints string |s|*/
+{@+pool_pointer j; /*current character code position*/
+if ((s >= str_ptr)||(s < 256)) printn(s);
+else{@+j=str_start[s];
+ while (j < str_start[s+1])
+ {@+printn(so(str_pool[j]));incr(j);
+ }
+ }
+}
+
+@ Here is the very first thing that \TeX\ prints: a headline that identifies
+the version number and format package. The |term_offset| variable is temporarily
+incorrect, but the discrepancy is not serious since we assume that the banner
+and format identifier together will occupy at most |max_print_line|
+character positions.
+
+k\TeX, according to the conventions of \TeX\ Live,
+ prints the |dump_name| if no format identifier is known.
+@<Initialize the output...@>=
+wterm("%s",banner);
+if (format_ident==0) wterm_ln(" (preloaded format=%s)", dump_name);
+else{@+slow_print(format_ident);print_ln();
+ }
+update_terminal;
+
+@ The procedure |print_nl| is like |print|, but it makes sure that the
+string appears at the beginning of a new line.
+
+@<Basic print...@>=
+void print_nl(char *@!s) /*prints string |s| at beginning of line*/
+{@+if (((term_offset > 0)&&(odd(selector)))||@|
+ ((file_offset > 0)&&(selector >= log_only))) print_ln();
+print(s);
+}
+
+@ The procedure |print_esc| prints a string that is preceded by
+the user's escape character (which is usually a backslash).
+
+@<Basic print...@>=
+static void printn_esc(str_number @!s) /*prints escape character, then |s|*/
+{@+int c; /*the escape character code*/
+@<Set variable |c| to the current escape character@>;
+if (c >= 0) if (c < 256) printn(c);
+slow_print(s);
+}
+
+static void print_esc(char *@!s) /*the fast way*/
+{@+int c; /*the escape character code*/
+@<Set variable |c| to the current escape character@>;
+if (c >= 0) if (c < 256) printn(c);
+print(s);
+}
+
+@ An array of digits in the range |0 dotdot 15| is printed by |print_the_digs|.
+
+@<Basic print...@>=
+static void print_the_digs(eight_bits @!k)
+ /*prints |dig[k-1]|$\,\ldots\,$|dig[0]|*/
+{@+while (k > 0)
+ {@+decr(k);
+ if (dig[k] < 10) print_char('0'+dig[k]);
+ else print_char('A'-10+dig[k]);
+ }
+}
+
+@ The following procedure, which prints out the decimal representation of a
+given integer |n|, has been written carefully so that it works properly
+if |n==0| or if |(-n)| would cause overflow. It does not apply |%| or |/|
+to negative arguments, since such operations are not implemented consistently
+by all \PASCAL\ compilers.
+
+@<Basic print...@>=
+void print_int(int @!n) /*prints an integer in decimal form*/
+{@+int k; /*index to current digit; we assume that $|n|<10^{23}$*/
+int @!m; /*used to negate |n| in possibly dangerous cases*/
+k=0;
+if (n < 0)
+ {@+print_char('-');
+ if (n > -100000000) negate(n);
+ else{@+m=-1-n;n=m/10;m=(m%10)+1;k=1;
+ if (m < 10) dig[0]=m;
+ else{@+dig[0]=0;incr(n);
+ }
+ }
+ }
+@/do at +{dig[k]=n%10;n=n/10;incr(k);
+}@+ while (!(n==0));
+print_the_digs(k);
+}
+
+@ Here is a trivial procedure to print two digits; it is usually called with
+a parameter in the range |0 <= n <= 99|.
+
+ at p static void print_two(int @!n) /*prints two least significant digits*/
+{@+n=abs(n)%100;print_char('0'+(n/10));
+print_char('0'+(n%10));
+}
+
+@ Hexadecimal printing of nonnegative integers is accomplished by |print_hex|.
+
+ at p static void print_hex(int @!n)
+ /*prints a positive integer in hexadecimal form*/
+{@+int k; /*index to current digit; we assume that $0\le n<16^{22}$*/
+k=0;print_char('"');
+@/do at +{dig[k]=n%16;n=n/16;incr(k);
+}@+ while (!(n==0));
+print_the_digs(k);
+}
+
+@ Old versions of \TeX\ needed a procedure called |print_ASCII| whose function
+is now subsumed by |print|. We retain the old name here as a possible aid to
+future software arch\ae ologists.
+
+ at d print_ASCII printn
+
+@ Roman numerals are produced by the |print_roman_int| routine. Readers
+who like puzzles might enjoy trying to figure out how this tricky code
+works; therefore no explanation will be given. Notice that 1990 yields
+\.{mcmxc}, not \.{mxm}.
+
+ at p static void print_roman_int(int @!n)
+{@+
+pool_pointer j, @!k; /*mysterious indices into |mystery|*/
+nonnegative_integer @!u, @!v; /*mysterious numbers*/
+const char mystery[] ="m2d5c2l5x2v5i";
+j=0;v=1000;
+loop at +{@+while (n >= v)
+ {@+print_char(so(mystery[j]));n=n-v;
+ }
+ if (n <= 0) return; /*nonpositive input produces no output*/
+ k=j+2;u=v/(so(mystery[k-1])-'0');
+ if (mystery[k-1]==si('2'))
+ {@+k=k+2;u=u/(so(mystery[k-1])-'0');
+ }
+ if (n+u >= v)
+ {@+print_char(so(mystery[k]));n=n+u;
+ }
+ else{@+j=j+2;v=v/(so(mystery[j-1])-'0');
+ }
+ }
+}
+
+@ The |print| subroutine will not print a string that is still being
+created. The following procedure will.
+
+ at p static void print_current_string(void) /*prints a yet-unmade string*/
+{@+pool_pointer j; /*points to current character code*/
+j=str_start[str_ptr];
+while (j < pool_ptr)
+ {@+print_char(so(str_pool[j]));incr(j);
+ }
+}
+
+@ Here is a procedure that asks the user to type a line of input,
+assuming that the |selector| setting is either |term_only| or |term_and_log|.
+The input is placed into locations |first| through |last-1| of the
+|buffer| array, and echoed on the transcript file if appropriate.
+
+This procedure is never called when |interaction < scroll_mode|.
+
+ at d prompt_input(A) {@+wake_up_terminal;print(A);term_input();
+ } /*prints a string and gets a line of input*/
+
+ at p static void term_input(void) /*gets a line from the terminal*/
+{@+int k; /*index into |buffer|*/
+update_terminal; /*now the user sees the prompt for sure*/
+if (!input_ln(&term_in, true)) fatal_error("End of file on the terminal!");
+ at .End of file on the terminal@>
+term_offset=0; /*the user's line ended with \<\rm return>*/
+decr(selector); /*prepare to echo the input*/
+if (last!=first) for (k=first; k<=last-1; k++) printn(buffer[k]);
+print_ln();incr(selector); /*restore previous status*/
+}
+
+@* Reporting errors.
+When something anomalous is detected, \TeX\ typically does something like this:
+$$\vbox{\halign{#\hfil\cr
+|print_err("Something anomalous has been detected");|\cr
+|help3("This is the first line of my offer to help.")|\cr
+|("This is the second line. I'm trying to")|\cr
+|("explain the best way for you to proceed.");|\cr
+|error;|\cr}}$$
+A two-line help message would be given using |help2|, etc.; these informal
+helps should use simple vocabulary that complements the words used in the
+official error message that was printed. (Outside the U.S.A., the help
+messages should preferably be translated into the local vernacular. Each
+line of help is at most 60 characters long, in the present implementation,
+so that |max_print_line| will not be exceeded.)
+
+The |print_err| procedure supplies a `\.!' before the official message,
+and makes sure that the terminal is awake if a stop is going to occur.
+The |error| procedure supplies a `\..' after the official message, then it
+shows the location of the error; and if |interaction==error_stop_mode|,
+it also enters into a dialog with the user, during which time the help
+message may be printed.
+@^system dependencies@>
+
+@<Error handling...@>=
+void print_err(char *s)
+{@+if (interaction==error_stop_mode) wake_up_terminal;
+ if (filelineerrorstylep) print_file_line(); /* k\TeX\ */
+ else print_nl("! ");
+ print(s);
+}
+
+@ The global variable |interaction| has four settings, representing increasing
+amounts of user interaction:
+
+ at d batch_mode 0 /*omits all stops and omits terminal output*/
+ at d nonstop_mode 1 /*omits all stops*/
+ at d scroll_mode 2 /*omits error stops*/
+ at d error_stop_mode 3 /*stops at every opportunity to interact*/
+
+@<Glob...@>=
+int @!interaction; /*current level of interaction*/
+
+@ @<Set init...@>=interaction=error_stop_mode;
+
+@ \TeX\ is careful not to call |error| when the print |selector| setting
+might be unusual. The only possible values of |selector| at the time of
+error messages are
+
+\yskip\hang|no_print| (when |interaction==batch_mode|
+ and |log_file| not yet open);
+
+\hang|term_only| (when |interaction > batch_mode| and |log_file| not yet open);
+
+\hang|log_only| (when |interaction==batch_mode| and |log_file| is open);
+
+\hang|term_and_log| (when |interaction > batch_mode| and |log_file| is open).
+
+@<Initialize the print |selector| based on |interaction|@>=
+if (interaction==batch_mode) selector=no_print;@+else selector=term_only
+
+@ A global variable |deletions_allowed| is set |false| if the |get_next|
+routine is active when |error| is called; this ensures that |get_next|
+and related routines like |get_token| will never be called recursively.
+A similar interlock is provided by |set_box_allowed|.
+@^recursion@>
+
+The global variable |history| records the worst level of error that
+has been detected. It has four possible values: |spotless|, |warning_issued|,
+|error_message_issued|, and |fatal_error_stop|.
+
+Another global variable, |error_count|, is increased by one when an
+|error| occurs without an interactive dialog, and it is reset to zero at
+the end of every paragraph. If |error_count| reaches 100, \TeX\ decides
+that there is no point in continuing further.
+
+ at d spotless 0 /*|history| value when nothing has been amiss yet*/
+ at d warning_issued 1 /*|history| value when |begin_diagnostic| has been called*/
+ at d error_message_issued 2 /*|history| value when |error| has been called*/
+ at d fatal_error_stop 3 /*|history| value when termination was premature*/
+
+@<Glob...@>=
+static bool @!deletions_allowed; /*is it safe for |error| to call |get_token|?*/
+static bool @!set_box_allowed; /*is it safe to do a \.{\\setbox} assignment?*/
+static int @!history; /*has the source input been clean so far?*/
+static int @!error_count; /*the number of scrolled errors since the
+ last paragraph ended*/
+
+@ The value of |history| is initially |fatal_error_stop|, but it will
+be changed to |spotless| if \TeX\ survives the initialization process.
+
+@<Set init...@>=
+deletions_allowed=true;set_box_allowed=true;
+error_count=0; /*|history| is initialized elsewhere*/
+
+@ Since errors can be detected almost anywhere in \TeX, we want to declare the
+error procedures near the beginning of the program. But the error procedures
+in turn use some other procedures, which need to be declared |forward|
+before we get to |error| itself.
+
+It is possible for |error| to be called recursively if some error arises
+when |get_token| is being used to delete a token, and/or if some fatal error
+occurs while \TeX\ is trying to fix a non-fatal one. But such recursion
+@^recursion@>
+is never more than two levels deep.
+
+@<Error handling...@>=
+static void normalize_selector(void);@/
+static void get_token(void);@/
+static void term_input(void);@/
+static void show_context(void);@/
+static void begin_file_reading(void);@/
+void open_log_file(void);@/
+static void close_files_and_terminate(void);@/
+static void clear_for_error_prompt(void);@/
+static void give_err_help(void);@/
+#ifdef @!DEBUG
+static void debug_help(void);
+#else
+#define debug_help() do_nothing
+#endif
+
+@ Individual lines of help are recorded in the array |help_line|, which
+contains entries in positions |0 dotdot(help_ptr-1)|. They should be printed
+in reverse order, i.e., with |help_line[0]| appearing last.
+
+ at d hlp1(A) help_line[0]=A;@+}
+ at d hlp2(A, B) help_line[1]=A;help_line[0]=B;@+}
+ at d hlp3(A, B, C) help_line[2]=A;help_line[1]=B;help_line[0]=C;@+}
+ at d hlp4(A, B, C, D) help_line[3]=A;help_line[2]=B;help_line[1]=C;help_line[0]=D;@+}
+ at d hlp5(A, B, C, D, E) help_line[4]=A;help_line[3]=B;help_line[2]=C;help_line[1]=D;help_line[0]=E;@+}
+ at d hlp6(A, B, C, D, E, F) help_line[5]=A;help_line[4]=B;help_line[3]=C;help_line[2]=D;help_line[1]=E;help_line[0]=F;@+}
+ at d help0 help_ptr=0 /*sometimes there might be no help*/
+ at d help1(A) @+{@+help_ptr=1;hlp1(A) /*use this with one help line*/
+ at d help2(A, B) @+{@+help_ptr=2;hlp2(A, B) /*use this with two help lines*/
+ at d help3(A, B, C) @+{@+help_ptr=3;hlp3(A, B, C) /*use this with three help lines*/
+ at d help4(A, B, C, D) @+{@+help_ptr=4;hlp4(A, B, C, D) /*use this with four help lines*/
+ at d help5(A, B, C, D, E) @+{@+help_ptr=5;hlp5(A, B, C, D, E) /*use this with five help lines*/
+ at d help6(A, B, C, D, E, F) @+{@+help_ptr=6;hlp6(A, B, C, D, E, F) /*use this with six help lines*/
+
+@<Glob...@>=
+static char *@!help_line[6]; /*helps for the next |error|*/
+static int @!help_ptr; /*the number of help lines present*/
+static bool @!use_err_help; /*should the |err_help| list be shown?*/
+
+@ @<Set init...@>=
+help_ptr=0;use_err_help=false;
+
+@ The |jump_out| procedure just cuts across all active procedure levels and
+goes to |end_of_TEX|. This is the only nontrivial |@!goto| statement in the
+whole program. It is used when there is no recovery from a particular error.
+
+Some \PASCAL\ compilers do not implement non-local |goto| statements.
+@^system dependencies@>
+In such cases the body of |jump_out| should simply be
+`|close_files_and_terminate|;\thinspace' followed by a call on some system
+procedure that quietly terminates the program.
+
+@<Error hand...@>=
+static void jump_out(void)
+{@+ close_files_and_terminate(); exit(0);
+}
+
+@ Here now is the general |error| routine.
+
+@<Error hand...@>=
+static void error(void) /*completes the job of error reporting*/
+{@+
+ASCII_code c; /*what the user types*/
+int @!s1, @!s2, @!s3, @!s4;
+ /*used to save global variables when deleting tokens*/
+if (history < error_message_issued) history=error_message_issued;
+print_char('.');show_context();
+if (interaction==error_stop_mode) @<Get user's advice and |return|@>;
+incr(error_count);
+if (error_count==100)
+ {@+print_nl("(That makes 100 errors; please try again.)");
+ at .That makes 100 errors...@>
+ history=fatal_error_stop;jump_out();
+ }
+@<Put help message on the transcript file@>;
+}
+
+@ @<Get user's advice...@>=
+loop at +{@+resume: clear_for_error_prompt();prompt_input("? ");
+ at .?\relax@>
+ if (last==first) return;
+ c=buffer[first];
+ if (c >= 'a') c=c+'A'-'a'; /*convert to uppercase*/
+ @<Interpret code |c| and |return| if done@>;
+ }
+
+@ It is desirable to provide an `\.E' option here that gives the user
+an easy way to return from \TeX\ to the system editor, with the offending
+line ready to be edited. But such an extension requires some system
+wizardry, so the present implementation simply types out the name of the
+file that should be
+edited and the relevant line number.
+@^system dependencies@>
+
+There is a secret `\.D' option available when the debugging routines haven't
+been commented~out.
+@^debugging@>
+
+@<Interpret code |c| and |return| if done@>=
+switch (c) {
+case '0': case '1': case '2': case '3': case '4': case '5': case '6': case '7': case '8': case '9': if (deletions_allowed)
+ @<Delete \(c)|c-"0"| tokens and |goto continue|@>@;@+break;
+ at t\4\4@>@;
+#ifdef @!DEBUG
+case 'D': {@+debug_help();goto resume;@+}
+#endif
+case 'E': if (base_ptr > 0)
+ {@+print_nl("You want to edit file ");
+ at .You want to edit file x@>
+ slow_print(input_stack[base_ptr].name_field);
+ print(" at line ");print_int(line);
+ interaction=scroll_mode;jump_out();
+ } @+break;
+case 'H': @<Print the help information and |goto continue|@>@;
+case 'I': @<Introduce new material from the terminal and |return|@>@;
+case 'Q': case 'R': case 'S': @<Change the interaction level and |return|@>@;
+case 'X': {@+interaction=scroll_mode;jump_out();
+ } @+break;
+default:do_nothing;
+} @/
+@<Print the menu of available options@>@;
+
+@ @<Print the menu...@>=
+{@+print("Type <return> to proceed, S to scroll future error messages,");@/
+ at .Type <return> to proceed...@>
+print_nl("R to run without stopping, Q to run quietly,");@/
+print_nl("I to insert something, ");
+if (base_ptr > 0) print("E to edit your file,");
+if (deletions_allowed)
+ print_nl("1 or ... or 9 to ignore the next 1 to 9 tokens of input,");
+print_nl("H for help, X to quit.");
+}
+
+@ Here the author of \TeX\ apologizes for making use of the numerical
+relation between |'Q'|, |'R'|, |'S'|, and the desired interaction settings
+|batch_mode|, |nonstop_mode|, |scroll_mode|.
+@^Knuth, Donald Ervin@>
+
+@<Change the interaction...@>=
+{@+error_count=0;interaction=batch_mode+c-'Q';
+print("OK, entering ");
+switch (c) {
+case 'Q': {@+print_esc("batchmode");decr(selector);
+ } @+break;
+case 'R': print_esc("nonstopmode");@+break;
+case 'S': print_esc("scrollmode");
+} /*there are no other cases*/
+print("...");print_ln();update_terminal;return;
+}
+
+@ When the following code is executed, |buffer[(first+1)dotdot(last-1)]| may
+contain the material inserted by the user; otherwise another prompt will
+be given. In order to understand this part of the program fully, you need
+to be familiar with \TeX's input stacks.
+
+@<Introduce new material...@>=
+{@+begin_file_reading(); /*enter a new syntactic level for terminal input*/
+ /*now |state==mid_line|, so an initial blank space will count as a blank*/
+if (last > first+1)
+ {@+loc=first+1;buffer[first]=' ';
+ }
+else{@+prompt_input("insert>");loc=first;
+ at .insert>@>
+ }
+first=last;
+cur_input.limit_field=last-1; /*no |end_line_char| ends this line*/
+return;
+}
+
+@ We allow deletion of up to 99 tokens at a time.
+
+@<Delete \(c)|c-"0"| tokens...@>=
+{@+s1=cur_tok;s2=cur_cmd;s3=cur_chr;s4=align_state;
+align_state=1000000;OK_to_interrupt=false;
+if ((last > first+1)&&(buffer[first+1] >= '0')&&(buffer[first+1] <= '9'))
+ c=c*10+buffer[first+1]-'0'*11;
+else c=c-'0';
+while (c > 0)
+ {@+get_token(); /*one-level recursive call of |error| is possible*/
+ decr(c);
+ }
+cur_tok=s1;cur_cmd=s2;cur_chr=s3;align_state=s4;OK_to_interrupt=true;
+help2("I have just deleted some text, as you asked.",@/
+"You can now delete more, or insert, or whatever.");
+show_context();goto resume;
+}
+
+@ @<Print the help info...@>=
+{@+if (use_err_help)
+ {@+give_err_help();use_err_help=false;
+ }
+else{@+if (help_ptr==0)
+ help2("Sorry, I don't know how to help in this situation.",@/
+ @t\kern1em@>"Maybe you should try asking a human?");
+ @/do at +{decr(help_ptr);print(help_line[help_ptr]);print_ln();
+ }@+ while (!(help_ptr==0));
+ }
+help4("Sorry, I already gave what help I could...",@/
+ "Maybe you should try asking a human?",@/
+ "An error might have occurred before I noticed any problems.",@/
+ "``If all else fails, read the instructions.'");@/
+goto resume;
+}
+
+@ @<Put help message on the transcript file@>=
+if (interaction > batch_mode) decr(selector); /*avoid terminal output*/
+if (use_err_help)
+ {@+print_ln();give_err_help();
+ }
+else while (help_ptr > 0)
+ {@+decr(help_ptr);print_nl(help_line[help_ptr]);
+ }
+print_ln();
+if (interaction > batch_mode) incr(selector); /*re-enable terminal output*/
+print_ln()
+
+@ A dozen or so error messages end with a parenthesized integer, so we
+save a teeny bit of program space by declaring the following procedure:
+
+ at p static void int_error(int @!n)
+{@+print(" (");print_int(n);print_char(')');error();
+}
+
+@ In anomalous cases, the print selector might be in an unknown state;
+the following subroutine is called to fix things just enough to keep
+running a bit longer.
+
+ at p static void normalize_selector(void)
+{@+if (log_opened) selector=term_and_log;
+else selector=term_only;
+if (job_name==0) open_log_file();
+if (interaction==batch_mode) decr(selector);
+}
+
+@ The following procedure prints \TeX's last words before dying.
+
+ at d succumb {@+if (interaction==error_stop_mode)
+ interaction=scroll_mode; /*no more interaction*/
+ if (log_opened) error();
+ if (interaction > batch_mode) debug_help();
+ history=fatal_error_stop;jump_out(); /*irrecoverable error*/
+ }
+
+@<Error hand...@>=
+void fatal_error(char *@!s) /*prints |s|, and that's it*/
+{@+normalize_selector();@/
+print_err("Emergency stop");help1(s);succumb;
+ at .Emergency stop@>
+}
+
+@ Here is the most dreaded error message.
+
+@<Error hand...@>=
+void overflow(char *@!s, int @!n) /*stop due to finiteness*/
+{@+normalize_selector();
+print_err("TeX capacity exceeded, sorry [");
+ at .TeX capacity exceeded ...@>
+print(s);print_char('=');print_int(n);print_char(']');
+help2("If you really absolutely need more capacity,",@/
+ "you can ask a wizard to enlarge me.");
+succumb;
+}
+
+@ The program might sometime run completely amok, at which point there is
+no choice but to stop. If no previous error has been detected, that's bad
+news; a message is printed that is really intended for the \TeX\
+maintenance person instead of the user (unless the user has been
+particularly diabolical). The index entries for `this can't happen' may
+help to pinpoint the problem.
+@^dry rot@>
+
+@<Error hand...@>=
+void confusion(char *@!s)
+ /*consistency check violated; |s| tells where*/
+{@+normalize_selector();
+if (history < error_message_issued)
+ {@+print_err("This can't happen (");print(s);print_char(')');
+ at .This can't happen@>
+ help1("I'm broken. Please show this to someone who can fix can fix");
+ }
+else{@+print_err("I can't go on meeting you like this");
+ at .I can't go on...@>
+ help2("One of your faux pas seems to have wounded me deeply...",@/
+ "in fact, I'm barely conscious. Please fix it and try again.");
+ }
+succumb;
+}
+
+@ Users occasionally want to interrupt \TeX\ while it's running.
+If the \PASCAL\ runtime system allows this, one can implement
+a routine that sets the global variable |interrupt| to some nonzero value
+when such an interrupt is signalled. Otherwise there is probably at least
+a way to make |interrupt| nonzero using the \PASCAL\ debugger.
+@^system dependencies@>
+@^debugging@>
+
+ at d check_interrupt {@+if (interrupt!=0) pause_for_instructions();
+ }
+
+@<Global...@>=
+static int @!interrupt; /*should \TeX\ pause for instructions?*/
+static bool @!OK_to_interrupt; /*should interrupts be observed?*/
+
+@ @<Set init...@>=
+interrupt=0;OK_to_interrupt=true;
+
+@ When an interrupt has been detected, the program goes into its
+highest interaction level and lets the user have nearly the full flexibility of
+the |error| routine. \TeX\ checks for interrupts only at times when it is
+safe to do this.
+
+ at p static void pause_for_instructions(void)
+{@+if (OK_to_interrupt)
+ {@+interaction=error_stop_mode;
+ if ((selector==log_only)||(selector==no_print))
+ incr(selector);
+ print_err("Interruption");
+ at .Interruption@>
+ help3("You rang?",@/
+ "Try to insert some instructions for me (e.g.,`I\\showlists'),",@/
+ "unless you just want to quit by typing `X'.");
+ deletions_allowed=false;error();deletions_allowed=true;
+ interrupt=0;
+ }
+}
+
+@* Arithmetic with scaled dimensions.
+The principal computations performed by \TeX\ are done entirely in terms of
+integers less than $2^{31}$ in magnitude; and divisions are done only when both
+dividend and divisor are nonnegative. Thus, the arithmetic specified in this
+program can be carried out in exactly the same way on a wide variety of
+computers, including some small ones. Why? Because the arithmetic
+calculations need to be spelled out precisely in order to guarantee that
+\TeX\ will produce identical output on different machines. If some
+quantities were rounded differently in different implementations, we would
+find that line breaks and even page breaks might occur in different places.
+Hence the arithmetic of \TeX\ has been designed with care, and systems that
+claim to be implementations of \TeX82 should follow precisely the
+@:TeX82}{\TeX82@>
+calculations as they appear in the present program.
+
+(Actually there are three places where \TeX\ uses |/| with a possibly negative
+numerator. These are harmless; see |/| in the index. Also if the user
+sets the \.{\\time} or the \.{\\year} to a negative value, some diagnostic
+information will involve negative-numerator division. The same remarks
+apply for |%| as well as for |/|.)
+
+@ Here is a routine that calculates half of an integer, using an
+unambiguous convention with respect to signed odd numbers.
+
+ at p static int half(int @!x)
+{@+if (odd(x)) return(x+1)/2;
+else return x/2;
+}
+
+@ Fixed-point arithmetic is done on {\sl scaled integers\/} that are multiples
+of $2^{-16}$. In other words, a binary point is assumed to be sixteen bit
+positions from the right end of a binary computer word.
+
+ at d unity 0200000 /*$2^{16}$, represents 1.00000*/
+ at d two 0400000 /*$2^{17}$, represents 2.00000*/
+
+@<Types...@>=
+typedef int scaled; /*this type is used for scaled integers*/
+typedef int32_t nonnegative_integer; /*$0\le x<2^{31}$*/
+typedef int8_t small_number; /*this type is self-explanatory*/
+
+@ The following function is used to create a scaled integer from a given decimal
+fraction $(.d_0d_1\ldots d_{k-1})$, where |0 <= k <= 17|. The digit $d_i$ is
+given in |dig[i]|, and the calculation produces a correctly rounded result.
+
+ at p static scaled round_decimals(small_number @!k)
+ /*converts a decimal fraction*/
+{@+int a; /*the accumulator*/
+a=0;
+while (k > 0)
+ {@+decr(k);a=(a+dig[k]*two)/10;
+ }
+return(a+1)/2;
+}
+
+@ Conversely, here is a procedure analogous to |print_int|. If the output
+of this procedure is subsequently read by \TeX\ and converted by the
+|round_decimals| routine above, it turns out that the original value will
+be reproduced exactly; the ``simplest'' such decimal number is output,
+but there is always at least one digit following the decimal point.
+
+The invariant relation in the \&{repeat} loop is that a sequence of
+decimal digits yet to be printed will yield the original number if and only if
+they form a fraction~$f$ in the range $s-\delta\le 10\cdot2^{16}f<s$.
+We can stop if and only if $f=0$ satisfies this condition; the loop will
+terminate before $s$ can possibly become zero.
+
+ at p void print_scaled(scaled @!s) /*prints scaled real, rounded to five
+ digits*/
+{@+scaled delta; /*amount of allowable inaccuracy*/
+if (s < 0)
+ {@+print_char('-');negate(s); /*print the sign, if negative*/
+ }
+print_int(s/unity); /*print the integer part*/
+print_char('.');
+s=10*(s%unity)+5;delta=10;
+@/do at +{if (delta > unity) s=s+0100000-50000; /*round the last digit*/
+print_char('0'+(s/unity));s=10*(s%unity);delta=delta*10;
+}@+ while (!(s <= delta));
+}
+
+@ Physical sizes that a \TeX\ user specifies for portions of documents are
+represented internally as scaled points. Thus, if we define an `sp' (scaled
+@^sp@>
+point) as a unit equal to $2^{-16}$ printer's points, every dimension
+inside of \TeX\ is an integer number of sp. There are exactly
+4,736,286.72 sp per inch. Users are not allowed to specify dimensions
+larger than $2^{30}-1$ sp, which is a distance of about 18.892 feet (5.7583
+meters); two such quantities can be added without overflow on a 32-bit
+computer.
+
+The present implementation of \TeX\ does not check for overflow when
+@^overflow in arithmetic@>
+dimensions are added or subtracted. This could be done by inserting a
+few dozen tests of the form `\ignorespaces|if (x >= 010000000000)|
+\\{report\_overflow}', but the chance of overflow is so remote that
+such tests do not seem worthwhile.
+
+\TeX\ needs to do only a few arithmetic operations on scaled quantities,
+other than addition and subtraction, and the following subroutines do most of
+the work. A single computation might use several subroutine calls, and it is
+desirable to avoid producing multiple error messages in case of arithmetic
+overflow; so the routines set the global variable |arith_error| to |true|
+instead of reporting errors directly to the user. Another global variable,
+|rem|, holds the remainder after a division.
+
+@<Glob...@>=
+static bool @!arith_error; /*has arithmetic overflow occurred recently?*/
+static scaled @!rem; /*amount subtracted to get an exact division*/
+
+@ The first arithmetical subroutine we need computes $nx+y$, where |x|
+and~|y| are |scaled| and |n| is an integer. We will also use it to
+multiply integers.
+
+ at d nx_plus_y(A, B, C) mult_and_add(A, B, C, 07777777777)
+ at d mult_integers(A, B) mult_and_add(A, B, 0, 017777777777)
+
+ at p static scaled mult_and_add(int @!n, scaled @!x, scaled @!y, scaled @!max_answer)
+{@+if (n < 0)
+ {@+negate(x);negate(n);
+ }
+if (n==0) return y;
+else if (((x <= (max_answer-y)/n)&&(-x <= (max_answer+y)/n)))
+ return n*x+y;
+else{@+arith_error=true;return 0;
+ }
+}
+
+@ We also need to divide scaled dimensions by integers.
+
+ at p static scaled x_over_n(scaled @!x, int @!n)
+{@+bool negative; /*should |rem| be negated?*/
+scaled x_over_n;
+negative=false;
+if (n==0)
+ {@+arith_error=true;x_over_n=0;rem=x;
+ }
+else{@+if (n < 0)
+ {@+negate(x);negate(n);negative=true;
+ }
+ if (x >= 0)
+ {@+x_over_n=x/n;rem=x%n;
+ }
+ else{@+x_over_n=-((-x)/n);rem=-((-x)%n);
+ }
+ }
+if (negative) negate(rem);
+return x_over_n;}
+
+@ Then comes the multiplication of a scaled number by a fraction |n/(double)d|,
+where |n| and |d| are nonnegative integers | <= @t$2^{16}$@>| and |d| is
+positive. It would be too dangerous to multiply by~|n| and then divide
+by~|d|, in separate operations, since overflow might well occur; and it
+would be too inaccurate to divide by |d| and then multiply by |n|. Hence
+this subroutine simulates 1.5-precision arithmetic.
+
+ at p static scaled xn_over_d(scaled @!x, int @!n, int @!d)
+{@+bool positive; /*was |x >= 0|?*/
+nonnegative_integer @!t, @!u, @!v; /*intermediate quantities*/
+scaled xn_over_d;
+if (x >= 0) positive=true;
+else{@+negate(x);positive=false;
+ }
+t=(x%0100000)*n;
+u=(x/0100000)*n+(t/0100000);
+v=(u%d)*0100000+(t%0100000);
+if (u/d >= 0100000) arith_error=true;
+else u=0100000*(u/d)+(v/d);
+if (positive)
+ {@+xn_over_d=u;rem=v%d;
+ }
+else{@+xn_over_d=-u;rem=-(v%d);
+ }
+return xn_over_d;}
+
+@ The next subroutine is used to compute the ``badness'' of glue, when a
+total~|t| is supposed to be made from amounts that sum to~|s|. According
+to {\sl The \TeX book}, the badness of this situation is $100(t/s)^3$;
+however, badness is simply a heuristic, so we need not squeeze out the
+last drop of accuracy when computing it. All we really want is an
+approximation that has similar properties.
+@:TeXbook}{\sl The \TeX book@>
+
+The actual method used to compute the badness is easier to read from the
+program than to describe in words. It produces an integer value that is a
+reasonably close approximation to $100(t/s)^3$, and all implementations
+of \TeX\ should use precisely this method. Any badness of $2^{13}$ or more is
+treated as infinitely bad, and represented by 10000.
+
+It is not difficult to prove that $$\hbox{|badness(t+1, s) >= badness(t, s)
+ >= badness(t, s+1)|}.$$ The badness function defined here is capable of
+computing at most 1095 distinct values, but that is plenty.
+
+ at d inf_bad 10000 /*infinitely bad value*/
+
+ at p halfword badness(scaled @!t, scaled @!s) /*compute badness, given |t >= 0|*/
+{@+int r; /*approximation to $\alpha t/s$, where $\alpha^3\approx
+ 100\cdot2^{18}$*/
+if (t==0) return 0;
+else if (s <= 0) return inf_bad;
+else{@+if (t <= 7230584) r=(t*297)/s; /*$297^3=99.94\times2^{18}$*/
+ else if (s >= 1663497) r=t/(s/297);
+ else r=t;
+ if (r > 1290) return inf_bad; /*$1290^3<2^{31}<1291^3$*/
+ else return(r*r*r+0400000)/01000000;
+ } /*that was $r^3/2^{18}$, rounded to the nearest integer*/
+}
+
+@ When \TeX\ ``packages'' a list into a box, it needs to calculate the
+proportionality ratio by which the glue inside the box should stretch
+or shrink. This calculation does not affect \TeX's decision making,
+so the precise details of rounding, etc., in the glue calculation are not
+of critical importance for the consistency of results on different computers.
+
+We shall use the type |glue_ratio| for such proportionality ratios.
+A glue ratio should take the same amount of memory as an
+|int| (usually 32 bits) if it is to blend smoothly with \TeX's
+other data structures. Thus |glue_ratio| should be equivalent to
+|short_real| in some implementations of \PASCAL. Alternatively,
+it is possible to deal with glue ratios using nothing but fixed-point
+arithmetic; see {\sl TUGboat \bf3},1 (March 1982), 10--27. (But the
+routines cited there must be modified to allow negative glue ratios.)
+@^system dependencies@>
+
+ at d set_glue_ratio_zero(A) A=0.0 /*store the representation of zero ratio*/
+ at d set_glue_ratio_one(A) A=1.0 /*store the representation of unit ratio*/
+ at d float(A) ((double)(A)) /*convert from |glue_ratio| to type |double|*/
+ at d unfloat(A) ((glue_ratio)(A)) /*convert from |double| to type |glue_ratio|*/
+ at d float_constant(A) ((double)(A)) /*convert |int| constant to |double|*/
+
+@<Types...@>=
+#if __SIZEOF_FLOAT__==4
+typedef float float32_t;
+#else
+#error float type must have size 4
+#endif
+typedef float @!glue_ratio; /*one-word representation of a glue expansion factor*/
+
+@* Packed data.
+In order to make efficient use of storage space, \TeX\ bases its major data
+structures on a |memory_word|, which contains either a (signed) integer,
+possibly scaled, or a (signed) |glue_ratio|, or a small number of
+fields that are one half or one quarter of the size used for storing
+integers.
+
+If |x| is a variable of type |memory_word|, it contains up to four
+fields that can be referred to as follows:
+$$\vbox{\halign{\hfil#&#\hfil&#\hfil\cr
+|x|&.|i|&(an |int|)\cr
+|x|&.|sc|\qquad&(a |scaled| integer)\cr
+|x|&.|gr|&(a |glue_ratio|)\cr
+|x.hh.lh|, |x.hh|&.|rh|&(two halfword fields)\cr
+|x.hh.b0|, |x.hh.b1|, |x.hh|&.|rh|&(two quarterword fields, one halfword
+ field)\cr
+|x.qqqq.b0|, |x.qqqq.b1|, |x.qqqq|&.|b2|, |x.qqqq.b3|\hskip-100pt
+ &\qquad\qquad\qquad(four quarterword fields)\cr}}$$
+This is somewhat cumbersome to write, and not very readable either, but
+macros will be used to make the notation shorter and more transparent.
+The \PASCAL\ code below gives a formal definition of |memory_word| and
+its subsidiary types, using packed variant records. \TeX\ makes no
+assumptions about the relative positions of the fields within a word.
+
+Since we are assuming 32-bit integers, a halfword must contain at least
+16 bits, and a quarterword must contain at least 8 bits.
+@^system dependencies@>
+But it doesn't hurt to have more bits; for example, with enough 36-bit
+words you might be able to have |mem_max| as large as 262142, which is
+eight times as much memory as anybody had during the first four years of
+\TeX's existence.
+
+N.B.: Valuable memory space will be dreadfully wasted unless \TeX\ is compiled
+by a \PASCAL\ that packs all of the |memory_word| variants into
+the space of a single integer. This means, for example, that |glue_ratio|
+words should be |short_real| instead of |double| on some computers. Some
+\PASCAL\ compilers will pack an integer whose subrange is `|0 dotdot 255|' into
+an eight-bit field, but others insist on allocating space for an additional
+sign bit; on such systems you can get 256 values into a quarterword only
+if the subrange is `|-128 dotdot 127|'.
+
+The present implementation tries to accommodate as many variations as possible,
+so it makes few assumptions. If integers having the subrange
+`|min_quarterword dotdot max_quarterword|' can be packed into a quarterword,
+and if integers having the subrange `|min_halfword dotdot max_halfword|'
+can be packed into a halfword, everything should work satisfactorily.
+
+It is usually most efficient to have |min_quarterword==min_halfword==0|,
+so one should try to achieve this unless it causes a severe problem.
+The values defined here are recommended for most 32-bit computers.
+
+ at d min_quarterword 0 /*smallest allowable value in a |quarterword|*/
+ at d max_quarterword 65535 /*largest allowable value in a |quarterword|*/
+ at d min_halfword 0 /*smallest allowable value in a |halfword|*/
+ at d max_halfword 0x3FFFFFFF /*largest allowable value in a |halfword|*/
+
+@ Here are the inequalities that the quarterword and halfword values
+must satisfy (or rather, the inequalities that they mustn't satisfy):
+
+@<Check the ``constant''...@>=
+#ifdef @!INIT
+if ((mem_min!=mem_bot)||(mem_max!=mem_top)) bad=10;
+#endif
+@;@/
+if ((mem_min > mem_bot)||(mem_max < mem_top)) bad=10;
+if ((min_quarterword > 0)||(max_quarterword < 127)) bad=11;
+if ((min_halfword > 0)||(max_halfword < 32767)) bad=12;
+if ((min_quarterword < min_halfword)||@|
+ (max_quarterword > max_halfword)) bad=13;
+if ((mem_min < min_halfword)||(mem_max >= max_halfword)||@|
+ (mem_bot-mem_min > max_halfword+1)) bad=14;
+if ((font_base < min_quarterword)||(font_max > max_quarterword)) bad=15;
+if (font_max > font_base+256) bad=16;
+if ((save_size > max_halfword)||(max_strings > max_halfword)) bad=17;
+if (buf_size > max_halfword) bad=18;
+if (max_quarterword-min_quarterword < 255) bad=19;
+
+@ The operation of adding or subtracting |min_quarterword| occurs quite
+frequently in \TeX, so it is convenient to abbreviate this operation
+by using the macros |qi| and |qo| for input and output to and from
+quarterword format.
+
+The inner loop of \TeX\ will run faster with respect to compilers
+that don't optimize expressions like `|x+0|' and `|x-0|', if these
+macros are simplified in the obvious way when |min_quarterword==0|.
+@^inner loop@>@^system dependencies@>
+
+ at d qi(A) A+min_quarterword
+ /*to put an |eight_bits| item into a quarterword*/
+ at d qo(A) A-min_quarterword
+ /*to take an |eight_bits| item out of a quarterword*/
+ at d hi(A) A+min_halfword
+ /*to put a sixteen-bit item into a halfword*/
+ at d ho(A) A-min_halfword
+ /*to take a sixteen-bit item from a halfword*/
+
+@ The reader should study the following definitions closely:
+@^system dependencies@>
+
+ at d sc i /*|scaled| data is equivalent to |int|*/
+
+@<Types...@>=
+typedef uint16_t quarterword; /*1/4 of a word*/
+typedef int32_t halfword; /*1/2 of a word*/
+typedef int8_t two_choices; /*used when there are two variants in a record*/
+typedef int8_t four_choices; /*used when there are four variants in a record*/
+typedef struct { @;@/
+ halfword @!rh;
+ union {
+ halfword @!lh;
+ struct { quarterword @!b0;quarterword @!b1;} ;
+ };} two_halves;
+typedef struct { @;@/
+ quarterword @!b0;
+ quarterword @!b1;
+ quarterword @!b2;
+ quarterword @!b3;
+ } four_quarters;
+typedef struct { @;@/
+ union {
+ int @!i;
+ glue_ratio @!gr;
+ two_halves @!hh;
+ four_quarters @!qqqq;
+ };} memory_word;
+typedef struct {@+FILE *f;@+memory_word@,d;@+} word_file;
+
+@ When debugging, we may want to print a |memory_word| without knowing
+what type it is; so we print it in all modes.
+@^dirty \PASCAL@>@^debugging@>
+
+ at p
+#ifdef @!DEBUG
+static void print_word(memory_word @!w)
+ /*prints |w| in all ways*/
+{@+print_int(w.i);print_char(' ');@/
+print_scaled(w.sc);print_char(' ');@/
+print_scaled(round(unity*float(w.gr)));print_ln();@/
+@^real multiplication@>
+print_int(w.hh.lh);print_char('=');print_int(w.hh.b0);print_char(':');
+print_int(w.hh.b1);print_char(';');print_int(w.hh.rh);print_char(' ');@/
+print_int(w.qqqq.b0);print_char(':');print_int(w.qqqq.b1);print_char(':');
+print_int(w.qqqq.b2);print_char(':');print_int(w.qqqq.b3);
+}
+#endif
+
+@* Dynamic memory allocation.
+The \TeX\ system does nearly all of its own memory allocation, so that it
+can readily be transported into environments that do not have automatic
+facilities for strings, garbage collection, etc., and so that it can be in
+control of what error messages the user receives. The dynamic storage
+requirements of \TeX\ are handled by providing a large array |mem| in
+which consecutive blocks of words are used as nodes by the \TeX\ routines.
+
+Pointer variables are indices into this array, or into another array
+called |eqtb| that will be explained later. A pointer variable might
+also be a special flag that lies outside the bounds of |mem|, so we
+allow pointers to assume any |halfword| value. The minimum halfword
+value represents a null pointer. \TeX\ does not assume that |mem[null]| exists.
+
+ at s pointer int
+ at d pointer halfword /*a flag or a location in |mem| or |eqtb|*/
+ at d null min_halfword /*the null pointer*/
+
+@<Glob...@>=
+static pointer @!temp_ptr; /*a pointer variable for occasional emergency use*/
+
+@ The |mem| array is divided into two regions that are allocated separately,
+but the dividing line between these two regions is not fixed; they grow
+together until finding their ``natural'' size in a particular job.
+Locations less than or equal to |lo_mem_max| are used for storing
+variable-length records consisting of two or more words each. This region
+is maintained using an algorithm similar to the one described in exercise
+2.5--19 of {\sl The Art of Computer Programming}. However, no size field
+appears in the allocated nodes; the program is responsible for knowing the
+relevant size when a node is freed. Locations greater than or equal to
+|hi_mem_min| are used for storing one-word records; a conventional
+\.{AVAIL} stack is used for allocation in this region.
+
+Locations of |mem| between |mem_bot| and |mem_top| may be dumped as part
+of preloaded format files, by the \.{INITEX} preprocessor.
+ at .INITEX@>
+Production versions of \TeX\ may extend the memory at both ends in order to
+provide more space; locations between |mem_min| and |mem_bot| are always
+used for variable-size nodes, and locations between |mem_top| and |mem_max|
+are always used for single-word nodes.
+
+The key pointers that govern |mem| allocation have a prescribed order:
+$$\advance\thickmuskip-2mu
+\hbox{|null <= mem_min <= mem_bot < lo_mem_max <
+ hi_mem_min < mem_top <= mem_end <= mem_max|.}$$
+
+Empirical tests show that the present implementation of \TeX\ tends to
+spend about 9\pct! of its running time allocating nodes, and about 6\pct!
+deallocating them after their use.
+
+@<Glob...@>=
+memory_word @!mem0[mem_max-mem_min+1], *const @!mem = @!mem0-mem_min; /*the big dynamic storage area*/
+static pointer @!lo_mem_max; /*the largest location of variable-size memory in use*/
+pointer @!hi_mem_min; /*the smallest location of one-word memory in use*/
+
+@ In order to study the memory requirements of particular applications, it
+is possible to prepare a version of \TeX\ that keeps track of current and
+maximum memory usage. When code between the delimiters |
+#ifdef @!STAT
+| $\ldots$
+|tats| is not ``commented out,'' \TeX\ will run a bit slower but it will
+report these statistics when |tracing_stats| is sufficiently large.
+
+@<Glob...@>=
+int @!var_used, @!dyn_used; /*how much memory is in use*/
+#ifdef @!STAT
+#define incr_dyn_used @[incr(dyn_used)@]
+#define decr_dyn_used @[decr(dyn_used)@]
+#else
+#define incr_dyn_used
+#define decr_dyn_used
+#endif
+
+@ Let's consider the one-word memory region first, since it's the
+simplest. The pointer variable |mem_end| holds the highest-numbered location
+of |mem| that has ever been used. The free locations of |mem| that
+occur between |hi_mem_min| and |mem_end|, inclusive, are of type
+|two_halves|, and we write |info(p)| and |link(p)| for the |lh|
+and |rh| fields of |mem[p]| when it is of this type. The single-word
+free locations form a linked list
+$$|avail|,\;\hbox{|link(avail)|},\;\hbox{|link(link(avail))|},\;\ldots$$
+terminated by |null|.
+
+ at d link(A) mem[A].hh.rh /*the |link| field of a memory word*/
+ at d info(A) mem[A].hh.lh /*the |info| field of a memory word*/
+
+@<Glob...@>=
+static pointer @!avail; /*head of the list of available one-word nodes*/
+static pointer @!mem_end; /*the last one-word node used in |mem|*/
+
+@ If memory is exhausted, it might mean that the user has forgotten
+a right brace. We will define some procedures later that try to help
+pinpoint the trouble.
+
+ at p @<Declare the procedure called |show_token_list|@>@/
+@<Declare the procedure called |runaway|@>@;
+
+@ The function |get_avail| returns a pointer to a new one-word node whose
+|link| field is null. However, \TeX\ will halt if there is no more room left.
+@^inner loop@>
+
+If the available-space list is empty, i.e., if |avail==null|,
+we try first to increase |mem_end|. If that cannot be done, i.e., if
+|mem_end==mem_max|, we try to decrease |hi_mem_min|. If that cannot be
+done, i.e., if |hi_mem_min==lo_mem_max+1|, we have to quit.
+
+ at p static pointer get_avail(void) /*single-word node allocation*/
+{@+pointer p; /*the new node being got*/
+p=avail; /*get top location in the |avail| stack*/
+if (p!=null) avail=link(avail); /*and pop it off*/
+else if (mem_end < mem_max) /*or go into virgin territory*/
+ {@+incr(mem_end);p=mem_end;
+ }
+else{@+decr(hi_mem_min);p=hi_mem_min;
+ if (hi_mem_min <= lo_mem_max)
+ {@+runaway(); /*if memory is exhausted, display possible runaway text*/
+ overflow("main memory size", mem_max+1-mem_min);
+ /*quit; all one-word nodes are busy*/
+@:TeX capacity exceeded main memory size}{\quad main memory size@>
+ }
+ }
+link(p)=null; /*provide an oft-desired initialization of the new node*/
+#ifdef @!STAT
+incr(dyn_used);
+#endif
+@; /*maintain statistics*/
+return p;
+}
+
+@ Conversely, a one-word node is recycled by calling |free_avail|.
+This routine is part of \TeX's ``inner loop,'' so we want it to be fast.
+@^inner loop@>
+
+ at d free_avail(A) /*single-word node liberation*/
+ {@+link(A)=avail;avail=A;
+ decr_dyn_used;
+ }
+
+@ There's also a |fast_get_avail| routine, which saves the procedure-call
+overhead at the expense of extra programming. This routine is used in
+the places that would otherwise account for the most calls of |get_avail|.
+@^inner loop@>
+
+ at d fast_get_avail(A) @t@>@;@/
+ {@+A=avail; /*avoid |get_avail| if possible, to save time*/
+ if (A==null) A=get_avail();
+ else{@+avail=link(A);link(A)=null;
+ incr_dyn_used;
+ }
+ }
+
+@ The procedure |flush_list(p)| frees an entire linked list of
+one-word nodes that starts at position |p|.
+@^inner loop@>
+
+ at p static void flush_list(pointer @!p) /*makes list of single-word nodes
+ available*/
+{@+pointer @!q, @!r; /*list traversers*/
+if (p!=null)
+ {@+r=p;
+ @/do at +{q=r;r=link(r);
+#ifdef @!STAT
+decr(dyn_used);
+#endif
+ }@+ while (!(r==null)); /*now |q| is the last node on the list*/
+ link(q)=avail;avail=p;
+ }
+}
+
+@ The available-space list that keeps track of the variable-size portion
+of |mem| is a nonempty, doubly-linked circular list of empty nodes,
+pointed to by the roving pointer |rover|.
+
+Each empty node has size 2 or more; the first word contains the special
+value |max_halfword| in its |link| field and the size in its |info| field;
+the second word contains the two pointers for double linking.
+
+Each nonempty node also has size 2 or more. Its first word is of type
+|two_halves|\kern-1pt, and its |link| field is never equal to |max_halfword|.
+Otherwise there is complete flexibility with respect to the contents
+of its other fields and its other words.
+
+(We require |mem_max < max_halfword| because terrible things can happen
+when |max_halfword| appears in the |link| field of a nonempty node.)
+
+ at d empty_flag max_halfword /*the |link| of an empty variable-size node*/
+ at d is_empty(A) (link(A)==empty_flag) /*tests for empty node*/
+ at d node_size(A) info(A) /*the size field in empty variable-size nodes*/
+ at d llink(A) info(A+1) /*left link in doubly-linked list of empty nodes*/
+ at d rlink(A) link(A+1) /*right link in doubly-linked list of empty nodes*/
+
+@<Glob...@>=
+static pointer @!rover; /*points to some node in the list of empties*/
+
+@ A call to |get_node| with argument |s| returns a pointer to a new node
+of size~|s|, which must be 2~or more. The |link| field of the first word
+of this new node is set to null. An overflow stop occurs if no suitable
+space exists.
+
+If |get_node| is called with $s=2^{30}$, it simply merges adjacent free
+areas and returns the value |max_halfword|.
+
+ at p pointer get_node(int @!s) /*variable-size node allocation*/
+{@+
+pointer p; /*the node currently under inspection*/
+pointer @!q; /*the node physically after node |p|*/
+int @!r; /*the newly allocated node, or a candidate for this honor*/
+int @!t; /*temporary register*/
+restart: p=rover; /*start at some free node in the ring*/
+@/do at +{@<Try to allocate within node |p| and its physical successors, and |goto found|
+if allocation was possible@>;
+@^inner loop@>
+p=rlink(p); /*move to the next node in the ring*/
+}@+ while (!(p==rover)); /*repeat until the whole list has been traversed*/
+if (s==010000000000)
+ {@+return max_halfword;
+ }
+if (lo_mem_max+2 < hi_mem_min) if (lo_mem_max+2 <= mem_bot+max_halfword)
+ @<Grow more variable-size memory and |goto restart|@>;
+overflow("main memory size", mem_max+1-mem_min);
+ /*sorry, nothing satisfactory is left*/
+@:TeX capacity exceeded main memory size}{\quad main memory size@>
+found: link(r)=null; /*this node is now nonempty*/
+#ifdef @!STAT
+var_used=var_used+s; /*maintain usage statistics*/
+#endif
+@;@/
+return r;
+}
+
+@ The lower part of |mem| grows by 1000 words at a time, unless
+we are very close to going under. When it grows, we simply link
+a new node into the available-space list. This method of controlled
+growth helps to keep the |mem| usage consecutive when \TeX\ is
+implemented on ``virtual memory'' systems.
+@^virtual memory@>
+
+@<Grow more variable-size memory and |goto restart|@>=
+{@+if (hi_mem_min-lo_mem_max >= 1998) t=lo_mem_max+1000;
+else t=lo_mem_max+1+(hi_mem_min-lo_mem_max)/2;
+ /*|lo_mem_max+2 <= t < hi_mem_min|*/
+p=llink(rover);q=lo_mem_max;rlink(p)=q;llink(rover)=q;@/
+if (t > mem_bot+max_halfword) t=mem_bot+max_halfword;
+rlink(q)=rover;llink(q)=p;link(q)=empty_flag;node_size(q)=t-lo_mem_max;@/
+lo_mem_max=t;link(lo_mem_max)=null;info(lo_mem_max)=null;
+rover=q;goto restart;
+}
+
+@ Empirical tests show that the routine in this section performs a
+node-merging operation about 0.75 times per allocation, on the average,
+after which it finds that |r > p+1| about 95\pct! of the time.
+
+@<Try to allocate...@>=
+q=p+node_size(p); /*find the physical successor*/
+@^inner loop@>
+while (is_empty(q)) /*merge node |p| with node |q|*/
+ {@+t=rlink(q);
+ if (q==rover) rover=t;
+ llink(t)=llink(q);rlink(llink(q))=t;@/
+ q=q+node_size(q);
+ }
+r=q-s;
+if (r > p+1) @<Allocate from the top of node |p| and |goto found|@>;
+if (r==p) if (rlink(p)!=p)
+ @<Allocate entire node |p| and |goto found|@>;
+node_size(p)=q-p /*reset the size in case it grew*/
+
+@ @<Allocate from the top...@>=
+{@+node_size(p)=r-p; /*store the remaining size*/
+@^inner loop@>
+rover=p; /*start searching here next time*/
+goto found;
+}
+
+@ Here we delete node |p| from the ring, and let |rover| rove around.
+
+@<Allocate entire...@>=
+{@+rover=rlink(p);t=llink(p);
+llink(rover)=t;rlink(t)=rover;
+goto found;
+}
+
+@ Conversely, when some variable-size node |p| of size |s| is no longer needed,
+the operation |free_node(p, s)| will make its words available, by inserting
+|p| as a new empty node just before where |rover| now points.
+@^inner loop@>
+
+ at p void free_node(pointer @!p, halfword @!s) /*variable-size node
+ liberation*/
+{@+pointer q; /*|llink(rover)|*/
+node_size(p)=s;link(p)=empty_flag;
+q=llink(rover);llink(p)=q;rlink(p)=rover; /*set both links*/
+llink(rover)=p;rlink(q)=p; /*insert |p| into the ring*/
+#ifdef @!STAT
+var_used=var_used-s;
+#endif
+@; /*maintain statistics*/
+}
+
+@ Just before \.{INITEX} writes out the memory, it sorts the doubly linked
+available space list. The list is probably very short at such times, so a
+simple insertion sort is used. The smallest available location will be
+pointed to by |rover|, the next-smallest by |rlink(rover)|, etc.
+
+ at p
+#ifdef @!INIT
+static void sort_avail(void) /*sorts the available variable-size nodes
+ by location*/
+{@+pointer p, @!q, @!r; /*indices into |mem|*/
+pointer @!old_rover; /*initial |rover| setting*/
+p=get_node(010000000000); /*merge adjacent free areas*/
+p=rlink(rover);rlink(rover)=max_halfword;old_rover=rover;
+while (p!=old_rover) @<Sort \(p)|p| into the list starting at |rover| and advance
+|p| to |rlink(p)|@>;
+p=rover;
+while (rlink(p)!=max_halfword)
+ {@+llink(rlink(p))=p;p=rlink(p);
+ }
+rlink(p)=rover;llink(rover)=p;
+}
+#endif
+
+@ The following |while | loop is guaranteed to
+terminate, since the list that starts at
+|rover| ends with |max_halfword| during the sorting procedure.
+
+@<Sort \(p)|p|...@>=
+if (p < rover)
+ {@+q=p;p=rlink(q);rlink(q)=rover;rover=q;
+ }
+else{@+q=rover;
+ while (rlink(q) < p) q=rlink(q);
+ r=rlink(p);rlink(p)=rlink(q);rlink(q)=p;p=r;
+ }
+
+@* Data structures for boxes and their friends.
+From the computer's standpoint, \TeX's chief mission is to create
+horizontal and vertical lists. We shall now investigate how the elements
+of these lists are represented internally as nodes in the dynamic memory.
+
+A horizontal or vertical list is linked together by |link| fields in
+the first word of each node. Individual nodes represent boxes, glue,
+penalties, or special things like discretionary hyphens; because of this
+variety, some nodes are longer than others, and we must distinguish different
+kinds of nodes. We do this by putting a `|type|' field in the first word,
+together with the link and an optional `|subtype|'.
+
+ at d type(A) mem[A].hh.b0 /*identifies what kind of node this is*/
+ at d subtype(A) mem[A].hh.b1 /*secondary identification in some cases*/
+
+@ A |@!char_node|, which represents a single character, is the most important
+kind of node because it accounts for the vast majority of all boxes.
+Special precautions are therefore taken to ensure that a |char_node| does
+not take up much memory space. Every such node is one word long, and in fact
+it is identifiable by this property, since other kinds of nodes have at least
+two words, and they appear in |mem| locations less than |hi_mem_min|.
+This makes it possible to omit the |type| field in a |char_node|, leaving
+us room for two bytes that identify a |font| and a |character| within
+that font.
+
+Note that the format of a |char_node| allows for up to 256 different
+fonts and up to 256 characters per font; but most implementations will
+probably limit the total number of fonts to fewer than 75 per job,
+and most fonts will stick to characters whose codes are
+less than 128 (since higher codes
+are more difficult to access on most keyboards).
+
+Extensions of \TeX\ intended for oriental languages will need even more
+than $256\times256$ possible characters, when we consider different sizes
+@^oriental characters@>@^Chinese characters@>@^Japanese characters@>
+and styles of type. It is suggested that Chinese and Japanese fonts be
+handled by representing such characters in two consecutive |char_node|
+entries: The first of these has |font==font_base|, and its |link| points
+to the second;
+the second identifies the font and the character dimensions.
+The saving feature about oriental characters is that most of them have
+the same box dimensions. The |character| field of the first |char_node|
+is a ``\\{charext}'' that distinguishes between graphic symbols whose
+dimensions are identical for typesetting purposes. (See the \MF\ manual.)
+Such an extension of \TeX\ would not be difficult; further details are
+left to the reader.
+
+In order to make sure that the |character| code fits in a quarterword,
+\TeX\ adds the quantity |min_quarterword| to the actual code.
+
+Character nodes appear only in horizontal lists, never in vertical lists.
+
+ at d is_char_node(A) (A >= hi_mem_min)
+ /*does the argument point to a |char_node|?*/
+ at d font(A) type(A) /*the font code in a |char_node|*/
+ at d character(A) subtype(A) /*the character code in a |char_node|*/
+
+@ An |hlist_node| stands for a box that was made from a horizontal list.
+Each |hlist_node| is seven words long, and contains the following fields
+(in addition to the mandatory |type| and |link|, which we shall not
+mention explicitly when discussing the other node types): The |height| and
+|width| and |depth| are scaled integers denoting the dimensions of the
+box. There is also a |shift_amount| field, a scaled integer indicating
+how much this box should be lowered (if it appears in a horizontal list),
+or how much it should be moved to the right (if it appears in a vertical
+list). There is a |list_ptr| field, which points to the beginning of the
+list from which this box was fabricated; if |list_ptr| is |null|, the box
+is empty. Finally, there are three fields that represent the setting of
+the glue: |glue_set(p)| is a word of type |glue_ratio| that represents
+the proportionality constant for glue setting; |glue_sign(p)| is
+|stretching| or |shrinking| or |normal| depending on whether or not the
+glue should stretch or shrink or remain rigid; and |glue_order(p)|
+specifies the order of infinity to which glue setting applies (|normal|,
+|fil|, |fill|, or |filll|). The |subtype| field is not used.
+
+ at d hlist_node 0 /*|type| of hlist nodes*/
+ at d box_node_size 9 /*number of words to allocate for a box, set, or pack node*/
+ at d width_offset 1 /*position of |width| field in a box node*/
+ at d depth_offset 2 /*position of |depth| field in a box node*/
+ at d height_offset 3 /*position of |height| field in a box node*/
+ at d width(A) mem[A+width_offset].sc /*width of the box, in sp*/
+ at d depth(A) mem[A+depth_offset].sc /*depth of the box, in sp*/
+ at d height(A) mem[A+height_offset].sc /*height of the box, in sp*/
+ at d shift_amount(A) mem[A+4].sc /*repositioning distance, in sp*/
+ at d list_offset 5 /*position of |list_ptr| field in a box node*/
+ at d list_ptr(A) link(A+list_offset) /*beginning of the list inside the box*/
+ at d glue_order(A) subtype(A+list_offset) /*applicable order of infinity*/
+ at d glue_sign(A) type(A+list_offset) /*stretching or shrinking*/
+ at d normal 0 /*the most common case when several cases are named*/
+ at d stretching 1 /*glue setting applies to the stretch components*/
+ at d shrinking 2 /*glue setting applies to the shrink components*/
+ at d glue_offset 6 /*position of |glue_set| in a box node*/
+ at d glue_set(A) mem[A+glue_offset].gr
+ /*a word of type |glue_ratio| for glue setting*/
+
+@ The |new_null_box| function returns a pointer to an |hlist_node| in
+which all subfields have the values corresponding to `\.{\\hbox\{\}}'.
+The |subtype| field is set to |min_quarterword|, since that's the desired
+|span_count| value if this |hlist_node| is changed to an |unset_node|.
+
+ at p pointer new_null_box(void) /*creates a new box node*/
+{@+pointer p; /*the new node*/
+p=get_node(box_node_size);type(p)=hlist_node;
+subtype(p)=min_quarterword;
+width(p)=0;depth(p)=0;height(p)=0;shift_amount(p)=0;list_ptr(p)=null;
+glue_sign(p)=normal;glue_order(p)=normal;set_glue_ratio_zero(glue_set(p));
+return p;
+}
+
+@ A |vlist_node| is like an |hlist_node| in all respects except that it
+contains a vertical list.
+
+ at d vlist_node 1 /*|type| of vlist nodes*/
+
+@ A |rule_node| stands for a solid black rectangle; it has |width|,
+|depth|, and |height| fields just as in an |hlist_node|. However, if
+any of these dimensions is $-2^{30}$, the actual value will be determined
+by running the rule up to the boundary of the innermost enclosing box.
+This is called a ``running dimension.'' The |width| is never running in
+an hlist; the |height| and |depth| are never running in a~vlist.
+
+ at d rule_node 2 /*|type| of rule nodes*/
+ at d rule_node_size 4 /*number of words to allocate for a rule node*/
+ at d null_flag -010000000000 /*$-2^{30}$, signifies a missing item*/
+ at d is_running(A) (A==null_flag) /*tests for a running dimension*/
+
+@ A new rule node is delivered by the |new_rule| function. It
+makes all the dimensions ``running,'' so you have to change the
+ones that are not allowed to run.
+
+ at p pointer new_rule(void)
+{@+pointer p; /*the new node*/
+p=get_node(rule_node_size);type(p)=rule_node;
+subtype(p)=0; /*the |subtype| is not used*/
+width(p)=null_flag;depth(p)=null_flag;height(p)=null_flag;
+return p;
+}
+
+@ Insertions are represented by |ins_node| records, where the |subtype|
+indicates the corresponding box number. For example, `\.{\\insert 250}'
+leads to an |ins_node| whose |subtype| is |250+min_quarterword|.
+The |height| field of an |ins_node| is slightly misnamed; it actually holds
+the natural height plus depth of the vertical list being inserted.
+The |depth| field holds the |split_max_depth| to be used in case this
+insertion is split, and the |split_top_ptr| points to the corresponding
+|split_top_skip|. The |float_cost| field holds the |floating_penalty| that
+will be used if this insertion floats to a subsequent page after a
+split insertion of the same class. There is one more field, the
+|ins_ptr|, which points to the beginning of the vlist for the insertion.
+
+ at d ins_node 3 /*|type| of insertion nodes*/
+ at d ins_node_size 5 /*number of words to allocate for an insertion*/
+ at d float_cost(A) mem[A+1].i /*the |floating_penalty| to be used*/
+ at d ins_ptr(A) info(A+4) /*the vertical list to be inserted*/
+ at d split_top_ptr(A) link(A+4) /*the |split_top_skip| to be used*/
+
+@ A |mark_node| has a |mark_ptr| field that points to the reference count
+of a token list that contains the user's \.{\\mark} text.
+This field occupies a full word instead of a halfword, because
+there's nothing to put in the other halfword; it is easier in \PASCAL\ to
+use the full word than to risk leaving garbage in the unused half.
+
+ at d mark_node 4 /*|type| of a mark node*/
+ at d small_node_size 2 /*number of words to allocate for most node types*/
+ at d mark_ptr(A) mem[A+1].i /*head of the token list for a mark*/
+
+@ An |adjust_node|, which occurs only in horizontal lists,
+specifies material that will be moved out into the surrounding
+vertical list; i.e., it is used to implement \TeX's `\.{\\vadjust}'
+operation. The |adjust_ptr| field points to the vlist containing this
+material.
+
+ at d adjust_node 5 /*|type| of an adjust node*/
+ at d adjust_ptr(A) mark_ptr(A) /*vertical list to be moved out of horizontal list*/
+
+@ A |ligature_node|, which occurs only in horizontal lists, specifies
+a character that was fabricated from the interaction of two or more
+actual characters. The second word of the node, which is called the
+|lig_char| word, contains |font| and |character| fields just as in a
+|char_node|. The characters that generated the ligature have not been
+forgotten, since they are needed for diagnostic messages and for
+hyphenation; the |lig_ptr| field points to a linked list of character
+nodes for all original characters that have been deleted. (This list
+might be empty if the characters that generated the ligature were
+retained in other nodes.)
+
+The |subtype| field is 0, plus 2 and/or 1 if the original source of the
+ligature included implicit left and/or right boundaries.
+
+ at d ligature_node 6 /*|type| of a ligature node*/
+ at d lig_char(A) A+1 /*the word where the ligature is to be found*/
+ at d lig_ptr(A) link(lig_char(A)) /*the list of characters*/
+
+@ The |new_ligature| function creates a ligature node having given
+contents of the |font|, |character|, and |lig_ptr| fields. We also have
+a |new_lig_item| function, which returns a two-word node having a given
+|character| field. Such nodes are used for temporary processing as ligatures
+are being created.
+
+ at p static pointer new_ligature(quarterword @!f, quarterword @!c, pointer @!q)
+{@+pointer p; /*the new node*/
+p=get_node(small_node_size);type(p)=ligature_node;
+font(lig_char(p))=f;character(lig_char(p))=c;lig_ptr(p)=q;
+subtype(p)=0;return p;
+}
+@#
+static pointer new_lig_item(quarterword @!c)
+{@+pointer p; /*the new node*/
+p=get_node(small_node_size);character(p)=c;lig_ptr(p)=null;
+return p;
+}
+
+@ A |disc_node|, which occurs only in horizontal lists, specifies a
+``dis\-cretion\-ary'' line break. If such a break occurs at node |p|, the text
+that starts at |pre_break(p)| will precede the break, the text that starts at
+|post_break(p)| will follow the break, and text that appears in the next
+|replace_count(p)| nodes will be ignored. For example, an ordinary
+discretionary hyphen, indicated by `\.{\\-}', yields a |disc_node| with
+|pre_break| pointing to a |char_node| containing a hyphen, |post_break==null|,
+and |replace_count==0|. All three of the discretionary texts must be
+lists that consist entirely of character, kern, box, rule, and ligature nodes.
+
+If |pre_break(p)==null|, the |ex_hyphen_penalty| will be charged for this
+break. Otherwise the |hyphen_penalty| will be charged. The texts will
+actually be substituted into the list by the line-breaking algorithm if it
+decides to make the break, and the discretionary node will disappear at
+that time; thus, the output routine sees only discretionaries that were
+not chosen.
+
+ at d disc_node 7 /*|type| of a discretionary node*/
+ at d replace_count(A) (subtype(A)&0x7F) /*how many subsequent nodes to replace*/
+ at d set_replace_count(A,B) (subtype(A)=(B)&0x7F)
+ at d set_auto_disc(A) (subtype(A)|=0x80)
+ at d is_auto_disc(A) (subtype(A)&0x80)
+ at d pre_break(A) llink(A) /*text that precedes a discretionary break*/
+ at d post_break(A) rlink(A) /*text that follows a discretionary break*/
+
+ at p pointer new_disc(void) /*creates an empty |disc_node|*/
+{@+pointer p; /*the new node*/
+p=get_node(small_node_size);type(p)=disc_node;
+set_replace_count(p,0);pre_break(p)=null;post_break(p)=null;
+return p;
+}
+
+@ A |whatsit_node| is a wild card reserved for extensions to \TeX. The
+|subtype| field in its first word says what `\\{whatsit}' it is, and
+implicitly determines the node size (which must be 2 or more) and the
+format of the remaining words. When a |whatsit_node| is encountered
+in a list, special actions are invoked; knowledgeable people who are
+careful not to mess up the rest of \TeX\ are able to make \TeX\ do new
+things by adding code at the end of the program. For example, there
+might be a `\TeX nicolor' extension to specify different colors of ink,
+@^extensions to \TeX@>
+and the whatsit node might contain the desired parameters.
+
+The present implementation of \TeX\ treats the features associated with
+`\.{\\write}' and `\.{\\special}' as if they were extensions, in order to
+illustrate how such routines might be coded. We shall defer further
+discussion of extensions until the end of this program.
+
+ at d whatsit_node 8 /*|type| of special extension nodes*/
+
+@ A |math_node|, which occurs only in horizontal lists, appears before and
+after mathematical formulas. The |subtype| field is |before| before the
+formula and |after| after it. There is a |width| field, which represents
+the amount of surrounding space inserted by \.{\\mathsurround}.
+
+ at d math_node 9 /*|type| of a math node*/
+ at d before 0 /*|subtype| for math node that introduces a formula*/
+ at d after 1 /*|subtype| for math node that winds up a formula*/
+
+ at p static pointer new_math(scaled @!w, small_number @!s)
+{@+pointer p; /*the new node*/
+p=get_node(small_node_size);type(p)=math_node;
+subtype(p)=s;width(p)=w;return p;
+}
+
+@ \TeX\ makes use of the fact that |hlist_node|, |vlist_node|,
+|rule_node|, |ins_node|, |mark_node|, |adjust_node|, |ligature_node|,
+|disc_node|, |whatsit_node|, and |math_node| are at the low end of the
+type codes, by permitting a break at glue in a list if and only if the
+|type| of the previous node is less than |math_node|. Furthermore, a
+node is discarded after a break if its type is |math_node| or~more.
+
+ at d precedes_break(A) (type(A) < math_node)
+ at d non_discardable(A) (type(A) < math_node)
+
+@ A |glue_node| represents glue in a list. However, it is really only
+a pointer to a separate glue specification, since \TeX\ makes use of the
+fact that many essentially identical nodes of glue are usually present.
+If |p| points to a |glue_node|, |glue_ptr(p)| points to
+another packet of words that specify the stretch and shrink components, etc.
+
+Glue nodes also serve to represent leaders; the |subtype| is used to
+distinguish between ordinary glue (which is called |normal|) and the three
+kinds of leaders (which are called |a_leaders|, |c_leaders|, and |x_leaders|).
+The |leader_ptr| field points to a rule node or to a box node containing the
+leaders; it is set to |null| in ordinary glue nodes.
+
+Many kinds of glue are computed from \TeX's ``skip'' parameters, and
+it is helpful to know which parameter has led to a particular glue node.
+Therefore the |subtype| is set to indicate the source of glue, whenever
+it originated as a parameter. We will be defining symbolic names for the
+parameter numbers later (e.g., |line_skip_code==0|, |baseline_skip_code==1|,
+etc.); it suffices for now to say that the |subtype| of parametric glue
+will be the same as the parameter number, plus~one.
+
+In math formulas there are two more possibilities for the |subtype| in a
+glue node: |mu_glue| denotes an \.{\\mskip} (where the units are scaled \.{mu}
+instead of scaled \.{pt}); and |cond_math_glue| denotes the `\.{\\nonscript}'
+feature that cancels the glue node immediately following if it appears
+in a subscript.
+
+ at d glue_node 10 /*|type| of node that points to a glue specification*/
+ at d cond_math_glue 98 /*special |subtype| to suppress glue in the next node*/
+ at d mu_glue 99 /*|subtype| for math glue*/
+ at d a_leaders 100 /*|subtype| for aligned leaders*/
+ at d c_leaders 101 /*|subtype| for centered leaders*/
+ at d x_leaders 102 /*|subtype| for expanded leaders*/
+ at d glue_ptr(A) llink(A) /*pointer to a glue specification*/
+ at d leader_ptr(A) rlink(A) /*pointer to box or rule node for leaders*/
+
+@ A glue specification has a halfword reference count in its first word,
+@^reference counts@>
+representing |null| plus the number of glue nodes that point to it (less one).
+Note that the reference count appears in the same position as
+the |link| field in list nodes; this is the field that is initialized
+to |null| when a node is allocated, and it is also the field that is flagged
+by |empty_flag| in empty nodes.
+
+Glue specifications also contain three |scaled| fields, for the |width|,
+|stretch|, and |shrink| dimensions. Finally, there are two one-byte
+fields called |stretch_order| and |shrink_order|; these contain the
+orders of infinity (|normal|, |fil|, |fill|, or |filll|)
+corresponding to the stretch and shrink values.
+
+ at d glue_spec_size 4 /*number of words to allocate for a glue specification*/
+ at d glue_ref_count(A) link(A) /*reference count of a glue specification*/
+ at d stretch(A) mem[A+2].sc /*the stretchability of this glob of glue*/
+ at d shrink(A) mem[A+3].sc /*the shrinkability of this glob of glue*/
+ at d stretch_order(A) type(A) /*order of infinity for stretching*/
+ at d shrink_order(A) subtype(A) /*order of infinity for shrinking*/
+ at d fil 1 /*first-order infinity*/
+ at d fill 2 /*second-order infinity*/
+ at d filll 3 /*third-order infinity*/
+
+@<Types...@>=
+typedef int8_t glue_ord; /*infinity to the 0, 1, 2, or 3 power*/
+
+@ Here is a function that returns a pointer to a copy of a glue spec.
+The reference count in the copy is |null|, because there is assumed
+to be exactly one reference to the new specification.
+
+ at p pointer new_spec(pointer @!p) /*duplicates a glue specification*/
+{@+pointer q; /*the new spec*/
+q=get_node(glue_spec_size);@/
+mem[q]=mem[p];glue_ref_count(q)=null;@/
+width(q)=width(p);stretch(q)=stretch(p);shrink(q)=shrink(p);
+return q;
+}
+
+@ And here's a function that creates a glue node for a given parameter
+identified by its code number; for example,
+|new_param_glue(line_skip_code)| returns a pointer to a glue node for the
+current \.{\\lineskip}.
+
+ at p pointer new_param_glue(small_number @!n)
+{@+pointer p; /*the new node*/
+pointer @!q; /*the glue specification*/
+p=get_node(small_node_size);type(p)=glue_node;subtype(p)=n+1;
+leader_ptr(p)=null;@/
+q=@<Current |mem| equivalent of glue parameter number |n|@>@t@>;
+glue_ptr(p)=q;incr(glue_ref_count(q));
+return p;
+}
+
+@ Glue nodes that are more or less anonymous are created by |new_glue|,
+whose argument points to a glue specification.
+
+ at p pointer new_glue(pointer @!q)
+{@+pointer p; /*the new node*/
+p=get_node(small_node_size);type(p)=glue_node;subtype(p)=normal;
+leader_ptr(p)=null;glue_ptr(p)=q;incr(glue_ref_count(q));
+return p;
+}
+
+@ Still another subroutine is needed: This one is sort of a combination
+of |new_param_glue| and |new_glue|. It creates a glue node for one of
+the current glue parameters, but it makes a fresh copy of the glue
+specification, since that specification will probably be subject to change,
+while the parameter will stay put. The global variable |temp_ptr| is
+set to the address of the new spec.
+
+ at p static pointer new_skip_param(small_number @!n)
+{@+pointer p; /*the new node*/
+temp_ptr=new_spec(@<Current |mem| equivalent of glue parameter...@>);
+p=new_glue(temp_ptr);glue_ref_count(temp_ptr)=null;subtype(p)=n+1;
+return p;
+}
+
+@ A |kern_node| has a |width| field to specify a (normally negative)
+amount of spacing. This spacing correction appears in horizontal lists
+between letters like A and V when the font designer said that it looks
+better to move them closer together or further apart. A kern node can
+also appear in a vertical list, when its `|width|' denotes additional
+spacing in the vertical direction. The |subtype| is either |normal| (for
+kerns inserted from font information or math mode calculations) or |explicit|
+(for kerns inserted from \.{\\kern} and \.{\\/} commands) or |acc_kern|
+(for kerns inserted from non-math accents) or |mu_glue| (for kerns
+inserted from \.{\\mkern} specifications in math formulas).
+
+ at d kern_node 11 /*|type| of a kern node*/
+ at d explicit 1 /*|subtype| of kern nodes from \.{\\kern} and \.{\\/}*/
+ at d acc_kern 2 /*|subtype| of kern nodes from accents*/
+
+@ The |new_kern| function creates a kern node having a given width.
+
+ at p static pointer new_kern(scaled @!w)
+{@+pointer p; /*the new node*/
+p=get_node(small_node_size);type(p)=kern_node;
+subtype(p)=normal;
+width(p)=w;
+return p;
+}
+
+@ A |penalty_node| specifies the penalty associated with line or page
+breaking, in its |penalty| field. This field is a fullword integer, but
+the full range of integer values is not used: Any penalty | >= 10000| is
+treated as infinity, and no break will be allowed for such high values.
+Similarly, any penalty | <= -10000| is treated as negative infinity, and a
+break will be forced.
+
+ at d penalty_node 12 /*|type| of a penalty node*/
+ at d inf_penalty inf_bad /*``infinite'' penalty value*/
+ at d eject_penalty (-inf_penalty) /*``negatively infinite'' penalty value*/
+ at d penalty(A) mem[A+1].i /*the added cost of breaking a list here*/
+
+@ Anyone who has been reading the last few sections of the program will
+be able to guess what comes next.
+
+ at p pointer new_penalty(int @!m)
+{@+pointer p; /*the new node*/
+p=get_node(small_node_size);type(p)=penalty_node;
+subtype(p)=0; /*the |subtype| is not used*/
+penalty(p)=m;return p;
+}
+
+@ You might think that we have introduced enough node types by now. Well,
+almost, but there is one more: An |unset_node| has nearly the same format
+as an |hlist_node| or |vlist_node|; it is used for entries in \.{\\halign}
+or \.{\\valign} that are not yet in their final form, since the box
+dimensions are their ``natural'' sizes before any glue adjustment has been
+made. The |glue_set| word is not present; instead, we have a |glue_stretch|
+field, which contains the total stretch of order |glue_order| that is
+present in the hlist or vlist being boxed.
+Similarly, the |shift_amount| field is replaced by a |glue_shrink| field,
+containing the total shrink of order |glue_sign| that is present.
+The |subtype| field is called |span_count|; an unset box typically
+contains the data for |qo(span_count)+1| columns.
+Unset nodes will be changed to box nodes when alignment is completed.
+
+ at d unset_node 13 /*|type| for an unset node*/
+ at d unset_set_node 32 /*|type| for an unset |set_node|*/
+ at d unset_pack_node 33 /*|type| for an unset |pack_node|*/
+ at d glue_stretch(A) mem[A+glue_offset].sc /*total stretch in an unset node*/
+ at d glue_shrink(A) shift_amount(A) /*total shrink in an unset node*/
+ at d span_count(A) subtype(A) /*indicates the number of spanned columns*/
+
+@ In fact, there are still more types coming. When we get to math formula
+processing we will see that a |style_node| has |type==14|; and a number
+of larger type codes will also be defined, for use in math mode only.
+
+@ Warning: If any changes are made to these data structure layouts, such as
+changing any of the node sizes or even reordering the words of nodes,
+the |copy_node_list| procedure and the memory initialization code
+below may have to be changed. Such potentially dangerous parts of the
+program are listed in the index under `data structure assumptions'.
+@!@^data structure assumptions@>
+However, other references to the nodes are made symbolically in terms of
+the \.{WEB} macro definitions above, so that format changes will leave
+\TeX's other algorithms intact.
+@^system dependencies@>
+
+@* Memory layout.
+Some areas of |mem| are dedicated to fixed usage, since static allocation is
+more efficient than dynamic allocation when we can get away with it. For
+example, locations |mem_bot| to |mem_bot+3| are always used to store the
+specification for glue that is `\.{0pt plus 0pt minus 0pt}'. The
+following macro definitions accomplish the static allocation by giving
+symbolic names to the fixed positions. Static variable-size nodes appear
+in locations |mem_bot| through |lo_mem_stat_max|, and static single-word nodes
+appear in locations |hi_mem_stat_min| through |mem_top|, inclusive. It is
+harmless to let |lig_trick| and |garbage| share the same location of |mem|.
+
+ at d zero_glue mem_bot /*specification for \.{0pt plus 0pt minus 0pt}*/
+ at d fil_glue zero_glue+glue_spec_size /*\.{0pt plus 1fil minus 0pt}*/
+ at d fill_glue fil_glue+glue_spec_size /*\.{0pt plus 1fill minus 0pt}*/
+ at d ss_glue fill_glue+glue_spec_size /*\.{0pt plus 1fil minus 1fil}*/
+ at d fil_neg_glue ss_glue+glue_spec_size /*\.{0pt plus -1fil minus 0pt}*/
+ at d lo_mem_stat_max fil_neg_glue+glue_spec_size-1 /*largest statically
+ allocated word in the variable-size |mem|*/
+@#
+ at d page_ins_head mem_top /*list of insertion data for current page*/
+ at d contrib_head mem_top-1 /*vlist of items not yet on current page*/
+ at d page_head mem_top-2 /*vlist for current page*/
+ at d temp_head mem_top-3 /*head of a temporary list of some kind*/
+ at d hold_head mem_top-4 /*head of a temporary list of another kind*/
+ at d adjust_head mem_top-5 /*head of adjustment list returned by |hpack|*/
+ at d active mem_top-7 /*head of active list in |line_break|, needs two words*/
+ at d align_head mem_top-8 /*head of preamble list for alignments*/
+ at d end_span mem_top-9 /*tail of spanned-width lists*/
+ at d omit_template mem_top-10 /*a constant token list*/
+ at d null_list mem_top-11 /*permanently empty list*/
+ at d lig_trick mem_top-12 /*a ligature masquerading as a |char_node|*/
+ at d garbage mem_top-12 /*used for scrap information*/
+ at d backup_head mem_top-13 /*head of token list built by |scan_keyword|*/
+ at d setpage_head mem_top-14 /*head of page template list build by |new_setpage_node|*/
+ at d max_page type(setpage_head) /* maximum page template number */
+ at d max_stream subtype(setpage_head) /* maximum stream number */
+ at d hi_mem_stat_min mem_top-14 /*smallest statically allocated word in
+ the one-word |mem|*/
+ at d hi_mem_stat_usage 15 /*the number of one-word nodes always present*/
+
+@ The following code gets |mem| off to a good start, when \TeX\ is
+initializing itself the slow~way.
+
+@<Local variables for init...@>=
+int @!k; /*index into |mem|, |eqtb|, etc.*/
+
+@ @<Initialize table entries...@>=
+for (k=mem_bot+1; k<=lo_mem_stat_max; k++) mem[k].sc=0;
+ /*all glue dimensions are zeroed*/
+@^data structure assumptions@>
+k=mem_bot;@+while (k <= lo_mem_stat_max)
+ /*set first words of glue specifications*/
+ {@+glue_ref_count(k)=null+1;
+ stretch_order(k)=normal;shrink_order(k)=normal;
+ k=k+glue_spec_size;
+ }
+stretch(fil_glue)=unity;stretch_order(fil_glue)=fil;@/
+stretch(fill_glue)=unity;stretch_order(fill_glue)=fill;@/
+stretch(ss_glue)=unity;stretch_order(ss_glue)=fil;@/
+shrink(ss_glue)=unity;shrink_order(ss_glue)=fil;@/
+stretch(fil_neg_glue)=-unity;stretch_order(fil_neg_glue)=fil;@/
+rover=lo_mem_stat_max+1;
+link(rover)=empty_flag; /*now initialize the dynamic memory*/
+node_size(rover)=1000; /*which is a 1000-word available node*/
+llink(rover)=rover;rlink(rover)=rover;@/
+lo_mem_max=rover+1000;link(lo_mem_max)=null;info(lo_mem_max)=null;@/
+for (k=hi_mem_stat_min; k<=mem_top; k++)
+ mem[k]=mem[lo_mem_max]; /*clear list heads*/
+@<Initialize the special list heads and constant nodes@>;
+avail=null;mem_end=mem_top;
+hi_mem_min=hi_mem_stat_min; /*initialize the one-word memory*/
+var_used=lo_mem_stat_max+1-mem_bot;dyn_used=hi_mem_stat_usage;
+ /*initialize statistics*/
+
+@ If \TeX\ is extended improperly, the |mem| array might get screwed up.
+For example, some pointers might be wrong, or some ``dead'' nodes might not
+have been freed when the last reference to them disappeared. Procedures
+|check_mem| and |search_mem| are available to help diagnose such
+problems. These procedures make use of two arrays called |is_free| and
+|was_free| that are present only if \TeX's debugging routines have
+been included. (You may want to decrease the size of |mem| while you
+@^debugging@>
+are debugging.)
+
+@<Glob...@>=
+#ifdef @!DEBUG
+static bool @!is_free0[mem_max-mem_min+1], *const @!is_free = @!is_free0-mem_min; /*free cells*/
+ at t\hskip10pt@>static bool @!was_free0[mem_max-mem_min+1], *const @!was_free = @!was_free0-mem_min;
+ /*previously free cells*/
+ at t\hskip10pt@>static pointer @!was_mem_end, @!was_lo_max, @!was_hi_min;
+ /*previous |mem_end|, |lo_mem_max|, and |hi_mem_min|*/
+ at t\hskip10pt@>static bool @!panicking; /*do we want to check memory constantly?*/
+#endif
+
+@ @<Set initial...@>=
+#ifdef @!DEBUG
+was_mem_end=mem_min; /*indicate that everything was previously free*/
+was_lo_max=mem_min;was_hi_min=mem_max;
+panicking=false;
+#endif
+
+@ Procedure |check_mem| makes sure that the available space lists of
+|mem| are well formed, and it optionally prints out all locations
+that are reserved now but were free the last time this procedure was called.
+
+ at p
+#ifdef @!DEBUG
+static void check_mem(bool @!print_locs)
+{@+ /*loop exits*/
+int p, @!q; /*current locations of interest in |mem|*/
+bool @!clobbered; /*is something amiss?*/
+for (p=mem_min; p<=lo_mem_max; p++) is_free[p]=false; /*you can probably
+ do this faster*/
+for (p=hi_mem_min; p<=mem_end; p++) is_free[p]=false; /*ditto*/
+@<Check single-word |avail| list@>;
+@<Check variable-size |avail| list@>;
+@<Check flags of unavailable nodes@>;
+if (print_locs) @<Print newly busy locations@>;
+for (p=mem_min; p<=lo_mem_max; p++) was_free[p]=is_free[p];
+for (p=hi_mem_min; p<=mem_end; p++) was_free[p]=is_free[p];
+ /*|was_free=is_free| might be faster*/
+was_mem_end=mem_end;was_lo_max=lo_mem_max;was_hi_min=hi_mem_min;
+}
+#endif
+
+@ @<Check single-word...@>=
+p=avail;q=null;clobbered=false;
+while (p!=null)
+ {@+if ((p > mem_end)||(p < hi_mem_min)) clobbered=true;
+ else if (is_free[p]) clobbered=true;
+ if (clobbered)
+ {@+print_nl("AVAIL list clobbered at ");
+ at .AVAIL list clobbered...@>
+ print_int(q);goto done1;
+ }
+ is_free[p]=true;q=p;p=link(q);
+ }
+done1:
+
+@ @<Check variable-size...@>=
+p=rover;q=null;clobbered=false;
+@/do at +{if ((p >= lo_mem_max)||(p < mem_min)) clobbered=true;
+ else if ((rlink(p) >= lo_mem_max)||(rlink(p) < mem_min)) clobbered=true;
+ else if (!(is_empty(p))||(node_size(p) < 2)||@|
+ (p+node_size(p) > lo_mem_max)||@|(llink(rlink(p))!=p)) clobbered=true;
+ if (clobbered)
+ {@+print_nl("Double-AVAIL list clobbered at ");
+ print_int(q);goto done2;
+ }
+for (q=p; q<=p+node_size(p)-1; q++) /*mark all locations free*/
+ {@+if (is_free[q])
+ {@+print_nl("Doubly free location at ");
+ at .Doubly free location...@>
+ print_int(q);goto done2;
+ }
+ is_free[q]=true;
+ }
+q=p;p=rlink(p);
+}@+ while (!(p==rover));
+done2:
+
+@ @<Check flags...@>=
+p=mem_min;
+while (p <= lo_mem_max) /*node |p| should not be empty*/
+ {@+if (is_empty(p))
+ {@+print_nl("Bad flag at ");print_int(p);
+ at .Bad flag...@>
+ }
+ while ((p <= lo_mem_max)&&!is_free[p]) incr(p);
+ while ((p <= lo_mem_max)&&is_free[p]) incr(p);
+ }
+
+@ @<Print newly busy...@>=
+{@+print_nl("New busy locs:");
+for (p=mem_min; p<=lo_mem_max; p++)
+ if (!is_free[p]&&((p > was_lo_max)||was_free[p]))
+ {@+print_char(' ');print_int(p);
+ }
+for (p=hi_mem_min; p<=mem_end; p++)
+ if (!is_free[p]&&
+ ((p < was_hi_min)||(p > was_mem_end)||was_free[p]))
+ {@+print_char(' ');print_int(p);
+ }
+}
+
+@ The |search_mem| procedure attempts to answer the question ``Who points
+to node~|p|?'' In doing so, it fetches |link| and |info| fields of |mem|
+that might not be of type |two_halves|. Strictly speaking, this is
+@^dirty \PASCAL@>
+undefined in \PASCAL, and it can lead to ``false drops'' (words that seem to
+point to |p| purely by coincidence). But for debugging purposes, we want
+to rule out the places that do {\sl not\/} point to |p|, so a few false
+drops are tolerable.
+
+ at p
+#ifdef @!DEBUG
+static void search_mem(pointer @!p) /*look for pointers to |p|*/
+{@+int q; /*current position being searched*/
+for (q=mem_min; q<=lo_mem_max; q++)
+ {@+if (link(q)==p)
+ {@+print_nl("LINK(");print_int(q);print_char(')');
+ }
+ if (info(q)==p)
+ {@+print_nl("INFO(");print_int(q);print_char(')');
+ }
+ }
+for (q=hi_mem_min; q<=mem_end; q++)
+ {@+if (link(q)==p)
+ {@+print_nl("LINK(");print_int(q);print_char(')');
+ }
+ if (info(q)==p)
+ {@+print_nl("INFO(");print_int(q);print_char(')');
+ }
+ }
+@<Search |eqtb| for equivalents equal to |p|@>;
+@<Search |save_stack| for equivalents that point to |p|@>;
+@<Search |hyph_list| for pointers to |p|@>;
+}
+#endif
+
+@* Displaying boxes.
+We can reinforce our knowledge of the data structures just introduced
+by considering two procedures that display a list in symbolic form.
+The first of these, called |short_display|, is used in ``overfull box''
+messages to give the top-level description of a list. The other one,
+called |show_node_list|, prints a detailed description of exactly what
+is in the data structure.
+
+The philosophy of |short_display| is to ignore the fine points about exactly
+what is inside boxes, except that ligatures and discretionary breaks are
+expanded. As a result, |short_display| is a recursive procedure, but the
+recursion is never more than one level deep.
+@^recursion@>
+
+A global variable |font_in_short_display| keeps track of the font code that
+is assumed to be present when |short_display| begins; deviations from this
+font will be printed.
+
+@<Glob...@>=
+int @!font_in_short_display; /*an internal font number*/
+
+@ Boxes, rules, inserts, whatsits, marks, and things in general that are
+sort of ``complicated'' are indicated only by printing `\.{[]}'.
+
+ at p void short_display(int @!p) /*prints highlights of list |p|*/
+{@+int n; /*for replacement counts*/
+while (p > mem_min)
+ {@+if (is_char_node(p))
+ {@+if (p <= mem_end)
+ {@+if (font(p)!=font_in_short_display)
+ {@+if ((font(p) < font_base)||(font(p) > font_max))
+ print_char('*');
+ at .*\relax@>
+ else@<Print the font identifier for |font(p)|@>;
+ print_char(' ');font_in_short_display=font(p);
+ }
+ print_ASCII(qo(character(p)));
+ }
+ }
+ else@<Print a short indication of the contents of node |p|@>;
+ p=link(p);
+ }
+}
+
+@ @<Print a short indication of the contents of node |p|@>=
+switch (type(p)) {
+case hlist_node: case vlist_node: case ins_node: case whatsit_node: case mark_node: case adjust_node:
+ case unset_node: case unset_set_node: case unset_pack_node: print("[]");@+break;
+case rule_node: print_char('|');@+break;
+case glue_node: if (glue_ptr(p)!=zero_glue) print_char(' ');@+break;
+case math_node: print_char('$');@+break;
+case ligature_node: short_display(lig_ptr(p));@+break;
+case disc_node: {@+short_display(pre_break(p));
+ short_display(post_break(p));@/
+ n=replace_count(p);
+ while (n > 0)
+ {@+if (link(p)!=null) p=link(p);
+ decr(n);
+ }
+ } @+break;
+default:do_nothing;
+}
+
+@ The |show_node_list| routine requires some auxiliary subroutines: one to
+print a font-and-character combination, one to print a token list without
+its reference count, and one to print a rule dimension.
+
+ at p static void print_font_and_char(int @!p) /*prints |char_node| data*/
+{@+if (p > mem_end) print_esc("CLOBBERED.");
+else{@+if ((font(p) < font_base)||(font(p) > font_max)) print_char('*');
+ at .*\relax@>
+ else@<Print the font identifier for |font(p)|@>;
+ print_char(' ');print_ASCII(qo(character(p)));
+ }
+}
+@#
+static void print_mark(int @!p) /*prints token list data in braces*/
+{@+print_char('{');
+if ((p < hi_mem_min)||(p > mem_end)) print_esc("CLOBBERED.");
+else show_token_list(link(p), null, max_print_line-10);
+print_char('}');
+}
+@#
+static void print_rule_dimen(scaled @!d) /*prints dimension in rule node*/
+{@+if (is_running(d)) print_char('*');else print_scaled(d);
+ at .*\relax@>
+}
+
+@ Then there is a subroutine that prints glue stretch and shrink, possibly
+followed by the name of finite units:
+
+ at p static void print_glue(scaled @!d, int @!order, char *@!s)
+ /*prints a glue component*/
+{@+print_scaled(d);
+if ((order < normal)||(order > filll)) print("foul");
+else if (order > normal)
+ {@+print("fil");
+ while (order > fil)
+ {@+print_char('l');decr(order);
+ }
+ }
+else if (s!=0) print(s);
+}
+
+@ The next subroutine prints a whole glue specification.
+
+ at p void print_spec(int @!p, char *@!s)
+ /*prints a glue specification*/
+{@+if ((p < mem_min)||(p >= lo_mem_max)) print_char('*');
+ at .*\relax@>
+else{@+print_scaled(width(p));
+ if (s!=0) print(s);
+ if (stretch(p)!=0)
+ {@+print(" plus ");print_glue(stretch(p), stretch_order(p), s);
+ }
+ if (shrink(p)!=0)
+ {@+print(" minus ");print_glue(shrink(p), shrink_order(p), s);
+ }
+ }
+}
+
+@ We also need to declare some procedures that appear later in this
+documentation.
+
+ at p @<Declare procedures needed for displaying the elements of mlists@>@;
+@<Declare the procedure called |print_skip_param|@>@;
+static void print_xdimen(pointer p)
+{ print_scaled(xdimen_width(p));
+ if (xdimen_hfactor(p)!=0)
+ { print_char('+');print_scaled(xdimen_hfactor(p));print("*hsize");}
+ if (xdimen_vfactor(p)!=0)
+ { print_char('+');print_scaled(xdimen_vfactor(p));print("*vsize");}
+}
+extern void print_baseline_skip(int i);
+
+@ Since boxes can be inside of boxes, |show_node_list| is inherently recursive,
+@^recursion@>
+up to a given maximum number of levels. The history of nesting is indicated
+by the current string, which will be printed at the beginning of each line;
+the length of this string, namely |cur_length|, is the depth of nesting.
+
+Recursive calls on |show_node_list| therefore use the following pattern:
+
+ at d node_list_display(A)
+ {@+append_char('.');show_node_list(A);flush_char;
+ } /*|str_room| need not be checked; see |show_box| below*/
+
+@ A global variable called |depth_threshold| is used to record the maximum
+depth of nesting for which |show_node_list| will show information. If we
+have |depth_threshold==0|, for example, only the top level information will
+be given and no sublists will be traversed. Another global variable, called
+|breadth_max|, tells the maximum number of items to show at each level;
+|breadth_max| had better be positive, or you won't see anything.
+
+@<Glob...@>=
+int @!depth_threshold; /*maximum nesting depth in box displays*/
+int @!breadth_max; /*maximum number of items shown at the same list level*/
+
+@ Now we are ready for |show_node_list| itself. This procedure has been
+written to be ``extra robust'' in the sense that it should not crash or get
+into a loop even if the data structures have been messed up by bugs in
+the rest of the program. You can safely call its parent routine
+|show_box(p)| for arbitrary values of |p| when you are debugging \TeX.
+However, in the presence of bad data, the procedure may
+@^dirty \PASCAL@>@^debugging@>
+fetch a |memory_word| whose variant is different from the way it was stored;
+for example, it might try to read |mem[p].hh| when |mem[p]|
+contains a scaled integer, if |p| is a pointer that has been
+clobbered or chosen at random.
+
+ at p void show_node_list(int @!p) /*prints a node list symbolically*/
+{@+
+int n; /*the number of items already printed at this level*/
+double @!g; /*a glue ratio, as a floating point number*/
+if (cur_length > depth_threshold)
+ {@+if (p > null) print(" []");
+ /*indicate that there's been some truncation*/
+ return;
+ }
+n=0;
+while (p > mem_min)
+ {@+print_ln();print_current_string(); /*display the nesting history*/
+ if (p > mem_end) /*pointer out of range*/
+ {@+print("Bad link, display aborted.");return;
+ at .Bad link...@>
+ }
+ incr(n);if (n > breadth_max) /*time to stop*/
+ {@+print("etc.");return;
+ at .etc@>
+ }
+ display_node(p);
+ p=link(p);
+ }
+
+}
+
+void display_node(pointer p)@t~@>
+{
+if (is_char_node(p)) print_font_and_char(p);
+else switch (type(p)) {
+ case hlist_node: case vlist_node: case unset_node: case unset_set_node: case unset_pack_node: @<Display box |p|@>@;@+break;
+ case rule_node: @<Display rule |p|@>@;@+break;
+ case ins_node: @<Display insertion |p|@>@;@+break;
+ case whatsit_node: @<Display the whatsit node |p|@>@;@+break;
+ case glue_node: @<Display glue |p|@>@;@+break;
+ case kern_node: @<Display kern |p|@>@;@+break;
+ case math_node: @<Display math node |p|@>@;@+break;
+ case ligature_node: @<Display ligature |p|@>@;@+break;
+ case penalty_node: @<Display penalty |p|@>@;@+break;
+ case disc_node: @<Display discretionary |p|@>@;@+break;
+ case mark_node: @<Display mark |p|@>@;@+break;
+ case adjust_node: @<Display adjustment |p|@>@;@+break;
+ @t\4@>@<Cases of |show_node_list| that arise in mlists only@>@;
+ default:print("Unknown node type!");
+ }
+}
+
+@ @<Display box |p|@>=
+{@+if (type(p)==hlist_node) print_esc("h");
+else if (type(p)==vlist_node) print_esc("v");
+else print_esc("unset");
+print("box(");print_scaled(height(p));print_char('+');
+print_scaled(depth(p));print(")x");print_scaled(width(p));
+if (type(p)==unset_set_node) print("set");
+else if (type(p)==unset_pack_node) print("pack");
+else if (type(p)==unset_node)
+ @<Display special fields of the unset node |p|@>@;
+else{@+@<Display the value of |glue_set(p)|@>;
+ if (shift_amount(p)!=0)
+ {@+print(", shifted ");print_scaled(shift_amount(p));
+ }
+ }
+node_list_display(list_ptr(p)); /*recursive call*/
+}
+
+@ @<Display special fields of the unset node |p|@>=
+{@+if (span_count(p)!=min_quarterword)
+ {@+print(" (");print_int(qo(span_count(p))+1);
+ print(" columns)");
+ }
+if (glue_stretch(p)!=0)
+ {@+print(", stretch ");print_glue(glue_stretch(p), glue_order(p), 0);
+ }
+if (glue_shrink(p)!=0)
+ {@+print(", shrink ");print_glue(glue_shrink(p), glue_sign(p), 0);
+ }
+}
+
+@ The code will have to change in this place if |glue_ratio| is
+a structured type instead of an ordinary |double|. Note that this routine
+should avoid arithmetic errors even if the |glue_set| field holds an
+arbitrary random value. The following code assumes that a properly
+formed nonzero |double| number has absolute value $2^{20}$ or more when
+it is regarded as an integer; this precaution was adequate to prevent
+floating point underflow on the author's computer.
+@^system dependencies@>
+@^dirty \PASCAL@>
+
+@<Display the value of |glue_set(p)|@>=
+g=float(glue_set(p));
+if ((g!=float_constant(0))&&(glue_sign(p)!=normal))
+ {@+print(", glue set ");
+ if (glue_sign(p)==shrinking) print("- ");
+ if (abs(mem[p+glue_offset].i) < 04000000) print("?.?");
+ else if (abs(g) > float_constant(20000))
+ {@+if (g > float_constant(0)) print_char('>');
+ else print("< -");
+ print_glue(20000*unity, glue_order(p), 0);
+ }
+ else print_glue(round(unity*g), glue_order(p), 0);
+@^real multiplication@>
+ }
+
+@ @<Display rule |p|@>=
+{@+print_esc("rule(");print_rule_dimen(height(p));print_char('+');
+print_rule_dimen(depth(p));print(")x");print_rule_dimen(width(p));
+}
+
+@ @<Display insertion |p|@>=
+{@+print_esc("insert");print_int(qo(subtype(p)));
+print("; split(");print_spec(split_top_ptr(p), 0);
+print_char(',');print_scaled(depth(p));
+print("); float cost ");print_int(float_cost(p));
+node_list_display(ins_ptr(p)); /*recursive call*/
+}
+
+@ @<Display glue |p|@>=
+if (subtype(p) >= a_leaders) @<Display leaders |p|@>@;
+else{@+print_esc("glue");
+ if (subtype(p)!=normal)
+ {@+print_char('(');
+ if (subtype(p) < cond_math_glue)
+ print_skip_param(subtype(p)-1);
+ else if (subtype(p)==cond_math_glue) print_esc("nonscript");
+ else print_esc("mskip");
+ print_char(')');
+ }
+ if (subtype(p)!=cond_math_glue)
+ {@+print_char(' ');
+ if (subtype(p) < cond_math_glue) print_spec(glue_ptr(p), 0);
+ else print_spec(glue_ptr(p),"mu");
+ }
+ }
+
+@ @<Display leaders |p|@>=
+{@+print_esc("");
+if (subtype(p)==c_leaders) print_char('c');
+else if (subtype(p)==x_leaders) print_char('x');
+print("leaders ");print_spec(glue_ptr(p), 0);
+node_list_display(leader_ptr(p)); /*recursive call*/
+}
+
+@ An ``explicit'' kern value is indicated implicitly by an explicit space.
+
+@<Display kern |p|@>=
+if (subtype(p)!=mu_glue)
+ {@+print_esc("kern");
+ if (subtype(p)!=normal) print_char(' ');
+ print_scaled(width(p));
+ if (subtype(p)==acc_kern) print(" (for accent)");
+ at .for accent@>
+ }
+else{@+print_esc("mkern");print_scaled(width(p));print("mu");
+ }
+
+@ @<Display math node |p|@>=
+{@+print_esc("math");
+if (subtype(p)==before) print("on");
+else print("off");
+if (width(p)!=0)
+ {@+print(", surrounded ");print_scaled(width(p));
+ }
+}
+
+@ @<Display ligature |p|@>=
+{@+print_font_and_char(lig_char(p));print(" (ligature ");
+if (subtype(p) > 1) print_char('|');
+font_in_short_display=font(lig_char(p));short_display(lig_ptr(p));
+if (odd(subtype(p))) print_char('|');
+print_char(')');
+}
+
+@ @<Display penalty |p|@>=
+{@+print_esc("penalty ");print_int(penalty(p));
+}
+
+@ The |post_break| list of a discretionary node is indicated by a prefixed
+`\.{\char'174}' instead of the `\..' before the |pre_break| list.
+
+@<Display discretionary |p|@>=
+{@+print_esc("discretionary");
+if (replace_count(p) > 0)
+ {@+print(" replacing ");print_int(replace_count(p));
+ }
+node_list_display(pre_break(p)); /*recursive call*/
+append_char('|');show_node_list(post_break(p));flush_char; /*recursive call*/
+}
+
+@ @<Display mark |p|@>=
+{@+print_esc("mark");print_mark(mark_ptr(p));
+}
+
+@ @<Display adjustment |p|@>=
+{@+print_esc("vadjust");node_list_display(adjust_ptr(p)); /*recursive call*/
+}
+
+@ The recursive machinery is started by calling |show_box|.
+@^recursion@>
+
+ at p void show_box(pointer @!p)
+{@+@<Assign the values |depth_threshold:=show_box_depth| and |breadth_max:=show_box_breadth|@>;
+if (breadth_max <= 0) breadth_max=5;
+if (pool_ptr+depth_threshold >= pool_size)
+ depth_threshold=pool_size-pool_ptr-1;
+ /*now there's enough room for prefix string*/
+show_node_list(p); /*the show starts at |p|*/
+print_ln();
+}
+
+@* Destroying boxes.
+When we are done with a node list, we are obliged to return it to free
+storage, including all of its sublists. The recursive procedure
+|flush_node_list| does this for us.
+
+@ First, however, we shall consider two non-recursive procedures that do
+simpler tasks. The first of these, |delete_token_ref|, is called when
+a pointer to a token list's reference count is being removed. This means
+that the token list should disappear if the reference count was |null|,
+otherwise the count should be decreased by one.
+@^reference counts@>
+
+ at d token_ref_count(A) info(A) /*reference count preceding a token list*/
+
+ at p void delete_token_ref(pointer @!p) /*|p| points to the reference count
+ of a token list that is losing one reference*/
+{@+if (token_ref_count(p)==null) flush_list(p);
+else decr(token_ref_count(p));
+}
+
+@ Similarly, |delete_glue_ref| is called when a pointer to a glue
+specification is being withdrawn.
+@^reference counts@>
+ at d fast_delete_glue_ref(A) @t@>@;@/
+ {@+if (glue_ref_count(A)==null) free_node(A, glue_spec_size);
+ else decr(glue_ref_count(A));
+ }
+
+ at p void delete_glue_ref(pointer @!p) /*|p| points to a glue specification*/
+fast_delete_glue_ref(p)
+void delete_xdimen_ref(pointer @!p) /*|p| points to a xdimen specification*/
+{@+if (xdimen_ref_count(p)==null) free_node(p, xdimen_node_size);
+ else decr(xdimen_ref_count(p));
+}
+
+@ Now we are ready to delete any node list, recursively.
+In practice, the nodes deleted are usually charnodes (about 2/3 of the time),
+and they are glue nodes in about half of the remaining cases.
+@^recursion@>
+
+ at p void flush_node_list(pointer @!p) /*erase list of nodes starting at |p|*/
+{@+ /*go here when node |p| has been freed*/
+pointer q; /*successor to node |p|*/
+while (p!=null)
+@^inner loop@>
+ {@+q=link(p);
+ if (is_char_node(p)) free_avail(p)@;
+ else{@+switch (type(p)) {
+ case hlist_node: case vlist_node: case unset_node: case unset_set_node: case unset_pack_node: {@+flush_node_list(list_ptr(p));
+ free_node(p, box_node_size);goto done;
+ }
+ case rule_node: {@+free_node(p, rule_node_size);goto done;
+ }
+ case ins_node: {@+flush_node_list(ins_ptr(p));
+ delete_glue_ref(split_top_ptr(p));
+ free_node(p, ins_node_size);goto done;
+ }
+ case whatsit_node: @<Wipe out the whatsit node |p| and |goto done|@>@;
+ case glue_node: {@+fast_delete_glue_ref(glue_ptr(p));
+ if (leader_ptr(p)!=null) flush_node_list(leader_ptr(p));
+ } @+break;
+ case kern_node: case math_node: case penalty_node: do_nothing;@+break;
+ case ligature_node: flush_node_list(lig_ptr(p));@+break;
+ case mark_node: delete_token_ref(mark_ptr(p));@+break;
+ case disc_node: {@+flush_node_list(pre_break(p));
+ flush_node_list(post_break(p));
+ } @+break;
+ case adjust_node: flush_node_list(adjust_ptr(p));@+break;
+ @t\4@>@<Cases of |flush_node_list| that arise in mlists only@>@;
+ default:confusion("flushing");
+@:this can't happen flushing}{\quad flushing@>
+ } @/
+ free_node(p, small_node_size);
+ done: ;}
+ p=q;
+ }
+}
+
+@* Copying boxes.
+Another recursive operation that acts on boxes is sometimes needed: The
+procedure |copy_node_list| returns a pointer to another node list that has
+the same structure and meaning as the original. Note that since glue
+specifications and token lists have reference counts, we need not make
+copies of them. Reference counts can never get too large to fit in a
+halfword, since each pointer to a node is in a different memory address,
+and the total number of memory addresses fits in a halfword.
+@^recursion@>
+@^reference counts@>
+
+(Well, there actually are also references from outside |mem|; if the
+|save_stack| is made arbitrarily large, it would theoretically be possible
+to break \TeX\ by overflowing a reference count. But who would want to do that?)
+
+ at d add_token_ref(A) incr(token_ref_count(A)) /*new reference to a token list*/
+ at d add_glue_ref(A) incr(glue_ref_count(A)) /*new reference to a glue spec*/
+ at d add_xdimen_ref(A) incr(xdimen_ref_count(A)) /*new reference to an xdimen*/
+
+@ The copying procedure copies words en masse without bothering
+to look at their individual fields. If the node format changes---for
+example, if the size is altered, or if some link field is moved to another
+relative position---then this code may need to be changed too.
+@^data structure assumptions@>
+
+ at p pointer copy_node_list(pointer @!p) /*makes a duplicate of the
+ node list that starts at |p| and returns a pointer to the new list*/
+{@+pointer h; /*temporary head of copied list*/
+pointer @!q; /*previous position in new list*/
+pointer @!r; /*current node being fabricated for new list*/
+int @!words; /*number of words remaining to be copied*/
+h=get_avail();q=h;
+while (p!=null)
+ {@+@<Make a copy of node |p| in node |r|@>;
+ link(q)=r;q=r;p=link(p);
+ }
+link(q)=null;q=link(h);free_avail(h);
+return q;
+}
+
+@ @<Make a copy of node |p|...@>=
+words=1; /*this setting occurs in more branches than any other*/
+if (is_char_node(p)) r=get_avail();
+else@<Case statement to copy different types and set |words| to the number of initial
+words not yet copied@>;
+while (words > 0)
+ {@+decr(words);mem[r+words]=mem[p+words];
+ }
+
+@ @<Case statement to copy...@>=
+switch (type(p)) {
+case hlist_node: case vlist_node: case unset_node: case unset_set_node: case unset_pack_node: {@+r=get_node(box_node_size);
+ mem[r+6]=mem[p+6];mem[r+5]=mem[p+5]; /*copy the last two words*/
+ list_ptr(r)=copy_node_list(list_ptr(p)); /*this affects |mem[r+5]|*/
+ words=5;
+ } @+break;
+case rule_node: {@+r=get_node(rule_node_size);words=rule_node_size;
+ } @+break;
+case ins_node: {@+r=get_node(ins_node_size);mem[r+4]=mem[p+4];
+ add_glue_ref(split_top_ptr(p));
+ ins_ptr(r)=copy_node_list(ins_ptr(p)); /*this affects |mem[r+4]|*/
+ words=ins_node_size-1;
+ } @+break;
+case whatsit_node: @<Make a partial copy of the whatsit node |p| and make |r| point
+to it; set |words| to the number of initial words not yet copied@>@;@+break;
+case glue_node: {@+r=get_node(small_node_size);add_glue_ref(glue_ptr(p));
+ glue_ptr(r)=glue_ptr(p);leader_ptr(r)=copy_node_list(leader_ptr(p));
+ } @+break;
+case kern_node: case math_node: case penalty_node: {@+r=get_node(small_node_size);
+ words=small_node_size;
+ } @+break;
+case ligature_node: {@+r=get_node(small_node_size);
+ mem[lig_char(r)]=mem[lig_char(p)]; /*copy |font| and |character|*/
+ lig_ptr(r)=copy_node_list(lig_ptr(p));
+ } @+break;
+case disc_node: {@+r=get_node(small_node_size);
+ pre_break(r)=copy_node_list(pre_break(p));
+ post_break(r)=copy_node_list(post_break(p));
+ } @+break;
+case mark_node: {@+r=get_node(small_node_size);add_token_ref(mark_ptr(p));
+ words=small_node_size;
+ } @+break;
+case adjust_node: {@+r=get_node(small_node_size);
+ adjust_ptr(r)=copy_node_list(adjust_ptr(p));
+ } @+break; /*|words==1==small_node_size-1|*/
+default:confusion("copying");
+@:this can't happen copying}{\quad copying@>
+}
+
+@* The command codes.
+Before we can go any further, we need to define symbolic names for the internal
+code numbers that represent the various commands obeyed by \TeX. These codes
+are somewhat arbitrary, but not completely so. For example, the command
+codes for character types are fixed by the language, since a user says,
+e.g., `\.{\\catcode \`\\\${} = 3}' to make \.{\char'44} a math delimiter,
+and the command code |math_shift| is equal to~3. Some other codes have
+been made adjacent so that |case| statements in the program need not consider
+cases that are widely spaced, or so that |case| statements can be replaced
+by |if (| statements.
+
+At any rate, here is the list, for future reference. First come the
+``catcode'' commands, several of which share their numeric codes with
+ordinary commands when the catcode cannot emerge from \TeX's scanning routine.
+
+ at d escape 0 /*escape delimiter (called \.\\ in {\sl The \TeX book\/})*/
+@:TeXbook}{\sl The \TeX book@>
+ at d relax 0 /*do nothing ( \.{\\relax} )*/
+ at d left_brace 1 /*beginning of a group ( \.\{ )*/
+ at d right_brace 2 /*ending of a group ( \.\} )*/
+ at d math_shift 3 /*mathematics shift character ( \.\$ )*/
+ at d tab_mark 4 /*alignment delimiter ( \.\&, \.{\\span} )*/
+ at d car_ret 5 /*end of line ( |carriage_return|, \.{\\cr}, \.{\\crcr} )*/
+ at d out_param 5 /*output a macro parameter*/
+ at d mac_param 6 /*macro parameter symbol ( \.\# )*/
+ at d sup_mark 7 /*superscript ( \.{\char'136} )*/
+ at d sub_mark 8 /*subscript ( \.{\char'137} )*/
+ at d ignore 9 /*characters to ignore ( \.{\^\^@@} )*/
+ at d endv 9 /*end of \<v_j> list in alignment template*/
+ at d spacer 10 /*characters equivalent to blank space ( \.{\ } )*/
+ at d letter 11 /*characters regarded as letters ( \.{A..Z}, \.{a..z} )*/
+ at d other_char 12 /*none of the special character types*/
+ at d active_char 13 /*characters that invoke macros ( \.{\char`\~} )*/
+ at d par_end 13 /*end of paragraph ( \.{\\par} )*/
+ at d match 13 /*match a macro parameter*/
+ at d comment 14 /*characters that introduce comments ( \.\% )*/
+ at d end_match 14 /*end of parameters to macro*/
+ at d stop 14 /*end of job ( \.{\\end}, \.{\\dump} )*/
+ at d invalid_char 15 /*characters that shouldn't appear ( \.{\^\^?} )*/
+ at d delim_num 15 /*specify delimiter numerically ( \.{\\delimiter} )*/
+ at d max_char_code 15 /*largest catcode for individual characters*/
+
+@ Next are the ordinary run-of-the-mill command codes. Codes that are
+|min_internal| or more represent internal quantities that might be
+expanded by `\.{\\the}'.
+
+ at d char_num 16 /*character specified numerically ( \.{\\char} )*/
+ at d math_char_num 17 /*explicit math code ( \.{\\mathchar} )*/
+ at d mark 18 /*mark definition ( \.{\\mark} )*/
+ at d xray 19 /*peek inside of \TeX\ ( \.{\\show}, \.{\\showbox}, etc.~)*/
+ at d make_box 20 /*make a box ( \.{\\box}, \.{\\copy}, \.{\\hbox}, etc.~)*/
+ at d hmove 21 /*horizontal motion ( \.{\\moveleft}, \.{\\moveright} )*/
+ at d vmove 22 /*vertical motion ( \.{\\raise}, \.{\\lower} )*/
+ at d un_hbox 23 /*unglue a box ( \.{\\unhbox}, \.{\\unhcopy} )*/
+ at d un_vbox 24 /*unglue a box ( \.{\\unvbox}, \.{\\unvcopy} )*/
+ at d remove_item 25 /*nullify last item ( \.{\\unpenalty},
+ \.{\\unkern}, \.{\\unskip} )*/
+ at d hskip 26 /*horizontal glue ( \.{\\hskip}, \.{\\hfil}, etc.~)*/
+ at d vskip 27 /*vertical glue ( \.{\\vskip}, \.{\\vfil}, etc.~)*/
+ at d mskip 28 /*math glue ( \.{\\mskip} )*/
+ at d kern 29 /*fixed space ( \.{\\kern})*/
+ at d mkern 30 /*math kern ( \.{\\mkern} )*/
+ at d leader_ship 31 /*use a box ( \.{\\shipout}, \.{\\leaders}, etc.~)*/
+ at d halign 32 /*horizontal table alignment ( \.{\\halign} )*/
+ at d valign 33 /*vertical table alignment ( \.{\\valign} )*/
+ at d no_align 34 /*temporary escape from alignment ( \.{\\noalign} )*/
+ at d vrule 35 /*vertical rule ( \.{\\vrule} )*/
+ at d hrule 36 /*horizontal rule ( \.{\\hrule} )*/
+ at d insert 37 /*vlist inserted in box ( \.{\\insert} )*/
+ at d vadjust 38 /*vlist inserted in enclosing paragraph ( \.{\\vadjust} )*/
+ at d ignore_spaces 39 /*gobble |spacer| tokens ( \.{\\ignorespaces} )*/
+ at d after_assignment 40 /*save till assignment is done ( \.{\\afterassignment} )*/
+ at d after_group 41 /*save till group is done ( \.{\\aftergroup} )*/
+ at d break_penalty 42 /*additional badness ( \.{\\penalty} )*/
+ at d start_par 43 /*begin paragraph ( \.{\\indent}, \.{\\noindent} )*/
+ at d ital_corr 44 /*italic correction ( \.{\\/} )*/
+ at d accent 45 /*attach accent in text ( \.{\\accent} )*/
+ at d math_accent 46 /*attach accent in math ( \.{\\mathaccent} )*/
+ at d discretionary 47 /*discretionary texts ( \.{\\-}, \.{\\discretionary} )*/
+ at d eq_no 48 /*equation number ( \.{\\eqno}, \.{\\leqno} )*/
+ at d left_right 49 /*variable delimiter ( \.{\\left}, \.{\\right} )*/
+ /*( or \.{\\middle} )*/
+ at d math_comp 50 /*component of formula ( \.{\\mathbin}, etc.~)*/
+ at d limit_switch 51 /*diddle limit conventions ( \.{\\displaylimits}, etc.~)*/
+ at d above 52 /*generalized fraction ( \.{\\above}, \.{\\atop}, etc.~)*/
+ at d math_style 53 /*style specification ( \.{\\displaystyle}, etc.~)*/
+ at d math_choice 54 /*choice specification ( \.{\\mathchoice} )*/
+ at d non_script 55 /*conditional math glue ( \.{\\nonscript} )*/
+ at d vcenter 56 /*vertically center a vbox ( \.{\\vcenter} )*/
+ at d case_shift 57 /*force specific case ( \.{\\lowercase}, \.{\\uppercase}~)*/
+ at d message 58 /*send to user ( \.{\\message}, \.{\\errmessage} )*/
+ at d extension 59 /*extensions to \TeX\ ( \.{\\write}, \.{\\special}, etc.~)*/
+ at d in_stream 60 /*files for reading ( \.{\\openin}, \.{\\closein} )*/
+ at d begin_group 61 /*begin local grouping ( \.{\\begingroup} )*/
+ at d end_group 62 /*end local grouping ( \.{\\endgroup} )*/
+ at d omit 63 /*omit alignment template ( \.{\\omit} )*/
+ at d ex_space 64 /*explicit space ( \.{\\\ } )*/
+ at d no_boundary 65 /*suppress boundary ligatures ( \.{\\noboundary} )*/
+ at d radical 66 /*square root and similar signs ( \.{\\radical} )*/
+ at d end_cs_name 67 /*end control sequence ( \.{\\endcsname} )*/
+ at d min_internal 68 /*the smallest code that can follow \.{\\the}*/
+ at d char_given 68 /*character code defined by \.{\\chardef}*/
+ at d math_given 69 /*math code defined by \.{\\mathchardef}*/
+ at d last_item 70 /*most recent item ( \.{\\lastpenalty},
+ \.{\\lastkern}, \.{\\lastskip} )*/
+ at d max_non_prefixed_command 70 /*largest command code that can't be \.{\\global}*/
+
+@ The next codes are special; they all relate to mode-independent
+assignment of values to \TeX's internal registers or tables.
+Codes that are |max_internal| or less represent internal quantities
+that might be expanded by `\.{\\the}'.
+
+ at d toks_register 71 /*token list register ( \.{\\toks} )*/
+ at d assign_toks 72 /*special token list ( \.{\\output}, \.{\\everypar}, etc.~)*/
+ at d assign_int 73 /*user-defined integer ( \.{\\tolerance}, \.{\\day}, etc.~)*/
+ at d assign_dimen 74 /*user-defined length ( \.{\\hsize}, etc.~)*/
+ at d assign_glue 75 /*user-defined glue ( \.{\\baselineskip}, etc.~)*/
+ at d assign_mu_glue 76 /*user-defined muglue ( \.{\\thinmuskip}, etc.~)*/
+ at d assign_font_dimen 77 /*user-defined font dimension ( \.{\\fontdimen} )*/
+ at d assign_font_int 78 /*user-defined font integer ( \.{\\hyphenchar},
+ \.{\\skewchar} )*/
+ at d set_aux 79 /*specify state info ( \.{\\spacefactor}, \.{\\prevdepth} )*/
+ at d set_prev_graf 80 /*specify state info ( \.{\\prevgraf} )*/
+ at d set_page_dimen 81 /*specify state info ( \.{\\pagegoal}, etc.~)*/
+ at d set_page_int 82 /*specify state info ( \.{\\deadcycles},
+ \.{\\insertpenalties} )*/
+ /*( or \.{\\interactionmode} )*/
+ at d set_box_dimen 83 /*change dimension of box ( \.{\\wd}, \.{\\ht}, \.{\\dp} )*/
+ at d set_shape 84 /*specify fancy paragraph shape ( \.{\\parshape} )*/
+ /*(or \.{\\interlinepenalties}, etc.~)*/
+ at d def_code 85 /*define a character code ( \.{\\catcode}, etc.~)*/
+ at d def_family 86 /*declare math fonts ( \.{\\textfont}, etc.~)*/
+ at d set_font 87 /*set current font ( font identifiers )*/
+ at d def_font 88 /*define a font file ( \.{\\font} )*/
+ at d internal_register 89 /*internal register ( \.{\\count}, \.{\\dimen}, etc.~)*/
+ at d max_internal 89 /*the largest code that can follow \.{\\the}*/
+ at d advance 90 /*advance a register or parameter ( \.{\\advance} )*/
+ at d multiply 91 /*multiply a register or parameter ( \.{\\multiply} )*/
+ at d divide 92 /*divide a register or parameter ( \.{\\divide} )*/
+ at d prefix 93 /*qualify a definition ( \.{\\global}, \.{\\long}, \.{\\outer} )*/
+ /*( or \.{\\protected} )*/
+ at d let 94 /*assign a command code ( \.{\\let}, \.{\\futurelet} )*/
+ at d shorthand_def 95 /*code definition ( \.{\\chardef}, \.{\\countdef}, etc.~)*/
+ at d read_to_cs 96 /*read into a control sequence ( \.{\\read} )*/
+ /*( or \.{\\readline} )*/
+ at d def 97 /*macro definition ( \.{\\def}, \.{\\gdef}, \.{\\xdef}, \.{\\edef} )*/
+ at d set_box 98 /*set a box ( \.{\\setbox} )*/
+ at d hyph_data 99 /*hyphenation data ( \.{\\hyphenation}, \.{\\patterns} )*/
+ at d set_interaction 100 /*define level of interaction ( \.{\\batchmode}, etc.~)*/
+ at d max_command 100 /*the largest command code seen at |big_switch|*/
+
+@ The remaining command codes are extra special, since they cannot get through
+\TeX's scanner to the main control routine. They have been given values higher
+than |max_command| so that their special nature is easily discernible.
+The ``expandable'' commands come first.
+
+ at d undefined_cs (max_command+1) /*initial state of most |eq_type| fields*/
+ at d expand_after (max_command+2) /*special expansion ( \.{\\expandafter} )*/
+ at d no_expand (max_command+3) /*special nonexpansion ( \.{\\noexpand} )*/
+ at d input (max_command+4) /*input a source file ( \.{\\input}, \.{\\endinput} )*/
+ /*( or \.{\\scantokens} )*/
+ at d if_test (max_command+5) /*conditional text ( \.{\\if}, \.{\\ifcase}, etc.~)*/
+ at d fi_or_else (max_command+6) /*delimiters for conditionals ( \.{\\else}, etc.~)*/
+ at d cs_name (max_command+7) /*make a control sequence from tokens ( \.{\\csname} )*/
+ at d convert (max_command+8) /*convert to text ( \.{\\number}, \.{\\string}, etc.~)*/
+ at d the (max_command+9) /*expand an internal quantity ( \.{\\the} )*/
+ /*( or \.{\\unexpanded}, \.{\\detokenize} )*/
+ at d top_bot_mark (max_command+10) /*inserted mark ( \.{\\topmark}, etc.~)*/
+ at d call (max_command+11) /*non-long, non-outer control sequence*/
+ at d long_call (max_command+12) /*long, non-outer control sequence*/
+ at d outer_call (max_command+13) /*non-long, outer control sequence*/
+ at d long_outer_call (max_command+14) /*long, outer control sequence*/
+ at d end_template (max_command+15) /*end of an alignment template*/
+ at d dont_expand (max_command+16) /*the following token was marked by \.{\\noexpand}*/
+ at d glue_ref (max_command+17) /*the equivalent points to a glue specification*/
+ at d shape_ref (max_command+18) /*the equivalent points to a parshape specification*/
+ at d box_ref (max_command+19) /*the equivalent points to a box node, or is |null|*/
+ at d data (max_command+20) /*the equivalent is simply a halfword number*/
+
+@* The semantic nest.
+\TeX\ is typically in the midst of building many lists at once. For example,
+when a math formula is being processed, \TeX\ is in math mode and
+working on an mlist; this formula has temporarily interrupted \TeX\ from
+being in horizontal mode and building the hlist of a paragraph; and this
+paragraph has temporarily interrupted \TeX\ from being in vertical mode
+and building the vlist for the next page of a document. Similarly, when a
+\.{\\vbox} occurs inside of an \.{\\hbox}, \TeX\ is temporarily
+interrupted from working in restricted horizontal mode, and it enters
+internal vertical mode. The ``semantic nest'' is a stack that
+keeps track of what lists and modes are currently suspended.
+
+At each level of processing we are in one of six modes:
+
+\yskip\hang|vmode| stands for vertical mode (the page builder);
+
+\hang|hmode| stands for horizontal mode (the paragraph builder);
+
+\hang|mmode| stands for displayed formula mode;
+
+\hang|-vmode| stands for internal vertical mode (e.g., in a \.{\\vbox});
+
+\hang|-hmode| stands for restricted horizontal mode (e.g., in an \.{\\hbox});
+
+\hang|-mmode| stands for math formula mode (not displayed).
+
+\yskip\noindent The mode is temporarily set to zero while processing \.{\\write}
+texts in the |ship_out| routine.
+
+Numeric values are assigned to |vmode|, |hmode|, and |mmode| so that
+\TeX's ``big semantic switch'' can select the appropriate thing to
+do by computing the value |abs(mode)+cur_cmd|, where |mode| is the current
+mode and |cur_cmd| is the current command code.
+
+ at d vmode 1 /*vertical mode*/
+ at d hmode (vmode+max_command+1) /*horizontal mode*/
+ at d mmode (hmode+max_command+1) /*math mode*/
+
+ at p static void print_mode(int @!m) /*prints the mode represented by |m|*/
+{@+if (m > 0)
+ switch (m/(max_command+1)) {
+ case 0: print("vertical");@+break;
+ case 1: print("horizontal");@+break;
+ case 2: print("display math");
+ }
+else if (m==0) print("no");
+else switch ((-m)/(max_command+1)) {
+ case 0: print("internal vertical");@+break;
+ case 1: print("restricted horizontal");@+break;
+ case 2: print("math");
+ }
+print(" mode");
+}
+
+@ The state of affairs at any semantic level can be represented by
+five values:
+
+\yskip\hang|mode| is the number representing the semantic mode, as
+just explained.
+
+\yskip\hang|head| is a |pointer| to a list head for the list being built;
+|link(head)| therefore points to the first element of the list, or
+to |null| if the list is empty.
+
+\yskip\hang|tail| is a |pointer| to the final node of the list being
+built; thus, |tail==head| if and only if the list is empty.
+
+\yskip\hang|prev_graf| is the number of lines of the current paragraph that
+have already been put into the present vertical list.
+
+\yskip\hang|aux| is an auxiliary |memory_word| that gives further information
+that is needed to characterize the situation.
+
+\yskip\noindent
+In vertical mode, |aux| is also known as |prev_depth|; it is the scaled
+value representing the depth of the previous box, for use in baseline
+calculations, or it is | <= -1000|pt if the next box on the vertical list is to
+be exempt from baseline calculations. In horizontal mode, |aux| is also
+known as |space_factor| and |clang|; it holds the current space factor used in
+spacing calculations, and the current language used for hyphenation.
+(The value of |clang| is undefined in restricted horizontal mode.)
+In math mode, |aux| is also known as |incompleat_noad|; if
+not |null|, it points to a record that represents the numerator of a
+generalized fraction for which the denominator is currently being formed
+in the current list.
+
+There is also a sixth quantity, |mode_line|, which correlates
+the semantic nest with the user's input; |mode_line| contains the source
+line number at which the current level of nesting was entered. The negative
+of this line number is the |mode_line| at the level of the
+user's output routine.
+
+A seventh quantity, |eTeX_aux|, is used by the extended features \eTeX.
+In vertical modes it is known as |LR_save| and holds the LR stack when a
+paragraph is interrupted by a displayed formula. In display math mode
+it is known as |LR_box| and holds a pointer to a prototype box for the
+display. In math mode it is known as |delim_ptr| and points to the most
+recent |left_noad| or |middle_noad| of a |math_left_group|.
+
+In horizontal mode, the |prev_graf| field is used for initial language data.
+
+The semantic nest is an array called |nest| that holds the |mode|, |head|,
+|tail|, |prev_graf|, |aux|, and |mode_line| values for all semantic levels
+below the currently active one. Information about the currently active
+level is kept in the global quantities |mode|, |head|, |tail|, |prev_graf|,
+|aux|, and |mode_line|, which live in a \PASCAL\ record that is ready to
+be pushed onto |nest| if necessary.
+
+ at d ignore_depth (-1000*unity) /*|prev_depth| value that is ignored*/
+ at d unknown_depth (-2000*unity) /*|prev_depth| value that is unknown*/
+
+@<Types...@>=
+typedef struct { int16_t @!mode_field;@+
+ pointer @!head_field, @!tail_field;
+ pointer @!eTeX_aux_field;
+ int @!pg_field, @!ml_field;@+
+ memory_word @!aux_field;
+ } list_state_record;
+
+@ @d mode cur_list.mode_field /*current mode*/
+ at d head cur_list.head_field /*header node of current list*/
+ at d tail cur_list.tail_field /*final node on current list*/
+ at d eTeX_aux cur_list.eTeX_aux_field /*auxiliary data for \eTeX*/
+ at d LR_save eTeX_aux /*LR stack when a paragraph is interrupted*/
+ at d LR_box eTeX_aux /*prototype box for display*/
+ at d delim_ptr eTeX_aux /*most recent left or right noad of a math left group*/
+ at d prev_graf cur_list.pg_field /*number of paragraph lines accumulated*/
+ at d aux cur_list.aux_field /*auxiliary data about the current list*/
+ at d prev_depth aux.sc /*the name of |aux| in vertical mode*/
+ at d space_factor aux.hh.lh /*part of |aux| in horizontal mode*/
+ at d clang aux.hh.rh /*the other part of |aux| in horizontal mode*/
+ at d incompleat_noad aux.i /*the name of |aux| in math mode*/
+ at d mode_line cur_list.ml_field /*source file line number at beginning of list*/
+
+@<Glob...@>=
+list_state_record @!nest[nest_size+1];
+int @!nest_ptr; /*first unused location of |nest|*/
+static int @!max_nest_stack; /*maximum of |nest_ptr| when pushing*/
+list_state_record @!cur_list; /*the ``top'' semantic state*/
+static int @!shown_mode; /*most recent mode shown by \.{\\tracingcommands}*/
+
+@ Here is a common way to make the current list grow:
+
+ at d tail_append(A) {@+link(tail)=A;tail=link(tail);
+ }
+
+@ We will see later that the vertical list at the bottom semantic level is split
+into two parts; the ``current page'' runs from |page_head| to |page_tail|,
+and the ``contribution list'' runs from |contrib_head| to |tail| of
+semantic level zero. The idea is that contributions are first formed in
+vertical mode, then ``contributed'' to the current page (during which time
+the page-breaking decisions are made). For now, we don't need to know
+any more details about the page-building process.
+
+@<Set init...@>=
+nest_ptr=0;max_nest_stack=0;
+mode=vmode;head=contrib_head;tail=contrib_head;
+eTeX_aux=null;
+prev_depth=ignore_depth;mode_line=0;
+prev_graf=0;shown_mode=0;
+@<Start a new current page@>;
+
+@ When \TeX's work on one level is interrupted, the state is saved by
+calling |push_nest|. This routine changes |head| and |tail| so that
+a new (empty) list is begun; it does not change |mode| or |aux|.
+
+ at p void push_nest(void) /*enter a new semantic level, save the old*/
+{@+if (nest_ptr > max_nest_stack)
+ {@+max_nest_stack=nest_ptr;
+ if (nest_ptr==nest_size) overflow("semantic nest size", nest_size);
+@:TeX capacity exceeded semantic nest size}{\quad semantic nest size@>
+ }
+nest[nest_ptr]=cur_list; /*stack the record*/
+incr(nest_ptr);head=get_avail();tail=head;prev_graf=0;mode_line=line;
+eTeX_aux=null;
+}
+
+@ Conversely, when \TeX\ is finished on the current level, the former
+state is restored by calling |pop_nest|. This routine will never be
+called at the lowest semantic level, nor will it be called unless |head|
+is a node that should be returned to free memory.
+
+ at p void pop_nest(void) /*leave a semantic level, re-enter the old*/
+{@+free_avail(head);decr(nest_ptr);cur_list=nest[nest_ptr];
+}
+
+@ Here is a procedure that displays what \TeX\ is working on, at all levels.
+
+ at p static void print_totals(void);
+static void show_activities(void)
+{@+int p; /*index into |nest|*/
+int @!m; /*mode*/
+memory_word @!a; /*auxiliary*/
+pointer @!q, @!r; /*for showing the current page*/
+int @!t; /*ditto*/
+nest[nest_ptr]=cur_list; /*put the top level into the array*/
+print_nl("");print_ln();
+for (p=nest_ptr; p>=0; p--)
+ {@+m=nest[p].mode_field;a=nest[p].aux_field;
+ print_nl("### ");print_mode(m);
+ print(" entered at line ");print_int(abs(nest[p].ml_field));
+ if (m==hmode) if (nest[p].pg_field!=040600000)
+ {@+print(" (language");print_int(nest[p].pg_field%0200000);
+ print(":hyphenmin");print_int(nest[p].pg_field/020000000);
+ print_char(',');print_int((nest[p].pg_field/0200000)%0100);
+ print_char(')');
+ }
+ if (nest[p].ml_field < 0) print(" (\\output routine)");
+ if (p==0)
+ {@+@<Show the status of the current page@>;
+ if (link(contrib_head)!=null)
+ print_nl("### recent contributions:");
+ }
+ show_box(link(nest[p].head_field));
+ @<Show the auxiliary field, |a|@>;
+ }
+}
+
+@ @<Show the auxiliary...@>=
+switch (abs(m)/(max_command+1)) {
+case 0: {@+print_nl("prevdepth ");
+ if (a.sc <= ignore_depth)
+ { if (a.sc <= unknown_depth) print("unknown"); else print("ignored"); }
+ else print_scaled(a.sc);
+ if (nest[p].pg_field!=0)
+ {@+print(", prevgraf ");
+ print_int(nest[p].pg_field);print(" line");
+ if (nest[p].pg_field!=1) print_char('s');
+ }
+ } @+break;
+case 1: {@+print_nl("spacefactor ");print_int(a.hh.lh);
+ if (m > 0) @+if (a.hh.rh > 0)
+ {@+print(", current language ");print_int(a.hh.rh);@+
+ }
+ } @+break;
+case 2: if (a.i!=null)
+ {@+print("this will be denominator of:");show_box(a.i);@+
+ }
+} /*there are no other cases*/
+
+@* The table of equivalents.
+Now that we have studied the data structures for \TeX's semantic routines,
+we ought to consider the data structures used by its syntactic routines. In
+other words, our next concern will be
+the tables that \TeX\ looks at when it is scanning
+what the user has written.
+
+The biggest and most important such table is called |eqtb|. It holds the
+current ``equivalents'' of things; i.e., it explains what things mean
+or what their current values are, for all quantities that are subject to
+the nesting structure provided by \TeX's grouping mechanism. There are six
+parts to |eqtb|:
+
+\yskip\hangg 1) |eqtb[active_base dotdot(hash_base-1)]| holds the current
+equivalents of single-character control sequences.
+
+\yskip\hangg 2) |eqtb[hash_base dotdot(glue_base-1)]| holds the current
+equivalents of multiletter control sequences.
+
+\yskip\hangg 3) |eqtb[glue_base dotdot(local_base-1)]| holds the current
+equivalents of glue parameters like the current baselineskip.
+
+\yskip\hangg 4) |eqtb[local_base dotdot(int_base-1)]| holds the current
+equivalents of local halfword quantities like the current box registers,
+the current ``catcodes,'' the current font, and a pointer to the current
+paragraph shape.
+
+\yskip\hangg 5) |eqtb[int_base dotdot(dimen_base-1)]| holds the current
+equivalents of fullword integer parameters like the current hyphenation
+penalty.
+
+\yskip\hangg 6) |eqtb[dimen_base dotdot eqtb_size]| holds the current equivalents
+of fullword dimension parameters like the current hsize or amount of
+hanging indentation.
+
+\yskip\noindent Note that, for example, the current amount of
+baselineskip glue is determined by the setting of a particular location
+in region~3 of |eqtb|, while the current meaning of the control sequence
+`\.{\\baselineskip}' (which might have been changed by \.{\\def} or
+\.{\\let}) appears in region~2.
+
+@ Each entry in |eqtb| is a |memory_word|. Most of these words are of type
+|two_halves|, and subdivided into three fields:
+
+\yskip\hangg 1) The |eq_level| (a quarterword) is the level of grouping at
+which this equivalent was defined. If the level is |level_zero|, the
+equivalent has never been defined; |level_one| refers to the outer level
+(outside of all groups), and this level is also used for global
+definitions that never go away. Higher levels are for equivalents that
+will disappear at the end of their group. @^global definitions@>
+
+\yskip\hangg 2) The |eq_type| (another quarterword) specifies what kind of
+entry this is. There are many types, since each \TeX\ primitive like
+\.{\\hbox}, \.{\\def}, etc., has its own special code. The list of
+command codes above includes all possible settings of the |eq_type| field.
+
+\yskip\hangg 3) The |equiv| (a halfword) is the current equivalent value.
+This may be a font number, a pointer into |mem|, or a variety of other
+things.
+
+ at d eq_level_field(A) A.hh.b1
+ at d eq_type_field(A) A.hh.b0
+ at d equiv_field(A) A.hh.rh
+ at d eq_level(A) eq_level_field(eqtb[A]) /*level of definition*/
+ at d eq_type(A) eq_type_field(eqtb[A]) /*command code for equivalent*/
+ at d equiv(A) equiv_field(eqtb[A]) /*equivalent value*/
+ at d level_zero min_quarterword /*level for undefined quantities*/
+ at d level_one (level_zero+1) /*outermost level for defined quantities*/
+
+@ Many locations in |eqtb| have symbolic names. The purpose of the next
+paragraphs is to define these names, and to set up the initial values of the
+equivalents.
+
+In the first region we have 256 equivalents for ``active characters'' that
+act as control sequences, followed by 256 equivalents for single-character
+control sequences.
+
+Then comes region~2, which corresponds to the hash table that we will
+define later. The maximum address in this region is used for a dummy
+control sequence that is perpetually undefined. There also are several
+locations for control sequences that are perpetually defined
+(since they are used in error recovery).
+
+ at d active_base 1 /*beginning of region 1, for active character equivalents*/
+ at d single_base (active_base+256) /*equivalents of one-character control sequences*/
+ at d null_cs (single_base+256) /*equivalent of \.{\\csname\\endcsname}*/
+ at d hash_base (null_cs+1) /*beginning of region 2, for the hash table*/
+ at d frozen_control_sequence (hash_base+hash_size) /*for error recovery*/
+ at d frozen_protection frozen_control_sequence /*inaccessible but definable*/
+ at d frozen_cr (frozen_control_sequence+1) /*permanent `\.{\\cr}'*/
+ at d frozen_end_group (frozen_control_sequence+2) /*permanent `\.{\\endgroup}'*/
+ at d frozen_right (frozen_control_sequence+3) /*permanent `\.{\\right}'*/
+ at d frozen_fi (frozen_control_sequence+4) /*permanent `\.{\\fi}'*/
+ at d frozen_end_template (frozen_control_sequence+5) /*permanent `\.{\\endtemplate}'*/
+ at d frozen_endv (frozen_control_sequence+6) /*second permanent `\.{\\endtemplate}'*/
+ at d frozen_relax (frozen_control_sequence+7) /*permanent `\.{\\relax}'*/
+ at d end_write (frozen_control_sequence+8) /*permanent `\.{\\endwrite}'*/
+ at d frozen_dont_expand (frozen_control_sequence+9)
+ /*permanent `\.{\\notexpanded:}'*/
+ at d frozen_null_font (frozen_control_sequence+10)
+ /*permanent `\.{\\nullfont}'*/
+ at d font_id_base (frozen_null_font-font_base)
+ /*begins table of 257 permanent font identifiers*/
+ at d undefined_control_sequence (frozen_null_font+257) /*dummy location*/
+ at d glue_base (undefined_control_sequence+1) /*beginning of region 3*/
+
+@<Initialize table entries...@>=
+eq_type(undefined_control_sequence)=undefined_cs;
+equiv(undefined_control_sequence)=null;
+eq_level(undefined_control_sequence)=level_zero;
+for (k=active_base; k<=undefined_control_sequence-1; k++)
+ eqtb[k]=eqtb[undefined_control_sequence];
+
+@ Here is a routine that displays the current meaning of an |eqtb| entry
+in region 1 or~2. (Similar routines for the other regions will appear
+below.)
+
+@<Show equivalent |n|, in region 1 or 2@>=
+{@+sprint_cs(n);print_char('=');print_cmd_chr(eq_type(n), equiv(n));
+if (eq_type(n) >= call)
+ {@+print_char(':');show_token_list(link(equiv(n)), null, 32);
+ }
+}
+
+@ Region 3 of |eqtb| contains the 256 \.{\\skip} registers, as well as the
+glue parameters defined here. It is important that the ``muskip''
+parameters have larger numbers than the others.
+
+ at d line_skip_code 0 /*interline glue if |baseline_skip| is infeasible*/
+ at d baseline_skip_code 1 /*desired glue between baselines*/
+ at d par_skip_code 2 /*extra glue just above a paragraph*/
+ at d above_display_skip_code 3 /*extra glue just above displayed math*/
+ at d below_display_skip_code 4 /*extra glue just below displayed math*/
+ at d above_display_short_skip_code 5
+ /*glue above displayed math following short lines*/
+ at d below_display_short_skip_code 6
+ /*glue below displayed math following short lines*/
+ at d left_skip_code 7 /*glue at left of justified lines*/
+ at d right_skip_code 8 /*glue at right of justified lines*/
+ at d top_skip_code 9 /*glue at top of main pages*/
+ at d split_top_skip_code 10 /*glue at top of split pages*/
+ at d tab_skip_code 11 /*glue between aligned entries*/
+ at d space_skip_code 12 /*glue between words (if not |zero_glue|)*/
+ at d xspace_skip_code 13 /*glue after sentences (if not |zero_glue|)*/
+ at d par_fill_skip_code 14 /*glue on last line of paragraph*/
+ at d thin_mu_skip_code 15 /*thin space in math formula*/
+ at d med_mu_skip_code 16 /*medium space in math formula*/
+ at d thick_mu_skip_code 17 /*thick space in math formula*/
+ at d glue_pars 18 /*total number of glue parameters*/
+ at d skip_base (glue_base+glue_pars) /*table of 256 ``skip'' registers*/
+ at d mu_skip_base (skip_base+256) /*table of 256 ``muskip'' registers*/
+ at d local_base (mu_skip_base+256) /*beginning of region 4*/
+@#
+ at d skip(A) equiv(skip_base+A) /*|mem| location of glue specification*/
+ at d mu_skip(A) equiv(mu_skip_base+A) /*|mem| location of math glue spec*/
+ at d glue_par(A) equiv(glue_base+A) /*|mem| location of glue specification*/
+ at d line_skip glue_par(line_skip_code)
+ at d baseline_skip glue_par(baseline_skip_code)
+ at d par_skip glue_par(par_skip_code)
+ at d above_display_skip glue_par(above_display_skip_code)
+ at d below_display_skip glue_par(below_display_skip_code)
+ at d above_display_short_skip glue_par(above_display_short_skip_code)
+ at d below_display_short_skip glue_par(below_display_short_skip_code)
+ at d left_skip glue_par(left_skip_code)
+ at d right_skip glue_par(right_skip_code)
+ at d top_skip glue_par(top_skip_code)
+ at d split_top_skip glue_par(split_top_skip_code)
+ at d tab_skip glue_par(tab_skip_code)
+ at d space_skip glue_par(space_skip_code)
+ at d xspace_skip glue_par(xspace_skip_code)
+ at d par_fill_skip glue_par(par_fill_skip_code)
+ at d thin_mu_skip glue_par(thin_mu_skip_code)
+ at d med_mu_skip glue_par(med_mu_skip_code)
+ at d thick_mu_skip glue_par(thick_mu_skip_code)
+
+@<Current |mem| equivalent of glue parameter number |n|@>=glue_par(n)
+
+@ Sometimes we need to convert \TeX's internal code numbers into symbolic
+form. The |print_skip_param| routine gives the symbolic name of a glue
+parameter.
+
+@<Declare the procedure called |print_skip_param|@>=
+static void print_skip_param(int @!n)
+{@+switch (n) {
+case line_skip_code: print_esc("lineskip");@+break;
+case baseline_skip_code: print_esc("baselineskip");@+break;
+case par_skip_code: print_esc("parskip");@+break;
+case above_display_skip_code: print_esc("abovedisplayskip");@+break;
+case below_display_skip_code: print_esc("belowdisplayskip");@+break;
+case above_display_short_skip_code: print_esc("abovedisplayshortskip");@+break;
+case below_display_short_skip_code: print_esc("belowdisplayshortskip");@+break;
+case left_skip_code: print_esc("leftskip");@+break;
+case right_skip_code: print_esc("rightskip");@+break;
+case top_skip_code: print_esc("topskip");@+break;
+case split_top_skip_code: print_esc("splittopskip");@+break;
+case tab_skip_code: print_esc("tabskip");@+break;
+case space_skip_code: print_esc("spaceskip");@+break;
+case xspace_skip_code: print_esc("xspaceskip");@+break;
+case par_fill_skip_code: print_esc("parfillskip");@+break;
+case thin_mu_skip_code: print_esc("thinmuskip");@+break;
+case med_mu_skip_code: print_esc("medmuskip");@+break;
+case thick_mu_skip_code: print_esc("thickmuskip");@+break;
+default:print("[unknown glue parameter!]");
+}
+}
+
+@ The symbolic names for glue parameters are put into \TeX's hash table
+by using the routine called |primitive|, defined below. Let us enter them
+now, so that we don't have to list all those parameter names anywhere else.
+
+@<Put each of \TeX's primitives into the hash table@>=
+primitive("lineskip", assign_glue, glue_base+line_skip_code);@/
+@!@:line\_skip\_}{\.{\\lineskip} primitive@>
+primitive("baselineskip", assign_glue, glue_base+baseline_skip_code);@/
+@!@:baseline\_skip\_}{\.{\\baselineskip} primitive@>
+primitive("parskip", assign_glue, glue_base+par_skip_code);@/
+@!@:par\_skip\_}{\.{\\parskip} primitive@>
+primitive("abovedisplayskip", assign_glue, glue_base+above_display_skip_code);@/
+@!@:above\_display\_skip\_}{\.{\\abovedisplayskip} primitive@>
+primitive("belowdisplayskip", assign_glue, glue_base+below_display_skip_code);@/
+@!@:below\_display\_skip\_}{\.{\\belowdisplayskip} primitive@>
+primitive("abovedisplayshortskip",
+ assign_glue, glue_base+above_display_short_skip_code);@/
+@!@:above\_display\_short\_skip\_}{\.{\\abovedisplayshortskip} primitive@>
+primitive("belowdisplayshortskip",
+ assign_glue, glue_base+below_display_short_skip_code);@/
+@!@:below\_display\_short\_skip\_}{\.{\\belowdisplayshortskip} primitive@>
+primitive("leftskip", assign_glue, glue_base+left_skip_code);@/
+@!@:left\_skip\_}{\.{\\leftskip} primitive@>
+primitive("rightskip", assign_glue, glue_base+right_skip_code);@/
+@!@:right\_skip\_}{\.{\\rightskip} primitive@>
+primitive("topskip", assign_glue, glue_base+top_skip_code);@/
+@!@:top\_skip\_}{\.{\\topskip} primitive@>
+primitive("splittopskip", assign_glue, glue_base+split_top_skip_code);@/
+@!@:split\_top\_skip\_}{\.{\\splittopskip} primitive@>
+primitive("tabskip", assign_glue, glue_base+tab_skip_code);@/
+@!@:tab\_skip\_}{\.{\\tabskip} primitive@>
+primitive("spaceskip", assign_glue, glue_base+space_skip_code);@/
+@!@:space\_skip\_}{\.{\\spaceskip} primitive@>
+primitive("xspaceskip", assign_glue, glue_base+xspace_skip_code);@/
+@!@:xspace\_skip\_}{\.{\\xspaceskip} primitive@>
+primitive("parfillskip", assign_glue, glue_base+par_fill_skip_code);@/
+@!@:par\_fill\_skip\_}{\.{\\parfillskip} primitive@>
+primitive("thinmuskip", assign_mu_glue, glue_base+thin_mu_skip_code);@/
+@!@:thin\_mu\_skip\_}{\.{\\thinmuskip} primitive@>
+primitive("medmuskip", assign_mu_glue, glue_base+med_mu_skip_code);@/
+@!@:med\_mu\_skip\_}{\.{\\medmuskip} primitive@>
+primitive("thickmuskip", assign_mu_glue, glue_base+thick_mu_skip_code);@/
+@!@:thick\_mu\_skip\_}{\.{\\thickmuskip} primitive@>
+
+@ @<Cases of |print_cmd_chr| for symbolic printing of primitives@>=
+case assign_glue: case assign_mu_glue: if (chr_code < skip_base)
+ print_skip_param(chr_code-glue_base);
+ else if (chr_code < mu_skip_base)
+ {@+print_esc("skip");print_int(chr_code-skip_base);
+ }
+ else{@+print_esc("muskip");print_int(chr_code-mu_skip_base);
+ } @+break;
+
+@ All glue parameters and registers are initially `\.{0pt plus0pt minus0pt}'.
+
+@<Initialize table entries...@>=
+equiv(glue_base)=zero_glue;eq_level(glue_base)=level_one;
+eq_type(glue_base)=glue_ref;
+for (k=glue_base+1; k<=local_base-1; k++) eqtb[k]=eqtb[glue_base];
+glue_ref_count(zero_glue)=glue_ref_count(zero_glue)+local_base-glue_base;
+
+@ @<Show equivalent |n|, in region 3@>=
+if (n < skip_base)
+ {@+print_skip_param(n-glue_base);print_char('=');
+ if (n < glue_base+thin_mu_skip_code) print_spec(equiv(n),"pt");
+ else print_spec(equiv(n),"mu");
+ }
+else if (n < mu_skip_base)
+ {@+print_esc("skip");print_int(n-skip_base);print_char('=');
+ print_spec(equiv(n),"pt");
+ }
+else{@+print_esc("muskip");print_int(n-mu_skip_base);print_char('=');
+ print_spec(equiv(n),"mu");
+ }
+
+@ Region 4 of |eqtb| contains the local quantities defined here. The
+bulk of this region is taken up by five tables that are indexed by eight-bit
+characters; these tables are important to both the syntactic and semantic
+portions of \TeX. There are also a bunch of special things like font and
+token parameters, as well as the tables of \.{\\toks} and \.{\\box}
+registers.
+
+ at d par_shape_loc local_base /*specifies paragraph shape*/
+ at d output_routine_loc (local_base+1) /*points to token list for \.{\\output}*/
+ at d every_par_loc (local_base+2) /*points to token list for \.{\\everypar}*/
+ at d every_math_loc (local_base+3) /*points to token list for \.{\\everymath}*/
+ at d every_display_loc (local_base+4) /*points to token list for \.{\\everydisplay}*/
+ at d every_hbox_loc (local_base+5) /*points to token list for \.{\\everyhbox}*/
+ at d every_vbox_loc (local_base+6) /*points to token list for \.{\\everyvbox}*/
+ at d every_job_loc (local_base+7) /*points to token list for \.{\\everyjob}*/
+ at d every_cr_loc (local_base+8) /*points to token list for \.{\\everycr}*/
+ at d err_help_loc (local_base+9) /*points to token list for \.{\\errhelp}*/
+ at d tex_toks (local_base+10) /*end of \TeX's token list parameters*/
+@#
+ at d etex_toks_base tex_toks /*base for \eTeX's token list parameters*/
+ at d every_eof_loc etex_toks_base /*points to token list for \.{\\everyeof}*/
+ at d etex_toks (etex_toks_base+1) /*end of \eTeX's token list parameters*/
+@#
+ at d toks_base etex_toks /*table of 256 token list registers*/
+@#
+ at d etex_pen_base (toks_base+256) /*start of table of \eTeX's penalties*/
+ at d inter_line_penalties_loc etex_pen_base /*additional penalties between lines*/
+ at d club_penalties_loc (etex_pen_base+1) /*penalties for creating club lines*/
+ at d widow_penalties_loc (etex_pen_base+2) /*penalties for creating widow lines*/
+ at d display_widow_penalties_loc (etex_pen_base+3) /*ditto, just before a display*/
+ at d etex_pens (etex_pen_base+4) /*end of table of \eTeX's penalties*/
+@#
+ at d box_base etex_pens /*table of 256 box registers*/
+ at d cur_font_loc (box_base+256) /*internal font number outside math mode*/
+ at d math_font_base (cur_font_loc+1) /*table of 48 math font numbers*/
+ at d cat_code_base (math_font_base+48)
+ /*table of 256 command codes (the ``catcodes'')*/
+ at d lc_code_base (cat_code_base+256) /*table of 256 lowercase mappings*/
+ at d uc_code_base (lc_code_base+256) /*table of 256 uppercase mappings*/
+ at d sf_code_base (uc_code_base+256) /*table of 256 spacefactor mappings*/
+ at d math_code_base (sf_code_base+256) /*table of 256 math mode mappings*/
+ at d int_base (math_code_base+256) /*beginning of region 5*/
+@#
+ at d par_shape_ptr equiv(par_shape_loc)
+ at d output_routine equiv(output_routine_loc)
+ at d every_par equiv(every_par_loc)
+ at d every_math equiv(every_math_loc)
+ at d every_display equiv(every_display_loc)
+ at d every_hbox equiv(every_hbox_loc)
+ at d every_vbox equiv(every_vbox_loc)
+ at d every_job equiv(every_job_loc)
+ at d every_cr equiv(every_cr_loc)
+ at d err_help equiv(err_help_loc)
+ at d toks(X) equiv(toks_base+X)
+ at d box(A) equiv(box_base+A)
+ at d cur_font equiv(cur_font_loc)
+ at d fam_fnt(A) equiv(math_font_base+A)
+ at d cat_code(A) equiv(cat_code_base+A)
+ at d lc_code(A) equiv(lc_code_base+A)
+ at d uc_code(A) equiv(uc_code_base+A)
+ at d sf_code(A) equiv(sf_code_base+A)
+ at d math_code(A) equiv(math_code_base+A)
+ /*Note: |math_code(c)| is the true math code plus |min_halfword|*/
+
+@<Put each...@>=
+primitive("output", assign_toks, output_routine_loc);
+@!@:output\_}{\.{\\output} primitive@>
+primitive("everypar", assign_toks, every_par_loc);
+@!@:every\_par\_}{\.{\\everypar} primitive@>
+primitive("everymath", assign_toks, every_math_loc);
+@!@:every\_math\_}{\.{\\everymath} primitive@>
+primitive("everydisplay", assign_toks, every_display_loc);
+@!@:every\_display\_}{\.{\\everydisplay} primitive@>
+primitive("everyhbox", assign_toks, every_hbox_loc);
+@!@:every\_hbox\_}{\.{\\everyhbox} primitive@>
+primitive("everyvbox", assign_toks, every_vbox_loc);
+@!@:every\_vbox\_}{\.{\\everyvbox} primitive@>
+primitive("everyjob", assign_toks, every_job_loc);
+@!@:every\_job\_}{\.{\\everyjob} primitive@>
+primitive("everycr", assign_toks, every_cr_loc);
+@!@:every\_cr\_}{\.{\\everycr} primitive@>
+primitive("errhelp", assign_toks, err_help_loc);
+@!@:err\_help\_}{\.{\\errhelp} primitive@>
+
+@ @<Cases of |print_cmd_chr|...@>=
+case assign_toks: if (chr_code >= toks_base)
+ {@+print_esc("toks");print_int(chr_code-toks_base);
+ }
+else switch (chr_code) {
+ case output_routine_loc: print_esc("output");@+break;
+ case every_par_loc: print_esc("everypar");@+break;
+ case every_math_loc: print_esc("everymath");@+break;
+ case every_display_loc: print_esc("everydisplay");@+break;
+ case every_hbox_loc: print_esc("everyhbox");@+break;
+ case every_vbox_loc: print_esc("everyvbox");@+break;
+ case every_job_loc: print_esc("everyjob");@+break;
+ case every_cr_loc: print_esc("everycr");@+break;
+ @/@<Cases of |assign_toks| for |print_cmd_chr|@>@/
+ default:print_esc("errhelp");
+ } @+break;
+
+@ We initialize most things to null or undefined values. An undefined font
+is represented by the internal code |font_base|.
+
+However, the character code tables are given initial values based on the
+conventional interpretation of ASCII code. These initial values should
+not be changed when \TeX\ is adapted for use with non-English languages;
+all changes to the initialization conventions should be made in format
+packages, not in \TeX\ itself, so that global interchange of formats is
+possible.
+
+ at d null_font font_base
+ at d var_code 070000 /*math code meaning ``use the current family''*/
+
+@<Initialize table entries...@>=
+par_shape_ptr=null;eq_type(par_shape_loc)=shape_ref;
+eq_level(par_shape_loc)=level_one;@/
+for (k=etex_pen_base; k<=etex_pens-1; k++)
+ eqtb[k]=eqtb[par_shape_loc];
+for (k=output_routine_loc; k<=toks_base+255; k++)
+ eqtb[k]=eqtb[undefined_control_sequence];
+box(0)=null;eq_type(box_base)=box_ref;eq_level(box_base)=level_one;
+for (k=box_base+1; k<=box_base+255; k++) eqtb[k]=eqtb[box_base];
+cur_font=null_font;eq_type(cur_font_loc)=data;
+eq_level(cur_font_loc)=level_one;@/
+for (k=math_font_base; k<=math_font_base+47; k++) eqtb[k]=eqtb[cur_font_loc];
+equiv(cat_code_base)=0;eq_type(cat_code_base)=data;
+eq_level(cat_code_base)=level_one;@/
+for (k=cat_code_base+1; k<=int_base-1; k++) eqtb[k]=eqtb[cat_code_base];
+for (k=0; k<=255; k++)
+ {@+cat_code(k)=other_char;math_code(k)=hi(k);sf_code(k)=1000;
+ }
+cat_code(carriage_return)=car_ret;cat_code(' ')=spacer;
+cat_code('\\')=escape;cat_code('%')=comment;
+cat_code(invalid_code)=invalid_char;cat_code(null_code)=ignore;
+for (k='0'; k<='9'; k++) math_code(k)=hi(k+var_code);
+for (k='A'; k<='Z'; k++)
+ {@+cat_code(k)=letter;cat_code(k+'a'-'A')=letter;@/
+ math_code(k)=hi(k+var_code+0x100);
+ math_code(k+'a'-'A')=hi(k+'a'-'A'+var_code+0x100);@/
+ lc_code(k)=k+'a'-'A';lc_code(k+'a'-'A')=k+'a'-'A';@/
+ uc_code(k)=k;uc_code(k+'a'-'A')=k;@/
+ sf_code(k)=999;
+ }
+
+@ @<Show equivalent |n|, in region 4@>=
+if ((n==par_shape_loc)||((n >= etex_pen_base)&&(n < etex_pens)))
+ {@+print_cmd_chr(set_shape, n);print_char('=');
+ if (equiv(n)==null) print_char('0');
+ else if (n > par_shape_loc)
+ {@+print_int(penalty(equiv(n)));print_char(' ');
+ print_int(penalty(equiv(n)+1));
+ if (penalty(equiv(n)) > 1) print_esc("ETC.");
+ }
+ else print_int(info(par_shape_ptr));
+ }
+else if (n < toks_base)
+ {@+print_cmd_chr(assign_toks, n);print_char('=');
+ if (equiv(n)!=null) show_token_list(link(equiv(n)), null, 32);
+ }
+else if (n < box_base)
+ {@+print_esc("toks");print_int(n-toks_base);print_char('=');
+ if (equiv(n)!=null) show_token_list(link(equiv(n)), null, 32);
+ }
+else if (n < cur_font_loc)
+ {@+print_esc("box");print_int(n-box_base);print_char('=');
+ if (equiv(n)==null) print("void");
+ else{@+depth_threshold=0;breadth_max=1;show_node_list(equiv(n));
+ }
+ }
+else if (n < cat_code_base) @<Show the font identifier in |eqtb[n]|@>@;
+else@<Show the halfword code in |eqtb[n]|@>@;
+
+@ @<Show the font identifier in |eqtb[n]|@>=
+{@+if (n==cur_font_loc) print("current font");
+else if (n < math_font_base+16)
+ {@+print_esc("textfont");print_int(n-math_font_base);
+ }
+else if (n < math_font_base+32)
+ {@+print_esc("scriptfont");print_int(n-math_font_base-16);
+ }
+else{@+print_esc("scriptscriptfont");print_int(n-math_font_base-32);
+ }
+print_char('=');@/
+printn_esc(hash[font_id_base+equiv(n)].rh);
+ /*that's |font_id_text(equiv(n))|*/
+}
+
+@ @<Show the halfword code in |eqtb[n]|@>=
+if (n < math_code_base)
+ {@+if (n < lc_code_base)
+ {@+print_esc("catcode");print_int(n-cat_code_base);
+ }
+ else if (n < uc_code_base)
+ {@+print_esc("lccode");print_int(n-lc_code_base);
+ }
+ else if (n < sf_code_base)
+ {@+print_esc("uccode");print_int(n-uc_code_base);
+ }
+ else{@+print_esc("sfcode");print_int(n-sf_code_base);
+ }
+ print_char('=');print_int(equiv(n));
+ }
+else{@+print_esc("mathcode");print_int(n-math_code_base);
+ print_char('=');print_int(ho(equiv(n)));
+ }
+
+@ Region 5 of |eqtb| contains the integer parameters and registers defined
+here, as well as the |del_code| table. The latter table differs from the
+|cat_code dotdot math_code| tables that precede it, since delimiter codes are
+fullword integers while the other kinds of codes occupy at most a
+halfword. This is what makes region~5 different from region~4. We will
+store the |eq_level| information in an auxiliary array of quarterwords
+that will be defined later.
+
+ at d pretolerance_code 0 /*badness tolerance before hyphenation*/
+ at d tolerance_code 1 /*badness tolerance after hyphenation*/
+ at d line_penalty_code 2 /*added to the badness of every line*/
+ at d hyphen_penalty_code 3 /*penalty for break after discretionary hyphen*/
+ at d ex_hyphen_penalty_code 4 /*penalty for break after explicit hyphen*/
+ at d club_penalty_code 5 /*penalty for creating a club line*/
+ at d widow_penalty_code 6 /*penalty for creating a widow line*/
+ at d display_widow_penalty_code 7 /*ditto, just before a display*/
+ at d broken_penalty_code 8 /*penalty for breaking a page at a broken line*/
+ at d bin_op_penalty_code 9 /*penalty for breaking after a binary operation*/
+ at d rel_penalty_code 10 /*penalty for breaking after a relation*/
+ at d pre_display_penalty_code 11
+ /*penalty for breaking just before a displayed formula*/
+ at d post_display_penalty_code 12
+ /*penalty for breaking just after a displayed formula*/
+ at d inter_line_penalty_code 13 /*additional penalty between lines*/
+ at d double_hyphen_demerits_code 14 /*demerits for double hyphen break*/
+ at d final_hyphen_demerits_code 15 /*demerits for final hyphen break*/
+ at d adj_demerits_code 16 /*demerits for adjacent incompatible lines*/
+ at d mag_code 17 /*magnification ratio*/
+ at d delimiter_factor_code 18 /*ratio for variable-size delimiters*/
+ at d looseness_code 19 /*change in number of lines for a paragraph*/
+ at d time_code 20 /*current time of day*/
+ at d day_code 21 /*current day of the month*/
+ at d month_code 22 /*current month of the year*/
+ at d year_code 23 /*current year of our Lord*/
+ at d show_box_breadth_code 24 /*nodes per level in |show_box|*/
+ at d show_box_depth_code 25 /*maximum level in |show_box|*/
+ at d hbadness_code 26 /*hboxes exceeding this badness will be shown by |hpack|*/
+ at d vbadness_code 27 /*vboxes exceeding this badness will be shown by |vpack|*/
+ at d pausing_code 28 /*pause after each line is read from a file*/
+ at d tracing_online_code 29 /*show diagnostic output on terminal*/
+ at d tracing_macros_code 30 /*show macros as they are being expanded*/
+ at d tracing_stats_code 31 /*show memory usage if \TeX\ knows it*/
+ at d tracing_paragraphs_code 32 /*show line-break calculations*/
+ at d tracing_pages_code 33 /*show page-break calculations*/
+ at d tracing_output_code 34 /*show boxes when they are shipped out*/
+ at d tracing_lost_chars_code 35 /*show characters that aren't in the font*/
+ at d tracing_commands_code 36 /*show command codes at |big_switch|*/
+ at d tracing_restores_code 37 /*show equivalents when they are restored*/
+ at d uc_hyph_code 38 /*hyphenate words beginning with a capital letter*/
+ at d output_penalty_code 39 /*penalty found at current page break*/
+ at d max_dead_cycles_code 40 /*bound on consecutive dead cycles of output*/
+ at d hang_after_code 41 /*hanging indentation changes after this many lines*/
+ at d floating_penalty_code 42 /*penalty for insertions heldover after a split*/
+ at d global_defs_code 43 /*override \.{\\global} specifications*/
+ at d cur_fam_code 44 /*current family*/
+ at d escape_char_code 45 /*escape character for token output*/
+ at d default_hyphen_char_code 46 /*value of \.{\\hyphenchar} when a font is loaded*/
+ at d default_skew_char_code 47 /*value of \.{\\skewchar} when a font is loaded*/
+ at d end_line_char_code 48 /*character placed at the right end of the buffer*/
+ at d new_line_char_code 49 /*character that prints as |print_ln|*/
+ at d language_code 50 /*current hyphenation table*/
+ at d left_hyphen_min_code 51 /*minimum left hyphenation fragment size*/
+ at d right_hyphen_min_code 52 /*minimum right hyphenation fragment size*/
+ at d holding_inserts_code 53 /*do not remove insertion nodes from \.{\\box255}*/
+ at d error_context_lines_code 54 /*maximum intermediate line pairs shown*/
+ at d tex_int_pars 55 /*total number of \TeX's integer parameters*/
+@#
+ at d etex_int_base tex_int_pars /*base for \eTeX's integer parameters*/
+ at d tracing_assigns_code etex_int_base /*show assignments*/
+ at d tracing_groups_code (etex_int_base+1) /*show save/restore groups*/
+ at d tracing_ifs_code (etex_int_base+2) /*show conditionals*/
+ at d tracing_scan_tokens_code (etex_int_base+3) /*show pseudo file open and close*/
+ at d tracing_nesting_code (etex_int_base+4) /*show incomplete groups and ifs within files*/
+ at d eTeX_state_code (etex_int_base+5) /*\eTeX\ state variables*/
+ at d etex_int_pars (eTeX_state_code+eTeX_states) /*total number of \eTeX's integer parameters*/
+@#
+ at d int_pars etex_int_pars /*total number of integer parameters*/
+ at d count_base (int_base+int_pars) /*256 user \.{\\count} registers*/
+ at d del_code_base (count_base+256) /*256 delimiter code mappings*/
+ at d dimen_base (del_code_base+256) /*beginning of region 6*/
+@#
+ at d del_code(A) eqtb[del_code_base+A].i
+ at d count(A) eqtb[count_base+A].i
+ at d int_par(A) eqtb[int_base+A].i /*an integer parameter*/
+ at d pretolerance int_par(pretolerance_code)
+ at d tolerance int_par(tolerance_code)
+ at d line_penalty int_par(line_penalty_code)
+ at d hyphen_penalty int_par(hyphen_penalty_code)
+ at d ex_hyphen_penalty int_par(ex_hyphen_penalty_code)
+ at d club_penalty int_par(club_penalty_code)
+ at d widow_penalty int_par(widow_penalty_code)
+ at d display_widow_penalty int_par(display_widow_penalty_code)
+ at d broken_penalty int_par(broken_penalty_code)
+ at d bin_op_penalty int_par(bin_op_penalty_code)
+ at d rel_penalty int_par(rel_penalty_code)
+ at d pre_display_penalty int_par(pre_display_penalty_code)
+ at d post_display_penalty int_par(post_display_penalty_code)
+ at d inter_line_penalty int_par(inter_line_penalty_code)
+ at d double_hyphen_demerits int_par(double_hyphen_demerits_code)
+ at d final_hyphen_demerits int_par(final_hyphen_demerits_code)
+ at d adj_demerits int_par(adj_demerits_code)
+ at d mag int_par(mag_code)
+ at d delimiter_factor int_par(delimiter_factor_code)
+ at d looseness int_par(looseness_code)
+ at d time int_par(time_code)
+ at d day int_par(day_code)
+ at d month int_par(month_code)
+ at d year int_par(year_code)
+ at d show_box_breadth int_par(show_box_breadth_code)
+ at d show_box_depth int_par(show_box_depth_code)
+ at d hbadness int_par(hbadness_code)
+ at d vbadness int_par(vbadness_code)
+ at d pausing int_par(pausing_code)
+ at d tracing_online int_par(tracing_online_code)
+ at d tracing_macros int_par(tracing_macros_code)
+ at d tracing_stats int_par(tracing_stats_code)
+ at d tracing_paragraphs int_par(tracing_paragraphs_code)
+ at d tracing_pages int_par(tracing_pages_code)
+ at d tracing_output int_par(tracing_output_code)
+ at d tracing_lost_chars int_par(tracing_lost_chars_code)
+ at d tracing_commands int_par(tracing_commands_code)
+ at d tracing_restores int_par(tracing_restores_code)
+ at d uc_hyph int_par(uc_hyph_code)
+ at d output_penalty int_par(output_penalty_code)
+ at d max_dead_cycles int_par(max_dead_cycles_code)
+ at d hang_after int_par(hang_after_code)
+ at d floating_penalty int_par(floating_penalty_code)
+ at d global_defs int_par(global_defs_code)
+ at d cur_fam int_par(cur_fam_code)
+ at d escape_char int_par(escape_char_code)
+ at d default_hyphen_char int_par(default_hyphen_char_code)
+ at d default_skew_char int_par(default_skew_char_code)
+ at d end_line_char int_par(end_line_char_code)
+ at d new_line_char int_par(new_line_char_code)
+ at d language int_par(language_code)
+ at d left_hyphen_min int_par(left_hyphen_min_code)
+ at d right_hyphen_min int_par(right_hyphen_min_code)
+ at d holding_inserts int_par(holding_inserts_code)
+ at d error_context_lines int_par(error_context_lines_code)
+@#
+ at d tracing_assigns int_par(tracing_assigns_code)
+ at d tracing_groups int_par(tracing_groups_code)
+ at d tracing_ifs int_par(tracing_ifs_code)
+ at d tracing_scan_tokens int_par(tracing_scan_tokens_code)
+ at d tracing_nesting int_par(tracing_nesting_code)
+
+@<Assign the values |depth_threshold:=show_box_depth|...@>=
+depth_threshold=show_box_depth;
+breadth_max=show_box_breadth
+
+@ We can print the symbolic name of an integer parameter as follows.
+
+ at p static void print_param(int @!n)
+{@+switch (n) {
+case pretolerance_code: print_esc("pretolerance");@+break;
+case tolerance_code: print_esc("tolerance");@+break;
+case line_penalty_code: print_esc("linepenalty");@+break;
+case hyphen_penalty_code: print_esc("hyphenpenalty");@+break;
+case ex_hyphen_penalty_code: print_esc("exhyphenpenalty");@+break;
+case club_penalty_code: print_esc("clubpenalty");@+break;
+case widow_penalty_code: print_esc("widowpenalty");@+break;
+case display_widow_penalty_code: print_esc("displaywidowpenalty");@+break;
+case broken_penalty_code: print_esc("brokenpenalty");@+break;
+case bin_op_penalty_code: print_esc("binoppenalty");@+break;
+case rel_penalty_code: print_esc("relpenalty");@+break;
+case pre_display_penalty_code: print_esc("predisplaypenalty");@+break;
+case post_display_penalty_code: print_esc("postdisplaypenalty");@+break;
+case inter_line_penalty_code: print_esc("interlinepenalty");@+break;
+case double_hyphen_demerits_code: print_esc("doublehyphendemerits");@+break;
+case final_hyphen_demerits_code: print_esc("finalhyphendemerits");@+break;
+case adj_demerits_code: print_esc("adjdemerits");@+break;
+case mag_code: print_esc("mag");@+break;
+case delimiter_factor_code: print_esc("delimiterfactor");@+break;
+case looseness_code: print_esc("looseness");@+break;
+case time_code: print_esc("time");@+break;
+case day_code: print_esc("day");@+break;
+case month_code: print_esc("month");@+break;
+case year_code: print_esc("year");@+break;
+case show_box_breadth_code: print_esc("showboxbreadth");@+break;
+case show_box_depth_code: print_esc("showboxdepth");@+break;
+case hbadness_code: print_esc("hbadness");@+break;
+case vbadness_code: print_esc("vbadness");@+break;
+case pausing_code: print_esc("pausing");@+break;
+case tracing_online_code: print_esc("tracingonline");@+break;
+case tracing_macros_code: print_esc("tracingmacros");@+break;
+case tracing_stats_code: print_esc("tracingstats");@+break;
+case tracing_paragraphs_code: print_esc("tracingparagraphs");@+break;
+case tracing_pages_code: print_esc("tracingpages");@+break;
+case tracing_output_code: print_esc("tracingoutput");@+break;
+case tracing_lost_chars_code: print_esc("tracinglostchars");@+break;
+case tracing_commands_code: print_esc("tracingcommands");@+break;
+case tracing_restores_code: print_esc("tracingrestores");@+break;
+case uc_hyph_code: print_esc("uchyph");@+break;
+case output_penalty_code: print_esc("outputpenalty");@+break;
+case max_dead_cycles_code: print_esc("maxdeadcycles");@+break;
+case hang_after_code: print_esc("hangafter");@+break;
+case floating_penalty_code: print_esc("floatingpenalty");@+break;
+case global_defs_code: print_esc("globaldefs");@+break;
+case cur_fam_code: print_esc("fam");@+break;
+case escape_char_code: print_esc("escapechar");@+break;
+case default_hyphen_char_code: print_esc("defaulthyphenchar");@+break;
+case default_skew_char_code: print_esc("defaultskewchar");@+break;
+case end_line_char_code: print_esc("endlinechar");@+break;
+case new_line_char_code: print_esc("newlinechar");@+break;
+case language_code: print_esc("language");@+break;
+case left_hyphen_min_code: print_esc("lefthyphenmin");@+break;
+case right_hyphen_min_code: print_esc("righthyphenmin");@+break;
+case holding_inserts_code: print_esc("holdinginserts");@+break;
+case error_context_lines_code: print_esc("errorcontextlines");@+break;
+@/@<Cases for |print_param|@>@/
+default:print("[unknown integer parameter!]");
+}
+}
+
+@ The integer parameter names must be entered into the hash table.
+
+@<Put each...@>=
+primitive("pretolerance", assign_int, int_base+pretolerance_code);@/
+@!@:pretolerance\_}{\.{\\pretolerance} primitive@>
+primitive("tolerance", assign_int, int_base+tolerance_code);@/
+@!@:tolerance\_}{\.{\\tolerance} primitive@>
+primitive("linepenalty", assign_int, int_base+line_penalty_code);@/
+@!@:line\_penalty\_}{\.{\\linepenalty} primitive@>
+primitive("hyphenpenalty", assign_int, int_base+hyphen_penalty_code);@/
+@!@:hyphen\_penalty\_}{\.{\\hyphenpenalty} primitive@>
+primitive("exhyphenpenalty", assign_int, int_base+ex_hyphen_penalty_code);@/
+@!@:ex\_hyphen\_penalty\_}{\.{\\exhyphenpenalty} primitive@>
+primitive("clubpenalty", assign_int, int_base+club_penalty_code);@/
+@!@:club\_penalty\_}{\.{\\clubpenalty} primitive@>
+primitive("widowpenalty", assign_int, int_base+widow_penalty_code);@/
+@!@:widow\_penalty\_}{\.{\\widowpenalty} primitive@>
+primitive("displaywidowpenalty",
+ assign_int, int_base+display_widow_penalty_code);@/
+@!@:display\_widow\_penalty\_}{\.{\\displaywidowpenalty} primitive@>
+primitive("brokenpenalty", assign_int, int_base+broken_penalty_code);@/
+@!@:broken\_penalty\_}{\.{\\brokenpenalty} primitive@>
+primitive("binoppenalty", assign_int, int_base+bin_op_penalty_code);@/
+@!@:bin\_op\_penalty\_}{\.{\\binoppenalty} primitive@>
+primitive("relpenalty", assign_int, int_base+rel_penalty_code);@/
+@!@:rel\_penalty\_}{\.{\\relpenalty} primitive@>
+primitive("predisplaypenalty", assign_int, int_base+pre_display_penalty_code);@/
+@!@:pre\_display\_penalty\_}{\.{\\predisplaypenalty} primitive@>
+primitive("postdisplaypenalty", assign_int, int_base+post_display_penalty_code);@/
+@!@:post\_display\_penalty\_}{\.{\\postdisplaypenalty} primitive@>
+primitive("interlinepenalty", assign_int, int_base+inter_line_penalty_code);@/
+@!@:inter\_line\_penalty\_}{\.{\\interlinepenalty} primitive@>
+primitive("doublehyphendemerits",
+ assign_int, int_base+double_hyphen_demerits_code);@/
+@!@:double\_hyphen\_demerits\_}{\.{\\doublehyphendemerits} primitive@>
+primitive("finalhyphendemerits",
+ assign_int, int_base+final_hyphen_demerits_code);@/
+@!@:final\_hyphen\_demerits\_}{\.{\\finalhyphendemerits} primitive@>
+primitive("adjdemerits", assign_int, int_base+adj_demerits_code);@/
+@!@:adj\_demerits\_}{\.{\\adjdemerits} primitive@>
+primitive("mag", assign_int, int_base+mag_code);@/
+@!@:mag\_}{\.{\\mag} primitive@>
+primitive("delimiterfactor", assign_int, int_base+delimiter_factor_code);@/
+@!@:delimiter\_factor\_}{\.{\\delimiterfactor} primitive@>
+primitive("looseness", assign_int, int_base+looseness_code);@/
+@!@:looseness\_}{\.{\\looseness} primitive@>
+primitive("time", assign_int, int_base+time_code);@/
+@!@:time\_}{\.{\\time} primitive@>
+primitive("day", assign_int, int_base+day_code);@/
+@!@:day\_}{\.{\\day} primitive@>
+primitive("month", assign_int, int_base+month_code);@/
+@!@:month\_}{\.{\\month} primitive@>
+primitive("year", assign_int, int_base+year_code);@/
+@!@:year\_}{\.{\\year} primitive@>
+primitive("showboxbreadth", assign_int, int_base+show_box_breadth_code);@/
+@!@:show\_box\_breadth\_}{\.{\\showboxbreadth} primitive@>
+primitive("showboxdepth", assign_int, int_base+show_box_depth_code);@/
+@!@:show\_box\_depth\_}{\.{\\showboxdepth} primitive@>
+primitive("hbadness", assign_int, int_base+hbadness_code);@/
+@!@:hbadness\_}{\.{\\hbadness} primitive@>
+primitive("vbadness", assign_int, int_base+vbadness_code);@/
+@!@:vbadness\_}{\.{\\vbadness} primitive@>
+primitive("pausing", assign_int, int_base+pausing_code);@/
+@!@:pausing\_}{\.{\\pausing} primitive@>
+primitive("tracingonline", assign_int, int_base+tracing_online_code);@/
+@!@:tracing\_online\_}{\.{\\tracingonline} primitive@>
+primitive("tracingmacros", assign_int, int_base+tracing_macros_code);@/
+@!@:tracing\_macros\_}{\.{\\tracingmacros} primitive@>
+primitive("tracingstats", assign_int, int_base+tracing_stats_code);@/
+@!@:tracing\_stats\_}{\.{\\tracingstats} primitive@>
+primitive("tracingparagraphs", assign_int, int_base+tracing_paragraphs_code);@/
+@!@:tracing\_paragraphs\_}{\.{\\tracingparagraphs} primitive@>
+primitive("tracingpages", assign_int, int_base+tracing_pages_code);@/
+@!@:tracing\_pages\_}{\.{\\tracingpages} primitive@>
+primitive("tracingoutput", assign_int, int_base+tracing_output_code);@/
+@!@:tracing\_output\_}{\.{\\tracingoutput} primitive@>
+primitive("tracinglostchars", assign_int, int_base+tracing_lost_chars_code);@/
+@!@:tracing\_lost\_chars\_}{\.{\\tracinglostchars} primitive@>
+primitive("tracingcommands", assign_int, int_base+tracing_commands_code);@/
+@!@:tracing\_commands\_}{\.{\\tracingcommands} primitive@>
+primitive("tracingrestores", assign_int, int_base+tracing_restores_code);@/
+@!@:tracing\_restores\_}{\.{\\tracingrestores} primitive@>
+primitive("uchyph", assign_int, int_base+uc_hyph_code);@/
+@!@:uc\_hyph\_}{\.{\\uchyph} primitive@>
+primitive("outputpenalty", assign_int, int_base+output_penalty_code);@/
+@!@:output\_penalty\_}{\.{\\outputpenalty} primitive@>
+primitive("maxdeadcycles", assign_int, int_base+max_dead_cycles_code);@/
+@!@:max\_dead\_cycles\_}{\.{\\maxdeadcycles} primitive@>
+primitive("hangafter", assign_int, int_base+hang_after_code);@/
+@!@:hang\_after\_}{\.{\\hangafter} primitive@>
+primitive("floatingpenalty", assign_int, int_base+floating_penalty_code);@/
+@!@:floating\_penalty\_}{\.{\\floatingpenalty} primitive@>
+primitive("globaldefs", assign_int, int_base+global_defs_code);@/
+@!@:global\_defs\_}{\.{\\globaldefs} primitive@>
+primitive("fam", assign_int, int_base+cur_fam_code);@/
+@!@:fam\_}{\.{\\fam} primitive@>
+primitive("escapechar", assign_int, int_base+escape_char_code);@/
+@!@:escape\_char\_}{\.{\\escapechar} primitive@>
+primitive("defaulthyphenchar", assign_int, int_base+default_hyphen_char_code);@/
+@!@:default\_hyphen\_char\_}{\.{\\defaulthyphenchar} primitive@>
+primitive("defaultskewchar", assign_int, int_base+default_skew_char_code);@/
+@!@:default\_skew\_char\_}{\.{\\defaultskewchar} primitive@>
+primitive("endlinechar", assign_int, int_base+end_line_char_code);@/
+@!@:end\_line\_char\_}{\.{\\endlinechar} primitive@>
+primitive("newlinechar", assign_int, int_base+new_line_char_code);@/
+@!@:new\_line\_char\_}{\.{\\newlinechar} primitive@>
+primitive("language", assign_int, int_base+language_code);@/
+@!@:language\_}{\.{\\language} primitive@>
+primitive("lefthyphenmin", assign_int, int_base+left_hyphen_min_code);@/
+@!@:left\_hyphen\_min\_}{\.{\\lefthyphenmin} primitive@>
+primitive("righthyphenmin", assign_int, int_base+right_hyphen_min_code);@/
+@!@:right\_hyphen\_min\_}{\.{\\righthyphenmin} primitive@>
+primitive("holdinginserts", assign_int, int_base+holding_inserts_code);@/
+@!@:holding\_inserts\_}{\.{\\holdinginserts} primitive@>
+primitive("errorcontextlines", assign_int, int_base+error_context_lines_code);@/
+@!@:error\_context\_lines\_}{\.{\\errorcontextlines} primitive@>
+
+@ @<Cases of |print_cmd_chr|...@>=
+case assign_int: if (chr_code < count_base) print_param(chr_code-int_base);
+ else{@+print_esc("count");print_int(chr_code-count_base);
+ } @+break;
+
+@ The integer parameters should really be initialized by a macro package;
+the following initialization does the minimum to keep \TeX\ from
+complete failure.
+@^null delimiter@>
+
+@<Initialize table entries...@>=
+for (k=int_base; k<=del_code_base-1; k++) eqtb[k].i=0;
+mag=1000;tolerance=10000;hang_after=1;max_dead_cycles=25;
+escape_char='\\';end_line_char=carriage_return;
+for (k=0; k<=255; k++) del_code(k)=-1;
+del_code('.')=0; /*this null delimiter is used in error recovery*/
+
+@ The following procedure, which is called just before \TeX\ initializes its
+input and output, establishes the initial values of the date and time.
+@^system dependencies@>
+k\TeX\ calls |tl_now| to obtain the current time as a |tm| structure.
+ at p static void fix_date_and_time(void)
+{@+struct tm *gmt=tl_now();
+ time= gmt->tm_hour*60+gmt->tm_min;
+ day= gmt->tm_mday;
+ month=gmt->tm_mon+1;
+ year=gmt->tm_year+1900;
+}
+
+@ @<Show equivalent |n|, in region 5@>=
+{@+if (n < count_base) print_param(n-int_base);
+else if (n < del_code_base)
+ {@+print_esc("count");print_int(n-count_base);
+ }
+else{@+print_esc("delcode");print_int(n-del_code_base);
+ }
+print_char('=');print_int(eqtb[n].i);
+}
+
+@ @<Set variable |c| to the current escape character@>=c=escape_char
+
+@ @<Character |s| is the current new-line character@>=s==new_line_char
+
+@ \TeX\ is occasionally supposed to print diagnostic information that
+goes only into the transcript file, unless |tracing_online| is positive.
+Here are two routines that adjust the destination of print commands:
+
+ at p void begin_diagnostic(void) /*prepare to do some tracing*/
+{@+old_setting=selector;
+if ((tracing_online <= 0)&&(selector==term_and_log))
+ {@+decr(selector);
+ if (history==spotless) history=warning_issued;
+ }
+}
+@#
+void end_diagnostic(bool @!blank_line)
+ /*restore proper conditions after tracing*/
+{@+print_nl("");
+if (blank_line) print_ln();
+selector=old_setting;
+}
+
+@ Of course we had better declare another global variable, if the previous
+routines are going to work.
+
+@<Glob...@>=
+static int @!old_setting;
+
+@ The final region of |eqtb| contains the dimension parameters defined
+here, and the 256 \.{\\dimen} registers.
+
+ at d par_indent_code 0 /*indentation of paragraphs*/
+ at d math_surround_code 1 /*space around math in text*/
+ at d line_skip_limit_code 2 /*threshold for |line_skip| instead of |baseline_skip|*/
+ at d hsize_code 3 /*line width in horizontal mode*/
+ at d vsize_code 4 /*page height in vertical mode*/
+ at d max_depth_code 5 /*maximum depth of boxes on main pages*/
+ at d split_max_depth_code 6 /*maximum depth of boxes on split pages*/
+ at d box_max_depth_code 7 /*maximum depth of explicit vboxes*/
+ at d hfuzz_code 8 /*tolerance for overfull hbox messages*/
+ at d vfuzz_code 9 /*tolerance for overfull vbox messages*/
+ at d delimiter_shortfall_code 10 /*maximum amount uncovered by variable delimiters*/
+ at d null_delimiter_space_code 11 /*blank space in null delimiters*/
+ at d script_space_code 12 /*extra space after subscript or superscript*/
+ at d pre_display_size_code 13 /*length of text preceding a display*/
+ at d display_width_code 14 /*length of line for displayed equation*/
+ at d display_indent_code 15 /*indentation of line for displayed equation*/
+ at d overfull_rule_code 16 /*width of rule that identifies overfull hboxes*/
+ at d hang_indent_code 17 /*amount of hanging indentation*/
+ at d h_offset_code 18 /*amount of horizontal offset when shipping pages out*/
+ at d v_offset_code 19 /*amount of vertical offset when shipping pages out*/
+ at d emergency_stretch_code 20 /*reduces badnesses on final pass of line-breaking*/
+ at d dimen_pars 21 /*total number of dimension parameters*/
+ at d scaled_base (dimen_base+dimen_pars)
+ /*table of 256 user-defined \.{\\dimen} registers*/
+ at d eqtb_size (scaled_base+255) /*largest subscript of |eqtb|*/
+@#
+ at d dimen(A) eqtb[scaled_base+A].sc
+ at d dimen_par(A) eqtb[dimen_base+A].sc /*a scaled quantity*/
+ at d dimen_hfactor(A) hfactor_eqtb[scaled_base+A].sc
+ at d dimen_vfactor(A) vfactor_eqtb[scaled_base+A].sc
+ at d dimen_par_hfactor(A) hfactor_eqtb[dimen_base+A].sc
+ at d dimen_par_vfactor(A) vfactor_eqtb[dimen_base+A].sc
+ at d par_indent dimen_par(par_indent_code)
+ at d math_surround dimen_par(math_surround_code)
+ at d line_skip_limit dimen_par(line_skip_limit_code)
+ at d hsize dimen_par(hsize_code)
+ at d vsize dimen_par(vsize_code)
+ at d max_depth dimen_par(max_depth_code)
+ at d split_max_depth dimen_par(split_max_depth_code)
+ at d box_max_depth dimen_par(box_max_depth_code)
+ at d hfuzz dimen_par(hfuzz_code)
+ at d vfuzz dimen_par(vfuzz_code)
+ at d delimiter_shortfall dimen_par(delimiter_shortfall_code)
+ at d null_delimiter_space dimen_par(null_delimiter_space_code)
+ at d script_space dimen_par(script_space_code)
+ at d pre_display_size dimen_par(pre_display_size_code)
+ at d display_width dimen_par(display_width_code)
+ at d display_indent dimen_par(display_indent_code)
+ at d overfull_rule dimen_par(overfull_rule_code)
+ at d hang_indent dimen_par(hang_indent_code)
+ at d h_offset dimen_par(h_offset_code)
+ at d v_offset dimen_par(v_offset_code)
+ at d emergency_stretch dimen_par(emergency_stretch_code)
+
+ at p static void print_length_param(int @!n)
+{@+switch (n) {
+case par_indent_code: print_esc("parindent");@+break;
+case math_surround_code: print_esc("mathsurround");@+break;
+case line_skip_limit_code: print_esc("lineskiplimit");@+break;
+case hsize_code: print_esc("hsize");@+break;
+case vsize_code: print_esc("vsize");@+break;
+case max_depth_code: print_esc("maxdepth");@+break;
+case split_max_depth_code: print_esc("splitmaxdepth");@+break;
+case box_max_depth_code: print_esc("boxmaxdepth");@+break;
+case hfuzz_code: print_esc("hfuzz");@+break;
+case vfuzz_code: print_esc("vfuzz");@+break;
+case delimiter_shortfall_code: print_esc("delimitershortfall");@+break;
+case null_delimiter_space_code: print_esc("nulldelimiterspace");@+break;
+case script_space_code: print_esc("scriptspace");@+break;
+case pre_display_size_code: print_esc("predisplaysize");@+break;
+case display_width_code: print_esc("displaywidth");@+break;
+case display_indent_code: print_esc("displayindent");@+break;
+case overfull_rule_code: print_esc("overfullrule");@+break;
+case hang_indent_code: print_esc("hangindent");@+break;
+case h_offset_code: print_esc("hoffset");@+break;
+case v_offset_code: print_esc("voffset");@+break;
+case emergency_stretch_code: print_esc("emergencystretch");@+break;
+default:print("[unknown dimen parameter!]");
+}
+}
+
+@ @<Put each...@>=
+primitive("parindent", assign_dimen, dimen_base+par_indent_code);@/
+@!@:par\_indent\_}{\.{\\parindent} primitive@>
+primitive("mathsurround", assign_dimen, dimen_base+math_surround_code);@/
+@!@:math\_surround\_}{\.{\\mathsurround} primitive@>
+primitive("lineskiplimit", assign_dimen, dimen_base+line_skip_limit_code);@/
+@!@:line\_skip\_limit\_}{\.{\\lineskiplimit} primitive@>
+primitive("hsize", assign_dimen, dimen_base+hsize_code);@/
+@!@:hsize\_}{\.{\\hsize} primitive@>
+primitive("vsize", assign_dimen, dimen_base+vsize_code);@/
+@!@:vsize\_}{\.{\\vsize} primitive@>
+primitive("maxdepth", assign_dimen, dimen_base+max_depth_code);@/
+@!@:max\_depth\_}{\.{\\maxdepth} primitive@>
+primitive("splitmaxdepth", assign_dimen, dimen_base+split_max_depth_code);@/
+@!@:split\_max\_depth\_}{\.{\\splitmaxdepth} primitive@>
+primitive("boxmaxdepth", assign_dimen, dimen_base+box_max_depth_code);@/
+@!@:box\_max\_depth\_}{\.{\\boxmaxdepth} primitive@>
+primitive("hfuzz", assign_dimen, dimen_base+hfuzz_code);@/
+@!@:hfuzz\_}{\.{\\hfuzz} primitive@>
+primitive("vfuzz", assign_dimen, dimen_base+vfuzz_code);@/
+@!@:vfuzz\_}{\.{\\vfuzz} primitive@>
+primitive("delimitershortfall",
+ assign_dimen, dimen_base+delimiter_shortfall_code);@/
+@!@:delimiter\_shortfall\_}{\.{\\delimitershortfall} primitive@>
+primitive("nulldelimiterspace",
+ assign_dimen, dimen_base+null_delimiter_space_code);@/
+@!@:null\_delimiter\_space\_}{\.{\\nulldelimiterspace} primitive@>
+primitive("scriptspace", assign_dimen, dimen_base+script_space_code);@/
+@!@:script\_space\_}{\.{\\scriptspace} primitive@>
+primitive("predisplaysize", assign_dimen, dimen_base+pre_display_size_code);@/
+@!@:pre\_display\_size\_}{\.{\\predisplaysize} primitive@>
+primitive("displaywidth", assign_dimen, dimen_base+display_width_code);@/
+@!@:display\_width\_}{\.{\\displaywidth} primitive@>
+primitive("displayindent", assign_dimen, dimen_base+display_indent_code);@/
+@!@:display\_indent\_}{\.{\\displayindent} primitive@>
+primitive("overfullrule", assign_dimen, dimen_base+overfull_rule_code);@/
+@!@:overfull\_rule\_}{\.{\\overfullrule} primitive@>
+primitive("hangindent", assign_dimen, dimen_base+hang_indent_code);@/
+@!@:hang\_indent\_}{\.{\\hangindent} primitive@>
+primitive("hoffset", assign_dimen, dimen_base+h_offset_code);@/
+@!@:h\_offset\_}{\.{\\hoffset} primitive@>
+primitive("voffset", assign_dimen, dimen_base+v_offset_code);@/
+@!@:v\_offset\_}{\.{\\voffset} primitive@>
+primitive("emergencystretch", assign_dimen, dimen_base+emergency_stretch_code);@/
+@!@:emergency\_stretch\_}{\.{\\emergencystretch} primitive@>
+
+@ @<Cases of |print_cmd_chr|...@>=
+case assign_dimen: if (chr_code < scaled_base)
+ print_length_param(chr_code-dimen_base);
+ else{@+print_esc("dimen");print_int(chr_code-scaled_base);
+ } @+break;
+
+@ @<Initialize table entries...@>=
+for (k=dimen_base; k<=eqtb_size; k++) hfactor_eqtb[k].sc=vfactor_eqtb[k].sc=eqtb[k].sc=0;
+
+@ @<Show equivalent |n|, in region 6@>=
+{@+if (n < scaled_base) print_length_param(n-dimen_base);
+else{@+print_esc("dimen");print_int(n-scaled_base);
+ }
+print_char('=');print_scaled(eqtb[n].sc);print("pt");
+}
+
+@ Here is a procedure that displays the contents of |eqtb[n]|
+symbolically.
+
+ at p @t\4@>@<Declare the procedure called |print_cmd_chr|@>@;@/
+#ifdef @!STAT
+static void show_eqtb(pointer @!n)
+{@+if (n < active_base) print_char('?'); /*this can't happen*/
+else if (n < glue_base) @<Show equivalent |n|, in region 1 or 2@>@;
+else if (n < local_base) @<Show equivalent |n|, in region 3@>@;
+else if (n < int_base) @<Show equivalent |n|, in region 4@>@;
+else if (n < dimen_base) @<Show equivalent |n|, in region 5@>@;
+else if (n <= eqtb_size) @<Show equivalent |n|, in region 6@>@;
+else print_char('?'); /*this can't happen either*/
+}
+#endif
+
+@ The last two regions of |eqtb| have fullword values instead of the
+three fields |eq_level|, |eq_type|, and |equiv|. An |eq_type| is unnecessary,
+but \TeX\ needs to store the |eq_level| information in another array
+called |xeq_level|.
+
+@<Glob...@>=
+memory_word @!eqtb0[eqtb_size-active_base+1], *const @!eqtb = @!eqtb0-active_base;
+memory_word hfactor_eqtb0[dimen_pars+256]={{{0}}}, *const @!hfactor_eqtb = @!hfactor_eqtb0-dimen_base;
+memory_word vfactor_eqtb0[dimen_pars+256]={{{0}}}, *const @!vfactor_eqtb = @!vfactor_eqtb0-dimen_base;
+scaled par_shape_hfactor=0, par_shape_vfactor=0;
+scaled hhsize=0,hvsize=0;
+static quarterword @!xeq_level0[eqtb_size-int_base+1], *const @!xeq_level = @!xeq_level0-int_base;
+
+@ @<Set init...@>=
+for (k=int_base; k<=eqtb_size; k++) xeq_level[k]=level_one;
+
+@ When the debugging routine |search_mem| is looking for pointers having a
+given value, it is interested only in regions 1 to~3 of~|eqtb|, and in the
+first part of region~4.
+
+@<Search |eqtb| for equivalents equal to |p|@>=
+for (q=active_base; q<=box_base+255; q++)
+ {@+if (equiv(q)==p)
+ {@+print_nl("EQUIV(");print_int(q);print_char(')');
+ }
+ }
+
+@* The hash table.
+Control sequences are stored and retrieved by means of a fairly standard hash
+table algorithm called the method of ``coalescing lists'' (cf.\ Algorithm 6.4C
+in {\sl The Art of Computer Programming\/}). Once a control sequence enters the
+table, it is never removed, because there are complicated situations
+involving \.{\\gdef} where the removal of a control sequence at the end of
+a group would be a mistake preventable only by the introduction of a
+complicated reference-count mechanism.
+
+The actual sequence of letters forming a control sequence identifier is
+stored in the |str_pool| array together with all the other strings. An
+auxiliary array |hash| consists of items with two halfword fields per
+word. The first of these, called |next(p)|, points to the next identifier
+belonging to the same coalesced list as the identifier corresponding to~|p|;
+and the other, called |text(p)|, points to the |str_start| entry for
+|p|'s identifier. If position~|p| of the hash table is empty, we have
+|text(p)==0|; if position |p| is either empty or the end of a coalesced
+hash list, we have |next(p)==0|. An auxiliary pointer variable called
+|hash_used| is maintained in such a way that all locations |p >= hash_used|
+are nonempty. The global variable |cs_count| tells how many multiletter
+control sequences have been defined, if statistics are being kept.
+
+A global boolean variable called |no_new_control_sequence| is set to
+|true| during the time that new hash table entries are forbidden.
+
+ at d next(A) hash[A].lh /*link for coalesced lists*/
+ at d text(A) hash[A].rh /*string number for control sequence name*/
+ at d hash_is_full (hash_used==hash_base) /*test if all positions are occupied*/
+ at d font_id_text(A) text(font_id_base+A) /*a frozen font identifier's name*/
+
+@<Glob...@>=
+two_halves @!hash0[undefined_control_sequence-hash_base], *const @!hash = @!hash0-hash_base;
+ /*the hash table*/
+static pointer @!hash_used; /*allocation pointer for |hash|*/
+static bool @!no_new_control_sequence; /*are new identifiers legal?*/
+static int @!cs_count; /*total number of known identifiers*/
+
+@ @<Set init...@>=
+no_new_control_sequence=true; /*new identifiers are usually forbidden*/
+next(hash_base)=0;text(hash_base)=0;
+for (k=hash_base+1; k<=undefined_control_sequence-1; k++) hash[k]=hash[hash_base];
+
+@ @<Initialize table entries...@>=
+hash_used=frozen_control_sequence; /*nothing is used*/
+cs_count=0;
+eq_type(frozen_dont_expand)=dont_expand;
+text(frozen_dont_expand)=s_no("notexpanded:");
+ at .notexpanded:@>
+
+@ Here is the subroutine that searches the hash table for an identifier
+that matches a given string of length |l > 1| appearing in |buffer[j dotdot
+(j+l-1)]|. If the identifier is found, the corresponding hash table address
+is returned. Otherwise, if the global variable |no_new_control_sequence|
+is |true|, the dummy address |undefined_control_sequence| is returned.
+Otherwise the identifier is inserted into the hash table and its location
+is returned.
+
+ at p static pointer id_lookup(int @!j, int @!l) /*search the hash table*/
+{@+ /*go here if you found it*/
+int h; /*hash code*/
+int @!d; /*number of characters in incomplete current string*/
+pointer @!p; /*index in |hash| array*/
+int @!k; /*index in |buffer| array*/
+@<Compute the hash code |h|@>;
+p=h+hash_base; /*we start searching here; note that |0 <= h < hash_prime|*/
+loop at +{@+if (text(p) > 0) if (length(text(p))==l)
+ if (str_eq_buf(text(p), j)) goto found;
+ if (next(p)==0)
+ {@+if (no_new_control_sequence)
+ p=undefined_control_sequence;
+ else@<Insert a new control sequence after |p|, then make |p| point to it@>;
+ goto found;
+ }
+ p=next(p);
+ }
+found: return p;
+}
+
+@ @<Insert a new control...@>=
+{@+if (text(p) > 0)
+ {@+@/do at +{if (hash_is_full) overflow("hash size", hash_size);
+@:TeX capacity exceeded hash size}{\quad hash size@>
+ decr(hash_used);
+ }@+ while (!(text(hash_used)==0)); /*search for an empty location in |hash|*/
+ next(p)=hash_used;p=hash_used;
+ }
+str_room(l);d=cur_length;
+while (pool_ptr > str_start[str_ptr])
+ {@+decr(pool_ptr);str_pool[pool_ptr+l]=str_pool[pool_ptr];
+ } /*move current string up to make room for another*/
+for (k=j; k<=j+l-1; k++) append_char(buffer[k]);
+text(p)=make_string();pool_ptr=pool_ptr+d;
+#ifdef @!STAT
+incr(cs_count);
+#endif
+@;@/
+}
+
+@ The value of |hash_prime| should be roughly 85\pct! of |hash_size|, and it
+should be a prime number. The theory of hashing tells us to expect fewer
+than two table probes, on the average, when the search is successful.
+[See J.~S. Vitter, {\sl Journal of the ACM\/ \bf30} (1983), 231--258.]
+@^Vitter, Jeffrey Scott@>
+
+@<Compute the hash code |h|@>=
+h=buffer[j];
+for (k=j+1; k<=j+l-1; k++)
+ {@+h=h+h+buffer[k];
+ while (h >= hash_prime) h=h-hash_prime;
+ }
+
+@ Single-character control sequences do not need to be looked up in a hash
+table, since we can use the character code itself as a direct address.
+The procedure |print_cs| prints the name of a control sequence, given
+a pointer to its address in |eqtb|. A space is printed after the name
+unless it is a single nonletter or an active character. This procedure
+might be invoked with invalid data, so it is ``extra robust.'' The
+individual characters must be printed one at a time using |print|, since
+they may be unprintable.
+
+@<Basic printing...@>=
+static void print_cs(int @!p) /*prints a purported control sequence*/
+{@+if (p < hash_base) /*single character*/
+ if (p >= single_base)
+ if (p==null_cs)
+ {@+print_esc("csname");print_esc("endcsname");print_char(' ');
+ }
+ else{@+printn_esc(p-single_base);
+ if (cat_code(p-single_base)==letter) print_char(' ');
+ }
+ else if (p < active_base) print_esc("IMPOSSIBLE.");
+ at .IMPOSSIBLE@>
+ else printn(p-active_base);
+else if (p >= undefined_control_sequence) print_esc("IMPOSSIBLE.");
+else if ((text(p) < 0)||(text(p) >= str_ptr)) print_esc("NONEXISTENT.");
+ at .NONEXISTENT@>
+else{@+printn_esc(text(p));
+ print_char(' ');
+ }
+}
+
+@ Here is a similar procedure; it avoids the error checks, and it never
+prints a space after the control sequence.
+
+@<Basic printing procedures@>=
+static void sprint_cs(pointer @!p) /*prints a control sequence*/
+{@+if (p < hash_base)
+ if (p < single_base) printn(p-active_base);
+ else if (p < null_cs) printn_esc(p-single_base);
+ else{@+print_esc("csname");print_esc("endcsname");
+ }
+else printn_esc(text(p));
+}
+
+@ We need to put \TeX's ``primitive'' control sequences into the hash
+table, together with their command code (which will be the |eq_type|)
+and an operand (which will be the |equiv|). The |primitive| procedure
+does this, in a way that no \TeX\ user can. The global value |cur_val|
+contains the new |eqtb| pointer after |primitive| has acted.
+
+ at p
+#ifdef @!INIT
+static void primitive(char *@!str, quarterword @!c, halfword @!o)
+{@+str_number s=s_no(str);
+int k; /*index into |str_pool|*/
+int @!j; /*index into |buffer|*/
+small_number @!l; /*length of the string*/
+if (s < 256) cur_val=s+single_base;
+else{@+k=str_start[s];l=str_start[s+1]-k;
+ /*we will move |s| into the (possibly non-empty) |buffer|*/
+ if (first+l > buf_size+1)
+ overflow("buffer size", buf_size);
+@:TeX capacity exceeded buffer size}{\quad buffer size@>
+ for (j=0; j<=l-1; j++) buffer[first+j]=so(str_pool[k+j]);
+ cur_val=id_lookup(first, l); /*|no_new_control_sequence| is |false|*/
+ flush_string;text(cur_val)=s; /*we don't want to have the string twice*/
+ }
+eq_level(cur_val)=level_one;eq_type(cur_val)=c;equiv(cur_val)=o;
+}
+#endif
+
+@ Many of \TeX's primitives need no |equiv|, since they are identifiable
+by their |eq_type| alone. These primitives are loaded into the hash table
+as follows:
+
+@<Put each of \TeX's primitives into the hash table@>=
+primitive(" ", ex_space, 0);@/
+@!@:Single-character primitives /}{\quad\.{\\\ }@>
+primitive("/", ital_corr, 0);@/
+@!@:Single-character primitives /}{\quad\.{\\/}@>
+primitive("accent", accent, 0);@/
+@!@:accent\_}{\.{\\accent} primitive@>
+primitive("advance", advance, 0);@/
+@!@:advance\_}{\.{\\advance} primitive@>
+primitive("afterassignment", after_assignment, 0);@/
+@!@:after\_assignment\_}{\.{\\afterassignment} primitive@>
+primitive("aftergroup", after_group, 0);@/
+@!@:after\_group\_}{\.{\\aftergroup} primitive@>
+primitive("begingroup", begin_group, 0);@/
+@!@:begin\_group\_}{\.{\\begingroup} primitive@>
+primitive("char", char_num, 0);@/
+@!@:char\_}{\.{\\char} primitive@>
+primitive("csname", cs_name, 0);@/
+@!@:cs\_name\_}{\.{\\csname} primitive@>
+primitive("delimiter", delim_num, 0);@/
+@!@:delimiter\_}{\.{\\delimiter} primitive@>
+primitive("divide", divide, 0);@/
+@!@:divide\_}{\.{\\divide} primitive@>
+primitive("endcsname", end_cs_name, 0);@/
+@!@:end\_cs\_name\_}{\.{\\endcsname} primitive@>
+primitive("endgroup", end_group, 0);
+@!@:end\_group\_}{\.{\\endgroup} primitive@>
+text(frozen_end_group)=text(cur_val);eqtb[frozen_end_group]=eqtb[cur_val];@/
+primitive("expandafter", expand_after, 0);@/
+@!@:expand\_after\_}{\.{\\expandafter} primitive@>
+primitive("font", def_font, 0);@/
+@!@:font\_}{\.{\\font} primitive@>
+primitive("fontdimen", assign_font_dimen, 0);@/
+@!@:font\_dimen\_}{\.{\\fontdimen} primitive@>
+primitive("halign", halign, 0);@/
+@!@:halign\_}{\.{\\halign} primitive@>
+primitive("hrule", hrule, 0);@/
+@!@:hrule\_}{\.{\\hrule} primitive@>
+primitive("ignorespaces", ignore_spaces, 0);@/
+@!@:ignore\_spaces\_}{\.{\\ignorespaces} primitive@>
+primitive("insert", insert, 0);@/
+@!@:insert\_}{\.{\\insert} primitive@>
+primitive("mark", mark, 0);@/
+@!@:mark\_}{\.{\\mark} primitive@>
+primitive("mathaccent", math_accent, 0);@/
+@!@:math\_accent\_}{\.{\\mathaccent} primitive@>
+primitive("mathchar", math_char_num, 0);@/
+@!@:math\_char\_}{\.{\\mathchar} primitive@>
+primitive("mathchoice"