[tex-k] Bug report for tex.web

[Andreas Scherer]

Am 31.08.24 um 01:09 schrieb Karl Berry:
> How does this affect the section name sorting?  It seems like it would
> be sorted under "Store t" anyway?
Yes, in this particular case, the final sorting order is not different.
However...

> but it seems that like his use of \(m) in
>    @<Store \(m)maximum values ...
> is likewise a no-op?
Not at all! If we leave out the '\(m)', then this section name is sorted 
_after_ '@<Store \(s)|save_stack...'. And if we had a section name with 
a capital letter at the inflection point, say '@<Store TeX stuff@>' 
(without the backslash), then this would come _before_ section name 
'@<Store \(c)|cur_box|@>'. ('@<Store \TeX\ stuff@>' would come _after_ 
'@<Store \(s)|save_stack...@>', but _before_ '@<Store maximum...@>'.)

> What's going on :)?
The reason for these curious results is that the sorting is done by 
C/WEAVE purely based on the ASCII (sic!) characters between '@<' and 
'@>' of a named section. TeX _eliminates_ the '\(...)' stuff only later 
when formatting the already sorted C/WEAVE output.

Here are a few simple relations of the 7-bit ASCII table:
* '[A-Z]' < '\' < '[a-z]', hence 'TeX' < '\(c)' and 'max' > '\(s)'
* '(' < '[A-Z]', hence '\TeX' > '\(s)' (and '\TeX' < 'max')

> This is not a feature I ever noticed before. Learn something new about
> TeX every day (still).
Yep, I stumpled upon this rather tricky stuff by pondering Point 11 of 
Appendix G in 'webman' (see 
https://mirrors.ctan.org/info/knuth-pdf/web/webman.pdf) regarding the 
'kludges' to make index entries and names-of-sections sorted to the 
user's liking. (It's also Point 11 in Appendix C in 'cwebman'; see 
<https://mirrors.ctan.org/info/knuth-pdf/cweb/cwebman.pdf>.)

While the trick with '\def\9#1{}' is well documented in both WEB amd 
CWEB in connection with the '@:' control code, I wasn't aware of DEK's 
method(s) to sort section names until very recently.

DEK defines '\def\(#1){}' in many WEB codes besides TEX.WEB, but 
actually uses this in section names only in VFTOVP.WEB. I have _not_ 
checked the typeset codes of DVICOPY, DVITYPE, GFTODVI, GFTOPK, GFTYPE, 
MF, PKTOGF, PKTYPE, PLTOTF, POOLTYPE, TFTOPL, VPTOVF for possible 
sorting errors; these WEB codes have '\def\(#1){}', but they do not 
_use_ this macro.

DEK also declares the slightly different 'kludge' '\def\({}' in 
TANGLE.WEB, WEAVE.WEB, and MFT.WEB (not!), and relies much more on 'the 
natural sorting order' and peculiar relations like 'P'>'(' to get
'@<Translate the \(definition part@>'
'@<Translate the \PASCAL/ part@>'
'@<Translate the \TEX/ part@>'
in correct order without much ado.

On August 2, I actually _removed_ '\def\({}' from CWEAVE.W (TeX Live 
revision 71951), because it was not used at all. And it wouldn't have 
worked in CWEB, because then the sorting order of the related sections 
would have been incorrect:
'@<Translate the \(definition part@>'
'@<Translate the \CEE/ part@>'
'@<Translate the \TEX/ part@>'

Two problems remain: (a) The names-of-sections of CWEAVE.W have sorting 
errors (see https://mirrors.ctan.org/info/knuth-pdf/cweb/cweave.pdf) and 
(b) the PDF outlines (bookmarks) of TEX.WEB (and all its variants in 
'knuth-pdf') and VFTOVP.WEB have spurious leftovers from the '\(...)' 
macro (see https://mirrors.ctan.org/info/knuth-pdf/tex/tex.pdf).

After a full day of furious hacking, I came up with a simple solution 
that will be made public in CWEB 4.13 and PWEBMAC.TEX 5.1 later. The 
gist is to use '\def\9#1{}' instead of '\def\(#1){}' and amend the PDF 
'sanitizer' accordingly. Here are a few links to the current state of 
affairs:
* https://github.com/ascherer/cweb/issues/55
* 
https://github.com/ascherer/web/commit/205a91912423c71c7ec1829f5fd003ad2da32da5

To conclude, the proposed change to TEX.WEB does not actually 'fix' a 
visible bug; it is just intended to improve the consistency, because 
'@<Store the...@>' is the single section name with siblings that require 
special treatment.

Whew, what a brain storm!

Cheers,
Andreas