[tex-live] General test suite for TeX-Live

[Reinhard Kotucha]

On 2016-06-18 at 14:59:18 +0900, Norbert Preining wrote:

 > Hi Uwe,
 > 
 > I think this is a nice idea, but needs good thinking. I don't want
 > just a bunch of examples - all must be automatized and best
 > included in our daily checks.

Hi Uwe and Norbert,
maybe it's worthwhile to know how Thomas Esser tested all the packages
before they were included in teTeX.

Thomas said that his tests were never supposed to detect all problems
but a package should at least be able to compile its own
documentation.  Thus he created a Makefile for each and every package
he supported.

Even this was an enormous amount of work.  teTeX was a small subset of
TeX Live, hence it was feasible.  But TeX Live was always much bigger
and grew rapidly since then.

Compiling package documentation is a good starting point.  If a
package comes with a .dtx file, the process can probably be
automatized, but I fear that most packages require manual
intervention.

An other point comes to my mind.  Occasionally people report that some
dependencies in texlive.tlpdb are incorrect.  Of course, it's
difficult to get everything correct.  But if there is a halfway
comprehensive test suite we can probably determine dependencies from
the log files.  Another approach is to run TeX with the --recorder
option on these files and evaluate the .fls files afterwards.

I don't think that the dependencies in texlive.tlpdb can be determined
automatically but I'm convinced that a halfway comprehensive test
suite can provide useful information.

Uwe's idea is, as far as I understand, to create files which load many
packages at once.  This reduces the mumber of required tests
significantly.  The caveat is that some packages are overlooked
because Uwe doesn't use them.

After all, I like Uwe's idea.  In TeX parlance:

  \let\uwe\proceed

Regards,
  Reinhard

-- 
------------------------------------------------------------------
Reinhard Kotucha                            Phone: +49-511-3373112
Marschnerstr. 25
D-30167 Hannover                    mailto:reinhard.kotucha at web.de
------------------------------------------------------------------