[tex-live] hyperref/puenc.def broken after upgrade

[Heiko Oberdiek]

On Fri, Sep 07, 2012 at 12:50:33PM +0200, Pander wrote:

> Would be nice that the people who told me to start working on this that
> some feedback would be given.

> On 2012-08-15 12:25, Pander wrote:
> > 
> > Please review:
> >   tlit - TeX Live Integration Testing
> >   https://github.com/PanderMusubi/tlit
> > 
> > The goal of integration testing of TeX Live is to automatically confirm
> > that an update did not break any major functionality. Detailed module
> > testing of individual packagese is up to the package maintaineres. This
> > is done in much more detail and is outside of the scope of integration
> > testing. Integration testing offered here is merely a simply check if
> > certain often used combinations of package can play together. The goal
> > is to prevent that simultaneous updateting of multiple packages, which
> > all passed their individual module tests without being aware that other
> > packages are also being updated on the same time, break TeX Live.

* It assumes that packages have individual module tests that are run on
  updates. Both is not true with few excaptions only.

* To some degree package testing already involves other packages.
  There is no sharp separation line for integration testing.

* I cannot find, how this is addressed:
  * There are quite a few TeX compilers with quite a few formats.
  * There are lots of classes and tons of packages.
  * Classes and packages provide none up to an endless number of options
    with multiple values and working modes.
  * The number of package combinations and loading order is overwhelming.

* What is tested:
  * Errors: expected or unexpected
  * Warnings: expected or unexpected
  * Messages: expected or unexpected
  * Output: DVI/PS/PDF
  * log file, aux files, ...

* Which system to manage the tests?

* How to deal with the result? But report to package authors?

* ...

Good testing is a huge task requiring lots of "(wo)man power".

Yours sincerely
  Heiko Oberdiek