[tex-live] General test suite for TeX-Live

[Joseph Wright]

On 19/06/2016 20:34, David Carlisle wrote:
> On 19 June 2016 at 20:30, Uwe Ziegenhagen <ziegenhagen at gmail.com> wrote:
>> Maybe but my Lua Skills are non-existing, my Python skills improving on a
>> daily basis.
>>
>> Let's see if it works at all, then we can find a way.
>>
>> Uwe
>>
> 
> the existing (lua) l3build framework could be used as a model of a
> cross platform tex test suite.
> 
> Using texlua as the scripting engine has proved pretty successful in
> making testing and building
> packages work in all texlive supported environments.
> 
> David

I'd strongly suggest taking a look over l3build before anyone expends
effort in scripting something new. Whilst it's about more than just
testing, the core idea goes back a long way (see
http://tug.org/TUGboat/tb35-3/tb111mitt-l3build.pdf).

As detailed in
http://mirrors.ctan.org/macros/latex/contrib/l3build/l3build.pdf, it
does log-based testing with Lua doing the 'grind' and uses system
utilities for the diff step (if required). (The team and so l3build
needs to work cross-platform, so this works on *nix and Windows.) As
Norbert has noted, we've recently added PDF-based comparisons as this
will be potentially-useful for cases where log-based methods are not so
useful.

I'd also point to the automated testing we've set up:
https://travis-ci.org/latex3/latex3. (Travis-CI is one of many choices,
this is just a convenient way of running the tests on a VM we don't have
to maintain.) In amongst our own tests there are a small number for
third-party code (https://github.com/latex3/latex3/tree/master/contrib).
I talked about this at the UK-TUG meeting last year:
http://river-valley.zeeba.tv/why-does-it-always-fail-on-me-automating-testing-using-travis-ci/.

Joseph