Can doc/ contain documentation independently of source/

[Joseph Wright]

Hello Jean-François,
> So far I have mainly interpreted TDS structure as asking for source/
> to contain the source not only of macro files but also of the complete
> documentation.

I think that's the usual reading.

> As a result, for my packages with source distributed as .dtx files
> (which is almost all my packages) I may embed into the
> .dtx files things such as README.md, CHANGES or even Makefiles needed
> to build documentation and reconstruct a full package.tds.zip.
 >
> I am wondering, accepting for the duraction of this discussion
> that the whole .dtx concept actually continues
> to make sense nowadays, whether I should not bundle things a bit
> differently and consider that xint.dtx should only be a container
> of the package documentation stricto sensu and macro files.
> 
> Things such as README, CHANGES, Manifest, Makefiles
> being left outside and only included in doc/ but not in the dtx
> which ends up in source/.  I.e. the shipped bundle would still
> provide the means to both produce the complete documentation
> and macro files, but this would go perhaps via using a contributed
> Makefile or other means located in doc/, which to work would
> of course would need to have foo.dtx available too.

For small packages, there is an attraction in having a single source. 
However, the LaTeX team's idea with .dtx files is rather more toward 
them being the source for the code + code docs, with other files 
separate. For example, all of the core team stuff has README files, 
scripts, etc. as separate files, and the scripts or whatever don't even 
go to CTAN. (Makefiles or whatever depend on having some known 
development set up, so are likely useful more in a source repository for 
a small number of devs than on CTAN for users.)

> But foo.dtx by itself would only be a container of macro files
> and perhaps of some user-manual.
> 
> For example for xint which I updated recently, in the process
> I removed some previous duplication (I had been going
> quite overboard with both a CHANGES.html and CHANGES.pdf
> for example). But currently the xint.dtx still contains
> CHANGES in markdown format, as well as shell scripts
> which can use pandoc to produce CHANGES.html
> as distributed in the doc/ section too.
> 
> I am considering removing entirely from the xint.dtx
> all these things ; and distribute directly only a CHANGES.html
> produced at home.
> 
> Thus CHANGES.html will qualify as its own source but be in doc/
> 
> Also a Makefile (currently extractable from xint.dtx as Makefile.mk)
> would end up in doc/ but not be in the source/xint.dtx
> 
> In fact source/ would contain only xint.dtx (I have also dropped
> xint.ins which always had been purely optional as xint.dtx
> self-extracts).
> 
> And doc/ would contain Makefile, CHANGES.html, README.md,
> xint.pdf and sourcexint.pdf.
> 
> The source/ would thus only contain the source for xint.pdf
> and sourcexint.pdf production, and of course of the macros files xint*.sty
> but not for the other stuff (which nevertheless are an indispensible
> part of the user documentation).
> 
> In the longer term, I think the whole concept of stripping xint*.sty files
> is somewhat dated and I would prefer an approach à la Python
> where the macro files have their docstrings and the CTAN uploads
> do not included a bulky PDF documentation byt only the
> macro files and the user has means to consult the docstrings.
> 
> This would not preclude having some dedicated xint.pdf user manual,
> but at least in the case of code documentation would include it
> in only one place, not duplicated in a "dtx" and in a "pdf".

One can take various views with code documentation. Again, the approach 
of the team has been to aim for one foo.tex file as the source for 
*user* documentation, and one foo.dtx as the source for documented code. 
One can skip typesetting the latter, particularly for smaller, simpler 
packages, but the source misses things like cross-refs, indexing, etc. 
So there it very much depends how you imagine people using your sources. 
(For the LaTeX kernel, things are quite possibly different from a 
one-file .dtx of a few 10s of lines.)

> I even wonder after all why one should include the TeX source
> of a pdf documentation at all and why not directly the PDF ?

Without the source, the PDF can't go into TeX Live as the ability to 
build binary files like PDFs is a key part of the requirements for e.g. 
Debian. If you are worried about space/size, the PDFs are also where the 
hit is: the sources will always be much smaller.

> Then what would actually source/ serve to ? There would be
> only tex/ for the macro files containing docstrings
> and doc/ for extra documentation including the means
> to extract the docstrings.
> 
> That TeX would mean a bit more time to load the macro files
> looks irrelevant nowadays.

One approach taken by some authors is to put the code docs at the end of 
the .sty, thus avoiding the load time issue. But you are right that the 
time hit for simply having comments in a .sty or similar is today 
trivial. (Things like pgf are documented that way without obvious impact.)

> For my package polexpr I distributed the documentation as
> an html file polexpr.html which is produced via DocUtils
> from reStructuredText source, polexpr.txt, but also there
> I wonder why distribute at all this "source" and not uniquely
> the html ?
> 
> In brief, shouldn't there be some project about reducing
> duplication in the TDS trees?

Well, 'duplication' depends what you mean: a PDF is not really 
'duplication' of a source, having the source file twice on the other 
hand is.

Joseph