[tex-live] tlmgr query

[Norbert Preining]

Some answers ...

On Fr, 24 Jul 2009, Rowland McDonnell wrote:
> What I have achieved in all that time is to have found out a little bit
> about how to tell TeX Live (MacTeX 2008 installation) to generate
> locally specified formats and how to add new fount map files.  I have

That is already more than most people know. Most people simply expect a 
working system and do not want to fiddle around with the details. They
never heard about updmap(-sys), fmtutil(-sys) etc.

> It is very slow and frustrating: no-one involved seems to understand

True, it took me long long time, probably some years, to understand
all the internals. Do you expect to understand everything without 
serious, and I mean really serious, investment of time? Anyone having
ever prepared a TeX system knows that this is a huge amount of work,
and that a few hours here and there is far from being enough to 
understand even the surface.

Sorry, that is life. But nobody expects you to understand the internals
of Microsoft Word, but with TeX you have at least the chance to understand
the internals, if you want to know them.

> expert at all.  In particular, I find it very hard to understand man
> pages and I am *NOT* familiar with the structure of TeX Live.  I have

man pages are nothing mythical, please. You are native English, so I 
assume you have the ability to comprehend even a longer text like the
man page of tlmgr. That has nothing to do with Unix or not Unix.

> Thank you again.  I had read a lot of things telling me that I should
> use tlmgr instead of texconfig.  It seems that the advice is not

You should use tlmgr for changing some configuration options, but
texconfig conf is something different, and in fact I don't see its
immediate use (in fact I never used it).

> >> I would also like to know what exactly tlmgr -gui does when the
> >> `Re-create all formats' button is pressed.  I have some custom
> >> formats
> >
> >It calls
> >   fmtutil-sys --all
> >not surprisingly.
> 
> Thank you.  It is not any surprise, but not something that could be
> relied upon.  For all I knew, tlmgr might have been calling a
> fmtutil-sys (or plain fmtutil) and asking for a special fmtutil.cnf
> file.

And which should that be? There is one fmtutil.cnf file that defines
the formats, and that is used.

> Now I have to find out what fmtutil-sys --all does and exactly how it
> differs from fmtutil --all.  I'm stuck at the moment.   I have read the

The difference between the -sys and none-sys variants has been introduced
in teTeX 3.0, so about 6 (8?) years ago? The differnece is where stuff
is written to/read from, 
	-sys:	TEXMFSYSVAR, TEXMFSYSCONFIG
	none-sys: TEXMFVAR, TEXMFCONFIG

> I've read the man page and asked questions.  I still do not understand
> how to use kpsewhich, nor do I understand exactly what it is that
> kpsewhich tells the user.

It tells you which and if at all a file is found by the respective 
program:
	$ kpsewhich -progname latex fancyhdr.sty
	..../tl/2009/texmf-dist/tex/latex/fancyhdr/fancyhdr.sty

> In short, kpsewhich has not yet been my friend.  I often have to use
> find to supplement kpsewhich.  I cannot work out which file in a texlive

find does not know about where the respective programs are searching for
files. All that is well documented in the kpathsea.info document, which
can also be formatted in pdf, or html.

> Remember that I'm a Mac user and I do not - yet - understand the
> structure of texlive.  A lot of things you take for granted are alien to
> me.

Not many understand the full structure and working of TeX Live, probably
less then 10. So don't expect to understand it without serious research.

> > it is for 
> > ROOT/texmf-var/web2c/fmtutil.cnf
> 
> Could you explain how you know that?  You said kpsewhich is my friend,
> but I don't see how kpsewhich could tell you that.

	kpsewhich -mode="web2c files" fmtutil.cnf

> I'm afraid I do not quite understand this part.  When you say `is
> installed', what does the installing (or removing for that matter)?

	tlmgr install ....
	tlmgr remove ....
	tlmgr update ...

Did you read, I mean READ not skim, the tlmgr documentation?

> Where does the information in the database come from?

>From the packages that you have installed, and originally from our
master copy.

> What processes add or remove data from it?

	tlmgr ...

(man page????)

> And what data exactly is the file ROOT/texmf-var/web2c/fmtutil.cnf
> generated from?

>From the data in the texlive.tlpdb, exactely from the lines
	execute AddFormat ....

> I've read all the man pages I can find that might be relevant.

Doesn't look like.

> I've spent some hours on this - and got only headaches.  I cannot find
> any documentation on the TeX Live Package Database.

As already mentioned, go to 
	http://www.logic.at/people/preining/Work
and you will find many things, the is a long article in the ArsTeXnica,
and many other things. 


ArsTeXnica article: http://www.logic.at/people/preining/pubs/at07.pdf
GuIT Meeting 2007 presentation: http://www.logic.at/people/preining/talks/guit07.pdf
BachoTeX 2008 presentation: http://www.logic.at/people/preining/talks/bachotex08-talk.pdf
ArsTeXnica 2008 TL and tlmgr article: http://www.logic.at/people/preining/pubs/at08.pdf
resp.presentation: http://www.logic.at/people/preining/talks/guit08.pdf
TL 2008 talk at the CSTUG: http://www.logic.at/people/preining/talks/cstug08.pdf


Plus all the documentation (containing most of the stuff from above) in
	http://www.tug.org/svn/texlive/trunk/Master/tlpkg/doc/

So don't tell me nothing is documented!

> >> What does `Re-initialize file database' mean?
> >
> >Calling
> >   mktexlsr
> >(was that difficult?)
> 
> I do not understand your question.
> 
> It was impossible for me to work out what `Re-initialize file database'
> meant.  I had absolutely no hints at all.  It was not hard: it was
> impossible.

Because you have no idea how TeX works.

Reading kpathsea.info would have told you that there is for each tree
a ls-R file listing all the files. And that texhash/mktexlsr is updating
these database.

> I am not familiar with the command mktexlsr.  I am not a Unix guru.  I

Nothing to do with Unix, same on Windows, Mac, ...

> am not a long-standing expert in the care and maintenence of texlive.  I

If you use TL and you don't have any special need, you don't need to,
tlmgr is caring for all that.

> The reason I asked about backups is that I've read the man page, found
> the section on backups, and I've found it impossible to work out what
> tlmgr does by way of backups.

Aemmm ... did you read tlmgr --help???

       --backup and --backupdir directory
           These two options control the creation of backups of packages
           before an update is started; that is, the backup is of the package
           as it's installed. ...

Isn't that clear enough? When you call
	tlmgr update
and give --backup or have set that option, a backup of the current status
of a package is done (.tar) before updating. Using 
	tlmgr restore
you can go back.

tlmgr -help reading helps, *REALLY*, trust me!

> sort of Unix expert; I cannot write shell scripts, I do not speak Perl,
> I'm not familiar with the structure and jargon of Unix or texlive).

No need do be Unix guru, only reading *carefully* the documentation.

> One reason texlive man pages are often not very helpful for me is their
> excessive use of the passive voice.
> 
> For example: `these two options control the creation of backups of
> packages before an update is started'.
> 
> Before an update is started by what, exactly?  I can't find anything
> that tells me and I can't work it out.

Aemmmm .... tlmgr update ... did you READ?

I mean, honestly, are you joking or do you want to steal our time here?
Did you read or not? And if you did, why don't you remember
	tlmgr update
?

> I don't even know what `backup' means in this context.  Nor do I know
> what a `package' is in this context.  Of course in /general/ terms I

Well, packages are packages in the CTAN sense. Like memoir, like
fancyhdr, like ...., one package shipping many files, or sometimes only
one file.

> what *exactly* will be saved (and how is that worked out)?

tar is used to package the files into an archive, the list of files
is determined from the texlive.tlpdb

> what does the saving?

tlmgr

> what events exactly prompt the save to happen?

the command line options or the options saved in the tlpdb, together
with the actions backup or update.

> what does `update' mean in this context?

tlmgr udpate

> And what's a package?

See above.

> I know about LaTeX packages (pkg files - and their dtx/ins source code
> etc), but I find it hard to credit that `something' makes backups only

Exactely, and the collection of these files is one package.

So you can say
	tlmgr install fancyhdr
and the full fancyhdr package will be installed

> Please believe me that I have put a lot of time and effort into finding
> my own answers to the above questions.  I'm not asking because I'm lazy:

No, I don't believe you. Not the least.

> Okay, that's where the database is.    But again I have more questions I
> cannot find the answers to myself.  What puts data in the database?  And

install-tl initially, i.e., the installation.
After that tlmgr

> where does this data come from?

>From the packages you install, thus from the master texlive.tlpdb on the
DVD/tlnet distribution on CTAN.

> Righto - thank you for that.  Do you know how I might find out more
> about it?  In particular, where does the information in that database
> come from?

>From the master tlpdb, and there it comes from the definitions WE put
into our tlpsrc files, see
	http://www.tug.org/svn/texlive/trunk/Master/tlpkg/tlpsrc/

Enough now.

I would say before you TRY to understand the inner working, please begin
to actually *USE* tlmgr. And when you are familiar with tlmgr and its
usage, then you can read more documentation like the above mentioned
articles and presentations.

Best wishes

Norbert

-------------------------------------------------------------------------------
Dr. Norbert Preining <preining at logic.at>        Vienna University of Technology
Debian Developer <preining at debian.org>                         Debian TeX Group
gpg DSA: 0x09C5B094      fp: 14DF 2E6C 0307 BE6D AD76  A9C0 D2BF 4AA3 09C5 B094
-------------------------------------------------------------------------------
FARNHAM (n.)
The feeling you get about four o'clock in the afternoon when you
haven't got enough done.
			--- Douglas Adams, The Meaning of Liff