[tex-live] tlmgr query

[Norbert Preining]

Hi Rowland,

Definition 1.
A package identifier is defined by the following grammar, S being the
initial symbol, Sigma = all lower and upper-case letters, plus underscore.
S ::= Sigma S
S ::= Sigma

Definition 2.
The "root of an installation" is the directory choosen by the user
when an installation is done. Usually on newer Mac and Unix systems
this is /usr/local/texlive/2009 (currently), but might be changed
at installation time.

Definition 3.
An "execute statement" is either:
- AddFormat
- addMap or addMixedMap
- AddHyphen
The semantics of that is defined straightforward by reformatting the
rest of the respective arguments to text lines suitable for fmtutil.cnf,
updmap.cfg, language.dat and language.def.

Example 3.1
The execute statement
  AddFormat name=amstex engine=pdftex options="-translate-file=cp227.tcx *amstex.ini"
triggers the addition of the following line to ROOT/texmf-var/web2c/fmtutil.cnf
	amstex pdftex - -translate-file=cp227.tcx *amstex.ini

Example 3.2
The execute statement
  addMap epiolmec.map
will trigger calling
  updmap-sys --enable Map epiolmec.map
on installation, and
  updmap-sys --disable Map epiolmec.map
on removal.
  
  
Definition 4.
A package is an entity of
- list of files
- short and long descriptions
- execute actions
a package is described by a text block (either saved in a text file
or being part of a larger document) of the format described in
TLPOBJ.pm in pod documentation.

Remark: This documentation can be obtained by calling pod2man, pod2html,
pod2text, pod2latex on the .pm file.

Definition 5.
A container of a package consists of a tar archive containing all the
files of the package (def 2) and the text document describing the
package saved in tlpkg/tlpobj/<package-identifier>.tlpobj within the
tar archive. The container may or may not be compressed with xz.

Definition 6.
The TeX Live Database is a text file containing descriptions of packages
(see definition 3) separated by at least one empty line.

Definition 7.
Installation of a package is the operation of:
- unpacking the tar archive in the root of the installation
- adding the description (as saved in the tar archive, see def 4)
  to the texlive database
- executing the "execute statements" listed in the description

Definition 8.
Removal of a package is the operation of:
- removing all files listed under the package identifier in the texlive
  database
- executing all execute statements for removal
- removing the description of the package from the texlive database

Definiton 9.
Upgrade of a package is the operation of:
- removal of the package
- installation of the package

Remark: Since the installation is done from a new container, it can
contain updated or changed files.
  

Enough joking, here some real remarks:

On Fr, 24 Jul 2009, Rowland McDonnell wrote:
> Unplug Babel

That is interesting, and not easily supported. So what you want are
special formats that do NOT contain babel, right?

The easiest way to achive that is to put your additional format 
definitions into
	ROOT/texmf-local/web2c/fmtutil-local.cnf
where ROOT is the root of your installation, AFAIR 
	/usr/local/texlive/2009
in your case.

Then these formats will be always included in the fmtutil.cnf file
generated by tlmgr and used by fmtutil-sys. It is the one in 
	ROOT/texmf-var/web2c/fmtutil.cnf

(explained in tlmgr help under action "generate")

> Plug in some additional founts

Simply put them into
	ROOT/texmf-local/
in a TDS compliant way and call as the one who installed TeX Live:
	updmap-sys --enable Map your.map

(explained in tlmgr help under action "generate")

> Add a local system-wide additional TEXMF tree.

use 
	ROOT/texmf-local
or add a line
	TEXMFLOCAL = /the/path/to/your/system/local/tree
in
	ROOT/texmf.cnf
(there is already that file but without any entry but comments).

(usual configuration)

> Have you not understood that I have /already/ invested a significant
> amount of time in this job?  And that I continue to do so despite

No, you do NOT have invest a significant time. Do you want to know how
much time I have invested? I don't count hours, I count man-month!

> But I need help to make progress - just some small, simple items of
> information, which are proving nearly impossible to discover.

Ok, above there are the things you want to know. Try it out now.

> ever studied it on my own, and Knuth is a really rotten teacher so I've

You already said that in several other threads where you discussed the
documentation of other packages. Do you always come up with the same
sentences. I just read *exactely* the same thing from you. What are
you trying to achieve? Solving your problem or making problems?

> <very puzzled> I did not claim that they were.  They're real, living,
> and nearly completely incomprehensible in most cases.

Maybe to you, but not to the rest of the world. At least there are enough
other people out there who managed to read the tlmgr help pages and
act according to them, with success. So there is either a problem on
your side or I am hallucinating, which could of course be possible, due
to the many times I have been in high altitude.

> > You are native English,
> 
> Unashamedly so - but I'd be careful about that sort of accusation if I
> were you.  Not all QEII's subjects are English, remember; FWIW, my
> surname's Irish.

That was not an accusation, it was a statement. If it was wrong, sorry 
for that. If you consider your mother tongue to be Irish, I still expect
that you are fluent in English.

> The length of the text is not the problem.  The contents of the text is
> the problem.

For you, as I said, not for most others. You can start a questionaire
about the usefullness and understandability of the tlmgr help page, and
I guess your evaluations will fall right to the bottom, and (at least 
I hope) many other will value it higher then your estimation.

> texconfig conf is the only method I know of for finding out what TeX
> system path variables are set and what paths they refer to.
> 
> Do you know of another way of doing that job?  I have failed to find any
> mention of another method in the documentation.

kpsewhich -var-value TEXMF
kpsewhich -var-value TEXMFCONFIG
kpsewhich -var-value TEXMFVAR
...


> Because I have two different MacTeX installations selectable via the TeX
> prefs pane, I have two different local hand-maintained fmtutil.cnf files
> - specifically:

That is not my problem. That is even less a problem of TeX Live, that is
MacTeX specific.

> /usr/local/texlive/2007/texmf/web2c/fmtutil.cnf
> /usr/local/texlive/2007/texmf-var/web2c/fmtutil.cnf
> 
> /usr/local/texlive/2008/texmf/web2c/fmtutil.cnf
> /usr/local/texlive/2008/texmf-var/web2c/fmtutil.cnf
> 
> So there are two different fmtutil.cnf files per TeX Live installation
> by default, and I use my own custom version (at the moment) for
> generating the formats that are in use (because I don't understand the
> automatic management processes, I dare not use them).
> 
> But the interesting thing is the two fmtutil.cnf files installed by
> default.

Right, the texmf/web2c/fmtutil.cnf is in fact useless.

> kpsewhich -mode="web2c files" fmtutil.cnf
> 
> returns:
> 
> /usr/local/texlive/2008/texmf-var/web2c/fmtutil.cnf
> 
> so that's the version which will be used by tlmgr, I assume.  What,
> though, is the other copy for?  That is not clear.

Nothing for. You can ignore it.

> >The differnece is where stuff
> >is written to/read from, 
> >   -sys:   TEXMFSYSVAR, TEXMFSYSCONFIG
> >   none-sys: TEXMFVAR, TEXMFCONFIG
> 
> Ah!  That is very useful information - thank you!  Where is it
> documented?  I've made a note of it in my current `How to manage TeX on
> my Mac' document.

	man updmap

tells me:
       ...         In the latter case, it tries to  use  the  tree  where  the
       variable  TEXMFVAR  -- or TEXMFSYSVAR if called as updmap-sys -- points
       to.

> I have so far been unable to work out what `respective program' means

tex, pdftex, dvips, latex, context, xetex, ...

the one given as -progname=XXXXXX

> kpsewhich cannot find kpathsea.info.

Of course not, kpsewhich is a program for TeX/MF input files, not for
info documentation.

> >> > ROOT/texmf-var/web2c/fmtutil.cnf
> >> 
> >> Could you explain how you know that?  You said kpsewhich is my
> >> friend, but I don't see how kpsewhich could tell you that.
> >
> >   kpsewhich -mode="web2c files" fmtutil.cnf
> 
> Thank you for telling me what incantation to use.  But I wonder how one
> might work out that that is the right form?  According to the kpathsea
> documentation I've got printed out here:

Sorry, my fault, that should have beein

	kpsewhich -format="web2c files" fmtutil.cnf
and under --format=NAME in the info documentation you fine all you need,
hopefully.

> >> I'm afraid I do not quite understand this part.  When you say `is
> >> installed', what does the installing (or removing for that matter)?
> >
> >   tlmgr install ....
> >   tlmgr remove ....
> >   tlmgr update ...
> 
> Thank you for listing the commands.

See above definitions.

> I have already told you that almost none of that style of documentation
> makes any sense to me at the moment due to undefined terms.  I can read

I have defined some of them above, some others can be defined, but
I am too lazy because everyone else understands the definitions
intuitively, everyone but you.

> As I've tried to explain many times, the tlmgr man page is almost
> totally useless for me at the moment because it is full of undefined
> terms that I am finding it very very hard to get defintions for because

Don't discuss with a mathematician about undefined terms. Or I will start
questioning YOUR emails about undefined words and terms you are using.

> I think there really is no excuse for writing such rotten documentation.

Thanks for the insults, great to hear that you are insulting those trying
to help you. Is that the way you achieve your goals in real life, too?

> >As already mentioned, go to 
> >   http://www.logic.at/people/preining/Work
> >and you will find many things,
> 
> I had looked <http://www.logic.at/staff/preining/TeX> and I discovered

Why did I write .../Work ???? There the articles I mentioned are listed.

> > the is a long article in the ArsTeXnica,
> >and many other things. 
> 
> I was unable to find a link to that.

because you were not looking at what I told you!

> >ArsTeXnica article: http://www.logic.at/people/preining/pubs/at07.pdf

Did you read that one? Did you read that one? Did you read that one?

If yes, then we can continue.

> <raised eyebrows>  I am somewhat surprised to come across that claim.  I
> can assure you I've got a very clear idea how TeX works and have had
> since the 1980s.

ok, maybe in the 80s, but since then many years have passed and the
ls-R databases are there since many years.

> I understand how TeX works, but I do not understand Unix or Unix TeX
> system organization and operation.

No, again, that has nothing to do with Unix. TeX Live works *the*very*same*
way on Unix, Windows, Mac. And it was like this with emTeX, it is the
same with miktex.

> Thank you again!

(Sorry one question: Have you been raised in the US? Your habit of
thanking me over and over again, and in the next sentence insulting me
and everyone here is a bit strange, it reminds me on the thoughtless
"Hi how ya doing" each time you enter a restaurant, bar, ... in the US,
where nobody is actually interested in the answer. Guys from the US, 
sorry for the rant, but I am really feeling like that, I can't hear
that "Hi how ya doing" anymore ;-)

> I've looked, and I can't see that the kpathsea documention says anything
> at all about what tlmgr does.  Where is the section in that document

Of course. kpathsea is the library using in tex programs, tlmgr is a 
configuration and management frontend.

> OzTeX did not operate in that way as far as I'm aware.

I cannot guarantee anything here, but i am quite sure that OzTeX had
something similar.

> OzTeX created a file database without user intervention on startup.

Might be, but it didn't have the ability to update a package (no don't
ask me for a definiton, see above). Now you can call
	tlmgr update memoir
and woops you have the fix for a bug of memoir on your computer. Did
OzTeX allow that?

> Thank you for that suggestion, but I've worked out that tlmgr does not
> perform as you claim it does.

Specifics please, no shakey claims. Where and under which circumstances
did it misbehave? (There are of course many bugs, I never said it is
bug free)

> If I can work out what a `package', `current status of a package', and a
> `backup' means *exactly* in this context, I might be able to understand
> it a little.  But I'm still ignorant of what `package' means and I've no

Please download
	http://mirror.ctan.org/systems/texlive/tlnet/2008/tlpkg/memoir.tar.lzma
that is a package. Unpack it with unlzma or lzma -d, and look into 
the tar archive with tar -tvf 

> Yes, and I totally failed to understand what "tlmgr update" does.

Each package has a revision saved in the local database, the one 
saved on your computer. tlmgr update contacts the texlive distribution
server and checks which packages on our server have a higher revision
number. These package can be updated.

> The fact that some of these terms might well be formally defined if one
> were to read and understand several other long and (to me) nearly
> totally incomprehensible documents doesn't help me at all.

Well, then you don't WANT to know it. The definition is there, but
one has to read it, even if it is long.

(Remember what I said about man-month?)

> Unfortunately, you seem to be angry with me for not understanding things
> that you do understand and I'm not getting many of the explanations I've
> asked for.

I explained it already several times, but it seems that my English
capabilities are so inferior that all my trials occur to you as brabbling
of small children or insanes.

> >Well, packages are packages in the CTAN sense. Like memoir, like
> >fancyhdr, like ...., one package shipping many files, or sometimes only
> >one file.
> 
> So where can I find out what `package' means in the CTAN sense?

LPPL ever heard? LaTeX Project Public License? That defines a "work"
which we call package (and everyone else). The entity created by an
author or group of authors.

> Okay, so that's what does the job and I get a tar file out.  Not what I
> asked for here, but useful information.
> 
> Which files, exactly, are put in the tar file?

Those listed under the package identifier in the local database.

> I shall not press you for an answer.  You've got angry enough with my
> inability to understand things which are very simple for you.

Do you always repeat the very same sentence in the same email several
times?

> Since I have not installed any packages except some of my own local ones

You did, by installing MacTeX a certain (unknown to me) selection of 
package of TeX Live are installed onto your computer and registered in
the local texlive database. If you have selected scheme-full, all
package available are installed, otherwise some might be missing.

> It would be very stupid of me to follow that advice, because doing so
> would certainly break my TeX installation.

Most certainly not, but anyway.

> I'm only really interested in the outer workings, but it seems that
> you're not at all happy about me learning them.

No, I am not happy about writing hours on emails answering questions
only you are posing.


> Normally, I just need a bit of help to sort out some basic questions at
> the start.  That's all I'm asking for at the moment, but my questions
> seem to anger you.

Why don't you try out
	tlmgr update --list
it will NOT change anything on your system. Same with
	tlmgr show
	tlmgr search
and you will slowly learn what is going on.

All the best, hope you read that email carefully, it cotains many hidden
pearls that might help you on your epic quest.


Best wishes

Norbert

-------------------------------------------------------------------------------
Dr. Norbert Preining <preining at logic.at>        Vienna University of Technology
Debian Developer <preining at debian.org>                         Debian TeX Group
gpg DSA: 0x09C5B094      fp: 14DF 2E6C 0307 BE6D AD76  A9C0 D2BF 4AA3 09C5 B094
-------------------------------------------------------------------------------
SHIFNAL (n.,vb.)
An awkward shuffling walk caused by two or more people in a hurry
accidentally getting into the same segment of revolving door. A
similar effect is achieved by people entering three-legged races
unwisely joined at the neck instead of the ankles.
			--- Douglas Adams, The Meaning of Liff