[tldistro] Experimental mapping of texlive packages to rpm packages
pcpa at mandriva.com.br
pcpa at mandriva.com.br
Mon Nov 7 16:37:24 CET 2011
> Hi Paulo,
Hi Norbert,
> On Mi, 02 Nov 2011, pcpa at mandriva.com.br wrote:
>> > THis is what I am currently doing in Debian (in preparation).
>>
>> Do you have some prototype, or more specifically, what I have interest
>> is in, how do you manage the config files updates and ls-R.
>
> That will not change and is more or less the same since 2005 (!!!)
> when I packaged the first version of TL for Debian. ANd actually,
> if you are serious about packaging, there is not many other ways.
>
> Let us the few important things in turn:
> - ls-R files
> - updmap-sys/updmap.cfg
> - fmtutil-sys/fmtutil.cnf
> - fmtutil-sys/language.dat/def/lua.def
Testing that all texlive-scheme-* install works, after installing
texlive-scheme-full I have:
$ l /usr/share/tlpkg/*.d
/usr/share/tlpkg/fmtutil.cnf.d:
aleph cslatex jadetex luatex mltex ptex xmltex
context csplain latex-bin mex pdftex xetex
/usr/share/tlpkg/language.dat.d:
dehyph-exptl hyphen-english hyphen-interlingua hyphen-sanskrit
hyphen-afrikaans hyphen-esperanto hyphen-irish hyphen-serbian
hyphen-ancientgreek hyphen-estonian hyphen-italian hyphen-slovak
hyphen-arabic hyphen-ethiopic hyphen-kurmanji hyphen-slovenian
hyphen-armenian hyphen-farsi hyphen-lao hyphen-spanish
hyphen-basque hyphen-finnish hyphen-latin hyphen-swedish
hyphen-bulgarian hyphen-french hyphen-latvian hyphen-turkish
hyphen-catalan hyphen-galician hyphen-lithuanian hyphen-turkmen
hyphen-chinese hyphen-german hyphen-mongolian hyphen-ukrainian
hyphen-coptic hyphen-greek hyphen-norwegian hyphen-uppersorbian
hyphen-croatian hyphen-hungarian hyphen-polish hyphen-welsh
hyphen-czech hyphen-icelandic hyphen-portuguese
hyphen-danish hyphen-indic hyphen-romanian
hyphen-dutch hyphen-indonesian hyphen-russian
/usr/share/tlpkg/language.def.d:
dehyph-exptl hyphen-english hyphen-interlingua hyphen-sanskrit
hyphen-afrikaans hyphen-esperanto hyphen-irish hyphen-serbian
hyphen-ancientgreek hyphen-estonian hyphen-italian hyphen-slovak
hyphen-arabic hyphen-ethiopic hyphen-kurmanji hyphen-slovenian
hyphen-armenian hyphen-farsi hyphen-lao hyphen-spanish
hyphen-basque hyphen-finnish hyphen-latin hyphen-swedish
hyphen-bulgarian hyphen-french hyphen-latvian hyphen-turkish
hyphen-catalan hyphen-galician hyphen-lithuanian hyphen-turkmen
hyphen-chinese hyphen-german hyphen-mongolian hyphen-ukrainian
hyphen-coptic hyphen-greek hyphen-norwegian hyphen-uppersorbian
hyphen-croatian hyphen-hungarian hyphen-polish hyphen-welsh
hyphen-czech hyphen-icelandic hyphen-portuguese
hyphen-danish hyphen-indic hyphen-romanian
hyphen-dutch hyphen-indonesian hyphen-russian
/usr/share/tlpkg/language.lua.d:
dehyph-exptl hyphen-english hyphen-interlingua hyphen-sanskrit
hyphen-afrikaans hyphen-esperanto hyphen-irish hyphen-serbian
hyphen-ancientgreek hyphen-estonian hyphen-italian hyphen-slovak
hyphen-arabic hyphen-ethiopic hyphen-kurmanji hyphen-slovenian
hyphen-armenian hyphen-farsi hyphen-lao hyphen-spanish
hyphen-basque hyphen-finnish hyphen-latin hyphen-swedish
hyphen-bulgarian hyphen-french hyphen-latvian hyphen-turkish
hyphen-catalan hyphen-galician hyphen-lithuanian hyphen-turkmen
hyphen-chinese hyphen-german hyphen-mongolian hyphen-ukrainian
hyphen-coptic hyphen-greek hyphen-norwegian hyphen-uppersorbian
hyphen-croatian hyphen-hungarian hyphen-polish hyphen-welsh
hyphen-czech hyphen-icelandic hyphen-portuguese
hyphen-danish hyphen-indic hyphen-romanian
hyphen-dutch hyphen-indonesian hyphen-russian
/usr/share/tlpkg/updmap.cfg.d:
adfsymbols cm-super initials psnfss venturisadf
amsfonts cs mathdesign ptex vntex
arphic fourier pl tex-gyre wadalab
And contents of those files are contents of the equivalent tlpobj
fmtutil_cnf_lines, updmap_cfg_lines, language_dat_lines, language_def_lines
or language_lua_lines. The file generation uses the same logic as the
call to TeXLive::TLUtils related function from install-tl, but in this
case, the post script does it without creating/loading a texlive.tlpdb
database in perl.
Most post scripts follow the same pattern, for example:
$ cat /usr/sbin/updmap.post
#!/bin/sh
(
flock -n 9 || exit 1
if [ -f /var/run/updmap ]; then
N=`cat /var/run/updmap`
N=`expr $N - 1`
else
N=0
fi
if [ $N -lt 1 ]; then
(
cat /usr/share/texmf/web2c/updmap-hdr.cfg
for file in `find /usr/share/tlpkg/updmap.cfg.d/ -type f`; do
cat $file
done
if [ -f /usr/share/texmf-local/web2c/updmap-local.cfg ]; then
cat /usr/share/texmf-local/web2c/updmap-local.cfg
fi
) > /etc/texmf/web2c/updmap.cfg
/usr/bin/updmap-sys --syncwithtrees > /dev/null
rm -f /var/run/updmap
else
echo $N > /var/run/updmap
fi
) 9>/var/run/updmap.lock
As anything that relies on external/shared state, there are failure
conditions, and so far the only automatic handling for that is the
"virtual" texlive package, that rescues the state if one "kill -9"
or something else a texlive update/install and starts over.
> All these files depend on the actually installed packages. So you
> should never never use the pre-shipped files (updmap.cfg, ...), because
> they refer to *all* fonts in a full installation.
Yes, I did that only in the very first prototype of the rework of the
packages :-)
> But if you want to allow for smooth upgrades etc, and partial installs,
> you cannot do that.
>
> So the Debian way for these files is:
> - at package build time we knoe which formats/font maps/hypehn patterns
> there are, and create snippets for these configuration file, and
> at install time put them into /etc/texmf/updmap.d, fmtutil.d, hyphen.d
> - after the package has been installed the configuration routine does:
> . assemble the pieces together to updmap.cfg/fmtutil.cnf/language.*
> . run the necessary commands (also determined at package *build* time,
> that is updmap-sys, fmtutil --whatever, etc)
>
> Furthermore, in Debian there is another requirement, namely that
> if a sysadm changes confiurations files (hereafter called conffiles)
> in /etc, and the package is only removed (that is, the conffiles are
> *NOT* deleted) but NOT purged (purged = also conffiles removed),
> then a later installation of the package has to honor the changes the
> sysadm made.
I can use a similar approach with rpm attributes like %config, %ghost,
etc. But at first, I think I will stick with the logic in install-tl
that is to use files in TEXMFLOCAL. But I agree that the usage of
/usr/share for config files is not wise...
$ egrep "TEXMFLOCAL\s+=" /usr/share/texmf/web2c/*
/usr/share/texmf/web2c/texmf.cnf:TEXMFLOCAL = /usr/share/texmf-local
/usr/share/texmf/web2c/texmfcnf.lua: TEXMFLOCAL =
"/usr/share/texmf-local",
> That means, there will remain snippets in /etc/updmap.d (for example)
> if a font package has been removed, but not purged. THese snippets
> should of course NOT be merged into the main config file. So we ship
> in addition a file /var/lib/texmf/fmt.d/.... and check at assemble
> time if this file is present. If it is present, then the snippet
> is used, if not, then it is NOT used.
> (BTW, this also happens during partial upgrades, so if you are serious
> about packaging, think about that, too).
I coded enough logic for partial upgrades and removal of single
packages, but support for keeping changes to /usr/share/tlpkg/*.d
files is not provided (as said above, should use an extra file
that is added at the bottom of the generated configuration file).
> Well, that is the story at least till 2007. In later version of the Debian
> packages we are now something called "triggers", where a central package
> (in our case "tex-common") shows interest in a certain directory, namely
> the directories /etc/texmf/fmtutil.d etc (all of them), and if a file
> is dropped there or changed there, only some specific code in tex-common
> is executed. THis way we can reduce the calls to mktexlsr and updmap-sys
> a *LOT* (think about installing every collection of TeX Live, and run
> mktexlsr, fmtutil-sys --all, updmap-sys after *EACH* collection, versus
> running it only once or twice!). But this is Debian specific, and probably
> no other distribution has a similar system (although very useful).
It should be possible to run only once or twice in the schema I am
working on, but would need to add support to external tools, so, for
now it should be done per transaction, and the amount of times it runs
is based on how many transactions are done. Default urpmi transaction
size is like 10 packages, unless there is some complex interdependecy
that obligates a longer transaction, so, it should run the post scripts
average every 10 packages installed. If installing from scratch, that
should be like 40-50 running mktexlsr.
> Puuuuh, long email. BTW, there are loads of documents in the tex-common
> package of Debian: http://packages.debian.org/sid/tex-common, especially
> the main documents Debian-TeX-Policy and TeX-on-Debian. Although a bit
> outdated, they describe the reality quite closely. Get the .deb and
> unpack it with ar ;-)
>
>> in the same "transaction", it runs mktexlsr only once.
>
> Good idea, but not sooo important, because after the first run
> normally all the files are in the file system cache of the kernel,
> and running mktexlsr once more is fast.
>
>> installing files in a "config.d" directory, and regenerating
>> the files somewhat like the same way install-tl does, just that
>> in that case, not using the texlive perl modules, e.g.:
>
> Sounds like the right approach.
>
>> $ cat /usr/share/tlpkg/Mandriva/fmtutil.cnf.d/aleph
>> aleph aleph \- \*aleph.ini
>> lamed aleph language.dat \*lambda.ini
>>
>> $ cat /usr/share/tlpkg/Mandriva/updmap.cfg.d/adfsymbols
>> Map ArrowsADF.map
>> Map BulletsADF.map
>>
>> $ cat /usr/share/tlpkg/Mandriva/language.dat.d/dehyph-exptl
>> % from dehyph-exptl:
>> german-x-2011-07-01 dehypht-x-2011-07-01.tex
>> =german-x-latest
>> ngerman-x-2011-07-01 dehyphn-x-2011-07-01.tex
>> =ngerman-x-latest
>
> Yes, this is the way to go.
>
> You probably have the advantage that you don't have to care for
> sysadm changes, as you don't allow them ;-))) Sometimes I wish
> I am not on Debian, which is soo conservative in keeping changes
> of sysadmins ;-))))
>
>> Another question, how do you plan to manage updates? I did
>> initial work not using the "revision" metadata in the tlpobj
>> files, but probably the best approach to automate some tool
>> do know when there are updates. And my current plan before
>> actually starting using the new approach is to write some
>> script to handle it, so that, from time to time can check
>> what has been updated.
>
> I will have debina packages of the form
> texlive-<collection-name>
> version: 2011.YYYY.MM.DD-d
> where YYYY.MM.DD is the tlnet of that day, and -d is the Debian version.
>
> THis way I can update whenever I want. I will not do it on a daily
> bases.
I wrote a script that reads a texlive.tlpdb file, and with a
checkout of the texlive packages, it quickly tells what is outdated,
what have changed or deprecated. I am working for several days now
in updating to have an "almost" one texlive package == one rpm package,
for example, right now checking updates (with a possibly outdated
texlive.tlpdb):
$ pwd
/home/pcpa/mandriva/svn/texlive-tlpkg
$ SOURCES/checkupdates.pl
texlive-ipaex: revision changed
texlive-tetex: revision changed
texlive-kdgreek: not in texlive.tlpdb
the script is
http://svn.mandriva.com/viewvc/packages/cooker/texlive-tlpkg/current/SOURCES/checkupdates.pl?view=markup
> Furthermore, in Debian we only have 5 source packages, and for every
> collection one binary package (all source packages build several binary
> packages).
>
> If I could have one TL package -> one Debian package, that would be *easy*
> and *nice*, but our ftpmaster (the guard of the holy grail) will not
> allow to enter 2000+ packages ;-)
Pssst, I am doing that right now in Mandriva :-)
> Best wishes
You too, and many thanks for the very valuable information. I should
have it in Mandriva cooker in the next few days. I have been already
using it for 2 weeks and making several tests. Just needs some time
to get built/processed by the build system.
> Norbert
> ------------------------------------------------------------------------
> Norbert Preining preining@{jaist.ac.jp, logic.at, debian.org}
> JAIST, Japan TeX Live & Debian Developer
> DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094
> ------------------------------------------------------------------------
> BILBSTER
> A pimple so hideous and enormous that you have to cover it with
> sticking plaster and pretend you've cut yourself shaving.
> --- Douglas Adams, The Meaning of Liff
Paulo
More information about the tldistro
mailing list