[tlu] TLU critical update still faliing

[Bruno Voisin]

> On 18 Nov 2023, at 02:13, Norbert Preining <norbert at preining.info> wrote:
> 
> You can do
> ./update-tlmgr-latest.sh --noexec --keep
> this will generate a tmp directory in .
> Within that is a `runme` script that does the update, and a `master`
> directory that contains all the necessary files.
> I just confirmed that xz for universal-darwin is there.

Since extraction doesn't work, I went pedestrian:

- Take update-tlmgr-latest.sh, delete the ascii preamble, ie everything up to and including line #500

	eval $finish; exit $res

- Change the extension to .gzip, double-click.

This gives exactly the structure you describe: a script runme.sh and a directory "master" containing all the required bits (including the xz binaries).

> I suggest that you run the runme script with 
> bash -x runme --debug
> collecting all output and send it here.


% sudo bash -x runme.sh --debug
Password:
+ creation_date='Thu Nov 16 02:12:22 CET 2023'
+ NETRELEASE=2023
+ MINRELEASE=2016
+ infrapkg=texlive.infra
++ kpsewhich --var-value=SELFAUTOPARENT
+ ROOT=/usr/local/texlive/2023
+ test -r /usr/local/texlive/2023/tlpkg/texlive.tlpdb
+ test -d /usr/local/texlive/2023/tlpkg/tlpobj/
+ echo 'runme.sh: updating in /usr/local/texlive/2023...'
runme.sh: updating in /usr/local/texlive/2023...
++ tlmgr version
++ grep '^TeX Live.*version'
++ awk '{print $NF}'
+ tlmgrversion=2023
+ test -z 2023
+ echo 'runme.sh: tlmgr version says this is TeX Live 2023,'
runme.sh: tlmgr version says this is TeX Live 2023,
+ echo 'runme.sh: and this updater script created: Thu Nov 16 02:12:22 CET 2023.'
runme.sh: and this updater script created: Thu Nov 16 02:12:22 CET 2023.
+ test x--debug = x--debug
+ shift
+ opt_debug=1
+ set -vx
+ test x = x--upgrade
+ echo 'runme.sh: proceeding with tlmgr update.'
runme.sh: proceeding with tlmgr update.
+ mv ./master/bin .
+ mkdir ./installer
+ mv ./master/tlpkg/installer/tar.exe ./installer
+ mv ./master/tlpkg/installer/xz ./installer
+ mv ./master/tlpkg/installer/wget ./installer
+ mv ./master/tlpkg/installer/lz4 ./installer
+ cd master
+ tar cf - LICENSE.CTAN LICENSE.TL README README.usergroups index.html readme-html.dir readme-txt.dir release-texlive.txt texmf-dist tlpkg
+ cd /usr/local/texlive/2023
+ tar xf -
+ t_instdir=/usr/local/texlive/2023/tlpkg/installer
+ t_xzdir=/usr/local/texlive/2023/tlpkg/installer/xz
+ t_wgetdir=/usr/local/texlive/2023/tlpkg/installer/wget
+ mkdir -p /usr/local/texlive/2023/tlpkg/installer/xz /usr/local/texlive/2023/tlpkg/installer/wget
+ src_tlpobj=master/tlpkg/tlpobj
+ tlpobjs=master/tlpkg/tlpobj/texlive.infra.tlpobj
+ test -n 1
pwd
++ pwd
+ : 'here (/Users/brunovoisin/Desktop/Test/Utah/update-tlmgr-latest):'
+ ls
bin installer master runme.sh
+ : installer:
+ ls installer
lz4 tar.exe wget xz
+ : installer/wget:
+ ls installer/wget
wget.amd64-freebsd wget.i386-freebsd wget.x86_64-darwinlegacy
wget.amd64-netbsd wget.i386-netbsd wget.x86_64-linuxmusl
wget.exe wget.i386-solaris wget.x86_64-solaris
+ : installer/xz:
+ ls installer/xz
xz.aarch64-linux xz.armhf-linux xz.i386-linux xz.universal-darwin xz.x86_64-linux
xz.amd64-freebsd xz.exe xz.i386-netbsd xz.x86_64-cygwin.exe xz.x86_64-linuxmusl
xz.amd64-netbsd xz.i386-freebsd xz.i386-solaris xz.x86_64-darwinlegacy xz.x86_64-solaris
+ : ROOT/bin=/usr/local/texlive/2023/bin:
+ ls /usr/local/texlive/2023/bin
universal-darwin
+ for a in '"$ROOT"/bin/*'
+ test -d /usr/local/texlive/2023/bin/universal-darwin
basename "$a"
++ basename /usr/local/texlive/2023/bin/universal-darwin
+ b=universal-darwin
+ test universal-darwin = man
+ echo 'runme.sh: updating /usr/local/texlive/2023/bin/universal-darwin ...'
runme.sh: updating /usr/local/texlive/2023/bin/universal-darwin ...
+ tlpobjs='master/tlpkg/tlpobj/texlive.infra.tlpobj master/tlpkg/tlpobj/texlive.infra.universal-darwin.tlpobj'
+ cd bin
+ tar cf - universal-darwin
+ cd /usr/local/texlive/2023/bin
+ tar xf -
+ test -r installer/xz/xz.universal-darwin
+ cp installer/xz/xz.universal-darwin /usr/local/texlive/2023/tlpkg/installer/xz/
+ test -r installer/wget/wget.universal-darwin

# Invoke special tlmgr action with the tlpobjs we found.
# Hopefully the result will be a clean tlpdb state.
# We are running the new tlmgr, from our own archive, here,
# due to the untar above.
tlmgr_path=`which tlmgr 2>/dev/null`      # provide complete path if we can
which tlmgr 2>/dev/null
++ which tlmgr
+ tlmgr_path=/Library/TeX/texbin/tlmgr
test -z "$tlmgr_path" && tlmgr_path=tlmgr # but no biggie if we can't
+ test -z /Library/TeX/texbin/tlmgr
echo "$0: $tlmgr_path including objects: $tlpobjs"
+ echo 'runme.sh: /Library/TeX/texbin/tlmgr including objects: master/tlpkg/tlpobj/texlive.infra.tlpobj master/tlpkg/tlpobj/texlive.infra.universal-darwin.tlpobj'
runme.sh: /Library/TeX/texbin/tlmgr including objects: master/tlpkg/tlpobj/texlive.infra.tlpobj master/tlpkg/tlpobj/texlive.infra.universal-darwin.tlpobj
#debug: (pwd; ls -l $tlpobjs; ls -lR) >/tmp/ls
#debug: -vv here gives more info. (Also -vx on the #! line above.)
if tlmgr -v _include_tlpobj $tlpobjs; then :; else
  echo "$0: $tlmgr_path failed, goodbye." >&2
  exit 1
fi
+ tlmgr -v _include_tlpobj master/tlpkg/tlpobj/texlive.infra.tlpobj master/tlpkg/tlpobj/texlive.infra.universal-darwin.tlpobj
D:no translations available for en_fr (nor en_fr); falling back to English
D:appending to package log file: /usr/local/texlive/2023/texmf-var/web2c/tlmgr.log
D:appending to command log file: /usr/local/texlive/2023/texmf-var/web2c/tlmgr-commands.log
D:tlmgr:main: ::tldownload_server hash:: {enabled:1,errorcount:0,initcount:1,ua:LWP::UserAgent=HASH(0x13e1fce58)}
D:setup_programs: preferring system versions
D:program df found in path
D:program curl found in path
D:program wget not usable from path
D:(unix) trying to set up wget, default /usr/local/texlive/2023/tlpkg/installer/wget/wget.universal-darwin, arg --version
D:program lz4 not usable from path
D:(unix) trying to set up lz4, default /usr/local/texlive/2023/tlpkg/installer/lz4/lz4.universal-darwin, arg --version
D: Using shipped /usr/local/texlive/2023/tlpkg/installer/lz4/lz4.universal-darwin for lz4 (tested).
D:program gzip found in path
D:program xz not usable from path
D:(unix) trying to set up xz, default /usr/local/texlive/2023/tlpkg/installer/xz/xz.universal-darwin, arg --version
D: Using shipped /usr/local/texlive/2023/tlpkg/installer/xz/xz.universal-darwin for xz (tested).
+ :
echo "$0: done."
+ echo 'runme.sh: done.'
runme.sh: done.


So everything looks OK.

It seems the problem is in the extraction: running update-tlmgr-latest.sh gives in the TLU log

2023-11-17 21:09:47 +0000 Warning tlu_ipctask[9107] + ls installer
2023-11-17 21:09:47 +0000 Warning tlu_ipctask[9107] lz4
2023-11-17 21:09:47 +0000 Warning tlu_ipctask[9107] tar.exe
2023-11-17 21:09:47 +0000 Warning tlu_ipctask[9107] wget
2023-11-17 21:09:47 +0000 Warning tlu_ipctask[9107] xz
2023-11-17 21:09:47 +0000 Warning tlu_ipctask[9107] + : installer/wget:
2023-11-17 21:09:47 +0000 Warning tlu_ipctask[9107] + ls installer/wget
2023-11-17 21:09:47 +0000 Warning tlu_ipctask[9107] + : installer/xz:
2023-11-17 21:09:47 +0000 Warning tlu_ipctask[9107] + ls installer/xz

so the bin directories are empty, while extracting the archive manually as above then running runme.sh gives

+ ls installer
lz4 tar.exe wget xz
+ : installer/wget:
+ ls installer/wget
wget.amd64-freebsd wget.i386-freebsd wget.x86_64-darwinlegacy
wget.amd64-netbsd wget.i386-netbsd wget.x86_64-linuxmusl
wget.exe wget.i386-solaris wget.x86_64-solaris
+ : installer/xz:
+ ls installer/xz
xz.aarch64-linux xz.armhf-linux xz.i386-linux xz.universal-darwin xz.x86_64-linux
xz.amd64-freebsd xz.exe xz.i386-netbsd xz.x86_64-cygwin.exe xz.x86_64-linuxmusl
xz.amd64-netbsd xz.i386-freebsd xz.i386-solaris xz.x86_64-darwinlegacy xz.x86_64-solaris

so the bin directories are populated and everything's right afterwards.

My setup: M1 chip (MacBook Pro 13" from November 2020), macOS Sonoma 14.2 beta 3 (build 23C5047e).


> On 18 Nov 2023, at 03:41, Norbert Preining <norbert at preining.info> wrote:
> 
> Ok, it is not a few days. I checked, the last version of
> update-tlmgr-latest was from April, svn 66822
> 
> Something seems to have changed, because the old version (66822) still
> runs on the macs, but not the new ones.
> 
> I need to dig in and compare the two.

I can't figure out where to access previous versions of update-tlmgr-r*.sh to compare.

runme.sh refers to

https://tug.org/svn/texlive/trunk/Master/tlpkg/bin/tl-makeself-from-tlnet?revision=65994&view=markup

which says update-tlmgr-r*.sh is created by a cron job (hence I imagine recreated every night).

Are the previous versions archived somewhere?

It may very well be the problem is several months old, and we only see it now because (as far as I remember) TLU hasn't had to apply an infrastructure update for quite some time.


Bruno