[tlbuild] wget in TL now needs https

[Mojca Miklavec]

On Wed, 21 Apr 2021 at 23:16, Karl Berry wrote:
>
> As of now, the CTAN mirror list contains only https mirrors, not http.
> This was necessary in order to cater to (broken/stupid) Chrome behavior.

Chrome has announced gradually breaking HTTP-related stuff a long time
ago already.

The question is whether we prefer a broken browser or a broken TeX
Live installation.

> Therefore our wget binaries provided in TL now need to support https to
> be useful. I know that several (all?) do not, because https is such a
> mess to compile, and induces so many shared lib dependencies.

I strongly suspect that the one by Akira might just be the only one.

Now, luckily, I guess that recent versions of macOS can use the system
curl, and Linux mostly has no issues as one can simply use the system
wget, which covers the great majority of users.

I guess that for Solaris and older OS X versions I would simply add
something like "please install wget via OpenCSW / MacPorts" to the
installation instructions. I'm anything but eager to compile a fully
functional wget.

If we ever want to support SSL from the standalone wget binaries, then
TeX Live also needs to ship the list of trusted certificate CA files
(which is something that generally needs regular updates, and someone
needs to keep that list up to date). Who would be taking care of
those?

> Nevertheless, we have no choice.

I still believe that CTAN could provide two separate lists of mirrors.
One for https and the other legacy one for http. Then tlmgr could
fetch files from one of the http mirrors in case the available wget
wouldn't support SSL.

> I don't know if it's practical to compile wget statically, but of course
> that would be ideal. Otherwise, it's ok to compile it with shared libs
> for whatever OS version makes the most sense.

What does it mean "with shared libs"? Installing MacPorts, linking
against MacPorts libraries when building wget and then uploading such
a binary that will be broken for anyone but those who have MacPorts
installed, with libraries at the exact same version as was current at
the time when wget was compiled?

> A system-provided binary is
> preferred (by default) in any case.
>
> If wget cannot be feasibly compiled with ssl support at all on a given
> platform, we should just remove it from TL, since it won't be useful.
> Let me know.

It cannot be feasibly compiled on most of the platforms. Now, I would
say that you can probably easily remove the ones for freebsd,
universal-darwin (don't do that just yet), ... as there are really
simple alternatives. But say for Mac OS X 10.6 and Solaris there is no
really simple alternative other than installing a third-party package
manager. Now the users can at least still manually override the
preferred mirror themselves and put in the one that still serves
contents without enforcing SSL. If you remove wget, there will be
pretty much nearly zero choices left.

> I tweaked the sample configuration invocation at
>   https://tug.org/texlive/build.html#wget
> to remove the --without-ssl, since that's clearly not what we want any
> more, but I'm sure more options are needed in practice. Let me know.
>
> I wonder if curl would make ssl support any easier. I kind of doubt it,
> but I guess we could go that way if it's useful.

No.

> Sorry, but such is life nowadays. --thanks, karl.
>
> Here is the list of wget binaries currently provided:
>
> wget.amd64-freebsd wget.i386-freebsd

SSL support is disabled.
wget may be installed from the official ports though, which shouldn't
be too difficult to do.

> wget.i386-solaris  wget.x86_64-solaris

SSL support is disabled.
A third-party package manager provides wget (but I'm not sure how many
people have it installed; Nelson doesn't, for example)
https://www.opencsw.org/packages/wget/

> wget.x86_64-darwinlegacy

SSL support is disabled.
One can install wget via a third-party package manager (MacPorts).

If I understood correctly, tlmgr can also use the system curl, but
that one doesn't work with SSL on all the systems that we try to
support (at least it should work with the recent macOS version
though).

Mojca