[tex-live] Primitive parity, \expanded and \Ucharcat

[Joseph Wright]

On 03/05/2018 22:38, Joseph Wright wrote:
> Hello all,
> 
> In adding features to expl3, the LaTeX team have been making use of a 
> variety of 'new' (post-e-TeX) 'utility' primitives in various engines. 
> Almost always these originate in pdfTeX and have migrated to other 
> engines, but are not in any way tied to PDF output, etc. Depending on 
> the exact engine in use, some or all of these primitives may be 
> unavailable, and that then limits macro-level features.
> 
> It seems sensible long-term to have cross-engine feature stay 'in sync' 
> with each other. In particular, (u)pTeX has picked up a number of pdfTeX 
> features, meaning that XeTeX often is the most 'limited' engine. The 
> team would like, if possible, to have a common feature set in all 
> engines in this regard. At the same time, there are a few 'bits and 
> pieces' that make sense to raise at the same time. I'll lay out the 
> various areas below.
> 
> Doing the work here is non-trivial, but luckily there is an automated 
> build system available via GitHub which is allowing us (me/David 
> Carlisle) to do some testing. I'm building up patches in various 
> branches at https://github.com/josephwright/texlive-source: assuming 
> these look good, I'll merge them as required and send diff files to 
> where/whoever is best. The branches on GitHub should hopefully have 
> clear names for what they address.
> 
> The areas we are keen to look at are as follows:
> 
> - 'pdfutils': (u)pTeX has picked up a number of pdfTeX primitives, and
>    a subset have made their way into XeTeX too. However, XeTeX is still
>    missing several, most notably an expandable RNG. We are part-way
>    though working out patches to add the rest to XeTeX (RNG is done,
>    file data and timer to do)
> 
> - banners: pdfTeX and LuaTeX have \<engine>banner, other engines lack
>    that. The banner includes TeX version and details of the TeX system,
>    so is potentially useful. Adding this to (u)pTeX/XeTeX looks
>    straight forward: still to-do.
> 
> - \expanded: This was slated for pdfTeX 1.50 but that has never
>    appeared, but the primitive is useful as it allows 'function-like'
>    expandable macros. We can see this begin very useful for simplifying
>    some macro code, and in many ways it feels like an e-TeX primitive.
>    The GitHub expanded branch adds it to pdfTeX/XeTeX/(u)pTeX
> 
> - Allowing \Ucharcat (XeTeX) to make \active tokens: this was raised
>    recently on the XeTeX list, but fits here as we've put a branch
>    together to show it works
> 
> It's likely I'll finish the outstanding patches by the weekend. Note 
> that at present each feature addition is in a separate Git branch, so to 
> add all of them I'll have to do a little tidying up: that will happen 
> once I know which of these suggestions are useful.
> 
> Feedback most welcome.
> 
> Joseph

Hello all,

Modulo any issues that show up in testing, all of the above is now done 
and on my GitHub fork 
(https://github.com/josephwright/texlive-source/tree/Ucharcat: this 
branch has 'all the stuff' on it).

I know that https://github.com/texjporg/tex-jp-build already has a 
branch for \expanded. What's the best way to request 'officially' that 
the changes go into pdfTeX/XeTeX? I can send a .diff to the pdfTeX dev 
list, and put in a pull request on SourceForge for XeTeX, if that's best.

Regards,

Joseph