[luatex] incompatible change to mathcode

[David Carlisle]

On 09/05/2012 09:16, Taco Hoekwater wrote:
>
> Hi all,
>
> On 05/08/2012 09:11 PM, David Carlisle wrote:
Taco thanks for CC'ing me, saves me waiting til it appears in the
archives:-)

>>
>> Luatex has changed mathcode to allow a larger numeric range however
>> changing TeX primitives without giving them new names seems suspect
>> and in this particular case, since \mathcode has changed but
>> \mathchardef has not the essential link between these two commands
>> has been broken.
>
> Not quite. You get the results below because the math code for `a
> has been set using \Umathcode, not the TeX82 \mathcode. In fact,
> assignment to \mathcode is not extended in LuaTeX at all.

Ah! that explains why the number changed, that bit I couldn't understand
at all.

>
> The output of \mathcode is indeed extended, and the value you get
> uses the same format as the input for \Umathcodenum (with that
> somewhat unusual XeTeX-invented format explained by Kazuki Maeda)
I missed some of the thread, I'll catch up on the archives and then
come back to this email.....

.
>
> The reason for this is that math character objects in luatex remember
> how they were defined.

> So if you do not want incompatibilities with TeX82, it would help if
> you did not use extended primitives to define stuff.

Yes but as you know, there isn't a single "you" involved here. If bm (or
amsmath) or any other ancient bit of code that's using \mathcode gets
used in lualatex, then the person who wrote the package and the person
using the package are typically not the person who set up the mathcode
of the letter a. So you are asking three people doing work decades apart
to cooperate:-)


>
> That said, probably the result of (\the)\mathcode can be reverted
> back to compatibility with TeX82, and output a warning (and zero) if
> the actual value is out-of-range (like XeTeX does).

I think that would be an improvement, code using \mathcode-returned
values just isn't expecting this extended format.
>
> Anyway, please do not panic and please all don't get your panties in
> a bunch.

Oh don't worry about me, I'm quite used to taking a long view of these
things so long as things get sorted out in the next decade or so that'll
be fine:-)


> The world is still just as safe a place to live in with this luatex
> bug in it. And actually, it helps if these bugs are reported in the
> tracker ;)

Well yes although it isn't really a bug (in that it was presumably
implemented as designed) rather than (I claim) a questionable design
decision when you take legacy code into account. Which is why I raised
it on this discussion list rather than as a bug.

>
> Best wishes, Taco

Kazuki Maeda wrote

>
>
> \luatexUmathcode`a="7"1"61 \the\mathcode`a            % 31457377 =
> 1*2^24 + 7*2^21 + 97

>
> I hope this helps you.

Thanks, it helps me (so I understand the number) I'll have to see if I
can put that understanding into bm. (What bm is trying to do is inspect
the mathcode of a letter, take it apart and then construct a mathcode
for a token that has the same mathclass and character code but uses the
fam number for a bold font. (actually for unicode fonts there would be
another possibility to get the bold glyph from the plane 1 math
alphabetic character range in the same font).

Given that bm has been completely stable for 20 years, it's not that big
a deal to have to update it for the unicode age, but it would be good
(for bm, and for every other math package author) if that only needs to
be done once. My main concern was that was not the case. But actually if
I understand the replies correctly I should be able to share the code
for xelatex and lualatex so long as I avoid \mathcode once I have
detected either of them?

David



________________________________________________________________________
The Numerical Algorithms Group Ltd is a company registered in England
and Wales with company number 1249803. The registered office is:
Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.

This e-mail has been scanned for all viruses by Star. The service is
powered by MessageLabs. 
________________________________________________________________________