[XeTeX] XeTeX and "ignore sub" substitution rules

[mskala at ansuz.sooke.bc.ca]

On Tue, 29 Nov 2011, Khaled Hosny wrote:
> The same here, but I get the expected result (aBa, abc) with LuaTeX as
> well as HarfBuzz. I tested with another application using ICU layout
> engine (fontmatrix) and got the same result as XeTeX, so I think it is
> an ICU bug.

On further testing, I don't think Apostolos's font works with XeTeX
either.  It may appear to at first glance, just because the effect of the
"ignore sub" rules in that font is very subtle, but if I modify the
alternate glyphs to be more obviously different, it's clear that they are
being put in in cases where the rules say they shouldn't.

I think I've also figured out just what XeTeX (presumably ICU) is doing
wrong:  it is failing to move the glyph pointer ahead on a successful
match.  As a result, later rules in the lookup still have the chance to
match again on the output of earlier rules, and "ignore sub" rules have
no effect.

I remember that you once commented that my "Terrible Secret" article was
wrong because I'd documented this behaviour, and this explains the
disagreement - I was documenting what I'd observed XeTeX to do, and at the
time, I hadn't tested "ignore sub" rules and didn't realize that it was
incorrect behaviour by XeTeX and would be a problem for "ignore sub"
rules.  Some of my own code both in that article and in my actual fonts
depends on the "later rules see output of earlier rules" behaviour and
will have to be fixed, but there's no help for that; it's more important
to have "ignore sub" work.

I will attempt to navigate ICU's bug tracking system and submit the bug to
them.  I don't know if XeTeX's practice is to track updates of ICU, though.
Unfortunately, it appears that in the short term I have to not only do
without "ignore sub," but also do without later rules seeing the output of
earlier rules, because I need my fonts to work both with widely-deployed
XeTeX and with correct implementations.
-- 
Matthew Skala
mskala at ansuz.sooke.bc.ca                 People before principles.
http://ansuz.sooke.bc.ca/