bug in fontinst 1.801: rounding problems (was: duplicated kernings)

[Lars Hellström]

On September 28, Vladimir Volovich wrote:
>And while we are at this topic, some more bug. :-)
>This time it can be called something like "rounding problems?".
>
>While looking into vpl files for ae fonts, i mentioned that some of
>them contain rather strange (unneeded) font references. E.g., the
>aer17.vpl contains the following stuff:
>
>---------------------------------------------
>(MAPFONT D 0 (FONTNAME cmr17) (FONTDSIZE R 17.28) (FONTAT R 17.2799))
>(MAPFONT D 1 (FONTNAME cmr17) (FONTDSIZE R 17.28) (FONTAT R 17.28))
>(MAPFONT D 2 (FONTNAME cmmi12) (FONTDSIZE R 12.0) (FONTAT R 13.82385))
>(MAPFONT D 3 (FONTNAME cmmi12) (FONTDSIZE R 12.0) (FONTAT R 17.28))
>(MAPFONT D 4 (FONTNAME cmsy10) (FONTDSIZE R 10.0) (FONTAT R 17.28))
>(MAPFONT D 5 (FONTNAME cmu10) (FONTDSIZE R 10.0) (FONTAT R 17.28))
>---------------------------------------------
>
>Note, that fonts number 0 and number 1 are *almost* the same, but
>differ by 0.0001 of `at size'. What is the reason of this? Obviously,
>this is a "bug", and the vpl file is overloaded with one unneeded (and
>incorrect) reference. The size and processing speed of the virtual
>font without this unneeded reference will improve. Is it possible to
>solve this problem as well? :-)

Last night, I came across something which probably is where the above
rounding error occurs. Recall the following piece of code in fontinst:

\def\vpl_scale#1#2{{
   \divide \scaled_design_size by 8
   \multiply \scaled_design_size by #1
   \divide \scaled_design_size by \one_thousand
   \multiply \scaled_design_size by 8
   #2
}}

What happens here if \scaled_design_size is 17.28pt and #1 is 1000? Let's
have a look!

*\dimen@=17.28pt \count@=\dimen@ \showthe\count@
> 1132462.

As we want to spot a small rounding error, we need to see exactly what
TeX's registers contain. In this case, TeX stores the dimension 17.28pt as
1132462sp. Next, we do what \vpl_scale would do, i.e.,

*\divide\dimen@ 8 \multiply\dimen@ 1000 \divide\dimen@ 1000 \multiply\dimen@ 8

and have a look at the result

*\count@=\dimen@ \showthe\dimen@ \showthe\count@
> 17.2799pt.

Exactly what Vladimir reported. A look at the exact representation shows
what has happened:
> 1132456.
The division and multiplication by 8 has truncated the dimension to an
integer multiple of 8sp.


Although this has (probably) shown where the rounding error occurs, it does
not show what has caused it. My guess is that there is an inconsistency
between the respective implementations of some mtx file commands in
fontinst, so that some will generate an explicit scaling with factor 1000
if given an (explicit or implicit) scaling factor of 1000, while others
will not generate any scaling at all in this case. I believe the search for
a fix should primarily be focused on finding this inconsistency.

The reason this error has not been noticed earlier is probably that it only
shows up for fonts at non-integer point sizes. I believe font metrics taken
from AFMs are always assigned a point size of 10pt, so the above problem
won't occur for them.

Lars Hellström