[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Round-off errors in metrics & fontinst/finst status...
>At 04:36 PM 97/12/17 -0800, Melissa O'Neill wrote:
>How serious is this? Well, the error of rounding obviously lies
>uniformly distributed in the [-0.5, 0.5] range.
Not obvious: does the software really round, or does it truncate? (I'll
assume it does proper rounding, because there's no excuse. But some
programmers are very lazy.)
> The average character
>width in CMR10 is 586 (on the 1000 per em scale). So the error is
>of the order of +/- 0.1% In 150mm we may see in the worst case
>+/- 0.15mm. Is that noticeable? Probably. Of course in the typical
>situations lots of positive errors of varying size are added to lots of
>negative errors of various size and so the overall error is less.
Assuming that the error is evenly distributed, the error will be *very*
much less. It'd take some horrible stats to work out what you could expect
normally - I'd guess that one order of magnitude less is about right.
So the typical error would be about 15 microns in 150 mm, which probably
>The time it does add up systematically is when one is using the same
>character repeatedly (as on a fill) or in a fixed width font. Note that the
>error can be worse than 0.15mm if the character is narrower than 586.
And it can be less if the rounding is less than 0.5 - this 0.15mm is
more-or-less worst-case. The rms rounding error is something like 0.3, so
the typical error with a repeated character will be nearer 0.05%, or 75
microns in 150mm; typical error with random characters will be nearer 7.5
microns. I'm not inclined to fret over errors of less than 0.1mm across a