[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

*To*: ajeffrey@cs.depaul.edu, rebecca@astrid.u-net.com*Subject*: RE: rounding problems*From*: mackay@cs.washington.edu (Pierre MacKay)*Date*: Fri, 2 Oct 1998 21:38:52 -0700*Cc*: fontinst@cogs.susx.ac.uk

> What amazes me about TeX is that someone like Knuth could write something > that's as bad at arithmetic as TeX is. Wow. Have you tried to think back to all the forms of floating-point representation that were there in the late 70's. I don't know what the status of the IEEE format was then, but I can be pretty confident that even if it was already taking shape, it was certainly not reliably available on a range of systems. Moreover, there was the special problem of the 36-bit word in the processors used by the Dec10 and the Dec20. (They were pretty terrific machines in their day, and the Dec20 inspired Tops20, which is the only archaic operating system that still raises twinges of regret at its disappearance. Knuth insisted on integer-only arithmetic for the precise reason that he did not trust the floating-point behaviour even of his much-loved Dec10. The infinitesimal Scaled Point, which you can always get at if you really need it, provides access to enough precision for anything that really needs to be done. Yes, it is a bit harder to do floating-point calculations that way, but you know that the rounding errors are limited to what can be expressed in scaled points if you do it right. Consider the extraordinary job done by PicTeX. TeX log files do express certain values in floating-point notation, and the minor variants in trip and trap log files show that floating point can still be an uncertain format when real precision is needed. But TeX never does the calculation in floating-point, only the expression of the results. I find it hard to believe that the precision of scaled points is ever to coarse to solve any reasonable typesetting problem. Integer arithmetic always involves the risk of overflow, either into a pseudo-negative number (bad underlying code) or an arbitrary truncation. Knuth has showed how to tame that problem in Web coding in dvitype and numerous other places. You sometimes have to cascade calculations to avoid overflow, but at the speed of present-day processors, so what? I do calculations to three and four decimal places in TeX, and it sometimes means taking four steps for something that a floating point processor would do in one. But I know that the four decimal places will be the same on every processor if I accept that slight extra effort. Email: mackay@cs.washington.edu Pierre A. MacKay Smail: Department of Classics Emeritus Druid for 218 Denny Hall, Box 353110 Unix-flavored TeX University of Washington Seattle, WA 98195 (206) 543-2268 (Message recorder)

- Prev by Date:
**RE: rounding problems** - Next by Date:
**Re: rounding problems** - Prev by thread:
**RE: rounding problems** - Next by thread:
**Re: rounding problems** - Index(es):