Representational error in binary and decimal numbering systems

The representation of a general rational number of the form A/B as a floating point number requires a conversion from the general form to a base specific form. This conversion often results in the generation of infinitely repeating non-zero strings of digits which are truncated to the size of the mantissa resulting in a loss of precision. It is shown that the proportion of repeating versus finite rational numbers specific to a base is expotentially related to the number of unique prime factors of the base. Simulation results are presented which show the relative proportions of finite representations for binary and decimal cases over a range of mantissa sizes. The representation of rational numbers in computer systems is typically implemented by modified forms of scientific notations that are referred to as floating point representations. That is, all rational numbers in general fractional form are converted to a rational number of a default base and stored as a mantissa of fixed precision scaled by a power of the base. In this form the denominator need not be explicity represented or manipulated. This simplification limits the computational overhead and extends the range of the representation at the price of precision. Generally the normalization and base of such numbers are assumed to be a default understood by the algorithms which manipulate them. These systems of floating point representation can be grouped into P e r m i s s i o n to copy w i t h o u t f e e a l l o r p a r t o f t h i s m a t e r i a l i s g r a n t e d p r o v i d e d t h a t t h e c o p i e s are not made or d i s t r i b u t e d f o r d i r e c t c o m m e r c i a l a d v a n t a g e , t h e ACM c o p y r i g h t n o t i c e and the t i t l e o f t h e p u b l i c a t i o n and i t s d a t e appear , and n o t i c e i s g i v e n t h a t c o p y i n g i s by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. 1982 ACM 0-89791-071-0/82/0400-0085 $00.75 three distinct categories: a binary mantissa and exponent a binary encodirg of decimal digits a binary encoding of a decimal mantissa Of these representations the first is a true base two scientific notation and has been the choice of the overwhelming majority of floating point systems. The remaining methods rely on only binary representation as a media for storage and representation of values. That is, their algorithms perform actual decimal manipulations. Therefore values represented in these methods are given binary encoding for rational numbers in decimal form. All of these representations require that rational numbers in fractional form be converted to a rational number in the appropriate base. To express a general rational number as a base-specific rational number requires conversion of the original fraction to a form in which the denominator is some power of the base of representation. For example, we typically convert the fraction 1/2 to 5/10 or its equivalent form . 5 to express it within a floating point representation. This power is saved as the "exponent" in floating point forms. The key problem in these systems of representation is that not all possible fractions (in fact a very limited subset) can be converted to a base specific representation with a finite numerator. For example, the rational number 1/3 (i/ii binary) cannot be represented in either decimal or binary systems with a finite numerator over a power of the base. Floating point systems of representation must therefore truncate the least significant digits of the numerator to fit within the finite size of the mantissa The result is a loss of significance and imprecision which is introduced into subsequent computations. It follows that this same phenomena (i.e. truncation of This work was funded in part by the Academic Grant Fund of Loyola University, New Orleans.