SSL II USER'S GUIDE - Lahey Computer Systems

2.9 UNIT ROUND OFF
SSL II subroutines frequently use the unit round off.
The unit round off is a basic concept in error analysis
for floating point arithmetic.
Definition
The unit round off of floating-point arithmetic are
defined as follows:
u = M 1-L / 2, for (correctly) rounded arithmetic
u = M 1-L , for chopped arithmetic,
where M is the base for a floating-point number system,
and L is the number of digits used to hold the mantissa.
In SSL II, the unit round off is used for convergence
criterion or testing loss of significant figures.
Error analysis for floating point arithmetic is covered in
the following references:
[1] Yamashita, S.
On the Error Estimation in Floating-point
Arithmetic
Information Processing in Japan Vol. 15, PP.935-
939, 1974
[2] Wilkinson, J.H.
Rounding Errors in Algebraic Process
Her Britannic Majesty’s Stationery Office,
London 1963
GENERAL RULES
2.10 ACCUMULATION OF SUMS
Accumulation of sums is often used in numerical
calculations. For instance, it occurs in solving a system
of linear equations as sum of products, and in
calculations of various vector operations.
On the theory of error analysis for floating point
arithmetic, in order to preserve the significant figures
during the operation, it is important that accumulation of
sums must be computed as exactly as possible.
As a rule, in SSL II the higher precision accumulation is
used to reduce the effect of round off errors.
2.11 Computer Constants
This manual uses symbols to express computer hardware
constants. The symbols are defined as follows:
• flmax: Positive maximum value for the floating-point
number system
(See AFMAX in Appendix A.)
• flmin: Positive minimum value for the floating-point
number system
(See AFMIN in Appendix A.)
• tmax: Upper limit of an argument for a trigonometric
function (sin and cos)
Upper limit of argument Application
Single
precision
8.23 x 10 5 FACOM M series
FACOM S series
Double
precision
3.53 × 10 15 SX/G 100/200 series
FM series
27