Numerical Methods Contents - SAM

10 for n=N
11 % Initialize dense vectors a, b, x (column vectors!)
12 a = ( 1 : n ) ’ ; b = ( n: −1:1) ’ ; x = rand ( n , 1 ) ;
13
14 % Measuring times using MATLAB tic-toc commands
15 t f o o l = 1000; for i =1: nruns , t i c ; y = ( a∗b ’ ) ∗x ; t f o o l =
min ( t f o o l , toc ) ; end ;
16 tsmart = 1000; for i =1: nruns , t i c ; y = a∗dot ( b ’ , x ) ; tsmart =
min ( tsmart , toc ) ; end ;
17 times = [ times ; n , t f o o l , tsmart ] ;
18 end
19
20 % log-scale plot for investigation of asymptotic complexity
21 figure ( ’name ’ , ’ d o t t e n s t i m i n g ’ ) ;
22 loglog ( times ( : , 1 ) , times ( : , 2 ) , ’m+ ’ , . . .
23 times ( : , 1 ) , times ( : , 3 ) , ’ r ∗ ’ , . . .
24 times ( : , 1 ) , times ( : , 1 ) ∗times ( 1 , 3 ) / times ( 1 , 1 ) , ’ k−’ , . . .
25 times ( : , 1 ) , ( times ( : , 1 ) . ^ 2 ) ∗times ( 2 , 2 ) / ( times ( 2 , 1 ) ^2) , ’ b−−’ ) ;
26 xlabel ( ’ { \ b f problem size n } ’ , ’ f o n t s i z e ’ ,14) ;
27 ylabel ( ’ { \ b f average runtime ( s ) } ’ , ’ f o n t s i z e ’ ,14) ;
28 t i t l e ( ’ t i c −toc timing , mininum over 10 runs ’ ) ;
29 legend ( ’ slow e v a l u a t i o n ’ , ’ e f f i c i e n t e v a l u a t i o n ’ , . . .
30 ’O( n ) ’ , ’O( n ^2) ’ , ’ l o c a t i o n ’ , ’ northwest ’ ) ;
Ôº ½º¿
31 p r i n t −depsc2 ’ . . / PICTURES/ d o t t e n s t i m i n g . eps ’ ;
Remark 1.3.5 (Relevance of asymptotic complexity).
Runtimes in Ex. 1.3.2 illustrate that the
asymptotic complexity of an algorithm need not be closely correlated with its overall runtime on
a particular platform,
because on modern computer architectures with multi-level memory hierarchies the memory access
pattern may be more important for efficiency than the mere number of floating point operations, see
[25].
Then, why do we pay so much attention to asymptotic complexity in this course ?
☛ To a certain extent, the asymptotic complexity allows to predict the dependence of the runtime
of a particular implementation of an algorithm on the problem size (for large problems). For
instance, an algorithm with asymptotic complexity O(n 2 ) is likely to take 4× as much time when
the problem size is doubled.
△
1.4 BLAS
Ôº ½º
BLAS = basic linear algebra subroutines
Remark 1.3.4 (Reading off complexity).
Available: “Measurements” t i = t i (n i ) for different n 1 ,n 2 ,...,n N , n i ∈ N
✸
BLAS provides a library of routines with standardized (FORTRAN 77 style) interfaces. These routines
have been implemented efficiently on various platforms and operating systems.
Grouping of BLAS routines (“levels”) according to asymptotic complexity, see [18, Sect. 1.1.12]:
Conjectured: Algebraic dependence t i = Cn α i , α ∈ R
t i = Cn α i ⇒ log(t i ) = log C + α log(n i ) , i = 1,...,N .
If the conjecture holds true, then the points (n i ,t i ) will lie on a straight line with slope α in a
doubly logarithmic plot.
➣ quick “visual test” of conjectured asymptotic complexity
More rigorous: Perform linear regression on (log n i , log t i ), i = 1, ...,N (→ Ch. 6)
△
Ôº ½º¿
• Level 1: vector operations such as scalar products and vector norms.
asymptotic complexity O(n), (with n ˆ= vector length),
e.g.: dot product: ρ = x T y
• Level 2: vector-matrix operations such as matrix-vector multiplications.
asymptotic complexity O(mn),(with (m, n) ˆ= matrix size),
e.g.: matrix×vector multiplication: y = αAx + βy
• Level 3: matrix operations such as matrix additions or multiplications.
asymptotic complexity O(nmk),(with (n,m, k) ˆ= matrix sizes),
e.g.: matrix product: C = AB
Ôº ½º