A Practical Introduction to Data Structures and Algorithm Analysis

Sec. 3.11 Empirical Analysis 893.11 Empirical AnalysisThis chapter has focused on asymptotic analysis. This is an analytic tool, wherebywe model the key aspects of an algorithm to determine the growth rate of the algorithmas the input size grows. As pointed out previously, there are many limitationsto this approach. These include the effects at small problem size, determiningthe finer distinctions between algorithms with the same growth rate, and the inherentdifficulty of doing mathematical modeling for more complex problems.An alternative to analytical approaches are empirical approaches. The most obviousempirical approach is simply to run two competitors and see which performsbetter. In this way we might overcome the deficiencies of analytical approaches.Be warned that comparative timing of programs is a difficult business, oftensubject to experimental errors arising from uncontrolled factors (system load, thelanguage or compiler used, etc.). The most important point is not to be biased infavor of one of the programs. If you are biased, this is certain to be reflected inthe timings. One look at competing software or hardware vendors’ advertisementsshould convince you of this. The most common pitfall when writing two programsto compare their performance is that one receives more code-tuning effort than theother. As mentioned in Section 3.10, code tuning can often reduce running time bya factor of ten. If the running times for two programs differ by a constant factorregardless of input size (i.e., their growth rates are the same), then differences incode tuning might account for any difference in running time. Be suspicious ofempirical comparisons in this situation.Another approach to analysis is simulation. The idea of simulation is to modelthe problem with a computer program and then run it to get a result. In the contextof algorithm analysis, simulation is distinct from empirical comparison of twocompetitors because the purpose of the simulation is to perform analysis that mightotherwise be too difficult. A good example of this appears in Figure 9.8. This figureshows the cost for inserting or deleting a record from a hash table under twodifferent assumptions for the policy used to find a free slot in the table. The y axesis the cost in number of hash table slots evaluated, and the x axes is the percentageof slots in the table that are full. The mathematical equations for these curves canbe determined, but this is not so easy. A reasonable alternative is to write simplevariations on hashing. By timing the cost of the program for various loading conditions,it is not difficult to construct a plot similar to Figure 9.8. The purpose of thisanalysis is not to determine which approach to hashing is most efficient, so we arenot doing empirical comparison of hashing alternatives. Instead, the purpose is toanalyze the proper loading factor that would be used in an efficient hashing systemto balance time cost versus hash table size (space cost).