Permission to copy, without fee, all or part of this material ... - PNSQC

y the final result of the analysis. In general, you should use more than one collaborative analysis
technique when possible in order to use multiple results to cross-validate your analysis. If multiple
techniques yield the same conclusion, you gain some measure of confidence in your results. If multiple
techniques do not yield the same results then you are well advised to reexamine your assumptions and
try and determine why you are getting different results. The situations where the collaborative techniques
described in this paper are used are often quite subjective so your experience and intuition should play a
big part in interpreting your results. In other words, you should not blindly accept the results of any
collaborative technique.
As a general rule of thumb, the pure plurality technique is only effective in situations where ranking all
alternatives is simply not feasible. Because the majority runoff technique has the potential to alienate up
to 49% of your evaluators, you should generally use this technique primarily to supplement other
techniques. The Borda count system, in spite of technical flaws, works quite well especially when the
evaluators are stakeholders in the analysis. The Condorcet principle is very simple, and so can often be
used to validate the results of other techniques. The Schulze method is an excellent general purpose
technique in many software quality situations except when you are dealing with a relatively small group of
evaluators who are influenced by the results, or evaluators who do not understand the Schulze algorithm.
Acknowledgment: I am grateful to Joshua Eisenberg (McAfee, Inc.) and Moss Drake (Dentists
Management Corp.) who reviewed this paper and made significant improvements to its quality.
== References
Gaetner, Wulf. "A Primer in Social Choice Theory", Oxford University Press, New York, 2006.
McCaffrey, James. "Group Determination of a Best Alternative in Software Testing", MSDN Magazine,
November 2008 (in press).
Nurmi, Hannu. "Comparing Voting Systems", Springer-Verlag, 1987.
Saari, D. G. "Basic Geometry of Voting", Springer-Verlag, New York, 1995.
Excerpt from 2008 Proceedings
PNSQC.ORG
Page 8 of 8