Assessing graduate screen production outputs in nineteen ...

Figure 8.2 Graph of 30 assessors’ mean locations (in logits) on the SPAS for eachof 45 productionsAs noted in the results in Table 8.4, the mean locations (scores) for each productionobtained from the SPAS vary (that is the quality of the productions are deemeddifferent from one another by the assessors) and the range of mean locations givento each production by each of the 30 assessors varies. In other words, someproductions attract mean locations that are more similar to one another than themean locations allotted to other productions. Put yet another way, assessors aremore consistent in their judgments about the quality of some productions than theyare for other productions. It is difficult to see these differences clearly in RescaledAnalysis Table 8.7, but Table 8.4 tells us that assessors, using the SPAS, weremost consistent in their judgments of productions such as P30 and P1 and leastconsistent in their judgments of productions such as P39 and P29. Interestingly,these two latter productions were the lowest scoring.There is a possibility that both high quality and very poor quality productions mayattract a wider range of opinions than productions of middling quality. If this is thecase, then the mean standard deviations for the highest quality productions shouldbe similar to that for the lowest quality and the mean standard deviation for middlingproductions should be less than either of these. Using the results in Table 5 (scoreson the SPAS), if we consider just the seven lowest scoring productions, the sevenhighest scoring, and the seven in the middle of the range, the mean standarddeviations are 1.631, 1.039 and 1.377, respectively. This tells us that the 30assessors have varied most in their scores for the lowest scoring productions, themiddle ranking productions have the next largest variation and the variations for thehighest ranking productions are the lowest. This means that the higher the quality ofthe productions, the more like each other are the judgments of assessors (usingSPAS). Judgments vary the most for the lowest quality productions and the least forthe highest quality productions. This patterning should be apparent from a negativecorrelation between the mean locations and the standard deviations in Table 8.4(negative because the relationships is expected to be inverse with high standarddeviations associated with low mean scores and low standard deviations associatedwith higher mean scores). The correlation coefficient between these is, in fact, -Assessing graduate screen production outputs in nineteen Australian film schools118