Research and the Theoretical Field - Pauldowling.me

55
Babbedge et al report that ‘the two [photometric] codes are in reasonable
agreement about the extinction of a given source’ (p. 12 in pdf version). This
may be taken to indicate a ‘reasonable’ level of reliability for the use of
photometrics. The term, reliability, generally has to do with repeatability. For
example, in my own research, I have developed the concepts esoteric
domain, descriptive domain, expressive domain and public domain in relation
to any given field of practice, such as school mathematics. The esoteric
domain may be glossed as those practices that are most highly specialised to
the field. At one point (see Dowling, 1998), I wanted to compare two sets of
school mathematics textbooks in terms of the proportion of esoteric domain
text that they contained. So, I divided each of a sample of pages from each
book into four and coded each quarter page as either esoteric domain or notesoteric
domain. I then produced tables showing the proportion of esoteric
domain text in each chapter of each book in each series. This enabled me to
compare the two series in terms of, shall we say, their respective levels of
specialisation and also to compare the level of specialisation within different
topics (ie chapters) within school mathematics. For the record, I was able to
confirm that books directed as ‘high ability’ students contained more esoteric
domain text and those directed at ‘low ability’ students contained more nonesoteric
domain text. I was also able to reveal that, even within the ‘high
ability’ books, the topic, ‘statistics and probability’ contained relatively little
esoteric domain text (plausibly a reason for its notorious difficulty amongst
school students). This was a small scale study and I was working alone. I had
generated a set of coding principles that enabled me to determine the
appropriate coding for each quarter page and I simply applied this to my
sample. But who is to say how much subjective judgement entered into my
application of these principles Well, this was only one aspect of my analysis
of the textbooks. I also engaged in close textual analysis in which I was able
to demonstrate how I was applying the principles by an extended or
elaborated description (Brown & Dowling, 1998) of the interpretation of a
given textual fragment. However, to augment this, I might have taught another
individual my coding principles and had them re-code part of my sample of
pages. I could then have computed a measure of correlation between the
results of the two coding operations and this would have been a measure of
inter-coder reliability. In the astrophysics example, two different ‘photometric
codes’ are deployed and the results found to exhibit a ‘reason able’ level of
correlation (though the researchers used a scatter diagram to represent this
rather than a coefficient of correlation). This speaks to the reliability of the use
of photometrics generally, as represented by these two codes: a method is
reliable if it consistently produces comparable results.
Babbedge et al also point out that although the results from the photometric
method do not exhibit a strong correlation with the Balmer-derived results,
when other factors—principally, a lack of precision in all three methods—are
taken into account, the correlation can be interpreted as ‘in fact quite good’ (p.
15 in pdf version). This statement might be taken to indicate that, whilst all of
the methods are fairly reliable, none is very accurate. Accuracy, here, refers
to the range of measurement within which the actual dust extinction level is
expected to lie. You may have come across the use of the term accuracy in
relation to opinion polls. Here, the figure most commonly stated is the
RESEARCH & THE THEORETICAL FIELD