isbn9789526046266

There are values that are internal to science and epistemically relevant (Phillips and Burbules, 2000,
p. 54). For instance, one must not falsify evidence and must be open to criticism. Without these values
– which “foster the epistemic concerns of science as an enterprise that produces competent warrants for
knowledge claims” – “scientific inquiry loses its point”.
External and epistemically irrelevant values range from the personal (e.g., “computing education
is important and we must make sure everyone who does our courses learns at least the basics”) to
externally given agendas (e.g., “we need to find out how best to advertise our department to potential
future students”). These values may and do (and must) guide the scientist as they choose which
research questions are important enough to merit investigation – a decision which may change during
an investigation. They also affect the courses of action that we take and recommend on the basis of our
results. However, we should minimize the intrusion of epistemically irrelevant values into the process of
establishing empirical evidence for knowledge claims about our object of research. In other words, our
results are ideally not affected by what we would prefer them to be.
Phillips and Burbules (2000) are careful to point out that “of course the judgment about what is
epistemically relevant or irrelevant is itself – like all judgments – potentially a fallible one” (p. 54).
On methods and results
Various pragmatists and mixed-methods scholars (recently, e.g., Patton, 2002; Johnson and Onwuegbuzie,
2004; Morgan, 2007; Tashakkori and Teddlie, 2010) have argued that there is a disconnect or only a very
loose connection between philosophy and research methods. Johnson and Onwuegbuzie (2004), for
example, caution that “there is rarely entailment from epistemology to methodology” (p. 15). Patton
(2002) makes a similar comment and concludes: “In short, in real-world practice, methods can be separated
from the epistemology out of which they have emerged” (p. 136). Similarly, a set of research findings
can be useful to researchers and practitioners who subscribe to different ontological and epistemological
assumptions than the original researchers. The final word on the epistemological status of our results lies
with our readers, who will interpret our results on the basis of their own beliefs.
I will return to the fundamentals of qualitative and quantitative research shortly to discuss the
trustworthiness of our research. Now, let us return to our research questions.
16.2.3 The following chapters describe several interrelated studies
The pragmatist may use any methods for empirical work that produce useful results and that suit the
research goals. That is what we have done.
To answer Question 1, “In what ways do novice programmers experience learning through visual
program simulation?”, we adopted a phenomenographic perspective. As discussed in Chapter 7,
phenomenography is a research approach that is geared towards studying the different ways in which
people experience phenomena in educational contexts. Compatibly with our research questions – and
typically of phenomenographic studies – we used qualitative methods to explore learners’ understandings
of VPS.
We also adopted a qualitative orientation to answer Question 2, “What happens during visual program
simulation sessions?” More specifically, we used a form of data-driven, qualitative content analysis, an
approach suitable for the exploration of complex data.
We approached Question 3, “Does a short VPS session help produce short-term improvement in
learners’ ability to predict the behavior of given programs?” quantitatively, and used an experimental
setup to measure the short-term impact of VPS in classroom use.
Finally, we sought to answer Question 4, “How do students react to the use of UUhistle in CS1 (and
why)?”, through both qualitative and quantitative analyses of course feedback questionnaires, which are
an inexpensive way of surveying the opinions of many students.
Table 16.1 presents an overview of our studies.
254