isbn9789526046266

Each assignment was worth a number of points; the overall assignment grade was determined by the
sum of the points received during the semester. The program-writing assignments were by far the most
valuable in terms of points. VPS assignments were worth considerably less (as was natural, since they
were meant to be easier and quicker to do than the program-writing assignments). Even the program
animations were worth a very small number of points that the student got just for watching the animations.
In total, the program-reading assignments accounted for 14% of all the assignment points available.
Getting the highest grade for the course required the production of a working solution to most
assignments. The overall assignment grade was averaged with the grade from a final examination to
produce the overall grade.
Students could freely choose whether to do any of the program-reading assignments in UUhistle.
Getting the highest grades required them to do at least some of them, but it was possible to pass the
course without ever using UUhistle at all.
Exercise sessions and lectures
All the assignments in CS1–Imp–Pyth – program-reading and program-writing alike – were ‘open labs’.
Students worked on them at their own pace, mostly outside of class. There were no ‘closed labs’ with a
fixed agenda. Instead, teaching assistants supervised ‘exercise sessions’. In these sessions, students worked
freely on whichever assignment they wanted, and could ask the assistant for guidance when they felt they
needed it. Attendance was voluntary. Students could also voluntarily attend lectures. Most students in
CS1–Imp–Pyth did not attend either the lectures or the exercise sessions, preferring to study on their own
or with friends.
The lecturer occasionally made use of UUhistle to illustrate program execution during the lectures. The
visualizations in UUhistle were the main (and almost only) explicit form of teaching about the underlying
execution model of programs in CS1–Imp–Pyth. During one lecture at least, the lecturer drew diagrams
of computer memory to illustrate references.
Staff
For a number of recent years, CS1–Imp–Pyth has been designed, lectured and organized by a teacher
who has not otherwise been involved in UUhistle’s development or this empirical study. She agreed
to adopt UUhistle after it was suggested to her during a demonstration of the tool by the authors.
The program-reading assignments were created by UUhistle’s authors and subsequently approved by the
teacher. UUhistle’s programmer, Teemu Sirkiä, was one of 18 part-time teaching assistants. He did not
play a direct role in conducting the empirical studies presented in the next chapters.
The teaching assistants were asked to familiarize themselves with UUhistle and VPS on their own and
did not receive any other training on the topic.
A few statistics
A total of 562 students did at least one of the program-reading assignments. That means 74% of all
enrolled students, and 81% of those who submitted at least one assignment of some sort – among the
enrolled students there are always some “ghosts” (Mason and Cooper, 2012) who fail to turn up at all or
at least do not put in nearly any work.
The students made 11,010 submissions through UUhistle in total, including a small number of
resubmissions.
The code of each of the program-reading assignments was fairly short, ranging from 2 to 28 lines in
length. The shortest program animation (assignment 1.1, that is, the first program-reading assignment
of round 1) was 13 (correct) steps long, and the longest (assignment 9.3) about 200 steps. The shortest
VPS exercise (assignment 1.2) was 12 steps, and the longest (assignment 4.9) about 80 steps.
The easiest VPS task as measured by the number of mistakes made was assignment 3.3 on while
loops, in which the submissions had a median of 1 misstep. The hardest was assignment 5.1 – which
combined function calls, boolean values, selection, and logical operators – with a median of 30 incorrect
steps per submission.
260