DISCUSSION

2 Design of our field experiments
2.1 Description of the sample
We ran our field experiments at a public university in the United States. Our subjects were
undergraduate students enrolled in a large semester-long introductory course. As described in
Section 2.2, we sought consent from all our subjects (the consent rate was ninety-eight percent).
Approximately four thousand students participated in total. We employed a between-subjects
design: each student was randomized into the Treatment group or the Control group immediately
on giving consent. Students in the Treatment group were asked to set goals while students in the
Control group were not asked to set any goals. As described in Section 2.3, in the Fall 2013 and
Spring 2014 semesters we studied the effects of performance-based goals on student performance
in the course (the ‘performance-based goals’ experiment). As described in Section 2.4, in the Fall
2014 and Spring 2015 semesters we studied the effects of task-based goals on task completion
and course performance (the ‘task-based goals’ experiment). 8
Table 1 provides statistics about participant numbers and treatment rates. We have information
about participant demographics from the university’s Registrar data: Tables SWA.1,
SWA.2 and SWA.3 in Supplementary Web Appendix I summarize the characteristics of our
participants and provide evidence that our sample is balanced. 9
All semesters
Fall 2013 & Spring 2014 Fall 2014 & Spring 2015
(Performance-based goals) (Task-based goals)
Number of participating students 3,971 1,967 2,004
Number of students in Treatment group 1,979 995 984
Number of students in Control group 1,992 972 1,020
Fraction of students in Treatment group 0.50 0.51 0.49
Notes: The number of participating students excludes: students who did not give consent to participate; students
who formally withdrew from the course; students who were under eighteen at the beginning of the semester;
students for whom the university’s Registrar data does not include SAT or equivalent aptitude test scores; and
one student for whom the Registrar data does not include information on gender.
Table 1: Participant numbers and treatment rates.
2.2 Course structure
In all semesters, a student’s letter grade for the course was based on the student’s total points
score out of one hundred. 10 Points were available for performance in two midterm exams, a final
exam and a number of online quizzes. Points were also available for taking an online syllabus
quiz and a number of online surveys. For the Fall 2013 semester Figure 1 gives a timeline of
8 We also ran a small-scale pilot in the summer of 2013 to test our software.
9 For each characteristic we test the null that the difference between the mean of the characteristic in the
Treatment Group and the Control group is zero, and we then test the joint null that all of the differences equal
zero. The joint test gives p-values of 0.636, 0.153 and 0.471 for, respectively, all semesters, Fall 2013 and Spring
2014 (the performance-based goals experiment), and Fall 2014 and Spring 2015 (the task-based goals experiment).
See Tables SWA.1, SWA.2 and SWA.3 for further details.
10 The grade key is at the bottom of Figure SWA.1 in Supplementary Web Appendix II.
5