Student Perceptions of Teaching Effectiveness - Harvard Graduate ...

Student Perceptions of Teaching Effectiveness
Discussion Brief
National Center for Teacher Effectiveness
and the Achievement Gap Initiative
Harvard University
Ronald F. Ferguson
(Ronald_Ferguson@Harvard.edu)
October 14, 2010
University administrators realized long ago that students could be important sources of
information about what happens in classrooms. Student feedback in colleges and universities adds
value for teachers and administrators alike in their efforts to improve teaching and learning. After all,
no one knows more about what happens in classrooms than the students and teachers who inhabit
them. The same is true in elementary and secondary schools. This discussion brief describes the origin
and selected features of the Tripod Assessment of Instructional Quality and Student Engagement for
elementary and secondary schools. In addition, it summarizes some interesting patterns in the data for
a large sample of secondary school classrooms that are participating in the Measuring Effective Teaching
(MET) project of the Bill and Melinda Gates Foundation. While the assessments cover both instruction
and student engagement, this discussion brief focuses only on student perceptions of instruction.
Origin of the Survey
The Tripod student perceptions assessment is based upon classroom-level surveys developed
over the past decade by the Tripod Project for School Improvement. The Tripod Project was founded by
the author of this brief and refined in consultation with K-12 teachers and administrators in participating
schools and districts and, as of 2007, colleagues at Cambridge Education. Cambridge Education deploys
the surveys and does associated reporting as a service to participating schools.
Over the past decade, several hundred thousand elementary, middle- and high-school students
have completed Tripod surveys in hundreds of schools across the United States, Canada and China.
Items have been added and replaced each year to develop gradually a stable set of valid and reliable
measures. By responding to these surveys, students have provided feedback to their schools and
teachers not only about their schools as whole institutions, but more importantly, concerning
instruction in the specific classrooms where they took the surveys. Tripod teachers receive summaries
for their own classrooms. Reports for administrators typically hide teachers’ identities, but nonetheless
show patterns of instructional disparity between classrooms and point to priorities for professional
development and school improvement.
Similar to their university-level counterparts, students in elementary-, middle- and high-school
classrooms have consistently been able report in valid and reliable ways on their own engagement, as
well as on the quality of the teaching that they experience. Individual students surveyed in multiple

classes—say, English, math and science—tend to respond differently in different classes, reflecting
variation in what they experience. Response patterns indicate that individuals often experience great
teaching in one subject, but then mediocre teaching another.
Measures of Teaching Quality
The primary measures of instructional quality in the Tripod surveys are gathered under seven
headings called the Seven C’s. The seven are: Care, Control, Clarify, Challenge, Captivate, Confer and
Consolidate. The Seven C’s are grounded upon a great deal of education research by many researchers
over the past several decades. They capture much of what researchers have suggested is important in
determining how well teachers teach and how much students learn. Each of the C’s is measured using
multiple survey items. The following are brief descriptions of each concept.
The Seven C’s
1. Care pertains to teacher behaviors that help students to feel emotionally safe and to rely on the
teacher to be a dependable ally in the classroom. Caring reduces anxiety and provides a sense
of positive affiliation and belonging. Caring goes beyond “niceness”; caring teachers work hard,
and they go out of their way to help. They signal to their students, “I want you to be happy and
successful, and I will work hard to serve your best interest; your success is an important source
of my personal satisfaction.” An example of a MET survey item measuring Care is: “My teacher
really tries to understand how students feel about things.”
2. Control pertains to classroom management. Teachers need skills to manage student
propensities towards off-task or out-of-order behaviors, in order to foster conditions in the
classroom that allow for effective communication and focus. Effective control helps to maintain
order and supplements caring in making the classroom calm and emotionally safe from such
things as negative peer pressures. An example of a MET survey item measuring Control is: “Our
class stays busy and doesn’t waste time.”
3. Clarify concerns teacher behaviors that promote understanding. Interactions that clear up
confusion and help students persevere are especially important. Each student comes with
particular gaps in understanding and with both correct and incorrect interpretations of the
world around them. To be most effective, teachers should be able to diagnose students’ skills
and knowledge, and they need multiple ways of explaining ideas that are likely to be difficult for
students to grasp. Teachers also must judge how much information students can absorb at any
one time, and they should differentiate instruction according to individual maturity and interest.
An example of a MET survey item measuring Clarify is: “My teacher has several good ways to
explain each topic that we cover in this class.”
4. Challenge concerns both effort and rigor -- pressing students to work hard and to think hard. .
Challenging teachers tend to monitor student effort and to confront students if their effort is
unsatisfactory. Students who do not devote enough time to their work or who give up too easily
in the face of difficulty are pushed to do more. Similarly, students who do not think deeply or to
reason their way through challenging questions are both supported and pushed. The teacher
may ask a series of follow-up questions intended to elicit deeper, more thorough reasoning. An
example of a MET survey question measuring Challenge for effort is: “In this class, my teacher
2 | P a g e

accepts nothing less than our full effort.” A question measuring Challenge for rigorous thinking
is: “My teacher wants us to use our thinking skills, not just memorize things.”
5. Captivate concerns teacher behaviors that make instruction stimulating, instead of boring.
Captivating teachers make the material interesting, often by making it seem relevant to things
about which students already care. Brain research establishes clearly that stimulating learning
experiences and relevant material make lessons easier to remember than when the experience
is boring and the material seems irrelevant. Examples of questions concerning stimulation and
relevance are: “My teacher makes lessons interesting,” and “[negatively worded] I often feel like
this class has nothing to do with real life outside school.”
6. Confer concerns seeking students’ points of view by asking them questions and inviting them to
express themselves. When students expect that the teacher might call on them to speak in
class, they have an incentive to stay alert. In addition, believing that the teacher values their
points of view provides positive reinforcement for the effort that it takes to formulate a
perspective in the first place. Further, if students are asked to respond not only to the teacher,
but to one another as well, a learning community may develop in the classroom, with all of the
attended social reinforcements. An example of a question concerning Confer is: “My teacher
gives us time to explain our ideas.”
7. Consolidate is the seventh C. Consolidation concerns how teachers help students to organize
material for more effective encoding in memory and for more efficient reasoning. These
practices include reviewing and summarizing material at the end of classes and connecting ideas
to material covered in previous lessons. Teachers who excel at consolidation talk about the
relationships between ideas and help students to see patterns. There is a large body of
evidence supporting the hypothesis that these types of instructional activities enhance retention
by building multiple brain pathways for retrieving knowledge and for combining disparate bits of
knowledge in effective reasoning. An example of a question concerning Consolidation is: “My
teacher takes the time to summarize what we learn each day.”
There are between four and eight survey items associated with each particular C. The result is seven
composite measures, one for each of the Seven C’s. Each item in a given Seven C’s index represents a
different way of measuring that particular one of the Seven C’s.
There is a useful distinction in the teaching quality literature between “support” and “press.”
Support refers to ways of serving and communicating with students that the students are likely to
perceive as helpful and welcome. Conversely, press imposes some degree of stress. It refers to
communication that students may experience as pressure to move outside of their comfort zones.
Below, we distinguish the Five Support C’s—i.e., Care, Clarify, Captivate, Confer and Consolidate—from
the two C’s that entail a higher degree of press, i.e. Control and Challenge. Challenge and Control
strongly connote teacher demandingness. However, Challenge is not so easy to classify. Our findings
indicate that Challenge may entail aspects of both support and press.
Quality of the Measures
Stephen Raudenbush of the University of Chicago has helped develop a multi-level
measurement model for the Tripod surveys. The model takes account of multiple types of
3 | P a g e

measurement error in calibrating reliability statistics. Recall that there are several survey items for each
student in each composite index for each of the Seven C’s. The Seven C’s indices defined at the student
level are used to create classroom-level measures that aggregate across students. Classroom-level
measures, in turn, are embedded inside schools. The measurement model assumes that random factors
introduce measurement error at each level of this system. Calculations that calibrate the measurement
errors are the basis of reliability measures. For each Seven C’s index, the reliability represents the
fraction of variation in that index that is true variation, not due to measurement error.
Using this measurement model, indices for the Seven C’s constructed from MET survey data
have proven highly reliable—in the range of 0.80 and above. They are also remarkably stable during the
school year. To examine this stability, a special sample of 141 classrooms was surveyed in December
2009 and again in March 2010. Classroom-level indices were constructed for each month and
compared. Corrected for measurement error, the correlations for given classroom quality indices,
comparing the December and March values, ranged between 0.70 and 0.85.
A more intuitive way to analyze the stability is the following. For each month, we used the
survey responses for all 7 of the Seven C’s to form a single composite quality measure. Then, we used
this measure to rank order the classrooms and form deciles. The bottom ten percent of classrooms in
the ranking comprises the first decile, and so on. There is one set of deciles defined using the December
data and another set defined using the March data.
For the 141 classrooms in the analysis, 78 percent moved no more than one decile up or down
between December and March. Ninety-three percent moved no more than two. Hence, there was
great stability. At the same time, some classrooms did get better and some got worse. We know this
because consistency between student responses within-classrooms at each point in time was sufficient
to distinguish classrooms that changed from those that did not. In other words, for a minority of
classrooms, we are able to reject the null-hypothesis that students perceived quality to be the same in
March as in December.
Dimensions of Teaching Quality
We analyzed data from 1264 English Language Arts (ELA) classrooms and 1094 math classrooms
to examine how strongly the Seven C’s distinguish dimensions of teaching quality. 1 The data were
classroom averages of student survey responses. The goal was to learn more about what combinations
of survey responses most strongly distinguish classrooms from one another. In addition to the Seven Cs,
we included two other constructs in the analysis reported in Panel A of Exhibit 1. One was a three-item
measure of how much time the class spent preparing for the state accountability exams (referred to as
Test-Prep Emphasis on Exhibit 1). The other was a three item measure of Class Emotional Climate,
measuring the degree to which students felt nervous, stressed or angry in the classrooms where they
completed the MET survey.
Panels A and B of Exhibit 1 summarize two distinct analyses. Each sheds light from a distinct
vantage point (which we explain) on dimensions of classroom quality. The analysis summarized in Panel
A included all of the survey items for each of the Seven C’s, as well as the Test Prep Emphasis and the
1 This is what researchers call “factor analysis.” Here, we explain the findings in nontechnical language.
4 | P a g e

Class Emotional Climate items. The analysis for Panel B included all of the items for six of the Seven C’s,
excluding Control.
Panel A: The Helicopter Vantage Point. Imagine sitting 300 feet above the ground in a
helicopter looking down at a row of apartment buildings. You are trying to describe what you see.
Some of the variation you observe—say, twenty percent of it—entails different styles of chimneys.
Another ten percent entails different sized air conditioning units on the rooftops. Twenty percent is flat
roofs and constituent materials. The remaining fifty percent might be the sides of the various buildings;
but because of the angle of vision and the light, the features (e.g., windows, doors, siding materials, etc.)
blur together and are difficult to distinguish.
Exhibit 1: Dimensions of Teaching Distinguished by Student Survey Responses
Secondary School Students in English Language Arts/Reading and Math
PANEL A: Dimensional Analysis of Classroom Conditions
English Language Arts (ELA)
Math
Dimensions: 1 2 3 4 5 1 2 3 4 5
The Five “Support” Cs
Control (Press)
Test-Prep Emphasis
Challenge (Press)
Class Emotional Climate
Proportion of Variance 0.52 0.22 0.09 0.08 0.05 0.52 0.23 0.09 0.07 0.05
PANEL B: Dimensional Analysis of Six C’s other than Control.
English Language Arts (ELA)
Math
Dimensions: 1 2 3 1 2 3 4
Care
Clarify
Challenge A: Effort
Challenge B: Rigor
Captivate
Confer
Consolidate
Proportion of Variance 0.47 0.33 0.12 0.31 0.29 0.23 0.13
Classrooms with no fewer than 10 and no more than 35 students. Based upon principal factors
analysis with orthogonal varimax rotation, using classroom-level means as the units of analysis.
Separately for each subject (i.e., ELA and math), the initial factor analysis distinguished the Control
index from the other six; Control loaded on one factor while the other six loaded on another. A
second factor analysis using only items for the other six indices produced the distinctions shown in
columns 1, 2 and 3. N=1264 Classrooms for ELA; N=1090 Classrooms for math.
Panel A is like the view from the helicopter. First, just as the vantage point of the helicopter
makes it difficult to distinguish windows, doors and textures on the sides of the buildings, the vantage
point for Panel A makes it difficult to distinguishing the Five Support C’s from one another and also from
5 | P a g e

Challenge. Nonetheless, the Five Support Cs and Challenge as an amorphous group (like the non-distinct
features on the sides of buildings) account for half of the classroom-to-classroom variation that we see
in this collection of variables (specifically, 52 percent). Like helicopter views of chimneys and air
conditioning units on rooftops, classroom-to-classroom differences in Test-Prep Emphasis and
Classroom Emotional Climate are easy to distinguish even though they do not account for much of the
observable variation between classrooms (9 percent and 5 percent, respectively). Of the two “press”
C’s, which are distinguishable from this view, Control is the most distinguishable and accounts for 22
percent of the between-classroom variation in student responses. Finally in Panel A, Challenge appears
both as a distinct dimension of classroom quality (Column 4)—a measure of press distinct from
Control—and also as part of the amorphous group with the other Five Support Cs (Column 1). 2
There is almost no difference between ELA and Math in the distinctions that Panel A highlights.
For both ELA and math, the major dimensions that distinguish classrooms from one another are Five
Support Cs and Challenge as an amorphous group, classroom Control, Challenge, Test-Prep Emphasis and
Classroom Emotional Climate.
Panel B: The Ground-Level View. Imagine now standing on the ground where all you can see is
the sides of the same buildings, not the tops. The chimneys and air conditioning units are hidden from
view, but features on the sides of the buildings are much more discernable than they were from the
helicopter. The analysis in Panel B omits the “rooftop” measures—Control, Test-Prep Emphasis and
Classroom Emotional Climate—that were most distinguishable in Panel A, but it includes Challenge
because it was part of the amorphous group with the Five Support C’s. Observing “from the ground
level”—as though looking up at windows, doors and siding materials—we can see that classrooms are
distinguished by different combinations of Care, Clarify, Challenge, Captivate, Confer and Consolidate.
We learn in Panel B that the Seven C’s function a bit differently for ELA as compared with math.
For ELA, the quality in that most strongly distinguishes classes from one another appears to be
Captivation (hence the blackened rectangle for Captivation in Column 1). But Care, Clarify, Confer and
Consolidate are associated with this dimension as well (thus shaded in grey), as though in support of
Captivation. One interpretation is that English teachers make lessons captivating when they connect
with students through caring, explaining lessons clearly, conferring with students about their views and
summarizing lessons to make them coherent. However, the second most distinguishing dimension for
ELA is Challenge (for both effort and rigor), with supporting roles for Clarify and Consolidate. An
interpretation here is that English teachers make lessons more challenging when they also make them
clear and coherent. The only additional distinguishing dimension for ELA is Confer. ELA teachers are
distinguished from one another by their tendency to confer with students, for example, by inviting them
to share ideas.
Math is different. Survey items for Care and Captivate dominate two major dimensions of
classroom quality, each of which strongly distinguishes math classrooms from one another. Perhaps
caring is more important for math than for ELA because math is a more difficult subject and caring
teachers are more effective at reducing math anxiety. Clarify does not dominate any dimension, but
2 In other words, the items in the Challenge index load heavily on the factor for the Five Support C’s and also on a
separate factor, where the Challenge items are the only ones with heavy loadings.
6 | P a g e

instead shows up in a supporting role on both the Care and Captivate dimensions. Apparently, the
capacity to clarify difficult math concepts supports the communication of caring and the ability to make
lessons captivating. A frustrated teacher may communicate less care; and math that is confusing is
unlikely to be captivating. Finally, for math but not for ELA, Challenge for effort and Challenge for rigor
appear as two separate dimensions. The math teachers that challenge students to work hard—for
example, to do a lot of homework—are not necessarily the same teachers that challenge them to think
hard. The two types of Challenge constitute separate ways of distinguishing among math teachers.
Classrooms Differ Tremendously
Finally, recall the discussion above where we used a composite index of survey items to consider
stability of classroom rankings from December to March. Here in this section, we use a similar ranking,
but we employ the data for all of the 1094 math and 1264 ELA classrooms from the analysis directly
above. This produces a ranking with 2358 classrooms that range from the lowest to the highest rated
overall. The ranking can be broken into deciles and we can examine the differences in levels of
agreement with survey items in different parts of the quality distribution. With 2358 classrooms, there
are roughly 236 classrooms and almost twenty times that many students in each decile.
How much do classrooms differ? Exhibit 2 shows average levels of agreement with each of the
Seven C’s in the first, fifth and tenth deciles of the overall composite teaching quality distribution. The
disparity is massive. Fewer than 20 percent of students in the first decile of the quality distribution
agree with statements in the Care index, while almost 80 percent in the tenth decile agree with the
same statements. Captivate follows a similar pattern to Care, except that the level of agreement is
somewhat higher, especially in the tenth decile, where 85 percent agree with statements in the
Captivate index. Items in the Challenge index generate the highest levels of agreement for all three
segments—at the bottom, middle and top of the quality ranking. Imagine how different life must be in
these three segments of the classroom quality distribution!
Conclusion
Researchers over many decades have suggested that students will engage more deeply and
master their lessons more thoroughly when their teachers care about them, control the classroom well,
clarify complex ideas, challenge them to work hard and think hard, deliver lessons in ways that
captivate, confer with them about their ideas and consolidate lessons to make learning coherent.
Building upon Tripod surveys developed over the past decade, MET student perception surveys are
giving students in thousands of classrooms the opportunity to demonstrate that they can distinguish
their classrooms on these key dimensions. Student responses are reliable, valid and stable over time at
the classroom level. MET constitutes an unprecedented opportunity to combine these student
perceptions with information collected by other methods and to measure how together they predict
learning outcomes. Classroom-level student surveys are a high-potential mechanism for incorporating
students’ voices in massive numbers into school, district, state and national efforts to improve teaching
and learning in elementary, middle and high schools.
7 | P a g e

1. Care
2. Control
3. Clarify
4. Challenge
5. Captivate
6. Confer
7. Consolidate
Exhibit 2: Mean percentage agreement with items in each Seven Cs
index, in the first, fifth and ninth deciles of the composite measure.
First Decile Fifth Decile Tenth Decile
0 10 20 30 40 50 60 70 80 90 100
8 | P a g e