Fall 2002

Taking Stock: Evaluations from Students
Robert F. Bruner
Distinguished Professor of Business Administration and Executive Director of the Batten Institute, Darden Graduate Business School

A packet of student evaluations of your course (and, by extension, of you) arrives on your desk. This packet represents larger forces of change in higher education. Virtually all schools are giving more attention to the quality of teaching and course design. Evaluations have gotten longer and more detailed. They carry more weight in decisions about promotion, tenure, and compensation. Qualitative comments from students seem to carry more bite. The pressures to please employers, the advent of school rankings by major magazines, and the general trend toward consumerism in education are some of the tectonic forces at work here.

Many instructors view student evaluations with indifference, fear, or anger. Surveys are awkward assessments of teaching. One wants to "earn the appreciation of honest critics," as Ralph Waldo Emerson said. But with the anonymity granted by the typical evaluation questionnaire, it is hard to know which critics said what, and therefore what weight to give to the comments. Ultimately, criticism from student evaluations is rarely pleasant to receive. One can't reply to the critics. And institutional pressures or one's career situation can amplify the stress surrounding the evaluations. It is understandable then, that some instructors file evaluations away for viewing later, perhaps never.

There is, however, a positive argument for studying the evaluations. The feedback provides yet another opportunity to listen to the students. The capacity to listen is a hallmark of a strong teacher. Also, to be a professional is to commit oneself to steady growth in that field. For the sake of growth, one should read and think about the feedback from students. Student feedback with all its flaws can lead to good outcomes; but the instructor must approach it in the right way. Here are some tips on how to take stock of your classroom work at the end of a course.

Reading the tea leaves

First, write your own evaluation. You, after all, have an important perspective on how things went. Flip through your course syllabus, and jot down some notes about the course design, materials, and classroom teaching experience. Do this while the course is still fresh in your mind. Having your own assessment serves to frame the students' comments in a way that can offer insights-not just "good" versus "bad" but also the degree of alignment between you and the students. Misalignment can point toward improvements that may not be readily apparent in the student evaluations only. Read the students' evaluations twice, the first time quickly to gain a sense of the whole. If the feedback is worse than you expected, catch your breath; put it aside for a while until you can reread it more objectively. If the feedback is complimentary, you have a different challenge, though no less important: find the humility to look for genuine opportunities for improvement. Re-read the evaluations slowly, and consider the following:

• Cross-sectional patterns. Student evaluations are very noisy; the objective should be to find the signal rather than marinate in the static (or even add to it). There may be clusters of comments that point to a common behavior. Try to separate comments by focus of criticism: teaching versus materials versus course design. Note what seems to be going well and should be continued. We focus too easily on the criticisms without acknowledging successes. Sort students' qualitative written comments into two piles, positive and negative. Make separate inventories of each.

• Command and connection. Look for evidence of these critically important qualities. Their absence is a showstopper. "Command" is shorthand for confidence, authority in the subject, apparent mastery. Effective teachers seem to know what they talk about. Command also has an interpersonal dimension: the ability to gain the attention, respect, and following of students. "Connection" is shorthand for attending to the students: to know who they are, to listen well in and out of class, to understand the learning opportunities and challenges they face, to sense where they might be struggling, and simply to be available. As important as it is ungrammatical is the complaint that "the teacher don't hear so good." Lack of connection is not consistent with student-centered teaching. Fortunately, issues of command and connection can be remedied. Numerous small tricks of the trade are helpful here, though often overlooked in the myopic belief offered by some that the material teaches itself. These tricks include arriving at class early, staying after class, being present at the student café during coffee breaks or at lunch-these have in common being where the students are.

• Look at the details. To a large extent the overall evaluation of your teaching is the least useful information in the feedback, because it says little about what you might do differently next time. The results are built up from the details of your teaching. In this regard, the qualitative comments offered by students are valuable. You can do something about criticisms such as "Her handwriting on the chalkboard is too small," or "He only calls on students in the front row," or "The three class meetings on X left me utterly confused," or "The technical notes were riddled with typos."

• Your norms versus theirs. Many community norms are not discovered until violated. New teachers and visiting instructors are especially vulnerable to this, but it can happen to anyone. Examples might involve cold-calling, handing out solutions to purportedly open-ended assignments, assigning a heavy workload on weekends, and requiring teamwork. In some schools these are expected; in others unusual. The point here is that a close reading of the evaluations can help crystallize an understanding of school norms, your own philosophy of teaching, and where they collide.

• Trends over time. How does the most recent evaluation compare to earlier evaluations? Do the same comments tend to reappear? What do those trends imply for your personal development agenda?

What is to be done?

Reflect first on the process of developing your instructional skills before embarking on a detailed inventory of changes to be made, taking into account that such assessments are always a work-in-progress

• Crystallize your priorities. Think critically about what students seem to want you to change. Not all requests must be granted. One good criterion for assessing the implied requests is to consider whether the change will promote better learning. For instance, suppose you were criticized for routinely not ending class on time. You may choose not to change your practice in the belief that what really matters is how the learning comes together toward the end of class and that occasionally a punctual ending is worth sacrificing for solid learning.

• Manage the feedback process. End-of-course student surveys have their drawbacks. Consider collecting other points of reference, such as mid-course surveys, videotapes, and classroom observations by your colleagues. Informal conversation with one's own students can be very revealing. Given the alternative sources of feedback, it is a mistake to wait until the end of the course to find out how you are doing. A feeling of surprise at seeing the evaluations may be stark evidence that you don't listen sufficiently well to the students or the classroom process.

• Get an attitude-the right attitude. Some novices worry that they don't have the instinct or charisma ever to get good evaluations. This flies in the face of abundant evidence and my own experience that the capacity to teach very well can be learned. You have a career as a teacher, of which any individual course is a small episode. Resolve to learn from it, and grow to be a better teacher.

• Be student-centered, and trust that decent evaluations will follow. Occasionally an instructor will try to teach in ways expressly designed to get good student evaluations. Such an attitude produces a bonfire of aberrant behavior, including easy homework assignments, inappropriate socializing with students, liberal distribution of "solutions" to the class discussions, and grade inflation. The underlying premises that campaigning works, and that challenging the students leads to bad evaluations may be true in some cases. But in general students value instructors from whom they have learned well. That suggests that the straightest path to positive evaluations is to focus on learning and the delivery of an intellectually valuable experience.

• Accept variance. Uncomplimentary feedback is the occasional companion of any instructor who takes risks with new material, tries new teaching styles, gets a poor draw of students, or believes that challenging students is good. You and your Deans should accept this reality, and acknowledge that if one relentlessly gets perfect evaluations, one might not be trying hard enough.

• Be action-oriented. Focus on what you can do differently next time rather than what happened in the past. Distill what you have learned from the evaluations into a few important "to do's" rather than a detailed inventory of all possible improvements. Those improvements should be priorities in your personal development agenda for the coming year. They should have action steps beginning soon: reading on teaching techniques; asking to observe a successful colleague's class; asking a colleague to observe you at a few points during your course; searching for more suitable course materials; tinkering with the course design to put your best foot forward. Above all, don't shrink from the task. The attention you give to this season can pay huge dividends in the years to come.

NB: The Teaching Resource Center offers a variety services to assist instructors, including consultations about interpreting and making use of feedback from student evaluations.