University of Virginia Teaching Resource Center

The written comments

Can written comments on student evaluations provide useful information about the quality of an instructor’s teaching and the quality of students’ learning? Many instructors argue that written comments are random and often contradictory. Who wouldn’t when it’s not uncommon to have some students write, “I learned so much in this class. This is the best course I’ve taken,” while others—in the same class—claim, “This course is terrible. I didn’t learn a thing.” Research indicates, however, that written comments do correlate well with numerical data, both in terms of frequency and intensity. One might suggest, then, that qualitative comments do indeed impart information about teaching and learning. Unfortunately, written comments don’t arrive packaged as neatly as numerical data, nor do they contain the statistical analysis necessary to weight each comment appropriately. To get at the information and to lend it context, an instructor can employ a relatively simple set of techniques.

Taking Pause
On first read of evaluations, many instructors’ typical impulse is to read the written comments straight through. Whether consciously or not, they begin to assign meaning and weight to the comments. And, it is human nature to place greater significance on the negative, so the “this class is horrible” comment is all they take away. To avoid this pitfall, consider this simple strategy (1):

Skim the comments for general trends, then stop reading and leave the evaluations for a couple days.

Next, carefully read some of the comments, focusing on those that are positive; be sure to set the evaluations aside again if feeling emotionally attached to them.

Finally, read all the comments carefully, trying to identify how each comment supports your teaching and learning goals or how the comment could lead to a positive improvement in the course.

Remember that written comments are simply data in written form—the random negative comment carries no more significance than a rogue number.

Sorting By Respondent
A straightforward way to provide context to written comments is to sort them according to how each student rated the overall course. For example, group all the comments together from those students who strongly agree with the statement, “Overall, this was a worthwhile course.” Look for trends among the different groups. If those who strongly agree and those who strongly disagree have a complaint in common, it’s probably significant and worth looking into.

Grouping Like Comments
To find trends in written comments, group like comments together. For example, group all comments which address “organization.” Generally, the comments can be placed into four or five major groups, greatly reducing the complexity of analysis. By separating out positive comments from others, analysis is even simpler.

Color-coding is a simple, but effective way to group the comments. One possibility is to use three different colored highlighters—one color for positive comments, one color for things which the students feel impeded their learning, and one color for useful suggestions. The key is not to mark everything, but instead only mark a handful of each type of comment (e.g. comments about the final examination). After color-coding the comments, read back through the comments, but ignore everything not highlighted. The overall themes in the comments are easily identified.

Analysis Grid (2)

After grouping the comments and sorting them by respondent, it is often useful to put the data into a two-dimensional matrix. One side of the matrix corresponds with the ratings and the other side corresponds with groupings of interest to the instructor, e.g. knowledge of subject matter, organization, student-instructor interaction, etc. An example is shown below:

Rating	Knowledge of Subject Matter	Organization	Student-Instructor Interaction

Strongly Agree	Place Student Comments Here
Agree
Neutral
Disagree
Strongly Disagree

Note: Italicize positive comments.

The completed matrix will clearly show any trends and help you make decisions regarding course and instructional improvement.

Authoring Questions

In order to get written comments which are most useful and that focus specifically on teaching and learning matters, consider authoring your own questions. Students are much more likely to give detailed feedback on the question, “What suggestions do you have to improve your learning in the course?” than they are to, “Please make any overall comments about this course.” If you have a particular concern (e.g. the pace of the course), be sure to ask a question about that as well.

(1) An excellent essay on one individual’s approach to reading evaluations is found in “How I Read Student Evaluations” by Cedar Riener.

(2) Adapted from Lewis, Karron G. “Making Sense of Student Written Comments.” New Directions for Teaching and Learning 87 (2001): 25-32.