Timing is everything, especially with course evaluations. The conventional system collects evaluations shortly before the end of the semester, a highly problematic time: too early for harried students to reflect upon or consider their final letter grade, but too late for an instructor to make really good use of the information. Why such an awkward time? In part, I suppose, to split the difference between two highly disparate applications of the information: first for people to evaluate the effectiveness of the instructor, and second for the instructor to alter the course. The first requires thoughtful distant reflection from students, while the second often requires rapid response from the instructor. It's difficult for any evaluation system to do both, and the conventional system doesn't do either terribly well.
Instead of trying to meet these dissimilar needs with one evaluation mechanism, I think we'll be better off using multiple mechanisms, each suited to its own purpose; and for some of the purposes, information technology can be extremely useful. I'd like to look at the particular problem of collecting information from students while a course is in progress, and discuss some of my experiences using the web to collect anonymous student feedback over the past two years in large classes (50 to 100), teaching structural engineering design to architecture students, a topic which most students find difficult and outside their mainstream interests.
Since I first started using the web to supplement my teaching in the fall of 1995, I've included a feedback page with a web-based form where students can make anonymous comments. The comments are given a time stamp and appended to a file to which only I have access. I promise to review it at least once a week, but in practice I do so nearly daily while courses are underway (the time stamp is for clarification when someone writes "Your lecture was terrible yesterday..."). Students know that the system is truly anonymous since they can sit down at any machine in a lab and type whatever they want; there's no handwriting, no one to see them dropping a note in a suggestion box, and they know it can't be traced. Complete anonymity.
What happens when you do that? The messages fall into a handful of categories. First, there are simple and practical requests, such as the following:
Receiving messages like these, I typically mention at the beginning of the next lecture that I received the message, and describe my response to it. That act, though simple and mundane, is an important gesture for many students. It shows I'm listening (at least part of the time).
Other practical messages are directed more at course content, such as this:
This kind of feedback is extremely useful. At the beginning of the next lecture, I can ask about that problem and who's been having trouble with it; and spending as little as five minutes going over that and getting the class to talk about it can help avoid much bigger problems down the road. This kind of time-sensitive feedback is difficult to collect any other way. Some students are aggressive enough to make such requests directly, but many are not, and the feedback page gives many of them a viable option.
And then there's simple venting, as demonstrated in the following (highly abridged) excerpt:
Or, more succinctly, in the following complete message:
I'll spare the reader more vehement examples, although I must add that I've never received a comment that was personal or profane (that may change after this article is published). While venting is not particularly pleasant reading in the feedback file, I must say that I would much rather read it there than on my course evaluations (not to mention all the others who may read my evaluations). Some students spend the better part of the semester looking forward to course evaluation day when they can unload several weeks of pent-up frustration. A feedback page lets them vent as they go along, and there is often useful advice underlying the anger. The in-progress feedback gives me a chance to respond, so that the problems do not grow as big, and the student response to the remaining problems is more rational and less emotional. Plus, even after a hard test late in the semester, when the class is grumpy and venting, you can get a message like this:
Definitely worth it.
Are there problems with anonymous feedback? Some. First, it's not for the faint of heart. You can avoid reading your evaluations, but you can't avoid the feedback file once you start, because if you don't visibly respond, then students will know it's a sham. Second, the most difficult aspect for me is when someone effectively demands an answer, but I can't respond to them directly and it's not appropriate to address the entire class. There are technologies that would allow an effective anonymous dialog, rather than one-way collection. I haven't yet decided whether that would be worth the effort, or even whether it would be a good thing. But I do know that immediate anonymous feedback is an important tool in teaching a large class, and it's an ideal--though widely overlooked--application of web-based technology in teaching.