The invisible hands behind the student evaluation of teaching: the rise of the new managerial elite in the governance of higher education.

The amount of academic research on the Student Evaluation of Teaching (SET) is staggering. Perhaps the fascination with SET stems from the very introspective nature of the issue: academics researching themselves. While previous research has grappled with the notion of teaching effectiveness and teaching score validity, we shift the focus to university governance. Thus, we set ourselves aside from the managerialist perspective on higher education. Managerialism deals with the technological problem of optimizing the operations of the organization within a given set of rules, while university governance analyzes the very choice of rules that govern the allocation of power and control among financial claims. Unlike managerialism, university governance does not claim to be a value-free exercise; it approaches decision-making from an economic and political perspective.

The emergence of modern student evaluation of teaching can be traced back to the 1960s. During that period, SET was used on an experimental basis, i.e., voluntarily. In the early 1980s it became the mainstay of academic practice in North America (Centra 1993; Wachtel 1998; Murray 2005; Lohman 2006). Today, the main justification for its use stems from the belief that SET is measuring teaching effectiveness. Many universities in the United States and Canada use teaching scores (to various extents) as a basis for tenure, promotion, re-appointment, and resource allocation.

Interestingly, the impetus gained by SET in the 1960s came in the wake of the civil rights movement. Back then, the most vocal group were the students who saw SET as a conduit for making their voice heard in the affairs of the university. The Faculty seized upon the opportunity to diversify the performance criteria on which tenure and promotion were based. And, the administrators probably sensed that the use of SET would provide an aura of accountability and legitimacy. Universities are entrusted with millions of dollars of public and private money, and it was important to show that the money was well spent. From its inception, SET mollified students, taxpayers, and private donors.

We argue that the notion of teaching effectiveness--invoked today to rationalize the use of SET--has no verifiable empirical content, and therefore the question of teaching score validity is misguided. Current research shows that, at most, teaching scores reveal the extent to which the professor is able to connect to the students' cultural beliefs and live up to their expectations. This expectation-fulfilling mindset, however, is not a meaningful approach. When used for administrative purposes, SET leads to collusion between students and teachers, and generates negative economic externalities. Why then are SETs still used on such a grand scale? The main thesis of this paper is that university administrators nurture teaching scores because they represent an enabling myth (Dugger 1989). SET legitimizes managerial claims to increasing control over the affairs of the university.

We organize our paper as follows: in the next two sections we discuss the empirical content of teaching scores. Then we discuss the conflict of interests that plague the use of SETs. We analyze the relationship among teaching scores, the rising managerial elite, and university governance in the latter sections, and suggestions and concluding comments are presented at the end of the article.

In the Eye of the Beholder

The central issue in the student evaluation of teaching is the notion of teaching effectiveness. The ability of the students to gauge the quality of the instruction process rests on the argument that the wisdom of crowds is more dependable than the wisdom of elites (Surowiecki 2004). About a century ago, Galton (1907) noted that any large crowd is better than any single individual at guessing the dressed weight of an ox. From here, Galton inferred that democratic judgments are more trustworthy than otherwise believed. Unfortunately, the wisdom of crowds argument cannot be readily applied to teaching in spite of its merit. For one, the dressed weight of an ox is a relatively straight forward notion that can be easily verified by a third independent party. Teaching effectiveness, however, is only an abstract construct, subject to various interpretations. In addition, the crowd participating in the weight-measuring contest was not subject to any conflict of interest. That is, the evaluation had no significant impact on their welfare. Each individual was an objective participant motivated only by the desire to test his/her guessing skill.