The Hubbub about Teaching Evaluations

There is some interesting debate (Mankiw, Cowen, and Jeff Ely) going on about measuring teacher quality at the postsecondary level. A couple of good sentences from this article, which recently appeared in the Journal of Political Economy (ungated version here).

In primary and secondary education, measures of teacher quality are
often based on contemporaneous student performance on standard-ized
achievement tests. In the postsecondary environment, scores on student
evaluations of professors are typically used to measure teaching
quality.

A lot of us question why this should be the case, and some of us actively doubt the competence of students when it comes to evaluating teacher quality. My own view is that these evaluations are a signal of some things the professor might be doing well and certain other things that the professor might improve, but they do a poor job of measuring how well the instructor did at teaching the actual material. Sometimes they measure little more than popularity (which is not altogether unimportant!). But here are some interesting results:

[O]ur results indicate that professors who excel at promoting contemporaneous student achievement, on average, harm the subsequent performance of their students in more advanced classes.

In other words, students of harder professors for introductory courses (who may struggle and give lower evaluations of that professor) do better in subsequent courses. Higher rank and experience (and thus lower pressured to get good evaluations for tenure) are negatively correlated with current "value-added" in introductory coursework, but positively correlated with value-added in subsequent courses.