Fans of the American sport of baseball enjoyed an unusual “double header” of activities this weekend. In addition to watching the annual ritual of spring training play out in Arizona and Florida, they also rooted for the baseball film Moneyball during Sunday evening’s Academy Awards telecast.
The film was nominated in three major categories: Best Picture, Best Actor in a Leading Role (Brad Pitt), and Best Actor in a Supporting Role (Jonah Hill). Brad Pitt himself was a producer on the film; he took a “leading role” (so to speak) in securing the required financing of the entire project.
The film itself was based on a best selling book that described how the Oakland Athletics employed advanced statistical methods to assess available ball players. Using such methods, they were able to select and promote the very best players in their own organization, and to identify and trade for the finest talent in other organizations.
In a sense, the Moneyball approach to rating ball players with statistical measurements is similar to the process that grade schools and college programs use to rank their students. Last week, however, the City of New York turned its ranking system on its head by publicizing the evaluation grades of its own teachers.
Baseball vs. Biology
In the film Moneyball, Brad Pitt struggles to convince his coterie of senior talent scouts that qualitative baseball skills can defined and measured in a manner that produces reliable ratings. His scouts, however, stubbornly cling to the traditional belief that talent and ability are innate traits and thus cannot be measured and compared quantitatively.
Until recently, most grade school administrators clung to similar beliefs about their own teachers. After all, they argued, how can we measure the effectiveness of ninth grade biology teachers if we are asking them to inspire their students to become doctors, nurses, and medical researchers many years later? In other words, how can we possibly expect to measure inspirational qualities that will necessarily take many years to generate any results?
If we define the primary role of teachers, though, in a narrow manner as the “teacher of facts” to students, then we can utilize standardized tests to measure effectiveness. That’s the approach that the City of New York utilized to compile its teacher rankings; nevertheless, many critics are now identifying problems with the City’s compilation process.
A Pair Of Problems
One set of problems appears to involve concerns about public disclosure. Unlike most organizations, which generally treat performance evaluations as private and confidential documents, the City of New York dumped its entire set of ratings onto public web pages. Many now criticize the City for publicly humiliating teachers with “incomplete, misleading, and often wrong” ratings.
A second set of problems appears to involve concerns about the validity of the ratings themselves. They appear to have been calculated with an extremely high confidence internal, which is a statistical term that refers to a measurement’s inherent margin of error.
Imagine that you are a teacher who has become comfortable teaching an introductory biology course in a wealthy community where parents routinely hire (at their own expense) private tutors to help their children “ace” their exams. Faced with such a rating system, would you ever agree to teach an advanced biology course in a poor community where children live in single parent households with no financial resources?
Teaching To The Test
Ironically, New York City’s decision comes at a time when the federal Department of Education is dismantling its national No Child Left Behind program. One of the most significant complaints about that program (and about New York City’s testing process as well) targets its high level of reliance on standardized student tests, a process that may compel teachers to focus on “teaching to the test” by emphasizing rote memorization activities over critical thinking assignments.
In fact, very shortly before New York City published its teacher rankings on the internet, Bill Gates authored a lengthy and impassioned editorial in the New York Times to ask that the City refrain from doing so. His request was a highly noteworthy one, considering the role that the Gates Foundation has played in developing advanced (and highly successful) methods to modernize our primary education systems in the United States and overseas.
He titled his editorial Shame Is Not The Solution, noting that New York’s “value added” method of rating teachers on the basis of their “impact on students’ test scores” is a “big mistake.” Although today’s professional baseball executives use statistics like Wins Above Replacement to evaluate talent, Gates claimed that the same concept cannot be applied to public school teachers.
Of course, that’s exactly the objection that Brad Pitt’s character faced in the film Moneyball. In New York City, though, the fate of a baseball team is not at stake. Instead, the future of generations of students hangs in the balance.