Disputatio Sine Fine: O’Connell, Yusuf, and Meier on evidence-based teacher accountability

The debate below is based on “Two models of teacher accountability: Toward an explanation for the disappointing results of some market-based school reforms” by Lenahan O’Connell and Juita-Elena Yusuf, published online September 7, 2015. There is a brief commentary by O’Connell and Yusuf, followed by a response from Ken Meier, and ending with a rejoinder by O’Connell and Yusuf.

O’Connell and Yusuf on their article:

Many school systems have turned to school choice, especially charter schools, to improve student achievement. This article reviews the numerous assessments of school choice programs to document that choice programs have had no significant effect on test scores. It describes two theoretical approaches to school teacher work effort and effectiveness: an economic model (agency theory) and a social actor model derived from sociology and management theory. The social actor model is used to describe the inter-personal pressures and norms operating in the public schools that foster teacher accountability and work effort even in the absence of competition between schools for students. Given the substantial level of accountability present in the public schools, the article concludes that charter schools or school choice are unlikely to improve student achievement without improvements in teaching methods and additional aid directed to the needs of low-income students.

A commentary on evidence-based public management—Meier’s response to the article:

Traditional public administration was reform oriented and recognized few limits on its prescriptive range; anything from the proper form of city government to the organization of the U.S. presidency was considered within its scope (Gulick and Urwick 1938; Waldo 1948). O’Connell and Yusuf return to this tradition with a critique of governance that is superficially about the use of incentives in public education but can be read as a fundamental challenge to a wide range of New Public Administration style efforts that seek to bring business values and processes to government. The authors provide extensive evidence on four questions that market-based reformers advocate and find that: 1) nonunion schools do not outperform unionized ones, 2) students in private schools do not outperform those in public schools, 3) students who use vouchers do not outperform those who stay in public schools, and 4) charter school students do not outperform public school students.

The basic core of O’Connell and Yusef’s argument is that these education reforms fail to recognize the multidimensional nature of human motivation and focus instead on money (pay for performance), competition (threats to lose one’s job), and collective choice to work as they do in microeconomic theory. O’Connell and Yusef rely on existing management theories and the extensive literature on public service motivation to argue that market incentives under cut many of the reasons why public employees work hard to improve organizational performance. They propose a social actor model to replace the principal agent model as the basic logic for designing programs. Let me raise three issues in regard to this excellent contribution.

First, the authors do not focus on one of the fundamental flaws of incentives, that individual incentives do not necessarily contribute to group performance. Since the classic work by Peter Blau (1955), we know that individual level incentives can generate suboptimization, goal displacement, and undercut overall organizational performance. The best teachers should be assigned to the most difficult classes, just as the best surgeons get the difficult cases. Although some pay for performance systems recognize this organizational fact of life and seek to create value added measures of performance, those measures are highly complex and as a result lack face validity for most teachers.

Second, although I am hardly an advocate of principal-agent models, principal-agent models can be far more informative than their economic advocates and than O’Connell and Yusuf portray them. Gary Miller’s (2000) excellent work shifts the focus from agent shirking to cheating by the principal. How can the agent (teacher) know that the principal (school district) will actually reward for better performance? The failure of the federal pay for performance bonuses is a classic example. Gailmard and Patty (2013), as another example, show that principals have to give agents more discretion, not less, to gain the benefits of professionalism and expertise.

Third, the article reveals another serious pathology of governance – policy making by anecdote. As the authors demonstrate, we have a wealth of evidence on education and performance, yet it appears that little of this truth breaches the fortress of power. Rather than rely on evidence-based public policy, policy makers respond to stories. Some of these are stories about individuals (the success of a few charters is generalized to all charters) and some are based on existing theoretical models that seem intuitively appealing. Indeed the logic of economic incentives is so obvious that the empirical evidence should be overwhelming. Unfortunately the evidence is not overwhelming; it is not even on balance positive. There are real problems in U.S. education, and in a variety of other policy areas. The more we design public policies based on a model with little empirical support of based on anecdotal claims, the more money, time and effort we waste that could be applied to actually fixing the problems.

References

Blau, Peter. 1955. Bureaucracy and Modern Society. New York: Random House.

Gailmard, Sean and John W. Patty. 2013. Learning While Governing: Expertise and Accountability in the Executive Branch. Chicago: University of Chicago Press.

Gulick, Luther H. and Lyndall Urwick. 1937. Papers on the Science of Administration. New York: Institute of Public Administration.

Miller, Gary. 2000.   “Above Politics: Credible Commitment and Efficiency in the Design of Public Agencies.” Journal of Public Administration Research and Theory 10: 289-327.

Waldo, C. Dwight. 1948. The Administrative State: A Study in the Political Theory of American Public Administration. New York: Holmes and Meier.

O’Connell and Yusuf’s rejoinder:

In this paper we describe the dynamics of the classroom and school environment from the perspective of the subordinate agents, the teachers. We underscore the norms and relationships that influence teachers as they interact with students, parents and school principals. Our discussion of incentives concentrated on those arising from the demands of the classroom environment and teacher professionalism.

Kenneth Meier makes three observations with which we agree. He is right to point out that superordinate principals can shirk their responsibilities. Clearly, school system superintendents and school boards often ignore the plight of low-performing schools and students, preferring to please local elites by shifting more resources to schools serving students from the upper-middle class.

One of our points is that the study of accountability requires an assessment of all the forces at work in a given setting. This fosters identification of all the factors—negative as well as positive—operating in the accountability environment. Meier recommends (and we agree) that the best teachers be assigned to the most difficult cases (i.e., students from low-income families). This, of course, rarely happens as the teacher assignment system in most cities tends to reward the best teachers with assignment to the schools with the most prepared students. This keeps the best teachers in the system but short changes the disadvantaged students who more often are taught by inexperienced or less able instructors.

Last, we couldn’t agree more that public policy must be informed by empirical study involving a close analysis of all the incentives and disincentives present in a given situation. This would be policy guided by research rather than highly abstract economic or social theory. Our paper documents and offers an explanation of the failure of market reforms based on economic theories to improve teacher work effort and effectiveness.

 

 

 

 


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s