How Does Political Ideology Influence Views On Accountability?
May 5, 2013 Leave a comment
Accountability is all the rage these days, whether it’s with regard to schools, hospitals, government agencies, or the local Geico car insurance branch. But not all accountability is the same, and a thought-provoking new study led by Penn’s Philip Tetlock examines how political ideology and trust can influence support for various accountability systems.
The study used a sample of MBA students to investigate the distinction between “outcome accountability” — where the focus is on a tangible end-result — and “process accountability” — where the focus is on effort and adherence to best practices. Tetlock and his team hypothesized that conservatives would be more likely to support outcome accountability while liberals would be more likely to support process accountability. They reasoned that conservatives tend to attribute outcomes to personal characteristics rather than external factors, and thus a person ought to be held responsible for whether they generate a desired outcome. Conservatives also value protecting organizations from free riders who create a facade of good faith, and thus they would be skeptical of the efficacy of process accountability. Liberals, on the other hand, would be more likely to want to protect lower status employees from being judged based on uncontrollable events, and thus they would prefer process accountability.
In the initial experiment, the researchers tested their hypothesis in an unspecified policy domain — i.e. they simply described different accountability systems in a “large company.” Sure enough, conservatives tended to prefer outcome accountability while liberals tended to prefer process accountability.
But things got more interesting once the researchers introduced two specific policy domains that emphasized different values. One domain was education, an area where efficiency is the central value accountability is designed to uphold. (Yes, you might disagree with this notion of efficiency, but I think it’s valid in the context of the experiment.) Participants were asked to consider an outcome accountability option, in which teachers were evaluated based on standardized test scores, and a process accountability option, in which teachers were monitored through occasional observations. The second domain was equal employment opportunity hiring, a domain where equality is the central value the accountability is designed to uphold. Participants were asked to consider an outcome accountability option, in which managers were evaluated based on the number of minority employees hired, and a process accountability option, in which managers received diversity training and had their hiring decisions monitored by the human resources department.
As expected, in the education scenario, conservatives preferred outcome accountability, and their preference was even stronger than in the initial “unspecified” scenario. However, in the diversity scenario, liberals were more likely to prefer outcome accountability and conservatives were more likely to prefer process accountability. In other words, when equality was at stake, it was liberals who wanted employees to reach certain metrics regardless of how it was done. The idelogical nature of these preferences also tended to make them fairly sticky. When confronted with evidence that employees had acted dishonestly to subvert the system, both liberals and conservatives were significantly more likely to say they would switch accountability systems when the subversion occurred in their non-preferred system (e.g. process accountability for liberals in the diversity domain). This may help explain some of the stalemate over teacher evaluations and affirmative action.
Thus, managers who initially prefer process or outcome accountability and then discover that employees have subverted their preferred system will opt to tinker on the margins by closing loopholes, not by adopting a system they initially disliked. When they learn that employees failed to implement promised best-practices for checking racial bias, conservative managers will not suddenly embrace outcome accountability in equality-salient policy domains. Nor will liberals suddenly embrace outcome accountability in efficiency-salient domains when they discover that employees failed to implement promised processes.
As the researchers point out, the broader lesson is that accountability preferences tend to emerge from a complex recipe rather than a single ingredient.
Our claims are modest: (1) when managers think about how to structure accountability systems in particular domains, the spotlight for some managers seems to be more on ‘‘getting the job done’’ and for others, more on treating employees equally/preventing prejudice; (2) which managers fall in which categories is a joint function of ideological outlook and the degree to which the work setting has become a focal point for policy debates (‘‘politicization’’).
It would be interesting to see how the study would play out in the domain of healthcare policy. For liberals, healthcare clearly involves equality, as the priority is for everybody to have health insurance. But the potential for a single-payer system to keeps prices low and ensure conditions don’t worsen due to a lack of treatment may also make it an efficiency issue. For conservatives, healthcare seems like an issue where efficiency would be important, but it’s clear there’s also a focus on the lack of fairness in having income redistributed to buy insurance for poor people. All of this is to say that I’m not sure what the result would be, and it would be interesting to find out.
Tetlock and his team also conducted a follow up experiment that sought to further investigate the role of ideology in situations where there was complete information about employee trustworthiness. Before doing so, they identified a positive and negative type of both process and outcome accountability.
(A) Opportunity-focused outcome accountability empowers employees to use their creativity to go beyond standard operating routines and gives them chances to benefit from upside uncertainties of effort-outcome links (Grant & Ash- ford, 2008; Simons, 2005, 2010).
(B) Punitive outcome accountability sends a no-excuses message to employees (Rodgers, 1993) and shifts the risk of uncertain effort-outcome linkages to employees (Williamson, 1991).
(C) Employee-protective process accountability rewards good faith effort and shifts the downside risk of uncertain effort-outcome links from employees onto management (Scholten, van Knippenberg, Nijstad, & De Dreu, 2007; Siegel-Jacobs & Yates, 1996), with potential benefits of reducing stress and fear of mistakes and failure (Lee, Edmondson, Thomke, & Worline, 2004; Schoemaker, 2011);
(D) Punitive process accountability increases monitoring of processes to prevent faking of good-faith efforts and shifts the risk of uncertain effort-outcome links onto employees (Patil, Vieider, & Tetlock, 2013).
In the second experiment participants were told that employees either wanted to do a good job, or get paid as much as possible for as little work as possible. Participants were also told that either a great deal of luck was involved in how employee effort translated into work quality, or that almost no luck was involved. They were then asked about accountability preferences.
The researchers found that regardless of ideology, people chose punitive accountability systems (B & D) when employees were untrustworthy and positive accountability systems (A & C) when employees were trustworthy. This occurred even when there was no reliable link between effort and outcome. In addition, there no significant relationship between ideology and whether outcome or process accountability was preferred. It would seem that ideology plays a role, but it may only serve as a heuristic for when no good information about employee trustworthiness is available.
What Can This Teach Us About Education Policy?
I think these findings do help illuminate the contours of the debate on teacher accountability. Regardless of ideology, almost everybody thinks teachers are trustworthy. The idea of “teacher bashing” is used to rally union supporters, but nobody serious actually believes that a significant number of teachers are slacking off because they know it’s hard to get fired. That means the acceptable accountability options are A and C. Many people, myself and Barack Obama included, don’t see most manifestations of option C, the process accountability system, as doing enough to ensure accountability. This view is based on the fact that observations rarely rate teachers at a level that necessitates consequences, and that a few annual observations may not be enough to ensure that a teacher is doing a good job. Therefore the clear choice becomes option A, which in practice involves using test scores as part of a teacher’s evaluation.
Others look at the idea of using test scores and all the see is option B — a system that unfairly punishes teachers and blames them for every shortcoming in the system. For these people, the clear choice is option C, even if it’s imperfect.
Unfortunately, there doesn’t seem to be much room for compromise between these two trenches. Value-added measures could get better at controlling for certain environmental factors, but at some point you run up against the wall of innovation in data collection and statistical analysis. This makes it difficult to convince people value-added measure are option A rather than option B. Similarly, we could strengthen the stakes tied to teacher observations, but any system with teeth would likely be seen as a punitive outcome accountability system akin to B, rather than a positive process accountability system akin to C.
One thing that makes this a difficult issue is that we’ve moved on to debating policies when the real disagreement is in the underlying philosophical issues. Ultimately, teacher accountability is about what level of teacher turnover will maximize the talent of a teaching staff, as well as the degree to which teachers should be immune from that turnover. Put another way, how many prospective new teachers would quickly become better than incumbent teachers, and how many incumbent teachers, because of the nature of their profession, should have their jobs protected regardless of how good the new candidates are? That’s basically what the teacher accountability debate is about. (Yes, teacher accountability is also about helping teachers get better. And we should definitely have a system focused on helping teachers get better. But these systems would hypothetically help new teachers get better too, and so whether or not a system helps teachers improve is largely irrelevant to whether a school would be better off replacing a teacher with somebody new.)
Now you might think that few teachers have good replacements and that it’s insane, unproductive, and possibly illegal for teachers to have such little job security. Alternatively you might think that new graduates from teacher programs are pretty good, and that teachers, like most other employees, ought to be replaced if their manager is able to find somebody significantly better. For years the consensus was basically the former view — that all competent teachers should have their jobs protected. More recently there has been a marginal shift toward the latter view — that we could improve teaching staffs by pushing to replace more teachers. Many think we should push even further, while many believe we should go back. This is the core of the debate about using test-scores to evaluate teachers, and debates about VAM models and teacher observations merely obfuscate the real issue of how much job security is appropriate. (And yes, in this case I’m essentially using test scores and VAM as a proxy for “policy that will lead to more teacher turnover.”) Again, I don’t think there’s a silver bullet compromise, I just wanted to vent about people dancing around the issues at the core of policy debates.
Tetlock, P., Vieider, F., Patil, S., & Grant, A. (2013). Accountability and ideology: When left looks right and right looks left Organizational Behavior and Human Decision Processes, 122 (1), 22-35 DOI: 10.1016/j.obhdp.2013.03.007